The Center for Nursing Research & Innovation

Robert Wood Johnson Rapid Response Team Initiative Evaluation Project

Nancy E. Donaldson RN, DNSc., Principal Investigator

Introduction

Robert Wood Johnson Foundation launched an ambitious initiative that provides support to nine health care systems/associations to establish learning networks to assist and accelerate the efforts of their member hospitals to implement and evaluate the impact of rapid response team (Rapid Response Teams) interventions with the aim of improving patient outcomes and the work environment of nurses. The literature suggests that Rapid Response Teams mobilize clinical expert responders to manage emergent changes in patient condition which are potentially life threatening and may be related to clinician or system errors and omissions.

Rapid Response Teams, also known as Medical Emergency Teams (METs), have been adopted as one of the IHI 100,000 Lives Campaign innovations based on preliminary evidence that they are a low risk, high benefit intervention that engages professionals in early rescue of patients when direct care providers note triggers in patient status that suggest possible deteriorating, perhaps life threatening, condition changes. Several published reports note that nurses, in the vast majority of instances, sound the Rapid Response Team alarm, summoning the team to the patient's bedside, thus it is posited that the implementation of Rapid Response Teams may ease the burden of nurse confronting worsening patient condition. It is also possible that variation in the deployment and expertise of nurses at the bedside may influence the effectiveness of the Rapid Response Team, for example, by delaying the Rapid Response Team call. Retrospective chart reviews have revealed a pattern in which one or more signs of patient deterioration were present by not apparently noted by nurses prior to cardiac or pulmonary arrest, two common emergent conditions the Rapid Response Team is intended to reduce.

In advancing Rapid Response Teams as a patient safety innovation ready for implementation, IHI proposed three core measures and offered a preliminary operational measurement approach. In responding to the RWJ RFP, successful grantees (8 out of 9) included the IHI measures in their evaluation approach, although the specifications for the measures were not fully presented and may vary widely across grantees and their hospital sites.

The UCSF RWJ Rapid Response Team Initiative Evaluation Project was designed to conduct a formal evaluation of the RWJ Rapid Response Team implementation impacts by working with grantees to standardize their measures, develop coding guidelines for data capture, provide training for data collectors/coders, as well as aggregate and analyze data submitted by grantees to trace the impacts of the Rapid Response Team Initiative at baseline, mid point and final points in time. In addition, we will integrate qualitative measures which explore important contextual factors related to “best practices” in Rapid Response Team impacts and factors that emerge as significant barriers to progress. Finally, we worked with several grantees who have access to nurse staffing data on the Rapid Response Team pilot test units, and engaged them in a descriptive analysis that examines the association between key staff variables (hours of care, skill mix, ratios, use of contract/agency workers, years of experience) to permit exploration of the potential contribution of these variables to understanding variation in the observed impacts of Rapid Response Teams on patient outcomes.

Project Principal Objectives and Target Audience

The principal objectives of this two year, descriptive evaluation project were to, in collaboration with RWJ Project Staff and Grantees:

1. Establish standardized metrics that build on IHI and Grantee proposed measures to establish a matrix of baseline, formative and summative measures that are feasible to collect and aggregate across sites and provide insight into the impacts of Rapid Response Team implementation overtime in the diverse RWJ Rapid Response Team grantee hospital settings.
2. .Identify, explore, validate and describe contextual factors, including variation in nurse staffing, that are associated with comparatively “best” implementation of Rapid Response Teams, and factors that emerge as barriers to effective Rapid Response Team implementation.
3. Develop and deploy educational and consultative assistance to grantees to build their capacity for evaluation data capture and optimize the integrity of the evaluation data they submit.
4. Conduct baseline, formative and summative point-in-time aggregation and analysis of “core” quantitative variables.

The UCSF Team worked in a highly collaborative relationship with grantees, to optimize their evaluation data capture and participation in the evaluation enterprise. It was our goal to provide evaluation data to the Foundation, Rapid Response Team grantees and the “community-at-large” that advances understanding of Rapid Response Team impacts, outcomes and effective implementation.

Project Rationale and Approach

This project is especially timely. In addition to IHI and RWJ, the Gordon and Betty Moore Foundation is supporting efforts to accelerate development and implementation of Rapid Response Teams in Bay Area hospitals in California. These efforts cannot, however, rely on a robust knowledge base with well developed measures. Metrics have varied widely across studies and much of the foundational work has been studies in the United Kingdom and Australia. The literature suggests much of the research related to Rapid Response Team/MET implementation is preliminary. As a result, there is not consensus on measures, rates and consensus on standards for judging the effectiveness of Rapid Response Teams has not been achieved.

Our approach built on the preliminary evaluation approaches proposed by RWJ Rapid Response Team grantees. Proposed Grantee Rapid Response Team Learning Network interventions vary widely in their content, contacts and delivery methods. While a pattern of proposed evaluation measures is noted, grantees varied in their metric foci and specifications. It is likely that the educational and supportive interventions proposed will vary greatly in their “effect power” and ultimate impacts over the 18 months of the project are preliminary. We therefore envision an approach that is iterative and integrates valid and reliable hospital level core metrics that are standardized, from the National Registry for Cardiopulmonary Resuscitation (NRCPR), including CPR immediate and final survival rates (more sensitive than gross measures of mortality). We developed additional metrics to describe the number of Rapid Response Team calls. Working with grantees, we then identified contextual factors which may confound outcomes, for example what percent of Rapid Response Team calls result in Do Not Resuscitate (DNR) action, which may be more an indicator of how advanced directives and DNR processes are managed in that setting, than Rapid Response Team impacts. In collaboration with grantees and gleaned from a sample of selected site visits, we expected to explore contextual factors that will contribute to interpreting the implementation outcomes, including how variation in the deployment of the nursing workforce may impact Rapid Response Team calls and patient survival. We also worked with several grantees to systematically examine how standardized measures of nurse staffing are associated with Rapid Response Team clinical outcomes. Nurse staffing measures were standardized based on the measures now used by the California Nursing Outcomes Coalition, which are consistent with the National Database for Nursing Quality and the VA and Military nursing outcomes datasets.

To optimize the integrity of the data collection process within grantee systems and across grantees, we will create and distribute evaluation data collection guidelines and tools, as well as provide grantees with conference call and web-based training and technical support as they begin baseline data collection. Using email and conference calls, grantees will have opportunities to receive supportive technical assistance from the UCSF Team and forge connections with one another that may enhance their efforts.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -