Introduction: Real time checklists, adopted from high reliability industries, have been used in many healthcare settings to assure that key care deliverables and processes are applied consistently. The purpose of this quality improvement initiative is to describe the development and implementation of a daily rounding critical cardiac care checklist and assess its impact on increasing compliance with guideline-recommended cardiac performance measures.

Methods & Design: This is a prospective, observational cohort study in the coronary care unit (CCU) of a university-affiliated major teaching hospital in southeastern Michigan, from Jan 2014 - Jan 2015. Each CCU cardiac patient served as their own control, utilizing each patient day for monitoring of adherence to 20 key care deliverables (based on eligibility) as audited by a quality technician using an aligned checklist. The rate of adherence to the items on the checklist, and the rate of corrections which occurred within 24 hours of being identified as an omission were measured monthly. The results were then summed up into four quarters and compared between teaching and non-teaching services.

Results: During the study period, the total number of patients enrolled was 550 patients. These patients were followed for a total number of 1457 days. The overall rate of initial adherence to the items on the checklist was 88.4%, and it improved to 95.5% with the use of the checklist in the final adherence rates. Both initial and final rates of adherence were found to be higher on the teaching service compared to the non-teaching service.

Conclusion: The use and application of a daily CCU checklist by an assigned quality technician can identify and correct deficiencies in desired care. Despite the intensity of the interventions, the enhanced engagement and alignment of the nursing and physician staff and the reliable processes put into place, the goal of achieving 100% adherence rates to eligible and appropriate clinical measures was not achieved. This project provided a great deal of insight into the barriers, challenges and limitations that need to be overcome for successful implementation, utilization, value and sustainability of an applied healthcare checklist, to improve intended and desired performance measures and drive favorable outcomes in a real world setting

The Institute of Medicine, in a landmark publication [1], pointed out that hospitals are not particularly safe environments, with frequent medical errors, omissions of care and adverse events related directly to healthcare, leading to significant morbidity and mortality. This is especially evident in patients being managed in intensive care units which involve a higher acuity and complexity of care [2-7] and often involve biased clinical decision making. The provision of optimal care requires integration and coordination of complex processes of care and collaboration by teams representing multiple disciplinary roles providing acute critical clinical decisions, evaluation and interventions. The sheer enormity, critical nature and complexity of care makes intensive care units especially vulnerable for cognitive and unintentional errors and breakdown of care communication and handoff coordination. There are often distractions caused by the need for acute interventions which may lead to occasionally omitting key elements of care, application of guideline recommended therapies or implementation of preventative practices by members of the multidisciplinary team.

Real time, quality audit checklists have been widely used in the aviation and high risk industries for many years to achieve safety by standardizing the steps to be followed in carrying out an action, ensuring all required and best practices, policies and procedures are applied consistently and reliably and team work and communication are optimized [8]. The Agency for Healthcare Research and Quality (AHRQ) defines a checklist as “an algorithmic listing of actions to be performed in a given clinical setting, the goal being to ensure that no step will be forgotten” thus preventing lapses in concentration and distractions [9]. Based on the remarkable results achieved, surgical safety and various care checklists have been successfully adopted and applied in various healthcare settings. Multiple studies have shown that the use of checklists in a variety of healthcare settings and situations result in positive impact on various patient outcomes [10-16] but this has not been entirely consistent [17-20]. Checklists are support tools to be used by healthcare providers, especially operating in challenging and complex environments such as intensive care units. They serve as cognitive and forcing function reminders to ensure safe practices, team communication and coordination and the application of evidence-based preventive strategies and guideline recommended therapies [21-37].

Despite proven effectiveness in mostly observational studies, especially when the tool is embedded into care reliably, checklists are not uniformly or consistently used. Many challenges exist in their implementation and sustainable use and proving a direct relationship between the use of the tool and outcomes [27- 29] has been evasive. Even though daily rounding checklists have been well described in medical, trauma, and pediatric and surgical intensive care units [21-37] with favorable results, there is no reported experience on the use and impact of a daily care checklist in the coronary care unit (CCU) setting, focusing on optimizing the management of complex and challenging critically ill cardiac patients and assuring the application of guideline recommended care.

The purpose of this quality improvement initiative is to describe the development and implementation of a daily rounding critical cardiac care checklist and impact on optimizing adherence to key care deliverables. This study was designed to audit and document patient care, highlight care deficiencies and correct concurrently. By successfully embedding the checklist into the daily work flow of a CCU in a large academic community hospital, the goals were to: 1) facilitate and optimize adherence to evidence-based preventative measures and guideline recommended cardiac specific therapeutic measures; 2) improve processes of care on a consistent daily basis; and 3) foster a shared mission and culture of safety by the healthcare providers in the CCU. It was expected that this project, would provide a great deal of insight into the challenges and barriers that needed to be overcome for successful implementation and sustainability to drive favorable outcomes in a real world setting.

This study took place in the Coronary Care Unit (CCU) of Beaumont Hospital-Dearborn (BH-D), formerly Oakwood and Medical Center, a university-affiliated major teaching hospital in southeastern Michigan. The CCU consists of 18 beds, and admits an average of 1100 patients yearly, consisting of both cardiac and non-cardiac intensive care patients. Patients with principal cardiac diagnoses who are admitted to the CCU are cared for by one of two services, a teaching service (which trains residents and cardiology fellows), or a non-teaching service, attended to by private internists and cardiologists.

Daily multidisciplinary quality and operational rounding (MDR) had been occurring prior to the implementation of this quality improvement project. These rounds were led by one of the nurse co-managers and involved the staff nurses, CCU Medical Director, a cardiac Pharm.D. (who participated in these rounds for education and quality improvement purposes, assuring evidencebased therapies and safe medication practices), a respiratory therapist, the CCU case manager, a dietician, often a palliative care nurse, and the residents assigned to the CCU. During these rounds, a worksheet (paper format, not part of the permanent record) had been used to assure that bundle order sets were used and updates on clinical status, daily management plans and challenges to problem resolution on each patient admitted to the unit were discussed by the team along with discharge or transfer escalation if appropriate. Active social, ethical, dispositional or operational concerns were also reviewed and addressed for every patient. Based on our performance outcomes from an internal database documenting the application of and adherence to the Michigan Health and Hospital Keystone Center for Patient Safety and Quality Keystone ICU Project Care deliverables [21], opportunities had been highlighted to reduce ventilator acquired pneumonia and sepsis, improve deep vein thrombosis (DVT) prophylaxis, and assure better blood sugar control. In response to this, and as the literature regarding the benefits of checklists evolved for ICU quality improvement [21, 23, 25, 26, 33], an opportunity and challenge presented itself to design, develop and implement a similar type of checklist applicable to the CCU. A number of prototypic checklists were trialed, with feedback and insight gained, with the goal being to develop and implement a checklist that ensured that key cardiac performance measures were ordered for every patient on a consistent and reliable basis.

For over 20 years, the principal investigator (ALR) had rounded on almost a daily and consistent basis with the residents rotating on the teaching services on the CCU. Based on the daily MDR’s in the CCU, and with a growing culture of safety and improved quality in our health system, a number of key cardiac specific care measures, preventative and operational throughput opportunities were deemed appropriate to be targeted for improvement. These included: evidence based cardiac therapies and prophylactic interventions, progressive activity guidelines, palliative and hospice care candidacy, presence of advanced directive, and daily auditing for transfer/discharge eligibility. The MDR was the ideal and fertile venue to introduce, embed, apply and operationalize this project and the checklist was the tool to identify and address omissions in management. Constant feedback and reinforcement were key to engage and embrace the nursing staff.

In the summer of 2011, a premedical student from the University of Michigan joined the CCU team for a rotation on quality improvement and clinical research. Her project was to review the literature on checklists, give input into the design and development of the checklist, and trial a prototype during rounds. She was mentored and trained on evidence based therapies and preventative strategies with assistance from the Pharm D, medical residents and nurses. After a short period of time, based on observations of the workflow, she modified the MDR worksheet which was already being applied on daily basis. During MDR’s, any deficiencies in care that were identified were brought to the attention of the physicians and nurses to assure implementation of corrective action. The student completed the checklist on a daily basis and conferred with the attending physicians and nurses regarding deficiencies in patient care, and reasons for deficiencies were recorded and measured. This proved to be a very valuable prototype of the checklist. After two weeks of pilot use, the student made a number of key observations regarding the use of the checklist. The checklist was effective in documenting current care. Deficiencies in care were commonly due to contraindications due to other medical problems rather than omissions or commission. Some logistic and appropriate exclusion prevented the application of certain preventative measures.

A “checklist quality technician”, who was an MD graduate waiting to start his formal residency (RK), was initially hired in 2012 to support the team in the checklist application as intended, and eventually embedding into the workflow. He proved to be a pivotal factor in the design and adaption of the final intervention process and prototypic checklist used for the project. On a daily basis, prior to MDR, he reviewed the cardiac CCU patients for adherence to items on the original checklist, identified gaps and omissions in care and deliverables, and during the MDR, verbally reported these omissions, reminding the team to apply the missing measures, and then assuring the recommendations were executed and deficiencies corrected within the following 24 hours. The checklist therefore served as a reminder tool and a source of documentation for a database, and helped identify opportunities for better compliance. In order to sustain the use of the checklist, the Medical Director of the CCU reinforced on a continuing basis the importance of the checklist and monitored its use (via the quality technician). The entire process of checklist review added only one minute per patient to the duration of the rounds (approximately 10 minutes for the average daily census of CCU patients).

The checklist was further refined and customized by a team consisting of the CCU Medical Director and nurse managers, a Pharm.D, and the cardiovascular quality specialist. After thorough review of the literature and practice guidelines of the American College of Cardiology/American Heart Association, the therapeutic core measures on the proposed checklist were defined; the proven prophylactic interventions to reduce ventilator-associated pneumonia [33], central line-related blood stream infections and sepsis [21], gastrointestinal bleeding [35], and deep vein thrombosis were included as key care deliverables to our CCU checklist [34]. The process of development and inclusion of care deliverables in the checklist was driven by consensus reached by the team based on clinical relevance, evidence based validation of these measures, ease of use and integration into the workflow of the unit. Review and requested feedback on these core measures was obtained from the staff nurses, medical residents and Attending Cardiologists, regarding ease of monitoring and auditing, clinical relevance and added value to quality and patient safety outcomes. These measures were then endorsed and accepted for the final checklist after a number of “small tests of change” (see Appendix for final version).

The role of the quality technician was then assumed by a CCU registered nurse (MA) in June 2013 and a cardiology research nurse coordinator (JG) who joined in September 2013. The two partnered to ensure that the checklist was used daily as intended. They were trained on reviewing patients’ charts for adherence to the checklist measures prior to MDR’s, evidence-based care, and identifying gaps and omissions in care deliverables, in order to verbally report them during rounds. They attended CCU MDR alternatively Monday-Friday and addressed the identified deficiencies in care needing follow-up. Although, as registered nurses (RN), they were not credentialed to directly affect the corrections (e.g. by ordering independently), they were empowered (by the Project) to provide ongoing reminders and feedback to the primary nurses and physicians and do serial auditing to document if the recommended deficiencies were corrected.

Until December 2013, a “paper” form checklist was used during MDR’s, and the data was converted to an Excel spreadsheet for database analysis. This data entry process was too time consuming, and not user friendly for running reports. In November of 2013, an “electronic rounding tool” to collect information via a rounding I Pad was developed; this tool was interactive and allowed for real time reports and concurrent review of the entered data [31]. From January 2014 to January 2015, the I Pad was used exclusively for the checklist data reports, and the results are based on this.

The primary CCU RN serving as the quality technician left the project in December 2014, to become a full time employed advance practice nurse in a primary care practice. The cardiology research nurse coordinator continued solely as the checklist quality technician until the end of the study in January 2015. JG from Corporate Data Analytics provided database support for the entire project.

The targeted patients for this study were the critical ill cardiac patients admitted to the coronary care unit at our hospital above the age of eighteen, with the following diagnoses: acute coronary syndromes (STEMI and high risk ACS), congestive heart failure (CHF) or any type of an arrhythmia (regardless of which service was taking care of the patient, whether teaching or non-teaching). The data analyzed for results was collected between January 2014 and January 2015.

Patients younger than eighteen, critical care patients placed in the CCU (due to shortage in ICU beds), and cardiac patients outside of the CCU were not included in the study.

The utilization and rationale of the checklist as a tool, the purpose and goals of the project, and the roles and expectations of all the members of the team were constantly reinforced during the MDR and feedback given on a continual basis. The intended and expected use and completion of the checklist, design and implementation problems and issues, ongoing barriers to overcome, concurrent results and compliance was assessed on a weekly basis by the team coordinating the project. In the event corrective actions were needed, we applied “small tests of change” [38] process improvement interventions along with developing and assigning specific corrective action plans. It was occasionally necessary to revise and clarify the checklist for improved functionality and relevance based on observational monitoring of its use and feedback. There was also continuous follow up, reminders to and recognition of members of the nursing staff who excelled with deficiency corrections within the 24 hours from their detection.

Comprehensive baseline data for adherence rates to the cardiac indicators pre-checklist implementation was not available because of lack of a reliable process of monitoring, the unavailability of a CCU clinical database or auditing and the lack of a fully integrated EMR at the time of the project. Patients enrolled in the project served as their own controls by utilizing each patient day for monitoring of adherence to a panel of the twenty key care deliverables (based on their eligibility).

As this project was as much a pre- and post-interventional study as observational, the goal was to achieve “perfect” adherence rates and optimize the application of the targeted checklist measures by correcting identified gaps and omissions in care. The primary data analysis was measuring the adherence rates to the items on the checklist for each patient (based on eligibility) at the time of admission. The secondary data outcome was to measure the rate of corrections to the omitted items. The data was summed up into monthly reports, aggregated into four quarters, which was reviewed by the leaders of the project. We also compared performance between teaching and non-teaching patients. The success of the project was gauged by the initial rates of compliance and adherence, the identified care indicators per patient needing correcting, and the actual correctional rates, with the expectation that all eligible care indicators per patient based on eligibility, be achieved 100% of times. Quarterly reports of the status of the project as required were shared with BCBSMF.

The Institutional Investigational Review Board of Oakwood Health System approved the study and the independent chart review and consent was waived because of the observational and quality assurance nature of the study.

The total number of patients enrolled in the project was 550, the majority of whom were being managed by the private attending services. Table 1 shows the number of patients and patient days for each service during the study period.

Table 2 shows a summary of the adherence and correction rates to the eligible indicators during the four quarters of implementing the check list. For example, in Quarter 1 the nonteaching service had 4312 eligible indicators, 86.4% of these were checked, which resulted in 13.6% omissions. A subsequent follow up effort corrected 55.1% of these omissions. Consequently, this led to 93.9% of total adherence (i.e. compliance) of non-teaching service in Quarter 1. The data of the other Quarters were calculated similarly for non-teaching, teaching, and total services.

It can be observed from table 2 that both initial adherence and final adherence rates are higher for the teaching than nonteaching services. The reasons for this difference are covered in the discussion section.

A very important observation is the significant impact of correcting the omissions on the final adherence rates for teaching, non-teaching, and the total services. This is evident in the observed improvement in adherence from an initial rate of 88.4% to a final rate of 95.5%. In fact, a very high correlation was

found between correction rates and final adherence rates data with a correlation coefficient r = 0.941 over the four quarters. This indicates that exerting a diligent and careful effort in correcting the missed eligible indicators will lead to higher final adherence rate.

Table 2 and figure 1 also show some degree of improvement in the total rates of initial adherence starting from the third quarter. At no point was the goal of 100% adherence reached during the study. On the other hand, the total rates of corrections showed a progressive increase over the first three quarters. Even though the last quarter showed a drop in the total correction rates (on both services), the final correction rates for that quarter was still higher compared to the first quarter.

The purpose of this project was to embed a checklist care tool within the work flow and operation of a CCU of a large teaching community hospital in order to assure that all cardiac specific performance measures, best practices and evidence-based and required care are applied at the bedside and documented on daily basis. By serving as a scripted reminder to and informing the healthcare providers during daily MDR in regards to their accountable-care, mistakes and omissions were avoided and corrected, respectively. This purposeful mitigation of care omissions and assurance of preventative strategies is especially impactful in critical care units which are especially vulnerable to medical errors and adverse events. The CCU quality audit checklist served as both a trigger to identify and report therapeutic and process of care deficiencies needing immediate correction, and a tool to monitor concurrently the effectiveness of reversing the gaps in care.

The rate of adherence to the desired eligible items on our 20 item checklist had improved from 88.4% to 95.9% in this study. Our goal of achieving total correction of all the identified deficiencies within 24 hours was not achieved, but steady progress was made. The third quarter demonstrated marked improvement in correction rates for both the teaching and private services, as the checklist became more accepted by the team, aligned with the mission and better integrated into the workflow of the unit. Similar degrees in improvement in compliance with intensive care unit best practices were achieved with specialty aligned checklists applied in surgical, trauma and intensive care units [21-37]. In addition, this study also validated that verbal review is an effective means to improve application of CCU best practices as has been shown in other ICU settings [23, 36].

In response to the increased worldwide risk of complications and death associated with surgery, the World Health Organization (WHO) in 2008 developed and published practice guidelines to promote and ensure patient safety [39]. Based on the very favorable experience in the aviation industry with checklists [8], which ensure effective communication and teamwork and that all steps and procedures are followed reliably and consistently during all phases of flight, healthcare derived safety checklists evolved and have been incorporated to improve the quality of care and prevent ICU associated adverse events [21-37]. Tarrago et al reported that by implementing a safety checklist in their pediatric intensive care unit, and supplementing with sending individual monthly reports to physicians about their compliance, they observed an improvement in the rate of adherence to safety measures, a reduction in the cost of care, and better compliance by physicians with standards of care [24]. Many studies (as above) demonstrated the successful implementation and application of checklists in various ICU settings, for both specific purposes and universal safety and quality improvement with favorable outcomes in reducing ventilator acquired pneumonia and catheter-related bloodstream infections and improving compliance with evidence-based intensive care unit practices [21, 22, 25, 33]. Most daily goal directed checklists in medical/surgical ICU settings have been shown to enhance patient safety outcomes via improved team work and interdisciplinary communication between nurses, physicians and pharmacists [24, 29, 30].

In a landmark study, Haynes and his colleagues implemented the 19 item World Health Organization Surgical Safety Checklist designed to improve team communication and ensure the application of consistent care in eight diverse hospitals around the world and demonstrated significant reductions in the rates of inpatient death and complications [10]. Implementing a more comprehensive multidisciplinary surgical safety checklist (SURPASS Collaborative Group) likewise resulted in a significant reduction in surgical complications and mortality in hospitals with a high standard of care in the Netherlands [11], confirming the results of the Haynes study, and further supported the use of preoperative checklists. Van Klei and associates in a retrospective cohort study in a tertiary university hospital, demonstrated a smaller impact on in-hospital mortality with use of the WHO Surgical Safety Checklist, but the effect was strongly related to checklist compliance [12].Using the American College of Surgeons National Surgical Quality Improvement Program Database to validate the use of a comprehensive surgical checklist and a structured team training curriculum, resulted in a significantly decrease in 30 day mortality [13]. In their systematic review of the English literature (with 16 high quality studies identified) of the use, compliance and impact on patient outcomes of the WHO Surgical Safety Checklist and staff perceptions of the checklist, Patel and colleagues concluded that surgical checklists improve patient outcomes, but continual feedback is necessary to maintain high checklist compliance [16]. However, subsequent studies failed to confirm the favorable results with the WHO surgical safety checklist and postoperative adverse events were inconsistently improved [17-20]. Another systematic literature review of 25 studies [19], concluded that complication rates decreased with the use of a checklist (with a lot of heterogeneity and incongruous results) but the tool was more effective in underdeveloped nations.

An often cited study from Canada, found that surgical safety checklists were not associated with significant reductions in postoperative complications or mortality after mandatory rapid cycle implementation and adoption [17]. Data was compared from the 3 months preceding the checklist implementation to data obtained after its mandated use. The benefits of improved communication and teamwork and promotion of a culture of safety did not translate into improvement in outcomes. This study calls into question the impact of mandatory adoption of checklists in healthcare and implies that there are other factors beyond the tool itself that influences clinical outcomes. More recently, using a cluster randomized trial design, the use of checklists and other interventions had no significant effect on quality improvement among critically ill ICU patients [40]. The checklist, which targeted 11 preventative, therapeutic and bundle care processes, was embedded into daily rounds, where goals of care were discussed, and the healthcare providers were reminded to follow through with adherence to the checklist and highlighted goals of care. Inhospital mortality and secondary exploratory clinical outcomes were not improved by the intervention processes.

The inconsistent results of the studies reviewed can be best explained by differences in and limitations of the research design and methodology; development , type, requirements, content, validity and implementation process of the checklist and compliance with intended use; selection bias; heterogeneous patient populations and characteristics; types and number of hospitals in the intervention groups; sources of data and methods assessing effectiveness; limited observation periods; impact of confounding variables and unadjusted comparisons; secular trends and the “Hawthorne Effect”[41]. The conditions, types of patients and healthcare providers targeted may be highly specific or involve multiple care needs and domains across different environments and phases of care with varying complexities and intensities. Few or multiple outcomes were reviewed. Other variables and opportunities include application of risk adjustment; assessment of the intensity, comprehensiveness and reinforcement of team training and ongoing feedback; the role of administrative support; impact of the culture of patient safety within each institution along with shared mission of the rationale and ownership of the checklist; generalization of the results of each individual study; and lack of well-designed and powered randomized controlled trials. All studies highlight the importance of intensive training, teamwork and communication; measuring and providing feedback on outcomes; encouraging a shared mission; reinforcing performance expectations into work flow, and monitoring and assurance of compliance, as predictors of significant effects on outcomes [42-48].

Our study is the first study evaluating the application and impact of a daily quality rounding checklist on increasing adherence to evidence-base cardiac specific therapies and prophylactic measures in a mixed use coronary care unit in which care is coordinated by either a teaching service or private attending. We were able to achieve an improved and incremental level of adherence to the care measures targeted by our rounding checklist, for both the medical residents and private physicians (but less so for the latter), by increasing consideration and application of CCU best practices, and standardizing care amongst varying practice patterns and providing some compensation for the discontinuity of care created by restricted resident work hours and frequent resident turnover. We feel, based on observation, that this was also related to the continuous feedback and reinforcement provided during the MDR in the CCU to the nursing staff, higher expectations placed on the nurse quality technicians and better integration and acceptance of the checklist within the daily CCU work flow. Over time, the majority of the staff and residents rotating through the unit valued the checklist and appreciated the verbal reminders, but we did not evaluate this systematically.

The decrease in the percentage of corrections made in the fourth quarter was disappointing. We attribute this, to the time of the year (holiday season), during which many of the regular CCU nurses whom had already gotten used to the work follow with the checklist in place, took off to spend time with their families, and were substituted by covering nurses pulled from different areas in the hospital. Substitute nurses were not familiar with the project and had to be reoriented and trained. Another reason which contributed to the drop in adherence and correction rates in that final quarter, was that the leadership of the project did not meet on a consistent regular basis (in part because of conflicting schedules, competing priorities and personal challenges). The key role that direct involvement of an endorsement by stakeholders and administrative/medical leadership play in the success of quality improvement projects has been emphasized repeatedly and specifically, the need to meet on a regular basis to review utilization of the checklist and performance metrics and overcome obstacles and provide feedback regarding performance to the front line staff [42-48].

The present day CCU is an advanced management unit with a high-level acuity and complexity of care and each patient has a number of co morbidities beyond the cardiac reason for admission. Medical errors and omissions of care are not uncommon, and a threat of complications exists. In addition, there are rapid staff shift changes and multiple physicians provide input via their consultative recommendations, often without, direct communication or therapeutic synergy. Furthermore, with the advent of restricted resident hours of duty in teaching hospitals, multiple opportunities exist for communication, educational and care handover breakdowns and gaps. Use of a neurointerventional procedural checklist, likewise the first of its kind, resulted in significant improvements in team communication and a significant reduction in total adverse events [37]. Applying a similar process and methodology as in our study, and engaging junior medical staff to identify gaps (during pre-rounding review) in preventative care bundles (despite the use of reminder mnemonics), the use of a simple checklist resulted in more compliant detection and correction of missed prophylactic measures on a daily basis. However, even with the use of the checklist, some indicated therapies were not applied on subsequent days – highlighting that the checklist has to be used and applied reliably, consistently and as intended [49].

Multiple real world challenges and barriers were encountered which provided a great deal of insight on the design, implementation, applicability and impact of our CCU checklist. In addition, our study had a number of limitations.

The major challenge encountered at the beginning of this project was securing buy in to the initiative from the leadership, staff nurses and physicians. The administrative and medical staff leadership was aware of the project and their unwavering commitment and support that the project is a priority needed to be secured upfront to be successful. The Director of research at the time of the study supported the submission of the required quarterly reports and was very engaged. The nursing staff initially perceived (and complained) that implementing the checklist would add significantly to the burden of their work, interfere with their workflow, and distract from their direct patient care responsibilities. In addition, they were not entirely informed regarding the rationale of the study, the intended and desired outcomes related to improved patient quality and safety, and as a result, there was very little motivation initially to participate and fully engage. They did not see the value of the study, especially since they perceived the interference with their workflow. We could have done a better job in explaining the rationale and purpose of the project and the expectations regarding full and complete compliance with the checklist. In order to overcome this barrier, the quality technicians were hired for the project and were responsible for most of the work required of the checklist, and supported their efforts to provide best practice care for their patients. They also constantly reinforced the value of the checklist. The initial paper checklist was developed into an electronic tool to ease the process and for real time reports. As shown, an electronic checklist instead of a paper one, does not add additional significant time needed for completing it, and it reduced the workload and errors when compared to using a paper form [31]. In addition, the nurses and staff were reminded on a continuing basis regarding the importance, rationale and value of the checklist, and its link to improved and safer patient care during the MDR. A reward system was also instituted for excellent performance in order to encourage high performance by nursing staff.

There was also variable engagement by the physicians – the private Attending who viewed the queries and requests for orders to correct the omissions somewhat of a nuisance and their responses were often not timely or reliable, or they “forgot,” again a reflection of lack of consensus, participation, ownership, not being one of their priorities, and institutional challenges regarding a shared mission around culture of safety. There were no readily transparent and tangible incentives for them. In addition, there was no accountability, expectations, or consequences built in; despite this, there was still improved rates of adherence over time, even with the private physicians, a direct result of the perseverance and resilience of the quality technicians to correct the deficiencies. The higher rates of adherence with the teaching services was in part due to their employed status and supervision by the Medical Director of the CCU – but even with this “control,” the teaching teams did not achieve perfect compliance and correction rates, and they too were not entirely reliable in implementing identified deficits, even after a number of reminders. The expectations for direct follow up by the residents on the recommended corrections were frequently emphasized by the lead of this project. The monthly rotational changes of the residents and high nursing staff turnover also was challenging because of the frequent need for orientation and training of the utilization, value and responsibility of the checklist. Having to deal with physician resistance left the nursing staff with a sense that physicians were not partnering with them and contributing to team work. Given the limited scope in which the checklist was applied, the nurses and physicians did not see any relationship between the enhanced adherence rates to the indicators and outcomes in the CCU phase of care. The accountability of the Medical and Nurse Managers and champions could have also been more transparent and robust.

Another barrier and limitation was that the roles and responsibilities of the CCU checklist quality technicians were somewhat finite and limited in scope. They were not empowered or expected to assure that the identified gap in care was corrected beyond their scripted reminders and raising awareness and repeatedly reviewing and auditing until the care deficit was corrected. As pointed out in one of the rationales of this project, “medicine by memory alone is not reliable.” To compensate for this challenge, there needs to be an integration of the checklist into the electronic medical record (EMR); as Tarrago et al demonstrated that embedding the checklist into the EMR led to improved physicians’ compliance and accountability [24]. Another strategy to improve physician compliance would be sending regular compliance reports, as peer pressure has shown to cause positive behavioral changes when it comes to compliance with the checklist [27].

1. Lack of complete buy-in and understanding of the project and appreciation of the potential value of the checklist by all the stakeholders

2. Not having baseline adherence rates for the care indicators pre-checklist implementation to compare performances postchecklist implementation

3. The lack of outcome data linked to the improvement in adherence rates to the care processes

4. The limited set of interventions, focused to a specific patient population during an isolated episode of care in the CCU

5. Lack of a formal process and monitoring of documenting compliance

6. Failure to include a formal mechanism to assess staff and physician satisfaction

7. High turnover in the staff serving as quality technicians with subtle differences in their perception of their roles and responsibilities

8. The intervention was only applied 5 days a week during usual work hours.

9. Need for more frequent reinforcing and feedback in various group and 1:1 venues

10. The use of the patients as their own controls

11. Setting a higher expectation on the quality technicians to monitor compliance and frequent feedback and audits until the gaps in care are corrected

12. Because of some inherent flaws in the project design and data collection, we were unable to apply any meaningful statistical analysis and relegated conclusions regarding the increased adherence rates based solely on observation of directional trends. We should have included a biostatistician from the early stage of study design, with ongoing data analysis and interpretation

13. The impact of compliance with the checklist recommendations and corrected deficiencies on the clinical outcomes in the CCU and after transfer from the CCU was not assessed

The insights derived from this study regarding challenges, barriers and limitations are very similar to the ones summarized above derived from the published literature.

The use and application of a daily CCU checklist by an assigned quality technician can identify and correct deficiencies in desired care. Despite the intensity of the interventions, the enhanced engagement of the nursing and physician staff and the reliable processes put into place, the goal of achieving 100% adherence rates to eligible and appropriate clinical measures was not achieved. This project provided a great deal of insight into the barriers, challenges and limitations that need to be overcome for successful implementation, utilization, value and sustainability of an applied healthcare checklist, to improve intended and desired performance measures and drive favorable outcomes in a real world setting.

From the Cardiology Section and Coronary Care Unit of Beaumont Hospital – Dearborn (formerly Oakwood Hospital). The project was supported by a generous grant from the Blue Cross/Blue Shield of Michigan Foundation. The authors greatly appreciate the incredible support and dedication of the CCU nurses to this project.

1. Institute of Medicine. To ERR is Human. Building a Safer Health System. 2000.
2. Valentin A, Bion J.  How safe is my intensive care unit? An overview of error causation and prevention. Curr Opin Crit Care. 2007;13(6):697-702. DOI: 10.1097/MCC.0b013e3282f12cc8
3. Rothschild JM, Landrigan CP, Cronin JW, Kaushal R, Lockley SW, Burdick E, et al. The Critical Care Safety Study: The incidence and nature of adverse events and serious medical errors in intensive care. Crit Care Med. 2005; 33(8):1694-1700.
4. Sinopoli DJ, Needham DM, Thompson DA, Holzmueller CG, Dorman T, Lubomski LH, et al. Intensive care safety events for medical vs surgical patients: a prospective multicenter study. J Crit Care. 2007; 22(3):177-183. DOI: 10.1016/j.jcrc.2006.11.002
5. McLaughlin N, Leslie GD, Williams TA, Dobb GJ. Examining the occurrence of adverse events within 72 hours of discharge from the intensive care unit. Anaesth Intensive Care. 2007;35(4):486-493
6. Valentine A, Capuzzo M, Guidet B, Moreno RP, Dolanski L, Bauer P, et al. Patient safety in intensive care: Results from the multinational Sentinel Events Evaluation (SEE) Study. Intensive Care Med. 2006;32(10):1591-1598. DOI: 10.1007/s00134-006-0290-7
7. Pronovost PJ, Rinke ML, Emery K, Dennison C, Blackledge C, Berenholtz SM. Interventions to reduce mortality among patients treated in intensive care units. J Crit Care. 2004;19(3):158-164. doi:10.1016/j.jcrc.2004.07.003
8. Eisen LA, Savel RH. What went wrong. Lessons for the intensivist from the crew of US Airways flight 154. Chest. 2009;136:910-917.
9. AHRQ. Agency for Healthcare Research and quality. U.S. Department of Health & Human Services.
10. Haynes AB, Weiser TG, Berry WR, Stuart R. Lipsitz, Abdel-Hadi S. Breizat, E. Patchen Dellinger,  et al. A surgical safety checklist to reduce Morbidity and Mortality in a global population. N Engl J Med. 2009;360:491-499. DOI: 10.1056/NEJMsa0810119
11. De Vries EN, Prins HA, Crolla, Rogier, Adriaan J den Outer, George van Andel, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010;363:1928-1937. DOI: 10.1056/NEJMsa0911535
12. Van Klei WA, Hoff RG, van Aarnhem, Simmermacher RK, Regli LP, Kappen TH, et al. Effects of the introduction of the World Health Organization Surgical Safety Checklist on In-Hospital Mortality. A cohort study. Ann Surg.  2012; 255(1):44-49. doi: 10.1097/SLA.0b013e31823779ae
13. Bliss LA, Ross-Richardson CB, Sanzari LJ, Shapiro DS, Lukianoff AE, Bernstein BA, et al. Thirty day outcomes support implementation of a surgical safety checklist. J Am Coll Surg. 2012;215(6):766-776. doi: 10.1016/j.jamcollsurg.2012.07.015
14. Haugen AS, Søfteland E, Almeland SK, Sevdalis N, Vonen B, Eide GE et al. Effect of the World Health Organization Checklist on patient outcomes. Ann Surg. 2015;261(5):821-828. doi: 10.1097/SLA.0000000000000716
15. Using surgical checklists helped lower length of stay, 90 day mortality, but not 30 day mortality in a small Italian study. JAMA Surgery. 2016.
16. Patel J, Ahmed K, Guru KA, Khan F, Marsh H, Shamim Khan M, et al. An overview of the use and implementation of checklists in surgical specialties – A systematic review. Internat J of Surgery. 2014;12(12):1317-1323. doi: 10.1016/j.ijsu.2014.10.031
17. Urbach DR, Govindarjan A, Saskin R, Andrew S Wilton , Nancy N. Baxter. Introduction of surgical safety checklists in Ontario, Canada. N Eng J Med. 2014;370:1029-1038. DOI: 10.1056/NEJMsa1308261
18. Rice S. One surgical checklist does not fit all, experts say. Modern Healthcare.                   
19. Impact of World Health Organization surgical safety checklists inconsistent; more effective in resource poor settings than in developed ones. World J Surg. 2016.
20. de Jager F, McKenna C, Bartlett L, Gunnarsson R, Ho YH. Postoperative adverse events inconsistently improved by the World Health Organization surgical safety checklist: a systematic literature review of 25 studies. World J Surg. 2016;40(8):1842-1858. Doi: 10.1007/s00268-016-3519-9
21. Anthes E. Hospital checklists are meant to save lives – so why do they often fail. Nature. 2015;523(7562):516-518. DOI: 10.1038/523516a
22. Pronovost P, Needham D, Berenholtz S, Sinopoli D, Chu H, Cosgrove S, et al. An intervention to decrease catheter-related bloodstream infections in the ICU. N Engl J Med. 2006;355(26):2725-2732. DOI: 10.1056/NEJMoa061115
23. Nama A, Sviri S, Abutbul A, Stav I, van Heerden, et al:  Successful introduction of a daily checklist to enhance compliance with accepted standards of care in the medical intensive care unit. Anaesth Intensive Care. 2016;44(4):498-500.
24. Weiss CH, Moazed F, McEvoy CA, Singer BD, Szleifer I, Amaral LA, et al. Prompting physicians to address a daily checklist and process of care and clinical outcomes: a single-site study. Am J Respir Crit Care Med. 2011;184(6):680-686. doi: 10.1164/rccm.201101-0037OC
25. Tarrago R, Nowak JE, Leonard CS , Payne NR. Reductions in invasive devise use and care costs after institution of a daily safety checklist in a pediatric critical care unit. Jt Comm J Qual Patient Saf. 2014;40(6):270-278.
26.  DuBose JJ, Inaba K, Shiflett A, Trankiem C, Teixeira PG, Salim A, et al. Measurable outcomes of quality improvement using a daily quality rounds checklist: one-year analysis in a trauma intensive care unit with sustained ventilator-associated pneumonia reduction. Journal of Trauma-Injury Infection & Critical Care. 2008;64(1):22-29. doi: 10.1097/TA.0b013e318161b0c8
27. Teixeira PG, Inaba K, Dubose J, Melo N, Bass M, Belzberg H, et al.  Measurable outcomes of quality improvement using a daily quality rounds checklist: two-year prospective analysis of sustainability in a surgical intensive care unit. J Trauma Acute Care Surg. 2013;75(4):717-721. doi: 10.1097/TA.0b013e31829d27b6
28. William G Carlos, D Patel G, Vannostrand KM, Gupta S, Cucci AR, Bosslet GT. Intensive care unit rounding checklist implementation. Ann Am Thorac Soc. 2015;12(4):533–538. doi: 10.1513/AnnalsATS.201410-494OC
29. Elsbeth CM. Ten Have, Raoul E. Nap, Jaap E. Tulleken. Measurement Properties and Implementation of a Checklist to Assess Leadership Skills during Interdisciplinary Rounds in the Intensive Care Unit. The Scientific World Journal. 2015. doi:10.1155/2015/951924.
30. Newkirk M, Pamplin JC, Kuwamoto R, Allen DA, Chung KK. Checklists change communication about key elements of patient care. J Trauma Acute Care Surg. 2012;73(1):S75-S82. doi: 10.1097/TA.0b013e3182606239
31. Tarrago R, Nowak J, Lenoard CS, Payne NR. Reductions in Invasive Device Use and Care Costs After Institution of a Daily Safety Checklist in a Pediatric Critical Care Unit. Jt Comm J Qual Patient Saf. 2014;40(6):270-278
32. Centofanti JE, Duan EH,  Hoad NC, Swinton ME, Perri D, Waugh L, et al. Use of a Daily Goals Checklist for Morning ICU Rounds: A Mixed-Mthods Study.  2014;42(8):1797-1803. doi: 10.1097/CCM.0000000000000331
33. Thongprayoon C, Harrison AM, Horo JC, Berrios RA, Pickering BW, Herasevich V.  The effect of an Electronic Checklist on Critical Care Provider Workload, Errors, and Performance. J of Intensive Care Medicine. 2016;31(3):205-212. doi: 10.1177/0885066614558015
34.  Lashoher A, Schneider E, Julliard C, Stevens K, Colantuoni E, Berry WR, et al. Implementation of the World Health Organization Trauma Care Checklist Program in 11 Centers Across Multiple Economic Strata: Effect on Care Process Measures. World J Surg. 2017;41(4):954-962. Doi: 10.1007/s00268-016-3759-8
35. Morris AC, Hay AW, Swann DG, Everingham K, McCulloch C, McNulty J, et al. Reducing Ventilator-Associate Pneumonia in Intensive Care: Impact of Implementing a Care Bundle. Crit Care Med. 2011;39(10):2218-2224. doi: 10.1097/CCM.0b013e3182227d52
36. Adriance SM, Murphy CV. Prophylaxis and treatment of Venous Thromboembolism in the Critically Ill. Int J Crit Illn Inj Sci. 2013;3:143-151. doi: 10.4103/2229-5151.114274
37. Mohebbi l, Hesch K. Stress Ulcer Prophylaxis in the Intensive Care Unit. Proc (Bayl Univ Med Cent). 2009;22(4):373-376.
38. Byrnes MC, Schuerer JE, Schallom, ME, Sona CS, Mazuski JE, Taylor BE, et al. Implementation of a mandatory checklist of protocols and objectives improves compliance with a wide range of evidence-based intensive care unit practices. Crit Care Med. 2009;37(10):2775-2781. doi: 10.1097/CCM.0b013e3181a96379
39. Fargen KM, Velat, GJ, Lawson MF, Lawson MF, Firment CS, Mocco J, Hoh BL. Enhanced staff communication and reduced near miss errors with a neurointerventional procedural checklist. J NeuroIntervent Surg. 2012:1-4.
40. Anderson JC, Rungtusanatham M, Schroeder, RG. A theory of quality management underlying the Deming Management Method. Acad Manag Rev. 1994;19:472-509
41. World Alliance for Patient Safety. World Health Organization guidelines for safe surgery. Geneva: WHO, 2008.
42. Cavalcanti AB, Bozza FA, Machado FR, Salluh JI, Campagnucci VP, Vendramim P. Writing Group for the Checklist-ICU Investigators and the Brazilean Research in Intensive Care Network (BRICNET). JAMA. 2016;315:1480-1490. doi: 10.1001/jama.2016.3463.
43. Mayo E. The human problems of an industrial civilization. New York. Mcmillan, 1933.
44. Simpson SQ, Peterson DA, O’Brien-Ladner AR. Development and Implementation of an ICU Quality Improvement Checklist. AACN Advanced Critical Care. 2007;18(2):183-189. doi: 10.1097/01.AACN.0000269262.37288.bf
45. Putnam LR, Levy SM, Sajid M, Dubuisson DA, Rogers NB, Kao LS, et al: Multifaceted interventions to improve adherence to the surgical checklist. Surgery. 2014;156(2):336-344. doi: 10.1016/j.surg.2014.03.032
46. Pyrek KM. Checklists and Bundles: Patient safety tools require appropriate implementation. 2016.
47. Russ S, Rout S, Caris J, Mansell J, Davies R, Mayer E, et al: Measuring variation in use of the WHO surgical safety checklist in the operating room: A multicenter prospective cross-sectional study. J Amer Coll Surg. 2015;220(1):1-11.e4. doi: 10.1016/j.jamcollsurg.2014.09.021
48. Cahill TJ, Stables R. Surgery: Tailor checklists to clinical teams. Nature. 2015;525:312. doi:10.1038/525321b
49. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: a systemic review of impacts and implementation. BMJ Qual Saf. 2014;23(4):299-318. doi: 10.1136/bmjqs-2012-001797
50. No improvement after statewide surgical checklist effort in Michigan; raises questions regarding checklist efficacy in non research settings. Med Care. 2015;53:87-94.