Impact of a multifaceted and multidisciplinary intervention on pain, agitation and delirium management in a Canadian community intensive care unit: a quality improvement study protocol
========================================================================================================================================================================================

* Mercedes Camargo Penuela
* Madelyn Law
* Han-Oh Chung
* Brent E. Faught
* Jennifer L.Y. Tsang

## Abstract

**Background:** Pain and agitation are closely linked to the development of delirium, which affects 60%–87% of critically ill patients. Delirium is associated with increased mortality and morbidity. Clinical guidelines that suggest routine assessment, treatment and prevention of pain, agitation and delirium (PAD) is crucial to improving patient outcomes. However, the adoption of and adherence to PAD guidelines remain suboptimal, especially in community hospitals. The aim of this quality improvement study is to evaluate the impact of a multifaceted and multidisciplinary intervention on PAD management in a Canadian community intensive care unit (ICU).

**Methods:** This is a quality improvement, uncontrolled, before-and-after study of a multifaceted and multidisciplinary intervention targeting nurses (educational modules, visual reminders), family members (interviews, educational pamphlets and an educational video), physicians (multidisciplinary round script) and the multidisciplinary team as a whole (delirium poster). We will collect data every day for 6 weeks before implementing the intervention. Data collection will include clinical information and information on process of care. We will then implement the intervention. Four weeks after, we will collect data daily for 6 weeks to evaluate the effect of the intervention. On the basis of the volume of the ICU, we expect to enroll approximately 280 patients. We have obtained local ethics approval from the Hamilton Integrated Research Ethics Board (HiREB 18-040-C).

**Interpretation:** The results of this quality improvement study will provide information on adherence to PAD guidelines in a Canadian community ICU setting. They will also supply information on the feasibility of implementing multifaceted and multidisciplinary PAD interventions in community ICUs.

Delirium is a harmful condition commonly encountered in the intensive care unit (ICU).1,2 It can affect 60%–87% of critically ill patients2–6 and it is associated with multiple adverse outcomes such as increased mortality, 1–3,6,7 prolonged hospital length of stay,2,3,6,8 increased health care costs2,4,5,9 and long-term cognitive dysfunction. 5,10–12 Pain and agitation are closely linked to the development of delirium.2 Pain is a common symptom in the ICU; the incidence of moderate to severe pain is up to 50%–80%.13–15 Agitation affects at least 71% of patients in the ICU.16 The causes of pain, agitation and delirium (PAD) are multifactorial.17 In 2013, the Society of Critical Care Medicine published clinical practice guidelines for the management of PAD in adult patients in the ICU, known as the PAD guidelines.18 These guidelines strongly recommended the routine assessment of PAD using validated tools and the treatment and prevention of PAD. The validated tools for the assessment of PAD are the Critical Care Pain Observation Tool (CPOT), the Richmond Agitation-Sedation Scale (RASS) and the Confusion Assessment Method for the ICU (CAM-ICU).18 The optimal screening rates are 4 times per 12-hour shift for pain and agitation and once per 12-hour shift for delirium.18 Although there is well-established evidence demonstrating the advantages of implementing the PAD guidelines, the adoption of the guidelines and adherence to them remain poor,18 especially in community ICUs.19

Barriers to guideline implementation include organizational, professional and personal factors such as lack of training, skills, knowledge and motivation for culture change.20–27 Only 3% of ICU nurses ranked delirium as the most important condition to evaluate; by comparison, 44% of them ranked level of consciousness as the most important, 23% ranked the presence of pain as the most important and 21% ranked improper placement of invasive device as the most important. 28 ICU nurses also reported many barriers to delirium assessment including intubation (38%), the complexity of the tools for assessing delirium (34%) and the inability to complete assessments of delirium in sedated patients (13%).29 Moreover, health care professionals considered delirium to be a complex but nonurgent condition, and there were variable management strategies as a result.30 These observations suggest that health care professionals’ attitudes about delirium management need to change if the barriers to the implementation of the PAD guidelines are to be overcome.

Multiple studies have looked at various interventions to improve the management of PAD, including education,31 monitoring31,32 and sedation quality feedback.31 The educational intervention was associated with a 50% relative reduction in sedation-related adverse event rates,31 whereas pain and delirium monitoring was associated with a decrease in mortality.32 Barnes-Daly and colleagues evaluated the ICU Liberation Collaborative PAD implementation method called the ABCDEF bundle (awakening and breathing coordination, choice of drugs, delirium monitoring and management, early mobility and family engagement) in a community setting. They found that compliance with the ABCDEF bundle was associated with improved survival and more days free of delirium and coma.33 Furthermore, Black and colleagues found that patients who received psychological care from family members demonstrated improved psychological outcomes at 4–12 weeks after critical illness.34 However, only a limited number of studies have investigated the impact of a multifaceted and multidisciplinary intervention on PAD management. Moreover, most of the existing studies were conducted in academic ICUs28,32,35 and thus it is difficult to extrapolate their findings to community ICUs. We therefore set out to conduct a quality improvement study to evaluate the effect of a multifaceted and multidisciplinary intervention to improve the assessment and treatment of PAD in a Canadian community ICU setting. We hypothesize that the implementation of a multidisciplinary intervention codeveloped by front-line health care staff would improve PAD management in a community ICU.

## Methods

### Design

This is a quality improvement study, with an uncontrolled before-and-after design. This study examines the impact of a multifaceted and multidisciplinary intervention targeting nurses, family members and physicians to improve PAD management in a Canadian community ICU.

### Setting

We will conduct this study in the ICU of the St. Catharines Site, Niagara Health, a community hospital. The hospital is located in a medium-sized city in Ontario, Canada. Approximately 90 registered nurses provide ICU care. The centre has 1 level III medical–surgical ICU capable of caring for 14 adult patients in private rooms. The ICU is a closed unit with 24-hour intensivist coverage; the intensivists act as the patients’ primary physician during their ICU stay. The nurse to patient ratio is 1:1 to 1:2. There is a dedicated ICU pharmacist, respiratory therapist, physiotherapist and dietitian. The health care team provides care to general medical, cardiac, respiratory, nephrological, oncological, general surgical, orthopedic and vascular surgical patients. The ICU is a locked unit but without restriction during family visiting hours. There is a nursing policy in place that stipulates that pain is to be assessed using the Numeric Pain Rating Scale (NPRS)36–38 at the beginning of each shift, after analgesic administration every 4 hours and as needed. Sedation level is assessed using the RASS39 at the beginning of each shift and as needed, and delirium is assessed using the CAM-ICU.40 Nurses are trained to use these validated tools during their orientation when they join the ICU team. The ICU nurses are responsible for titrating the doses and frequency of analgesics and sedatives according to patients’ clinical status. There is no preexisting protocol for medication titration.

### Eligibility criteria and sample size calculation

All adult patients (aged 18 yr and above) admitted to the ICU for more than 24 hours will be included in this study. There are no exclusion criteria for this study, as the PAD guidelines can be applied to all adult patients in the ICU. This study will incorporate nonprobability consecutive sampling. Using a priori determination, we calculated the sample size for this study using a 95% confidence interval.41 On the basis of a previously conducted nurse-focused quality improvement study, expected differences in pain (8.2%), agitation (14.4%) and delirium (14.8%) management are anticipated (Carolyn Tan, Mercedes Camargo, Franziska Miller, et al. Niagara Health: unpublished data, 2019). Using a power of 80% (Zβ = 0.20), we calculated that a minimum sample size of 277 patients is required. As a conservative estimate, we plan to include at least 280 patients.

### Preintervention data collection

A dedicated research assistant will collect prospective data daily on all admitted ICU patients for 6 weeks (Appendix 1 available at [www.cmajopen.ca/content/7/2/E430/suppl/DC1](http://www.cmajopen.ca/content/7/2/E430/suppl/DC1)). On the basis of the volume of this ICU, the estimated number of patients enrolled in this period will be approximately 140. See the data analysis section for the details of the data collection.

### Intervention

#### PAD Advisory Committee

The intervention for this project is multifaceted and multidisciplinary. It focuses on the social and medical needs of the patients in the ICU in relation to the PAD guidelines.18 The intervention targets 3 groups: nurses, family members and physicians. To facilitate the development of the intervention, we formed a PAD Advisory Committee. It comprises the ICU manager, 5 nurses, 2 physicians (intensivists), an ICU pharmacist and the ICU research coordinator. In addition to informing the development of our intervention, the members of the PAD Advisory Committee will act as local champions to engage, motivate and support front-line health care staff during the implementation of the intervention. More importantly, the PAD Advisory Committee will cultivate positive relationships with end users of the program to enhance PAD protocol adherence, obtain feedback from front-line staff and stimulate collaborative practice among front-line staff. We will hold regular meetings with the PAD Advisory Committee to ensure that we receive regular feedback on the progress of the study, monitor the implementation of the PAD intervention and make changes according to their feedback.

### Development of intervention

We designed and developed the intervention over a 1-year period (September 2017 to August 2018) using the Model for Improvement to allow for the development and refinement of the intervention through plan–do–study–act test cycles42–44 to optimize adherence. Specifically, the PAD Advisory Committee designed and developed the nurse-focused components (educational modules and visual reminders), the family member focused components (interviews, educational pamphlet and educational video), the physician-focused component (multidisciplinary round script) and the multidisciplinary-focused component (delirium poster). We circulated the interventional materials among front-line health care staff to receive feedback and we have iteratively refined the contents and the layout of the interventional materials to ensure optimal adoption by front-line staff during implementation. We developed and refined the interview guide for the family member interviews through plan–do–study–act test cycles by conducting interviews with family members and iteratively revising the interview guide.

### Nurse-focused components

#### Educational modules

We will implement an online educational program for all ICU nurses with 4 modules that were developed using the 2013 PAD guidelines.18 The first module is a PAD program overview. This module will include information on our local PAD research program, postintensive care syndrome and basic pharmacology and pharmacokinetic properties of common ICU drugs in critically ill patients. The second module is the pain module. This module will introduce the CPOT.18 It will also provide nurses with information on the pharmacology of commonly used nonopioid and opioid analgesics in the ICU. The third module is the agitation module. This module will introduce the RASS.39 It will also provide nurses with information on the pharmacology of commonly used sedatives. The fourth module is the delirium module. This module will give an overview on delirium and will introduce the CAM-ICU40 as a validated tool to screen for delirium in the ICU. It will also provide nurses with information on pharmacological and nonpharmacological treatment of delirium.

#### Visual reminders

We will place CPOT, RASS and CAM-ICU cue cards by each bedside to remind front-line nurses how to properly use these validated tools to screen for pain, agitation and delirium, respectively (Appendix 2 available at [www.cmajopen.ca/content/7/2/E430/suppl/DC1](http://www.cmajopen.ca/content/7/2/E430/suppl/DC1)).

### Family member focused components

#### Interviews

A team of PAD volunteers (undergraduate students) will conduct in-person interviews (approximately 20 minutes in duration) with family members of all newly admitted ICU patients within 48–72 hours of admission. We will obtain consent before all interviews. The purpose of this intervention is to empower family members to participate in the PAD care of the patients by providing us with important information about patients’ baseline cognitive function, mobility and use of visual and hearing aids (Appendix 3 available at [www.cmajopen.ca/content/7/2/E430/suppl/DC1](http://www.cmajopen.ca/content/7/2/E430/suppl/DC1)).

#### Educational pamphlet

We will provide an educational pamphlet (developed with permission from [www.icudelirium.org](http://www.icudelirium.org)) in the ICU waiting room for family members. The purpose of this component is to provide educational materials to family members about delirium (Appendix 4 available at [www.cmajopen.ca/content/7/2/E430/suppl/DC1](http://www.cmajopen.ca/content/7/2/E430/suppl/DC1)).

#### Educational video

An educational video on delirium (produced by Osmosis) will be made available on a dedicated computer in the ICU waiting room for family members. The purpose of this component is to provide information on delirium to family members through a different educational medium.

### Physician-focused component

#### Multidisciplinary round script

We will post a script (Appendix 5 available at [www.cmajopen.ca/content/7/2/E430/suppl/DC1](http://www.cmajopen.ca/content/7/2/E430/suppl/DC1)) for use during multidisciplinary rounds on the workstation on wheels as a reminder to intensivists to order target RASS score, to discuss PAD assessment and treatment and to encourage nurses to achieve adequate pain control and light sedation.

### Multidisciplinary-focused component

#### Poster

We will post our delirium poster in the ICU to remind all front-line multidisciplinary health care staff, patients and family members about the importance of detection, treatment and prevention of delirium (Appendix 6 available at [www.cmajopen.ca/content/7/2/E430/suppl/DC1](http://www.cmajopen.ca/content/7/2/E430/suppl/DC1)).

### Postintervention data collection

Four weeks after the implementation of the intervention, a dedicated research assistant will collect data on all admitted ICU patients each day for 6 weeks. On the basis of the volume of this ICU, we estimated that approximately 140 patients will be enrolled in this period.

### Data analysis

We will analyze quantitative data with descriptive and analytical statistics, using SPSS version 26. We will examine numerical data using means and standard deviations or medians and 25th to 75th percentiles according to data distribution. We will use paired *t* tests for parametric analyses and Wilcoxon signed-rank paired difference tests for nonparametric analyses comparing before-and-after intervention data of 2 related samples. As our objective is to evaluate the adherence to guideline recommendations on pain, agitation and delirium assessment by nurses per day, we will measure their adherence to our intervention in patient-day units. The level of significance is set at α = 0.05.

Our primary outcomes will be (a) the proportion of patient-days with pain assessment using the NPRS (for patients who can verbally report pain) or the CPOT (for patients who cannot verbally report pain) at least 4 times per shift, (b) the proportion of patient-days with agitation assessment using the RASS at least 4 times per shift and (c) the proportion of patient-days with delirium assessment using the CAM-ICU at least once per shift.

Our secondary outcomes will be (a) the average number per patient-day of pain assessments using the NPRS (for patients who can verbally report pain) or the CPOT (for patient who cannot verbally report pain), (b) the average number per patient-day of agitation assessments using the RASS, (c) the average number per patient-day of delirium assessments using the CAM-ICU, (d) the proportion of patient-days with benzodiazepine use, (e) the proportion of patient-days with significant pain defined by NPRS scores of 4 or higher or CPOT scores of 3 or higher, (f) the proportion of patient-days with optimal sedation level defined by a RASS score of between −2 and 0 or a target RASS score at least 50% of the time, (g) the proportion of patient-days with oversedation defined by a RASS score less than −2 at least 50% of the time, (h) the proportion of patient-days with agitation defined by a RASS score greater than 0 at least 50% of the time and (i) daily percent patient-days with a positive delirium screen using the CAM-ICU.

Our balancing measures will include the proportion of patient-days with physical restraint use and inadvertent extubation. Our control measures will include the proportion of patient-days during which mechanically ventilated patients are on stress ulcer prophylaxis and the proportion of patient-days during which pharmacological or mechanical prophylaxis for deep vein thrombosis is used.

### Ethics approval

We have obtained local ethics approval from the Hamilton Integrated Research Ethics Board (HiREB 18-040-C), with a waiver of the need to obtain consent as this is a quality improvement study. All data will be anonymized and stored in password-protected computers in a locked research office.

### Knowledge translation

We will present our data at the annual scientific meeting of the European Society of Intensive Care Medicine and the Critical Care Canada Forum. We will also publish the results of this study in peer-reviewed academic journals. Should this multifaceted and multidisciplinary intervention be effective in improving PAD management, we will implement it in other Canadian community ICUs.

## Interpretation

This quality improvement study will report on the baseline rate at which target RASS scores were ordered by physicians; the rate of PAD screening, the rate of optimal pain and sedation management and the rate of delirium documented by bedside nurses in a Canadian community ICU. We selected these process and outcome metrics on the basis of the recommendations of the 2013 PAD guidelines.18 This study will also report on the impact of a multifaceted and multidisciplinary intervention on PAD management as reflected by the process and outcome metrics described above. The results of this study will provide information on the feasibility of implementing multifaceted and multidisciplinary PAD interventions in community ICUs.

### Limitations

The uncontrolled before-and-after study design may overestimate the effect of the intervention. However, this study design represents a pragmatic approach and reflects real-life experience in a Canadian community ICU setting, where research infrastructure is generally lacking. We chose this study design because it allowed us to conduct a quality improvement study with the greatest possible degree of scientific rigour given the limitations of a community ICU. Lastly, because resources are limited in the community ICU setting, we will not collect information on patient demographics that would allow for adjustment for confounders (e.g., severity of illness, age, history of dementia, delirium, substance use). However, our primary outcomes should not be affected by any major patient-related confounders because according to the PAD guidelines all ICU patients, regardless of severity of illness, should have regular PAD assessments.

### Conclusion

This uncontrolled before-and-after study in a Canadian community ICU proposes to determine the effect of a multifaceted and multidisciplinary intervention on the management of PAD.

## Acknowledgements

The authors would like to thank Cynthia Wan, Zac Franklin and the PAD Advisory Committee (Elayn Young, Dimitra Fleming, Francie Davidson, Paige Gehrke and Anna Plant) for their contribution to the development of the intervention.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** Mercedes Camargo Penuela, Madelyn Law and Jennifer Tsang conceived the idea and drafted the protocol. Brent Faught assisted in the design of the study and developed the statistical analysis plan. Mercedes Camargo Penuela, Madelyn Law, Jennifer Tsang and Han-Oh Chung revised the protocol. Mercedes Camargo Penuela drafted the manuscript and Madelyn Law, Jennifer Tsang, Han-Oh Chung and Brent Faught revised it for important intellectual content. All authors approved the final version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** This study is funded by Physicians’ Services Incorporated (PSI) Foundation and Medbuy.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/7/2/E430/suppl/DC1](http://www.cmajopen.ca/content/7/2/E430/suppl/DC1).

## References

1.  Ely EW, Shintani A, Truman B, et al. (2004) Delirium as a predictor of mortality in mechanically ventilated patients in the intensive care unit. JAMA 291:1753–62.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.291.14.1753&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15082703&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000220788700033&link_type=ISI) 

2.  Limpawattana P, Panitchote A, Tangvoraphonkchai K, et al. (2016) Delirium in critical care: a study of incidence, prevalence, and associated factors in the tertiary care hospital of older Thai adults. Aging Ment Health 20:74–80.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1080/13607863.2015.1035695&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25902330&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

3.  Ouimet S, Kavanagh BP, Gottfried SB, et al. (2007) Incidence, risk factors and consequences of ICU delirium. Intensive Care Med 33:66–73.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s00134-006-0399-8&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17102966&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000243464300012&link_type=ISI) 

4.  Pisani MA, Murphy TE, Van Ness PH, et al. (2007) Characteristics associated with delirium in older patients in a medical intensive care unit. Arch Intern Med 167:1629–34.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/archinte.167.15.1629&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17698685&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000248694500008&link_type=ISI) 

5.  Pandharipande P, Cotton BA, Shintani A, et al. (2008) Prevalence and risk factors for development of delirium in surgical and trauma intensive care unit patients. J Trauma 65:34–41.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/TA.0b013e31814b2c4d&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18580517&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000257767300006&link_type=ISI) 

6.  Girard TD, Pandharipande PP, Ely EW (2008) Delirium in the intensive care unit. Crit Care 12(Suppl 3):S3.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/cc6149&link_type=DOI) 

7.  Lin SM, Liu CY, Wang CH, et al. (2004) The impact of delirium on the survival of mechanically ventilated patients. Crit Care Med 32:2254–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/01.CCM.0000145587.16421.BB&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15640638&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000225211400011&link_type=ISI) 

8.  Ely EW, Gautam S, Margolin R, et al. (2001) The impact of delirium in the intensive care unit on hospital length of stay. Intensive Care Med 27:1892–900.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s00134-001-1132-2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11797025&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000173199800010&link_type=ISI) 

9.  Milbrandt EB, Deppen S, Harrison PL, et al. (2004) Costs associated with delirium mechanically ventilated patients. Crit Care Med 32:955–62.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/01.CCM.0000119429.16055.92&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15071384&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000220872700008&link_type=ISI) 

10. Girard TD, Jackson JC, Pandharipande PP, et al. (2010) Delirium as a predictor of long-term cognitive impairment in survivors of critical illness. Crit Care Med 38:1513–20.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/CCM.0b013e3181e47be1&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20473145&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000278920000001&link_type=ISI) 

11. Dubois MJ, Bergeron N, Dumont M, et al. (2001) Delirium in an intensive care unit: a study of risk factors. Intensive Care Med 27:1297–304.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s001340101017&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11511942&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000170824100010&link_type=ISI) 

12. McAvay GJ, Van Ness PH, Bogardus ST Jr., et al. (2006) Older adults discharged from the hospital with delirium: 1-year outcomes. J Am Geriatr Soc 54:1245–50.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1532-5415.2006.00815.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16913993&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000239961400013&link_type=ISI) 

13. Adam VN, Matolic M, Ilic MK, et al. (2015) Pain management in critically ill patients. Period Biol 117:225–30.
    
    

14. de Jong A, Molinari N, de Lattre S, et al. (2013) Decreasing severe pain and serious adverse events while moving intensive care unit patients: a prospective interventional study (the NURSE-DO project). Crit Care 17:R74.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/cc12683&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23597243&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

15. Alderson SM, Mckechnie SR (2014) Unrecognised, undertreated, pain in ICU: causes, effects, and how to do better. Open J Nurs 3:108–13.
    
    

16. Siegel MD (2003) Management of agitation in the intensive care unit. Clin Chest Med 24:713–25.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0272-5231(03)00104-7&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=14710699&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

17. Fong TG, Tulebaev SR, Inouye SH (2009) Delirium in elderly adults: diagnosis, prevention and treatment. Nat Rev Neurol 5:210–20.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1038/nrneurol.2009.24&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19347026&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000265854600008&link_type=ISI) 

18. Barr J, Fraser GL, Puntillo K, et al. (2013) Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit Care Med 41:263–306.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/CCM.0b013e3182783b72&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23269131&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000313150300029&link_type=ISI) 

19. Devlin JW, Marquis F, Riker RR, et al. (2008) Combined didactic and scenario-based education improves the ability of intensive care unit staff to recognize delirium at the bedside. Crit Care 12:R19.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/cc6793&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18291021&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

20. Brotons C, Lobos JM, Royo-Bordonada MA, et al. (2013) Implementation of Spanish adaptation of the European guidelines on cardiovascular disease prevention in primary care. BMC Fam Pract 14:36–42.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2296-14-36&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23506390&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

21. Abrahamson KA, Fox RL, Doebbeling BN (2012) Facilitators and barriers to clinical practice guideline use among nurses. Am J Nurs 112:26–35.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/01.NAJ.0000418920.45600.7a&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22546733&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

22. Jun J, Kovner CT, Stimpfel AW (2016) Barriers and facilitators of nurses’ use of clinical practice guidelines: an integrative review. Int J Nurs Stud 60:54–68.
    
    

23. Ploeg J, Davies B, Edwards N, et al. (2007) Factors influencing best-practice guideline implementation: lessons learned from administrators, nursing staff, and project leaders. Worldviews Evid Based Nurs 4:210–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1741-6787.2007.00106.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18076464&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000251436100005&link_type=ISI) 

24. Ferrante D, Konfino J, Linetzky B, et al. (2013) Barriers to prevention of cardiovascular disease in primary care settings in Argentina. Rev Panam Salud Publica 33:259–66.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1590/S1020-49892013000400004&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23698174&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

25. Hobbs FD, Erhardt L (2002) Acceptance of guideline recommendations and perceived implementation of coronary heart disease prevention among primary care physicians in five European countries: the Reassessing European Attitudes about Cardiovascular Treatment (REACT) survey. Fam Pract 19:596–604.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/fampra/19.6.596&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12429661&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000179252500006&link_type=ISI) 

26. Erhardt L, Komajda M, Hobbs FD, et al. (2008) Cardiologists’ awareness and perceptions of guidelines for chronic heart failure. The ADDress your Heart survey. Eur J Heart Fail 10:1020–5.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.ejheart.2008.08.001&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18793869&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000260749400007&link_type=ISI) 

27. Hadely KA, Power E, O’Halloran R (2014) Speech pathologists’ experiences with stroke clinical practice guidelines and the barriers and facilitators influencing their use: a national descriptive study. BMC Health Serv Res 14:110.
    
    

28. Devlin JW, Fong JJ, Howard EP, et al. (2008) Assessment of the delirium in the intensive care unit: nursing practices and perceptions. Am J Crit Care 17:555–65.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpjYyI7czo1OiJyZXNpZCI7czo4OiIxNy82LzU1NSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNy8yL0U0MzAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

29. Kitson AL, Rycroft-Malone J, Harvey G, et al. (2008) Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges. Implement Sci 3:1–12.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1748-5908-3-1&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18179688&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

30. Palacios-Ceña D, Cachón-Pérez JM, Martínez-Piedrola R, et al. (2016) How do doctors and nurses manage delirium in intensive care units? A qualitative study using focus groups. BMJ Open 6:e009678.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiNi8xL2UwMDk2NzgiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzcvMi9FNDMwLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

31. Walsh TS, Kydonaki K, Antonelli J, et al. (2016) Staff education, regular sedation and analgesia quality feedback, and a sedation monitoring technology for improving sedation and analgesia quality for critically ill, mechanically ventilated patients: a cluster randomised trial. Lancet Respir Med 4:807–17.
    
    

32. Radtke FM, Heymann A, Franck M, et al. (2012) How to implement monitoring tools for sedation, pain and delirium in the intensive care unit: an experimental cohort study. Intensive Care Med 38:1974–81.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s00134-012-2658-1&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22945432&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000311025600008&link_type=ISI) 

33. Barnes-Daly MA, Phillips G, Ely EW (2017) Improving hospital survival and reducing brain dysfunction at seven California community hospitals: implementing PAD guidelines via the ABCDEF bundle in 6064 patients. Crit Care Med 45:171–8.
    
    

34. Black P, Boore J, Parahoo K (2011) The effect of nurse-facilitated family participation in the psychological care of the critically ill patient. J Adv Nurs 67:1091–101.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1365-2648.2010.05558.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21214624&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

35. Pun BT, Gordon SM, Peterson JF, et al. (2005) Large-scale implementation of sedation and delirium monitoring in the intensive care unit: a report from two medical centers. Crit Care Med 33:1199–205.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/01.CCM.0000166867.78320.AC&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15942331&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000230064100002&link_type=ISI) 

36. Breivik H, Borchgrevink PC, Allen SM, et al. (2008) Assessment of pain. Br J Anaesth 101:17–24.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/bja/aen103&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18487245&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000256757700004&link_type=ISI) 

37. Chanques G, Viel E, Constantin JM, et al. (2010) The measurement of pain in intensive care unit: comparison of 5 self-report intensity scales. Pain 151:711–21.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.pain.2010.08.039&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20843604&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000283657300024&link_type=ISI) 

38. Rahu MA, Grap MJ, Ferguson P, et al. (2015) Validity and sensitivity of 6 pain scales in critically ill, intubated adults. Am J Crit Care 24:514–23.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYWpjYyI7czo1OiJyZXNpZCI7czo4OiIyNC82LzUxNCI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNy8yL0U0MzAuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

39. Sessler CN, Grap MJ, Ramsay MA (2008) Evaluating and monitoring analgesia and sedation in the intensive care unit. Crit Care (Suppl 3):S2.
    
    

40. Gusmao-Flores D, Salluh JI, Chalhub RA, et al. (2012) The confusion assessment method for the intensive care unit (CAM-ICU) and intensive care delirium screening checklist (ICDSC) for the diagnosis of delirium: a systematic review and meta-analysis of clinical studies. Crit Care 16:R115.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/cc11407&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22759376&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 

41. Hirsch RP, Riegelman RK (1992) Statistical first aid: interpretation of health research data (Blackwell Scientific Publications, Boston).
    
    

42. Coury J, Schenider J, Rivelli JS, et al. (2017) Applying the plan-do-study-act (PDSA) approach to a large pragmatic study involving safety net clinics. BMC Health Serv Res 17:411.
    
    

43. Greene SM, Reid RJ, Larson EB (2012) Implementing the learning health system: from concept to action. Ann Intern Med 157:207–10.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-157-3-201208070-00012&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22868839&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000307789900008&link_type=ISI) 

44. Luce BR, Kramer JM, Goodman SN, et al. (2009) Rethinking randomized clinical trials for comparative effectiveness research: the need for transformational change. Ann Intern Med 151:206–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-151-3-200908040-00126&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19567619&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE430.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000268947800008&link_type=ISI) 

*   Copyright 2019, Joule Inc. or its licensors