An online educational module on transfusion safety and appropriateness for resident physicians: a controlled before–after quality-improvement study
=====================================================================================================================================================

* Todd C. Lee
* Jennifer Murray
* Emily G. McDonald

## Abstract

**Background:** Several professional societies have made value-based statements in support of restrictive transfusion strategies. The aim of this study was to determine whether completion of an accredited online training program in transfusion safety could improve transfusion knowledge among medical residents and increase transfusion appropriateness.

**Methods:** We performed a controlled before–after evaluation of a mandatory accredited self-directed training program (Bloody Easy Lite for Physicians) that provides education about transfusion medicine on a 47-bed medical clinical teaching unit at a university-affiliated hospital centre in Montréal. The program consists of 2 modules and takes about 30 minutes to complete. We used the 45-bed medical teaching unit at another Montréal hospital as a contemporary control. We compared resident physicians’ pre- and posttest scores and evaluated the impact on transfusion appropriateness by comparing the proportion occurring below a hemoglobin concentration of 80 g/L before (April 2013–June 2015) and after (July 2015–January 2016) the intervention.

**Results:** Of the 55 residents on the intervention unit, 53 (96%) completed the training. The median pretest score was 50% (inter-quartile range [IQR] 40%–60%). The median posttest score was 90% (IQR 80%–90%) for module 1 and 80% (IQR 80%–90%) for module 2 (*p* < 0.001 for both pre–post comparisons). The proportion of transfusions below 80 g/L increased from 80.1% to 86.9% (*p* = 0.04) on the intervention unit and remained relatively unchanged on the control unit (75.6% v. 71.1%, *p* = 0.4). Although there was no statistically significant difference between the units in the proportion of transfusions below 80 g/L before the intervention (*p* = 0.07), a significant difference was observed after the intervention (*p* = 0.002).

**Interpretation:** Mandatory training in transfusion safety via an online program resulted in improved transfusion knowledge among residents and an increase in the proportion of transfusions occurring at a hemoglobin concentration below 80 g/L. This low-cost educational initiative may improve transfusion appropriateness.

The Choosing Wisely campaign contains more than 300 recommendations from specialty societies addressing the overuse of common tests and procedures. The lists were created to improve health care through the avoidance of low-value practices.1 The campaign has evolved in order to place a stronger emphasis on implementation with the development of toolkits and the evaluation of clinically meaningful interventions.2 An important recommendation shared by several professional societies is the avoidance of transfusions in patients in stable condition for arbitrary hemoglobin values.3 In most circumstances (including trauma, critical care, anemia in patients with gastrointestinal bleeding whose condition is stable, in the postoperative setting and for medical inpatients in stable condition), restrictive transfusion practices have been shown to be as safe as or safer than liberal transfusions.4–8 An updated Cochrane review comparing liberal versus restrictive transfusion strategies showed no increase in mortality or other outcomes (e.g., myocardial ischemia, length of stay, stroke, thromboembolism or infection) with a restrictive strategy, and such a strategy decreased the risk of receiving a blood transfusion by 43% across a wide range of clinical scenarios.8 In addition, a restrictive strategy may prevent adverse events related to transfusion that are likely underreported outside of clinical trials,9 packed red blood cells are expensive and indirectly cost about $1200 per unit transfused,10 blood products can be a limited resource in smaller centres, and packed red blood cells are often overused.8,11–13

At our centre, residents receive teaching about transfusion reactions, but the curriculum does not formally address restrictive transfusion strategies. The aim of the current study was to determine whether the universal mandatory completion of a freely available, accredited, online educational module (Bloody Easy Lite for Physicians14) could lead to improved trainee knowledge of transfusion safety (measured through pre- and posttest results) and consequently increase transfusion appropriateness (measured by the proportion of transfusions occurring at or below a hemoglobin threshold of 80 g/L).

## Methods

### Study sites

This study was performed on the medical clinical teaching units of the Royal Victoria Hospital (47 beds), where one of the authors (T.C.L.) was the medical director of the unit at that time. As a comparator, we used the 45-bed medical teaching unit at the Montreal General Hospital as a contemporary control. These 2 hospitals belong to the McGill University Health Centre, an 832-bed tertiary care centre in Montréal. Both medical units admit predominantly acutely ill patients who do not require surgery, acute chemotherapy, critical care or specialized cardiac care via the emergency department. The 2 units care for a similar proportion of patients with coronary artery disease, congestive heart failure and gastrointestinal hemorrhage. On both units, most transfusions are ordered by the senior resident, with variable direct staff oversight, and the blood bank is not involved in determining the appropriateness of transfusions.

### Staffing of clinical teaching units

On the intervention unit, there were 55 residents who rotated through the unit, all of whom were included in the intervention. There were also 15 faculty physicians, who were invited to complete the training modules. At the control site, there were 50 residents, including 13 first-year residents who crossed between sites. Seventeen staff physicians attended on the control unit.

### Intervention

At baseline, all resident physicians in the first 3 years of the internal medicine core residency program receive a 1-hour session, given every 2 years, on the management of transfusion reactions; however, this session does not emphasize transfusion appropriateness. Beginning on July 1, 2015, in addition to the standard 1-hour session, resident physicians who would be working on the intervention unit were required to complete an online training program called Bloody Easy Lite for Physicians (freely available after registration from the Ontario Regional Blood Coordinating Network website [[http://belite.transfusionontario.org/](http://belite.transfusionontario.org/)]). Program completion was required before the end of the first week of the first rotation on the unit. We used administrative support to send the residents up to 2 email reminders during their first week on the teaching unit. Residents were asked to email the administrator once their training was complete. On the intervention unit, medical students were also encouraged to complete the program, but because they cannot order blood products, they were not required to participate. Faculty members attending on the intervention unit were also invited to complete the program.

Bloody Easy Lite for Physicians is a standardized, self-directed learning program accredited by the Royal College of Physicians and Surgeons of Canada that provides education about transfusion medicine. Sample screenshots are presented in Appendix 1 (available at [www.cmajopen.ca/content/7/3/E492/suppl/DC1](http://www.cmajopen.ca/content/7/3/E492/suppl/DC1)). Trainees can complete the program on their own time, including from home. The learning program consists of 2 modules, both of which include a premodule knowledge test. Module 1 focuses on the indications for blood transfusion and contains subcategories related to pretransfusion testing and transfusion of red blood cells, plasma and cryoprecipitate. Specifically related to red blood cells, there are sections on transfusion basics, managing acute blood loss, anemia in inpatients and transfusion for perioperative patients. Trials that are referenced include the Transfusion Requirements in Critical Care trial6 and the Transfusion Trigger Trial for Functional Outcomes in Cardiovascular Patients Undergoing Surgical Hip Fracture Repair.5 Module 2 focuses on the diagnosis and management of transfusion reactions. Both modules take about 30 minutes to complete and are followed by a test and a course evaluation. The entire training can be completed in a single session. Posttests consist of 10 multiple-choice questions and can be completed immediately after the module or at a later time, at the learner’s discretion. Pre- and posttesting allow for objective measurement of improvement in knowledge, although the posttest can be retaken multiple times. Once the learner has successfully passed (score ≥ 80% on both tests), he or she can print a certificate of completion.

### Data sources

We obtained transfusion data from the blood bank through TRACE LINE (MAK-SYSTEM International Group), a software package designed for large hospital transfusion services and used in over 80 hospitals in Quebec. Individual transfusions were manually cross-referenced with corresponding pretransfusion hemoglobin values through our electronic medical record, Oacis (Telus Health). The earliest data available were from Apr. 1, 2013. The most recent hemoglobin value up to 48 hours before transfusion was extracted and used in the adjudication of the primary outcome. In the rare cases in which the hemoglobin concentration was not measured between 2 or more transfused units, we inferred the subsequent pretransfusion hemoglobin concentration by adding 10 g/L to the initial pretransfusion value for each unit transfused.

### Outcomes

Based on high-quality evidence, the Clinical Practice Guideline on red blood cell transfusion from the American Association of Blood Banks strongly recommends “adhering to a restrictive transfusion strategy (70 to 80 g/L) in hospitalized, stable patients.”4 In conjunction with Choosing Wisely Canada15 and the Ontario Regional Blood Coordinating Network, we decided on a value of 80 g/L or more as being a reasonable indicator of potentially inappropriate transfusions. Thus, although 80 g/L does not imply a clinical threshold for transfusion, we used this value for detecting potentially inappropriate transfusions. Secondary outcomes were the rates of transfusion per 100 admissions and per 1000 patient days. Tertiary outcomes were trainee pre- and posttest scores on the learning module.

### Statistical analysis

Based on a previous audit, 70%–75% of transfusions on the medical units were occurring at a hemoglobin threshold of less than 80 g/L (T.C.L. and E.G.M., unpublished observations, 2013). In contrast, based on a 2014 audit of packed red blood cell transfusion conducted by the Ontario Regional Blood Coordinating Network,15 a comparable tertiary care hospital in Ontario was able to achieve 80% of transfusions below a threshold of 80 g/L. To have 80% power to show an absolute difference of 8% in the proportion of transfusions occurring below a hemoglobin threshold of 80 g/L, we calculated that we would need about 125 transfusions. Given an average of 25 transfusions per month, we estimated this would take 6 months and allowed an extra month in case rates were lower than expected. We therefore compared outcomes in the period before (Apr. 1, 2013, to June 30, 2015) and after (July 1, 2015, to Jan. 31, 2016) the intervention. We compared proportions using the χ2 test and compared transfusion rates using a *Z*-test of the summary rate difference using inverse variance weights. We compared anonymized trainee pre- and posttest performance using the paired-sample Wilcoxon test.

### Ethics approval

This study was approved by the McGill University Health Centre Research Ethics Board.

## Results

Of the 55 resident physicians who worked on the intervention units, 53 (96%) successfully completed the training. One resident switched to a different program, and 1 resident was on a medical leave of absence during the intervention. Most residents required 1–2 email reminders to complete the training on time. Four (27%) of the 15 faculty physicians completed the training. No faculty physician or senior resident at the control site completed the training; however, 13 first-year residents at the control site did 1-month rotations at the intervention site at various times during the study (Figure 1). The median pretest score was 50% (interquartile range [IQR] 40%–60%). The median posttest score was 90% (IQR 80%–90%) for module 1 and 80% (IQR 80%–90%) for module 2 (*p* < 0.001 for both pre–post comparisons).

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/7/3/E492/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/7/3/E492/F1)

Figure 1: 
Flow diagram showing numbers of residents at the control and intervention sites.

A total of 1410 units of red blood cells were transfused over the entire study period. Two patients who received 4 or more units of packed red blood cells within a single day for major bleeding were excluded from the analysis. The pre- and postintervention data for the intervention and control units are shown in Table 1. On the intervention unit, the appropriateness of transfusions improved, with the proportion of transfusions with a hemoglobin concentration below 80 g/L increasing from 80.1% to 86.9% (*p* = 0.04), an absolute improvement of 6.8%. On the control unit, the proportion remained similar (75.6% v. 71.1%, *p* = 0.4). Before the intervention, there was no statistically significant difference in the proportion of transfusions occurring with a hemoglobin concentration below 80 g/L between the intervention and control units (*p* = 0.07); however, following the intervention, there was a significant difference (*p* = 0.002).

View this table:
[Table 1:](http://www.cmajopen.ca/content/7/3/E492/T1)

Table 1: 
Proportion of transfusions below a hemoglobin threshold of 80 g/L and population-based transfusion rates on the control and intervention units before and after the training intervention

In contrast, both population-based transfusion rates were significantly lower at both times on the control unit than on the intervention unit, and the number of transfusions per 1000 patient days and per 100 admissions decreased significantly over time on the control unit (*p* < 0.001) but not on the intervention unit (*p* = 0.4 and 0.2, respectively) (Table 1).

## Interpretation

In this pragmatic before–after study conducted on our medical clinical teaching units, the mandatory completion by residents of a self-directed learning program that provides education about transfusion medicine was associated with an absolute improvement of 6.8% in appropriate transfusions based on a hemoglobin threshold of 80 g/L. Uptake of the training program was nearly complete (96%), and transfusion knowledge, as evidenced by standardized posttest performance, was greatly improved over pretest performance. The only cost associated with this intervention was about 60 minutes of the residents’ time, often outside of normal working hours. House staff time is valuable beyond monetary considerations, and so it is important to recognize that this time was spent in participating in an effective, relevant and accredited educational activity.

On the intervention unit, 2 population-based metrics — transfusions per 100 admissions and transfusions per 1000 patient days — did not capture the increase seen in the proportion of appropriate transfusions. Both these metrics decreased on the control unit. Population-based metrics measure the rates of transfusion; however, this may not reflect transfusion appropriateness when trying to compare units or sites, particularly where patient populations may differ in the need for appropriate transfusions. This is important to keep in mind when designing future quality-improvement initiatives in this area. For example, in our study, although the overall demographic characteristics of the 2 units were similar, the number of patients with hematological malignant disorders was higher on the intervention unit than on the control unit, and the difference has continued to increase since hospital restructuring in April 2015. This population requires a large number of transfusions, many of which would be appropriate; hence, higher numbers of such patients on the intervention unit would result in higher population-based transfusion rates on the unit despite an improvement in transfusion appropriateness and a reduction in overall use on the control site without a corresponding change in appropriateness. The pros and cons of various measures of transfusion use are presented in Appendix 2 (available at [www.cmajopen.ca/content/7/3/E492/suppl/DC1](http://www.cmajopen.ca/content/7/3/E492/suppl/DC1)).

There have been several publications involving traditional educational interventions to improve transfusion appropriateness, ranging from frequent verbal presentations to more formal behavioural interventions.16–19 These have had mixed results depending on the training personnel, clinical champions and specific curriculum used. Our use of an online training program that can be accessed from home remains relatively novel. A recent meta-analysis of behaviour modification interventions (including protocols, education, electronic medical record alerts, audit and feedback, and policy interventions) to optimize red blood cell transfusion practices showed that use of an intervention decreased the pooled odds of inappropriate transfusion (pooled odds ratio 0.46, 95% confidence interval 0.36–0.59).20 The authors of the meta-analysis noted that no study to date had examined the cost-effectiveness of such interventions.

In many studies, multimodal interventions, especially the subsequent addition of computerized decision support at the time of ordering a transfusion, led to even further reductions, which suggest that this pairing is more effective than educational efforts alone.16–18,21 However, at our institution, we currently have no means of implementing such computerized decision support, although some degree of decision support could be introduced into paper-based systems with standardized transfusion order forms.22 Importantly, we tested this intervention on a medical unit where most patients are acutely ill, have a variety of diagnoses and comorbidities, and often require transfusions. If potentially inappropriate transfusions can be limited in this environment, training may prove effective in other inpatient settings as well.

### Limitations

A major limitation of our study is that it was a nonrandomized before–after study at a single academic health centre on a medical teaching unit. Clearly, the external validity needs to be considered, particularly as applies to centres with more extensive preexisting transfusion education, on surgical or critical care units, or in nonteaching centres. Before–after studies also have methodological limitations that can raise concerns for internal validity. The use of a contemporary control medical teaching unit does help strengthen our conclusions, and our primary outcome metric is easily obtained and relatively independent of patient volume or case-mix. Although several junior residents rotated through both sites and completed the training, there was no contamination of the senior residents or faculty physicians. Importantly, any effect of junior resident contamination would have biased the results toward the null hypothesis and led to an underestimate of any effects. As with many educational interventions, there is a concern that the effect may wane with time. In the long term, we will be providing report cards, both by ordering physician (resident and most responsible faculty physician) and by unit, with the proportion of transfusions under 80 g/L compared to the overall institutional target of 80%. Finally, it should be cautioned that the measure of transfusion with a hemoglobin threshold of 80 g/L is a proxy for appropriateness, which itself is a process outcome. Improvements in transfusion appropriateness may not necessarily translate into reductions in harder outcomes such as transfusion-associated adverse events, and, in turn, these are too rare for a study of this size to detect.

### Conclusion

Interventions to improve the appropriateness of packed red blood cell transfusion are necessary and could have an important impact if deployed broadly and early on in a physician’s training. An online training program that is widely available, accredited and inexpensive presents a very promising option. If implemented at a more central level, this type of online learning module could be applied systematically across a training program or institution, thereby promoting change and reaching a larger and broader audience than might be achieved with monthly unit-based teaching sessions, grand rounds or specific educational campaigns. Subsequent studies are needed to evaluate this educational initiative in a broader context. Given the promising findings of the present study, in June 2019 our institution will mandate Bloody Easy Lite training for all postgraduate trainees who can order blood transfusions. Such pragmatic interventions provide promise for directed learning as a means of turning value-based avoidance statements into meaningful advances in high-value health care.

## Acknowledgements

The authors acknowledge the Canadian Society of Internal Medicine’s Choosing Wisely Workgroup for their participation in the genesis of this project. They thank the Ontario Regional Blood Coordinating Network for the use of their online education module. They also thank Patricia Pelletier and Ann Wilson from the institutional blood bank for their assistance in study design, expert hematology review and with obtaining the relevant data.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** Emily McDonald and Todd Lee contributed to the study conception and design. Todd Lee performed the statistical analysis. All of the authors contributed to data acquisition and drafting and revising the manuscript, approved the version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** Funding was provided by the Royal Victoria Hospital 9C Clinical Teaching Unit Quality Improvement Fund and the McGill University Health Centre Association of Physicians. Todd Lee and Emily McDonald receive salary support awards from the Fonds de recherche du Québec – Santé.

*   **Disclaimer:** The funders had no role in the design, conduct or reporting of the study.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/7/3/E492/suppl/DC1](http://www.cmajopen.ca/content/7/3/E492/suppl/DC1).

## References

1.  Choosing Wisely (ABIM Foundation, Philadelphia) Available: [choosingwisely.org](http://choosingwisely.org). accessed 2019 May 1.
    
    

2.  Rosenberg A, Agiro A, Gottlieb M, et al. (2015) Early trends among seven recommendations from the Choosing Wisely Campaign. JAMA Intern Med 175:1913–20.
    
    

3.  Canadian Society of Internal Medicine Five things physicians and patients should question (Choosing Wisely Canada, Toronto) Updated September 2018. Available: [https://choosingwiselycanada.org/internal-medicine/](https://choosingwiselycanada.org/internal-medicine/). accessed 2019 May 1.
    
    

4.  Carson JL, Guyatt G, Heddle NM, et al. (2016) Clinical practice guidelines from the AABB: red blood cell transfusion thresholds and storage. JAMA 316:2025–35.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2016.9185&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=27732721&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 

5.  Carson JL, Terrin ML, Noveck H, et al. (2011) Liberal or restrictive transfusion in high-risk patients after hip surgery. N Engl J Med 365:2453–62.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMoa1012452&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22168590&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000298544600006&link_type=ISI) 

6.  Hébert PC, Wells G, Blajchman MA, et al. (1999) A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion Requirements in Critical Care Investigators, Canadian Critical Care Trials Group. N Engl J Med 340:409–17.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJM199902113400601&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=9971864&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000078571000001&link_type=ISI) 

7.  Villanueva C, Colomo A, Bosch A, et al. (2013) Transfusion strategies for acute upper gastrointestinal bleeding. N Engl J Med 368:11–21.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMoa1211801&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23281973&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000312930600001&link_type=ISI) 

8.  Carson JL, Stanworth SJ, Roubinian N, et al. (2016) Transfusion thresholds and other strategies for guiding allogeneic red blood cell transfusion. Cochrane Database Syst Rev 10:CD002042.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=27731885&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 

9.  Narvios AB, Lichtiger B, Neumann JL (2004) Underreporting of minor transfusion reactions in cancer patients. MedGenMed 6:17.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15775844&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 

10. Shander A, Hofmann A, Ozawa S, et al. (2010) Activity-based costs of blood transfusions in surgical patients at four hospitals. Transfusion 50:753–65.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1537-2995.2009.02518.x&link_type=DOI) 

11. Spahn DR, Goodnough LT (2013) Alternatives to blood transfusion. Lancet 381:1855–65.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(13)60808-9&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23706802&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000319443700032&link_type=ISI) 

12. Goodnough LT, Verbrugge D, Vizmeg K, et al. (1992) Identifying elective orthopedic surgical patients transfused with amounts of blood in excess of need: the transfusion trigger revisited. Transfusion 32:648–53.
    
    

13. Shander A, Fink A, Javidroozi M, et al. (2011) Appropriateness of allogeneic red blood cell transfusion: the international consensus conference on transfusion outcomes. Transfus Med Rev 25:232–46.e53.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.tmrv.2011.02.001&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21498040&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 

14. (2016) Bloody Easy Lite for Physicians (Ontario Regional Blood Coordinating Network, Toronto) Available: [http://belite.transfusionontario.org/](http://belite.transfusionontario.org/). accessed 2019 May 1.
    
    

15. Ontario Transfusion Quality Improvement Plan Committee, Ontario Regional Blood Coordinating Network, Ontario Transfusion Coordinators and Choosing Wisely Canada (2017) Why give two when one will do? A toolkit for reducing unnecessary red blood cell transfusions in hospitals. Version 1.2 (Choosing Wisely Canada, Toronto) Available: [https://choosingwiselycanada.org/wp-content/uploads/2017/07/CWC\_Transfusion\_Toolkit\_v1.2\_2017-07-12.pdf](https://choosingwiselycanada.org/wp-content/uploads/2017/07/CWC\_Transfusion_Toolkit_v1.2_2017-07-12.pdf). accessed 2019 June 12.
    
    

16. Rothschild JM, McGurk S, Honour M, et al. (2007) Assessment of education and computerized decision support interventions for improving transfusion practice. Transfusion 47:228–39.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1537-2995.2007.01093.x&link_type=DOI) 

17. Corwin HL, Theus JW, Cargile CS, et al. (2014) Red blood cell transfusion: impact of an education program and a clinical guideline on transfusion practice. J Hosp Med 9:745–9.
    
    

18. Goodnough LT, Shieh L, Hadhazy E, et al. (2014) Improved blood utilization using real-time clinical decision support. Transfusion 54:1358–65.
    
    

19. Wilson K, MacDougall L, Fergusson D, et al. (2002) The effectiveness of interventions to reduce physician’s levels of inappropriate transfusion: what can be learned from a systematic review of the literature. Transfusion 42:1224–9.
    
    

20. Soril LJJ, Noseworthy TW, Dowsett LE, et al. (2018) Behaviour modification interventions to optimise red blood cell transfusion practices: a systematic review and meta-analysis. BMJ Open 8:e019912.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOC81L2UwMTk5MTIiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzcvMy9FNDkyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

21. Hibbs SP, Nielsen ND, Brunskill S, et al. (2015) The impact of electronic decision support on transfusion practice: a systematic review. Transfus Med Rev 29:14–23.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.tmrv.2014.10.002&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25535095&link_type=MED&atom=%2Fcmajo%2F7%2F3%2FE492.atom) 

22. (2016) Ontario Transfusion Quality Improvement Plan guidance document for institutional implementation (Ontario Regional Blood Coordinating Network, Toronto) Available: [http://transfusionontario.org/en/wp-content/uploads/sites/4/2016/04/Quality-Improvement-Plan-Guidance-Document-final.pdf](http://transfusionontario.org/en/wp-content/uploads/sites/4/2016/04/Quality-Improvement-Plan-Guidance-Document-final.pdf). accessed 2019 May 1.
    
    

*   Copyright 2019, Joule Inc. or its licensors