Accelerating surgical quality improvement in Ontario through a regional collaborative: a quality-improvement study
==================================================================================================================

* Timothy Jackson
* David Schramm
* Husein Moloo
* Lee Fairclough
* Azusa Maeda
* Tricia Beath
* Avery Nathens

## Abstract

**Background:** The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) collaborative in Ontario, the Ontario Surgical Quality Improvement Network (ON-SQIN), was launched in January 2015. We describe its approaches to support surgical quality improvement and examine its early impact on member hospitals.

**Methods:** All Ontario hospitals that participated in the ON-SQIN and NSQIP were included in this quality-improvement study. The primary intervention was the introduction of the ON-SQIN, and the secondary interventions included a community of practice and access to quality-improvement resources and tools. Outcome measures included the level of quality-improvement capacity, collaborative-wide aggregate data on postoperative complications, and self-reported rates of surgical site and urinary tract infections.

**Results:** Eighteen hospitals that enrolled in the ON-SQIN in 2015 reported an increase in their capacity for quality improvement after 18 months. Analysis of the collaborative-wide aggregate data in a 6-month period (14 748 surgical cases) revealed a substantial reduction of acute renal failure (relative risk 0.48, 95% confidence interval 0.25–0.95) and urinary tract infection (relative risk 0.77, 95% confidence interval 0.61–0.97). Most hospitals that targeted prevention of surgical site infection and urinary tract infection reported reduction of these occurrences during a 1-year period.

**Interpretation:** The ON-SQIN supported the uptake of the NSQIP in Ontario hospitals and promoted targeted surgical quality-improvement initiatives, resulting in increased quality-improvement capacity and development of the community of practice. Furthermore, our early experience suggests that improvements in surgical care are being realized.

Postsurgical complications are associated with increased mortality and health care costs, as well as impaired psychosocial well-being of patients.1–4 With the goal to improve the quality of surgical care, the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) was implemented to provide hospitals with tools on surgical quality improvement and reports on their surgical outcomes.5 Although participation in the NSQIP results in the reduction of postoperative adverse events,6,7 studies suggest that the reporting mechanism alone may not be sufficient to achieve long-term improvements.8,9 Thus, with support from the NSQIP,5 several institutions implemented a regional collaborative approach to quality improvement.10–13 Notably, Share and colleagues12 showed that surgical outcomes in hospitals participating in a collaborative improved faster than those in other NSQIP hospitals that were not part of a collaborative. Therefore, additional support and resources provided by a collaborative may accelerate quality improvement.

In Canada, an NSQIP regional collaborative was implemented in British Columbia.14 Similarly, the Ontario Surgical Quality Improvement Network (ON-SQIN) was launched in January 2015 to support surgical teams to accelerate improvement through a community of practice, defined as “groups of people who share a concern, a set of problems, or a passion for a topic, and who deepen their knowledge and expertise in an area by interacting on an ongoing basis.”15 The ON-SQIN was established by Health Quality Ontario with funding support from the Ontario Ministry of Health and Long-Term Care and adapted the quality-improvement plan mandated by the Ontario Excellent Care for All Act.16 The objective of this quality-improvement report was to describe our initial experience in building a collaborative in Ontario and its impact on hospital quality-improvement capacity and surgical outcomes.

## Methods

### Participating hospitals

The ON-SQIN was launched in January 2015 with 4 academic organizations (5 hospitals) that were already enrolled in the NSQIP. In 2015, 13 additional hospitals (2 academic, 6 community, 5 small/rural) participated in the NSQIP with support from the ON-SQIN. Of the hospitals that expressed their interest, those that enrolled in the ON-SQIN and the NSQIP were included in the study.

### Collaborative structure

The ON-SQIN is structured with a model for surveillance and monitoring of progress. Specifically, it comprises 1) the Steering Committee, which includes representation from member hospitals to provide the overall strategic direction and oversight, 2) the Network Delivery Team, which manages implementation activities, and 3) the provincial surgical lead, who acts as the clinical liaison between the Steering Committee and surgeon champions to facilitate knowledge transfer.

### Quality-improvement resources and community of practice

Hospitals were provided with access to resources and tools including site visits, an established quality-improvement program (Improving & Driving Excellence Across Sectors [IDEAS]),17 an Institute for Healthcare Improvement e-learning module18 and support to implement enhanced recovery after surgery.19 Furthermore, an online platform was developed to provide information on the best-practice bundles and comprehensive unit-based safety program approach,20 forums for member discussion and a document library. In addition, hospitals were provided with financial support to help offset the initial costs for the first 18 months of the program. This budget covered the NSQIP licensing fee, a stipend for the surgeon champion and a portion of salary for a surgical clinical reviewer. Last, a community of practice was established to provide opportunities for interaction among peers, including monthly surgeon champion and surgical clinical reviewer conference calls, annual conferences, webinars and newsletters.

### Data source and outcome measures

Eighteen hospitals that participated in the ON-SQIN in 2015 were included. We assessed the level of quality-improvement capacity at each hospital via a Web-based questionnaire at the time of their initial enrolment in the ON-SQIN and after 18 months. The questionnaire was developed by the Network Delivery Team and was completed by surgeon champions, surgical clinical reviewers, quality-improvement team members and/or surgical administrative staff at each hospital. The initial questionnaire was used as a tool to determine appropriate quality-improvement resources needed for each hospital. The questionnaire was a closed survey distributed to all 18 hospitals by email with the goal of tailoring support for participants based on the organization’s needs, skill level and experience in quality improvement. A total of 1–3 questions were displayed per page for a total of 9 pages (including the title page), and respondents were able to change answers at any time. The questionnaire included questions pertaining to access to quality-improvement resources, collaboration within the surgical team, active surgical quality-improvement initiatives and organizational engagement in surgical quality improvement (Appendix 1, available at [www.cmajopen.ca/content/6/3/E353/suppl/DC1](http://www.cmajopen.ca/content/6/3/E353/suppl/DC1)).

We determined the initial performance status of the 18 hospitals relative to all hospitals in the NSQIP based on the July 2016 risk-adjusted semiannual report encompassing 13 indicators for general, vascular, colorectal or all-case surgical procedures between January and December 2015. The semiannual report is prepared by the NSQIP to compare risk-adjusted surgical outcomes to those of other participating sites.21 We obtained the odds ratio for the indicators to calculate the proportion of hospitals that were categorized as “needs improvement” (high statistical outlier or in 10th decile), “as expected” (neither an outlier nor in first or 10th decile) or “exemplary” (low statistical outlier or in first decile) with respect to all hospitals for each indicator. We collected collaborative-wide aggregate data on postoperative occurrences between August 2015 and January 2016 in accordance with the established data-sharing methods. Data after January 2016 were not considered in this analysis as they included data from new hospitals that participated in the ON-SQIN.

Last, we collected self-reported outcomes from member hospitals through a Surgical Quality Improvement Plan, which was submitted by hospitals every 6 months to report on their chosen indicators and change ideas. We analyzed rates of surgical site infections and urinary tract infections for hospitals that indicated reduction of these target measures in their Surgical Quality Improvement Plan between September 2015 and September 2016. Hospitals that reported odds ratios in their Surgical Quality Improvement Plan and those that did not submit the year-end Surgical Quality Improvement Plan were excluded from the analysis.

### Statistical analysis

We analyzed the questionnaire data by assigning questions a value using an ordinal scale, and an overall score between 0 and 1 was used. We analyzed collaborative-wide surgical outcomes by calculating percent change and relative risk for postoperative occurrences from the baseline measurement, and *p* < 0.05 was considered statistically significant.

### Ethics approval

Consent was not required as the study patients were anonymized. No incentives were offered to participate in the survey.

## Results

### Building quality-improvement capacity across hospitals

All respondents from the 18 hospitals completed the questionnaire. At the initial stage of participation in the ON-SQIN, members reported limited availability of quality-improvement capacity and initiatives (Table 1). Collaboration within the surgical team was markedly lacking, with only 8 hospitals (44%) indicating that they had sufficient ongoing collaboration. At 18 months, all 17 hospitals that answered the questionnaire indicated that they had access to resources, ongoing quality-improvement initiatives and engagement within their organization. Four hospitals (24%) still indicated lack of collaboration within their surgical team. Resources provided by the ON-SQIN were used over the 18-month period, including the online platform (> 200 visits per month), Improving and Driving Excellence Across Sectors (7/18 sites), Open School e-Learning module (15/18 sites), enhanced recovery after surgery programs (15/18 sites) and Canadian Surgical Site Infection Prevention Audit (8/18 sites).

View this table:
[Table 1:](http://www.cmajopen.ca/content/6/3/E353/T1)

Table 1: 
Self-assessment of available quality-improvement capacity by hospitals that participated in the Ontario Surgical Quality Improvement Network in 2015

### Establishing community of practice

Members received comprehensive support through the community of practice (Appendix 2, available at [www.cmajopen.ca/content/6/3/E353/suppl/DC1](http://www.cmajopen.ca/content/6/3/E353/suppl/DC1)). In-person events, such as the surgeon champion meetings, conferences and workshops, were particularly well-represented by both surgeon champions and surgical clinical reviewers, with 83%–100% representation of the hospitals.

### Impact on surgical outcomes

The performance of the ON-SQIN hospitals on postoperative indicators showed that there was a room for improvement in the prevention of morbidity and surgical site and urinary tract infections (Table 2). As expected, many hospitals indicated surgical site and/or urinary tract infections as their targets for improvement in their initial Surgical Quality Improvement Plan submitted in September 2015. Common initiatives for prevention of surgical site infections included implementation of enhanced recovery after surgery, participation in the Canadian Surgical Site Infection Prevention Audit, normothermia and development of standardized evidence-based procedures such as a surgical dressing protocol. Common initiatives for prevention of urinary tract infections included implementation of enhanced recovery after surgery, reduction of the use of indwelling catheters and staff education.

View this table:
[Table 2:](http://www.cmajopen.ca/content/6/3/E353/T2)

Table 2: 
Performance of the Ontario Surgical Quality Improvement Network hospitals on postoperative indicators relative to all National Surgical Quality Improvement Program (NSQIP) hospitals, all surgical procedures collected in NSQIP, 2015

We analyzed surgical outcomes of the member hospitals using the aggregated data, which included all surgical cases performed at the 18 hospitals: 4806 cases in August–September 2015 (baseline), 4821 cases in October–November 2015 and 5121 cases in December 2015–January 2016. Overall, there was a significant aggregate improvement in the occurrence of acute renal failure (relative risk 0.48, 95% confidence interval 0.25–0.95) and urinary tract infection (relative risk 0.77, 95% confidence interval 0.61–0.97) in the 4-month period following the initial submission of the Surgical Quality Improvement Plan (Table 3). Most hospitals elected to target surgical site infection and urinary tract infection in the general surgery population. Among hospitals that specifically targeted reduction of rates of surgical site and urinary tract infection, 7 of 8 hospitals reported a decrease in their surgical site infection rate 1 year after the initial submission of their Surgical Quality Improvement Plan (Figure 1, A), and all 3 hospitals that targeted reduction of the rate of urinary tract infections reported a decrease in the rate (Figure 1, B).

View this table:
[Table 3:](http://www.cmajopen.ca/content/6/3/E353/T3)

Table 3: 
Frequency and relative risk of postoperative wound occurrences and urinary tract infections

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/3/E353/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/6/3/E353/F1)

Figure 1: 
Self-reported rates of (A) surgical site infection (*n* = 8 hospitals) and (B) urinary tract infection (*n* = 3 hospitals) between September 2015 and September 2016.

## Interpretation

Implementation of the collaborative in Ontario resulted in the development of the community of practice, and an increase in hospital quality-improvement capacity and other ongoing surgical quality-improvement activities captured within the Surgical Quality Improvement Plan framework. Many hospitals identified reduction of surgical site and urinary tract infections as their targets for improvement and implemented best-practice bundles,22 with resultant reduction of these postoperative occurrences. Of 2257 morbidity cases reported in the July 2016 semiannual report for Ontario hospitals, 1568 (69.5%) were due to surgical site or urinary tract infections. Therefore, the reduction of such infections may translate into a reduction of overall morbidity. The number of member hospitals grew to 33 by the end of 2016, and 11 of the 14 Ontario Local Health Integration Networks have hospitals participating in the ON-SQIN (Figure 2). As of January 2017, member hospitals performed 46.4% of all adult surgical procedures in Ontario. In addition, 3 of 5 pediatric hospitals in Ontario participated (as of January 2017) in the ON-SQIN.

![Figure 2:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/6/3/E353/F2.medium.gif)

[Figure 2:](http://www.cmajopen.ca/content/6/3/E353/F2)

Figure 2: 
Location of the Ontario Surgical Quality Improvement Network hospitals (blue circles) within the Local Health Integration Networks.

1: Erie St. Clair, 2: South West, 3: Waterloo, Wellington, 4: Hamilton, Niagara, Haldimand, Brant, 5: Central West, 6: Mississauga, Halton, 7: Toronto Central, 8: Central, 9: Central East, 10: South East, 11: Champlain, 12: North Simcoe, Muskoka, 13: North East, 14: North West.

Participation in the NSQIP has been clearly associated with a reduction of adverse postoperative events.7 In addition, a collaborative approach provides an added value to further accelerate improvements in surgical care.12 Although it is difficult to identify specific components of the collaborative that contribute to improvement, we believe that the collaborative structure, including the community of practice, provides a model for spread, scale and sustainability across member hospitals. The community of practice was built on a previously described model for quality improvement in cancer surgery, which is structured with 1) access to data, 2) access to evidence and methodology, 3) access to education and professional development activities, 4) project management support and 5) communication strategies.23 Based on this model, the ON-SQIN provided access to evidence-based best practice, training opportunities and resources, communication tools such as the online platform and in-person meetings, and support through its structure (Steering Committee, Network Delivery Team and provincial surgical lead) to enable sharing of best practices within the province. In addition, hospitals used the Surgical Quality Improvement Plan to select their target quality-improvement indicators and activities to track their progress over time. Although other hospital-led quality-improvement initiatives may exist independent of the collaborative, we believe that the model of the community of practice and the active reporting mechanism contributed to the reported increase in the available quality-improvement capacity across member hospitals to achieve long-term changes and surveillance.

Hospitals participated in the ON-SQIN at the time of their enrolment in the NSQIP. This mechanism ensured that there was no delay in providing support to build necessary capacity to maximize the use of data collected through the NSQIP. Hospitals collected data for at least 4 months before determining their targets for improvement. During this initial phase, hospital sites were able to review their unadjusted data benchmarked to hospitals participating in the NSQIP, as well as those in the ON-SQIN at any time. These raw data are informative for front-line staff at the beginning of the quality-improvement initiative as they can be easily interpreted and give the perspective of the number of patients affected. Hospitals were also able to obtain the on-demand, risk-adjusted report, which can be used at an early stage to identify opportunities for improvement. The on-demand report is also useful in assessing the impact of quality-improvement initiatives at each hospital since the time period for data comparison can be specified. Nevertheless, risk-adjusted data in the semiannual report, although less optimal in terms of timeliness, are still the gold standard with respect to accuracy and should be used to examine the targets for improvement and impact of quality-improvement activities.24

Hospitals were provided with financial support during the first 18 months to ensure that they were equipped with sufficient resources for quality-improvement activities. To address the potential variability in existing resources, the Web-based questionnaire was used to customize the support offered to individual hospitals. Following the 18-month period, hospitals committed to financially supporting their NSQIP fees, in-house human resources and other costs required to sustain their initiatives, while the ON-SQIN maintained the community resources and support. However, 3 rural and small hospitals were unable to maintain their participation in the NSQIP owing to the associated cost, which highlights challenges to sustain and disseminate the collaborative. Nevertheless, those hospitals continued to receive support in quality improvement outside of NSQIP, including conferences and access to resources provided by the ON-SQIN. To reduce the cost of administering the program, the NSQIP collaborative of Florida launched the NSQIP “lite” system, which applies the data collection process to a limited sample, with evidence of quality improvement and cost saving.11 This strategy may also be useful in Ontario. In addition, Share and colleagues12 and Englesbe and colleagues25 reported that the cost savings associated with the care for postoperative complications exceeds the cost of administering the NSQIP and suggested that a reduction of about 2% in the complication rate is sufficient to offset the cost of maintaining participation in the NSQIP. Similar cost analysis in the context of Ontario hospitals is underway to promote further dissemination of the NSQIP and the ON-SQIN program in community and small/ rural hospitals.

### Limitations

We used unadjusted aggregate data reports on all cases in the ON-SQIN to examine the change in postoperative outcomes. As the ON-SQIN was initially set up as a blinded collaborative, we did not have direct access to any risk-adjusted hospital-level data at the time of the analysis. Limitations of the use of aggregated data include the inability to consider the impact of hospital size and varying capacities. We were also unable to analyze the aggregate data beyond 2016 since additional hospitals joined in 2016. Furthermore, reasons for aggregate change in postoperative occurrences are not readily obvious.10 To address these limitations, the ON-SQIN has recently revised the data-sharing agreement to enable collection of nonblinded hospital-level data, which will be made available exclusively to member hospitals. This will allow for consideration of hospital size and capacity and will foster further collaboration across member hospitals to apply successful quality-improvement initiatives.

The questionnaire used in the study was not validated. Furthermore, the community of practice was evaluated based on attendance only. To further evaluate the effect of the community of practice, we plan to measure compliance with the provincial evidence-based clinical guidelines as well as standardization and implementation of these pathways through a provincial campaign aimed to reduce postoperative infections.

### Conclusion

Establishment of the ON-SQIN led to the rapid uptake of the NSQIP among Ontario hospitals and development of the strong community of practice dedicated to improving surgical care. Our initial findings suggest that the support provided by the ON-SQIN improved the capacity for quality improvement and the member hospitals’ ability to respond to their own data. To scale and spread early success, a provincial campaign aimed at reducing postoperative surgical site infection, urinary tract infection and pneumonia is underway to continue supporting member hospitals. Ongoing data collection and analysis will help determine the long-term effect of the collaborative on the growth of quality-improvement capabilities, its impact on quality of surgical care and potential cost saving.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** Timothy Jackson, Azusa Maeda and Tricia Beath drafted the manuscript, and David Schramm, Husein Moloo, Lee Fairclough and Avery Nathens revised it critically for important intellectual content. All of the authors contributed to study conception and design, data acquisition, and data analysis and interpretation, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/6/3/E353/suppl/DC1](http://www.cmajopen.ca/content/6/3/E353/suppl/DC1).

## References

1.  Dimick JB, Chen SL, Taheri PA, et al. (2004) Hospital costs associated with surgical complications: a report from the private-sector National Surgical Quality Improvement Program. J Am Coll Surg 199:531–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jamcollsurg.2004.05.276&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15454134&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000224368400002&link_type=ISI) 

2.  McAleese P, Odling-Smee W (1994) The effect of complications on length of stay. Ann Surg 220:740–4.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=7986140&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1994PW18300006&link_type=ISI) 

3.  Pinto A, Faiz O, Davis R, et al. (2016) Surgical complications and their impact on patients’ psychosocial well-being: a systematic review and meta-analysis. BMJ Open 6:e007224.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiNi8yL2UwMDcyMjQiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzYvMy9FMzUzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

4.  Tevis SE, Kennedy GD (2013) Postoperative complications and implications on patient-centered outcomes. J Surg Res 181:106–13.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jss.2013.01.032&link_type=DOI) 

5.  Ingraham AM, Richards KE, Hall BL, et al. (2010) Quality improvement in surgery: the American College of Surgeons National Surgical Quality Improvement Program approach. Adv Surg 44:251–67.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.yasu.2010.05.003&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20919525&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

6.  Hall BL, Hamilton BH, Richards K, et al. (2009) Does surgical quality improve in the American College of Surgeons National Surgical Quality Improvement Program: an evaluation of all participating hospitals. Ann Surg 250:363–76.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/SLA.0b013e3181b4148f&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19644350&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000269361400002&link_type=ISI) 

7.  Cohen ME, Liu Y, Ko CY, et al. (2016) Improved surgical outcomes for ACS NSQIP hospitals over time: evaluation of hospital cohorts with up to 8 years of participation. Ann Surg 263:267–73.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/SLA.0000000000001192&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25723845&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

8.  Etzioni DA, Wasif N, Dueck AC, et al. (2015) Association of hospital participation in a surgical outcomes monitoring program with inpatient complications and mortality. JAMA 313:505–11.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2015.90&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25647206&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

9.  Osborne NH, Nicholas LH, Ryan AM, et al. (2015) Association of hospital participation in a quality reporting program with surgical outcomes and expenditures for Medicare beneficiaries. JAMA 313:496–504.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2015.25&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25647205&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

10. Guillamondegui OD, Gunter OL, Hines L, et al. (2012) Using the National Surgical Quality Improvement Program and the Tennessee Surgical Quality Collaborative to improve surgical outcomes. J Am Coll Surg 214:709–14, discussion 714–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jamcollsurg.2011.12.012&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22265639&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

11. Tepas JJ 3rd., Kerwin AJ, deVilla J, et al. (2014) Macro vs micro level surgical quality improvement: a regional collaborative demonstrates the case for a national NSQIP initiative. J Am Coll Surg 218:599–604.
    
    

12. Share DA, Campbell DA, Birkmeyer N, et al. (2011) How a regional collaborative of hospitals and physicians in Michigan cut costs and improved the quality of care. Health Aff (Millwood) 30:636–45.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjMwLzQvNjM2IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby82LzMvRTM1My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

13. Wandling MW, Minami CA, Johnson JK, et al. (2016) Development of a conceptual model for surgical quality improvement collaboratives: facilitating the implementation and evaluation of collaborative quality improvement. JAMA Surg 151:1181–3.
    
    

14. Tackling complex problems with team-based solutions: NSQIP in BC 2014 (BC Patient Safety & Quality Council, Vancouver) Available: [https://bcpsqc.ca/resource/tackling-complex-problems-with-team-based-solutions-nsqip-in-bc-2014/](https://bcpsqc.ca/resource/tackling-complex-problems-with-team-based-solutions-nsqip-in-bc-2014/). accessed 2017 Sept 15.
    
    

15. Wenger E, McDermott R, Snyder W (2002) Cultivating communities of practice: a guide to managing knowledge (Harvard Business Review Press, Watertown (MA)), 1st ed.
    
    

16. Chan B (2012) Transforming healthcare in Ontario through integration, evidence, and building capacity for improvement. Healthc Manage Forum 25:191–3.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.hcmf.2012.09.005&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23387138&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

17. IDEAS: Improving & Driving Excellence Across Sectors, Available: [https://www.ideasontarioca/](https://www.ideasontarioca/). accessed 2017 Sept 15.
    
    

18. IHI Open School online course (Institute for Healthcare Improvement, Boston) Available: [www.ihi.org/education/IHIOpenSchool/Courses/Pages/default.aspx](http://www.ihi.org/education/IHIOpenSchool/Courses/Pages/default.aspx). accessed 2017 Sept 15.
    
    

19. McLeod RS, Aarts MA, Chung F, et al. (2015) Development of an Enhanced Recovery After Surgery guideline and implementation strategy based on the knowledge-to-action cycle. Ann Surg 262:1016–25.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/SLA.0000000000001067&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25692358&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

20. Wick EC, Hobson DB, Bennett JL, et al. (2012) Implementation of a surgical comprehensive unit-based safety program to reduce surgical site infections. J Am Coll Surg 215:193–200.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jamcollsurg.2012.03.017&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22632912&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

21. Cohen ME, Ko CY, Bilimoria KY, et al. (2013) Optimizing ACS NSQIP modeling for evaluation of surgical quality and risk: patient risk adjustment, procedure mix adjustment, shrinkage adjustment, and surgical focus. J Am Coll Surg 217:336–46, e1.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jamcollsurg.2013.02.027&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23628227&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 

22. Thakker A, Briggs N, Maeda A, et al. (2018) Reducing the rate of post-surgical urinary tract infections in orthopedic patients. BMJ Open Qual 7:e000177.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiYm1qcWlyIjtzOjU6InJlc2lkIjtzOjExOiI3LzIvZTAwMDE3NyI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNi8zL0UzNTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

23. Fung-Kee-Fung M, Boushey RP, Morash R (2014) Exploring a “community of practice” methodology as a regional platform for large-scale collaboration in cancer surgery — the Ottawa approach. Curr Oncol 21:13–8.
    
    

24. Cohen ME, Liu Y, Huffman KM, et al. (2016) On-demand reporting of risk-adjusted and smoothed rates for quality profiling in ACS NSQIP. Ann Surg 264:966–72.
    
    

25. Englesbe MJ, Dimick JB, Sonnenday CJ, et al. (2007) The Michigan surgical quality collaborative: Will a statewide quality improvement initiative pay for itself? Ann Surg 246:1100–3.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/SLA.0b013e31815c3fe5&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18043116&link_type=MED&atom=%2Fcmajo%2F6%2F3%2FE353.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000251638000028&link_type=ISI) 

*   Copyright 2018, Joule Inc. or its licensors