Impact of a provincial quality-improvement program on primary health care in Ontario: a population-based controlled before-and-after study
==========================================================================================================================================

* Michael E. Green
* Stewart B. Harris
* Susan Webster-Bogaert
* Han Han
* Jyoti Kotecha
* Alexander Kopp
* Minnie M. Ho
* Richard V. Birtwhistle
* Richard H. Glazier

## Abstract

**Background:** In Ontario, a province-wide quality-improvement program (Quality Improvement and Innovation Partnership [QIIP]) was implemented between 2008 and 2010 to support improved outcomes in Family Health Teams, a care model that includes many features of the patient-centred medical home. We assessed the impact of this program on diabetes management, colorectal and cervical cancer screening and access to health care.

**Methods:** We used comprehensive linked administrative data sets to conduct a population-based controlled before-and-after study. Outcome measures included diabetes process-of-care measures (test ordering, retinal examination, medication prescribing and completion of billing items specific to diabetes management), colorectal and cervical cancer screening measures and use of health care services (emergency department visits, hospital admission for ambulatory-care-sensitive conditions and rates of readmission to hospital). The control group consisted of Family Health Team physicians with at least 100 assigned patients during the study follow-up period (November 2009-February 2013).

**Results:** There were 53 physicians in the intervention group and 1178 physicians in the control group. Diabetes process-of-care measures improved more in the intervention group than in the control group: hemoglobin A1c testing 4.3% (95% confidence interval [CI] 1.2-7.5) more, retinal examination 2.5% (95% CI 0.8-4.4) more and preventive care visits 8.9% (95% CI 2.9-14.9) more. Medication prescribing also improved for use of statins (3.4% [95% CI 0.8-6.0] more) and angiotensin-converting-enzyme inhibitors or angiotensin receptor blockers (4.1% [95% CI 1.8-6.4] more). Colorectal cancer screening improved 5.4% (95% CI 3.1-7.8) more in the intervention group than in the control group, and cervical cancer screening improved 2.7% (95% CI 0.9-4.6) more. There were no significant differences in any of the measures of use of health care services.

**Interpretation:** This large controlled evaluation of a broadly implemented quality-improvement initiative showed improvement for diabetes process of care and cancer screening outcomes, but not for proxy measures of access related to use of health care services.

Primary health care plays a key role in health care systems in Canada and around the world.1,2 Studies consistently show that the vast majority of care is delivered in primary care settings3-7 and that strong primary care systems are associated with improved outcomes and decreased health care costs.5,8,9 In Ontario, primary care is the backbone of the publicly funded health care system, accounting for about 80% of all visits annually.7,10 Over the past decade, improving and strengthening primary health care have been key priorities of successive governments in Ontario, which have implemented a series of reforms and initiatives in this key sector.10 These include changes to payment models for physicians, formal enrolment of patients with primary care providers, support for multidisciplinary teams, support for the adoption of electronic health records and province-wide quality-improvement initiatives.8,11,12 Family Health Teams represent the most highly reformed model of primary health care in Ontario and include all of the elements described above.13 There is also consistent evidence that there is room for improvement in the quality of delivery of primary health care in Canada and in Ontario. The much-cited 2010 Commonwealth Fund International Health Policy Survey shows that Canada's performance, last or next to last on many measures, leaves much to be desired.14 Recent studies comparing aspects of the new payment and organizational models in Ontario have shown modest improvements or no differences between models but have shown improvement in quality of care over time.15-20

To help maximize the impact of the new models, a provincial quality-improvement initiative, the Quality Improvement and Innovation Partnership (QIIP) (subsequently incorporated into Health Quality Ontario), was created to assist Family Health Teams in 3 targeted areas - diabetes management, colorectal cancer screening and access to care - through a learning collaborative program based on the Institute for Healthcare Improvement model.21-23. Quality-improvement teams in participating practices received training on quality-improvement methods and the chronic care model as described by Wagner and colleagues24 and others10 as well as professional practice facilitation and participated in learning sessions with other teams. Tricco and colleagues25 conducted a systematic review and meta-analysis of any approach to quality improvement for diabetes management. Within their selected studies, only 4 cluster randomized trials were classified as "continuous quality improvement," which was the closest match for the learning collaborative model in their taxonomy of interventions. They did not find any significant improvement in any process of care measures for this approach. Schouten and colleagues26 conducted a systematic review of quality-improvement collaboratives and concluded that, although there is evidence that this approach can be effective, further evaluation, particularly with controlled study designs, is warranted, as outcome improvements in prior studies have been modest and not always consistent or predictable. Of the controlled studies included in those authors' review, only the single-clinic study on diabetes by Benedetti and colleagues27 reported on any of the outcomes targeted by the QIIP program. The QIIP program and the overall results of a comprehensive mixed-methods evaluation are described in detail elsewhere.23,28 The mixed-methods evaluation did not show statistically significant differences in the primary outcomes, but it was not powered to detect modest effect sizes. We conducted a supplementary population-level controlled before-and-after evaluation to assess the impact of the QIIP learning collaborative program on outcomes that could be assessed with the use of administrative data.

## Methods

### Participants and setting

We used a controlled before-and-after-study design. All patients of physicians included in the study during the study period were included in the study population. Patients were assigned to physicians if they had formally enrolled as a patient with the physician or if they could be virtually assigned by means of an established method of allocating patients to the physician who delivered the majority of a basket of primary care services.19,20,29 All Family Health Team physicians in Ontario with at least 100 assigned patients during the study follow-up period (November 2009-February 2013) were included in the study. We asked physicians who had participated in the QIIP learning collaborative program to provide their registration number and to consent to the use of this number to identify their data within the health administrative data. Physicians who agreed and who met the inclusion criteria constituted the intervention group. Privacy limitations meant that we were unable to determine which physicians who had participated in the QIIP program did not consent, so all other eligible physicians made up the control group. Therefore, the control group included physicians who had participated in the QIIP program but did not consent to take part in the current evaluation.

### QIIP learning collaborative program (intervention)

The QIIP learning collaborative program was available to all Family Health Teams in Ontario on a voluntary basis. The learning collaboratives used the Institute for Healthcare Improvement's Breakthrough Series learning model21 and included three 2-day learning sessions, action periods and a summative congress. The learning sessions enabled teams to learn quality-improvement methods and concepts from experts, and also provided a rich opportunity to network and learn from each other. Teams were educated in the use of the Ontario Chronic Disease Prevention and Management Framework's plan-do-study-act methodology and clinical practice guidelines.10 The program provided each team the support of a dedicated quality-improvement coach, whose role was to facilitate and mentor participants throughout the program. Participating practices were required to form a core interdisciplinary quality-improvement team that would be representative of the interdisciplinary composition of their organization. Participating teams included family physicians and allied health care clinicians (e.g., pharmacists, social workers, nurses) as well as senior administrators and front-line office staff. Practices could focus on any or all of the 3 priority areas of diabetes management, colorectal cancer screening and access to care. The program was offered in 3 waves between April 2008 and October 2010, with each wave occurring over about 15 months.

### Design and data sources

Administrative data sets were linked with the use of unique, encoded identifiers and were analyzed at the Institute for Clinical Evaluative Sciences to measure the impact of the quality-improvement program on diabetes management, screening for colorectal cancer and cervical cancer, and access to care (emergency department visits, hospital admission for ambulatory-care-sensitive conditions and rates of readmission to hospital). We included cervical cancer screening to see whether participation in the program resulted in process changes that affected cancer screening in general, not just the targeted condition (spread); we considered both 2-year and 3-year screening intervals on account of changes in practice guidelines over the study period. We also assessed continuity of care with the primary care physician. The data sets included payments to physicians from the Ontario Health Insurance Plan, the hospital Discharge Abstracts Database, the National Ambulatory Care Reporting System (for emergency department visits), laboratory and diagnostic imaging ordering data, the Ontario Cancer Registry, the Ontario Drug Benefit database, physician workforce data, data on patient enrolment in primary care models, the Ontario Registered Persons Database, the census and vital statistics. We randomly assigned control physicians index dates corresponding to the 3 waves between 2008 and 2010. We assessed physician and practice demographic characteristics including case-mix, health status of the practice population, patterns of use of health care services, and measures for chronic disease prevention and management. Data definitions and sources for each of the measures used in this paper are in alignment with the *Primary Care in Ontario: ICES Atlas*.29

### Statistical analysis

We compared results 12-24 months before the program to results 12-24 months after the program using generalized linear regression with adjustment for baseline value, patient demographic characteristics (age, sex, rurality) and case-mix (using the Johns Hopkins Adjusted Diagnosis Groups30). All analysis was conducted with the use of SAS version 13.1. We conducted the analysis at the level of the physician practice as that is where the intervention was targeted. A *p* value less than 0.05 was considered statistically significant.

### Ethics approval

This study was approved by the research ethics boards at Sunnybrook Hospital (Institute for Clinical Evaluative Sciences Central), Western University and Queen's University.

## Results

We approached 118 physicians who had participated in the QIIP program. Of the 118, 45 did not respond and 10 refused, for a recruitment rate of 53.4% (63/118). Ten physicians were subsequently found to be ineligible (4 had a panel size less than 100 patients, 4 did not have rostered patients, and 2 did not have a Family Health Team designation) and were removed from the study.

A total of 1178 control physicians with at least 100 patients were identified and randomly assigned to one of the 3 index dates. Demographic and practice characteristics of the QIIP and control physicians are presented in Table 1. QIIP physicians were slightly more likely to be male, Canadian trained and from rural areas than the control physicians, but only the mean Rurality Index of Ontario score was significantly different between the 2 groups (*p* = 0.006).

View this table:
[Table 1:](http://www.cmajopen.ca/content/5/2/E281/T1)

Table 1:   Demographic and practice characteristics of physicians in the QIIP and control groups

There were no clinically relevant differences in demographic and clinical characteristics between the patients of the 2 groups of physicians other than a higher proportion residing in a rural location in the QIIP group than in the control group (18.1% v. 15.1%) (Table 2). No statistical analysis is presented because, owing to the large sample, all reported differences were statistically significant.

View this table:
[Table 2:](http://www.cmajopen.ca/content/5/2/E281/T2)

Table 2:   Demographic and clinical characteristics of patients of physicians in the QIIP and control groups

Table 3 presents the results for diabetes process-of-care measures including test ordering, completion of billing items specific to diabetes management and medications prescribed for eligible patients. Process-of-care measures improved more in the intervention group than the control group: hemoglobin A1c testing improved 4.3% (95% confidence interval [CI] 1.2-7.5) more, and retinal examination improved 2.5% (95% CI 0.8-4.4) more. The differential increase in the completion of lipid testing, 1.3%, was not significant. Billing items specific to diabetes management also improved more in the intervention group than in the control group: flow sheet completion improved 8.8% (95% CI 4.1-13.5) more, and preventive care visits improved 8.9% (95% CI 2.9-14.9) more. Differential increases in medication prescribing were noted for statins (3.4% [95% CI 0.8-6.0]) and angiotensin-converting-enzyme inhibitors/angiotensin receptor blockers (4.1% [95% CI 1.8-6.4]). There were no significant differences in the use of orally administered hypoglycemic agents or insulin.

View this table:
[Table 3:](http://www.cmajopen.ca/content/5/2/E281/T3)

Table 3:   Rates of completion of diabetes management measures by physicians in the QIIP and control groups

Screening measures for colorectal cancer increased more in the intervention group than in the control group: fecal occult blood testing improved 8.5% (95% CI 5.1-12.0) more, and any screening improved 5.4% (95% CI 3.1-7.8) more (Table 4). There was a small but statistically significant increase in cervical cancer screening in the intervention group, for both 2- and 3-year intervals (percent adjusted change 2.3% [95% CI 0.5-4.1] and 2.7% [95% CI 0.9-4.6), respectively).

View this table:
[Table 4:](http://www.cmajopen.ca/content/5/2/E281/T4)

Table 4:   Rates of completion of screening measures for colorectal cancer and cervical cancer by physicians in the QIIP and control groups

Table 5 presents the results for access to care measures. There were no significant changes in emergency department visits, hospital admission for ambulatory-care-sensitive conditions or hospital readmission, with the exception of admission for chronic obstructive pulmonary disease, for which there was a small but statistically significant increase in the intervention group (percent adjusted change 6.0% [95% CI 1.0-11.0]). Continuity of care with the usual provider was similar at baseline for the intervention and control groups (72.2% and 72.0%, respectively), and there were no significant changes over the study period (increase of 0.7% v. decrease of 0.3%) (*p* = 0.5) (data not shown).

View this table:
[Table 5:](http://www.cmajopen.ca/content/5/2/E281/T5)

Table 5:   Rates of access to care (emergency department visits, hospital admission and hospital readmission) for physicians in the QIIP and control groups

## Interpretation

For most of the measures that were targeted by the QIIP program, outcomes were improved compared with the control group, with rates of change about double those in the control group. These differences were statistically significant owing to the relatively large samples that are possible with population-based analyses but were in general modest in magnitude (absolute differences 2%-11%). However, even small changes can be important if they are broadly applied to the population, so findings such as ours have the potential to provide system- or population-level benefits despite their modest effect size.

The observed changes in diabetes process-of-care measures are consistent with those noted in other studies. Benedetti and colleagues27 found improvements in both process measures (test ordering, documentation of self-management plans) in diabetes-focused learning collaboratives in Washington State. Valk and colleagues31 compared programs to improve the quality of diabetes care in the Netherlands and the United States and also found both process and outcome (hemoglobin A1c testing) improvements in both groups. The magnitude of changes (crude rate) observed in our intervention group after participation in the QIIP program is similar to that noted in an uncontrolled before-and-after evaluation of a quality-improvement initiative (Partnerships for Health) that also used the Institute for Healthcare Improvement model (9% increase in annual hemoglobin A1c testing and 9% increase in testing for low-density lipoprotein cholesterol).32 Our adjusted analysis, which took into account temporal changes in the control group, showed more modest increases, which highlights the importance of including a control group.

In contrast to the Supporting Colorectal Cancer Outcomes through Participatory Enhancements trial,33 which showed no significant improvement in colorectal cancer screening after an intervention that combined learning collaboratives and on-site facilitation, in this population-based study we found an improvement in screening comparable to that with other interventions, such as audit and feedback, noted in the Colorectal Cancer Screening in Primary Care Practice study34 and Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice study.35 Our results also contrast with those of our chart audit, which failed to show a significant difference in screening rates in a small subsample of patients.28 This highlights one of the advantages of using population-level data to achieve the power required to detect changes in situations in which event frequency or effect sizes mean that resource-intensive audits of patient records are not feasible. Analysis of rates of cervical cancer screening showed a statistically significant increase in the intervention group, but the magnitude of the difference between the intervention and control groups was smaller than that for colorectal cancer screening. This could be an indication of spread if, for example, recall processes for cancer screening were introduced that covered more than just colorectal cancer.

We found no clinically relevant changes in measures of use of health care services such as low-acuity emergency department visits and hospital admission for ambulatory-care-sensitive conditions that are commonly used as outcome measures or proxy measures for access to primary health care. This is consistent with the limited number of other studies that have explored the relation between advanced access scheduling (the focus of the QIIP program) and use of health care services. In a 2011 systematic review of advanced access scheduling, Rose and colleagues36 found that, although most studies showed improvements in time to third-next available appointment and reduced no-show rates, the effects on patient satisfaction were mixed, and the data on use were limited to very few studies. Those authors identified only 2 studies that reported on use of health care services, and only 1 of these was controlled. Solberg and colleagues37 reported on an uncontrolled before-and-after study of advanced access for patients with chronic conditions and found that, despite significant improvements in access and improved continuity of care, there was very limited change in use of health care services, including emergency department visits and hospital admissions, or overall costs. Finally, in a controlled study of primary care clinics in Indiana that transitioned or did not transition to advanced access, no significant differences were found in emergency department and urgent care visits, hospital admissions or total outpatient visits.38

### Limitations

First, by using administrative data, we were limited to collected process measures and measures of use of health care services. Second, as we were not permitted access to the list of all participating physicians because of privacy restrictions, physicians who participated in the QIIP program but who did not complete consent to take part in the current evaluation were included in the control group. Although this may have introduced a bias toward a null result, nonparticipants represented only 5% of the control group, which makes bias toward a null result unlikely. Third, this was a program implementation rather than a trial of an intervention, so the degree to which recommended processes were implemented and the way in which they were implemented likely varied significantly across the participating teams. This reduced implementation fidelity, would generally result in a bias toward a null result. In addition, the implementation took place during a time of reform, with other changes and initiatives being implemented concurrently. In particular, province-wide programs to improve the quality of diabetes care and cancer screening were being conducted that targeted all primary care providers, including physicians in Family Health Teams. To mitigate against the risk of bias toward a null result we applied a controlled before-and-after design and also limited patients in the control group to Family Health Team patients, as the concurrent changes would likely have been similar in the control and intervention practices. Fourth, as this was a voluntary program, it is possible that participating physicians were more motivated to improve their practices than nonparticipating physicians. A final caveat is that this evaluation was based on a relatively limited follow-up period. Repeating this analysis later in time to assess for the sustainability of the changes would provide important additional information about the role of the learning collaborative approach in improving quality of primary care.

### Conclusion

Our overall results were positive for diabetes management and for colorectal and cervical cancer screening, with both clinically and statistically significant improvements in multiple outcomes noted. These findings support the use of the Institute for Healthcare Improvement's learning collaborative strategy for quality improvement in multidisciplinary team practices such as Family Health Teams. There were no improvements in measures of use of health care services that were used as a proxy for improved primary care access. This is likely because there are only weak links between actual improved access and these outcomes. Direct assessment of access to primary health care as well as patient experience with access would be better measures of success in this domain. Despite the relatively modest absolute levels of improvement noted, the success in achieving improvements of this scale for large populations is important. In our setting, the learning collaborative approach seemed to be beneficial for outcomes that were more directly under the control of health care providers when applied at large scale to a broad range of primary care practices. It seemed less effective in changing outcomes such as medication use or use of health care services that are dependent on patient or system factors outside the control of the practice.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/5/2/E281/suppl/DC1](http://www.cmajopen.ca/content/5/2/E281/suppl/DC1)

## Acknowledgements

This study was supported by the Quality Improvement and Innovation Partnership (QIIP), Health Quality Ontario and a Canadian Institutes of Health Research Knowledge Dissemination grant. Michael Green is supported by the Clinical Teachers' Association of Queen's University Chair in Applied Health Policy/Health Economics. Stewart Harris is supported by the Canadian Diabetes Association Chair in Diabetes Management and the Ian McWhinney Chair of Family Medicine Studies, Western University. Richard Glazier is supported as a Clinician Scientist in the Department of Family and Community Medicine at the University of Toronto and at St. Michael's Hospital. No endorsement by Health Quality Ontario is intended or should be inferred. Parts of this material are based on based on data and information compiled and provided by the Canadian Institutes of Health Information (CIHI) or Cancer Care Ontario (CCO). The analysis, conclusions, opinions and statements reported in this paper are those of the authors and do not necessarily reflect those of the CIHI or CCO. No endorsement by CCO or the CIHI is intended or inferred.

## Footnotes

*   **Competing interests:** See the end of the article.

*   **Contributors:** All authors contributed to the study design. Minnie Ho and Alexander Kopp conducted the analysis. Michael Green produced the initial draft of the manuscript. All of the authors contributed to data interpretation, contributed substantially to revising the manuscript, approved the final version to be published and agreed to act as guarantors of the work.

*   **Competing interests:** This work was supported by an award to Stewart Harris, Richard Birtwhistle, Michael Green and Richard Glazier from Health Quality Ontario to conduct the evaluation of the Quality Improvement and Innovation Partnership program. Stewart Harris reports grants and personal fees from Abbott, AstraZeneca, Boehringer Ingelheim/Lilly, Janssen and Novo Nordisk, and personal fees from Merck outside the submitted work. Richard Birtwhistle reports a contract with the Ontario Ministry of Health and Long-Term Care and a grant from the Canadian Institutes of Health Research outside the submitted work. No other competing interests were declared.

*   **Funding:** Health Quality Ontario, Canadian Institutes of Health Research.

*   **Prior presentations:** Health Services Research Association of Australia and New Zealand biennial conference, Melbourne, Australia, Dec. 7, 2015; Canadian Association for Health Services and Policy Research meeting, Montréal, May 2016.

*   **ICES disclaimer:** This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this article are those of the authors and are independent from the funding sources. No endorsement by the ICES or the Ontario MOHLTC is intended or should be inferred.

## References

1.  (2008) The World health report 2008 - primary health care (now more than ever). Geneva: World Health Organization.
    
    

2.  Schoen C, Osborn R, Huynh PT, et al. (2006) On the front lines of care: primary care doctors' office systems, experiences, and views in seven countries. Health Aff (Millwood) 25:w555–71.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjk6IjI1LzYvdzU1NSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNS8yL0UyODEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  White KL, Williams TF, Greenberg BG (1961) The ecology of medical care. N Engl J Med 265:885–92.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJM196111022651805&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=14006536&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A19618492B00012&link_type=ISI) 

4.  Green LA, Cifuentes M, Glasgow RE, et al. (2008) Redesigning primary care practice to incorporate health behavior change: prescription for health round-2 results. Am J Prev Med 35(Suppl):S347–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.amepre.2008.08.013&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18929980&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

5.  Starfield B, Shi L, Macinko J (2005) Contribution of primary care to health systems and health. Milbank Q 83:457–502.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1468-0009.2005.00409.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16202000&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000232231200005&link_type=ISI) 

6.  Boucai L, Zonszein J (2007) Effects of quality improvement strategies for type 2 diabetes in Bronx, N.Y. Clin Diabetes 25:155–9.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZGlhY2xpbiI7czo1OiJyZXNpZCI7czo4OiIyNS80LzE1NSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNS8yL0UyODEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

7.  McPherson C, Kothari A, Sibbald SQuality improvement in primary health care in Ontario: an environmental scan and capacity map. Final report to the Quality Improvement in Primary Healthcare Project Planning Group. Toronto: Cancer Quality Council of Ontario2010Available[www.cqco.ca/common/pages/UserFile.aspx?fileId=250528](http://www.cqco.ca/common/pages/UserFile.aspx?fileId=250528)accessed 2015 
    
    

8.  O'Brien BD, Brown MG, Kephart G (2001) Estimation of hospital costs for colorectal cancer care in Nova Scotia. Can J Gastroenterol 15:43–7.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11173325&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

9.  Friedberg MW, Hussey PS, Schneider EC (2010) Primary care: a critical review of the evidence on quality and costs of health care. Health Aff (Millwood) 29:766–72.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjg6IjI5LzUvNzY2IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby81LzIvRTI4MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

10. (2007) Preventing and managing chronic disease: Ontario's framework. Toronto: Ministry of Health and Long-Term Care. Available[www.health.gov.on.ca/en/pro/programs/cdpm/pdf/framework_full.pdf](http://www.health.gov.on.ca/en/pro/programs/cdpm/pdf/framework_full.pdf). accessed 2014 Nov. 2.
    
    

11. Hutchison B, Levesque JF, Strumpf E, et al. (2011) Primary health care in Canada: systems in motion. Milbank Q 89:256–88.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1468-0009.2011.00628.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21676023&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000292083600004&link_type=ISI) 

12. Kates N, Hutchison B, O'Brien P, et al. (2012) Framework for advancing improvement in primary care. Healthc Pap 12:8–21.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22842927&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

13. Rosser WW, Colwill JM, Kasperski J, et al. (2011) Progress of Ontario's family health team model: a patient-centred medical home. Ann Fam Med 9:165–71.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6NzoiOS8yLzE2NSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNS8yL0UyODEuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

14. How engaged are Canadians in their primary care? Results from the 2010 Commonwealth Fund International Health Policy Survey. Canadian Health Care Matters Bulletin 5. Toronto: Health Council of Canada; 2011. Available[www.healthcouncilcanada.ca/tree/2.30-Commonwealth5\_EN\_Final.pdf](http://www.healthcouncilcanada.ca/tree/2.30-Commonwealth5_EN_Final.pdf). accessed 2017 Mar. 26.
    
    

15. Dahrouge S, Hogg W, Russell G, et al. (2009) The comparison of models of primary care in Ontario (COMP-PC) study: methodology of a multifaceted cross-sectional practice-based study. Open Med 3:e149–64.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21603051&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

16. Collier R (2011) Verdict still out on family health teams. CMAJ 183:1131–2.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTgzLzEwLzExMzEiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvMi9FMjgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

17. Jaakkimainen L, Glazier R, Barnsley J, et al. (2014) Waiting to see the specialist: patient and provider characteristics of wait times from primary to specialty care. BMC Fam Pract 15:16.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2296-15-16&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24460619&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

18. Russell GM, Hogg W, Lemelin J (2010) Integrated primary care organizations: the next step for primary care reform. Can Fam Physician 56:216–8, e87–9.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2ZwIjtzOjU6InJlc2lkIjtzOjg6IjU2LzMvMjE2IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby81LzIvRTI4MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. Final report: an external evaluation of the Family Health Team (FHT) initiative. Ottawa: Conference Board of Canada2014Available[www.conferenceboard.ca/e-library/abstract.aspx?did=6711](http://www.conferenceboard.ca/e-library/abstract.aspx?did=6711)accessed 2015 
    
    

20. Kiran T, Kopp A, Moineddin R, et al. (2015) Longitudinal evaluation of physician payment reform and team-based care for chronic disease management and prevention. CMAJ 187:E494–502.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTg3LzE3L0U0OTQiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvMi9FMjgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

21. (2003) The Breakthrough Series: IHI's collaborative model for achieving breakthrough improvement. Boston: Institute for Healthcare Improvement.
    
    

22. Jones K, Piterman L (2008) The effectiveness of the breakthrough series methodology. Aust J Prim Health 14:59–65.
    
    

23. Kotecha J, Brown JB, Han H, et al. (2015) Influence of a quality improvement learning collaborative program on team functioning in primary healthcare. Fam Syst Health 33:222–30.
    
    

24. Wagner EH, Austin BT, Davis C, et al. (2001) Improving chronic illness care: translating evidence into action. Health Aff (Millwood) 20:64–78.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToiaGVhbHRoYWZmIjtzOjU6InJlc2lkIjtzOjc6IjIwLzYvNjQiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvMi9FMjgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

25. Tricco AC, Ivers NM, Grimshaw JM, et al. (2012) Effectiveness of quality improvement strategies on the management of diabetes: a systematic review and meta-analysis. Lancet 379:2252–61.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(12)60480-2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22683130&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000305511400032&link_type=ISI) 

26. Schouten LMT, Hulscher MEJL, van Everdingen JE, et al. (2008) Evidence for the impact of quality improvement collaborative: systematic review. BMJ 336:1491.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEzOiIzMzYvNzY1OS8xNDkxIjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby81LzIvRTI4MS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

27. Benedetti R, Flock B, Pedersen S, et al. (2004) Improved clinical outcomes for fee-for-service physician practices participating in a diabetes care collaborative. Jt Comm J Qual Saf 30:187–94.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15085784&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

28. Harris SB, Green ME, Brown JB, et al. (2015) Impact of a quality improvement program on primary healthcare in Canada: a mixed-method evaluation. Health Policy 119:405–16.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.healthpol.2014.10.019&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25476553&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

29. Jaakkimainen L, Upshur REG, Klein-Geltink JE, et al.editors. Primary care in Ontario: ICES atlas. Toronto: Institute for Clinical Evaluative Sciences2006Available[www.ices.on.ca/flip-publication/primary-care-2006/index.html](http://www.ices.on.ca/flip-publication/primary-care-2006/index.html)accessed 2015 
    
    

30. The Johns Hopkins ACG SystemAvailable: https://www.hopkinsacg.org/ accessed 2017 Mar. 29
    
    

31. Valk GD, Renders CM, Kriegsman DMW, et al. (2004) Quality of care for patients with type 2 diabetes mellitus in the Netherlands and the United States: a comparison of two quality improvement programs. Health Serv Res 39:709–25.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1475-6773.2004.00254.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15230924&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000226710600002&link_type=ISI) 

32. Harris S, Paquette-Warren J, Roberts S, et al. (2013) Results of a mixed-methods evaluation of partnerships for health: a quality improvement initiative for diabetes care. J Am Board Fam Med 26:711–9.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjYvNi83MTEiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvMi9FMjgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

33. Shaw EK, Obman-Strickland PA, Piasecki A, et al. (2013) Effects of facilitated teammeetings and learning collaboratives on colorectal cancer screening rates in primary care practices: a cluster randomized trial. Ann Fam Med 11:220–8, S1–8.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTEvMy8yMjAiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvMi9FMjgxLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

34. Ornstein S, Nemeth LS, Jenkins RG, et al. (2010) Colorectal cancer screening in primary care: translating research into practice. Med Care 48:900–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/MLR.0b013e3181ec5591&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20808257&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000282068900007&link_type=ISI) 

35. Grunfeld E, Manca D, Moineddin R, et al., BETTER Trial Investigators (2013) Improving chronic disease prevention and screening in primary care: results of the BETTER pragmatic cluster randomized controlled trial. BMC Fam Pract 14:175.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2296-14-175&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24252125&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 

36. Rose KD, Ross JS, Horwitz LI (2011) Advanced access scheduling outcomes: a systematic review. Arch Intern Med 171:1150–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/archinternmed.2011.168&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21518935&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000292658100002&link_type=ISI) 

37. Solberg LI, Maciosek MV, Sperl-Hillen JM, et al. (2004) Does improved access to care affect utilization and costs for patients with chronic conditions? Am J Manag Care 10:717–22.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15521163&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000224504600010&link_type=ISI) 

38. Subramanian U, Ackermann RT, Brizendine EJ, et al. (2009) Effect of advanced access scheduling on processes and intermediate outcomes of diabetes care and utilization. J Gen Intern Med 24:327–33.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s11606-008-0888-2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19132326&link_type=MED&atom=%2Fcmajo%2F5%2F2%2FE281.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000263379300006&link_type=ISI) 

*   Copyright 2017, Joule Inc. or its licensors