Effect of a multimorbidity intervention on health care utilization and costs in Ontario: randomized controlled trial and propensity-matched analyses
====================================================================================================================================================

* Bridget L. Ryan
* Luke Mondor
* Walter P. Wodchis
* Richard H. Glazier
* Leslie Meredith
* Martin Fortin
* Moira Stewart

## Abstract

**Background:** Patients with multimorbidity require coordinated and patient-centred care. Telemedicine IMPACT Plus provides such care for complex patients in Toronto, Ontario. We conducted a randomized controlled trial (RCT) comparing health care utilization and costs at 1-year postintervention for an intervention group and 2 control groups (RCT and propensity matched).

**Methods:** Data for 82 RCT intervention and 74 RCT control participants were linked with health administrative data. We created a second control group using health administrative data–derived propensity scores to match (1:5) intervention participants with comparators. We evaluated 5 outcomes: acute hospital admissions, emergency department visits, costs of all insured health care, 30-day hospital readmissions and 7-day family physician follow-up after hospital discharge using generalized linear models for RCT controls and generalized estimating equations for propensity-matched controls.

**Results:** There were no significant differences between intervention participants and either control group. For hospital admissions, emergency department visits, costs and readmissions, the relative differences ranged from 1.00 (95% confidence interval [CI] 0.39–2.60) to 1.67 (95% CI 0.82–3.38) with intervention costs at about Can$20 000, RCT controls costs at around Can$15 000 and propensity controls costs at around Can$17 000. There was a higher rate of follow-up with a family physician for the intervention participants compared with the RCT controls (53.13 v. 21.43 per 100 hospital discharges; relative difference 2.48 [95% CI 0.98–6.29]) and propensity-matched controls (49.94 v. 28.21 per 100 hospital discharges; relative difference 1.81 [95% CI 0.99–3.30]).

**Interpretation:** Despite a complex patient-centred intervention, there was no significant improvement in health care utilization or cost. Future research requires larger sample sizes and should include outcomes important to patients and the health care system, and longer follow-up periods.

**Ontario [ClinicalTrials.gov](http://ClinicalTrials.gov):** 104191

Multimorbidity is common1,2 and is associated with a burden on patients and the health care system.3 Care for patients with multimorbidity must consider the incremental challenges that multiple chronic conditions confer upon patients over and above the burden conferred by each individual condition.2,3 As such, patients with multimorbidity require care coordinated by teams of providers and care that attends to them as a whole person, not as a sum of their diseases.3–5 Although interventions aimed at improving health care, including for complex patients, have been mounted in Ontario, few have been evaluated.6 Further, interventions specifically targeted for patients with multimorbidity worldwide have had mixed results;7–12 only 211,12 have examined the effect of interventions on health care utilization.

We report the effect on health services outcomes for patients enrolled in the Ontario arm of a clinical trial called Patient-centred Innovations for Persons with Multimorbidity (PACE in MM).13 The first objective was to compare health care utilization and costs between intervention and control patients before and after the intervention. Anticipating a small sample size for a community-based complex intervention, the decision was made a priori13 to include a second objective where health care utilization and costs for intervention patients were examined relative to propensity-matched controls derived from health administrative data.

## Methods

With values of coordinated team-based and patient-centred care as pillars, the PACE in MM project conducted 2 randomized controlled trials (RCTs) of primary care delivery for people with multimorbidity — 1 in Quebec and 1 in Ontario.13 In Ontario, the patient-reported outcomes were as follows: the Health Education Impact Questionnaire (heiQ), Self-Efficacy for Managing Chronic Disease scale (SEM-CD), Veteran’s Rand 12-Item Health Survey (VR-12), the EuroQol 5 Dimension quality of life measure (EQ-5D), Kessler Psychological Distress Scale (K6) and the health behaviour survey.14 The effects of the intervention on these patient-reported health outcomes14,15 are reported elsewhere.

The Ontario intervention took place in the highly diverse neighbourhoods of central and east Toronto from 2016 to 2017. Nine team-based family practices, along with solo practices and emergency departments affiliated with those teams, provided care for complex patients with high health care utilization through Telemedicine IMPACT Plus, hereafter referred to as the intervention.16 The intervention consisted of a meeting between the patient and a nurse, where the patient’s goals for care were elaborated. There was also a subsequent case conference of about 6 providers relevant to the patient’s needs, including the patient and a family physician known to the patient.14 Patients aged 18 to 80 years were the target population and had to have 3 or more chronic conditions and be referred by their provider to receive the intervention. Patients were excluded if they had cognitive impairment, were illiterate or had previously received the intervention. For this current analysis, the follow-up period was 1 year postintervention. Further details of the Ontario RCT can be found in Appendix 1, available at [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1) and Appendix 2, available at [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1).

### Data sources

The data for participant information for the PACE in MM Ontario RCT were gathered from patient questionnaires completed through a telephone interview by a research assistant upon patient enrolment in the study. The research assistant was unaware of the participants’ RCT assignment during interviews. Questionnaire data were transferred from paper, then verified by another research assistant and stored in the study database. The questionnaire asked participants about their chronic condition history, coordination of health care, patient-centredness, patient-reported outcomes (heiQ, SEM-CD, VR-12, EQ-5D, K6, and health behaviours), costs of uninsured health care and demographic characteristics. Variables relevant to the health administrative analysis were age, sex, education, household income, marital status, employment and a list of patients’ chronic conditions. These variables were stored in the PACE in MM Ontario RCT database (hereafter referred to as the PACE database) along with participants’ assigned index dates, RCT assignments (intervention or control) and Ontario Health Insurance Plan (OHIP) numbers.

Health administrative data were obtained through ICES (Toronto). ICES is an independent, nonprofit research institute whose legal status under Ontario’s health information privacy law allows it to collect and analyze health care and demographic data, without consent, for health system evaluation and improvement. In January 2020, the PACE database was transferred to ICES (Toronto). Data from the PACE database and ICES health administrative databases were linked using unique encoded identifiers (derived from OHIP numbers) and analyzed at ICES. Appendix 3, available at [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1), describes the health administrative data sets used in analyses.

### Overall methods and outcome measures

We conducted 2 analyses. The first analysis compared intervention participants to RCT control patients (hereafter called RCT analysis). The second compared intervention participants to propensity-matched controls identified in health administrative data (hereafter called propensity-matched analysis). A priori, we expected the sample size for the RCT to be modest and therefore included a 5:1 propensity-matched analysis to increase power.13 This process created an analytical sample in which measured confounding factors were balanced between intervention arms. Below, the 2 analyses are described separately. All analyses were conducted using SAS version 9.4 (SAS Institute).

For both analyses, 5 outcomes were obtained from health administrative data from the 1-year postindex date: acute hospital admissions, emergency department visits, costs (total included all health care expenditures that had been allocated to patient encounters for insured health care), 30-day hospital readmissions for patients who had at least 1 hospital admission and 7-day follow-up with a family physician after hospital discharge for patients who had at least 1 hospital admission. Appendix 4, available at [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1) provides definitions for each outcome, including data sources used. Measures were chosen a priori to show important markers for PACE in MM success.13 Minor adjustments from protocol were made to some outcome definitions to align with measures available in health administrative data.

### Samples

For the RCT analysis, there were 86 intervention participants and 77 control participants.14 Participants were included if they were successfully linked to health administrative databases (Appendix 1).

For the propensity-matched analysis, participants from the PACE intervention group were included in this analysis if they were successfully linked to health administrative databases. The index date used for intervention participants was the date they received the intervention.

To create a pool of eligible comparators, we assigned all Ontarians in the Registered Persons Database a pseudo-index date. This date was based on the quarterly distribution of all index dates for only the intervention participants in the PACE database. From this pool, we excluded people who did not have a physician encounter recorded in OHIP 1 year before their pseudo-index date, were in hospital at pseudo-index date, were enrolled in Family Health Teams that participated in PACE in MM, were a rural resident or resided outside of the forward sortation areas (i.e., first 3 digits of a postal code) of PACE participants, were missing income or rurality data, were a resident of a long-term care facility before pseudo-index or died within 1 year of pseudo-index (Figure 1 describes the propensity-matched study hierarchy).

![Figure 1:](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/11/1/E45/F1.medium.gif)

[Figure 1:](http://www.cmajopen.ca/content/11/1/E45/F1)

Figure 1: 
Propensity-matched analysis study hierarchy. Note: FHT = Family Health Team, FSA = forward sortation area, OHIP = Ontario Health Insurance Plan, PACE in MM = Patient-centred Innovations for Persons with Multimorbidity, RPDB = Registered Persons Database, SD = standard deviation.

### Covariates

For the RCT analysis, covariates were obtained from the PACE database: age, sex, education, household income, marital status, employment and a list of the patient’s chronic conditions; a variable was created for the number of chronic conditions. Covariates were used to describe and compare intervention and control groups.

For the propensity-matched analysis (for intervention participants and for the comparator pool population from which we drew the propensity-matched controls), we defined covariates — at index or pseudo-index date, respectively — including age, sex, rurality (defined using the Rurality Index of Ontario17) and neighbourhood-level income quintile. The history of 17 conditions were defined based on retrospective data from ICES databases as described in Appendix 5, available at [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1). The 17 conditions represent a subset of the most substantial conditions from a population perspective; these have been used extensively for multimorbidity research in Ontario.1,18–23 In addition, we identified the number of urgent hospital admissions (Discharge Abstract Database), emergency department visits (National Ambulatory Care Reporting System), visits to family physicians and specialists (OHIP) and total costs24 incurred across the health care system. These utilization variables were derived for the 1 year before index or pseudo-index as well as quarterly, leading to index or pseudo-index.

To create the propensity-matched cohort, people in the comparator pool were matched 5-to-1 to people in the intervention group using propensity score methods without replacement. Propensity scores were derived from logistic regression modelling the probability of enrolment in the intervention as a function of variables relevant to patients with multimorbidity including age (modelled as a restricted cubic spline), sex, income quintile, rurality, history of 17 chronic conditions and quarterly counts of health care utilization including urgent admissions, emergency department visits, visits to family physicians and specialists and total costs. Utilization variables were transformed using a square-root function before modelling. We created a propensity-score–matched cohort using the nearest-neighbour greedy algorithm to match 3 to 5 comparators for every person in the intervention group.25 People were matched by sex (hard match), the logit of the propensity score (within 0.20 standard deviations [SD]),26 age (hard match within 2 yr), index or pseudo-index date (within 90 d) and total costs in the year before index or pseudo-index (hard match within 3 SD in overall population). To assess quality of the match, we used standardized differences, weighted for many-to-1 matching,27 to compare baseline characteristics of the intervention group and comparators. A standardized difference of less than 0.10 is indicative of good balance between groups.28

### Statistical analysis

The effects of the intervention on acute hospital admissions and emergency department visits were estimated using a negative binomial distribution, log link function and log of persontime offset term. Costs were modelled using a γ distribution, log link function and log of person-time offset. In addition, for these 3 outcomes we conducted a sensitivity analysis disregarding person-time contribution (i.e., no offset was used). For 30-day hospital readmissions and 7-day follow-up with a family physician, data were modelled using a Poisson distribution and a log link function with number of index discharges as the offset term (i.e., people without a hospital discharge were excluded from the analyses). For these 2 outcomes, estimated values were presented as a rate per 100 discharges.

For the RCT analysis, differences in health care utilization 1 year after index dates for PACE intervention group compared with the control group (control used as reference group) were estimated using univariate regression with generalized linear model including a single covariate for RCT assignment (intervention or control).

For the propensity-matched analysis, differences in health care utilization in the year before and after index or pseudo-index for the PACE intervention group compared with the propensity-score-matched controls were estimated using generalized estimating equations to account for correlated data. Each model included a binary covariate for treatment group, a binary variable for time (pre- or postindex or pseudo-index) and the 2-way interaction between treatment and time. This latter term is the ratio of 2 ratios; that is, the risk ratios for the post compared with the pre differences for the intervention group and control group. We have termed this a relative difference, which represents the ratio of the relative changes in the outcomes post-compared with pre-intervention. An exchangeable correlation structure was used to account for correlation between records. The parallel trends of quarterly data were checked visually to ensure model assumptions were valid.

### Ethics approval

The Western University Health Sciences Research Ethics Board (106921) approved this study.

## Results

A total of 82 of 86 (95.3%) participants from the PACE intervention group and 74 of 77 (96.1%) from the control group were successfully linked to health administrative data. It was not possible to link 7 participants as the unique encoded identifiers required to link were invalid; therefore, these 7 participants had to be excluded from the analysis. Table 1 reports the baseline characteristics. Three control group participants died in the 1-year follow-up period.

View this table:
[Table 1:](http://www.cmajopen.ca/content/11/1/E45/T1)

Table 1: 
Randomized controlled trial analysis: baseline characteristics

Table 2 reports the results of the generalized linear model analysis comparing health care utilization and costs at the 1-year postindex date between the intervention and control participants. There were no significant differences for any of the outcomes of acute hospital admissions, emergency department visits, costs, 30-day hospital readmissions and 7-day follow-up with a family physician. For 7-day follow-up with a family physician, intervention participants had 2.48 times the follow-up compared with control participants (*p* = 0.06). Appendix 6a, available at [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1), provides the observed values for these outcomes.

View this table:
[Table 2:](http://www.cmajopen.ca/content/11/1/E45/T2)

Table 2: 
Randomized controlled trial analysis: results for outcomes 1-year postindex date through generalized estimating equations

### Propensity-matched analysis

A total of 82 participants from the PACE intervention group were successfully matched to health administrative data. The mean age at index for this group was 62 (SD 14) years and nearly two-thirds were women (65%). People from the lowest (29%) and highest (24%) area-based income quintiles were overrepresented in the data. On average, people in the PACE intervention group had a history of 5.4 (SD 2.4) out of 17 conditions. The most common diagnoses were osteoarthritis (78% of participants), mood and anxiety disorders (78%), hypertension (61%) and cancer (59%). In the year before intervention, the PACE group had on average 9.5 family physician visits (SD 9.9), 12.0 specialist visits (SD 15.3), 0.4 urgent hospital admissions (SD 0.9), 1.3 emergency department visits (SD 2.7) and incurred Can$19 900 in health care costs (SD $27 900). Appendix 6b, available at [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1), provides the observed values for these outcomes.

In propensity-score matching, we matched the 82 intervention participants to 401 comparators. Seventy-nine intervention participants were matched to 5 comparators; 3 were matched to only 2 comparators. Baseline covariates were balanced between matched groups (Table 3) with the exception of family physician visits in the quarter before index or pseudo-index (standardized difference = 0.126) and specialist visits in the second quarter nearest to index or pseudo-index (standardized difference = 0.191, data not shown).

View this table:
[Table 3:](http://www.cmajopen.ca/content/11/1/E45/T3)

Table 3: 
Propensity-matched analysis: comparison of characteristics of intervention and comparator groups before and after matching

Table 4 reports the relative differences between intervention participants and controls. Across the 5 outcomes, there were no significant differences. This suggests that the change in utilization or costs before compared with after index for the PACE intervention group was no different than the matched comparator group. Plots of crude quarterly data validated the parallel trend assumption required for the analysis (not shown).

View this table:
[Table 4:](http://www.cmajopen.ca/content/11/1/E45/T4)

Table 4: 
Propensity-matched analysis: results for outcomes 1-year postindex date relative differences

## Interpretation

We report the analysis of 5 hospital-based and cost outcomes for an RCT that provided patient-centred care for people with multimorbidity. The RCT and the propensity-matched analyses found no significant postindex differences in health care utilization or costs between intervention and control participants. For one outcome (7-d follow-up with a family physician), intervention participants had 2.48 times (RCT analysis) and 1.8 times (propensity-matched analysis) the follow-up compared with control participants, but in both analyses, 95% confidence intervals included the null value. The difference found in this study can suggest clinical and policy importance and provide hypothesis generation for future studies.

The RCT findings from this health administrative analysis (RCT and propensity-matched comparisons) are congruent with findings on patient-reported outcomes at 4 months in the Ontario arm of the PACE in MM RCT.14 Results are also consistent with similar interventions.11,12 Another propensity-matched study of a community intervention called Health Links for people with multiple chronic conditions in Ontario found no effect on acute hospital admissions, readmissions or timely follow-up with primary care providers 7 days after hospital discharge.11 This study included components similar to those in the PACE in MM RCT, such as intensive care coordination, multidisciplinary care and a patient-centred coordinated care plan outlining the patient’s needs, goals, providers, treatments and appointments.11 Our results regarding the hospital admission outcome also correspond with the findings of Salisbury and colleagues,12 who reported no difference in 15-month hospital admissions between intervention and control patients in a patient-centred RCT for management of multimorbidity. We identified other studies that tested complex interventions for people with multimorbidity, but these did not have health care utilization outcomes.7–10

### Limitations

We conducted a pragmatic trial in which the providers, rather than the researchers, were responsible for identifying eligible patients and then recruiting them into the trial; therefore, our sample size was based on the numbers our sites were able to recruit. However, we recruited until we reached the minimum sample size to detect a difference in the means of our primary outcome, which was a total sample size of 128, not including dropouts, as described in Stewart and colleagues.14

Regarding the outcomes of 30-day hospital readmissions and 7-day follow-up with a family physician for patients who had at least 1 hospital admission, the comparison group is no longer equivalent to the intervention group, as we restricted the analysis to those who had a hospital admission, which happened postrandomization.

In the propensity-matched analysis, for propensity-matched controls, it was not possible to match for every baseline characteristic collected in the questionnaires as these individual-level characteristics were not available in the health administrative data and therefore could not be included. We cannot rule out the possibility of unmeasured confounding owing to the unavailability of variables such as lifestyle behaviours. In addition, the propensity-matched controls would not have been subject to any placebo effects that might have occurred within the RCT. We excluded participants who died within 1 year of the pseudo-index date, which was not a criterion applied in the RCT; therefore, the controls could have been healthier than the intervention participants. However, we excluded 4292 people who died within 1 year of the pseudo-index date (0.465% of the population), leaving a pool of 919 027 for the propensity match. From the 4292, only a small number would have been accepted into the analysis; therefore, it is unlikely that the exclusion of the 4292 would have biased the results. No participants in the intervention group died in the 1 year after the intervention.

For the propensity-matched analysis, there could have been regression to the mean, in that participants were selected for high complexity, and then could have reverted to their long-term average over the 1 year postintervention. However, we attempted to mitigate this concern, by aiming to ensure equivalence in risk of regression to mean by including prior utilization patterns in the propensity matching process.

The majority of the control group in this RCT received usual care within a primary care team-based model. Usual care may have had similarities to the team-based care provided in the intervention which may affect health care utilization.

Our outcomes were limited to hospital care and direct costs of health care services, outcomes relevant to policymakers. The intervention may confer benefits aligned with outcomes that are situated in primary care, such as continuity of care or that consider patient preferences such as improved function, but not confer benefits aligned with health care utilization and costs. Our follow-up period of 1 year may not have been sufficiently long to see benefits from this complex intervention.

### Conclusion

The results of this study add to the body of evidence that improving health for people with multimorbidity continues to challenge us.12,14,15 Despite a complex patient-centred intervention, there was no significant improvement in health care utilization or cost outcomes for patients who received the PACE in MM intervention. Future research requires larger sample sizes, longer follow-up periods, recruitment of patients from family physicians in non-team-based practice models and the incorporation of a wider range of outcomes important to patients and the health care system.

## Acknowledgements

The authors acknowledge the contributions of the entire Patient-Centred Innovations for Persons with Multimorbidity (PACE in MM) team, some of whom are authors of this article. They acknowledge team members Jocelyn Charles, Pauline Pariser and Thuy-Nga Pham for their contributions to the design and delivery of the Telemedicine IMPACT Plus program. The authors thank IQVIA Solutions Canada Inc. for use of its drug information file.

## Footnotes

*   **Competing interests:** Bridget Ryan was funded by the Canadian Institutes of Health Research Transformative Community-based Primary Health Care Innovation Team, Patient-centred Innovations for Persons with Multimorbidity (2014–2018). Walter Wodchis is supported as a Research Chair in Implementation and Evaluation Science by Trillium Health Partners (2017–). Richard Glazier is supported as a Clinician Scientist in the Department of Family and Community Medicine at St. Michael’s Hospital and at the University of Toronto. Moira Stewart was funded by the Dr. Brian W. Gilbert Canada Research Chair (Tier 1) in Primary Health Care Research (2003–2017). No other competing interests were declared.

*   This article has been peer reviewed.

*   **Contributors:** Moira Stewart and Martin Fortin conceived the study. Bridget Ryan was responsible for the analytic plan for this reported analysis with all authors contributing to the plan. Luke Mondor was responsible for data preparation and conducting the statistical analysis. All authors interpreted the data. Bridget Ryan drafted the manuscript and all authors revised it critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** The Patient-Centred Innovations for Persons with Multimorbidity team was funded by the Canadian Institutes of Health Research’s Transformative Community-based Primary Healthcare Signature Initiative. This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and the Ministry of Long-Term Care. Parts of this material are based on data and information compiled and provided by the Canadian Institute for Health Information and IMS Brogan. The analyses, conclusions, opinions and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.

*   **Data sharing:** The data set from this study is held securely in coded form at ICES. Although data sharing agreements prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at [https://www.ices.on.ca/DAS](https://www.ices.on.ca/DAS). The full data set creation plan and underlying analytic code are available from the authors on request, understanding that the computer programs may rely on coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/11/1/E45/suppl/DC1](http://www.cmajopen.ca/content/11/1/E45/suppl/DC1).

This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use) and no modifications or adaptations are made. See: [https://creativecommons.org/licenses/by-nc-nd/4.0/](https://creativecommons.org/licenses/by-nc-nd/4.0/)

## References

1.  Ryan BL, Bray Jenkyn K, Shariff SZ, et al. (2018) Beyond the grey tsunami: a cross-sectional population-based study of multimorbidity in Ontario. Can J Public Health 109:845–54.
    
    

2.  Steffler M, Li Y, Weir S, et al. (2021) Trends in prevalence of chronic disease and multimorbidity in Ontario, Canada. CMAJ 193:E270–7.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTkzLzgvRTI3MCI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vMTEvMS9FNDUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

3.  Boyd CM, Fortin M (2010) Future of multimorbidity research: how should understanding of multimorbidity inform health system design? Public Health Rev 32:451–74.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/pubmed/fdq057&link_type=DOI) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000281186000024&link_type=ISI) 

4.  (2010) Multiple chronic conditions: a strategic framework optimum health and quality of life for individuals with multiple chronic conditions (US Department of Health and Human Services, Washington (DC)) Available[https://www.hhs.gov/sites/default/files/ash/initiatives/mcc/mcc_framework.pdf](https://www.hhs.gov/sites/default/files/ash/initiatives/mcc/mcc_framework.pdf). accessed 2021 Mar. 8.
    
    

5.  (2016) Multimorbidity: clinical assessment and management (National Institute for Health and Care Excellence, London (UK)) Available: [https://www.nice.org.uk/guidance/ng56/resources/multimorbidity-clinical-assessment-andmanagement-pdf-1837516654789](https://www.nice.org.uk/guidance/ng56/resources/multimorbidity-clinical-assessment-andmanagement-pdf-1837516654789). accessed 2021 Mar. 8.
    
    

6.  Ryan BL, Stewart M (2012) Patient-Centered Innovations for Persons with Multimorbidity, Environmental scan of primary care-linked chronic disease prevention and management programs in Ontario. Report to the Canadian Institutes of Health Research Community-based Primary Health Care Innovation Team (Western University, London (ON)).
    
    

7.  Ford JA, Lenaghan E, Salter C, et al. (2019) Can goal-setting for patients with multimorbidity improve outcomes in primary care? Cluster randomised feasibility trial. BMJ Open 9:e025332.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiYm1qb3BlbiI7czo1OiJyZXNpZCI7czoxMToiOS82L2UwMjUzMzIiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzExLzEvRTQ1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

8.  Mercer SW, Fitzpatrick B, Guthrie B, et al. (2016) The CARE Plus study — a whole-system intervention to improve quality of life of primary care patients with multimorbidity in areas of high socioeconomic deprivation: exploratory cluster randomised controlled trial and cost-utility analysis. BMC Med 14:88.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/s12916-016-0634-2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

9.  Spoorenberg SLW, Wynia K, Uittenbroek RJ, et al. (2018) Effects of a population-based, person-centred and integrated care service on health, wellbeing and self-management of community-living older adults: a randomised controlled trial on Embrace. PLoS One 13:e0190751.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

10. Verdoorn S, Kwint H-F, Blom JW, et al. (2019) Effects of a clinical medication review focused on personal goals, quality of life, and health problems in older persons with polypharmacy: a randomised controlled trial (DREAMeR-study). PLoS Med 16:e1002798.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1371/journal.pmed.1002798&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

11. Mondor L, Walker K, Bai YQ, et al. (2017) Use of hospital-related health care among Health Links enrollees in the Central Ontario health region: a propensity-matched difference-in-differences study. CMAJ Open 5:E753–9.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiY21ham8iO3M6NToicmVzaWQiO3M6ODoiNS80L0U3NTMiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzExLzEvRTQ1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

12. Salisbury C, Man M-S, Bower P, et al. (2018) Management of multimorbidity using a patient-centred care model: a pragmatic cluster-randomised trial of the 3D approach. Lancet 392:41–50.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(18)31308-4&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

13. Stewart M, Fortin M (2017) Patient-centred innovations for persons with multimorbidity: funded evaluation protocol. CMAJ Open 5:E365–72.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiY21ham8iO3M6NToicmVzaWQiO3M6ODoiNS8yL0UzNjUiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzExLzEvRTQ1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

14. Stewart M, Fortin M, Brown JB, et al. (2021) Patient-centred innovation for multimorbidity care: a mixed-methods, randomised trial and qualitative study of the patients’ experience. Br J Gen Pract 71:e320–30.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMToiNzEvNzA1L2UzMjAiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzExLzEvRTQ1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

15. Fortin M, Stewart M, Ngangue P, et al. (2021) Scaling up patient-centered interdisciplinary care for multimorbidity: a pragmatic mixed-methods randomized controlled trial. Ann Fam Med 19:126–34.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTkvMi8xMjYiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzExLzEvRTQ1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

16. Pariser P, Pham TT, Brown JB, et al. (2019) Connecting people with multimorbidity to interprofessional teams using telemedicine. Ann Fam Med 17:S57–62.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6MTQ6IjE3L1N1cHBsXzEvUzU3IjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby8xMS8xL0U0NS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

17. Kralj B (2008) Measuring rurality - RIO2008 BASIC: methodology and results (Ontario Medical Association, Toronto).
    
    

18. Gruneir A, Bronskill SE, Maxwell CJ, et al. (2016) The association between multimorbidity and hospitalization is modified by individual demographics and physician continuity of care: a retrospective cohort study. BMC Health Serv Res 16:154.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

19. Kone AP, Mondor L, Maxwell C, et al. (2021) Rising burden of multimorbidity and related socio-demographic factors: a repeated cross-sectional study of Ontarians. Can J Public Health 112:737–47.
    
    

20. Thavorn K, Maxwell CJ, Gruneir A, et al. (2017) Effect of socio-demographic factors on the association between multimorbidity and healthcare costs: a population-based, retrospective cohort study. BMJ Open 7:e017264.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.ncbi.nlm.nih.gov/pubmed/28988178&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

21. Mondor L, Cohen D, Khan AI, et al. (2018) Income inequalities in multimorbidity prevalence in Ontario, Canada: a decomposition analysis of linked survey and health administrative data. Int J Equity Health 17:90.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

22. Rosella L, Kornas K, Huang A, et al. (2018) Accumulation of chronic conditions at the time of death increased in Ontario from 1994 to 2013. Health Aff (Millwood) 37:464–72.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

23. Lane NE, Maxwell CJ, Gruneir A, et al. (2015) Absence of a socioeconomic gradient in older adults’ survival with multiple chronic conditions. EBioMedicine 2:2094–100.
    
    

24. Wodchis WP, Bushmeneva K, Nikitovic M, et al. (2013) Guidelines on person-level costing using administrative databases in Ontario (Health System Performance Research Network, Toronto) Available: [https://tspace.library.utoronto.ca/bitstream/1807/87373/1/Wodchis%20et%20al\_2013_Guidelines%20on%20Person-Level%20Costing.pdf](https://tspace.library.utoronto.ca/bitstream/1807/87373/1/Wodchis%20et%20al_2013_Guidelines%20on%20Person-Level%20Costing.pdf). accessed 2021 Mar. 8.
    
    

25. Rosenbaum P (2002) Observational studies, Overt bias in observational studies (Springer, New York), pp 71–104.
    
    

26. Austin PC (2011) Optimal caliper widths for propensity-score matching when estimating differences in means and differences in proportions in observational studies. Pharm Stat 10:150–61.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/pst.433&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20925139&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 

27. Austin PC (2008) Assessing balance in measured baseline covariates when using many-to-one matching on the propensity-score. Pharmacoepidemiol Drug Saf 17:1218–25.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/pds.1674&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18972455&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000261912200011&link_type=ISI) 

28. Austin PC (2011) An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behav Res 46:399–424.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1080/00273171.2011.568786&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21818162&link_type=MED&atom=%2Fcmajo%2F11%2F1%2FE45.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000291533400002&link_type=ISI) 

*   © 2023 CMA Impact Inc. or its licensors