Challenges in conducting research on collaborative mental health care: a qualitative study
==========================================================================================

* Nadiya Sunderji
* Allyson Ion
* Annie Zhu
* Athina Perivolaris
* David Rodie
* Benoit H. Mulsant

## Abstract

**Background:** We sought to understand poor uptake of the Primary Care Assessment and Research of a Telephone Intervention for Neuropsychiatric Conditions with Education and Resources study (PARTNERs), a pragmatic randomized controlled trial of a collaborative care intervention for people experiencing depression, anxiety or at-risk drinking. We explored primary care providers’ experience with PARTNERs, and preferences regarding collaborative care models and trials.

**Methods:** In this qualitative study, we interviewed primary care providers across Ontario who had participated in PARTNERs, using stratified sampling to reach high-, low- and nonreferring providers in urban and rural settings. We audio-recorded, transcribed and thematically analyzed the interviews between May and December 2017, collecting and analyzing data concurrently until achieving saturation.

**Results:** We interviewed 23 primary care providers. They valued the unique availability of telephone-based coaching for patients but desired greater integration of the coach into their practice. They appreciated expert psychiatric recommendations but rarely changed their practices. Sites varied in organizational adoption and implementation of the study, including whether they designated a local champion, proactively identified eligible patients, integrated the study into existing workflows and reflected on (and revised) practices. These behaviours affected continuing awareness of the study and referral rates.

**Interpretation:** Study uptake was influenced by the limited relationship between PARTNERs coaches and primary care providers, and variable attention to leadership, training and quality improvement as vital elements of collaborative care. Study designs focusing on implementation could promote reach and penetration of novel interventions in the practice setting and more successfully advance collaborative care implementation.

Collaborative mental health care involves primary care providers and mental health specialists working together in structured ways to improve access to care, quality of care and outcomes.1,2 It is one of the most empirically supported approaches to achieving good outcomes in primary mental health care2–7 and is integral to mental health and primary care strategies in Canada.8–12 However, well-studied effective models of collaborative care have not been implemented in Canada.13,14

The Primary Care Assessment and Research of a Telephone Intervention for Neuropsychiatric Conditions with Education and Resources study (PARTNERs) ([Clinicaltrials.gov](http://Clinicaltrials.gov), no. [NCT02345122](http://www.cmajopen.ca/lookup/external-ref?link_type=CLINTRIALGOV&access_num=NCT02345122&atom=%2Fcmajo%2F7%2F2%2FE405.atom)) was designed to address geographic and human resource barriers that impede the implementation of collaborative care. It was a pragmatic randomized controlled trial assessing the implementation and effectiveness of a collaborative care model delivered by telephone versus enhanced usual care for people experiencing depression, anxiety or at-risk drinking, including mild to moderate alcohol use disorders.15 The study intervention included telephone-based symptom monitoring and self-management support provided by a care manager, and recommendations from a psychiatrist to guide pharmacotherapy provided by primary care providers. The study outcomes were reduction in symptom severity and improved quality of life, measured 4, 8 and 12 months after baseline. The study had lower than expected referral rates.16 We conducted a qualitative study with the aim of exploring primary care providers’ experiences of PARTNERs, understanding barriers to and facilitators of collaborative care interventions and research, and informing the design of future studies.

## Methods

The full qualitative study protocol is provided in Appendix 1 (available at [www.cmajopen.ca/content/7/2/E405/suppl/DC1](http://www.cmajopen.ca/content/7/2/E405/suppl/DC1)). We attempted to answer the following questions:

*   Perceptions and preferences regarding collaborative care models: How do primary care providers perceive the role for, and the advantages and disadvantages of, the PARTNERs collaborative care intervention, including measurement-based care, care management and psychiatrist decision support?

*   Implementation and uptake of the collaborative care model: What aspects of the care model and its implementation in PARTNERs enabled or hindered participation of primary care providers in the provision of collaborative care?

*   Referrals to the study: What provider, practice, intervention and study factors influenced the rate of referral to the study? How did primary care providers decide who and when to refer?

*   Future research: What features of collaborative care interventions, study design and study processes could increase engagement of primary care providers in future studies?

### Theoretical frameworks

We drew on the Consolidated Framework for Implementation Research17,18 to inform sampling, data collection and initial organization of the data. Our data collection was also guided by the theory of planned behaviour.19,20

### Sampling and recruitment

Eligible participants for this qualitative study were family physicians, nurse practitioners, directors, nurses and social workers involved in implementing research in 1 of the 14 primary care organizations that participated in PARTNERs as of May 2017. These organizations included 8 Family Health Teams, 3 solo-physician family practices, 2 community health centres and 1 student health clinic. Eight sites were located in the Greater Toronto Area, and 6 were in rural Ontario communities; the latter had populations of less than 165 000 and were outside metropolitan commuting zones. The PARTNERs research team knew all eligible participants as referring providers or study liaisons; the qualitative research team did not.

The lead PARTNERs researchers (B.H.M. and A.P.) provided the qualitative study team members (N.S., A.I. and A.Z.) with referral data, from which 2 research team members (N.S. and A.I.) developed a stratified purposive sampling framework whereby we recruited family physicians and nurse practitioners for qualitative interviews based on their individual and clinic referral rates (high, low or nonreferral) and their practice settings (urban v. rural).21,22 We created definitions of high and low referral rates based on the referral data. Practices with high referral rates averaged more than 0.15 referrals per physician or nurse practitioner per month they were involved in PARTNERs. Providers with high referral rates referred more than 0.2 participants per month they were involved in the study, and those with low referral rates referred 0.2 participants or fewer per month. We did not adjust for providers’ practice sizes.

We used criterion sampling to recruit the study participants who were directors, nurses or social workers who had contributed to the conduct of research and typically had acted as PARTNERs study liaisons.

A research coordinator contacted eligible participants by fax, telephone and/or email and invited them to participate in a telephone interview about 60 minutes in length. We contacted 10–20 eligible participants at a time across the various recruitment strata and criteria. Owing to lower response rates among providers in the low- and nonreferring strata, we contacted more of these providers. Ultimately, to maximize participation, we contacted most primary care providers and study liaisons (180 of the 190 who were eligible). We conducted qualitative interviews with 23 people across 12 of the 14 PARTNERs sites between May and December 2017. All participants gave informed consent and received a $200 honorarium.

### Data collection

Our interviews followed a semistructured interview guide that addressed characteristics of the PARTNERs intervention, outer and inner settings (domains of the Consolidated Framework for Implementation Research), implementation processes, and individual perceptions and beliefs that influence intentions and, in turn, behaviour (see Box 1 for further description of topics and Appendix 2, available at [www.cmajopen.ca/content/7/2/E405/suppl/DC1](http://www.cmajopen.ca/content/7/2/E405/suppl/DC1), for the interview guide).17–20 One research team member (A.I.) with experience in qualitative research conducted the interviews, which were 46–74 minutes in length, audio-recorded and transcribed.

Box 1: 
### **Topics explored in interviews**

First impressions of the Primary Care Assessment and Research of a Telephone Intervention for Neuropsychiatric Conditions with Education and Resources study (PARTNERs) Beliefs about research and randomization Perceptions of collaborative care model components:

*   In general

*   As applied in PARTNERs

Factors influencing motivation to refer

Factors influencing actual completion of referral

Organizational culture:

*   Social networks

*   Climate

*   Leadership

Implementation processes:

*   Planning

*   Engaging (e.g., marketing or training)

*   Leading or championing

*   Reflecting

### Data analysis

We conducted a qualitative thematic analysis that drew on grounded theory to develop a mid-level theory to explain primary care providers’ referral patterns in PARTNERs.23,24 Data collection was concurrent with analysis and continued until theoretical and informational saturation was reached, with no emerging themes. Three research team members (N.S., A.I. and A.Z.) created an initial set of codes based on Consolidated Framework for Implementation Research constructs to categorize the interview data, then independently read several transcripts and generated additional codes. The qualitative research team then met and revised the initial codebook, from which A.Z. coded the remaining transcripts and organized the data using NVivo11 software (QSR International). The analysis team met regularly to discuss broader themes and interrelations emerging from the codes, exploring convergent and divergent themes from the various interviews, looking for patterns, building explanations iteratively and considering rival explanations.23,25 We slightly revised the interview guide to saturate certain codes or test the developing theory by seeking confirming or disconfirming cases.21 We also met with the PARTNERs research team twice to triangulate our preliminary analysis with their experiences and perspectives. We used nonleading interviews, triangulation of multiple data sources and a team approach to data analysis to ensure that diverse perspectives emerged, and we maintained a research audit trail to provide transparency about our choices.

### Ethics approval

The qualitative study was approved by the research ethics boards of St. Michael’s Hospital and the Centre for Addiction and Mental Health.

## Results

We interviewed 23 participants, who varied greatly in their degree of engagement with PARTNERs (Tables 1 and 2).

View this table:
[Table 1:](http://www.cmajopen.ca/content/7/2/E405/T1)

Table 1: 
Characteristics of study participants

View this table:
[Table 2:](http://www.cmajopen.ca/content/7/2/E405/T2)

Table 2: 
Study participants by referral stratum

### Perceptions and preferences regarding collaborative care models

Participants generally liked the concepts in the collaborative care model and had specific feedback about how these concepts were enacted in PARTNERs (Table 3). They strongly appreciated the provision of care management and identified telephone-based service as helpful and appealing to some patients; however, they had difficulty establishing personal relationships with the off-site care manager and wanted more reciprocal communication. Participants endorsed the idea of health care system redesign toward more team-based care. However, with limited penetration of the intervention in their setting, most primary care providers did not identify the intervention as changing their clinical practices. Finally, participants identified leadership and training as critical to successful implementation of the study and lacking among the core components of the PARTNERs intervention.

View this table:
[Table 3:](http://www.cmajopen.ca/content/7/2/E405/T3)

Table 3: 
Perceived strengths and weaknesses of collaborative care in PARTNERs

### Study referrals

Primary care providers stated they were more likely to refer patients with depression or anxiety versus at-risk drinking based on their assumptions about who would and would not benefit from the intervention (Table 4). Referrals were typically visit-triggered and therefore subject to the provider’s remembering in the moment; providers were more likely to think of referring for new onset of depression (v. ongoing depression). However, if the patient appeared to be in crisis, randomization became a deterrent; primary care providers prioritized connecting these patients to immediate sources of support (e.g., on-site social worker) and deprioritized discussion of PARTNERs even though these were not mutually exclusive. Providers sometimes referred patients known to be ineligible (e.g., due to an excluded condition) out of a sense of desperation to access mental health services.

View this table:
[Table 4:](http://www.cmajopen.ca/content/7/2/E405/T4)

Table 4: 
Primary care providers’ perspectives on patient and study characteristics that influenced referrals

Despite the PARTNERs team’s continuous outreach efforts, some primary care providers felt they lacked knowledge about the particulars of the study and did not feel confident in making referrals. A recommendation from a colleague within their practice influenced their initial intention to refer to PARTNERs, but after making the decision to refer, peer opinions and norms diminished in importance. If providers perceived that a referred patient had greatly benefitted, this reinforced their expectation of positive outcomes for patients and increased their intention to refer.

Primary care providers often presented PARTNERs to their patient as part of a “menu” of options. Some patients preferred other options (e.g., face-to-face therapy) if it was available in the primary care setting and did not perceive an added value or need for the PARTNERs intervention. Some patients experienced language barriers to participation. Beyond that, patients’ reasons for declining referral to the study are largely unknown. Notably, some primary care providers whom we thought were nonreferrers or low referrers stated that they had discussed the study with many patients who declined to be referred.

### Implementation and uptake of collaborative care model

Various factors facilitated or inhibited primary care providers’ engagement in and referrals to PARTNERs (Appendix 3, available at [www.cmajopen.ca/content/7/2/E405/suppl/DC1](http://www.cmajopen.ca/content/7/2/E405/suppl/DC1)). Leadership and training, team buy-in, infrastructure and integration into existing workflows were important factors contributing to site involvement. Perceived time burden stemming from poor integration into one’s usual workflow or cognitive burden of remembering about the study posed barriers for some. Enabling factors included perceiving PARTNERs as a much-needed resource for patients and the community, and ease of the referral process. Practices that referred many patients had developed site-specific practical, time-efficient and proactive strategies to identify and engage eligible patients, for example, contacting patients who were awaiting telepsychiatry consultation, or embedding assessment of patient eligibility and interest into existing interprofessional intake processes.

## Interpretation

In this study, we explored the experiences, perceptions and preferences of primary care providers and study site liaisons who were involved in a pragmatic randomized controlled trial of a collaborative mental health care intervention. Our participants had positive perceptions of collaborative care models; they appreciated the accessibility of telephone-based care management and “indirect” (provider-to-provider) psychiatric consultation, and expressed interest in team-based care. However, their engagement was influenced by organizational characteristics (e.g., how the decision to participate in the study was made, the involvement of champions and available alternative resources), implementation activities (e.g., integration of the patient identification and referral process into existing workflows) and, to a lesser extent, beliefs of individual primary care providers (e.g., expected outcomes of the intervention, preferences regarding randomization). Our findings confirm Ontario primary care providers’ interest in participating in collaborative care but also identify barriers that may supersede their intentions.

Despite the promise of collaborative care models in improving mental health care in primary care, their successful implementation remains a major gap. Several initiatives have failed to achieve their objectives. Solberg and colleagues26,27 attempted large-scale implementation of collaborative care for depression in the Depression Improvement Across Minnesota — Offering a New Direction (DIAMOND) initiative but experienced variable uptake. As in our study, Whitebird and colleagues28 found that leadership support, primary care provider champions and well-integrated care managers influenced DIAMOND’s implementation and effectiveness, including key intervention processes such as patient activation, at the clinic level. Finally, the Care of Mental, Physical and Substance-Use Syndromes (COMPASS) initiative implemented collaborative care for patients with depression and poorly controlled diabetes or cardiovascular disease in 172 clinics across 8 US states and found large variations between clinic outcomes.29 The authors concluded that these differences were related to many contextual factors that were “unmeasured and unmeasurable.” Our findings and those of the DIAMOND initiative26,27 suggest that these contextual differences are actually measurable. We argue for greater attention toward the following factors when implementing and evaluating future studies: leadership, change management, primary care provider champions, integration of care managers, integration into existing workflows and training for all team members.

More broadly, engagement of primary care physicians is crucial to quality-improvement initiatives aiming to improve health care system integration.30 Successful transformation of the health care system considers the adopter’s perspective and gains his or her investment through meaningful engagement early in the change process and true partnership in decision-making and codesign.31,32

Our findings also resonate with prior research suggesting common challenges in conducting research trials in primary care. Time constraints and assimilation into workflows are widely viewed as hindering research uptake.33–38 Explaining research studies to patients and completing referral processes take time in an already busy clinic schedule, and resources to support these actions are often perceived as inadequate.30,35,37 In PARTNERs, the referral was easy to complete, but the cognitive burden of remembering to offer PARTNERs during clinic visits and the perceived difficulties in engaging patients in mental health care hampered participation in the study. In some cases, primary care providers’ doubts about the clinical relevance of particular research studies can pose a barrier; however, our study participants viewed PARTNERs as highly relevant.34,39–43 Here again, active involvement of champions and peers would facilitate participation by explaining the relevance of the research (making it seem worthwhile) and setting up efficient workflows for recruitment.44–46 These issues might have been anticipated had primary care providers initially been involved in the study design (e.g., as part of the study team).

### Limitations

Our definitions of high and low referral rates were not derived from the literature. We did not collect information on participants’ gender. The low referrers and non-referrers we interviewed had limited knowledge of the PARTNERs intervention (variously attributed to such factors as missed meetings, unread emails and staff turnover), such that their recommendations were based on their practice experience and their conceptual understanding of collaborative care but were not grounded in direct experience with PARTNERs; this may have affected the quality of their interviews. Participants’ descriptions of their experience with PARTNERs may have also been affected by the time elapsed since referring or having patients actively engaged in the study, and by not knowing whether their patient(s) had been allocated to the intervention or control arm of the trial. Participants may have been biased by the honorarium in their decision to participate or in the perspectives they expressed. Patients were not interviewed for this qualitative study. Feedback from patients who completed the PARTNERs study (described elsewhere15) was positive but does not help us understand the perspectives of those who chose not to participate.

### Conclusion

Future studies should test approaches to enhance collaborative care delivery at a distance and to build relationships between the central and distal sites. Leadership and training are key but often overlooked components of the chronic care model that should be emphasized in future studies. Finally, implementation research in collaborative care is critically needed to understand how to modify day-to-day practice and sustain these changes over time. Using implementation science and study designs conducive to practice-level implementation (e.g., cluster randomization or stepped-wedge trials) is essential to ensure adoption and sustainability, and to generate new and useful knowledge about the real-world effectiveness of collaborative care.

## Acknowledgement

The authors thank Laura Carvalhal Schoffel for assistance with recruitment for qualitative interviews.

## Footnotes

*   **Competing interests:** Benoit Mulsant reports grants from Brain Canada, the Canadian Institutes of Health Research, the Centre for Addiction and Mental Health Foundation, the Patient-Centered Outcomes Research Institute and the US National Institutes of Health, and nonfinancial support from the US National Institutes of Health, Eli Lilly, Pfizer, Capital Solution Design, HAPPYneuron and General Electric outside the submitted work. No other competing interests were declared.

*   This article has been peer reviewed.

*   **Contributors:** Nadiya Sunderji, Allyson Ion, Benoit Mulsant and Athina Perivolaris conceived and designed the study. Nadiya Sunderji, Allyson Ion and Annie Zhu acquired and analyzed the data. Nadiya Sunderji, Allyson Ion and Annie Zhu drafted the manuscript, and Benoit Mulsant, Athina Perivolaris and David Rodie revised it critically for important intellectual content. All of the authors contributed to interpreting the data, approved the version to be published and agreed to be accountable for all aspects of the work.

*   **Funding:** This research was funded by Bell Canada and the Centre for Addiction and Mental Health Foundation.

*   **Disclaimer:** The opinions, results and conclusions reported in this article are those of the authors and are independent from the funding source. The funders had no role in the design and conduct of the study, collection, management, analysis and interpretation of the data, and preparation, review or approval of the manuscript.

*   **Supplemental information:** For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/7/2/E405/suppl/DC1](http://www.cmajopen.ca/content/7/2/E405/suppl/DC1).

## References

1.  Kates N, Mazowita G, Lemire F, et al. (2011) The evolution of collaborative mental health care in Canada: a shared vision for the future. Can J Psychiatry 56:1–10.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22014688&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

2.  Vanderlip ER, Rundell J, Avery M, et al. (2016) Dissemination of integrated care within adult primary care settings: the collaborative care model (American Psychiatric Association and Academy of Psychosomatic Medicine, Washington) Available: [www.psychiatry.org/psychiatrists/practice/professional-interests/integrated-care/collaborative-care-model](http://www.psychiatry.org/psychiatrists/practice/professional-interests/integrated-care/collaborative-care-model). accessed 2016 Sept 14.
    
    

3.  Archer J, Bower P, Gilbody S, et al. (2012) Collaborative care for depression and anxiety problems. Cochrane Database Syst Rev 10:CD006525.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1002/14651858.CD006525.pub2&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23076925&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

4.  Butler M, Kane RL, McAlpine D, et al. (2008) Evidence report no 173, Integ ration of mental health/substance abuse and primary care (Agency for Healthcare Research and Quality, Rockville (MD)) Available: [www.ahrq.gov/downloads/pub/evidence/pdf/mhsapc/mhsapc.pdf](http://www.ahrq.gov/downloads/pub/evidence/pdf/mhsapc/mhsapc.pdf). accessed 2012 Feb 26.
    
    

5.  Gilbody S, Bower P, Fletcher J, et al. (2006) Collaborative care for depression: a cumulative meta-analysis and review of longer-term outcomes. Arch Intern Med 166:2314–21.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/archinte.166.21.2314&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17130383&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000242312700003&link_type=ISI) 

6.  Woltmann E, Grogan-Kaylor A, Perron B, et al. (2012) Comparative effectiveness of collaborative chronic care models for mental health conditions across primary, specialty, and behavioral health care settings: systematic review and meta-analysis. Am J Psychiatry 169:790–804.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1176/appi.ajp.2012.11111616&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22772364&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000306888900007&link_type=ISI) 

7.  (2008) Integrating mental health into primary care: a global perspective (World Health Organization), (World Organization of Family Doctors, Geneva), (Singapore) Available: [www.who.int/mental_health/policy/services/mentalhealthintoprimarycare/en/](http://www.who.int/mental_health/policy/services/mentalhealthintoprimarycare/en/). accessed 2013 Nov 13.
    
    

8.  (2011) Open minds, healthy minds: Ontario’s comprehensive mental health and addictions strategy (Ontario Ministry of Health and Long-Term Care, Toronto) Available: [www.health.gov.on.ca/en/common/ministry/publications/reports/mental\_health2011/mentalhealth_rep2011.pdf](http://www.health.gov.on.ca/en/common/ministry/publications/reports/mental_health2011/mentalhealth_rep2011.pdf). accessed 2014 Apr 14.
    
    

9.  The action plan for mental health in New Brunswick 2011–18 (Province of New Brunswick, Fredericton) Available: [www.gnb.ca/0055/pdf/2011/7379%20english.pdf](http://www.gnb.ca/0055/pdf/2011/7379%20english.pdf). accessed 2016 Nov 16.
    
    

10. (2011) Creating connections: Alberta’s addiction and mental heath action plan: 2011–2016 (Government of Alberta), Available: [www.albertaaddictionserviceproviders.org/ccactionplan2011.pdf](http://www.albertaaddictionserviceproviders.org/ccactionplan2011.pdf). accessed 2016 Nov 16.
    
    

11. (2010) Healthy minds, healthy people: a 10-year plan to address mental health and substance use in British Columbia (BC Ministry of Health Services and Ministry of Children and Family Development, Victoria) Available: [www.health.gov.bc.ca/library/publications/year/2010/healthy\_minds\_healthy\_people.pdf](http://www.health.gov.bc.ca/library/publications/year/2010/healthy_minds_healthy_people.pdf). accessed 2016 Nov 16.
    
    

12. (2011) *A vision for Canada: family practice — the patient’s medical home* [position paper] (College of Family Physicians of Canada, Mississauga (ON)) Available: [www.cfpc.ca/uploadedFiles/Resources/Resource\_Items/PMH\_A\_Vision\_for\_Canada.pdf](http://www.cfpc.ca/uploadedFiles/Resources/Resource\_Items/PMH\_A_Vision_for_Canada.pdf). accessed 2013 Nov 10.
    
    

13. Sunderji N, Ghavam-Rassoul A, Ion A, et al. (2016) Driving improvements in the implementation of collaborative mental health care: a quality framework to guide measurement, improvement and research (St. Michael’s Hospital, Toronto) Available: [www.researchgate.net/publication/312539646\_Driving\_Improvements\_in\_the\_Implementation\_of\_Collaborative\_Mental\_Health\_Care\_A\_Quality\_Framework\_to\_Guide\_Measurement\_Improvement\_and\_Research](http://www.researchgate.net/publication/312539646\_Driving\_Improvements\_in\_the\_Implementation\_of\_Collaborative\_Mental\_Health\_Care\_A\_Quality\_Framework\_to\_Guide\_Measurement_Improvement_and_Research). accessed 2017 Jan 11.
    
    

14. Mulvale G, Danner U, Pasic D (2008) Advancing community-based collaborative mental health care through interdisciplinary family health teams in Ontario. Can J Commun Ment Health 27:55–73.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7870/cjcmh-2008-0018&link_type=DOI) 

15. Zaheer S, Garofalo V, Rodie D, et al., PARTNERs Study Group (2018) Computer-aided telephone support for primary care patients with common mental health conditions: randomized controlled trial. JMIR Ment Health 5:e10224.
    
    

16. Navaratnam N, Fitzgibbon K, Garofalo V, et al. Canadian Psychiatric Association annual conference (2016 Sept 22–24, Toronto), CAMH Integrated Care Research Project: a telephone-based intervention for mental health problems in primary care patients [poster].
    
    

17. Consolidated Framework for Implementation Research (main page), Available: [http://cfirguide.org/](http://cfirguide.org/). accessed 2017 Jan 30.
    
    

18. Damschroder LJ, Aron DC, Keith RE, et al. (2009) Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 4:50.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1748-5908-4-50&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19664226&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

19. Ajzen I (2006) Behavioral interventions based on the theory of planned behavior (University of Massachusetts Amherst, Amherst (MA)) Available: [https://people.umass.edu/aizen/pdf/tpb.intervention.pdf](https://people.umass.edu/aizen/pdf/tpb.intervention.pdf). accessed 2017 Jan 29.
    
    

20. Ajzen I (1991) The theory of planned behavior. Organ Behav Hum Decis Process 50:179–211.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/0749-5978(91)90020-T&link_type=DOI) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1991GQ64400003&link_type=ISI) 

21. Palinkas LA, Horwitz SM, Green CA, et al. (2015) Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Adm Policy Ment Health 42:533–44.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s10488-013-0528-y&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24193818&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

22. Onwuegbuzie AJ, Collins KMT (2007) A typology of mixed methods sampling designs in social science research. Qual Rep 12:281–316.
    
    

23. Strauss AL, Corbin JM (1990) Basics of qualitative research: grounded theory procedures and techniques (Sage Publications, Newbury Park (CA)).
    
    

24. 1.  Strauss AL, 
    2.  Corbin JM
    
    , eds (1997) Grounded theory in practice (Sage Publications, Thousand Oaks (CA)).
    
    

25. Glaser BG, Strauss AL (1999) The discovery of grounded theory: strategies for qualitative research (Transaction Publishers, Piscataway (NJ)).
    
    

26. Solberg LI, Crain AL, Maciosek MV, et al. (2015) A stepped-wedge evaluation of an initiative to spread the collaborative care model for depression in primary care. Ann Fam Med 13:412–20.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiYW5uYWxzZm0iO3M6NToicmVzaWQiO3M6ODoiMTMvNS80MTIiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzcvMi9FNDA1LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

27. Solberg LI, Crain AL, Jaeckels N, et al. (2013) The DIAMOND initiative: implementing collaborative care for depression in 75 primary care clinics. Implement Sci 8:135.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1748-5908-8-135&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24238225&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

28. Whitebird RR, Solberg LI, Jaeckels NA, et al. (2014) Effective implementation of collaborative care for depression: What is needed? Am J Manag Care 20:699–707.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25365745&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

29. Rossom RC, Solberg LI, Magnan S, et al. (2017) Impact of a national collaborative care initiative for patients with depression and diabetes or cardiovascular disease. Gen Hosp Psychiatry 44:77–85.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=http://www.n&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

30. Goodyear-Smith F, York D, Petousis-Harris H, et al. (2009) Recruitment of practices in primary care research: the long and the short of it. Fam Pract 26:128–36.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/fampra/cmp015&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19251761&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000264190000009&link_type=ISI) 

31. Dickinson H, Ham C (2008) Engaging doctors in leadership: review of the literature (University of Birmingham, Birmingham (UK)) Available: [www.birmingham.ac.uk/Documents/college-social-sciences/social-policy/HSMC/research/leadership-literature-review.pdf](http://www.birmingham.ac.uk/Documents/college-social-sciences/social-policy/HSMC/research/leadership-literature-review.pdf). accessed 2019 Jan 31.
    
    

32. (2011) Engagement: the grey literature — What’s known about engagement in the NHS, and what do we still need to find out? (Aston Business School, Aston University, Birmingham (UK)) Available: [https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment\_data/file/215456/dh_129659.pdf](https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/215456/dh_129659.pdf). accessed 2019 Jan 31.
    
    

33. Bevan H (2018) Being ready for change (Guy’s and St Thomas’ Foundation Trust, London (UK)) Available: [www.slideshare.net/helenbevan/being-ready-for-change](http://www.slideshare.net/helenbevan/being-ready-for-change). accessed 2019 Jan 31.
    
    

34. McKinstry B, Hammersley V, Daly F, et al. (2007) Recruitment and retention in a multicentre randomised controlled trial in Bell’s palsy: a case study. BMC Med Res Methodol 7:15.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2288-7-15&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17391510&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

35. Page MJ, French SD, McKenzie JE, et al. (2011) Recruitment difficulties in a primary care cluster randomised trial: investigating factors contributing to general practitioners’ recruitment of patients. BMC Med Res Methodol 11:35.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2288-11-35&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21453543&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

36. van der Windt DA, Koes BW, van Aarst M, et al. (2000) Practical aspects of conducting a pragmatic randomised trial in primary care: patient recruitment and outcome assessment. Br J Gen Pract 50:371–4.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czoxMDoiNTAvNDU0LzM3MSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNy8yL0U0MDUuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

37. Rosemann T, Szecsenyi J (2004) General practitioners’ attitudes towards research in primary care: qualitative results of a cross sectional study. BMC Fam Pract 5:31.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2296-5-31&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15613246&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

38. Salmon P, Peters S, Rogers A, et al. (2007) Peering through the barriers in GPs’ explanations for declining to participate in research: the role of professional autonomy and the economy of time. Fam Pract 24:269–75.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/fampra/cmm015&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17504773&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000248846900011&link_type=ISI) 

39. Ngune I, Jiwa M, Dadich A, et al. (2012) Effective recruitment strategies in primary care research: a systematic review. Qual Prim Care 20:115–23.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22824564&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

40. Foy R, Parry J, Duggan A, et al. (2003) How evidence based are recruitment strategies to randomized controlled trials in primary care? Experience from seven studies. Fam Pract 20:83–92.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/fampra/20.1.83&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12509377&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000180892800016&link_type=ISI) 

41. Johnston S, Liddy C, Hogg W, et al. (2010) Barriers and facilitators to recruitment of physicians and practices for primary care health services research at one centre. BMC Med Res Methodol 10:109.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2288-10-109&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21144048&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

42. Kenealy TW, Hao’uli S, Arroll B (2015) A qualitative study of recruiting for investigations in primary care: plan, pay, minimise intermediaries and keep it simple. SAGE Open Med 3:2050312115596649.
    
    

43. Peto V, Coulter A, Bond A (1993) Factors affecting general practitioners’ recruitment of patients into a prospective study. Fam Pract 10:207–11.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/fampra/10.2.207&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=8359613&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1993LK45000018&link_type=ISI) 

44. Asch S, Connor SE, Hamilton EG, et al. (2000) Problems in recruiting community-based physicians for health services research. J Gen Intern Med 15:591–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1046/j.1525-1497.2000.02329.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=10940152&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000089054300010&link_type=ISI) 

45. Graffy J, Grant J, Boase S, et al. (2009) UK research staff perspectives on improving recruitment and retention to primary care research; nominal group exercise. Fam Pract 26:48–55.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1093/fampra/cmn085&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19011173&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000262961800008&link_type=ISI) 

46. Leathem CS, Cupples ME, Byrne MC, et al. (2009) Identifying strategies to maximise recruitment and retention of practices and patients in a multicentre randomised controlled trial of an intervention to optimise secondary prevention for coronary heart disease in primary care. BMC Med Res Methodol 9:40.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2288-9-40&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19545366&link_type=MED&atom=%2Fcmajo%2F7%2F2%2FE405.atom) 

*   Copyright 2019, Joule Inc. or its licensors