Choosing Wisely Canada Students and Trainees Advocating for Resource Stewardship (STARS) campaign: a descriptive evaluation
===========================================================================================================================

* Franco Cardone
* Daphne Cheung
* Angela Han
* Karen B. Born
* Lisa Alexander
* Wendy Levinson
* Brian M. Wong

## Abstract

**Background:** Resource stewardship is being increasingly recognized as an essential competency for physicians, but medical schools are just beginning to integrate this into education. We describe the evaluation of Choosing Wisely Canada's Students and Trainees Advocating for Resource Stewardship (STARS) campaign, a student-led campaign to advance resource stewardship education in medical schools across Canada.

**Methods:** We evaluated the campaign 6 months after its launch, in November 2015. STARS students were administered a telephone survey eliciting a description of the initiatives that they had implemented or planned to implement at their schools to promote resource stewardship, and exploring their perceptions of facilitators of and barriers to successful implementation of their initiatives. We used a mixed-methods approach to analyze and summarize the data.

**Results:** Twenty-seven (82%) of the 33 eligible students representing all 17 medical schools responded. In 14 schools (82%), students led various local activities (e.g., interest groups, campaign weeks) to raise awareness about resource stewardship among medical students and faculty. Students contributed to curriculum change (both planned and implemented) at 10 schools (59%). Thematic analysis revealed key program characteristics that facilitated success (e.g., pan-Canadian student network, local faculty champion) as well as barriers to implementing change (e.g., complex processes to change curriculum, hierarchical nature of medical school).

**Interpretation:** This student-led campaign, with support from local faculty and Choosing Wisely Canada staff, led to awareness-building activities and early curricula change at medical schools across Canada. Future plans will build on the initial momentum created by the STARS campaign to sustain and spread local initiatives.

Choosing Wisely Canada is a campaign that promotes resource stewardship by helping physicians and patients engage in conversations about unnecessary tests and treatments.1 Physicians' ordering behaviours and resource use are strongly influenced by practices that they were exposed to during training.2,3 In addition, the Royal College of Physicians and Surgeons of Canada updated its CanMEDS Physician Competency Framework in 2015 and now clearly identifies resource stewardship as a key physician competency within the "Leader" role that must be addressed in residency training.4 However, given that these changes are relatively recent, many curricula and residency programs at Canadian medical schools do not have consistent content on resource stewardship.5,6 A recent survey showed that US medical students feel that physicians have a responsibility to contribute to health care system stewardship and should play a more prominent role in reducing unnecessary care.7

Given the importance of training in establishing practice habits and the gap in medical curricula, Choosing Wisely Canada established medical education as a key strategic priority. As a first step, Choosing Wisely Canada partnered with national medical student associations to release a list of "Six things medical students and trainees should question" to highlight the importance of integrating resource stewardship in medical education.8 Next, in November 2015, Choosing Wisely Canada launched the Students and Trainees Advocating for Resource Stewardship (STARS) campaign (see Box 1 for a detailed overview and Table 1 for the key program elements) and engaged 2 medical students from each of Canada's 17 medical schools to catalyze grassroots, student-led initiatives to advance resource stewardship in medical education at their schools. The main objectives of this descriptive study were to 1) describe how the STARS campaign was implemented, 2) summarize the STARS initiatives and 3) identify systemic facilitators and barriers encountered by STARS students that could guide the improvement of the STARS campaign to optimize its impact.

Box 1:  
### Overview of Students and Trainees Advocating for Resource Stewardship (STARS) campaign

• Choosing Wisely Canada leadership communicated with the deans of all 17 medical schools to inform them about STARS and asked each of them to sponsor 2 medical students to participate in the campaign.

• Students were then recruited to apply through an open call using local medical school email distribution lists and social media, facilitated by the Canadian Federation of Medical Students (CFMS) and Fédération médicale étudiante du Québec (FMÉQ).

• 33 students were selected (16 medical schools sponsored 2 student leaders, and 1 medical school sponsored 1 student leader) from the 114 applications received to attend a 1-day Leadership Summit hosted by Choosing Wisely Canada. Wherever possible, 1 first-year and 1 second-year student from each school were selected. The rationale was that this would allow first-year students to carry forward and facilitate sustainability of STARS initiatives at their schools. Second-year students also have more experience and may have more reach within the student body.

• The deans at each of the medical schools funded all travel-related costs for their school's students. 

• The CFMS and FMÉQ both identified a student to represent the organization and participate in the summit.

• The main goals of the Leadership Summit were to introduce student leaders to core resource stewardship principles and the broader Choosing Wisely Canada campaign, and to equip them with leadership, advocacy and communication skills to launch grassroots initiatives at their local schools to promote resource stewardship.

• Additional program elements, such as Cloud-based resource sharing, ongoing communication through social media and teleconferences, and identification of faculty to support STARS student-led activities, supported local change efforts and fostered the establishment of a pan-Canadian network of STARS students.

View this table:
[Table 1:](http://www.cmajopen.ca/content/5/4/E864/T1)

Table 1:   Key elements of Students and Trainees Advocating for Resource Stewardship campaign

## Methods

### Study design and sources of data

This descriptive study consisted of a telephone survey of STARS students (Appendix 1, available at [www.cmajopen.ca/content/5/4/E864/suppl/DC1](http://www.cmajopen.ca/content/5/4/E864/suppl/DC1)) and 1 question emailed to all medical school deans. The student survey elicited a description of the initiatives that the students had implemented or planned to implement at their schools to promote resource stewardship. The survey also explored the students' perceptions of facilitators of and barriers to successful implementation of their initiatives, and included open-ended questions to solicit feedback on the usefulness of the various program elements (Table 1). We pilot-tested the survey with 3 past Choosing Wisely Canada summer students and modified the survey accordingly.

We invited all STARS students to participate. Two summer students (D.C. and F.C.) administered the survey. To minimize bias, the summer students who collected the data were not involved in the organization of STARS. When possible, students from the same school answered the survey together on the same call; however, some students responded individually owing to scheduling issues. Interviews ranged from 30 minutes to 1 hour in duration. We audiorecorded the telephone conversations to ensure accuracy of data collection and entered numeric and narrative data into our study database, removing any potential identifying information. Student participants were anonymized to ensure confidentiality.

We asked the deans for a narrative description of the impact of STARS at their school in order to triangulate data collected from our medical student questionnaire. We expected that an in-depth questionnaire would result in a low response rate given how busy the deans are and so elected to send them the following question and asked them to provide a detailed response: "What impact have the STARS students had with respect to resource stewardship at your medical school?"

### Descriptive analysis

We used descriptive statistics to summarize survey results, employing counts and percentages to summarize results. The medical school was the unit of analysis, and so in cases in which 2 students from the same school responded, we combined their responses. We analyzed the open-ended narrative responses from STARS students and deans inductively using a thematic analysis. Three study team members (F.C., D.C., K.B.B.) independently read the interview transcripts and identified key themes regarding facilitators and barriers; consensus was reached through discussion. Our analysis was reflexively mindful of our own subject positions in the research context; we remained cognizant of the forces that affected our analysis of the data (e.g., our own sex, professional roles and social positions). In particular, 3 coauthors (B.W., K.B.B., W.L.) hold national leadership positions in Choosing Wisely Canada, and 1 student coauthor is enrolled in a physician leadership stream in our medical school.

### Ethics approval

The study was approved by the University of Toronto Research Ethics Board.

## Results

Twenty-seven (82%) of the 33 students representing the 17 medical schools provided consent and responded to the survey. Students who did not participate were unavailable owing to logistical reasons or did not respond to the interview request. In 19 cases, students from the same school answered the survey together on the same telephone call, and 8 students responded individually. Of the 27 students, 17 were women and 10 were men. Fourteen will graduate in 2019, 12 will graduate in 2018, and 1 graduated in 2017. Eleven (65%) of the 17 deans or a delegate responded to the email request.

### Student-led initiatives

Students at 6 schools (35%) planned curricular changes. Planning activities included meeting with educational leaders, reviewing course materials to highlight existing resource stewardship content or identify opportunities to integrate resource stewardship principles, and carrying out needs assessment. At 4 schools (24%), students successfully implemented curricular changes, including the introduction of resource stewardship content into small-group learning sessions, lectures and clinical skills sessions.

Many students led a variety of other initiatives intended to raise awareness of and generate interest in Choosing Wisely Canada and resource stewardship. Since STARS was designed to be grassroots and student-led, initiatives varied substantially across schools (Table 2), reflecting the personal interests of the students as well as the unique educational environment of each school. Examples of student-led initiatives included creating a Choosing Wisely Canada student interest group (implemented at 2 schools [12%] and approved at 4 schools [24%]), organizing a campaign week to raise awareness about Choosing Wisely Canada and resource stewardship (4 schools [24%]), publishing newsletters or blogs about resource stewardship (3 schools [18%]), organizing special presentations on resource stewardship (6 schools [35%]) and introducing articles focused on resource stewardship in journal club discussions (2 schools [12%]). In additional, at 8 schools (47%), students performed a needs assessment (outside of curriculum planning activities) to determine baseline student knowledge and attitudes with regard to resource stewardship.

View this table:
[Table 2:](http://www.cmajopen.ca/content/5/4/E864/T2)

Table 2:   Student-led initiatives

### Medical school deans' perceptions

The nature of the deans' responses did not provide sufficient detail to allow for rigorous thematic analysis. In several cases, deans provided a summary that they had asked their STARS students to prepare. However, their responses gave the impression that they were aware of the STARS students' contributions and that they had a favourable view of the STARS campaign in general.

### Facilitators of student-led initiatives

Table 3 summarizes STARS program elements and student perceptions of how these facilitated successful implementation of their STARS initiatives. Overall, program elements fell into 3 broad categories. First, there were program elements students felt were extremely helpful in facilitating their overall success. For example, students reported that the Leadership Summit itself was essential and provided the foundational knowledge and skills to create change. The summit also contributed to establishing a pan-Canadian network of STARS students. Students recounted numerous examples of connecting with each other both in person and through videoconference in order to share resources, provide peer feedback and, in some instances, work together to organize interschool activities. Students also described the conference calls that took place every 2 months as a highly useful forum to discuss project progress and seek advice from other students and Choosing Wisely Canada leadership.

View this table:
[Table 3:](http://www.cmajopen.ca/content/5/4/E864/T3)

Table 3:   Students' perceptions of facilitators of student-led initiatives and the usefulness of program elements of the Students and Trainees Advocating for Resource Stewardship campaign

Second, some program elements, particularly those based at the local medical school, were perceived to be helpful facilitators but were not consistently accessible or available to all students. For example, at 10 schools (59%), students had support from a local faculty mentor, which was critical to their overall success. They described finding like-minded faculty engaged in aligned work to be akin to finding "an ally."

Finally, some program elements, although generally seen as "nice to have," were not perceived to be critical facilitators of students' overall success in implementing change at their school. For example, the Facebook group, which was widely accessed by students, was seen as a nice venue to share stories and provide updates but was not frequently used by students to seek advice from one another.

### Barriers to implementation of student-led initiatives

Students identified barriers to implementing STARS initiatives at their medical schools (Table 4), which we grouped into 3 broad categories: 1) students limited as change agents, 2) curricular change is complex and 3) structural barriers. The multiple commitments and competing demands of medical school left some students with limited time to lead local initiatives and implement curricular change. Students also expressed difficulty recommending changes to the curriculum for upper-year students when they themselves had not yet reached that stage of medical training. A few students reported that it was difficult to influence change at their schools owing to the perceived hierarchical nature of medical school.

View this table:
[Table 4:](http://www.cmajopen.ca/content/5/4/E864/T4)

Table 4:   Thematic analysis of perceived barriers to change

Students who struggled to initiate curricular change found it difficult to navigate the complexities of medical school administration. For some, advocating to incorporate resource stewardship as a new topic was seen as competing for limited curricular time. Furthermore, several schools are undergoing curricular reform, such that the introduction of new content was not seen to be a priority.

Finally, structural barriers, which included local policies and logistical challenges, also posed problems. For example, many schools have deadlines for proposals to approve new interest groups, and these had already elapsed by the time students returned from the Leadership Summit.

## Interpretation

Within a relatively short time, STARS students implemented a wide range of awareness-building activities and initiated plans to integrate, and in several cases added, resource stewardship content into the curriculum. These findings suggest that a national grassroots campaign that capitalizes on student advocacy and leadership can accelerate change and serve as a catalyst to address the urgent need to teach medical students about resource stewardship. STARS students clearly self-identified as leaders, and the time spent learning together in-person at the leadership summit and cocreating a vision for STARS was highly beneficial and led to the establishment of a national network of STARS students. We believe that this pan-Canadian network, with Choosing Wisely Canada playing a key central coordination role, served to fuel the implementation of local initiatives by providing students with a sounding board and a forum that they could turn to for advice and support.

Educational leaders have emphasized the importance of combining culture and curriculum change to promote resource stewardship in medical education.9 A recent systematic review identified the "creation of a supportive environment in which … the presence of role models of delivering high-value, cost-conscious care, and a culture of high-value, cost-conscious care reinforce the desired training goals" as critical to advancing resource stewardship education.10 The STARS campaign, which combines a bottom-up approach enabled by top-down support to catalyze change, encapsulates these principles and has the potential to shift the culture and initiate a conversation with faculty and students about the importance of high-value care.11

Our evaluation of the STARS campaign identified several challenges and has informed further refinements needed to maximize its impact and ensure its sustainability. Although the campaign has sowed the seeds for grassroots student leadership for resource stewardship across Canada, much work is still needed to translate early wins into lasting change. Given the transient nature of medical students and the inevitable transition to clinical clerkship, when students generally have less time to devote to initiatives like STARS, the sustainability of the campaign ultimately requires the infusion of a new cohort of student leaders to expand on the efforts already underway and to lead new initiatives.

The next cohort of student leaders assembled at a Leadership Summit in February 2017 that marked the official launch of the next wave of STARS. Elements that enabled success were continued, and several key modifications based on the program evaluation were made. First, we dovetailed the Leadership Summit with the Choosing Wisely Canada National Meeting to expose students to the broader campaign. We also updated the Leadership Summit to include content on curriculum change because past STARS students identified challenges navigating this process. We also took a more proactive approach to identifying at least 1 faculty member at each school to help mentor students. Perhaps most important, we partnered with students to help codesign the Leadership Summit, and several students delivered presentations, including past STARS students discussing lessons learned from their experiences.

### Limitations

Our study has several potential limitations. Since Choosing Wisely Canada led the evaluation of the STARS campaign, students may have treated the interview like an evaluation of their individual effectiveness, which may have rendered our findings susceptible to social desirability bias. The decision to combine student responses may have resulted in some loss of information. The self-reported nature of our data did not allow for evaluation of whether the initiatives led by students actually resulted in increased awareness or knowledge of resource stewardship principles among students at their local medical schools. Finally, the fact that the landscape nationally with respect to resource stewardship is rapidly shifting makes it difficult to know whether some of the curricular changes reported by students would have occurred independent of the STARS campaign. However, we believe that students' efforts at a minimum contributed to changes to the medical school curriculum.

### Conclusion

By combining bottom-up, student-led change with top-down support from local faculty and the national Choosing Wisely Canada campaign, the STARS campaign led to the implementation of a wide range of awareness-building activities and early curricular change at medical schools across Canada. Our approach could be used by others to promote resource stewardship in medical education. In fact, programs modelled on STARS were launched in September 2017 in the Netherlands and will be launched this month in the United States. Next steps include building on the initial momentum created by the STARS campaign to sustain and spread local initiatives, and evaluating the longer-term impact of curricular change on medical student knowledge, skills and attitudes.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/5/4/E864/suppl/DC1.](http://www.cmajopen.ca/content/5/4/E864/suppl/DC1.)

## Footnotes

*   **Competing interests:** None declared.

*   **Contributors:** Franco Cardone, Daphne Cheung, Angela Han, Karen Born, Lisa Alexander and Brian Wong contributed substantially to the study conception and design. Franco Cardone, Daphne Cheung and Angela Han contributed substantially to data acquisition, and Franco Cardone, Daphne Cheung, Angela Han, Karen Born and Brian Wong contributed substantially to data analysis and interpretation. Franco Cardone, Daphne Cheung, Angela Han, Karen Born, Wendy Levinson and Brian Wong drafted the article and revised it critically for important intellectual content. Franco Cardone and Daphne Cheung contributed equally to the intellectual and other work associated with the role of first author. All of the authors gave final approval of the version to be published and agreed to be accountable for all aspects of the work.

## References

1.  Levinson W, Huynh T (2014) Engaging physicians and patients in conversations about unnecessary tests and procedures: Choosing Wisely Canada. CMAJ 186:325–26.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxODYvNS8zMjUiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzUvNC9FODY0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

2.  Chen C, Petterson S, Phillips R, et al. (2014) Spending patterns in region of residency training and subsequent expenditures for care provided by practicing physicians for Medicare beneficiaries. JAMA 312:2385–93.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2014.15973&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25490329&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE864.atom) 

3.  Sirovich BE, Lipner RS, Johnston M, et al. (2014) The association between residency training and internists' ability to practice conservatively. JAMA Intern Med 174:1640–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jamainternmed.2014.3337&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25179515&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE864.atom) 

4.  Frank JR, Snell L, Sherbino J (2015) editors. CanMEDS 2015 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada.
    
    

5.  Korenstein D (2015) Charting the route to high-value care: the role of medical education. JAMA 314:2359–61.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.2015.15406&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26647256&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE864.atom) 

6.  Nasca TJ, Wess KB, Bagian JP (2014) Improving clinical learning environments for tomorrow's physicians. N Engl J Med 370:991–3.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMp1314628&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24467307&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE864.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000332689100005&link_type=ISI) 

7.  Leep Hunderfund AN, Dyrbye LN, Starr SR, et al. (2017) Role modeling and regional health care intensity: U.S. medical student attitudes toward and experiences with cost-conscious care. Acad Med 92:694–702.
    
    

8.  Lakhani A, Lass E, Silverstein WK, et al. (2016) Choosing wisely for medical education: six things medical students and trainees should question. Acad Med 91:1374–8.
    
    

9.  Levy AE, Shah N, Moriates C, et al. (2014) Fostering value in clinical practice among future physicians: time to consider COST. Acad Med 89:1440.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/ACM.0000000000000496&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=25350335&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE864.atom) 

10. Stammen LA, Stalmeijer RE, Paternotte E, et al. (2015) Training physicians to provide high-value, cost-conscious care: a systematic review. JAMA 314:2384–400.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=26647260&link_type=MED&atom=%2Fcmajo%2F5%2F4%2FE864.atom) 

11. Moriates C, Wong BM, et al. (2016) High-value care programmes from the bottom-up… and the top-down. BMJ Qual Saf 25:821–3.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjk6IjI1LzExLzgyMSI7czo0OiJhdG9tIjtzOjIwOiIvY21ham8vNS80L0U4NjQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

*   Copyright 2017, Joule Inc. or its licensors