Readiness of hospital-based internists to embrace and discuss high-value care with patients and family members: a single-centre cross-sectional survey study
============================================================================================================================================================

* Daniel Brandt Vegas
* Wendy Levinson
* Geoff Norman
* Sandra Monteiro
* John J. You

## Abstract

**Background:** Choosing Wisely Canada is a campaign that fosters conversations between physicians and patients about high-value health care. However, little is known about physicians' readiness to have these conversations. Our objective was to determine how ready practising internists were to embrace and openly address high-value care during conversations with patients or their families.

**Methods:** Practising internists in hospitals affiliated with McMaster University, Hamilton, Ontario, were invited to complete an electronic survey with 3 clinical scenarios: each had 3 low-value interventions that had been requested by the patient or family member. For each request, participants chose 1 of 3 statements reflecting how they would respond: a low-value statement agreeing to provide the intervention, an implicit high-value statement declining to provide the intervention without mentioning value or an explicit high-value statement declining to provide the intervention with mention of value.

**Results:** Forty-four of 62 eligible physicians (71.0% response rate) participated in the survey. High-value statements were selected in 91% of cases. The implicit high-value statement was chosen more often than the explicit high-value statement (65.7% v. 25.5% of all responses, respectively; χ2 range 4.46-56.23, *p* < 0.05).

**Interpretation:** Physicians favoured high-value care but frequently chose not to explicitly address value in their statements. Physicians seemed ready to embrace high-value health care practice, although they were not ready to openly discuss it with patients and their families.

Value in health care is becoming increasingly important as concerns regarding sustainability and accessibility continue to rise.1-3 Wasteful interventions are common in clinical practice, and inefficiencies seem to be deeply embedded in our medical culture.2-7 Stakeholders at many different levels of the health care system are actively seeking to improve efficiency of care.2,6,8-11 Cost can be considered a financial incurrence, resource use or harm that occurs as a direct consequence of a medical intervention. Value is an assessment of both the medical benefit of a given intervention and its associated costs. A small medical benefit combined with high costs will yield a low‐value intervention and vice versa.

The need for payers and policy-makers to optimize allocation of limited resources at a health system or societal level can conflict with medical decision-making at the patient level, because it challenges physicians' traditional notion of "doing the best for your patient at whatever cost."1,6,8-10 The Choosing Wisely campaign, originally launched in the United States in 2012 and launched in Canada in 2014, aims to educate the public and practitioners about high-value patient care and foster an open conversation about value in health care between patients and their physicians.3,12 Despite these efforts, it is unclear if physicians are ready to embrace this concept in clinical practice.

Currently, there is no consistent framework available to assist physicians in discussing the value of interventions with their patients. We conducted a survey of practising internists at McMaster University, Hamilton, Ontario, to determine how ready they were to embrace and openly address high-value care during conversations with patients or their families by presenting them with common hypothetical clinical scenarios.

## Methods

### Design, setting and participants

We tried to answer 3 different questions: 1) would physicians choose high-value interventions over low-value ones, 2) would physicians openly state to the patient or family member that value had a role in their choice, and 3) would physicians consider cost as an important influence on their choice.

With this purpose we conducted a cross-sectional survey of all physicians practising inpatient internal medicine at 4 teaching hospitals affiliated with McMaster University. The study received full approval from the Hamilton Integrated Research Ethics Board. Given the local culture embracing evidence-based medicine and innovation in health care, which is shown by McMaster University's contributions to the development of evidence-based medicine, our sample may not have been representative of practising physicians across Canada.

#### Study questionnaire

The survey consisted of 3 hypothetical patient scenarios that simulated common patient interactions in hospital-based internal medicine practice (Appendix 1, available at [www.cmajopen.ca/content/3/4/E382/suppl/DC1](http://www.cmajopen.ca/content/3/4/E382/suppl/DC1)). The clinical scenarios were first developed by the authors (D.B.V., W.L., J.J.Y.) based on their clinical expertise as general internists. The clinical sensibility of the scenarios was confirmed during pilot testing, and the content of the scenarios was further refined based on feedback obtained during pilot testing. The scenarios described an elderly woman from a senior care facility with acute delirium, a middle-aged man seen before and after an elective cholecystectomy and a woman with metastatic lung cancer who was diagnosed with a pulmonary embolism resulting in hemodynamic instability. Each patient scenario contained a series of 3 questions in which the patient or their family member asks the physician to provide a low-value intervention (small or no clinical benefit and potentially costly) as determined by the Choosing Wisely and Choosing Wisely Canada recommendations12,13 and clinical practice guidelines.14-17 For each question, participants were asked to choose the response option that most resembled the way they would respond to such a request in clinical practice. There were 3 response options for each question: a statement in which the respondent does not comply with the request and describes a high-value care plan explicitly mentioning value as a reason (explicit high-value option), a statement in which the respondent does not comply with the patient's request and describes a high-value care plan without mentioning value (implicit high-value option) and a statement in which the respondent does comply with the request and agrees to a low-value care plan (low-value option). We also asked participants to rate the importance of each of 6 factors (patient-centred care, prior clinical experience, evidence-based medicine, local hospital practice, medico-legal concerns and cost to the health care system) in influencing their response to each patient or family member request on a 5-point Likert scale with clear anchors (1 = unimportant to 5 = very important).

We pilot-tested an initial draft of the survey with 5 practising and highly experienced internal medicine physicians and received written feedback regarding the similarity of the survey scenarios to actual clinical experiences and the clarity of the response options. Based on their feedback, we developed a revised version of the survey using the same 3 hypothetical patient scenarios but substantially shortened the descriptions. A second round of pilot testing was done with 3 additional nonacademic general internists. Based on their feedback, we modified the language for some of the response options to better reflect what a physician might actually say. A final round of pilot testing was done with 3 recent internal medicine graduates who had not participated in the previous pilot-test rounds. After completing the survey, we held a focus group with them to discuss the clinical sensibility and clarity of each scenario, the questions, the response options, and to obtain feedback on the length of the survey. Based on the focus group feedback we further shortened the description of the clinical scenarios in the final version of the survey.

#### Recruitment and survey delivery

Between June 2013 and January 2014, we invited eligible physicians to complete the survey via email that included a personalized link to the survey, which allowed tracking of survey completion, preserving the anonymity of the survey response data (LimeSurvey, available at [www.limesurvey.org](http://www.limesurvey.org)). Reminders were sent at weeks 1, 3, 7 and 10 after the first email to those who had not completed the survey, following a modified Dillman total design survey method.18-20 After week 10, we contacted nonrespondents by email or telephone to invite them to complete the survey.

### Statistical analysis

We analyzed the study data using IBM SPSS Statistics for Windows, version 22.0. We used descriptive statistics to describe the distribution of responses to each question ("explicit high-value option," "implicit high-value option" and "low-value option"). The frequency of each type of response was analyzed using nonparametric statistics with a χ2 test to compare frequencies within and across all scenarios. Finally, we conducted a repeated-measures analysis of variance to compare the differences of the mean importance of each of the 6 factors (patient-centred care, prior clinical experience, evidence-based medicine, local hospital practice, medico-legal concerns and cost to the health care system) in influencing physicians' responses to requests for low-value interventions from patients or families throughout the entire survey.

## Results

### Participants and response rate

Forty-four of 62 eligible physicians completed the survey, for a response rate of 71.0%. For participant characteristics see Table 1.

View this table:
[Table 1:](http://www.cmajopen.ca/content/3/4/E382/T1)

Table 1:   Survey participant characteristics **(***n*** = 44)**

### Participant responses to requests for low-value interventions

Across the 3 clinical scenarios, most participants (91%) chose 1 of the 2 high-value responses (Table 2). For scenario 1 (patient from nursing home with acute delirium), the low-value response was selected by 8.0% of participants, whereas 1 of the 2 high-value responses was selected by 92.0% of participants. For scenario 2 (patient pre- and postelective cholecystectomy), the low-value response was selected by 8.0% of participants, whereas 1 of the 2 high-value responses was selected by 92.0% of participants. For scenario 3 (patient with metastatic cancer and acute massive pulmonary embolism with prior expressed wishes for a comfort-oriented care plan), the low-value response was selected by 11.0% of participants, whereas 1 of the 2 high-value responses was selected by 89.0% of participants.

View this table:
[Table 2:](http://www.cmajopen.ca/content/3/4/E382/T2)

Table 2:   Physician responses to requests from patients or family members for low-value interventions

### Explicit versus implicit value statements

Although participants almost always selected a high-value option in response to patient or family requests for a low-value intervention, they frequently selected the implicit high-value statement that did not directly mention value rather than the high-value response in which there was an explicit mention of value (Table 2).

### How important was cost in influencing response selection?

Across all scenarios and all questions, on a 5-point Likert scale (1 = unimportant and 5 = very important), cost had a mean importance rating of 3.5 (SD 0.99) in influencing participants' response to hypothetical requests for low-value interventions, between somewhat important and important. The modal response was 4 (important). Overall, cost was rated lower in importance than patient-centred care, evidence-based medicine and prior clinical experience, but more important than local hospital practice and medico-legal concerns. The differences between the mean importance ratings of each of the factors were statistically significant (*F*5,215 = 55.2, *p* < 0.001).

## Interpretation

### Main findings

We found that when a patient or family member requested a low-value medical intervention, most physicians would not endorse it, suggesting participant readiness to embrace high-value care. Although physicians in this study rated cost as a somewhat important influence in their response, we also found that they often chose not to mention value to the patient or family member, which suggests some unwillingness to openly address this during care plan discussions. We did not find evidence for differences in the pattern of responses based on the sex or number of years in practice of the respondents.

### Explanation and comparison with other studies

Cost seems to be a relevant factor in shaping physicians' decision-making about care plans, although it is clearly not the only factor or the most important one. Respondents in our study considered it more important than other well-recognized factors, such as medico-legal concerns and local hospital practice, but ranked it less important than evidence-based medicine, prior clinical experience and patient-centred care in influencing their decisions. This suggests a hierarchy of priorities, in which cost is relevant but not the leading factor in determining a care plan, appropriately less important than scientific knowledge, patient-centredness and clinical experience. These findings are both reassuring and encouraging in the context of the Choosing Wisely Canada campaign, which seeks to introduce value in health care as a part of the care plan discussion but not as the single reason informing medical decisions.

Our findings are also in keeping with previous studies showing that physicians routinely make health care decisions based on cost and value.6,8-10 In 1998, Ubel and Goold6 outlined how most value-based decisions made by physicians are implicit, often occur regarding interventions seen as beneficial but not indispensable and involve resources considered limited but not absolutely scarce. They hypothesized that a more explicit consideration of cost or value may raise questions of rationing, a term frequently associated with negative perceptions.2,9,21 The fear of raising issues of rationing health care resources or perhaps undermining the therapeutic relationship by including value in care plan discussions may have played a part in our findings; however, we did not directly explore the reasons for answers given by the physicians in our survey.

This study offers a novel insight into the perspectives of physicians in practice on the value in health care and their inclination to not openly discuss these issues with patients or their family members. In the context of the recent launching of the Choosing Wisely Canada campaign, which encourages physicians to talk to their patients about value in health care, our findings are encouraging as they suggest physicians are largely ready to embrace high-value care in their daily clinical practice. Our results will help physicians reconsider if value plays a part in their care plan recommendations to patients and if they should explicitly disclose this to the patient or family. Our findings will inform future research to determine if specific clinical situations call for a more explicit discussion of cost and others a more implicit approach.

Our findings also suggest that physicians prefer not to openly mention value in conversations with patients. This apparent contradiction likely reflects a difference between the way high-value care is considered among the medical community and the general population. It further highlights the need for a societal shift of perspective, in which high-value care is better understood, more positively seen and widely embraced by the general population. Previous studies have shown that patients would like to be informed of health care costs; however, patients would rather hear about this from someone other than the physician.22,23Strengths of our study include 3 iterations of pilot testing that led to clearer statements and clinical sensibility of the scenarios. Another strength is the response rate of 71.0%, which increases the representativeness of our study findings.

### Limitations

First, because of the enrolment of participants from hospitals within a single academic institution in Canada, our findings are not necessarily representative of values and practices at other institutions, especially in other countries with different models of health care financing. Second, we used hypothetical patient scenarios rather than direct observations of real interactions with patients, which had the potential to introduce a bias for participants to select socially desirable answers. We tried to reduce this bias by not explicitly stating that cost was the topic being addressed in the survey. Nevertheless, it is possible that in actual practice, physicians might respond differently to patient or family requests for low-value interventions and, for example, may be more likely to agree to low-value care plans when faced with these types of requests in clinical practice. Third, we included a relatively small number (3) of patient scenarios in our survey and did not seek stakeholder input in its initial development. However, we did receive stakeholder input during pilot testing about the clinical sensibility and representativeness of the scenarios to help refine and finalize our survey instrument. Each patient scenario included 3 clinical decisions; therefore, each respondent was in fact confronted with a total of 9 clinical decisions. Although we considered including more scenarios and clinical decisions, we elected to restrict the number of clinical decisions to 9 to not overburden respondents, in the interest of maintaining a high enough response rate to preserve internal validity.

### Conclusion and implications for practice and future research

Future larger studies using direct observation of physician behaviours would be helpful to confirm our findings and to look at clinical scenario and intervention specificity as other potential factors influencing physicians' responses to requests for low-value interventions. Future studies should also focus on the perspective of patients in this context about having care plan discussions openly addressing the value of different interventions and options. This perspective would complement our findings and help develop recommendations to guide clinicians on when and how to address value in conversations with their patients, as well as to inform public health campaigns that will eventually help change the perception of value and resource stewardship in health care by the general population.

Physicians appear ready to embrace high-value care in their clinical practice, although they are less willing to openly address value when discussing care plans with patients or their families. Initiatives such as the Choosing Wisely Canada campaign should encourage and expect physicians to further consider and implement high-value care. However, there is a need to determine if patients are ready to have open conversations about health care value with their doctors, to eventually help shift the general population's perception of high-value care to a better understanding and more positive consideration, and to inform guidelines on how and when these conversations should occur.

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/3/4/E382/suppl/DC1](http://www.cmajopen.ca/content/3/4/E382/suppl/DC1)

## Footnotes

*   Competing interests: None declared.

*   Contributors: Daniel Brandt Vegas, John You, Geoff Norman and Wendy Levinson contributed to the study design. Daniel Brandt Vegas, John You and Geoff Norman obtained the data. Daniel Brandt Vegas, John You, Geoff Norman and Sandra Monteiro analyzed the data. Daniel Brandt Vegas and John You wrote the manuscript. All of the authors criticially revised the manuscript for intellectual content, approved the final version to be published and agreed to act as guarantors of the work.

*   **Funding:** This study was done at the Department of Medicine of McMaster University, funded by an Alternate Funding Plan Grant within the Department of Medicine. The Department of Medicine had no role in the design, implementation, analysis, manuscript writing or dissemination of the study findings.

## References

1.  Emanuel EJ, Fuchs VR (2008) The perfect storm of overutilization. JAMA 299:2789–91.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.299.23.2789&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18560006&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000256805200027&link_type=ISI) 

2.  Rosenbaum L, Lamas D (2012) Cents and sensitivity - teaching physicians to think about costs. N Engl J Med 367:99–101.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMp1205634&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22784112&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000306270800004&link_type=ISI) 

3.  Rao VM, Levin DC (2012) The overuse of diagnostic imaging and the Choosing Wisely initiative. Ann Intern Med 157:574–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-157-8-201210160-00535&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22928172&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000309906000019&link_type=ISI) 

4.  McKenzie JK, Moss AH, Feest TG, et al. (1998) Dialysis decision making in Canada, the United Kingdom, and the United States. Am J Kidney Dis 31:12–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1053/ajkd.1998.v31.pm9428446&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=9428446&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000071345700003&link_type=ISI) 

5.  Eisenberg JM, Williams SV (1981) Cost containment and changing physicians' practice behavior. Can the fox learn to guard the chicken coop? JAMA 246:2195–201.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/jama.1981.03320190053032&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=7197307&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=A1981MP46600026&link_type=ISI) 

6.  Ubel PA, Goold SD (1998) 'Rationing' health care. Not all definitions are created equal. Arch Intern Med 158:209–14.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/archinte.158.3.209&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=9472199&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000071883900001&link_type=ISI) 

7.  Brody H (2012) From an ethics of rationing to an ethics of waste avoidance. N Engl J Med 366:1949–51.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMp1203365&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22551106&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000304353000002&link_type=ISI) 

8.  Hurst SA, Slowther A-M, Forde R, et al. (2006) Prevalence and determinants of physician bedside rationing. J Gen Intern Med 21:1138–43.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1525-1497.2006.00551.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16836629&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 

9.  Wendler D (2010) Are physicians obligated always to act in the patient's best interests? J Med Ethics 36:66–70.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToibWVkZXRoaWNzIjtzOjU6InJlc2lkIjtzOjc6IjM2LzIvNjYiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzMvNC9FMzgyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

10. Ubel PA, Goold S (1998) Does bedside rationing violate patients' best interests? An exploration of "moral hazard." Am J Med 104:64–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0002-9343(97)00172-1&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=9528721&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000071640700010&link_type=ISI) 

11. Weinberger SE (2011) Providing high-value, cost-conscious care: a critical seventh general competency for physicians. Ann Intern Med 155:386, 8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-155-6-201109200-00007&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21930856&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000295033400018&link_type=ISI) 

12. Choosing Wisely [Internet]. Available:[www.choosingwisely.org/](http://www.choosingwisely.org/). accessed 2015 Oct. 21.
    
    

13. Choosing Wisely Canada [Internet]. Available:[www.choosingwiselycanada.org/](http://www.choosingwiselycanada.org/). accessed 2015 Oct. 21.
    
    

14. Nicolle LE, Bradley S, Colgan R, et al. (2005) Infectious Diseases Society of America guidelines for the diagnosis and treatment of asymptomatic bacteriuria in adults. Clin Infect Dis 40:643–54.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiY2lkIjtzOjU6InJlc2lkIjtzOjg6IjQwLzUvNjQzIjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby8zLzQvRTM4Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

15. Rao VM, Levin DC (2012) The overuse of diagnostic imaging and the choosing wisely initiative. Ann Intern Med 157:574–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-157-8-201210160-00535&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22928172&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000309906000019&link_type=ISI) 

16. Rogers G, O'Flynn N (2011) NICE Guideline: transient loss of consciousness (blackouts) in adults and young people. Br J Gen Pract 61:40–2.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiYmpncCI7czo1OiJyZXNpZCI7czo5OiI2MS81ODIvNDAiO3M6NDoiYXRvbSI7czoyMDoiL2NtYWpvLzMvNC9FMzgyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

17. Kearon C, Akl EA, Comerota AJ, et al. (2012) American College of Chest Physicians. Antithrombotic therapy for VTE disease: Antithrombotic therapy and prevention of thrombosis, 9th ed: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines. Chest 141(2 Suppl):e419S–94S.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1378/chest.11-2301&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22315268&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000208839900013&link_type=ISI) 

18. Dillman DA (2000) Mail and Internet surveys: the tailored design method. New York: Wiley;.
    
    

19. Schaefer DR, Dillman DA (1998) Development of a standard e-mail methodology: results of an experiment. Public Opin Q 62:378–97.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1086/297851&link_type=DOI) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000077502500005&link_type=ISI) 

20. Fraenkel JR, Wallen NE, Hyun HH (2012) How to design and evaluate research in education. New York, NY: McGraw Hill;.
    
    

21. Cooke M (2010) Cost consciousness in patient care-What is medical education's responsibility? N Engl J Med 362:1253–5.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMp0911502&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20357275&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000276393700001&link_type=ISI) 

22. Schrag D, Hanger M (2007) Medical oncologists' views on communicating with patients about chemotherapy costs: a pilot survey. J Clin Oncol 25:233–7.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiamNvIjtzOjU6InJlc2lkIjtzOjg6IjI1LzIvMjMzIjtzOjQ6ImF0b20iO3M6MjA6Ii9jbWFqby8zLzQvRTM4Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

23. Danis M, Sommers R, Logan J, et al. (2014) Exploring public attitudes towards approaches to discussing costs in the clinical encounter. J Gen Intern Med 29:223–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1007/s11606-013-2543-9&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=23881272&link_type=MED&atom=%2Fcmajo%2F3%2F4%2FE382.atom) 

*   Copyright 2015, 8872147 Canada Inc.