Abstract
Background The aim of the Building on Existing Tools to Improve Chronic Disease Prevention and Screening in Family Practice (BETTER) randomized controlled trial is to improve the primary prevention of and screening for multiple conditions (diabetes, cardiovascular disease, cancer) and some of the associated lifestyle factors (tobacco use, alcohol overuse, poor nutrition, physical inactivity). In this article, we describe how we harmonized the evidence-based clinical practice guideline recommendations and patient tools to determine the content for the BETTER trial.
Methods We identified clinical practice guidelines and tools through a structured literature search; we included both indexed and grey literature. From these guidelines, recommendations were extracted and integrated into knowledge products and outcome measures for use in the BETTER trial. End-users (family physicians, nurse practitioners, nurses and dieticians) were engaged in reviewing the recommendations and tools, as well as tailoring the content to the needs of the BETTER trial and family practice.
Results In total, 3–5 high-quality guidelines were identified for each condition; from these, we identified high-grade recommendations for the prevention of and screening for chronic disease. The guideline recommendations were limited by conflicting recommendations, vague wording and different taxonomies for strength of recommendation. There was a lack of quality evidence for manoeuvres to improve the uptake of guidelines among patients with depression. We developed the BETTER clinical algorithms for the implementation plan. Although it was difficult to identify high-quality tools, 180 tools of interest were identified.
Interpretation The intervention for the BETTER trial was built by integrating existing guidelines and tools, and working with end-users throughout the process to increase the intervention’s utility for practice. Trial registration: ISRCTN07170460
Most of the health care activities for improving the prevention of and screening for chronic diseases are implemented in family practice.1–5 About 7.4 hours per day of provider time are required to deliver all manoeuvres recommended within current practice structures.6–13 Because of the multiple demands of family practice, this is not feasible. Primary care practitioners also require effective strategies to concurrently address multiple conditions, because 45% of people have more than one chronic disease.14 In particular, depression — a prevalent comorbidity in family practice — is associated with poor uptake of prevention and screening manoeuvres.15–19 Therefore, promoting practice organization that delivers prevention and screening in an effective and time-efficient manner is a priority.
Cardiovascular disease, diabetes and breast, colorectal and cervical cancer, as well as their associated lifestyle risk factors (e.g., tobacco use, alcohol overuse, poor nutrition, physical inactivity), are prevalent in primary care. There are evidence-based recommendations for the prevention of and screening for these conditions, and following these recommendations can reduce morbidity and mortality. However, there are dozens of clinical practice guidelines with recommendations for each of these conditions, which creates a substantial challenge for providers. This is one of the reasons why the effectiveness of guidelines as tools to facilitate the translation of evidence into practice has been inconsistent.20–22
The harmonization of guidelines and their integration into an implementation plan represent important first steps in addressing this issue.23 In this article, we describe the process by which we harmonized recommendations from clinical practice guidelines and implementation tools for the target conditions. The results of this process were used to inform the interventions in the Building on Existing Tools to Improve Primary Prevention and Screening in Family Practice (BETTER) randomized controlled trial.24
Methods
The knowledge-to-action cycle was the starting framework for the design of the BETTER evidence integration process, the core of which is “knowledge creation.”25 With engagement from end-users (family physicians, nurse practitioners, nurses and dieticians), we identified high-quality clinical practice guidelines, harmonized them to standardize the recommendations, and created knowledge products for the implementation process. Building on previous work by the Centre for Effective Practice and incorporating processes of guideline knowledge translation, we developed a structured approach to the evidence integration (Figure 1).23,25–27
From November 2009 to March 2010, a clinical working group was convened and facilitated by the Centre for Effective Practice; this group included clinicians (family physicians, a nurse practitioner, registered nurses, a licensed practical nurse and a dietician) and methodologists, including a medical librarian. Appendix 1 presents details on the members of the clinical working group (available online at www.cmajopen.ca/content/2/1/E1/suppl/DC1).
The purpose of the working group was to select and integrate high-quality recommendations for the chronic conditions selected for inclusion in the BETTER trial. These included cardiovascular disease, diabetes and breast, colorectal, lung and cervical cancer, as well as the associated lifestyle risk factors (e.g., tobacco use, alcohol overuse, poor nutrition, physical inactivity). The working group considered recommendations relevant to patients aged 40–65 years and those with strong evidence linked to a target or clear health outcome. A separate search was performed for interventions for depression and for patients with strong family history of disease.
Harmonization process
The key steps in the harmonization process are outlined in Figure 1; the complete process, including implementation, based on the knowledge-to-action cycle, is shown in Figure 2.
Step 1: Identify, evaluate and select high-quality clinical practice guidelines and tools
We identified high-quality guidelines through a focused 4-step process (Box 1). This process was based on the search strategy developed by the Guidelines Advisory Committee at the Centre for Effective Practice (Canadian Medical Association Handbook on Clinical Practice Guidelines10). The inclusion criteria were diabetes, cardiovascular disease, breast cancer, colorectal cancer, cervical cancer, skin cancer and lung cancer. The search was limited to guidelines published in English between 2004 and 2009. Guidelines were excluded if they did not include any recommendations related to screening or primary prevention for the areas of interest or were not applicable to family practice. We also performed a search for effective interventions for patients with depression (Box 1).
Four-step search process for high-quality clinical practice guidelines
1. Guideline repositories:
• CMA Infobase: Clinical Practice Guidelines: http://mdm.ca/cpgsnew/cpgs/index.asp
• National Guideline Clearinghouse: www.guidelines.gov
2. Renowned developers:
• National Institute for Health and Clinical Excellence: www.nice.org.uk/
• New Zealand Guidelines Group: www.nzgg.org.nz
• Scottish Intercollegiate Guidelines Network: www.sign.ac.uk/guidelines/index.html
• University of Michigan: http://cme.med.umich.edu/iCME/default.asp
• US Preventive Services Task Force: www.ahrq.gov/clinic/prevnew.htm
3. Websites of national and international specialty societies (e.g., for diabetes: the Canadian Diabetes Association, American Diabetes Association)
4. General Internet Search:
• Google: [Topic] AND [Guideline(s)]. First 3 pages of results examined
5. For the depression CDPS uptake literature review
• Medline and PsychINFO were searched using relevant depression, primary care and prevention MeSH search terms; 239 abstracts were found. A clinical filter was applied; a clinician reviewed the abstracts and identified 17 articles of interest. These articles were then reviewed by 3 other investigators for relevance; debate was resolved by consensus. Of these, 3 were considered of sufficient quality and applicability for subsequent review with the larger clinical working group of investigators for consideration of clinical applicability and acceptance in primary care.
We retrieved the full-text versions of the guidelines identified and evaluated them using the AGREE (Appraisal of Guidelines for Research and Evaluation) instrument,27 with a particular focus on the domains of “rigour of development” and “editorial independence.” This ensured that a systematic search was conducted, recommendations in the guidelines were linked directly to the level of evidence, and that the guideline was developed with minimal bias. In an effort to build on existing work in guideline assessment and adaptation, we asked the Canadian Partnership Against Cancer to provide the full AGREE scores from their previous work for all guidelines for breast, cervical, colorectal, lung and skin cancer published between 2004 and 2008. We identified and assessed additional guidelines in these areas published between 2008 and 2009. We located 3–5 high-quality clinical practice guidelines in each topic area. Canadian guidelines and those published most recently were included if they satisfied the quality criteria.
The BETTER trial’s search and appraisal strategy for existing patient tools is shown in Box 2. Using a structured search of the grey literature, we identified and reviewed 180 clinical tools (> 1000 were considered) for potential inclusion in the intervention component of the BETTER trial (the BETTER tool kit). A selection of tools was chosen for the assessment of alcohol, depression, smoking, diet, exercise and family history. To avoid redundancy, we used a ranking and indexing scheme to help facilitate cataloguing and retrieving high-quality tools to decide which tools to include in the implementation.
Search process for high-quality clinical tools
1. Criteria for consideration
• Published in English
• Address screening and primary prevention of topic area recommendations
2. Search strategy
• Public Health Agency of Canada’s Canadian Best Practices Portal
• Agency for Healthcare Research and Quality
• Preventive, chronic and primary care
• Internet search using Google:
• [Topic] AND [tools] AND [prevention/screening] AND [Canada] AND [patients] AND [providers]
• [Country] AND [topic]. The first 3 pages of results were examined
• Websites of national and provincial specialty societies
3. Criteria for inclusion:
• Screening and prevention tools
• More weight was given to the most up-to-date tools and those from Canadian and American organizations
4. Extraction of tools:
• Unique name, description of tool, focus, source, organization
• Reviewer instructions (type/usefulness and quality/applicability)
• Clinical working group’s overall review for inclusion in tool kit
5. Review process:
• Two independent reviewers, process for resolving differences
• Clear process for criteria to evaluate tools:
• Type of tool/clinical usefulness
• Quality of the tool/applicability
• Validation of the tool in multiple settings
6. Placing tools into tables:
• Includes name, description of tool, focus, source and organization
• Reviewer of Instructions (type/usefulness and quality/applicability)
• Clinical working group’s overall review for inclusion in the tool kit
7. Need a method to index, search and retrieve tools in the literature:
• Dedicated repositories exist, but they would benefit from indexing and preappraisal of the tools
Step 2: Extraction of the relevant recommendations for review
We extracted into evidence tables all relevant recommendations from the highest-quality guidelines in each topic area. These recommendations, along with the level of evidence and strength of recommendation according to the evidence taxonomy used by the particular guideline developer, were presented to the working group for review. The levels of evidence taxonomy for each guideline recommendation for a patient intervention were compared with the prespecified standards for quality. The exception to this quality metric was areas for which a decision for practice implementation had to be made on the basis of lower-quality recommendations, such as the frequency of repeat manoeuvres.
Step 3: Evidence review
In this phase, the guidelines were reviewed by smaller, disease-specific theme groups created from the members of the full clinical working group. The themes included diabetes, cardiovascular disease, cancer, lifestyle modification (smoking, alcohol, physical activity, nutrition), family history and depression. Members with content expertise were active in each group. At least 2 members of each group independently reviewed each recommendation and clinical tool. Disagreements were resolved through discussion among the group. All recommendations identified were presented to the entire working group for final review. For clinically controversial recommendations (i.e., because of information that may have postdated the publication of the guideline), we considered the primary literature. A literature review was conducted to identify strategies to increase the uptake of manoeuvres among patients with depression. To ensure that recommendations were appropriate within the specific provincial contexts, we identified local guidelines and policies from provincial bodies (e.g., provincial cancer screening policies). We compared these local guidelines with the selected recommendations to ensure consistency and applicability.
Step 4: Recommendation integration and knowledge product development
The clinical working group reviewed the recommendations for final inclusion. The group integrated the selected recommendations into a format that was explicit, actionable and measurable. Appendix 2 (available online at www.cmajopen.ca/content/2/1/E1/suppl/DC1) summarizes one example of this, showing the recommendations from 4 different guidelines concerning the timing of screening for diabetes. Each of guideline had a different taxonomy for indicating level of evidence and strength of the recommendation. In the example in Appendix 2, recommendation number 2 is the only one that is specific, comprehensive and actionable.
The clinical working group, Centre for Effective Practice and the research team worked together to integrate the recommendations into knowledge products that would inform the measures to be used in the BETTER trial (defining the manoeuvres, targets and outcomes); inform the clinical approach that would be used to implement the recommendations in family practice; and assist in the training of the trial team and primary care providers about the evidence and how to engage patients in meaningful discussions about the prevention of and screening for chronic diseases.
Results
Integrated recommendations from guidelines
The selected guidelines are presented in Appendix 3 (available online at www.cmajopen.ca/content/2/1/E1/suppl/DC1). We found strong evidence for manoeuvres for the prevention of and screening for breast, colorectal and cervical cancer, diabetes and cardiovascular disease and for the prevention of lung cancer. There was an absence of evidence for screening the general population for skin and lung cancer and for screening for colorectal and breast cancer among patients aged 40–49 years. Given this, the working group recommended that these conditions not be included in the BETTER trial. Screening for breast and colorectal cancer was only recommended for patients aged 40–49 years with a family history indicative of higher risk. The working group requested additional searches for prostate cancer guidelines. Following an additional review, the working group concluded that there was insufficient evidence to recommend screening. The working group concluded that there was no high-quality evidence showing effectiveness of strategies to improve screening and primary prevention in patients with depression.
Knowledge products
We developed clinical algorithms to relay the principal results of the guideline review of the strongest available recommendations for the prevention of and screening for chronic diseases. The algorithms provide a clear approach specifically tailored to the intervention. We developed algorithms for the primary prevention of and screening for cardiovascular disease and diabetes (Figure 3), the prevention of and screening for cardiovascular disease among patients with existing type 2 diabetes (Figure 4), and the prevention of and screening for breast, cervical and colorectal cancer and for the primary prevention of lung cancer (Figure 5).
Interpretation
The dynamic, iterative process used to integrate the guideline recommendations was critical for effective implementation of the intervention in the BETTER trial. Our findings are consistent with the literature: for a recommendation to be clinically useful, it not only has to have robust evidence, it also has to be formulated in a way that is actionable.20–23,28
We disassembled high-quality clinical practice guideline recommendations into their core components and reassembled them into integrated, implementable and measurable recommendations for use in the BETTER trial. We accomplished this using a participatory approach through direct engagement with end-users and researchers, ensuring direct interactivity between the “knowledge creation” process and the “knowledge application” process from the knowledge-to-action cycle, while facilitating greater uptake within the BETTER trial. With end-user engagement, we reformatted the recommendations for patients aged 40–65 years into an algorithm to facilitate clinical decision-making and encourage implementation. Figure 2 illustrates the process of harmonizing of guideline recommendations and implementing them at the practice and patient level. At the patient level, the recommendations implemented were tailored to patients’ personal risk factors and family history.
There are a number of effective, yet under-used, evidence-based manoeuvres for the screening for and prevention of chronic diseases. These manoeuvres are particularly under-used for patients with depression.6–8 No robust primary studies demonstrating solid outcomes improving uptake of manoeuvres were identified for patients with depression in primary care; the lack of high-level evidence for such interventions suggests that further research into this area is needed. Although there is a paucity of studies on harmonizing guidelines across multiple conditions, there is interesting work in this direction with the C-CHANGE initiative in cardiovascular disease diagnosis and management.29
The BETTER trial’s evidence integration process highlights practical considerations contributing to our understanding of guideline implementation challenges in family practice settings and for guidelines encompassing multiple clinical areas. Our findings are consistent with previously published studies showing the complexity in facilitating a process of adaptation or harmonization.23,28 Our iterative review of guideline recommendations, with an eye to clinical applicability, showed that the imprecise language of recommendations results in a discussion about what we should actually “do.” We realized that for use across multiple diseases, we needed to focus on the recommendations that have the greatest effect and best evidence. The variation in taxonomy regarding the strength of recommendations challenged the proper interpretation of the strength of evidence for each guideline. We believe that this is a limitation to the uptake of guidelines and concur with others that to facilitate uptake there is potential value in adopting a consistent taxonomy for level of evidence and strength of recommendation.30 Some have advised that clinicians need to “find guidelines that are systematic and transparent, and then make informed judgments;”31 however, the half-life of all clinical research is 5.5 years,32 making the primary appraisal of all guidelines at point of care impractical. There is a need for guideline developers and knowledge users to develop processes to harmonize clinical practice guidelines.
An important aspect of this work was the iterative approach used to ensure that the evidence and knowledge products were informed by the needs of the users. The process for this continues to be explored through the participatory approach of the ADAPTE process and emerging process for integrated knowledge translation.23,28,33 The BETTER trial gives one example of bringing together end-users and researchers to collaboratively develop knowledge products and training approaches to support integrated guidelines for chronic disease prevention and screening implementation in family practice.
Limitations
It is possible that we may have missed a relevant clinical practice guideline that may have had different recommendations. However, we are confident that our sample was robust because we included recommendations with good evidence supporting their impact on patients, we achieved general thematic consensus across the different guidelines, and our clinical working group did not identify any areas of deficit.
The working group, while attempting to follow best evidence, was required to rely on consensus when the evidence was insufficient or unclear, as in the case of the frequency of repeat measures in practice. Varying interpretations of this evidence, along with differing values with regards to harms and benefits, while ultimately resolved by consensus, relied on group dynamics and decision-making, which are subject to bias.
The rapid rate of new knowledge across the included topics means that the algorithms created for the BETTER trial need to be modified regularly. We have recently undertaken this for the BETTER2 project, which is implementing the patient-level intervention in urban, rural and remote communities in Canada.
Conclusion
A solid evidence base and the inclusion of target end-users in the development of knowledge products are required for initiatives to improve clinical care processes in family practice. An important aspect of this work was the iterative, deliberative process that yielded clinical recommendations that were explicit and measurable, as well as the BETTER clinical algorithms, which were an effective way to summarize the integrated recommendations for implementation. Clinical practice guidelines are useful syntheses of information about single diseases. Improved uptake in practice may be achieved by increasing collaboration between guideline developers and end-users, with a focus on strategies for implementation.
Supplemental information
For reviewer comments and the original submission of this manuscript, please see www.cmajopen.ca/content/2/1/E1/suppl/DC1
Acknowledgements
We acknowledge the clinical contributions of Dr. Andrew Cave, Department of Family Medicine University of Alberta; Dr. Sheila Dunn, Department of Family and Community Medicine, University of Toronto; all of the BETTER clinical working group and implementation team clinicians; Ms. Rachael Wymer and the prevention practitioners. We thank Ms. Shelley Bible RN for her ongoing engagement in training the next generation of prevention practitioners for BETTER2. We also acknowledge the work of the Guidelines Advisory Committee, and specifically thank Dr. Valerie Palda and Dr. Melissa Brouwers for their willingness to share the AGREE scores for the cancer guidelines.
Footnotes
-
Competing interests: Jess Rogers was employed by the Centre for Effective Practice through the BETTER project to facilitate the guideline development process. Sandy Buchman has served as the regional primary care lead for the Toronto Regional Cancer Program. None declared for Denise Campbell-Scherer, Donna Manca, Kelly Lang-Robertson, Stephanie Bell, Ginetta Salvalaggio, Michelle Greiver, Christina Korownyk, Doug Klein, June Carroll, Mel Kahan, Jamie Meuser, Rebekah Barrett and Eva Grunfeld.
-
Contributors: All of the authors contributed substantially to the conception and design of the article and to the drafting and revising of the manuscript. All of the authors approved the final version submitted for publication.
-
Funding: This research was made possible through a financial contribution from the Canadian Partnership Against Cancer and Health Canada. This research is supported by the Heart and Stroke Foundation of Ontario, Grant No. PG 10-0479. Eva Grunfeld is supported by a clinician scientist award from the Ontario Institute for Cancer Research (OICR) with funds from the Ontario Ministry of Research and Innovation (MRI). The opinions, results and conclusions reported are those of the authors and are independent of the funding sources. No endorsement by OICR or MRI is intended or should be inferred.
References
- © 2014 Canadian Medical Association or its licensors