Quality indicators for the detection and management of chronic kidney disease in primary care in Canada derived from a modified Delphi panel approach
=====================================================================================================================================================

* Karen Tu
* Lindsay Bevan
* Katie Hunter
* Jess Rogers
* Jacqueline Young
* Gihad Nesrallah

## Abstract

**Background:** The detection and management of chronic kidney disease lies within primary care; however, performance measures applicable in the Canadian context are lacking. We sought to develop a set of primary care quality indicators for chronic kidney disease in the Canadian setting and to assess the current state of the disease's detection and management in primary care.

**Methods:** We used a modified Delphi panel approach, involving 20 panel members from across Canada (10 family physicians, 7 nephrologists, 1 patient, 1 primary care nurse and 1 pharmacist). Indicators identified from peer-reviewed and grey literature sources were subjected to 3 rounds of voting to develop a set of quality indicators for the detection and management of chronic kidney disease in the primary care setting. The final indicators were applied to primary care electronic medical records in the Electronic Medical Record Administrative data Linked Database (EMRALD) to assess the current state of primary care detection and management of chronic kidney disease in Ontario.

**Results:** Seventeen indicators made up the final list, with 1 under the category Prevalence, Incidence and Mortality; 4 under Screening, Diagnosis and Risk Factors; 11 under Management; and 1 under Referral to a Specialist. In a sample of 139 993 adult patients not on dialysis, 6848 (4.9%) had stage 3 or higher chronic kidney disease, with the average age of patients being 76.1 years (standard deviation [SD] 11.0); 62.9% of patients were female. Diagnosis and screening for chronic kidney disease were poorly performed. Only 27.1% of patients with stage 3 or higher disease had their diagnosis documented in their cumulative patient profile. Albumin-creatinine ratio testing was only performed for 16.3% of patients with a low estimated glomerular filtration rate (eGFR) and for 28.5% of patients with risk factors for chronic kidney disease. Family physicians performed relatively better with the management of chronic kidney disease, with 90.4% of patients with stage 3 or higher disease having an eGFR performed in the previous 18 months and 83.1% having a blood pressure recorded in the previous 9 months.

**Interpretation:** We propose a set of measurable indicators to evaluate the quality of the management of chronic kidney disease in primary care. These indicators may be used to identify opportunities to improve current practice in Canada.

Chronic kidney disease is common. The median prevalence among adults aged 30 years and older is estimated to be 7.2%, and between 23.4% and 35.8% for people more than 64 years of age.1 Chronic kidney disease has a prevalence similar to that of diabetes2 and engenders at least the same amount of risk for cardiovascular events and death,3-6 yet it does not get as much attention with respect to quality improvement. Studies in Canada,7,8 the United States,9-11 the United Kingdom12,13 and Australia14,15 have universally identified gaps in care and knowledge about chronic kidney disease among patients and providers in both primary care and specialist settings.

Many countries have guidelines for the management of chronic kidney disease,16-18 including Canada.19 In 2014, the Kidney Disease Improving Global Outcomes group published international chronic kidney disease guidelines.20 Guideline statements typically represent important processes of care, such as appropriate use of diagnostic tests and medications. Processes that are supported by high-quality evidence are more likely to improve outcomes; hence, measuring guideline adherence is increasingly used to assess care quality.21 Measures of quality for chronic kidney disease care are uncommon, and are not currently included in major measurement sets such as the American Healthcare Effectiveness Data and Information Set measures.22 The Quality and Outcomes Framework in the UK includes only 1 measure specific to chronic kidney disease.23 National primary care quality indicators in Canada24 currently include no measures related specifically to chronic kidney disease.

We set out to develop a set of primary care quality indicators for chronic kidney disease in the Canadian setting and to assess the current state of the disease's detection and management in the primary care setting using electronic medical record (EMR) data from a representative sample of Ontario physicians and patients.

## Methods

### Quality indicator selection

We used a modified Delphi approach to establish a chronic kidney disease quality indicator measurement set for primary care. We used a multifaceted search strategy of peer-reviewed and grey literature sources to identify chronic kidney disease-related measures used by other organizations. We then performed a focused search to identify clinical practice guidelines specific to the diagnosis and management of chronic kidney disease. We used AGREE II25 criteria (for quality assessment of guidelines) to only include high-quality guidelines, from which we extracted recommendations for consideration by the Delphi panel as evidentiary support for the identified measures. We did not use recommendations from clinical practice guidelines to develop new indicators; rather, the information from the guidelines was used to shape the indicators during the Delphi process (Appendix 1, available at [www.cmajopen.ca/content/5/1/E74/suppl/DC1](http://www.cmajopen.ca/content/5/1/E74/suppl/DC1)).

Identified quality measures were reviewed by the project clinical leads in primary care (KT) and nephrology (GN) for relevance to chronic kidney disease in the primary care setting, and the retained measures were submitted to the Delphi panel for prioritization.

### Panel members and modified Delphi process

We invited 20 panel members from across Canada for participation in a modified Delphi process. Ten were family physicians, 7 were nephrologists with clinical and methodological expertise, 1 was a patient, 1 was a primary care nurse and 1 was a pharmacist. Each panel member completed conflict of interest and consent forms. Panelists completed 3 rounds of ratings of candidate measures using a Web-based tool and a criteria matrix based on an adaptation from previously established criteria26,27 (Figure 1). Panelists participated in a webinar after the first round, to allow for discussion and consensus building. Panelists provided qualitative feedback during the review, and could propose new measures. Measures were excluded at each round according to the criteria in Box 1. Panelists reviewed their own responses, the panel's aggregate responses and qualitative feedback at each round. The study team considered qualitative feedback and Canadian practice guidelines in modifying selected candidate measures to align with Canadian standards (e.g., blood pressure targets). After 3 rounds of rating and the webinar, indicators that met the inclusion criteria (Box 1) were reviewed by the panel and clinical leads for face validity and comprehensiveness to derive the final measurement set.

![Figure 1](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/5/1/E74/F1.medium.gif)

[Figure 1](http://www.cmajopen.ca/content/5/1/E74/F1)

Figure 1 
Indicator rating criteria matrix.

Box 1: 
### Filter criteria

**Exclusion criteria:**

• 75% or more of panel member ratings of the "overall" criteria fell within the bottom 2 tertiles (between 1 and 6 on a 9-point Likert scale)

OR

• 75% or more of panel members' composite ratings (sum of ratings for all 7 sub-criteria) fell within the bottom 2 tertiles (7-48)

**Inclusion criteria:**

• 75% or more of panel member ratings of the "overall" criteria fell within the top tertile (between 7 and 9 on a 9-point Likert scale)

OR

• 75% or more of panel members' composite ratings (sum of ratings for all 7 sub-criteria) fell within the top tertile (49-63)

AND

• Median "overall" score ≥ 7

### Measurement of quality indicators

Using our final measurement set, we performed a current state analysis of detection and management practices in a convenience sample that consisted of Ontario residents and physicians. We used the Electronic Medical Record Administrative data Linked Database (EMRALD), which captures clinically relevant data contained in nearly 400 family physician EMRs distributed across Ontario. The representativeness of EMRALD patients and physicians, in addition to the quality and comprehensiveness of EMRALD data, has been previously found to be generally reflective of the Ontario population.28,29 Young adults and people with lower socioeconomic status are slightly under represented in EMRALD compared with the general Ontario population; however, this is likely characteristic of the types of people that see a physician and not anything specific to EMRALD patients. Relative to all Ontario physicians, EMRALD physicians practise more in rural locations and fewer are foreign-trained physicians.28

To operationalize our measures using EMR data, we introduced a number of additional specifications. We included patients greater than 18 years of age at index date with an EMR record that began at least 1 year before the extraction of the EMR data in the summer/fall of 2014. For indicators for patients with chronic kidney disease, we identified patients with stage 3 or greater disease as having a most recent estimated glomerular filtration rate (eGFR) of less than 60 mL/min per 1.73 m2 and a second abnormal reading at least 3 months earlier. We excluded patients with chronic kidney disease who were receiving dialysis as documented in the cumulative patient profile, because patients on dialysis were considered to be beyond the domain of primary care for management of their condition. For the first indicator, "The primary care provider can identify patients in their practice aged 18 or over with chronic kidney disease," the proxy EMR measure was a recording of chronic kidney disease or it's synonyms in the cumulative patient profile, a discrete EMR field, typical of family physician patient records, which contains a "history of past health" and active "problem list." For the 2 indicators that were written as "… initial eGFR less than 60 mL/min per 1.73 m2 …," we defined "initial" as the first eGFR at least 6 months before the date the data was extracted to allow for at least 6 months to look for a repeat test or albumin-creatinine ratio (ACR) test. For the "percentage of patients with chronic kidney disease that had a serum potassium test 7-30 days after the initial angiotensin-converting enzyme (ACE) inhibitor-angiotensin II receptor blocker (ARB) prescription," we only included patients with ACE inhibitor-ARB prescriptions after a full year of no ACE inhibitor-ARB prescription, thus ensuring new-user status among patients included in the denominator. For the indicator "percentage of patients with chronic kidney disease simultaneously receiving both an ACE inhibitor and an ARB," the numerator only included patients that received a prescription for both types of medications on the same day.

We used validated methods for identifying patients with diabetes30,31 and included patients with hypertension who had hypertension recorded in their cumulative patient profile, who had an elevated blood pressure and a prescription for an antihypertensive on the same day and a prescription for an antihypertensive in the past 18 months, or who met Canadian Hypertension Education Program Criteria for hypertension at any time in their EMR record and an elevated blood pressure or antihypertensive prescription in the past 18 months. This algorithm had a sensitivity of 81.1%, specificity of 97.7%, positive predictive value of 93.2% and negative predictive value of 93.1% in a validation study involving 969 randomly selected adults that compared EMRALD with chart-abstracted data.32-34

All analyses were done in SQL Server Management Studio 2012. 

### Ethics approval

The Sunnybrook Health Sciences Research Ethics Board approved both phases of the project.

## Results

### Modified-Delphi panel results

We identified 174 measures published by 26 sources (Figure 2). After review by clinical leads, 89 measures (Appendix 2, available at [www.cmajopen.ca/content/5/1/E74/suppl/DC1](http://www.cmajopen.ca/content/5/1/E74/suppl/DC1) were submitted to the panel. The response rate for all 3 of the rating rounds of the panel process was 100%. Seventeen indicators made up the final list, with 1 under the category Prevalence, Incidence and Mortality; 4 under Screening, Diagnosis and Risk Factors; 11 under Management; and 1 under Referral to a Specialist (Box 2). There were 2 categories, System Level and Lifestyle, for which no indicators met the inclusion criteria. The panel acknowledged through discussion that, though important, System Level indicators are likely outside of the family physicians' control. Lifestyle-related indicators (e.g., smoking cessation, dietary, exercise counselling) did not get included because the panel rated them as low in their feasibility to measure in the EMR.

![Figure 2](http://www.cmajopen.ca/https://www.cmajopen.ca/content/cmajo/5/1/E74/F2.medium.gif)

[Figure 2](http://www.cmajopen.ca/content/5/1/E74/F2)

Figure 2 
Modified Delphi panel process.

Box 2: 
### Quality indicators resulting from Delphi panel process

Prevalence, incidence and mortality

1. The primary care providers can identify patients in their practice aged 18 years and older with chronic kidney disease.

Screening, diagnosis and risk factors

2. Percentage of patients with an initial estimated glomerular filtration rate (eGFR) < 60 mL/min per 1.73 m2 that is followed by a repeat test within 6 months.

3. Percentage of patients with an initial eGFR < 60 mL/min/1.73 m2 with an albumin-creatinine ratio (ACR) test conducted within 6 months.

4. Percentage of patients with risk factors for chronic kidney disease (diabetes, hypertension) with an eGFR in the past 18 months.

5. Percentage of patients with risk factors for chronic kidney disease (diabetes, hypertension) with an ACR in the past 18 months.

Management

6. Percentage of patients with chronic kidney disease with an eGFR in the past 18 months.

7. Percentage of patients with chronic kidney disease with an ACR in the past 18 months.

8. Percentage of patients with chronic kidney disease with a blood pressure recorded in the past 18 months.

9. Percentage of patients with diabetes and albuminuria (moderately or severely increased ACR ≥ 3 mg/mmol) with a blood pressure recorded in the past 9 months.

10. Percentage of patients with chronic kidney disease with a most recent blood pressure < 140/90 mm Hg, or with chronic kidney disease and diabetes with a most recent blood pressure < 130/80 mm Hg.

11. Percentage of patients with diabetes and albuminuria (moderately or severely increased ACR ≥ 3 mg/mmol) who were prescribed an angiotensin-converting enzyme (ACE) inhibitor or angiotensin II receptor blocker (ARB) unless a contraindication or adverse effects are recorded.

12. Percentage of patients with chronic kidney disease who had a serum potassium test 7-30 days after initial ACE inhibitor-ARB prescription.

13. Percentage of patients with chronic kidney disease simultaneously receiving both an ACE inhibitor and an ARB.

14. Percentage of patients with stage 3-5 chronic kidney disease and a prescription for a nonsteroidal anti-inflammatory drug for more than 2 weeks.

15. Percentage of patients ≥ 50 years and ≤ 80 years of age with stage 3-5 chronic kidney disease and taking a statin unless contraindicated.

16. Percentage of patients with chronic kidney disease who received an influenza vaccine in the past year unless contraindicated.

Referral to a specialist

17. Percentage of patients age < 80 years with a referral to a nephrologist for eGFR < 30 mL/min per 1.73 m2.

### Measurement of quality indicators

Overall, of the 140 147 eligible adult patients in EMRALD, 101 561 (72.5%) had at least 1 eGFR in their chart and although only 76 935 (54.9%) patients had at least 2 eGFRs recorded in their chart, 16 585 of 17 299 (95.9%) of the patients with an eGFR less than 60 mL/min per 1.73 m2 had at least 1 additional eGFR test. There were 154 dialysis recipients who were removed from analysis; of the remaining patients, 6848 of 139 993 (4.9%) had stage 3 or greater chronic kidney disease. The mean age of the EMRALD cohort was 50.0 (standard deviation [SD] 18.3) years, with 57.2% of the cohort being female. The mean age of patients in our cohort with stage 3 or greater chronic kidney disease was 76.1(SD 11.0) years, with 62.9% being female. The mean duration of the EMR record was 5.8 (SD 2.9) years. Among our patients with stage 3 or greater disease, 32.9% had diabetes and 70.3% had hypertension, compared with 10.6% with diabetes and 23.1% with hypertension in the general EMRALD population.

Family physician performance was highest in avoiding nonsteroidal anti-inflammatory drugs (NSAIDs), avoiding simultaneous prescription of ACE inhibitors and ARBs, and measuring eGFR and blood pressure for patients with stage 3 or greater chronic kidney disease, with over 80% adherence in these measures (Table 1). In addition, more than 70% of patients with a clinical indication were prescribed an ACE inhibitor or ARB and had an eGFR measured if at high risk for chronic kidney disease. Less than 70% of applicable patients had a referral to a nephrologist, had received influenza vaccine, met blood pressure targets, were prescribed a statin, or had a potassium test 7-30 days after starting an ACE inhibitor or ARB. It is notable that ACR testing was part of 3 of the 5 most poorly performed indicators. Less than 50% of patients at risk for chronic kidney disease had an ACR done, had their chronic kidney disease documented in their cumulative patient profile, had an ACR done or a repeat eGFR if their initial eGFR was less than 60 mL/min per 1.73 m2 within 6 months, or had an ACR in the past 18 months if they had chronic kidney disease.

View this table:
[Table 1:](http://www.cmajopen.ca/content/5/1/E74/T1)

Table 1:   Results as applied in Electronic Medical Record Administrative data Linked Database (EMRALD)

## Interpretation

Through a modified Delphi process, we established a set of primary care indicators for detecting and managing chronic kidney disease in a Canadian context. In addition, we were able to show the feasibility of measuring these indicators to gain an understanding of the current state of the detection and management of this disease in the primary care setting in Ontario, which could allow us to identify gaps in care and determine areas that should be targeted for improvement.

Our measures span a broad range of identified measurement domains and concepts. A US-based panel identified 12 measures for the primary care management of chronic kidney disease.35 Most were conceptually similar to ours in terms of identifying important actions in the management of chronic kidney disease, although they differed slightly in their definitions of time frames for actions. An annual complete blood count for patients with stage 3b-5 disease and avoidance of bisphosphonates in patients with an eGFR of less than 30 mL/min per 1.73 m2 were included in the US indicators, but not in ours. Recently, a Japanese team followed a modified Delphi method to identify a set of quality indicators for the care of chronic kidney disease in the primary care setting.36 They selected 11 indicators, of which 7 were conceptually similar to ours, with 4 measures not included in our set: prevention of contrast-induced nephropathy, glycemic control of diabetes in chronic kidney disease, avoidance of biguanides in diabetes and quarterly urine testing. With respect to lipid management in chronic kidney disease, our indicator measured statin prescribing, which is within the provider's control. The US-based panels' indicator required cholesterol testing, and the Japanese indicator was based on achieving a cholesterol target, which is not necessarily within the provider's control.

We found the lack of documentation in problem lists or past medical history of chronic kidney disease by family physicians consistent with other studies that identified the lack of recognition of chronic kidney disease by primary care physicians.13,37 Although our indicator methods differed slightly from previous measures in the primary care setting in the US, we found similar rates of lipid-lowering medication use (~60%) and avoidance of NSAID prescribing.38 Our prevalence rates for stage 3 or greater disease (4.9%) were lower than identified elsewhere, but other studies based their diagnosis on a single eGFR measure and did not exclude patients who were undergoing dialysis.13,38 The higher prevalence of chronic kidney disease that we found among women was similar to those found in the UK and in the US, and the rates of diabetes and hypertension were similar to the American rates, but higher than those found in the UK.13,38

### Limitations

It was necessary to modify some of the indicators for measurement because we only had laboratory data as far back as the date of creation of each EMR record. Therefore, we could not confirm disease onset and duration because we could not be certain that the first occurrence of an eGFR of less than 60 mL/min per 1.73 m2 in the EMR was the first ever for a given patient. This limitation required us to redefine "initial" in indicators that measured repeat testing after the initial eGFR of less than 60 mL/min per 1.73 m2. It is possible that if we had the initial eGFR, our measured performance rate for repeat or ACR testing may have been higher. However, it is unlikely that it would have been significantly higher given the low rate of ACR testing in general. 

In Ontario, the eGFR is typically provided when serum creatinine testing is ordered and is calculated at the laboratory using the Modification of Diet in Renal Disease (MDRD) equation.39 However, the MDRD equation only takes into account white and black ethnicity, and does not consider the ethnic diversity of the Ontario population. Ethnicity is not typically provided when the laboratory test is ordered, thus it is likely that even the correction factor for black patients is not applied. The MDRD equation may underestimate eGFR and therefore may have led to overdiagnosis. More recently, in 2015, Ontario laboratories have switched to using the Chronic Kidney Disease Epidemiology Collaboration equation,40 which is considered to be more accurate, particularly for women and the younger population. However, this change occurred after the conduct of our analysis.

It is possible that influenza vaccines were given to patients but not recorded in the EMR because patients in Ontario may receive influenza vaccines outside of the family physician's office (e.g., at shopping malls, pharmacies or public health units), and the completeness of this recording in EMRs is unknown. 

We did not have access to medication duration and precise medication discontinuation dates; therefore, we were required to make estimates on timing and duration for medication indicators. 

Finally, we are limited by identifying patients with chronic kidney disease through the laboratory tests in the EMR. It is likely that laboratory tests ordered by specialists or in hospital are not accounted for in our analysis.

### Conclusion

We have developed a set of quality indicators for the detection and management of chronic kidney disease that are feasible to measure. Through our application of these indicators to real-world primary care EMR data, we have identified areas that need improvement. Future studies that focus on understanding why physicians are not ordering ACR tests and appear not to be recognizing chronic kidney disease are needed. We have been unable to identify similar quality indicator measurement in other provinces in Canada; however, it is hopeful with the release of these indicators that other provinces may be able to do comparable analyses in the future. 

### Supplemental information

For reviewer comments and the original submission of this manuscript, please see [www.cmajopen.ca/content/5/1/E74/suppl/DC1](http://www.cmajopen.ca/content/5/1/E74/suppl/DC1)

## Acknowledgements

This project was made possible by a grant from the Ontario Renal Network. The authors would like to thank all the members of the panel for their participation and expertise (Dr. Adeera Levin, Dr. Allan Grill, Dr. Amit Garg, Betty Hogeterp, Dr. Blaise Clarkson, Dr. Brenda Hemmelgarn, Dr. Brian Hutchison, Dr. Elizabeth Muggah, Dr. Francois Madore, Dr. Kevin Samson, Dr. Liisa Jaakkimainen, Michael McCormick, Dr. Noah Ivers, Dr. Patricia Marr, Dr. Roland Grad, Dr. Scott Brimble, Dr. Sharon Johnston and Dr. Sheldon Tobe).

## Footnotes

*   **Competing interests:** Karen Tu, Jess Rogers, Lindsay Bevan and Katie Hunter report grants or support from the Ontario Renal Network. No other competing interests were declared.

*   **Contributors:** Karen Tu conceived, participated in the design, acquisition and interpreted the data, drafted and revised the manuscript and agrees to be accountable for all aspects of the work. Lindsay Bevan, Katie Hunter, Jess Rogers and Gihad Nesrallah designed, acquired, analysed and interpreted the data for the modified Delphi panel process, provided critically important revision of the manuscript for intellectual content, approved the final version to be published and agree to be accountable for all of the indicator development components of this work. Jacqueline Young assisted in the design, analysis and interpretation of the measurement application portion of this work, revised it critically for important intellectual content, approve the final version to be published and agrees to be accountable for all aspects indicator measurement portion of this work. All of the authors approved the final version to be published and agreed to act as guarantors of the work.

*   **Disclaimer:** This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this article are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intented or should be inferred. 

## References

1.  Zhang QL, Rothenbacher D (2008) Prevalence of chronic kidney disease in population-based studies: systematic review. BMC Public Health 8:117.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2458-8-117&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18405348&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

2.  Lipscombe LL, Hux JE (2007) Trends in diabetes prevalence, incidence, and mortality in Ontario, Canada 1995-2005: a population-based study. Lancet 369:750–6.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0140-6736(07)60361-4&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17336651&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000244726700031&link_type=ISI) 

3.  Go AS, Chertow GM, Fan D, et al. (2004) Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med 351:1296–305.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1056/NEJMoa041031&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15385656&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000223997700007&link_type=ISI) 

4.  Keith DS, Nichols GA, Gullion CM, et al. (2004) Longitudinal follow-up and outcomes among a population with chronic kidney disease in a large managed care organization. Arch Intern Med 164:659–63.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1001/archinte.164.6.659&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=15037495&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000220313600011&link_type=ISI) 

5.  Johnson DW (2004) Evidence-based guide to slowing the progression of early renal insufficiency. Intern Med J 34:50–7.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1111/j.1444-0903.2004.t01-6-.x&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=14748914&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000188541200010&link_type=ISI) 

6.  Trivedi HS, Pang MM, Campbell A, et al. (2002) Slowing the progression of chronic renal failure: economic benefits and patients' perspectives. Am J Kidney Dis 39:721–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1053/ajkd.2002.35679&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11920337&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000174835000007&link_type=ISI) 

7.  Tonelli M, Bohm C, Pandeya S, et al. (2001) Cardiac risk factors and the use of cardioprotective medications in patients with chronic renal insufficiency. Am J Kidney Dis 37:484–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1053/ajkd.2001.22070&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=11228171&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000167495600004&link_type=ISI) 

8.  Stevens L, Cooper S, Singh R, et al. (2005) Detection of chronic kidney disease in non-nephrology practices An important focus for intervention. BCMJ  47:305–311.
    
    

9.  Coresh J, Byrd-Holt D, Astor BC, et al. (2005) Chronic kidney disease awareness, prevalence, and trends among US adults, 1999 to 2000. J Am Soc Nephrol 16:180–8.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiam5lcGhyb2wiO3M6NToicmVzaWQiO3M6ODoiMTYvMS8xODAiO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzUvMS9FNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. Drawz PE, Miller RT, Singh S, et al. (2012) Impact of a chronic kidney disease registry and provider education on guideline adherence - a cluster randomized controlled trial. BMC Med Inform Decis Mak 12:62.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1472-6947-12-62&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22765882&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

11. Lenz O, Mekala DP, Patel DV, et al. (2005) Barriers to successful care for chronic kidney disease. BMC Nephrol 6:11.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1471-2369-6-11&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=16250919&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

12. Stevens PE, O'Donoghue DJ, de Lusignan S, et al. (2007) Chronic kidney disease management in the United Kingdom: NEOERICA project results. Kidney Int 72:92–9.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1038/sj.ki.5002273&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=17440495&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000247909800013&link_type=ISI) 

13. de Lusignan S, Chan T, Stevens P, et al. (2005) Identifying patients with chronic kidney disease from general practice computer records. Fam Pract 22:234–41.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoiZmFtcHJhY3QiO3M6NToicmVzaWQiO3M6ODoiMjIvMy8yMzQiO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzUvMS9FNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

14. White SL, Polkinghorne KR, Cass A, et al. (2008) Limited knowledge of kidney disease in a survey of AusDiab study participants. Med J Aust 188:204–8.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=18279125&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000255914800004&link_type=ISI) 

15. Razavian M, Heeley EL, Perkovic V, et al. (2012) Cardiovascular risk management in chronic kidney disease in general practice (the AusHEART study). Nephrol Dial Transplant 27:1396–402.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoibmR0IjtzOjU6InJlc2lkIjtzOjk6IjI3LzQvMTM5NiI7czo0OiJhdG9tIjtzOjE5OiIvY21ham8vNS8xL0U3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

16. Levey AS, Coresh J, Balk E, et al. (2003) National Kidney Foundation practice guidelines for chronic kidney disease: evaluation, classification, and stratification. Ann Intern Med 139:137–47.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-139-2-200307150-00013&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=12859163&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000184163100007&link_type=ISI) 

17. Chronic kidney disease in adults: assessment and management. London (UK): 2014. Available:[www.nice.org.uk/guidance/cg182](http://www.nice.org.uk/guidance/cg182). accessed 2016 July 15.
    
    

18. Chronic kidney disease (CKD) management in general practice (2nd ed.). Melbourne (AU): Kidney Health Australia; 2012
    
    

19. Levin A, Hemmelgarn B, Culleton B, et al. (2008) Guidelines for the management of chronic kidney disease. CMAJ 179:1154–62.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTc5LzExLzExNTQiO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzUvMS9FNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

20. Kidney Disease Improving Global Outcomes (KDIGO) (2013) KDIGO 2012 Clinical Practice Guideline for the Evaluation and Management of Chronic Kidney Disease. Kidney Int Suppl 3.
    
    

21. Kötter T, Blozik E, Scherer M (2012) Methods for the guideline-based development of quality indicators - a systematic review. Implement Sci 7:21.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1186/1748-5908-7-21&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=22436067&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

22. Ganz GMF (2005) Guidelines for the management of chronic kidney disease: Rationale, development, and implementation. BCMJ  47:292–5.
    
    

23. 2016/17 General Medical Services (GMS) contract Quality and Outcomes Framework (QOF). United Kingdom: BMA/NHS Employers/NHS England; 2016. Available:[www.nhsemployers.org/~/media/Employers/Documents/Primary%20care%20contracts/QOF/2016-17/2016-17%20QOF%20guidance%20documents.pdf](http://www.nhsemployers.org/~/media/Employers/Documents/Primary%20care%20contracts/QOF/2016-17/2016-17%20QOF%20guidance%20documents.pdf). accessed 2016 July 15.
    
    

24. Pan-Canadian primary health care indicator update. Toronto: Canadian Institute for Health Information (CIHI); 2012.
    
    

25. Brouwers MC, Kho ME, Browman GP, et al. (2010) AGREE II: advancing guideline development, reporting and evaluation in health care. CMAJ 182:E839–42.
    
    [FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMToiMTgyLzE4L0U4MzkiO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzUvMS9FNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

26. Spertus JA, Eagle KA, Krumholz HM, et al. (2005) American College of Cardiology and American Heart Association methodology for the selection and creation of performance measures for quantifying the quality of cardiovascular care. Circulation 111:1703–12.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTQ6ImNpcmN1bGF0aW9uYWhhIjtzOjU6InJlc2lkIjtzOjExOiIxMTEvMTMvMTcwMyI7czo0OiJhdG9tIjtzOjE5OiIvY21ham8vNS8xL0U3NC5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

27. To T, Guttmann A, Lougheed MD, et al. (2010) Evidence-based performance indicators of primary care for asthma: a modified RAND Appropriateness Method. Int J Qual Health Care 22:476–85.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NjoiaW50cWhjIjtzOjU6InJlc2lkIjtzOjg6IjIyLzYvNDc2IjtzOjQ6ImF0b20iO3M6MTk6Ii9jbWFqby81LzEvRTc0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

28. Tu K, Mitiku TF, Ivers NM, et al. (2014) Evaluation of Electronic Medical Record Administrative data Linked Database (EMRALD). Am J Manag Care 20:e15–21.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24669409&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000330599000010&link_type=ISI) 

29. Tu K, Widdifield J, Young J, et al. (2015) Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective. BMC Med Inform Decis Mak 15:67.
    
    

30. Tu K, Manuel D, Lam K, et al. (2011) Diabetics can be identified in an electronic medical record using laboratory tests and prescriptions. J Clin Epidemiol 64:431–5.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2010.04.007&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20638237&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

31. Ivers NM, Tu K, Francis J, et al. (2010) Feedback GAP: study protocol for a cluster-randomized trial of goal setting and action plans to increase the effectiveness of audit and feedback interventions in primary care. Implement Sci 5:98.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=21167034&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

32. Tu K, Mitiku T, Guo H, et al. (2010) Myocardial infarction and the validation of physician billing and hospitalization data using electronic medical records. Chronic Dis Can 30:141–6.
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20946715&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

33. Tu K, Mitiku T, Lee DS, et al. (2010) Validation of physician billing and hospitalization data to identify patients with ischemic heart disease using data from the Electronic Medical Record Administrative data Linked Database (EMRALD). Can J Cardiol 26:e225–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1016/S0828-282X(10)70412-8&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=20847968&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

34. Ivers N, Pylypenko B, Tu K (2011) Identifying patients with ischemic heart disease in an electronic medical record. J Prim Care Community Health 2:49–53.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToic3BqcGMiO3M6NToicmVzaWQiO3M6NjoiMi8xLzQ5IjtzOjQ6ImF0b20iO3M6MTk6Ii9jbWFqby81LzEvRTc0LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

35. Litvin CB, Ornstein SM (2014) Quality indicators for primary care: an example for chronic kidney disease. J Ambul Care Manage 37:171–8.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.1097/JAC.0000000000000015&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=24594565&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 

36. Fukuma S, Shimizu S, Niihata K, et al. (2016) Development of quality indicators for care of chronic kidney disease in the primary care setting using electronic health data: a RAND-modified Delphi method. Clin Exp Nephrol.
    
    

37. Vest BM, York TRM, Sand J, et al. (2015) Chronic kidney disease guideline implementation in primary care: a qualitative report from the TRANSLATE CKD Study. J Am Board Fam Med 28:624–31.
    
    [Abstract/FREE Full Text](http://www.cmajopen.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NToiamFiZnAiO3M6NToicmVzaWQiO3M6ODoiMjgvNS82MjQiO3M6NDoiYXRvbSI7czoxOToiL2NtYWpvLzUvMS9FNzQuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

38. Litvin CB, Nietert PJ, Wessell AM, et al. (2011) Recognition and management of CKD in primary care. Am J Kidney Dis 37:646–7.
    
    

39. Levey AS, Bosch JP, Lewis JB, et al. (1999) A more accurate method to estimate glomerular filtration rate from serum creatinine: a new prediction equation. Modification of Diet in Renal Disease Study Group. Ann Intern Med 130:461–70.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-130-6-199903160-00002&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=10075613&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000079165500001&link_type=ISI) 

40. Levey AS, Stevens LA, Schmid CH, et al. (2009) A new equation to estimate glomerular filtration rate. Ann Intern Med 150:604–12.
    
    [CrossRef](http://www.cmajopen.ca/lookup/external-ref?access_num=10.7326/0003-4819-150-9-200905050-00006&link_type=DOI) 
    
    [PubMed](http://www.cmajopen.ca/lookup/external-ref?access_num=19414839&link_type=MED&atom=%2Fcmajo%2F5%2F1%2FE74.atom) 
    
    [Web of Science](http://www.cmajopen.ca/lookup/external-ref?access_num=000265903800004&link_type=ISI) 

*   Copyright 2017, Joule Inc. or its licensors