Abstract
The validity of estimates of resource utilisation based on patient recall has not been firmly established. A comprehensive literature search was conducted to identify studies that used patient self-reports to derive estimates of healthcare resource utilisation, direct nonmedical costs and indirect costs. Previous work in this area was examined to determine the issues that were most important in determining whether patient self-reports lead to valid and unbiased estimates in cost-effectiveness studies.
This study reviews and highlights areas where patient self-reports lead to reliable estimates and where their use may lead to erroneous conclusions. In particular, it is noted that patient recall of resource utilisation declines over time, that the salience of a treatment episode affects recall and that the perceived social acceptability of a condition and other confounding factors may influence patient reporting. Moreover, it appears that not all elements of healthcare consumption are recalled to the same degree: medication use tends to be recalled with less accuracy than hospitalisation. In terms of the potential impact on cost-effectiveness results, the main concern is with problems of validity rather than bias, although bias may occur when the results are applied to league tables.
Similar content being viewed by others
References
Nevitt M, Cummings S, Browner W, et al. The accuracy of self report of fractures in elderly women: evidence from a prospective study. Am J Epidemiol 1992; 135 (5): 490–9
Gold D, Weiss S, Tager I, et al. Comparison of questionnaire and diary methods in acute childhood respiratory illness surveillance. Am Rev Respir Dis 1989; 139: 847–9
Werler M, Pober B, Nelson K, et al. Reporting accuracy among mothers of malformed and nonmalformed infants. Am J Epidemiol 1989; 129 (2): 415–21
Boyer G, Templin D, Wedell G, et al. Discrepancy between patient recall and the medical record: potential impact on diagnosis and clinical assessment of chronic disease. Arch Intern Med 1995: 155: 1868–72
Paganinini-Hill A, Chao A. Accuracy of recall of hip fracture, heart attack, and cancer: a comparison of postal survey data and medical records. Am J Epidemiol 1993; 138 (2): 101–6
Rittenhouse B, O’Brien B. Threats to the validity of pharmacoeconomics analyses based on clinical trial data. In: Spilker B, editor. Quality of life in clinical trials. Philadelphia (PA): Lippincott-Raven Publishers, 1996
Ellewein LB, Drummond M. Economic analysis alongside clinical trials: bias in the assessment of economic outcomes. Int J Technol Assess Heath Care 1996; 12 (4): 691–7
Stewart LA, Parmar MKB. Bias in the analysis and reporting of randomized controlled trials. Int J Technol Assess Health Care 1996; 12 (2): 264–75
Jacobs P, Bachynsky J, Baladi J-F. A comparative review of pharmacoeconomic guidelines. Pharmacoeconomics 1995; 8 (3): 182–9
Torrance G, Blaker D, Detsky A, et al. Canadian guidelines for economic evaluation of pharmaceuticals. Pharmacoeconomics 1996; 9 (6): 535–59
Garattini L, Grilli R, Scopelliti D, et al. A proposal for Italian guidelines in pharmacoeconomics. Pharmacoeconomics 1995; 7 (1): 1–6
Commonwealth Department of Health, Housing and Community Services. Guidelines for the pharmaceutical industry on the preparation of submissions to the pharmaceutical benefits advisory committee. Canberra: Australian Government Publishing Services, 1995
Gold M, Siegel J, Russell L, et al., editors. Cost-effectiveness in health and medicine. New York (NY): Oxford University Press, 1996
Evans C. The use of consensus methods and expert panels in pharmacoeconomic studies: practical applications and methodological shortcomings. Pharmacoeconomics 1997; 12 (2 Pt 1): 121–9
US Department of Health, Education and Welfare. Health interview responses compared with medical records: vital and health statistics series 2 no. 7. Washington, DC: National Center for Health Statistics, 1965
US Department of Health, Education and Welfare. Comparison of hospitalization reporting in three survey procedures: vital and health statistics series 2 no. 8. Washington, DC: National Center for Health Statistics, 1965
Norrish A, North D, Kirkman P, et al. Validity of self-reported hospital admission in a prospective study. Am J Epidemiol 1994; 140: 938–42
Licciardone J, Kotsanos J, Brinkman-Kaplan V, et al. Resource utilization and work or school loss reported by patients with diabetes: experience in diabetes training programs. Am J Manage Care 1997; 3: 777–82
Roberts R, Bergstralh E, Schmidt, et al. Comparison of selfreported and medical record health care utilization measures. J Clin Epidemiol 1996; 49 (9): 989–95
Law M, Hurley S, Carlin J, et al. A comparison of patient interview data with pharmacy and medical records for patients with acquired immunodeficiency syndrome or human immunodeficiency virus infection. J Clin Epidemiol 1996; 49 (9): 997–1002
Clark R, Ricketts S, McHugo G. Measuring hospital use without claims: a comparison of patient and provider reports. Health Serv Res 1996: 31 (2): 153–69
Hennesy K, Reed S. Validating self-reports of mental health service use in a chronic population. J Nerv Ment Dis 1992; 180 (6): 399–400
Dansky K, Dellasega C, Shellenbarger T, et al. After hospitalization: home health care for elderly persons. Clin Nurs Res 1996; 5 (2): 185–98
Coutler A, McPherson K, Elliott S, et al. Accuracy of recall of surgical histories: a comparison of postal survey data and general practice records. Community Med 1985; 7: 186–9
Cannell C, Miller P, Oksenberg L. Sociological methodology. Washington (DC): Josey-Bass Publishers, 1981
Yelin E, Criswell L, Feigenbaum P. Health care utilization and outcomes among persons with rheumatoid arthritis in fee-forservice and prepaid group practice settings. JAMA 1996; 276 (13): 1048–53
Remafedi G, Lauer T. The health and psychosocial status of HIV-seropositive youth in Minnesota. J Adolesc Health 1996; 18: 263–9
Weissman J, Levin K, Chasan-Taber S, et al. The validity of self-reported health care utilization by AIDS patients. AIDS 1996; 10: 775–83
Salmon P, Calderbank S. The relationship of childhood physical and sexual abuse to adult illness behavior. J Psychosom Res 1996; 40 (3): 329–36
Dahlquist G, Wall S, Ivarsson JI, et al. Health problems and care in young families: an evaluation of survey procedures. Int J Epidemiol 1984; 13: 221–8
Reifler D, Kessler H, Bernhard E, et al. Impact of screening for mental heath concerns on health service utilization and functional status in primary care patients. Arch Intern Med 1996; 156: 2593–9
McCormick M, Bernbaum J, Eisenberg J, et al. Costs incurred by patients with very low birth weight infants after the initial neonatal hospitalization. Pediatrics 1991; 88 (3): 533–41
Greiner D, Addy S. Sumatriptan use in a large group model health maintenance organization. Am J Health Syst Pharm 1996; 53: 633–8
Means B, Nigam A, Zarrow M, et al. Autobiographical memory for health related events: vital and health statistics series 6 no. 2. Hyattsville (MD): National Center for Health Statistics, 1989
Golding J, Gongla P, Brownwell A. Feasibility of validating survey self reports of mental health service use. Am J Community Psychol 1988; 16 (1): 39–51
Spector P, Bedell J. Measuring program effectiveness: self-report versus objective indicators of recidivism. Am J Community Psychol 1982; 10 (5): 613–6
Petrou S, Dooley M, Whitaker L, et al. The economic costs of caring for people with HIV infection and AIDS in England and Wales. Pharmacoeconomics 1996; 9 (4): 332–40
Reuben D, Wong R, Walsh K, et al. Feasibility and accuracy of a postcard diary system for tracking healthcare utilization of community dwelling older persons. J Am Geriatr Soc 1995; 43: 550–2
Green S, Kaufert J, Corkhill R, et al. The collection of service utilization data: a research note on validity. Soc Sci Med 1979; 13A: 231–4
Yaffe R, Shapiro S, Fuchsberg R, et al. Medical economics survey methods study: cost effectiveness of alternative survey strategies. Med Care 1978; 16 (8): 641–59
Jobe J, White A, Kelley C, et al. Recall strategies and memory for health-care visits. Millbank Q 1990; 68 (2): 171–89
Perloff J, Morris N. Asking about the usual source of care: an appraisal of health care survey alternatives. Med Care 1992; 30 (10): 950–7
Laupacis A, Bournbe R, Rorabeck C, et al. Costs of elective total hip arthroplasty during the first year: cemented versus noncemented. J Arthroplasty 1994; 9 (5): 481–7
Glandon G, Counte M, Tancredi D. An analysis of physician utilization by elderly persons: systematic differences between self-report and archival information. J Gerontol 1992; 47 (5): S245–52
Coulton C, Frost A. Use of social and health services by the elderly. J Health Soc Behav 1982; 23: 330–9
Cleary P, Jette A. The validity of self-reported physician utilization measures. Med Care 1984; 22 (9): 796–803
Siemiatycki J. A comparison of mail, telephone and home interview strategies for household health surveys. Am J Public Health 1979; 69 (3): 238–45
Marquis K, Marquis MS, Newhouse J. The measurement of expenditures for outpatient physician and dental services: methodological findings from the health insurance study. Med Care 1976; 14 (11): 913–31
Coutler A, Vessey M, McPherson K, et al. The ability of women to recall their oral contraceptive histories. Contraception 1986; 33 (2): 127–37
Paganini-Hill A, Ross R. Reliability of recall of drug usage and other health related information. Am J Epidemiol 1982; 116 (1): 114–22
Al Mahdy H, Seymour D. How much can elderly patients tell us about their medications? Postgrad Med J 1990; 66: 116–21
Landahl S. Drug treatment in 70-82 year old persons: a longitudinal study. Acta Med Scand 1987; 221: 179–84
Harlow S, Linet M. Agreement between questionnaire data and medical records: the evidence for accuracy of recall. Am J Epidemiol 1989; 129: 233–48
Bryant H, Visser N, Love E. Records, recall loss and recall bias in pregnancy: a comparison of interview and medical records data of pregnant and postnatal women. Am J Public Health 1989; 79: 78–80
Hulka B, Kupper L, Cassel J, et al. Medication use and misuse: physician-patient discrepancies. J Chronic Dis 1975; 28: 7–21
Anie K, Jones P, Hilton S, et al. A computer assisted telephone interview technique for assessment of asthma morbidity and drug use in adult asthma. J Clin Epidemiol 1996; 49 (6): 653–6
Gerbert B, Stone G, Stulbarg M, et al. Agreement among physician assessment methods; searching for the truth among fallible methods. Med Care 1988; 26: 519–35
Brown J, Adams M. Patients as reliable reporters of medical care process: recall of ambulatory encounter events. Med Care 1992; 30 (5): 400–11
Revecki D, Irwin D, Reblando J, et al. The accuracy of self reported disability days. Med Care 1994; 32 (4): 401–4
Widlack P, Greenley J, McKee D. Validity of case manager reports of clients’ functioning in the community: independent living, income, employment, family contact, and problem behaviours. Community Ment Health J 1992; 28 (6): 505–17
Broadhead WE, Blazere D, George L, et al. Depression, disability days and days lost from work in a prospective epidemiological survey. JAMA 1990; 264 (19): 2524–8
Mushet G, Miller D, Clements B, et al. Impact of sumatriptan on workplace productivity, nonwork activities, and health related quality of life among hospital employees with migraine. Headache 1996; 36: 137–43
Houts P, Lipton A, Harvey H, et al. Nonmedical costs to patients and their families associated with outpatient chemotherapy. Cancer 1984; 53: 2388–92
Barr R, Furlong W, Horsman J, et al. The monetary costs of childhood cancer to the families of patients. Int J Oncol 1996; 8 (5): 933–40
Bradburn N, Sudman S, Blair E, et al. Question threat and response bias. Public Opin Q 1978; 42: 221–34
Rosner T, Namazi K, Wykle M. Health diaries and interviews: consistency in reporting by older adults. Res Aging 1992; 14 (2): 248–66
Field A, Colditz G, Herzog D, et al. Disordered eating: can women accurately recall their binging and purging behaviors 10 years later? Obes Res 1996; 4 (2): 153–9
Burman M. Health diaries in nursing research and practice. J Nurs Scholarship 1995; 27 (2): 147–52
Carp F, Carp A. The validity, reliability and generalizability of diary data. Exp Aging Res 1981; 7 (3): 281–96
Parkin DM, Henney CR, Quirk J, et al. Deviation from prescribed drug treatment after discharge from hospital. BMJ 1976; 2: 686–8
Mechanic D. Medical sociology: some tensions among theory, method and substance. J Health Soc Behav 1989; 30: 147–60
Verbrugge L. Health diaries. Med Care 1980; 18 (1): 73–95
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Evans, C., Crawford, B. Patient Self-Reports in Pharmacoeconomic Studies. Pharmacoeconomics 15, 241–256 (1999). https://doi.org/10.2165/00019053-199915030-00004
Published:
Issue Date:
DOI: https://doi.org/10.2165/00019053-199915030-00004