Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Identification of adults with symptoms suggestive of obstructive airways disease: Validation of a postal respiratory questionnaire

Identification of adults with symptoms suggestive of obstructive airways disease: Validation of a... Background: Two simples scoring systems for a self-completed postal respiratory questionnaire were developed to identify adults who may have obstructive airways disease. The objective of this study was to validate these scoring systems. Method: A two-stage design was used. All adults in two practice populations were sent the questionnaire and a stratified random sample of respondents was selected to undergo full clinical evaluation. Three respiratory physicians reviewed the results of each evaluation. A majority decision was reached as to whether the subject merited a trial of obstructive airways disease medication. This clinical decision was compared with two scoring systems based on the questionnaire in order to determine their positive predictive value, sensitivity and specificity. Results: The PPV (positive predictive value) of the first scoring system was 75.1% (95% CI 68.6– 82.3), whilst that of the second system was 82.3% (95% CI 75.9–89.2). The more stringent second system had the greater specificity, 97.1% (95% CI 96.0–98.2) versus 95.3% (95% CI 94.0–96.7), but poorer sensitivity 46.9% (95% CI 33.0–66.8) versus 50.3% (95% CI 35.3–71.6). Conclusion: This scoring system based on the number of symptoms/risk factors reported via a postal questionnaire could be used to identify adults who would benefit from a trial of treatment for obstructive airways disease. signed to identify adult patients likely to have asthma/ Background The aim of the study was to validate two simple scoring COPD (obstructive airways disease). systems for a self-completed postal questionnaire de- Page 1 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 Table 1: Six key questions from the respiratory questionnaire and two markers of disease severity Six key questions: 1) Have you had wheezing or whistling in your chest at any time*? 2) Have you woken up with a feeling of tightness in your chest*? 3) Have you been woken by an attack of shortness of breath at any time*? 4) Have you been woken by an attack of coughing at any time*? 5) Has any person in your family (parents, grandparents, sisters, brothers, or your children) had asthma? 6) Have you ever had hay fever or eczema? *(in the last 12 months) Markers of severity: a) Breathlessness when wheezing was present. b) Wheezing or whistling without a cold. It has been reported that asthma is under-diagnosed and formed consent was obtained from all subjects attending under-treated in adults [1,2]. In the short term this prob- for clinical review. ably leads to increased morbidity for sufferers, and in the long term it may have a detrimental effect on their lung Method function and clinical state [3]. Chronic obstructive pul- In September 1995 postal questionnaires [2] were sent to monary disease is also thought to be under-diagnosed in all adults registered with two practices in South Manches- adults [4]. This may lead to increased morbidity and the ter, with reminders sent after four and eight weeks to those loss of an effective opportunity for giving smoking cessa- who had not yet responded. From the respondents, a strat- tion advice. There may be clinical benefits to the individ- ified random sample of adults was selected to undergo full ual and health economic benefits to society from clinical examination. To determine the sample, adults identifying and treating patients who have asthma or were first stratified according to the number of symptoms/ COPD and who are unknown to the medical services. Be- risk factors reported from six key questions (Table 1). fore this hypothesis can be tested, however, mechanisms for identifying such patients need to be established. Full Random samples were taken from each of five strata clinical review of everyone in a community is neither cost (those with one to three positive responses were aggregat- efficient nor practical. An alternative approach is to use a ed for sampling purposes). The sampling fraction for each screening questionnaire, designed to identify individuals stratum was chosen in advance in order to give approxi- most likely to have the condition, and therefore most like- mately equal numbers of subjects with and without ob- ly to benefit from clinical review. structive airways disease in the overall sample. As this was a new scoring system, estimates of the expected prevalence The Wythenshawe Community Asthma Project (WYCAP) of obstructive airways disease in each stratum had to be is a long term prospective study of the natural history of derived empirically from the researchers' clinical experi- respiratory symptoms in two general practice populations ence. The number from each stratum invited to the clini- in Manchester, UK. One of its aims is to develop a method cal review and the number who attended is shown in for identifying patients in the community with asthma Table 2. Subjects were invited via telephone or home visit and COPD as the first stage in evaluating the benefits and by their general practitioner to attend their local hospital costs of treating such individuals. In 1993 and 1995, post- for clinical review. al respiratory questionnaire surveys were carried out in both practice populations. We did not feel that it would Clinical assessment be possible to differentiate asthma from COPD using a The clinical assessment (carried out by a clinician, TLF) in- simple postal questionnaire and so two simple scoring volved a structured medical history and physical examina- systems were developed, based on the number of symp- tion. A research assistant performed the investigations, toms/risk factors reported from key questions, both hav- supervised and assisted by the clinician. Neither individu- ing the aim of identifying adults with either asthma or al had access to the postal questionnaire results for the COPD. This paper examines the validity of these scoring subject being examined. systems, using data relating to respondents to the 1995 postal survey. A similar exercise has been performed on The investigations performed have been described in a children in the two practices [5]. This study had ethical ap- previous study [5] and included forced expiratory volume proval from the local research ethics committee and in- in one second (FEV ), forced vital capacity (FVC), FEV / 1 1 FVC, a peak expiratory flow diary, skin prick testing using Page 2 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 Table 2: Response categories of respondents, those invited for and those attending clinical review Number of symptoms/risk factors 0 1–3 4 5 6 Total All respondents 2158 3765 521 355 148 6947 Number invited for clinical review 60 60 150 105 45 420 Number who attended review (% of those invited) 26 (43.3) 30 (50.0) 75 (50.0) 53 (50.5) 18 (40.0) 202 (48.1) Table 3: Level of consultant agreement Number of adults categorised according to whether they merit a trial of obstructive airways disease medication by the level of consultant agreement. Meriting a trial of obstructive airways disease medication Consultant agreement Yes No Total All agreed 73 46 119 2 out of 3 agreed 43 39 82 Final categorisation 116 85 201 One subject was not categorised by one of the consultants and the other two disagreed on the decision on whether a trial of obstructive airways disease treatment was indicated so no majority decision could be made. This individual was excluded from analysis. the skin prick/puncture technique [6] and reversibility stage process [9]. The prevalence of subjects who would testing to β agonists [7]. A bronchial challenge test to his- merit a trial of obstructive airways disease medication was tamine was also performed by the YAN technique [8]. A also estimated using the two-stage process. positive histamine challenge was defined as a fall in FEV of 20% from the pre challenge FEV . Results Of 10,429 questionnaires sent out in 1995, 7,580 were re- turned after three mailings. After a 5.5% adjustment for For each subject, three independent consultant chest phy- sicians were sent full details of the clinical assessment but adults no longer at the mailing address [2], this represent- excluding the details of the postal questionnaire. The con- ed a 78% response rate. 6947 responses contained suffi- sultants were asked to decide whether the subject merited cient information for the present analyses. 420 adults a trial of asthma/COPD (obstructive airways disease) were invited for clinical review of whom 202 (48%) medication. Patients warranting a trial of treatment were attended. said to have a diagnosis of possible obstructive airways disease. Agreement between consultants about the decision to treat is detailed in Table 3. There was unanimous agree- The consultants' opinions were compared with two sets of ment concerning the merits of a trial of treatment in 119 questionnaire scoring systems chosen in advance on the (59.2 %) adults. The majority verdict was used to classify basis of clinical experience as likely predictors of obstruc- the remaining subjects. tive airways disease: Table 4 compares the majority expert opinions against the 1) Four or more symptoms/risk factors reported from the two sets of questionnaire scoring systems. Of 146 adults six key questions. with four or more symptoms/risk factors, 109 (75%) mer- ited a trial of medication. 100 (82%) of the 122 adults 2) Four or more symptoms/risk factors reported from the who reported four or more symptoms/risk factors and six key questions plus at least one marker of severity (Ta- who had a marker of severity were thought by the consult- ble 1). ants to merit a trial of asthma medication. Statistical analysis The positive predictive values (PPV), sensitivity and spe- The positive predictive value, sensitivity, specificity and cificity of the two scoring systems are shown in Table 5. their confidence intervals were calculated for each scoring The PPV for the first scoring system when compared system using the majority consultant opinion as the "gold against the consultants' opinion was 75.1% (95% confi- standard". The method used was appropriate for the two- dence interval (CI) 68.6–82.3), whilst that of the second Page 3 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 Table 4: Comparison between majority expert opinions and questionnaire scoring systems Categories of score "Meriting a trial of "Not meriting a trial of Total medication" medication" All responses 116 85 201* System 1 Four or more symptoms/risk factors 109 37 146 Fewer than four or more symptoms/risk factors 7 48 55 System 2 Four or more symptoms/risk factors plus one marker of severity 100 22 122 Other responses- 16 63 79 * One subject was not categorised by one of the consultants and the other two disagreed on the decision on whether a trial of obstructive airways disease treatment was indicated so no majority decision could be made. This individual was excluded from analysis. Table 5: Positive predictive value, sensitivity and specificity of the two scoring systems Discriminative properties of two scoring systems for questionnaire responses compared with the majority expert opinion about whether the patients merit a trial of obstructive airways disease therapy. Scoring system 1. Four or more symptoms/risk factors 2. Four or more symptoms/risk factors plus one marker of severity PPV % [95% CI] 75.1 [68.6–82.3] 82.3 [75.9–89.2] Sensitivity % [95% CI] 50.3 [35.3–71.6] 46.9 [33.0–66.8] Specificity % [95% CI] 95.3 [94.0–96.7] 97.1 [96.0–98.2] PPV = Positive predictive value CI = Confidence interval system was 82.3% (95% CI 75.9–89.2). The more strin- groups were compared with respect to their age, gender gent second scoring system had a greater specificity but and the number of cigarettes smoked as reported on the poorer sensitivity than the first scoring system. questionnaire. No significant differences were found be- tween the two groups. The prevalence of adults "meriting a trial of obstructive airways disease therapy" (a surrogate for obstructive air- The purpose of a screening test is to identify individuals ways disease) in our survey was calculated [5] to be 22% with a good chance of having disease and who require (95% CI 15.5–31.4). further clinical assessment to confirm or refute the diag- nosis. When defining the threshold at which a screening test is deemed to be positive, consideration has to be tak- Discussion Two simple scoring systems, (based on the number of en of the balance between false positive results (which can symptoms/risk factors reported on a respiratory question- lead to extra distress because of unnecessary further inves- naire) produced high positive predictive values when tigations) and false negatives (which result in some cases compared with majority opinion of three respiratory phy- of disease being missed). The positive predictive value of sicians regarding possible obstructive airways disease. a test reflects the frequency of disease in those with a pos- itive screening test. The cost effectiveness of a screening Two important sources of selection bias could have oc- programme will depend on the cut-off values chosen for curred. First, non-respondents to the questionnaire survey the screening procedure as these determine the number of could have been materially different from respondents. new cases detected and requiring treatment, unnecessary This was examined after the first questionnaire survey in investigations undertaken etc. 1993, when a comparison was made between the practice medical records of a random sample of 100 respondents When assessing a screening test it can be difficult to know and 100 non-respondents. No important differences were what diagnostic standard to use, particularly for condi- found with respect to age, gender, and total number of tions such as obstructive airways disease which do not consultations and consultations for respiratory problems have a universally accepted clinical definition. In our in the previous year. There is no reason to suspect that the study, we defined adults as having possible obstructive situation was different in 1995. Second, the adults attend- airways disease if the majority opinion of three consultant ing for clinical review may have been different from those physicians was that a trial of treatment for obstructive air- who were invited but who did not attend. These two ways disease was merited (majority decisions of specialists Page 4 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 has been used in previous studies to diagnose asthma) Neither the experts nor the examining team were aware of [1,5]. Many clinicians would probably agree that it is rea- the results of the postal questionnaire at the time of their sonable to assess an adult who might merit a trial of ob- involvement in the study. Total agreement by the consult- structive airways disease treatment. Comparing the two ants on whether a subject warranted a trial of medication scoring systems against this "standard", we were able to was reached in 59.2% of adults. Difficulties in making a determine the test characteristics (sensitivity, specificity diagnosis without personally seeing the patient may also and positive predictive value) of each system. The more have contributed to disagreement between consultants al- stringent system (four or more symptoms/risk factors plus though it is likely that some difference of opinion would a marker of severity) had a greater positive predictive val- have persisted even with personal examination. Expert ue and greater specificity (so gave fewer false positive re- opinion concerning asthma diagnosis has been used to sults) than the less stringent system (four or more define asthma in previous epidemiological studies symptoms/risk factors alone), but at the cost of reduced [1,11,12]., whilst others have used patient recall of asth- sensitivity (more cases were missed). ma diagnosis or treatment when measuring prevalence of the disease or its underdiagnosis [13]. The majority of di- Any system, which increased sensitivity, would almost agnostic decisions are made in primary care and we ac- certainly have reduced positive predictive value probably knowledge that choosing three consultant respiratory resulting in and unacceptable rate of false positives. In physicians as the diagnostic gold standard therefore has contrast with screening for malignant conditions when limitations. They were however chosen to reflect a spread detection of all cases is of prime importance (i.e. high sen- of secondary and tertiary respiratory opinions. sitivity), it could be argued that in population screening for diseases such as asthma/COPD the PPV carries more An advantage of the scoring systems used was their sim- weight than sensitivity. plicity. Techniques such as discriminant analysis and lo- gistic regression could have been used to assess the A recent article by Grimes and Schultz [10] illustrates this predictive value of each question and a different scoring point: system developed from them. This would have the disad- vantage that validation would then have been based on "Although sensitivity and specificity are of interest to pub- the same data as was used to derive the scoring system and lic-health policymakers, they are of little use to the clini- therefore would tend to be over optimistic. In addition cian. Stated alternatively, sensitivity and specificity the scoring system might be more difficult to implement. (population measures) look backward (at results gathered over time). Clinicians have to interpret test results to those In clinical practice, a simple scoring system to identify pa- tested. Thus what clinicians need to know are the predic- tients requiring further review is attractive. Choices about tive values of the test (individual measures, which look which system to use will depend on a number of factors; forward)". the balance between positive predictive value and sensi- tivity of each system and available health care resources. It In a previous paper [2], 1112 patients (13.8% of respond- is important to remember that if this questionnaire were ers) were identified positively by the screening question- to be used in another setting the positive predictive value naire. The PPV would indicate that approximately 834 of would have to be recalculated as this value is dependent these had obstructive airways disease. Further, 529 of on the prevalence of disease in the population studied. those who screened positively had no recorded diagnosis of obstructive airways disease or received inhaled medica- Conclusions tion in the previous 12 months. Thus despite the disad- When assessed against the majority expert opinion of vantage of a relatively low sensitivity for the scoring "meriting a trial of obstructive airways disease medica- system significant numbers of patients with obstructive tion", two simple scoring systems based on responses to a airways disease would be identified. It should be noted brief postal questionnaire provided a good method for that the prevalence of adults "meriting a trial of obstruc- identifying adults likely to benefit from a trial of treat- tive airways disease therapy" in our survey was calculated ment for obstructive airways disease and so warranting [5] to be 22% (95% CI 15.5–31.4). This may be an over- clinical review. estimate of the true prevalence of obstructive airways dis- ease as it will include patients who would not benefit Authors' contributions from a trial of treatment. TLF and PIF conceived the study and TLF, PIF, PCH, RMM, SH, and CACP participated in its design and coordination. The three specialists reviewing the clinical information re- TLF, JAC and MLH contributed to data analysis and inter- lied on detailed written data supplied to them by the re- pretation. RRM provided statistical advice, data analysis search team; they did not have direct contact with patient. and interpretation. TLF wrote the manuscript and all the Page 5 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 authors critically revised it and approved the final manuscript Acknowledgements The authors would like to thank the three consultants, Dr J Miles, Dr S Burge and Dr J Williams who provided our expert majority diagnosis and Tamsin Wright who helped with the clinical evaluations; the staff of Bow- land Medical Practice and Tregenna Group Practice and the North West Lung Research Centre; and the subjects who answered the postal question- naire and who attended for clinical review. We would like to thank the Roy- al College of General Practitioners (RCGP) Manchester Research Unit (now relocated to Aberdeen). This study was supported by Glaxo-Well- come, Zeneca Pharmaceuticals, Manchester Airport, The Royal College of General Practitioners and The North West Lung Centre. References 1. De Marco R, Cerveri I, Bugiani M, Ferrari M and Verlato G An unde- tected burden of asthma in Italy: the relationship between clinical and epidemiological diagnosis of asthma Eur Respir J 1998, 11:599-605 2. Frank PI, Ferry S, Moorhead T and Hannaford P Use of a postal questionnaire to estimate the likely under-diagnosis of asth- ma-like illness in adults Br J Gen Pract 1996, 46:295-297 3. Agertoft L and Pedersen S Effects of long-term treatment with an inhaled corticosteroid on growth and pulmonary function in asthmatic children Respir Med 1994, 88:373-381 4. van den Boom G, Rutten-van Molken MP, Folgering H, van Weel C and van Schayck CP The economic effects of screening for ob- structive airway disease: an economic analysis of the DIMCA program Preventive Medicine 2000, 30:302-308 5. Frank TL, Frank PI, McNamee R, Wright T, Hannaford P and Morrison J Assessment of a simple scoring system applied to a screen- ing questionnaire for asthma in children aged 5–15 yrs Eur Respir J 1999, 14:1190-1197 6. Basomba A, Sastre A, Peleaz A, Romar A, Campos A and Garcia-Vil- lalmanzo A Standardisation of the prick test Allergy 1985, 40:395- 7. American Thoracic Society. Standards for the diagnosis and care of patients with chronic obstructive pulmonary disease (COPD) and asthma Am Rev Respir Dis 1987, 225-244 8. Yan K, Salome C and Woolcock AJ. Rapid method for measure- ment of bronchial responsiveness Thorax 1983, 38:760-765 9. McNamee R Optimal designs of two-stage studies for estima- tion of sensitivity, specificity and positive predictive value Sta- tistics in Medicine 2002, 21:3609-3625 10. Grimes DA and Schulz KF Uses and abuses of screening tests Lancet 2002, 359:881-884 11. Jenkins MA, Clarke JR, Carlin JB, Robertson CF, Hopper JL and Dalton MF Validation of questionnaire and bronchial hyperrespon- siveness against respiratory physician assessment in the di- agnosis of asthma Int J Epidemiol 1996, 25:609-616 12. Burr ML Editorial: Diagnosing asthma by questionnaire in ep- idemiological surveys Clin Exp Allergy 1992, 22:509-510 13. Kaur B, Anderson HR, Austin J, Burr M, Harkins LS and Strachan DP Prevalence of asthma symptoms, diagnosis, and treatment in 12–14 year old children across Great Britain (international Publish with Bio Med Central and every study of asthma and allergies in childhood, ISAAC UK) BMJ 1998, 316:118-124 scientist can read your work free of charge "BioMed Central will be the most significant development for Pre-publication history disseminating the results of biomedical researc h in our lifetime." The pre-publication history for this paper can be accessed Sir Paul Nurse, Cancer Research UK here: Your research papers will be: available free of charge to the entire biomedical community http://www.biomedcentral.com/1471-2296/4/5/prepub peer reviewed and published immediately upon acceptance cited in PubMed and archived on PubMed Central yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 6 of 6 (page number not for citation purposes) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BMC Family Practice Springer Journals

Identification of adults with symptoms suggestive of obstructive airways disease: Validation of a postal respiratory questionnaire

Loading next page...
 
/lp/springer-journals/identification-of-adults-with-symptoms-suggestive-of-obstructive-0Djyo6mgc0

References (17)

Publisher
Springer Journals
Copyright
Copyright © 2003 by Frank et al; licensee BioMed Central Ltd.
Subject
Medicine & Public Health; General Practice / Family Medicine; Primary Care Medicine
eISSN
1471-2296
DOI
10.1186/1471-2296-4-5
Publisher site
See Article on Publisher Site

Abstract

Background: Two simples scoring systems for a self-completed postal respiratory questionnaire were developed to identify adults who may have obstructive airways disease. The objective of this study was to validate these scoring systems. Method: A two-stage design was used. All adults in two practice populations were sent the questionnaire and a stratified random sample of respondents was selected to undergo full clinical evaluation. Three respiratory physicians reviewed the results of each evaluation. A majority decision was reached as to whether the subject merited a trial of obstructive airways disease medication. This clinical decision was compared with two scoring systems based on the questionnaire in order to determine their positive predictive value, sensitivity and specificity. Results: The PPV (positive predictive value) of the first scoring system was 75.1% (95% CI 68.6– 82.3), whilst that of the second system was 82.3% (95% CI 75.9–89.2). The more stringent second system had the greater specificity, 97.1% (95% CI 96.0–98.2) versus 95.3% (95% CI 94.0–96.7), but poorer sensitivity 46.9% (95% CI 33.0–66.8) versus 50.3% (95% CI 35.3–71.6). Conclusion: This scoring system based on the number of symptoms/risk factors reported via a postal questionnaire could be used to identify adults who would benefit from a trial of treatment for obstructive airways disease. signed to identify adult patients likely to have asthma/ Background The aim of the study was to validate two simple scoring COPD (obstructive airways disease). systems for a self-completed postal questionnaire de- Page 1 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 Table 1: Six key questions from the respiratory questionnaire and two markers of disease severity Six key questions: 1) Have you had wheezing or whistling in your chest at any time*? 2) Have you woken up with a feeling of tightness in your chest*? 3) Have you been woken by an attack of shortness of breath at any time*? 4) Have you been woken by an attack of coughing at any time*? 5) Has any person in your family (parents, grandparents, sisters, brothers, or your children) had asthma? 6) Have you ever had hay fever or eczema? *(in the last 12 months) Markers of severity: a) Breathlessness when wheezing was present. b) Wheezing or whistling without a cold. It has been reported that asthma is under-diagnosed and formed consent was obtained from all subjects attending under-treated in adults [1,2]. In the short term this prob- for clinical review. ably leads to increased morbidity for sufferers, and in the long term it may have a detrimental effect on their lung Method function and clinical state [3]. Chronic obstructive pul- In September 1995 postal questionnaires [2] were sent to monary disease is also thought to be under-diagnosed in all adults registered with two practices in South Manches- adults [4]. This may lead to increased morbidity and the ter, with reminders sent after four and eight weeks to those loss of an effective opportunity for giving smoking cessa- who had not yet responded. From the respondents, a strat- tion advice. There may be clinical benefits to the individ- ified random sample of adults was selected to undergo full ual and health economic benefits to society from clinical examination. To determine the sample, adults identifying and treating patients who have asthma or were first stratified according to the number of symptoms/ COPD and who are unknown to the medical services. Be- risk factors reported from six key questions (Table 1). fore this hypothesis can be tested, however, mechanisms for identifying such patients need to be established. Full Random samples were taken from each of five strata clinical review of everyone in a community is neither cost (those with one to three positive responses were aggregat- efficient nor practical. An alternative approach is to use a ed for sampling purposes). The sampling fraction for each screening questionnaire, designed to identify individuals stratum was chosen in advance in order to give approxi- most likely to have the condition, and therefore most like- mately equal numbers of subjects with and without ob- ly to benefit from clinical review. structive airways disease in the overall sample. As this was a new scoring system, estimates of the expected prevalence The Wythenshawe Community Asthma Project (WYCAP) of obstructive airways disease in each stratum had to be is a long term prospective study of the natural history of derived empirically from the researchers' clinical experi- respiratory symptoms in two general practice populations ence. The number from each stratum invited to the clini- in Manchester, UK. One of its aims is to develop a method cal review and the number who attended is shown in for identifying patients in the community with asthma Table 2. Subjects were invited via telephone or home visit and COPD as the first stage in evaluating the benefits and by their general practitioner to attend their local hospital costs of treating such individuals. In 1993 and 1995, post- for clinical review. al respiratory questionnaire surveys were carried out in both practice populations. We did not feel that it would Clinical assessment be possible to differentiate asthma from COPD using a The clinical assessment (carried out by a clinician, TLF) in- simple postal questionnaire and so two simple scoring volved a structured medical history and physical examina- systems were developed, based on the number of symp- tion. A research assistant performed the investigations, toms/risk factors reported from key questions, both hav- supervised and assisted by the clinician. Neither individu- ing the aim of identifying adults with either asthma or al had access to the postal questionnaire results for the COPD. This paper examines the validity of these scoring subject being examined. systems, using data relating to respondents to the 1995 postal survey. A similar exercise has been performed on The investigations performed have been described in a children in the two practices [5]. This study had ethical ap- previous study [5] and included forced expiratory volume proval from the local research ethics committee and in- in one second (FEV ), forced vital capacity (FVC), FEV / 1 1 FVC, a peak expiratory flow diary, skin prick testing using Page 2 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 Table 2: Response categories of respondents, those invited for and those attending clinical review Number of symptoms/risk factors 0 1–3 4 5 6 Total All respondents 2158 3765 521 355 148 6947 Number invited for clinical review 60 60 150 105 45 420 Number who attended review (% of those invited) 26 (43.3) 30 (50.0) 75 (50.0) 53 (50.5) 18 (40.0) 202 (48.1) Table 3: Level of consultant agreement Number of adults categorised according to whether they merit a trial of obstructive airways disease medication by the level of consultant agreement. Meriting a trial of obstructive airways disease medication Consultant agreement Yes No Total All agreed 73 46 119 2 out of 3 agreed 43 39 82 Final categorisation 116 85 201 One subject was not categorised by one of the consultants and the other two disagreed on the decision on whether a trial of obstructive airways disease treatment was indicated so no majority decision could be made. This individual was excluded from analysis. the skin prick/puncture technique [6] and reversibility stage process [9]. The prevalence of subjects who would testing to β agonists [7]. A bronchial challenge test to his- merit a trial of obstructive airways disease medication was tamine was also performed by the YAN technique [8]. A also estimated using the two-stage process. positive histamine challenge was defined as a fall in FEV of 20% from the pre challenge FEV . Results Of 10,429 questionnaires sent out in 1995, 7,580 were re- turned after three mailings. After a 5.5% adjustment for For each subject, three independent consultant chest phy- sicians were sent full details of the clinical assessment but adults no longer at the mailing address [2], this represent- excluding the details of the postal questionnaire. The con- ed a 78% response rate. 6947 responses contained suffi- sultants were asked to decide whether the subject merited cient information for the present analyses. 420 adults a trial of asthma/COPD (obstructive airways disease) were invited for clinical review of whom 202 (48%) medication. Patients warranting a trial of treatment were attended. said to have a diagnosis of possible obstructive airways disease. Agreement between consultants about the decision to treat is detailed in Table 3. There was unanimous agree- The consultants' opinions were compared with two sets of ment concerning the merits of a trial of treatment in 119 questionnaire scoring systems chosen in advance on the (59.2 %) adults. The majority verdict was used to classify basis of clinical experience as likely predictors of obstruc- the remaining subjects. tive airways disease: Table 4 compares the majority expert opinions against the 1) Four or more symptoms/risk factors reported from the two sets of questionnaire scoring systems. Of 146 adults six key questions. with four or more symptoms/risk factors, 109 (75%) mer- ited a trial of medication. 100 (82%) of the 122 adults 2) Four or more symptoms/risk factors reported from the who reported four or more symptoms/risk factors and six key questions plus at least one marker of severity (Ta- who had a marker of severity were thought by the consult- ble 1). ants to merit a trial of asthma medication. Statistical analysis The positive predictive values (PPV), sensitivity and spe- The positive predictive value, sensitivity, specificity and cificity of the two scoring systems are shown in Table 5. their confidence intervals were calculated for each scoring The PPV for the first scoring system when compared system using the majority consultant opinion as the "gold against the consultants' opinion was 75.1% (95% confi- standard". The method used was appropriate for the two- dence interval (CI) 68.6–82.3), whilst that of the second Page 3 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 Table 4: Comparison between majority expert opinions and questionnaire scoring systems Categories of score "Meriting a trial of "Not meriting a trial of Total medication" medication" All responses 116 85 201* System 1 Four or more symptoms/risk factors 109 37 146 Fewer than four or more symptoms/risk factors 7 48 55 System 2 Four or more symptoms/risk factors plus one marker of severity 100 22 122 Other responses- 16 63 79 * One subject was not categorised by one of the consultants and the other two disagreed on the decision on whether a trial of obstructive airways disease treatment was indicated so no majority decision could be made. This individual was excluded from analysis. Table 5: Positive predictive value, sensitivity and specificity of the two scoring systems Discriminative properties of two scoring systems for questionnaire responses compared with the majority expert opinion about whether the patients merit a trial of obstructive airways disease therapy. Scoring system 1. Four or more symptoms/risk factors 2. Four or more symptoms/risk factors plus one marker of severity PPV % [95% CI] 75.1 [68.6–82.3] 82.3 [75.9–89.2] Sensitivity % [95% CI] 50.3 [35.3–71.6] 46.9 [33.0–66.8] Specificity % [95% CI] 95.3 [94.0–96.7] 97.1 [96.0–98.2] PPV = Positive predictive value CI = Confidence interval system was 82.3% (95% CI 75.9–89.2). The more strin- groups were compared with respect to their age, gender gent second scoring system had a greater specificity but and the number of cigarettes smoked as reported on the poorer sensitivity than the first scoring system. questionnaire. No significant differences were found be- tween the two groups. The prevalence of adults "meriting a trial of obstructive airways disease therapy" (a surrogate for obstructive air- The purpose of a screening test is to identify individuals ways disease) in our survey was calculated [5] to be 22% with a good chance of having disease and who require (95% CI 15.5–31.4). further clinical assessment to confirm or refute the diag- nosis. When defining the threshold at which a screening test is deemed to be positive, consideration has to be tak- Discussion Two simple scoring systems, (based on the number of en of the balance between false positive results (which can symptoms/risk factors reported on a respiratory question- lead to extra distress because of unnecessary further inves- naire) produced high positive predictive values when tigations) and false negatives (which result in some cases compared with majority opinion of three respiratory phy- of disease being missed). The positive predictive value of sicians regarding possible obstructive airways disease. a test reflects the frequency of disease in those with a pos- itive screening test. The cost effectiveness of a screening Two important sources of selection bias could have oc- programme will depend on the cut-off values chosen for curred. First, non-respondents to the questionnaire survey the screening procedure as these determine the number of could have been materially different from respondents. new cases detected and requiring treatment, unnecessary This was examined after the first questionnaire survey in investigations undertaken etc. 1993, when a comparison was made between the practice medical records of a random sample of 100 respondents When assessing a screening test it can be difficult to know and 100 non-respondents. No important differences were what diagnostic standard to use, particularly for condi- found with respect to age, gender, and total number of tions such as obstructive airways disease which do not consultations and consultations for respiratory problems have a universally accepted clinical definition. In our in the previous year. There is no reason to suspect that the study, we defined adults as having possible obstructive situation was different in 1995. Second, the adults attend- airways disease if the majority opinion of three consultant ing for clinical review may have been different from those physicians was that a trial of treatment for obstructive air- who were invited but who did not attend. These two ways disease was merited (majority decisions of specialists Page 4 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 has been used in previous studies to diagnose asthma) Neither the experts nor the examining team were aware of [1,5]. Many clinicians would probably agree that it is rea- the results of the postal questionnaire at the time of their sonable to assess an adult who might merit a trial of ob- involvement in the study. Total agreement by the consult- structive airways disease treatment. Comparing the two ants on whether a subject warranted a trial of medication scoring systems against this "standard", we were able to was reached in 59.2% of adults. Difficulties in making a determine the test characteristics (sensitivity, specificity diagnosis without personally seeing the patient may also and positive predictive value) of each system. The more have contributed to disagreement between consultants al- stringent system (four or more symptoms/risk factors plus though it is likely that some difference of opinion would a marker of severity) had a greater positive predictive val- have persisted even with personal examination. Expert ue and greater specificity (so gave fewer false positive re- opinion concerning asthma diagnosis has been used to sults) than the less stringent system (four or more define asthma in previous epidemiological studies symptoms/risk factors alone), but at the cost of reduced [1,11,12]., whilst others have used patient recall of asth- sensitivity (more cases were missed). ma diagnosis or treatment when measuring prevalence of the disease or its underdiagnosis [13]. The majority of di- Any system, which increased sensitivity, would almost agnostic decisions are made in primary care and we ac- certainly have reduced positive predictive value probably knowledge that choosing three consultant respiratory resulting in and unacceptable rate of false positives. In physicians as the diagnostic gold standard therefore has contrast with screening for malignant conditions when limitations. They were however chosen to reflect a spread detection of all cases is of prime importance (i.e. high sen- of secondary and tertiary respiratory opinions. sitivity), it could be argued that in population screening for diseases such as asthma/COPD the PPV carries more An advantage of the scoring systems used was their sim- weight than sensitivity. plicity. Techniques such as discriminant analysis and lo- gistic regression could have been used to assess the A recent article by Grimes and Schultz [10] illustrates this predictive value of each question and a different scoring point: system developed from them. This would have the disad- vantage that validation would then have been based on "Although sensitivity and specificity are of interest to pub- the same data as was used to derive the scoring system and lic-health policymakers, they are of little use to the clini- therefore would tend to be over optimistic. In addition cian. Stated alternatively, sensitivity and specificity the scoring system might be more difficult to implement. (population measures) look backward (at results gathered over time). Clinicians have to interpret test results to those In clinical practice, a simple scoring system to identify pa- tested. Thus what clinicians need to know are the predic- tients requiring further review is attractive. Choices about tive values of the test (individual measures, which look which system to use will depend on a number of factors; forward)". the balance between positive predictive value and sensi- tivity of each system and available health care resources. It In a previous paper [2], 1112 patients (13.8% of respond- is important to remember that if this questionnaire were ers) were identified positively by the screening question- to be used in another setting the positive predictive value naire. The PPV would indicate that approximately 834 of would have to be recalculated as this value is dependent these had obstructive airways disease. Further, 529 of on the prevalence of disease in the population studied. those who screened positively had no recorded diagnosis of obstructive airways disease or received inhaled medica- Conclusions tion in the previous 12 months. Thus despite the disad- When assessed against the majority expert opinion of vantage of a relatively low sensitivity for the scoring "meriting a trial of obstructive airways disease medica- system significant numbers of patients with obstructive tion", two simple scoring systems based on responses to a airways disease would be identified. It should be noted brief postal questionnaire provided a good method for that the prevalence of adults "meriting a trial of obstruc- identifying adults likely to benefit from a trial of treat- tive airways disease therapy" in our survey was calculated ment for obstructive airways disease and so warranting [5] to be 22% (95% CI 15.5–31.4). This may be an over- clinical review. estimate of the true prevalence of obstructive airways dis- ease as it will include patients who would not benefit Authors' contributions from a trial of treatment. TLF and PIF conceived the study and TLF, PIF, PCH, RMM, SH, and CACP participated in its design and coordination. The three specialists reviewing the clinical information re- TLF, JAC and MLH contributed to data analysis and inter- lied on detailed written data supplied to them by the re- pretation. RRM provided statistical advice, data analysis search team; they did not have direct contact with patient. and interpretation. TLF wrote the manuscript and all the Page 5 of 6 (page number not for citation purposes) BMC Family Practice 2003, 4 http://www.biomedcentral.com/1471-2296/4/5 authors critically revised it and approved the final manuscript Acknowledgements The authors would like to thank the three consultants, Dr J Miles, Dr S Burge and Dr J Williams who provided our expert majority diagnosis and Tamsin Wright who helped with the clinical evaluations; the staff of Bow- land Medical Practice and Tregenna Group Practice and the North West Lung Research Centre; and the subjects who answered the postal question- naire and who attended for clinical review. We would like to thank the Roy- al College of General Practitioners (RCGP) Manchester Research Unit (now relocated to Aberdeen). This study was supported by Glaxo-Well- come, Zeneca Pharmaceuticals, Manchester Airport, The Royal College of General Practitioners and The North West Lung Centre. References 1. De Marco R, Cerveri I, Bugiani M, Ferrari M and Verlato G An unde- tected burden of asthma in Italy: the relationship between clinical and epidemiological diagnosis of asthma Eur Respir J 1998, 11:599-605 2. Frank PI, Ferry S, Moorhead T and Hannaford P Use of a postal questionnaire to estimate the likely under-diagnosis of asth- ma-like illness in adults Br J Gen Pract 1996, 46:295-297 3. Agertoft L and Pedersen S Effects of long-term treatment with an inhaled corticosteroid on growth and pulmonary function in asthmatic children Respir Med 1994, 88:373-381 4. van den Boom G, Rutten-van Molken MP, Folgering H, van Weel C and van Schayck CP The economic effects of screening for ob- structive airway disease: an economic analysis of the DIMCA program Preventive Medicine 2000, 30:302-308 5. Frank TL, Frank PI, McNamee R, Wright T, Hannaford P and Morrison J Assessment of a simple scoring system applied to a screen- ing questionnaire for asthma in children aged 5–15 yrs Eur Respir J 1999, 14:1190-1197 6. Basomba A, Sastre A, Peleaz A, Romar A, Campos A and Garcia-Vil- lalmanzo A Standardisation of the prick test Allergy 1985, 40:395- 7. American Thoracic Society. Standards for the diagnosis and care of patients with chronic obstructive pulmonary disease (COPD) and asthma Am Rev Respir Dis 1987, 225-244 8. Yan K, Salome C and Woolcock AJ. Rapid method for measure- ment of bronchial responsiveness Thorax 1983, 38:760-765 9. McNamee R Optimal designs of two-stage studies for estima- tion of sensitivity, specificity and positive predictive value Sta- tistics in Medicine 2002, 21:3609-3625 10. Grimes DA and Schulz KF Uses and abuses of screening tests Lancet 2002, 359:881-884 11. Jenkins MA, Clarke JR, Carlin JB, Robertson CF, Hopper JL and Dalton MF Validation of questionnaire and bronchial hyperrespon- siveness against respiratory physician assessment in the di- agnosis of asthma Int J Epidemiol 1996, 25:609-616 12. Burr ML Editorial: Diagnosing asthma by questionnaire in ep- idemiological surveys Clin Exp Allergy 1992, 22:509-510 13. Kaur B, Anderson HR, Austin J, Burr M, Harkins LS and Strachan DP Prevalence of asthma symptoms, diagnosis, and treatment in 12–14 year old children across Great Britain (international Publish with Bio Med Central and every study of asthma and allergies in childhood, ISAAC UK) BMJ 1998, 316:118-124 scientist can read your work free of charge "BioMed Central will be the most significant development for Pre-publication history disseminating the results of biomedical researc h in our lifetime." The pre-publication history for this paper can be accessed Sir Paul Nurse, Cancer Research UK here: Your research papers will be: available free of charge to the entire biomedical community http://www.biomedcentral.com/1471-2296/4/5/prepub peer reviewed and published immediately upon acceptance cited in PubMed and archived on PubMed Central yours — you keep the copyright BioMedcentral Submit your manuscript here: http://www.biomedcentral.com/info/publishing_adv.asp Page 6 of 6 (page number not for citation purposes)

Journal

BMC Family PracticeSpringer Journals

Published: Apr 25, 2003

There are no references for this article.