Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Utility of a Clinical Decision Support System in Weight Loss Prediction After Head and Neck Cancer Radiotherapy

Utility of a Clinical Decision Support System in Weight Loss Prediction After Head and Neck... original report abstract Utility of a Clinical Decision Support System in Weight Loss Prediction After Head and Neck Cancer Radiotherapy 1 2 1 1 1 1 Zhi Cheng, MD, MPH ; Minoru Nakatsugawa, PhD ; Xian Chong Zhou, MS ; Chen Hu, PhD ; Stephen Greco, MD ; Ana Kiess, MD, PhD ; 1 1 3 2 2 1 Brandi Page, MD ; Sara Alcorn, MD, PhD ; John Haller, PhD ; Kazuki Utsunomiya, MS ; Shinya Sugiyama, MS ; Wei Fu, MS ; 1 1 1 1 John Wong, PhD ; Junghoon Lee, PhD ; Todd McNutt, PhD ; and Harry Quon, MD, MS PURPOSE To evaluate the utility of a clinical decision support system (CDSS) using a weight loss prediction model. METHODS A prediction model for significant weight loss (loss of greater than or equal to 7.5% of body mass at 3-month post radiotherapy) was created with clinical, dosimetric, and radiomics predictors from 63 patients in an independent training data set (accuracy, 0.78; area under the curve [AUC], 0.81) using least absolute shrinkage and selection operator logistic regression. Four physicians with varying experience levels were then recruited to evaluate 100 patients in an independent validation data set of head and neck cancer twice (ie, a pre- post design): first without and then with the aid of a CDSS derived from the prediction model. At both evaluations, physicians were asked to predict the development (yes/no) and probability of significant weight loss for each patient on the basis of patient characteristics, including pretreatment dysphagia and weight loss and information from the treatment plan. At the second evaluation, physicians were also provided with the prediction model’s results for weight loss probability. Physicians’ predictions were compared with actual weight loss, and accuracy and AUC were investigated between the two evaluations. RESULTS The mean accuracy of the physicians’ ability to identify patients who will experience significant weight loss (yes/no) increased from 0.58 (range, 0.47 to 0.63) to 0.63 (range, 0.58 to 0.72) with the CDSS (P = .06). The AUC of weight loss probability predicted by physicians significantly increased from 0.56 (range, 0.46 to 0.64) to 0.69 (range, 0.63 to 0.73) with the aid of the CDSS (P , .05). Specifically, more improvement was observed among less-experienced physicians (P , .01). CONCLUSION Our preliminary results demonstrate that physicians’ decisions may be improved by a weight loss CDSS model, especially among less-experienced physicians. Additional study with a larger cohort of patients and more participating physicians is thus warranted for understanding the usefulness of CDSSs. Clin Cancer Inform. © 2019 by American Society of Clinical Oncology Licensed under the Creative Commons Attribution 4.0 License INTRODUCTION patients at high risk of weight loss may trigger per- sonalized weight management strategies, including Patients with head and neck cancer (HNC) are at high nutritionist referrals, speech and swallow assess- risk of weight loss as a result of their disease site and ASSOCIATED ments, and even prophylactic prevention of malnu- 1,2 CONTENT treatment, particularly with radiation therapy (RT). trition by placement of a feeding tube when indicated. Appendix RT-induced adverse effects, such as painful mucosal In particular, patients who are afraid of and refuse Author affiliations inflammation of the oropharynx, secretion change prophylactic feeding tube placement before the and support from the salivary gland injury, taste or smell alteration, information (if 3 treatment would benefit from the accurate prediction and damage to muscles involved with swallowing, can applicable) appear at of weight loss. Although several studies have in- result in decreased oral intake. In turn, patients may the end of this vestigated predictive factors of weight loss during or experience significant weight loss, which is defined as article. 8,9 after the treatment, it is still unclear how these loss of greater than or equal to 5% of body mass Accepted on factors can be used in clinical practice. December 14, 2018 between the start of RT through week 4 or greater than and published at or equal to 7.5% loss through week 12. Moreover, A clinical decision support system (CDSS) may benefit ascopubs.org/journal/ severe weight loss leads to compromised quality of life, patient care and potentiate a new paradigm of medical cci on March 12, 5-7 10-12 delayed recovery, and unfavorable prognosis. Tox- practice. Although previous models have been 2019: DOI https://doi. icities may become progressively worse after treat- developed and tested with internal and external vali- org/10.1200/CCI.18. 13-15 00058 ment when less monitored; therefore, identification of dation, few studies have assessed clinical utility 1 Cheng et al and effectiveness. In a recent pilot study comparing pre- Experiment system section). In the second evaluation, the dictive accuracy between physicians and models for survival physicians were also provided with the probability of weight and toxicity outcomes after chemoradiation (CRT) in lung loss predicted by the model, both the overall probability of cancer, management models consistently outperformed weight loss and the probability of weight loss associated physicians for all outcomes. This lends early support for the with each category of the predictors, such as tumor loca- potential impact of including robust models into clinical tion, chemotherapy, radiation dosage, and image features practice. of parotid glands, submandibular glands, and larynx (Fig 3; see Experiment system section). At both evaluation time Our group has previously developed a predictive model for points, the physicians were asked to provide predictions of significant weight loss in patients with HNC treated with significant weight loss (yes/no [Y/N]) and the probability of 16,17 concurrent CRT. This study seeks to evaluate the utility the significant weight loss (0 to 1) using an electronic of this weight loss prediction model within a CDSS. We questionnaire (Appendix Fig A1; see Experiment system hypothesize that a CDSS including a weight loss prediction section). Physicians’ predictions were compared with the algorithm for patients with HNC can improve clinicians’ actual weight loss, and accuracy and area under the curve predictions for significant weight loss, thus possibly en- (AUC) were investigated between the two evaluations and abling earlier intervention. the model-only prediction. This study was approved by the institutional review board. METHODS Details of the Study Overview of the Study Procedures Weight loss prediction model using radiomic features. Figure 1 shows an overview of the study procedures, and Clinical and dosimetric data were systematically captured detailed information is in the Experiment system section. A during routine clinical care into our database Onco- prediction model for significant weight loss (loss of greater 18,19 space. The structured database facilitated prospective than or equal to 7.5% of body mass at 3 months post RT) collection of patient clinical assessments and treatment in- was created using clinical, dosimetric, and radiomic pre- formation captured as a part of the routine clinical workflow. dictor data from a training set of 63 patients. An in- Clinical data were routinely captured at the point of care dependent evaluation set of 100 HNC cases was selected during weekly on-treatment evaluations and follow-up for this study. Four physicians were assigned to evaluate visits, including tumor location, type of chemotherapy, each test set case via an electronic platform at two time and clinical assessment of RT-induced adverse effects. For points separated by 30 days: first without and second with this study, 163 patients with HNC who were treated with RT the aid of the CDSS. In both evaluations, a narrative patient between 2009 and 2015 and had measured weight at summary with a treatment plan report including isodose baseline and 3-month visit (61 to 119 days post RT) were lines and dose volume histograms were provided (Fig 2; see included. Dose-volume histograms from these patients’ plans were queried from the departmental database to provide CDSS model development dosimetric information for structures including parotid and Weight loss prediction model was built by using external 63 submandibular glands, larynx, and superior constrictor patients' data. muscles. Planning computed tomography (CT) images and contours were extracted from the picture archiving and communication system. An in-house imaging feature engine First evaluation by four physicians without CDSS calculated the radiomics imaging features from contoured Physician's prediction of weight loss was answered for 100 regions of interest on planning CT images. Radiomics features patients. Information of patient summary with RT plan was provided. included volume, shape, first-order statistics for the distri- bution of intensities, and texture for the parotid and sub- mandibular glands, larynx, and superior constrictor muscles. Two evaluations were separated by 30 days These data were combined and used in the analysis and model development as displayed in Appendix Figure A2. Of the 163 patients included, 100 patients were randomly Second evaluation by the same four physicians with CDSS selected to compose the evaluation dataset. The remaining Prediction by the CDSS was also provided. Physician's prediction of weight loss was answered for the 63 patients were assigned to the training set for the pre- 100 patients again. diction model for significant weight loss, which was per- formed using least absolute shrinkage and selection operator logistic regression (Appendix). Predictive fac- Statistical comparison for the first and second evaluation results tors selected by least absolute shrinkage and selection operator included tumor location, type of chemotherapy, dose to parotid glands, shape features of parotid glands, FIG 1. Overview of study procedures. CDSS, clinical decision support system; RT, radiation therapy. and texture features of parotid glands/submandibular 2 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss All measurements are at planning stage (0-1 week of radiotherapy). Mr. Pt 1 is a 57-year-old white male with recently diagnosed T4 N0 M0 tonsil carcinoma. Currently, he presents with dysphagia and weight loss. His weight is 79.4 kg. We recommend chemoradiation. The tumor volume is 83 cc. Dose Volume Histogram DVH Calculation 1.0 DVH Legend Brain 0.9 PTV58.1_eval 0.8 PTV63_eval 0.7 PTV70_eval 0.6 brainstem 0.5 cord 0.4 cord4mm mandible 0.3 0.2 0.1 0.0 0 1000 2000 3000 4000 5000 6000 7000 80 Dose (cGy) DVH Calculation Dose Volume Histogram 1.0 DVH Legend esophagus 0.9 larynx for edema 0.8 It brachial plexus 0.7 It parotid 0.6 rt brachial plexus 0.5 rt parotid 0.4 0.3 0.2 0.1 0.0 0 1000 2000 3000 4000 5000 6000 7000 80 Dose (cGy) FIG 2. Clinical decision support system case presentation. glands/larynx (Appendix Table A1). The model indicated practice post residency; Drs C and D: junior physician,, 5 years that the imaging features can predict weight loss in com- of practice post residency) as participants in our study. bination with the dosimetric and clinical features. Leave- Experiment system. An experimental system displaying an one-out cross-validation was used to evaluate the AUC of anonymized patient list, a clinical case presentation, the model. The prediction performance of leave-one-out a display of CDSS information, and a questionnaire for cross-validation by the 63 cases had accuracy of 0.78 and physician predictions was developed for evaluation (Figs 2 AUC of 0.81. and 3; Appendix Fig A1). Participating physicians used this system to assess the 100 patients in the evaluation set. Sample size for evaluation. The primary objective was to Physicians were provided with written and video instruction assess if a physician’s prediction of significant weight loss and had a practice session using 11 nonstudy patient cases improves with the support of the CDSS. To properly account to familiarize themselves with the system before the for the correlation between the first and second evaluation evaluation. for each patient, the sample size calculation and primary analysis were performed on the basis of one-sided For the clinical case presentation, the system first provided McNemar test. Assuming that a physician would revise the physicians with a deidentified clinical case narrative his or her prediction after CDSS ≤ 25% of the time, including patient age, sex, race, TNM staging, tumor lo- a sample size of 100 was selected to provide ≥ 80% power cation, pretreatment weight (kilograms), presence of pre- to detect a 15% improvement in rate of correctly predicting treatment dysphagia (Y/N), pretreatment weight loss (Y/N), the clinical outcome. treatment modality (CRT or RT alone), planning target volume, isodose lines, and dose-volume histograms of Participating physicians. Because a physician’s ability to regions of interest and their contours on RT-planning CT accurately predict significant weight loss is likely to be images (example in Fig 2). After presentation of these data, influenced by his or her clinical experience in the manage- the physicians asked to answer the following questions: (1) ment of irradiated HNC, we recruited four physicians of dif- Based on the case presentations, do you believe this patient ferent levels of experience (Drs A and B: senior, . 5 years of will have significant weight loss (defined as ≥ 7.5% weight JCO Clinical Cancer Informatics 3 Norm.Volume Norm.Volume Cheng et al FIG 3. Clinical decision support system display for prediction results from model. loss) as measured at 3 months post-RT to the weight (Fig 3). Physicians were blinded to their previous answers measured at the start of the radiation? Yes/No. (2) Based on as well as to the model performance and the outcome of each the case presentation, what is the probability of significant case. Each physician completed the evaluation independently. weight loss that you would estimate? Estimate (%) as free text (Appendix Fig A2). Statistical Analysis After the physicians’ initial evaluation of all 100 patients was Descriptive statistics were used to summarize patient and completed without the aid of the CDSS, a 30-day wash-out treatment characteristics. The physicians’ evaluations were period was taken. Then, the physicians were asked to compared with actual post-RT weight loss (ground truth) repeat the evaluation for the same 100 patients, this time from patient records. For analysis, binary weight loss (Y/N) with the CDSS results included. The display of CDSS in- was used to calculate accuracy, sensitivity, specificity, formation showed the overall probability of weight loss (0 to 1) positive predictive value (PPV), and negative predictive predicted by the model, its predictors, and the probability of value (NPV), and probability of weight loss (%) was used to weight loss associated with each predictor for the patient calculate AUC. The improvement of physicians’ prediction 4 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss before and after CDSS was evaluated using one-sided TABLE 1. Study Population Characteristics (N = 100) Parameter No. (%) McNemar test for accuracy, and Delong test for AUC, re- spectively. Accuracy is defined as the number of correct Age, mean (SD) 58 (10) predictions (true positive plus true negative) divided by the PTV volume, mL, mean (SD) 130 (116) number of all assessments. Here, true positive represents Sex a correct prediction of weight loss, and true negative Male 79 (79) represents a correct prediction of no weight loss. The Female 21 (21) maximum value of the AUC is 1.0, indicating a perfect prediction model; a value of 0.5 is equivalent to a random Race classifier. Statistical analysis was performed by SAS 9.3 White 74 (74) software, with significance level at the 0.05. African American 20 (20) RESULTS Asian and others 6 (6) Chemotherapy Baseline characteristics for all 100 patients are presented in Table 1. The mean age was 57 years, and the majority of Yes 77 (77) patients were white (75%) and male (79%). For the primary No 23 (23) diagnosis, 32% had an advanced T stage (T3 or T4), and Tumor site 58% had an advanced N stage (N2b or N3). Site of disease Nasal cavity/nasopharynx 14 (14) was 13% nasal cavity/nasopharynx, 48% oropharynx, 10% Oral cavity 9 (9) oral cavity, and 20% hypopharynx/larynx. Most treatment regimens (77%) included chemotherapy. Patient-reported Oropharynx 49 (49) pretreatment dysphagia and weight loss were noted in 14% Hypopharynx/larynx/thyroid 23 (23) and 19% of cases, respectively. In terms of weight mea- Others 5 (5) surements before and after RT, 60% of patients experi- T stage enced significant weight loss of greater than or equal to T1-T2 62 (62) 7.5% of their body mass at 3 months post RT. T3-T4 32 (32) The prediction performance of the prediction model for the Missing 6 (6) 100 test samples used for the CDSS evaluation had accuracy of 0.73 and AUC of 0.75. The sensitivity, specificity, PPV, N stage and NPV were 0.80, 0.63, 0.76, and 0.68, respectively. N0-N2a 36 (36) Figure 4 presents the average performance of participating . N2b 58 (58) physicians with and without CDSS. Before using CDSS, the Missing 6 (6) accuracy, sensitivity, specificity, PPV, NPV, and AUC of M stage physicians’ performance were 0.58, 0.69, 0.42, 0.64, 0.47, M1 2 (2) and 0.56, respectively. With CDSS, physicians’ perfor- M0 92 (92) mance improved in almost every parameter except for Missing 6 (6) specificity; accuracy, sensitivity, specificity, PPV, NPV, and AUC of physicians’ performance were 0.63, 0.84, 0.31, Baseline dysphagia 0.64, 0.56, and 0.69, respectively. Yes 14 (14) Table 2 illustrates the performance by each physician. As No 86 (86) shown, physicians had fair to moderate accuracy and AUC Baseline weight loss for predicting significant weight loss without CDSS. Mean Yes 19 (19) accuracy was 0.58 (ranging from 0.47 to 0.63) without No 81 (81) CDSS and improved to 0.63 (ranging from 0.58 to 0.72) with CDSS. The improvement was not statistically signifi- Abbreviation: PTV, XXXX. cant (P = .06). In terms of AUC, physicians’ predictions improved from 0.56 (ranging from 0.46 to 0.64) to 0.69 DISCUSSION (ranging from 0.63 to 0.73), which was statistically sig- nificant (P , .05). Clinically, this increase in AUC repre- There is an emerging interest in using prediction models sents increased discernment of patients at high versus low derived from machine learning algorithms to aid clinical 14,15,23-25 risk of weight loss with use of the CDSS. More improve- decision making. In this study, we developed and ment was observed among less-experienced physicians validated a prediction model for significant weight loss (Dr C: AUC before, 0.46; AUC after, 0.70; P , .01; Dr D: among patients with HNC receiving RT and then further AUC before, 0.55; AUC after, 0.72; P , .01). Table 2 also evaluated whether the model improved physicians’ pre- shows that sensitivity and specificity vary among physician. dictions for weight loss. In general, our model predicted the JCO Clinical Cancer Informatics 5 Cheng et al by category of predictive factors such as image features, tumor location, chemotherapy, and radiation dosage (Fig 3). 80 Notably, our study showed that the effect of the CDSS varied among the physicians. Even without use of the CDSS, there was appreciable variation in weight loss pre- dictions among the four physicians. Yet the availability of CDSS influenced the decisions made by all physicians, with impact most noticeable among those with less clinical 30 experience. This suggests that junior physicians may benefit more from use of the CDSS. Moreover, in addition to clinical applications, the CDSS may have utility as an ed- ucational tool for simulation training among junior physi- cians and trainees. Indeed, by displaying predictive Accuracy Sensitivity Specificity PPV NPV AUC Physician alone Physician + CDSS dosimetric variables and resultant outcomes (Fig 3), the CDSS may provide a novel means for improving physicians’ FIG 4. Comparison of prediction performance between physician clinical reasoning relative to these parameters. Conversely, alone and physician with clinical decision support system (CDSS). accuracy for the binary classification of patients with/ AUC, area under the curve; NPV, negative predictive value; PPV, without weight loss did not change significantly with the use positive predictive value. of the CDSS. This binary classification is dependent on an unobserved threshold for predicted probability of weight outcome of weight loss more accurately than physicians’ loss held by each physician. In the case of Drs C and D, it is predictions. We also demonstrated that physician’s predictions assumed that the threshold was risk averse and much lower improved with the aid of a CDSS derived from our model, most than the optimal value, leading to high sensitivity and low noticeably among junior physicians. Moreover, the CDSS specificity. Additional training or supplementary educational enabled physicians to better distinguish between patients at tools may be required to address such unobserved thresholds. higher versus lower risk of weight loss, perhaps allowing them Performance of the CDSS according to the various metrics to be targeted for clinical interventions. To our knowledge, this can affect its role in clinical use. For example, variation in is the first published study to evaluate the utility of a CDSS for feeding tube use suggests that CDSS could play a role in weight loss among patients with HNC undergoing RT. facilitating more appropriate and timely use. Reducing its overuse would require the CDSS tool be highly specific, and Several unique features of our model and resultant CDSS may reducing its underuse would require the tool to be highly explain the improvement in predictions that we observed. sensitive. We expect that the tool we developed will only get Specifically, the predictions may be optimized by our in- better in future iterations, but already we believe that with clusion of radiomics features into the model. Radiomics data a specificity and sensitivity of 0.63 and 0.80 it could have are concrete and comprehensive, and their predictive value a role in clinical practices, particularly those that are has been increasingly appreciated in recent years. How- underusing feeding tubes. ever, such data can be complicated and unwieldy, perhaps explaining why radiomics parameters are not yet commonly Our study has the potential for recall bias, because the accepted and adapted into clinical settings. Because pre- same set of patient cases was presented and evaluated sentation of such information in an understandable way is twice. However, a 30 day wash-out period was provided in critical to its optimal use, our CDSS may have provided a means an attempt to decrease this bias. Our study is also limited for physicians to apply valuable radiomics information. In our by a small number of physicians and the use of study, we provided the summarized probability of weight loss single-institution data. Given the limited scope of patient TABLE 2. Comparison Between Without and With CDSS: Accuracy, Area Under the Curve, Sensitivity, and Specificity Accuracy Area Under the Curve Sensitivity Specificity Physician Without CDSS With CDSS P Without CDSS With CDSS P Without CDSS With CDSS Without CDSS With CDSS A 0.47 0.58 .08 0.60 0.63 .58 0.22 0.55 0.85 0.63 B 0.61 0.72 .08 0.64 0.73 .18 0.63 0.82 0.58 0.58 C 0.63 0.59 .25 0.46 0.70 , .01 0.92 0.98 0.2 0 D 0.61 0.61 1.00 0.55 0.72 , .01 0.98 1.00 0.05 0.03 Overall 0.58 0.63 .06 0.56 0.69 , .05 0.69 0.84 0.42 0.31 Abbreviation: CDSS, clinical decision support system 6 © 2019 by American Society of Clinical Oncology Prediction Performance (%) Utility of a Clinical Decision Support System for Weight Loss TABLE 3. Odds Ratios and Regularized β Coefficients Selected Out Using Least the tool may enable physicians to better identify patients Absolute Shrinkage and Selection Operator Logistic Regression with higher versus lower probability of weight loss, evoking 27,28 Regularized β Odds targeted implementation of anti–weight loss strategies. Predictive Factor Coefficients Ratio Because increasing weight loss is correlated with more Tumor location, tongue −0.070 0.93 hospital admissions, higher infection rates, and decreased Tumor location, nasopharynx 1.757 5.80 survival among patients receiving CRT for HNC, such targeted interventions may also improve important clinical Tumor location, nasal cavities −1.342 0.26 outcomes for patients. Given this potential impact, more Tumor location, larynx −1.792 0.17 comprehensive studies of the CDSS, with refined clinical Chemotherapy, cisplatin 0.905 2.47 information, inclusion of more patients and practitioners, Combined parotid d25 0.018 1.02 validation with external data sets, and possibly prospective Combined parotid d30 0.032 1.03 clinical trials are warranted. Combined parotid D80 0.029 1.03 An additional limitation is the simulated nature of the study Parotid fractal dimension 0.160 1.17 design. This was a retrospective evaluation, which limited Parotid GLCM cluster prominence −0.834 0.43 our access to patient characteristics to those that were collected at the point of care, and so we were not able to Parotid GLCM correlation −0.392 0.68 adjust for important factors like comorbidities, social Parotid RLE short emphasis −0.077 0.93 support, family support, insurance status, and the patient’s Parotid RLE long emphasis 0.337 1.40 investment in their own care. In a prospective evaluation, Submandibular GLCM inverse difference −0.128 0.88 we hope to account for these factors. Furthermore, defi- moment nition of significant weight loss for symptom management Submandibular GLCM inverse correlation 0.292 1.34 may vary among clinicians. Other factors, such as prolonged narcotics use, feeding tube dependency, patient’sactivity Submandibular RLE run length uniformity 0.244 1.28 level post treatment, and quality of life in relation to weight Submandibular RLE short/low emphasis −0.012 0.99 loss, may have shaped what is viewed as significant weight Larynx GLCM contrast 2 0.329 1.39 loss. Converting their own thresholds to 7.5% may affect the Intercept −1.643 prediction during the study and also the use of the CDSS in the clinical setting. This also reflects the complexity of Abbreviations: GLCM, Gray-level co-occurrence matrix; RLE, XXXX. medicine and the realistic limitations on our ability to quantify the patient’s condition, as opposed to using the physician’s information provided for review and the simplicity of the impressions when making decisions. study design, it is recognized that the study cannot evaluate the full clinical benefits of the CDSS for weight loss man- In summary, we have described the development of a ro- agement. Instead, the study aimed to develop and describe bust prediction model that can outperform physicians’ how the CDSS might be deployed and to suggest the po- predictions for weight loss in patients with HNC. We have tential for clinical impact on patient care. In particular, provided preliminary evidence that a CDSS that is based on improvement of AUC with use of the CDSS indicates that this model can be used to improve physicians’ predictions. AUTHOR CONTRIBUTIONS AFFILIATIONS Conception and design: Zhi Cheng, Minoru Nakatsugawa, Chen Hu, Ana Johns Hopkins University, Baltimore, MD Kiess, John Haller, Shinya Sugiyama, John Wong, Junghoon Lee, Todd Canon Medical Systems, Otawara, Japan McNutt, Harry Quon Canon Medical Research USA, Vernon Hills, IL Financial support: Zhi Cheng, John Haller, Kazuki Utsunomiya, Shinya Sugiyama, John Wong, Todd McNutt CORRESPONDING AUTHOR Administrative support: Zhi Cheng, John Haller, Shinya Sugiyama, Zhi Cheng, MD, MPH, 401 North Broadway, B141, Baltimore, MD John Wong 21231; e-mail: zcheng4@jhmi.edu. Provision of study material or patients: Zhi Cheng, Ana Kiess, Brandi Page, Todd McNutt, Harry Quon PRIOR PRESENTATION Collection and assembly of data: Zhi Cheng, Stephen Greco, Ana Kiess, Presented in part at the Annual Meeting of the Radiological Society of Brandi R. Page, Sara Alcorn, Junghoon Lee, Todd McNutt, Harry Quon North America, Chicago, IL, November 26-December 1, 2017. Data analysis and interpretation: Zhi Cheng, Minoru Nakatsugawa, Xian Chong Zhou, Chen Hu, Stephen Greco, Ana Kiess, Brandi Page, Sara R. Alcorn, Kazuki Utsunomiya, Wei Fu, Junghoon Lee, Todd McNutt, SUPPORT Harry Quon Supported by Canon Medical Systems Corporation, Radiation Oncology Manuscript writing: All authors Institute, the Commonwealth Foundation, Elekta, Philips Radiation Oncology Systems, and the Johns Hopkins University. Final approval of manuscript: All authors JCO Clinical Cancer Informatics 7 Cheng et al AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST John Haller AND DATA AVAILABILITY STATEMENT Employment: Canon Medical Research USA The following represents disclosure information provided by authors of Research Funding: Canon Medical Research USA this manuscript. All relationships are considered compensated. Patents, Royalties, Other Intellectual Property: Patent: Method and apparatus Relationships are self-held unless noted. I = Immediate Family Member, for determining treatment region and mitigating radiation toxicity Inst = My Institution. Relationships may not relate to the subject matter of Travel, Accommodations, Expenses: Canon Medical Research USA this manuscript.For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/site/ifc. Kazuki Utsunomiya Employment: Canon Medical Systems Zhi Cheng Shinya Sugiyama Research Funding: Toshiba (Inst) Employment: Canon Medical Systems Minoru Nakatsugawa Employment: Canon Medical Systems Junghoon Lee Research Funding: Canon Medical Systems (Inst) Xiang Chong Zhou Todd McNutt Employment: PRA Health Sciences Stock and Other Ownership Interests: Oncospace Stock and Other Ownership Interests: PRA Health Sciences Research Funding: Canon (Inst), Philips Healthcare (Inst) Chen Hu Patents, Royalties, Other Intellectual Property: Radiation dose calculation Consulting or Advisory Role: Merck Sharp & Dohme algorithm Expert Testimony: Elekta Ana Kiess Research Funding: Advanced Accelerator Applications/Novartis (Inst) No other potential conflicts of interest were reported Travel, Accommodations, Expenses: Augmenix ACKNOWLEDGMENT We thank Canon Medical Systems Corporation and Radiation Oncology Institute for the funding provided for this research. REFERENCES 1. Munshi A, Pandey MB, Durga T, et al: Weight loss during radiotherapy for head and neck malignancies: What factors impact it? Nutr Cancer 47:136-140, 2003 2. Pressoir M, Desne S, Berchery D, et al: Prevalence, risk factors and clinical implications of malnutrition in French Comprehensive Cancer Centres. Br J Cancer 102:966-971, 2010 3. Chasen MR, Bhargava R: A descriptive review of the factors contributing to nutritional compromise in patients with head and neck cancer. Support Care Cancer 17:1345-1351, 2009 4. White JV, Guenter P, Jensen G, et al: Consensus statement: Academy of Nutrition and Dietetics and American Society for Parenteral and Enteral Nutrition: characteristics recommended for the identification and documentation of adult malnutrition (undernutrition). JPEN J Parenter Enteral Nutr 36:275-283, 2012 5. Langius JAE, Zandbergen MC, Eerenstein SEJ, et al: Effect of nutritional interventions on nutritional status, quality of life and mortality in patients with head and neck cancer receiving (chemo)radiotherapy: A systematic review. Clin Nutr 32:671-678, 2013 6. Langius JAE, Bakker S, Rietveld DHF, et al: Critical weight loss is a major prognostic indicator for disease-specific survival in patients with head and neck cancer receiving radiotherapy. Br J Cancer 109:1093-1099, 2013 7. Langius JA, van Dijk AM, Doornaert P, et al: More than 10% weight loss in head and neck cancer patients during radiotherapy is independently associated with deterioration in quality of life. Nutr Cancer 65:76-83, 2013 8. Mallick I, Gupta SK, Ray R, et al: Predictors of weight loss during conformal radiotherapy for head and neck cancers - how important are planning target volumes? Clin Oncol (R Coll Radiol) 25:557-563, 2013 9. Langius JAE, Twisk J, Kampman M, et al: Prediction model to predict critical weight loss in patients with head and neck cancer during (chemo)radiotherapy. Oral Oncol 52:91-96, 2016 10. Lambin P, Roelofs E, Reymen B, et al: Rapid Learning health care in oncology - an approach towards decision support systems enabling customised radiotherapy. Radiother Oncol 109:159-164, 2013 11. Lambin P, van Stiphout RGPM, Starmans MHW, et al: Predicting outcomes in radiation oncology--multifactorial decision support systems. Nat Rev Clin Oncol 10:27-40, 2013 12. Lambin P, Zindler J, Vanneste BGL, et al: Decision support systems for personalized and participative radiation oncology. Adv Drug Deliv Rev 109:131-153, 13. Lustberg T, Bailey M, Thwaites DI, et al: Implementation of a rapid learning platform: Predicting 2-year survival in laryngeal carcinoma patients in a clinical setting. Oncotarget 7:37288-37296, 2016 14. Dehing-Oberije C, De Ruysscher D, Petit S, et al: Development, external validation and clinical usefulness of a practical prediction model for radiation-induced dysphagia in lung cancer patients. Radiother Oncol 97:455-461, 2010 15. Oberije C, Nalbantov G, Dekker A, et al: A prospective study comparing the predictions of doctors versus models for treatment outcome of lung cancer patients: A step toward individualized care and shared decision making. Radiother Oncol 112:37-43, 2014 16. Cheng Z, Nakatsugawa M, Hu C, et al: Evaluation of classification and regression tree (CART) model in weight loss prediction following head and neck cancer radiotherapy. Adv Radiat Oncol 3:346-355, 2017 17. Nakatsugawa M, Cheng Z, Goatman KA, et al: Radiomic analysis of salivary glands and its role for predicting xerostomia in irradiated head and neck cancer patients. Int J Radiat Oncol Biol Phys 96:S217, 2016 (suppl) 18. Bowers M, Robertson S, Moore J, et al: SU-E-P-26: Oncospace: A shared radiation oncology database system designed for personalized medicine, decision support, and research. Med Phys 42:3232, 2015 8 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss 19. Robertson SP, Quon H, Kiess AP, et al: A data-mining framework for large scale analysis of dose-outcome relationships in a database of irradiated head and neck cancer patients. Med Phys 42:4329-4337, 2015 20. Aerts HJWL, Velazquez ER, Leijenaar RTH, et al: Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006, 2014 21. Tibshirani R: Regression shrinkage and selection via the lasso: a retrospective. 73:273-282, 2011 22. Rice M, Harris GT: Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law Hum Behav 29:615-620, 2005 23. Mattonen SA, Palma DA, Johnson C, et al: Detection of local cancer recurrence after stereotactic ablative radiation therapy for lung cancer: physician performance versus radiomic assessment. Int J Radiat Oncol Biol Phys 94:1121-1128, 2016 24. Bornstein BH, Emler AC: Rationality in medical decision making: a review of the literature on doctors’ decision-making biases. J Eval Clin Pract 7:97-107, 2001 25. Liu J, Wyatt JC, Altman DG: Decision tools in health care: focus on the problem, not the solution. BMC Med Inform Decis Mak 6:4, 2006 26. Lambin P, Leijenaar RTH, Deist TM, et al: Radiomics: The bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749-762, 2017 27. Rosenthal DI, Lewin JS, Eisbruch A: Prevention and treatment of dysphagia and aspiration after chemoradiation for head and neck cancer. J Clin Oncol 24:2636-2643, 2006 28. Paccagnella A, Morello M, Da Mosto MC, et al: Early nutritional intervention improves treatment tolerance and outcomes in head and neck cancer patients undergoing concurrent chemoradiotherapy. Support Care Cancer 18:837-845, 2010 29. Capuano G, Grosso A, Gentile PC, et al: Influence of weight loss on outcomes in patients with head and neck cancer undergoing concomitant chemo- radiotherapy. Head Neck 30:503-508, 2008 nn n JCO Clinical Cancer Informatics 9 Cheng et al of the contoured ROIs. The ROIs included were both ipsilateral/ APPENDIX contralateral parotid and submandibular glands. The following types of Development of Prediction Model imaging features were calculated: volume; shape:aspect ratio, con- volutedness and compactness, sphericity, radius, and fractal di- The weight loss prediction model was developed with dosimetric mension; first-order statistics for the distribution of intensities: mean, features, clinical features, and imaging features. The dosimetric fea- median, maximum, minimum, range, interquartile range, quantiles, tures included the DVH for the combined parotid and submandibular standard deviation, skew, and kurtosis; texture measures: Gray-level glands at every 5% volume increments: for example, combo parotid co-occurrence matrix, Gray-level run-length matrix, neighborhood D05, D10, D15, and so on. The clinical features included patient gray-tone difference matrix, and Laws’ filter. Least absolute shrinkage demographics (age, sex), tumor characteristics (TNM stage, location and selection operator logistic regression was applied to predict weight by International Classification of Diseases, 9th revision code), and loss greater than or equal to 7.5% at 3 months post radiation therapy. existence and type of chemotherapy. A total of 5,086 imaging features were calculated by an in-house radiomics calculation engine for each FIG A1. Clinical decision support system display for entering evaluation answer. 10 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss Imaging data Dosimetry data Clinical data Diagnosis, assessments Dose distribution Planning CT with contours Informatics platform Imaging feature calculation engine (Oncospace) Radiomics Prediction model Machine learning algorithm of weight loss LASSO logistic regression Imaging features of OARs FIG A2. Development of weight loss prediction model. CT, computed tomography; LASSO, least absolute shrinkage and selection operator; OAR, XXXX. JCO Clinical Cancer Informatics 11 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JCO Clinical Cancer Informatics Wolters Kluwer Health

Utility of a Clinical Decision Support System in Weight Loss Prediction After Head and Neck Cancer Radiotherapy

Download PDF
11 pages

Loading next page...
 
/lp/wolters-kluwer-health/utility-of-a-clinical-decision-support-system-in-weight-loss-CT0EtBedAN

References (30)

Publisher
Wolters Kluwer Health
Copyright
(C) 2019 by Lippincott Williams & Wilkins, Inc.
ISSN
2473-4276
DOI
10.1200/CCI.18.00058
Publisher site
See Article on Publisher Site

Abstract

original report abstract Utility of a Clinical Decision Support System in Weight Loss Prediction After Head and Neck Cancer Radiotherapy 1 2 1 1 1 1 Zhi Cheng, MD, MPH ; Minoru Nakatsugawa, PhD ; Xian Chong Zhou, MS ; Chen Hu, PhD ; Stephen Greco, MD ; Ana Kiess, MD, PhD ; 1 1 3 2 2 1 Brandi Page, MD ; Sara Alcorn, MD, PhD ; John Haller, PhD ; Kazuki Utsunomiya, MS ; Shinya Sugiyama, MS ; Wei Fu, MS ; 1 1 1 1 John Wong, PhD ; Junghoon Lee, PhD ; Todd McNutt, PhD ; and Harry Quon, MD, MS PURPOSE To evaluate the utility of a clinical decision support system (CDSS) using a weight loss prediction model. METHODS A prediction model for significant weight loss (loss of greater than or equal to 7.5% of body mass at 3-month post radiotherapy) was created with clinical, dosimetric, and radiomics predictors from 63 patients in an independent training data set (accuracy, 0.78; area under the curve [AUC], 0.81) using least absolute shrinkage and selection operator logistic regression. Four physicians with varying experience levels were then recruited to evaluate 100 patients in an independent validation data set of head and neck cancer twice (ie, a pre- post design): first without and then with the aid of a CDSS derived from the prediction model. At both evaluations, physicians were asked to predict the development (yes/no) and probability of significant weight loss for each patient on the basis of patient characteristics, including pretreatment dysphagia and weight loss and information from the treatment plan. At the second evaluation, physicians were also provided with the prediction model’s results for weight loss probability. Physicians’ predictions were compared with actual weight loss, and accuracy and AUC were investigated between the two evaluations. RESULTS The mean accuracy of the physicians’ ability to identify patients who will experience significant weight loss (yes/no) increased from 0.58 (range, 0.47 to 0.63) to 0.63 (range, 0.58 to 0.72) with the CDSS (P = .06). The AUC of weight loss probability predicted by physicians significantly increased from 0.56 (range, 0.46 to 0.64) to 0.69 (range, 0.63 to 0.73) with the aid of the CDSS (P , .05). Specifically, more improvement was observed among less-experienced physicians (P , .01). CONCLUSION Our preliminary results demonstrate that physicians’ decisions may be improved by a weight loss CDSS model, especially among less-experienced physicians. Additional study with a larger cohort of patients and more participating physicians is thus warranted for understanding the usefulness of CDSSs. Clin Cancer Inform. © 2019 by American Society of Clinical Oncology Licensed under the Creative Commons Attribution 4.0 License INTRODUCTION patients at high risk of weight loss may trigger per- sonalized weight management strategies, including Patients with head and neck cancer (HNC) are at high nutritionist referrals, speech and swallow assess- risk of weight loss as a result of their disease site and ASSOCIATED ments, and even prophylactic prevention of malnu- 1,2 CONTENT treatment, particularly with radiation therapy (RT). trition by placement of a feeding tube when indicated. Appendix RT-induced adverse effects, such as painful mucosal In particular, patients who are afraid of and refuse Author affiliations inflammation of the oropharynx, secretion change prophylactic feeding tube placement before the and support from the salivary gland injury, taste or smell alteration, information (if 3 treatment would benefit from the accurate prediction and damage to muscles involved with swallowing, can applicable) appear at of weight loss. Although several studies have in- result in decreased oral intake. In turn, patients may the end of this vestigated predictive factors of weight loss during or experience significant weight loss, which is defined as article. 8,9 after the treatment, it is still unclear how these loss of greater than or equal to 5% of body mass Accepted on factors can be used in clinical practice. December 14, 2018 between the start of RT through week 4 or greater than and published at or equal to 7.5% loss through week 12. Moreover, A clinical decision support system (CDSS) may benefit ascopubs.org/journal/ severe weight loss leads to compromised quality of life, patient care and potentiate a new paradigm of medical cci on March 12, 5-7 10-12 delayed recovery, and unfavorable prognosis. Tox- practice. Although previous models have been 2019: DOI https://doi. icities may become progressively worse after treat- developed and tested with internal and external vali- org/10.1200/CCI.18. 13-15 00058 ment when less monitored; therefore, identification of dation, few studies have assessed clinical utility 1 Cheng et al and effectiveness. In a recent pilot study comparing pre- Experiment system section). In the second evaluation, the dictive accuracy between physicians and models for survival physicians were also provided with the probability of weight and toxicity outcomes after chemoradiation (CRT) in lung loss predicted by the model, both the overall probability of cancer, management models consistently outperformed weight loss and the probability of weight loss associated physicians for all outcomes. This lends early support for the with each category of the predictors, such as tumor loca- potential impact of including robust models into clinical tion, chemotherapy, radiation dosage, and image features practice. of parotid glands, submandibular glands, and larynx (Fig 3; see Experiment system section). At both evaluation time Our group has previously developed a predictive model for points, the physicians were asked to provide predictions of significant weight loss in patients with HNC treated with significant weight loss (yes/no [Y/N]) and the probability of 16,17 concurrent CRT. This study seeks to evaluate the utility the significant weight loss (0 to 1) using an electronic of this weight loss prediction model within a CDSS. We questionnaire (Appendix Fig A1; see Experiment system hypothesize that a CDSS including a weight loss prediction section). Physicians’ predictions were compared with the algorithm for patients with HNC can improve clinicians’ actual weight loss, and accuracy and area under the curve predictions for significant weight loss, thus possibly en- (AUC) were investigated between the two evaluations and abling earlier intervention. the model-only prediction. This study was approved by the institutional review board. METHODS Details of the Study Overview of the Study Procedures Weight loss prediction model using radiomic features. Figure 1 shows an overview of the study procedures, and Clinical and dosimetric data were systematically captured detailed information is in the Experiment system section. A during routine clinical care into our database Onco- prediction model for significant weight loss (loss of greater 18,19 space. The structured database facilitated prospective than or equal to 7.5% of body mass at 3 months post RT) collection of patient clinical assessments and treatment in- was created using clinical, dosimetric, and radiomic pre- formation captured as a part of the routine clinical workflow. dictor data from a training set of 63 patients. An in- Clinical data were routinely captured at the point of care dependent evaluation set of 100 HNC cases was selected during weekly on-treatment evaluations and follow-up for this study. Four physicians were assigned to evaluate visits, including tumor location, type of chemotherapy, each test set case via an electronic platform at two time and clinical assessment of RT-induced adverse effects. For points separated by 30 days: first without and second with this study, 163 patients with HNC who were treated with RT the aid of the CDSS. In both evaluations, a narrative patient between 2009 and 2015 and had measured weight at summary with a treatment plan report including isodose baseline and 3-month visit (61 to 119 days post RT) were lines and dose volume histograms were provided (Fig 2; see included. Dose-volume histograms from these patients’ plans were queried from the departmental database to provide CDSS model development dosimetric information for structures including parotid and Weight loss prediction model was built by using external 63 submandibular glands, larynx, and superior constrictor patients' data. muscles. Planning computed tomography (CT) images and contours were extracted from the picture archiving and communication system. An in-house imaging feature engine First evaluation by four physicians without CDSS calculated the radiomics imaging features from contoured Physician's prediction of weight loss was answered for 100 regions of interest on planning CT images. Radiomics features patients. Information of patient summary with RT plan was provided. included volume, shape, first-order statistics for the distri- bution of intensities, and texture for the parotid and sub- mandibular glands, larynx, and superior constrictor muscles. Two evaluations were separated by 30 days These data were combined and used in the analysis and model development as displayed in Appendix Figure A2. Of the 163 patients included, 100 patients were randomly Second evaluation by the same four physicians with CDSS selected to compose the evaluation dataset. The remaining Prediction by the CDSS was also provided. Physician's prediction of weight loss was answered for the 63 patients were assigned to the training set for the pre- 100 patients again. diction model for significant weight loss, which was per- formed using least absolute shrinkage and selection operator logistic regression (Appendix). Predictive fac- Statistical comparison for the first and second evaluation results tors selected by least absolute shrinkage and selection operator included tumor location, type of chemotherapy, dose to parotid glands, shape features of parotid glands, FIG 1. Overview of study procedures. CDSS, clinical decision support system; RT, radiation therapy. and texture features of parotid glands/submandibular 2 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss All measurements are at planning stage (0-1 week of radiotherapy). Mr. Pt 1 is a 57-year-old white male with recently diagnosed T4 N0 M0 tonsil carcinoma. Currently, he presents with dysphagia and weight loss. His weight is 79.4 kg. We recommend chemoradiation. The tumor volume is 83 cc. Dose Volume Histogram DVH Calculation 1.0 DVH Legend Brain 0.9 PTV58.1_eval 0.8 PTV63_eval 0.7 PTV70_eval 0.6 brainstem 0.5 cord 0.4 cord4mm mandible 0.3 0.2 0.1 0.0 0 1000 2000 3000 4000 5000 6000 7000 80 Dose (cGy) DVH Calculation Dose Volume Histogram 1.0 DVH Legend esophagus 0.9 larynx for edema 0.8 It brachial plexus 0.7 It parotid 0.6 rt brachial plexus 0.5 rt parotid 0.4 0.3 0.2 0.1 0.0 0 1000 2000 3000 4000 5000 6000 7000 80 Dose (cGy) FIG 2. Clinical decision support system case presentation. glands/larynx (Appendix Table A1). The model indicated practice post residency; Drs C and D: junior physician,, 5 years that the imaging features can predict weight loss in com- of practice post residency) as participants in our study. bination with the dosimetric and clinical features. Leave- Experiment system. An experimental system displaying an one-out cross-validation was used to evaluate the AUC of anonymized patient list, a clinical case presentation, the model. The prediction performance of leave-one-out a display of CDSS information, and a questionnaire for cross-validation by the 63 cases had accuracy of 0.78 and physician predictions was developed for evaluation (Figs 2 AUC of 0.81. and 3; Appendix Fig A1). Participating physicians used this system to assess the 100 patients in the evaluation set. Sample size for evaluation. The primary objective was to Physicians were provided with written and video instruction assess if a physician’s prediction of significant weight loss and had a practice session using 11 nonstudy patient cases improves with the support of the CDSS. To properly account to familiarize themselves with the system before the for the correlation between the first and second evaluation evaluation. for each patient, the sample size calculation and primary analysis were performed on the basis of one-sided For the clinical case presentation, the system first provided McNemar test. Assuming that a physician would revise the physicians with a deidentified clinical case narrative his or her prediction after CDSS ≤ 25% of the time, including patient age, sex, race, TNM staging, tumor lo- a sample size of 100 was selected to provide ≥ 80% power cation, pretreatment weight (kilograms), presence of pre- to detect a 15% improvement in rate of correctly predicting treatment dysphagia (Y/N), pretreatment weight loss (Y/N), the clinical outcome. treatment modality (CRT or RT alone), planning target volume, isodose lines, and dose-volume histograms of Participating physicians. Because a physician’s ability to regions of interest and their contours on RT-planning CT accurately predict significant weight loss is likely to be images (example in Fig 2). After presentation of these data, influenced by his or her clinical experience in the manage- the physicians asked to answer the following questions: (1) ment of irradiated HNC, we recruited four physicians of dif- Based on the case presentations, do you believe this patient ferent levels of experience (Drs A and B: senior, . 5 years of will have significant weight loss (defined as ≥ 7.5% weight JCO Clinical Cancer Informatics 3 Norm.Volume Norm.Volume Cheng et al FIG 3. Clinical decision support system display for prediction results from model. loss) as measured at 3 months post-RT to the weight (Fig 3). Physicians were blinded to their previous answers measured at the start of the radiation? Yes/No. (2) Based on as well as to the model performance and the outcome of each the case presentation, what is the probability of significant case. Each physician completed the evaluation independently. weight loss that you would estimate? Estimate (%) as free text (Appendix Fig A2). Statistical Analysis After the physicians’ initial evaluation of all 100 patients was Descriptive statistics were used to summarize patient and completed without the aid of the CDSS, a 30-day wash-out treatment characteristics. The physicians’ evaluations were period was taken. Then, the physicians were asked to compared with actual post-RT weight loss (ground truth) repeat the evaluation for the same 100 patients, this time from patient records. For analysis, binary weight loss (Y/N) with the CDSS results included. The display of CDSS in- was used to calculate accuracy, sensitivity, specificity, formation showed the overall probability of weight loss (0 to 1) positive predictive value (PPV), and negative predictive predicted by the model, its predictors, and the probability of value (NPV), and probability of weight loss (%) was used to weight loss associated with each predictor for the patient calculate AUC. The improvement of physicians’ prediction 4 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss before and after CDSS was evaluated using one-sided TABLE 1. Study Population Characteristics (N = 100) Parameter No. (%) McNemar test for accuracy, and Delong test for AUC, re- spectively. Accuracy is defined as the number of correct Age, mean (SD) 58 (10) predictions (true positive plus true negative) divided by the PTV volume, mL, mean (SD) 130 (116) number of all assessments. Here, true positive represents Sex a correct prediction of weight loss, and true negative Male 79 (79) represents a correct prediction of no weight loss. The Female 21 (21) maximum value of the AUC is 1.0, indicating a perfect prediction model; a value of 0.5 is equivalent to a random Race classifier. Statistical analysis was performed by SAS 9.3 White 74 (74) software, with significance level at the 0.05. African American 20 (20) RESULTS Asian and others 6 (6) Chemotherapy Baseline characteristics for all 100 patients are presented in Table 1. The mean age was 57 years, and the majority of Yes 77 (77) patients were white (75%) and male (79%). For the primary No 23 (23) diagnosis, 32% had an advanced T stage (T3 or T4), and Tumor site 58% had an advanced N stage (N2b or N3). Site of disease Nasal cavity/nasopharynx 14 (14) was 13% nasal cavity/nasopharynx, 48% oropharynx, 10% Oral cavity 9 (9) oral cavity, and 20% hypopharynx/larynx. Most treatment regimens (77%) included chemotherapy. Patient-reported Oropharynx 49 (49) pretreatment dysphagia and weight loss were noted in 14% Hypopharynx/larynx/thyroid 23 (23) and 19% of cases, respectively. In terms of weight mea- Others 5 (5) surements before and after RT, 60% of patients experi- T stage enced significant weight loss of greater than or equal to T1-T2 62 (62) 7.5% of their body mass at 3 months post RT. T3-T4 32 (32) The prediction performance of the prediction model for the Missing 6 (6) 100 test samples used for the CDSS evaluation had accuracy of 0.73 and AUC of 0.75. The sensitivity, specificity, PPV, N stage and NPV were 0.80, 0.63, 0.76, and 0.68, respectively. N0-N2a 36 (36) Figure 4 presents the average performance of participating . N2b 58 (58) physicians with and without CDSS. Before using CDSS, the Missing 6 (6) accuracy, sensitivity, specificity, PPV, NPV, and AUC of M stage physicians’ performance were 0.58, 0.69, 0.42, 0.64, 0.47, M1 2 (2) and 0.56, respectively. With CDSS, physicians’ perfor- M0 92 (92) mance improved in almost every parameter except for Missing 6 (6) specificity; accuracy, sensitivity, specificity, PPV, NPV, and AUC of physicians’ performance were 0.63, 0.84, 0.31, Baseline dysphagia 0.64, 0.56, and 0.69, respectively. Yes 14 (14) Table 2 illustrates the performance by each physician. As No 86 (86) shown, physicians had fair to moderate accuracy and AUC Baseline weight loss for predicting significant weight loss without CDSS. Mean Yes 19 (19) accuracy was 0.58 (ranging from 0.47 to 0.63) without No 81 (81) CDSS and improved to 0.63 (ranging from 0.58 to 0.72) with CDSS. The improvement was not statistically signifi- Abbreviation: PTV, XXXX. cant (P = .06). In terms of AUC, physicians’ predictions improved from 0.56 (ranging from 0.46 to 0.64) to 0.69 DISCUSSION (ranging from 0.63 to 0.73), which was statistically sig- nificant (P , .05). Clinically, this increase in AUC repre- There is an emerging interest in using prediction models sents increased discernment of patients at high versus low derived from machine learning algorithms to aid clinical 14,15,23-25 risk of weight loss with use of the CDSS. More improve- decision making. In this study, we developed and ment was observed among less-experienced physicians validated a prediction model for significant weight loss (Dr C: AUC before, 0.46; AUC after, 0.70; P , .01; Dr D: among patients with HNC receiving RT and then further AUC before, 0.55; AUC after, 0.72; P , .01). Table 2 also evaluated whether the model improved physicians’ pre- shows that sensitivity and specificity vary among physician. dictions for weight loss. In general, our model predicted the JCO Clinical Cancer Informatics 5 Cheng et al by category of predictive factors such as image features, tumor location, chemotherapy, and radiation dosage (Fig 3). 80 Notably, our study showed that the effect of the CDSS varied among the physicians. Even without use of the CDSS, there was appreciable variation in weight loss pre- dictions among the four physicians. Yet the availability of CDSS influenced the decisions made by all physicians, with impact most noticeable among those with less clinical 30 experience. This suggests that junior physicians may benefit more from use of the CDSS. Moreover, in addition to clinical applications, the CDSS may have utility as an ed- ucational tool for simulation training among junior physi- cians and trainees. Indeed, by displaying predictive Accuracy Sensitivity Specificity PPV NPV AUC Physician alone Physician + CDSS dosimetric variables and resultant outcomes (Fig 3), the CDSS may provide a novel means for improving physicians’ FIG 4. Comparison of prediction performance between physician clinical reasoning relative to these parameters. Conversely, alone and physician with clinical decision support system (CDSS). accuracy for the binary classification of patients with/ AUC, area under the curve; NPV, negative predictive value; PPV, without weight loss did not change significantly with the use positive predictive value. of the CDSS. This binary classification is dependent on an unobserved threshold for predicted probability of weight outcome of weight loss more accurately than physicians’ loss held by each physician. In the case of Drs C and D, it is predictions. We also demonstrated that physician’s predictions assumed that the threshold was risk averse and much lower improved with the aid of a CDSS derived from our model, most than the optimal value, leading to high sensitivity and low noticeably among junior physicians. Moreover, the CDSS specificity. Additional training or supplementary educational enabled physicians to better distinguish between patients at tools may be required to address such unobserved thresholds. higher versus lower risk of weight loss, perhaps allowing them Performance of the CDSS according to the various metrics to be targeted for clinical interventions. To our knowledge, this can affect its role in clinical use. For example, variation in is the first published study to evaluate the utility of a CDSS for feeding tube use suggests that CDSS could play a role in weight loss among patients with HNC undergoing RT. facilitating more appropriate and timely use. Reducing its overuse would require the CDSS tool be highly specific, and Several unique features of our model and resultant CDSS may reducing its underuse would require the tool to be highly explain the improvement in predictions that we observed. sensitive. We expect that the tool we developed will only get Specifically, the predictions may be optimized by our in- better in future iterations, but already we believe that with clusion of radiomics features into the model. Radiomics data a specificity and sensitivity of 0.63 and 0.80 it could have are concrete and comprehensive, and their predictive value a role in clinical practices, particularly those that are has been increasingly appreciated in recent years. How- underusing feeding tubes. ever, such data can be complicated and unwieldy, perhaps explaining why radiomics parameters are not yet commonly Our study has the potential for recall bias, because the accepted and adapted into clinical settings. Because pre- same set of patient cases was presented and evaluated sentation of such information in an understandable way is twice. However, a 30 day wash-out period was provided in critical to its optimal use, our CDSS may have provided a means an attempt to decrease this bias. Our study is also limited for physicians to apply valuable radiomics information. In our by a small number of physicians and the use of study, we provided the summarized probability of weight loss single-institution data. Given the limited scope of patient TABLE 2. Comparison Between Without and With CDSS: Accuracy, Area Under the Curve, Sensitivity, and Specificity Accuracy Area Under the Curve Sensitivity Specificity Physician Without CDSS With CDSS P Without CDSS With CDSS P Without CDSS With CDSS Without CDSS With CDSS A 0.47 0.58 .08 0.60 0.63 .58 0.22 0.55 0.85 0.63 B 0.61 0.72 .08 0.64 0.73 .18 0.63 0.82 0.58 0.58 C 0.63 0.59 .25 0.46 0.70 , .01 0.92 0.98 0.2 0 D 0.61 0.61 1.00 0.55 0.72 , .01 0.98 1.00 0.05 0.03 Overall 0.58 0.63 .06 0.56 0.69 , .05 0.69 0.84 0.42 0.31 Abbreviation: CDSS, clinical decision support system 6 © 2019 by American Society of Clinical Oncology Prediction Performance (%) Utility of a Clinical Decision Support System for Weight Loss TABLE 3. Odds Ratios and Regularized β Coefficients Selected Out Using Least the tool may enable physicians to better identify patients Absolute Shrinkage and Selection Operator Logistic Regression with higher versus lower probability of weight loss, evoking 27,28 Regularized β Odds targeted implementation of anti–weight loss strategies. Predictive Factor Coefficients Ratio Because increasing weight loss is correlated with more Tumor location, tongue −0.070 0.93 hospital admissions, higher infection rates, and decreased Tumor location, nasopharynx 1.757 5.80 survival among patients receiving CRT for HNC, such targeted interventions may also improve important clinical Tumor location, nasal cavities −1.342 0.26 outcomes for patients. Given this potential impact, more Tumor location, larynx −1.792 0.17 comprehensive studies of the CDSS, with refined clinical Chemotherapy, cisplatin 0.905 2.47 information, inclusion of more patients and practitioners, Combined parotid d25 0.018 1.02 validation with external data sets, and possibly prospective Combined parotid d30 0.032 1.03 clinical trials are warranted. Combined parotid D80 0.029 1.03 An additional limitation is the simulated nature of the study Parotid fractal dimension 0.160 1.17 design. This was a retrospective evaluation, which limited Parotid GLCM cluster prominence −0.834 0.43 our access to patient characteristics to those that were collected at the point of care, and so we were not able to Parotid GLCM correlation −0.392 0.68 adjust for important factors like comorbidities, social Parotid RLE short emphasis −0.077 0.93 support, family support, insurance status, and the patient’s Parotid RLE long emphasis 0.337 1.40 investment in their own care. In a prospective evaluation, Submandibular GLCM inverse difference −0.128 0.88 we hope to account for these factors. Furthermore, defi- moment nition of significant weight loss for symptom management Submandibular GLCM inverse correlation 0.292 1.34 may vary among clinicians. Other factors, such as prolonged narcotics use, feeding tube dependency, patient’sactivity Submandibular RLE run length uniformity 0.244 1.28 level post treatment, and quality of life in relation to weight Submandibular RLE short/low emphasis −0.012 0.99 loss, may have shaped what is viewed as significant weight Larynx GLCM contrast 2 0.329 1.39 loss. Converting their own thresholds to 7.5% may affect the Intercept −1.643 prediction during the study and also the use of the CDSS in the clinical setting. This also reflects the complexity of Abbreviations: GLCM, Gray-level co-occurrence matrix; RLE, XXXX. medicine and the realistic limitations on our ability to quantify the patient’s condition, as opposed to using the physician’s information provided for review and the simplicity of the impressions when making decisions. study design, it is recognized that the study cannot evaluate the full clinical benefits of the CDSS for weight loss man- In summary, we have described the development of a ro- agement. Instead, the study aimed to develop and describe bust prediction model that can outperform physicians’ how the CDSS might be deployed and to suggest the po- predictions for weight loss in patients with HNC. We have tential for clinical impact on patient care. In particular, provided preliminary evidence that a CDSS that is based on improvement of AUC with use of the CDSS indicates that this model can be used to improve physicians’ predictions. AUTHOR CONTRIBUTIONS AFFILIATIONS Conception and design: Zhi Cheng, Minoru Nakatsugawa, Chen Hu, Ana Johns Hopkins University, Baltimore, MD Kiess, John Haller, Shinya Sugiyama, John Wong, Junghoon Lee, Todd Canon Medical Systems, Otawara, Japan McNutt, Harry Quon Canon Medical Research USA, Vernon Hills, IL Financial support: Zhi Cheng, John Haller, Kazuki Utsunomiya, Shinya Sugiyama, John Wong, Todd McNutt CORRESPONDING AUTHOR Administrative support: Zhi Cheng, John Haller, Shinya Sugiyama, Zhi Cheng, MD, MPH, 401 North Broadway, B141, Baltimore, MD John Wong 21231; e-mail: zcheng4@jhmi.edu. Provision of study material or patients: Zhi Cheng, Ana Kiess, Brandi Page, Todd McNutt, Harry Quon PRIOR PRESENTATION Collection and assembly of data: Zhi Cheng, Stephen Greco, Ana Kiess, Presented in part at the Annual Meeting of the Radiological Society of Brandi R. Page, Sara Alcorn, Junghoon Lee, Todd McNutt, Harry Quon North America, Chicago, IL, November 26-December 1, 2017. Data analysis and interpretation: Zhi Cheng, Minoru Nakatsugawa, Xian Chong Zhou, Chen Hu, Stephen Greco, Ana Kiess, Brandi Page, Sara R. Alcorn, Kazuki Utsunomiya, Wei Fu, Junghoon Lee, Todd McNutt, SUPPORT Harry Quon Supported by Canon Medical Systems Corporation, Radiation Oncology Manuscript writing: All authors Institute, the Commonwealth Foundation, Elekta, Philips Radiation Oncology Systems, and the Johns Hopkins University. Final approval of manuscript: All authors JCO Clinical Cancer Informatics 7 Cheng et al AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST John Haller AND DATA AVAILABILITY STATEMENT Employment: Canon Medical Research USA The following represents disclosure information provided by authors of Research Funding: Canon Medical Research USA this manuscript. All relationships are considered compensated. Patents, Royalties, Other Intellectual Property: Patent: Method and apparatus Relationships are self-held unless noted. I = Immediate Family Member, for determining treatment region and mitigating radiation toxicity Inst = My Institution. Relationships may not relate to the subject matter of Travel, Accommodations, Expenses: Canon Medical Research USA this manuscript.For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/site/ifc. Kazuki Utsunomiya Employment: Canon Medical Systems Zhi Cheng Shinya Sugiyama Research Funding: Toshiba (Inst) Employment: Canon Medical Systems Minoru Nakatsugawa Employment: Canon Medical Systems Junghoon Lee Research Funding: Canon Medical Systems (Inst) Xiang Chong Zhou Todd McNutt Employment: PRA Health Sciences Stock and Other Ownership Interests: Oncospace Stock and Other Ownership Interests: PRA Health Sciences Research Funding: Canon (Inst), Philips Healthcare (Inst) Chen Hu Patents, Royalties, Other Intellectual Property: Radiation dose calculation Consulting or Advisory Role: Merck Sharp & Dohme algorithm Expert Testimony: Elekta Ana Kiess Research Funding: Advanced Accelerator Applications/Novartis (Inst) No other potential conflicts of interest were reported Travel, Accommodations, Expenses: Augmenix ACKNOWLEDGMENT We thank Canon Medical Systems Corporation and Radiation Oncology Institute for the funding provided for this research. REFERENCES 1. Munshi A, Pandey MB, Durga T, et al: Weight loss during radiotherapy for head and neck malignancies: What factors impact it? Nutr Cancer 47:136-140, 2003 2. Pressoir M, Desne S, Berchery D, et al: Prevalence, risk factors and clinical implications of malnutrition in French Comprehensive Cancer Centres. Br J Cancer 102:966-971, 2010 3. Chasen MR, Bhargava R: A descriptive review of the factors contributing to nutritional compromise in patients with head and neck cancer. Support Care Cancer 17:1345-1351, 2009 4. White JV, Guenter P, Jensen G, et al: Consensus statement: Academy of Nutrition and Dietetics and American Society for Parenteral and Enteral Nutrition: characteristics recommended for the identification and documentation of adult malnutrition (undernutrition). JPEN J Parenter Enteral Nutr 36:275-283, 2012 5. Langius JAE, Zandbergen MC, Eerenstein SEJ, et al: Effect of nutritional interventions on nutritional status, quality of life and mortality in patients with head and neck cancer receiving (chemo)radiotherapy: A systematic review. Clin Nutr 32:671-678, 2013 6. Langius JAE, Bakker S, Rietveld DHF, et al: Critical weight loss is a major prognostic indicator for disease-specific survival in patients with head and neck cancer receiving radiotherapy. Br J Cancer 109:1093-1099, 2013 7. Langius JA, van Dijk AM, Doornaert P, et al: More than 10% weight loss in head and neck cancer patients during radiotherapy is independently associated with deterioration in quality of life. Nutr Cancer 65:76-83, 2013 8. Mallick I, Gupta SK, Ray R, et al: Predictors of weight loss during conformal radiotherapy for head and neck cancers - how important are planning target volumes? Clin Oncol (R Coll Radiol) 25:557-563, 2013 9. Langius JAE, Twisk J, Kampman M, et al: Prediction model to predict critical weight loss in patients with head and neck cancer during (chemo)radiotherapy. Oral Oncol 52:91-96, 2016 10. Lambin P, Roelofs E, Reymen B, et al: Rapid Learning health care in oncology - an approach towards decision support systems enabling customised radiotherapy. Radiother Oncol 109:159-164, 2013 11. Lambin P, van Stiphout RGPM, Starmans MHW, et al: Predicting outcomes in radiation oncology--multifactorial decision support systems. Nat Rev Clin Oncol 10:27-40, 2013 12. Lambin P, Zindler J, Vanneste BGL, et al: Decision support systems for personalized and participative radiation oncology. Adv Drug Deliv Rev 109:131-153, 13. Lustberg T, Bailey M, Thwaites DI, et al: Implementation of a rapid learning platform: Predicting 2-year survival in laryngeal carcinoma patients in a clinical setting. Oncotarget 7:37288-37296, 2016 14. Dehing-Oberije C, De Ruysscher D, Petit S, et al: Development, external validation and clinical usefulness of a practical prediction model for radiation-induced dysphagia in lung cancer patients. Radiother Oncol 97:455-461, 2010 15. Oberije C, Nalbantov G, Dekker A, et al: A prospective study comparing the predictions of doctors versus models for treatment outcome of lung cancer patients: A step toward individualized care and shared decision making. Radiother Oncol 112:37-43, 2014 16. Cheng Z, Nakatsugawa M, Hu C, et al: Evaluation of classification and regression tree (CART) model in weight loss prediction following head and neck cancer radiotherapy. Adv Radiat Oncol 3:346-355, 2017 17. Nakatsugawa M, Cheng Z, Goatman KA, et al: Radiomic analysis of salivary glands and its role for predicting xerostomia in irradiated head and neck cancer patients. Int J Radiat Oncol Biol Phys 96:S217, 2016 (suppl) 18. Bowers M, Robertson S, Moore J, et al: SU-E-P-26: Oncospace: A shared radiation oncology database system designed for personalized medicine, decision support, and research. Med Phys 42:3232, 2015 8 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss 19. Robertson SP, Quon H, Kiess AP, et al: A data-mining framework for large scale analysis of dose-outcome relationships in a database of irradiated head and neck cancer patients. Med Phys 42:4329-4337, 2015 20. Aerts HJWL, Velazquez ER, Leijenaar RTH, et al: Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach. Nat Commun 5:4006, 2014 21. Tibshirani R: Regression shrinkage and selection via the lasso: a retrospective. 73:273-282, 2011 22. Rice M, Harris GT: Comparing effect sizes in follow-up studies: ROC Area, Cohen’s d, and r. Law Hum Behav 29:615-620, 2005 23. Mattonen SA, Palma DA, Johnson C, et al: Detection of local cancer recurrence after stereotactic ablative radiation therapy for lung cancer: physician performance versus radiomic assessment. Int J Radiat Oncol Biol Phys 94:1121-1128, 2016 24. Bornstein BH, Emler AC: Rationality in medical decision making: a review of the literature on doctors’ decision-making biases. J Eval Clin Pract 7:97-107, 2001 25. Liu J, Wyatt JC, Altman DG: Decision tools in health care: focus on the problem, not the solution. BMC Med Inform Decis Mak 6:4, 2006 26. Lambin P, Leijenaar RTH, Deist TM, et al: Radiomics: The bridge between medical imaging and personalized medicine. Nat Rev Clin Oncol 14:749-762, 2017 27. Rosenthal DI, Lewin JS, Eisbruch A: Prevention and treatment of dysphagia and aspiration after chemoradiation for head and neck cancer. J Clin Oncol 24:2636-2643, 2006 28. Paccagnella A, Morello M, Da Mosto MC, et al: Early nutritional intervention improves treatment tolerance and outcomes in head and neck cancer patients undergoing concurrent chemoradiotherapy. Support Care Cancer 18:837-845, 2010 29. Capuano G, Grosso A, Gentile PC, et al: Influence of weight loss on outcomes in patients with head and neck cancer undergoing concomitant chemo- radiotherapy. Head Neck 30:503-508, 2008 nn n JCO Clinical Cancer Informatics 9 Cheng et al of the contoured ROIs. The ROIs included were both ipsilateral/ APPENDIX contralateral parotid and submandibular glands. The following types of Development of Prediction Model imaging features were calculated: volume; shape:aspect ratio, con- volutedness and compactness, sphericity, radius, and fractal di- The weight loss prediction model was developed with dosimetric mension; first-order statistics for the distribution of intensities: mean, features, clinical features, and imaging features. The dosimetric fea- median, maximum, minimum, range, interquartile range, quantiles, tures included the DVH for the combined parotid and submandibular standard deviation, skew, and kurtosis; texture measures: Gray-level glands at every 5% volume increments: for example, combo parotid co-occurrence matrix, Gray-level run-length matrix, neighborhood D05, D10, D15, and so on. The clinical features included patient gray-tone difference matrix, and Laws’ filter. Least absolute shrinkage demographics (age, sex), tumor characteristics (TNM stage, location and selection operator logistic regression was applied to predict weight by International Classification of Diseases, 9th revision code), and loss greater than or equal to 7.5% at 3 months post radiation therapy. existence and type of chemotherapy. A total of 5,086 imaging features were calculated by an in-house radiomics calculation engine for each FIG A1. Clinical decision support system display for entering evaluation answer. 10 © 2019 by American Society of Clinical Oncology Utility of a Clinical Decision Support System for Weight Loss Imaging data Dosimetry data Clinical data Diagnosis, assessments Dose distribution Planning CT with contours Informatics platform Imaging feature calculation engine (Oncospace) Radiomics Prediction model Machine learning algorithm of weight loss LASSO logistic regression Imaging features of OARs FIG A2. Development of weight loss prediction model. CT, computed tomography; LASSO, least absolute shrinkage and selection operator; OAR, XXXX. JCO Clinical Cancer Informatics 11

Journal

JCO Clinical Cancer Informatics – Wolters Kluwer Health

Published: Mar 12, 2019

There are no references for this article.