Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Impact on Quality of Documentation and Workload of the Introduction of a National Information Standard for Tumor Board Reporting

Impact on Quality of Documentation and Workload of the Introduction of a National Information... original reports abstract Impact on Quality of Documentation and Workload of the Introduction of a National Information Standard for Tumor Board Reporting 1 1 2 3 Kees C. W. J. Ebben, MSc ; Melle S. Sieswerda, MD, MSc ; Ernest J. T. Luiten, MD, PhD ; Joan B. Heijns, MD ; 4 5,6 6,7 8 Carmen C. van der Pol, MD ; Maud Bessems, MD, PhD ; Aafke H. Honkoop, MD, PhD ; Mathijs P. Hendriks, MD ; 1 1 Janneke Verloop, PhD ; and Xander A. A. M. Verbeek, PhD PURPOSE Tumor boards, clinical practice guidelines, and cancer registries are intertwined cancer care quality instruments. Standardized structured reporting has been proposed as a solution to improve clinical docu- mentation, while facilitating data reuse for secondary purposes. This study describes the implementation and evaluation of a national standard for tumor board reporting for breast cancer on the basis of the clinical practice guideline and the potential for reusing clinical data for the Netherlands Cancer Registry (NCR). METHODS Previously, a national information standard for breast cancer was derived from the corresponding Dutch clinical practice guideline. Using data items from the information standard, we developed three different tumor board forms: preoperative, postoperative, and postneoadjuvant-postoperative. The forms were imple- mented in Amphia Hospital’s electronic health record. Quality of clinical documentation and workload before and after implementation were compared. RESULTS Both draft and final tumor board reports were collected from 27 and 31 patients in baseline and effect measurements, respectively. Completeness of final reports increased from 39.5% to 45.4% (P = .04). The workload for tumor board preparation and discussion did not change significantly. Standardized tumor board reports included 50% (61/122) of the data items carried in the NCR. An automated process was developed to upload information captured in tumor board reports to the NCR database. CONCLUSION This study shows implementation of a national standard for tumor board reports improves quality of clinical documentation, without increasing clinical workload. Simultaneously, our work enables data reuse for secondary purposes like cancer registration. JCO Clin Cancer Inform 4:346-356. © 2020 by American Society of Clinical Oncology Licensed under the Creative Commons Attribution 4.0 License INTRODUCTION form the basis of the tumor board recommendations. 1 2 However, the format of most guidelines is far from ideal Tumor boards, clinical practice guidelines, and 3 for consultation at the point of (multidisciplinary) de- cancer registries are intertwined cancer care quality cision making. instruments. Tumor boards perform two separate tasks. First, they perform a multidisciplinary review of the Cancer registries, such as the Netherlands Cancer patient status, during which data previously reported by Registry (NCR), collect patient data generated during ASSOCIATED ancillary services (eg, radiology or pathology) may be routine care. They are the basis for epidemiologic CONTENT 4-7 outcomes research, results of which are used to aggregated or reinterpreted. For example, a tumor Appendix evaluate and refine the guidelines used in tumor board board may decide that, for a particular case, a tumor Author affiliations decision making. diameter is better approximated on ultrasound than on and support magnetic resonance imaging, which may lead to information (if In the current situation, there are several areas of im- applicable) appear at readjustment of the tumor stage. Subsequently, on the provement: tumor board report quality, clinician the end of this basis of the outcome of the review, the tumor board will workload, and data reuse. First, quality of tumor board article. recommend a course of action. This final recommen- reports varies widely between hospitals, while at the Accepted on February dation, together with any (potentially readjusted) find- same time it is easily understood that proper docu- 24, 2020 and ings that drive it, should be documented in the mentation may directly influence patient outcomes. published at ascopubs.org/journal/ electronic health record (EHR) in a tumor board report. Second, although valuable for patient care, tumor cci on April 23, 2020: Clinical practice guidelines are the embodiment of the board meetings place a burden on physicians’ time. DOI https://doi.org/10. 1200/CCI.19.00050 current status of scientific knowledge and (should) Because of the meeting’s quick pace, preparation is 346 Standardized Structured Tumor Board Reporting CONTEXT Key Objective To develop a national standard for a structured tumor board report for breast cancer and implement this in an electronic health record and work processes in a Dutch hospital. This was followed by a clinical evaluation of the impact on quality of the tumor board report, workload, and interoperability (ie, the degree to which stakeholders are able to electronically exchange information). Knowledge Generated This study shows that standardized structured tumor board reporting can successfully be implemented in complex multi- disciplinary processes. The adherence and implementation of a national standard benefits quality of tumor board reporting but also improves interoperability in cancer care without significant additional workload. Relevance Improving interoperability among stakeholders in health care is an important topic. The development and implementation of standards is seen as a prerequisite for interoperability. This study proposes and evaluates a generalizable approach to standard development that is also applicable beyond breast cancer and even oncology. required, which consists of summarizing patient history, (effect measurement) implementing the national standard clinical findings, and ancillary information. This is complex for breast cancer tumor board forms (forms are defined as work: it requires knowledge and skill to reconstruct a pa- predefined questionnaires). The study design was sub- tient’s medical timeline from progress notes and to de- mitted to a medical ethics committee but was considered termine what is relevant for the upcoming discussion. After exempt from approval. the meeting, it is customary to notify the patient’s general Definition of the National Information Standard practitioner (GP) of the outcome by means of a clinical 11 Previously, we derived a national information standard for letter. This too places a burden on time. breast cancer from the corresponding Dutch clinical Third, cancer registries to date either rely on self-reporting practice guideline. Briefly, the guideline was analyzed and by hospitals or use professionally trained data managers translated into clinical decision trees. In a decision tree, (cancer registrars) to obtain data from medical records. In nodes, branches, and leaves represent data items (patient both cases, this requires significant effort. or disease characteristics; eg, tumor diameter), values or cutoff points (eg, ≤ 5 mm), and guideline recommenda- Standardized structured reporting has been proposed as tions (eg, perform a lumpectomy), respectively. We encoded a solution to support clinicians to produce more complete 13,14 the data items, together with any value sets, using in- and consistent documentation. It entails capturing data ternational standards (eg, SNOMED) where possible. The in discrete fields using (international) terminology systems, resulting list of data items makes up the information standard like Systematized Nomenclature of Medicine (SNOMED). for breast cancer. As a result, reuse of information for secondary purposes is 15,16 facilitated at the same time. The information standard was approved by the EHR standardization workgroup of the National Breast Cancer In this study, we describe the implementation and evalu- Network of the Netherlands, members of which are sur- ation of a national standard for tumor board reporting for geons, physicians, radiation oncologists, radiologists, breast cancer care on the basis of the national guideline pathologists, clinical geneticists, nuclear medicine spe- affected, according to the method described by Hendriks 17 cialists, and nurses with a formal mandate of their re- et al. We investigated the effects of the implementation spective Dutch national associations. It is published online regarding the quality of clinical documentation, with the (in Dutch). associated workload, and reuse of data for the NCR and automatic text generation for GP letters. In addition, we Implementation investigated the effect on additional data entry and changes The study was carried out at Amphia Hospital (Breda, the required during the tumor board meeting and whether Netherlands), an 837-bed hospital treating 380 new breast cancer registrars can play a role in supporting tumor board cancer cases annually. The multidisciplinary team meeting preparation. in the Amphia hospital takes place once a week. There are 5-9 new patients with breast cancer presented every week. METHODS Here, tumor board preparation is performed by nurse Design practitioners and consists of collection and entry of relevant For this study, a before-after design was used. Data col- patient data in a form in the EHR. During the tumor board lection took place before (baseline measurement) and after this form is updated, and after approval its status changes JCO Clinical Cancer Informatics 347 Ebben et al from draft to final. All related clinical documentation within higher values indicating better usability. Questionnaire the EHR is performed by surgical and medical oncologist responses were obtained from the medical professionals departments (ancillary departments have their own in- (n = 4) who were actively involved in tumor board formation systems). preparations. We distinguished three different tumor boards: preoperative, Assessing Quality of (Draft) Tumor Board Reports postoperative, and postneoadjuvant-postoperative. For The goal of a tumor board report is to reflect the outcome of each, the standardized and structured tumor board forms the multidisciplinary case review and communicate the were composed using the data items from the national recommended course of action. As such, it should contain information standard that are relevant for the different all tumor board data items substantiating the final rec- tumor board types (eg, the field “cT” [clinical tumor stage; ommendation. For an individual case, this corresponds to Classification of Malignant Tumors UICC/AJCCe] was in- a set of data items (and their values) that make up a path cluded in the preoperative form where “pT” [pathologic through the decision tree(s) in question. tumor stage; Classification of Malignant Tumours UICC/ This also means that the minimally required set of data AJCC] was included in the postoperative forms; see Ap- varies from case to case. For example, according to the pendix Table A1). guideline for breast cancer, if a patient has metastatic For the purpose of the study, these forms were imple- disease, details about the primary tumor or lymph node mented in the hospital’s EHR (Epic Hyperspace, Verona, involvement are irrelevant when selecting primary treat- WI). EHR functionality was configured for generating full- ment. However, for nonmetastatic disease, these details are text clinical notes from entered form data. The generated required. To complicate matters, there may be multiple clinical notes were subsequently reused in correspondence paths in a tree leading to a single recommendation. As to the patient’s GP. End users were trained to work with the a result, determining which (and consequently how many) standardized forms before introduction into daily clinical data items should have been reported for individual cases practice. becomes difficult or even impossible in case of missing data It should be noted that Amphia already used forms in the (Fig 1). EHR for structured tumor board reporting at baseline. Quality of a tumor board report was therefore operation- However, the previous forms were less comprehensive, not alized as follows. Relevant subsets defined previously associated with any terminology system, and not aligned (see “Definition of the National Information Standard”) with the national information standard and did not generate were considered the gold standard for each type of tumor full-text clinical notes. board. Completeness was defined as the number of data Finally, a data extraction and transformation process items contained in the report divided by the number of data was developed to automatically upload the information items in the relevant subset. Sample size calculations re- captured in the tumor board reports to the NCR garding this primary objective indicated a minimum number database. Electronic messaging used Health Level 7–Fast of 30 reports for demonstrating a statistically significant Healthcare Interoperability Resources (HL7 FHIR, Ann difference. Arbor, MI). Considering it is common that only a few data items are required to complete a path through a decision tree and Data Collection thus determine the appropriate guideline recommendation, Both draft and final tumor board reports, created as part of completeness , 100% is expected and does not indicate routine clinical care, were collected in baseline and effect a low-quality report (Fig 1). Therefore, the measure should measurements. Cancer registrars prepared tumor board not be used in an absolute sense. However, it can be used reports in a sandbox environment (a 1-day-old copy of the to measure changes (eg, in a baseline and intervention production environment) of the EHR. setting). In addition, time required for tumor board preparation (by To additionally measure the impact on quality of tumor nurse practitioners) and tumor board discussion was board preparation, we compared the scored data items of measured by manually clocking each case. Before the the tumor board reports in draft status with their final measurements, we defined start and stop indications per version and compared the scored data items of the drafts task. Likewise, it was decided how to deal with potential prepared by nurse practitioners with those prepared by interruptions. To check consistency in time measurements, cancer registrars. the first batch was measured by two researchers. Assessing Feasibility of Data Reuse for Cancer Registries Finally, a questionnaire, consisting of 25 statements re- garding the usability of the tumor board forms was cre- To assess the potential for reusing tumor board data for ated. The statements were divided over the domains cancer registries, we determined the overlap between the simplicity, clarity, readability, and general impression. data items in the tumor board forms and the data items Each statement was scored on a 4-point scale, with currently registered in the NCR for breast cancer. 348 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting Hypothetical Hypothetical clinical clinical desicion tree desicion tree cM cM1 cM cM1 cM0 cM0 ≥ cN2 cN ≥ cN2 cN cNo cN1 cN0 cN1 cT ≥ cT2 cT cT ≥ cT2 cT ≥ cT2 ≥ cT2 ≤ cT1 ≤ cT1 ≤ cT1 ≤ cT1 Grade Grade 3 Grade Grade 3 Grade 1; Grade 1; Grade 2 Grade 2 Breast- Breast- Breast- Breast- conserving Mastectomy conserving Mastectomy Breast- conserving Mastectomy Breast- conserving Mastectomy therapy AND therapy AND conserving therapy AND conserving therapy AND AND lymph node AND lymph node therapy AND sentinel node therapy AND sentinel Node lymph node dissection lymph node dissection sentinel node sentinel node dissection dissection FIG 1. Example of a hypothetical clinical decision tree (CDT). The nodes (diamonds) represent patient and disease characteristics, the branches represent values of these characteristics, and the leaves at the bottom (rectangles) contain recommendations. Every patient runs through the CDT (top down) on a single, individual path, passing a selection of the characteristics leading to a recommendation. As indicated by the green paths, on the left panel two data items are required to be provided with a recommendation; on the right panel this is four. Statistical Evaluation data items, 113 of which were found in the guideline. The number of data items for preoperative, post- For all statistical analyses, two-sided unpaired t tests were operative, and postneoadjuvant-postoperative tumor used to compare data from before and after implementa- boards forms were 37, 39, and 37, respectively tion. A P value , .05 was considered to be statistically (Table 1). significant. Standardized structured tumor board forms, automatic text RESULTS generation for GP letters, and HL7 FHIR–based message Implementation exchange to the NCR were successfully implemented. The FHIR message definitions can be found online on The national information standard for breast cancer was derived as described in Methods and is composed of 121 simplifier.net. TABLE 1. Overview of the Number of Data Items per Tumor Board Form and Number of Tumor Board Reports Collected No. of Draft Reports No. of Draft and Final Prepared by Data Reports Collected Managers Form No. of Items in Tumor Board Form Baseline Effect Baseline Effect Preoperative 37 14 17 5 14 Postoperative 39 11 8 10 5 Postneoadjuvant-postoperative 37 2614 Total 113 27 31 16 23 NOTE. The draft reports that were subsequently finalized were prepared by nurse practitioners. Draft reports prepared by data managers were not involved in the clinical process. JCO Clinical Cancer Informatics 349 Ebben et al Assessing Quality of Tumor Board Reports Workload. Mean time involved with tumor board prepa- ration by nurse practitioners did not change significantly Clinical documentation. Draft and final tumor board reports (from 4:06 minutes [SD = 1:44] to 4:39 minutes [SD = 1: were collected from 27 and 31 patients in baseline and 59]; P = .28). Time for tumor board discussion per patient effect measurements, respectively (Table 1). Measure- did not change significantly (from 2:19 minutes [SD, 1:27] ments for every patient were performed for 5 subsequent to 2:43 minutes [SD, 1:41]; P = .35; Table 5). weeks in baseline and in effect setting. Completeness of final tumor board reports increased from 39.5% in baseline Assessing Feasibility of Data Reuse for Cancer Registries to 45.4% in effect measurements (P = .04; Table 2). Standardized tumor board reports included 50% (61/122) During the tumor board meeting in baseline on average of the data items carried in the NCR (Appendix Table A1). (14.9 − 13 =) 1.9 data items were added to the report. In End-User Satisfaction the effect measurements this delta was (17 − 16.6 =) 0.4 The mean overall usability score (range, 1-4) was 2.63 in data items (Table 2). This change was not statistically baseline and 2.84 in effect measurement, suggesting an significant. overall improvement. Distributed over the subdomains, the At baseline, when comparing values of individual data mean scores were: simplicity, 2.75 and 2.75; clarity, 2.50 items between the draft and final report, 26 out of 414 and 2.96; readability, 2.71 and 3.00; and general im- (6.3%) were documented with different values. In the effect pression, 2.69 and 3.22 in baseline and effect measure- measurements, different values were recorded in 22 out of ments, respectively. 531 (4.1%) data items (Table 3). This change was not DISCUSSION statistically significant. This study shows that implementation of a national stan- Tumor board preparation by cancer registrars. Cancer dard for tumor board reports improves the quality of clinical registrars prepared 16 and 23 cases in baseline and effect documentation and is possible without increasing the measurements, respectively (Table 1). When comparing clinical workload. At the same time, our work enables data draft tumor board reports prepared by nurse practitioners reuse for secondary purposes like cancer registration. and cancer registrars, no statistically significant difference was found regarding completeness in baseline and effect Potential downsides of structured reporting may include measurements (Table 2). perceived disproportionate burden to physician workload and limitations in reporting freedom. On the basis of the In 16 (draft) tumor board reports in the baseline mea- results on user satisfaction, our study did not corroborate surement, an absolute number of 210 data items were these presumed downsides. scored either by nurse practitioners or by cancer registrars. Out of these, 3 (1.4%) data items were recorded by nurse However, as mentioned in Methods, it should be noted that practitioners only and 11 (5.2%) data items by cancer Amphia already used structured EHR forms for tumor registrars only. A total of 173 (82.4%) data items were board reporting at baseline. Compared with coming from recorded by both with equal values, and 23 (11.0%) were a completely free-text baseline, this could have led to recorded with discordant values. underestimation of the extra effort required for structured reporting but might also have led to underestimation of the Similarly, in the effect measurement, 361 data items were improvement in clinical documentation quality. Our results scored across 23 (draft) tumor board reports. Here, 14 are comparable to those in pathology, where a national (3.9%) data items were only scored by nurse practitioners, standard for structured reporting also improved com- 46 (12.7%) were only scored by cancer registrars, 286 pleteness of clinically relevant data. (79.2%) data items were documented by both with cor- responding values, and 15 (4.2%) data items were The completion rate of approximately 50% may seem low, recorded with disagreeing values (Table 4). but it needs to be interpreted carefully. Indeed, it can be TABLE 2. Mean Number of Data Items Scored With Completeness (in parentheses), in the Draft and Final Tumor Board Reports, for Each Type of Tumor Board Report Draft Final Nurse Practitioner Data Manager Tumor Board Form Baseline Effect Baseline Effect Baseline Effect Preoperative 16 (43) 18.5 (50) 18.4 (50) 18.2 (49) 18.2 (49) 18 (50) Postoperative 9.8 (25) 13.5 (35) 14.3 (37) 15.2 (39) 15.2 (39) 15 (38) Postneoadjuvant- postoperative 10 (27) 15.2 (41) 12 (32) 16 (43) 16 (43) 15.5 (42) Average 13.0 (34.7) 16.6 (44.3) 15.8 (43.1) 17.2 (45.3) 14.9 (39.5) 17.0 (45.4) NOTE. Data are presented as No. (% completeness). 350 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting TABLE 3. Overview of Changes Between Draft (prepared by nurse practitioners) and Final Tumor Board Reports, for Each Type in Baseline and Effect Measurements No. of Items Added No. of Items Changed No. of Items No. of Items Measurement by Tumor Board by Tumor Board Unchanged Total Baseline Preoperative 14 12 195 221 Postoperative 65 14 90 169 Postneoadjuvant-postoperative 12 0 12 24 Total 91 (22.0) 26 (6.3) 297 (71.7) 414 (100) Effect Preoperative 14 11 300 325 Postoperative 13 5 102 120 Postneoadjuvant-postoperative 8 6 72 86 Total 35 (6.6) 22 (4.1) 474 (89.3) 531 (100) NOTE. Data presented as No. or No. (%).The first column shows the number of data items added to the draft by the tumor board. The second and third columns display the number of items that were changed and remained unchanged, respectively. partly explained by certain data items not being filled in in minimize frustration were that no items were considered clinical practice. Yet, as demonstrated by the logic tree in mandatory, taking into account that in clinical practice Figure 1, on a per-case basis this low completion does not cases do occur where information simply is not available. imply that a similar number of data items required for Second, the forms to a degree allow hiding of information guideline-based treatment decisions were missing. that was not required in specific cases. For example, de- tailed information required for a lesion is only shown to the Another reason for lower completion may reside in usability user if a lesion is actually present. issues related to EHR systems. Indeed, there are recent studies showing a relation between physician burnout and With respect to data reuse for cancer registries, approxi- registration burden and EHR usability. The degree to mately 50% of data items defined in tumor board forms are which usability can be taken into account when imple- currently carried in the NCR. The actual degree of reuse menting an information standard is limited by the possi- potential depends on the completeness of the tumor board bilities of the EHR system. Measures that were taken to reports. Despite the improvement in documentation TABLE 4. Comparison of Draft Tumor Board Reports as Prepared by Data Managers and Nurse Practitioners for Each Type of Tumor Board in Baseline and Effect Measurements Scored by Data Managers Scored by Nurse Scored With Different Scored With Corresponding Measurement Only Practitioners Only Values Values Total Baseline Preoperative 4 1 8 61 74 Postoperative 6 0 15 104 125 Postneoadjuvant- 12 0 8 11 postoperative Total 11 (5.2) 3 (1.4) 23 (11.0) 173 (82.4) 210 (100) Effect Preoperative 26 8 11 190 241 Postoperative 10 0 0 56 66 Postneoadjuvant- 10 6 4 40 61 postoperative Total 46 (12.7) 14 (3.9) 15 (4.2) 286 (79.2) 361 (100) NOTE. Data presented as No. or No. (%). The first two columns show the number of items that were documented by either data managers or by nurse practitioners. The third and fourth columns show the number of items they disagreed and agreed on, respectively. JCO Clinical Cancer Informatics 351 Ebben et al TABLE 5. Mean Time (in minutes) Spent per Patient for Tumor Board–Related scoring values that were not part of the official TNM Activities in Baseline and Effect Measurements classification system (eg, cMX). To evaluate the remaining Activity Baseline (SD) Effect (SD) Delta discordant values and get a better understanding of the Tumor board preparation 00:04:06 (1:44) 00:04:39 (1:59) +00:00:33 cancer registrar drafts, additional investigation would be recommended. Tumor board discussion 00:02:19 (1:27) 00:02:43 (1:41) +00:00:24 To evaluate another avenue that might reduce tumor board preparation workload, we estimated the amount of in- quality, we observed an overall low completeness of tumor formation in the tumor board form that is generated by board reports. This may limit ability for data reuse in practice. Low completeness could be partially explained by ancillary services, like radiology and pathology (Appendix not documenting negative findings (eg, not explicitly Table A1). This suggested that 59% of the data items in the documenting “M0” in absence of metastatic disease). Lack forms could be automatically prepopulated if supported by of disciplined use of structured reporting forms by clinicians the underlying technology. Actual benefits, like with reuse is a well-known phenomenon often attributed to poor EHR of data from the tumor board report, depend on the degree usability and the aforementioned (perceived) dispropor- to which ancillary departments report required information. tionate burden and limitations in reporting freedom. Evaluation regarding usability in the effect measurement We investigated the possibility to have cancer registrars showed an improvement over the baseline, although the assist in tumor board preparation. As there were no sig- number of participants in the survey was low. This was nificant differences in completeness and only a limited partially due to the fact that only a limited number of cli- number of discrepancies in scored data items between nicians are actively involved in the information manage- draft tumor board forms prepared by nurse practitioners ment tasks surrounding tumor board meetings. The survey and cancer registrars, the results hint at the possibility of results were consistent with the positive personal feedback using cancer registrars for this purpose. In the absence of we received from these clinicians. a ground truth, we were not able to evaluate discrepancies in documentation between nurse practitioners and cancer EHR implementations of structured reporting are not registrars and establish which party was correct (if any). unique but are usually based on local physician prefer- However, the number of data items scored with different ences. The strength of our approach is that it is based on values decreased from 11% in baseline to 4.1% in effect a national standard that is derived from the national measurements. guideline and enables evaluation of guideline adherence. As such, this study provides a road map for tumor board This change was largely explained by the fact that the (introduced) standardized tumor board forms prevented meetings for tumors other than breast cancer. AFFILIATIONS AUTHOR CONTRIBUTIONS Department of Research and Development, Netherlands Comprehensive Conception and design: Kees C. W. J. Ebben, Melle S. Sieswerda, Joan B. Cancer Organization, Utrecht, the Netherlands Heijns, Aafke H. Honkoop, Xander A. A. M. Verbeek Department of Surgical Oncology, Amphia Hospital, Breda, the Administrative support: Kees C. W. J. Ebben, Aafke H. Honkoop Netherlands Provision of study material or patients: Joan B. Heijns, Carmen C. van der Department of Medical Oncology, Amphia Hospital, Breda, the Pol, Aafke H. Honkoop Netherlands Collection and assembly of data: Kees C. W. J. Ebben, Melle S. Sieswerda, Department of Surgical Oncology, Alrijne Hospital, Leiden, the Carmen C. van der Pol, Maud Bessems, Janneke Verloop, Xander A. A. M. Netherlands Verbeek Department of Surgical Oncology, Jeroen Bosch Hospital, ’s- Data analysis and interpretation: Kees C. W. J. Ebben, Melle S. Sieswerda, Hertogenbosch, the Netherlands Ernest J. T. Luiten, Joan B. Heijns, Carmen C. van der Pol, Mathijs P. National Breast Cancer Network Netherlands (NABON), Utrecht, the Hendriks, Janneke Verloop, Xander A. A. M. Verbeek Netherlands Manuscript writing: All authors Department of Medical Oncology, Isala Hospital, Zwolle, the Final approval of manuscript: All authors Netherlands Accountable for all aspects of the work: All authors Department of Medical Oncology, Northwest Clinics, Alkmaar, the Netherlands AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST CORRESPONDING AUTHOR The following represents disclosure information provided by authors of Kees C. W. J. Ebben, MSc, Godebaldkwartier 419, 3511 DT Utrecht, the this manuscript. All relationships are considered compensated unless Netherlands; e-mail: k.ebben@iknl.nl. otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO’s EQUAL CONTRIBUTION conflict of interest policy, please refer to www.asco.org/rwc or ascopubs. K.C.W.J.E. and M.S.S. contributed equally to this work. org/cci/author-center. 352 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting Open Payments is a public database containing information reported by Joan B. Heijns companies about payments made to US-licensed physicians (Open Travel, Accommodations, Expenses: Pfizer, Tesaro Payments). Aafke H. Honkoop Melle S. Sieswerda Consulting or Advisory Role: Eli Lilly Employment: United Neuroscience (I) Mathijs P. Hendriks Ernest J. T. Luiten Consulting or Advisory Role: MSD Consulting or Advisory Role: Agendia No other potential conflicts of interest were reported. Speakers’ Bureau: Agendia REFERENCES 1. Lamb BW, Green JSA, Benn J, et al: Improving decision making in multidisciplinary tumor boards: Prospective longitudinal evaluation of a multicomponent intervention for 1,421 patients. J Am Coll Surg 217:412-420, 2013 2. Institute of Medicine: Clinical Practice Guidelines We Can Trust. Washington, DC, National Acadamies Press, 2011 3. Parkin DM: The role of cancer registries in cancer control. Int J Clin Oncol 13:102-111, 2008 4. Newman EA, Guest AB, Helvie MA, et al: Changes in surgical management resulting from case review at a breast cancer multidisciplinary tumor board. Cancer 107:2346-2351, 2006 5. Thenappan A, Halaweish I, Mody RJ, et al: Review at a multidisciplinary tumor board impacts critical management decisions of pediatric patients with cancer. Pediatr Blood Cancer 64:254-258, 2017 6. van Hagen P, Spaander MCW, van der Gaast A, et al: Impact of a multidisciplinary tumour board meeting for upper-GI malignancies on clinical decision making: A prospective cohort study. Int J Clin Oncol 18:214-219, 2013 7. Wheless SA, McKinney KA, Zanation AM: A prospective study of the clinical impact of a multidisciplinary head and neck tumor board. Otolaryngol Head Neck Surg 143:650-654, 2010 8. Pillay B, Wootten AC, Crowe H, et al: The impact of multidisciplinary team meetings on patient assessment, management and outcomes in oncology settings: A systematic review of the literature. Cancer Treat Rev 42:56-72, 2016 9. Peleg M, Tu S, Bury J, et al: Comparing computer-interpretable guideline models: A case-study approach. J Am Med Inform Assoc 10:52-68, 2003 [Erratum: J Am Med Inform Assoc 20:801] 10. Darko-Yawson S, Ellingsen G: Assessing and improving EHRs data quality through a socio-technical approach. Procedia Comput Sci 98:243-250, 2016 11. El Saghir NS, Keating NL, Carlson RW, et al: Tumor boards: Optimizing the structure and improving efficiency of multidisciplinary management of patients with cancer worldwide. Am Soc Clin Oncol Ed Book e461-e466, 2014 12. Schouten LJ, Jager JJ, van den Brandt PA: Quality of cancer registry data: A comparison of data provided by clinicians with those of registration personnel. Br J Cancer 68:974-977, 1993 13. Farrugia DJ, Fischer TD, Delitto D, et al: Improved breast cancer care quality metrics after implementation of a standardized tumor board documentation template. J Oncol Pract 11:421-423, 2015 14. Woods YL, Mukhtar S, McClements P, et al: A survey of reporting of colorectal cancer in Scotland: Compliance with guidelines and effect of proforma reporting. J Clin Pathol 67:499-505, 2014 15. Renshaw AA, Mena-Allauca M, Gould EW, et al: Synoptic reporting: Evidence-based review and future directions. JCO Clin Cancer Inform 10.1200/CCI.17. 16. Sluijter CE, van Lonkhuijzen LRCW, van Slooten H-J, et al: The effects of implementing synoptic pathology reporting in cancer diagnosis: A systematic review. Virchows Arch 468:639-649, 2016 17. Hendriks MP, Verbeek XAAM, van Vegchel T, et al: Transformation of the National Breast Cancer guideline into data-driven clinical decision trees. JCO Clin Cancer Inform 10.1200/CCI.18.00150 18. NABON: Information Standard Breast Cancer. https://decor.nictiz.nl/art-decor/decor-project–onco-mamma- 19. NABON: Breast Cancer Tumor Board questionnaires. Simplefier.net, 2017. https://simplifier.net/breastcancertumorboardquestionnaires/~resources 20. Swillens JEM, Sluijter CE, Overbeek LIH, et al: Identification of barriers and facilitators in nationwide implementation of standardized structured reporting in pathology: A mixed method study. Virchows Arch 475:551-561, 2019 21. Brown T, Bergman S: Doctors, nurses and the paperwork crises that could unite them. New York Times, 2019. https://www.nytimes.com/2019/12/31/opinion/ doctors-nurses-and-the-paperwork-crisis-that-could-unite-them.html 22. Gawande A: Why doctors hate their computers. New Yorker, 2018. https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers nn n JCO Clinical Cancer Informatics 353 Ebben et al (14.2%), pathologic 51 (45.1%). For 22 data items (19.5%), the tumor APPENDIX board itself is the primary source. Standardized tumor board reports Distribution of Data Items in the Information Standard included 50% (61/122) of data items also registered retrospectively in the NCR (Appendix Table A1). The distribution of tumor board report data items by their primary source is: administrative 12 (10.6%), clinical 12 (10.6%), radiologic 16 354 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting JCO Clinical Cancer Informatics 355 TABLE A1. The Implemented TBR Forms, Their Included Data Items, and Their Presence in the NCR and Primary Source Reports Data items (population/disease characteristics and workflow data) Preoperative TBR Postoperative TBR Postneoadjuvant-Postoperative TBR NCR Primary Source Reports Tumor board date X X X X Administrative Tumor board report approval date X X X Administrative Present specialists X X X Administrative Clinical question X X X Administrative Pregnancy X X Clinical Disease laterality X X X X Clinical Disease localization X X X X Clinical BI-RADS X X Radiology Calcifications in breast X Radiology Solid mass in breast X Radiology Tumor laterality X X Radiology Tumor localization X X Radiology cT diameter X X Radiology cT4 characteristics X Radiology Morphology X X X X Pathology Grade X X X X Pathology cT X X TBR Tumor distribution X X Radiology Solid component on mammography X Radiology ER percentage X X X X Pathology HER2 status X X X X Pathology Lymph node laterality X X X Radiology Fixed axillary lymph nodes X Radiology Clinical positive axillary lymph nodes X Radiology Clinical positive parasternal lymph nodes X X X Radiology Clinical positive infraclavicular lymph nodes X Radiology Clinical positive supraclavicular lymph nodes X Radiology cN X X TBR Localization metastasis X X Radiology ER percentage metastases X Pathology HER2 status metastases X Pathology cM X X TBR cTNM X X TBR Clinical study advice X X X TBR Clinical study X X X X TBR (Continued on following page) Ebben et al 356 © 2020 by American Society of Clinical Oncology TABLE A1. The Implemented TBR Forms, Their Included Data Items, and Their Presence in the NCR and Primary Source Reports (Continued) Data items (population/disease characteristics and workflow data) Preoperative TBR Postoperative TBR Postneoadjuvant-Postoperative TBR NCR Primary Source Reports Policy X X X X TBR Motivation for CPG deviation X X X TBR pT diameter X X Pathology pT4 characteristics X Pathology Cutting edge X X X Pathology Angio invasion X X X Pathology pT X X Pathology Positive supraclavicular lymph nodes in SN X X Pathology Positive parasternal lymph nodes in SN X X Pathology Positive infraclavicular lymph nodes X X Pathology No. of axillary lymph nodes with micro- or macrometastases in SN X X X Pathology No. of axillary lymph nodes with macrometastases in SN X X X Pathology No. of axillary lymph nodes with micrometastases in SN X X X Pathology No. of axillary lymph nodes with isolated tumor cells in SN X X X Pathology Extranodal growth in SN X X X Pathology Extranodal growth in lymph node dissection X X X Pathology Positive axillary top lymph node X X X Pathology pN X X Pathology pM X X Pathology pTNM X X TBR Genetic mutation X X X Pathology N0 risk status X Pathology First re-excision X X X TBR Lymph node surgery X X X TBR ypT 4 diameter X X Pathology ypT 4 characteristics X Pathology Pathologic response primary tumor X X Pathology ypT X X Pathology ypN X X Pathology ypM X X Pathology ypTNM X X TBR Total 37 39 37 61 Abbreviations: BI-RADS, Breast Imaging-Reporting and Data System; CPG, clinical practice guideline; ER, estrogen receptor; NCR, Netherlands Cancer Registry; SN, sentinel node; TBR, tumor board report. Several NCR data items are registered multiple times throughout the clinical pathway. Additional data items (not CPG) identified by care professionals to optimally support the tumor board processes. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png JCO Clinical Cancer Informatics Wolters Kluwer Health

Impact on Quality of Documentation and Workload of the Introduction of a National Information Standard for Tumor Board Reporting

Loading next page...
 
/lp/wolters-kluwer-health/impact-on-quality-of-documentation-and-workload-of-the-introduction-of-oxnqqAxxxV

References (52)

Publisher
Wolters Kluwer Health
Copyright
(C) 2020 American Society of Clinical Oncology
ISSN
2473-4276
DOI
10.1200/CCI.19.00050
Publisher site
See Article on Publisher Site

Abstract

original reports abstract Impact on Quality of Documentation and Workload of the Introduction of a National Information Standard for Tumor Board Reporting 1 1 2 3 Kees C. W. J. Ebben, MSc ; Melle S. Sieswerda, MD, MSc ; Ernest J. T. Luiten, MD, PhD ; Joan B. Heijns, MD ; 4 5,6 6,7 8 Carmen C. van der Pol, MD ; Maud Bessems, MD, PhD ; Aafke H. Honkoop, MD, PhD ; Mathijs P. Hendriks, MD ; 1 1 Janneke Verloop, PhD ; and Xander A. A. M. Verbeek, PhD PURPOSE Tumor boards, clinical practice guidelines, and cancer registries are intertwined cancer care quality instruments. Standardized structured reporting has been proposed as a solution to improve clinical docu- mentation, while facilitating data reuse for secondary purposes. This study describes the implementation and evaluation of a national standard for tumor board reporting for breast cancer on the basis of the clinical practice guideline and the potential for reusing clinical data for the Netherlands Cancer Registry (NCR). METHODS Previously, a national information standard for breast cancer was derived from the corresponding Dutch clinical practice guideline. Using data items from the information standard, we developed three different tumor board forms: preoperative, postoperative, and postneoadjuvant-postoperative. The forms were imple- mented in Amphia Hospital’s electronic health record. Quality of clinical documentation and workload before and after implementation were compared. RESULTS Both draft and final tumor board reports were collected from 27 and 31 patients in baseline and effect measurements, respectively. Completeness of final reports increased from 39.5% to 45.4% (P = .04). The workload for tumor board preparation and discussion did not change significantly. Standardized tumor board reports included 50% (61/122) of the data items carried in the NCR. An automated process was developed to upload information captured in tumor board reports to the NCR database. CONCLUSION This study shows implementation of a national standard for tumor board reports improves quality of clinical documentation, without increasing clinical workload. Simultaneously, our work enables data reuse for secondary purposes like cancer registration. JCO Clin Cancer Inform 4:346-356. © 2020 by American Society of Clinical Oncology Licensed under the Creative Commons Attribution 4.0 License INTRODUCTION form the basis of the tumor board recommendations. 1 2 However, the format of most guidelines is far from ideal Tumor boards, clinical practice guidelines, and 3 for consultation at the point of (multidisciplinary) de- cancer registries are intertwined cancer care quality cision making. instruments. Tumor boards perform two separate tasks. First, they perform a multidisciplinary review of the Cancer registries, such as the Netherlands Cancer patient status, during which data previously reported by Registry (NCR), collect patient data generated during ASSOCIATED ancillary services (eg, radiology or pathology) may be routine care. They are the basis for epidemiologic CONTENT 4-7 outcomes research, results of which are used to aggregated or reinterpreted. For example, a tumor Appendix evaluate and refine the guidelines used in tumor board board may decide that, for a particular case, a tumor Author affiliations decision making. diameter is better approximated on ultrasound than on and support magnetic resonance imaging, which may lead to information (if In the current situation, there are several areas of im- applicable) appear at readjustment of the tumor stage. Subsequently, on the provement: tumor board report quality, clinician the end of this basis of the outcome of the review, the tumor board will workload, and data reuse. First, quality of tumor board article. recommend a course of action. This final recommen- reports varies widely between hospitals, while at the Accepted on February dation, together with any (potentially readjusted) find- same time it is easily understood that proper docu- 24, 2020 and ings that drive it, should be documented in the mentation may directly influence patient outcomes. published at ascopubs.org/journal/ electronic health record (EHR) in a tumor board report. Second, although valuable for patient care, tumor cci on April 23, 2020: Clinical practice guidelines are the embodiment of the board meetings place a burden on physicians’ time. DOI https://doi.org/10. 1200/CCI.19.00050 current status of scientific knowledge and (should) Because of the meeting’s quick pace, preparation is 346 Standardized Structured Tumor Board Reporting CONTEXT Key Objective To develop a national standard for a structured tumor board report for breast cancer and implement this in an electronic health record and work processes in a Dutch hospital. This was followed by a clinical evaluation of the impact on quality of the tumor board report, workload, and interoperability (ie, the degree to which stakeholders are able to electronically exchange information). Knowledge Generated This study shows that standardized structured tumor board reporting can successfully be implemented in complex multi- disciplinary processes. The adherence and implementation of a national standard benefits quality of tumor board reporting but also improves interoperability in cancer care without significant additional workload. Relevance Improving interoperability among stakeholders in health care is an important topic. The development and implementation of standards is seen as a prerequisite for interoperability. This study proposes and evaluates a generalizable approach to standard development that is also applicable beyond breast cancer and even oncology. required, which consists of summarizing patient history, (effect measurement) implementing the national standard clinical findings, and ancillary information. This is complex for breast cancer tumor board forms (forms are defined as work: it requires knowledge and skill to reconstruct a pa- predefined questionnaires). The study design was sub- tient’s medical timeline from progress notes and to de- mitted to a medical ethics committee but was considered termine what is relevant for the upcoming discussion. After exempt from approval. the meeting, it is customary to notify the patient’s general Definition of the National Information Standard practitioner (GP) of the outcome by means of a clinical 11 Previously, we derived a national information standard for letter. This too places a burden on time. breast cancer from the corresponding Dutch clinical Third, cancer registries to date either rely on self-reporting practice guideline. Briefly, the guideline was analyzed and by hospitals or use professionally trained data managers translated into clinical decision trees. In a decision tree, (cancer registrars) to obtain data from medical records. In nodes, branches, and leaves represent data items (patient both cases, this requires significant effort. or disease characteristics; eg, tumor diameter), values or cutoff points (eg, ≤ 5 mm), and guideline recommenda- Standardized structured reporting has been proposed as tions (eg, perform a lumpectomy), respectively. We encoded a solution to support clinicians to produce more complete 13,14 the data items, together with any value sets, using in- and consistent documentation. It entails capturing data ternational standards (eg, SNOMED) where possible. The in discrete fields using (international) terminology systems, resulting list of data items makes up the information standard like Systematized Nomenclature of Medicine (SNOMED). for breast cancer. As a result, reuse of information for secondary purposes is 15,16 facilitated at the same time. The information standard was approved by the EHR standardization workgroup of the National Breast Cancer In this study, we describe the implementation and evalu- Network of the Netherlands, members of which are sur- ation of a national standard for tumor board reporting for geons, physicians, radiation oncologists, radiologists, breast cancer care on the basis of the national guideline pathologists, clinical geneticists, nuclear medicine spe- affected, according to the method described by Hendriks 17 cialists, and nurses with a formal mandate of their re- et al. We investigated the effects of the implementation spective Dutch national associations. It is published online regarding the quality of clinical documentation, with the (in Dutch). associated workload, and reuse of data for the NCR and automatic text generation for GP letters. In addition, we Implementation investigated the effect on additional data entry and changes The study was carried out at Amphia Hospital (Breda, the required during the tumor board meeting and whether Netherlands), an 837-bed hospital treating 380 new breast cancer registrars can play a role in supporting tumor board cancer cases annually. The multidisciplinary team meeting preparation. in the Amphia hospital takes place once a week. There are 5-9 new patients with breast cancer presented every week. METHODS Here, tumor board preparation is performed by nurse Design practitioners and consists of collection and entry of relevant For this study, a before-after design was used. Data col- patient data in a form in the EHR. During the tumor board lection took place before (baseline measurement) and after this form is updated, and after approval its status changes JCO Clinical Cancer Informatics 347 Ebben et al from draft to final. All related clinical documentation within higher values indicating better usability. Questionnaire the EHR is performed by surgical and medical oncologist responses were obtained from the medical professionals departments (ancillary departments have their own in- (n = 4) who were actively involved in tumor board formation systems). preparations. We distinguished three different tumor boards: preoperative, Assessing Quality of (Draft) Tumor Board Reports postoperative, and postneoadjuvant-postoperative. For The goal of a tumor board report is to reflect the outcome of each, the standardized and structured tumor board forms the multidisciplinary case review and communicate the were composed using the data items from the national recommended course of action. As such, it should contain information standard that are relevant for the different all tumor board data items substantiating the final rec- tumor board types (eg, the field “cT” [clinical tumor stage; ommendation. For an individual case, this corresponds to Classification of Malignant Tumors UICC/AJCCe] was in- a set of data items (and their values) that make up a path cluded in the preoperative form where “pT” [pathologic through the decision tree(s) in question. tumor stage; Classification of Malignant Tumours UICC/ This also means that the minimally required set of data AJCC] was included in the postoperative forms; see Ap- varies from case to case. For example, according to the pendix Table A1). guideline for breast cancer, if a patient has metastatic For the purpose of the study, these forms were imple- disease, details about the primary tumor or lymph node mented in the hospital’s EHR (Epic Hyperspace, Verona, involvement are irrelevant when selecting primary treat- WI). EHR functionality was configured for generating full- ment. However, for nonmetastatic disease, these details are text clinical notes from entered form data. The generated required. To complicate matters, there may be multiple clinical notes were subsequently reused in correspondence paths in a tree leading to a single recommendation. As to the patient’s GP. End users were trained to work with the a result, determining which (and consequently how many) standardized forms before introduction into daily clinical data items should have been reported for individual cases practice. becomes difficult or even impossible in case of missing data It should be noted that Amphia already used forms in the (Fig 1). EHR for structured tumor board reporting at baseline. Quality of a tumor board report was therefore operation- However, the previous forms were less comprehensive, not alized as follows. Relevant subsets defined previously associated with any terminology system, and not aligned (see “Definition of the National Information Standard”) with the national information standard and did not generate were considered the gold standard for each type of tumor full-text clinical notes. board. Completeness was defined as the number of data Finally, a data extraction and transformation process items contained in the report divided by the number of data was developed to automatically upload the information items in the relevant subset. Sample size calculations re- captured in the tumor board reports to the NCR garding this primary objective indicated a minimum number database. Electronic messaging used Health Level 7–Fast of 30 reports for demonstrating a statistically significant Healthcare Interoperability Resources (HL7 FHIR, Ann difference. Arbor, MI). Considering it is common that only a few data items are required to complete a path through a decision tree and Data Collection thus determine the appropriate guideline recommendation, Both draft and final tumor board reports, created as part of completeness , 100% is expected and does not indicate routine clinical care, were collected in baseline and effect a low-quality report (Fig 1). Therefore, the measure should measurements. Cancer registrars prepared tumor board not be used in an absolute sense. However, it can be used reports in a sandbox environment (a 1-day-old copy of the to measure changes (eg, in a baseline and intervention production environment) of the EHR. setting). In addition, time required for tumor board preparation (by To additionally measure the impact on quality of tumor nurse practitioners) and tumor board discussion was board preparation, we compared the scored data items of measured by manually clocking each case. Before the the tumor board reports in draft status with their final measurements, we defined start and stop indications per version and compared the scored data items of the drafts task. Likewise, it was decided how to deal with potential prepared by nurse practitioners with those prepared by interruptions. To check consistency in time measurements, cancer registrars. the first batch was measured by two researchers. Assessing Feasibility of Data Reuse for Cancer Registries Finally, a questionnaire, consisting of 25 statements re- garding the usability of the tumor board forms was cre- To assess the potential for reusing tumor board data for ated. The statements were divided over the domains cancer registries, we determined the overlap between the simplicity, clarity, readability, and general impression. data items in the tumor board forms and the data items Each statement was scored on a 4-point scale, with currently registered in the NCR for breast cancer. 348 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting Hypothetical Hypothetical clinical clinical desicion tree desicion tree cM cM1 cM cM1 cM0 cM0 ≥ cN2 cN ≥ cN2 cN cNo cN1 cN0 cN1 cT ≥ cT2 cT cT ≥ cT2 cT ≥ cT2 ≥ cT2 ≤ cT1 ≤ cT1 ≤ cT1 ≤ cT1 Grade Grade 3 Grade Grade 3 Grade 1; Grade 1; Grade 2 Grade 2 Breast- Breast- Breast- Breast- conserving Mastectomy conserving Mastectomy Breast- conserving Mastectomy Breast- conserving Mastectomy therapy AND therapy AND conserving therapy AND conserving therapy AND AND lymph node AND lymph node therapy AND sentinel node therapy AND sentinel Node lymph node dissection lymph node dissection sentinel node sentinel node dissection dissection FIG 1. Example of a hypothetical clinical decision tree (CDT). The nodes (diamonds) represent patient and disease characteristics, the branches represent values of these characteristics, and the leaves at the bottom (rectangles) contain recommendations. Every patient runs through the CDT (top down) on a single, individual path, passing a selection of the characteristics leading to a recommendation. As indicated by the green paths, on the left panel two data items are required to be provided with a recommendation; on the right panel this is four. Statistical Evaluation data items, 113 of which were found in the guideline. The number of data items for preoperative, post- For all statistical analyses, two-sided unpaired t tests were operative, and postneoadjuvant-postoperative tumor used to compare data from before and after implementa- boards forms were 37, 39, and 37, respectively tion. A P value , .05 was considered to be statistically (Table 1). significant. Standardized structured tumor board forms, automatic text RESULTS generation for GP letters, and HL7 FHIR–based message Implementation exchange to the NCR were successfully implemented. The FHIR message definitions can be found online on The national information standard for breast cancer was derived as described in Methods and is composed of 121 simplifier.net. TABLE 1. Overview of the Number of Data Items per Tumor Board Form and Number of Tumor Board Reports Collected No. of Draft Reports No. of Draft and Final Prepared by Data Reports Collected Managers Form No. of Items in Tumor Board Form Baseline Effect Baseline Effect Preoperative 37 14 17 5 14 Postoperative 39 11 8 10 5 Postneoadjuvant-postoperative 37 2614 Total 113 27 31 16 23 NOTE. The draft reports that were subsequently finalized were prepared by nurse practitioners. Draft reports prepared by data managers were not involved in the clinical process. JCO Clinical Cancer Informatics 349 Ebben et al Assessing Quality of Tumor Board Reports Workload. Mean time involved with tumor board prepa- ration by nurse practitioners did not change significantly Clinical documentation. Draft and final tumor board reports (from 4:06 minutes [SD = 1:44] to 4:39 minutes [SD = 1: were collected from 27 and 31 patients in baseline and 59]; P = .28). Time for tumor board discussion per patient effect measurements, respectively (Table 1). Measure- did not change significantly (from 2:19 minutes [SD, 1:27] ments for every patient were performed for 5 subsequent to 2:43 minutes [SD, 1:41]; P = .35; Table 5). weeks in baseline and in effect setting. Completeness of final tumor board reports increased from 39.5% in baseline Assessing Feasibility of Data Reuse for Cancer Registries to 45.4% in effect measurements (P = .04; Table 2). Standardized tumor board reports included 50% (61/122) During the tumor board meeting in baseline on average of the data items carried in the NCR (Appendix Table A1). (14.9 − 13 =) 1.9 data items were added to the report. In End-User Satisfaction the effect measurements this delta was (17 − 16.6 =) 0.4 The mean overall usability score (range, 1-4) was 2.63 in data items (Table 2). This change was not statistically baseline and 2.84 in effect measurement, suggesting an significant. overall improvement. Distributed over the subdomains, the At baseline, when comparing values of individual data mean scores were: simplicity, 2.75 and 2.75; clarity, 2.50 items between the draft and final report, 26 out of 414 and 2.96; readability, 2.71 and 3.00; and general im- (6.3%) were documented with different values. In the effect pression, 2.69 and 3.22 in baseline and effect measure- measurements, different values were recorded in 22 out of ments, respectively. 531 (4.1%) data items (Table 3). This change was not DISCUSSION statistically significant. This study shows that implementation of a national stan- Tumor board preparation by cancer registrars. Cancer dard for tumor board reports improves the quality of clinical registrars prepared 16 and 23 cases in baseline and effect documentation and is possible without increasing the measurements, respectively (Table 1). When comparing clinical workload. At the same time, our work enables data draft tumor board reports prepared by nurse practitioners reuse for secondary purposes like cancer registration. and cancer registrars, no statistically significant difference was found regarding completeness in baseline and effect Potential downsides of structured reporting may include measurements (Table 2). perceived disproportionate burden to physician workload and limitations in reporting freedom. On the basis of the In 16 (draft) tumor board reports in the baseline mea- results on user satisfaction, our study did not corroborate surement, an absolute number of 210 data items were these presumed downsides. scored either by nurse practitioners or by cancer registrars. Out of these, 3 (1.4%) data items were recorded by nurse However, as mentioned in Methods, it should be noted that practitioners only and 11 (5.2%) data items by cancer Amphia already used structured EHR forms for tumor registrars only. A total of 173 (82.4%) data items were board reporting at baseline. Compared with coming from recorded by both with equal values, and 23 (11.0%) were a completely free-text baseline, this could have led to recorded with discordant values. underestimation of the extra effort required for structured reporting but might also have led to underestimation of the Similarly, in the effect measurement, 361 data items were improvement in clinical documentation quality. Our results scored across 23 (draft) tumor board reports. Here, 14 are comparable to those in pathology, where a national (3.9%) data items were only scored by nurse practitioners, standard for structured reporting also improved com- 46 (12.7%) were only scored by cancer registrars, 286 pleteness of clinically relevant data. (79.2%) data items were documented by both with cor- responding values, and 15 (4.2%) data items were The completion rate of approximately 50% may seem low, recorded with disagreeing values (Table 4). but it needs to be interpreted carefully. Indeed, it can be TABLE 2. Mean Number of Data Items Scored With Completeness (in parentheses), in the Draft and Final Tumor Board Reports, for Each Type of Tumor Board Report Draft Final Nurse Practitioner Data Manager Tumor Board Form Baseline Effect Baseline Effect Baseline Effect Preoperative 16 (43) 18.5 (50) 18.4 (50) 18.2 (49) 18.2 (49) 18 (50) Postoperative 9.8 (25) 13.5 (35) 14.3 (37) 15.2 (39) 15.2 (39) 15 (38) Postneoadjuvant- postoperative 10 (27) 15.2 (41) 12 (32) 16 (43) 16 (43) 15.5 (42) Average 13.0 (34.7) 16.6 (44.3) 15.8 (43.1) 17.2 (45.3) 14.9 (39.5) 17.0 (45.4) NOTE. Data are presented as No. (% completeness). 350 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting TABLE 3. Overview of Changes Between Draft (prepared by nurse practitioners) and Final Tumor Board Reports, for Each Type in Baseline and Effect Measurements No. of Items Added No. of Items Changed No. of Items No. of Items Measurement by Tumor Board by Tumor Board Unchanged Total Baseline Preoperative 14 12 195 221 Postoperative 65 14 90 169 Postneoadjuvant-postoperative 12 0 12 24 Total 91 (22.0) 26 (6.3) 297 (71.7) 414 (100) Effect Preoperative 14 11 300 325 Postoperative 13 5 102 120 Postneoadjuvant-postoperative 8 6 72 86 Total 35 (6.6) 22 (4.1) 474 (89.3) 531 (100) NOTE. Data presented as No. or No. (%).The first column shows the number of data items added to the draft by the tumor board. The second and third columns display the number of items that were changed and remained unchanged, respectively. partly explained by certain data items not being filled in in minimize frustration were that no items were considered clinical practice. Yet, as demonstrated by the logic tree in mandatory, taking into account that in clinical practice Figure 1, on a per-case basis this low completion does not cases do occur where information simply is not available. imply that a similar number of data items required for Second, the forms to a degree allow hiding of information guideline-based treatment decisions were missing. that was not required in specific cases. For example, de- tailed information required for a lesion is only shown to the Another reason for lower completion may reside in usability user if a lesion is actually present. issues related to EHR systems. Indeed, there are recent studies showing a relation between physician burnout and With respect to data reuse for cancer registries, approxi- registration burden and EHR usability. The degree to mately 50% of data items defined in tumor board forms are which usability can be taken into account when imple- currently carried in the NCR. The actual degree of reuse menting an information standard is limited by the possi- potential depends on the completeness of the tumor board bilities of the EHR system. Measures that were taken to reports. Despite the improvement in documentation TABLE 4. Comparison of Draft Tumor Board Reports as Prepared by Data Managers and Nurse Practitioners for Each Type of Tumor Board in Baseline and Effect Measurements Scored by Data Managers Scored by Nurse Scored With Different Scored With Corresponding Measurement Only Practitioners Only Values Values Total Baseline Preoperative 4 1 8 61 74 Postoperative 6 0 15 104 125 Postneoadjuvant- 12 0 8 11 postoperative Total 11 (5.2) 3 (1.4) 23 (11.0) 173 (82.4) 210 (100) Effect Preoperative 26 8 11 190 241 Postoperative 10 0 0 56 66 Postneoadjuvant- 10 6 4 40 61 postoperative Total 46 (12.7) 14 (3.9) 15 (4.2) 286 (79.2) 361 (100) NOTE. Data presented as No. or No. (%). The first two columns show the number of items that were documented by either data managers or by nurse practitioners. The third and fourth columns show the number of items they disagreed and agreed on, respectively. JCO Clinical Cancer Informatics 351 Ebben et al TABLE 5. Mean Time (in minutes) Spent per Patient for Tumor Board–Related scoring values that were not part of the official TNM Activities in Baseline and Effect Measurements classification system (eg, cMX). To evaluate the remaining Activity Baseline (SD) Effect (SD) Delta discordant values and get a better understanding of the Tumor board preparation 00:04:06 (1:44) 00:04:39 (1:59) +00:00:33 cancer registrar drafts, additional investigation would be recommended. Tumor board discussion 00:02:19 (1:27) 00:02:43 (1:41) +00:00:24 To evaluate another avenue that might reduce tumor board preparation workload, we estimated the amount of in- quality, we observed an overall low completeness of tumor formation in the tumor board form that is generated by board reports. This may limit ability for data reuse in practice. Low completeness could be partially explained by ancillary services, like radiology and pathology (Appendix not documenting negative findings (eg, not explicitly Table A1). This suggested that 59% of the data items in the documenting “M0” in absence of metastatic disease). Lack forms could be automatically prepopulated if supported by of disciplined use of structured reporting forms by clinicians the underlying technology. Actual benefits, like with reuse is a well-known phenomenon often attributed to poor EHR of data from the tumor board report, depend on the degree usability and the aforementioned (perceived) dispropor- to which ancillary departments report required information. tionate burden and limitations in reporting freedom. Evaluation regarding usability in the effect measurement We investigated the possibility to have cancer registrars showed an improvement over the baseline, although the assist in tumor board preparation. As there were no sig- number of participants in the survey was low. This was nificant differences in completeness and only a limited partially due to the fact that only a limited number of cli- number of discrepancies in scored data items between nicians are actively involved in the information manage- draft tumor board forms prepared by nurse practitioners ment tasks surrounding tumor board meetings. The survey and cancer registrars, the results hint at the possibility of results were consistent with the positive personal feedback using cancer registrars for this purpose. In the absence of we received from these clinicians. a ground truth, we were not able to evaluate discrepancies in documentation between nurse practitioners and cancer EHR implementations of structured reporting are not registrars and establish which party was correct (if any). unique but are usually based on local physician prefer- However, the number of data items scored with different ences. The strength of our approach is that it is based on values decreased from 11% in baseline to 4.1% in effect a national standard that is derived from the national measurements. guideline and enables evaluation of guideline adherence. As such, this study provides a road map for tumor board This change was largely explained by the fact that the (introduced) standardized tumor board forms prevented meetings for tumors other than breast cancer. AFFILIATIONS AUTHOR CONTRIBUTIONS Department of Research and Development, Netherlands Comprehensive Conception and design: Kees C. W. J. Ebben, Melle S. Sieswerda, Joan B. Cancer Organization, Utrecht, the Netherlands Heijns, Aafke H. Honkoop, Xander A. A. M. Verbeek Department of Surgical Oncology, Amphia Hospital, Breda, the Administrative support: Kees C. W. J. Ebben, Aafke H. Honkoop Netherlands Provision of study material or patients: Joan B. Heijns, Carmen C. van der Department of Medical Oncology, Amphia Hospital, Breda, the Pol, Aafke H. Honkoop Netherlands Collection and assembly of data: Kees C. W. J. Ebben, Melle S. Sieswerda, Department of Surgical Oncology, Alrijne Hospital, Leiden, the Carmen C. van der Pol, Maud Bessems, Janneke Verloop, Xander A. A. M. Netherlands Verbeek Department of Surgical Oncology, Jeroen Bosch Hospital, ’s- Data analysis and interpretation: Kees C. W. J. Ebben, Melle S. Sieswerda, Hertogenbosch, the Netherlands Ernest J. T. Luiten, Joan B. Heijns, Carmen C. van der Pol, Mathijs P. National Breast Cancer Network Netherlands (NABON), Utrecht, the Hendriks, Janneke Verloop, Xander A. A. M. Verbeek Netherlands Manuscript writing: All authors Department of Medical Oncology, Isala Hospital, Zwolle, the Final approval of manuscript: All authors Netherlands Accountable for all aspects of the work: All authors Department of Medical Oncology, Northwest Clinics, Alkmaar, the Netherlands AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST CORRESPONDING AUTHOR The following represents disclosure information provided by authors of Kees C. W. J. Ebben, MSc, Godebaldkwartier 419, 3511 DT Utrecht, the this manuscript. All relationships are considered compensated unless Netherlands; e-mail: k.ebben@iknl.nl. otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO’s EQUAL CONTRIBUTION conflict of interest policy, please refer to www.asco.org/rwc or ascopubs. K.C.W.J.E. and M.S.S. contributed equally to this work. org/cci/author-center. 352 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting Open Payments is a public database containing information reported by Joan B. Heijns companies about payments made to US-licensed physicians (Open Travel, Accommodations, Expenses: Pfizer, Tesaro Payments). Aafke H. Honkoop Melle S. Sieswerda Consulting or Advisory Role: Eli Lilly Employment: United Neuroscience (I) Mathijs P. Hendriks Ernest J. T. Luiten Consulting or Advisory Role: MSD Consulting or Advisory Role: Agendia No other potential conflicts of interest were reported. Speakers’ Bureau: Agendia REFERENCES 1. Lamb BW, Green JSA, Benn J, et al: Improving decision making in multidisciplinary tumor boards: Prospective longitudinal evaluation of a multicomponent intervention for 1,421 patients. J Am Coll Surg 217:412-420, 2013 2. Institute of Medicine: Clinical Practice Guidelines We Can Trust. Washington, DC, National Acadamies Press, 2011 3. Parkin DM: The role of cancer registries in cancer control. Int J Clin Oncol 13:102-111, 2008 4. Newman EA, Guest AB, Helvie MA, et al: Changes in surgical management resulting from case review at a breast cancer multidisciplinary tumor board. Cancer 107:2346-2351, 2006 5. Thenappan A, Halaweish I, Mody RJ, et al: Review at a multidisciplinary tumor board impacts critical management decisions of pediatric patients with cancer. Pediatr Blood Cancer 64:254-258, 2017 6. van Hagen P, Spaander MCW, van der Gaast A, et al: Impact of a multidisciplinary tumour board meeting for upper-GI malignancies on clinical decision making: A prospective cohort study. Int J Clin Oncol 18:214-219, 2013 7. Wheless SA, McKinney KA, Zanation AM: A prospective study of the clinical impact of a multidisciplinary head and neck tumor board. Otolaryngol Head Neck Surg 143:650-654, 2010 8. Pillay B, Wootten AC, Crowe H, et al: The impact of multidisciplinary team meetings on patient assessment, management and outcomes in oncology settings: A systematic review of the literature. Cancer Treat Rev 42:56-72, 2016 9. Peleg M, Tu S, Bury J, et al: Comparing computer-interpretable guideline models: A case-study approach. J Am Med Inform Assoc 10:52-68, 2003 [Erratum: J Am Med Inform Assoc 20:801] 10. Darko-Yawson S, Ellingsen G: Assessing and improving EHRs data quality through a socio-technical approach. Procedia Comput Sci 98:243-250, 2016 11. El Saghir NS, Keating NL, Carlson RW, et al: Tumor boards: Optimizing the structure and improving efficiency of multidisciplinary management of patients with cancer worldwide. Am Soc Clin Oncol Ed Book e461-e466, 2014 12. Schouten LJ, Jager JJ, van den Brandt PA: Quality of cancer registry data: A comparison of data provided by clinicians with those of registration personnel. Br J Cancer 68:974-977, 1993 13. Farrugia DJ, Fischer TD, Delitto D, et al: Improved breast cancer care quality metrics after implementation of a standardized tumor board documentation template. J Oncol Pract 11:421-423, 2015 14. Woods YL, Mukhtar S, McClements P, et al: A survey of reporting of colorectal cancer in Scotland: Compliance with guidelines and effect of proforma reporting. J Clin Pathol 67:499-505, 2014 15. Renshaw AA, Mena-Allauca M, Gould EW, et al: Synoptic reporting: Evidence-based review and future directions. JCO Clin Cancer Inform 10.1200/CCI.17. 16. Sluijter CE, van Lonkhuijzen LRCW, van Slooten H-J, et al: The effects of implementing synoptic pathology reporting in cancer diagnosis: A systematic review. Virchows Arch 468:639-649, 2016 17. Hendriks MP, Verbeek XAAM, van Vegchel T, et al: Transformation of the National Breast Cancer guideline into data-driven clinical decision trees. JCO Clin Cancer Inform 10.1200/CCI.18.00150 18. NABON: Information Standard Breast Cancer. https://decor.nictiz.nl/art-decor/decor-project–onco-mamma- 19. NABON: Breast Cancer Tumor Board questionnaires. Simplefier.net, 2017. https://simplifier.net/breastcancertumorboardquestionnaires/~resources 20. Swillens JEM, Sluijter CE, Overbeek LIH, et al: Identification of barriers and facilitators in nationwide implementation of standardized structured reporting in pathology: A mixed method study. Virchows Arch 475:551-561, 2019 21. Brown T, Bergman S: Doctors, nurses and the paperwork crises that could unite them. New York Times, 2019. https://www.nytimes.com/2019/12/31/opinion/ doctors-nurses-and-the-paperwork-crisis-that-could-unite-them.html 22. Gawande A: Why doctors hate their computers. New Yorker, 2018. https://www.newyorker.com/magazine/2018/11/12/why-doctors-hate-their-computers nn n JCO Clinical Cancer Informatics 353 Ebben et al (14.2%), pathologic 51 (45.1%). For 22 data items (19.5%), the tumor APPENDIX board itself is the primary source. Standardized tumor board reports Distribution of Data Items in the Information Standard included 50% (61/122) of data items also registered retrospectively in the NCR (Appendix Table A1). The distribution of tumor board report data items by their primary source is: administrative 12 (10.6%), clinical 12 (10.6%), radiologic 16 354 © 2020 by American Society of Clinical Oncology Standardized Structured Tumor Board Reporting JCO Clinical Cancer Informatics 355 TABLE A1. The Implemented TBR Forms, Their Included Data Items, and Their Presence in the NCR and Primary Source Reports Data items (population/disease characteristics and workflow data) Preoperative TBR Postoperative TBR Postneoadjuvant-Postoperative TBR NCR Primary Source Reports Tumor board date X X X X Administrative Tumor board report approval date X X X Administrative Present specialists X X X Administrative Clinical question X X X Administrative Pregnancy X X Clinical Disease laterality X X X X Clinical Disease localization X X X X Clinical BI-RADS X X Radiology Calcifications in breast X Radiology Solid mass in breast X Radiology Tumor laterality X X Radiology Tumor localization X X Radiology cT diameter X X Radiology cT4 characteristics X Radiology Morphology X X X X Pathology Grade X X X X Pathology cT X X TBR Tumor distribution X X Radiology Solid component on mammography X Radiology ER percentage X X X X Pathology HER2 status X X X X Pathology Lymph node laterality X X X Radiology Fixed axillary lymph nodes X Radiology Clinical positive axillary lymph nodes X Radiology Clinical positive parasternal lymph nodes X X X Radiology Clinical positive infraclavicular lymph nodes X Radiology Clinical positive supraclavicular lymph nodes X Radiology cN X X TBR Localization metastasis X X Radiology ER percentage metastases X Pathology HER2 status metastases X Pathology cM X X TBR cTNM X X TBR Clinical study advice X X X TBR Clinical study X X X X TBR (Continued on following page) Ebben et al 356 © 2020 by American Society of Clinical Oncology TABLE A1. The Implemented TBR Forms, Their Included Data Items, and Their Presence in the NCR and Primary Source Reports (Continued) Data items (population/disease characteristics and workflow data) Preoperative TBR Postoperative TBR Postneoadjuvant-Postoperative TBR NCR Primary Source Reports Policy X X X X TBR Motivation for CPG deviation X X X TBR pT diameter X X Pathology pT4 characteristics X Pathology Cutting edge X X X Pathology Angio invasion X X X Pathology pT X X Pathology Positive supraclavicular lymph nodes in SN X X Pathology Positive parasternal lymph nodes in SN X X Pathology Positive infraclavicular lymph nodes X X Pathology No. of axillary lymph nodes with micro- or macrometastases in SN X X X Pathology No. of axillary lymph nodes with macrometastases in SN X X X Pathology No. of axillary lymph nodes with micrometastases in SN X X X Pathology No. of axillary lymph nodes with isolated tumor cells in SN X X X Pathology Extranodal growth in SN X X X Pathology Extranodal growth in lymph node dissection X X X Pathology Positive axillary top lymph node X X X Pathology pN X X Pathology pM X X Pathology pTNM X X TBR Genetic mutation X X X Pathology N0 risk status X Pathology First re-excision X X X TBR Lymph node surgery X X X TBR ypT 4 diameter X X Pathology ypT 4 characteristics X Pathology Pathologic response primary tumor X X Pathology ypT X X Pathology ypN X X Pathology ypM X X Pathology ypTNM X X TBR Total 37 39 37 61 Abbreviations: BI-RADS, Breast Imaging-Reporting and Data System; CPG, clinical practice guideline; ER, estrogen receptor; NCR, Netherlands Cancer Registry; SN, sentinel node; TBR, tumor board report. Several NCR data items are registered multiple times throughout the clinical pathway. Additional data items (not CPG) identified by care professionals to optimally support the tumor board processes.

Journal

JCO Clinical Cancer InformaticsWolters Kluwer Health

Published: Apr 23, 2020

There are no references for this article.