Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Interventions on health care providers to improve seasonal influenza vaccination rates among patients: a systematic review and meta-analysis of the evidence since 2000

Interventions on health care providers to improve seasonal influenza vaccination rates among... Abstract Background Seasonal influenza vaccination (SIV) rates remain suboptimal in many populations, even in those with universal SIV. Objective To summarize the evidence on interventions on health care providers (physicians/nurses/pharmacists) to increase SIV rates. Methods We systematically searched/selected full-text English publications from January 2000 to July 2019 (PROSPERO-CRD42019147199). Our outcome was the difference in SIV rates between patients in intervention and non-intervention groups. We calculated pooled difference using an inverse variance, random-effects model. Results We included 39 studies from 8370 retrieved citations. Compared with no intervention, team-based training/education of physicians significantly increased SIV rates in adult patients: 20.1% [7.5–32.7%; I2 = 0%; two randomized controlled trials (RCTs)] and 13.4% [8.6–18.1%; I2 = 0%; two non-randomized intervention studies (NRS)]. A smaller increase was observed in paediatric patients: 7% (0.1–14%; I2 = 0%; two NRS), and in adult patients with team-based training/education of physicians and nurses together: 0.9% (0.2–1.5%; I2 = 30.6%; four NRS). One-off provision of guidelines/information to physicians, and to both physicians and nurses, increased SIV rates in adult patients: 23.8% (15.7–31.8%; I2 = 45.8%; three NRS) and paediatric patients: 24% (8.1–39.9%; I2 = 0%; two NRS), respectively. Use of reminders (prompts) by physicians and nurses slightly increased SIV rates in paediatric patients: 2.3% (0.5–4.2%; I2 = 0%; two RCTs). A larger increase was observed in adult patients: 18.5% (14.8–22.1%; I2 = 0%; two NRS). Evidence from both RCTs and NRS showed significant increases in SIV rates with varied combinations of interventions. Conclusions Limited evidence suggests various forms of physicians’ and nurses’ education and use of reminders may be effective for increasing SIV rates among patients. Health personnel, human influenza, intervention, meta-analysis, systematic review, vaccination Key messages We reviewed 39 studies [7 RCTs and 32 non-randomized intervention studies (NRS)]. Physicians/nurses’ education/reminders increased vaccination among patients. Combinations of education and reminders increased vaccination among patients. Education and reminders were both effective among adult and paediatric patients. Introduction Vaccination is one of the most successful interventions against vaccine-preventable diseases and is considered to be the most effective and practical strategy for influenza prevention. Advisory bodies such as the World Health Organization (WHO) (1), the European Centre for Disease Prevention and Control (ECDC) (2) and the advisory committees on immunization in the United States of America (USA) (ACIP) (3) and Canada (NACI) (4) recommend annual seasonal influenza vaccination (SIV) for individuals 6 months of age and above, with emphasis on subpopulations that are at increased risk of developing influenza-related complications such as infants, older adults, pregnant women and persons with certain morbidities. Consequently, many countries have annual SIV programmes (5). Despite these programmes, SIV rates remain below national and WHO targets, even in jurisdictions (health regions) with universal (free-for-all) SIV programmes. Refusal or a delay in acceptance of a vaccine despite its availability and accessibility (vaccine hesitancy) (6) varies across time and place, and may explain the suboptimal SIV rates. This phenomenon is thought to be influenced by an individual’s complacency (believing disease is not serious or that vaccination is not necessary), convenience (vaccine availability, accessibility and affordability) and confidence (belief that vaccine works and is safe) in regard to influenza vaccination (6). However, health care providers (physicians, nurses, pharmacists, etc.) have been identified as the most important advocates and the main source of vaccination information for the public, and therefore could facilitate vaccine uptake (7–9). There have been a few published systematic reviews of interventions focussed on health care providers to increase SIV rates among patients. A Cochrane review focussed only on older community-based patients in high-income countries (10). Other reviews were limited to community-based adult patients (11), or specific patient subpopulations and were mainly narrative without meta-analysis (12,13). These reviews did not assess if any differences existed in the efficacy/effectiveness of interventions across patient populations and study types, and across subtypes of an intervention. To address these gaps in knowledge, we summarized the evidence to determine if the efficacy/effectiveness of the interventions vary by study type, health care provider group, patient population and jurisdiction. Methods We conducted a systematic review and meta-analysis of the available evidence, in accordance with the Methodological Expectations of the Cochrane Intervention Reviews (MECIR) guidelines (14), and reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines (15). We developed and registered a protocol for the review in the International Prospective Register of Systematic Reviews (CRD42019147199) prior to commencement of the review. We included published studies of interventions on health care providers aimed at improving SIV rates. Only studies conducted from the year 2000 onwards, with a full-text publication in English, were included. We excluded mathematical modelling studies and grey literature. Search strategy and selection criteria A knowledge synthesis librarian designed a literature search strategy for Medline (Ovid) (Supplementary Table 1). The search strategy was peer-reviewed by a second, independent professional knowledge synthesis librarian using the PRESS checklist (16). The final search strategy was then adapted for EMBASE (Ovid), CINAHL (EBSCO) and Scopus (Web) bibliographic databases, and executed. All literature searches were conducted on 24 July 2019. Identified citations from the executed searches were imported into a specially designed Microsoft Access 2016 database (Microsoft Corporation, Redmond, WA), and screened for eligibility by two independent systematic reviewers using a two-stage sifting approach to review the title/abstract and full-text article. The number of ineligible citations at the title/abstract screening stage was recorded, and both the number and reasons for ineligibility were recorded at the full-text article screening stage. Disagreements during these screening stages were resolved by discussion between the two systematic reviewers, and a third systematic reviewer adjudicated when necessary. Data extraction Data extraction was conducted using Microsoft Excel 2016 (Microsoft Corporation, Redmond, WA) by one reviewer after a pilot was conducted on a small selection of the included studies by the reviewer and another independent reviewer. A second reviewer independently checked the extracted data for errors. Disagreements were resolved by discussion between the two reviewers, and a third reviewer adjudicated when necessary. We extracted data such as study first author’s name, duration, publication year, funder, patient population, health care provider information, intervention(s) assessed, and the reported difference in SIV rates between patients in health care provider intervention and control groups or between patients before and after application of a health care provider intervention. The associated 95% confidence interval (CI) of the reported differences was also extracted. In addition, we extracted details relevant to risk of bias or study quality assessment. If a study also administered patient-focussed interventions, it was only included if patients in both the health care provider intervention and control groups (or patients before and during the health care provider intervention) received the same exact patient intervention(s). We did not include studies in which patients in just one of the compared groups (or intervention periods) received patient-focussed intervention(s). Data synthesis and analysis We summarized study characteristics and pooled reported differences in SIV rates between intervention and no intervention groups, and associated 95% CIs using an inverse variance, random-effects model implemented in STATA (version 13; StataCorp LP, College Station, TX). Educational interventions were categorized as either one-off provision of guidelines/information or team-based training/education sessions. If a study reported differences in SIV rates by population subgroups, we first pooled the differences before pooling the pooled estimate with appropriate differences in SIV rates from other studies. Statistical heterogeneity between pooled differences in SIV rates was assessed and quantified using the I-squared statistic (I2) (17). Where appropriate, publication bias was assessed visually using funnel plots, and statistically using Egger’s regression test (18). We conducted subgroup analyses by study type, intervention, health care provider and patient population (adults versus paediatrics). Risk of bias and study quality assessments One reviewer assessed risk of bias using the Cochrane risk of bias assessment tool for randomized controlled trials (RCTs) (we also used the tool for a few non-/quasi-RCTs by adding a confounder adjustment domain) (19). The reviewer also assessed study quality of ‘before-and-after intervention studies’ (where outcome assessment is made before and after the introduction of an intervention) using the National Institutes of Health (NIH) quality assessment tool for this study type (20). Another reviewer independently checked the assessments for errors. Disagreements were resolved by discussion between the two reviewers, and a third reviewer adjudicated when necessary. The Cochrane risk of bias assessment tool considers 6 main domains of potential biases, while the NIH quality assessment tool considers 12 domains of study quality assessment. If there was a high risk of bias in randomization (sequence generation domain assessment), a study was automatically judged to be at a high risk of bias. A study was also judged to be at a high risk of bias if any one of the other domains were judged to be of high risk, and an additional domain judged as having an unclear or high risk of bias. A study was judged as having a low risk of bias if all six domains were judged to be of low risk of bias, or if just one domain (not sequence generation) was judged as having an unclear risk of bias. A study was judged to be of good quality if all 12 domains were judged not to be lacking, or if just one domain (not pre-specified and uniformly measured outcome domain) was judged to be lacking. A moderate quality was assigned if two to four domains were judged to be lacking, while a low quality rating was given if five or more domains were judged to be lacking. Results Thirty-nine studies met our eligibility criteria, from a total of 8370 identified unique citations (Fig. 1) (21–59). Most of the studies (n = 28) were conducted in the USA (Table 1). The rest were conducted mostly in developed countries: two studies each for the UK and Australia, and one study each for UK/Australia, Canada, Denmark, Netherlands, Switzerland, India and Turkey. There were 7 RCTs (3 parallel and 4 cluster randomized) (22,28,47,49,52,56,59) and 32 non-randomized intervention studies (NRS) (2 non-RCTs (33,39), 1 quasi-RCT (54) and 29 before-and-after intervention studies). A total of 20 studies were focussed on physicians, while a smaller number of studies examined interventions targeting nurses (2 studies), pharmacists (1 study), both physicians and nurses (14 studies), both physicians and pharmacists (1 study) and both physicians separately and physicians and nurses together (1 study). Eleven studies examined the use of team-based training/education sessions, and six the use of one-off provision of guidelines/information. Eleven studies examined the use of reminders (nine using electronic prompts, one using paper-based prompt and one using letters), two studies examined the use of incentives (pay-for-performance) whereas nine studies examined the use of multiple interventions. There were five studies each on children/adolescents, pregnant women, adults and older adults, irrespective of health status, and three studies each specifically on chronic obstructive pulmonary disease, and inflammatory bowel disease patients. There were two studies each on adult diabetic and rheumatology patients, and one study each on preterm infants, paediatric asthma, respiratory tract infection, musculoskeletal/neurological patients, coronary heart disease, chronic kidney disease, Hodgkin’s lymphoma patients and patients with a risk of cardiovascular disease. One study was on a mixed patient population. The health care provider sample size among the studies that clearly reported this parameter ranged from 4 to 136 individuals. Six studies were funded by industry (pharmaceutical companies), while non-industry funding was reported by 25 studies. One study was funded by both industry and non-industry sources, and seven studies did not report a funding source. Nearly all RCTs and non-/quasi-RCTs were judged to have a high risk of bias with regard to lack of blinding of participants and study personnel. A majority of the before-and-after intervention studies were deficient with regard to study participants being a good representation of the studied population, having sufficiently large sample size, and blinding of outcome assessors. Overall, the included RCTs and non-/quasi-RCTs were judged to be at a moderate to high risk of bias (Supplementary Table 2), and the before-and-after intervention studies were judged to be of moderate to low quality (Supplementary Table 3). Figure 1. Open in new tabDownload slide Summary of literature search and screening process (modified PRISMA flow chart). Figure 1. Open in new tabDownload slide Summary of literature search and screening process (modified PRISMA flow chart). Table 1. Summary of study characteristics Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel a,b,cRepresent different cohorts (study participants) in the studies. EU, European Union; GSA, Gastroenterological Society of Australia; IBD, inflammatory bowel disease; MRC, Medical Research Council; NHS, National Health Service; NR, not reported. Open in new tab Table 1. Summary of study characteristics Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel a,b,cRepresent different cohorts (study participants) in the studies. EU, European Union; GSA, Gastroenterological Society of Australia; IBD, inflammatory bowel disease; MRC, Medical Research Council; NHS, National Health Service; NR, not reported. Open in new tab Pooled difference in SIV rates for education compared with no intervention Two RCTs (56,59) and 15 NRS (23,25,30,32,33,38,39,41,43,45,46,50,54,57,58) contributed to the pooled estimates. Evidence from RCTs showed that team-based training/education of physicians significantly increased SIV rates in adult patients: 20.1% (95% CI 7.5–32.7%; I2 = 0%; two studies) (Fig. 2). Evidence from NRS also showed that team-based training/education of physicians significantly increased SIV rates in adult patients: 21.6% (3.4–39.9%; I2 = 97.5%; three studies) and paediatric patients: 7% (0.1–14%; I2 = 0%; two studies) (Supplementary Fig. 1). The high heterogeneity observed in adult patients was due to higher estimates from Turkey, a country with a dissimilar health system from the other countries (Switzerland and USA). Exclusion of the result from Turkey significantly reduced the high heterogeneity observed in the pooled estimates (13.4%, 8.6–18.1; I2 = 0%). The evidence from NRS also showed that education of physicians and nurses marginally (though significantly) increased SIV rates in adult patients: 0.9% (0.2–1.5%; I2 = 30.6%; four studies) (Supplementary Fig. 1). Similarly, evidence from NRS showed that one-off provision of guidelines/information to physicians, and to both physicians and nurses, significantly increased SIV rates in adult patients: 23.8% (15.7–31.8%; I2 = 45.8%; three studies) and paediatric patients: 24% (8.1–39.9%; I2 = 0%; two studies), respectively (Supplementary Fig. 2). One study focussed on nurses and adult patients, and the study reported no significant benefit. Figure 2. Open in new tabDownload slide Forest plot of team-based training/education of physicians compared with no intervention, in adult patients (RCTs). Figure 2. Open in new tabDownload slide Forest plot of team-based training/education of physicians compared with no intervention, in adult patients (RCTs). Pooled difference in SIV rates for reminders compared with no intervention Three RCTs (22,47,49) and seven out of eight NRS (21,29,34,36,42,48,53) contributed to the pooled estimates. One NRS was excluded from the meta-analysis because the authors reported that there was vaccine shortage in the study year compared with the preceding year (the comparator), and as a result, many patients who requested vaccination could not be vaccinated (31). Result from this study differed significantly from the rest. Evidence from RCTs showed that use of reminders (prompts) by physicians and nurses significantly increased SIV rates in paediatrics patients by 2.3% (0.5–4.2%; I2 = 0%; two studies) (Fig. 3). There was one study on the use of letter reminders among physicians in adult patients, and the study reported no significant benefit. Evidence from NRS showed that physicians and both physicians’ and nurses’ use of reminders (prompts) significantly increased SIV rates in adult patients: 15.3% (7.4–23.2%; I2 = 98.1%; three studies) and 18.5% (14.8–22.1%; I2 = 0%; two studies), respectively (Supplementary Fig. 3). There were no major differences between the studies among physicians that could explain the high heterogeneity in the pooled estimates. However, all pooled results were in favour of the intervention. There was one study each among nurses alone and physicians and nurses together in paediatric patients and both were in favour of the intervention. Figure 3. Open in new tabDownload slide Forest plot of reminders (prompts) compared with no intervention (RCTs). Figure 3. Open in new tabDownload slide Forest plot of reminders (prompts) compared with no intervention (RCTs). Pooled difference in SIV rates for incentives compared with no intervention One NRS each examined the impact of physicians’ pay-for-performance (27), and physicians’ practices performance bonus (37) on SIV rates in adult patients, and found significantly increased SIV rate for pay-for-performance: 28.1% (1.3% to 54.9%), and no difference for performance bonus: −0.1% (−42.1% to 42%), respectively. Pooled difference in SIV rates for combined interventions compared with no intervention Two RCTs (28,52) and two out of five NRS (40,44) contributed to the pooled estimates. Evidence from RCTs showed that team-based training/education of physicians and nurses, together with use of reminders, significantly increased SIV rates in adult patients: 10.8% (3.3–18.2%; I2 = 0%; two studies) (Fig. 4). Evidence from NRS showed that one-off provision of guidelines/information to physicians together with and use of reminders significantly increased SIV rates in adult patients: 19.5% (14.9–24.1%; I2 = 0%; two studies) (Supplementary Fig. 4). One study among physicians and nurses in adult patients was also in favour of the interventions combination (Supplementary Fig. 4) (35). One study each reported on team-based training/education of physicians and of pharmacists together with the use of reminders in adult patients and both studies reported no difference: 10.9% (−3.2% to 26.5%) and 24% (−0.0% to 48%), respectively. One study each also reported on physician pay-for-performance together with reminders (26), and team-based training/education together with fee-for-service for physicians and for pharmacists and physicians (51), all in adult patients. All three were in favour of the intervention combinations: 15.9% (7.9–23.8%), 15% (5.6–24%) and 3.7% (2.2–5.2%), respectively. Figure 4. Open in new tabDownload slide Forest plot of combined team-based training/education of physicians and nurses, and reminders (prompts) compared with no intervention, in adult patients (RCTs). Figure 4. Open in new tabDownload slide Forest plot of combined team-based training/education of physicians and nurses, and reminders (prompts) compared with no intervention, in adult patients (RCTs). Discussion This review indicates that team-based training/education and one-off provision of guidelines/information to physicians and nurses may be effective in increasing SIV rates in adult and paediatric patient populations. The review also indicates that use of reminder prompts by physicians and nurses may be effective in increasing SIV rates in adult and patient populations, and that various combinations of educational and reminder interventions may also be effective in increasing SIV rates. The evidence supporting these findings are, however, limited and are of moderate to low quality. While patient population types varied considerably across the included studies, almost all belonged to the high-risk groups for whom SIV is highly recommended. Physician specialty also varied slightly across studies. All but one study were conducted in developed countries with similar health system characteristics, although there would have been potentially differing public health practices across the countries with respect to influenza vaccination programs. There were substantial variabilities in the number of studied health care providers and patients, and the forms, methods of delivery and durations of an intervention. Furthermore, a majority of the studies were conducted in the USA; thus, our findings may not be generalizable and may be more applicable to the USA. That said, our findings are substantial contributions to the evidence base and will help in the development and evaluation of health care provider-focussed interventions to increase seasonal influenza vaccine uptake and, potentially, uptake of other vaccines across important patient populations. SIV is proven to be a safe, effective and cost-effective preventive measure against influenza, but annual rates remain suboptimal in many countries even among the most at-risk populations. Vaccination behaviour is influenced by so many factors including perceptions of vaccines and vaccination, and intrinsic factors such as an individual’s knowledge, perceptions and attitudes which are often shaped by many external factors. Despite immense influence and importance of the media, health care professionals have been identified as the most important advocates and the main source of vaccination information for the public (7). The summit of independent European vaccination experts in 2007 concluded that it is of utmost importance that health care providers are offered adequate education on vaccinology and an unhindered access to up-to-date information on vaccines (7). The experts also concluded that it is important that vaccine information systems are set up for health care providers, to facilitate promotion of vaccination. Education can play an important role in prompting positive change in behaviour (60,61). Increases in knowledge and awareness have been found to lead to changes in attitudes (62). Education of health care providers on specific health issues is therefore expected to help increase their knowledge and awareness and, ultimately, promote a positive change in behaviours. This may explain our finding that one-off provision of guidelines/information or team-based training/education of physicians and physicians together with nurses are effective in increasing SIV rates in both adult and paediatric patient populations. Contrary to our findings, a Cochrane review (Thomas and Lorenzetti) found that physician education was ineffective (10). The review included only RCTs focussing on SIV among community-based patients (≥60 years) in high-income countries, unlike our review which considered all available evidence irrespective of provider type, patient population and country of study. In addition, Thomas and Lorenzetti considered all educational interventions the same, whereas we appropriately categorized the interventions, considering that types and methods of education differ. An older systematic review (Lau) also found physician education to be ineffective (11). This review also focussed on community-based adults and, although the review considered both RCTs and non-RCTs, educational interventions were examined as one group without categorization. It was also not clear which studies contributed to the pooled analysis. Reminder systems serve as alerting tools. In clinical settings, they alert a health care provider to the key procedures and necessary preventive care that a patient may need; thus, informing of the importance of a needed care (63). The Centers for Disease Control and Prevention (CDC) stress that a reminder system helps to reduce missed opportunities for recommending and providing influenza vaccine to eligible patients when vaccine is available (63). Therefore, implementing a SIV reminder system for health care providers is expected to increase the vaccine recommendation and, consequently, vaccination rates as our review suggests. Implementing a reminder system for both physicians and nurses provides more than one level of quality assurance check, and decreases the risk of failing to provide an important recommendation to an eligible patient. This has been demonstrated by studies which have found that a reminder system for both physicians and nurses leads to increased vaccination uptake (64,65). Our findings are in line with those of some clinical studies that have shown that computerized reminders increase the use of preventive care in outpatient settings (66,67). However, some studies in inpatient settings using standard orders and computerized reminders failed to demonstrate the effectiveness of this intervention (67,68), probably due to a focus on treatment of the cause of hospitalization rather than the provision of preventive care. That said, as a study suggested, it is worth noting that reminder systems for physicians are effective only when applied to all eligible patients (69,70). Our decision to include only English publications may have limited the number of potentially relevant studies for inclusion. Limiting to studies since 2000 may have also limited the inclusion of studies, but this allowed us to focus on studies conducted during the period in which SIV became publicly funded in many jurisdictions. We were unable to determine if SIV was offered free-of-charge to patients or whether some of the patients had to pay for the vaccine out-of-pocket. As such, we could not examine the impact of free vaccination on the efficacy/effectiveness of the assessed interventions. Furthermore, nearly all reported SIV rates from the NRS were crude rates. For the before-and-after intervention studies, the populations at baseline were not always exactly the same as those post-intervention. Nevertheless, this review has many merits, including full compliance with the Cochrane Handbook for Systematic Reviews of Interventions guidelines, utilization of the expertise of knowledge synthesis librarians in developing and peer reviewing a comprehensive search strategy, and searching of appropriate bibliographic databases for literature. Where necessary, we requested additional data from corresponding authors of the included studies, to ensure completeness of the analysed data. We conducted the review to the expected standards and reported in accordance with the PRISMA guidelines. Conclusions Limited low- to moderate-quality evidence suggests that various forms of education for physicians and nurses, and use of reminder prompts by physicians and nurses, may be effective for increasing SIV rates in both adult and paediatric patient populations. High-quality studies are needed for a stronger evidence base to better inform and understand the efficacy/effectiveness of various interventions focussed on health care providers to increase SIV. Acknowledgements We thank Drs Thomas J Van Hoof, James M Gill, Ulrike Dapp, Louis Vernacchio, William Trick, Ari Pollack, Timothy Loo, David Nace, Valerie Weber, Eric Ruderman and Jan van Lieshout for responding kindly to our request for additional data on their publication(s). Declaration Funding: this work was supported by Sanofi Pasteur, USA (grant number not applicable). The funder contributed towards specifying the research questions but had no role in the study design, data collection and analysis. Ethical approval: not applicable. Conflict of interest: Salaheddin M Mahmud (SMM) has received unrestricted research grants from GlaxoSmithKline, Merck, Sanofi Pasteur, Pfizer and Roche-Assurex for unrelated studies, and fees as an advisory board member for Sanofi Pasteur. SMM is supported, in part, by funding from the Canada Research Chairs Program. E Thommes and A Chit are employees of Sanofi Pasteur, USA. The other authors have no conflicts of interest to declare. Data availability The data underlying this article are available in the article and in its online supplementary material. References 1. The World Health Organization Strategic Advisory Group of Experts (SAGE) Working Group . Background Paper on Influenza Vaccines and Immunization . Geneva, Switzerland : WHO , 2012 . https://www.who.int/influenza/vaccines/SAGE_information/en/ Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 2. European Centre for Disease Prevention and Control . Seasonal Influenza Vaccination and Antiviral Use in EU/EEA Member States: Overview of Vaccine Recommendations for 2017–2018 and Vaccination Coverage Rates for 2015–2016 and 2016–2017 Influenza Seasons . Stockholm, Sweden : ECDC , 2018 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 3. Grohskopf LA , Sokolow LZ, Broder KR et al. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices—United States, 2017–18 influenza season . Morb Mortal Wkly Rep 2017 ; 66 ( 2 ): 1 – 20 . Google Scholar OpenURL Placeholder Text WorldCat 4. National Advisory Committee on Immunization (NACI) . Canadian Immunization Guide Chapter on Influenza and Statement on Seasonal Influenza Vaccine for 2017–2018 . Ottawa, Ontario, Canada : Public Health Agency of Canada , 2017 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 5. Ortiz JR , Perut M, Dumolard L et al. A global review of national influenza immunization policies: analysis of the 2014 WHO/UNICEF Joint Reporting Form on immunization . Vaccine 2016 ; 34 ( 45 ): 5400 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 6. MacDonald NE ; SAGE Working Group on Vaccine Hesitancy. Vaccine hesitancy: definition, scope and determinants . Vaccine 2015 ; 33 ( 34 ): 4161 – 4 . Google Scholar Crossref Search ADS PubMed WorldCat 7. Schmitt HJ , Booy R, Aston R et al. How to optimise the coverage rate of infant and adult immunisations in Europe . BMC Med 2007 ; 5 : 11 . Google Scholar Crossref Search ADS PubMed WorldCat 8. Smith PJ , Kennedy AM, Wooten K, Gust DA, Pickering LK. Association between health care providers’ influence on parents who have concerns about vaccine safety and vaccination coverage . Pediatrics 2006 ; 118 ( 5 ): e1287 – 92 . Google Scholar Crossref Search ADS PubMed WorldCat 9. Vorsters A , Bonanni P, Maltezou HC et al. The role of healthcare providers in HPV vaccination programs—a meeting report . Papillomavirus Res 2019 ; 8 : 100183 . Google Scholar Crossref Search ADS PubMed WorldCat 10. Thomas RE , Lorenzetti DL. Interventions to increase influenza vaccination rates of those 60 years and older in the community (review) . Cochrane Database Syst Rev 2018 ;( 5 ): CD005188 . Google Scholar OpenURL Placeholder Text WorldCat 11. Lau D , Hu J, Majumdar SR et al. Interventions to improve influenza and pneumococcal vaccination rates among community-dwelling adults: a systematic review and meta-analysis . Ann Fam Med 2012 ; 10 ( 6 ): 538 – 46 . Google Scholar Crossref Search ADS PubMed WorldCat 12. Sanftenberg L , Brombacher F, Schelling J, Klug SJ, Gensichen J. Increasing influenza vaccination rates in people with chronic illness . Dtsch Arztebl Int 2019 ; 116 ( 39 ): 645 – 52 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 13. Gosselin Boucher V , Colmegna I, Gemme C et al. Interventions to improve vaccine acceptance among rheumatoid arthritis patients: a systematic review . Clin Rheumatol 2019 ; 38 ( 6 ): 1537 – 44 . Google Scholar Crossref Search ADS PubMed WorldCat 14. Higgins JPT , Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews (MECIR) . London, UK : The Cochrane Collaboration , 2016 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 15. Moher D , Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . Int J Surg 2010 ; 8 ( 5 ): 336 – 41 . Google Scholar Crossref Search ADS PubMed WorldCat 16. McGowan J , Sampson M, Salzwedel DM et al. PRESS peer review of electronic search strategies: 2015 guideline statement . J Clin Epidemiol 2016 ; 75 : 40 – 6 . Google Scholar Crossref Search ADS PubMed WorldCat 17. Higgins JP , Thompson SG. Quantifying heterogeneity in a meta-analysis . Stat Med 2002 ; 21 ( 11 ): 1539 – 58 . Google Scholar Crossref Search ADS PubMed WorldCat 18. Sterne JA , Egger M, Smith GD. Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis . BMJ 2001 ; 323 ( 7304 ): 101 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 19. Higgins JPT , Altman DG, Gøtzsche PC et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials . BMJ 2011 ; 343 : d5928 . Google Scholar OpenURL Placeholder Text WorldCat 20. National Hearth L, and Blood Institute . Quality Assessment Tool for Before-After (Pre-Post) Studies with No Control Group . National Institutes of Health (NIH) . https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools. (accessed on 31 December 2019 ). Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 21. Gill JM , Saldarriaga AM. The impact of a computerized physician reminder and a mailed patient reminder on influenza immunizations for older patients . Del Med J 2000 ; 72 ( 10 ): 425 – 30 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 22. Chan L , MacLehose RF, Houck PM. Impact of physician reminders on the use of influenza vaccinations: a randomized trial . Arch Phys Med Rehabil 2002 ; 83 ( 3 ): 371 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 23. Gill JM , DiPrinzio MJ. The Medical Society of Delaware’s Uniform Clinical Guidelines for diabetes: did they have a positive impact on quality of diabetes care? Del Med J 2004 ; 76 ( 3 ): 111 – 22 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 24. Hutt E , Ruscin JM, Corbett K et al. A multifaceted intervention to implement guidelines improved treatment of nursing home-acquired pneumonia in a state veterans home . J Am Geriatr Soc 2006 ; 54 ( 11 ): 1694 – 700 . Google Scholar Crossref Search ADS PubMed WorldCat 25. Ogburn T , Espey EL, Contreras V, Arroyo P. Impact of clinic interventions on the rate of influenza vaccination in pregnant women . J Reprod Med 2007 ; 52 ( 9 ): 753 – 6 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 26. Weber V , Bloom F, Pierdon S, Wood C. Employing the electronic health record to improve diabetes care: a multifaceted intervention in an integrated delivery system . J Gen Intern Med 2008 ; 23 ( 4 ): 379 – 82 . Google Scholar Crossref Search ADS PubMed WorldCat 27. McGovern MP , Boroujerdi MA, Taylor MW et al. The effect of the UK incentive-based contract on the management of patients with coronary heart disease in primary care . Fam Pract 2008 ; 25 ( 1 ): 33 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat 28. Hogg W , Lemelin J, Graham ID et al. Improving prevention in primary care: evaluating the effectiveness of outreach facilitation . Fam Pract 2008 ; 25 ( 1 ): 40 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 29. Ledwich LJ , Harrington TM, Ayoub WT, Sartorius JA, Newman ED. Improved influenza and pneumococcal vaccination in rheumatology patients taking immunosuppressants using an electronic health record best practice alert . Arthritis Rheum 2009 ; 61 ( 11 ): 1505 – 10 . Google Scholar Crossref Search ADS PubMed WorldCat 30. Snow V , Reynolds CE, Bennett L et al. Closing the gap-cardiovascular risk and primary prevention: results from the American College of Physicians quality improvement program . Am J Med Qual 2010 ; 25 ( 4 ): 261 – 7 . Google Scholar Crossref Search ADS PubMed WorldCat 31. Loo TS , Davis RB, Lipsitz LA et al. Electronic medical record reminders and panel management to improve primary care of elderly patients . Arch Intern Med 2011 ; 171 ( 17 ): 1552 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 32. Nace DA , Perera S, Handler SM, Muder R, Hoffman EL. Increasing influenza and pneumococcal immunization rates in a nursing home network . J Am Med Dir Assoc 2011 ; 12 ( 9 ): 678 – 84 . Google Scholar Crossref Search ADS PubMed WorldCat 33. Dapp U , Anders JA, von Renteln-Kruse W et al. ; PRO-AGE Study Group . A randomized trial of effects of health risk appraisal combined with group sessions or home visits on preventive behaviors in older adults . J Gerontol A Biol Sci Med Sci 2011 ; 66 ( 5 ): 591 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 34. Patwardhan A , Kelleher K, Cunningham D, Menke J, Spencer C. The use of a mandatory best practice reminder in the electronic record improves influenza vaccination rate in a pediatric rheumatology clinic . Clin Govern 2011 ; 16 ( 4 ): 308 – 19 . Google Scholar Crossref Search ADS WorldCat 35. Sherman MJ , Raker CA, Phipps MG. Improving influenza vaccination rates in pregnant women . J Reprod Med 2012 ; 57 ( 9–10 ): 371 – 6 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 36. Klatt TE , Hopp E. Effect of a best-practice alert on the rate of influenza vaccination of pregnant women . Obstet Gynecol 2012 ; 119 ( 2 Pt 1 ): 301 – 5 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 37. Kirschner K , Braspenning J, Akkermans RP, Jacobs JE, Grol R. Assessment of a pay-for-performance program in primary care designed by target users . Fam Pract 2013 ; 30 ( 2 ): 161 – 71 . Google Scholar Crossref Search ADS PubMed WorldCat 38. Satman I , Akalin S, Cakir B, Altinel S; diaVAX Study Group. The effect of physicians’ awareness on influenza and pneumococcal vaccination rates and correlates of vaccination in patients with diabetes in Turkey: an epidemiological Study “diaVAX” . Hum Vaccin Immunother 2013 ; 9 ( 12 ): 2618 – 26 . Google Scholar Crossref Search ADS PubMed WorldCat 39. Ulrik CS , Sørensen TB, Højmark TB, Olsen KR, Vedsted P. Adherence to COPD guidelines in general practice: impact of an educational programme delivered on location in Danish general practices . Prim Care Respir J 2013 ; 22 ( 1 ): 23 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 40. Walsh AJ , Weltman M, Burger D et al. Implementing guidelines on the prevention of opportunistic infections in inflammatory bowel disease . J Crohns Colitis 2013 ; 7 ( 10 ): e449 – 56 . Google Scholar Crossref Search ADS PubMed WorldCat 41. McCreary L . Increasing the rate of influenza vaccination in children with asthma using a clinic staff and provider educational intervention . J Asthma Allergy Educ 2013 ; 4 ( 6 ): 277 – 81 . Google Scholar Crossref Search ADS WorldCat 42. Pollack AH , Kronman MP, Zhou C, Zerr DM. Automated screening of hospitalized children for influenza vaccination . J Pediatr Infect Dis Soc 2014 ; 3 ( 1 ): 7 – 14 . Google Scholar Crossref Search ADS WorldCat 43. Hull S , Mathur R, Lloyd-Owen S, Round T, Robson J. Improving outcomes for people with COPD by developing networks of general practices: evaluation of a quality improvement project in east London . NPJ Prim Care Respir Med 2014 ; 24 : 14082 . Google Scholar Crossref Search ADS PubMed WorldCat 44. Mendu ML , Schneider LI, Aizer AA et al. Implementation of a CKD checklist for primary care providers . Clin J Am Soc Nephrol 2014 ; 9 ( 9 ): 1526 – 35 . Google Scholar Crossref Search ADS PubMed WorldCat 45. Vernacchio L , Francis ME, Epstein DM et al. Effectiveness of an asthma quality improvement program designed for maintenance of certification . Pediatrics 2014 ; 134 ( 1 ): e242 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 46. Crawford NW , Barfield C, Hunt RW, Pitcher H, Buttery JP. Improving preterm infants’ immunisation status: a follow-up audit . J Paediatr Child Health 2014 ; 50 ( 4 ): 314 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 47. Szilagyi PG , Serwint JR, Humiston SG et al. Effect of provider prompts on adolescent immunization rates: a randomized trial . Acad Pediatr 2015 ; 15 ( 2 ): 149 – 57 . Google Scholar Crossref Search ADS PubMed WorldCat 48. Pierson RC , Malone AM, Haas DM. Increasing influenza vaccination rates in a busy urban clinic . J Nat Sci 2015 ; 1 ( 3 ): e57 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 49. Stockwell MS , Catallozzi M, Camargo S et al. Registry-linked electronic influenza vaccine provider reminders: a cluster-crossover trial . Pediatrics 2015 ; 135 ( 1 ): e75 – 82 . Google Scholar Crossref Search ADS PubMed WorldCat 50. Sapir T , Moreo K, Carter JD et al. Continuing medical education improves gastroenterologists’ compliance with inflammatory bowel disease quality measures . Dig Dis Sci 2016 ; 61 ( 7 ): 1862 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat 51. Luder H , Frede S. Community pharmacist collaboration with a patient-centered medical home: establishment of a patient-centered medical neighborhood and payment model . J Am Pharm Assoc 2016 ; 56 ( 3 ): e18 – 9 . Google Scholar OpenURL Placeholder Text WorldCat 52. Zwar NA , Bunker JM, Reddel HK et al. Early intervention for chronic obstructive pulmonary disease by practice nurse and GP teams: a cluster randomized trial . Fam Pract 2016 ; 33 ( 6 ): 663 – 70 . Google Scholar Crossref Search ADS PubMed WorldCat 53. Patel MS , Volpp KG, Small DS et al. Using active choice within the electronic health record to increase influenza vaccination rates . J Gen Intern Med 2017 ; 32 ( 7 ): 790 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 54. Real FJ , DeBlasio D, Beck AF et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal . Acad Pediatr 2017 ; 17 ( 4 ): 431 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 55. Lin JL , Bacci JL, Reynolds MJ et al. Comparison of two training methods in community pharmacy: project VACCINATE . J Am Pharm Assoc (2003) 2018 ; 58 ( 4S ): S94 – 100.e3 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 56. Parker PA , Banerjee SC, Matasar MJ et al. Efficacy of a survivorship-focused consultation versus a time-controlled rehabilitation consultation in patients with lymphoma: a cluster randomized controlled trial . Cancer 2018 ; 124 ( 23 ): 4567 – 76 . Google Scholar Crossref Search ADS PubMed WorldCat 57. Whitaker JA , Poland CM, Beckman TJ et al. Immunization education for internal medicine residents: a cluster-randomized controlled trial . Vaccine 2018 ; 36 ( 14 ): 1823 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat 58. Feuerstein JD , Papamichael K, Popejoy S et al. Targeted physician education and standardizing documentation improves documented reporting with inflammatory bowel disease quality measures in a large academic and private practice . Dig Dis Sci 2018 ; 63 ( 1 ): 36 – 45 . Google Scholar Crossref Search ADS PubMed WorldCat 59. Giduthuri JG , Purohit V, Maire N et al. Influenza vaccination of pregnant women: engaging clinicians to reduce missed opportunities for vaccination . Vaccine 2019 ; 37 ( 14 ): 1910 – 7 . Google Scholar Crossref Search ADS PubMed WorldCat 60. Aminrad Z , Zarina S, Sayed B et al. Relationship between awareness, knowledge and attitudes towards environmental education among secondary school students in Malaysia . World Appl Sci 2013 ; 22 ( 9 ): 1326 – 33 . Google Scholar OpenURL Placeholder Text WorldCat 61. Ho HJ , Chan YY, Ibrahim MAB et al. A formative research-guided educational intervention to improve the knowledge and attitudes of seniors towards influenza and pneumococcal vaccinations . Vaccine 2017 ; 35 ( 47 ): 6367 – 74 . Google Scholar Crossref Search ADS PubMed WorldCat 62. Rose KA , Ayad S. Factors associated with changes in knowledge and attitude towards public health concepts among chiropractic college students enrolled in a community health class . J Chiropr Educ 2008 ; 22 ( 2 ): 127 – 37 . Google Scholar Crossref Search ADS PubMed WorldCat 63. Centers for Disease Control and Prevention . Epidemiology and Prevention of Vaccine—Preventable Diseases: Immunization Strategies for Healthcare Practices and Providers . 13th edn. Washington, DC : Public Health Foundation , 2015 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 64. Donato AA , Motz LM, Wilson G, Lloyd BJ. Efficacy of multiple influenza vaccine delivery systems in a single facility . Infect Control Hosp Epidemiol 2007 ; 28 ( 2 ): 219 – 21 . Google Scholar Crossref Search ADS PubMed WorldCat 65. Rees S , Stevens L, Drayton J, Engledow N, Sanders J. Improving inpatient pneumococcal and influenza vaccination rates . J Nurs Care Qual 2011 ; 26 ( 4 ): 358 – 63 . Google Scholar Crossref Search ADS PubMed WorldCat 66. Coyle CM , Currie BP. Improving the rates of inpatient pneumococcal vaccination: impact of standing orders versus computerized reminders to physicians . Infect Control Hosp Epidemiol 2004 ; 25 ( 11 ): 904 – 7 . Google Scholar Crossref Search ADS PubMed WorldCat 67. Shojania KG , Jennings A, Mayhew A et al. Effect of point-of-care computer reminders on physician behaviour: a systematic review . CMAJ 2010 ; 182 ( 5 ): E216 – 25 . Google Scholar Crossref Search ADS PubMed WorldCat 68. Fiks AG , Hunter KF, Localio AR et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma . Pediatrics 2009 ; 124 ( 1 ): 159 – 69 . Google Scholar Crossref Search ADS PubMed WorldCat 69. Chambers CV , Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center . J Am Board Fam Pract 1991 ; 4 ( 1 ): 19 – 26 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 70. Tierney WM , Overhage JM, Murray MD et al. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial . Health Serv Res 2005 ; 40 ( 2 ): 477 – 97 . Google Scholar Crossref Search ADS PubMed WorldCat © The Author(s) 2021. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial reproduction and distribution of the work, in any medium, provided the original work is not altered or transformed in any way, and that the work is properly cited. For commercial re-use, please contact journals.permissions@oup.com © The Author(s) 2021. Published by Oxford University Press. http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Family Practice Oxford University Press

Interventions on health care providers to improve seasonal influenza vaccination rates among patients: a systematic review and meta-analysis of the evidence since 2000

Download PDF
13 pages

Loading next page...
 
/lp/oxford-university-press/interventions-on-health-care-providers-to-improve-seasonal-influenza-hoV3RUkhra

References (149)

Publisher
Oxford University Press
Copyright
Copyright © 2022 Oxford University Press
ISSN
0263-2136
eISSN
1460-2229
DOI
10.1093/fampra/cmaa149
Publisher site
See Article on Publisher Site

Abstract

Abstract Background Seasonal influenza vaccination (SIV) rates remain suboptimal in many populations, even in those with universal SIV. Objective To summarize the evidence on interventions on health care providers (physicians/nurses/pharmacists) to increase SIV rates. Methods We systematically searched/selected full-text English publications from January 2000 to July 2019 (PROSPERO-CRD42019147199). Our outcome was the difference in SIV rates between patients in intervention and non-intervention groups. We calculated pooled difference using an inverse variance, random-effects model. Results We included 39 studies from 8370 retrieved citations. Compared with no intervention, team-based training/education of physicians significantly increased SIV rates in adult patients: 20.1% [7.5–32.7%; I2 = 0%; two randomized controlled trials (RCTs)] and 13.4% [8.6–18.1%; I2 = 0%; two non-randomized intervention studies (NRS)]. A smaller increase was observed in paediatric patients: 7% (0.1–14%; I2 = 0%; two NRS), and in adult patients with team-based training/education of physicians and nurses together: 0.9% (0.2–1.5%; I2 = 30.6%; four NRS). One-off provision of guidelines/information to physicians, and to both physicians and nurses, increased SIV rates in adult patients: 23.8% (15.7–31.8%; I2 = 45.8%; three NRS) and paediatric patients: 24% (8.1–39.9%; I2 = 0%; two NRS), respectively. Use of reminders (prompts) by physicians and nurses slightly increased SIV rates in paediatric patients: 2.3% (0.5–4.2%; I2 = 0%; two RCTs). A larger increase was observed in adult patients: 18.5% (14.8–22.1%; I2 = 0%; two NRS). Evidence from both RCTs and NRS showed significant increases in SIV rates with varied combinations of interventions. Conclusions Limited evidence suggests various forms of physicians’ and nurses’ education and use of reminders may be effective for increasing SIV rates among patients. Health personnel, human influenza, intervention, meta-analysis, systematic review, vaccination Key messages We reviewed 39 studies [7 RCTs and 32 non-randomized intervention studies (NRS)]. Physicians/nurses’ education/reminders increased vaccination among patients. Combinations of education and reminders increased vaccination among patients. Education and reminders were both effective among adult and paediatric patients. Introduction Vaccination is one of the most successful interventions against vaccine-preventable diseases and is considered to be the most effective and practical strategy for influenza prevention. Advisory bodies such as the World Health Organization (WHO) (1), the European Centre for Disease Prevention and Control (ECDC) (2) and the advisory committees on immunization in the United States of America (USA) (ACIP) (3) and Canada (NACI) (4) recommend annual seasonal influenza vaccination (SIV) for individuals 6 months of age and above, with emphasis on subpopulations that are at increased risk of developing influenza-related complications such as infants, older adults, pregnant women and persons with certain morbidities. Consequently, many countries have annual SIV programmes (5). Despite these programmes, SIV rates remain below national and WHO targets, even in jurisdictions (health regions) with universal (free-for-all) SIV programmes. Refusal or a delay in acceptance of a vaccine despite its availability and accessibility (vaccine hesitancy) (6) varies across time and place, and may explain the suboptimal SIV rates. This phenomenon is thought to be influenced by an individual’s complacency (believing disease is not serious or that vaccination is not necessary), convenience (vaccine availability, accessibility and affordability) and confidence (belief that vaccine works and is safe) in regard to influenza vaccination (6). However, health care providers (physicians, nurses, pharmacists, etc.) have been identified as the most important advocates and the main source of vaccination information for the public, and therefore could facilitate vaccine uptake (7–9). There have been a few published systematic reviews of interventions focussed on health care providers to increase SIV rates among patients. A Cochrane review focussed only on older community-based patients in high-income countries (10). Other reviews were limited to community-based adult patients (11), or specific patient subpopulations and were mainly narrative without meta-analysis (12,13). These reviews did not assess if any differences existed in the efficacy/effectiveness of interventions across patient populations and study types, and across subtypes of an intervention. To address these gaps in knowledge, we summarized the evidence to determine if the efficacy/effectiveness of the interventions vary by study type, health care provider group, patient population and jurisdiction. Methods We conducted a systematic review and meta-analysis of the available evidence, in accordance with the Methodological Expectations of the Cochrane Intervention Reviews (MECIR) guidelines (14), and reported in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines (15). We developed and registered a protocol for the review in the International Prospective Register of Systematic Reviews (CRD42019147199) prior to commencement of the review. We included published studies of interventions on health care providers aimed at improving SIV rates. Only studies conducted from the year 2000 onwards, with a full-text publication in English, were included. We excluded mathematical modelling studies and grey literature. Search strategy and selection criteria A knowledge synthesis librarian designed a literature search strategy for Medline (Ovid) (Supplementary Table 1). The search strategy was peer-reviewed by a second, independent professional knowledge synthesis librarian using the PRESS checklist (16). The final search strategy was then adapted for EMBASE (Ovid), CINAHL (EBSCO) and Scopus (Web) bibliographic databases, and executed. All literature searches were conducted on 24 July 2019. Identified citations from the executed searches were imported into a specially designed Microsoft Access 2016 database (Microsoft Corporation, Redmond, WA), and screened for eligibility by two independent systematic reviewers using a two-stage sifting approach to review the title/abstract and full-text article. The number of ineligible citations at the title/abstract screening stage was recorded, and both the number and reasons for ineligibility were recorded at the full-text article screening stage. Disagreements during these screening stages were resolved by discussion between the two systematic reviewers, and a third systematic reviewer adjudicated when necessary. Data extraction Data extraction was conducted using Microsoft Excel 2016 (Microsoft Corporation, Redmond, WA) by one reviewer after a pilot was conducted on a small selection of the included studies by the reviewer and another independent reviewer. A second reviewer independently checked the extracted data for errors. Disagreements were resolved by discussion between the two reviewers, and a third reviewer adjudicated when necessary. We extracted data such as study first author’s name, duration, publication year, funder, patient population, health care provider information, intervention(s) assessed, and the reported difference in SIV rates between patients in health care provider intervention and control groups or between patients before and after application of a health care provider intervention. The associated 95% confidence interval (CI) of the reported differences was also extracted. In addition, we extracted details relevant to risk of bias or study quality assessment. If a study also administered patient-focussed interventions, it was only included if patients in both the health care provider intervention and control groups (or patients before and during the health care provider intervention) received the same exact patient intervention(s). We did not include studies in which patients in just one of the compared groups (or intervention periods) received patient-focussed intervention(s). Data synthesis and analysis We summarized study characteristics and pooled reported differences in SIV rates between intervention and no intervention groups, and associated 95% CIs using an inverse variance, random-effects model implemented in STATA (version 13; StataCorp LP, College Station, TX). Educational interventions were categorized as either one-off provision of guidelines/information or team-based training/education sessions. If a study reported differences in SIV rates by population subgroups, we first pooled the differences before pooling the pooled estimate with appropriate differences in SIV rates from other studies. Statistical heterogeneity between pooled differences in SIV rates was assessed and quantified using the I-squared statistic (I2) (17). Where appropriate, publication bias was assessed visually using funnel plots, and statistically using Egger’s regression test (18). We conducted subgroup analyses by study type, intervention, health care provider and patient population (adults versus paediatrics). Risk of bias and study quality assessments One reviewer assessed risk of bias using the Cochrane risk of bias assessment tool for randomized controlled trials (RCTs) (we also used the tool for a few non-/quasi-RCTs by adding a confounder adjustment domain) (19). The reviewer also assessed study quality of ‘before-and-after intervention studies’ (where outcome assessment is made before and after the introduction of an intervention) using the National Institutes of Health (NIH) quality assessment tool for this study type (20). Another reviewer independently checked the assessments for errors. Disagreements were resolved by discussion between the two reviewers, and a third reviewer adjudicated when necessary. The Cochrane risk of bias assessment tool considers 6 main domains of potential biases, while the NIH quality assessment tool considers 12 domains of study quality assessment. If there was a high risk of bias in randomization (sequence generation domain assessment), a study was automatically judged to be at a high risk of bias. A study was also judged to be at a high risk of bias if any one of the other domains were judged to be of high risk, and an additional domain judged as having an unclear or high risk of bias. A study was judged as having a low risk of bias if all six domains were judged to be of low risk of bias, or if just one domain (not sequence generation) was judged as having an unclear risk of bias. A study was judged to be of good quality if all 12 domains were judged not to be lacking, or if just one domain (not pre-specified and uniformly measured outcome domain) was judged to be lacking. A moderate quality was assigned if two to four domains were judged to be lacking, while a low quality rating was given if five or more domains were judged to be lacking. Results Thirty-nine studies met our eligibility criteria, from a total of 8370 identified unique citations (Fig. 1) (21–59). Most of the studies (n = 28) were conducted in the USA (Table 1). The rest were conducted mostly in developed countries: two studies each for the UK and Australia, and one study each for UK/Australia, Canada, Denmark, Netherlands, Switzerland, India and Turkey. There were 7 RCTs (3 parallel and 4 cluster randomized) (22,28,47,49,52,56,59) and 32 non-randomized intervention studies (NRS) (2 non-RCTs (33,39), 1 quasi-RCT (54) and 29 before-and-after intervention studies). A total of 20 studies were focussed on physicians, while a smaller number of studies examined interventions targeting nurses (2 studies), pharmacists (1 study), both physicians and nurses (14 studies), both physicians and pharmacists (1 study) and both physicians separately and physicians and nurses together (1 study). Eleven studies examined the use of team-based training/education sessions, and six the use of one-off provision of guidelines/information. Eleven studies examined the use of reminders (nine using electronic prompts, one using paper-based prompt and one using letters), two studies examined the use of incentives (pay-for-performance) whereas nine studies examined the use of multiple interventions. There were five studies each on children/adolescents, pregnant women, adults and older adults, irrespective of health status, and three studies each specifically on chronic obstructive pulmonary disease, and inflammatory bowel disease patients. There were two studies each on adult diabetic and rheumatology patients, and one study each on preterm infants, paediatric asthma, respiratory tract infection, musculoskeletal/neurological patients, coronary heart disease, chronic kidney disease, Hodgkin’s lymphoma patients and patients with a risk of cardiovascular disease. One study was on a mixed patient population. The health care provider sample size among the studies that clearly reported this parameter ranged from 4 to 136 individuals. Six studies were funded by industry (pharmaceutical companies), while non-industry funding was reported by 25 studies. One study was funded by both industry and non-industry sources, and seven studies did not report a funding source. Nearly all RCTs and non-/quasi-RCTs were judged to have a high risk of bias with regard to lack of blinding of participants and study personnel. A majority of the before-and-after intervention studies were deficient with regard to study participants being a good representation of the studied population, having sufficiently large sample size, and blinding of outcome assessors. Overall, the included RCTs and non-/quasi-RCTs were judged to be at a moderate to high risk of bias (Supplementary Table 2), and the before-and-after intervention studies were judged to be of moderate to low quality (Supplementary Table 3). Figure 1. Open in new tabDownload slide Summary of literature search and screening process (modified PRISMA flow chart). Figure 1. Open in new tabDownload slide Summary of literature search and screening process (modified PRISMA flow chart). Table 1. Summary of study characteristics Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel a,b,cRepresent different cohorts (study participants) in the studies. EU, European Union; GSA, Gastroenterological Society of Australia; IBD, inflammatory bowel disease; MRC, Medical Research Council; NHS, National Health Service; NR, not reported. Open in new tab Table 1. Summary of study characteristics Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel Study (Country) . Funder (Study period) . Study type . Study unit (Number of participants) . Patients . Provider type (Male %) . Interventions (Types) . Intervention(s) description . Gill (21) (USA) Delaware Foundation for Medical Services and DuPont (1997/1998) Before-and-after intervention study Health care provider (31) Elderly (≥65 years) Physicians (NR) Reminders (Prompts) A fully automated office medical record with a computerized physician reminder Chan (22) (USA) NR (1997/1998) Parallel RCT Health care provider (44)a and (13)b Musculoskeletal or neurologic issues Physicians (NR) Reminders (Letters) Four separate monthly mailed physician reminders to immunize their patients during the influenza season Gill (23) (USA) NR (2001/2002) Before-and-after intervention study Health care provider (28) Adult diabetics Physicians (NR) Education (One-off provision of guidelines/information) Provision and explanation of clinical guidelines to doctors Hutt (24) (USA) Department of Veterans Affairs (2004/2005) Before-and-after intervention study Health care provider (40) Lower respiratory tract infection (veterans) Physicians (NR) Education and reminders (Team-based training/ education sessions and reminder) A prototype educational program with case studies, provision of selected materials and reprints to physicians including annotated copy of the guidelines, meetings to help educate physicians, posters and cards Ogburn (25) (USA) NR (2003/2004) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education (Team-based training/ education sessions) Educational sessions for providers and clinic personnel, and a protocol for nurses to screen patients for vaccination Weber (26) (USA) NR (2005/2007) Before-and-after intervention study Health care provider (136) Adults Physicians (NR) Incentives and reminders (Pay-for-performance and prompts) Electronic health maintenance and best practice alert to draw physician’s attention that an item was needed, and physician bonuses if the practices met a criteria McGovern (27) (UK) Scottish Executive Health Department, University of Aberdeen (2004/2005) Before-and-after intervention study General practice or hospital (310) Coronary heart disease (adults) Physicians (NR) Incentives (Pay-for-performance) General Medical Services contract Hogg (28) (Canada) Canadian Institutes of Health Research (CIHR) (NR) Cluster RCT General practice or hospital (54) Adults Physicians and nurses (66.8) Education and reminders (Team-based training/ education sessions and prompts) A nurse-led preventative educational outreach and training sessions on use of medical office electronic computer systems Ledwich (29) (USA) NR (2006/2007) Before-and-after intervention study General practice or hospital (2) Adult rheumatology patients taking immunosuppressive drugs Physicians and nurses (NR) Reminders (Prompts) An electronic health record best practice alert clinical reminder Snow (30) (USA) Pfizer (2006) Before-and-after intervention study Health care provider (54) Increased risk of cardiovascular disease Physicians and nurses (NR) Education (Team-based training/ education sessions) Three training sessions and two conference calls for a team of one physician, one nurse or an allied health professional over a 6-month period Loo (31) (USA) Donald W. Reynolds Foundation (2009/2010) Before-and-after intervention study Health care provider (17) Elderly (≥65 years) Physicians (52.9) Reminders (Prompts) A set of electronic medical records and provider’s panel list for patients reminders Nace (32) (USA) NR (2002/2003) Before-and-after intervention study General practice or hospital (3) Elderly in long-term care (≥65 years) Nurses (NR) Education (One-off provision of guidelines/information) A written educational toolkit and shared information through email and a single half-day collaborative training session Dapp (33) (Switzerland) EU, Federal Ministry, Bern, Switzerland, Federal Ministry, Berlin, Germany, Max and Ingeburg & Heart Robert Bosch Foundations (NR) Non-RCT General practice or hospital (14) Elderly (≥60 years) Physicians (NR) Education (Team-based training/ education sessions) Bimonthly 2-hour training sessions for general practitioners led by an experienced geriatrician using a newly developed evidence-based manual with guidance notes Patwardhan (34) (USA) NR (2007/2010) Before-and-after intervention study Health care provider (6) Rheumatic disease (paediatric) Physicians and nurses (NR) Reminder (Prompt) An electronic medical record reminder during inpatient visit and relatively early on in outpatient encounter Sherman (35) (USA) NR (2003/2005) Before-and-after intervention study Health care provider (NR) Pregnant women Physicians and nurses (NR) Education and reminders (One-off provision of guidelines/information, and reminder) Education of staff (knowledge base discussions) and reminder (prompts) system containing a simple outline of recommendations for vaccination, answers to common questions and a list of high-risk groups Klatt (36) (USA) NR (2008/2009) Before-and-after intervention study Health care provider (22) Pregnant women Physicians and nurses (NR) Reminders (Prompts) An electronic reminder within patient electronic prenatal record Kirschner (37) (Netherlands) CZ and VGZ Health Insurance Companies (NR) Before-and-after intervention study General practice or hospital (60) High risk and elderly (≥65 years) Physicians (NR) Incentives (Pay-for-performance) Practices received a bonus according to their performance, based on comparing their quality indicator scores of clinical care, practice management and patient experience Satman (38) (Turkey) Sanofi-Pasteur (2010/2011) Before-and-after intervention study Health care provider (44) Adult diabetics Physicians (NR) Education (Team-based training/ education sessions) A physician training program designed in accordance with the immunization recommendations of clinical practice guidelines on diabetes and its complications Ulrik (39) (Denmark) GlaxoSmithKline (GSK) (2008, 2009, 2010) Non-RCT General practice or hospital (202) Adults COPD (≥40 years) Physicians and nurses (NR) Education (Team-based training/ education sessions) Three-hour teaching lesson to GPs conducted by a specialist in respiratory medicine. Five training visits by a GSK representative. Written educational material to GPs and their staff Walsh (40) (Australia and UK) NHS, MRC, National Institute of Clinical Studies, GSA (2010) Before-and-after intervention study Health care provider (30) Adult IBD patients Physicians (NR) Education and reminders (One-off provision of guidelines/information, and reminder) A copy of clinical practice guidelines was sent to physicians. Reminder emails were also sent to them on a regular basis McCreary (41) (USA) NR (2011/2012) Before-and-after intervention study Health care provider (11) Asthmatics (paediatric) Physicians and nurses (NR) Education (One-off provision of guidelines/information) Educational session during a free luncheon in which a 30-minute lecture with slide presentation was offered Pollack (42) (USA) National Institutes of Health (NIH) (2003/2012) Before-and-after intervention study Health care provider (NR) Children Nurses (NR) Reminders (Prompts) An electronic device for screening of patients at the time of admission or transfer and automatically placing order for vaccination without requiring direct intervention from a physician Hull (43) (UK) Tower Hamlets Primary Care Trust, UK (2010/2013) Before-and-after intervention study General practice or hospital (8) Adult COPD Physicians and nurses (NR) Education (Team-based training/ education sessions) Training and multidisciplinary team meetings Mendu (44) (USA) NR (2012/2013) Non-RCT Health care provider (4) Chronic kidney disease Physicians (0) Education and reminders (One-off provision of guidelines/information, and reminder) Education materials provided to physicians and a 30-minute lecture and email reminders Vernacchio (45) (USA) Paediatric Physicians’ Organization (2009/2010) Before-and-after intervention study Health care provider (15)a, (24)b and (17)c Children and adolescents Physicians (46.7)a, (37.5)b and (29.4)c Education (Team-based training/education sessions) Two-hour face-to-face meetings using Institute for Healthcare Improvement’s learning collaborative model, a longitudinal learning curriculum and monthly communication via email or telephone Crawford (46) (Australia) NR (2007–11) Before-and-after intervention study Health care provider (NR) Preterm infants Physicians and nurses (NR) Education (One-off provision of guidelines/information) Education of health care professionals regarding immunization of preterm infants and the scientific evidence behind the current clinical guidelines Szilagyi (47) (USA) CDC (2011/2013) Parallel RCT General practice or hospital (10)a and (12)b Adolescents Physicians and nurses (NR) Reminders (Prompts) Educational sessions and one or two 1-hour training sessions on use of electronic health records prompts Pierson (48) (USA) Indiana University CPTRM and NIH-NIGMS (2011/2012) Before-and-after intervention study Health care provider (NR) Pregnant women (underserved and needy) Physicians (NR) Reminders (Prompts) A simple paper-based prompt attached to the front of a patient’s chart during the check-in process to record whether patient had been previously vaccinated or not, and patient desire for vaccination Stockwell (49) (USA) Agency for Healthcare Research and Quality (2011/2012) Cluster RCT General practice or hospital (4) Children and adolescents Physicians and nurses (NR) Reminders (Prompts) A short in-person in-service as well as visual aids training on an automatic electronic reminder display Sapir (50) (USA) AbbVie Inc. and Takeda Pharmaceuticals USA Inc. (2013/2013) Before-and-after intervention study Health care provider (20) Adult with ulcerative colitis Physicians (70) Education (Team-based training/education sessions) A private audit feedback session through web conferencing, a total of five webinars, series of 30- to 60-minute interactive videos online/mobile toolkit that included accredited CME activities Luder (51) (USA) Community Pharmacy Foundation (2013/2014) Before-and-after intervention study Health care provider (12) Adult diabetic or hypertensive or dyslipidaemia or three or more chronic conditions Physicians and pharmacists (NR) Education and incentives (Team-based training/education sessions, and fee-for-service) Education and capitated payment per patient per month for a predetermined number of 1000 high-risk patients Zwar (52) (Australia) Australian National Health and MRC Cluster RCT General practice or hospital (36) Middle age to elderly adult smokers with risk factors for COPD Physicians and nurses (NR) Education and reminders (Team-based training/education sessions and prompts) Computer-based distance learning for physicians, a 3-hour combined workshop for physicians and nurses, provision of copy of national guidelines and computerized clinical record and care planning systems Patel (53) (USA) University of Pennsylvania Leonard Davis Institute of Health Economics and NIA (2010/2013) Before-and-after intervention study General practice or hospital (3) Adults Physicians (NR) Reminders (Prompts) An automated electronic health records alert at the time of patient check-in to medical assistants who could pend orders for the physician to review and potentially sign. Regardless, physicians also received the alert when first opening a patient’s chart Real (54) (USA) Health Resources and Services Admin. (2015/2016) Quasi-RCT Health care provider (45) <5 years old Physicians (28.89) Education (Team-based training/education sessions) Three immersive virtual reality simulations to teach communication skills when discussing influenza vaccine hesitancy Lin (55) (USA) None (2016) Before-and-after intervention study Health care provider (19) Adults Pharmacists (NR) Education and reminders (Team-based training/education sessions and reminder) An in-person 2-hour training session, and discussions, and an electronic platform reminder Parker (56) (USA) National Cancer Institute and National Institutes of Health (2011/2019) [study on-going] Cluster RCT General practice or hospital (4) Hodgkin’s lymphoma or diffuse large B cell lymphoma in remission Physicians (67) Education (Team-based training/education sessions) Five hours of in-person communication skills training, which included role playing, and discussions Whitaker (57) (USA) Pfizer (2014/2015) Before-and-after intervention study Health care provider (47)a and (52)b Adults Physicians (63) Education (One-off provision of guidelines/information) Power-Point presentation focussed on Advisory Committee on Immunization Practices (ACIP) guidelines for influenza immunizations administered over 60 minutes Feuerstein (58) (USA) The Hellenic Group for the Study of Inflammatory Bowel Disease (NR) Before-and-after intervention study Health care provider (30)a and (16)b Adult IBD patients Physicians (NR)a Physicians and nurses (NR)b Education (One-off provision of guidelines/information) Emailed copy of quality measures, and handout of quality measures placed in clinic room Giduthuri (59) (India) WHO, CDC, Swiss Government Excellence Fellowship (2015/2016) Parallel RCT Health care provider (19)a and (18)b Pregnant women (third trimester) Physicians (7.14)a and (43.75)b Education (Team-based training/education sessions) Discussions with physicians of WHO recommendations on vaccination, a review article and a policy report of an expert panel a,b,cRepresent different cohorts (study participants) in the studies. EU, European Union; GSA, Gastroenterological Society of Australia; IBD, inflammatory bowel disease; MRC, Medical Research Council; NHS, National Health Service; NR, not reported. Open in new tab Pooled difference in SIV rates for education compared with no intervention Two RCTs (56,59) and 15 NRS (23,25,30,32,33,38,39,41,43,45,46,50,54,57,58) contributed to the pooled estimates. Evidence from RCTs showed that team-based training/education of physicians significantly increased SIV rates in adult patients: 20.1% (95% CI 7.5–32.7%; I2 = 0%; two studies) (Fig. 2). Evidence from NRS also showed that team-based training/education of physicians significantly increased SIV rates in adult patients: 21.6% (3.4–39.9%; I2 = 97.5%; three studies) and paediatric patients: 7% (0.1–14%; I2 = 0%; two studies) (Supplementary Fig. 1). The high heterogeneity observed in adult patients was due to higher estimates from Turkey, a country with a dissimilar health system from the other countries (Switzerland and USA). Exclusion of the result from Turkey significantly reduced the high heterogeneity observed in the pooled estimates (13.4%, 8.6–18.1; I2 = 0%). The evidence from NRS also showed that education of physicians and nurses marginally (though significantly) increased SIV rates in adult patients: 0.9% (0.2–1.5%; I2 = 30.6%; four studies) (Supplementary Fig. 1). Similarly, evidence from NRS showed that one-off provision of guidelines/information to physicians, and to both physicians and nurses, significantly increased SIV rates in adult patients: 23.8% (15.7–31.8%; I2 = 45.8%; three studies) and paediatric patients: 24% (8.1–39.9%; I2 = 0%; two studies), respectively (Supplementary Fig. 2). One study focussed on nurses and adult patients, and the study reported no significant benefit. Figure 2. Open in new tabDownload slide Forest plot of team-based training/education of physicians compared with no intervention, in adult patients (RCTs). Figure 2. Open in new tabDownload slide Forest plot of team-based training/education of physicians compared with no intervention, in adult patients (RCTs). Pooled difference in SIV rates for reminders compared with no intervention Three RCTs (22,47,49) and seven out of eight NRS (21,29,34,36,42,48,53) contributed to the pooled estimates. One NRS was excluded from the meta-analysis because the authors reported that there was vaccine shortage in the study year compared with the preceding year (the comparator), and as a result, many patients who requested vaccination could not be vaccinated (31). Result from this study differed significantly from the rest. Evidence from RCTs showed that use of reminders (prompts) by physicians and nurses significantly increased SIV rates in paediatrics patients by 2.3% (0.5–4.2%; I2 = 0%; two studies) (Fig. 3). There was one study on the use of letter reminders among physicians in adult patients, and the study reported no significant benefit. Evidence from NRS showed that physicians and both physicians’ and nurses’ use of reminders (prompts) significantly increased SIV rates in adult patients: 15.3% (7.4–23.2%; I2 = 98.1%; three studies) and 18.5% (14.8–22.1%; I2 = 0%; two studies), respectively (Supplementary Fig. 3). There were no major differences between the studies among physicians that could explain the high heterogeneity in the pooled estimates. However, all pooled results were in favour of the intervention. There was one study each among nurses alone and physicians and nurses together in paediatric patients and both were in favour of the intervention. Figure 3. Open in new tabDownload slide Forest plot of reminders (prompts) compared with no intervention (RCTs). Figure 3. Open in new tabDownload slide Forest plot of reminders (prompts) compared with no intervention (RCTs). Pooled difference in SIV rates for incentives compared with no intervention One NRS each examined the impact of physicians’ pay-for-performance (27), and physicians’ practices performance bonus (37) on SIV rates in adult patients, and found significantly increased SIV rate for pay-for-performance: 28.1% (1.3% to 54.9%), and no difference for performance bonus: −0.1% (−42.1% to 42%), respectively. Pooled difference in SIV rates for combined interventions compared with no intervention Two RCTs (28,52) and two out of five NRS (40,44) contributed to the pooled estimates. Evidence from RCTs showed that team-based training/education of physicians and nurses, together with use of reminders, significantly increased SIV rates in adult patients: 10.8% (3.3–18.2%; I2 = 0%; two studies) (Fig. 4). Evidence from NRS showed that one-off provision of guidelines/information to physicians together with and use of reminders significantly increased SIV rates in adult patients: 19.5% (14.9–24.1%; I2 = 0%; two studies) (Supplementary Fig. 4). One study among physicians and nurses in adult patients was also in favour of the interventions combination (Supplementary Fig. 4) (35). One study each reported on team-based training/education of physicians and of pharmacists together with the use of reminders in adult patients and both studies reported no difference: 10.9% (−3.2% to 26.5%) and 24% (−0.0% to 48%), respectively. One study each also reported on physician pay-for-performance together with reminders (26), and team-based training/education together with fee-for-service for physicians and for pharmacists and physicians (51), all in adult patients. All three were in favour of the intervention combinations: 15.9% (7.9–23.8%), 15% (5.6–24%) and 3.7% (2.2–5.2%), respectively. Figure 4. Open in new tabDownload slide Forest plot of combined team-based training/education of physicians and nurses, and reminders (prompts) compared with no intervention, in adult patients (RCTs). Figure 4. Open in new tabDownload slide Forest plot of combined team-based training/education of physicians and nurses, and reminders (prompts) compared with no intervention, in adult patients (RCTs). Discussion This review indicates that team-based training/education and one-off provision of guidelines/information to physicians and nurses may be effective in increasing SIV rates in adult and paediatric patient populations. The review also indicates that use of reminder prompts by physicians and nurses may be effective in increasing SIV rates in adult and patient populations, and that various combinations of educational and reminder interventions may also be effective in increasing SIV rates. The evidence supporting these findings are, however, limited and are of moderate to low quality. While patient population types varied considerably across the included studies, almost all belonged to the high-risk groups for whom SIV is highly recommended. Physician specialty also varied slightly across studies. All but one study were conducted in developed countries with similar health system characteristics, although there would have been potentially differing public health practices across the countries with respect to influenza vaccination programs. There were substantial variabilities in the number of studied health care providers and patients, and the forms, methods of delivery and durations of an intervention. Furthermore, a majority of the studies were conducted in the USA; thus, our findings may not be generalizable and may be more applicable to the USA. That said, our findings are substantial contributions to the evidence base and will help in the development and evaluation of health care provider-focussed interventions to increase seasonal influenza vaccine uptake and, potentially, uptake of other vaccines across important patient populations. SIV is proven to be a safe, effective and cost-effective preventive measure against influenza, but annual rates remain suboptimal in many countries even among the most at-risk populations. Vaccination behaviour is influenced by so many factors including perceptions of vaccines and vaccination, and intrinsic factors such as an individual’s knowledge, perceptions and attitudes which are often shaped by many external factors. Despite immense influence and importance of the media, health care professionals have been identified as the most important advocates and the main source of vaccination information for the public (7). The summit of independent European vaccination experts in 2007 concluded that it is of utmost importance that health care providers are offered adequate education on vaccinology and an unhindered access to up-to-date information on vaccines (7). The experts also concluded that it is important that vaccine information systems are set up for health care providers, to facilitate promotion of vaccination. Education can play an important role in prompting positive change in behaviour (60,61). Increases in knowledge and awareness have been found to lead to changes in attitudes (62). Education of health care providers on specific health issues is therefore expected to help increase their knowledge and awareness and, ultimately, promote a positive change in behaviours. This may explain our finding that one-off provision of guidelines/information or team-based training/education of physicians and physicians together with nurses are effective in increasing SIV rates in both adult and paediatric patient populations. Contrary to our findings, a Cochrane review (Thomas and Lorenzetti) found that physician education was ineffective (10). The review included only RCTs focussing on SIV among community-based patients (≥60 years) in high-income countries, unlike our review which considered all available evidence irrespective of provider type, patient population and country of study. In addition, Thomas and Lorenzetti considered all educational interventions the same, whereas we appropriately categorized the interventions, considering that types and methods of education differ. An older systematic review (Lau) also found physician education to be ineffective (11). This review also focussed on community-based adults and, although the review considered both RCTs and non-RCTs, educational interventions were examined as one group without categorization. It was also not clear which studies contributed to the pooled analysis. Reminder systems serve as alerting tools. In clinical settings, they alert a health care provider to the key procedures and necessary preventive care that a patient may need; thus, informing of the importance of a needed care (63). The Centers for Disease Control and Prevention (CDC) stress that a reminder system helps to reduce missed opportunities for recommending and providing influenza vaccine to eligible patients when vaccine is available (63). Therefore, implementing a SIV reminder system for health care providers is expected to increase the vaccine recommendation and, consequently, vaccination rates as our review suggests. Implementing a reminder system for both physicians and nurses provides more than one level of quality assurance check, and decreases the risk of failing to provide an important recommendation to an eligible patient. This has been demonstrated by studies which have found that a reminder system for both physicians and nurses leads to increased vaccination uptake (64,65). Our findings are in line with those of some clinical studies that have shown that computerized reminders increase the use of preventive care in outpatient settings (66,67). However, some studies in inpatient settings using standard orders and computerized reminders failed to demonstrate the effectiveness of this intervention (67,68), probably due to a focus on treatment of the cause of hospitalization rather than the provision of preventive care. That said, as a study suggested, it is worth noting that reminder systems for physicians are effective only when applied to all eligible patients (69,70). Our decision to include only English publications may have limited the number of potentially relevant studies for inclusion. Limiting to studies since 2000 may have also limited the inclusion of studies, but this allowed us to focus on studies conducted during the period in which SIV became publicly funded in many jurisdictions. We were unable to determine if SIV was offered free-of-charge to patients or whether some of the patients had to pay for the vaccine out-of-pocket. As such, we could not examine the impact of free vaccination on the efficacy/effectiveness of the assessed interventions. Furthermore, nearly all reported SIV rates from the NRS were crude rates. For the before-and-after intervention studies, the populations at baseline were not always exactly the same as those post-intervention. Nevertheless, this review has many merits, including full compliance with the Cochrane Handbook for Systematic Reviews of Interventions guidelines, utilization of the expertise of knowledge synthesis librarians in developing and peer reviewing a comprehensive search strategy, and searching of appropriate bibliographic databases for literature. Where necessary, we requested additional data from corresponding authors of the included studies, to ensure completeness of the analysed data. We conducted the review to the expected standards and reported in accordance with the PRISMA guidelines. Conclusions Limited low- to moderate-quality evidence suggests that various forms of education for physicians and nurses, and use of reminder prompts by physicians and nurses, may be effective for increasing SIV rates in both adult and paediatric patient populations. High-quality studies are needed for a stronger evidence base to better inform and understand the efficacy/effectiveness of various interventions focussed on health care providers to increase SIV. Acknowledgements We thank Drs Thomas J Van Hoof, James M Gill, Ulrike Dapp, Louis Vernacchio, William Trick, Ari Pollack, Timothy Loo, David Nace, Valerie Weber, Eric Ruderman and Jan van Lieshout for responding kindly to our request for additional data on their publication(s). Declaration Funding: this work was supported by Sanofi Pasteur, USA (grant number not applicable). The funder contributed towards specifying the research questions but had no role in the study design, data collection and analysis. Ethical approval: not applicable. Conflict of interest: Salaheddin M Mahmud (SMM) has received unrestricted research grants from GlaxoSmithKline, Merck, Sanofi Pasteur, Pfizer and Roche-Assurex for unrelated studies, and fees as an advisory board member for Sanofi Pasteur. SMM is supported, in part, by funding from the Canada Research Chairs Program. E Thommes and A Chit are employees of Sanofi Pasteur, USA. The other authors have no conflicts of interest to declare. Data availability The data underlying this article are available in the article and in its online supplementary material. References 1. The World Health Organization Strategic Advisory Group of Experts (SAGE) Working Group . Background Paper on Influenza Vaccines and Immunization . Geneva, Switzerland : WHO , 2012 . https://www.who.int/influenza/vaccines/SAGE_information/en/ Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 2. European Centre for Disease Prevention and Control . Seasonal Influenza Vaccination and Antiviral Use in EU/EEA Member States: Overview of Vaccine Recommendations for 2017–2018 and Vaccination Coverage Rates for 2015–2016 and 2016–2017 Influenza Seasons . Stockholm, Sweden : ECDC , 2018 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 3. Grohskopf LA , Sokolow LZ, Broder KR et al. Prevention and control of seasonal influenza with vaccines: recommendations of the Advisory Committee on Immunization Practices—United States, 2017–18 influenza season . Morb Mortal Wkly Rep 2017 ; 66 ( 2 ): 1 – 20 . Google Scholar OpenURL Placeholder Text WorldCat 4. National Advisory Committee on Immunization (NACI) . Canadian Immunization Guide Chapter on Influenza and Statement on Seasonal Influenza Vaccine for 2017–2018 . Ottawa, Ontario, Canada : Public Health Agency of Canada , 2017 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 5. Ortiz JR , Perut M, Dumolard L et al. A global review of national influenza immunization policies: analysis of the 2014 WHO/UNICEF Joint Reporting Form on immunization . Vaccine 2016 ; 34 ( 45 ): 5400 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 6. MacDonald NE ; SAGE Working Group on Vaccine Hesitancy. Vaccine hesitancy: definition, scope and determinants . Vaccine 2015 ; 33 ( 34 ): 4161 – 4 . Google Scholar Crossref Search ADS PubMed WorldCat 7. Schmitt HJ , Booy R, Aston R et al. How to optimise the coverage rate of infant and adult immunisations in Europe . BMC Med 2007 ; 5 : 11 . Google Scholar Crossref Search ADS PubMed WorldCat 8. Smith PJ , Kennedy AM, Wooten K, Gust DA, Pickering LK. Association between health care providers’ influence on parents who have concerns about vaccine safety and vaccination coverage . Pediatrics 2006 ; 118 ( 5 ): e1287 – 92 . Google Scholar Crossref Search ADS PubMed WorldCat 9. Vorsters A , Bonanni P, Maltezou HC et al. The role of healthcare providers in HPV vaccination programs—a meeting report . Papillomavirus Res 2019 ; 8 : 100183 . Google Scholar Crossref Search ADS PubMed WorldCat 10. Thomas RE , Lorenzetti DL. Interventions to increase influenza vaccination rates of those 60 years and older in the community (review) . Cochrane Database Syst Rev 2018 ;( 5 ): CD005188 . Google Scholar OpenURL Placeholder Text WorldCat 11. Lau D , Hu J, Majumdar SR et al. Interventions to improve influenza and pneumococcal vaccination rates among community-dwelling adults: a systematic review and meta-analysis . Ann Fam Med 2012 ; 10 ( 6 ): 538 – 46 . Google Scholar Crossref Search ADS PubMed WorldCat 12. Sanftenberg L , Brombacher F, Schelling J, Klug SJ, Gensichen J. Increasing influenza vaccination rates in people with chronic illness . Dtsch Arztebl Int 2019 ; 116 ( 39 ): 645 – 52 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 13. Gosselin Boucher V , Colmegna I, Gemme C et al. Interventions to improve vaccine acceptance among rheumatoid arthritis patients: a systematic review . Clin Rheumatol 2019 ; 38 ( 6 ): 1537 – 44 . Google Scholar Crossref Search ADS PubMed WorldCat 14. Higgins JPT , Lasserson T, Chandler J, Tovey D, Churchill R. Methodological Expectations of Cochrane Intervention Reviews (MECIR) . London, UK : The Cochrane Collaboration , 2016 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 15. Moher D , Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . Int J Surg 2010 ; 8 ( 5 ): 336 – 41 . Google Scholar Crossref Search ADS PubMed WorldCat 16. McGowan J , Sampson M, Salzwedel DM et al. PRESS peer review of electronic search strategies: 2015 guideline statement . J Clin Epidemiol 2016 ; 75 : 40 – 6 . Google Scholar Crossref Search ADS PubMed WorldCat 17. Higgins JP , Thompson SG. Quantifying heterogeneity in a meta-analysis . Stat Med 2002 ; 21 ( 11 ): 1539 – 58 . Google Scholar Crossref Search ADS PubMed WorldCat 18. Sterne JA , Egger M, Smith GD. Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis . BMJ 2001 ; 323 ( 7304 ): 101 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 19. Higgins JPT , Altman DG, Gøtzsche PC et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials . BMJ 2011 ; 343 : d5928 . Google Scholar OpenURL Placeholder Text WorldCat 20. National Hearth L, and Blood Institute . Quality Assessment Tool for Before-After (Pre-Post) Studies with No Control Group . National Institutes of Health (NIH) . https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools. (accessed on 31 December 2019 ). Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 21. Gill JM , Saldarriaga AM. The impact of a computerized physician reminder and a mailed patient reminder on influenza immunizations for older patients . Del Med J 2000 ; 72 ( 10 ): 425 – 30 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 22. Chan L , MacLehose RF, Houck PM. Impact of physician reminders on the use of influenza vaccinations: a randomized trial . Arch Phys Med Rehabil 2002 ; 83 ( 3 ): 371 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 23. Gill JM , DiPrinzio MJ. The Medical Society of Delaware’s Uniform Clinical Guidelines for diabetes: did they have a positive impact on quality of diabetes care? Del Med J 2004 ; 76 ( 3 ): 111 – 22 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 24. Hutt E , Ruscin JM, Corbett K et al. A multifaceted intervention to implement guidelines improved treatment of nursing home-acquired pneumonia in a state veterans home . J Am Geriatr Soc 2006 ; 54 ( 11 ): 1694 – 700 . Google Scholar Crossref Search ADS PubMed WorldCat 25. Ogburn T , Espey EL, Contreras V, Arroyo P. Impact of clinic interventions on the rate of influenza vaccination in pregnant women . J Reprod Med 2007 ; 52 ( 9 ): 753 – 6 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 26. Weber V , Bloom F, Pierdon S, Wood C. Employing the electronic health record to improve diabetes care: a multifaceted intervention in an integrated delivery system . J Gen Intern Med 2008 ; 23 ( 4 ): 379 – 82 . Google Scholar Crossref Search ADS PubMed WorldCat 27. McGovern MP , Boroujerdi MA, Taylor MW et al. The effect of the UK incentive-based contract on the management of patients with coronary heart disease in primary care . Fam Pract 2008 ; 25 ( 1 ): 33 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat 28. Hogg W , Lemelin J, Graham ID et al. Improving prevention in primary care: evaluating the effectiveness of outreach facilitation . Fam Pract 2008 ; 25 ( 1 ): 40 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 29. Ledwich LJ , Harrington TM, Ayoub WT, Sartorius JA, Newman ED. Improved influenza and pneumococcal vaccination in rheumatology patients taking immunosuppressants using an electronic health record best practice alert . Arthritis Rheum 2009 ; 61 ( 11 ): 1505 – 10 . Google Scholar Crossref Search ADS PubMed WorldCat 30. Snow V , Reynolds CE, Bennett L et al. Closing the gap-cardiovascular risk and primary prevention: results from the American College of Physicians quality improvement program . Am J Med Qual 2010 ; 25 ( 4 ): 261 – 7 . Google Scholar Crossref Search ADS PubMed WorldCat 31. Loo TS , Davis RB, Lipsitz LA et al. Electronic medical record reminders and panel management to improve primary care of elderly patients . Arch Intern Med 2011 ; 171 ( 17 ): 1552 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 32. Nace DA , Perera S, Handler SM, Muder R, Hoffman EL. Increasing influenza and pneumococcal immunization rates in a nursing home network . J Am Med Dir Assoc 2011 ; 12 ( 9 ): 678 – 84 . Google Scholar Crossref Search ADS PubMed WorldCat 33. Dapp U , Anders JA, von Renteln-Kruse W et al. ; PRO-AGE Study Group . A randomized trial of effects of health risk appraisal combined with group sessions or home visits on preventive behaviors in older adults . J Gerontol A Biol Sci Med Sci 2011 ; 66 ( 5 ): 591 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 34. Patwardhan A , Kelleher K, Cunningham D, Menke J, Spencer C. The use of a mandatory best practice reminder in the electronic record improves influenza vaccination rate in a pediatric rheumatology clinic . Clin Govern 2011 ; 16 ( 4 ): 308 – 19 . Google Scholar Crossref Search ADS WorldCat 35. Sherman MJ , Raker CA, Phipps MG. Improving influenza vaccination rates in pregnant women . J Reprod Med 2012 ; 57 ( 9–10 ): 371 – 6 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 36. Klatt TE , Hopp E. Effect of a best-practice alert on the rate of influenza vaccination of pregnant women . Obstet Gynecol 2012 ; 119 ( 2 Pt 1 ): 301 – 5 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 37. Kirschner K , Braspenning J, Akkermans RP, Jacobs JE, Grol R. Assessment of a pay-for-performance program in primary care designed by target users . Fam Pract 2013 ; 30 ( 2 ): 161 – 71 . Google Scholar Crossref Search ADS PubMed WorldCat 38. Satman I , Akalin S, Cakir B, Altinel S; diaVAX Study Group. The effect of physicians’ awareness on influenza and pneumococcal vaccination rates and correlates of vaccination in patients with diabetes in Turkey: an epidemiological Study “diaVAX” . Hum Vaccin Immunother 2013 ; 9 ( 12 ): 2618 – 26 . Google Scholar Crossref Search ADS PubMed WorldCat 39. Ulrik CS , Sørensen TB, Højmark TB, Olsen KR, Vedsted P. Adherence to COPD guidelines in general practice: impact of an educational programme delivered on location in Danish general practices . Prim Care Respir J 2013 ; 22 ( 1 ): 23 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 40. Walsh AJ , Weltman M, Burger D et al. Implementing guidelines on the prevention of opportunistic infections in inflammatory bowel disease . J Crohns Colitis 2013 ; 7 ( 10 ): e449 – 56 . Google Scholar Crossref Search ADS PubMed WorldCat 41. McCreary L . Increasing the rate of influenza vaccination in children with asthma using a clinic staff and provider educational intervention . J Asthma Allergy Educ 2013 ; 4 ( 6 ): 277 – 81 . Google Scholar Crossref Search ADS WorldCat 42. Pollack AH , Kronman MP, Zhou C, Zerr DM. Automated screening of hospitalized children for influenza vaccination . J Pediatr Infect Dis Soc 2014 ; 3 ( 1 ): 7 – 14 . Google Scholar Crossref Search ADS WorldCat 43. Hull S , Mathur R, Lloyd-Owen S, Round T, Robson J. Improving outcomes for people with COPD by developing networks of general practices: evaluation of a quality improvement project in east London . NPJ Prim Care Respir Med 2014 ; 24 : 14082 . Google Scholar Crossref Search ADS PubMed WorldCat 44. Mendu ML , Schneider LI, Aizer AA et al. Implementation of a CKD checklist for primary care providers . Clin J Am Soc Nephrol 2014 ; 9 ( 9 ): 1526 – 35 . Google Scholar Crossref Search ADS PubMed WorldCat 45. Vernacchio L , Francis ME, Epstein DM et al. Effectiveness of an asthma quality improvement program designed for maintenance of certification . Pediatrics 2014 ; 134 ( 1 ): e242 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 46. Crawford NW , Barfield C, Hunt RW, Pitcher H, Buttery JP. Improving preterm infants’ immunisation status: a follow-up audit . J Paediatr Child Health 2014 ; 50 ( 4 ): 314 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 47. Szilagyi PG , Serwint JR, Humiston SG et al. Effect of provider prompts on adolescent immunization rates: a randomized trial . Acad Pediatr 2015 ; 15 ( 2 ): 149 – 57 . Google Scholar Crossref Search ADS PubMed WorldCat 48. Pierson RC , Malone AM, Haas DM. Increasing influenza vaccination rates in a busy urban clinic . J Nat Sci 2015 ; 1 ( 3 ): e57 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 49. Stockwell MS , Catallozzi M, Camargo S et al. Registry-linked electronic influenza vaccine provider reminders: a cluster-crossover trial . Pediatrics 2015 ; 135 ( 1 ): e75 – 82 . Google Scholar Crossref Search ADS PubMed WorldCat 50. Sapir T , Moreo K, Carter JD et al. Continuing medical education improves gastroenterologists’ compliance with inflammatory bowel disease quality measures . Dig Dis Sci 2016 ; 61 ( 7 ): 1862 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat 51. Luder H , Frede S. Community pharmacist collaboration with a patient-centered medical home: establishment of a patient-centered medical neighborhood and payment model . J Am Pharm Assoc 2016 ; 56 ( 3 ): e18 – 9 . Google Scholar OpenURL Placeholder Text WorldCat 52. Zwar NA , Bunker JM, Reddel HK et al. Early intervention for chronic obstructive pulmonary disease by practice nurse and GP teams: a cluster randomized trial . Fam Pract 2016 ; 33 ( 6 ): 663 – 70 . Google Scholar Crossref Search ADS PubMed WorldCat 53. Patel MS , Volpp KG, Small DS et al. Using active choice within the electronic health record to increase influenza vaccination rates . J Gen Intern Med 2017 ; 32 ( 7 ): 790 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 54. Real FJ , DeBlasio D, Beck AF et al. A virtual reality curriculum for pediatric residents decreases rates of influenza vaccine refusal . Acad Pediatr 2017 ; 17 ( 4 ): 431 – 5 . Google Scholar Crossref Search ADS PubMed WorldCat 55. Lin JL , Bacci JL, Reynolds MJ et al. Comparison of two training methods in community pharmacy: project VACCINATE . J Am Pharm Assoc (2003) 2018 ; 58 ( 4S ): S94 – 100.e3 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 56. Parker PA , Banerjee SC, Matasar MJ et al. Efficacy of a survivorship-focused consultation versus a time-controlled rehabilitation consultation in patients with lymphoma: a cluster randomized controlled trial . Cancer 2018 ; 124 ( 23 ): 4567 – 76 . Google Scholar Crossref Search ADS PubMed WorldCat 57. Whitaker JA , Poland CM, Beckman TJ et al. Immunization education for internal medicine residents: a cluster-randomized controlled trial . Vaccine 2018 ; 36 ( 14 ): 1823 – 9 . Google Scholar Crossref Search ADS PubMed WorldCat 58. Feuerstein JD , Papamichael K, Popejoy S et al. Targeted physician education and standardizing documentation improves documented reporting with inflammatory bowel disease quality measures in a large academic and private practice . Dig Dis Sci 2018 ; 63 ( 1 ): 36 – 45 . Google Scholar Crossref Search ADS PubMed WorldCat 59. Giduthuri JG , Purohit V, Maire N et al. Influenza vaccination of pregnant women: engaging clinicians to reduce missed opportunities for vaccination . Vaccine 2019 ; 37 ( 14 ): 1910 – 7 . Google Scholar Crossref Search ADS PubMed WorldCat 60. Aminrad Z , Zarina S, Sayed B et al. Relationship between awareness, knowledge and attitudes towards environmental education among secondary school students in Malaysia . World Appl Sci 2013 ; 22 ( 9 ): 1326 – 33 . Google Scholar OpenURL Placeholder Text WorldCat 61. Ho HJ , Chan YY, Ibrahim MAB et al. A formative research-guided educational intervention to improve the knowledge and attitudes of seniors towards influenza and pneumococcal vaccinations . Vaccine 2017 ; 35 ( 47 ): 6367 – 74 . Google Scholar Crossref Search ADS PubMed WorldCat 62. Rose KA , Ayad S. Factors associated with changes in knowledge and attitude towards public health concepts among chiropractic college students enrolled in a community health class . J Chiropr Educ 2008 ; 22 ( 2 ): 127 – 37 . Google Scholar Crossref Search ADS PubMed WorldCat 63. Centers for Disease Control and Prevention . Epidemiology and Prevention of Vaccine—Preventable Diseases: Immunization Strategies for Healthcare Practices and Providers . 13th edn. Washington, DC : Public Health Foundation , 2015 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 64. Donato AA , Motz LM, Wilson G, Lloyd BJ. Efficacy of multiple influenza vaccine delivery systems in a single facility . Infect Control Hosp Epidemiol 2007 ; 28 ( 2 ): 219 – 21 . Google Scholar Crossref Search ADS PubMed WorldCat 65. Rees S , Stevens L, Drayton J, Engledow N, Sanders J. Improving inpatient pneumococcal and influenza vaccination rates . J Nurs Care Qual 2011 ; 26 ( 4 ): 358 – 63 . Google Scholar Crossref Search ADS PubMed WorldCat 66. Coyle CM , Currie BP. Improving the rates of inpatient pneumococcal vaccination: impact of standing orders versus computerized reminders to physicians . Infect Control Hosp Epidemiol 2004 ; 25 ( 11 ): 904 – 7 . Google Scholar Crossref Search ADS PubMed WorldCat 67. Shojania KG , Jennings A, Mayhew A et al. Effect of point-of-care computer reminders on physician behaviour: a systematic review . CMAJ 2010 ; 182 ( 5 ): E216 – 25 . Google Scholar Crossref Search ADS PubMed WorldCat 68. Fiks AG , Hunter KF, Localio AR et al. Impact of electronic health record-based alerts on influenza vaccination for children with asthma . Pediatrics 2009 ; 124 ( 1 ): 159 – 69 . Google Scholar Crossref Search ADS PubMed WorldCat 69. Chambers CV , Balaban DJ, Carlson BL, Grasberger DM. The effect of microcomputer-generated reminders on influenza vaccination rates in a university-based family practice center . J Am Board Fam Pract 1991 ; 4 ( 1 ): 19 – 26 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 70. Tierney WM , Overhage JM, Murray MD et al. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial . Health Serv Res 2005 ; 40 ( 2 ): 477 – 97 . Google Scholar Crossref Search ADS PubMed WorldCat © The Author(s) 2021. Published by Oxford University Press. This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial reproduction and distribution of the work, in any medium, provided the original work is not altered or transformed in any way, and that the work is properly cited. For commercial re-use, please contact journals.permissions@oup.com © The Author(s) 2021. Published by Oxford University Press.

Journal

Family Practice – Oxford University Press

Published: Jul 28, 2021

Keywords: adult; health personnel; influenza vaccines; nurses; pediatrics; vaccination; guidelines; pharmacists

There are no references for this article.