Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Physicians’ electronic inbox work patterns and factors associated with high inbox work duration

Physicians’ electronic inbox work patterns and factors associated with high inbox work duration Abstract Objectives Electronic health record systems are increasingly used to send messages to physicians, but research on physicians’ inbox use patterns is limited. This study’s aims were to (1) quantify the time primary care physicians (PCPs) spend managing inboxes; (2) describe daily patterns of inbox use; (3) investigate which types of messages consume the most time; and (4) identify factors associated with inbox work duration. Materials and Methods We analyzed 1 month of electronic inbox data for 1275 PCPs in a large medical group and linked these data with physicians’ demographic data. Results PCPs spent an average of 52 minutes on inbox management on workdays, including 19 minutes (37%) outside work hours. Temporal patterns of electronic inbox use differed from other EHR functions such as charting. Patient-initiated messages (28%) and results (29%) accounted for the most inbox work time. PCPs with higher inbox work duration were more likely to be female (P < .001), have more patient encounters (P < .001), have older patients (P < .001), spend proportionally more time on patient messages (P < .001), and spend more time per message (P < .001). Compared with PCPs with the lowest duration of time on inbox work, PCPs with the highest duration had more message views per workday (200 vs 109; P < .001) and spent more time on the inbox outside work hours (30 minutes vs 9.7 minutes; P < .001). Conclusions Electronic inbox work by PCPs requires roughly an hour per workday, much of which occurs outside scheduled work hours. Interventions to assist PCPs in handling patient-initiated messages and results may help alleviate inbox workload. electronic mail, electronic health records, medical informatics applications, in-basket, work connectivity after hours INTRODUCTION The electronic inbox forms a crucial hub for physicians to communicate with other clinicians, staff, and patients via electronic health record (EHR)–based messages. The ability to communicate with patients and families via email-like secure messages helps physicians build relationships.1–8 Electronic inbox management has become a progressively more important component of physicians’ work as EHR adoption and patient use of secure messages have increased.9–12 In business and information work, inbox management has been associated with stress and burnout due to the time required to handle the ever-increasing volume of emails, the task demands associated with emails, and the interruptions they create.12–14 In the same way, electronic inbox management has been purported to contribute to physician stress and burnout.1,15 One study of the burden of EHR inbox notifications reported that primary care physicians (PCPs) received an average 77 inbox notifications per day, significantly higher than specialists’ average of 29 notifications per day.16 Another found that inbox management accounted for 24% of physicians’ total time spent on EHR work.17 Scant research on physicians’ electronic inbox use patterns currently exists. Previous studies of daily EHR work patterns have evaluated overall EHR use throughout the day,18,19 but inbox management has received limited attention beyond studies that have quantified the time spent on it. To facilitate more adept use of electronic inboxes, it is critical to gain a more detailed understanding of how physicians manage them. Understanding the temporal patterns of inbox work and identifying factors associated with high duration of inbox work are key steps toward learning how to potentially reduce stress associated with inbox management. This study’s aims were to (1) quantify the amount of time physicians spend on electronic inbox management, (2) describe daily patterns of inbox use, (3) describe variation in inbox work by the type of message received, and (4) identify factors associated with high duration of time on electronic inbox work. MATERIALS AND METHODS Study setting We analyzed the electronic inbox use data of internists and family practitioners in the Permanente Medical Group, the largest medical group in the United States. This group has 9200 physicians and serves 4.4 million members of Kaiser Permanente via 21 hospital-based medical centers and more than 250 medical offices in Northern California. Physicians are salaried, and each primary care practitioner has a defined panel of affiliated patients to manage. Since 2008, the medical group has used a comprehensive EHR (Epic Systems, Verona, WI) that integrates inpatient, emergency, and outpatient care, including primary care, specialty, laboratory, pharmacy, and imaging data. The EHR inbox is the primary message hub that physicians use for clinical care. The inbox receives messages patients send via a portal website (also available through patient-facing mobile applications), as well as messages from other physicians, clinical staff, the pharmacy, laboratory, and other departments. Physicians can access the inbox on computers or mobile devices. Physicians are expected to reply to each patient message within 2 business days. Patients are encouraged to use the messaging functionality of the EHR to enhance access to their physicians and the care experience. The practice environment was relatively consistent across physicians in this study. Clinical work hours were Monday to Friday, 8:30 am to 12:30 pm and 1:30 pm to 5:30 pm; medical offices were open during these hours. Clinical work hours were scheduled predominantly with patient appointments conducted in person or via telephone or video visits. Within clinical work hours, some departments earmarked periods of time without scheduled patient visits called “panel management time,” to protect time for physicians to manage inboxes and do other types of work. Physicians sometimes worked clinically on weekends, with work hours that could differ from weekdays. This setting’s patient population was diverse, with 54% of adults being White, 21% being Asian, 16% being Latino, and 6% being Black. Approximately half of adult members had a college degree, more than three-fourths were employed or self-employed, and most lived in urban or suburban areas.20 Data The Epic EHR creates daily static datasets. These logs contain second-by-second records of the EHR components physicians have accessed and records of the associated actions performed. The logged data for the inbox includes the type of message being accessed, the time, user ID, and the access mode (desktop or mobile). In this study’s analysis, the logged time spent on the inbox only included time spent on the inbox page, and did not count time spent on other EHR components that physicians might use to gather information to respond to messages. Hence, the inbox time in this analysis was an underestimate of the total time physicians spent on inbox-related work. EHR log data were extracted from March 1 to 31, 2018, for all internists and family practitioners who were doing at least 70% full-time equivalent (FTE) outpatient clinical practice at the time of the analysis. We chose March because it is a representative month: neither extremely busy due to influenza season nor slow due to summer vacations. We also obtained physicians' age, sex, years in practice, and their FTE in clinical practice, a measure of their workload in scheduled clinical hours. Internal analyses by this medical group reveal that FTE is strongly correlated with panel size. We thus include patient panel characteristics per physician (mean age, percentage over 65 years of age, percentage female). After preliminary analyses of the 1275 physicians in the dataset, we excluded physicians with <10 working days in the month, and those with no time spent on the inbox on all workdays, leaving 1257 physicians in the study population. We defined a workday for a physician as a day with at least 1 scheduled in-person visit or more than 7 telephone and video visits. We chose this cut point based on the number of telephone encounters among physicians on weekends in March 2018 to properly classify days off when a physician may make a few telephone calls to relay lab results to patients while not scheduled to work. We removed weekend workdays (Saturdays and Sundays) from our analysis of workdays to better distinguish work and nonwork hours, as work hours can vary when physicians are working on weekends. The system logs included a “message type description” field, with 59 different labels. We categorized these labels into higher-level groupings based on an analysis of frequencies and input from this study’s clinical collaborators, who were familiar with the meanings and patterns of different message types. No message content or metadata (ie, sender, receiver, or timestamp) were included in this study, which was approved by the Kaiser Permanente Northern California Institutional Review Board. Statistical analyses To identify factors associated with duration of time spent on EHR inbox work, we compared physicians in the top and bottom quartiles of average inbox use duration, and created multiple regression models. To compare factors between these groups, we conducted a series of 2-group tests (t tests for normally distributed variables, Mann-Whitney U tests for nonparametrically distributed variables, and chi-square tests for categorical variables). We corrected for multiple comparisons using the Bonferroni correction, testing each P value against an alpha value of .05/18 tests = .003. For the regression models, the distributions of dependent variables were inspected to confirm the normality assumption. Multicollinearity was tested using the variance inflation factor, and independent variables with variance inflation factor >5 were removed. Analyses were conducted using the Scipy and Statsmodels packages in Python. RESULTS Participants Of the 1257 physicians in the study, 57% were female. Their ages ranged from 29 to 72 years (mean 46.7 ± 8.4 years of age). Years since medical school graduation ranged from 4 to 45 years of age (mean 19.3 ± 9.0 years of age). Participants’ workload ranged from 0.7 to 1.0 FTE (mean = 0.85 ± 0.11 FTE). On average, physicians in the dataset had 20 ± 3 workdays and 11 ± 3 nonworkdays. The average physician in the study had a patient panel with an average age of 46.78 ± 6.25 years, with 19 ± 10% of patients being older than 65 years of age, and 52 ± 18% being female patients. The average number of patient encounters per physician for the month was 335.26 ± 95.1, of which 237.39 ± 60.58 were face-to-face encounters and 97.87 ± 57.05 were telephone and video encounters. Time spent on inbox management On workdays, the average time spent on inbox work was a total of 52 minutes: 33 minutes during work hours and 19 minutes outside of work hours (Table 1). Thus, 37% of total time on the inbox on a workday occurred outside of formal work hours. On nonworkdays, physicians spent an average of 12 minutes on the inbox, with a range of 0-93 minutes. Table 1. Electronic inbox message views and time spent per day by primary care physicians in the Permanente Medical Group, March 2018 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 Open in new tab Table 1. Electronic inbox message views and time spent per day by primary care physicians in the Permanente Medical Group, March 2018 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 Open in new tab On workdays, physicians spent most of their inbox time on desktop or laptop computers, whether within or outside of work hours. On nonworkdays, however, the inbox was mostly accessed through mobile devices. On average, physicians had 100 message views a day during work hours, and 53 views outside work hours (Table 1). On nonworkdays, physicians had an average of 33 message views daily. It is important to note that we cannot ascertain if these were unique message counts, or if they were the number of times physicians started a period of inbox work; rather, they were the number of times physicians switched into an inbox page. For example, it would count as 2 message views if a physician viewed a message, then shifted to another EHR system page to retrieve information, then returned to reply to the same message. Daily patterns of electronic inbox work compared with other EHR work Figure 1 contrasts the daily patterns of time spent on the inbox component of the EHR with time spent on other components of the EHR. For many physicians (Figure 1A), the most time spent on the EHR inbox was before and after formal work hours, and during the lunch hour. Averaging across all users showed small peaks at the beginning and end of the workday. Figure 1. Open in new tabDownload slide Time spent on (A) the EHR inbox and (B) EHR functionality other than the inbox. Top figures show daily averages for each user (1257 users) and the bottom figures show overall average across user averages. Figure 1. Open in new tabDownload slide Time spent on (A) the EHR inbox and (B) EHR functionality other than the inbox. Top figures show daily averages for each user (1257 users) and the bottom figures show overall average across user averages. In contrast, time on other EHR components followed a different pattern. As Figure 1B shows, time spent on EHR functionality other than the inbox (including chart review, order entry, and charting) increased during work hours compared with nonwork hours. There was a sharp decrease in EHR use during the lunch hour as well. On nonworkdays, both overall EHR and inbox use had flatter patterns than workdays. As can be expected, there is a large variance across individual users in the amount of time spent on the EHR and the inbox per hour. Inbox use patterns associated with type of message received We categorized messages into 4 types: (1) messages from patients; (2) results, such as lab test results; (3) requests, which ask the physician to perform an action such as approving a medication refill or signing clinical orders; and (4) informational and administrative messages. An additional category (5) was added for other time on the inbox that was not tied to a specific message. Of all time spent on the EHR inbox, physicians spent 28%, 29%, 25%, and 11% on patient messages, results, requests, and administrative messages, respectively (Table 2). Table 2. Average time spent on each message type during workdays and non-workdays. Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Values are mean ± SD. The overall column indicates mean percentages of total inbox time over the month. Open in new tab Table 2. Average time spent on each message type during workdays and non-workdays. Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Values are mean ± SD. The overall column indicates mean percentages of total inbox time over the month. Open in new tab Factors associated with high duration of time on inbox work To further explore different patterns of inbox use, we first compared physicians in our dataset in the top and lowest quartiles of average duration of time spent on the inbox on workdays (during and outside of work hours). This segmentation resulted in 314 users per group. The high-duration group (top quartile) had an average inbox duration of 72 ± 9 minutes per workday, while the low-duration group (lowest quartile) had an average of 33 ± 7 minutes. Table 3 shows these comparisons. The high-duration user group had a higher proportion of women (62%) compared with low-duration user group (50%) (P = .003). There was no statistically significant difference between high- and low-duration users in the number of days worked or FTE. Both groups had an average of 19 (SD = 3) workdays during the month and an average FTE of 0.85 ± 0.1. However, high-duration users had more patient encounters, older patients, and a higher percentage of patients older than 65 years of age and female patients. Table 3. Comparisons of high-duration and low-duration users of the electronic health record–based inbox among primary care physicians in the Permanente Medical Group, 2018 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 Values are mean ± SD, unless otherwise indicated. FTE: full-time equivalent. a Significance of P < .003 based on the Bonferroni correction. Open in new tab Table 3. Comparisons of high-duration and low-duration users of the electronic health record–based inbox among primary care physicians in the Permanente Medical Group, 2018 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 Values are mean ± SD, unless otherwise indicated. FTE: full-time equivalent. a Significance of P < .003 based on the Bonferroni correction. Open in new tab High-duration users spent more than twice the duration on messages after work hours on workdays (30 minutes vs 9.7 minutes; P < .001), compared with users in the low-duration group (Table 2). Comparing the ratio of after-hours inbox use duration to all day inbox use duration, we found that a larger average proportion of time on the inbox occurred outside of work hours for high-duration users (41%) compared with low-duration users (29%) (P < .001). We also found differences in the time spent on the inbox on nonworkdays. On nonworkdays, high-duration users spent almost double the time on inbox work relative to low-duration users. Users in the high-duration group averaged almost twice the message views on workdays (199 views vs 109 views; P < .001), and spent slightly more time per message view (22.3 seconds vs 18.9 seconds; P < .001). The duration of time spent on the inbox on mobile devices was also more than doubled for high-duration users, compared with low-duration users, although the ratio of inbox time on mobile to all inbox time was similar between the groups, as both groups spent 23%-25% of their total EHR inbox time on mobile devices. To identify factors independently associated with inbox work duration, we created multiple linear regression models for 2 dependent variables: all-day inbox use duration and after-hours inbox use duration. Both measures were averages over each physician’s workdays. Independent variables included physician characteristics (age and sex), patient panel characteristics (age), patient encounters (face-to-face and telephone and video visits), average percentage of inbox time spent on patient messages, average time per message view, and average percentage of inbox time using mobile devices. For predicting after-hours inbox time, work-hours inbox time was also added as an independent variable. Owing to multicollinearity, we excluded patient panel percentage of female patients, percentage of patients over 65 years of age, physician years of experience, and FTE. Model results are presented in Table 4. Table 4. Regression models predicting duration of all-day inbox work (per 24-hour period) and after-hours inbox work . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 Open in new tab Table 4. Regression models predicting duration of all-day inbox work (per 24-hour period) and after-hours inbox work . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 Open in new tab For the model of all-day inbox duration all predictors, except physician age, had a positive relationship with inbox duration (F = 63.71, P < .001, adj R2 = .29). The average age of the patient panel, the number of face-to-face patient encounters, and time per message view had the largest effect sizes (standardized coefficients >0.3). For predicting after-hours inbox duration, the analysis showed a negative relationship of the more time physicians spent on the inbox during work hours, the less time they spent on the inbox after hours. A negative relationship was also found between physician age and after-hours inbox use, although the effect size was small. Face-to-face patient encounters, time per message view, and percentage of inbox time that is on mobile devices had the largest effect sizes (standardized coefficients >0.3). Female physicians and physicians who had older patients spent more time on the inbox after hours. The model explains 33% of the variation in after-hours inbox time (F = 68.36, P < .001, adj R2 = .33). DISCUSSION Major findings To our knowledge, this study is the first to describe how electronic inbox work fits temporally into the workdays of PCPs. We found that, on average, PCPs spent 52 minutes on electronic inbox work on workdays, and that more than one-third of this time occurred outside formally scheduled work hours. This study also was unique in our ability to identify characteristics of physicians who spent higher than average amounts of time on inbox work. Those with the highest duration of inbox work were more likely to be female, have a higher percentage of female patients, have older patients, have more patient encounters, and be doing inbox work after hours, as well as spend more time per message view and spend a higher proportion of their inbox time on patient-initiated messages. Interpretation and comparison with past studies Previous studies of electronic inbox use have described the amount and timing of inbox work but have neither studied temporal patterns in depth nor described variation among physicians. A 2017 study found that PCPs spent an average of 5.9 hours per workday on EHR systems, including 1.4 hours outside clinic hours, with 24% of the total time spent in the inbox and a higher proportion of inbox time spent after hours compared with other EHR activities.17 Our study went beyond previous studies17–19 in comparing temporal patterns of electronic inbox use as distinct from use of other parts of the EHR (eg, chart review, order entry, creating notes). We found that inbox use patterns did not mirror work hours, but rather increased before and after work hours and during lunch hours. We also found that the more time physicians spent on the inbox during work hours, the less time they spent on the inbox after hours. These findings were not surprising because during workdays, PCPs usually are scheduled to see patients via in-person, telephone, or video visits for most or all of the available time. Electronic inbox work during formal work hours is typically fit in between patient visits, and work that cannot be finished during those hours is usually addressed just before work hours, just after work hours, or during lunch. Given that the average time spent on the inbox after hours is 19 minutes, which is small relative to the total time spent on EHR activities in other studies, it is possible that some physicians may be prioritizing inbox work during work hours and completing notes outside work hours. Future work analyzing the trade-off between different EHR actions within and outside of work hours could clarify this supposition. This study found that physicians switch to view electronic messages 100 times a day. This number is higher than previous studies of information workers, who checked their emails an average of 77 times a day during work hours.21 The high number of views may be due to physicians needing to switch screens between the inbox and other parts of the EHR to find information to reply to messages or to take other actions, indicating a high rate of task switching within inbox work itself. Given the associations between multitasking and stress,22,23 physician patterns of multitasking with the electronic inbox warrant further study. Our finding that physicians who spent more time on inbox work tended to be female is novel, and consistent with a recent finding that female physicians tend to spend more time on the EHR.24 This contrasts with another recent study in which women reported less EHR-related stress and higher efficiency than men.25 Studies of physicians have suggested that burnout symptoms may be associated with being female, and that work-home conflict may play a role.15,26,27 Our observation that female physicians tend to spend more time on inbox work suggests that this deserves further exploration as a possible factor in sex differences in physician stress. We also found that high inbox work duration was associated with more work outside of work hours. Not only was the absolute amount of time spent working after hours longer, but also the proportion of all time spent on the inbox that occurred after hours was higher. Research in office settings has found spending more time on email associated with greater work overload, frustration, and stress.13 Our finding that inbox work tends to extend beyond work hours for high-duration users suggests that further study is warranted regarding how to best support this group. In this study, inbox work duration was independently associated with clinician workload as measured by the number of appointments seen during the month studied. This is in accord with a previous finding that physicians with more clinical time were disproportionately burdened by after-hours EHR work,28 and another study that found that work relative value units (ie, work volume and complexity) were positively associated with EHR time within and outside of work hours.24 In contrast, we did not find that lower FTE was associated with reduced inbox work. This finding may reflect that physicians who adopt reduced clinical FTE schedules may not experience a commensurate decrease in their amount of electronic inbox work, or conversely, that physicians who feel overburdened may adopt reduced FTE schedules to allow themselves more time to complete clinical work in general, including inbox work. A recent study found that provider-to-provider variation was the largest source of variation in after-hours EHR usage.24 Similarly, the wide variation we found in inbox work duration and use patterns among PCPs in this study suggests that individual preferences and approaches most likely play a role in inbox work patterns. Future research could attempt to identify different types of users based on the strategies they adopt (eg, batching inbox message work),21 the temporal patterns of their work, and their use of mobile devices for inbox work. In addition, future work could investigate the effect of organizational efforts such as designating time for panel management within clinical work hours. It would be useful to study whether inbox use patterns are associated with physician stress or productivity, or patient satisfaction with electronic message communication with their PCPs. The results suggest practical implications for inbox system design. Given the high number of message views (ie counts of switching to the inbox page), a system design consideration would be to implement an interface that incorporates information that physicians need from sources outside the inbox page (eg patient data) to process an inbox message, thus reducing potential frequent switching between the inbox page and other windows. Batching (ie attending to the inbox once or twice a day rather than consistently checking messages throughout the day) and checking the inbox fewer times has been found to be associated with less time on email and less stress.14,21,29 While email batching may be desirable in some work settings, our study’s results show that the temporal patterns of inbox use are different in medical settings, in which physicians spend smaller amounts of time on the inbox during periods when they are scheduled to see patients. It is possible that the messages being checked in between patient visits are those that are more clinically urgent, making batching infeasible. Thus, the feasibility and desirability of batching inbox time in medical contexts is yet to be evaluated. Another practical implication for inbox system design is to implement screening and categorization of patient-initiated messages, automatically or by assistants, which can help PCPs prioritize or delegate some messages. Limitations This study was conducted in a large medical group that encourages patients to use EHR portal messages to communicate with physicians.8 The group also tries to limit the amount of system-generated messages and administrative reminders sent to physicians’ inboxes. Thus, this setting’s volume of inbox messages from patients may be higher and the balance of different types of messages may differ from those in less integrated settings. The dataset for this study had message views but not unique message counts. Thus, we did not analyze the volume of messages, time spent per message, or how many times a message was revisited. As noted in the Materials and Methods, inbox work duration counted time the inbox window was open, but did not count (for example) time when the inbox window was in the background while the user was accessing other parts of the EHR such as chart review or order entry in response to a message. Thus, the duration of inbox work in this study is likely an underestimate of the time physicians spent on inbox-related work. Finally, our analysis did not control for panel management time (ie, time designated by departments specifically for tasks like inbox management). Thus, we cannot make assumptions about why inbox work patterns and peak use times differ among physicians. Our analysis shows an inverse relationship between time on the inbox during and after work hours. It is possible that physicians who have dedicated panel management time during work hours are those who spend less time on inbox work after hours. The effect of panel management time on inbox use patterns needs further study. CONCLUSIONS PCPs in the largest medical group in the United States spend roughly an hour per workday on inbox management, and much of this work occurs outside scheduled work hours. In this setting, patient-initiated messages and results consume the highest proportion of inbox work time. Interventions to assist PCPs in handling patient-initiated messages and results may help alleviate inbox workload. FUNDING This work was supported by the Permanente Medical Group via its Delivery Science Research Program, and by the National Science Foundation under Grant No. 1704889. The funder (Delivery Science grant) supported the effort of the study team and had no specific requirements regarding interpretation of results or framing of the manuscript. The National Science Foundation grant partially supported FA in collection, management, analysis, and interpretation of the data and GM in interpretation, preparation, and review. The views, opinions, and/or findings contained in this publication are those of the authors and do not necessarily reflect the views of Kaiser Permanente and should not be construed as an official position, policy or decision of Kaiser Permanente unless so designated by other documentation. No official endorsement should be made. AUTHOR CONTRIBUTIONS EMW collected and preprocessed the data. FA analyzed the data and drafted the manuscript. TAL, MR, and GM obtained funding and provided supervision. All authors contributed to interpreting results, refining all sections and critically editing the manuscript. ACKNOWLEDGMENTS We appreciate the contributions of Manuel Ballesca, MD, and Brian Hoberman, MD, to the design of this study, as well as the participation of Andres Montoya, BS, and Lynn Emerson, MS, on the project team. We are grateful to the study’s sponsors, TPMG Associate Executive Directors Yi-Fen Irene Chen, MD; Sameer Awsare, MD; Patricia Conolly, MD; Edward Lee, MD; and Philip Madvig, MD. CONFLICT OF INTEREST STATEMENT None declared. REFERENCES 1 Lieu TA Freed GL. Unbounded–parent-physician communication in the era of portal messaging . JAMA Pediatr 2019 ; 173 ( 9 ): 811 – 2 . Google Scholar Crossref Search ADS WorldCat 2 Reed ME Huang J Brand R , et al. Communicating through a patient portal to engage family care partners . JAMA Intern Med 2018 ; 178 ( 1 ): 142 – 4 . Google Scholar Crossref Search ADS PubMed WorldCat 3 Reed ME Huang J Millman A , et al. Portal use among patients with chronic conditions: patient-reported care experiences . Med Care 2019 ; 57 ( 10 ): 809 – 14 . Google Scholar Crossref Search ADS PubMed WorldCat 4 Patt MR Houston TK Jenckes MW Sands DZ Ford DE. Doctors who are using e-mail with their patients: a qualitative exploration . J Med Internet Res 2003 ; 5 ( 2 ): e9 . Google Scholar Crossref Search ADS PubMed WorldCat 5 Car J Sheikh A. Email consultations in health care: 1—scope and effectiveness . BMJ 2004 ; 329 ( 7463 ): 435 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 6 Moyer CA Stern DT Katz SJ Fendrick AM. “ We got mail”: electronic communication between physicians and patients . Am J Manag Care 1999 ; 5 ( 12 ): 1513 – 22 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 7 Reed M Graetz I Gordon N Fung V. Patient-initiated e-mails to providers: associations with out-of-pocket visit costs, and impact on care-seeking and health . Am J Manag Care 2015 ; 21 ( 12 ): e632 – 9 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 8 Lieu TA Altschuler A Weiner JZ , et al. Primary care physicians’ experiences with and strategies for managing electronic messages . JAMA Netw Open 2019 ; 2 ( 12 ): e1918287 . Google Scholar Crossref Search ADS PubMed WorldCat 9 Adler-Milstein J Holmgren AJ Kralovec P Worzala C Searcy T Patel V. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide . J Am Med Inform Assoc 2017 ; 24 ( 6 ): 1142 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 10 Crotty BH Tamrat Y Mostaghimi A Safran C Landon BE. Patient-to-physician messaging: volume nearly tripled as more patients joined system, but per capita rate plateaued . Health Aff (Millwood) 2014 ; 33 ( 10 ): 1817 – 22 . Google Scholar Crossref Search ADS PubMed WorldCat 11 Shenson JA Cronin RM Davis SE Chen Q Jackson GP. Rapid growth in surgeons’ use of secure messaging in a patient portal . Surg Endosc 2016 ; 30 ( 4 ): 1432 – 40 . Google Scholar Crossref Search ADS PubMed WorldCat 12 Renaud K Ramsay J Hair M. “ You’ve got e-mail!”… shall I deal with it now? Electronic mail from the recipient’s perspective . Int J Hum Comput Interact 2006 ; 21 ( 3 ): 313 – 32 . Google Scholar Crossref Search ADS WorldCat 13 Barley SR Meyerson DE Grodal S. E-mail as a source and symbol of stress . Organ Sci 2011 ; 22 ( 4 ): 887 – 906 . Google Scholar Crossref Search ADS WorldCat 14 Mark G Voida S Cardello A. A pace not dictated by electrons: an empirical study of work without email. In: proceedings of the SIGCHI Conference on Human Factors in Computing Systems ; 2012 : 555 – 564 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 15 Tai-Seale M Dillon EC Yang Y , et al. Physicians’ well-being linked to in-basket messages generated by algorithms in electronic health records . Health Aff (Millwood) 2019 ; 38 ( 7 ): 1073 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 16 Murphy DR Meyer AN Russo E Sittig DF Wei L Singh H. The burden of inbox notifications in commercial electronic health records . JAMA Intern Med 2016 ; 176 ( 4 ): 559 – 60 . Google Scholar Crossref Search ADS PubMed WorldCat 17 Arndt BG Beasley JW Watkinson MD , et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations . Ann Fam Med 2017 ; 15 ( 5 ): 419 – 26 . Google Scholar Crossref Search ADS PubMed WorldCat 18 Wang JK Ouyang D Hom J Chi J Chen JH. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data . PLoS One 2019 ; 14 ( 2 ): e0205379 . Google Scholar Crossref Search ADS PubMed WorldCat 19 Ouyang D Chen JH Hom J Chi J. Internal medicine resident computer usage: an electronic audit of an inpatient service . JAMA Intern Med 2016 ; 176 ( 2 ): 252 – 4 . Google Scholar Crossref Search ADS PubMed WorldCat 20 Gordon N. Highlights of Results of the Kaiser Permanente Northern California 2014/2015 Member Health Survey . Oakland, CA : Division of Research, Kaiser Permanente Medical Care Program ; 2017 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 21 Mark G Iqbal ST Czerwinski M Johns P Sano A Lutchyn Y. Email duration, batching and self-interruption: Patterns of email use on productivity and stress In: proceedings of the 2016 CHI Conference on Human Factors in Computing Systems ; 2016 : 1717 – 28 . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC 22 Mark G Wang Y Niiya M. Stress and multitasking in everyday college life: an empirical study of online activity In: proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems - CHI ’14; 2014 : 41 – 50 . doi: 10.1145/2556288.2557361 23 Reinecke L Aufenanger S Beutel ME , et al. Digital stress over the life span: the effects of communication load and internet multitasking on perceived stress and psychological health impairments in a German probability sample . Media Psychol 2017 ; 20 ( 1 ): 90 – 115 . Google Scholar Crossref Search ADS WorldCat 24 Attipoe S Huang Y Schweikhart S Rust S Hoffman J Lin S. Factors associated with electronic health record usage among primary care physicians after hours: retrospective cohort study . JMIR Hum Factors 2019 ; 6 ( 3 ): e13779 . Google Scholar Crossref Search ADS PubMed WorldCat 25 Khairat S Coleman C Ottmar P Bice T Koppel R Carson SS. Physicians’ gender and their use of electronic health records: findings from a mixed-methods usability study . J Am Med Inform Assoc 2019 ; 26 ( 12 ): 1505 – 14 . Google Scholar Crossref Search ADS PubMed WorldCat 26 Houkes I Winants Y Twellaar M Verdonk P. Development of burnout over time and the causal order of the three dimensions of burnout among male and female GPs. A three-wave panel study . BMC Public Health 2011 ; 11 ( 1 ): 240 . Google Scholar Crossref Search ADS PubMed WorldCat 27 Innstrand ST Langballe EM Falkum E Aasland OG. Exploring within-and between-gender differences in burnout: 8 different occupational groups . Int Arch Occup Environ Health 2011 ; 84 ( 7 ): 813 – 24 . Google Scholar Crossref Search ADS PubMed WorldCat 28 Saag HS Shah K Jones SA Testa PA Horwitz LI. Pajama time: working after work in the electronic health record . J Gen Intern Med 2019 ; 34 ( 9 ): 1695 – 2 . Google Scholar Crossref Search ADS PubMed WorldCat 29 Kushlev K Dunn EW. Checking email less frequently reduces stress . Comput Hum Behav 2015 ; 43 : 220 – 8 . Google Scholar Crossref Search ADS WorldCat © The Author(s) 2020. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Journal of the American Medical Informatics Association Oxford University Press

Physicians’ electronic inbox work patterns and factors associated with high inbox work duration

Download PDF
8 pages

Loading next page...
 
/lp/oxford-university-press/physicians-electronic-inbox-work-patterns-and-factors-associated-with-z7IqUyAcP8

References (38)

Publisher
Oxford University Press
Copyright
Copyright © 2021 American Medical Informatics Association
ISSN
1067-5027
eISSN
1527-974X
DOI
10.1093/jamia/ocaa229
Publisher site
See Article on Publisher Site

Abstract

Abstract Objectives Electronic health record systems are increasingly used to send messages to physicians, but research on physicians’ inbox use patterns is limited. This study’s aims were to (1) quantify the time primary care physicians (PCPs) spend managing inboxes; (2) describe daily patterns of inbox use; (3) investigate which types of messages consume the most time; and (4) identify factors associated with inbox work duration. Materials and Methods We analyzed 1 month of electronic inbox data for 1275 PCPs in a large medical group and linked these data with physicians’ demographic data. Results PCPs spent an average of 52 minutes on inbox management on workdays, including 19 minutes (37%) outside work hours. Temporal patterns of electronic inbox use differed from other EHR functions such as charting. Patient-initiated messages (28%) and results (29%) accounted for the most inbox work time. PCPs with higher inbox work duration were more likely to be female (P < .001), have more patient encounters (P < .001), have older patients (P < .001), spend proportionally more time on patient messages (P < .001), and spend more time per message (P < .001). Compared with PCPs with the lowest duration of time on inbox work, PCPs with the highest duration had more message views per workday (200 vs 109; P < .001) and spent more time on the inbox outside work hours (30 minutes vs 9.7 minutes; P < .001). Conclusions Electronic inbox work by PCPs requires roughly an hour per workday, much of which occurs outside scheduled work hours. Interventions to assist PCPs in handling patient-initiated messages and results may help alleviate inbox workload. electronic mail, electronic health records, medical informatics applications, in-basket, work connectivity after hours INTRODUCTION The electronic inbox forms a crucial hub for physicians to communicate with other clinicians, staff, and patients via electronic health record (EHR)–based messages. The ability to communicate with patients and families via email-like secure messages helps physicians build relationships.1–8 Electronic inbox management has become a progressively more important component of physicians’ work as EHR adoption and patient use of secure messages have increased.9–12 In business and information work, inbox management has been associated with stress and burnout due to the time required to handle the ever-increasing volume of emails, the task demands associated with emails, and the interruptions they create.12–14 In the same way, electronic inbox management has been purported to contribute to physician stress and burnout.1,15 One study of the burden of EHR inbox notifications reported that primary care physicians (PCPs) received an average 77 inbox notifications per day, significantly higher than specialists’ average of 29 notifications per day.16 Another found that inbox management accounted for 24% of physicians’ total time spent on EHR work.17 Scant research on physicians’ electronic inbox use patterns currently exists. Previous studies of daily EHR work patterns have evaluated overall EHR use throughout the day,18,19 but inbox management has received limited attention beyond studies that have quantified the time spent on it. To facilitate more adept use of electronic inboxes, it is critical to gain a more detailed understanding of how physicians manage them. Understanding the temporal patterns of inbox work and identifying factors associated with high duration of inbox work are key steps toward learning how to potentially reduce stress associated with inbox management. This study’s aims were to (1) quantify the amount of time physicians spend on electronic inbox management, (2) describe daily patterns of inbox use, (3) describe variation in inbox work by the type of message received, and (4) identify factors associated with high duration of time on electronic inbox work. MATERIALS AND METHODS Study setting We analyzed the electronic inbox use data of internists and family practitioners in the Permanente Medical Group, the largest medical group in the United States. This group has 9200 physicians and serves 4.4 million members of Kaiser Permanente via 21 hospital-based medical centers and more than 250 medical offices in Northern California. Physicians are salaried, and each primary care practitioner has a defined panel of affiliated patients to manage. Since 2008, the medical group has used a comprehensive EHR (Epic Systems, Verona, WI) that integrates inpatient, emergency, and outpatient care, including primary care, specialty, laboratory, pharmacy, and imaging data. The EHR inbox is the primary message hub that physicians use for clinical care. The inbox receives messages patients send via a portal website (also available through patient-facing mobile applications), as well as messages from other physicians, clinical staff, the pharmacy, laboratory, and other departments. Physicians can access the inbox on computers or mobile devices. Physicians are expected to reply to each patient message within 2 business days. Patients are encouraged to use the messaging functionality of the EHR to enhance access to their physicians and the care experience. The practice environment was relatively consistent across physicians in this study. Clinical work hours were Monday to Friday, 8:30 am to 12:30 pm and 1:30 pm to 5:30 pm; medical offices were open during these hours. Clinical work hours were scheduled predominantly with patient appointments conducted in person or via telephone or video visits. Within clinical work hours, some departments earmarked periods of time without scheduled patient visits called “panel management time,” to protect time for physicians to manage inboxes and do other types of work. Physicians sometimes worked clinically on weekends, with work hours that could differ from weekdays. This setting’s patient population was diverse, with 54% of adults being White, 21% being Asian, 16% being Latino, and 6% being Black. Approximately half of adult members had a college degree, more than three-fourths were employed or self-employed, and most lived in urban or suburban areas.20 Data The Epic EHR creates daily static datasets. These logs contain second-by-second records of the EHR components physicians have accessed and records of the associated actions performed. The logged data for the inbox includes the type of message being accessed, the time, user ID, and the access mode (desktop or mobile). In this study’s analysis, the logged time spent on the inbox only included time spent on the inbox page, and did not count time spent on other EHR components that physicians might use to gather information to respond to messages. Hence, the inbox time in this analysis was an underestimate of the total time physicians spent on inbox-related work. EHR log data were extracted from March 1 to 31, 2018, for all internists and family practitioners who were doing at least 70% full-time equivalent (FTE) outpatient clinical practice at the time of the analysis. We chose March because it is a representative month: neither extremely busy due to influenza season nor slow due to summer vacations. We also obtained physicians' age, sex, years in practice, and their FTE in clinical practice, a measure of their workload in scheduled clinical hours. Internal analyses by this medical group reveal that FTE is strongly correlated with panel size. We thus include patient panel characteristics per physician (mean age, percentage over 65 years of age, percentage female). After preliminary analyses of the 1275 physicians in the dataset, we excluded physicians with <10 working days in the month, and those with no time spent on the inbox on all workdays, leaving 1257 physicians in the study population. We defined a workday for a physician as a day with at least 1 scheduled in-person visit or more than 7 telephone and video visits. We chose this cut point based on the number of telephone encounters among physicians on weekends in March 2018 to properly classify days off when a physician may make a few telephone calls to relay lab results to patients while not scheduled to work. We removed weekend workdays (Saturdays and Sundays) from our analysis of workdays to better distinguish work and nonwork hours, as work hours can vary when physicians are working on weekends. The system logs included a “message type description” field, with 59 different labels. We categorized these labels into higher-level groupings based on an analysis of frequencies and input from this study’s clinical collaborators, who were familiar with the meanings and patterns of different message types. No message content or metadata (ie, sender, receiver, or timestamp) were included in this study, which was approved by the Kaiser Permanente Northern California Institutional Review Board. Statistical analyses To identify factors associated with duration of time spent on EHR inbox work, we compared physicians in the top and bottom quartiles of average inbox use duration, and created multiple regression models. To compare factors between these groups, we conducted a series of 2-group tests (t tests for normally distributed variables, Mann-Whitney U tests for nonparametrically distributed variables, and chi-square tests for categorical variables). We corrected for multiple comparisons using the Bonferroni correction, testing each P value against an alpha value of .05/18 tests = .003. For the regression models, the distributions of dependent variables were inspected to confirm the normality assumption. Multicollinearity was tested using the variance inflation factor, and independent variables with variance inflation factor >5 were removed. Analyses were conducted using the Scipy and Statsmodels packages in Python. RESULTS Participants Of the 1257 physicians in the study, 57% were female. Their ages ranged from 29 to 72 years (mean 46.7 ± 8.4 years of age). Years since medical school graduation ranged from 4 to 45 years of age (mean 19.3 ± 9.0 years of age). Participants’ workload ranged from 0.7 to 1.0 FTE (mean = 0.85 ± 0.11 FTE). On average, physicians in the dataset had 20 ± 3 workdays and 11 ± 3 nonworkdays. The average physician in the study had a patient panel with an average age of 46.78 ± 6.25 years, with 19 ± 10% of patients being older than 65 years of age, and 52 ± 18% being female patients. The average number of patient encounters per physician for the month was 335.26 ± 95.1, of which 237.39 ± 60.58 were face-to-face encounters and 97.87 ± 57.05 were telephone and video encounters. Time spent on inbox management On workdays, the average time spent on inbox work was a total of 52 minutes: 33 minutes during work hours and 19 minutes outside of work hours (Table 1). Thus, 37% of total time on the inbox on a workday occurred outside of formal work hours. On nonworkdays, physicians spent an average of 12 minutes on the inbox, with a range of 0-93 minutes. Table 1. Electronic inbox message views and time spent per day by primary care physicians in the Permanente Medical Group, March 2018 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 Open in new tab Table 1. Electronic inbox message views and time spent per day by primary care physicians in the Permanente Medical Group, March 2018 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 . Mode of inbox access . Desktop/laptop . Mobile . Total: desktop/laptop + mobile . Workdays  Message views   During work hours 94 ± 38 6 ± 11 100 ± 38   Outside work hours 36 ± 25 17 ± 20 53 ± 31   Total 131 ± 47 23 ± 25 153 ± 47  Time spent (min:s)   During work hours 30:55 ± 11:34 2:01 ± 3:34 32:56 ± 11:14   Outside work hours 12:45 ± 9:31 6:09 ± 7:27 18:49 ± 11:52   Total 43:37 ± 15:29 08:10 ± 8:59 51:47 ± 15:30 Nonworkdays  Message views 4 ± 13 29 ± 31 33 ± 34  Time spent (min:s) 1:12 ± 4:30 11:07 ± 12:33 12:19 ± 13:16 Open in new tab On workdays, physicians spent most of their inbox time on desktop or laptop computers, whether within or outside of work hours. On nonworkdays, however, the inbox was mostly accessed through mobile devices. On average, physicians had 100 message views a day during work hours, and 53 views outside work hours (Table 1). On nonworkdays, physicians had an average of 33 message views daily. It is important to note that we cannot ascertain if these were unique message counts, or if they were the number of times physicians started a period of inbox work; rather, they were the number of times physicians switched into an inbox page. For example, it would count as 2 message views if a physician viewed a message, then shifted to another EHR system page to retrieve information, then returned to reply to the same message. Daily patterns of electronic inbox work compared with other EHR work Figure 1 contrasts the daily patterns of time spent on the inbox component of the EHR with time spent on other components of the EHR. For many physicians (Figure 1A), the most time spent on the EHR inbox was before and after formal work hours, and during the lunch hour. Averaging across all users showed small peaks at the beginning and end of the workday. Figure 1. Open in new tabDownload slide Time spent on (A) the EHR inbox and (B) EHR functionality other than the inbox. Top figures show daily averages for each user (1257 users) and the bottom figures show overall average across user averages. Figure 1. Open in new tabDownload slide Time spent on (A) the EHR inbox and (B) EHR functionality other than the inbox. Top figures show daily averages for each user (1257 users) and the bottom figures show overall average across user averages. In contrast, time on other EHR components followed a different pattern. As Figure 1B shows, time spent on EHR functionality other than the inbox (including chart review, order entry, and charting) increased during work hours compared with nonwork hours. There was a sharp decrease in EHR use during the lunch hour as well. On nonworkdays, both overall EHR and inbox use had flatter patterns than workdays. As can be expected, there is a large variance across individual users in the amount of time spent on the EHR and the inbox per hour. Inbox use patterns associated with type of message received We categorized messages into 4 types: (1) messages from patients; (2) results, such as lab test results; (3) requests, which ask the physician to perform an action such as approving a medication refill or signing clinical orders; and (4) informational and administrative messages. An additional category (5) was added for other time on the inbox that was not tied to a specific message. Of all time spent on the EHR inbox, physicians spent 28%, 29%, 25%, and 11% on patient messages, results, requests, and administrative messages, respectively (Table 2). Table 2. Average time spent on each message type during workdays and non-workdays. Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Values are mean ± SD. The overall column indicates mean percentages of total inbox time over the month. Open in new tab Table 2. Average time spent on each message type during workdays and non-workdays. Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Message type . Workdays (min:s) . Nonworkdays (min:s) . Overall (%) . Patient-initiated messages 14:56 ± 6:49 2:24 ± 3:14 28 ± 10 Results 14:01 ± 5:20 4:45 ± 5:45 29 ± 6 Requests 13:21 ± 5:44 2:00 ± 2:33 25 ± 8 Admin 5:30 ± 2:44 1:51 ± 2:32 11 ± 5 Other 2:48 ± 1:37 1:16 ± 2:03 6 ± 3 Values are mean ± SD. The overall column indicates mean percentages of total inbox time over the month. Open in new tab Factors associated with high duration of time on inbox work To further explore different patterns of inbox use, we first compared physicians in our dataset in the top and lowest quartiles of average duration of time spent on the inbox on workdays (during and outside of work hours). This segmentation resulted in 314 users per group. The high-duration group (top quartile) had an average inbox duration of 72 ± 9 minutes per workday, while the low-duration group (lowest quartile) had an average of 33 ± 7 minutes. Table 3 shows these comparisons. The high-duration user group had a higher proportion of women (62%) compared with low-duration user group (50%) (P = .003). There was no statistically significant difference between high- and low-duration users in the number of days worked or FTE. Both groups had an average of 19 (SD = 3) workdays during the month and an average FTE of 0.85 ± 0.1. However, high-duration users had more patient encounters, older patients, and a higher percentage of patients older than 65 years of age and female patients. Table 3. Comparisons of high-duration and low-duration users of the electronic health record–based inbox among primary care physicians in the Permanente Medical Group, 2018 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 Values are mean ± SD, unless otherwise indicated. FTE: full-time equivalent. a Significance of P < .003 based on the Bonferroni correction. Open in new tab Table 3. Comparisons of high-duration and low-duration users of the electronic health record–based inbox among primary care physicians in the Permanente Medical Group, 2018 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 . High-duration inbox users (n = 314) . Low-duration inbox users (n = 314) . P value . Physician characteristics  Age, y 47.2 ± 8.5 46 ± 8.6 .098  Years since medical school graduation 20.3 ± 9 18.2 ± 9.3 .005  Female, % 62 50 .003a Workload  Workdays 18.9 ± 3 18.8 ± 3 .286  FTE 0.85 ± 0.1 0.85 ± 0.1 .291   Total number of patient encounters 347.1 ± 82.7 311.7 ± 112.7 <.001a  Number of face-to-face patient encounters 247 ± 57 216 ± 68.8 <.001a  Number of phone/video patient encounters 100.1 ± 42.1 95.8 ± 66.8 .005 Patient panel characteristics  Patient age, y 48.3 ± 5.9 44.8 ± 6.4 <.001a  Patients older than 65 y, % 21 ± 10% 16 ± 9 <.001a  Female, % 54 ± 18 48 ± 18 <.001a Temporal patterns of inbox work  Ratio of after-hours inbox duration to total inbox duration on workdays, % 41 ± 15 29 ± 16% <.001a  After-hours inbox duration on workdays, min 30.1 ± 13.1 9.7 ± 5.6 <.001a  Inbox duration on nonworkdays, min 16.7 ± 14.9 8.5 ± 10.5 <.001a Message view patterns  Message views per workday (including after hours) 199.9 ± 42.1 108.8 ± 31.2 <.001a  Time per message view, s 22.3 ± 3.8 18.9 ± 3.7 <.001a Use of mobile devices  Duration of inbox work on mobile devices per workday, min 12 ± 11.5 5 ± 5.8 <.001a  % of all inbox time spent in mobile device use 25 ± 18 23 ± 22 .015 Values are mean ± SD, unless otherwise indicated. FTE: full-time equivalent. a Significance of P < .003 based on the Bonferroni correction. Open in new tab High-duration users spent more than twice the duration on messages after work hours on workdays (30 minutes vs 9.7 minutes; P < .001), compared with users in the low-duration group (Table 2). Comparing the ratio of after-hours inbox use duration to all day inbox use duration, we found that a larger average proportion of time on the inbox occurred outside of work hours for high-duration users (41%) compared with low-duration users (29%) (P < .001). We also found differences in the time spent on the inbox on nonworkdays. On nonworkdays, high-duration users spent almost double the time on inbox work relative to low-duration users. Users in the high-duration group averaged almost twice the message views on workdays (199 views vs 109 views; P < .001), and spent slightly more time per message view (22.3 seconds vs 18.9 seconds; P < .001). The duration of time spent on the inbox on mobile devices was also more than doubled for high-duration users, compared with low-duration users, although the ratio of inbox time on mobile to all inbox time was similar between the groups, as both groups spent 23%-25% of their total EHR inbox time on mobile devices. To identify factors independently associated with inbox work duration, we created multiple linear regression models for 2 dependent variables: all-day inbox use duration and after-hours inbox use duration. Both measures were averages over each physician’s workdays. Independent variables included physician characteristics (age and sex), patient panel characteristics (age), patient encounters (face-to-face and telephone and video visits), average percentage of inbox time spent on patient messages, average time per message view, and average percentage of inbox time using mobile devices. For predicting after-hours inbox time, work-hours inbox time was also added as an independent variable. Owing to multicollinearity, we excluded patient panel percentage of female patients, percentage of patients over 65 years of age, physician years of experience, and FTE. Model results are presented in Table 4. Table 4. Regression models predicting duration of all-day inbox work (per 24-hour period) and after-hours inbox work . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 Open in new tab Table 4. Regression models predicting duration of all-day inbox work (per 24-hour period) and after-hours inbox work . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 . All-day inbox work . After-hours inbox work . . Coefficient (SE) . P value . Coefficient (SE) . P value . Physician characteristics  Female 306 (46) <.001 159 (34.64) <.001  Age −17 (3) <.001 −6 (2) .018 Patient panel characteristics  Mean age 50 (4) <.001 30 (3) <.001  Face-to-face appointments 126 (9) <.001 101 (7) <.001  Phone/video appointments 33 (10) .001 21 (7.72) .007 Inbox use patterns  % patient messages 1297 (226) <.001 485 (170) .004  Time per message view 76 (5.76) <.001 55 (4) <.001  % on mobile 347 (145) .017 1431 (113) <.001  Workhours inbox use — — −0.14 (0.03) <.001 Open in new tab For the model of all-day inbox duration all predictors, except physician age, had a positive relationship with inbox duration (F = 63.71, P < .001, adj R2 = .29). The average age of the patient panel, the number of face-to-face patient encounters, and time per message view had the largest effect sizes (standardized coefficients >0.3). For predicting after-hours inbox duration, the analysis showed a negative relationship of the more time physicians spent on the inbox during work hours, the less time they spent on the inbox after hours. A negative relationship was also found between physician age and after-hours inbox use, although the effect size was small. Face-to-face patient encounters, time per message view, and percentage of inbox time that is on mobile devices had the largest effect sizes (standardized coefficients >0.3). Female physicians and physicians who had older patients spent more time on the inbox after hours. The model explains 33% of the variation in after-hours inbox time (F = 68.36, P < .001, adj R2 = .33). DISCUSSION Major findings To our knowledge, this study is the first to describe how electronic inbox work fits temporally into the workdays of PCPs. We found that, on average, PCPs spent 52 minutes on electronic inbox work on workdays, and that more than one-third of this time occurred outside formally scheduled work hours. This study also was unique in our ability to identify characteristics of physicians who spent higher than average amounts of time on inbox work. Those with the highest duration of inbox work were more likely to be female, have a higher percentage of female patients, have older patients, have more patient encounters, and be doing inbox work after hours, as well as spend more time per message view and spend a higher proportion of their inbox time on patient-initiated messages. Interpretation and comparison with past studies Previous studies of electronic inbox use have described the amount and timing of inbox work but have neither studied temporal patterns in depth nor described variation among physicians. A 2017 study found that PCPs spent an average of 5.9 hours per workday on EHR systems, including 1.4 hours outside clinic hours, with 24% of the total time spent in the inbox and a higher proportion of inbox time spent after hours compared with other EHR activities.17 Our study went beyond previous studies17–19 in comparing temporal patterns of electronic inbox use as distinct from use of other parts of the EHR (eg, chart review, order entry, creating notes). We found that inbox use patterns did not mirror work hours, but rather increased before and after work hours and during lunch hours. We also found that the more time physicians spent on the inbox during work hours, the less time they spent on the inbox after hours. These findings were not surprising because during workdays, PCPs usually are scheduled to see patients via in-person, telephone, or video visits for most or all of the available time. Electronic inbox work during formal work hours is typically fit in between patient visits, and work that cannot be finished during those hours is usually addressed just before work hours, just after work hours, or during lunch. Given that the average time spent on the inbox after hours is 19 minutes, which is small relative to the total time spent on EHR activities in other studies, it is possible that some physicians may be prioritizing inbox work during work hours and completing notes outside work hours. Future work analyzing the trade-off between different EHR actions within and outside of work hours could clarify this supposition. This study found that physicians switch to view electronic messages 100 times a day. This number is higher than previous studies of information workers, who checked their emails an average of 77 times a day during work hours.21 The high number of views may be due to physicians needing to switch screens between the inbox and other parts of the EHR to find information to reply to messages or to take other actions, indicating a high rate of task switching within inbox work itself. Given the associations between multitasking and stress,22,23 physician patterns of multitasking with the electronic inbox warrant further study. Our finding that physicians who spent more time on inbox work tended to be female is novel, and consistent with a recent finding that female physicians tend to spend more time on the EHR.24 This contrasts with another recent study in which women reported less EHR-related stress and higher efficiency than men.25 Studies of physicians have suggested that burnout symptoms may be associated with being female, and that work-home conflict may play a role.15,26,27 Our observation that female physicians tend to spend more time on inbox work suggests that this deserves further exploration as a possible factor in sex differences in physician stress. We also found that high inbox work duration was associated with more work outside of work hours. Not only was the absolute amount of time spent working after hours longer, but also the proportion of all time spent on the inbox that occurred after hours was higher. Research in office settings has found spending more time on email associated with greater work overload, frustration, and stress.13 Our finding that inbox work tends to extend beyond work hours for high-duration users suggests that further study is warranted regarding how to best support this group. In this study, inbox work duration was independently associated with clinician workload as measured by the number of appointments seen during the month studied. This is in accord with a previous finding that physicians with more clinical time were disproportionately burdened by after-hours EHR work,28 and another study that found that work relative value units (ie, work volume and complexity) were positively associated with EHR time within and outside of work hours.24 In contrast, we did not find that lower FTE was associated with reduced inbox work. This finding may reflect that physicians who adopt reduced clinical FTE schedules may not experience a commensurate decrease in their amount of electronic inbox work, or conversely, that physicians who feel overburdened may adopt reduced FTE schedules to allow themselves more time to complete clinical work in general, including inbox work. A recent study found that provider-to-provider variation was the largest source of variation in after-hours EHR usage.24 Similarly, the wide variation we found in inbox work duration and use patterns among PCPs in this study suggests that individual preferences and approaches most likely play a role in inbox work patterns. Future research could attempt to identify different types of users based on the strategies they adopt (eg, batching inbox message work),21 the temporal patterns of their work, and their use of mobile devices for inbox work. In addition, future work could investigate the effect of organizational efforts such as designating time for panel management within clinical work hours. It would be useful to study whether inbox use patterns are associated with physician stress or productivity, or patient satisfaction with electronic message communication with their PCPs. The results suggest practical implications for inbox system design. Given the high number of message views (ie counts of switching to the inbox page), a system design consideration would be to implement an interface that incorporates information that physicians need from sources outside the inbox page (eg patient data) to process an inbox message, thus reducing potential frequent switching between the inbox page and other windows. Batching (ie attending to the inbox once or twice a day rather than consistently checking messages throughout the day) and checking the inbox fewer times has been found to be associated with less time on email and less stress.14,21,29 While email batching may be desirable in some work settings, our study’s results show that the temporal patterns of inbox use are different in medical settings, in which physicians spend smaller amounts of time on the inbox during periods when they are scheduled to see patients. It is possible that the messages being checked in between patient visits are those that are more clinically urgent, making batching infeasible. Thus, the feasibility and desirability of batching inbox time in medical contexts is yet to be evaluated. Another practical implication for inbox system design is to implement screening and categorization of patient-initiated messages, automatically or by assistants, which can help PCPs prioritize or delegate some messages. Limitations This study was conducted in a large medical group that encourages patients to use EHR portal messages to communicate with physicians.8 The group also tries to limit the amount of system-generated messages and administrative reminders sent to physicians’ inboxes. Thus, this setting’s volume of inbox messages from patients may be higher and the balance of different types of messages may differ from those in less integrated settings. The dataset for this study had message views but not unique message counts. Thus, we did not analyze the volume of messages, time spent per message, or how many times a message was revisited. As noted in the Materials and Methods, inbox work duration counted time the inbox window was open, but did not count (for example) time when the inbox window was in the background while the user was accessing other parts of the EHR such as chart review or order entry in response to a message. Thus, the duration of inbox work in this study is likely an underestimate of the time physicians spent on inbox-related work. Finally, our analysis did not control for panel management time (ie, time designated by departments specifically for tasks like inbox management). Thus, we cannot make assumptions about why inbox work patterns and peak use times differ among physicians. Our analysis shows an inverse relationship between time on the inbox during and after work hours. It is possible that physicians who have dedicated panel management time during work hours are those who spend less time on inbox work after hours. The effect of panel management time on inbox use patterns needs further study. CONCLUSIONS PCPs in the largest medical group in the United States spend roughly an hour per workday on inbox management, and much of this work occurs outside scheduled work hours. In this setting, patient-initiated messages and results consume the highest proportion of inbox work time. Interventions to assist PCPs in handling patient-initiated messages and results may help alleviate inbox workload. FUNDING This work was supported by the Permanente Medical Group via its Delivery Science Research Program, and by the National Science Foundation under Grant No. 1704889. The funder (Delivery Science grant) supported the effort of the study team and had no specific requirements regarding interpretation of results or framing of the manuscript. The National Science Foundation grant partially supported FA in collection, management, analysis, and interpretation of the data and GM in interpretation, preparation, and review. The views, opinions, and/or findings contained in this publication are those of the authors and do not necessarily reflect the views of Kaiser Permanente and should not be construed as an official position, policy or decision of Kaiser Permanente unless so designated by other documentation. No official endorsement should be made. AUTHOR CONTRIBUTIONS EMW collected and preprocessed the data. FA analyzed the data and drafted the manuscript. TAL, MR, and GM obtained funding and provided supervision. All authors contributed to interpreting results, refining all sections and critically editing the manuscript. ACKNOWLEDGMENTS We appreciate the contributions of Manuel Ballesca, MD, and Brian Hoberman, MD, to the design of this study, as well as the participation of Andres Montoya, BS, and Lynn Emerson, MS, on the project team. We are grateful to the study’s sponsors, TPMG Associate Executive Directors Yi-Fen Irene Chen, MD; Sameer Awsare, MD; Patricia Conolly, MD; Edward Lee, MD; and Philip Madvig, MD. CONFLICT OF INTEREST STATEMENT None declared. REFERENCES 1 Lieu TA Freed GL. Unbounded–parent-physician communication in the era of portal messaging . JAMA Pediatr 2019 ; 173 ( 9 ): 811 – 2 . Google Scholar Crossref Search ADS WorldCat 2 Reed ME Huang J Brand R , et al. Communicating through a patient portal to engage family care partners . JAMA Intern Med 2018 ; 178 ( 1 ): 142 – 4 . Google Scholar Crossref Search ADS PubMed WorldCat 3 Reed ME Huang J Millman A , et al. Portal use among patients with chronic conditions: patient-reported care experiences . Med Care 2019 ; 57 ( 10 ): 809 – 14 . Google Scholar Crossref Search ADS PubMed WorldCat 4 Patt MR Houston TK Jenckes MW Sands DZ Ford DE. Doctors who are using e-mail with their patients: a qualitative exploration . J Med Internet Res 2003 ; 5 ( 2 ): e9 . Google Scholar Crossref Search ADS PubMed WorldCat 5 Car J Sheikh A. Email consultations in health care: 1—scope and effectiveness . BMJ 2004 ; 329 ( 7463 ): 435 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 6 Moyer CA Stern DT Katz SJ Fendrick AM. “ We got mail”: electronic communication between physicians and patients . Am J Manag Care 1999 ; 5 ( 12 ): 1513 – 22 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 7 Reed M Graetz I Gordon N Fung V. Patient-initiated e-mails to providers: associations with out-of-pocket visit costs, and impact on care-seeking and health . Am J Manag Care 2015 ; 21 ( 12 ): e632 – 9 . Google Scholar PubMed OpenURL Placeholder Text WorldCat 8 Lieu TA Altschuler A Weiner JZ , et al. Primary care physicians’ experiences with and strategies for managing electronic messages . JAMA Netw Open 2019 ; 2 ( 12 ): e1918287 . Google Scholar Crossref Search ADS PubMed WorldCat 9 Adler-Milstein J Holmgren AJ Kralovec P Worzala C Searcy T Patel V. Electronic health record adoption in US hospitals: the emergence of a digital “advanced use” divide . J Am Med Inform Assoc 2017 ; 24 ( 6 ): 1142 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 10 Crotty BH Tamrat Y Mostaghimi A Safran C Landon BE. Patient-to-physician messaging: volume nearly tripled as more patients joined system, but per capita rate plateaued . Health Aff (Millwood) 2014 ; 33 ( 10 ): 1817 – 22 . Google Scholar Crossref Search ADS PubMed WorldCat 11 Shenson JA Cronin RM Davis SE Chen Q Jackson GP. Rapid growth in surgeons’ use of secure messaging in a patient portal . Surg Endosc 2016 ; 30 ( 4 ): 1432 – 40 . Google Scholar Crossref Search ADS PubMed WorldCat 12 Renaud K Ramsay J Hair M. “ You’ve got e-mail!”… shall I deal with it now? Electronic mail from the recipient’s perspective . Int J Hum Comput Interact 2006 ; 21 ( 3 ): 313 – 32 . Google Scholar Crossref Search ADS WorldCat 13 Barley SR Meyerson DE Grodal S. E-mail as a source and symbol of stress . Organ Sci 2011 ; 22 ( 4 ): 887 – 906 . Google Scholar Crossref Search ADS WorldCat 14 Mark G Voida S Cardello A. A pace not dictated by electrons: an empirical study of work without email. In: proceedings of the SIGCHI Conference on Human Factors in Computing Systems ; 2012 : 555 – 564 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 15 Tai-Seale M Dillon EC Yang Y , et al. Physicians’ well-being linked to in-basket messages generated by algorithms in electronic health records . Health Aff (Millwood) 2019 ; 38 ( 7 ): 1073 – 8 . Google Scholar Crossref Search ADS PubMed WorldCat 16 Murphy DR Meyer AN Russo E Sittig DF Wei L Singh H. The burden of inbox notifications in commercial electronic health records . JAMA Intern Med 2016 ; 176 ( 4 ): 559 – 60 . Google Scholar Crossref Search ADS PubMed WorldCat 17 Arndt BG Beasley JW Watkinson MD , et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations . Ann Fam Med 2017 ; 15 ( 5 ): 419 – 26 . Google Scholar Crossref Search ADS PubMed WorldCat 18 Wang JK Ouyang D Hom J Chi J Chen JH. Characterizing electronic health record usage patterns of inpatient medicine residents using event log data . PLoS One 2019 ; 14 ( 2 ): e0205379 . Google Scholar Crossref Search ADS PubMed WorldCat 19 Ouyang D Chen JH Hom J Chi J. Internal medicine resident computer usage: an electronic audit of an inpatient service . JAMA Intern Med 2016 ; 176 ( 2 ): 252 – 4 . Google Scholar Crossref Search ADS PubMed WorldCat 20 Gordon N. Highlights of Results of the Kaiser Permanente Northern California 2014/2015 Member Health Survey . Oakland, CA : Division of Research, Kaiser Permanente Medical Care Program ; 2017 . Google Scholar Google Preview OpenURL Placeholder Text WorldCat COPAC 21 Mark G Iqbal ST Czerwinski M Johns P Sano A Lutchyn Y. Email duration, batching and self-interruption: Patterns of email use on productivity and stress In: proceedings of the 2016 CHI Conference on Human Factors in Computing Systems ; 2016 : 1717 – 28 . Google Scholar Crossref Search ADS Google Preview WorldCat COPAC 22 Mark G Wang Y Niiya M. Stress and multitasking in everyday college life: an empirical study of online activity In: proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems - CHI ’14; 2014 : 41 – 50 . doi: 10.1145/2556288.2557361 23 Reinecke L Aufenanger S Beutel ME , et al. Digital stress over the life span: the effects of communication load and internet multitasking on perceived stress and psychological health impairments in a German probability sample . Media Psychol 2017 ; 20 ( 1 ): 90 – 115 . Google Scholar Crossref Search ADS WorldCat 24 Attipoe S Huang Y Schweikhart S Rust S Hoffman J Lin S. Factors associated with electronic health record usage among primary care physicians after hours: retrospective cohort study . JMIR Hum Factors 2019 ; 6 ( 3 ): e13779 . Google Scholar Crossref Search ADS PubMed WorldCat 25 Khairat S Coleman C Ottmar P Bice T Koppel R Carson SS. Physicians’ gender and their use of electronic health records: findings from a mixed-methods usability study . J Am Med Inform Assoc 2019 ; 26 ( 12 ): 1505 – 14 . Google Scholar Crossref Search ADS PubMed WorldCat 26 Houkes I Winants Y Twellaar M Verdonk P. Development of burnout over time and the causal order of the three dimensions of burnout among male and female GPs. A three-wave panel study . BMC Public Health 2011 ; 11 ( 1 ): 240 . Google Scholar Crossref Search ADS PubMed WorldCat 27 Innstrand ST Langballe EM Falkum E Aasland OG. Exploring within-and between-gender differences in burnout: 8 different occupational groups . Int Arch Occup Environ Health 2011 ; 84 ( 7 ): 813 – 24 . Google Scholar Crossref Search ADS PubMed WorldCat 28 Saag HS Shah K Jones SA Testa PA Horwitz LI. Pajama time: working after work in the electronic health record . J Gen Intern Med 2019 ; 34 ( 9 ): 1695 – 2 . Google Scholar Crossref Search ADS PubMed WorldCat 29 Kushlev K Dunn EW. Checking email less frequently reduces stress . Comput Hum Behav 2015 ; 43 : 220 – 8 . Google Scholar Crossref Search ADS WorldCat © The Author(s) 2020. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model)

Journal

Journal of the American Medical Informatics Association – Oxford University Press

Published: Oct 15, 2020

There are no references for this article.