Measurement of clinical documentation burden among physicians and nurses using e | Open-access article

Abstract

Objective

Electronic health records (EHRs) are linked with documentation burden resulting in clinician burnout. While clear classifications and validated measures of burnout exist, documentation burden remains ill-defined and inconsistently measured. We aim to conduct a scoping review focused on identifying approaches to documentation burden measurement and their characteristics.

Materials and Methods

Based on Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Extension for Scoping Reviews (ScR) guidelines, we conducted a scoping review assessing MEDLINE, Embase, Web of Science, and CINAHL from inception to April 2020 for studies investigating documentation burden among physicians and nurses in ambulatory or inpatient settings. Two reviewers evaluated each potentially relevant study for inclusion/exclusion criteria.

Results

Of the 3482 articles retrieved, 35 studies met inclusion criteria. We identified 15 measurement characteristics, including 7 effort constructs: EHR usage and workload, clinical documentation/review, EHR work after hours and remotely, administrative tasks, cognitively cumbersome work, fragmentation of workflow, and patient interaction. We uncovered 4 time constructs: average time, proportion of time, timeliness of completion, activity rate, and 11 units of analysis. Only 45.0% of studies assessed the impact of EHRs on clinicians and/or patients and 40.0% mentioned clinician burnout.

Discussion

Standard and validated measures of documentation burden are lacking. While time and effort were the core concepts measured, there appears to be no consensus on the best approach nor degree of rigor to study documentation burden.

Conclusion

Further research is needed to reliably operationalize the concept of documentation burden, explore best practices for measurement, and standardize its use.

INTRODUCTION

Rapid adoption of electronic health records (EHRs) following the passage of the Health Information Technology for Economic and Clinical Health (HITECH) Act has led to advances in both individual- and population-level health.¹ HITECH has improved healthcare quality, patient safety, and diagnostic accuracy through enhanced data management and timely reuse; interoperable systems have facilitated care continuity and monitoring of compliance metrics.^2–5 EHR-facilitated, guideline-based care has been associated with reduced redundancies⁶^,⁷ and streamlined billing administration.⁸

Largely still in its infancy, the implementation of EHRs has also resulted in unintended consequences on clinical practice and healthcare systems, including significant increases in clinician documentation time.^9–13 Extended work hours, time constraints, clerical workload, and disruptions to the patient-provider encounter, have led to a rise in discontent with existing documentation methods in EHR systems.⁶^,¹⁴^,¹⁵ This documentation burden has been linked to increases in medical errors,³^,⁹^,¹⁶ threats to patient safety,³^,⁹^,¹⁶ inferior documentation quality,¹⁷^,¹⁸ job attrition, and, ultimately, burnout among nurses and physicians.³^,^9–11^,¹⁴^,^16–22

In concert with Affordable Care Act (ACA) reimbursement models, Meaningful Use (MU) mandates, and a regulatory-rich environment, EHRs have drastically altered clinical documentation workflow and communication in routine healthcare.¹³^,¹⁵^,²³ Physicians have reported willingness to remain out of compliance with EHR incentive programs (eg, MU and the Physician Quality Reporting System²⁴) in favor of mitigating documentation burden (hereinafter referred interchangeably as “burden”).¹⁵^,²⁵ Still, studies consistently demonstrate that physicians spend twice as much time on electronic documentation and clerical tasks as compared to time providing direct patient care.¹⁴^,^26–30 Similarly, nurses devote more than half of their shift time to EHR data entry and retrieval¹⁹^,²⁰ and report reduced direct patient contact.³¹^,³²

While researchers have discussed the challenges of burden and its implications for clinician burnout due to EHRs over the past decade,⁵^,¹⁵^,³³ limited attention has been paid to discriminating the antecedent concept of burden (defined as a duty, responsibility, etc, that causes worry, difficulty, or hard work),³⁴ from burnout (defined as long-term work-related stress reaction marked by emotional exhaustion, depersonalization, and a lack of sense of personal accomplishment).³⁵^,³⁶ Clinician burnout has been well-documented and widely quantified using surveys and psychological measurements throughout peer-reviewed literature.^37–40 Yet, to our best knowledge, there is a lack of consensus on approaches to measure burden.¹⁵^,³⁷^,^41–45

While EHR dissatisfaction has been extensively studied and some clinician activity metrics have been proposed,⁴⁶ few empirically-based readily-available solutions to reduce burden exist.¹¹ Interventions to assuage burden have ranged from the utilization of scribes and remote transcription services ²⁷ to text summarization and dictation software.¹⁶^,⁴⁷ In March 2020, the Department of Health and Human Services (HHS) released a report outlining 3 primary goals to reduce EHR-related clinician burdens that influence care: reduce the time and effort clinicians require to document health information, reduce the effort required to meet regulatory requirements, and improve EHR ease of use.⁴⁸ Evaluating the impact of interventions that target these goals will necessitate standardized, quantitative measurements.

OBJECTIVE

The purpose of this scoping review is to assess the state of science, identify gaps in knowledge, and synthesize characteristics of documentation burden measurement among physicians and nurses using EHRs.

MATERIALS AND METHODS

We conducted a scoping review using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) extension for Scoping Reviews (ScR) guidelines.⁴⁹ A scoping review fit our objective to describe the breadth of methods used to measure documentation burden.⁴⁹

Search strategy and selection criteria

We systematically searched the MEDLINE, Embase, Web of Science, and CINAHL databases for all English-language studies published in peer-reviewed journals and conference proceedings, investigating documentation burden among physicians and/or nurses in ambulatory and/or inpatient settings from inception to April 20, 2020. We evaluated all relevant literature identified through in-text references among eligible studies. Burden is not specifically represented in Medical Subject Headings (MeSH); therefore, we explored both keyword and MeSH terms for 2 burden-related concepts outlined in the HHS report Strategy on Reducing Burden Relating to the Use of Health IT and EHRs⁴⁸ documentation: (a) effort, and (b) time. We also focused our search on: (a) the EHR and (b) physicians or nurses. The finalized search strategy is summarized in Table 1.

Table 1. Summary of search terms and query employed to each academic literature database in our review

Concept	Search Strings	Operator
documentation time	(“Task Performance and Analysis”[Mesh]) OR (“Costs and Cost Analysis”[Mesh]) OR (“Time Factors”[Mesh]) OR (“Process Assessment, Health Care”[Mesh]) OR (“time”) OR (“Measure”) OR (“measurement”) OR (“quantify”) OR (“quanti*”) OR (“metric”)	AND
documentation effort	(“Documentation”[Mesh]) OR (“documentation”) OR (“note”) or (“unstructured data”) OR (“narrative”) OR (“Burnout, Professional”[Mesh]) OR (“Cognition”[Mesh]) OR (“Cognitive load”) OR (“Burnout”) OR (“burden”)	AND
EHR	(“Electronic Health Records”[Mesh]) OR (“electronic health record”) OR (“electronic medical record”) OR (“EHR”) OR (“EMR”) OR (“computerized medical record”)	AND
physicians/nurses	(“Physicians”[Mesh]) OR (“Nurses”[Mesh]) OR (“nurse”) OR (“physician”)

Study selection and selection criteria

We selected inclusion and exclusion parameters a priori, and iteratively modified them to exclude studies involving niche clinical systems and those strictly comparing to paper-based documentation (Table 2). We included all peer-reviewed primary studies that focused on EHR utilization with an objective time or effort measure⁴⁸ (eg, EHR usage logs, which report time stamped documentation events) in the review.

Table 2. Inclusion and exclusion criteria

Inclusion	Exclusion
Peer-reviewed publications and conference proceedingsPrimary studiesObjective time measures via EHR usage logs or other digital time capture toolsFocus on EHR utilizationAmbulatory setting and/or inpatient settingPhysicians and/or nurses	Comparison to paper-based systems onlyNiche clinical systems (eg, radiology system, medication ordering system,) and applicationsLaboratory-based studies of prototype systems (ie, not yet used in clinical setting)Qualitative studyNot accessible in the full textNot English language

The term “physicians” encompassed attending physicians, fellows, resident physicians, and interns; “nurses” referred to registered nurses. We focused on physicians and nurses given our aim of identifying interprofessional measurements of documentation burden. We excluded studies comparing EHR documentation to paper-based systems if they were not focused on measuring burden, but rather on EHR implementation evaluation.

After removing duplicates, 2 reviewers (AJM and JMS, AJM and RC, AJM and SS, or AJM and EL) independently screened article titles and abstracts for relevance using Covidence.⁵⁰ Two authors (with a third serving as a tiebreaker) reviewed each potentially relevant abstract for eligibility criteria in the full-text. We included full-text articles with concordant decisions by the 2 reviewers in the final analysis; for discordant decisions, all reviewers reexamined and adjudicated until a consensus was reached.

Data extraction and analysis

One author (AJM) performed data charting for all articles meeting full-text inclusion criteria (see online Supplementary Table), which was reviewed by all authors and discussed. We extracted the following information: publication year, geographic location, time source, unit(s) of analysis, activity, sample size, sample characteristics, EHR system, provider role/specialty, clinical setting, study design and objectives, study type (eg, quantitative or mixed-methods), site type (eg, single or multisite), exposure and outcome measures, analytical and statistical methods, study limitations/bias, and major findings. We reported study limitations and biases such as threats to internal and external validity to appraise rigor. We used the HHS concepts to organize our reporting of measurement characteristics.⁴⁸ HHS does not elaborate further on definitions of: (a) time, (b) effort, and (c) outcomes assessed⁴⁸; therefore, we conducted purposeful thematic analysis to identify proxies and synthesize these 3 recurring concepts.⁵¹ We iteratively combined themes until we achieved a consensus.

RESULTS

Sources of evidence

Our search strategy yielded 3482 potentially relevant manuscripts from MEDLINE (n = 507), Embase (n = 1143), Web of Science (n = 1007), and CINAHL (n = 825). Seven additional manuscripts were identified through in-text references. After eliminating duplicates, 1946 titles/abstracts were screened; of those, 166 were eligible for full-text review. Consensus was achieved for all disagreements concerning the inclusion of full-text articles. Thirty-five studies meeting criteria were summarized in the final analysis (Figure 1).

Figure 1. PRISMA flow diagram for scoping review of eligible studies. (Figure not reproduced here; see the original article.)

Study characteristics

Studies were conducted in the United States (n = 31),¹³^,¹⁴^,²²^,²⁷^,²⁸^,⁴³^,^52–76 Europe (n = 1),²⁹ and Asia (n = 3).^77–79 Studies included a mix of ambulatory (n = 22)¹⁴^,²²^,^27–29^,^52–54^,⁵⁶^,^61–66^,⁶⁹^,^71–75 and hospital (ie, inpatient and emergency) settings (n = 11)¹³^,⁵⁵^,⁵⁷^,⁵⁹^,⁶⁰^,⁶⁷^,⁶⁸^,⁷⁰^,⁷⁶^,⁷⁸^,⁷⁹ with 2 involving both.⁵⁸^,⁷⁷ A majority of those studies involved single sites (77.1%) and were affiliated with an academic institution/teaching hospital (80.0%). One third used Epic systems (n = 13),¹³^,²²^,²⁷^,⁴³^,⁵³^,⁵⁴^,⁵⁶^,⁵⁷^,⁶⁴^,⁶⁶^,⁶⁹^,⁷²^,⁷³ followed by multiple/other/unspecified (n = 12),¹⁴^,²⁸^,²⁹^,⁵²^,⁶³^,⁶⁷^,⁶⁸^,^75–79 Cerner (n = 6),^58–60^,⁶⁵^,⁷¹^,⁷⁴ Allscripts (n = 2),⁶¹^,⁶² and Eclipsys (n = 2).⁵⁵^,⁷⁰

Articles were published between 2010 and 2020 with 2018 (n = 8)¹³^,¹⁴^,²⁹^,⁵⁶^,⁵⁷^,⁶³^,⁶⁷^,⁷⁶ and 2019 (n = 8)^{22,54,58,69,72,74,6066} representing the highest volumes. Range of study sample sizes was expansive among the studies (4 ≤ n ≤ 154 719). Most studies exclusively focused on physicians (n = 25)^13–2942 as compared to nurses (n = 5)⁵⁸^,⁶⁷^,^76–78 or an interprofessional sample of providers (n = 5).²²^,⁵⁵^,⁵⁶^,⁶⁹^,⁷³ Clinician specialties were heterogeneous; over half the studies involved single specialties (general [n = 11],¹⁴^,²⁷^,⁵²^,⁵³^,⁶¹^,⁶²^,⁶⁴^,^71–73^,⁷⁵ emergency [n = 2],⁵⁷^,⁷⁹ intensivist [n = 2],⁶⁷^,⁷⁰ other [(n = 5]¹³^,⁵⁴^,⁵⁶^,⁷⁴^,⁷⁸), while the remaining were multiple subspecialties (n = 13)⁹^,²²^,²⁸^,⁴³^,^58–60^,⁶³^,⁶⁵^,⁶⁶^,⁶⁹^,⁷⁶^,⁷⁷ or unspecified (n = 2).⁵⁵^,⁶⁸ Across all studies, most involved general medicine (n = 17)¹⁴^,²²^,²⁷^,²⁸^,⁴³^,⁵²^,⁵³^,^59–66^,⁶⁹^,^71–73^,^75–77 followed by surgical subspecialties (n = 8),¹³^,²⁹^,⁵⁸^,⁵⁹^,⁶⁶^,⁷⁴^,⁷⁷^,⁷⁸ intensive care (n = 6),⁵⁸^,⁵⁹^,⁶⁷^,⁷⁰^,⁷⁶ and emergency medicine (n = 4) ⁵⁷^,⁵⁸^,⁶⁰^,⁷⁹; 10 included other subspecialties.²²^,²⁸^,²⁹^,⁵⁴^,⁵⁶^,⁵⁸^,⁶⁰^,⁶³^,⁶⁵^,⁶⁶

Thirty were strictly quantitative studies. While purely qualitative studies were excluded, 5 studies employed mixed methods²⁸^,⁵²^,⁵⁵^,⁶¹^,⁶² (see online Supplementary Table). Study designs varied, including time-and-motion (TM [n = 5]²⁸^,⁶¹^,⁶²^,⁶⁷^,⁶⁸), validation of TM (n = 2),²⁷^,⁷⁰ cohort (n = 15),¹³^,²⁷^,⁴³^,⁵⁴^,⁵⁵^,⁵⁷^,⁵⁹^,⁶⁰^,⁶³^,⁶⁴^,⁶⁶^,⁷²^,⁷⁴^,⁷⁵^,⁷⁷ experimental/quasi-experimental (n = 8),¹⁴^,²²^,²⁹^,⁵³^,⁵⁶^,⁵⁸^,⁷⁸^,⁷⁹ and cross-sectional studies (n = 4).⁶⁹^,⁷¹^,⁷³^,⁷⁶ Eight studies evaluated an intervention,¹⁴^,²²^,⁵²^,⁵³^,⁵⁶^,⁵⁸^,⁷⁵^,⁷⁸ including scribes (n = 3),¹⁴^,⁵²^,⁵³ documentation redesign (n = 3),⁵⁸^,⁷⁵^,⁷⁸ or EHR training programs (n = 2)²²^,⁵⁶; the remaining were descriptive studies on EHR activities and usage (n = 27)—2 of which involved the implementation of new EHR systems.²⁹^,⁷⁹

A diversity of analytical methods was employed. Most studies to which statistical testing were relevant (n = 23) applied parametric (n = 19) as opposed to non-parametric methods (n = 12). Qualitative methods employed in the mixed-methods studies involved informal interviews,⁶² social network analysis,⁵⁵ thematic analysis,⁶² focus groups,⁵² and self-reported diary.²⁸ Few studies addressed validity or reliability of measurements in their studies (n = 11) ²²^,⁵²^,⁵³^,⁵⁹^,⁶⁰^,⁶³^,⁶⁴^,⁶⁷^,⁶⁹^,⁷³^,⁷⁸; 2 examined interobserver reliability,²⁸^,⁶⁸ 2 employed TM approaches to validate novel analytical methods to examine workflow⁷⁰ and the use of EHR usage logs to estimate workload,²⁷ 2 examined correlations between self-reported and objective EHR usage log times,²²^,⁷³ and 1 employed video recording timers to validate EHR usage log times.⁵⁸

Characterization of effort

Seven overarching effort constructs emerged (Table 3): (a) general workload such as overall EHR usage (n = 4)⁵³^,⁵⁶^,⁶⁸^,⁶⁹; (b) clinical documentation/review (n = 15)²⁸^,²⁹^,⁵⁵^,^57–61^,⁶⁷^,⁷²^,^75–79; (c) excess workload including EHR usage after hours (n = 15)¹³^,²²^,²⁷^,^52–54^,⁵⁹^,^63–66^,⁶⁹^,⁷¹^,⁷³^,⁷⁴ and remote access (n = 1)⁷²; (d) administrative tasks, such as inbox management (n = 2)⁶⁹^,⁷³; (e) cognitively cumbersome work, such as multitasking (n = 3)⁶¹^,⁶²^,⁶⁸; (f) fragmentation of EHR workflow (n = 1)⁷⁰; and (g) patient interaction/in-person visits (n = 7).¹⁴^,²⁸^,²⁹^,⁴³^,⁵³^,⁶²^,⁶⁸ Several terms were employed referring to EHR usage afterhours including “work after work,”⁶⁶ “pajama time,”⁶⁶ and “Clinician Logged-In Outside Clinic” (CLOC) time.²² For example, Cox et al proposed the “amount of EHR usage taking place after scheduled duty hours” specifically for surgical residents.¹³

Table 3. Identified measurement characteristics from study findings

Documentation Burden Concepts	Measurement Constructs
Effort	EHR usage and workload
	Clinical documentation/review
	EHR work afterhours and remotely
	Administrative tasks (eg, inbox management)
	Cognitively cumbersome work (eg, multitasking)
	Fragmentation of workflow
	Patient interaction
Time	Average time spent
	Proportion or percentage of time spent
	Binary of timeliness of completion (eg, documenting within shift or policy time frame)
	Activity rate
Units of analysis	Clinically-oriented units of analysis	Temporally-oriented units of analysis
	Encounter	Seconds
	Encounter	Minutes
	Provider	Minutes
	Patient	Seconds
	Patient	Minutes
	Event/Task	Seconds
		Minutes
		Hours
		Shifts
		Days
		Weeks
		Months

Measurement of time

Time spent documenting was assessed in all studies and was measured using 3 key data collection strategies: EHR usage logs (n = 28),¹³^,¹⁴^,²²^,²⁷^,⁴³^,^53–60^,^63–67^,⁶⁹^,^71–79 activity capture applications (n = 8),^27–29^,⁵²^,⁶¹^,⁶²^,⁶⁸^,⁸⁰ and video recordings (n = 1).⁵⁸ Few studies triangulated these data through multiple data collection strategies (n = 2).²⁷^,⁵⁸ Time constructs identified (Table 3) include (a) average time spent (n = 20),²²^,²⁷^,²⁹^,⁴³^,⁵⁴^,⁵⁵^,⁵⁷^,^59–61^,^63–67^,⁶⁹^,^71–73^,⁷⁸ (b) proportion or percentage of time spent (n = 10),¹³^,²⁸^,⁵³^,⁵⁶^,⁶²^,⁶⁸^,⁷⁰^,⁷²^,⁷⁴^,⁷⁵ (c) binary of timeliness of completion (n = 1),⁷⁷ and (d) activity rate (n = 2).⁶¹^,⁷⁶ Units of analysis varied within and across studies (Table 3), including time reported per: (a) encounter (n = 5),⁵⁴^,⁶⁰^,⁶⁵^,⁶⁷^,⁶⁹ (b) provider (n = 2),¹⁴^,⁷³ (c) patient (n = 3),⁵⁷^,⁵⁹^,⁷⁸ or (d) event/task (n = 28).¹³^,¹⁴^,²²^,^27–29^,⁴³^,^53–56^,⁵⁸^,^61–64^,⁶⁶^,⁶⁸^,^70–79 Units of analysis also included average hours per day, per week, or per month (n = 6)²²^,²⁹^,⁴³^,⁶³^,⁷¹^,⁷² and average minutes per day, per week, per shift, or per clinical full-time equivalent per week (n = 7).²⁷^,⁵⁵^,⁶¹^,⁶⁴^,⁶⁶^,⁷³^,⁷⁸ We have organized these units of analysis into 2 levels for combination in individual measures: (a) a clinically oriented unit of analysis, such as “per encounter,” and (b) a temporally oriented unit of analysis, such as “per hour” (see Table 3). Operationalization of a shift and “active versus idle” time in the EHR also varied. Among the 15 studies that examined shifts,¹³^,²²^,²⁷^,⁵⁴^,⁵⁶^,⁵⁷^,⁵⁹^,⁶⁴^,⁶⁵^,⁶⁸^,⁶⁹^,⁷¹^,⁷³^,⁷⁴^,⁷⁶ 9 distinct shift times were identified with 6:00 am–6:00 pm (n = 4),¹³^,⁶⁵^,⁷¹^,⁷⁴ 7:00 am–7:00 pm (n = 3),⁶⁹^,⁷³^,⁷⁶ and 8:00 am–6:00 pm (n = 2)²²^,²⁷ representing the most frequently reported intervals. Meanwhile, only half the studies employing EHR usage logs explicitly operationalized active versus idle time in the EHR to account for the time a clinician is logged in but not actively using the system. However, determination of “active and idle” time were measured at different levels of granularity (ie, complete system time-out [n = 3]¹³^,⁴³^,⁷³ vs “active versus idle” between tasks [n = 11]²²^,²⁷^,⁵⁶^,⁵⁹^,⁶⁰^,⁶⁴^,⁶⁵^,⁶⁹^,⁷¹^,⁷²^,⁷⁴). “Active versus idle” activity time was largely vendor defined (n = 7),²²^,⁵⁹^,⁶⁰^,⁶⁵^,⁶⁹^,⁷¹^,⁷⁴ relied on mouse clicks and keystrokes (n = 5),⁵⁹^,⁶⁰^,⁶⁵^,⁷¹^,⁷⁴ and/or idle time between 30 seconds and 10 minutes of length (n = 5).²⁷^,⁵⁶^,⁶⁴^,⁶⁹^,⁷²

Outcome assessment

Less than half the studies assessed the impact of documentation burden on clinicians and/or patients (n = 16). Among those studies, authors referenced the temporal relationship between burden and burnout at a higher proportion (68.8%) compared to those that did not extend beyond measuring time and effort alone (50.0%). Outcomes measured included clinical process measures [n = 8 (ie, treatment time, encounter closure, length of stay)¹⁴^,⁵⁴^,⁵⁷^,⁶⁹^,⁷⁹], clinician (n = 7)¹⁴^,²²^,⁵²^,⁵³^,⁷⁵^,⁷⁸^,⁷⁹ and patient satisfaction (n = 4),¹⁴^,⁵²^,⁵³^,⁶³ burnout/stress (n = 5),²²^,⁶⁴^,⁶⁹^,⁷³^,⁷⁵ patient census/mortality (n = 2),⁵⁹ response to messages (n = 1),²² and team interactions (n = 1).⁵⁵ Primary predictors and outcomes of interest are summarized in the online Supplementary Table.

Limitations and biases reported

Two limitations were ubiquitous across included studies (Table 4): (a) threats to generalizability due to constraints in sample size (n = 19),¹⁴^,²⁸^,²⁹^,^52–54^,⁵⁷^,^59–64^,⁶⁷^,⁶⁹^,⁷⁰^,⁷³^,⁷⁴^,⁷⁹ study setting (n = 21),²²^,²⁸^,^52–55^,^59–62^,⁶⁴^,^68–70^,^72–74^,^76–79 patient population,⁵⁷^,⁷⁷ EHR system (n = 6),⁵⁸^,⁶⁰^,⁶¹^,⁷⁰^,⁷⁵^,⁷⁸^,⁸¹ activity type,⁷⁶ clinician role or seniority,⁵⁷^,⁵⁹^,⁶¹^,⁶⁹^,⁷⁰ early adoption,⁴³ and/or subspecialty;⁶²^,⁶⁴^,⁷⁰ and, (b) measurement error including the inability of logs to distinguish between “idle and active” time (n = 6),²⁷^,⁴³^,⁵⁵^,⁶⁴^,⁷³^,⁸⁰ uncertainty regarding the definition of “afterhours,”⁵⁹^,⁷³ incomplete measurement of tasks (n = 15),¹³^,²⁷^,²⁹^,⁴³^,^56–58^,⁶⁵^,^68–71^,⁷⁶^,⁷⁸^,⁸⁰ imprecision of time capture,²⁷^,⁴³^,⁵⁵^,⁷³^,⁸⁰ information bias (n = 10),^27–29^,⁵⁶^,⁶¹^,⁶⁸^,⁷⁰ and validity of measures.⁵³^,⁵⁴^,⁶⁴

Table 4. Study limitations identified in the review

Author (Year)	generalizability	small sample size	selection bias	response bias	measurement error	misclassification	information bias	no data triangulation	confounding	self-reported data
Adler-Milstein et al (2020)	•	•			•	•				•
Ahn et al (2016)	•
Anderson et al (2020)	•				•				•
Arndt et al (2017)	•		•			•	•
Aziz et al (2019)	•	•			•	•				•
Carlson et al (2015)	•		•	•	•				•	•
Collins et al (2018)	•				•			•
Cox et al (2018)	•				•	•		•	•
DiAngi et al (2019)	•		•	•	•					•
Earls et al (2017)	•	•	•		•		•	•	•	•
Gidwani et al (2017)	•	•			•			•		•
Goldstein et al (2019)	•	•			•			•	•
Hripcsak et al (2011)	•				•			•
Hsieh et al (2016)	•				•					•
Inokuchi et al (2015)	•	•	•	•						•
Joukes et al (2018)	•	•	•		•		•		•
Kadish et al (2018)	•			•	•		•	•		•
Kannampallil et al (2018)	•	•	•		•	•			•
Karp et al (2019)	•				•
Krawiec et al (2019)	•	•			•			•
Krawiec et al (2020)	•	•			•				•
Mamykina et al (2012)	•	•			•		•
Mamykina et al (2016)	•	•	•		•		•		•
Marmor et al (2018)	•	•			•			•	•	•
Micek et al (2020)	•	•		•	•	•				•
Mishra et al (2018)		•	•				•	•		•
Overhage et al (2020)	•				•	•			•
Saag et al (2019)								•
Sinsky et al (2016)	•	•	•		•		•		•	•
Smith et al (2018)	•	•			•
Tai-Seale et al (2017)	•		•		•	•
Tipping et al (2010)	•				•		•
Tran et al (2019)	•	•			•				•	•
Wang et al (2019)	•				•
Zheng et al (2010)	•	•	•		•		•

Six studies cited selection bias derived from both the presence of self-selection and voluntary participation among high-performing subjects²⁷^,²⁸ and the presence of low response.²²^,⁵⁶^,⁶⁴^,⁷⁵ Eleven studies noted a lack of data triangulation, such as combining log data with direct observations, encounter information or qualitative data to offer contextual information corresponding to types of EHR interfaces used (eg, remote, inpatient, outpatient) for login timestamps, direct patient care, and other data.¹³^,¹⁴^,^52–56^,⁶⁰^,⁶³^,⁶⁶^,⁷⁶ Twelve studies identified the presence of potential confounding.¹³^,²⁸^,²⁹^,⁵²^,⁵⁴^,⁵⁷^,⁵⁹^,⁶²^,⁶³^,⁶⁵^,⁶⁹^,⁷¹^,⁷⁵

DISCUSSION

In this scoping review, we identified 35 studies that explored the measurement of documentation burden among physicians and nurses, underlining the overall paucity of research in the domain. As may be expected, all 35 studies were published post-HITECH Act. Seven effort constructs, 4 time constructs, and 11 units of analysis were uncovered. Our effort constructs—except workflow fragmentation and cognitively cumbersome work (eg, multitasking)—largely align with “proposed core EHR use measures (for practice efficiency)” published by Sinsky and colleagues which indicates burden may be quantified through existing metrics.⁴⁶ Generated with expert stakeholders, Sinsky’s core measures include total EHR time, work outside of work, time on documentation, time on prescriptions, inbox time, teamwork for orders, and undivided attention to patients.⁴⁶ Further efforts should examine these measures for validity and reliability. Fewer than half (n = 16) of the studies investigated the impact of burden on clinicians and/or patients. Methodologies varied across study design, suggesting there is no current consensus regarding best approach or standard to study burden, although it is possible an ensemble of methods coupled with the triangulation of multiple data sources will emerge as a best practice.

Historically, TM studies have been considered the gold standard for quantifying the effects of computer systems on task-based clinical workflow and duration.⁸²^,⁸³ Despite yielding valid results,⁷⁰^,⁸⁴^,⁸⁵ TM studies are costly and time-consuming to perform⁸³ and engage only a handful of participants per study. In addition to concerns regarding the generalizability of TM studies, prior research has identified widespread methodological inconsistencies in their design and conduct as well as in their quantitative analyses and reporting of results, making it difficult to synthesize findings across studies.⁷⁰^,⁸⁶ Readily accessible and scalable, and less subject to the Hawthorne effect, evidence may suggest that analyzing EHR usage logs is a more feasible alternative as these data were used in the overwhelming majority of included studies (80.0%). Nevertheless, research on the use of EHR usage logs to evaluate clinical activity has revealed a dearth of validation, cross-study analyses, and, most critically, defined terminology (eg, access log, audit log) and measures.⁴⁶^,⁸⁷ These inconsistencies parallel those found in TM studies, as described above. TM studies provide valuable contextual information on time and sequence of activities performed which can be triangulated with EHR usage logs to better understand burden in the context of clinical workflows. In recognizing that all methods have strengths and weaknesses, we anticipate that future work will identify the methods of measurement and triangulation of data that best align with different research objectives related to burden.

One major finding of this review was the absence of quantitative studies assessing the reliability and validity of time and effort measures. Of the 35 studies included, only 1 study intended to develop a measure of burden (ie, EHR usage outside shift),¹³ while 2 studies individually employed TM studies to empirically validate proposed measures of workflow and the use of EHR usage log data in characterizing workload.²⁷^,⁷⁰ Interobserver reliability was reported in only 2 studies.²⁸^,⁶⁸ As described above, previous studies on quantifying physician EHR activity through EHR usage logs have noted similar challenges.⁸⁷ The lack of studies developing and validating burden measures confirms that limited efforts have been dedicated to formally and objectively quantifying and measuring burden, despite increasing references to it in public policy and lay literature. Researchers have often used unstandardized proxies to quantify burden which elucidates why no objective proxies exist.⁶^,¹³^,¹⁴^,²⁵ Reinforcing the absence of empirical validation studies, there is a lack of an agreed-upon definition for burden and a plethora of definitions throughout the literature.⁶^,¹³^,¹⁴^,²⁵^,²⁸^,⁴³^,⁴⁷^,⁵²^,^88–90 We found that many related—but different—concepts were used in the context of studies quantifying time and effort, such as workload,²⁷^,⁷⁸ workflow,¹³^,⁷⁴ work disruption,⁷⁵ efficiency,²²^,⁵² cognitive burden,⁵⁶ usability,⁷⁴ and productivity, among others.⁶⁹ In contrast, burnout is identifiable in controlled vocabularies including, the International Classification of Diseases (ICD), in addition to the Diagnostic and Statistical Manual for Mental Disorders (DSM) and MeSH.⁹¹^,⁹² Furthermore, validated measures of burnout, such as the Maslach Burnout Inventory and the Mini Z burnout survey are often applied,⁶⁹^,⁷³ whereas no known analog for burden is currently available. Likewise, in a literature review conducted on the impact of EHRs on documentation time among physicians and nurses, Poissant and colleagues suggested that a lack of research evaluating EHR time efficiency is likely associated with the poverty of rigorous methods accurately capturing time.¹² We found that generalizability and measurement error issues were partially driven by the use of distinct EHR systems with some instances of proprietary and opaque vendor-defined time metrics for shift and active EHR time.⁷³ There was also imprecision in time capture among EHR usage log studies. Reported elsewhere in the literature, EHR usage logs have exhibited unreliable degrees of accuracy for both clinician activity and time durations captured.⁸⁷ Intended for troubleshooting technical problems and HIPAA compliance, EHR usage logs originate from many interconnected information systems and sources (eg, devices).⁹³ Vendor-defined time metrics may not be generalizable between, or within, institutions or provide precise estimates in real-world settings. Therefore, given the value in measuring clinician EHR time, researchers should explore novel algorithmic methods to validate these metrics and EHR usage log data. For example, Dziorny and colleagues developed an automated algorithm to quantify shift duration among physicians in an inpatient setting and internally validated it against scheduled shift-time.⁸³ Likewise, DiAngi et al proposed the “calculated EHR time outside of clinic” (CLOC) metric for ambulatory settings to measure after clinic hours using EHR usage logs and were able to correlate their findings with self-reported time spent in the EHR after clinic hours.²²

The HHS Report—Strategy on Reducing Burden Relating to the Use of Health IT and EHRs—aims to evaluate the clinical impact of burden (ie, time and effort) on clinicians and/or patients;⁴⁸ however, fewer than half the studies reviewed investigated an outcome of interest (n = 16). Of those studies (note: outcomes were not mutually exclusive), the majority examined clinician satisfaction and burnout (n = 12), while only half examined clinical process measures as an end goal. Half evaluated patient satisfaction and health indicators. Research questions and study objectives were widespread across included studies.

In this review, scribes represented 1 of 3 areas of study concerning proposed interventions to mitigate burden (n = 3);¹⁴^,⁵²^,⁵³ however, associated costs and high turnover rates among scribes suggest that this solution may not be broadly feasible or sustainable.⁴⁷ In the context of reducing documentation burden, implementing and measuring the impact of scribes does not solve the higher-level information processing issues that informatics research should be investigating (eg, reduction in data entry requirements, improvement of system usability) and possibly diverts resources away from more sophisticated biomedical informatics approaches. Other identified interventions, such as training on EHR use (n = 2)²²^,⁵⁶ and documentation redesign (n = 3)⁵⁸^,⁷⁵^,⁷⁸ also have their strengths and weaknesses. Training may represent a lower cost method of mitigating burden than scribes, while documentation redesign may be more costly but likely more effective at solving information processing and usability concerns. Moreover, lack of standardized measures leads to the inability to conduct comparative effectiveness studies on design modifications within EHR systems¹⁵ or across distinct burden-alleviating interventions.

In summary, our findings identified distinct, but not necessarily comprehensive, characteristics of measuring burden: 7 effort constructs, 4 time constructs, and 11 units of analysis (see Table 3).

Limitations

While this study sought to investigate literature on the operationalization of documentation burden and the development and/or validation of quantitative burden measures, research in this domain has not yet matured. Despite employing broad search terms and queries, the majority of the literature retrieved did not detail how to conceptualize and/or measure burden. We extracted manuscripts using keywords, as extant MeSH terms were unable to capture the phenomenon of study interest; in fact, no term for burden used in this specific context exists. It is conceivable that some articles were not captured because: (a) our keywords were limited, and/or (b) our queries were not sufficiently broad or narrow.

Future directions

Future research should build upon existing burden evidence, focusing on strengthening objectivity and generalizability. Proposed quantitative measures of burden such as the after scheduled duty hours measure described by Cox and colleagues should undergo rigorous testing and validation across settings and specialties.¹³ Additionally, HHS links time and effort concepts to clinical impact; ⁴⁸ therefore, research should directly connect measurement of these concepts with specific outcome measures to be able to accurately evaluate documentation burden over time. This remains a difficult undertaking as studies have shown that neither burden nor task value in the clinical context are identical across all EHR interactions or across different roles and specialties.¹⁰^,²⁰^,²⁵ Examining tradeoffs between specific tasks within the EHR, Rao and colleagues discovered that EHR functionalities are not equally burdensome.²⁵ They also found that settings are not equally burdensome, reporting that shift-based work may be associated with less burden and that ambulatory clinical documentation is rated equally valuable and burdensome.²⁵ Perceptions of distinct documentation types among nurses have also been studied, yet no objective criteria have been established to evaluate value.¹⁹ We found that only 1 study investigating EHR work afterhours (ie, “pajama time”) included nurses.⁷⁷ While “pajama time” connotes remotely accessing the EHR from home to document, few inpatient nurses do so given the immediacy of their documentation. Thus, data entry rates may be more suitable for measuring nurse burden.⁷⁶ Because physicians working in general medicine were most represented in our findings, future work should be dedicated to characterizing and measuring burden among understudied professions and settings (ie, nurses and subspecialties).

However, promisingly, burden measures identified were not strictly unique to individual professions and workflows, supporting the opportunity for defining interprofessional measures of burden in future work. We propose that burden be examined as a global composite measure, indicative of magnitude and directionality, consistent with the characteristics uncovered in this review. This would require: (a) developing a universally agreed-upon inventory for key EHR tasks and activities weighted for relative value according to burden (ie, a taxonomy) that could be linked to clinical outcomes such as “quality, financial or professional satisfaction”¹⁵^,²⁷^,²⁸; and (b) quantifying the relationship between “pain points” and specific features in the EHR with more granularity. This furnishes the examination of task value, as indicated by task relationship with burden, a high priority area for future research. Such research would allow the identification of tasks that are of high burden but low value so that EHR design and intervention efforts may target the elimination or mitigation of these tasks.

CONCLUSION

Documentation burden among interprofessional clinical roles remains understudied and under-measured in both inpatient and ambulatory settings. This review suggests that concrete, validated measures of burden in research are lacking, which pales in comparison to burnout literature.³⁶ Moreover, this review demonstrates that the existing evidence is imprecise and fragmentary. While there is a multitude of measures for both effort and time among the included studies, the majority lack generalizability across study setting, patient population, EHR system, activity type, role, and subspecialty. In the absence of standardization, these studies additionally run the risk of measurement error including misclassification of idle and active time, completeness of task measurement, and precision of time capture. Hence, it would be prudent to further explore easily accessible, scalable alternatives, such as EHR usage log data. Targeting burden to evaluate the impact of quality improvement strategies and interventions requires quantifiable measures that are comparable and consistent across time, settings, professions, and contexts. We propose that burden should be examined as a global composite measure based on task value, consistent with burden measurement characteristics uncovered in this review. Further research is needed to reliably operationalize and standardize the concept of burden and to explore how it is best measured across clinical settings.

FUNDING

This study was supported by the US National Library of Medicine of the National Institutes of Health (NIH) under the training fellowship award 5T15LM007079 and the National Institute for Nursing Research (NINR) under grant numbers 1R01NR016941 and 5T32NR007969.

AUTHOR CONTRIBUTIONS

AJM and SCR conceptualized the scope of this review. AJM, JMS, RC, SS, and EL conducted the initial and full-text screenings. AJM drafted the manuscript with significant revisions and feedback from JMS, RC, KDC, and SCR.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

DATA AVAILABILITY STATEMENT

The data underlying this article are available in the article and in its online supplementary material.

CONFLICT OF INTEREST STATEMENT

None declared.

Supplementary Material

References

Hripcsak G , BloomrosenM, FlatelyBrennanP, et alHealth data use, stewardship, and governance: ongoing gaps and challenges: a report from AMIA’s 2012 health policy meeting. J Am Med Inform Assoc2014; 21 (2): 204–11.2416927510.1136/amiajnl-2013-002117PMC3932468 doi:10.1136/amiajnl-2013-002117 PubMed
Paterick ZR , PatelNJ, PaterickTE. Unintended consequences of the electronic medical record on physicians in training and their mentors. Postgrad Med J2018; 94 (1117): 659–61.3055417110.1136/postgradmedj-2018-135849 doi:10.1136/postgradmedj-2018-135849 PubMed
Coiera E , AshJ, BergM. The unintended consequences of health information technology revisited. Yearb Med Inform2016; 25 (01): 163–9.10.15265/IY-2016-014PMC517157627830246 doi:10.15265/IY-2016-014 PubMed
Palabindala V , PamarthyA, JonnalagaddaNR. Adoption of electronic health records and barriers. J Community Hosp Intern Med Perspect2016; 6 (5): 32643.2780285710.3402/jchimp.v6.32643PMC5089148 doi:10.3402/jchimp.v6.32643 PubMed
Khairat S , BurkeG, ArchambaultH, et alFocus section on health IT usability: perceived burden of EHRs on physicians at different stages of their career. Appl Clin Inform2018; 09 (02): 336–47.10.1055/s-0038-1648222PMC595571729768634 doi:10.1055/s-0038-1648222 PubMed
Howard J , ClarkEC, FriedmanA, et alElectronic health record impact on work burden in small, unaffiliated, community-based primary care practices. J Gen Intern Med2013; 28 (1): 107–13.2292663310.1007/s11606-012-2192-4PMC3539023 doi:10.1007/s11606-012-2192-4 PubMed
Chaudhry B , WangJ, WuS, et alSystematic review: impact of health information technology on quality, efficiency, and costs of medical care. Ann Intern Med2006; 144 (10): 742–52.1670259010.7326/0003-4819-144-10-200605160-00125 doi:10.7326/0003-4819-144-10-200605160-00125 PubMed
Hillestad R , BigelowJ, BowerA, et alCan electronic medical record systems transform health care? Potential health benefits, savings, and costs. Health Aff2005; 24 (5): 1103–17.10.1377/hlthaff.24.5.110316162551 doi:10.1377/hlthaff.24.5.1103 PubMed
Ash JS , BergM, CoieraE. Some unintended consequences of information technology in health care: the nature of patient care information system-related errors. J Am Med Inform Assoc2003; 11 (2): 104–12.1463393610.1197/jamia.M1471PMC353015 doi:10.1197/jamia.M1471 PubMed
DesRoches CM , CampbellEG, RaoSR, et alElectronic health records in ambulatory care—a national survey of physicians. N Engl J Med2008; 359 (1): 50–60.1856585510.1056/NEJMsa0802005 doi:10.1056/NEJMsa0802005 PubMed
DiAngi YT , LonghurstCA, PayneTH. Taming the EHR (electronic health record): there is hope. J Fam Med2016; 3 (6): 1072.27830215PMC5098336 PubMed
Poissant L , PereiraJ, TamblynR, et alThe impact of electronic health records on time efficiency of physicians and nurses: a systematic review. J Am Med Inform Assoc2005; 12 (5): 505–16.1590548710.1197/jamia.M1700PMC1205599 doi:10.1197/jamia.M1700 PubMed
Cox ML , FarjatAE, RisoliTJ, et alDocumenting or operating: where is time spent in general surgery residency?J Surg Educ2018;75 (6): e97–106.3052282810.1016/j.jsurg.2018.10.010PMC10765321 doi:10.1016/j.jsurg.2018.10.010 PubMed
Mishra P , KiangJC, GrantRW. Association of medical scribes in primary care with physician workflow and patient experience. JAMA Intern Med2018; 178 (11): 1467–72.3024238010.1001/jamainternmed.2018.3956PMC6248201 doi:10.1001/jamainternmed.2018.3956 PubMed
Shanafelt TD , DyrbyeLN, SinskyC, et alRelationship between clerical burden and characteristics of the electronic environment with physician burnout and professional satisfaction. Mayo Clin Proc2016; 91 (7): 836–48.2731312110.1016/j.mayocp.2016.05.007 doi:10.1016/j.mayocp.2016.05.007 PubMed
Kaufman DR , SheehanB, StetsonP, et alNatural language processing-enabled and conventional data capture methods for input to electronic health records: a comparative usability study. JMIR Med Inform2016; 4 (4): e35.2779379110.2196/medinform.5544PMC5106560 doi:10.2196/medinform.5544 PubMed
Ollapally V. EHRs add to surgeons’ administrative burdens: the ACS responds. 2018. http://bulletin.facs.org/2018/09/ehrs-add-to-surgeons-administrative-burdens-the-acs-responds/ Accessed February 6, 2019.
Dykstra RH , AshJS, CampbellE, et alPersistent paper: the myth of going paperless. AMIA Annu Symp Proc2009; 2009: 158–62.20351841PMC2815440 PubMed
Olivares Bøgeskov B , Grimshaw-AagaardSLS. Essential task or meaningless burden? Nurses’ perceptions of the value of documentation. Nord J Nurs Res2019; 39 (1): 9–19.
Kossman SP , ScheidenhelmSL. Nurses’ perceptions of the impact of electronic health records on work and patient outcomes. Comput Inform Nurs2008; 26 (2): 69–77.1831725710.1097/01.NCN.0000304775.40531.67 doi:10.1097/01.NCN.0000304775.40531.67 PubMed
Shanafelt TD , DyrbyeLN, WestCP. Addressing physician burnout: the way forward. Jama2017; 317 (9): 901–2.2819620110.1001/jama.2017.0076 doi:10.1001/jama.2017.0076 PubMed
DiAngi YT , StevensLA, Halpern-FelsherB, et alElectronic health record (EHR) training program identifies a new tool to quantify the EHR time burden and improves providers’ perceived control over their workload in the EHR. JAMIA Open2019; 2 (2): 222–30.3198435710.1093/jamiaopen/ooz003PMC6952029 doi:10.1093/jamiaopen/ooz003 PubMed
Goldenberg JN. The breadth and burden of data collection in clinical practice. Neurol Clin Pract2016; Feb;6 (1): 81–6.2944327210.1212/CPJ.0000000000000209PMC5765897 doi:10.1212/CPJ.0000000000000209 PubMed
Center for Medicare and Medicaid Services. An introduction to the Medicare EHR Incentive Program for eligible professionals. 2014; https://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Downloads/EHR_Medicare_Stg1_BegGuide.pdf Accessed August 14, 2019.
Rao SK , KimballAB, LehrhoffSR, et alThe impact of administrative burden on academic physicians. Acad Med2017; 92 (2): 237–43.2812168710.1097/ACM.0000000000001461 doi:10.1097/ACM.0000000000001461 PubMed
Colicchio TK , CiminoJJ. Clinicians’ reasoning as reflected in electronic clinical note-entry and reading/retrieval: a systematic review and qualitative synthesis. J Am Med Inform Assoc2019; 26 (2): 172–84.3057656110.1093/jamia/ocy155PMC7647174 doi:10.1093/jamia/ocy155 PubMed
Arndt BG , BeasleyJW, WatkinsonMD, et alTethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med2017; 15 (5): 419–26.2889381110.1370/afm.2121PMC5593724 doi:10.1370/afm.2121 PubMed
Sinsky C , ColliganL, LiL, et alAllocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med2016; 165 (11): 753–60.2759543010.7326/M16-0961 doi:10.7326/M16-0961 PubMed
Joukes E , Abu-HannaA, CornetR, et alTime spent on dedicated patient care and documentation tasks before and after the introduction of a structured and standardized electronic health record. Appl Clin Inform2018; 09 (01): 46–53.10.1055/s-0037-1615747PMC580188129342479 doi:10.1055/s-0037-1615747 PubMed
Wang SJ , MiddletonB, ProsserLA, et alA cost-benefit analysis of electronic medical records in primary care. Am J Med2003; 114 (5): 397–403.1271413010.1016/s0002-9343(03)00057-3 doi:10.1016/s0002-9343(03)00057-3 PubMed
Wang N , YuP, HaileyD. The quality of paper-based versus electronic nursing care plan in Australian aged care homes: a documentation audit study. Int J Med Inform2015; 84 (8): 561–9.2600434010.1016/j.ijmedinf.2015.04.004 doi:10.1016/j.ijmedinf.2015.04.004 PubMed
Lavander P , MeriläinenM, TurkkiL. Working time use and division of labour among nurses and health-care workers in hospitals: a systematic review. J Nurs Manag2016; 24 (8): 1027–40.2758109310.1111/jonm.12423 doi:10.1111/jonm.12423 PubMed
Schulte F , FryE. Death by a thousand clicks: where electronic health records went wrong. Kaiser Health News.2019. https://khn.org/news/death-by-a-thousand-clicks/ Accessed June 16, 2020.
Oxford English Dictionary. 2019. https://www.oed.com/view/Entry/24885? rskey=DnyH6B&result=1&isAdvanced=false#eidAccessed July 16, 2019
Agency for Healthcare Research and Quality. Physician burnout. 2017. https://www.ahrq.gov/professionals/clinicians-providers/ahrq-works/burnout/index.html Accessed July 16, 2019
West CP , DyrbyeLN, ErwinPJ, et alInterventions to prevent and reduce physician burnout: a systematic review and meta-analysis. Lancet2016; 388 (10057): 2272–81.2769246910.1016/S0140-6736(16)31279-X doi:10.1016/S0140-6736(16)31279-X PubMed
Heinemann LV , HeinemannT. Burnout research: emergence and scientific investigation of a contested diagnosis. SAGE Open2017; 7 (1): 215824401769715.10.1177/2158244017697154 doi:10.1177/2158244017697154
Dyrbye LN , ShanafeltTD, JohnsonPO, JohnsonLA, SateleD, WestCP. A cross-sectional study exploring the relationship between burnout, absenteeism, and job performance among American nurses. BMC Nurs2019; 18 (1): 57. 10.1186/s12912-019-0382-7PMC687374231768129 doi:10.1186/s12912-019-0382-7 PubMed
Dyrbye LN , ShanafeltTD, SinskyCA, et alBurnout among health care professionals: a call to explore and address this underrecognized threat to safe, high-quality care. NAM Perspectives2017; 7 (7). 10.31478/201707b doi:10.31478/201707b
Moss M , GoodVS, GozalD, et alAn official critical care societies collaborative statement—burnout syndrome in critical care health-care professionals: a call for action. Crit Care Med2016; 44 (7): 1414–21.2730915710.1097/CCM.0000000000001885 doi:10.1097/CCM.0000000000001885 PubMed
Babbott S , ManwellLB, BrownR, et alElectronic medical records and physician stress in primary care: results from the MEMO Study. J Am Med Inform Assoc2014; 21 (e1): e100-6–e106.2400579610.1136/amiajnl-2013-001875PMC3957395 doi:10.1136/amiajnl-2013-001875 PubMed
Meigs SL , SolomonM. Electronic health record use a bitter pill for many physicians. Perspect Heal Inf Manag2016; 13: 1–17.PMC473944326903782 PubMed
Tai-Seale M , OlsonCW, LiJ, et alElectronic health record logs indicate that physicians split time evenly between seeing patients and desktop medicine. Health Aff2017; 36 (4): 655–62.10.1377/hlthaff.2016.0811PMC554641128373331 doi:10.1377/hlthaff.2016.0811 PubMed
Maslach C , JacksonSE. The measurement of experienced burnout. J Organiz Behav1981; 2 (2): 99–113.
Olson K , SinskyC, RinneST, et alCross-sectional survey of workplace stressors associated with physician burnout measured by the Mini-Z and the Maslach Burnout Inventory. Stress Health2019; 35 (2): 157–75.3046794910.1002/smi.2849 doi:10.1002/smi.2849 PubMed
Sinsky CA , RuleA, CohenG, et alMetrics for assessing physician activity using electronic health record log data. J Am Med Inform Assoc2020; 27 (4): 639–43.3202736010.1093/jamia/ocz223PMC7075531 doi:10.1093/jamia/ocz223 PubMed
Dela Cruz JE , ShaboskyJC, AlbrechtM, et alTyped versus voice recognition for data entry in electronic health records: emergency physician time use and interruptions. West J Emerg Med2014;15 (4): 541–7.2503576510.5811/westjem.2014.3.19658PMC4100865 doi:10.5811/westjem.2014.3.19658 PubMed
Office of the National Coordinator for Health Information Technology. Strategy on reducing burden relating to the use of health IT and EHRs. 2020. https://www.healthit.gov/topic/usability-and-provider-burden/strategy-reducing-burden-relating-use-health-it-and-ehrs Accessed April 2, 2020.
Tricco AC , LillieE, ZarinW, et alPRISMA extension for scoping reviews (PRISMA-ScR): Checklist and explanation. Ann Intern Med2018;169 (7): 467–73.3017803310.7326/M18-0850 doi:10.7326/M18-0850 PubMed
Veritas Health Innovation. Covidence Systematic Review Software. Melbourne, Australia: Veritas Health Innovation; 2018.
Nowell LS , NorrisJM, WhiteDE, et alThematic analysis: striving to meet the trustworthiness criteria. Int J Qual Methods2017; 16 (1): 160940691773384.
Earls ST , SavageauJA, BegleyS, et alCan scribes boost FPs’ efficiency and job satisfaction?J Fam Pract2017;66 (4): 206–14.28375393 PubMed
Gidwani R , NguyenC, KofoedA, et alImpact of scribes on physician satisfaction, patient satisfaction, and charting efficiency: a randomized controlled trial. Ann Fam Med2017; 15 (5): 427–33.2889381210.1370/afm.2122PMC5593725 doi:10.1370/afm.2122 PubMed
Goldstein IH , HwangT, GowrisankaranS, et alChanges in electronic health record use time and documentation over the course of a decade. Ophthalmology2019;126 (6): 783–91.3066489310.1016/j.ophtha.2019.01.011PMC6534421 doi:10.1016/j.ophtha.2019.01.011 PubMed
Hripcsak G , VawdreyDK, FredMR, et alUse of electronic clinical documentation: time spent and team interactions. J Am Med Inform Assoc2011; 18 (2): 112–7.2129270610.1136/jamia.2010.008441PMC3116265 doi:10.1136/jamia.2010.008441 PubMed
Kadish SS , MayerEL, JackmanDM, et alImplementation to optimization: a tailored, data-driven approach to improve provider efficiency and confidence in use of the electronic medical record. J OncolPract2018; 14 (7): e421–ee428.10.1200/JOP.18.0009329939808 doi:10.1200/JOP.18.00093 PubMed
Kannampallil TG , DentonCA, ShapiroJS, et alEfficiency of emergency physicians: insights from an observational study using EHR log files. Appl Clin Inform2018; 9 (1): 99–104.3018424110.1055/s-0037-1621705PMC5802998 doi:10.1055/s-0037-1621705 PubMed
Karp EL , FreemanR, SimpsonKN, et alChanges in efficiency and quality of nursing electronic health record documentation after implementation of an admission patient history essential data set. Comput Inform Nurs2019; 37 (5): 260–5.3109491510.1097/CIN.0000000000000516 doi:10.1097/CIN.0000000000000516 PubMed
Krawiec C , StetterC, KongL, et alImpact of patient census and admission mortality on pediatric intensive care unit attending electronic health record activity: a preliminary study. Appl Clin Inform2020; 11 (02): 226–34.3221589410.1055/s-0040-1705108PMC7096317 doi:10.1055/s-0040-1705108 PubMed
Krawiec C , MarkerC, StetterC, et alTracking resident pre-rounding electronic health record usage. Int J Health Care Qual Assur2019; 32 (3): 611–20.3101879810.1108/IJHCQA-06-2018-0137 doi:10.1108/IJHCQA-06-2018-0137 PubMed
Mamykina L , VawdreyDK, StetsonPD, et alClinical documentation: composition or synthesis?J Am Med Inform Assoc2012; 19 (6): 1025–31.2281376210.1136/amiajnl-2012-000901PMC3534467 doi:10.1136/amiajnl-2012-000901 PubMed
Mamykina L , VawdreyDK, HripcsakG. How do residents spend their shift time? A time and motion study with a particular focus on the use of computers. Acad Med2016; 91 (6): 827–32.2702802610.1097/ACM.0000000000001148PMC4879085 doi:10.1097/ACM.0000000000001148 PubMed
Marmor RA , ClayB, MillenM, et alThe impact of physician EHR usage on patient satisfaction. Appl Clin Inform2018; (01): 011–4. doi: 10.1055/s-0037-162026310.1055/s-0037-1620263PMC580188629298451 doi:10.1055/s-0037-1620263 PubMed
Micek MA , ArndtB, TuanW-J, et alPhysician burnout and timing of electronic health record use. ACI Open2020; 04 (01): e1–e8.10.1055/s-0039-3401815PMC1055336737800093 doi:10.1055/s-0039-3401815 PubMed
Overhage JM , McCallieD. Physician time spent using the electronic health record during outpatient encounters a descriptive study. Ann Intern Med2020; 172 (3): 169–74.3193152310.7326/M18-3684 doi:10.7326/M18-3684 PubMed
Saag HS , ShahK, JonesSA, et alPajama time: working after work in the electronic health record. J Gen Intern Med2019; 34 (9): 1695–6.3107385610.1007/s11606-019-05055-xPMC6712097 doi:10.1007/s11606-019-05055-x PubMed
Smith B , SreeramakavachamS, KimJH, et al Improving computerized charting in an intensive care unit. In: proceedings of the Digital Human Modeling: Applications in Safety, Ergonomics, and Risk Management; July 15-20, 2018; Las Vegas, NV. New York (USA): Springer International Publishing; 2018: 537–46.
Tipping MD , ForthVE, O'LearyKJ, et alWhere did the day go? A time-motion study of hospitalists. J Hosp Med2010; 5 (6): 323–8.2080366910.1002/jhm.790 doi:10.1002/jhm.790 PubMed
Tran B , LenhartA, RossR, et alBurnout and EHR use among academic primary care physicians with varied clinical workloads. AMIA Jt Summits Transl Sci Proceedings AMIA Jt Summits Transl Sci2019; 2019: 136–44.PMC656807631258965 PubMed
Zheng K , HaftelHM, HirschlRB, et alQuantifying the impact of health IT implementations on clinical workflow: a new methodological perspective. J Am Med Inf Assoc2010; 17 (4): 454–61.10.1136/jamia.2010.004440PMC299565420595314 doi:10.1136/jamia.2010.004440 PubMed
Anderson J , LeubnerJ, BrownSR. EHR overtime: an analysis of time spent after hours by family physicians. Fam Med2020; 52 (2): 135–7.3205027010.22454/FamMed.2020.942762 doi:10.22454/FamMed.2020.942762 PubMed
Wang JK , OuyangD, HomJ, et alCharacterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS One2019;14 (2): e0205379.3072620810.1371/journal.pone.0205379PMC6364867 doi:10.1371/journal.pone.0205379 PubMed
Adler-Milstein J , ZhaoW, Willard-GraceR, et alElectronic health records and burnout: time spent on the electronic health record after hours and message volume associated with exhaustion but not with cynicism among primary care clinicians. J Am Med Informatics Assoc2020; 27 (4): 531–8.10.1093/jamia/ocz220PMC764726132016375 doi:10.1093/jamia/ocz220 PubMed
Aziz F , TalhelmL, KeeferJ, et alVascular surgery residents spend one fifth of their time on electronic health records after duty hours. J Vasc Surg2019; 69 (5): 1574–9.3101052110.1016/j.jvs.2018.08.173 doi:10.1016/j.jvs.2018.08.173 PubMed
Carlson KL , McFaddenSE, BarkinS. Improving documentation timeliness: a “brighter future” for the electronic medical record in resident clinics. Acad Med2015;90 (12): 1641–5.2610788010.1097/ACM.0000000000000792 doi:10.1097/ACM.0000000000000792 PubMed
Collins S , CoutureB, KangMJ, et alQuantifying and visualizing nursing flowsheet documentation burden in acute and critical care. AMIA. Annu Symp Proc AMIA Symp2018; 2018: 348–57.30815074PMC6371331 PubMed
Ahn M , ChoiM, KimY. Factors associated with the timeliness of electronic nursing documentation. Healthc Inform Res2016; 22 (4): 270–6.2789595810.4258/hir.2016.22.4.270PMC5116538 doi:10.4258/hir.2016.22.4.270 PubMed
Hsieh H-Y , HenkerR, RenD, et alImproving effectiveness and satisfaction of an electronic charting system in Taiwan. Clin Nurse Spec2016; 30 (6): E1–6.10.1097/NUR.000000000000025027753675 doi:10.1097/NUR.0000000000000250 PubMed
Inokuchi R , SatoH, IwagamiM, et alImpact of a new medical record system for emergency departments designed to accelerate clinical documentation: a crossover study. Med (United States)2015; 94 (26): e856.10.1097/MD.0000000000000856PMC450457226131837 doi:10.1097/MD.0000000000000856 PubMed
Zheng K , HaftelHMM, HirschlRBB, et alCharacterizing electronic health record usage patterns of inpatient medicine residents using event log data. PLoS One2018; 9 (6): 531–8.10.1371/journal.pone.0205379PMC636486730726208 doi:10.1371/journal.pone.0205379 PubMed
Goldstein IH , HwangT, GowrisankaranS, et alChanges in electronic health record use time over a decade of use. Investig Ophthalmol Vis Sci2019; 126 (6): 783–91.10.1016/j.ophtha.2019.01.011PMC653442130664893 doi:10.1016/j.ophtha.2019.01.011 PubMed
Lopetegui M , YenP-Y, LaiA, et alTime motion studies in healthcare: what are we talking about?J Biomed Inform2014; 49: 292–9.2460786310.1016/j.jbi.2014.02.017PMC4058370 doi:10.1016/j.jbi.2014.02.017 PubMed
Dziorny A , OrensteinE, LindellR, et alAutomatic detection of front-line clinician hospital shifts: a novel use of electronic health record timestamp data. Appl Clin Inform2019; 10 (01): 28–37.3062550210.1055/s-0038-1676819PMC8438622 doi:10.1055/s-0038-1676819 PubMed
Ampt A , WestbrookJ, CreswickN, et alA comparison of self-reported and observational work sampling techniques for measuring time in nursing tasks. J Health Serv Res Policy2007; 12 (1): 18–24.1724439310.1258/135581907779497576 doi:10.1258/135581907779497576 PubMed
Burke TA , McKeeJR, WilsonHC, et alA comparison of time-and-motion and self-reporting methods of work measurement. J Nurs Adm2000; 30 (3): 118–25.1072594010.1097/00005110-200003000-00003 doi:10.1097/00005110-200003000-00003 PubMed
Zheng K , GuoMH, HanauerDA. Using the time and motion method to study clinical work processes and workflow: methodological inconsistencies and a call for standardized research. J Am Med Inform Assoc2011; 18 (5): 704–10.2152740710.1136/amiajnl-2011-000083PMC3168304 doi:10.1136/amiajnl-2011-000083 PubMed
Rule A , ChiangMF, HribarMR. Using electronic health record audit logs to study clinical activity: a systematic review of aims, measures, and methods. J Am Med Inform Assoc2020; 27 (3): 480–90. doi: 10.1093/jamia/ocz196.3175091210.1093/jamia/ocz196PMC7025338 doi:10.1093/jamia/ocz196 PubMed
Koshy S , FeustelPJ, HongM, et alScribes in an ambulatory urology practice: patient and physician satisfaction. J Urol2010; 184 (1): 258–62.2048315310.1016/j.juro.2010.03.040 doi:10.1016/j.juro.2010.03.040 PubMed
Park SY , LeeSY, ChenY. The effects of EMR deployment on doctors’ work practices: a qualitative study in the emergency department of a teaching hospital. Int J Med Inform2012; 81 (3): 204–17.2221780210.1016/j.ijmedinf.2011.12.001 doi:10.1016/j.ijmedinf.2011.12.001 PubMed
Senathirajah Y , KaufmanDR, BakkenSR. User-composable electronic health record improves efficiency of clinician data viewing for patient case appraisal: a mixed-methods study. eGEMs2016; 4 (1): 7.10.13063/2327-9214.1176PMC486276327195306 doi:10.13063/2327-9214.1176 PubMed
Korczak D , HuberB, KisterC. Differential diagnostic of the burnout syndrome. GMS Health Technol Assess2010; 6: Doc09.2128988210.3205/hta000087PMC3010892 doi:10.3205/hta000087 PubMed
Hall LH , JohnsonJ, WattI, et alHealthcare staff wellbeing, burnout, and patient safety: a systematic review. PLoS One2016; 11 (7): e0159015–12.2739194610.1371/journal.pone.0159015PMC4938539 doi:10.1371/journal.pone.0159015 PubMed
Wickramage C , SahamaT, FidgeC. Anatomy of log files: implications for information accountability measures. In: proceedings of the 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom); September 14–17, 2016; Munich, Germany. New Jersey (USA): IEEE Press; 2016: 1–6.

Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review