Advertisement

Exploratory Descriptive Analysis of Smart Speaker Utilization in the Emergency Department during the COVID-19 pandemic

      Abstract

      Background

      In March 2020, the US Department of Health and Human Services Office for Civil Rights stated they would use discretion when enforcing HIPAA regarding remote communication technologies which promoted telehealth delivery during the COVID-19 pandemic. This was in an effort to protect patients, clinicians, and staff. More recently, smart speakers, voice activated, hands-free devices, are being proposed as productivity tools within hospitals.

      Objective

      We aimed to characterize the novel usage of smart speakers in the emergency department (ED).

      Methods

      A retrospective observational study of Amazon Echo Show® utilization from May 2020 to October 2020 in a large academic Northeast health system ED. Voice commands and queries were classified as either patient care related or non-patient care related, and then further subcategorized to explore the content of given commands.

      Results

      Of 1,232 commands analyzed, 200 (16.23%) were determined to be patient care related. Of these commands, 155 (77.5%) were clinical in nature (i.e., “drop in on triage”) and 23 (11.5%) were environment enhancing commands (i.e., “play calming sounds”). Among non-patient care related commands, 644 (62.4%) were for entertainment. Among all commands, 804 (65.3%) were during night shift hours, which was statistically significant (p<0.001).

      Conclusions

      Smart speakers showed notable engagement, primarily being used for patient communication and entertainment. Future studies should examine content of patient care conversations using these devices, effects on frontline staff wellbeing, productivity, patient satisfaction, and even explore opportunities for “smart” hospital rooms.

      Keywords

      INTRODUCTION

      The COVID-19 pandemic has led to rapid changes in protocols and aggressive adjustments to previously routine clinical workflows to slow viral spread and improve staff safety. A variety of community recommendations were implemented including social-distancing, capacity limitations on businesses, and use of video conferencing to substitute in-person interactions1. In the hospital setting, however, such distancing precautions have been more difficult to apply2.
      Hospitals have worked to minimize viral transmission with diligent hand-hygiene and appropriate use of personal protective equipment (PPE), among other forms of infection control3. Remote communication to provide telehealth services has increased dramatically, with focus most heavily placed on offering outpatient services via video consultation4. In these outpatient care settings, virtual encounters have demonstrated feasibility, maintained patient satisfaction, promoted social distancing, and helped to relieve heavy patient volume in emergency departments (ED)5. Limited video monitoring and communication are also established in several inpatient hospital settings, specifically tele-consults and tele-ICU monitoring capabilities6-8. The expansion of telehealth services has been enabled in part by the US Department of Health and Human Services Office for Civil Rights (HHS OCR) guidance that discretion would be used when enforcing HIPAA in regard to good faith provision of telehealth during the COVID-19 public health emergency which has since expired in July 20229. Thus far, however, there has been less consideration of how remote communication technology may supplement encounters that require immediate contact, such as ED visits. Specifically, ED use of smart speakers as patient care tools has not been well defined.
      We envisioned smart speakers might be used as communication tools with patients to obtain histories or answer questions while the ED workup was pending. Alternatively, they might be used by healthcare workers to call for additional supplies and prevent unnecessary don/doffing of PPE. This study seeks to explore the use of a smart speaker within a large Northeast health system that experienced a high COVID-19 burden by characterizing the content of voice commands and pattern of smart speaker utilization.

      METHODS

      Setting and Design

      A retrospective observational analysis from May 2020 to October 2020 in three EDs. These EDs are adult academic, pediatric academic, and community EDs that see a combined annual patient volume greater than 190,000.

      Smart Speaker

      Amazon Echo Show® is a hands-free smart speaker device that is activated by voice commands and designed to respond when applicable. This touchscreen device uses Amazon's® virtual assistant, Alexa®, to respond to user queries using sound and video. It can also conduct video calls through its “drop in on [device name]” voice command which serves as an intercom between two smart speakers allowing two users to have a conversation by video. This smart speaker tracks verbal commands, time stamps, and any device response. During the device implementation in physical patient rooms and clinical (nursing and provider) workstations, an email was sent to all ED staff inclusive of clinicians, nursing, and registration, notifying them of device placement and providing them with brief instructions on use. The device was installed in 51 patient rooms and 23 clinical workstations (appendix 1). Devices were prioritized to the adult patient rooms given COVID-19 burden during the early pandemic. Clinical workstations are organized into clusters of computers with each cluster of 2-3 having a device. This installation was made feasible under governmental guidance for good faith provision of telehealth services9. Of note, any nursing station device could connect to any patient room within the same ED. Conversely, devices in patient rooms could connect to any nursing station. Given the rapidly changing operational environment of COVID-19 and privacy concerns, only voice commands were recorded and the devices were not linked to the patient's electronic health record.

      Objectives

      The primary objective was to categorize the content of smart speaker voice commands which could serve as a proxy for device utilization in the ED. The secondary objective was to determine if there was a difference in voice commands performed by shift given the tendency for decreased ED staffing at night might translate to increased remote communication to improve workflow.

      Analysis

      Commands were downloaded from the Amazon® account and analyzed for content. All interactions with the device, indicated by device awakening with the key word “Alexa” were considered as content. Of note, device activation due to ambient background noise was excluded (Appendix 2). This noise was defined as data entries that included an incidental awakening of the device without any subsequent command or query. All individual data entries were reviewed by a study researcher (JF) for patterns of usage and categorized accordingly. A random subset (determined with a random number generator) of 200 entries was confirmed for categorization accuracy and interrater agreement by a second investigator (RS), consistent with manual chart review research guidelines10-11. Raw interrater reliability was 96.4% and kappa was 92.2 (95% CI 0.89-0.95).
      Classification of commands was done conservatively, being considered non-patient care related and non-clinical in nature unless there was clear contextual evidence to do otherwise (Table 1). A chi-square test of independence was performed to examine the relationship between device usage on day shift and night shift using Excel 2013. This study was approved by the Institutional Review Board.
      Table 1Category definitions for command classification
      CategorizationDefinitionExample Command
      Patient Care Related
      Smart Speaker ToolsTools offered by this smart speaker that appeared to be used to augment the patient experience“set timer ten minutes”
      ClinicalRelating to clinical patient care“drop in on triage”
      EnvironmentRelating to the hospital environment“turn off the lights”
      InformationalInquiries that may supplement patient care related activities“give me a list of available devices”
      PhonePhone usage that was in-house“call the front desk”
      PhotoAttempted use of the device to take and/or upload photos“how do I upload this photo”
      Non-Patient Care Related
      Smart Speaker ToolsTools offered by this smart speaker that would not necessarily augment the patient experience“let me see the calendar”
      ConversationalInquiries of the smart speaker“Alexa, how are you?”
      Device CommandsCommands with a device task“power down”
      EntertainmentRequesting entertainment“play country music”
      EnvironmentCommands that appeared to be relating to the hospital environment, but not clearly interpreted enough to be categorized as patient care related
      InformationalInquiries unrelated to patient care activities“what's the weather for tomorrow”
      PhonePhone usage to mobile devices or devices that may have been located off-campus“call +1(203)xxx-xxxx”
      PhotoAttempted use of the device to take and/or view photos“take a selfie”

      RESULTS

      Of the 1,232 total commands analyzed, 200 (16.23%) were determined to be patient care related and 1,032 (83.76%) were non-patient care related. Data is summarized in Table 2 and visually displayed in appendix 2.
      Table 2Classification statistics of collected smart speaker commands and queries
      Smart Speaker Commands(N=1232)
      Overall N, (%)Night Shift 1900-0659N, (%)Day Shift 0700-1859 N, (%)P value
      Patient Care Related20012476<0.001
      Clinical155, (77.5%)92, (74.2%)63, (82.9%)
      Environment23, (11.5%)13, (10.5%)10, (13.2%)
      Smart Speaker Tools11, (5.5%)10, (8.1%)1, (1.3%)
      Phone7, (3.5%)6, (4.8%)1, (1.3%)
      Photo3, (1.5%)2, (1.6%)1, (1.3%)
      Informational1, (0.5%)1, (0.8%)0, (0%)
      Non-Patient Care Related1032680352<0.001
      Entertainment644, (62.4%)451, (66.3%)193, (54.8%)
      Informational183, (17.7%)113, (16.6%)70, (19.9%)
      Conversational85, (8.2%)57, (8.4%)28, (8.0%)
      Device Command63, (6.1%)32, (4.7%)31, (8.8%)
      Phone29, (2.8%)12, (1.8%)17, (4.8%)
      Smart Speaker Tools18, (1.7%)11, (1.6%)7, (2.0%)
      Photo6, (0.6%)3, (0.4%)3, (0.9%)
      Environment4, (0.4%)1, (0.2%)3, (0.9%)
      Of the 200 patient care related commands, 155 (77.5%) were clinical in nature. Further breakdown of the patient care related commands included 23 (11.5%) environment enhancing commands, 11 (5.50%) smart speaker tools, 7 (3.50%) phone usage commands, 3 (1.50%) photo commands, and 1 (0.5%) informational. Among non-patient care related commands and queries, devices were used 644 (62.4%) times to access entertainment. Other non-patient care related commands included 183 (17.73%) informational, 85 (8.24%) conversational, 63 (6.10%) device commands, 29 (2.81%) phone usage, 18 (1.74%) smart speaker tools, 6 (0.58%) photo commands, and 4 (0.39%) environment commands.
      An analysis of smart speaker utilization throughout the workday revealed that 124 (64%) of all patient care commands were given during night shift, from 1900-0659, which was statistically significant (p<0.001). The distribution included 92 (74.19%) clinical commands, 13 (10.48%) patient care related environment enhancing commands, 10 (8.06%) patient care related access of smart speaker tools, 6 (4.84%) phone usage commands, 2 (1.61%) photo commands, and 1 (0.81%) informational. As with patient care related commands, devices were also significantly more utilized for non-patient care related commands during the night shift hours (p<0.001). We found that 680 (65.89%) of all non-patient care related commands were given from 1900-0659. This breakdown included 451 (66.32%) entertainment commands, 113 (16.62%) informational, 57 (8.38%) conversational, 32 (4.71%) device commands, 12 (1.76%) phone usage, 11 (1.62%) smart speaker tools, 3 (0.44%) photo commands, and 1 (0.15%) environment command.

      DISCUSSION

      We found fairly common use of a novel application of smart speakers in the ED during the COVID-19 pandemic as a clinical communication tool, televisit terminal, and entertainment hub for providers and patients. To our knowledge this is the first examination of smart speaker usage in the ED. Although there was no formal orientation provided in the use of these smart speakers, this exploratory analysis reveals the devices were used for a range of patient and non-patient care activities suggesting a modest staff acceptance, particularly during night shifts.
      The initial hope of this smart speaker implementation was to help protect staff from unnecessary exposure and conserve PPE. By allowing staff to communicate with a patient from outside the room (ie answer questions) or allow a healthcare worker to stay in a patient room and obtain supplies via a runner (ie obtain a forgotten medical item) theoretically conserves PPE. While we were unable to track PPE consumption in synchrony with device usage, 200 patient care related commands over 6 months is unlikely to have made an impact on overall PPE usage. Recently, Amazon® announced that it would be installing similar smart speakers in partnering hospitals.12 They hypothesize this will increase productivity and conservation of medical supplies and PPE, as smart speaker utilization decreases the need for staff to enter patient rooms unnecessarily.12 Devices might even be used to provide patients with quick status updates, offering greater transparency and enhancing communication and the patient experience. In the future, given the pandemic limited family visitation, these ED devices could enable virtual family visits or participation in medical decision making, building on ICU experience.13,14 More comprehensive integration of smart speakers in patient rooms may also offer additional resources to those delivering patient care, such as ease of access to medical translating services. These all present ideas for future investigation if smart speakers could be integrated with patient call bell, supply chain, and patient satisfaction data streams. Of additional interest was our finding that devices were more frequently used during night shift hours. The mechanism is unclear and could be considered for further investigation. This might be a consequence of staffing ratios motivating staff to find workflow improvements or possibly greater need for patient and staff relaxation during night hours.
      In our analysis of non-patient care related commands, devices were most frequently used to access entertainment, suggesting a potential use case for the devices to enhance patient and staff wellness. The high incidence of burnout among those on the frontlines in healthcare, particularly in EDs, has long been under consideration15. Given the exacerbating effects that COVID-19 has had on these burnout rates16, there has been recent use of voice-activated devices to manufacture immersive environment rooms in an effort to decrease acute stress among staff17. By employing similar concepts and integrating smart speakers into patient rooms, hospitals may be able to offer wellness benefits to patients as well. Closer analysis of commands revealed that users expected more advanced integration of the devices with patient rooms than the existing hospital infrastructure offered. Environmental commands such as “turn off the lights” and “set the heat” were suggestive of an opportunity to use smart speakers to increase patient comfort while reducing staff contact with potential fomites.

      Limitations

      Given the novelty of smart speaker installation in the hospital setting coupled with the rapidly changing operational environment of the first waves of COVID-19, there are study limitations that present a focus for future studies. While the HHS OCR policy (expired July 2022) allowed more permissive use of this technology, lack of an existing vendor-health system contractual agreement, device connection to the electronic health record, and concerns regarding patient privacy, prevent conversation recording. This limited in-depth analysis of patient care activities, sentiments, and other linguistics. As a similar privacy protection, we were unable to retrieve usable demographic information that would have offered a better understanding of which specific patients or the roles of those healthcare workers who used the devices. Specifically, if a voice command originated from a patient room, we are unable to differentiate if this was initiated by a patient or healthcare worker. These limitations also preclude analysis to patients who did not have smart speaker used. Future study should look to operationalize smart speakers such that privacy concerns are addressed, maintain compliance with HHS OCR, to allow for more in-depth analysis of equity and care outcomes. In aggregate, our analysis is contextually limited and it was for this reason that we approached data categorization conservatively, ultimately focusing our attention on an evaluation of overall device utilization.

      Conclusions

      We noted smart speaker engagement over a range of activities by both frontline ED staff and patients with most use for entertainment and direct patient communication. After addressing privacy concerns, future studies should explore the use of devices on staff wellness and patient satisfaction. This exploratory analysis also suggests an interest in “smart” patient care rooms to enhance efficacy of patient care and improve patient experience.

      Article Summary

      • 1
        Why is this topic important?
        • a
          Smart speakers are a household device that were allowed during the COVID-19 pandemic to help prevent staff infections. Their usage amongst ED staff is unknown.
      • 2
        What does this brief report attempt to show?
        • a
          This exploratory analysis examines the novel usage of smart speakers and their usage across a range of activities and were not limited to patient care.
      • 3
        What are the key findings?
        • a
          Smart speakers were primarily used for entertainment and patient clinical care. However privacy concerns regarding devices limited deeper contextual analysis
      • 4
        How is patient care affected?
        • a
          Entertainment may have implications for staff wellness and patient satisfaction and this implementation revealed a patient desire for “smart” hospital rooms.

      References

      • 1
        Weible CM, Nohrstedt D, Cairney P, et al. COVID-19 and the policy sciences: initial reactions and perspectives. Policy Sci. 2020;1-17.
      • 2
        Liu Q, Luo D, Haase JE, et al. The experiences of health-care providers during the COVID-19 crisis in China: a qualitative study. The Lancet Global Health. (2020)
      • 3
        Sangal RB, Scofi JE, Parwani V, et al. Less Social Emergency Departments: Implementation of Workplace Contact Reduction During COVID-19. Emergency Medicine Journal 2020;37:463-466.
      • 4
        Medicare Telemedicine Health Care Provider Fact Sheet. CMS.gov. https://www.cms.gov/newsroom/fact-sheets/medicare-telemedicine-health-care-provider-fact-sheet. Published March 17, 2020. Accessed May 11, 2020.
      • 5
        Mann DM, Chen J, Chunara R, Testa PA, Nov O. COVID-19 Transforms Health Care Through Telemedicine: Evidence from the Field. Journal of the American Medical Informatics Association 2020;27(7):1132-1134.
      • 6
        Essay P, Shahin TB, Balkan B, et al. The Connected Intensive Care Unit Patient: Exploratory Analyses and Cohort Discovery from a Critical Care Telemedicine Database. JMIR Medical Informatics 2019;7(1).
      • 7
        Subramanian S, Pamplin JC, Hravnak M, et al. Tele-Critical Care: An Update from the Society of Critical Care Medicine Tele-ICU Committee. Critical Care Medicine 2020;48(4):553-561.
      • 8
        Udeh C, Udeh B, Rahman N, et al. Telemedicine/Virtual ICU: Where Are We and Where Are We Going? Methodist DeBakey Cardiovascular Journal 2018;14(2):126-133.
      • 9
        Secretary of the Department of Health and Human Services for the Office of Civil Rights. Notification of Enforcement Discretion for Telehealth. HHS.gov. https://www.hhs.gov/hipaa/for-professionals/special-topics/emergency-preparedness/notification-enforcement-discretion-telehealth/index.html. Last Reviewed January 20, 2021. Accessed August 19, 2021.
      • 10
        Cantor AB. Sample-size calculations for cohen's kappa. Psychol Methods 1996;1:150-153.
      • 11
        Kaji AH, Schringer D, Green S. Looking through the retrospectoscope: reducing bias in emergency medicine chart review studies. Ann Emerg Med 2014;64:292-298.
      • 12
        Schuster-Bruce, C. Amazon is putting Alexa next to hospital beds throughout the US, it says it will boost productivity because staff can go into patients’ rooms less. Business Insider, Yahoo! News. https://news.yahoo.com/amazon-putting-alexa-next-hospital-123026473.html?guccounter=1. Published October 26, 2021. Accessed November 12, 202.
      • 13
        Kennedy NR, Steinberg A, Arnold RM et al. Perspectives on Telephone and Video Communication in the Intensive Care Unit during COVID-19. Ann Am Thorac Soc. 2021 May;18(5):838-847. doi: 10.1513/AnnalsATS.202006-729OC. PMID: 33181033; PMCID: PMC8086546.
      • 14
        Rose L, Yu L, Casey J et al. Communication and Virtual Visiting for Families of Patients in Intensive Care during the COVID-19 Pandemic: A UK National Survey. Ann Am Thorac Soc. 2021 Oct;18(10):1685-1692. doi: 10.1513/AnnalsATS.202012-1500OC. PMID: 33617747; PMCID: PMC8522289.
      • 15
        15 Nesbitt TS. Development of a telemedicine program. Western Journal of Medicine 2000;173(3):169-174.
      • 16
        Chor WP, Ng WM, Cheng L, et al. Burnout amongst emergency healthcare workers during the COVID-19 pandemic: A multi-center study. The American Journal of Emergency Medicine 2020.
      • 17
        Putrino D, Ripp J, Herrera JE, et al. Multisensory, Nature-Inspired Recharge Rooms Yield Short-Term Reductions in Perceived Stress Among Frontline Healthcare Workers. Front Psychol 2020;11:560833.

      Appendix. Supplementary materials