Advertisement

INTERRATER RELIABILITY OF POINT-OF-CARE CARDIOPULMONARY ULTRASOUND IN PATIENTS WITH SEPTIC SHOCK: AN ANALYSIS OF AGREEMENT BETWEEN TREATING CLINICIAN AND EXPERT REVIEWERS

Published:December 22, 2022DOI:https://doi.org/10.1016/j.jemermed.2022.12.020

      Abstract

      Background

      Cardiopulmonary ultrasound (CPUS) is commonly used to assess cardiac function and preload status in patients with septic shock. However, the reliability of CPUS findings at the point of care is unknown.

      Objective

      To assess interrater reliability (IRR) of CPUS in patients with suspected septic shock between treating emergency physicians (EPs) vs emergency ultrasound (EUS) experts.

      Methods

      Single-center, prospective, observational cohort enrolling patients (n = 51) with hypotension and suspected infection. Treating EPs performed and interpreted CPUS for cardiac function parameters (left ventricular [LV] function and right ventricular [RV] function and size) and preload volume parameters (inferior vena cava [IVC] diameter and pulmonary B-lines). The primary outcome was IRR (assessed by Kappa values [κ] and intraclass correlation coefficient [ICC]) between EP and EUS-expert consensus. Secondary analyses examined the effects on IRR of operator experience, respiratory rate, and known difficult views on a Cardiology-performed echocardiogram.

      Results

      IRR was fair for LV function, κ = 0.37, 95% confidence interval (CI) 0.1–0.64; poor for RV function, κ = −0.05, 95% CI −0.6–0.5; moderate for RV size, κ = 0.47, 95% CI 0.07–0.88; and substantial for B-lines, κ = 0.73, 95% CI 0.51–0.95 and IVC size, ICC = 0.87, 95% CI 0.2–0.99. Involvement of ultrasound-trained faculty was associated with improved IRR of RV size (p = 0.002), but not other CPUS domains.

      Conclusions

      Our study demonstrated high IRR for preload volume parameters (IVC size and presence of B-lines), but not for cardiac parameters (LV function and RV function and size) in patients presenting with concern for septic shock. Future research must focus on determining sonographer and patient-specific factors affecting CPUS interpretation in real-time.

      Keywords

      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Journal of Emergency Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      REFERENCES

        • Jones AE
        • Tayal VS
        • Sullivan DM
        • Kline JA.
        Randomized, controlled trial of immediate versus delayed goal-directed ultrasound to identify the cause of nontraumatic hypotension in emergency department patients.
        Crit Care Med. 2004; 32: 1703-1708
        • Volpicelli G
        • Lamorte A
        • Tullio M
        • et al.
        Point-of-care multiorgan ultrasonography for the evaluation of undifferentiated hypotension in the emergency department.
        Intensive Care Med. 2023; 39: 1290-1298
        • Sasmaz MI
        • Gungor F
        • Guven R
        • Akyol KC
        • Kozaci N
        • Kesapli M.
        Effect of focused bedside ultrasonography in hypotensive patients on the clinical decision of emergency physicians.
        Emerg Med Int. 2017; 20176248687
        • Lichtenstein DA
        • Meziere GA.
        Relevance of lung ultrasound in the diagnosis of acute respiratory failure: the BLUE protocol.
        Chest. 2008; 134: 117-125
        • Mantuani D
        • Frazee BW
        • Fahimi J
        • Nagdev A.
        Point-of-care multi-organ ultrasound improves diagnostic accuracy in adults presenting to the emergency department with acute dyspnea.
        West J Emerg Med. 2016; 17: 46-53
        • Bekgoz B
        • Kilicaslan I
        • Bildik F
        • et al.
        BLUE protocol ultrasonography in Emergency Department patients presenting with acute dyspnea.
        Am J Emerg Med. 2019; 37: 2020-2027
        • Shokoohi H
        • Boniface KS
        • Pourmand A
        • et al.
        Bedside ultrasound reduces diagnostic uncertainty and guides resuscitation in patients with undifferentiated hypotension.
        Crit Care Med. 2015; 43: 2562-2569
      1. Ultrasound guidelines: emergency, point-of-care and clinical ultrasound guidelines in medicine.
        Ann Emerg Med. 2017; 69: e27-e54
        • Stowell JR
        • Kessler R
        • Lewiss RE
        • et al.
        Critical care ultrasound: a national survey across specialties.
        J Clin Ultrasound. 2018; 46: 167-177
        • Prager R
        • Wu K
        • Bachar R
        • et al.
        Blinding practices during acute point-of-care ultrasound research: the BLIND-US meta-research study.
        BMJ Evid Based Med. 2021; 26: 110-111
        • Chiem AT
        • Chan CH
        • Ander DS
        • Kobylivker AN
        • Manson WC.
        Comparison of expert and novice sonographers' performance in focused lung ultrasonography in dyspnea (FLUID) to diagnose patients with acute heart failure syndrome.
        Acad Emerg Med. 2015; 22: 564-573
        • Choi BG
        • Mukherjee M
        • Dala P
        • et al.
        Interpretation of remotely downloaded pocket-size cardiac ultrasound images on a web-enabled smartphone: validation against workstation evaluation.
        J Am Soc Echocardiogr. 2011; 24: 1325-1330
        • Thomas-Mohtat R
        • Sable C
        • Breslin K
        • et al.
        Interpretation errors in focused cardiac ultrasound by novice pediatric emergency medicine fellow sonologists.
        Crit Ultrasound J. 2018; 10: 33
        • Gravel CA
        • Monuteaux MC
        • Levy JA
        • Miller AF
        • Vieira RL
        • Bachur RG.
        Interrater reliability of pediatric point-of-care lung ultrasound findings.
        Am J Emerg Med. 2020; 38: 1-6
        • DeSanti RL
        • Cowan EA
        • Kory PD
        • Lasarev MR
        • Schmidt J
        • Al-Subu AM
        The inter-rater reliability of pediatric point-of-care lung ultrasound interpretation in children with acute respiratory failure.
        J Ultrasound Med. 2022; 41: 1159-1167
        • Zhang H
        • He W
        • Lian H
        • et al.
        Physicians' abilities to obtain and interpret focused cardiac ultrasound images from critically ill patients after a 2-day training course.
        BMC Cardiovasc Disord. 2020; 20: 151
        • Schnittke N
        • Schmidt J
        • Barvalia U
        • Emmerich K
        • Kory P
        • Damewood S.
        Assessment of dynamic changes in cardiac function during resuscitation of patients with suspected septic shock: a prospective, observational, cohort study.
        Am J Emerg Med. 2020; 38: 2653-2657
        • Emergency Ultrasound Imaging Criteria Compendium
        Ann Emerg Med. 2016; 68: e11-e48
        • Lang RM
        • Badano LP
        • Mor-Avi V
        • et al.
        Recommendations for cardiac chamber quantification by echocardiography in adults: an update from the American Society of Echocardiography and the European Association of Cardiovascular Imaging.
        Eur Heart J Cardiovasc Imaging. 2015; 16: 233-270
        • Rudski LG
        • Lai WW
        • Afilalo J
        • et al.
        Guidelines for the echocardiographic assessment of the right heart in adults: a report from the American Society of Echocardiography endorsed by the European Association of Echocardiography, a registered branch of the European Society of Cardiology, and the Canadian Society of Echocardiography.
        J Am Soc Echocardiogr. 2010; 23 (quiz 786–8): 685-713
        • Feissel M
        • Michard F
        • Faller JP
        • Teboul JL.
        The respiratory variation in inferior vena cava diameter as a guide to fluid therapy.
        Intensive Care Med. 2004; 30: 1834-1837
        • Lee CW
        • Kory PD
        • Arntfield RT.
        Development of a fluid resuscitation protocol using inferior vena cava and lung ultrasound.
        J Crit Care. 2016; 31: 96-100
        • Landis JR
        • Koch GG.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Benjamini Y
        • Krieger AM
        • Yekutieli D.
        Adaptive linear step-up procedures that control the false discovery rate.
        Biometrika. 2006; 93: 491-507
        • Orde S
        • Slama M
        • Yastrebov K
        • et al.
        Subjective right ventricle assessment by echo qualified intensive care specialists: assessing agreement with objective measures.
        Crit Care. 2019; 23: 70
        • Feinstein AR
        • Cicchetti DV.
        High agreement but low kappa: I. The problems of two paradoxes.
        J Clin Epidemiol. 1990; 43: 543-549
        • Moore CL
        • Rose GA
        • Tayal VS
        • Sullivan DM
        • Arrowood JA
        • Kline JA.
        Determination of left ventricular function by emergency physician echocardiography of hypotensive patients.
        Acad Emerg Med. 2002; 9: 186-193
        • Randazzo MR
        • Snoey ER
        • Levitt MA
        • Binder K.
        Accuracy of emergency physician assessment of left ventricular ejection fraction and central venous pressure using echocardiography.
        Acad Emerg Med. 2003; 10: 973-977
        • Bahl A
        • Johnson S
        • Altwail M
        • et al.
        Left ventricular ejection fraction assessment by emergency physician-performed bedside echocardiography: a prospective comparative evaluation of multiple modalities.
        J Emerg Med. 2021; 61: 711-719
        • American College of Emergency Physicians
        Emergency ultrasound imaging criteria compendium. American College of Emergency Physicians.
        Ann Emerg Med. 2006; 48: 487-510
        • Kumar A
        • Weng Y
        • Graglia S
        • et al.
        Interobserver agreement of lung ultrasound findings of COVID-19.
        J Ultrasound Med. 2021; 40: 2369-2376
        • Evans L
        • Rhodes A
        • Alhazzani W
        • et al.
        Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic Shock 2021.
        Crit Care Med. 2021; 49: e1063-e1143
        • Lafon T
        • Appert A
        • Hadj M
        • et al.
        Comparative early hemodynamic profiles in patients presenting to the emergency department with septic and nonseptic acute circulatory failure using focused echocardiography.
        Shock. 2020; 53: 695-700