Abstract
Background
Physicians tend to overestimate patients’ pretest probability of having bacteremia. The low yield of blood cultures and contaminants is associated with significant financial cost, as well as increased length of stay and unnecessary antibiotic treatment.
Objective
This study examined the abilities of the National Early Warning Score (NEWS), the Quick Sequential Organ Failure Assessment (qSOFA), the Modified Sequential Organ Failure Assessment (mSOFA), and two versions of the causal probabilistic network, SepsisFinder™ (SF) to predict bacteremia in adult emergency department (ED) patients.
Methods
This cohort study included adult ED patients from a large urban, academic tertiary hospital, with blood cultures obtained within 24 h of admission between 2016 and 2017. The outcome measure was true bacteremia. NEWS, qSOFA, mSOFA, and the two versions of SF score were calculated for all patients based on the first available full set of vital signs within 2 h and laboratory values within 6 h after drawing the blood cultures. Area under the receiver operating characteristic curve (AUROC) was calculated for each scoring system.
Results
The study included 3106 ED patients, of which 199 (6.4%) patients had true bacteremia. The AUROCs for prediction of bacteremia were: NEWS = 0.65, qSOFA = 0.60, SF I = 0.65, mSOFA = 0.71, and SF II = 0.80.
Conclusions
Scoring systems using only vital signs, NEWS, and SF I showed moderate abilities in predicting bacteremia, whereas qSOFA performed poorly. Scoring systems using both vital signs and laboratory values, mSOFA and especially SF II, showed good abilities in predicting bacteremia.
Keywords
Introduction
Patients with suspected infections are at risk of bacteremia (
1
). Bacteremia, confirmed by a positive blood culture, is common, with an incidence rate of 114–166 per 100,000 person-years, among patients treated in public hospitals in the North Denmark Region (2
). The decision to obtain a blood culture is often subjective; it is typically the result of clinical judgement, justified by the presence of fever (3
,4
). However, the prevalence of bacteremia is below 8% (5
,6
). Physicians tend to overestimate patients’ pretest probability of having bacteremia (7
). Physicians probably do this to limit patient risks associated with missed bacteremia, as missed bacteremia can be associated with high mortality (6
,8
). The tendency to overestimate pretest probability may be due to a sparse set of guidelines clearly describing which patients should have blood cultures drawn (4
,6
). The low yield of blood cultures and contaminants is associated with significant financial costs, as well as increased length of stay and unnecessary antibiotic treatment (9
).Prediction models for risk stratification of patients with suspected bacteremia have been developed and tested to support physicians in deciding whether to obtain blood cultures, but none are widely used in clinical practice (
10
,11
). This raises the question of whether already implemented scoring systems used for other purposes with readily available values could also predict bacteremia, such as the National Early Warning Score (NEWS), the Quick Sequential Organ Failure Assessment (qSOFA), and the Modified Sequential Organ Failure Assessment (mSOFA) (12
,13
). In addition, a causal probabilistic network (CPN), SepsisFinder™ (SF), which is not yet in routine clinical use, has been suggested for predicting bacteremia (3
,10
). This study's aim was to examine the abilities of the scoring systems—NEWS, qSOFA, mSOFA, and SF—to predict bacteremia in adult patients admitted to the emergency department (ED) who had blood cultures drawn.Materials and Methods
Study Design and Setting
This was a retrospective cohort study of adult patients admitted to a large urban, academic tertiary ED at Aarhus University Hospital (AUH), Aarhus, Denmark, who had a blood culture drawn. The ED is a short-stay unit (usually up to 24 h), has approximately 63,000 annual patient visits, and provides 24-h emergency care to all acute patients, except those with myocardial infarction or stroke and women in labor. The health care system in Denmark is free and tax-supported for patients with a Danish Civil Personal Registration (CPR) number.
The Danish Patient Safety Authority (No. 31-1522-47) and the Danish Data Protection Agency (No. 1-16-02-371-18) approved the study. According to Danish law, ethics committee approval was not required.
Study Population
We included adult ED patients (age ≥ 18 years) with a valid Danish CPR number who had a blood culture performed in the ED between July 1, 2016 and June 30, 2017, within 24 h of ED arrival. Patients discharged or transferred to other departments within the 24-h period but who had a blood culture performed in the ED were kept in the cohort. We considered a performed blood culture as a proxy for suspicion of bacteremia. The attending physician decided whether to draw a blood culture in the ED without knowledge of this study. Only the first admission with blood cultures drawn during the inclusion period was included. The patient demographics, the first full set of vital signs within 2 h, and the laboratory values within 6 h of the blood culture being drawn were retrieved from the electronic health record and from the microbiology data system for all adult patients admitted to the ED.
Scoring Systems
We used the locally implemented version of NEWS called “Tidlig Opsporing af Kritisk Sygdom (TOKS),” which is used in EDs and on regular wards to monitor adult patients after triage in Central Denmark Region. It is based on respiration rate, saturation, systolic blood pressure, pulse, temperature, and level of consciousness using the alert, voice, pain, and unresponsive scale (AVPU). Each vital sign is weighted and summed to a final score (Appendix A). To calculate a TOKS score as soon as possible, values were taken during triage, converting the Glasgow Coma Scale (GCS) to AVPU (GCS 15 = A, GCS 14 = V, GCS 9–13 = P, and GCS ≤ 8 = U) (
14
). The TOKS score is subsequently referred to as NEWS in this study.The qSOFA uses simple bedside criteria to identify adult patients with suspected infection who are likely to have poor outcomes, and includes systolic blood pressure ≤ 100 mm Hg, a respiratory rate ≤ 22 breaths/min, and a GCS score < 15. A positive score is defined as two or more out of three criteria (Appendix B) (
15
).The Danish Society of Infectious Diseases has developed a modified version of the Sequential Organ Failure Assessment (SOFA), called mSOFA, which is suitable for patients outside of the intensive care unit. The mSOFA is based on the GCS, PaO2 (kPa), systolic blood pressure (mm Hg), bilirubin (μmol/L), creatinine (μmol/L), and platelet count (× 103/μL). A score from 0 to 4 based on a set of reference intervals is given for each organ with a final sum score (Appendix C) (). Ventilator and vasopressor treatments are not included in the calculations.
All variables had to be available prior to calculation in the above-mentioned scoring systems.
The SF is a stochastic model that uses CPN as a framework for representing medical knowledge that can be coded as dependence between stochastic variables (
17
). It was developed to predict the probability of bacteremia and 30-day mortality for patients with suspected sepsis (18
). To predict the probabilities, SF uses vital signs and laboratory values, but does not depend on a specific number of variables being present to predict the probability of bacteremia (19
). Therefore, two versions of SF were investigated: SF I, which only used vital signs, and SF II, which used vital signs and laboratory values, if and when available (Appendix D).Bacteremia
Each patient had one or more sets of blood cultures drawn, consisting of one aerobic and one anaerobic bottle, with a nominal blood sample volume of 10 mL per bottle (BacT/Alert; bioMérieux, Marcy l'Etoile, France). A set was considered positive when bacteria grew in at least one bottle.
Outcome
The outcome was true bacteremia, defined as bacterial or fungal growth in blood cultures deemed to have an etiological role on the basis of joint clinical and microbiological assessment. All blood culture isolates were classified prospectively as either bacteremia, possible contamination, or definite contamination. In accordance with Weinstein et al., coagulase-negative Staphylococci, Corynebacterium spp., Bacillus spp., and Propionibacterium acnes were considered definite contaminants unless isolated from two or more separate blood culture sets (
20
). They were considered true bacteremia or possible contamination if two of four bottles were positive for the above-mentioned bacteria. A microbiologist (MTR) reviewed all true bacteremia and possible contaminations. If a blood culture was determined as exclusively positive for contaminants, the blood culture was considered negative. If a blood culture consisted of both a contaminant and a significant bacterium, it was considered positive. We included all patients who had a blood culture drawn in the ED irrespective of the final result of the blood culture.Statistical Analysis
We used means with standard deviations, medians with interquartile ranges, and proportions as appropriate for descriptive statistics. We examined the scoring systems’ abilities to predict bacteremia using sensitivity, specificity, and positive and negative predictive values (PPV and NPV) with 95% confidence intervals (95% CI). We used Youden's index to determine which cut-off values to report in this study and calculated the area under the receiver operating characteristic curve (AUROC) using the sum score given by the scoring systems (
21
). We report the average scores for bacteremia and nonbacteremia patients. We analyzed data using STATA 16.1 (StataCorp, College Station, Texas).Results
Population Characteristics
During the 1-year study period, 4200 patients had blood cultures drawn in the ED, and 3106 patients met the inclusion criteria. Among the 1094 excluded patients were 12 patients without a valid Danish CPR number, 252 patients who were not admitted, 151 patients whose blood culture was drawn later than 24 h after admission, and 679 readmissions during the inclusion period. No patients were excluded due to missing data points.
Patient characteristics are reported in Table 1, stratified by bacteremia and nonbacteremia.
Table 1Patient Characteristics
| Bacteremia (n = 199) | Nonbacteremia (n = 2907) | ||
|---|---|---|---|
| Demographics, mean (SD) | |||
| Age, years | 69.9 (15) | 65.2 (20) | |
| Male sex, n (%) | 114 (57) | 1,406 (48) | |
| Vital signs, mean (SD) | |||
| Systolic blood pressure (mm Hg) | 129 (26) | 139 (26) | |
| Mean arterial pressure (mm Hg) | 91 (18) | 98 (18) | |
| Heart rate (beats/min) | 99 (21) | 92 (20) | |
| Respiratory rate (breaths/min) | 23 (7) | 21 (7) | |
| Temperature (°C) | 38.4 (1) | 37.8 (1) | |
| Glasgow Coma Scale - score < 15, n (%) | 42 (22) | 406 (15) | |
| SpO2 %, median [IQR] | 96 [94–98] | 97 [95–99] | |
| Blood samples, median [IQR] | |||
| C-reactive protein (mg/L) | 119 [40–213] | 49 [12–114] | |
| B-neutrophil leukocyte, fraction (%) | 89 [84–92] | 78 [70–85] | |
| P-creatinine (μmol/L) | 99 [77–152] | 78 [63–102] | |
| P-albumin (g/L), mean (SD) | 30 (5) | 33 (5) | |
| P-bilirubin (μmol/L) | 14 [8–21] | 8 [6–13] | |
| B-platelets (× 109/L) | 190 [151–237] | 241 [185–309] | |
| Arterial blood gas, median [IQR] | |||
| P(aB)-pO2 (kPa) | 9.9 [8.6–12.7] | 10.1 [8.6–12.3] | |
| P(aB)-lactate (mmol/L) | 2.5 [1.7–4.3] | 1.5 [1.1–2.4] | |
| Admission & mortality, median [IQR] | |||
| Cause of admission: fever or infection, n (%) | 65 (33) | 584 (20) | |
| Time in emergency department (hours) | 21 [8–35] | 21 [9–33] | |
| Total admission time (hours) | 150 [86–240] | 73 [30–147] | |
| Time until blood culture (min.) | 42 [30–70] | 49 [31–82] | |
| Intensive care during admission, n (%) | 30 (15) | 117 (4) | |
| Mortality during admission n (%) | 13 (7) | 112 (4) | |
| 30 days mortality from admission n (%) | 24 (12) | 257 (9) | |
SD = standard deviation; IQR = interquartile range.
Microbiologic Diagnosis
A total of 199 patients (6.4%) had true bacteremia; 71 (36%) cases were due to Gram-positive bacteria, 104 (52%) were due to Gram-negative bacteria, and 24 (12%) patients had polymicrobial bacteremia. Seventy-three patients (2%) were considered contaminated, of which 70 (2%) were definitely contaminated and three (0.1%) had possible contaminated blood cultures, which were all regarded as negative blood cultures (Appendix E).
Performance of Scoring Systems
The patients with bacteremia had higher scores compared with patients with nonbacteremia (Table 2). The data availability of the variables for calculating the five scoring systems—NEWS, qSOFA, SF I, mSOFA, and SF II—are shown in Appendix F. The AUROC values for the prediction of bacteremia were as follows: NEWS = 0.65 (95% CI 0.60–0.70), qSOFA = 0.60 (95% CI 0.56–0.64), SF I = 0.65 (95% CI 0.60–0.69), mSOFA = 0.71 (95% CI 0.65–0.76), and SF II = 0.80 (95% CI 0.76–0.83) (Figure. 1).
Table 2Average Scores for Bacteremia and Nonbacteremia Patients
| Bacteremia | Nonbacteremia | |
|---|---|---|
| NEWS, median [IQR] | 5 [2.5; 6] | 3 [1; 4] |
| qSOFA, median [IQR] | 1 [0; 1] | 0 [0; 1] |
| SF I, mean (SD) | 7.96% (4.7) | 5.87% (2.7) |
| mSOFA, median [IQR] | 3 [2; 5] | 2 [1; 3] |
| SF II, mean (SD) | 10.34% (5.2) | 5.69% (3.6) |
NEWS = National Early Warning Score; IQR = interquartile range; qSOFA = quick Sequential Organ Failure Assessment; SF = SepsisFinder; SD = standard deviation; mSOFA = modified Sequential Organ Failure Assessment.
Figure 1Area under the receiver operating characteristic (AUROC) curve for predicting bacteremia. *AUROC [95% confidence interval]. SF = SepsisFinder; mSOFA = modified Sequential Organ Failure Assessment; NEWS = National Early Warning Score; qSOFA = quick Sequential Organ Failure Assessment.
The calculated endpoints for all the cut-off values are shown in Appendix G. Table 3 presents the sensitivity, specificity, PPV, NPV, potential reduction in blood cultures, and missed bacteremia with the given cut-offs for NEWS, qSOFA, SF I, mSOFA, and SF II.
Table 3Performance of NEWS, qSOFA, SF I, mSOFA, and SF II
| Cut-Off ≥∗ | Number of Patients ≥ Cut-Off n (%) | Sensitivity % [95% CI] | Specificity % [95% CI] | PPV % [95% CI] | NPV % [95% CI] | Reduction in Blood Cultures n (%) | Missed Bacteremia n (%) § n (%) = number of patients with bacteremia under the cut-off value and the percentage of missed bacteremia based on the total number of bacteremia a given scoring system could calculate. NEWS = National Early Warning Score; qSOFA = quick Sequential Organ Failure Assessment; SF = SepsisFinder; mSOFA = modified Sequential Organ Failure Assessment; CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value. | |
|---|---|---|---|---|---|---|---|---|
| NEWS | ||||||||
| 5 | 576 (25.6) | 52.1 [43.5-60.7] | 76.1 [74.2-77.9] | 12.7 [10.1-15.7] | 96 [94.9-96.9] | 1670 (74.4) | 67/140 (47.9) | |
| qSOFA | ||||||||
| 1 | 1367 (46.7) | 62 [54.7-68.9] | 54.4 [52.5-56.3] | 8.7 [7.3-10.3] | 95.3 [94.2-96.3] | 1561 (53.3) | 73/192 (38) | |
| SF I | ||||||||
| 6.34% | 931 (30) | 52.8 [45.6-59.9] | 71.6 [69.9-73.2] | 11.3 [9.3-13.5] | 95.7 [94.7-96.5] | 2175 (70) | 94/199 (47.2) | |
| mSOFA | ||||||||
| 3 | 373 (37.7) | 65.8 [54-76.3] | 64.7 [61.5-67.8] | 13.4 [10.1-17.3] | 95.8 [93.9-97.2] | 617 (62.3) | 26/76 (34.2) | |
| SF II | ||||||||
| 5.99% | 1243 (40) | 83.4 [77.5-88.3] | 63 [61.2-64.8] | 13.4 [11.5-15.4] | 98.2 [97.5-98.8] | 1863 (60) | 33/199 (16.6) |
*Youden's index was used to determine which cut-off values to report in this table.
† n (%) = number of patients ≥ the cut-off value and the percentage of the total number of times a given scorings system could be calculated.
‡ n (%) = number of patients under the cut-off value and the percentage of the total number of times a given scorings system could be calculated.
§ n (%) = number of patients with bacteremia under the cut-off value and the percentage of missed bacteremia based on the total number of bacteremia a given scoring system could calculate.NEWS = National Early Warning Score; qSOFA = quick Sequential Organ Failure Assessment; SF = SepsisFinder; mSOFA = modified Sequential Organ Failure Assessment; CI = confidence interval; PPV = positive predictive value; NPV = negative predictive value.
Based on the highest Youden's Index, the preferred cut-offs and endpoints for the scoring systems’ abilities in predicting bacteremia are reported in Table 3 and were as follows (95% CI); NEWS cut-off ≥ 5: sensitivity 52.1% (43.5–60.7%) and specificity 76.1% (74.2–77.9%); qSOFA cut-off ≥ 1: sensitivity 62% (54.7–68.9%) and specificity 54.4% (52.5–56.3%); SF I cut-off ≥ 6.34%: sensitivity 52.8% (45.6–59.9%) and specificity 71.6% (69.9–73.2%); mSOFA cut-off ≥ 3: sensitivity of 65.8% (54–76.3%) and specificity 64.7% (61.5–67.8%); and SF II cut-off ≥ 5.99%: sensitivity of 83.4% (77.5–88.3%) and specificity 63% (61.2–64.8%).
Discussion
This study's results suggest that scoring systems using both vital signs and laboratory values have better abilities in predicting bacteremia compared to scoring systems using only vital signs. Among the scoring systems using only vital signs, SF I and NEWS showed better abilities in predicting bacteremia compared to qSOFA. The scoring systems that also included laboratory values, both mSOFA and SF II, showed good abilities in predicting bacteremia. However, SF II was able to include more patients through its CPN by continuously predicting the likelihood of bacteremia using selected values as they became available. Of the five scoring systems, SF II showed the best abilities in predicting bacteremia with the highest sensitivity, NPV, and AUROC, along with the lowest percentages of missed bacteremia.
To predict bacteremia, scoring systems are used as tools supporting clinical suspicion, but should always be secondary to good clinical judgement. Previously developed prediction tools for the prediction of bacteremia in the ED do not seem to have been clinically implemented (
10
,11
,22
). Scoring systems like NEWS, qSOFA, and mSOFA have the advantage of being well-known bedside scoring systems in the clinical setting compared with a CPN such as SF I and SF II. Furthermore, mSOFA and SF II use laboratory values, which postpone the decision-making, raising the question of whether to postpone the blood draw for cultures until laboratory values are available. Laboratory values in EDs in the Central Denmark Region are available within a maximum of 2 h. Several possible consequences of postponing blood cultures include more blood sampling and laboratory analyses, changes in workflow and procedures, and delayed treatment, as national and international guidelines require blood cultures prior to starting antibiotic treatment (23
). A 1-h delay in antibiotic treatment in sepsis patients can increase hospital mortality by 0.3–1.8%, depending on the severity of sepsis (24
). Nevertheless, this study's results showed that scoring systems using laboratory values had better abilities in predicting bacteremia compared with scoring systems that did not include laboratory values.Usman et al. retrospectively investigated the original NEWS's applicability as an early predictor of severe sepsis and septic shock in an ED and found an AUROC of 0.91 (
25
). This supports the Danish version of NEWS's potential for predicting bacteremia, together with the AUROC of 0.65 for NEWS found in this study.The qSOFA score had the lowest AUROC (0.60) among the five scoring systems in this study. This could be explained by the fact that qSOFA used the fewest variables and was the simplest scoring system of those compared in this study. In Otani et al.’s study, qSOFA score likewise showed a low AUROC of 0.59 when predicting bacteremia (
26
). The Surviving Sepsis Campaign recently published the International Guidelines for Management of Sepsis and Septic Shock 2021, recommending against using qSOFA compared with, among others, NEWS as a single screening tool for sepsis or sepsis shock (27
). Accordingly, qSOFA as a single screening tool for predicting bacteremia cannot be recommended.The literature on mSOFA's performance in predicting bacteremia is sparse (
13
). The mSOFA is highly comparable with and simpler to use in the ED setting than the regular SOFA score due to its use of systolic blood pressure instead of mean arterial pressure and PaO2 instead of PaO2/FiO2. The mSOFA achieved an AUROC of 0.71 (95% CI 0.65–0.76) based on the 990 patients in this study. Similar results have been shown in a recent study outside the intensive care unit, where the full SOFA score had an AUROC of 0.71 (95% CI 0.62–0.80) based on 427 patients (28
). The comparability of the results may contribute to future comparisons of the two scoring systems and help emphasize mSOFA's ability to predict bacteremia.The major advantage of SF is that it is not dependent on all variables being available for calculating a score. A new score can be calculated every time a new variable is added or changed during the admission process. However, SF's weakness, as shown in this study, is that to achieve a high level of discrimination for bacteremia patients, it must have laboratory values, which are often available only after requisition and sampling of blood culture. Different versions of SF are described in the literature, but to the best of our knowledge, SF has not been studied when limiting its use to include only vital signs, as in SF I.
In Andreassen et al.’s recent study on SF II, they found an AUROC of 0.75 (95% CI 0.70–0.76) in predicting bacteremia in a combined dataset from Denmark and Israel (
28
). The current study's version of SF II did not include chills, mental status, and age, but added PaO2 to compare with the later version of SF that Andreassen et al. used (28
). The AUROC of 0.80 in this study is the highest AUROC SF II achieved in predicting bacteremia compared with previous studies (18
,19
). The variation in AUROC values may also be influenced by different settings.Machine learning, smart algorithms and networks, and other terms associated with artificial intelligence (AI) have become common in the health care literature (
29
,30
). AI seems to have the potential to improve early identification of bacteremia in patients who may benefit from antibiotic administration. AI is not yet used in our ED setting; however, in the future, it seems very likely that it can be implemented for predicting and flagging patients at risk of specific events.This study's results showed the potential of scoring systems to supplement clinical judgement, as guidelines are sparse and do not clearly describe which patients should have blood cultures drawn. A prospective study should be initiated to validate the scoring systems prior to use in clinical practice, because prospective studies validating prediction models often show lower predictive abilities than validation studies (
10
). For scoring systems with sparse evidence in predicting bacteremia, validation at other sites prior to implementation in clinical practice is required.Limitations
This study has some limitations. There was a lack of clear guidelines for the decision to draw blood cultures. This may have resulted in patients with bacteremia not having blood cultures drawn and thus not being included in the study. In addition, the classification of bacteremia and nonbacteremia patients could be faulty with a misclassification bias, although this should be minimized in the study. The study's retrospective design did not permit evaluation of the potential harm caused by not drawing blood cultures from the patients with scores below the cut-offs. Furthermore, the admission, treatment, and consequences for the hypothetical missed bacteremia patients were not studied.
Conclusions
In conclusion, the five scoring systems—NEWS, qSOFA, SF I, mSOFA, and SF II—were able to predict bacteremia in adult patients who had blood cultures drawn in the ED. Using only vital signs, NEWS and SF I showed moderate abilities in predicting bacteremia, both with an AUROC of 0.65, whereas qSOFA was inferior (AUROC 0.60). Using both vital signs and laboratory values, mSOFA (AUROC 0.71), and especially SF II (AUROC 0.80), showed good abilities in predicting bacteremia. With an increasing focus on AI, SF II, especially, has potential.
Article Summary
1. Why is this topic important?
Physicians tend to overestimate patients’ pretest probability of having bacteremia, as guidelines are sparse. The low yield of blood cultures and contaminants is associated with significant financial costs, increased length of stay, and unnecessary antibiotic treatment.
2. What does this study attempt to show?
This study attempts to demonstrate National Early Warning Score (NEWS), quick Sequential Organ Failure Assessment (qSOFA), modified Sequential Organ Failure Assessment (mSOFA), and SepsisFinder's ability to predict bacteremia in adult emergency department patients.
3. What are the key findings?
The scoring systems that used vital signs and laboratory values showed better abilities in predicting bacteremia compared with scoring systems that only used vital signs.
4. How is patient care impacted?
Scoring systems can supplement physicians’ clinical judgment when deciding whether to perform a blood culture and start treatment.
Appendix. Supplementary materials
References
- Bacteremia: a consideration of some experimental and clinical aspects.Yale J Biol Med. 1954; 26: 241-262
- Temporal changes in the incidence and 30-day mortality associated with bacteremia in hospitalized patients from 1992 through 2006: a population-based cohort study.Clin Infect Dis. 2011; 52: 61-69
- Blood cultures for febrile patients in the acute care setting: too quick on the draw?.J Am Acad Nurse Pract. 2008; 20: 539-546
- Culture if spikes? Indications and yield of blood cultures in hospitalized medical patients.J Hosp Med. 2016; 11: 336-340
- Predicting bacteremia in hospitalized patients. A prospectively validated model.Ann Intern Med. 1990; 113: 495-500
- Does this adult patient with suspected bacteremia require blood cultures?.JAMA. 2012; 308: 502-511
- Availability, wishful thinking, and physicians’ diagnostic judgments for patients with suspected bacteremia.Med Decis Making. 1991; 11: 159-168
- Best clinical practice: blood culture utility in the emergency department.J Emerg Med. 2016; 51: 529-539
- Contaminant blood cultures and resource utilization. The true consequences of false-positive results.JAMA. 1991; 265: 365-369
- Predicting bacteraemia in validated models–a systematic review.Clin Microbiol Infect. 2015; 21: 295-301
- Who needs a blood culture? A prospectively derived and validated prediction rule.J Emerg Med. 2008; 35: 255-264
- National Early Warning Score (NEWS): standardising the assessment of acute-illness severity in the NHS.Royal College of Physicians, London2012
- A modified sequential organ failure assessment score for critical care triage.Disaster Med Public Health Prep. 2010; 4: 277-284
- Should age be included as a component of track and trigger systems used to identify sick adult patients?.Resuscitation. 2008; 78: 109-115
- The third international consensus definitions for sepsis and septic shock (Sepsis-3).JAMA. 2016; 315: 801-810
- Rekommandationer for initial behandling af sepsis og septisk shock.Dansk Selskab for Infektionsmedicin;. 2021; ([Danish]): 11
- Generic causal probabilistic networks: a solution to a problem of transferability in medical decision support.Comput Methods Programs Biomed. 2008; 89: 189-201
- Prediction of bacteraemia and of 30-day mortality among patients with suspected infection using a CPN model of systemic inflammation.IFAC-PapersOnLine. 2018; 51: 116-121
- Prediction of bacteremia using TREAT, a computerized decision-support system.Clin Infect Dis. 2006; 42: 1274-1282
- The clinical significance of positive blood cultures: a comprehensive analysis of 500 episodes of bacteremia and fungemia in adults. I. Laboratory and epidemiologic observations.Rev Infect Dis. 1983; 5: 35-53
- Index for rating diagnostic tests.Cancer. 1950; 3: 32-35
- Prediction of bacteremia in the emergency department: an external validation of a clinical decision rule.Eur J Emerg Med. 2016; 23: 44-49
- Surviving sepsis campaign: international guidelines for management of sepsis and septic shock: 2016.Intensive Care Med. 2017; 43: 304-377
- The timing of early antibiotics and hospital mortality in sepsis.Am J Respir Crit Care Med. 2017; 196: 856-863
- Comparison of SIRS, qSOFA and NEWS for the early identification of sepsis in the emergency department.Am J Emerg Med. 2019; 37: 1490-1497
- Clinical prediction rule is more useful than qSOFA and the Sepsis-3 definition of sepsis for screening bacteremia.Am J Emerg Med. 2021; 46: 84-89
- Surviving sepsis campaign: international guidelines for management of sepsis and septic shock 2021.Crit Care Med. 2021; 49: e1063-e1143
- A comparison of predictors for mortality and bacteraemia in patients suspected of infection.BMC Infect Dis. 2021; 21: 864
- Artificial intelligence in healthcare: review and prediction case studies.Engineering. 2020; 6: 291-301
- Global evolution of research in artificial intelligence in health and medicine: a bibliometric study.J Clin Med. 2019; 8: 360
Article Info
Publication History
Published online: September 15, 2022
Accepted:
September 4,
2022
Received in revised form:
July 2,
2022
Received:
March 29,
2022
Publication stage
In Press Corrected ProofIdentification
Copyright
© 2022 The Authors. Published by Elsevier Inc.
User License
Creative Commons Attribution (CC BY 4.0) | How you can reuse 
Elsevier's open access license policy
Creative Commons Attribution (CC BY 4.0)
Permitted
- Read, print & download
- Redistribute or republish the final article
- Text & data mine
- Translate the article
- Reuse portions or extracts from the article in other works
- Sell or re-use for commercial purposes
Elsevier's open access license policy
00531-5&id=gr1.jpg){kind=link}