A widely used sepsis model has been found to correctly identify more than two-thirds of cases of hospitalised patients suspected of having the condition.
The early detection and appropriate treatment of patients with sepsis has been shown to have a huge mortality benefit. With several sepsis models available to enable the early detection of sepsis, there is evidence to suggest that clinical staff find these alerts to be helpful for identifying suspected cases. The incorporation of alert systems into the electronic health record has meant that a sepsis model alert system is commonly found and used within healthcare systems.
In the US, an extensively used sepsis alert system is the Epic Sepsis Model (ESM) and this has been developed and validated by the manufacturer, Epic Systems Corporation. Nevertheless, as a proprietary system, there is limited publicly available information on the performance of the sepsis model. This lack of transparency led a from the Department of Internal Medicine, Michigan University, US, to undertake an independent and retrospective, external validation of the ESM using data from their academic medical centre. The researchers included all adult patients (18 years and over) admitted to their medical centre over a 10-month period and calculated ESM scores for all patients throughout their hospital stay to predict the onset of sepsis. The threshold for the ESM sepsis model alert was set at 6, which is broadly in line with the recommendations of the manufacturer (5 – 9).
Findings
A total of 27,697 patients with a median age of 56 years (57% female) were included in the analysis. Sepsis occurred in 2552 patients and the ESM identified 183 (7%) cases which were missed by clinicians and therefore did not receive timely administration of antibiotics. However, the ESM failed to identify 1709 (67%) patients who were clinician diagnosed with sepsis. A further problem highlighted by the evaluation was how the ESM system generated alerts (i.e., where the sepsis model calculated a score of > 6 for a patient) in 18% of all patients, who were found not to have sepsis, raising the potential for alert fatigue. In fact, based on these finding, the team estimated that if clinicians followed the alerts generated by the ESM system, they would have to evaluate 109 patients to find a single case of sepsis.
Commenting on their results, the authors noted that while the manufacturer had reported that the ESM could identify 76 to 83% of sepsis cases, their real-world analysis indicated the much lower level of 63%. They concluded that the widespread use of the ESM system raised some concern over sepsis management, especially since the sepsis model had been implemented in a large number of hospitals across the country.
Citation
Wong A et al. External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalised Patients. JAMA Int Med 2021
