Skip to main content
Download PDF
Download ePub

Development and validation of the early warning system scores ontology

Journal of Biomedical Semantics volume 14, Article number: 14 (2023) Cite this article

Abstract

Background

Clinical early warning scoring systems, have improved patient outcomes in a range of specializations and global contexts. These systems are used to predict patient deterioration. A multitude of patient-level physiological decompensation data has been made available through the widespread integration of early warning scoring systems within EHRs across national and international health care organizations. These data can be used to promote secondary research. The diversity of early warning scoring systems and various EHR systems is one barrier to secondary analysis of early warning score data. Given that early warning score parameters are varied, this makes it difficult to query across providers and EHR systems. Moreover, mapping and merging the parameters is challenging. We develop and validate the Early Warning System Scores Ontology (EWSSO), representing three commonly used early warning scores: the National Early Warning Score (NEWS), the six-item modified Early Warning Score (MEWS), and the quick Sequential Organ Failure Assessment (qSOFA) to overcome these problems.

Methods

We apply the Software Development Lifecycle Framework—conceived by Winston Boyce in 1970—to model the activities involved in organizing, producing, and evaluating the EWSSO. We also follow OBO Foundry Principles and the principles of best practice for domain ontology design, terms, definitions, and classifications to meet BFO requirements for ontology building.

Results

We developed twenty-nine new classes, reused four classes and four object properties to create the EWSSO. When we queried the data our ontology-based process could differentiate between necessary and unnecessary features for score calculation 100% of the time. Further, our process applied the proper temperature conversions for the early warning score calculator 100% of the time.

Conclusions

Using synthetic datasets, we demonstrate the EWSSO can be used to generate and query health system data on vital signs and provide input to calculate the NEWS, six-item MEWS, and qSOFA. Future work includes extending the EWSSO by introducing additional early warning scores for adult and pediatric patient populations and creating patient profiles that contain clinical, demographic, and outcomes data regarding the patient.

Introduction

Early detection and response to patient deterioration are critical to avoid adverse events, preventable morbidity, cardiac arrest, unexpected ICU admission, and mortality [1]. Patients often display indicators of clinical deterioration up to 24 h before a significant clinical event arises needing extensive intervention [2]. Failure to manage these deteriorations appropriately leads to negative patient outcomes [3]. However, clinical deterioration may be masked or not obvious to the treatment team [4, 5]. Thus, the patient may deteriorate to the point of instability or even death before this deterioration is noticed and acted upon. Early Warning Scores (EWSs), also known as track and trigger systems, can assist clinicians in recognizing and responding to patient deterioration at an earlier stage of progression and in many cases improve patient care outcomes [6,7,8,9,10].

Since their introduction in the mid-1990s [11], different EWSs have been developed and implemented within different types of electronic health record (EHR) systems and within diverse healthcare settings. Twenty-eight different early warning system scores have been developed and validated in adult patients [12]. Twenty-seven unique pediatric track and trigger systems have been validated in hospitalized children [13]. The proliferation of EWSs within EHR systems across national and international health care institutions has provided a wealth of patient-level physiological decompensation clinical data that can also be used to advance secondary research, such as quality/safety protocols or population health initiatives. One roadblock to secondary analysis of EWS data is the variability of EWSs across multiple providers and multiple EHR systems. This hinders querying across providers and EHR systems, since EWS types use different parameters mapping and merging the parameters is also not always easily possible. We propose to use ontologies to overcome these problems.

In this paper we describe the development and validation of the early warning system scores ontology (EWSSO), representing three commonly used EWSs: the National Early Warning Score (NEWS), the six-item modified Early Warning Score (MEWS), and the quick Sequential Organ Failure Assessment (qSOFA) [14]. In addition, we demonstrate how the EWSSO can be used to generate and query health system data on vital signs and calculate the NEWS, six-item MEWS, and qSOFA using synthetic data. Our goal is to establish a platform that can eventually foster semantic integration of data captured using different EWSs and generate EWS data for the secondary use of clinical data from EHR systems. The ability to semantically integrate EWS data across multiple healthcare settings with the use of ontologies will advance our goal of using real-word data from EHR systems to improve individual-level and population-level health.

Background

Early Warning Scores, if implemented within the clinical setting, can become an integral part of the healthcare delivery process by assessing the early signs of clinical deterioration of a patient [12, 15, 16]. When EWSs are used as a track and trigger system a patient’s EWSs can either alert the healthcare team to intervene on behalf of the patient or continue to monitor the patient’s condition [12, 17, 18]. Although the validity of EWSs to predict adverse outcomes has been established across different healthcare settings and among diverse patient populations [19,20,21], these data are not being maximized by healthcare researchers and informaticists to advance secondary research of clinical data and outcomes stored within EHR systems across multiple healthcare settings.

Researchers of individual institutions may be able to extract EWS data, clean it, and append it with patient demographic and clinical characteristics using traditional data cleaning and merging techniques; but integrating data in this format, which may exist as either structured data or free text, can be inefficient and time consuming [22]. This traditional approach to manually transform and append disparate data sources to answer a single research question can be inefficient, because the procedure is not easily reproduced by other researchers, and it does not transform heterogeneous data sources into actionable clinical information. For instance, for a researcher to have confidence in the use of a colleague's transformed dataset to extract clinical information that may be useful to answer a different scientific question, agreement with the meaning of each term used to define a particular feature must exist. Oftentimes, however, the labels or terms used by individual researchers to provide meaning to features within a dataset may differ in subtle but important ways from other researchers’ ideas. Without a standardized data dictionary that is openly accessible and includes terms agreed upon by all members of the research team and is updated overtime, use of such a data source will decrease or worse it continues to be used, but the clinical information derived is questionable. This is where ontologies can help.

Ontologies are formalized representations of the entities in a domain of discourse that are machine interpretable [23]. They are frequently used to achieve semantic integration of biomedical data [24,25,26,27,28]. The lack of semantic integration between different EWSs implemented within different EHR systems across disparate healthcare institutions is currently hampering our ability to query EWS data to identify factors relevant to different EWSs. Our goal is to achieve semantic integration of EWS data to allow easier appending of heterogeneous patient-level and environmental-level data sets to investigate relationships that extend beyond the clinical setting. For instance, examining the correlation between aspects of social determinants, a patient’s EWSs, and negative health outcomes, such as unplanned ICU readmissions, morbidity, and mortality.

Our ontology aims to represent both the structure and planned process of the National Early Warning Score [29], the six-item modified Early Warning Score [30, 31], and the quick Sequential Organ Failure Assessment [32]. The National Early Warning Score system uses inputs from seven vital function measurement datum: respiratory rate; systolic blood pressure; heart rate; Alert, Voice, Pain, Unresponsiveness (AVPU); body temperature; oxygen saturation; and supplemental oxygen use [29]. There are several Modified Early Warning Score systems that are differentiated by the number of physiological parameters used to derive the final score. For the purposes of our study, we will focus on the six-item MEWS, which uses inputs from six vital function measurement datum, respiratory rate, systolic blood pressure, urine output, heart rate, AVPU, and body temperature [31]. The final representation is of the qSOFA, which uses inputs from three vital function measurement datum, the Glasgow coma scale, respiratory rate, and systolic blood pressure [32].

Materials and methods

Ontology development

We followed the Software Development Life Cycle (SDLC) framework depicted in Fig. 1 to model the tasks involved in planning, creating, and testing the Early Warning Systems Score Ontology [33]. While EWSSO is available from a public repository, we have not fully gone live with our ontology. Thus, we have not finalized the deployment, operations, and maintenance phases of the SDLC.

Fig. 1
figure 1

Representation of the seven phases of the SDLC framework

Full size image

SDLC phase 1 and phase 2: planning and requirements

During the planning phase of the SDLC relevant information and methods to be applied were discussed to assess the feasibility of developing and validating the EWSSO. During the design phase the team agreed to apply methods described by Arp, Smith & Spear for “Building Ontologies with Basic Formal Ontology” [34] particularly, the principles of best practice for ontology design, terms, definitions, and classifications to meet BFO requirements for ontology building. Additionally, we followed Arp, Smith, and Spear’s guidance on using Aristotelian definitions—i.e. genus-differentia definitions—which define an A as a B with the properties C, where A is the class of things being defined (e.g., national early warning score) and B is the immediate superclass (e.g., early warning scores) and C are a set of properties that make the members of A different from the members of B that are not members of A.

As an example, two Aristotelian definitions relevant to our domain are described here.

  1. 1.

    early warning score = def. an information content entity that is about clinical deterioration.

  2. 2.

    national early warning score = def. an early warning score that is the specified output of the national early warning score summation process that has specified input respiratory rate, systolic blood pressure, heart rate, AVPU, body temperature, oxygen saturation, and supplemental oxygen use measurement data.

In the first definition, we specify A [early warning score] as a B [an information content entity] that Cs [is about clinical deterioration]. In the second definition, we specify A [national early warning score] as a B [early warning score] that Cs [is the specified output of the national early warning score summation process that has specified input respiratory rate, systolic blood pressure, heart rate, AVPU, body temperature, oxygen saturation, and supplemental oxygen use measurement data]. If we were to replace the term early warning score in the second sentence with the definition of early warning score in the first sentence the unpacked definition for national early warning score would be an information content entity that is about clinical deterioration and is the specified output of the national early warning score summation process. The result is that we have the same meaning of national early warning score as the sentence with which we began. In keeping in line with creating high-quality ontological definitions we use singular noun terms and avoid acronyms. Aristotelian definitions are helpful for ontology development and maintenance since they bind the definitions to the taxonomy underlying our ontology.

SDLC phase 3 and phase 4: designing and building

OBO foundry principles

The OBO Foundry Principles [35,36,37] guidelines are meant to serve as the norm for OBO Foundry ontologies and are used in assessing the ontologies that are submitted to the OBO Foundry for review. Because we have plans to submit the EWSSO for the Foundry's assessment, our ontology development process follows these principles. An assessment of how our ontology in the early development phase meets certain OBO Foundry principles is outlined in the results section of this paper. The OBO Foundry aims to be an orthogonal library of interactive ontologies [37]. To prevent conflicting representation and enable optimal semantic integration among resources using OBO Foundry ontologies, a key strategy in developing an ontology for submission to the OBO Foundry is identifying potential areas of overlap with existing OBO Foundry ontologies and prevent conflicting representations [37]. The OBO Foundry principles prescribe the reuse of relations [35], e.g., object properties but considering the reuse of classes is also a common and useful strategy [38]. To address the issue of effective reuse of individual entities or sets of individual entities, OBO Foundry contributors have developed methods of importing classes, object properties, and other entities from existing ontologies [38], called MIREOT. There are multiple implementations of MIREOT. We use the MIREOT Protégé plugin developed by Hanna et al. [39] Reusing entities from preexisting ontologies using MIREOT copies over annotation values from the source ontology. Such annotation values sometimes provide cross-references to other ontologies. E.g., the EWSSO class respiratory rate measurement datum point to the class respiratory rate [40], which in turn points to several similar terms in medical terminologies. These cross-references are coming from an outside source and not subject to our development. Our aim is to follow the strategy to prevent duplicate representation of entities in EWSSO and OBO Foundry ontologies.

Reused classes

The 4 reused classes and definitions from OBO Foundry ontologies are listed in Table 1.

Table 1 OBO foundry ontologies, reused classes, and definitions
Full size table

Reused object properties

We reused four object properties from OBO Foundry ontologies, listed in Table 2.

Table 2 OBO Foundry Ontologies, reused object properties, and definitions
Full size table

Every early warning score is an information content entity that is about clinical deterioration. Every early warning score is specified by property is about. The Information Artifact Ontology (IAO) defines object property is about, as “a (currently) primitive relation that relates an information artifact to an entity.” In this case, the early warning score is the information artifact that is about some entity, clinical deterioration, that is “the process of the diminishing realization of the patient's vital functions.” The Ontology for Biomedical Investigations (OBI) defines object property is specified output of as “a relation between a planned process and a continuant participating in that process. The presence of the continuant at the end of the process is explicitly specified in the objective specification which the process realizes the concretization of.” In this case, the national early warning score is the continuant at the end of the national early warning score summation process that is “…explicitly specified in the objective specification which the process realizes the concretization of.”

According to BFO, reality is made up of two types of phenomena: (1) continuants, which are things like objects, qualities, and functions that last over time; and (2) occurrents, which are the activities that continuants engage in [50,51,52]. Continuants may be independent or dependent [52]. Entities we encounter daily (e.g., person, ball, pencil) are primary examples of independent continuants that also act as carriers of dependent continuants (e.g., a smile, the color of a ball, the ability to write) [52]. We mention these BFO fundamental concepts of reality because we classify early warning system scores, such as the NEWS, six-item MEWS, and qSOFA as generically dependent continuants, more specifically an information content entity, that are the specified output of a planned process (occurrent) that uses attribute specific measurement inputs to calculate a patient’s early warning score, which warns about clinical deterioration.

We created our ontology in OWL2 using the Protégé ontology editor [53] and provided visualization of our ontology using the online MIRO software [54]

SDLC phase 5: testing

To determine the usefulness of our ontology to generate and query heterogeneous vital sign data for early warning score calculation and secondary data analysis we performed the following steps.

Step 1: create synthetic datasets

The purpose for creating synthetic datasets was to simulate heterogeneous data sources of critical care patients whose degree of physiological decompensation is typically evaluated with an early warning system score, such as the NEWS, MEWS, and / or qSOFA. We created two different synthetic datasets. The first included vital sign measurement features specific to each of our early warning scores. The second dataset included other clinical and demographic features, unnecessary for early warning score calculator. This was done to test whether our process with ontological foundation could differentiate between features necessary and unnecessary for score calculation. Temperature measurement data were added as either degree Fahrenheit or degree Celsius. This was done so that the system would then calculate the early warning score with the appropriate temperature measurement scale. Additionally, a random number of values were left blank to represent missingness, common to all datasets. Our process is not meant to impute missing data. However, if a feature necessary to calculate an early warning score is missing the system will return the output as unavailable. Each dataset included 1,000 patient rows.

To create our synthetic datasets of vital measurements used in various early warning scores, we first needed a method to generate a normal distribution of values with a set median, interquartile range (IQR), and extreme range. Using a JavaScript example as a base [55] a Python script was created to generate N samples with configurable parameters (median, IQR, and range) for each vital measurement needed. For the dataset to prove useful in a comparison of early warning scores, we needed to base the vitals on a set of data pertaining to emergency / critical care. We used a Danish study by Nissen et. al. [56] of the performance of NEWS as the baseline parameters to determine most of our measurements since the Danish Multicenter Cohort focused on emergency department admissions. These patients would have distribution and ranges of EWS variables that are more indicative of the types of patients who would be subject to these types of early warning scores for deterioration assessment. The listed median and IQR for each measurement were used in conjunction with a set of extreme ranges that ensured the full range of NEWS scoring would be represented [56]. Any missing measurements were filled from similar studies regarding critical patients in European cohorts [16]. To better align the results of different early warning scores using the distinct but related measurements for consciousness, AVPU vs GCS, we generated the score for AVPU and then converted to GCS per Romanelli and Farrell [57].

Step 2: convert synthetic dataset into triples

The synthetic datasets were converted into triples in an RDF database informed by our early warning system scores ontology. We transformed the entirety of each dataset to triples.

Step 3: visualization of RDF database and SPARQL queries

Once the heterogeneous synthetic datasets were converted into triples representing each patient and their respective vital sign measurements, the results were loaded into GraphDB. We then ran a SPARQL query to GraphDB that returned all measurement data for a specific patient (patient identifier). A visualization of the triples in the RDF database, RDF graph, SPARQL query, and the answer for the SPARQL query used to extract a person’s measurement data can be seen in Fig. 2.

Fig. 2
figure 2

Representation of RDF database

Full size image

Step 4: calculate the early scores for each person

To calculate the early warning scores for each person, three projects were created in Python. The intention was to define the required calculations in separate modules that would be imported and utilized by the base project. One project defined each early warning score’s requirements and the formula for the score calculation. We used the guide charts for NEWS, six-item MEWS, and qSOFA scoring system methodology provided by Aseptika Limited [58], the Institute for Healthcare Improvement [59], and Omni Calculator [60], respectively. Another project contained formulas for unit conversions (e.g., converting temperature to degrees Celsius when a formula requires Celsius, but the provided data is Fahrenheit). The general flow of the application was that the base Python project would query the data from the triplestore using SPARQL, representing a person with their semantically tagged measurements. The EWS-specific module would be dynamically imported by the base project and used to calculate each early warning score that meets the minimum data requirements for the score. The calculation is possible because the data representing the measurements is semantically tagged, and the formulas for each score use those tags to determine the correct parameters. In turn, the EWS module would import the unit conversion module to perform any necessary unit conversions on the data. This approach allows new early warning scores or unit conversion formulas to be added to their respective projects without interfering with the base project or other projects created to take advantage of the centralized specification.

Step 5: validate early warning system score output

Researcher CEZ randomly selected ten patients and manually calculated the early warning system scores, based on the heterogeneous datasets, and compared to system output.

Results

Ontology development

OBO foundry principles

The list below indicates how the early-stage development of the EWSSO fulfills several of the OBO Foundry Principles.

  • P1) Open. Our ontology is openly available to all users under the Creative Commons CC-BY 3.0 license. This means it may be shared and adopted if appropriately credited and changes, if made, are indicated.

  • P2) Common Format. Our ontology is made available in the OWL Web Ontology Language and the Resource Description Framework (RDF).

  • P5) Scope. The domain of the ontology is early warning system scores for clinical deterioration in inpatient populations. We start with the simplest and most easily defined features of our relevant domain and will work our way outward to more complex features in future updates of our ontology.

  • P6) Textual Definitions. All newly created classes have a textual definition (see Table 3).

  • P7) Relations. Relations reused from the Relations Ontology (RO), Basic Formal Ontology (BFO), Ontology for Biomedical Investigations (OBI), and Information Artifact Ontology (IAO).

Table 3 New classes and definitions
Full size table

Our long-term goal is to submit the EWSSO to the OBO Foundry. Hence, the GitHub repository is created following OBO Foundry practices and rules. The repository is modeled to be similar to GitHub repositories, such as OBI [https://github.com/obi-ontology/obi] or OBIB [https://github.com/biobanking/biobanking] among others.

EWSSO is freely available on the GitHub https://github.com/zcilia/ews-ontology.

The data for our experiment can be found here: https://github.com/EWSSO/documentation

Newly created classes

Table 3 lists the 29 new classes created and their respective definitions. We also include references reviewed that inspired our new definitions, or the Internationalized Resource Identifier (IRI) of the OBO Foundry ontology classes that we based our new definitions on.

Representation of early warning system score ontology

The images of the early warning system score ontology in Figs. 3, 4, and 5 represent the national early warning score, six-item MEWS and qSOFA.

Fig. 3
figure 3

Early warning system score ontology of the national early warning score

Full size image
Fig. 4
figure 4

Early warning system score ontology of the six-item modified early warning score

Full size image
Fig. 5
figure 5

Early warning system score ontology of the quick sequential organ failure assessment

Full size image

Ontology performance evaluation

Our evaluation of EWSSO was to generate and query health systems data and correctly retrieve the data elements required to calculate the NEWS, six-item MEWS, and qSOFA and no other data elements. To test performance, we created two synthetic datasets with 1,000 patient rows each. The first synthetic dataset included vital sign measurement features specific to each of our early warning scores. The second synthetic dataset included other clinical and demographic features, unnecessary for early warning score calculator. Each time we queried the data, our ontology-based process could differentiate between features necessary and unnecessary for score calculation.

Discussion

One of the core terms for early warning scores is clinical deterioration. We checked multiple ontology lookup resources, e.g., OntoBee [68] and the NCBO Bioportal [69] for a term to re-use. The only term we could find was from Medical Subject Headings (MeSH) resource and defines clinical deterioration as “A critical disease progression, often measured by a set of clinical parameters, which activates HOSPITAL RAPID RESPONSE TEAM” [70]. This definition is not optimal, since it defines clinical deterioration by the activation of a hospital rapid response team. The authors hold that clinical deterioration may exist without a hospital rapid response team present and activated. A recent concept analysis of clinical deterioration did not result in a definition referring to the activation of a hospital rapid response team or any other healthcare procedure [61]. We propose to define clinical deterioration as the process of the diminishing realization of the patient’s vital functions.

While the current project is of limited scope, as only NEWS, MEWS, and qSOFA are considered it serves as the basis for future work with all early warning scoring systems. As the project is consolidated, we are going to seek to integrate additional early warning scores. Although the EWSS ontology does not perform mathematical calculations, it is expected to help users query relevant ontological relationships and measurement data. Using Microsoft Excel, SAS, or other computer programming languages researchers can use the queried data to calculate the early warning scores for secondary analyses of clinical data.

It is our plan to create a pipeline that allows clinical service providers and clinical researchers to perform data acquisition and EWS calculation without the need to run SPARQL queries or write Python commands. During the development of this pipeline, we will recruit a small number of independent users to test it and provide feedback on its usability. During these trials EWSSO will begin to fulfill the independent user requirement of the OBO Foundry. Once EWSSO fulfills all criteria for applying for an OBO Foundry namespace we will apply for a namespace—preferably “EWSSO”—assigned by the OBO Foundry.

Conclusion

The objective of this paper was to describe the development and validation of the early warning system scores ontology using the principles of best practice for domain ontology design, terms, definitions, and classification as described by Arp, Smith & Spear Building Ontologies with Basic Formal Ontology [14] and aiming to follow OBO Foundry principles [35,36,37]. We have developed the EWSS Ontology covering three commonly used early warning scores in clinical practice: NEWS, six-item MEWS, and qSOFA. We demonstrated that EWSSO provides the functionality to facilitate search and extraction of data necessary to calculate the scores.

Further work includes extending the EWSS Ontology by introducing additional early warning scores for adult and pediatric patient populations and creating patient profiles which contain clinical, demographic, and outcomes data regarding the patient. The early warning system scores ontology can be implemented and personalized for a particular healthcare system by adding classes for organizational characteristics including clinical staff, non-clinical staff, and management staff. Users of the ontology can develop queries based on the semantically integrated data created, thereby reducing the inefficiency involved with compiling voluminous datasets of heterogeneous EHR data for research or quality improvement initiatives.

Availability of data and materials

EWSSO is freely available on the GitHub https://github.com/zcilia/ews-ontology.

The data for our experiment can be found here:https://github.com/EWSSO/documentation.

• Ontosheep (application full instructions):https://github.com/jmwhorton/ontosheep/tree/main.

• Ontosheep (container instructions):https://github.com/jmwhorton/ontosheep/tree/main/test-deploy.

• Ontosheep modules (These are the two 'helper' modules for Ontosheep. Modules in separate projects for reusabiity.):https://github.com/jmwhorton/ontosheep-ews and https://github.com/jmwhorton/ontosheep-conversion.

Abbreviations

BFO:

Basic Formal Ontology

EWSSO:

Early Warning Systems Score Ontology

IAO:

Information Artifact Ontology

MEWS:

Modified Early Warning Score

NEWS:

National Early Warning Score

OBI:

Ontology for Biomedical Investigations

qSOFA:

Quick Sepsis Related Organ Failure Assessment

References

  1. Maharaj R, Raffaele I, Wendon J. Rapid response systems: a systematic review and meta-analysis. Crit Care. 2015;19(1):254. https://doi.org/10.1186/s13054-015-0973-y.

    Article  Google Scholar 

  2. McGaughey J, Alderdice F, Fowler R, Kapila A, Mayhew A, Moutray M. “Outreach and Early Warning Systems (EWS) for the prevention of intensive care admission and death of critically ill adult patients on general hospital wards,” Cochrane Database Syst Rev 2007(3);CD005529. https://doi.org/10.1002/14651858.CD005529.pub2.

  3. Sun L, Joshi M, Khan SN, Ashrafian H, Darzi A. Clinical impact of multi-parameter continuous non-invasive monitoring in hospital wards: a systematic review and meta-analysis. J R Soc Med. 2020;113(6):217–24. https://doi.org/10.1177/0141076820925436.

    Article  Google Scholar 

  4. Smith AF, Wood J. Can some in-hospital cardio-respiratory arrests be prevented? A prospective survey. Resuscitation. 1998;37(3):133–7. https://doi.org/10.1016/S0300-9572(98)00056-2.

    Article  Google Scholar 

  5. Buist MD, Burton PR, Bernard SA, Waxman BP, Anderson J. Recognising clinical instability in hospital patients before cardiac arrest or unplanned admission to intensive care: a pilot study in a tertiary-care hospital. Med J Aust. 1999;171(1):22–5. https://doi.org/10.5694/j.1326-5377.1999.tb123492.x.

    Article  Google Scholar 

  6. Credland N, Dyson J, Johnson MJ. Do early warning track and trigger tools improve patient outcomes? A systematic synthesis without meta-analysis. J Adv Nurs. 2021;77(2):622–34. https://doi.org/10.1111/jan.14619.

    Article  Google Scholar 

  7. Penney SW, et al. A quality improvement initiative to reduce unnecessary rapid responses using early warning scores. Pediatrics. 2021;147(3):e20191947. https://doi.org/10.1542/peds.2019-1947.

    Article  Google Scholar 

  8. Wu C-L, et al. Implementation of an electronic national early warning system to decrease clinical deterioration in hospitalized patients at a tertiary medical center. Int J Environ Res Public Health. 2021;18(9):4550. https://doi.org/10.3390/ijerph18094550.

    Article  Google Scholar 

  9. Mathukia C, Fan W, Vadyak K, Biege C, Krishnamurthy M. Modified early warning system improves patient safety and clinical outcomes in an academic community hospital. J Commun Hosp Internal Med Perspect. 2015;5:26716. https://doi.org/10.3402/jchimp.v5.26716.

    Article  Google Scholar 

  10. Nishijima I, et al. Use of a modified early warning score system to reduce the rate of in-hospital cardiac arrest. J Intensive Care. 2016;4:12. https://doi.org/10.1186/s40560-016-0134-7.

    Article  Google Scholar 

  11. Lee A, Bishop G, Hillman KM, Daffurn K. The medical emergency team. Anaesth Intensive Care. 1995;23(2):183–6. https://doi.org/10.1177/0310057X9502300210.

    Article  Google Scholar 

  12. Wuytack F, et al. The effectiveness of physiologically based early warning or track and trigger systems after triage in adult patients presenting to emergency departments: a systematic review. BMC Emerg Med. 2017;17:38. https://doi.org/10.1186/s12873-017-0148-z.

    Article  Google Scholar 

  13. Trubey R, et al. Validity and effectiveness of paediatric early warning systems and track and trigger tools for identifying and reducing clinical deterioration in hospitalised children: a systematic review. BMJ Open. 2019;9(5):e022105.https://doi.org/10.1136/bmjopen-2018-022105.

    Article  Google Scholar 

  14. Liu VX, et al. Comparison of early warning scoring systems for hospitalized patients with and without infection at risk for in-hospital mortality and transfer to the intensive care unit. JAMA Netw Open. 2020;3(5):e205191. https://doi.org/10.1001/jamanetworkopen.2020.5191.

    Article  Google Scholar 

  15. Burgos-Esteban A, et al. Effectiveness of early warning scores for early severity assessment in outpatient emergency care: a systematic review. Front Public Health. 2022;10:894906. https://doi.org/10.3389/fpubh.2022.894906.

    Article  Google Scholar 

  16. Cooksley T, Kitlowski E, Haji-Michael P. Effectiveness of modified early warning score in predicting outcomes in oncology patients. QJM. 2012;105(11):1083–8. https://doi.org/10.1093/qjmed/hcs138.

    Article  Google Scholar 

  17. M. E. B. Smith et al., Early Warning System Scores: A Systematic Review. in VA Evidence-based Synthesis Program Reports. Washington (DC): Department of Veterans Affairs, 2014. Available: http://www.ncbi.nlm.nih.gov/books/NBK259026/.Accessed 11 Apr 2022

  18. F. Creed, J. Dawson, and K. Looker, “Assessment tools and track-and-trigger systems,” in Care of the Acutely Ill Adult: An essential guide for nurses, F. Creed and C. Spiers, Eds., Oxford University Press, 2010 0. https://doi.org/10.1093/med/9780199564385.003.11.

  19. Fang AHS, Lim WT, Balakrishnan T. Early warning score validation methodologies and performance metrics: a systematic review. BMC Med Inform Decis Mak. 2020;20:111. https://doi.org/10.1186/s12911-020-01144-8.

    Article  Google Scholar 

  20. Gerry S, Birks J, Bonnici T, Watkinson PJ, Kirtley S, Collins GS. Early warning scores for detecting deterioration in adult hospital patients: a systematic review protocol. BMJ Open. 2017;7(12):e019268. https://doi.org/10.1136/bmjopen-2017-019268.

    Article  Google Scholar 

  21. Alhmoud B, Bonnici T, Patel R, Melley D, Williams B, Banerjee A. Performance of universal early warning scores in different patient subgroups and clinical settings: a systematic review. BMJ Open. 2021;11(4):e045849. https://doi.org/10.1136/bmjopen-2020-045849.

    Article  Google Scholar 

  22. Ridzuan F, Zainon WM. A review on data cleansing methods for big data. Proc Comp Sci. 2019;161:731–8. https://doi.org/10.1016/j.procs.2019.11.177.

    Article  Google Scholar 

  23. Gruber TR. Toward principles for the design of ontologies used for knowledge sharing? Int J Hum Comput Stud. 1995;43(5):907–28. https://doi.org/10.1006/ijhc.1995.1081.

    Article  Google Scholar 

  24. Livingston KM, Bada M, Baumgartner WA, Hunter LE. KaBOB: ontology-based semantic integration of biomedical databases. BMC Bioinform. 2015;16(1):126. https://doi.org/10.1186/s12859-015-0559-3.

    Article  Google Scholar 

  25. Bona J, et al. Semantic integration of multi-modal data and derived neuroimaging results using the Platform for Imaging in Precision Medicine (PRISM) in the Arkansas Imaging Enterprise System (ARIES). Front Artif Intell. 2021;4:649970. https://doi.org/10.3389/frai.2021.649970.

    Article  Google Scholar 

  26. Bona JP, Prior FW, Zozus MN, Brochhausen M. Enhancing clinical data and clinical research data with biomedical ontologies - insights from the knowledge representation perspective. Yearb Med Inform. 2019;28(1):140–51. https://doi.org/10.1055/s-0039-1677912.

    Article  Google Scholar 

  27. Zaman S, Sarntivijai S, Abernethy DR. Use of biomedical ontologies for integration of biological knowledge for learning and prediction of adverse drug reactions. Gene Regul Syst Bio. 2017;11:1177625017696075. https://doi.org/10.1177/1177625017696075.

    Article  Google Scholar 

  28. O. Bodenreider, “Biomedical Ontologies in Action: Role in Knowledge Management, Data Integration and Decision Support,” Yearb Med Inform 2008;67–79. Available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2592252/. Accessed 15 Jul 2023

  29. Royal College of Physicians of London. National Early Warning Score (NEWS): standardising the assessment of acute-illness severity in the NHS. London: Royal College of Physicians; 2012.

    Google Scholar 

  30. Morgan R, Williams F, Wright M. An early warning scoring system for detecting developing critical illness. Clin Intensive Care. 1997;8(2):100.

    Google Scholar 

  31. Gardner-Thorpe J, Love N, Wrightson J, Walsh S, Keeling N. The value of Modified Early Warning Score (MEWS) in surgical in-patients: a prospective observational study. Ann R Coll Surg Engl. 2006;88(6):571–5. https://doi.org/10.1308/003588406X130615.

    Article  Google Scholar 

  32. Wagner T, et al. qSOFA score is useful to assess disease severity in patients with heart failure in the setting of a Heart Failure Unit (HFU). Front Cardiovasc Med. 2020;7:574768. https://doi.org/10.3389/fcvm.2020.574768.

    Article  Google Scholar 

  33. Laveena Ramchandani, “The World of Software Development Life Cycle,” TestProject by Tricentis. https://blog.testproject.io/2021/03/08/the-world-of-software-development-life-cycle/ (Accessed 16 Mar 2023).

  34. Arp R, Smith B, Spear AD. Building Ontologies With Basic Formal Ontology. The MIT Press. 2015. https://doi.org/10.7551/mitpress/9780262527811.001.0001.

    Article  Google Scholar 

  35. “OBO Foundry: Priciples Overview.” https://obofoundry.org/principles/fp-000-summary.html(Accessed 16 Feb 2023).

  36. Jackson R, et al. OBO foundry in 2021: operationalizing open data principles to evaluate ontologies. Database (Oxford). 2021;2021:069. https://doi.org/10.1093/database/baab069.

    Article  Google Scholar 

  37. Smith B, et al. The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol. 2007;25(11):1251–5. https://doi.org/10.1038/nbt1346.

    Article  Google Scholar 

  38. Courtot M, et al. “MIREOT: the minimum information to reference an external ontology term,” Nat Prec 2009;1–1. https://doi.org/10.1038/npre.2009.3576.1.

  39. Hanna J, Chen C, Crow WA, Hall R, Liu J, Pendurthi T, Schmidt T, Jennings SF, Brochhausen M, Hogan W., “Simplifying MIREOT: A MIREOT Protégé Plugin. Presented at the International Semantic Web Conference, Boston, Massachusetts. NIHMSID: 446691.,”

  40. “respiratory rate.” https://www.ontobee.org/ontology/HTN?iri=http://purl.obolibrary.org/obo/VSO_0000034 (Accessed 10 May 2022).

  41. “process.” http://purl.obolibrary.org/obo/BFO_0000015 (Accessed 16 Jul 2023).

  42. “planned process.” http://purl.obolibrary.org/obo/OBI_0000011 (Accessed 16 Jul 2023).

  43. “information content entity.” http://purl.obolibrary.org/obo/IAO_0000027 (Accessed 16 Jul 2023).

  44. “measurement datum.” http://purl.obolibrary.org/obo/IAO_0000109 (Accessed 16 Jul 2023).

  45. “‘is_about.’” http://purl.obolibrary.org/obo/IAO_0000136 (Accessed 10 May 2022).

  46. “‘is_specified_output_of.’” http://purl.obolibrary.org/obo/OBI_0000312 (Accessed 10 May 2022).

  47. “‘has_specified_input.’” http://purl.obolibrary.org/obo/OBI_0000293 (Accessed 10 May 2022).

  48. “is_specified_input_of.” http://purl.obolibrary.org/obo/OBI_0000295 (Accessed 16 Jul 2023).

  49. “‘has_specified_output.’” http://purl.obolibrary.org/obo/OBI_0000299 (Accessed 10 May 2022).

  50. Grenon P, Smith B. SNAP and SPAN: towards dynamic spatial ontology. Spat Cogn Comput. 2004;4(1):69–104. https://doi.org/10.1207/s15427633scc0401_5.

    Article  Google Scholar 

  51. Smith B, Grenon P. The Cornucopia of formal-ontological relations. Dialectica. 2004;58(3):279–96. https://doi.org/10.1111/j.1746-8361.2004.tb00305.x.

    Article  Google Scholar 

  52. Arp R, Smith B. “Realizable entities in basic formal ontology” 2008.

  53. Musen MA. The protégé project: a look back and a look forward. AI Matters. 2015;1(4):4–12. https://doi.org/10.1145/2757001.2757003.

    Article  Google Scholar 

  54. “The Visual Collaboration Platform for Every Team | Miro,” https://miro.com/. https://miro.com/ (Accessed 10 May 2022).

  55. Dinesh, “Answer to ‘How to create a list of numbers with given range, quartiles and median?,’” Stack Overflow. https://stackoverflow.com/a/26478053 (Accessed 17 Mar 2023).

  56. Nissen SK, et al. The impact of age on predictive performance of national early warning score at arrival to emergency departments: development and external validation. Ann Emerg Med. 2022;79(4):354–63. https://doi.org/10.1016/j.annemergmed.2021.09.434.

    Article  Google Scholar 

  57. D. Romanelli and M. W. Farrell, “AVPU Score,” in StatPearls, Treasure Island (FL): StatPearls Publishing, 2022. Available: http://www.ncbi.nlm.nih.gov/books/NBK538431/. Accessed 16 Feb 2023.

  58. “National Early Warning Score (NEWS).” https://www.activ8rlives.com/support/data-collected/cardiovascular-and-respiratory/national-early-warning-score-news (Accessed 16 Feb 2023).

  59. “Early Warning Systems: Scorecards That Save Lives | IHI - Institute for Healthcare Improvement.” https://www.ihi.org:443/resources/Pages/ImprovementStories/EarlyWarningSystemsScorecardsThatSaveLives.aspx (Accessed 16 Feb 2023).

  60. Pawlik, K, “qSOFA Score Calculator | Formula | Definition.” https://www.omnicalculator.com/health/q-sofa (Accessed 16 Feb 2023).

  61. Padilla RM, Mayo AM. Clinical deterioration: a concept analysis. J Clin Nurs. 2018;27(7–8):1360–8. https://doi.org/10.1111/jocn.14238.

    Article  Google Scholar 

  62. “Alert verbal painful unresponsive scale clinical classification.” http://purl.obolibrary.org/obo/NCIT_C161656 (Accessed 10 May 2022).

  63. “Glasgow coma scale-NINDS version clinical classification.” http://purl.obolibrary.org/obo/NCIT_C124710 (Accessed 10 May 2022).

  64. “Oxygen Saturation Measurement.” http://purl.obolibrary.org/obo/NCIT_C60832 (Accessed 10 May 2022).

  65. “body temperature.” https://www.ontobee.org/ontology/HTN?iri=http://purl.obolibrary.org/obo/VSO_0000007 (Accessed 10 May 2022).

  66. T. D. Homan, S. Bordes, and E. Cichowski, “Physiology, Pulse Pressure,” in StatPearls, Treasure Island (FL): StatPearls Publishing, 2022. Available: http://www.ncbi.nlm.nih.gov/books/NBK482408/. Accessed 10 May 2022

  67. “Heart Rate.” http://purl.obolibrary.org/obo/NCIT_C49677 (Accessed 10 May 2022).

  68. Z. Xiang, C. Mungall, A. Ruttenberg, and Y. He, “Ontobee: A Linked Data Server and Browser for Ontology Terms,” presented at the International Conference on Biomedical Ontology, 2011. Available: https://www.semanticscholar.org/paper/Ontobee%3A-A-Linked-Data-Server-and-Browser-for-Terms-Xiang-Mungall/4b32148dc40590475444881ae92b379ba71f857b. Accessed 20 Apr 2023.

  69. Noy NF, et al. BioPortal: ontologies and integrated data resources at the click of a mouse. Nucleic Acids Res. 2009;37(Web Server issue):170–3. https://doi.org/10.1093/nar/gkp440.

    Article  Google Scholar 

  70. “Medical Subject Headings - Clinical Deterioration - Classes | NCBO BioPortal.” https://bioportal.bioontology.org/ontologies/MESH?p=classes&conceptid=http%3A%2F%2Fpurl.bioontology.org%2Fontology%2FMESH%2FD000075902 (Accessed 26 Apr 2023).

Download references

Acknowledgements

Not applicable.

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Biomedical Informatics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA

    Cilia E. Zayas, Justin M. Whorton, Kevin W. Sexton, S. Clint Dowland & Mathias Brochhausen

  2. Department of Surgery, College of Medicine, University of Arkansas for Medical Sciences, Little Rock, AR, USA

    Kevin W. Sexton & Charles D. Mabry

  3. University of Arkansas for Medical Sciences, Institute for Digital Health & Innovation, 4301 West Markham Street, Slot 781, Little Rock, AR, 72205, USA

    Kevin W. Sexton

  4. Department of Medical Humanities and Bioethics, University of Arkansas for Medical Sciences, Little Rock, Arkansas, USA

    Mathias Brochhausen

Authors
  1. Cilia E. Zayas
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Justin M. Whorton
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kevin W. Sexton
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Charles D. Mabry
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. S. Clint Dowland
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mathias Brochhausen
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

CEZ: participated in the development and evaluation of the EWSSO, created the graphical representations depicted, and led the effort of writing the manuscript. JW: developed the mechanisms used to determine the usefulness of our ontology to generate and query heterogeneous vital sign data for early warning score calculator. Mechanisms included creating synthetic dataset, transforming the entirety of the datasets to triples, visualization of RDF database and SPARQL queries, and developing Python scripts for early warning score calculator. JW also created the EWSSO Documentation GitHub repository and participated in writing the materials and methods sections. KS: provided background knowledge on the clinical relevance and implementation of EWSs, provided input on the study design and use case. CM: provided background knowledge on the clinical relevance and implementation of EWSs, provided input on the study design and use case. SCD: provided feedback on ontology development and participated in refining the manuscript, and the diagrams within it. MB: participated in the development of the EWSSO and participated in writing the manuscript. All authors contributed to manuscript discussion and preparation. The authors read and approved the final manuscript.

Corresponding author

Correspondence to Cilia E. Zayas.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zayas, C.E., Whorton, J.M., Sexton, K.W. et al. Development and validation of the early warning system scores ontology. J Biomed Semant 14, 14 (2023). https://doi.org/10.1186/s13326-023-00296-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13326-023-00296-6

Keywords

Journal of Biomedical Semantics

ISSN: 2041-1480