Skip to main content
Download PDF
Download ePub
  • Research Article
  • Open access
  • Published:

Annotation-based feature extraction from sets of SBML models

Journal of Biomedical Semantics volume 6, Article number: 20 (2015) Cite this article

Abstract

Background

Model repositories such as BioModels Database provide computational models of biological systems for the scientific community. These models contain rich semantic annotations that link model entities to concepts in well-established bio-ontologies such as Gene Ontology. Consequently, thematically similar models are likely to share similar annotations. Based on this assumption, we argue that semantic annotations are a suitable tool to characterize sets of models. These characteristics improve model classification, allow to identify additional features for model retrieval tasks, and enable the comparison of sets of models.

Results

In this paper we discuss four methods for annotation-based feature extraction from model sets. We tested all methods on sets of models in SBML format which were composed from BioModels Database. To characterize each of these sets, we analyzed and extracted concepts from three frequently used ontologies, namely Gene Ontology, ChEBI and SBO. We find that three out of the methods are suitable to determine characteristic features for arbitrary sets of models: The selected features vary depending on the underlying model set, and they are also specific to the chosen model set. We show that the identified features map on concepts that are higher up in the hierarchy of the ontologies than the concepts used for model annotations. Our analysis also reveals that the information content of concepts in ontologies and their usage for model annotation do not correlate.

Conclusions

Annotation-based feature extraction enables the comparison of model sets, as opposed to existing methods for model-to-keyword comparison, or model-to-model comparison.

Introduction

Thanks to standardization efforts in Systems Biology [1], modelers today have access to high-quality, curated models in standard formats. The Systems Biology Markup Language (SBML) [2] is an XML-based standard format to encode models as interactions between biological entities. The emerging networks are furthermore enriched with semantic annotations [3] which link model parts to external knowledge in domain-specific ontologies (bio-ontologies) [4]. Many SBML models live in open model repositories such as BioModels Database [5], the Physiome Model Repository [6], or JWS Online [7]. These repositories distribute computational models and associated data in standard formats. They support necessary management tasks, including curation, annotation, search, version control, data visualization etc. to different extents.

BioModels Database implements a native, SQL-based search [5]. An alternative search is the ranked model retrieval [8]. Here, models and their annotations are mapped on pre-defined model features (e. g., model organism, author, biological entity), leading to a characteristic term vector for each model. The properties of this vector are numeric values mostly describing term frequency and inverse document frequency (TF-IDF) [9]. The ranking is determined by the comparison of search terms (i. e. provided keywords) with the extracted characteristic term vector per model. Current approaches are solely capable of comparing a set of keywords against an indexed corpus of models and retrieve matching models. In addition, it is possible to create a characteristic term vector directly from a model and, subsequently, query a corpus by example.

For example, a standard search for the keywords “cell cycle” in BioModels Database retrieves all models in the corpus that are relevant to the term “cell cycle”. Together, all models returned by this search can be seen as a new, cell cycle focused, model set (or corpus). The same is possible for keywords such as “apoptosis”, “calcium oscillation” or “NF- κB”. At this point, we end up with different sets of thematically related models. To characterize such a set and, later on, compare them, features describing this specific model set will be helpful. However, it is problematic to identify suitable characteristics for arbitrary or thematically focused sets of models.

In this paper we present four methods for annotation-based feature extraction from arbitrary sets of SBML models. Our methods build on combinations of existing approaches for feature extraction [10-13]. We exemplify our methods by comparing the characteristic features of thematic sets to the features of arbitrary sets of SBML models. The thematic sets were extracted from BioModels Database and represent the cell cycle, apoptosis, calcium oscillation, and NF- κB. Concepts, i. e. terms in the ontology, were extracted from three major bio-ontologies used to semantically enrich models (GO, ChEBI, SBO). We argue that our methods contribute to the determination of similarity between sets of SBML models. They also provide statistics on the use of ontology terms in SBML models, and on the relation between ontology terms and models.

Background

Bio-ontologies

SBML is an XML format. It uses an RDF scheme to add semantic annotations to model parts [14]. Among the ontologies that are used to enrich SBML models, we chose here the following three ontologies, which we believe are the most relevant in model annotation: An ontology of gene and gene product attributes, the Gene Ontology (GO) [15]; an ontology of chemical entities, the Chemical Entities in BIology (ChEBI) [16]; and an ontology for modeling in biology, the Systems Biology Ontology (SBO) [3].

The GO is proposed and maintained by the Gene Ontology Consortium. It aims at standardizing the representation of gene and gene product attributes across species and databases by a structured, precisely defined, common, controlled vocabulary. GO covers three domains. The most important relationships within each domain are is-a and part-of. Additionally, each concept is linked to other kinds of information, including many gene and protein keyword databases.

ChEBI is an ontology of chemical entities of biological interest. All database entries are is_a linked within the ontology. Chemical classifications of ChEBI are aligned with the classification of chemical processes in the GO, and the majority of chemical processes in GO are defined in terms of the ChEBI entities that participate in them.

The SBO provides a set of controlled vocabularies of terms commonly used in Systems Biology. It consists of seven orthogonal branches. Terms within each branch are linked by standard is_a relationships. Formal ties to SBO have been developed for several representation formats in Systems Biology. SBML elementsa, for example, carry an optional sboTerm attribute, which allows for a precise definition of the meaning of encoded model entities and their relationships.

Feature extraction from ontologies

For feature extraction it is important to group similar items and to find categories that represent the content of the objects.

Several techniques to determine similarity use distance measures as a basis. Common techniques are euclidian or cosinus distance in vector space [17] or the editing distance for text [9,17-19]. In the context of this work the techniques to distances in ontologies and tree structures are of significance.

The hierarchical structure of the ontology can be used to determine the (semantic) similarity between objects [17]. A distinction is made between two approaches; the graph-theoretic and information-theoretic approach.

Examples for the graph-theoretic approach are the works of Bernstein et al. [17] and Wang et al. [20]. They describe the traditional approach for distance determination in ontologies using the number of edges between the nodes. The inheritance structure is represented in a directed acyclic graph in which the specialization of objects increases with each level. In such a graph the ontology distance can be described as the shortest path between two nodes. The shorter the distance between two nodes, the more similar they are. The problem with this approach is the assumption that the edges represent uniform distances within a taxonomy; i.e. the semantic connections are of equal weight. Li et al. therefore investigate in [21] how path length, depth and local semantic density influence the quality of the similarity function. They come to the conclusion, that for a semantic knowledge base especially path length and depth are important to get similarity results that compare to the human perception of similarity. The similarity values are used in cluster analysis approaches for hierarchical clustering [22]. Applied to the feature extraction task, we group concepts based on their distance in the ontology graph for one bio-ontology at a time. The top-down approach starts with a cluster containing all concepts and then splits this cluster into smaller groups. The bottom-up approach starts with clusters only containing one concept. Those clusters are merged into larger clusters.

The most prominent representative of the information-theoretic approach is Resnik [12,13]. This approach exploits the information content of objects to compare. The more information two objects have in common, the more similar they are. The information content of a concept c is dependent on the concept’s probability. The probability p(c) is calculated by the frequency f r e q(c) of the concept and the count N of all concepts of the ontology. It is formally defined by Resnik [12]:

$$ p(c) = \frac{freq(c)}{N} $$
((1))

If all concepts in an ontology are subordinate to one item, then this item has the greatest probability of 1, because its classification always applies. However, the smaller the probability of a concept is, the higher is its information content. The information content IC can be calculated by the negative logarithm of the likelihood:

$$ IC(c) = - \log_{2} p(c) $$
((2))

For example, the root term of the Gene Ontology summarizes all concepts of the ontology and consequently has an information content of zero. A child concept such as establishment of localization (GO_0051234) that summarizes 1408 concepts has a higher information content of 3.34 and a leaf concept such as natural killer cell mediated cytotoxicity directed against tumor cell target (GO_0002420) has the highest information content of 10.59.

In order to determine the common information content of two objects, one considers the deepest element that classifies both objects together. The information content of this element is the degree of mutual information content.

The Information Content can be used to address the problem of overgeneralization when using parent concepts as representatives for child concepts [23]. The challenge of feature extraction in ontologies is to find summarizing features that do not generalize too strongly. Concepts further up in the ontology are less specific than concepts further down in the ontology and, thus, have less “information content”. Counting the number of references of a concept and its successor concepts would rank the general concept always highest, as it has more references. The counting approach does not consider the loss of specificity when moving up the ontology. Trißl et al. propose a similarity-based scoring function where a general concept must be supported by more references to yield a good score of representativeness.

For our work we identified the information-theoretic approach and especially the notion of the information content to be of interest. Furthermore, we considered existing approaches for feature extraction in other areas, such as text classification, and selected the document frequency to be to some extend applicable in extracting a pre-defined number of features from sets of SBML models.

The document frequency describes the number of documents in which a term occurs [10,11]. It is used to reduce a vocabulary by removing to rare or common words, respectively. In text classification, common words are removed, because they are not discriminating for any particular class. Rare words are eliminated because they are considered non-informative for category prediction and not influential in global performance. In our specific application, common concepts from bio-ontologies are kept because they are very convenient as features. The discriminating power of a concept is given by the feature value that is saved for each model. However, rarely used concepts are removed during the feature extraction process.

For example, the Gene Ontology Term mRNA catabolic process (GO_0006402) is referenced in over 40 documents, terms of the branch establishment of localization (GO_0051234) are contained in over 200 documents, while terms of cell killing (GO_0001906) are rarely annotated. While the first two terms could be suitable as features, cell killing is not suitable at all, because only a few annotated documents could be found by this term.

Implementation

As a proof of concept, we implemented the four different methods described in Section “Results and discussion” in a prototype applicationb. We then tested all methods on seven different model sets, which we extracted from BioModels Database.

Prototype

The prototype implementation incorporates two major technologies. First, ontologies are imported using the OWL API [24] and the JFact [25] reasoner. The Web Ontology Language (OWL) is a specification of the World Wide Web Consortium (W3C) to create, publish and to distribute ontologies based on a formal description language [26]. Most bio-ontologies are available in OWL format.

Second, all relevant information about the models and the ontologies is stored in a graph database [27]. A graph database is well suited for models in SBML structure and ontologies alike. It supports links between ontology concepts and SBML models, and it allows for efficient queries [28]. For evaluation purposes, we imported the ontology concepts and their taxonomic relationships and counted the number of annotations referring from a model to a particular ontology concept. The storage approach has been described in detail in an earlier publication [29].

Test sets

We generated seven different test sets containing SBML models from BioModels Database [30]. Two model sets contain arbitrary models, four model sets have a certain biological focus, and one model set contains the complete BioModels Database (Additional file 1: Table S1).

The cell cycle set (CC) contains only models from the curated branch. This ensures ground truth in model annotation as annotations in the curated branch are manually reviewed [5]. In addition to the cell cycle set, the two random sets (RS1 and RS2), the thematic test sets for apoptosis (APOP), calcium oscillation (CA) and NF- κB (NFKB), and the set containing all 490 curated models (BMDB) were assembled from the curated branch. In contrast to the CC set (containing 30 models) the thematic test sets APOP, CA and NFKB only contain about 13 models each.

Consequently, we rely on the cell cycle set in our analysis of methods, we use the three other thematic sets for evaluation purposes.

The models for all model sets were pre-selected using our previously developed retrieval algorithm [8]. For example, the first test set is a thematic set containing SBML encodings of published cell cycle models. We used the term “cell cycle” for a keyword based search to retrieve a list of relevant models. To exclude possible false positive search results we manually validated the retrieved models based on their reference publications, resulting in the 34 given models for cell cycle. The model sets APOP, CA and NFKB were compiled in the same way.

From the biological point of view, the test sets CC, APOP, and NFKB are thematically similar. NFKB, which is one of the most prominent transcription factors, is able to manipulate cyclins that drive the cell cycle [31] and additionally has stimulus dependent pro- or anti-apoptotic functions [32]. Moreover, the connection between cell cycle and apoptosis is presented by many cells starting their apoptotic cell fate decision from the cell cycle arrest (G1/S checkpoint), i. e. after caspase activation [33]. More recently, calcium oscillations were shown to influence NF- κB activity depending on the calcium spike duration [34]. We deliberately introduced the NFKB set with strong relations to the CC and APOP sets to evaluate if our methods reflect these relations in terms of similarity of extracted features. The assumption is that biologically similar model sets share semantic annotations.

Results and discussion

Our main hypothesis is that it should be possible to extract characteristic features from semantic annotations, both for thematic sets of models and for arbitrary ones. The following subsections explain our four methods for feature identification, based on the aforementioned feature extraction methods (Section “Implemented feature extraction methods”); discuss their applicability to feature extraction from model sets (Section “Applicability of methods”); show the distribution of model annotations in BioModels Database (Section “Distribution of SBO concepts in SBML models”); and discuss the results obtained from two selected methods when applied to the abovementioned test sets (Section “Feature extraction from arbitrary model sets”). We conclude that it is indeed possible to identify characteristic features. These features can, for example, help with model retrieval, comparison and clustering.

Implemented feature extraction methods

Our methods are designed to identify a predefined, maximum number of features for each compiled set of models. All methods incorporate the structure of the underlying ontology when grouping the concepts within it. Parent concepts represent the group containing their child concepts. Consequently, the developed methods are only applicable to taxonomy-shaped ontologies. Method 1 depends only on the chosen ontology, but not on the input set of models. All other methods additionally consider the annotations in the given set of models.

Method 1 is a top-down clustering. To decide on the suitability of a concept for characterization, the probability p of each concept in the ontology is determined, following Resnik’s definition (Equation 1). In the context of this work, the frequency f r e q(c) refers to the number of all concepts that are summarized by a parent concept c.

Method 2 is a top-down clustering that considers both the ontology structure and the annotations used in models of the given set. Consequently, the real distribution of references to ontology concepts used in models is regarded. Selected features depend on the given set of models. For each concept in the ontology, we count the number of annotations that refer to it. We call this number entity frequency. Additionally, we store the sum of a concept’s entity frequency and its descendants’ entity frequencies as aggregated entity frequency EF. All concepts with E F>0 provide the basis for feature extraction. Method 2 re-uses the algorithm of Method 1. The algorithm is adjusted to the dynamic setting by using the entity frequency metric instead of the probability p(c). To better compare the balance of the branches, we will normalize EF as entity probability e p(c):

$$ ep(c) = \frac{EF(c)}{EF(root)} $$
((3))

Method 3 is a bottom-up clustering relying on the same input as Method 2. It also uses the entity probability e p(c) but begins with the individual concepts, which are gradually merged to form greater clusters. The results of this method are nearly identical to the ones of Method 2, but the performance of Method 2 is much better.

Method 4 is a bottom-up clustering that addresses the problem of overgeneralization. It uses an adaptation of the scoring function as described in [23]:

$$ Score_{T}(c) = IC(c) \cdot EF(c) $$
((4))

The S c o r e T (c) for a grouping represented by the concept c considers the information content and the aggregated entity frequency. The information content is calculated depending on the probability of c (see Equations 1 and 2). A group is formed by merging concepts with the ancestor that reaches the highest possible score.

Applicability of methods

We tested the applicability of all described methods on sets of SBML models taken from BioModels Database. Method 1 calculates the probability to hit a certain node in an ontology with a model entity. It condenses a given ontology to a defined number of features, based on the probability of a concept in the ontology only. Thus, the results obtained from Method 1 do not depend on the actual ontology concepts that are referenced in the model set. Consequently, it does not adapt to the specifics of the corpus under study. Therefore, Method 1 is only suitable to provide a static set of features, solely based on the underlying ontology. As a result we dismissed Method 1 for the problem of finding characteristics for arbitrary model sets. However, Method 1 calculates the distribution of concepts in bio-ontologies, as shown in Section “Distribution of SBO concepts in SBML models”. Method 2 and Method 3 rely on entity probabilities. Our evaluations show that Method 2 (top-down) and Method 3 (bottom-up) produce almost identical results. The direction is only relevant in the rare constellation that two concepts are subsumed to the same score. In the following, we consider Method 2 for further evaluations. Method 4 is a dynamic approach that calculates the score value by entity frequency and information content. Based on the unique scoring and the absence of splits, Method 4 generally finds fewer features than the prior methods. It also selects more specific features (located further down in the ontology tree) that are still representative for the model sets. In Section “Feature extraction from arbitrary model sets” we use Method 2 and Method 4 to discuss the specificity and distinctness of extracted features.

Distribution of SBO concepts in SBML models

Using Method 1, we compare the distributions of concepts in the SBO with the frequency of annotations as they occur in all models from BioModels Database. It becomes obvious that the concepts are unequally distributed across seven top-level branches (Figure 1, top). This is explained by the design of the SBO and its orthogonal branches. For example, the branch modeling framework (SBO:0000004) lists a “set of assumptions that underlay a mathematical description” whereas the branch mathematical expression (SBO:0000064) contains “formal representation of a calculus linking parameters and variables of a model”. Consequently, one expects more entries for mathematical expression than for modeling framework.

Figure 1
figure 1

Concept vs. annotation distribution in SBO. Overview of the concept distribution in the seven branches of the Systems Biology Ontology (SBO). The size of the colored circles visualizes the number of concepts summarized by each branch. The bottom mirrored image visualizes the distribution of annotations from all models in the BioModels Database test set (BMDB). Figure adapted from [3].

Full size image

In conjunction with the application of SBO in model annotation, concepts of some branches are annotated more frequently (Figure 1, bottom). For example, the branch physical entity representation (SBO:0000236), which is a “representation of an entity that may participate in an interaction, a process or relationship of significance”, contains only 10% of SBO concepts, but 47% of the model annotations link to that branch. We expect that the characteristic features follow the distribution of the model annotations as seen in the lower part of the figure. Indeed, after applying Method 4, the selected SBO features show a distribution (66.6% physical entity representation (SBO:0000236), 6.6% participant role (SBO:00000003), 13.3% occurring entity representation (SBO:0000231), 6.6% mathematical expression (SBO:0000064), and 6.6% systems description parameter (SBO:0000545) that is closer to Figure 1 (bottom) than before; please refer to Table 1, Method 4, SBO, 15 features).

Table 1 Extracted features for different sets (CC, RS1, RS2 and BMDB), methods and feature size
Full size table

We also investigated for each model set the distribution of the depth of annotated concepts in the ontology tree. This knowledge helps us to decide on how specific a model annotation is. Figure 2 shows the distribution for model annotations using ChEBI, GO and SBO (Additional file 2).

Figure 2
figure 2

Concept depth of annotations. Distribution of annotation depth. Overview of the distribution of annotated model entities in relation to the depth of the annotation. The x-axis shows the depth of the annotated concepts in the corresponding ontology, the y-axis shows the number of annotated entities on a logarithmic scale (exact values are stated at the bottom of the figure). The figure legend states the ontology name, the model set and the average depth.

Full size image

Here, we plotted the distribution of annotations for the CC and the BMDB sets. As one would expect, both test sets show normal distributions. The 30 models contained in the CC set make up 6% of the 490 models in the BMDB set. However, the number of annotations in the CC set that refer to ChEBI is less than 1% compared to the number of annotations in the BMDB set. It should be considered that very sparsely annotated model set may be inferior in terms of specificity and distinctness. This information helps us later on in Section “Feature extraction from arbitrary model sets” to decide on the value of the extracted features.

Feature extraction from arbitrary model sets

We hypothesize that the vast property space of a set of models can be condensed into a smaller, but still descriptive, number of features. To establish such “characteristic features”, we collect the models’ annotations and analyze the semantics behind the linked ontology terms. We focus on the semantics behind the model elements because we believe that this information will be most influential. All our methods require setting a maximum number of features.

Here, we chose to run our extraction methods with five and 15 features as an upper limit. The resulting sets of features for all feature extraction algorithms, models, and ontologies are shown in Table 1.

Specificity of selected ontology concepts. Table 1 shows the average depth of concepts in all three ontologies for all identified features in the CC and BMDB sets. Additionally, Figure 2 contains the average depth of annotation for the CC and BMDB sets before applying the feature extraction methods. The data confirms that the average depth of annotations decreases for Methods 2 and 4 (for all three ontologies and both model sets). Thus, selected concepts are higher up in the ontology, and more generic. This behavior is expected as the feature extraction process also involves generalization. However, the features extracted by Method 4 are more specific than the features extracted by Method 2. This is in accordance with the design of Method 4 to prevent overgeneralization. Moreover, the average annotation depth for the CC set is higher than for the corresponding BMDB set. This supports our assumption that thematically similar models share more annotations, and consequently the extracted features are more specific. For example, the concepts that were selected from ChEBI by Method 2 with a maximum of 15 features for the CC set have an average annotation depth of 5.9. In contrast, the concepts that were selected for the BMDB set only have an average depth of 4.8. According to our obtained data we infer that Method 4, in general, provides features that correspond to deeper concepts in the ontology than the features obtained from Method 2. We conclude from our test data that the depth of chosen concepts decreases with the increased randomness in the sets of models. This is not unexpected, as a broader data basis should not be characterizable by very specific ontology concepts. Rather, an arbitrary model set should cover many different semantic concepts, leading to more generic features being extracted. This behavior is also reflected in our data. In summary, both methods extract features that are specific to the model set. However, features extracted by Method 4 are mostly more specific than those extracted by Method 2. An exemption where the average depth slightly increases is Method 4 for SBO and 15 features. SBO is relatively small compared to GO or ChEBI. As Method 2 is required to select 15 features and Method 4 is only required to select up to 15 features, Method 4 selects only the most relevant features whereas Method 2 selects exactly 15 features. Due to the size of SBO, Method 2 adds features that are not best matches, nevertheless have a higher depth within SBO. This phenomenon did not occur for the lager ontologies, GO and ChEBI.

Distinctness of feature sets. Another important question is how distinct the obtained features are for our test sets. If the methods retrieved similar concepts for the four test sets, then the extracted features could not be regarded specific to the set of models. Consequently, we measure overlap of concepts between the different characteristic features that we calculated with Method 2 and Method 4. Ideally, there would be almost no overlap of features selected for the CC set with any other selected set, whereas an overlap between BMDB and the random sets is expectable. Our results are shown in Figure 3.

Figure 3
figure 3

Feature overlaps. Visualization of feature overlaps of the four test sets. Each diagram shows the overlap of the results of one ontology (SBO, GO or ChEBI), method (M2 or M4) and number of features (F5 or F15).

Full size image

A good result is achieved for Method 4 using 15 features and GO. Here, the cell cycle features have almost no overlap. The result achieved for Method 2 using 15 features and GO is not satisfiable. Here, the cell cycle features largely overlap with at least two other sets. However, the Venn diagrams, in general, confirm that both methods determine features that are specific to the model sets. They contain higher numbers of overlapping features at the intersection between arbitrary sets and very few overlapping features at the intersection between the CC and the BMDB sets. This is particularly visible for the results obtained from Method 4.

Similarity of model sets. We are also interested in how characteristic the sets of extracted features are for a given set of models. We first calculate the similarity of two concepts within the same ontology, as described by Li et al. [21]:

$$ S(c_{1}, c_{2}) = \mathrm{e}^{- \alpha l} \cdot {{\mathrm{e}^{\beta h} - \mathrm{e}^{- \beta h}} \over {\mathrm{e}^{\beta h} + \mathrm{e}^{- \beta h} }} $$
((5))

The variable h is the depth of the least common subsumer of the concepts c 1 and c 2, and the variable l is the length of the shortest path between both concepts. Following [21], the parameters are set to α=0.2 and β=0.6. We calculate this similarity value for each possible combination of features from two sets of models.

Afterwards we apply an adaptation of the Hungarian method [35] to the matrix resulting from the above calculations. The Hungarian method, a solution for the assignment problem, aligns pairs of features, in a way that ensures a global maximum similarity. Based on this similarity of features, we then calculate the total similarity of two sets of features, which corresponds to the similarity of the associated sets of models. The results are shown in Table 2.

Table 2 Similarity between thematic and arbitrary model sets, calculated based on the similarity of their characteristic features
Full size table

Firstly, we discuss specificity of extracted features for the cell cycle set versus the set containing all curated models from Biomodels Database, and one random set. Desirable are low similarities for BMDB vs CC as well as CC vs RS1. As CC is a thematic set, its extracted features should differ from the features extracted from the BMDB and arbitrary model sets. A higher similarity is expected for BMDB vs RS1, as both sets represent a wide range of model topics. The results in Table 2 reflect our expectations. Particularly, the similarity values for Method 4 using 15 features clearly distinguish the extracted features of two sets. Method 2 using five features still shows the desired result, but due to the limited number of features the selected ones are more general and not very distinguishable. Even though results of Method 2 show the expected behavior, we conclude that the results of Method 4 are superior.

Secondly, we discuss the specificity for all thematic sets. Here, we narrow our scope to the Gene Ontology. As Table 3 indicates only the number of distinct annotations using GO is sufficient for all four thematic sets. In addition, we manually reviewed the extracted features and deduced that the features extracted for GO have the highest biological meaning. We use the aforementioned approach to calculate similarity between extracted features of six sets (BMDB, RS1, CC, APOP, CA, NFKB), as shown in Tables 1 and 4. Results for five and 15 selected sets are shown in Table 5. It becomes obvious that the similarities for Method 2 are to high in general, this supports our previous assumption of Method 2 over-generalizing the extracted features. An example of over-generalization is Method 2 using 5 features and the sets RS2 and NFKB. Both sets perfectly match. The reason for this match is, that Method 2 selected only top and second level representatives (both sets have an average depth of 1.8).

Table 3 Number of curated model contained in each thematic data set
Full size table
Table 4 Extracted features for thematic test sets, methods and feature size
Full size table
Table 5 Similarity between two model sets, calculated based on the similarity of their characteristic GO features
Full size table

Desirable are low similarities for each thematic set versus the BMDB, RS1 or RS2 set, respectively. Both result tables show according similarity values for Method 4. We expect NFKB to have a slightly higher similarity to the other three thematic sets as NF- κB has a regulatory effect on cell cycle, apoptosis and calcium oscillation. For five and 15 selected features Method 4 fits our expectation. The relation between CC and APOP is also visible as many cells start apoptosis from the cell cycle arrest. This is also supported by Method 4 for five and 15 features, respectively. In contrast, we predict CA to be distinct from CC and APOP as calcium oscillation has low overlap with cell cycle or apoptosis. Again, Method 4 advocates our prediction. In conclusion, Method 4 was able to support all our assumptions, even if only five characteristic features are provided per set.

Conclusions

This paper presents and discusses methods for the annotation-based extraction of characteristic features from sets of SBML models. The methods consider clustering and text classification techniques to extract characterizing features for sets of annotated computational models in biology. Annotation-based feature extraction enables the comparison of sets of models, as opposed to existing methods for model-to-keyword comparison, or model-to-model comparison.

We evaluated four different methods for feature extraction and conclude that Method 4 is the most suitable. This method considers both, the semantic annotations in a set of models, and the information content of the ontology concepts. For our seven test sets, we showed that the extracted features are specific and distinct. In addition, we demonstrated that the extracted features are not overgeneralized. Thus, our expectations have been met: A thematic set of models, for example cell cycle models, can computationally be distinguished from arbitrary and other thematic sets of models. Finally, we suggested how to assign a similarity value to sets of models, based on the similarity of the extracted features.

Our applied methods are format agnostic and expandable. They can be adapted to other model representation formats such as CellML [36] or NeuroML [37]. Interestingly, these extensions enable a comparison between sets of models of arbitrary formats. It is also possible to incorporate further bio-ontologies, e. g. BRENDA [38].

For the near future, we plan to integrate Method 4 in our system for ranked model retrieval [8]. We wish to test the implications of feature extraction on model comparison and, in particular, model retrieval. We will also incorporate a larger set of ontologies into our system and ultimately in the process of feature extraction.

Endnotes

a Since Level 2 Version 2.

b our code repository is available at https://bitbucket.org/ronhenkel/masymos.

References

  1. Hucka M, Nickerson DP, Bader GD, Bergmann FT, Cooper J, Demir E, et al. Promoting coordinated development of community-based information standards for modeling in biology: the COMBINE initiative. Frontiers in bioengineering and biotechnology. 2015; 3:19.

    Article  Google Scholar 

  2. Hucka M, Finney A, Sauro HM, Bolouri H, Doyle JC, Kitano H, et al. The systems biology markup language (SBML): a medium for representation and exchange of biochemical network models. Bioinformatics. 2003; 19(4):524–31.

    Article  Google Scholar 

  3. Courtot M, Juty N, Knüpfer C, Waltemath D, Zhukova A, Dräger A, et al. Controlled vocabularies and semantics in systems biology. Molecular systems biology. 2011; 7(1):543.

    Article  Google Scholar 

  4. Robinson PN, Bauer S. Introduction to Bio-ontologies. Boca Raton, FL, USA: CRC Press; 2011.

    Google Scholar 

  5. Li C, Donizelli M, Rodriguez N, Dharuri H, Endler L, Chelliah V, et al. BioModels Database: An enhanced, curated and annotated resource for published quantitative kinetic models. BMC systems biology. 2010; 4(1):92.

    Article  Google Scholar 

  6. Yu T, Lloyd CM, Nickerson DP, Cooling MT, Miller AK, Garny A, et al. The physiome model repository 2. Bioinformatics. 2011; 27(5):743–44.

    Article  Google Scholar 

  7. Olivier BG, Snoep JL. Web-based kinetic modelling using jws online. Bioinformatics. 2004; 20(13):2143–2144.

    Article  Google Scholar 

  8. Henkel R, Endler L, Peters A, Le Novère N, Waltemath D. Ranked retrieval of Computational Biology models. BMC Bioinformatics. 2010; 11(1):423.

    Google Scholar 

  9. Baeza-Yates R, Ribeiro-Neto B. Modern Information Retrieval. Harlow, England: Addison-Wesley; 1999. http://books.google.de/books?id=nsjla44zAfwC

    Google Scholar 

  10. Yang Y, Pedersen JO. A Comparative Study on Feature Selection in Text Categorization. In: Proceedings of the Fourteenth International Conference on Machine Learning. ICML ’97. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc.: 1997. p. 412–20. http://dl.acm.org/citation.cfm?id=645526.657137

    Google Scholar 

  11. Forman G. An extensive empirical study of feature selection metrics for text classification. The Journal of machine learning research. 2003; 3:1289–1305.

    MATH  Google Scholar 

  12. Resnik P. Using information content to evaluate semantic similarity in a taxonomy. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence. Montreal, Canada: Morgan Kaufmann Publishers Inc.: 1995. p. 445–53.

    Google Scholar 

  13. Resnik P. Semantic similarity in a taxonomy: An information-based measure and its application to problems of ambiguity in natural language. Journal of Artificial Intelligence Research. 1999; 11:95–130.

    MATH  Google Scholar 

  14. Waltemath D, Swainston N, Lister AL, Bergmann F, Henkel R, Hoops S, et al. SBML Level 3 Package Proposal: Annot. available online, Nature Preceedings. 2011. http://precedings.nature.com/documents/5610/version/1

  15. Ashburner M, Ball CA, Blake JA, Botstein D, Butler H, Cherry JM, et al. Gene Ontology: tool for the unification of biology. Nature Genetics. 2000; 25(1):25–9.

    Article  Google Scholar 

  16. Hastings J, de Matos P, Dekker A, Ennis M, Harsha B, Kale N, et al. The ChEBI reference database and ontology for biologically relevant chemistry: enhancements for 2013. Nucleic acids research. 2013; 41(D1):D456–63. doi:10.1093/nar/gks1146.

    Article  Google Scholar 

  17. Bernstein A, Kaufmann E, Bürki C, Klein M. How Similar Is It? Towards Personalized Similarity Measures in Ontologies In: Ferstl OK, Sinz EJ, Eckert S, Isselhorst T, editors. Proceedings of 7. Internationale Tagung Wirtschaftsinformatik. Bamberg, Germany: Physica-Verlag HD: 2005. p. 1347–66. http://dx.doi.org/10.1007/3-7908-1624-8_71

    Google Scholar 

  18. Algergawy A, Nayak R, Saake G. Element similarity measures in xml schema matching. Inform Sci. 2010; 180(24):4975–98. doi:10.1016/j.ins.2010.08.022.

    Article  Google Scholar 

  19. Schallehn E, Sattler K-U, Saake G. Efficient similarity-based operations for data integration. Data Knowl Eng. 2004; 48(3):361–87. doi:10.1016/j.datak.2003.08.004.

    Article  Google Scholar 

  20. Wang BB, Mckay RI, Abbass HA, Barlow M. A comparative study for domain ontology guided feature extraction. In: Proceedings of the 26th Australasian Computer Science Conference - Volume 16. ACSC ’03. Darlinghurst, Australia: Australian Computer Society, Inc.: 2003. p. 69–78. http://dl.acm.org/citation.cfm?id=783106.783115

    Google Scholar 

  21. Li Y, Bandar ZA, McLean D. An approach for measuring semantic similarity between words using multiple information sources. Transactions on Knowledge and Data Engineering. 2003; 15(4):871–82. doi:10.1109/TKDE.2003.1209005.

    Article  Google Scholar 

  22. Hastie T, Tibshirani R, Friedman J. Hierarchical Clustering. In: The elements of statistical learning. New York, NY, USA: Springer: 2009. p. 520–28. http://link.springer.com/content/pdf/10.1007/978-0-387-84858-7.pdf

    Chapter  Google Scholar 

  23. Trißl S, Hussels P, Leser U. InterOnto – Ranking Inter-Ontology Links In: Bodenreider O, Rance B, editors. Data Integration in the Life Sciences. Lecture Notes in Computer Science, vol. 7348. New York, NY, USA: Springer: 2012. p. 5–20. http://dx.doi.org/10.1007/978-3-642-31040-9_2

    Google Scholar 

  24. Horridge M, Bechhofer S. The owl api: A java api for owl ontologies. Semantic Web. 2011; 2(1):11–21.

    Google Scholar 

  25. JFact DL Reasoner: http://jfact.sourceforge.net/

  26. McGuinness DL, Van Harmelen F. Owl web ontology language overview. W3C recommendation. 2004; 10(2004-03):10.

    Google Scholar 

  27. Henkel R, Wolkenhauer O, Walthemath D. Combining computational models, semantic annotations and simulation experiments in a graph database. Database. 2015; 2015:bau130.

    Article  Google Scholar 

  28. Waltemath D, Wolkenhauer O, Le Novère N, Dumontier M. Possibilities for integrating model-related data in computational biology. In: CEUR Workshop Proceedings of the 9th International Conference on Data Integration in the Life Sciences: 2013. Available online, www2.unb.ca/csas/data/ws/dils2013/

  29. Henkel R, Le Novère N, Wolkenhauer O, Waltemath D. Considerations of graph-based concepts to manage computational biology models and associated simulations. GI-Jahrestagung. 2012; 2012:1545–51.

    Google Scholar 

  30. Release 25 of BioModels Database: ftp://ftp.ebi.ac.uk/pub/databases/biomodels/releases/2013-06-18/BioModels_Database-r25_pub-sbml_files.tar.bz2

  31. Joyce D, Albanese C, Steer J, Fu M, Bouzahzah B, Pestell RG. NF- κB and cell-cycle regulation: the cyclin connection. Cytokine Growth Factor Rev. 2001; 12(1):73–90.

    Article  Google Scholar 

  32. Kaltschmidt B, Kaltschmidt C, Hofmann TG, Hehner SP, Dröge W, Schmitz ML. The pro- or anti-apoptotic function of NF- κB is determined by the nature of the apoptotic stimulus. Eur J Biochem. 2000; 267(12):3828–35. doi:10.1046/j.1432-1327.2000.01421.x.

    Article  Google Scholar 

  33. Pucci B, Kasten M, Giordano A. Cell cycle and apoptosis. Neoplasia. 2000; 2(4):291–9.

    Article  Google Scholar 

  34. Zhu L, Song S, Pi Y, Yu Y, She W, Ye H, et al. Cumulated Ca2+ spike duration underlies ca2+ oscillation frequency-regulated NF κB transcriptional activity. J Cell Sci. 2011; 124(15):2591–601.

    Article  Google Scholar 

  35. Kuhn HW. The hungarian method for the assignment problem. Naval research logistics quarterly. 1955; 2(1–2):83–97.

    Article  MathSciNet  Google Scholar 

  36. Cuellar AA, Lloyd CM, Nielsen PF, Bullivant DP, Nickerson DP, Hunter PJ. An overview of CellML 1.1, a biological model description language. Simulation. 2003; 79(12):740–747.

    Article  Google Scholar 

  37. Gleeson P, Crook S, Cannon RC, Hines ML, Billings GO, Farinella M, et al. NeuroML: a language for describing data driven models of neurons and networks with a high degree of biological detail. PLoS computational biology. 2010; 6(6):1000815.

    Article  MathSciNet  Google Scholar 

  38. Schomburg I, Chang A, Placzek S, Söhngen C, Rother M, Lang M, et al. BRENDA in 2013: integrated reactions, kinetic data, enzyme function data, improved disease classification: new options and contents in BRENDA. Nucleic Acids Research. 2013; 41(D1):D764–72.

    Article  Google Scholar 

Download references

Acknowledgements

The authors like to thank Kurt Sandkuhl for the lively and fruitful discussions. This paper has been part of DILS2014. DW was funded by the German Federal Ministry of Education and Research (e:Bio program SEMS, FKZ 031 6194).

Author information

Authors and Affiliations

  1. Department of Multimedia Communication, University of Rostock, Joachim-Jungius-Str. 11, Rostock, 18051, Germany

    Rebekka Alm

  2. Fraunhofer Institute for Computer Graphics Research IGD, Joachim-Jungius-Str. 11, Rostock, 18059, Germany

    Rebekka Alm

  3. Department of Systems Biology and Bioinformatics, University of Rostock, Ulmenstr. 69, Rostock, 18051, Germany

    Dagmar Waltemath, Markus Wolfien & Olaf Wolkenhauer

  4. Department of Mobile Multimedia Information Systems, University of Rostock, Albert-Einstein-Str. 22, Rostock, 18051, Germany

    Ron Henkel

  5. Stellenbosch Institute for Advanced Study (STIAS), Wallenberg Research Centre at Stellenbosch University, Stellenbosch, South Africa

    Olaf Wolkenhauer

Authors
  1. Rebekka Alm
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dagmar Waltemath
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Markus Wolfien
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Olaf Wolkenhauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ron Henkel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rebekka Alm.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

RA designed and implemented the classification methods. DW developed the study and revised the manuscript. MW compiled the test sets and helped with biological context. RH developed the study and conducted the evaluation. All authors wrote the manuscript. All authors read and approved the final manuscript.

Additional files

Additional file 1

Supplementary material. A landscape table. This table lists for each of our seven test sets the contained models by their Biomodels Database ID.

Additional file 2

Depth of ontology entries. This file lists the number of annotations pointing to a certain depth within an ontology for each model set.

Additional file 3

Extracted features. This file lists extracted features and corresponding depth for each model set, feature size and ontology.

Rights and permissions

This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alm, R., Waltemath, D., Wolfien, M. et al. Annotation-based feature extraction from sets of SBML models. J Biomed Semant 6, 20 (2015). https://doi.org/10.1186/s13326-015-0014-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13326-015-0014-4

Keywords

Journal of Biomedical Semantics

ISSN: 2041-1480