Developing VISO: Vaccine Information Statement Ontology for patient education
© Amith et al. 2015
Received: 9 December 2014
Accepted: 28 March 2015
Published: 1 May 2015
To construct a comprehensive vaccine information ontology that can support personal health information applications using patient-consumer lexicon, and lead to outcomes that can improve patient education.
The authors composed the Vaccine Information Statement Ontology (VISO) using the web ontology language (OWL). We started with 6 Vaccine Information Statement (VIS) documents collected from the Centers for Disease Control and Prevention (CDC) website. Important and relevant selections from the documents were recorded, and knowledge triples were derived. Based on the collection of knowledge triples, the meta-level formalization of the vaccine information domain was developed. Relevant instances and their relationships were created to represent vaccine domain knowledge
The initial iteration of the VISO was realized, based on the 6 Vaccine Information Statements and coded into OWL2 with Protégé. The ontology consisted of 132 concepts (classes and subclasses) with 33 types of relationships between the concepts. The total number of instances from classes totaled at 460, along with 429 knowledge triples in total. Semiotic-based metric scoring was applied to evaluate quality of the ontology.
KeywordsBiomedical informatics Vaccines Vaccine Information Statements Knowledge based systems Ontology Ontology construction
In the present information age, patients are afforded many options to educate themselves on vaccines. Some of these options included valid and reputable websites, books, and other media sources while other options may appear reputable but are not. Regardless of the source’s credibility, one researcher found that 70% of individuals who seek vaccine information online are influenced by what they find on the Internet . Hopefully, patients and parents are able to identify the most reliable resources such as the Centers for Disease Control and Prevention’s (CDC) Vaccine Information Statements (VIS). VISs were developed in response to the National Childhood Vaccine Injury Act (NCVIA) which was passed in an effort to minimize provider liability and respond to public health concerns. The NCVIA requires healthcare providers to provide a VIS to the person receiving the vaccine or his/her guardian. The VIS must be given each time a vaccine is administered and provides information on the benefits and risks of the vaccine as well as other relevant disease information. They are usually provided to the patient as a handout .
Some researchers have identified specific issues with the dissemination of vaccine information through these documents. Both Lieu, et al., and St. Amour, et al. have revealed that vaccine documentation handed to patients are rarely read or fully understood by the patient [3,4]. This kind of education is called passive education, because the patients may or may not choose to read the materials. In addition, while the purpose of the VIS is to inform parents and initiate potential questions, limited clinical staff time and resources may hinder the optimal interactions ideal for vaccine education. This demonstrates a pervasive issue with medical education for patients, where the delivery and presentation should be more consumer-friendly in order to effectively impact patient education and increase knowledge [5-7].
Following in the examples described, we developed an ontology-driven mobile application system to improve patient learning and comprehension of vaccine knowledge . While a proof of concept prototype, the system was limited by the test ontology which comprised of 19 classes and 82 instances, a relatively small knowledge base with which to interact. To further extend this project, as a first step, we aimed to create a comprehensive vaccine ontology with which patients can interact.
The objective of this paper is to formalize the vaccine knowledge for patient education using ontology tools as a solution for vaccine education for the public. We propose modeling vaccine knowledge using OWL2 (Web Ontology Language) , which will be encoded with Protégé . We focus on the knowledge in the VISs developed by the CDC. By publishing the Vaccine Information Statement Ontology (VISO) in a domain ontology and offering the benefit of a serialized format that can be processed by a machine and reused , we introduce opportunities to develop ontology-driven personal health agents to improve patient learning and comprehension of vaccine knowledge. Overall, the overarching goal is to develop a scalable ontological model that can reliably cover any new and applicable vaccine knowledge for a consumer audience. With a scalable conceptual-level model, opportunities in natural language ontology learning and population would be a future possibility to investigate for automated upkeep and maintenance of the ontology.
This paper will start by briefly discussing the source material, the VISs. Afterwards, the paper will segue way into describing the development of the class-level schema and the instance level of VISO. This will include discussion of the initial common design patterns encountered and basic ontology metrics. The Results section will elaborate the quantitative and qualitative aspects of VISO, that also includes an initial evaluation scoring. The paper will then close with challenges in developing an ontology from the CDC VISs and future direction.
The focus of this paper is the ontology representation of the knowledge in the VISs developed by the CDC. Currently there are 25 VISs available from the CDC’s website, and these documents vary between 1 to 2 pages . The VISs describe patient-level information about vaccines ranging from the historical burden of disease to the clinical indications for the vaccine.
All of the vaccines selected are recommended for children on or before the first year of life. Each of the VIS documents are available for download from the CDC’s website in both PDF and RTF format, along with the HTML version.
General vaccine and disease information.
Possible vaccine reactions and/or side effects.
Populations for whom the vaccine is indicated.
General vaccine recommendations and dose information.
Each of the six VISs acquired from the CDC’s website were examined, and sentences were identified as relevant selections to inform the design of VISO. Ignoring headers and standard text that appeared on VIS documents, the relevant passages were transcribed onto a tracking spreadsheet and coded for simple identification. Any fragment with bullet points was also recorded. Each phrase in the VIS was broken down into a knowledge triple, which is a piece of factual information that is decomposed to a subject-predicate-object format that can be visually represented or modeled in an ontology. Separate tracking documents were used for individual VIS documents for management purposes. Later, separate knowledge models for each of the VIS documents were realized, and then collated to obtain a comprehensive model of vaccine information. The proceeding subsections will discuss the high-level conceptualization of the VISO model, detailing the class-level organization and formalization of knowledge, and the last subsection will discuss the encoding of the instances and triples, along with reoccurring design patterns in the VIS domain. The end goal was to develop common classes that can accommodate a large corpus of vaccine information, and have a formalized model to cover an expansive vaccine domain that is relevant for patients.
Example subclasses or Instances (in italics)
A class description to categorize vaccines documented by the CDC’s VIS documentation
Provides class specification for virus, bacteria, diseases, etc. that are prevented by Vaccine Disease
Virus, SeriousDisease, Bacteria
Categorizes various types of patients or groups of individuals impacted by vaccines, diseases, reactions, and other health conditions
PeopleWithCondition, Infants, PeopleWithIllness, PeopleWithModerateIllness, PeopleOfAge, Children, Adults
Used to describe an origin of a vaccine’s target (bacteria, disease). Can be reused in relations with other classes.
bacteria, Hepatitis B virus
Class of medium of travel for vaccine targets.
PeopleChannel, ObjectChannel, PeopleWithConditionChannel, DermalChannel, HumanActivityChannel, FluidChannel s
For a description of a condition as a result of a vaccine target. E.g. infection, coughing spells.
ear infection, long-term illness, coughing spells, pneumonia
Type to categorize location, specifically area of the body, affected by a heath condition or reaction
FacialLocation, ThroatLocation, ArmLocation
Classification for types of probabilities enumerated in VIS documents
QualitativeProbability, QuantitativeProbability, ProbabilityInCases
Types of effects resulting from causation, reactions, or chain of outcomes.
RareOutcome, AdultOutcome, ChildOutcome, FatalOutcome
Used for various types of descriptions for qualitative length of time for effects of health conditions or signs of conditions.
Classification of kinds of substance for vaccines and possibly other class groupings.
LiquidSubstance, GaseousSubstance, SolidSubstance
For various artifacts that interact with vaccines.
Groups and classifies inoculation methods for vaccines.
InhaleMethod, InjectionMethod, OralMethod
Enumerates the maximum number of doses for vaccine.
Designates the types of dose or the dose interval. E.g. 1st Dose, 3rd Dose.
Categorizes types of elements of a vaccine.
Enumerates the type of quantitative classification of age ? years, months, weeks, etc.
Enumerates the types of quantitative classification of date ? days, months, weeks, etc.
DateInYears, DateInMonths, DateInDays
Classification of types of events
Types of patient recourse in response to reactions or actions required vaccine patients before inoculation.
InactiveAction, ActiveAction, EmergencyAction
For classes of substances leading to an allergic reaction.
Used to categorizes types of reactions that may result from vaccination.
MildReaction, TemporaryReaction, SeriousReaction, ModerateReaction, AdultReaction
Class of indicators for vaccine reaction effects.
fast heartbeat, crying, stomach pain, hives
Total instances of classes
The VISO model’s high-level classes utilizes extensive subclassing to cover specific abstraction of entities. One apparent use was for categorization purposes to organize the knowledge for better identification, such as the case for the People class. For example, the VIS documents often refer to a specific population that may react differently to vaccines, or may need a different vaccine type due to age or certain medical conditions, justifying the need to create subclasses to refer to specific groups of people.
Another motivation for subclasses is to facilitate descriptions from the text that are in the form of adjectives or prepositions. If the documentation refers to the number of optional doses of a vaccine, a subclass called OptionalNumberOfDoses, which is a child of NumberOfDoses, was defined. Attention was given to classes that have “universal” subclasses that may not have been observed from the VIS documentation. Some examples are Organ class (“Heart”, “Liver”, “Lungs”) or Substance class (“GaseousSubstance”, “LiquidSubstance”, “SolidSubstance”).
Value sets and partitions
To address quantitative, descriptive, or ranks in classes, value set representations, as described in a W3C working draft , was utilized for the VISO meta-ontology. More specifically, we use subclasses to represent permissible values in a value set. For example, the VIS documentation alluded to a three-point scale, especially when describing severity of conditions - mild, moderate, severe (or serious). We created a Reaction class that has the subclasses MildReaction, ModerateReaction, and SeriousReaction to describe the degree of vaccine effect severity. Other classes that employ value sets include the People class and the Target class. For any other units of measure revealed in the VIS documents, specific classes yielded subclasses that handled units of measure, such as, AgeInYears and AgeInMonths for the Age class or DateInMonths and DateInDays for the Date class.
Knowledge triples evoked by the VIS documents suggested common relationships or properties between classes across the VIS corpus. In all of the 6 documents, it was common to describe a vaccine preventing a bacterial infection or virus, or a vaccine can potentially cause a rare reaction following administration. Many of these properties between the classes were identified and normalized to a standard representation. One example in the DTaP VIS, the evoked triple - “Another vaccine, called Td, protects against tetanus and diphtheria...” - utilized the predicate “protect against”, which essentially means “Td prevents tetanus and diphtheria”. In result, “Td protects against tetanus” was rendered as “Td prevents tetanus”, where “prevents” is the standard property, or controlled term, to describe that specific relationship.
Target, Cause, Reaction
Cause, Outcome, Reaction
has number of
is a substance of
is safe for
is safe with
Target, Reaction, Sign, ObjectChannel
Target, Cause, Outcome, Reaction
Total number of triples
also known as
is described as
Total number of triples
Instances and triples
Afterwards, the meta-level ontology development led into the encoding of instances from the collected VISs. Referring to Table 1, the People class accounted for most of instances, with 93 instances of the People classes. Since the initial set of VIS documents was small, some of the classes had one instance stemming from its class. There is a possibility that the remaining VIS documents may add more instances to these classes.
Earlier, we indicated that across the VIS documents there exist some consistency that influence the representation of knowledge in the VISO. This was also reflected in how the instances are conceptualized. The following subsections list some noticeable design patterns observed in the development of VISO - dosage pattern for vaccine dose representation, target pattern for representing the object of vaccine prevention, and the reaction pattern for possible side effects of the vaccine.
The patterns described in this section are the most evident types in the mapping of the information, but additional vaccine information design patterns also exist. It is also understood that more patterns may emerge with the inclusion of the remaining VIS documents.
To evaluate the overall quality of VISO, the ontology was scored using the ontology metrics suite proposed by Burton-Jones, et al. . Inspired by semiotics concepts, this simple, yet extensive ontology scoring metric evaluates an ontology based on four criteria - semantic, syntactic, social, and pragmatic levels. The semantic criteria evaluates the term meanings and word sense of each of the terms. The syntactic criteria assesses syntax of the ontology, while social criteria examines how other ontologist use the ontology through links. The pragmatic criteria evaluates the practicality of the ontology, relating to its usage and its construction. Also, the evaluation metric is flexible to allow individual tailoring if any of the criteria does not apply, as demonstrated in reference . The resulting final score is a value between 0 and 1.
Because of the initial design of the ontology, the scoring metric needed to be tailored to provide a progress indicator of its development. For example, social criteria which is dependent on other ontologies linking to VISO was not possible to determine, therefore, it was excluded from the complete calculation score.
Results and discussion
VISO metrics data
Object property count
Data property count
Unique individual count
Instances asserted from classes
Total knowledge triples
Instances of triples in VISO
Table 2 indicate the number of knowledge triples present in the current VISO version. Most of the triples were mainly of the “Vaccine may cause Reaction” type, with 54 instances of that triple. Total number of instances of triples in VISO numbered at 429 (derived from Tables 2 and 3), that includes the 15 instances of triples denoted by data property relationships from the table in Table 3.
Both the HermiT (version 1.3.8) and the FaCT+ reasoners with the Protégé editor did not reveal any discrepancies with this version of the model.
The final quality score for evaluation, Q, based on the tailored equation (4) calculated to 0.54. Observing the other components of the quality score, the semantic quality, E, amounted to 0.59. Pragmatic quality and syntactic quality equated to 0.75 (P) and 0.27 (S), respectively.
While the construction of VISO is in the early stages and covering 6 VISs, the quality score appears to reflect the early developmental state of the ontology. However, looking at the decomposition of the score can reveal some insight to VISO and possibly inform the future direction of the VISO development.
Observing specifics of the semantic quality, the “Clarity” component score (0.89) reveals less ambiguity among the terms. The “Interpertability” value (0.74) also indicates the use of meaningful terms in the ontology, and the inconsistency of the terms was low (0.04).
With pragmatic quality (0.75) of VISO, we ignored the “Relevance” quality due to unavailability for deployment in an application environment. “Comprehensiveness” quality (1.2) was exceedingly high due to being a large ontology with many classes and properties. Based on reports from two expert reviewers, VISO’s inaccuracy was relatively low at 0.30.
Syntactic quality revealed no violations with syntax, however, the “Richness” quality computed at 0.54, reflecting that VISO was utilizing a little over half of the syntactic features available.
Based on the results, the VISO’s strength lies in its pragmatic quality which indicates its overall usability based on the two of the three components described earlier - “Accuracy” and “Comprehensiveness”. The excluded component, “Relevance”, could provide a more holistic score in the future. Syntactic outcome was the weakest of the three aspects of VISO’s quality. This is partially due to the minimal usage of ontology features available. Focus on this aspect will warrant attention in future development of VISO.
Relating to Consistency, some terms, according to reviewers, were improperly used or required nuanced descriptions. One example is the term “Dosage” which is technically used to describe measurement of a vaccine and not a synonym for “dose”. Another example are property labels like “cause”. Some instances in the ontology should have been labeled “may cause” to imply a possible causation, rather than an expected outcome. Proper and precise term usage will be another focus that the next version of VISO will rectify.
Because children are protected by the vaccine, hospitalizations, and emergency visits for rotavirus have dropped dramatically. 
It is debatable whether the historical information may be useful to patients, or if summarized statements, which are naturally repetitious in these documents, could be integrated into the VISO model. In this initial version of VISO the repetitive and historical texts were not mapped but may be considered in future versions of the ontology.
Another challenge were gaps in the knowledge where the information was incomplete, brief, or needed medical understanding beyond the lay person. Some of the language in the documents may not be readily evident to a parent or patient, and would require a medical professional to provide interpretation. For example, if a certain vaccine should not be given to a child who is “moderately ill”, how does the reader of the VIS determine the exact signs of a “moderately ill” child as opposed to a child who may be “mildly ill”? If the documents refer to a sign as “physical weakness”, how does the patient or the reader determine features or indicators of “physical weakness”? Issues like these limited the scope of the VISO knowledge-base. In practice, the healthcare provider, rather than the parent, will determine if the child is too ill to receive the vaccine. It is the medical professional’s responsibility to provide judgment and guidance to the patient. Similarly, there were issues with limited number of VIS documents to develop the VISO models and to create instances. Lack of additional information resulted in some of the classes not having any subclasses to suggest.
Expressing the knowledge contained within the VIS documents posed some challenges as well. While the meta-level definition was designed with subclasses to handle descriptive instances, there are often passages with complex nouns and adjectives where each word carried important meaning for the instance. Examples such as “painful tightening of muscles” or “difficult for infants to breathe” posed a predicament of whether these instances should be decomposed to additional classes and relationships; use a subclass or create a new subclass; apply polymorphism; or keep as is as an instance. In most cases, they were realized as a single instance, until the meta-level model is further developed to map difficult passages. Moreover, given the historical nature of the VISs and considering that several VISs were originally developed more than 30 years ago in response to NCVIA, it is assumed that subsequent versions contain much of the original language. This likely contributes to the variability in the semantic language of the ontology.
Conclusion and future direction
Expand the Vaccine Information Statement Domain. This version of the VISO models 6 VIS documents as an initial iteration to examine the ontology. The next few iterations of VISO will include more domain knowledge from the remaining 19 VIS documents, available from the CDC website. There is also an awareness that additional knowledge can be modeled from outside the CDC’s VIS documents. It is also assumed that the meta-level design will mature as we realize alternative interpretation of vaccine information, or discover abstractions that could integrate some of the ignored passages and phrase, like negation statements or summary passages. With an expanded version and throughout the development cycle, we plan to evaluate the ontology using semiotic-based metric suite, and also adapt the suite to also include aspects previously excluded, like social quality or Relevance.
Link VISO with existing relevant ontologies to expand the knowledge domain. Some third party ontologies that could be aligned with the VISO may include the Vaccine Ontology (VO) , Ontology of Vaccine Adverse Effect (OVAE)  or Medical Dictionary for Regulatory Activities (MedDRA)  to address reaction or conditions in the VISO. This approach would comprise of code-linking particular classes with matching classes in the VISO model, which would lead into providing a comprehensive and expanded knowledge-base for patient learning. However, because the knowledge-base is intended for patient use, it will be essential to determine the appropriate since the aforementioned third party ontologies utilize professional vocabulary which may not be understood by patients.
Integrating patient-level synonymous terms and multi-language equivalents into the lexicon. A multi-lingual VISO would presumably expand to reach potential patients who may be excluded because of language or socio-economic barriers. There is also an interest in using the Open Access, Collaborative Consumer Health Vocabulary Initiative  as a resource to integrate consumer-level terms or synonyms.
Applying natural language processing. We intend to explore the possibility of applying natural language processing (NLP) for both ontology learning and knowledge retrieval. We will implement NLP methods to facilitate automatic knowledge extraction to expand the VISO. This will also provide intrinsic value for applications using natural language processing and dialogue systems.
Intelligent mobile agents. Ontology-driven applications could introduce the potential for intelligent agents for learning. Realistic and engaging agents are proven to be more effective for increasing involvement and learning [32-35], persuasion , and trustworthiness [37,38]. It provides a cost-effective way to address patient’s concerns and answer their questions about vaccination, which otherwise requires healthcare professionals to address in person. This will presumably improve the efficiency of healthcare delivery workflows and reduce the cost. They also provide flexibilities for vaccine education. Patients or parents can spend as much time as they need with intelligent mobile devices, and interact with intelligent devices anytime they have access to a computer or tablet. Last but not least, personalized agents can be automatically built according to users’ preference to improve the usability and acceptability of the system.
Examples like VAMATA, which is an ontology-driven mobile application with a speech interface designed for combat medical settings, reveal applicable synergy between natural language processing and ontology in mobile applications .
We have previously developed a proof-of-concept ontology-driven mobile application with a natural language interface to query a VISO knowledge-base . The ongoing evolution of VISO will assist in the continuing development of that project.
This project is partially supported by the National Library Of Medicine of the National Institutes of Health under Award Number R01LM011829.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Fox S, Rainie L. The online health care revolution: How the web helps americans take better care of themselves. Technical report, Pew Research Center 2000. http://www.pewinternet.org/2000/11/26/the-online-health-care-revolution/. Accessed 26 Nov 2014.
- Vaccine Information Statement: Facts About VISs - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/about/facts-vis.html. Accessed 9 Jul 2014.
- Lieu T, Glauber J, Fuentes-Afflick E, Lo B. Effects of vaccine information pamphlets on parents’ attitudes. Arch Pediatrics Adolescent Med. 1994; 148(9):921–5. doi:10.1001/archpedi.1994.02170090035004.View ArticleGoogle Scholar
- St-Amour M, Guay M, Perron L, Clément P, Baron G, Petit G, et al. Are vaccination information leaflets useful for vaccinators and parents?. Vaccine. 2006; 24(14):2491–6. doi:10.1016/j.vaccine.2005.12.026.View ArticleGoogle Scholar
- Redlich FC. The patient’s language: An investigation into the use of medical terms. The Yale J Biol Med. 1945; 17(3):427. PMID: 21434217. Accessed 2014-08-02.Google Scholar
- Zeng QT, Tse T. Exploring and developing consumer health vocabularies. J Am Med Inf Assoc: JAMIA. 2006; 13(1):24–9. doi:10.1197/jamia.M1761. PMID: 16221948 PMCID: PMC1380193 Accessed 6 Jul 2014.View ArticleGoogle Scholar
- Smith CA, Stavri PZ, Chapman WW. In their own words? a terminological analysis of e-mail to a cancer information service. In: Proceedings of the AMIA Symposium, 697 2002. PMID: 12463914. Accessed 2 Aug 2014.Google Scholar
- Adnan M, Warren J, Orr M. Ontology based semantic recommendations for discharge summary medication information for patients. In: Computer-Based Medical Systems (CBMS), 2010 IEEE 23rd International Symposium On. IEEE: 2010. p. 456–61. doi:10.1109/CBMS.2010.6042688.Google Scholar
- Cardillo E, Serafini L, Tamilin A. A lexical-ontological resource for consumer healthcare In: Kostkova P, editor. Electronic Healthcare. Berlin: Springer: 2010. p. 131–8.Google Scholar
- Bailin SC, Lehmann HB. Facilitating physician-patient dialogue through ontology negotiation. In: Computer-Based Medical Systems, 2003 Proceedings 16th IEEE Symposium. IEEE: 2003. p. 248–53.Google Scholar
- Beveridge M, Fox J. Automatic generation of spoken dialogue from medical plans and ontologies. J Biomed Inf. 2006; 39(5):482–99. doi:10.1016/j.jbi.2005.12.008. Accessed 6 Jul 2014.View ArticleGoogle Scholar
- Amith M, Tao C. A prototype mobile virtual assistant for semantic-based vaccine information retrieval In: Zheng X, Zeng D, Chen H, Zhang Y, Xing C, Neill D, editors. Smart Health. Lecture Notes in Computer Science. New York: Springer: 2014. p. 200–5. http://dx.doi.org/10.1007/978-3-319-08416-9_21.Google Scholar
- OWL 2 Web Ontology Language Document Overview (Second Edition). http://www.w3.org/TR/owl2-overview/ Accessed 9 Jul 2014.
- Protégé. http://protege.stanford.edu/ Accessed 9 Jul 2014.
- Yu L. A Developer’s Guide to the Semantic Web, 2011edition. Heidelberg; New York: Springer; 2011.View ArticleGoogle Scholar
- Vaccine Information Statement: Home Page - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/index.html.
- Vaccine Information Statement: Diphtheria, Tetanus, and Pertussis - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/vis-statements/dtap.html Accessed 9 Jul 2014.
- Vaccine Information Statement: Rotavirus - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/vis-statements/rotavirus.html Accessed 9 Jul 2014.
- Vaccine Information Statement: Hepatitis B - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/vis-statements/hep-b.html Accessed 9 Jul 2014.
- Vaccine Information Statement: Haemophilus Influenzae Type B (Hib) - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/vis-statements/hib.html Accessed 9 Jul 2014.
- Vaccine Information Statement: MMR (Measles, Mumps, Rubella) - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/vis-statements/mmr.html Accessed 9 Jul 2014.
- Vaccine Information Statement: Pneumococcal Conjugate - Vaccines - CDC. http://www.cdc.gov/vaccines/hcp/vis/vis-statements/pcv13.html Accessed 9 Jul 2014.
- Atkinson W, Wolfe C, Hamborsky J. Centers for Disease Control and Prevention. Epidemiology and Prevention of Vaccine-Preventable Diseases, 12th edn. Washington DC: Public Health Foundation; 2012.Google Scholar
- Representing Specified Values in OWL: “value Partitions” and “value Sets”. http://www.w3.org/TR/2004/WD-swbp-specified-values-20040803/ Accessed 11 Aug. 2014.
- Burton-Jones A, Storey VC, Sugumaran V, Ahluwalia P. A semiotic metrics suite for assessing the quality of ontologies. Data Knowledge Eng. 2005; 55(1):84–102. doi:10.1016/j.datak.2004.11.010. Accessed 15 Oct 2014.View ArticleGoogle Scholar
- Ws, 4j - WordNet Similarity for Java - Google Project Hosting. https://code.google.com/p/ws4j/ Accessed 7 Oct 2014.
- OWL API. http://owlapi.sourceforge.net/ Accessed 7 Oct 2014.
- VO: Vaccine Ontology. http://www.violinet.org/vaccineontology/ Accessed 13 Aug 2014.
- OVAE: The Ontology of Vaccine Adverse Events. http://www.violinet.org/ovae/introduction.php Accessed 9 Jul 2014.
- Medical Dictionary for Regulatory Activities - Summary | NCBO BioPortal. http://bioportal.bioontology.org/ontologies/MEDDRA Accessed 9 Jul 2014.
- Open Access, Collaborative Consumer Health Vocabulary Initiative. http://consumerhealthvocab.org Accessed 5 Jul 2014.
- Christel MG. The role of visual fidelity in computer-based instruction. Human-Computer Interact. 1994; 9(2):183.View ArticleGoogle Scholar
- Moreno R, Mayer RE, Spires HA, Lester JC. The case for social agency in computer-based teaching: do students learn more deeply when they interact with animated pedagogical agents?Cogn Instruction. 2001; 19(2):177–213. Accessed 9 Jul 2014.View ArticleGoogle Scholar
- Moundridou M, Virvou M. Evaluating the persona effect of an interface agent in a tutoring system. Journal of computer assisted learning. 2002; 18(3):253–61. Accessed 29 Jul 2014.View ArticleGoogle Scholar
- Hongpaisanwiwat C, Lewis M. Attentional effect of animated character. In: Proceedings of the human-computer interaction. iOS Press: 2003. p. 423–30. Accessed 29 Jul 2014.Google Scholar
- Guadagno RE, Blascovich J, Bailenson JN, Mccall C. Virtual humans and persuasion: the effects of agency and behavioral realism. Media Psychol. 2007; 10(1):1–22. Accessed 2014-07-29.Google Scholar
- Van Mulken S, André E, Müller J. An empirical study on the trustworthiness of life-like interface agents. In: HCI (2). New York: ACM: 1999. p. 152–6.Google Scholar
- Rickenberg R, Reeves B. The effects of animated characters on anxiety, task performance, and evaluations of user interfaces. In: Proceedings of the SIGCHI conference on human factors in computing systems. New York: ACM: 2000. p. 49–56. Accessed 29 Jul 2014.Google Scholar
- Rodger J, George J. Mobile speech and the armed services: making a case for adding siri-like features to vamta (voice-activated medical tracking application) In: Neustein A, Markowitz JA, editors. Mobile Speech and Advanced Natural Language Solutions. New York: Springer: 2013. p. 319–32.Google Scholar