Skip to main content

Table 1 Methods for expert stakeholder engagement in ontology development

From: Why and how to engage expert stakeholders in ontology development: insights from social and behavioural sciences

Method Description Examples
Advisory Board and Working Groups Small and select group of domain experts evaluate progress of ontology development and providing unbiased strategic and scientific recommendations.
Delivered via in-person or online meetings, or by email
Exposure Ontology (EXO) [35]
Human Phenotype Ontology (HPO) [36]
Scientific Advisory Board of the Human Behaviour-Change Project [23]
Expert Panels
Feedback tasks Experts asked to provide feedback on the whole ontology or specified elements of it
Delivered in-person or using online survey tools (e.g. Qualtrics, Limesurvey, Google Forms).
Development of the Behaviour Change Intervention ontology (BCIO) within the Human Behaviour-Change Project [29, 30, 37]
Related example from social and behavioural sciences:
Development of COVID-19 research priorities in psychology [38]
Development of the Theories and Techniques Tool [39]
Self-Determination Taxonomy [40]
Consensus exercises Consists of two or more sequential rounds of questions and feedback to experts, aiming to achieve consensus.
Delivered using online consensus (e.g. Loomio, DelphiManager) or survey tools (e.g. Qualtrics)
Related examples from social and behavioural sciences:
Links between Behaviour Change Techniques and Mechanisms of Action [41]
Delphi consensus to identify priorities for methodological research in behavioural trials in health research [42]
Behaviour Change Techniques Taxonomy v1 [21]
Discussions Experts are invited to engage in online discussions with other experts on a specific topic, e.g. relationships between certain ontology entities.
Delivered using project websites or online consensus tools (e.g. Loomio, DelphiManager). It can be synchronous or asynchronous.
Cognitive Atlas [43]
Related examples from social and behavioural sciences:
Links between Behaviour Change Techniques and Mechanisms of Action [41]
Workshops Small-group discussions and presentation of specific proposals related to ontology development. Decisions are taken by the group during the workshop.
Delivered via in-person or online meetings
Computable Exposures Workshop [44]
NCI Semantic Competency Query Review [45]
Phenotype Ontologies Traversing All The Organisms (POTATO) workshop [46]
Workshops at International Conference of Biomedical Ontologies (ICBO)
Workshops at bi-annual Formal Ontologies in Information Systems
Related examples from social and behavioural sciences:
Current development of the E-cigarette ontology [47]