Browse by author
Lookup NU author(s): Dr Phillip Lord
This is the authors' accepted manuscript of an article that has been published in its final definitive form by Elsevier BV, 2019.
For re-use rights please refer to the publisher's terms and conditions.
© 2018 Background: Classifying entities in domain ontologies under upper ontology classes is a recommended task in ontology engineering to facilitate semantic interoperability and modelling consistency. Integrating upper ontologies this way is difficult and, despite emerging automated methods, remains a largely manual task. Problem: Little is known about how well experts perform at upper ontology integration. To develop methodological and tool support, we first need to understand how well experts do this task. We designed a study to measure the performance of human experts at manually classifying classes in a general knowledge domain ontology with entities in the Basic Formal Ontology (BFO), an upper ontology used widely in the biomedical domain. Method: We recruited 8 BFO experts and asked them to classify 46 commonly known entities from the domain of travel with BFO entities. The tasks were delivered as part of a web survey. Results: We find that, even for a well understood general knowledge domain such as travel, the results of the manual classification tasks are highly inconsistent: the mean agreement of the participants with the classification decisions of an expert panel was only 51%, and the inter-rater agreement using Fleiss’ Kappa was merely moderate (0.52). We further follow up on the conjecture that the degree of classification consistency is correlated with the frequency the respective BFO classes are used in practice and find that this is only true to a moderate degree (0.52, Pearson). Conclusions: We conclude that manually classifying domain entities under upper ontology classes is indeed very difficult to do correctly. Given the importance of the task and the high degree of inconsistent classifications we encountered, we further conclude that it is necessary to improve the methodological framework surrounding the manual integration of domain and upper ontologies.
Author(s): Stevens R, Lord P, Malone J, Matentzoglu N
Publication type: Article
Publication status: Published
Journal: Journal of Web Semantics
Year: 2019
Volume: 57
Print publication date: 01/08/2019
Online publication date: 06/09/2018
Acceptance date: 26/08/2018
Date deposited: 12/11/2018
ISSN (print): 1570-8268
ISSN (electronic): 1873-7749
Publisher: Elsevier BV
URL: https://doi.org/10.1016/j.websem.2018.08.004
DOI: 10.1016/j.websem.2018.08.004
Altmetrics provided by Altmetric
