Automated SNOMED CT concept and attribute relationship detection through a web-based implementation of cTAKES

Kersloot, Martijn G.; Lau, Francis; Abu-Hanna, Ameen; Arts, Derk L.; Cornet, Ronald

doi:10.1186/s13326-019-0207-3

Journal of Biomedical Semantics

Table 3 Precision, recall and calculated F-scores from the evaluation outcomes

From: Automated SNOMED CT concept and attribute relationship detection through a web-based implementation of cTAKES

				Development set (n = 30)			Test set (n = 68)
Implementation	Algorithm	Concept	Approach	Precision	Recall	F-score	Precision	Recall	F-score
UMLS2011	Named-entity recognition	Lung cancer^a	Implied	0.938	0.556	0.698	1.000	0.686	0.814
cTAKES with UMLS 2011		Lung cancer^a	Strict	0.938	0.652	0.769	0.943	0.825	0.880
		Non-small cell lung cancer^b	Implied	0.917	0.647	0.759	0.967	0.806	0.879
		Non-small cell lung cancer^b	Strict	0.917	0.647	0.759	0.967	0.935	0.951
		Recurrence^c		0.000	0.000	0.000	0.000	0.000	0.000
UMLS2016	Named-entity recognition	Lung cancer^a	Implied	1.000	0.370	0.541	1.000	0.569	0.725
cTAKES with UMLS 2016		Lung cancer^a	Strict	1.000	0.435	0.606	1.000	0.725	0.841
		Non-small cell lung cancer^b	Implied	1.000	0.294	0.455	0.947	0.500	0.655
		Non-small cell lung cancer^b	Strict	1.000	0.294	0.455	0.947	0.581	0.720
		Recurrence^c		0.000	0.000	0.000	0.000	0.000	0.000
UMLS2016 + Dict.	Named-entity recognition	Lung cancer^a	Implied	1.000	0.852	0.920	1.000	0.706	0.828
cTAKES with UMLS 2016 and custom dictionary		Lung cancer^a	Strict	1.000	1.000	1.000	1.000	0.900	0.947
		Non-small cell lung cancer^b	Implied	1.000	0.765	0.867	0.957	0.611	0.746
		Non-small cell lung cancer^b	Strict	1.000	0.765	0.867	0.957	0.710	0.815
		Recurrence^c		1.000	1.000	1.000	0.882	1.000	0.938
DIRECT	Named-entity recognition	Lung cancer^a	Implied	1.000	0.852	0.920	1.000	0.706	0.828
cTAKES with UMLS 2016, custom dictionary, and additional processing		Lung cancer^a	Strict	1.000	1.000	1.000	1.000	0.900	0.947
		Non-small cell lung cancer^b	Implied	1.000	1.000	1.000	0.966	0.778	0.862
		Non-small cell lung cancer^b	Strict	1.000	1.000	1.000	0.966	0.903	0.933
		Recurrence^c		1.000	1.000	1.000	0.879	0.967	0.921
	Attribute relationship detection	Recurrent lung cancer^d		1.000	1.000	1.000	1.000	0.750	0.857

^aSNOMED CT concept 93880001 | Primary malignant neoplasm of lung (disorder)
^bSNOMED CT concept 254637007 | Non-small cell lung cancer (disorder)
^cSNOMED CT concept 255227004 | Recurrent (qualifier value)
^dRelationship between three SNOMED CT concepts: 93880001 | Primary malignant neoplasm of lung (disorder) |: 263502005 | Clinical course (attribute) | = 255227004 | Recurrent (qualifier value)

Back to article page

ISSN: 2041-1480

Contact us

General enquiries: journalsubmissions@springernature.com