Skip to main content

Table 7 Evaluation outcomes of the different implementations and datasets

From: Automated SNOMED CT concept and attribute relationship detection through a web-based implementation of cTAKES

  Named-entity recognition Attribute relation detection
Implementation Set Approach Outcome Lung cancera Non-small cell lung cancerb Recurrencec Recurrent lung cancerd
Present Absent Present Absent Present Present Absent
1 Development set (n = 30) Relaxed Detected 15 1 11 1 0   
cTAKES with UMLS 2011 Not detected 12 2 6 12 15   
Strict Detected 15 1 11 1    
Not detected 8 6 6 12    
Test set (n = 68) Relaxed Detected 35 0 29 1 0   
Not detected 16 17 7 31 30   
Strict Detected 33 2 29 1    
Not detected 7 26 2 36    
2 Development set (n = 30) Relaxed Detected 10 0 5 0 0   
cTAKES with UMLS 2016 Not detected 17 3 12 13 15   
Strict Detected 10 0 5 0    
Not detected 13 7 12 13    
Test set (n = 68) Relaxed Detected 29 0 18 1 0   
Not detected 22 17 18 31 30   
Strict Detected 29 0 18 1    
Not detected 11 28 13 36    
3 Development set (n = 30) Relaxed Detected 23 0 13 0 15   
cTAKES with UMLS 2016 and custom dictionary Not detected 4 3 4 13 0   
Strict Detected 23 0 13 0    
Not detected 0 7 4 13    
Test set (n = 68) Relaxed Detected 36 0 22 1 30   
Not detected 15 17 14 31 0   
Strict Detected 36 0 22 1    
Not detected 4 28 9 36    
4 Development set (n = 30) Relaxed Detected 23 0 17 0 15 6 0
DIRECT cTAKES with UMLS 2016, custom dictionary, and post-processing Not detected 4 3 0 13 0 0 10
Strict Detected 23 0 17 0    
Not detected 0 7 0 13    
Test set (n = 68) Relaxed Detected 36 0 28 1 29 9 0
Not detected 15 17 8 31 1 3 18
Strict Detected 36 0 28 1    
Not detected 4 28 3 36    
  1. aSNOMED CT concept 93880001 | Primary malignant neoplasm of lung (disorder)
  2. bSNOMED CT concept 254637007 | Non-small cell lung cancer (disorder)
  3. cSNOMED CT concept 255227004 | Recurrent (qualifier value)
  4. dRelationship between three SNOMED CT concepts: 93880001 | Primary malignant neoplasm of lung (disorder) |: 263502005 | Clinical course (attribute) | = 255227004 | Recurrent (qualifier value)