Development set (n = 30) | Test set (n = 68) | |||||||||
---|---|---|---|---|---|---|---|---|---|---|
Algorithm | Concept | Approach | UMLS2011 | UMLS2016 | UMLS2016D | DIRECT | UMLS2011 | UMLS2016 | UMLS2016D | DIRECT |
Named-entity recognition | Lung cancera | Implied | 0.698 | 0.541 | 0.920 | 0.920 | 0.814 | 0.725 | 0.828 | 0.828 |
Strict | 0.769 | 0.606 | 1.000 | 1.000 | 0.880 | 0.841 | 0.947 | 0.947 | ||
Non-small cell lung cancerb | Implied | 0.759 | 0.455 | 0.867 | 1.000 | 0.879 | 0.655 | 0.746 | 0.862 | |
Strict | 0.759 | 0.455 | 0.867 | 1.000 | 0.951 | 0.720 | 0.815 | 0.933 | ||
Recurrencec | 0.000 | 0.000 | 1.000 | 1.000 | 0.000 | 0.000 | 0.938 | 0.921 | ||
Relationship detection | Recurrent lung cancerd | 1.000 | 0.857 |