|
Evaluation: Reference standard
|
|
Manual annotations
|
40 (52%)
|
[11, 12, 32, 34,35,36, 38,39,40, 42, 43, 45, 47, 48, 51,52,53, 56, 59, 60, 62, 64, 70, 77,78,79,80,81,82, 84,85,86, 91,92,93, 96, 97, 99, 101, 103]
|
|
Existing annotated corpus
|
24 (31%)
|
[30, 33, 37, 41, 44, 46, 49, 55, 58, 63, 68, 69, 72,73,74,75,76, 83, 87, 90, 95, 98, 100, 104]
|
|
Existing EHR data
|
7 (9.1%)
|
[29, 50, 57, 61, 66, 71, 88]
|
|
Manual retrospective review
|
6 (7.8%)
|
[31, 65, 67, 89, 94, 102]
|
|
Evaluation: Validation
|
|
Hold-out validation
|
40 (52%)
|
[11, 12, 29, 31, 34, 37, 41, 42, 45, 48,49,50,51,52, 55, 56, 58,59,60, 63, 65, 68, 69, 74, 76, 79,80,81, 83, 84, 87, 88, 90, 94,95,96, 98, 99, 102, 104]
|
|
Cross-validation
|
12 (16%)
|
[32, 39, 44, 53, 57, 62, 66, 73, 78, 88, 99, 101]
|
|
External validation
|
9 (12%)
|
[30, 32, 35, 42, 45, 46, 48, 72, 100]
|
|
Solely external validation
|
5 (6.5%)
|
[30, 35, 46, 72, 100]
|
|
In addition to another type of validation
|
4 (5.2%)
|
[32, 42, 45, 48]
|
|
Not performed or not listed
|
22 (29%)
|
[33, 36, 38, 40, 43, 47, 61, 64, 67, 70, 71, 75, 77, 82, 85, 86, 89, 91,92,93, 97, 103]
|
|
Generalizability
|
|
Claimed
|
23 (30%)
|
[30,31,32, 35, 38, 45, 49, 51, 58, 59, 65, 73, 74, 78,79,80, 83, 85, 87, 94, 96, 97, 100]
|
|
Externally validated
|
5 (6.5%)
|
[30, 32, 35, 45, 100]
|
|
Comparison
|
|
Compared to other existing algorithms or models
|
24 (31%)
|
[30, 35, 39, 45,46,47, 49, 58, 60, 63, 64, 72, 75, 80, 83, 87, 90, 94, 95, 98,99,100,101, 104]
|
|
Tested difference in outcomes for statistical significance
|
4 (5.2%)
|
[35, 39, 60, 63]
|
