Skip to main content

Table 1 Aggregated results for global events

From: Towards cross-lingual alerting for bursty epidemic events

English only news (Worldwide)

model

Se

Sp

PPV

NPV

AlarmsA

DaysB

F1

C3

0.52

0.95

0.54

0.91

7.6

3.2

0.53

 

(0.44,0.59)

(0.94,0.96)

(0.46,0.61)

(0.93,0.96)

   

C2

0.42

0.97

0.57

0.92

5.1

3.1

0.48

 

(0.34,0.50)

(0.96,0.98)

(0.48,0.66)

(0.93,0.96)

   

W2

0.42

0.97

0.59

0.92

5.2

3.1

0.49

 

(0.34,0.50)

(0.96,0.98)

(0.49,0.68)

(0.93,0.95)

   

F-stat

0.67

0.88

0.45

0.85

16.2

4.0

0.54

 

(0.61,0.73)

(0.86,0.89)

(0.40,0.51)

(0.93,0.96)

   

EWMA

0.44

0.95

0.51

0.90

6.5

3.0

0.47

 

(0.37,0.52)

(0.94,0.96)

(0.42,0.59)

(0.92,0.95)

   

All language news (Worldwide)

model

Se

Sp

PPV

NPV

AlarmsA

DaysB

F1

C3

0.67

0.91

0.48

0.89

12.0

4

0.56

 

(0.59,0.73)

(0.90,0.93)

(0.41,0.54)

(0.95,0.97)

   

C2

0.54

0.95

0.49

0.91

7.1

3.7

0.51

 

(0.46,061)

(0.94,0.96)

(0.42,0.56)

(0.95,0.97)

   

W2

0.55

0.95

0.52

0.91

10.6

3.7

0.54

 

(0.47,0.63)

(0.94,0.96)

(0.44,0.60)

(0.94,0.97)

   

F-stat

0.87

0.80

0.45

0.80

26.6

5.3

0.60

 

(0.83,0.91)

(0.77,0.81)

(0.41,0.50)

(0.96,0.98)

   

EWMA

0.48

0.93

0.44

0.89

7.8

3.7

0.46

 

(0.40,0.56)

(0.92,0.94)

(0.36,0.52)

(0.93,0.95)

   
  1. Aggregated evaluation metrics for data sets e1 to e16 stratified by source language. The mean number of ProMED-mail alerts per 100 days was 7.4. A Model alarms per 100 days; B Mean number of days that alerts were given before ProMED-mail reports. Figures in parentheses show 95% CI.