Table 1 Inter-rater agreement on page reliability for each corpus: 5-classes.
From: Reliability and type of consumer health documents on the World Wide Web: an annotation study
Accuracy/ Kappa | Reliability: 5 classes | ||
|---|---|---|---|
Set\Raters | X-Y | X-Z | Y-Z |
IBS train | 0.47 / 0.30 | 0.33 / 0.12 | 0.40 / 0.19 |
IBS test | 0.33 / 0.11 | 0.40 / 0.25 | 0.43 / 0.28 |
MMED100 | 0.51 / 0.32 | 0.35 / 0.12 | 0.38 / 0.14 |