Skip to main content

Table 3 Agreement in terms of Cohen’s (\(\kappa\)) between annotators for the first round, for each label (cf. 3.1), and for the overall corpus. An agreement is moderate if the (\(\kappa\)) is higher than 0.6, and strong if higher than 0.8 (cf. [29])

From: Automatic classification of experimental models in biomedical literature to support searching for alternative methods to animal experiments

Labels

Kappa

Level Of agreement

invertebrates

0.82

almost perfect

in vivo

0.78

substantial

in silico

0.72

substantial

human

0.63

substantial

organs

0.62

substantial

immortal cell lines

0.49

moderate

primary cell lines

0.45

moderate

others

0.42

moderate

overall

0.62

substantial