Skip to main content

Table 9 Learn to rank on test data

From: Concept selection for phenotypes and diseases using learn to rank

Learn to rank performance

Individual system contribution

Top-K

Strategy

Model

P

R

F1

M1

M2

M3

M4

M5

M6

M7

M8

M9

Top-1

Union

SVMRank

0.1712

0.6426

0.2703

0.17

0.04

0.50

-

-

0.01

0.00

-

0.28

  

ListNet

0.1271

0.6170

0.2108

0.65

0.04

0.26

0.05

-

0.00

-

-

-

  

RankNet

0.0923

0.5096

0.1562

1.00

-

-

-

-

-

-

-

-

  

RankBoost

0.1408

0.6524

0.2316

0.43

0.05

0.50

-

-

0.01

0.28

0.28

0.28

 

Oracle

SVMRank

0.1712

0.6426

0.2703

0.17

0.04

0.50

-

-

0.01

0.00

-

0.28

  

ListNet

0.1271

0.6170

0.2108

0.65

0.04

0.26

0.05

-

0.00

-

-

-

  

RankNet

0.0923

0.5096

0.1562

1.00

-

-

-

-

-

-

-

-

  

RankBoost

0.1872

0.6504

0.2907

0.23

0.05

0.51

-

-

0.01

0.20

0.00

-

Top-2

Union

SVMRank

0.1244

0.6986

0.2112

0.51

0.07

0.62

-

-

0.01

0.00

0.28

0.50

  

ListNet

0.1107

0.7109

0.1915

0.87

0.07

0.88

0.13

0.03

0.02

0.00

-

-

  

RankNet

0.1070

0.7028

0.1857

1.00

-

1.00

-

-

-

-

-

-

  

RankBoost

0.1188

0.7034

0.2032

0.53

0.09

0.62

0.01

0.01

0.28

0.76

0.76

0.76

 

Oracle

SVMRank

0.2350

0.6869

0.3501

0.07

0.00

0.63

-

-

0.00

0.00

-

0.29

  

ListNet

0.2534

0.6981

0.3718

0.14

0.00

0.77

0.06

0.02

0.00

0.00

-

-

  

RankNet

0.2629

0.6905

0.3808

0.15

-

0.85

-

-

-

-

-

-

  

RankBoost

0.2420

0.6908

0.3584

0.10

0.00

0.58

-

0.01

0.02

0.29

0.01

-

Top-3

Union

SVMRank

0.1157

0.7081

0.1990

0.53

0.10

0.63

-

0.01

0.28

 

0.50

0.93

  

ListNet

0.1019

0.7287

0.1788

0.91

0.57

0.92

0.44

0.09

0.06

0.02

0.00

-

  

RankNet

0.1029

0.7045

0.1796

1.00

1.00

1.00

-

-

-

-

-

-

  

RankBoost

0.1128

0.7109

0.1947

0.54

0.11

0.64

0.29

0.22

0.75

0.98

0.98

0.98

 

Oracle

SVMRank

0.2444

0.6933

0.3615

0.06

0.00

0.59

-

 

0.01

0.00

-

0.33

  

ListNet

0.2773

0.7126

0.3993

0.11

0.00

0.72

0.12

0.04

0.01

0.00

-

-

  

RankNet

0.2643

0.6914

0.3824

0.15

0.01

0.85

-

-

-

-

-

-

  

RankBoost

0.2593

0.6956

0.3777

0.09

0.00

0.57

-

0.10

0.02

0.22

0.00

-

  1. Macro precision, recall and F1 at different top K levels. The highest scoring system F1 for each level (both union and oracle strategies) is shown in bold. The table also shows the individual contribution of the systems to the final score where italics scores indicate the highest contributing individual system(s) to each ensemble.