A complete description is available at the Bio-ML documentation.
We accept direct result submission via this google forms based on trust. We will also release results for systems based on our implementations and for systems submitted via MELT. These three categories will be specified on the result tables.
Note: Click the column names (evaluation metrics) to sort the table; Cells with empty values suggest that the corresponding scores are not available.
The super-script symbols indicate that the results come from MELT-wrapped systems (†), and direct result submission (∗) . It is important to notice that direct result submissions are based on trust.
Note: New results of the submitted systems are being updated.
For equivalence matching, we report both the global matching and local ranking results.
For the global matching evaluation, the test mapping sets for unsupervised (not using training mappings) and semi-supervised (using training mappings) systems are different; the unsupervised test set is the full reference mapping set while the semi-supervised test set is the 70% reference mapping set (excluding 30% training mappings). Some systems may not use the training mappings (e.g., BERTMapLt, LogMap, etc.), but we still report their performances on the semi-supervised test set. The use of training mappings for the semi-supervised setting is indicated by ✔ (used) and ✘ (not used).
For the local ranking evaluation, we keep one ranking test set for both unsupervised and semi-supervised systems.