ICFHR 2016 Competition on Recognition of On-line Handwritten Mathematical Expressions (ICFHR-CROHME-2016)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Formula Recognition: Isolated formula recognition from handwritten strokes (main task)
The difficulty to recognize mathematical expressions depends of the number of different symbols, number of allowed layouts and the used grammar. In 2016, the Formulas recognition task uses 101 symbols and the most complex grammar from CROHME 2014.
The competition defines an evaluation protocol :
- participants can use available training and validation datasets (and more)
- the candidate systems take as input an inkml file (without ground-truth) and have to write as output a LG (label graph) file which provides the symbol segmentation, recognition and their layout.
- the provided evaluation tool compare the resulting LG files with the ground-truth. For the official competition, a website was available to compute the final evaluation.
The evaluation tool compute recall and precision of labels at different levels: primitives (strokes), objets (symbols) and expressions. Furthermore, metrics which merge these measurements are available (D_E...). A special tool computes confusion matrices at structure level which allow a deep and fine analysis of the systems.