Signature Verification and Writer Identification Competitions for On- and Offline Skilled Forgeries (SigWiComp2013)

Keywords

Forensic analysis, signatures, handwriting, off-line, on-line, verification, identification, evaluation, likelihood ratios

Description

The objective of this competition is to allow researchers and practitioners from academia and industries to compare their performance in signature verification on new unpublished forensic-like datasets (Dutch, Japanese). Skilled forgeries and genuine signatures were collected while writing on a paper in some cases it was attached to a digitizing tablet. The collected signature data are available in an offline format and some signatures are also available in online format. Participants can choose to compete on the online data or offline data only, or can choose to combine both data formats.

Similar to the last ICDAR competition, our aim is to compare different signature verification algorithms systematically for the forensic community, with the objective to establish a benchmark on the performance of such methods (providing new unpublished forensic-like datasets with authentic and skilled forgeries in both on- and offline format).

The Forensic Handwriting Examiner (FHE) weighs the likelihood of the observations given (at least) two hypotheses:

• H1: The questioned signature is an authentic signature of the reference writer;
• H2: The questioned signature is written by a writer other than the reference writer;

The interpretation of the observed similarities/differences in signature analysis is not as straightforward as in other forensic disciplines such as DNA or fingerprint evidence, because signatures are a product of a behavioral process that can be manipulated by the reference writer himself, or by another person than the reference writer. In this competition, we ask to produce two probability scores: The probability of observing the evidence (e.g. a certain similarity score) given H1 is true, and the probability of observing the evidence given H2 is true. In this competition only such cases of H2 exist, where the forger is not the reference writer. This continues the last successful competitions on ICDAR 2009 and 2011.

Technical Details

General information

Note that the password for opening the data files is mentioned at the end of the Disclaimer: “IherebyaccepttheSigWiComp2013disclaimer” (without quotes).

Description of the Tasks

The tasks are defined according to the two handwritten modalities, i.e., signatures and text. In particular, we offer three tasks for the signatures modality and one task for the handwritten text modality.

Modality: Signatures

We provide offline and online signature data. The offline datasets are constituted of PNG images, scanned at 400 dpi, RGB color. The online dataset consists of ascii files with the format: X, Y, and pressure. Sampling rate 200 Hz, resolution 2000 lines/cm, precision of 0.25 mm. Collection device: WACOM Intuos3 A3 Wide USB Pen Tablet. Collection software: MovAlyzer. A preprinted paper was used with 12 numbered boxes (width 59mm, height 23mm). The preprinted paper was placed underneath the blank writing paper. Four extra blank pages were added underneath the first two pages to ascertain a soft writing surface.

Here the aim is,

"given a handwritten signature and a bunch of reference signatures, classify the signature as being forged or genuine."

We define the following three tasks for the signatures modality.

Task SigDutch (Dutch Signatures: Offline):

• Training set: Data from SigComp2009 and SigComp2011 Competitions;
• Evaluation set: 27 specimen writers; each with 10 reference signatures, 10 corresponding forgeries;

Task SigJapanese (Japanese Signatures: Offline):

• Training set: 11 specimen authors; each with 42 reference signatures, 36 corresponding forgeries;
• Evaluation set: 20 specimen authors; each with 42 reference signatures, 36 corresponding forgeries;

Task SigJapanese (Japanese Signatures: Online):

• Training set: 11 specimen authors; each with 42 reference signatures, 36 corresponding forgeries;
• Evaluation set: 20 specimen authors; each with 42 reference signatures, 36 corresponding forgeries;

Modality: Handwritten Text

Task Wi (Writer Identification & Retrieval):

We have data available at the NFI in the form of English handwriting (including the London Letter). The total set consists of the writings of 55 subjects. Each writer produced 6 texts and wrote them in different writing styles. We plan to focus on linking different handwriting writing styles from one writer which is a real challenge – and forensically relevant!

In particular, the aim of this task is,

"given a handwritten text, retrieve the texts which were produced by the same writer."

• Training set: three samples of handwritten text produced by each of the 55 writers (totaling 165 sample texts);
• Evaluation set: another three samples of handwritten text produced by the same 55 writers (totaling 165 sample texts);

Evaluation of the Tasks

Evaluation of the Sig Tasks

The system will get as an input parameter the mode (online, offline or both), the questioned signature and up to N authentic signatures from the reference writer (Note N depends on the task N<=50). In this competition we ask to produce a comparison score (e.g. a degree of similarity or difference), and the evidential value of that score, expressed as the ratio of the probabilities of finding that score when Hypothesis 1 is true and when Hypothesis 2 is true (i.e. the likelihood ratio). We will calculate the Cllr as in 2011 and 2012 (contact Muhammad Imran Malik {muhammad.malik@dfki.de} for any details required about the likelihood ratios).

Evaluation of the Wi Task

The algorithm for handwriting comparison does not have to deal with the possibility of forgery. Thus, the results should be reported differently than those of the signature comparisons. We will calculate the standard precision and recall.