Summary: A Test Suite for Inference Involving Adjectives
Marilisa Amoia # , Claire Gardent +
# University of Saarland
Campus Scientifique BP 239, VandoeuvrelesNancy, France
Recently, most of the research in NLP has concentrated on the creation of applications coping with textual entailment. However, there
still exist very few resources for the evaluation of such applications. We argue that the reason for this resides not only in the novelty
of the research field but also and mainly in the difficulty of defining the linguistic phenomena which are responsible for inference.
As the TSNLP project has shown test suites provide optimal diagnostic and evaluation tools for NLP applications, as contrary to text
corpora they provide a deep insight in the linguistic phenomena allowing control over the data. Thus in this paper, we present a test suite
specifically developed for studying inference problems shown by English adjectives. The construction of the test suite is based on the
deep linguistic analysis and following classification of entailment patterns of adjectives and follows the TSNLP guidelines on linguistic
databases providing a clear coverage, systematic annotation of inference tasks, large reusability and simple maintenance. With the design
of this test suite we aim at creating a resource supporting the evaluation of computational systems handling natural language inference
and in particular at providing a benchmark against which to evaluate and compare existing semantic analysers.