Conference Paper

Evaluation of Japanese Dialogue Processing Method Based on Similarity Measure Using tf· AoI

Hokkai-Gakuen University, Sapporo, Hokkaido, Japan
DOI: 10.1007/978-3-540-24630-5_45 Conference: Computational Linguistics and Intelligent Text Processing, 5th International Conference, CICLing 2004, Seoul, Korea, February 15-21, 2004, Proceedings
Source: DBLP


In this paper, we propose a Japanese dialogue processing method based on a similarity measure using tf· AoI(termfrequency × Amountof
Information). Keywords are specially used in a spoken dialogue system because a user utterance includes an erroneous recognition, filler
and a noise. However, when a system uses keywords for robustness, it is difficult to realize detailed differences. Therefore,
our method calculates similarity between two sentences without deleting any word from an input sentence, and we use a weight
which multiplies term frequency and amount of information(tf · AoI). We use 173 open data sets which are collected from 12,095 sentences in SLDB. The experimental result using our method has
a correct response rate of 67.1%. We confirmed that correct response rate of our method was 11.6 points higher than that of
the matching rate measure between an input sentence and a comparison sentence. Furthermore that of our method was 7.0 points
higher than that of tf · idf.

Full-text preview

Available from:
  • Source
    • "Baseline Baseline plus MAIQA Majority decision or Web-based verifier Input Figure 1. The subsystems of HOKUM Hybrid System baseline presented in [4] "
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper is a report from collective participation in NTCIR-5 Question Answering Challenge between researchers from Mie University, Hokkaido University and Otaru University of Commerce. Although our re- sults were not impressive, we would like to share our experiences with everyone who think about participat- ing in the challenge but is afraid of his or her lack of experience in the field. Understanding the prob- lems of QA from the practical side was very instruc- tive and gave us a stronger base for future trials. We briefly introduce our preparations and participation then conclude with analysis what can be simply done with freely available tools.
    Full-text · Conference Paper · Dec 2005