Noriko Kando

• PhD (Library & Information Science), Keio University
• Professor (Full) at National Institute of Informatics

Community Question Answering (CQA) sites such as Yahoo! Answers have emerged as rich knowledge resources for information seekers. However, answers posted to CQA sites can be irrelevant, incomplete, redundant, incorrect, biased, ill-formed or even abusive. Hence, automatic selection of "good" answers for a given posted question is a practical resear...

We focus on clarifying the unique characteristics of seeking behaviors after obtaining results from a search engine. We conducted an experimental study of Web information-seeking behaviors by comparing two different types of tasks by recording browser logs and screen capture video. The participants were 11 undergraduate and 5 graduate students. The...

Word Sense Disambiguation (WSD) often relies on a context model or vector constructed from the words that co-occur with the target word within the same text windows. In most cases, a fixed-sized window is used, which is determined by trial and error. In addition, words within the same window are weighted uniformly regardless to their distance to th...

The special issue of Transactions on Asian Language Information Processing (TALIP) discusses six research papers on Indian language Information Retrieval (IR). The first article, 'The FIRE 2008 Evaluation Exercise' by Prasenjit Majumder and co-workers, provides the motivation and background for the FIRE initiative. It describes how the FIRE 2008 te...

Argument mining has typically been researched for specific corpora belonging to concrete languages and domains independently in each research work. Human argumentation, however, has domain-and language-dependent linguistic features that determine the content and structure of arguments. Also, when deploying argument mining systems in the wild, we mi...

A spectrum of human-artificial intelligence collaboration in assessing relevance.

This study focuses on a method for differentiating between the stance of citizens and city councilors on political issues (i.e., in favor or against) and attempts to compare the arguments of both sides. We created a dataset by annotating citizen tweets and city council minutes with labels for four attributes: stance, usefulness, regional dependence...

This report documents the program and the outcomes of Dagstuhl Seminar 23031 "Frontiers of Information Access Experimentation for Research and Education", which brought together 38 participants from 12 countries. The seminar addressed technology-enhanced information access (information retrieval, recommender systems, natural language processing) an...

This paper proposes a method for enhancing museum visits by turning them into a personalized treasure hunt game. This approach uses gamification to augment visitor experiences, explicitly addressing the issue of retaining information during casual museum visits. This is achieved by creating quizzes customized to each visitor’s interests and providi...

Gathering citizen feedback, analyzing it, and comparing the results against other cities is essential for improving government policy and service quality. However, because different cities have different policies and services, the opinions of citizens in different cities also differ. This makes it difficult to analyze citizen opinions adapted for m...

This is a report on the NTCIR-16 conference held online in June 2022. NTCIR is a series of parallel and collective evaluation efforts designed to enhance research on diverse information access technologies, including, but not limited to, cross-language and multimedia information access, question-answering, text mining, and summarization. 53 active...

In local government, clarifying the gap between the political opinions of citizens and city councilors is important for reflecting the will of the people in politics. In this study, we focus on the difference between the stances (i.e., favor or against) of citizens and city councilors on political issues, and attempt to compare the arguments of bot...

Citizen feedback is essential for improving hospitality in government policies and customer services. In this study, we propose a method for extracting citizen feedback from social media according to appraisal opinion type by filtering tweets based on multiple viewpoints such as regional dependency, status of citizen, and polarity. To improve the F...

This is a report on the NTCIR-15 conference held online in December 2020. NTCIR is a sesquiannual research project designed to evaluate various information access technologies, including information retrieval, information recommendation, question answering, natural language processing, etc. 55 active research groups from 22 countries\regions have p...

This paper focuses on analyzing (Japanese) splogs based on various characteristics of keywords contained in them. We estimate the behavior of spammers when creating splogs from other sources by analyzing the characteristics of keywords contained in splogs. Since splogs often cause noises in word occurrence statistics in the blogosphere, we assume t...

The Third Workshop on Evaluation of Personalisation in Information Retrieval (WEPIR 2021) was held in conjunction with the ACM SIGIR Conference on Human Information Interaction & Retrieval (CHIIR 2021) in Canberra, Australia, as a virtual event. WEPIR 2021 followed on from the first and second WEPIRs held at CHIIR 2018 and 2019. The purpose of the...

This open access book summarizes the first two decades of the NII Testbeds and Community for Information access Research (NTCIR). NTCIR is a series of evaluation forums run by a global team of researchers and hosted by the National Institute of Informatics (NII), Japan. The book is unique in that it discusses not just what was done at NTCIR, but al...

The NTCIR-14 QA Lab-PoliInfo aims to achieve real-world complex question-answering (QA) technologies using Japanese political information, such as local assembly minutes and newsletters. QA Lab-PoliInfo has three tasks, namely, segmentation, summarization and classification. We describe the dataset used, formal run results, and comparison between h...

There has been little research considering eye movement as a measure when assessing user interactions with music information retrieval (MIR) systems, whereas many studies have adopted conventional user-centered measures such as user effectiveness and user perception. To bridge this research gap, this study investigates users' eye movement patterns...

Important issues, such as abortion governmental laws, are discussed everyday online involving different opinions that could be favorable or not. Often the debates change tone and become more aggressive undermining the discussion. In this paper, we analyze the relation between abusive language and the stances of disapproval toward some controversial...

This book constitutes the refereed proceedings of the 14th International Conference on NII Testbeds and Community for Information Access Research, NTCIR 2019, held in Tokyo, Japan, in June 2019. The 15 full papers presented in this book were carefully reviewed and selected from 55 submissions. This NTCIR 2019 proceedings was structured in the foll...

Academic papers cite other literature for various purposes: such as sharing research motivations, comparing with previous methods or achieved accuracies, and so on. At some annual conference, high impact papers have been selected and honored after about 10 years from the publication to demonstrate both the emphasis research field and great contribu...

The amount of procedural texts, which describe the steps to reach a goal or solve a problem, continues to increase. This paper proposes flow graph generation methods for visualizing procedural texts. Examples of typical procedural texts include cooking texts, operation manuals for appliances/devices, and technical papers and patents that consist of...

This is a report on the NTCIR-13 conference held in December 2017, in Tokyo, Japan. NTCIR is a series of parallel and collective evaluation efforts designed to enhance research on diverse information access technologies, including, but not limited to, cross-language and multimedia information access, question-answering, text mining and summarizatio...

People often conduct exploratory search to explore unfamiliar information space and learn new knowledge. While supporting the highly dynamic and interactive exploratory search is still challenging for the search system, we want to investigate which factors can make the exploratory search successful and satisfying from the user’s perspective. Previo...

Evaluation in empirical computer science is essential to show progress and assess technologies developed. Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the Cranfield paradigm of creating shared test collections, defining search tasks, and collecting ground truth for these...

Reading Comprehension tests are commonly used to assess the degree to which people comprehend what they read. This is why we work with the hypothesis that it is reasonable to use these tests to assess the degree to which a machine “comprehends” what it is reading. In this work, we evaluate Question Answering systems using Reading Comprehension test...

The purpose of the Strategic Workshop in Information Retrieval in Lorne is to explore the long-range issues of the Information Retrieval field, to recognize challenges that are on-or even over-the horizon, to build consensus on some of the key challenges, and to disseminate the resulting information to the research community. The intent is that thi...

With the rapid growth of mobile search, it is important to understand how users browse the mobile SERPs and allocate their limited attention to each result. To address this problem, we introduce a two-stage examination model that can separately capture the position bias with a skimming model and the attractiveness bias with an attractiveness model....

In this paper, to effectively visualize the browsing order of scientific articles, we propose a visualization method for citation networks focusing on the directed acyclic graph (DAG) structure. In our method, all article nodes are embedded into polar coordinate plane, where angular and radial coordinates express the citation relations and order re...

The volume of procedural texts, which describe the steps to reach a goal state, continue to increase. This paper proposes structural analysis methods for generating flow graphs from procedural texts. Examples of typical procedural texts include cooking texts, operation manuals for appliances/devices, and technical papers and patents that consist of...

Purpose Twitter reflects events and trends in users’ real lives because many of them post tweets related to their experiences. Many studies have succeeded in detecting events along with real-life information from a large amount of tweets by assuming users as social sensors. To collect a large amount of tweets based on specific users for successful...

There has been little research on task complexity and difficulty in music information retrieval (MIR), whereas many studies in the text retrieval domain have found that task complexity and difficulty have significant effects on user effectiveness. This study aimed to bridge the gap by exploring i) the relationship between task complexity and diffic...

The NII Testbeds and Community for Information access Research (ntcir) has been the first benchmarking campaign that created a test collection specifically for patent retrieval, in 2001/2002. Over the course of just over a decade, organisers and participants at NTCIR patent-related challenges have addressed the problem of mono- and multilingual pat...

Many people share their daily events and opinions on Twitter. Some tweets are beneficial and others are related to such aspects of a user's real-life as eating, traffic conditions, and weather. In this paper, we propose an inference method of the real-life aspect distribution of tweets using labeled tweets. Our method infers the aspect probability...

This is a report on the NTCIR-12 conference held in June 2016, in Tokyo, Japan. NTCIR-12 is the twelfth sesquiannual research project for evaluating information access technologies that organizes a diverse set of tasks related to information retrieval, question answering, and natural language processing. The NTCIR-12 conference is a venue in which...

This paper addresses the problem of identifying irrelevant items from a small set of similar documents using Web search engine suggests. Specifically, we collected volumes of Web pages through Web search engines and inspected the page contents using topic models. Among each cluster of pages sharing the same topic indicated by the topic model, our t...

This paper presents techniques of retrieving useful information from a mixture of Web pages collected from either question-answer sites (Q&A sites) or Web search engines. The proposed techniques are designed to discover the maximum possible amount of know-how knowledge from such collections of Web pages, where know-how knowledge is defined as text...

Twitter reflects events and trends in users' real lives because many of them post tweets related to their experiences. Many studies have succeeded in detecting events such as earthquakes and influenza epidemics, along with real-life information from a large amount of tweets, by assuming users as social sensors. On the other hand, inactive users who...

We propose a novel centrality measure that is called Content Centrality for a given network that considers the feature vector of each node generated from its posting activities in social media, its own properties and so forth, in order to extract nodes who have neighbors with similar features. We assume that nodes with similar features are located...

Thousands of different recipes are posted on recipe sites by consumers who often refer to them when they cook. Such users occasionally select new recipes. In this paper, we propose for users a recipe selection model composed of both preference and challenging viewpoints to appropriately predict recipes that users are more likely to cook next in con...

Purpose Many Twitter users post tweets that are related to their particular interests. Users can also collect information by following other users. One approach clarifies user interests by tagging labels based on the users. A user tagging method is important to discover candidate users with similar interests. This paper aims to propose a new user t...

Many Twitter users build various relationships through communication activity such as replies and retweets. For example, friends engage in conversations through replies. Fans unilaterally send many replies to celebrities. In this paper, we focus on such relationships between users. We assume that such relationships are classified into several patte...

The "Search as Learning" (SAL) workshop is focused on an area within the information retrieval field that is only beginning to emerge: supporting users in their learning whilst interacting with information content.

The Dagstuhl Seminar on "Reproducibility of Data-Oriented Experiments in e-Science", held on 24-29 January 2016, focused on the core issues and approaches to reproducibility of experiments from a multidisciplinary point of view, sharing the experience coming from several fields of computer science. In this paper, we discuss, summarize, and adapt th...

Evaluation in empirical computer science is essential to show progress and assess technologies developed. Several research domains such as information retrieval have long relied on systematic evaluation to measure progress: here, the Cranfield paradigm of creating shared test collections, defining search tasks, and collecting ground truth for these...

Many Twitter users post tweets that are related to their particular interests. Users can also collect information by following other users. One approach clarifies user interests by tagging labels based on the users. A user tagging method is important to discover candidate users with similar interests. Typical approaches estimate user interests with...

In this report, we summarize the outcome of the "Evaluation-as-a-Service" workshop that was held on the 5th and 6th March 2015 in Sierre, Switzerland. The objective of the meeting was to bring together initiatives that use cloud infrastructures, virtual machines, APIs (Application Programming Interface) and related projects that provide evaluation...

We propose a graph-based ranking method for query-biased summarization in a three-layer graph model consisting of document, sentence and word-layers. The model has a representation that fuses three kinds of heterogeneous information: part-whole relationships between different linguistic units, similarity using the overlap of the Basic Elements (BEs...

This chapter presents a framework of detecting bursty topics of correlated news and twitter, and discusses how to integrate the framework into government services. Especially, as a specific application of the proposed framework of detecting bursty topics of correlated news and twitter, this chapter gives an example of collecting news and twitter th...

In this paper we describe the Searching as Learning Workshop (SAL 2014) taking place at IIiX 2014 in Regensburg, Germany.

This paper first studies how to apply a topic model to Chinese and Japanese blog posts collected from a few hundred Chinese and Japanese bloggers and then to classify bloggers into topics. The estimated topics are exploited in the task of over viewing the Chinese and Japanese bloggers' concerns, opinions, and cultures. Those topics are also quite h...

We propose two methods for exploratory image search systems using gaze data for continuous learning of the users' interests and relevance calculation. The first system uses the fixation time over the images selected by gaze in the search results pages, whereas the second one utilizes the fixation time over the clicked images and fixations over the...

The application of word sense disambiguation (WSD) techniques to information retrieval (IR) has yet to provide convincing retrieval results. Major obstacles to effective WSD in IR include coverage and granularity problems of word sense inventories, sparsity of document context, and limited information provided by short queries. In this paper, to al...

In this paper, risk communication as it relates to the issue of personal information leakage is discussed with the help of participants using the Multiplex Risk Communicator support system. We propose an information acquisition method that is divided into three phases corresponding to cognition models by the clarity of the information acquisition o...

Query suggestions have become pervasive in modern web search, as a mechanism to guide users towards a better representation of their information need. In this article, we propose a ranking approach for producing effective query suggestions. In particular, ...

The first Asian Summer School in Information Access (ASSIA 2013) was held between 22nd and 24th June, 2013 in Tsukuba, Japan. The summer school offered 9 lectures in Information Retrieval, Web Search, and related topics, along with two panel discussions and a poster session. This reports a successful international summer school in Asia attracting a...

It is now possible to use various geographical open data sources such as GeoNames and Wikipedia to construct geographic information systems. In addition, these open data sources are integrated by the concept of Linked Open Data. There have been several attempts to identify links between existing data, but few studies have focused on the quality of...

Traditional ad hoc IR test collections were built using a relatively large pool depth (e.g. 100), and are usually assumed to be reusable. Moreover, when they are reused to compare a new system with another or with systems that contributed to the pools (“contributors”), an even larger measurement depth (e.g. 1,000) is often used for computing evalua...

The present article addresses an attempt to apply questions in university entrance examinations to the evaluation of textual entailment recognition. Questions in several fields, such as history and politics, primarily test the examinee’s knowledge ...

We propose a novel framework that enables multifaceted analysis of news articles. This system uses semantic annotated information (e.g., person, place) as facets and can be used to construct structured queries for comparing the differences between sets of articles.

In this work, we will discuss how to improve Wikilabel, an approach which makes use of titles in Wikipedia pages to generate labels for documents, by retooling ideas from story link detection (SLD). A comparison of our approach against Elastic Net, a powerful machine learner, on the real world data, finds the visible superiority of our approach ove...

This work is concentrated on analyzing possibilities for retrieving and presenting the information available on the web, without the explicit need to formulate precise queries. The research question of our work can be formulated as: "Is it possible to retrieve information without clearly formulated verbal query?" For this consortium we present our...

This paper proposes a prototype system called Gaze-Learning-Access-and-Search-Engine 0.1 (GLASE), which can perform image relevance ranking based on gaze data and within-session learning. We developed a search user interface that uses an eye-tracker as an input device and employed a relevance re-ranking algorithm based on the gaze length. The preli...

Recently, query suggestions have become quite useful in web searches. Most provide additional and correct terms based on the initial query entered by users. However, query suggestions often recommend queries that differ from the user's search intentions due to different contexts. In such cases, faceted query expansions and their usages are quite ef...

This paper focuses on two types of modeling of information flow in news stream, namely, burst analysis and topic modeling. First, when one wants to detect a kind of topics that are paid much more attention than usual, it is usually necessary for him/her to carefully watch every article in news stream at every moment. In such a situation, it is well...

User evaluation in the domain of Music Information Retrieval (MIR) has been very scarce, while algorithms and systems in MIR have been improving rapidly. With the maturity of system-centered evaluation in MIR, time is ripe for MIR evaluation to involve users. In this study, we compare user-centered measures to a system effectiveness measure on the...

Among various types of recent information explosion, that in news stream is also a kind of serious problems. This paper studies issues regarding topic modeling of information flow in multilingual news streams. If someone wants to find differences in the topics of Japanese news and Chinese news, it is usually necessary for him/her to carefully watch...

コミュニティ型質問応答サイト(CQA)は，ユーザが自身の状況に応じた情報を得ることができる新たな手段である．しかし投稿された回答の質は様々であるため，その中から良質な回答を選択する方法が求められている．そこで本研究は，まず Yahoo!知恵袋データにおける良質回答を人手で分析し，その結果に基づいて良質回答自動予測システムを構築した．具体的には，「恋愛相談」「パソコン」「一般教養」「政治」の4つのカテゴリからそれぞれ無作為に50問の質問を抽出し，判定者2名によって手作業で良質回答を決定した．次に，その結果を分析し，良質回答の特徴として「詳しさ」「根拠」「丁寧さ」に基づく機械学習システムを構築した．機械学習システムの評価結果は，「パソコン」と「一般教養」カテゴリでは判定者らを上回った．「恋愛相談...

Introduction. This paper reports on a method for closely observing, eliciting and visualizing exploratory search processes with embedded information encountering in context. Method. Exploratory Web search processes of sixteen participants for report task and trip task with thinking aloud in Japanese were captured in video, together with their eye-m...

In this study, we investigate the changes in users' mental representations of a topic during their exploratory search on the Web. We use concept maps to compare the users' knowledge before and after a search. Participants are required to gather information on the Web as preparation for a regular feature of a magazine on environment and trip topics....

This paper presents a study on the use of word context features for Word Sense Disambiguation (WSD). State-of-the-art WSD systems achieve high accuracy by using resources such as dictionaries, taggers, lexical analyzers or topic modeling packages. However, these resources are either too heavy or don’t have sufficient coverage for large-scale tasks...

Our final goal is to understand exploratory searches as four levels of search processes: search task, intent unit, search unit, and link unit. To complete these objectives, we used qualitative data to categorize participants’ information needs for search units and quantitatively analyzed whether differences in the information needs of search units...

Semantic questions require semantic processing to deliver results beyond bag-of-words search. Geo-temporal search concerns search which has both geographic and temporal constraints. In particular the search for events or to answer questions about events contains, often, specificity of location (where) and time (when). NTCIR-GeoTime, the first evalu...

Among other domains and topics on which some issues are frequently argued in the blogosphere, the domain of crime is one of the most seriously discussed by various kinds of bloggers. Such information on crimes in blogs is especially valuable for those who are not familiar with tips for preventing being victimized. This paper proposes a framework of...