Content uploaded by Edgard Marx
Author content
All content in this area was uploaded by Edgard Marx on Apr 27, 2018
Content may be subject to copyright.
Content uploaded by Edgard Marx
Author content
All content in this area was uploaded by Edgard Marx on May 02, 2017
Content may be subject to copyright.
Content uploaded by Edgard Marx
Author content
All content in this area was uploaded by Edgard Marx on May 01, 2017
Content may be subject to copyright.
Semantic Search User Interface Patterns : An Introduction
Edgard Marx
University of Leipzig
Augustusplatz 10
04109, Leipzig, Saxony
marx@informatik.uni-
leipzig.de
Ali Khalili
VU University Amsterdam
The Netherlands
a.khalili@vu.nl
André Valdestilhas
University of Leipzig
Augustusplatz 10
04109, Leipzig, Saxony
valdestilhas@studserv.uni-
leipzig.de
ABSTRACT
Within the past few years, many patterns and principles have been proposed towards the enhancement of search user
interfaces and experience. However, to access and explore information efficiently is still significantly challenging.
Recently, we have seen the rise of a new kind of information retrieval approach, the so-called semantic search
systems. These systems promise more accurate results while exploring semantics of the data. Although there exist
several search user interfaces tailored to semantic search, there is still a lack of usability studies as well as good
practices. In this work, we discuss the applicability of traditional search user interfaces in semantic search systems.
Furthermore, we propose a new interaction model based on four patterns: Poli-Communicative, Discrete Display,
Heterogeneous Data-face and Dive in-place.
Keywords
Semantic Search User Interface, Interaction Design, Human-Computer Interaction, Semantic Search
1 INTRODUCTION
Although significant efforts have been devoted to re-
search and development of search engines, the search is
not a solved problem. In fact, users still find it challeng-
ing to access and explore information. With the advent
of the Internet, personal computers, mobile devices as
well as Smart TVs, the amount of information gener-
ated by users is increasing day by day. More and more
users resort to search engines to find the information
needed. Indeed, search engines are shaping how we ac-
cess and learn information. However, apart from the so-
phisticated algorithms behind search engines, there are
other aspects as important as a good search algorithm:
the user interface.
User interfaces are the gateway of the search engines,
they facilitate users to find, explore, and understand the
information. Since the inception of the first search en-
gine [3], many ideas have been developed to enable and
facilitate content access and exploration. Some of these
ideas have become repeatable good practices to solve
common problems and thereby have been established as
patterns. Examples of such patterns are Faceted Search
and Autocomplete.
Permission to make digital or hard copies of all or part of
this work for personal or classroom use is granted without
fee provided that copies are not made or distributed for profit
or commercial advantage and that copies bear this notice and
the full citation on the first page. To copy otherwise, or re-
publish, to post on servers or to redistribute to lists, requires
prior specific permission and/or a fee.
In this work, we discuss the applicability of search pat-
terns and propose an extension of those for semantic
search interfaces. By semantic search, we do not re-
strict our view to engines that make use of RDF data,
but to applications that try to understand the searcher’s
intent by using the contextual meaning of the terms in
the query. Moreover, we extend the concept of seman-
tics to visual aspects of the results been displayed to
the user. As it is shown by many researchers [5, 8, 9],
these aspects do influence the user’s cognitive percep-
tion and thus should be explored. Our aim is to develop
a semantic search user interface as part of the openQA
framework [7]. openQA is a framework designed for
fast and easy development of questions answering and
semantic search approaches. The remaining of this pa-
per is structured as follows. Section 2 discusses four
patterns for semantic search that we propose. Finally,
Section 3 concludes with an outlook of the future work.
2 A SEMANTIC SEARCH USER IN-
TERFACE
In this section, we present and illustrate four patterns –
(1) Poli-Communicative, (2) Discrete Display, (3) Het-
erogeneous Data-face and (4) Dive in-place – that ex-
tend the ten previously introduced patterns. Some of the
proposed patterns are (partially) implemented in some
of the existing semantic search applications, but they
were not previously defined [8]. The four patterns are
based on user experience as well as an analytical review
of the literature.
3
1
2
(a) Layout 1. Examples of 1 - 2 Discrete Display and 2
Heterogeneous Data-face patterns.
3
1
2
(b) Layout 2. Example of 3 Dive in-place pattern.
Figure 1: Examples of proposed semantic user interface patterns.
2.1 Poli-Communicative
The Poli-Communicative pattern is about the query
type and presentation of the result. Semantic search
systems are becoming smarter every day. In the past
24 years, search interfaces have changed from support-
ing simple text [3] to gestures [6]. However, a funda-
mental principle is that all aspects involved in a search
are related to the communication process. Communica-
tion “is the purposeful activity of information exchange
between two or more participants in order to convey
or receive the intended meanings through a shared sys-
tem of signs and semiotic rules"1. Thus communica-
tion is a complex process in which the goal is to trans-
port a message from one interlocutor to another. There
are two categories of communication: (1) verbal and
(2) non-verbal. Some of the non-verbal communica-
tions include, but are not restricted to, gestures, facial
expressions, eyes contact and even behavior. Further-
more, search interface users can have cognition prob-
lems or environmental disturbances such as noise en-
vironments that can affect their interaction and under-
standing. Thus, we propose that a good search inter-
face should be a good interlocutor, that is, be poli-
communicative.
The interface should be able to communicate efficiently
in different ways. An efficient search interface is able
to interpret different input formats and present the re-
sults in a comprehensive and customized manner to the
users. A good example is a teaching class. Teachers in a
classroom use a wide range of communication methods
to transmit the knowledge. For instance, when a teacher
asks a question to his students, those who raise their
1https://en.wikipedia.org/wiki/
Communication
hands are indicating that they want to answer the ques-
tion. The professor then uses his hand to indicate which
student should answer. In this scenario, the communi-
cation interchanges between verbal and gestures. Other
scenarios can impose interaction restrictions. The inter-
locutors involved in the communication process choose
unconsciously or consciously the most efficient com-
munication method which is available. The interaction
between the user and a search user interface should flow
likewise. The user should be able to use all the avail-
able communication methods for querying. The inter-
face, on the other hand, should present the result effi-
ciently. For instance, when being in a noisy environ-
ment, typing might be a better communication option.
However, when cooking or practicing sports, the use of
voice might be a better option.
The case of “Vossa mercê” Vossa mercê is a pronoun
in Portugues languge that means in yours concession
or in yours grace. During several years, this pronoun
has evolved to other forms, chronologically, vossemecê,
vosmecê,vancê and later você. In Brazil, its evolu-
tion did not stop which had generated other colloquial
forms, more often used in Minas Gerais state, as ocê
and cê. With the advent of keyword typing, the last col-
loquial form was expanded to just c. The case of Vossa
mercê is important to (1) show that efficient commu-
nication also affects the language, which, in this case,
means the use of the smallest number of symbols to ex-
press the same thing. We call it ‘express more and say
less’. Another important observation illustrated in this
case is that (2) communication through devices does not
use the same sort of symbols as speaking or writing and
(3) it usually tends to be more efficient. That makes the
design of search user interfaces a hard task.
We propose that semantic search interfaces should not
limit themselves to merely support Natural Language
techniques such as query expansion or better under-
standing complex queries, but be poli-communicative.
That is, displaying the information more efficiently, em-
bracing users with cognitive difficulties as well as the
many aspects of the communication process.
2.2 Discrete Display
Information can be more or less important for a given
input query. Krug et al. [5] argue that users spend a lit-
tle time reading most Web pages. Instead, they scan
them, looking for words or phrases that catch their
eye. Therefore, Krug concludes that clear visual hi-
erarchy is the best way to make a page easy to grasp
in a hurry. Most of the semantic search interfaces re-
strict the differentiation of result relevance by its posi-
tion in the resulting list. However, the layout is very
important for user’s interaction and perception, it has
to be with the organization of the data being displayed.
We propose that the relevance of an information should
not just affect its position, but also its style as well
as the mode it is being displayed with. For instance,
data with more importance should be displayed with
more details and more evidence (big fonts, big boxes,
big images) than other data. Google’s interface pro-
vides these capabilities to some extent. For instance,
when displaying the result for the query "Michael Schu-
macher" Google shows the required information on the
right while the documents sorted by relevance are on
the left side. However, the user might be looking for
the result in the right, but when displaying documents
on the left, Google is clearly giving more emphasis to
the documents.
This design pattern clears is incosistent with usability
studies [2, 9, 4]2that shows that user’s attention fol-
lows the same read/write patterns (F-Shaped Pattern).
That is, the user’s attention goes descending from left
to right and top to bottom. Furthermore, the documents
are being displayed with the same font and emphasis.
The only apparent difference is the position in the re-
sult list.
Figure 2 shows one of most emblematic examples of
Discrete Display, the newspaper. The newspaper dis-
plays information accordingly with its relevance. The
most relevant information occupied more space, have
bigger fonts and appears on the first pages. Figure 1(a)
depicts an example of 1 Discrete Display for seman-
tic search interfaces in which the most relevant result is
positioned on top with more evidence.
2https://www.nngroup.com/articles/
f-shaped- pattern-reading-web- content
Figure 2: Example of Discrete Display pattern in the
daily journal New York Times.
Figure 3: Layout 3. Google displaying results for 1994
F1 videos.
2.3 Heterogeneous Data-face
A good semantic search interface should support differ-
ent data presentation. Most of the data semantic search
interfaces do not take into consideration the type of
data being displayed. Heterogeneous Data-face (HD)
is an extension of the pattern Structured Results. The
main difference is that Structured Results is about hav-
ing structured data in the result page mainly focusing
on user’s intent, while HD is about displaying results
from different data sources in the resulting page–e.g.
video, audio, documents, structured information, and
image. Data can have different types and properties,
thereby are heterogeneous. In this sense, a good se-
mantic search interface must support different types of
data presentation. Many search interfaces already ex-
plore this concept such as Google, Yahoo, and Bing.
For instance, geographic data can be displayed on a
map while video can be displayed in a video canvas.
However, there is still a place for improvement. One
example is the result displayed in Google for the query
"videos of formula one"—Figure 3—in which there are
documents instead of the required videos.
The problem is that—for the previous query “videos of
formula one"—there is a need to switch between dif-
ferent tabs (Web, Video, among others) in order to ob-
tain the required information. One of the big challenges
is how to improve the way the content is being dis-
played. Although there are different contents related
to the query (e.g. Video, Web pages, and Structured
Data), the top one structured information is displayed
on the right and the top ten documents are listed on
the left. These patterns often repeat between semantic
search engines, but: (1) Why not display videos, im-
ages and other content related to the query? and; (2)
Why—instead of displaying the top one structured data
and top ten web pages—search engines do not display
the top ten results, independently of the data type?
We propose that semantic search interfaces should have
a heterogeneous data-face. Figure 1(a) shows a seman-
tic search interface displaying heterogeneous results, an
example of the pattern 2 HD. The given example also
implements a better version of Actionable Results than
the one used by Google in Figure 3. Different from the
previous models, it can promote a better interaction ex-
perience to users because all query related contents are
displayed in a single canvas where it can also be acti-
vated.
2.4 Dive in-place
Dive in-place is an extension of the pattern Actionable
Results. The idea behind this concept is that the inter-
face should be self-contained. That is, the user should
be allowed to explore the displayed information in-
place. Dive in-place is not the same as Faceted Search.
Faceted Search necessarily involves a change of the el-
ements in the view and is also known as Faceted Nav-
igation or Faceted Browsing, which involves the appli-
cation of filters. Dive in-place is followed close by the
design principle of Cooper et al. [1] that emphasizes as
a good design principle the reduction of places to go.
According to Cooper, reducing the number of screens,
pages, and modes on Web sites increases people’s abil-
ity to stay oriented.
Dive in-place involves the addition of new elements,
it allows a more detailed version of the content with-
out leaving the view. Some applications implement this
pattern as a modal window or as a more button, where
a detailed content is revealed to the user. However, a
good example of Dive in-place is the Google’s Image
tab which allows users to interact with the images by
giving an expanded version in-place. Different from the
Image tab, a bad example is the Map tab. When a user
requests the Map tab of the current search, the content
is displayed in a new page. A good behavior is to open
an extended view in-place in the same canvas in which
the user can interact with the content. Search engines,
usually force users to open the content in different win-
dows in order to find the information needed.
By switching back and forth in order to find the infor-
mation, the exploration can become tedious. Dive in-
place is a good pattern to avoid such an experience.
Figure 1(b) demonstrates the pattern Dive in-place in
action. In the given interface, an extended view of the
page content is shown enabling further exploration.
3 CONCLUSION AND FUTURE
WORK
In this work, we presented four patterns for semantic
search interface: (i) Poli-Communicative, (ii) Discrete
Display, (iii) Heterogeneous Data-face and (iv) Dive in-
place. As future work, we plan to do an extensible and
detailed evaluation as well literature review. We see
this work as the first step towards the enhancement of
semantic search user interfaces.
4 ACKNOWLEDGMENTS
This work was partly supported by a grant from the
German Research Foundation (DFG) for the project
Professorial Career Patterns of the Early Modern His-
tory: Development of a scientific method for research
on online available and distributed research databases
of academic history under the grant agreement No GL
225/9-1, by CNPq under the program Ciências Sem
Fronteiras process 200527/2012-6.
5 REFERENCES
[1] Alan Cooper and Robert Reimann. About Face
3.0: The Essentials of Interaction Design. Wiley &
Sons, 3rd revised edition edition, 2007.
[2] Edward Cutrell and Zhiwei Guan. What are you
looking for?: an eye-tracking study of informa-
tion usage in web search. In Proceedings of the
SIGCHI conference on Human factors in comput-
ing systems, pages 407–416. ACM, 2007.
[3] Alan Emtage and Peter Deutsch. Archie: An elec-
tronic directory service for the internet. In Pro-
ceedings of the Winter 1992 Usenix Conference,
pages 93–110, 1992.
[4] Laura Granka, Matthew Feusner, and Lori Lorigo.
Eyetracking in online search.
[5] Steve Krug. Don’t Make Me Think : A Common
Sense Approach to the Web. New Riders Publish-
ing, Berkeley, Calif., 2. ed. edition, 2005.
[6] Yang Li. Gesture Search: A Tool for Fast Mobile
Data Access. In Proceedings of the 23Nd Annual
ACM Symposium on User Interface Software and
Technology, UIST ’10, pages 87–96, New York,
NY, USA, 2010. ACM.
[7] Edgard Marx, Ricardo Usbeck, Axel-Cyrille
Ngomo Ngonga, Konrad Höffner, Jens Lehmann,
and Sören Auer. Towards an Open Question An-
swering Architecture. In SEMANTiCS 2014, 2014.
[8] Peter Morville and Jefferey Callender. Search Pat-
terns - Design for Discovery. O’Reilly, 2010.
[9] Jakob Nielsen and Kara Pernice. Eyetracking web
usability. New Riders, 2010.
