No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Uncovering Hidden Qualities – Benefits of Quality Measures for Automatically Generated Metadata
Abstract
Today, digital libraries more and more have to rely on semantic techniques during the workflows of metadata generation, search
and navigational access. But, due to the statistical and/or collaborative nature of such techniques, the underlying quality
of automatically generated metadata is questionable. Since data quality is essential in digital libraries, we present a user
study on one hand evaluating metrics for quality assessment, on the other hand evaluating their benefit for the individual
user during interaction. To observe the interaction of domain experts in the sample field of chemistry, we transferred the
abstract metrics’ outcome for a sample semantic technique into three different kinds of visualizations and asked the experts
to evaluate these visualizations first without, later augmented with the quality information. We show that the generated quality
information is indeed not only essential for data quality assurance in the curation step of digital libraries, but will also
be helpful for designing intuitive interaction interfaces for end-users.
ResearchGate has not been able to resolve any citations for this publication.
Tagclouds are visual presentations of a set of words, typically a set of "tags" selected by some rationale, in which attributes of the text such as size, weight, or color are used to represent features, such as frequency, of the associated terms. This note describes two studies to evaluate the effectiveness of differently constructed tagclouds for the various tasks they can be used to support, including searching, browsing, impression formation and recognition. Based on these studies, we propose a paradigm for evaluating tagclouds and ultimately guidelines for tagcloud construction.
JeromeDL is a Semantic Digital Library engine. It uses Se- mantic Web and Social Networking technologies to improve browsing and searching for resources. With JeromeDL's social and semantic services every library user can bookmark interesting books, articles, or other ma- terials in semantically annotated directories. Users can allow others to see their bookmarks and annotations and share their knowledge within a social network. JeromeDL can also treat a single library resource as a blog post. Users can comment the content of the resource and reply to others' comments and this way create new knowledge. All data stored in the library is available in RDF format for querying and processing by other applications. Also the result of every search or browse action is immediately available as a link to RDF. Innovative MBB (MultiBeeBrowse) component oers multifaceted navigation and provides SOA for integration with other applications. TagsTreeMaps al- lows to easily filter out resources using clustered tags and presented with treemaps layout. Users can also navigate through the presented search results using SIMILE Exhibit component. The home page of the project is http://www.jeromedl.org and a demo can be found at http://bleedingedge.jeromedl.org.
Using keyword search to find relevant objects in digital libraries often results in way too large result sets. Based on the metadata associated with such objects, the faceted search paradigm allows users to structure and filter the result set, for example, using a publication type facet to show only books or videos. These facets usually focus on clear-cut characteristics of digital items, however it is very difficult to also organize the actual semantic content information into such a facet. The Semantic GrowBagapproach, presented in this paper, uses the keywords provided by many authors of digital objects to automatically create light-weight topic categorization systems as a basis for a meaningful and dynam- ically adaptable topic facet. Using such emergent semantics enables an alternative way to filter large result sets according to the objects' content without the need to manually classify all objects with respect to a pre-specified vocabulary. We present the details of our algorithm using the DBLP collection of computer sci- ence documents and show some experimental evidence about the quality of the achieved results.
Managing and distributing published information is traditionally the mission of libraries. But in times of digital information provisioning and perso-nalized content delivery, the processes to fulfill this mission have to be re-considered. Beyond simple keyword indexing using library categorization sys-tems, digital corpora need to be preprocessed for later access directly by the end user. Thus, major functions of the classical librarian like assessing the actual in-formation need and mediating between the library categorization and the end user are to some degree bypassed and have to be compensated for. Moreover, also the quality control of a digital library's metadata annotations used for sub-sequent querying of collections has to be guaranteed. In this paper we discuss the importance of metadata quality control for large eBook collections.
In digital libraries semantic techniques are often deployed to reduce the expensive manual overhead for indexing documents, maintaining metadata, or caching for future search. However, using such techniques may cause a de- crease in a collection's quality due to their statistical nature. Since data quality is a major concern in digital libraries, it is important to be able to measure the (loss of) quality of metadata automatically generated by semantic techniques. In this paper we present a user study based on a typical semantic technique used for automatic metadata creation, namely taxonomies of author keywords and tag clouds. We observed experts assessing typical relations between keywords and documents over a small corpus in the field of chemistry. Based on the eval- uation of this experiment, we focused on communalities between the experts' perception and thus draw a first roadmap on how to evaluate semantic tech- niques by proposing some preliminary metrics.
Tag clouds have become a popular visualization and navigation interface on the Web. Despite their popularity, little is known
about tag cloud perception and performance with respect to different user goals. This paper presents results from a comparative
study of several tag cloud layouts. The results show differences in task performance, leading to the conclusion that interface
designers should carefully select the appropriate tag cloud layout according to the expected user goals. Furthermore, the
analysis of eye tracking data provides insights into the visual exploration strategies of tag cloud users.
The weighted list, known popularly as a `tag cloud', has appeared on many popular folksonomy-based web-sites. Flickr, Delicious, Technorati and many others have all featured a tag cloud at some point in their history. However, it is unclear whether the tag cloud is actually useful as an aid to finding information. We conducted an experiment, giving participants the option of using a tag cloud or a traditional search interface to answer various questions. We found that where the information-seeking task required specific information, participants preferred the search interface. Conversely, where the information-seeking task was more general, participants preferred the tag cloud. While the tag cloud is not without value, it is not sufficient as the sole means of navigation for a folksonomy-based dataset.
The folksonomy tag cloud: when is it useful?Getting our head in the clouds: toward evaluation studies of tagcloudsComparison of Tag Cloud Layouts: Task- Related Performance and Visual Exploration
- J Sinclair
- M Cardew-Hall
- A W Rivadeneira
- D M Gruen
- M J Muller
- D R Millen
- S Lohmann
- J Ziegler
- L Tetzlaff
J. Sinclair and M. Cardew-Hall, "The folksonomy tag cloud: when is it useful?,"
Journal of Information Science, vol. 34, 2007, pp. 15-29.
[7]
A.W. Rivadeneira, D.M. Gruen, M.J. Muller, and D.R. Millen, "Getting our head in
the clouds: toward evaluation studies of tagclouds," Conference on Human Factors in
Computing Systems, 2007.
[8]
S. Lohmann, J. Ziegler, and L. Tetzlaff, "Comparison of Tag Cloud Layouts: Task-
Related Performance and Visual Exploration," 12th IFIP TC 13 International Conference, Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 392-404.