No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Effect of Sense of Direction on Open Card Sorts for Websites
Informal caregivers play an essential role in caring for persons who require assistance and in managing the health of their loved ones. Unfortunately, they need more health, leisure, and relaxation time. Nature interaction is one of many kinds of self-care intervention. It has long been regarded as a refreshing break from stressful routines, and research suggests exposure to nature interventions to improve the quality of life of caregivers. Despite not being the real thing, technology allows us alternatives that can still have some beneficial effects. In this preliminary study, we explore the benefits of natural environment videos on informal caregivers as an alternative to exposure to nature. Specifically, we are interested in the effects of their own choices versus a random video. We found that natural environment videos improve the well-being of informal caregivers in at least three key areas: valence, arousal, and negative affect. Furthermore, the effect increases when they choose the video they want to watch instead of a random video. This effect benefits the studied subjects because they need more time and energy to visit real natural environments.KeywordsInformal caregiversSelf-careWell-beingNature videos
Open card sorting is the most used method for developing user-centered information architectures. One important question for every HCI method is how many users to involve. Existing studies that address this question for open card sorts have involved trained professionals sorting content items of rather specialized domains. In addition, they employ data analysis approaches that might decrease the confidence one can place on the reported findings. This paper investigates the minimum number of participants for open card sorts performed on a general public website domain (e-commerce). In specific, it involves 203 and 210 participants sorting content items of two real-world e-commerce websites. Results from all the participants were compared with those of different-sized and randomly selected samples of the participants. It was found that 15 to 20 participants is a cost-effective way to obtain reliable open card sort data for general public websites.KeywordsCard SortingInformation ArchitectureSample size
Open card sorting is the most widely used HCI technique for designing user-centered Information Architectures (IAs). The method has a straightforward data collection process, but data analysis can be challenging. Open card sorting has been also criticized as an inherently content-centric technique that may lead to unusable IAs when users are attempting tasks. This paper proposes a new variant of open card sorting, the Task-Based Open Card Sorting (TB-OCS), which considers users’ tasks and simplifies data analysis. The proposed method involves two phases. First, small groups of participants perform classic open card sorting. Then, each participant performs findability tasks using each IA produced by the rest participants of the same group and their first-click success is measured. Analysis of the collected data involves simply calculating the first-click success rate per participants’ IA and selecting the one with the highest value. We have also developed a web-based software tool to facilitate the conduction of TB-OCS. A within-subjects user testing study found that open card sorting produced IAs that had significantly higher first-click success rates and perceived usability ratings compared to the IAs produced by TB-OCS. However, this may be due to parameters of the new method that require finetuning, thus further research is required.KeywordsCard SortingInformation ArchitectureIATask-Based Open Card Sorting
Card sorting is a popular way for creating website information architectures based on users’ mental models. This paper explores the effect of participants’ self-efficacy on card sorting results. A two-phase study was carried out. The first phase involved 40 participants rating their self-efficacy on a standardized scale, followed by an open card sort experiment. The median self-efficacy score was used to split the open card sort data into two groups: one for low and one for high participants’ self-efficacy. These two datasets were analyzed following state-of-the-art techniques for open card sort data analysis, which resulted in two information architectures for the eshop. In the second phase, two functional prototypes were first created for the eshop, one for each information architecture of the first phase. Subsequently, 30 participants interacted with both prototypes in a user testing study. This paper found that users interacting with the information architecture produced by open card sort participants with low self-efficacy made statistically significantly more correct first clicks, significantly less time to find content items, rated the tasks as significantly easier, and provided higher perceived usability ratings compared to when they interacted with the information architecture produced by users with high self-efficacy.
Information architecture forms the foundation of users' navigation experience. Open card sorting is a widely-used method to create information architectures based on users' groupings of the content. However, little is known about the method's cross-study reliability: Does it produce consistent content groupings for similar profile participants involved in different card sort studies? This paper presents an empirical evaluation of the method's cross-study reliability. Six card sorts involving 140 participants were conducted: three open sorts for a travel website, and three for an eshop. Results showed that participants provided highly similar card sorting data for the same content. A rather high agreement of the produced navigation schemes was also found. These findings provide support for the cross-study reliability of the open card sorting method.
Previous research suggests an important role for self-tracking in promoting mental wellness. Recent studies with college student populations have examined the feasibility of collecting everyday mood, activity, and social data. However, these studies do not account for students' experiences and challenges adopting self-tracking technologies to support mental wellness goals. We present two studies conducted to better understand self-tracking for stress management and mental wellness in student populations. First, focus groups and card sorting activities with 14 student health professionals reveal expert perspectives on the usefulness of tracking for three scenarios. Second, an online survey of 297 students examines personal experiences with self-tracking and attitudes toward sharing self-tracked data with others. We draw on findings from these studies to characterize students' motivations, challenges and preferences in collecting and viewing self-tracked data related to mental wellness, and we compare findings between students with diagnosed mental illnesses and those without. We conclude with a discussion of challenges and opportunities in leveraging self-tracking for mental wellness, highlighting several design considerations.
This paper presents a taxonomy of design guidelines for robot teleoperation developed from a focused literature review of robot teleoperation. A list of user interface design guidelines was assembled, open card sorting and a focus group were used to classify them, and closed card sorting was employed to validate and further refine the proposed taxonomy. The initially obtained set of 70 guidelines is grouped into eight categories: platform architecture and scalability, error prevention and recovery, visual design, information presentation, robot state awareness, interaction effectiveness and efficiency, robot environment/surroundings awareness, and cognitive factors. Agricultural robots were used as an application case study for implementation and field evaluation. The proposed guideline taxonomy was used heuristically to evaluate the usability of an existing user interface of a teleoperated agricultural robot.
Perceived usability affects greatly student’s learning effectiveness and overall learning experience, and thus is an important requirement of educational software. The System Usability Scale (SUS) is a well-researched and widely used questionnaire for perceived usability evaluation. However, surprisingly few studies have used SUS to evaluate the perceived usability of Learning Management Systems (LMSs). This paper presents an empirical evaluation of the SUS questionnaire in the context of LMSs’ perceived usability evaluation. Eleven studies involving 769 students were conducted, in which participants evaluated the usability of two LMSs (eClass, and Moodle) used within courses of their curriculum. It was found that the perceived usability of the evaluated LMSs is at a satisfactory level (mean SUS score 76.27). Analysis of the results also demonstrated the validity and reliability of SUS for LMSs’ evaluation, and that it remains robust even for small sample sizes. Moreover, the following SUS attributes were investigated in the context of LMSs evaluation: gender, age, prior experience with the LMS, Internet self-efficacy, attitude towards the Internet and usage frequency of the LMS.
Transitioning from bespoke single-purpose displays to multi-purpose public interactive displays entails a number of challenges. One challenge is the development of usable mechanisms that allow users to explore the functionality and services on such displays. In this paper we present a field trial that employs AutoCardSorter, our tool that uses semantic similarity and clustering algorithms, to automatically group the available applications of a public interactive display into categories based on the developer-provided descriptions of each application. The results demonstrate that the grouping generated by AutoCardSorter improved both performance and self-reported usability measures compared to practitioners’ existing grouping. In addition, the study investigated the interplay between grouping and interaction modality (i.e. public display vs. desktop). Results tend to support that grouping affects more the user experience with a multi-purpose interactive display, but findings were insignificant. Our work provides a way for public displays to dynamically update their offered services without sacrificing usability.
Netsorting is a web based cardsorting tool. Our research group used Netsorting to run a number of experiments on cognitive science and on information architecture. In the study we are presenting here we compared two couples of card sorts with the same data: two performed with Netsorting, the other two with the traditional paper based card sorting. We measured the performance of the two groups with two indices: the number of correct classification and the correlation among sampled subsets of participants. The participants who used Netsorting performed as good as the ones who used the paper sorting.
Three experiments with 77 undergraduates studied sense of direction as a verbal expression of people's estimation of their own spatial orientation ability, rather than as a special mental faculty. Self-reports of sense of direction were shown to reflect spatial orientation ability. Ss with good senses of direction were better than ones with poor senses of direction at pointing to unseen goals (local buildings) in a familiar environment. In a novel environment (a maze) after minimal experience during which the need for orientation was not emphasized, good and poor sense-of-direction Ss did not differ in the accuracy of their mental representation of the area. Given additional exposure in which orientation was emphasized, good sense-of-direction Ss showed improved accuracy of their representation of the area, whereas poor sense-of-direction Ss showed no hint of improved performance. It is concluded, then, that the improved orientation of people with a good sense of direction is not automatic or facile, but requires possibly both (a) a conscious effort to orient and (b) repeated exposure to an environment. (28 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Perceived usability is an important requirement of educational software, affecting greatly student’s learning effectiveness and overall learning experience. Although the System Usability Scale (SUS) questionnaire is considered as a de facto standard for perceived usability evaluation, surprisingly, few studies are reporting results from evaluations related to Learning Management Systems (LMSs). In addition, given that the questionnaire comprises 10 questions presented in English, its applicability in studies involving speakers of other languages is questionable. In this paper, a first step towards standardization of a Greek version of SUS in the context of LMSs perceived usability evaluation is reported. To this end, three studies involving 280 university students in both blended and distance learning education were conducted. Analysis of the results demonstrated the validity and reliability of the Greek version of SUS.
This paper reports on a study of professional web designers and developers. We provide a detailed characterization of their knowledge of fundamental programming concepts elicited through card sorting. Additionally, we present qualitative findings regarding their motivation to learn new concepts and the learning strategies they employ. We find a high level of recognition of basic concepts, but we identify a number of concepts that they do not fully understand, consider difficult to learn, and use infrequently. We also note that their learning process is motivated by work projects and often follows a pattern of trial and error. We conclude with implications for end-user programming researchers.
This paper presents the methodology and the results of the development of a set of age-centered research-based web design guidelines. An initial set of guidelines was first developed through careful literature review of the HCI \& Aging literature. Then a seires of classification methods (card sorting, affinity diagrams) were employed as a means for obtaining a revised and more robust classified set of guidelines. Finally the revised set of guidelines and the original set were tested through their application to a number of age-related websites.
Post-task ratings of difficulty in a usability test have the potential to provide diagnostic information and be an additional measure of user satisfaction. But the ratings need to be reliable as well as easy to use for both respondents and researchers. Three one-question rating types were compared in a study with 26 participants who attempted the same five tasks with two software applications. The types were a Likert scale, a Usability Magnitude Estimation (UME) judgment, and a Subjective Mental Effort Question (SMEQ). All three types could distinguish between the applications with 26 participants, but the Likert and SMEQ types were more sensitive with small sample sizes. Both the Likert and SMEQ types were easy to learn and quick to execute. The online version of the SMEQ question was highly correlated with other measures and had equal sensitivity to the Likert question type. Author Keywords
In this paper, we describe an innovative tool that supports the design and evaluation of the information architecture of a Web site. The tool uses Latent Semantic Analysis and hierarchical clustering algorithms to provide optimal information navigation schemes in an automated manner. The proposed, tool-based, approach addresses the problem of reasonable content structuring, which established techniques such as card sorting also address. A real world case study depicted substantial effectiveness gain, without expense in the quality of results. We argue that such an approach could facilitate information-rich applications design, like most Web sites, by reducing time and resources required.
Structuring of the content is an important step in web site design, affecting greatly navigability and the overall user experience.
Automated support of this task is the object of this paper. AutoCardSorter, a computational tool that supports clustering
of the web pages of a site, is introduced. The proposed tool-based methodology uses semantic similarity measures, such as
latent semantic analysis, and hierarchical clustering algorithms, in order to suggest suitable information navigation schemes.
In the paper, after introducing AutoCardSorter design and functionality, three independent studies are discussed. The studies,
that were conducted in order to validate the proposal, compared the proposed method with the established card-sorting approach,
in different domains. It was found that substantial gain in effectiveness was achieved without expense in the quality of results,
therefore, reducing the required time and human resources.
Structural design of shared information environments is rather challenging. Card sorting constitutes a valuable method towards this end. However, there is a lack of free software tools for conducting card sorts, especially from a distance, and analysing the collected data. This paper presents CardSorter, an open source, web-based, card sorting tool. Currently, CardSorter offers functionality for running open card sorts online and analysing the collected data in real time. In a user testing study involving 108 participants, CardSorter was found to be from Good to Excellent in terms of perceived usability. A heuristic evaluation of the tool involving 25 teams of novice evaluators identified issues for further improvement.
This paper presents a study aimed at better understanding how children categorize different games. The paper reports the results of an open card sort where participants were asked to categorize games from the Google Play Store (formerly the 'Android Marketplace'). The key contribution of the paper is that when compared with existing categories in the Google Play Store, children used categorization criteria much more aligned to the goals of the game rather than more abstract categories currently found in mobile phone application stores. The paper provides examples of existing categories that are not generally used by children and provides new examples of categorization criteria that are used by children to categorize existing games.
The effects of spatial visualization ability (SVA) and domain expertise on information seeking skills, is described. SVA relates to hierarchical menus systems navigation within databases, online learning environments, and virtually all internet web sites. Information problems should be addressed incrementally with the system providing feedback which would help the user narrow or broaden the scope of their search or additional domain knowledge regarding the topic of their search. The Human Computer Interaction (HCI) should evolve into an environment in which the computer becomes invisible and the user has a sense of interacting directly with the problem domain.
Software and product designers use card sorting to understand item groups and relationships. In the usability community, a common method of formal statistical analysis for open card sort data is hierarchical cluster analysis, which results in a tree of the items sorted into distinct, nested clusters. Hierarchical cluster analysis is appropriate for highly structured settings, like software menus. However, many situations call for softer clusters, such as designing websites where multiple pages link to the same target page. Factor analysis summarizes the categories created in card sorts and generates clusters that can overlap. This paper explains how to prepare card sort data for statistical analysis, describes the results of factor analysis and how to interpret them, and discusses when hierarchical cluster analysis and factor analysis are appropriate.
The EZSort tool helps interface designers organize information for Web sites based on users' expectations gathered from card sorting exercises. It includes two packages: Usort and EZCalc. Usort provides a simple user interface for the card sort study participants to group cards by direct manipulation within a Graphical User Interface. EZCalc analyzes the card sort data gathered from Usort using cluster analysis statistical tool and generates tree diagrams that present clearly the page groupings suggested by the data. The output diagrams from EZCalc feature directly adjustable criteria bars and dynamic feedback on the resulting groupings. This tool is available for download from the IBM Ease of Use Web site.
Environmental spatial abilities are involved in everyday tasks such as finding one's way in the environment and learning the layout of a new environment. Self-report measures of environmental abilities, e.g., asking people to rate their “sense of direction (SOD),” have been found to predict objective measures of these abilities quite highly. In this study, we developed a standardized self-report scale of environmental spatial ability, the Santa Barbara Sense of Direction Scale (SBSOD). The scale proved to be internally consistent and had good test–retest reliability. A series of four validity studies examined its relation to measures of spatial updating and acquisition of spatial knowledge at different scales and acquired from different learning experiences. These studies suggested that the SBSOD is related to tasks that require one to update location in space as a result of self-motion. It is more highly correlated with tests of spatial knowledge that involve orienting oneself within the environment than with tests that involve estimating distances or drawing maps. Self-report SOD is also somewhat more highly correlated with measures of spatial knowledge acquired from direct experience in the environment than with measures of knowledge acquired from maps, video, or virtual environments (VE).
Watching DVDs can be a frustrating experience, because DVD menus often miss out on usability and are complex and difficult to navigate through. Similar to the early years of web development, there is a lack of design standards. In this paper, we show the development of user interface guidelines for DVD menus. These guidelines can be used to design and evaluate DVD menus. We built a prototype according to the guidelines, conducted usability tests with the prototype and evaluated other movie DVDs using the guidelines to show the applicability, utility and usability of the guidelines.
The relationship of sense of direction in the real world to Internet use, as well as to cognitive maps of the Internet, was investigated. In Study 1, 28 participants were asked to draw a sketch map of the Internet. They also completed a questionnaire containing scales for measuring Internet usage, knowledge, and skill, and completed the Sense of Direction Questionnaire (SDQ-S), which is composed of two scales, awareness of orientation and memory for usual spatial behavior. In Study 2, 48 participants performed a modified sketch map task and completed the same questionnaires. Awareness of orientation improved the Internet skills of novices but not of experts. In addition, awareness of orientation affected the structure of the cognitive maps of the Internet. These results suggest that sense of direction in the real world influence skill with the Web for novice users.
This article presents nearly 10 year's worth of System Usability Scale (SUS) data collected on numerous products in all phases of the development lifecycle. The SUS, developed by Brooke (1996)2.
Brooke , J. 1996. “SUS: A “quick and dirty” usability scale”. In Usability evaluation in industry, Edited by:
Jordan , P. W. ,
Thomas , B. A. Weerdmeester and
McClelland , I. L. 189–194. London: Taylor & Francis. View all references, reflected a strong need in the usability community for a tool that could quickly and easily collect a user's subjective rating of a product's usability. The data in this study indicate that the SUS fulfills that need. Results from the analysis of this large number of SUS scores show that the SUS is a highly robust and versatile tool for usability professionals. The article presents these results and discusses their implications, describes nontraditional uses of the SUS, explains a proposed modification to the SUS to provide an adjective rating that correlates with a given score, and provides details of what constitutes an acceptable SUS score.
Psychological research in human-computer interaction. Annual review of information science and technology 19 33-64. C. L Borgman. 1984. Psychological research in human-computer interaction
- C Borgman