Evaluating system utility and conceptual fit using CASSM

International Journal of Human-Computer Studies (Impact Factor: 1.29). 06/2008; 66(6). DOI: 10.1016/j.ijhcs.2007.11.005
Source: OAI


There is a wealth of user-centred evaluation methods (UEMs) to support the analyst in assessing interactive systems. Many of these support detailed aspects of use—for example: is the feedback helpful? Are labels appropriate? Is the task structure optimal? Few UEMs encourage the analyst to step back and consider how well a system supports users’ conceptual understandings and system utility. In this paper, we present CASSM, a method, which focuses on the quality of ‘fit’ between users and an interactive system. We describe the methodology of conducting a CASSM analysis and illustrate the approach with three contrasting worked examples (a robotic arm, a digital library system and a drawing tool) that demonstrate different depths of analysis. We show how CASSM can help identify re-design possibilities to improve system utility. CASSM complements established evaluation methods by focusing on conceptual structures rather than procedures. Prototype tool support for completing a CASSM analysis is provided by Cassata, an open source development.

Download full-text


Available from: Dominic John Furniss,
  • Source
    • "First, the needs analysis is the first step of a design process aiming to collect [22] [45] or to imagine [13] [54] [55] the needs, and to select and translate them into specifications for the implementation in the software. Secondly, the needs analysis corresponds to actions of the needs elaboration process that may take place throughout the design, even during the use of software [38] [39] [40] [41]. In ergonomics and in requirement engineering, authors have proposed various methods and tools to support these two approaches of needs analysis. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Context The participation of users in the design process is recognized as a positive and a necessary element as artifacts suit their needs. Two complementary approaches of users’ involvement co-exist: the user-centred design and the participatory design. These approaches involve learning process from users to designers and vice versa. However, there has no research in design of virtual reality (VR)-based software dealing with how the elaboration of needs is actually distributed in time and among users, designers and project leaders, as well as how it is actually supported by tools and methods. Objective This paper aims to observe, in a real design project of a virtual reality-based software, how the various stakeholders (users, designers, project leaders) actually participate by sharing and pulling pieces of information from the process of needs elaboration, and how these contributions evolve throughout the decisions made in the course of the project. Method Our method, based on the observation of the practices in collective design, allows us to collect and analyze the relationship between each possible action (e.g., elicitation), each stakeholder who initiates these actions (e.g., users) and each phase of the design process (e.g., evaluation phase), and the dynamics of the construction of needs. Results Our results detail how the elicited needs are dealt with by designers, users and / or project leaders: 1) we show a strong contribution of users in the design, compared to others stakeholders, 2) among the needs elicited by users, most have been validated by the designers, 3) some elicited needs could have been firstly rejected and finally validated and implemented. Conclusion We identify the reasons which justify and explain our results confronting them to the literature. We underline the conditions have been satisfied in our study in order to involve effectively users in the design of emerging technologies.
    Information and Software Technology 08/2014; 56:1049-1061. DOI:10.1016/j.infsof.2014.04.008 · 1.05 Impact Factor
  • Source
    • "One way is for the user to have an inaccurate mental model of a system (Vicente, 1999). If so, the challenge is often in supporting users in acquiring a more accurate model (Blandford et al., 2008). In other cases, a poor fit can occur because the system doesn't represent concepts (i.e. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Purpose – This research aims to identify some requirements for supporting user interactions with electronic current-awareness alert systems based on data from a professional work environment. Design/methodology/approach – Qualitative data were gathered using contextual inquiry observations with 21 workers at the London office of an international law firm. The analysis uses CASSM (“Concept-based Analysis of Surface and Structural Misfits”), a usability evaluation method structured around identifying mismatches, or “misfits”, between user-concepts and concepts represented within a system. Findings – Participants were frequently overwhelmed by e-mail alerts, and a key requirement is to support efficient interaction. Several misfits, which act as barriers to efficient reviewing and follow-on activities, are demonstrated. These relate to a lack of representation of key user-concepts at the interface and/or within the system, including alert items and their properties, source documents, “back-story”, primary sources, content categorisations and user collections. Research limitations/implications – Given these misfits, a set of requirements is derived to improve the efficiency with which users can achieve key outcomes with current-awareness information as these occur within a professional work environment. Originality/value – The findings will be of interest to current-awareness providers. The approach is relevant to information interaction researchers interested in deriving design requirements from naturalistic studies.
    Journal of Documentation 01/2011; 67(1):33-55. DOI:10.1108/00220411111105443 · 1.06 Impact Factor
  • Source

Show more