Evaluating Usability Evaluation Methods: Criteria, Method and a Case Study

DOI: 10.1007/978-3-540-73105-4_63 In book: Human-Computer Interaction, Part I, HCII 2007, LNCS 4550, Publisher: Springer, Editors: J. Jacko, pp.569-578

ABSTRACT The paper proposes an approach to comparative usability evaluation that incorporates important relevant criteria identified
in previous work. It applies the proposed approach to a case study of a comparative evaluation of an academic website employing
four widely-used usability evaluation methods (UEMs): heuristic evaluation, cognitive walkthroughs, think-aloud protocol and
co-discovery learning.

  • [Show abstract] [Hide abstract]
    ABSTRACT: To avoid use errors when handling medical equipment, it is important to develop products with a high degree of usability. This can be achieved by performing usability evaluations in the product development process to detect and mitigate potential usability problems. A commonly used method is cognitive walkthrough (CW), but this method shows three weaknesses: poor high-level perspective, insufficient categorisation of detected usability problems, and difficulties in overviewing the analytical results. This paper presents a further development of CW with the aim of overcoming its weaknesses. The new method is called enhanced cognitive walkthrough (ECW). ECW is a proactive analytical method for analysis of potential usability problems. The ECW method has been employed to evaluate user interface designs of medical equipment such as home-care ventilators, infusion pumps, dialysis machines, and insulin pumps. The method has proved capable of identifying several potential use problems in designs.
    Advances in Human-Computer Interaction 10/2013; 2013.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Web applications should be usable in order to be accepted by users and to improve their success probability. Despite the fact that this requirement has promoted the emergence of several usability evaluation methods, there is a need for empirically validated methods that provide evidence about their effectiveness and that can be properly integrated into early stages of Web development processes. Model-driven Web development processes have grown in popularity over the last few years, and offer a suitable context in which to perform early usability evaluations due to their intrinsic traceability mechanisms. These issues have motivated us to propose a Web Usability Evaluation Process (WUEP) which can be integrated into model-driven Web development processes. This paper presents a family of experiments that we have carried out to empirically validate WUEP. The family of experiments was carried out by 64 participants, including PhD and Master’s computer science students. The objective of the experiments was to evaluate the participants’ effectiveness, efficiency, perceived ease of use and perceived satisfaction when using WUEP in comparison to an industrial widely used inspection method: Heuristic Evaluation (HE). The statistical analysis and meta-analysis of the data obtained separately from each experiment indicated that WUEP is more effective and efficient than HE in the detection of usability problems. The evaluators were also more satisfied when applying WUEP, and found it easier to use than HE. Although further experiments must be carried out to strengthen these results, WUEP has proved to be a promising usability inspection method for Web applications which have been developed by using model-driven development processes.
    Journal of Systems and Software 01/2013; 86(1):161-186. · 1.14 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: The physical interaction aspects of embedded systems have been neglected in comparison to internal software issues in the software engineering field. Physical interfaces suffer from interaction complexities leading to usage difficulty and poor acceptance by the end-users. Meanwhile, the usability techniques focus on the overall usability issues while overlooking the in-depth physicality aspects of the interface and interaction. This study proposes a physicality-focused quantitative evaluation method to assist the embedded system developers in managing the interaction complexities of their products. The acceptance of the embedded system developers towards the proposed method was assessed by means of a user study. The results suggested their strong acceptance. The aim of this study is to re-emphasise the natural physical aspects of embedded system interfaces leading to intuitive interaction.
    Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration; 11/2013

Full-text (2 Sources)

Available from
May 31, 2014