No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Towards User Interface Guidelines for Musical Grid Interfaces
... A notation system for musical grid interfaces can enable the education, creation, and live performance of digitally produced works, leading to new educational paradigms, artistic possibilities, and forms of dissemination [7]. As musical instruments with grid interfaces offer 'an utterly unique method of controlling a computer system' [6], a notation system that accounts for grid playing surface topographies can significantly contribute to Human-Computer Interaction (HCI) in digital music practices, as the grid controller is becoming an industry standard and an "object of research in the HCI community" [8]. ...
... However, understanding that 'fragmented musical knowledge results if reading is not taught hand-in-hand with playing the instrument' underscores the importance of musical instruments and notation for holistic music education [30,31,32]. As grid user interfaces (UIs) are designed to 'reduce learning' in terms of cognitive dissonance by 'eliminating confusion,' they can streamline the learning process [8,12]. In this context, 'reduce learning' is viewed positively, as in minimizing the amount of new information or skills that a user must acquire to effectively use a system. ...
TABstaff+ is a hybrid music notation developed for grid-based user interfaces. The system builds on notational elements and conventions of tablature, standard five-line staff notation, and chord diagrams. TABstaff+ strives to facilitate teaching and learning, composition and production, and performance using grid-based tangible user interfaces (TUIs) and graphical user interfaces (GUIs). For usability testing, the study involved seven participants, music production and composition students (ages 13 to 19) with prior musical experience. The paper considers the Ableton Push instrument to illustrate the application and adaptability of the TAB+, Staff+, and Charts+ notation systems. These notation systems aim to further the development of postdigital practices by leveraging Human-Computer Interaction (HCI) and pre-digital practices of reading, playing , and teaching music using instruments and notation. TABstaff+ aims to be a transferable music notation system that allows educators and practicing musicians to utilize the pedagogical and creative capabilities of musical grid interfaces.
In this mini-review, I summarize and interpret the current status of sex/gender differences in terms of brain anatomy, brain function, behavior, and cognition. Based on this review and the reported findings, I conclude that most of these sex/gender differences are not large enough to support the assumption of sexual dimorphism in terms of brain anatomy, brain function, cognition, and behavior. Instead, I suggest that many brain and cognitive features are modulated by environment, culture, and practice (and several other influences). These influences interact with the menstrual cycle, the general hormone level, and current gender stereotypes in a way that has not yet been fully understood.
The Internet of Things (IoT) enabled through sensor-rich environments and smart devices allows us to collect and exchange vast quantities of data. The advent of new markets, such as the smart home sector, and movements, such as the quantified self, indicate the IoT’s huge economic and social impact. With the increased availability of IoT services, it becomes important to enable users with intuitive mechanisms for accessing the gathered data. In this work, we present findings from an exploratory design case study, in which we deployed a low-res lighting display in three family households to visualize domestic energy performance data. Our study showed that the standalone lighting display was preferred over a commercially available web-based application. Further, we found that in two of the three households those participants, who did not use the mobile application before, became the main user of the display and actively engaged with the visualized data. The paper concludes with design implications for pervasive displays connected as ambient gateways to smart devices.
Low-resolution (low-res) lighting displays are increasingly used by HCI researchers, designers, and in the industry as a versatile and aesthetic medium for deploying ambient interfaces in various contexts. These display types distinguish themselves from conventional high-res screens through: high contrasts, hi-power LED technology which allows visibility even in bright environments, and their ability to take on three-dimensional free forms. However, to date most work on low-res displays has been either of experimental nature or carried out in isolated industry contexts. This paper addresses this gap through an analysis of our own experiences from previous experimental design studies and related work, which led us to five domain challenges for designing low-res displays. We then describe how we approached these challenges in a deployment study, which involved the implementation of a prototype guided by a low-res prototyping toolkit. Based on an analysis of our design process and findings from the deployment study, we present ten design recommendations for low-res lighting displays.
Human-Computer Interaction (HCI) is dynamic and interdisciplinary. Diverse areas of knowledge contribute to form and advance the state of the art, but also bring different views about how to do research in HCI. In this paper we aim at promoting discussion and increased awareness on different kinds of research by comparing the three HCI waves with concepts of Philosophy of Science and general research paradigms. We believe this awareness can play an important role in improving the fairness of the peer review process and support researchers in positioning themselves and their investigations.
To what extent can information be successfully communicated through a media façade and what are the relevant parameters? In this paper, we focus on the issue of information design for media façades, which is not often discussed separately. As a thought experiment, we propose homing in on this topic as a core domain for the deployment of low-resolution, ambient displays in the city. We discuss the advantages and limitations of five techniques for encoding information: color, movement, text, images, and shape. Though designers may not always be aiming to convey explicit information, onlookers may still seek additional layers of meaning and end-users may re-appropriate an infrastructure over time. Two examples from our recent practice, a series of single-pixel way-finding beacons and a low-resolution media façade, serve to illustrate these techniques. By linking the broad notions of 'content' and 'meaning' to a set of purpose-driven and actionable parameters, we invite designers to scrutinize the low-level communication processes facilitated by media architecture.
This article describes five experiments that were conducted to evaluate viewing-distance preferences as a function of video image size and resolution. Viewing-distance preferences are a key design parameter for work involving human factors in video teleconferencing, distance learning, and other multimedia applications. In addition, viewing-distance preferences have been treated as a subjective measure of image quality and used in ongoing discussions of proposed standards for high-definition television (HDTV). Based on previous literature, it was hypothesized that the ratio of viewing-distance preferences to image height would be a constant, would vary based on image resolution, and would average around 7 for standard NTSC video (with 525 scan lines of resolution). Contrary to the predictions, the data showed that the ratio decreases as image size increases, and that the ratios are relatively uninfluenced by image resolution. The data for the five experiments were combined to generate a guideline for estimating viewing-distance preferences across a wide range of video image sizes.
Opt-in surveys are the most widespread method used to study participation in online communities, but produce biased results in the absence of adjustments for non-response. A 2008 survey conducted by the Wikimedia Foundation and United Nations University at Maastricht is the source of a frequently cited statistic that less than 13% of Wikipedia contributors are female. However, the same study suggested that only 39.9% of Wikipedia readers in the US were female - a finding contradicted by a representative survey of American adults by the Pew Research Center conducted less than two months later. Combining these two datasets through an application and extension of a propensity score estimation technique used to model survey non-response bias, we construct revised estimates, contingent on explicit assumptions, for several of the Wikimedia Foundation and United Nations University at Maastricht claims about Wikipedia editors. We estimate that the proportion of female US adult editors was 27.5% higher than the original study reported (22.7%, versus 17.8%), and that the total proportion of female editors was 26.8% higher (16.1%, versus 12.7%).
Currently, a musician working with digital technology is faced with a panoply of musical tools that can be roughly characterized by a split between ready-made music production software on the one hand, and audio-programming environments such as SuperCollider, CSound, Pure Data, Max/MSP, ChucK, or Audiomulch (to name but a few) on the other. Problems with the former lie in the conceptual and compositional constraints imposed upon users by software tools that clearly define the scope of available musical expressions. It is for this reason that many musicians, determined to fight the fossilization of music into stylistic boxes, often choose to work with programming environments that allow for more extensive experimentation. However, problems here include the practically infinite expressive scope of the environment, sometimes resulting in a creative paralysis or in the frequent symptom of a musician-turned-engineer. Consequently, a common strategy can be detected, defined here as that of designing constraints, where the instrument designer, the composer, or the performer (a distinction often irrelevant in these systems; see Drummond 2009) devises a relatively high-level system of constraints, encapsulating a defined space for potential expression, whether of compositional or gestural nature. This article engages with this situation by exploring the Human–Computer Interaction (HCI) concepts of affordances, constraints ,a ndmapping. We examine how computational systems of musical expression always involve the establishment of a particular stratum that provides certain affordances to the musician, while concurrently posing important constraints. The article studies how constraints can serve as sources for creative explorations. For this purpose, three systems are analyzed: the mLog, Phalanger, and the ixi lang. All these systems are instantiations that proscribe complexity in favor
Informal histories of HCI commonly document two major intellectual waves that have formed the field: the first orienting from engineering/human factors with its focus on optimizing man-machine fit, and the second stemming from cognitive science, with an increased emphasis on theory and on what is happening not only in the computer but, simultaneously, in the human mind. In this paper, we document underlying forces that constitute a third wave in HCI and suggest systemic consequences for the CHI community. We provisionally name this the ‘phenomenological matrix’. In the course of creating technologies such as ubiquitous computing, visualization, affective and educational technology, a variety of approaches are addressing issues that are bad fits to prior paradigms, ranging from embodiment to situated meaning to values and social issues. We demonstrate the underlying unity of these approaches, and document how they suggest the centrality of currently marginal criteria for design, evaluation, appreciation, and developmental methodology in CHI work.
This paper describes a demo prototype of a tangible user interface (TUI) concept that is derived from the expressive play of musical string instruments. We translated this interaction paradigm to an interactive demo which offers a novel gesture vocabulary (strumming, picking, etc.). In this work we present our interaction concepts, prototype description, technical details and insights on the rapid and low-cost manufacturing and design process. (Video demonstration: https://vimeo.com/309265370)
This paper introduces a tangible user interface (TUI) concept designed for child-oriented musical interaction and education by the name of COMB. The interaction concept of the interface is based upon the natural behavior and metaphors found in children's play during construction with building-blocks. This paradigm is used to increase the accessibility of the otherwise expert-focused digital and electronic music to children. We evaluated our prototype during two different study setups. We found preliminary indications that this concept fosters imitation during learning. Therefore, the usage of shape as a meaningful element of interaction could be a promising design strategy for interfaces addressing children. In this paper we present the theoretical foundation of the concept as well as technical details of the prototype. Furthermore, we discuss how this concept can be applied to increase accessibility of technology in various other domains.
Seemingly, trivial software does important cultural work, both reflecting hegemonic norms and providing opportunities for transforming them. Software applications for music production (music apps) within the iOS app store promise to broaden the potential for musical participation through simple, “fun,” user-friendly interface design. Yet, within the dominant user interface convention, “fun” is synonymous with the experience of instant success and effortless musical mastery. Drawing on semistructured interviews conducted with developers, and an analysis of shared user interface design conventions across three case studies of apps, ThumbJam, iMaschine 2, and Skram, I argue that normative conceptions of human perfectibility are assumed to be what generates an optimal user experience. Exploring theories of “queer fun,” and the importance of “failure” in studies of video gaming, I propose alternative conceptions of “fun,” and consider how, and with what effects, these might be implemented in the world of music apps.
Displays are ubiquitous in our everyday lives. Since most smart phones or tablets try to avoid control elements, touch screens are primarily the only remaining component of interaction. Beyond the restriction of input methods, static screens define the frame for our applications. Designers and developers, as well as users have to align with standard manufactured screen sizes and resolutions. Modular displays are able to overcome this restriction. Their arrangement can be adapted to fit momentary needs, but an even greater potential lies in their capability to create meaningful objects. Derived from toddlers constructive play with blocks, we introduce COMB. Our prototype demo includes an interaction concept centered around the idea of modular low-resolution displays representing accessible functionality by their arrangement and provide educational benefits for children.
Color mapping is an important technique used in visualization to build visual representations of data and information. With output devices such as computer displays providing a large number of colors, developers sometimes tend to build their visualization to be visually appealing, while forgetting the main goal of clear depiction of the underlying data. Visualization researchers have profited from findings in adjoining areas such as human vision and psychophysics which, combined with their own experience, enabled them to establish guidelines that might help practitioners to select appropriate color scales and adjust the associated color maps, for particular applications. This survey presents an overview on the subject of color scales by focusing on important guidelines, experimental research work and tools proposed to help non-expert users.
This paper examines Research Through Design as an orientation to so-called 'Third Wave' Human Computer Interaction (HCI). A number of recent critical reflections are reviewed and the 'disciplinary anxieties', which this approach to HCI has aroused, are discussed. Drawing on Feyerabend's philosophical scepticism over methods and contributions to the Sociology of Science, it is suggested that design research might build its own 'limited rationality' rather than be brought in line with supposed norms for good research or criteria for rigour and relevance of unfamiliar provenance. To this end, a concept of 'annotated portfolio' is advanced, and detailed, as a means for capturing the family resemblances that exist in a collection of artefacts, simultaneously respecting the particularity of specific designs and engaging with broader concerns. The concept is demonstrated through annotating nine well-known pieces created by the Goldsmiths Interaction Research Studio. Treating this collection as an annotated portfolio highlights, formulates and collates interaction design issues in this work in a novel manner. On this basis, annotated portfolios are proposed as a viable means for communicating design thinking in HCI in a descriptive yet generative and inspirational fashion, without having recourse to standards of 'theory' which fit design practice uncomfortably.
This paper is concerned with the design of interactive virtual musical instruments. An interaction design strategy which uses on-screen objects that respond to user actions in physically realistic ways is described. This approach allows musicians to ‘play’ the virtual instruments using the sound of their familiar acoustic instruments. An investigation of user experience identified three modes of interaction that characterise the musicians' approach to the virtual instruments: instrumental, ornamental and conversational. When using the virtual instruments in instrumental mode, musicians prioritise detailed control; in ornamental mode, they surrender detailed control to the software and allow it to transform their sound; in conversational mode, the musicians allow the virtual instrument to ‘talk back’, helping to shape the musical direction of performance much as a human playing partner might. Finding a balance between controllability and complexity emerged as a key issue in facilitating ‘conversational’ interaction.
Insights into human visual attention have benefited many areas of computing, but perhaps most significantly visualisation and UI design [3]. With the proliferation of mobile devices capable of supporting significantly complex applications on small screens, demands on mobile UI design and the user's visual system are becoming greater. In this paper, we report results from an empirical study of human visual attention, specifically the Change Blindness phenomenon, on handheld mobile devices and its impact on mobile UI design. It is arguable that due to the small size of the screen - unlike a typical computer monitor - a greater visual coverage of the mobile device is possible, and that these phenomena may occur less frequently during the use of the device, or even that they may not occur at all. Our study shows otherwise.
We tested for Change Blindness (CB) and Inattentional Blindness (IB) in a single-modal, mobile context and attempted to establish factors in the application interface design that induce and/or reduce their occurrences. The results show that both CB and IB can and do occur while using mobile devices. The results also suggest that the number of separate attendable items on-screen is directly proportional to rates of CB. Newly inserted objects were correctly identified more often than changes applied to existing on-screen objects. These results suggest that it is important for mobile UI designers to take these aspects of visual attention into account when designing mobile applications that attempt to deliver information through visual changes or notifications.
NASA-TLX is a multi-dimensional scale designed to obtain workload estimates from one or more operators while they are performing a task or immediately afterwards. The years of research that preceded subscale selection and the weighted averaging approach resulted in a tool that has proven to be reasonably easy to use and reliably sensitive to experimentally important manipulations over the past 20 years. Its use has spread far beyond its original application (aviation), focus (crew complement), and language (English). This survey of 550 studies in which NASA-TLX was used or reviewed was undertaken to provide a resource for a new generation of users. The goal was to summarize the environments in which it has been applied, the types of activities the raters performed, other variables that were measured that did (or did not) covary, methodological issues, and lessons learned
The use of a laptop computer for musical performance has become widespread in the electronic music community. It brings with it many issues pertaining to the communication of musical intent. Critics argue that performances of this nature fail to engage audiences because many performers use the mouse and/or computer keyboard to control their musical works, leaving no visual cues to guide the audience as to the correlation between performance gestures and musical outcomes. The author will argue that interfaces need to communicate something of their task and that cognitive affordances (Gibson 1979) associated with the performance interface become paramount if the musical outcomes are to be perceived as clearly tied to real-time performance gestures – in other words, that the audience are witnessing the creation of the music in that moment as distinct to the manipulation of pre-recorded or pre-sequenced events. Interfaces of his kind lend themselves particularly to electroacoustic and computer music performance where timbre, texture and morphology may be paramount.
In such cases the visual field is perceived as articulated into two components, the figure (patch) on the ground (surround). This figure-ground articulation may seem obvious, but it is not trivial. This type of field organization has a number of remarkable features, first described in the work of Rubin (1915/1921), predating Wertheimer's publication. The two components are perceived as two segments of the visual field differing not only in color, but in some other phenomenal characteristics as well. The figure has an object-like character, whereas the ground has less perceptual saliency and appears as 'mere' background. The areas of the figure and the ground usually do not appear juxtaposed in a common plane, as in a mosaic, but rather as stratified in depth: there is a tendency to see the figure as positioned in front, and the ground at a further depth plane and continuing to extend behind the figure, as if occluded by it. Furthermore, the border separating the two segments is perceived as belonging to the figure rather than to the ground, and as delineating the figure's shape as its contour, whereas it is irrelevant to the shape of the ground. Certain displays are bi-stable, in that what is perceived as figure can also be perceived as ground and vice-versa. However, in displays structured such as Figure 1, in which a smaller region is wholly surrounded by a larger region, it is usually the former that appears as figure (although it may also be seen as a hole), and the latter as ground.
Usability has been an essential component of the Web User Experience (UX) and a focal research topic. In recent years, the penetration of interactive technologies in all aspects of everyday life challenges the way UX is understood and designed. The past decade, Human-Computer Interaction (HCI) scholars have been continuously attempting to introduce and explore new and non-traditional factors in the UX arena, such as aesthetics, emotions, affect, and trust. This study contributes to the field by exploring the relationship between aesthetics and UX; specifically the impact of the classical design element of white space on the perceived attractiveness and perceived usability of an e-commerce website. A between-subject research design involves the manipulation a website’s white space. Three different versions were constructed using 25, 50 and 75% of the white space, respectively. Findings offer support for the relationship between aesthetics and the perceived usability of a website. The study results suggest that the usability of a website is impacted negatively when white space increases over 50%. Practitioners should consider that in the context of eCommerce Web design, reduction of content and shrinking of visual elements, in favor of white space, is likely to negatively impact the usability of a website.
The problem addressed in this essay is that of the poetic impact of recorded sound; and more specifically the connection between recorded sound, music, and poetry. This is not, according to one way of thinking, an inevitable connection, for in the phonograph and its successor devices the modern world has a way of recording not just the discrete tones of music but all sounds, the messy glissandi and dissonances of the natural world. In Douglas Kahn's account of the move towards sonic transcription—and this is also implicit in the work of Richard Leppert, Friedrich Kittler, and other recent media theorists—devices like the gramophone and Edouard Léon Scott's earlier phonautograph (1857) means that the musical model for sonic reception which runs from Aristotle to Helmholtz must be rejected in order to open up the regime of sonic modernity, characterized by what Kahn calls "all sound" including noise:
Beginning in the late eighteenth and pervading the nineteenth century, three new inscriptive practices as applied to sound—graphic techniques in general, visible sound techniques, and automatic recording instruments as represented by the phonautograph and phonography—contributed to a loosening up of the reliance of acoustics on music. A plethora of lines made sound tangible and textual by making the invisible visible and holding the time of sound still.
For Kittler, in a parallel fashion, this is the shift from the note to the frequency; from the arts to media. But that seems to me a premature judgement, or at least one which ignores the possibility that music itself already carries the notion of redeemed time later attached to the gramophone. This essay aims to examine that possibility—in which music is already a technology—and its metaphoric location in the player piano, a device which might be approached in the spirit of Walter Benjamin's suggestion that technologies become most interesting at the point of their obsolescence.
We can begin with three moments. Our first stop is a study in Wandsworth, South London, in the summer of 1878. A man suddenly rises from his desk and dashes out. In a sentence which comes as a mechanical rush of syllables, he later recalled that
he had heard a street barrel-organ of the kind that used to be called a "harmoniflute," playing somewhere near at hand the very quadrille over which the jaunty young man who had reached the end of his time at Hicks's had spread such a bewitching halo more than twenty years earlier by describing the glories of dancing round to its beats on the Cremorne platform or at the Argyle Rooms, and which Hardy had never been able to identify.
Thomas Hardy, for it is he, goes on to describe how that haunting dance tune was brought down to Dorset around 1860 by the "jaunty young man"; when Hardy moved to London he searched for the music, without success. Now here it is outside his door, albeit in a flattened guise—the organ-grinder has little English and can only point at the marking on the organ: "Quadrille."
Our second stop is again in London, the Silenus beer-hall, that hangout of suspicious characters—two of whom, a small shabby man and a younger yellow-haired man, are plotting over their ale. In the corner an "upright semi-grand piano" suddenly lurches into life, its playerless keys moving up and down to a variety of popular tunes—mazurkas, airs. This is the player piano of Conrad's The Secret Agent (1906), an obscurely potent symbol for evacuated meanings and automaticity which returns at the novel's end as a suitable chorus for Ossipon's derangement of memory, his obsessive repetition of that journalistic phrase about the "impenetrable mystery" surrounding Winnie's suicide.
Our third stop is an old poet, meditating in his study in Hartford, Connecticut in the dark days of the Second World War on those "times of inherent excellence" when the world collects itself. This is the formula he arrived at:
incalculable balances, At which a kind of Swiss perfection comes And a familiar music of the machine Sets up its Schwärmerei, not balances That we achieve but balances that happen...
Post-WIMP User Interfaces
- B Rossmy
- A Wiethoff
Rossmy, B., & Wiethoff, A. (2021, April 29). Musical Grid Interfaces: Past, Present,
and Future Directions. NIME 2021. https://doi.org/10.21428/92fbeb44.6a2451e6 ↩
8. van Dam, A. (1997). Post-WIMP User Interfaces. Commun. ACM, 40(2), 63-67.
The Modular Backward Evolution -Why to
- B Rossmy
- A Wiethoff
Rossmy, B., & Wiethoff, A. (2019). The Modular Backward Evolution -Why to
Designing for new generation electronic musical instruments: Strategies to improve interaction, user experience and live performance
Modernism/Modernity, 14(1), 1-19. https://doi.org/10.1353/mod.2007.0001 ↩
22. Öke, E. H. (2020). Designing for new generation electronic musical instruments:
Strategies to improve interaction, user experience and live performance [Phdthesis].