Mus ink : composing music through augmented drawing

01/2009; DOI: 10.1145/1518701.1518827

ABSTRACT We focus on the creative use of paper in the music composition process, particularly the interaction between paper and end-user programming. When expressing musical ideas, composers draw in a precise way, not just sketch. Working in close collaboration with composers, we designed Musink to provide them with a smooth transition between paper drawings and OpenMusic, a flexible music composition tool. Musink's built-in recognizers handle common needs, such as scoping and annotation. Users can also define new gestures and associate them with their own or predefined software functions. Musink supports semi-structured, delayed interpretation and serves as a customizable gesture browser, giving composers significant freedom to create their own, individualized composition languages and to experiment with music, on-paper and on-line.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Digital pen technology has allowed for the easy transfer of pen data from paper to the computer. However, linking handwritten content with the digital world remains a hard problem as it requires the translation of unstructured and highly personal vocabularies into structured ones that computers can easily understand and process. Automatic recognition can help to this direction, but as it is not always reliable, solutions require the active cooperation between users and recognition algorithms. This work examines the use of portable touch-screen devices in connection with pen and paper to help users direct and refine the interpretation of their strokes on paper. We explore four techniques of bi-manual interaction that combine touch and pen-writing, where user attention is divided between the original strokes on paper and their interpretation by the electronic device. We demonstrate the techniques through a mobile interface for writing music that complements the automatic recognition with interactive user-driven interpretation. An experiment evaluates the four techniques and provides insights about their strengths and limitations.
    Proceedings of the 25th annual ACM symposium on User interface software and technology; 10/2012
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: A Tonnetz, or "tone-network" in German, is a two-dimensional representation of the relationships among musical pitches. In this paper, we present PaperTonnetz, a tool that lets musicians explore and compose music with Tonnetz representations by making gestures on interactive paper. In addition to triggering musical notes with the pen as a button based-interface, the drawn gestures become interactive paths that can be used as chords or melodies to support composition.
  • [Show abstract] [Hide abstract]
    ABSTRACT: We describe the design, development, and evaluation of a personal digital music library application designed to assist musicians in capturing, developing, and managing their musical ideas over time. The target user group is musicians who primarily use audio and text for composition and arrangement, rather than with formal music notation. The software design was guided by a formative user study which suggested five requirements for the software to support: capturing, overdubbing, developing, archiving, and organizing. This led to a spatial hypermedia approach forming the basis for the developed application. Furthermore, the underlying spatial data-model was exploited to give raw audio compositions a hierarchical structure, and—to aid musicians in retrieving previous ideas—a search facility was provided to support both query by humming and text-based queries. A user evaluation of the implemented environment indicated that the target musicians would find the hypermedia environment useful for capturing and managing their moments of musical creativity. More specifically, they would make use of the query by humming facility and the hierarchical track organization, but not the overdubbing facility as implemented.
    International Journal on Digital Libraries 08/2012; 12(2-3).

Full-text (3 Sources)

Available from
May 22, 2014