Article

Mus ink : composing music through augmented drawing

01/2009; DOI: 10.1145/1518701.1518827

ABSTRACT We focus on the creative use of paper in the music composition process, particularly the interaction between paper and end-user programming. When expressing musical ideas, composers draw in a precise way, not just sketch. Working in close collaboration with composers, we designed Musink to provide them with a smooth transition between paper drawings and OpenMusic, a flexible music composition tool. Musink's built-in recognizers handle common needs, such as scoping and annotation. Users can also define new gestures and associate them with their own or predefined software functions. Musink supports semi-structured, delayed interpretation and serves as a customizable gesture browser, giving composers significant freedom to create their own, individualized composition languages and to experiment with music, on-paper and on-line.

0 Bookmarks
 · 
68 Views
  • [Show abstract] [Hide abstract]
    ABSTRACT: Digital pen technology has allowed for the easy transfer of pen data from paper to the computer. However, linking handwritten content with the digital world remains a hard problem as it requires the translation of unstructured and highly personal vocabularies into structured ones that computers can easily understand and process. Automatic recognition can help to this direction, but as it is not always reliable, solutions require the active cooperation between users and recognition algorithms. This work examines the use of portable touch-screen devices in connection with pen and paper to help users direct and refine the interpretation of their strokes on paper. We explore four techniques of bi-manual interaction that combine touch and pen-writing, where user attention is divided between the original strokes on paper and their interpretation by the electronic device. We demonstrate the techniques through a mobile interface for writing music that complements the automatic recognition with interactive user-driven interpretation. An experiment evaluates the four techniques and provides insights about their strengths and limitations.
    Proceedings of the 25th annual ACM symposium on User interface software and technology; 10/2012
  • Source
    AVI; 05/2012
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: We present paper substrates, interactive paper components that support the creation and manipulation of complex musical data. Substrates take different forms, from whole pages to movable strips, and contain or control typed data representations. We conducted participatory design sessions with five professional musicians with extensive experience with music creation tools. All generated innovative uses of paper substrates, manipulating their data, linking multiple representation layers and creating modular, reusable paper elements. The substrates reflect the structure of their computer-based data, but in a much more flexible and adaptable form. We use their prototypes to provide concrete examples of substrates, identify their roles, properties and functions. Finally, we explore their physical and interaction design with an interactive prototype.
    01/2012;

Full-text (3 Sources)

View
24 Downloads
Available from
May 22, 2014