Conference Paper

Midas: a declarative multi-touch interaction framework.

DOI: 10.1145/1935701.1935712 Conference: Proceedings of the 5th International Conference on Tangible and Embedded Interaction 2011, Funchal, Madeira, Portugal, January 22-26, 2011
Source: DBLP

ABSTRACT Over the past few years, multi-touch user interfaces emerged from research prototypes into mass market products. This evolution has been mainly driven by innovative devices such as Apple's iPhone or Microsoft's Surface tabletop computer. Unfortunately, there seems to be a lack of software engineering abstractions in existing multi-touch development frameworks. Many multi-touch applications are based on hard-coded procedural low level event processing. This leads to proprietary solutions with a lack of gesture extensibility and cross-application reusability. We present Midas, a declarative model for the definition and detection of multi-touch gestures where gestures are expressed via logical rules over a set of input facts. We highlight how our rule-based language approach leads to improvements in gesture extensibility and reusability. Last but not least, we introduce JMidas, an instantiation of Midas for the Java programming language and describe how JMidas has been applied to implement a number of innovative multi-touch gestures.

0 Bookmarks
 · 
180 Views
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Using imperative programming to process event streams, such as those generated by multi-touch devices and 3D cameras, has significant engineering drawbacks. Declarative approaches solve common problems but so far, they have not been able to scale on multicore systems while providing guaranteed response times. We propose PARTE, a parallel scalable complex event processing engine that allows for a declarative definition of event patterns and provides soft real-time guarantees for their recognition. The proposed approach extends the classical Rete algorithm and maps event matching onto a graph of actor nodes. Using a tiered event matching model, PARTE provides upper bounds on the detection latency by relying on a combination of non-blocking message passing between Rete nodes and safe memory management techniques. The performance evaluation shows the scalability of our approach on up to 64 cores. Moreover, it indicates that PARTE's design choices lead to more predictable performance compared to a PARTE variant without soft real-time guarantees. Finally, the evaluation indicates further that gesture recognition can benefit from the exposed parallelism with superlinear speedups.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Using imperative programming to process event streams, such as those generated by multi-touch devices and 3D cameras, has significant engineering drawbacks. Declarative approaches solve common problems but so far, they have not been able to scale on multicore systems while providing guaranteed response times. We propose PARTE, a parallel scalable complex event processing engine that allows for a declarative definition of event patterns and provides soft real-time guarantees for their recognition. The proposed approach extends the classical Rete algorithm and maps event matching onto a graph of actor nodes. Using a tiered event matching model, PARTE provides upper bounds on the detection latency by relying on a combination of non-blocking message passing between Rete nodes and safe memory management techniques. The performance evaluation shows the scalability of our approach on up to 64 cores. Moreover, it indicates that PARTE's design choices lead to more predictable performance compared to a PARTE variant without soft real-time guarantees. Finally, the evaluation indicates further that gesture recognition can benefit from the exposed parallelism with superlinear speedups.
    Science of Computer Programming 04/2014; · 0.57 Impact Factor
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Water Ball Z is a novel interactive two-player water game that allows kids and young adults to "fight" in a virtual world with actual physical feedback. The body movement of a player is captured via an RGB-D sensor and analysed by a 3D~gesture recognition engine. In order to enable tactile feedback without the need for wearable devices, a number of water nozzles are positioned around each user's play area. The idea is to translate the input gesture of one player to the corresponding water spray hitting the other player. Besides severely reducing the risk of injury in a fight, Water Ball~Z engages people in a real and fun experience where a hit is physically manifested via a water spray. Furthermore, power up moves and a live scoreboard extension bring the virtual world of Dragon Ball Z and Mortal Kombat cartoons into real (augmented) life. In addition to a detailed description of the physically augmented game, we discuss two new control parameters for mapping gesture input to haptic output which, to the best of our knowledge, are not present in existing 3D~gesture recognition approaches.
    Proceedings of TEI 2014, 8th International Conference on Tangible, Embedded and Embodied Interaction, Munich, Germany, February 2014; 02/2014

Full-text (2 Sources)

View
53 Downloads
Available from
May 17, 2014