Reinventing the Share Button for Physical Spaces
ABSTRACT Imagine if your favorite social networking website provided a "Share to Screen" button that automatically dimmed the room lights and streamed your web-based photos and videos to a nearby television, projector or photo frame. Unfortunately, such fluid interactions between the Web and the physical environment remain challenging, since conventional Web browsers are still unable to directly interact with the broad range of context sources and physical actuators available in highly heterogeneous mobile environments. In our previous work, we presented our plug-and-play context framework, called Ambient Dynamix (Dynamix), which enables advanced context sensing and acting capabilities to be deployed on-demand to mobile devices as plug-ins; enabling entirely new interaction techniques for the Web. To illustrate some of these possibilities, we demonstrate a fully functional Web application that leverages Dynamix plug-ins to stream shared photos and videos from a popular online social network to nearby media rendering devices discovered in the user's physical environment. For a better viewing experience, the demo also adjusts the room's ambient light dimmers through ad-hoc discovery and control of home automation equipment. The demo highlights how Dynamix enables Web applications to discover rich, high-order contextual information, perform context-aware adaptations, and directly influence the physical environment - all from within unmodified mobile Web browsers, such as Google Chrome and Firefox.
[Show abstract] [Hide abstract]
- "As Dynamix plug-ins are based on OSGi (the Open Services Gateway Initiative), they can bundle complex resources, such as support libraries, native code, and protocol stacks. We recently demonstrated that complex service protocol stacks could be successfully bundled and deployed within Dynamix plug-ins . For this work, we enhanced Dynamix with comprehensive inter-plug-in communication capabilities that Fig. 1. "
ABSTRACT: The number of networked smart devices available in everyday environments is rapidly increasing; however, many current devices adopt mutually incompatible networks, protocols, and application programming interfaces. As such, creating mobile applications that dynamically discover and integrate ambient functionality across multiple vertical markets remains challenging. In this paper, we introduce a novel middleware framework, called Ambient Control, which enables commodity mobile devices (e.g., smart-phones) to dynamically mediate control messaging between incompatible smart devices situated in the user's environment. Ambient Control enables a variety of control capabilities and protocol translation services to be dynamically installed into a user's mobile device on-demand using plug-ins. The approach also features an intuitive "Tap to Interact" workflow that allows a user to tap nearby smart devices with a smart-phone to install required interaction plug-ins and automatically "wire" them together in interesting and potentially unforeseen ways. In this paper, we present a detailed overview of the Ambient Control framework, introduce our fully functional prototype, and present an evaluation indicating that the prototype provides efficient messaging performance and imposes low processing and memory overhead, making it suitable for deployment on many commodity mobile devices.IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks, Boston, MA, USA; 07/2015
Conference Paper: An Ad-hoc Smart Gateway Platform for the Web of Things.[Show abstract] [Hide abstract]
ABSTRACT: The Web of Things (WoT) aims to extend the Web into the physical world by promoting the adoption of Web protocols by situated services and smart objects (ambient artifacts). However, real-world ambient artifacts often adopt proprietary and/or non-Web protocols, making them invisible to Web search engines and inaccessible to conventional Web agents. Smart Gateways have been proposed as a way to “Web-enable” proprietary ambient artifacts through intermediary proxy nodes; however, the requisite infrastructure is difficult to deploy at Web scale. To address such challenges, we are developing Ambient Dynamix (Dynamix): a plug-and-play context framework for mobile devices, which enables Web agents to interoperate with non-Web ambient artifacts – directly from the browser. In this paper, we describe how Dynamix can be used to transform the user’s device into an ad-hoc Smart Gateway in-situ, enabling Web applications (in the device’s browser) to seamlessly interact with non-Web ambient artifacts in the physical environment. We describe an operational prototype implementation, which enables Web apps to discover and control nearby UPnP and AirPlay media devices uniformly. We also present a performance evaluation that indicates the prototype imposes low processing and memory overhead, and is suitable for deployment on many commodity mobile devices.IEEE International Conference on Internet of Things (iThings 2013); 08/2013
- [Show abstract] [Hide abstract]
ABSTRACT: Advances in Human Computer Interaction techniques continue to enrich Natural User Interface (NUI) research. While dozens of novel NUI interaction techniques are proposed every year, the potential of the human body’s sensory and motor systems is not yet fully utilized. Hence, new pressing calls have emerged for exploring the potential of the whole body in motion when interacting with real-world pervasive and ubiquitous computing ecosystems (ambient spaces). Given the adoption of NUI paradigm in ambient spaces, users will be increasingly expected to interact with multiple interactions techniques simultaneously. Whilst NUIs provide rich interaction possibilities and alternatives, they also introduce critical challenges for interactive ambient spaces. This dissertation aims to tackle three of these challenges; namely, large-scale dynamic runtime deployment of existing and future interaction techniques; long-term and adequate record-keeping and dissemination practices for interaction techniques; and in-situ adaptation of interaction possibilities. These challenges are often fueled by users’ increased mobility; the increasingly heterogeneity and availability of interaction resources; and the increasing diversity of the physical abilities of many user populations (e.g., elderly users). This dissertation presents a novel approach for adapting the interaction modalities available to a given application at runtime (as deployable interaction plugins). Accordingly, the capabilities and behaviour of an interactive system are optimized to fit the users’ physical abilities, needs, and context. The approach includes a theoretical concept (called Interaction Ensemble) that relies on decoupling the often tight binding between devices, interaction techniques, and applications. A reference implementation (called the STAGE framework) is presented as an evaluation of the concept.08/2014, Degree: Doctoral Degree (Dr.-Ing.)