Conference PaperPDF Available

Abstract and Figures

In this paper we present a novel software-based home control platform as an add-on to a set-top box (STB) for digital television. By means of using merely a remote controller, traditional STB users become able to control lights, appliances and media playback in their homes. Intelligence and awareness is achieved with a support for execution of recipes – pre-prepared scripts that define timely actions and respond to triggers obtained from sensors. Software abstraction layer facilitates integration of any desired communication protocol. In our prototype, we supported Zigbee and DMX for light control, X10 for light/appliances control over power line, as well as Ethernet-based optical cameras as motion / presence sensors and UPnP / DLNA based equipment for distributed media playback.
Content may be subject to copyright.
Set-top box-based home controller
Milan Z. Bjelica
Faculty of Technical Sciences
University of Novi Sad
Novi Sad, Serbia
milan.bjelica@rt-rk.com
Istvan Papp
Faculty of Technical Sciences
University of Novi Sad
Novi Sad, Serbia
istvan.papp@rt-rk.com
Nikola Teslic
RT-RK LLC
Novi Sad, Serbia
nikola.teslic@rt-rk.com
Jean-Marc Coulon
J-M Creativ
Nivelles, Belgium
jean-marc@jmcreativ.com
Abstract — In this paper we present a novel software-based home
control platform as an add-on to a set-top box (STB) for digital
television. By means of using merely a remote controller,
traditional STB users become able to control lights, appliances
and media playback in their homes. Intelligence and awareness is
achieved with a support for execution of recipes – pre-prepared
scripts that define timely actions and respond to triggers
obtained from sensors. Software abstraction layer facilitates
integration of any desired communication protocol. In our
prototype, we supported Zigbee and DMX for light control, X10
for light/appliances control over power line, as well as Ethernet-
based optical cameras as motion / presence sensors and UPnP /
DLNA based equipment for distributed media playback.
Keywords: domotics, smart home, set-top box, Zigbee, DMX.
I. INTRODUCTION
Attempts to make everyday living more convenient and in-
house computing ubiquitous are not new. Achieving true,
context-aware and user-aware, intelligent ambience has been
traditionally assuming a complex installation of various
equipment. Integrating sensors, lighting and multimedia
seamlessly into one system has always been a daunting task.
Such systems have been hardly accessible for an average
home in terms of cost and installation levity. Instead, users are
offered solutions to a number of heterogeneous home
applications [1]. Making the home intelligent or autonomous
requires the user to acquire, mount and install different pieces
of hardware, often including lots of wiring and non-uniform
standards. An appealing alternative would be to utilize
traditional equipment users are accustomed to, such as set-top
boxes, as home/ambience controllers.
Most modern STBs run control software upon a POSIX
compliant operating system (recently, Linux). This control
software mostly runs on 32-bit processing unit that is often
under-utilized. Moreover, network-ready solutions are
equipped with an Ethernet connector, while USB connectors
are also embedded. The way to turn a STB into a home
controller without any hardware changes is to connect a USB
stick with home control software on it, and to activate home
control graphical user interface (GUI) by pressing a remote
controller button. In this paper we present such a solution.
Our solution responds to several key challenges. Portability
is guaranteed by the use of open standards (POSIX/C).
Concept of recipes enables the use of the controller for various
home setups, consisting of heterogeneous devices, making the
solution scalable. Existence of an abstraction layer provides
extendibility, when support for new devices/protocols equals
to writing additional plug-ins. Ability to integrate virtual
sensorial plugins that provide behavioral estimation, imparts
the creation of an awareness system. Abstraction layer helps
bridge confronted approaches in home automation nowadays,
that complicate standardization efforts (new wires, existing
wires, or wireless?). Customizability is achieved by a software
model [2] that provides separation of core functions from UI,
so the appearance of the software can be tailored to
accommodate different interaction requirements. Finally, the
solution can be used to intelligently control lights and
appliances, with the goal to reduce their energy consumption,
contributing to energy savings in the household.
In the next section we give an overview of current situation
in the home automation, and present works related to this
paper. In Section 3 we describe the architecture of our
solution, while in Section 4 we provide details of the
demonstration setup and our prototype, together with a use
case that underlines immersive experience for users inside a
living room. Section 5 presents experimental results that show
how our controller can help save energy in an average
household.
II. RELATED WORKS
Commercial influence to standards in home automation
nowadays is overwhelming. Markets across the globe,
dominantly in North America, Europe and Japan are
introducing new, yet mutually incompatible wiring/wireless
technologies and standards. Potpourri of available protocols
renders the situation very complex on global scale when
achieving interoperability gets harder each day. Nonetheless,
three main trends are clearly distinguishable: (1) dedicated
wired networks; (2) existing wires reuse and (3) wireless
connectivity.
The provision of a dedicated wired network is a most
frequent approach. In this regard proprietary solutions are
available (e.g. LonWorks [3]), but more general technologies
are increasingly popular (e.g. Ethernet), preferably for
networks of general purpose devices (computers) and high
bandwidth requirements. Based on these physical protocols,
various data transport protocols are implemented (X10 [4],
CEBus [5], HBS [6]) while neither one of them is yet globally
dominant. Global interoperability in wired connectivity for
home automation is pursued by a European initiative, called
Konnex open standard [7], which involves EIB, EHS and
BatiBus protocols (earlier developed independently).
Making use of the existing wirings is also very interesting
approach, given the relief to the users in terms of installation
levity. For example, using power line for carrying data is a
This work was partially supported by the Ministry of Science and
Technological Development of Republic Serbia under
the project No. 11005,
year 2008.
2010 IEEE 14th International Symposium on Consumer Electronics
978-1-4244-6673-3/10/$26.00 ©2010 IEEE
basis for EHS, X10, or Homeplug [8]. Main problems in this
area revolve around higher noise level then dedicated wirings
and safety concern with simple power line protocols (X10 is in
theory even forbidden in Europe).
Lastly, wireless technologies are also very attractive, where
infrared, or radio link technologies (e.g. IEEE802.11b) are
most frequently utilized. Radio link is usually adopted for
guaranteeing connectivity at higher distances.
Nevertheless, with the availability of complex controllers
and processors at lower costs, technologies that are well
established for common PCs are being revisited for numerous
embedded devices. It is not unusual therefore, that simple
computer systems are being used to bridge this
interconnection gap, having the cheap PC act as a
home/ambient controller. On top of PC-based architectures,
user interfacing is often provided by a simple web server,
when PC is exposed to the global network and can be accessed
from any internet hot spot. Also, GSM or GPRS networks can
be used with a standard mobile provider, to allow control of
the system via short message (SMS) or a phone call. PC-based
software packages that provide management of supported
devices with the goal of home automation are scarce, at least
when it comes to providing ambient intelligence and
awareness (ETS [9], commercial; EIBcontrol [10], open
source). Dedicated home automation controllers are also
deprived of high level ambient functions. For example, the
OSGi Alliance [11] has already proposed a rather complete
and complex specification for a residential gateway (and
alike). Here, protocol and integration issues, as well as
intelligence related behaviors, are demanded to third-party
“bundles”, that can be dynamically plugged into the OSGi
framework at run-time. An additional effort is therefore
required to enhance the system with a compact yet general
approach to intelligent and automatic interoperability among
the involved devices. Connecting devices together and
controlling them from a unique spot is a subject that is actively
pursued by another open source community, under the project
Open Remote [12].
Generalized home control is also the subject of additional,
independent researches. Coyle et al. in [13] presented an idea
for home control middleware, trying to integrate components,
sensors and different applications. Similarly, Casimiro et al.
presented their work on the architectural framework for smart
components in [14]. Pellegrino et al. in [15] proposed a
layered software concept for applications in domotics.
Application of sensor-actuator networks in home automation,
with accent to lighting was given by Gauger et al. in [16].
There have also been works regarding GUIs for home control.
Koskela and Vaananen evaluated PC, TV and a cell phone as
GUI devices for home control [17]. Using smart phones to
control appliances, with a notion of Personal Universal
Controller, was proposed by Nichols and Myers [18]. Home
networking and control based on the popular UPnP/DLNA
stack of protocols was a foundation for a prototype
implemented by Yiquin, Fang and Wei [19].
It can be noted, to the best of our knowledge, that there
have been no researches which tried to extend a STB or any
existing home device with home control functions. In this
paper, we have implemented software that can easily fit STBs
of different architectures, possibly by a B2B model with a
manufacturer. This proposal is commercially exciting because
it complies with users’ habits, trying to introduce new
functions under the helmet of already proven and widely
accepted devices and applications.
III. HOME CONTROLLER PROTOTYPE
The software is divided into a two components: the core
with core drivers and the UI. Software architecture is
presented in Figure 1.
Core
Protocol Driver
Protocol Driver
Protocol Driver Plug-in
mechanism
Drivers
layer
Drivers controller
Drivers
control layer
Modules controller
Modules
control layer
Abstraction layer
Recipes controller
Recipes interpreter
XML parser
System time
Kitchen layer
User sessions manager
OBEX-based protocol
Session layer
UI
Communication library
UI client back-end
SWF framework
DirectFB
POSIX
device
Figure 1. Layered software architecture
A. Core component
Core component is “hidden” within a standard POSIX
device, and acts as a home control server providing all desired
functions for client UI applications (listing home devices,
playing/stopping recipes, issuing a command etc).
Communication to this device is done with standard read/write
calls, by using an open communication protocol based on
OBEX [20]. This way it is possible to access core software
even from a remote platform in charge of user interaction,
leveraging on the total processor load. For example, while
core home control runs on a STB, user can invoke functions
from his cell phone or via web.
Core component is developed as an atomic, monolithic
software module written purely in C language, avoiding
operations that rely on concepts of stack, endianess or
threading. This module contains API that is wrapped in a
POSIX device and rolled into Linux kernel as a .ko object.
While nursing the intention to facilitate porting of core module
to any other architecture, its linkage to Linux kernel in a
monolithic manner (like kernel itself is), rounds up the
robustness and seals the most essential part of the system.
Core software comprises of several needed software layers.
Communication with a device (Zigbee, UPnP, Ethernet
camera, X10 etc) is provided by drivers layer, which can be
extended by plugins to support new protocols. A driver is in
charge for passing on commands to physical devices and to
report events that occurred on a device. Drivers layer defines a
simple C-based interface that defines structures holding
pointers to functions with a fixed API, that all plugins must be
conformant to. Software also supports a concept of a virtual
driver that can have behavioral modeling capabilities,
therefore providing awareness. These virtual awareness
drivers are by no means different in interfacing to the core
component than regular, physical drivers. Behavioral
modeling and classification results (e.g. detected user activity)
are regarded as raw data by the upper layers. In this case,
virtual drivers must handle physical communication to all
sensors they are dependable upon. Behavioral estimation
concepts and virtual drivers inside the system described in this
paper are presented in separate researches in [21] and [22].
A unified view to all drivers, their initialization and storage
is provided by drivers control layer. This layer loads all driver
plugins, enumerates them and provides mapping of symbolic
names coming from upper layers, to a driver/device pair of
identifiers (and its corresponding structure holding API
functions). This layer provides a mechanism for automatic
profiling, by measuring busyness of each driver (average time
needed for execution of its functions). Based on this profiling,
drivers control layer sets priorities to each function and orders
a priority-based delta-list. This way the real-time constraints
of a home automation system are respected (timely reaction to
sensor triggers, synchronization of light effects etc).
Since the household consists of modules (lights, appliances,
sensors) as control objects, modules control layer is
introduced. This layer enables issuing commands to modules
(e.g. toggle light, play video file, set color of a DMX led
lamp), as well as registering for different events that modules
may generate (e.g. presence detection, light level change). The
basic purpose of this layer is to provide abstraction to above
scripting, eliminating the need of knowing physical details of
addressed devices. Primitives provided by modules control
layer can be invoked indirectly, from upper layers that
interpret recipes, or directly, by a user command from UI.
Drivers layer, drivers control layer and modules control
layer together form an abstraction layer, hiding all details on
devices, protocols, behavior modeling and storage from layers
above.
Pre-scripted recipes are the most useful means of home
control within the developed system. Home behavior is
defined by XML-based recipes that are composed of sequence
of module control operations. Recipes define a sequence of
action when a certain complex condition is met. The criteria
based on which the condition is evaluated rely on the
reception of events, coming from physical devices or virtual
behavioral drivers. For example, user may want to trigger an
ambient lightshow and a multimedia presentation on an LCD
panel in the room, if there are more than four people in the
room on Saturday evening. Although recipes can be scripted
manually and provided directly by the user, the intention is
that these are generated automatically based on user actions on
GUI. XML-based language used for scripting uses constructs
common to high-level programming languages (loops,
selections, declaring variables, basic arithmetic) but remains
fast and easy to interpret (we adapted and used extremely light
McbXML parser for this purpose). A basic concept for the
creation of the interpreter is reused from our previous works
[23]. Example of a recipe for the aforementioned party
ambient setting is given in Figure 2. Interpreting recipes,
storing them and controlling their playback are all functions
provided by the kitchen layer.
<driver type=”signal” module=”time” stat=”reached” val=”20: 00”>
<linked type=”signal” module=”date” stat=”is” val=”Sat”/>
<linked type=”signal” module=”presence” stat=”reached” val= ”4”/>
<print> Recipe for triggering party </print>
<print>
Description: When there are more than 4 people
on Saturday evening, trigger lightshow and multim edia.
</print>
<print> Author: John Doe </print>
<print> </print>
<var name="counter" val="1"/>
<for>
<if>
<case namea="counter" val="1" rel="=">
<for loop=3>
<instruct module=”rgb1” function=”show1” />
<instruct module=”rgb2” function=”show2” />
<instruct module=”video” function=”par.avi”
/>
</for>
</case>
<case namea="counter" val="2" rel="=">
<for loop=3>
<instruct module=”rgb1” function=”show2” />
<instruct module=”rgb2” function=”show1” />
</for>
</case>
<default>
<instruct module=”video” function=”fadeout” />
<var name="counter" val="1"/>
</default>
</if>
<var name="counter" val="++"/>
</for>
</driver>
Figure 2. Example recipe for party ambient creation
Finally, there is a user sessions layer, in charge of
communication with (G)UI layers by implementing an OBEX-
based session protocol. Communication primitives are
attached directly to the read/write calls of the home control
POSIX device that represents core to the user. Therefore, to
use functions of the core, user needs to establish the
connection first. Each connection is uniquely identified, and a
separate session is created for each user. Interaction to the
core based on the communication protocol facilitates
integration of different remote UI units (e.g. mobile phone,
PDA computer) without the need to implement additional
communication protocols. The only concern when using core
for such UI applications is how to provide adequate data flow
from the communication socket used for connecting remote UI
device to the POSIX home control device. For this purpose, a
switching daemon application is used. This application listens
to different communication interfaces (Bluetooth, Ethernet,
RS232) for data. When data is received, it is read to a buffer
(POSIX read) and written to home control POSIX device
(POSIX write). Daemon also listens to home control POSIX
device and writes data back to the originating interface.
Therefore, daemon serves only as a connection point for data,
while core component handles all the communication logic.
POSIX home control device is also targeted from an
application residing locally on the STB. This is illustrated
inside the diagram in Figure 3.
The main characteristic of core component is its lightness,
with about 400Kb in size of sources (20 pure C source files,
without core drivers) and ~70kb the footprint of the kernel
object at runtime. So far, core component has been tested on
32-bit big-endian MIPS processor (with uCLinux [24]), as
well as on 32-bit and 64-bit PC processors, running XUbuntu
[25].
Figure. 3 A concept of interfacing the core home control component
B. Core drivers
To provide physical communication with home control
devices, we developed/integrated several drivers, placed
within drivers layer. Given the extensible plugin mechanism,
additional drivers can be written and added at any time. The
following drivers were provided: (1) DMX light driver (for
controlling DMX-based RGB lights connected over KWL
DMX2USB adapter [26]; (2) Smart sockets driver (for
controlling power on power outlets over Zigbee and UZBee
USB2Zigbee dongle [27]); (3) Powerline light driver (for
dimming lights using X10 protocol and Marmitek CM11 X10
transceiver [28]); (4) Ethernet camera driver (for
communication with AXIS M1011 Ethernet camera [28]); (5)
Multimedia driver (for controlling UPnP/DLNA playback on
compatible devices attached to Ethernet) and (6) behavioral
driver (gesture recognition using accelerometer – development
is in progress). We believe that given set of drivers is
sufficient for most home applications (light effects control and
ambient light over DMX, basic light control over X10,
multimedia control over UPnP, user detection with camera and
gestures (accelerometer), mechanical actions – curtains and
windows blinds control, garage door control etc. over X10).
DMX is a connectionless protocol with a simple addressing
scheme (512 addresses per bus) where in practice each address
corresponds to one light channel. For example, DMX Led
lamp we used in development had four address channels (Red,
Green, Blue and Mode, the fourth one defining light behavior,
e.g. strobe). When the Mode channel is set to 189, lights are
always of solid colors. Developed driver provided the
following functions: (1) Set color of a device to R, G, B; (2)
Fade color of a device to R, G, B in time T and (3) start a
lightshow with given name. Parameters of each DMX light are
its addresses and their purpose. Real-time constraints for
fading effects were respected by implementing fading for all
addressed DMX devices within a single thread, approaching
the target value in steps different for each DMX light,
depending on given T. Light shows were predefined in driver
to make the scripting easy for users that do not want highly
customized effects.
Smart socket driver was based on a system developed in a
research by Radin et. al in [30], with the addition of Zigbee
protocol support [31]. This driver enabled users to set a
specific socket (defined by its ID) to the given output power.
Sockets itself are based on Zigbee communication module and
TRIAC component for setting power. Therefore, sockets are
intended to be used with simple devices, like lamps and bulbs.
The driver provides the ability to read actual power value on
each socket, making the commands reliable.
X10 driver implements communication with Marmitek X10
transceiver. This device is able to send dim/bright commands
over power line to counterpart receivers connected to sockets
in the same power network within the household (LW12 [32],
AM12 [33]). X10 driver provides the ability to
increment/decrement current brightness value on a target
module (and therefore the light connected over it), or to switch
relay-based modules on or off (and therefore switch a target
device on or off – garage door motors, window blinds etc).
Ethernet camera driver was originally tested with single
camera, Axis M1011, but it may work well with any Ethernet
camera supporting Motion JPEG [34] formats. By default,
camera driver performs basic presence detection and people
counting. These algorithms are extremely simple (given the
constraints of target processing power), working for small
resolution images (320x240 pixels) while the output is
calculated by comparing last received image with a reference
background image taken periodically (this period can be
adjusted). If target STB does not have enough processing
power even for this operation, driver can be set to listening
mode, receiving event notifications on presence detection
reported by Axis camera (using one of the embedded
functionalities of this camera model).
Multimedia driver is based on AVSXLib library [35], acting
as a control unit for UPnP/DLNA compatible devices.
Therefore, serving multimedia content and its presentation is
not performed within the core. This approach appears very
convenient, given the number of UPnP/DLNA compatible
servers/renderers on the market nowadays (and growing). User
is able to start/stop/pause/resume multimedia playback on a
target renderer device specified by its friendly name. Content
provider (server) is also defined by stating URI (Uniform
Resource Identifier) of the served content.
Behavioral driver is currently in development, and its
intention is to detect hand gesture-based commands of users in
the household, wearing accelerometer-equipped bracelets with
RF. As a result, users can make meaningful gestures, setting
currently used ambient profile in the household. For example,
a recipe can be written to trigger change to the specific room
ambient, when a user starts waving his hand.
C. User Interface
User interface construction is dependant on the STB device
used (since GUI applications are utmost processor and
memory consumers). In our case, STB was based on 32-bit
MIPS with 128MB of memory. Accordingly, UI layer in our
prototype consists of a shockwave flash based GUI, with a
simple client back-end in charge of communication with the
core. This approach provides portability and facilitates GUI
development. For rendering swf file format and showing it on
screen, swfdec Linux library was used [36], on top of libgtk2
(cairo) [37] used for drawing operations and event system.
Drawing board for the creative designers in this case was very
convenient, since all the development could be done on their
home PC. Socket connection is established to back-end part of
GUI application, running locally that receives important GUI
events from Flash ActionScript [38]. All requests made by
users inside this flash application (start recipe playback, turn
Figure 4. Flash-based GUI page of home automation system rendered
by Set-top box
the light on/off), are packed into TCP/IP socket packets and
sent to the back-end. Then, a communication primitive of core
software is invoked, to activate actual home control operation.
The appearance of one GUI page (recipe creation) on LCD
screen is given in Figure 4. Depending on the STB
possibilities and target STB environment, different GUI
wrappers can be created. For example, usage of GUI based on
GTK widgets only is also viable [39][40].
IV. CASE STUDY
STB used for the implementation and demonstration was
based on a chipset for digital television from Trident
Microsystems. The core of the chipset is the 32-bit MIPS
processor with a real-time uCLinux OS. Prototype setup
consisted of a core and UI applications located on a USB stick
connected to USB hub and then to the available USB port on
the STB. To USB hub we also connected Marmitek CM11
adapter for lights/appliances control over power line, KWL
USB2DMX adapter to control RGB led lamps over DMX bus,
and UZBee USB2Zigbee dongle to control intelligent power
sockets. We used Ethernet to connect STB with Axis M1011
Network Camera with embedded motion detection abilities
and with UPnP software on another PC: Twonky Media
Server as multimedia content provider and Simple Center as a
renderer application. Setup is presented in Figure 5.
By using a simple on screen menu users were able to choose
one of the available recipes and set up ambient profiles for
their home. Each mode consisted of a specific DMX lightshow
and multimedia playback, tuned additionally depending on the
number of people in the room. Ceiling bulbs were controlled
over X10, while bedside lamps were connected to Zigbee
power sockets (and one of them over AM12 appliance module
with a relay, for demonstration purposes). Ambient was
created by using two DMX EcoLed RGB lamps. When no
people were present, ceiling lights and bedside lamps were
dimmed or switched off to save energy.
No formal survey of users has been conducted, but their
impressions were generally positive and the ambience has
proven to be indeed immersive. The main positive feedback
was addressing the simplicity of control, while one particular
Figure 5. System architecture used for demonstrating the prototype
user emphasized the control seemed “natural”. When asked to
clarify, he replied: Laid back in a chair and just pressed a
button on the remote as if I was changing the channel and
voila! There I was, even closer to the football match”. This
comment is encouraging in the sense that our choice of STB
(TV set) as a device that users are traditionally fond of is
likely to be a winner. When referring to the football match,
user selected ambient profile that was closely related to the
program watched. Therefore, we plan to enhance the system to
set the profile automatically, based on EPG data fetched from
the DVB stream of currently watched program.
V. EXPERIMENTAL RESULTS
Given the increasing trend of providing energy-saving
applications, we tested our prototype to see if and how it can
be used to help save energy in an average household. Energy
saving was performed by utilizing two common approaches,
both supported natively in our system (by an appropriate
recipe): (1) if there is nobody present in the room for more
than n minutes, turn all lights in that room off, and (2)
decrease brightness of all lights in the household by 10%
(what should be unnoticeable to users).
For our experiment, we installed the system to one of our
co-worker’s living room. We used Zigbee power sockets to
control a bedside lamp and two ceiling bulbs in the living
room, and Ethernet camera for presence detection. On the STB
we had our software running a recipe that was detecting if a
light was on to change its brightness by 10% (in small steps,
so that the change does not appear sudden). Another recipe
running in parallel was monitoring the state on the camera.
When there were no people in the room for more than 2
minutes, all three lights would be switched off. The setup
effort was marginal due to the scalability of our solution.
To be able to provide comparison, we extended our original
smart socket driver, so that it logs the following: (1) current
power consumption on all three sockets (bulbs) periodically
(once every 10 seconds), (2) switch toggling moments (change
in power on any socket - on to off and vice versa). We also
extended camera driver to log changes in presence detection.
Then, we recorded energy consumption, switch toggling
0
50
100
150
200
250
300
350
0
0
1
1
2
3
3
4
5
5
6
7
7
8
9
9
10
10
11
12
12
13
14
14
15
16
16
17
18
18
19
19
20
21
21
22
23
23
Hour (t)
Watts (W)
Without energy saving
With energy saving
moments and presence changes for a day of operation in the
living room without energy saving recipes activated, to be
used as a control result. Using the same bulbs and equipment,
experiment was repeated the next day. Smart socket driver
was previously modified to be able to play back switch
toggling data, in order to simulate user actions and control
bulbs accordingly. Camera driver was also modified, to play
back presence changes and emit virtual events. In the repeated
experiment, energy saving recipes were activated. This way
we achieved the comparison of two experiments, in which
people behavior was identical: one without and the other with
energy saving functionality. Note that we assumed that
automatic light control would not affect original people
behavior in toggling switches. Turning light on when it was
already on, or off when it was already off, are events that we
disregarded in the second experiment. Results of the
experiment are shown in Figure 6 (consumption during the
day) and in Table 1 (total daily consumption).
TABLE I. ENERGY SAVINGS WITH STB-BASED HOME CONTROLLER
Energy consumption
(one day, 3 bulbs of 100W)
STB-based HC control Regular control
472 W 1256 W
Figure 6. Energy consumption during a day without energy saving
(blue) and with energy saving (red)
VI. CONCLUSION
In this paper we presented a prototype system for home
automation and ambient intelligence, fit into a standard set-top
box device. One of the commercial benefits of implemented
prototype is the introduction of new technology to users
seamlessly (through SW for their familiar device) therefore
helping STB manufacturers to enter new market with
minimum risk. Apart from increasing users’ ambient
experience, the solution provides energy saving because the
brightness of lights and their toggling are controlled by
recipes, depending on the need. Future work would include
steps toward achieving better ambient intelligence and user
awareness, by introducing advanced behavioral modeling and
user activity detection capabilities.
REFERENCES
[1] Home Automation Index, http://www.homeautomationindex.com
[2] Bjelica, M, Z, Papp, I, Samardzija, D, Jovanovic, Z, A Software Model
with Remote User Interface for Embedded Systems, in proceedings of
conference TELFOR 2009, pp. 1205-1208
[3] LonWorks, www.echelon.com
[4] X10 Knowledge Base, http://kbase.x10.com/wiki/Main_Page
[5] CEBus community forum, http://www.cbusforums.com/forums/
[6] Sugihara, K, Kobatake, S, Shirai, H, Oowada, H, Yoshitomi, K, HBS-
standard-compatible home bus protocol controller, IEEE Transactions
on Consumer Electronics, Vol. 35, No. 3, 1989, pp. 605-607
[7] KNX, www.knx.org
[8] HomePlug Powerline Alliance, http://www.homeplug.org/home/
[9] Eng. Tool Software (ETS), http://www.knx.org/knx-tools/ets/description/
[10] EIB Home Server, http://sourceforge.net/projects/eibcontrol/
[11] OSGi Alliance, www.osgi.org
[12] Open Remote: The Digital Home Operating System
http://www.openremote.org/display/HOME/OpenRemote
[13] Coyle, L, Neely, S, Stevenson, G, Sullivan, M, Dobson, S, Nixon, P,
Sensor Fusion-Based Middleware for Smart Homes, International
Journal of ARM, Vol. 8, No. 2, June 2007, pp. 53-60
[14] Casimiro, A, Kaiser, J, Verissimo, P, An Architectural Framework and a
Middleware for Cooperating Smart Components, in proceedings of
CF’04 Italy, ACM, 2004, pp. 28-39
[15] Pellegrino, P, Bonino, D, Corno, F, Domotic House Gateway, in
proceedings of SAC’06 France, ACM, 2006, pp. 1915-1920
[16] Gauger, M, Minder, D, Marron, P. J, Wacker, A, Lachenmann, A,
Prototyping Sensor-Actuator Networks for Home Automation, in
proceedings of REALWSN’08 United Kingdom, ACM, 2008, pp. 56-60
[17] Koskela, T, Vaananen-Vainio-Matilla, K, Evolution towards smart home
environments: empirical evaluation of three user interfaces, Personal
Ubiquitous Computing, Vol. 8, 2004, pp. 234-240
[18] Nichols, J, Myers, B. A, Controlling Home and Office Appliances with
Smart Phones, IEEE Pervasive Computing, 2006, pp. 60-67
[19] Yiquin, L, Fang, F, Wei, L, Home Networking and Control Based on
UPnP: An Implementation, in proceedings of Second International
Workshop on Comp. Sci. and Engineering, Vol. 2, 2009, pp. 385-389
[20] Megowan, P, Suvak, D, Kogan, D, IrDA Object Exchange Protocol
OBEXTM, Infrared Data Association, Version 1.2, 1999
[21] Bjelica, M. Z, Teslic, N, A concept of usability assessment for user-
centered multimedia applications, in proceedings of International
Multiconf. on Comp. Sci. and Technology, Poland, 2009, pp. 443-450
[22] Bjelica, M. Z, Teslic, N, Multi-purpose User Awareness Kit for
Consumer Electronic Devices, in proceedings of International Conf. on
Consumer Electronics (ICCE), Las Vegas, USA, 2010, pp. 239-240.
[23] Bjelica, M. Z, Savic, M, Vujanovic, V, Temerinac, M, Realization of a
PC application for communication subsystem of an integrated circuit
emulation, in proceedings of conference TELFOR 2008, pp. 759-762
[24] UCLinux OS, http://www.uclinux.org/
[25] XUbuntu – Linux for human beings, http://www.xubuntu.org/
[26] KWL DMX2USB, www.feno.com/products/detail/fc-dmx-512u.html
[27] UZBee USB2Zigbee dongle, http://www.rfsolutions.co.uk/acatalog/
UZBee_ZigBee_USB_Dongle.html
[28] CM11 transceiver, http://www.marmitek.com/en/manual/9647.pdf
[29] Axis M1011, http://www.axis.com/products/cam_m1011/accessories.htm
[30] Radin, B, Nuhijevic, V, Pjevalica, N, An Intelligent Power Supply Outlet
as a Solution in Smart House, TELFOR 2009, pp. 815-818
[31] Zigbee protocol, http://www.zigbee.org
[32] LW12 Module, http://www.marmitek.com/en/manual/9611.pdf
[33] AM12 Module, http://www.marmitek.com/en/manual/8915.pdf
[34] Motion JPEG formats, http://en.wikipedia.org/wiki/Motion_JPEG
[35] Maric, S, Bjelica, M. Z, Mrazovac, B, Teslic, N, An implementation of
scripted control for a distributed multimedia system, in proceedings of
conference ETRAN 2010 (to be published)
[36] Swfdec Linux library, http://swfdec.freedesktop.org/wiki/
[37] GTK, www.gtk.org
[38] Flash ActionScript, http://www.actionscript.org/
[39] Bjelica, M, Z, Teslic, N, A concept and implementation of the
Embeddable Home Controller, in proc. MIPRO 2010 (to be published)
[40] Bjelica, M, Z, Teslic, N, Papp, I, Savic, M, A characterization to
evaluate graphical user interface frameworks for television receivers, in
proceedings of conference TELSIKS 2009, pp. 285-288
... Particularly, combined use of device and simple sensors, with the help of a dedicated visual platform, could overcome the abovementioned barriers obtaining conscious use and easy installation of sensors, user engagement and misleading feedback perception if the provided information are consistent with the desired knowledge goal [13][14][15]. In order to establish the communication between devices and end users, the approach is to use the so called recipes [16][17][18]. As stated by [17]: "recipes define a sequence of action when a certain complex condition is met. ...
... In order to establish the communication between devices and end users, the approach is to use the so called recipes [16][17][18]. As stated by [17]: "recipes define a sequence of action when a certain complex condition is met. The criteria based on which the condition is evaluated rely on the reception of events, coming from physical devices or virtual behavioural drivers." ...
... Les passerelles de services permettent à des fournisseurs de services de pouvoir déployer E conduits en ce sens. On peut citer par exemple le projet SIRLAN (Caelen et al., 2002) EnTiMid (Nain et al., 2009), les travaux de Donsez (Bottaro et al., 2007), les travaux de Bjelica (Bjelica et al., 2010) ainsi que les travaux de Bonino (Pellegrino et al., 2006;Bonino et al., 2008). C nt dynamique ainsi que le besoin (Helal et al., 2005) L comporte principalement trois couches : une couche bas niveau, ou physique rents réseaux dédiés en interne (accès aux compteurs intelligents, télévision connectée, réseaux domotiques...) C représenter les données issues de la couche bas niveau. ...
... La synchronisation des processus et la programmation horaire sont des fonctionnalités jugées importantes dans le cadre du bâtiment intelligent (Bjelica et al., 2010) L s pour être connectés à un serveur de temps, ce ûteuse à obtenir. La virtualisation des automatismes nous permet dans notre cadre de pouvoir utiliser des NTP Network Time Protocol, spécifié par la RFC 1305 49 ) pour redistribuer cette information au sein des autres systèmes. ...
Article
Full-text available
The notion of comfort in homes is a major problem to solve environmental problems (consumption and emissions of buildings), economic (reduction of operating costs) and social (maintenance and home care) that define sustainable development. However, this notion of comfort is complicated by the number of parameters that integrates both human (perception) and physical (measurement) parameters. Our study aims to model the concept of comfort in a smart home. Smart homes emerged in the early 2000s, and are positioned as heir to home automation, benefiting from technological advances illustrated by ubiquitous computing and artificial intelligence, ambient intelligence concepts formants. The first part of this study was to define habitat intelligent, formalizing the gains (home automation) and research issues, in terms of knowledge representation by the models. Our approach to intelligent building allowed us to define a framework for interoperability: a middleware able to focus and control the parameters of an environment. This interoperability is required by the heterogeneity of communicating objects that make up a habitat: heterogeneity of applications, communication protocols, know-how and practices. This first part of the study allowed us to instrument an experimental platform: the platform Domus. Thus, by restoring an apartment, and by providing it with smart objects, we could implement, through interoperability, an intelligent environment, environment characterized by high information density and capacity reaction. The realization of this platform is needed to address various topics related to housing, such as comfort. Indeed, ambient intelligence brings a new dimension in this research framework: ubiquity. The increasing density of sensors allows us to collect more information, not only the environment but also on the user and its behavior, thus defining a new approach to comfort: Adaptive comfort. Most of the work focusing on thermal comfort, we are interested in multi-sensory comfort. This allows one hand to take into account all the parameters that enhances an environment (air, sound, sight) but also allows our attention to cross-sensory effects that can induce these terms on the occupier. For example, it is suspected the temperature light (red light / heat, light blue / cold) to affect the perception of heat. Experiments have shown the practical approach and the theoretical approach of multi-sensory effects. The results of this study will be led to participate in the definition of "comfortmeter", a tool to sense comfort for both habitat and the inhabitant.
... En domotique, plusieurs travaux ont déjà été conduits en ce sens. On peut citer par exemple le projet SIRLAN (Caelen et al. 2002), l'intergiciel EnTiMid (Nain et al. 2009), les travaux de Donsez (Bottaro et al. 2007), les travaux de Bjelica (Bjelica et al. 2010) ainsi que les travaux de Bonino (Pellegrino et al. 2006;Bonino et al. 2008). Ces travaux expriment tous la nécessité d'un environnement dynamique ainsi que le besoin d'abstraire les données. ...
... La synchronisation des processus et la programmation horaire sont des fonctionnalités jugées importantes dans le cadre du bâtiment intelligent (Bjelica et al. 2010). L'heure et la date sont des variables primordiales, mais cependant, les systèmes d'automatismes n'étant pas prévu pour être connectés à un serveur de temps, cette information peut s'avérer couteuse. ...
Article
Dans le domaine du bâtiment intelligent, et plus généralement de l'intelligence ambiante, il est nécessaire, pour prototyper et évaluer l'usage des services proposés, de pouvoir commander et lire l'état des nombreux objets présents au sein de l'environnement. Or ces objets ont le plus souvent des moyens de contrôle et des modèles de comportement hétérogènes, même s'ils sont de nature semblable. Ainsi, les intergiciels constituent un élément essentiel pour l'intelligence ambiante. Ces composants permettent de collecter des données issues d'un ou plusieurs objets pour permettre à des services de plus haut niveaux de s'adapter. Ils permettent également, dans certains cas, de pouvoir réagir sur l'environnement. Nous proposons dans cet article un modèle d'interopérabilité pour un tel intergiciel. Nos travaux s'appuient sur une approche centrée sur les objets communicants, tout en intégrant une culture proche du terrain. Nos travaux ont pour particularité d'avoir été développés et validés au sein d'une plateforme expérimentale. ABSTRACT. In the field of intelligent building research, and more generally with ambient intelligence, it is necessary to control and read states of intelligent objects in the given environment in order to prototype and evaluate proposed services. Even if these objects are similar in terms of usage and functionality, the way to control them is heterogeneous, and so is their behavioral model. In this context, middleware is an essential component. They can collect data from multiple and different sources, acting as a gateway towards higher and external services. They also, in most cases, allow reacting back on the local environment. We propose in this paper a model for such interoperability within a middleware. Our work is based onto a object centered approach, with field and usage knowledge. Our work has the particularity to have been developed and validated in an experimental platform.
... Bjelica et al. [79] and Garzotto and Valoriani [80] considered that single devices, like controllers and small displays, are the best option for controlling smart home environments given the requirements of mobility within the house and the easy access to a general controller of the system. Additionally, it is required that such a device needs to be capable of interacting with several communication protocols (in cases where no device manager is available) for absolute command of the system. ...
Article
Full-text available
Current lifestyles and technologies have inspired new ways to manage home appliances free of restrictions previously imposed by distance and time. So-called home automation technologies allow users to control and schedule common home tasks, often without human intervention. Several types of home automation systems have been proposed and classified according to their functionality, mode of user interaction, scheduling systems and cost-effectiveness. This paper reviews proposals in the smart home field from the last five years, considering both technical and functional aspects. The paper seeks to define automated or smart home systems by identifying common aspects of current such systems. © 2016 International Journal of Automation and Smart Technology.
... Digital television and set-top boxes can be extended to provide control over home appliances in a smart home. References [8]- [9] provide examples of such systems. Remote control which was used to control television set or set-top box can now be used to control all home appliances. ...
Conference Paper
Full-text available
Cloud computing is making its way towards use at home. This paper presents a cloud for smart home. We present review of technologies for smart home and give an integrated cloud-based energy- and carbon-efficient architecture.
Article
A set-top box (STB) increases its energy efficiency with active standby and passive standby modes. Compared to the active standby mode, the passive standby mode saves more power by turning off more STB functions, but it results in longer activation latency. For an energyefficient STB, in this paper, power saving with the passive standby is studied. The STB operates the passive standby mode when it is not in use, and the passive standby mode is switched to the active standby mode before a user turns on the STB in order to reduce the activation latency. To do that, the passive standby uses activity logs, which are represented in a bitmap form, to find a user pattern and to predict the next user activity. In the presented power saving method with the passive standby, recent activities is reflected with more weights to quickly find a new user pattern, instead of waiting until activity logs for a new user pattern are accumulated. The method is evaluated via simulations and discussed1.
Article
Smartphones have a great potential as tools to adapt daily living spaces to the specific needs of its inhabitants. These devices usually host selected content, personal information and useful applications for their users, apart from continuously gathering their context data and preferences. Taking this fact as starting point, this paper describes PERSEO, a system that implements a mobile-instrumented interaction concept to personalize the smart space by using a network of presence sensors and media objects. PERSEO is device-centric and relies on physical interaction. This means that on touch, grasp or proximity with any of the trigger objects in the presence network (e.g. NFC or Bluetooth enabled devices), the smartphone seamlessly transfers contents or application control to the available (media) objects, in order to improve the user experience. The transfer actions are governed by an event-condition-action logic that may be customized by the user in the smartphone. Actions depend on the identification of the foreground applications running in the smartphone hosting PERSEO's logic. To enable the interaction concept, it has been necessary to design and implement a custom Digital Media Controller based on REST interfaces compatible with DLNA standard for media streaming. The paper explains the interaction flows and details the architecture components through an Android-based mobile prototype working in a smart living room. The system performance has been positively evaluated in terms of response time. Additionally, a group of users have tried PERSEO and favorably informed about this mobile-object instrumented personalization approach.
Article
Most modern smart TV set-top boxes support internet connection and allow the user to install and run more advanced application programs or plugins/add-ons based on a specific platform. It means that a smart TV set-top box can be a good candidate to act as a hub that combines a wide variety of solutions for smart home services. This paper proposes a home appliance control framework based on a smart TV set-top box. As a core component, a home gateway software module is installed on the smart TV set-top box. And the architecture of the home gateway module is designed so that a home appliance control driver is installed only when necessary. To enable the user to control their home appliances in any place using a Web-based application or a Web browser, the proposed framework supports a WebSocket protocol-based data communication and comprises a Web server where each set-top box???s network location is registered. The proposed framework is verified with a demonstration on a small-scale home model. Both IP-based and serial communication-based home appliances are included in the demonstration system. The demonstration examples show that consumers can control various types of home appliances in any place through a smart TV set-top box solution.
Conference Paper
The legacy TV is now being replaced by IP-connected Smart TV which offers more advanced computing ability and connectivity. Smart TV also allows the user to install and run more advanced applications based on a specific platform. Smart TV is expected to play the role of a hub which combines wide variety of solutions for digital living services in home in the near future. In this paper, we present a home appliance control framework based on Smart TV set-top box. The purpose of this framework is to keep users leaning back and watching TV screen and then to provide them with an intuitive and convenient way to monitor and control home appliances. The proposed framework supports not only IP-based but also conventional RS-485-based home appliances.
Article
This paper present a conceptual solution, implementation details, results of performance analysis and tests of system for controlling lighting effects from TV receiver over wireless network. The system represents an extension of a home automation system. It is based on embedded computer system that is integrated into the TV receiver, and enables the use of a single central device for monitoring and controlling household devices such as RGB lamps. Lamps are used to provide lighting effects and regulation of user's ambient. For controlling this and other devices user can use a standard remote controller.
Conference Paper
Full-text available
This paper proposes a concept of a home controller for home automation that is embeddable to existing consumer electronic devices in the household, such as TV sets and set-top boxes. The concept is investigated through implementation in a digital set-top box extended with units for both power line (X10) and wireless (Zigbee) communication to control lights and appliances. Main goal was to propose a generic software model that would be able to support the majority of field-proven protocols for automation and lighting. Additional important aspects, such as home behavior modeling, comprehensive graphical user interface (GUI) and economic use of resources were covered.
Conference Paper
Full-text available
In a future networked physical world, a myriad of smart sensors and actuators assess and control aspects of their en- vironments and autonomously act in response to it. Exam- ples range in telematics, tra-c management, team robotics or home automation to name a few. To a large extent, such systems operate proactively and independently of direct hu- man control driven by the perception of the environment and the ability to organize respective computations dynamically. The challenging characteristics of these applications include sentience and autonomy of components, issues of responsive- ness and safety criticality, geographical dispersion, mobility and evolution. A crucial design decision is the choice of the appropriate abstractions and interaction mechanisms. Looking to the basic building blocks of such systems we may flnd components which comprise mechanical compo- nents, hardware and software and a network interface, thus these components have difierent characteristics compared to pure software components. They are able to spontaneously disseminate information in response to events observed in the physical environment or to events received from other component via the network interface. Larger autonomous components may be composed recursively from these build- ing blocks. The paper describes an architectural framework and a middleware supporting a component-based system and an integrated view on events-based communication comprising the real world events and the events generated in the sys- tem. It starts by an outline of the component-based system construction. The generic event architecture GEAR is intro- duced which describes the event-based interaction between the components via a generic event layer. The generic event layer hides the difierent communication channels including ⁄ This work was partially supported by the EC, through project IST-2000-26031 (CORTEX), and by the FCT, through the Large-Scale Informatic Systems Labora- tory (LaSIGE) and project POSI/1999/CHS/33996 (DE- FEATS).
Conference Paper
Full-text available
This paper presents a domotic house gateway capable of seamlessly interacting with different devices from heterogeneous domotic systems and appliances. Such a gateway also provides the possibility to automate device cooperation through an embedded rule-based engine, which can be dynamically and automatically updated to accommodate necessities and anticipate users' actions. Some practical applications will show the effectiveness of the system.
Article
Full-text available
Today most of the home and office appliances that we interact with contain microprocessors. All of these appliances have some user interface, but many users become frustrated with the difficulty of using the complex functions of their appliances. We are developing a framework that allows users to interact with appliances through a separate user interface device that they are already carrying. Smart phones are good candidates for providing interfaces because they are common, have communication capabilities to allow connection to appliances, and are already being used for a wide range of different applications. Our framework includes an abstract specification language for describing appliances, a two-way communication protocol, and automatic interface generation software that allows user interfaces to be customized to users and the devices they are using. This article overviews our personal universal controller system and describes in detail our design and implementation of automatic interface generation for Microsoft's Smartphone platform.
Conference Paper
Full-text available
This paper presents a solution of a UPnP/DLNA control library for the use within a distributed multimedia system. Apart from standard playback control functions, library supports execution of scripts that define time transitions and response to external events, faster response to play command and better playback synchronization across multiple renderers.
Conference Paper
Full-text available
Making systems user-aware can be a daunting task. Problems that should be attended range from sensing a complex environment and recognizing user activity, to prediction and personalization of system services. This paper proposes a User Awareness Kit (UAK) that can be used as a slave system with a variety of consumer electronic devices. UAK combines sensor inputs (passive infrared sensor, microphone array, time of flight camera) with behavior models based on system attentiveness and user-system interference concepts, to provide information of the ecosystem of users to its host. UAK is designed as a system on chip (SoC) that shares that information by using a simple interprocessor protocol. As a result, host device would be able to anticipate user actions and adapt accordingly. We argue that equipping consumer electronic devices with UAK can mitigate risks of device rejection by users. Due to the all-growing number of electronic products that people use, carry around and even depend on reducing the impact on their attention and time should be an ultimate goal.
Conference Paper
Full-text available
In this paper we present a software model for the embedded systems, that separates functionality (core) from presentation (user interface). The model provides means to easily swap user interfaces, as well as their remote execution. We propose standards, protocols and guidelines for creating portable and scalable embedded software.
Article
Universal Plug and Play (UPnP) is a popular technology for home networking and control. However, to construct UPnP-based home networks is not easy. Especially, most existing appliances have no UPnP interface, how to connect them to the network is an important issue. A solution of this problem is proposed, where, UPnP control point is embedded in residential gateway, while UPnP device is embedded in so-called UPnP Adapter. The UPnP Adapter has an interface translating the UPnP control messages into the private protocol of the appliance. Besides, user can operate appliances via any control terminal connected to the network. The solution has been implemented with Intel SDK for UPnP Devices. The technical details of the implementation are given in this paper.
Article
Smart homes are sensor-rich environments that contain dynamic sets of interacting components. These components often use competing and closed standards and form a message-based architecture. This complicates the development of applications that require information from disparate sources. It becomes difficult to add new components or to allow components from different applications to interact with each another. In this paper we describe Construct, a pervasive computing middleware that is ideally suited for deployment in the smart home. Construct acts as a sensor fusion layer that takes output from each smart home component and makes it available to all applications. This makes it easy to develop applications that require access to heterogeneous sources of sensor data, and to add sensors to existing systems to improve their performance. This paper demonstrates two Construct-enabled smart home applications and shows how access to new sensors leads to improvements in their performance.
Article
Smart home environments have evolved to the point where everyday objects and devices at home can be networked to give the inhabitants new means to control them. Familiar information appliances can be used as user interfaces (UIs) to home functions to achieve a more convenient user experience. This paper reports an ethnographic study of smart home usability and living experience. The purpose of the research was to evaluate three UIs—a PC, a media terminal, and a mobile phone—for smart home environments. The results show two main types of activity patterns, pattern control and instant control, which require different UI solutions. The results suggest that a PC can act as a central unit to control functions for activity patterns that can be planned and determined in advance. The mobile phone, on the other hand, is well suited for instant control. The mobile phone turned out to be the primary and most frequently used UI during the 6-month trial period in the smart apartment.