Conference Paper

Deriving Privacy and Security Considerations for CORE: An Indoor IoT Adaptive Context Environment

To read the full-text of this research, you can request a copy directly from the authors.


The internet-of-things (IoT) consists of embedded devices and their networks of communication as they form decentralized frameworks of ubiquitous computing services. Within such decentralized systems the potential for malicious actors to impact the system is significant, with far-reaching consequences. Hence this work addresses the challenge of providing IoT systems engineers with a framework to elicit privacy and security design considerations, specifically for indoor adaptive smart environments. It introduces a new ambient intelligence indoor adaptive environment framework (CORE) which leverages multiple forms of data, and aims to elicit the privacy and security needs of this representative system. This contributes both a new adaptive IoT framework, but also an approach to systematically derive privacy and security design requirements via a combined and modified OCTAVE-Allegro and Privacy-by-Design methodology. This process also informs the future developments and evaluations of the CORE system, toward engineering more secure and private IoT systems.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... This leads to three overall contributions; i) a new IoT-Avatar architectural framework; ii) a bi-directional communication approach between an IoT system and a virtual avatar character representation; and iii) an early descriptive exploration of how such systems could be explored in future applications. This extends the authors' work on context-driven IoT design considerations (including privacy and security) [10] and highlights immersive interfaces for the future IoT. This section has covered the primary motivation for merging IoT interfaces with immersive and information rich approaches. ...
... Section 2 provides a background on IoT and HCI needs, dashboards, and mixed reality for IoT. Section 3 presents an IoT Avatar architecture based on [10] and presents a design for a Plant-Avatar proof-of-concept. Section 4 provides a discussion of related work and future investigations. ...
... These displays typically present information like the speedometer, GPS, and in some cases even directions. More recently, for vehicles, consumers can now purchase devices to convert a phone display into a temporary HUD, with products such as Hudwayglass 10 . Such applications of HUDs are meant to enhance and inform the driver of the vehicle situation, and may also be relevant to IoT-related information presentation. ...
Full-text available
The Internet of Things (IoT) continues its growth, adoption, toward ubiquitous usage but is not without the inevitable communication bandwidth challenge. Human-computer-interaction in this space must account for the multiple facets of human-in-the-loop considerations in IoT, yet current mechanisms are at present limited by display dimensions and unclear indicators. Mixed Reality (MR) may be a solution to this human communication bandwidth problem, as smart glasses and other head mounted displays could provide an ideal interface platform for IoT human-computer interaction, while handheld mobile MR can be used as a testbed. To bring MR interfaces to the IoT, this work contributes; i) a new IoT-Avatar architectural framework; ii) a bi-directional communication approach between an IoT system and a virtual avatar character representation; and iii) an early descriptive exploration of how such systems could be explored in future applications toward MR interfaces in ambient intelligent environments.
... This statement summarizes a maker's approach, where prototyping first is preferred to planning. Moreover, most maker-related discussions and tools do not place much emphasis on security [4], [5]. While various parties made efforts to promote secure IoT systems,such as bestpractice security guidelines outlined by public cloud providers, semiconductor companies, the burden still falls on the maker to properly design and secure their creations. ...
Full-text available
The Internet of Things (IoT) is an emerging paradigm focusing on the connection of devices, objects, or “things” to each other, to the Internet, and to users. IoT technology is anticipated to become an essential requirement in the development of smart homes, as it offers convenience and efficiency to home residents so that they can achieve better quality of life. Application of the IoT model to smart homes, by connecting objects to the Internet, poses new security and privacy challenges in terms of the confidentiality, authenticity, and integrity of the data sensed, collected, and exchanged by the IoT objects. These challenges make smart homes extremely vulnerable to different types of security attacks, resulting in IoT-based smart homes being insecure. Therefore, it is necessary to identify the possible security risks to develop a complete picture of the security status of smart homes. This article applies the operationally critical threat, asset, and vulnerability evaluation (OCTAVE) methodology, known as OCTAVE Allegro, to assess the security risks of smart homes. The OCTAVE Allegro method focuses on information assets and considers different information containers such as databases, physical papers, and humans. The key goals of this study are to highlight the various security vulnerabilities of IoT-based smart homes, to present the risks on home inhabitants, and to propose approaches to mitigating the identified risks. The research findings can be used as a foundation for improving the security requirements of IoT-based smart homes.
Full-text available
The high penetration rate of new technologies in all the activities of everyday life is fostering the belief that for any new societal challenge there is always an ICT solution able to successfully deal with it. Recently, the solution that is proposed almost anytime is the “Internet of Things ”(IoT). This apparent panacea of the ICT world takes different aspects on and, actually, is identified with different (often very different) technological solutions. As a result, many think that IoT is just RFIDs, others think that it is sensor networks, and yet others that it is machine-to-machine communications. In the meanwhile, industrial players are taking advantage of the popularity of IoT to use it as a very trendy brand for technology solutions oriented to the consumer market. The scientific literature sometimes does not help much in clarifying, as it is rich in definitions of IoT often discordant between them. Objective of this paper is to present the evolutionary stages, i.e., generations , that have characterized the development of IoT, along with the motivations of their triggering. Besides, it analyzes the role that IoT can play in addressing the main societal challenges and the set of features expected from the relevant solutions. The final objective is to give a modern definition of the phenomenon, which de facto shows a strong pervasive nature, and, if not well understood in its theories, technologies, methodologies, and real potentials, then runs the risk of being regarded with suspicion and, thus, rejected by users.
Conference Paper
Full-text available
DIY, hacking, and craft have recently drawn attention in HCI and CSCW, largely as a collaborative and creative hobbyist practice. We shift the focus from the recreational elements of this practice to the ways in which it democratizes design and manufacturing. This democratized technological practice, we argue, unifies playfulness, utility, and expressiveness, relying on some industrial infrastructures while creating demand for new types of tools and literacies. Thriving on top of collaborative digital systems, the Maker movement both implicates and impacts professional designers. As users move more towards personalization and reappropriation, new design opportunities are created for HCI.
Conference Paper
Full-text available
This paper presents a manifesto directed at developers and designers of internet-of-things creation platforms. Currently, most existing creation platforms are tailored to specific types of end-users, mostly people with a substantial background in or affinity with technology. The thirteen items presented in the manifesto however, resulted from several user studies including non-technical users, and highlight aspects that should be taken into account in order to open up internet-of-things creation to a wider audience. To reach out and involve more people in internet-of-things creation, a relation is made to the social phenomenon of doit- yourself, which provides valuable insights into how society can be encouraged to get involved in creation activities. Most importantly, the manifesto aims at providing a framework for do-it-yourself systems enabling non-technical users to create internet-of-things applications.
Conference Paper
Full-text available
In this paper we define the notion of a privacy design strategy. These strategies help IT architects to support privacy by design early in the software development life cycle, during concept development and analysis. Using current data protection legislation as point of departure we derive the following eight privacy design strategies: minimise, hide, separate, aggregate, inform, control, enforce, and demonstrate. The strategies also provide a useful classification of privacy design patterns and the underlying privacy enhancing technologies. We therefore believe that these privacy design strategies are not only useful when designing privacy friendly systems, but also helpful when evaluating the privacy impact of existing IT systems.
We propose a method for multi-person detection and 2-D keypoint localization (human pose estimation) that achieves state-of-the-art results on the challenging COCO keypoints task. It is a simple, yet powerful, top-down approach consisting of two stages. In the first stage, we predict the location and scale of boxes which are likely to contain people; for this we use the Faster RCNN detector with an Inception-ResNet architecture. In the second stage, we estimate the keypoints of the person potentially contained in each proposed bounding box. For each keypoint type we predict dense heatmaps and offsets using a fully convolutional ResNet. To combine these outputs we introduce a novel aggregation procedure to obtain highly localized keypoint predictions. We also use a novel form of keypoint-based Non-Maximum-Suppression (NMS), instead of the cruder box-level NMS, and a novel form of keypoint-based confidence score estimation, instead of box-level scoring. Our final system achieves average precision of 0.636 on the COCO test-dev set and the 0.628 test-standard sets, outperforming the CMU-Pose winner of the 2016 COCO keypoints challenge. Further, by using additional labeled data we obtain an even higher average precision of 0.668 on the test-dev set and 0.658 on the test-standard set, thus achieving a roughly 10% improvement over the previous best performing method on the same challenge.
Despite significant recent advances in the field of face recognition, implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure of face similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings as feature vectors. Our method uses a deep convolutional network trained to directly optimize the embedding itself, rather than an intermediate bottleneck layer as in previous deep learning approaches. To train, we use triplets of roughly aligned matching / non-matching face patches generated using a novel online triplet mining method. The benefit of our approach is much greater representational efficiency: we achieve state-of-the-art face recognition performance using only 128-bytes per face. On the widely used Labeled Faces in the Wild (LFW) dataset, our system achieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves 95.12%. Our system cuts the error rate in comparison to the best published result by 30% on both datasets.
We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 dif-ferent classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make train-ing faster, we used non-saturating neurons and a very efficient GPU implemen-tation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called "dropout" that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.
Conference Paper
We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in understanding an object's precise 2D location. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old along with per-instance segmentation masks. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.
Ubiquitous sensing enabled by Wireless Sensor Network (WSN) technologies cuts across many areas of modern day living. This offers the ability to measure, infer and understand environmental indicators, from delicate ecologies and natural resources to urban environments. The proliferation of these devices in a communicating-actuating network creates the Internet of Things (IoT), wherein, sensors and actuators blend seamlessly with the environment around us, and the information is shared across platforms in order to develop a common operating picture (COP). Fuelled by the recent adaptation of a variety of enabling device technologies such as RFID tags and readers, near field communication (NFC) devices and embedded sensor and actuator nodes, the IoT has stepped out of its infancy and is the the next revolutionary technology in transforming the Internet into a fully integrated Future Internet. As we move from www (static pages web) to web2 (social networking web) to web3 (ubiquitous computing web), the need for data-on-demand using sophisticated intuitive queries increases significantly. This paper presents a cloud centric vision for worldwide implementation of Internet of Things. The key enabling technologies and application domains that are likely to drive IoT research in the near future are discussed. A cloud implementation using Aneka, which is based on interaction of private and public clouds is presented. We conclude our IoT vision by expanding on the need for convergence of WSN, the Internet and distributed computing directed at technological research community.
In view of rapid and dramatic technological change, it is important to take the special requirements of privacy protection into account early on, because new technological systems often contain hidden dangers which are very difficult to overcome after the basic design has been worked out. So it makes all the more sense to identify and examine possible data protection problems when designing new technology and to incorporate privacy protection into the overall design, instead of having to come up with laborious and time-consuming “patches” later on. This approach is known as “Privacy by Design” (PbD). Keywords Privacy by Design -Electronic health card-Electronic ID card-ELENA
Nowadays, numerous journals and conferences have published articles related to context-aware systems, indicating many researchers’ interest. Therefore, the goal of this paper is to review the works that were published in journals, suggest a new classification framework of context-aware systems, and explore each feature of classification framework. This paper is based on a literature review of context-aware systems from 2000 to 2007 using a keyword index and article title search. The classification framework is developed based on the architecture of context-aware systems, which consists of the following five layers: concept and research layer, network layer, middleware layer, application layer and user infrastructure layer. The articles are categorized based on the classification framework. This paper allows researchers to extract several lessons learned that are important for the implementation of context-aware systems.
Socio-technical systems research aims to optimize two of the most important parts of the organization, the social network, and the technological network. The field is highly multi-disciplinary, and covers a host of issues, ranging from the management of complex systems, teams and work groups, interactions, and cognitive factors. Many approaches in the literature have adopted aspects of the socio-technical ideas, leading to four general perspectives. Depending on the view of the organization, solutions proposed are either those of the abstract organizational scientist, the social scientist, the technologist/engineer, or the complex systems engineer. This work surveys the field with these perspectives in mind, and highlights literature exploring such systems.
The relation between ambient intelligence and multimedia systems is discussed. Distributed multimedia applications and their processing on embedded static and mobile platforms will play a major role in the development of ambient intelligent environments. Ambient intelligence aims to improve integration, provided by ubiquitous computing. The new paradigm will improve people's quality of life by creating the desired atmosphere and functionality through intelligent, personalized, interconnected systems and services. In an ambient-intelligent environment, electronic system consisting of networked-intelligent devices will provide people with information, communication, services, and entertainment.
Conference Paper
This paper tries to serve as an introductory reading to privacy issues in the field of ubiquitous computing. It develops six principles for guiding system design, based on a set of fair information practices common in most privacy legislation in use today: notice, choice and consent, proximity and locality, anonymity and pseudonymity, security, and access and recourse. A brief look at the history of privacy protection, its legal status, and its expected utility is provided as a background.
Alexa Skills Kit - Build for Voice with Amazon
  • Amazon
  • Com 2018. Alexa Skills Kit -Build for Voice with Amazon. Retrieved July 17, 2018 from
Echo and Alexa - Amazon Devices
  • Amazon
  • Com 2018. Echo and Alexa -Amazon Devices. Retrieved July 17, 2018 from node=9818047011
Flask-a Python Microframework
  • Pocoo Flask
  • Org 2018. Flask-a Python Microframework. Retrieved July 17, 2018 from
Google Home - Smart Speaker and Home Assistant
  • Google
  • Com 2018. Google Home -Smart Speaker and Home Assistant. Retrieved July 17, 2018 from
Ngrok-Secure Introspectable Tunnels to Localhost
  • Ngrok
  • Com 2018. Ngrok-Secure Introspectable Tunnels to Localhost. Retrieved July 17, 2018 from
RaspberryPi - Teach, Learn, and Make with Raspberry Pi
  • Raspberrypi
  • Org 2018. RaspberryPi -Teach, Learn, and Make with Raspberry Pi. Retrieved July 17, 2018 from
Google Assistant - Your own personal Google
  • Google
  • Com 2018. Google Assistant -Your own personal Google. Retrieved July 17, 2018 from
Building the Web of Things: With examples in Node. js and Raspberry Pi
  • Dominique Guinard
  • Vlad Trifa
Dominique Guinard and Vlad Trifa. 2016. Building the Web of Things: With examples in Node. js and Raspberry Pi. (2016).
Pose Detection in the Browser: PoseNet Model
  • Github
  • Com 2018. Pose Detection in the Browser: PoseNet Model. Retrieved July 17, 2018 from
Google Cloud Platform
  • Google
  • Com 2018. Google Cloud Platform. Retrieved July 17, 2018 from
Google's Newest Feature: Find My Home
  • Tripwire
  • Com 2018. Google's Newest Feature: Find My Home. Retrieved July 17, 2018 from googles-newest-feature-find-my-home/
TensorFlow - An open source machine learning framework for everyone
  • Tensorflow
  • Org 2018. TensorFlow -An open source machine learning framework for everyone. Retrieved July 17, 2018 from
  • Apache
  • Org 2018. The Apache HTTP Server Project. Retrieved July 17, 2018 from
Katriina Kilpi, and An Jacobs. 2012. I would DiYSE for it!: a manifesto for do-it-yourself internet-of-things creation
  • Karin Dries De Roeck
  • Johan Slegers
  • Marc Criel
  • Laurence Godon
  • Claeys
Dries De Roeck, Karin Slegers, Johan Criel, Marc Godon, Laurence Claeys, Katriina Kilpi, and An Jacobs. 2012. I would DiYSE for it!: a manifesto for do-it-yourself internet-of-things creation. In Proceedings of the 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design. ACM, 170-179.
  • Redis
  • Io 2018. Redis. Retrieved July 17, 2018 from
  • Mqtt
  • Org 2018. MQTT. Retrieved July 17, 2018 from
  • Charith Perera
  • Mahmoud Barhamgi
  • K Arosha
  • Muhammad Bandara
  • Blaine Ajmal
  • Bashar Price
  • Nuseibeh
Charith Perera, Mahmoud Barhamgi, Arosha K Bandara, Muhammad Ajmal, Blaine Price, and Bashar Nuseibeh. 2017. Designing Privacy-aware Internet of Things Applications. arXiv preprint arXiv:1703.03892 (2017).
  • Alexander Wong
  • Mohammad Javad Shafiee
  • Francis Li
  • Brendan Chwyl
Alexander Wong, Mohammad Javad Shafiee, Francis Li, and Brendan Chwyl. 2018. Tiny SSD: A Tiny Single-shot Detection Deep Convolutional Neural Network for Real-time Embedded Object Detection. arXiv preprint arXiv:1802.06488 (2018).
Joseph Redmon and Ali Farhadi
  • Joseph Redmon
  • Ali Farhadi
Alex Krizhevsky Ilya Sutskever and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks
  • Alex Krizhevsky
  • Ilya Sutskever
  • Geoffrey E Hinton
Luigi Atzori Antonio Iera and Giacomo Morabito. 2010. The internet of things: A survey
  • Antonio Iera
  • Giacomo Morabito