Simon Harper

The University of Manchester · Department of Computer Science

Honey can be used to biomonitor contaminants, as honeybees can bioaccumulate both organic and inorganic (e.g., metal(loid)) contaminants from the total environment. Honey chemical composition can be influenced by natural and anthropogenic activities. Metal(loid) concentrations in honey samples collected by citizen scientist beekeepers in northwest...

This paper presents a prototypical accessibility tool and associated framework. Our tool supports a framework of six metrics: coverage, representativeness, complexity, popularity, freshness and accessibility, designed to optimise the web accessibility evaluation process through representative pages. We divided our tool into three stages: an initial...

Web accessibility evaluation is a costly and complex process due to limited time, resources and ambiguity. To optimise the accessibility evaluation process, we aim to reduce the number of pages auditors must review by employing statistically representative pages, reducing a site of thousands of pages to a manageable review of archetypal pages. Our...

Finding desired information can still be a complex task, which is particularly challenging on specialist search engines. We propose a methodology to model search behavior evolution to better understand the familiarization process. As a case study, we analyzed features derived from search queries as well as user interface interactions of 239 users f...

This systematic literature review presents the computational methods of human behaviour prediction research from Pentland and Liu’s seminal 1999 paper on human behaviour prediction to the latest research to date. The PRISMA framework for systematic reviews was used as the review methodology to structure this information aggregation. This review pro...

Honey can be used to biomonitor contaminants, as honeybees can bioaccumulate both organic and inorganic (e.g., metal(loid)) contaminants from the total environment. Honey chemical composition can be influenced by natural and anthropogenic activities. Metal(loid) concentrations in honey samples collected by citizen scientist beekeepers in northwest...

Complex human activity prediction is a difficult problem for computer science. Simple behaviours can be mapped to sequence prediction algorithms with good results; however, real-world examples of activity are generally stochastic and much more computationally difficult to infer. One method for solving this problem is to utilise contextual data - cl...

A significant proportion of smartphone notifications are indicative of human behaviour (e.g. delivery updates for purchased items, physical activity summaries, and notification of updates to subscribed content). However, present attempts to understand human behaviour from smartphone traces typically focus on sensors such as location, accelerometer...

Smartphones are widely used hubs of personal communication. With their many sensors, they are capable of monitoring social behaviours. Calls, messages, application usage and even face-to-face conversations can be captured by smartphones. These data are then used to study psychological, behavioural and clinical issues, thus providing an objective an...

Deriving human behaviour from smartphone location data is a multitask enrichment process that can be of value in behavioural studies. Optimising the algorithmic details of the enrichment tasks has shaped the current advances in the literature. However, the lack of a processing framework built around those advances complicates the planning for imple...

Human behaviours that are motivated by and indicative of personal interests can be utilised to personalise behavioural recommendations used to promote health and well-being. Behavioural and psychological studies show that (1) personal interests are demonstrated differently in individuals’ daily activities; and (2) drawbacks of self-reporting method...

Ambient Intelligence (AmI) is the application and embedding of artificial intelligence into everyday environments to seamlessly provide assistive and predictive support in a multitude of scenarios via an invisible user interface. These can be as diverse as autonomous vehicles, smart homes, industrial settings, and healthcare facilities—referred to...

We explore whether interactive navigational behaviours can be used as a reliable and effective source to measure the progress, achievement, and engagement of a learning process. To do this, we propose a data-driven methodology involving sequential pattern mining and thematic analysis of the low-level navigational interactions. We applied the method...

Eye-tracking research enables us to understand how users interact with visual stimuli. It has widely been used to understand user interaction with web pages for various purposes, such as to assess the usability and accessibility of web pages. However, individual eye movement sequences (i.e. scanpaths) tend to be complicated and differ from each oth...

Background: Hand tremor typically has a negative impact on a person’s ability to complete many common daily activities. Previous research has investigated how to quantify hand tremor with smartphones and wearable sensors, mainly under controlled data collection conditions. Solutions for daily real-life settings remain largely underexplored. Objec...

Understanding visual complexity as it relates to websites has been an emergent area for many years. However, predicting the visual complexity of a website as perceived by users has been a real challenge. Perception is important because it influences user engagement, dictating if they will find it dull, engaging, or too complex. While others have su...

Diagnosing COVID-19 early in domestic settings is possible through smart home devices that can classify audio input of coughs, and determine whether they are COVID-19. Research is currently sparse in this area and data is difficult to obtain. However, a few small data collection projects have enabled audio classification research into the applicati...

Background Governments promote behavioral policies such as social distancing and phased reopening to control the spread of COVID-19. Digital phenotyping helps promote the compliance with these policies through the personalized behavioral knowledge it produces. Objective This study investigated the value of smartphone-derived digital phenotypes in (...

BACKGROUND Governments promote behavioral policies such as social distancing and phased reopening to control the spread of COVID-19. Digital phenotyping helps promote the compliance with these policies through the personalized behavioral knowledge it produces. OBJECTIVE This study investigated the value of smartphone-derived digital phenotypes in...

Diagnosing COVID-19 early in domestic settings is possible through smart home devices that can classify audio input of coughs, and determine whether they are covid or not_covid. Research is currently sparse in this area and data is difficult to obtain. However, a few small data collection projects have enabled audio classification research into the ap...

Audio classification is an important area of research for smart environment technology. Enabling systems to understand real world audio data that is noisy or low quality is necessary for developing practical applied audio classifiers. We demonstrate a method for improving audio classification accuracy of MFCCs, a type of spectrogram image of the au...

This systematic review presents the computational methods of human behaviour prediction research, beginning with Pentland and Liu's 1999 paper on human behaviour prediction, and covering the subsequent 20 years of research in this area. This review answers a set of research questions that define gaps in the literature as opportunities for further r...

BACKGROUND Hand tremor typically has a negative impact on a person’s ability to complete many common daily activities. Previous research has investigated how to quantify hand tremor with smartphones and wearable sensors, mainly under controlled data collection conditions. Solutions for daily real-life settings remain largely underexplored. OBJECTI...

BACKGROUND Smartphones are widely used and have become hubs of personal communication. Combined with multiple sensors, they are capable of monitoring social interaction passively. OBJECTIVE To review the published empirical English literature of social sensing with smartphones passively. METHODS Following the PRISMA guidelines, we constructed a s...

Background: People with lung cancer often wait for several months before presenting symptoms to health services. Some patients report seeking information online to help them appraise symptoms. No research has evaluated whether websites about lung cancer present information in an optimal manner to encourage help-seeking. Objective: To evaluate the...

Autism diagnostic procedure is a subjective, challenging and expensive procedure and relies on behavioral, historical and parental report information. In our previous, we proposed a machine learning classifier to be used as a potential screening tool or used in conjunction with other diagnostic methods, thus aiding established diagnostic methods. T...

Health conditions, both chronic and acute, are often accompanied by disability-like impairments that might affect mobility, cognition, or perception. These impairments are often pernicious because they are difficult to isolate, vary in intensity and extent over time, and are under-investigated. Here, we make the case that solutions to these impairm...

Web pages are composed of different kinds of elements (menus, adverts, etc.). Segmenting pages into their elements has long been important in understanding how people experience those pages and in making those experiences “better.” Many approaches have been proposed that relate the resultant elements with the underlying source code; however, they d...

Parkinson's disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. Measuring the symptoms and progress of the disease, and medication effectiveness is currently performed using subjective measures and visual estimation. We developed and evaluated a mobile application, STOP fo...

Background/Aims This article aims to improve the understanding of the applied cognitive processes when interpreting electrocardiograms in clinical practice. It will do this by examining the self-reported approach practitioners take to interpret any barriers they encounter. Methods This was a qualitative study in which medical practitioners, who rou...

Arousal detection has been used as a proxy to sense frustration, cognitive load, anxiety and stress, which are relevant to user experience. The mechanisms provide limited potential for widespread use beyond the lab. We used eye-tracking to capture pupillary response and gaze behaviour during user interaction. Pupillary response is used to sense cha...

We examine the impact of the presentation of a patient’s clinical history on subsequent visual appraisal and interpretation accuracy of electrocardiograms (ECGs). Healthcare-practitioners (N = 31) skilled in 12-lead ECG interpretation took part in a repeated-measures experiment with counterbalancing viewing 9 ECGs on a computer screen in two separa...

People with autism often exhibit different visual behaviours from neurotypical users. To explore how these differences are exhibited on the Web, we model visual behaviour by combining pupillary response, which is an unobtrusive measure of physiological arousal, with eye-tracking scan paths that indicate visual attention. We evaluated our approach w...

Accessibility has, at its heart, the aim of removing developer and content creator preconceptions as to the technology, people, working environment, and use their systems support. The move to mobile has been nothing but positive for the accessibility of people with a visual disability. Similarly in the next 10 years as the Web becomes a utility, im...

Anecdotal evidence suggests there is a disjoint between the interaction experiences of sighted and visually disabled web users. However, we propose the converse and suggest that this disjoint is created by the lack of understanding of the interplay between the two domains. Current research shows that there is one single locus of attention at a give...

Welcome to my insecurities, and the lonely questions I ask myself at 03:00 in the morning. I'll present five 'Notes to My Past Self' which outline the things I should have considered -- and am now considering -- which would have made my accessibility work better. I'll take a look at: Impact and Public Dissemination; Reproducibility; I'll ask 'What...

Objective: The study sought to explore to what extent geolocation data has been used to study serious mental illness (SMI). SMIs such as bipolar disorder and schizophrenia are characterized by fluctuating symptoms and sudden relapse. Currently, monitoring of people with an SMI is largely done through face-to-face visits. Smartphone-based geolocati...

The use of mobile devices during television viewing is now commonplace, and broadcasters are increasingly supplying programme-related ‘companion content’. To produce an optimal user experience, it is important to determine how the delivery of companion content affects and is perceived by the viewer — without this we risk distracting the viewer, lea...

Anecdotal evidence suggests that people with autism may have different processing strategies when accessing the web. However, limited empirical evidence is available to support this. This paper presents an eye tracking study with 18 participants with high-functioning autism and 18 neurotypical participants to investigate the similarities and differ...

Parkinson's disease (PD) is the second most common neurodegenerative disorder, impacting an estimated seven to ten million people worldwide. It is commonly accepted that improving medication adherence alleviates symptoms and maintains motor capabilities. Not following the medication regimen (e.g., skipping or over-medicating) may worsen side-effect...

The use of mobile phones has transcended its initial use as a communication device to become a medium to fulfilling social needs. While this has been beneficial to some, studies have exposed adverse effects such as depression and social media addiction. Since it is not always clear which category the user belongs, we propose a mobile application th...

There are many mechanisms to sense arousal. Most of them are either intrusive, prone to bias, costly, require skills to set-up or do not provide additional context to the user's measure of arousal. We present arousal detection through the analysis of pupillary response from eye trackers. Using eye-trackers, the user's focal attention can be detecte...

Our world is increasingly interconnected via a wide variety of computers, IoT, wearable and mobile devices. The information provided collectively through these devices offers insightful information on our everyday lives, daily patterns, mood, behaviour, and surrounding environment. Our workshop brings together researchers interested in collecting a...

Poster presenting our latest results about the method we are developing for personalised monitoring of Parkinson's Disease using smartphones and human behaviour.

Objectives To detail the development method used to produce an online, tailored, theory‐based, user‐centred intervention to encourage help‐seeking for potential lung cancer symptoms. Design Intervention development was structured around the person‐based approach. The feasibility study involved a randomized controlled trial design. Methods Interve...

Eye tracking studies are increasingly used for understanding how people interact with web pages. However, these studies are typically costly and time-consuming. Some of the main reasons are the difficulty in finding participants, the cost of equipment and the need for a separate session for each participant. Due to these reasons, eye tracking studi...

Measuring emotions objectively in human-computer interaction is complicated because it involves selecting the appropriate detection mechanism, developing the computational technique to analyse it, and evaluating the correctness of the technique. Existing solutions are either manual, multimodal (requiring multiple sensors), have latency in their res...

Presentation delivered at ACM CHI 2018. Montreal, Canada. It is based on our paper with the same name (https://doi.org/10.1145/3173574.3173648). Consider Analogue over Digital if UX suggests it is best. We present a paper diary and 5 design implications for Parkinson's Disease self-reporting. We introduce PaperStream, a software tool to create and...

We report the process used to create artefacts for self-reporting Parkinson's Disease symptoms. Our premise was that a technology-based approach would provide participants with an effective, flexible, and resilient technique. After testing four prototypes using Bluetooth, NFC, and a microcontroller we accomplished almost full compliance and high ac...

In this paper, we explore how a number of novel methods for visualizing and analyzing differences in eye-tracking data, including scanpath length, Levenshtein distance, and visual transition frequency, can help to elucidate the methods clinicians use for interpreting 12-lead electrocardiograms (ECGs). Visualizing the differences between multiple pa...

Does reading a description of an artwork affect how a person subsequently views it? In a controlled study, we show that in most cases, textual description does not influence how people subsequently view paintings, contrary to participants’ self-report that they believed it did. To examine whether the description affected transition behaviour, we de...

User studies are typically difficult, recruiting enough users is often problematic and each experiment takes a considerable amount of time to be completed. In these studies, eye tracking is increasingly used which often increases time, therefore, the lower the number of users required for these studies the better for making these kinds of studies m...

Poster presented in ASSETS 2017. Parkinson’s Disease monitoring based on human behaviour inferred from smartphone collected data. Back to Analogue: testing a paper diary to collect self-reported Parkinson’s symptoms. Answers can be transcribed automatically to a computer using the open source program PaperStream. The diary had 99% compliance over 1...

Parkinson's Disease impairs the motor, cognitive and emotional functioning of people. Clinicians do not get an accurate image of the disease because patients visit them every six months and their symptoms can change within hours. Technology has been used to tackle this problem, but most approaches disrupt people's routines or are uncomfortable to u...

Web-based interactive systems can be verified and validated with eye tracking which provides valuable insights for understanding user interactions. However, individual eye movements (scanpaths) tend to be complicated and different from each other. Existing algorithms focus on clustering multiple scanpaths into a single representative path, but do n...

Background The Web has become an important information source for appraising symptoms. We need to understand the role it currently plays in help seeking and symptom evaluation to leverage its potential to support health care delivery. Objective The aim was to systematically review the literature currently available on Web use for symptom appraisal...

Background Delays to diagnosis in lung cancer can lead to reduced chance of survival, and patients often wait for several months before presenting symptoms. The time between first symptom recognition until diagnosis has been theorized into three intervals: symptom appraisal, help-seeking, and diagnostic interval (here: “pathway to diagnosis”). Inte...

Battery capacity (energy density) is increasing at around 3% per year. However, the increasing requirements of the mobile platform is placing higher demands on this capacity. In this case, there are three options: decrease our expectations of the mobile platform, increase the capacity and therefore size and weight of our batteries, or create energy...

We propose a set of algorithms to detect navigation problems in real-time. To do so, we operationalise some navigation strategies suggested by the literature and investigate the extent to which the exhibition of these strategies is an indicator of navigation problems. Our Firefox extension senses behaviour indicative of a user experiencing interact...

Elements related to cognitive disability are given lower priority in web accessibility guidelines due to limited understanding of the requirements of neurodiverse web users. Meanwhile, eye tracking has received a lot of interest in the accessibility community as a way to understand user behaviours. In this study, we combine results from information...

Interpretation of electrocardiograms (ECGs) is a complex task involving visual inspection. This paper aims to improve understanding of how practitioners perceive ECGs, and determine whether visual behaviour can indicate differences in interpretation accuracy. A group of healthcare practitioners (n = 31) who interpret ECGs as part of their clinical...

We present an experiment comparing performance of 20 novice evaluators of accessibility carrying out Web Content Accessibility Guidelines 2.0 conformance reviews working individually to performance obtained when they work in teams of two. They were asked to first carry out an individual assessment of a web page. Later on, they were matched randomly...

Eye tracking studies have widely been used in improving the design and usability of web pages and in the research of understanding how users navigate them. However, there is limited research in clustering users’ eye movement sequences (i.e., scanpaths) on web pages to identify a general direction they follow. Existing research tends to be reduction...

This paper describes the use of the Levenshtein distance and nearest neighbour index to visualise and analyse differences in eye-tracking scanpaths applied to the field of electrocardiology. Data was obtained from clinicians as they interpreted 12-lead electrocardiograms (ECGs). The main aim of the work is provide methods of visualising the differe...

Eye tracking has been widely used to investigate user interactions with the Web to improve user experience. In our previous work, we developed an algorithm called Scanpath Trend Analysis (STA) that analyses eye movement sequences (i.e., scanpaths) of multiple users on a web page and identifies their most commonly followed path in terms of the visua...

Early prototyping of user interfaces is an established good practice in interactive system development. However, prototypes cover only some usage scenarios, and questions dealing with number of required steps, possible interaction paths or impact of possible user errors can be answered only for the specific scenarios and only after tedious manual i...

In this paper, we introduce a new concept called trending scanpath that represents the most popular scanpath among users.