Lawrence Thatcher’s research while affiliated with Brigham Young University and other places

Inexpensive, low-power sensors and microcontrollers are widely available along with tutorials about how to use them in systems that sense the world around them. Despite this progress, it remains difficult for non-experts to design and implement event recognizers that find events in raw sensor data streams. Such a recognizer might identify specific events, such as gestures, from accelerometer or gyroscope data and be used to build an interactive system. While it is possible to use machine learning to learn event recognizers from labeled examples in sensor data streams, non-experts find it difficult to label events using sensor data alone. We combine sensor data and video recordings of example events to create a better interface for labeling examples. Non-expert users were able to collect video and sensor data and then quickly and accurately label example events using the video and sensor data together. We include 3 example systems based on event recognizers that were trained from examples labeled using this process.

Training classifiers for human activity recognition systems often relies on large corpora of annotated sensor data. Crowd sourcing is one way to collect and annotate large amounts of sensor data. Crowd sourcing often depends on unskilled workers to collect and annotate the data. In this paper we explore machine learning of classifiers based on human activity data collected and annotated by non-experts. We consider the entire process starting from data collection through annotation including machine learning and ending with the final application implementation. We focus on three issues 1) can non-expert annotators overcome the technical challenges of data acquisition and annotation, 2) can they annotate reliably, and 3) to what extent might we expect their annotations to yield accurate and generalizable event classifiers. Our results suggest that non-expert users can collect video and data as well as produce annotations which are suitable for machine learning.

We experiment with using sensors and a machine learning algorithm to detect and label turns in alpine skiing. Previous work in this area involves data from more sensors and turns are detected using either a physics-based model or custom signal processing algorithm. We recorded accelerometer and gyroscope data using a single sensor placed on a skier's knee. Left and right turns in the data were labeled for use in machine learner. Although skiing data proved to be difficult to label precisely, a classifier trained on 37 labelled examples correctly label all 13 examples from a different test data set with 2 false positives. This method allows for the use of a single sensor and may be generalizable to other applications.