Content uploaded by James Mckanna
Author content
All content in this area was uploaded by James Mckanna
Content may be subject to copyright.
Abstract—Maintaining cognitive performance is a key factor
influencing elders’ ability to live independently with a high
quality of life. We have been developing unobtrusive measures
to monitor cognitive performance and potentially predict
decline using information from routine computer interactions
in the home. Early detection of cognitive decline offers the
potential for intervention at a point when it is likely to be more
successful. This paper describes recommendations for the
conduct of studies monitoring cognitive function based on
routine computer interactions in elders’ home environments.
I. INTRODUCTION
LDERS are the fastest growing demographic in many
countries, with a concomitant escalation of health care
resources being spent on conditions associated with aging.
One of the key functional losses at risk with aging is a
decline in cognitive abilities. Estimates vary depending on
assessment protocols and populations studied, but up to 50%
of all individuals over age 85 are found to have measurable
decline in cognitive function [1]. These individuals are at
high risk for dementia, requiring increasing levels of
assisted living. Even mild cognitive declines lead to
degraded quality of life. Thus a major goal of seniors and
their families is to optimize their quality of life and remain
not only physically, but also mentally fit.
Researchers have demonstrated the importance of
identifying decline in cognitive function [2,3,4]. These
changes may be observed as short-term effects, such as
medication side effects or unrecognized medical illnesses.
Optimally, reliable early detection of future cognitive
decline may provide clinicians an opportunity to intervene at
an earlier point in time where therapy could be more
successful.
The goal of our work in the project described in this paper
was to develop and test monitoring software that would
provide trend information on metrics that are likely to be
Manuscript received December 24, 2005. This work was supported by
ORCATECH, Oregon’s Roybal Center for Aging & Technology with a
grant from NIA (Grant NIA P30AG024978).
H.B. Jimison is Associate Professor of Medical Informatics at Oregon
Health & Science University and a Senior Research Scientist at Spry
Learning (phone: 503-418-2277; fax: 503-494-4551; e-mail:
jimisonh@ohsu.edu).
N.Jessey is a Research Assistant with the Layton Aging & Alzheimer’s
Disease Center at Oregon Health & Science University (email:
jesseyn@ohsu.edu).
J. McKanna is a Senior Software Engineer with the Spry Learning
Company in Portland, Oregon (email: zephy@sprylearning.com).
T. Zitzelberger is a Senior Research Coordinator with the Layton Aging
& Alzheimer’s Disease Center at Oregon Health & Science University
(email: zitzelbe@ohsu.edu).
J. Kaye is Professor of Neurology at Oregon Health & Science
University and Director of the Layton Aging & Alzheimer’s Disease Center
at Oregon Health & Science University.
useful in predicting an elder’s degree of cognitive health.
These metrics include relative typing speed and accuracy,
relative mouse movement efficiency, and relative
performance on computer games that have been designed for
cognitive monitoring. Our continuous monitoring of
cognitive indicators allows us to perform trend detection of
individual performance, so that we are less susceptible to
unwanted variability due to educational, language, and
cultural backgrounds, as compared to traditional cognitive
assessment tests. In this pilot project, we deployed our
monitoring software in the homes of 15 elders in the
Portland, Oregon metropolitan area for a period of 6 months
and tested our ability to monitor computer activities and
keep elders motivated and engaged in using the computer.
This test was in preparation for a larger prospective long-
term study that will determine our ability to use these types
of metrics to predict future cognitive decline. Based on our
pilot test of the monitoring software in the homes of elders,
we developed a set of recommendations for effectively
managing a large number of elderly participants in a long-
term study, and eventually, as a routine service available to
the public at large.
II. BACKGROUND AND MOTIVATION
A. Computer Usage by Elders
There has been a rapid growth in the number of elders
using computers in the United States. According to a 2004
Pew Internet and American Life survey, 15% of adults 65
and older use the Internet [5]. However, adults 65 and older
are participating at faster rates, and that trend is expected to
continue. The survey also showed that 93% of seniors with
Internet access have sent or received email, and that seniors
are more inclined to go online and check email on any given
day than any other group of Internet users. The second most
common online activity for seniors is researching
information, especially health topics. Additionally, over a
third of the population of seniors online play computer
games.
B. Previous Work with Elder Computer Use
In a previous study done with the Spry Learning Company
and sponsored by the Intel Corporation, we conducted focus
groups with older computer users to provide us with
necessary background on existing computing and
communications needs, as well as interface preferences. We
studied 15 residents (mean age 79.5 ± 8.5) in the Calaroga
Terrace retirement community in Portland, Oregon.
Participants were selected because of their interest and use
of computers. Participants identified e-mail, Web browsing,
and computer games as the applications they used most
Monitoring Computer Interactions to
Detect Early Cognitive Impairment in Elders
Holly Jimison, Member, IEEE, Nicole Jessey, James McKanna, and Tracy Zitzelberger, Jeffrey Kaye
E
Proceedings of the 1st Distributed Diagnosis
and Home Healthcare (D2H2) Conference
Arlington, Virginia, USA, April 2-4, 2006
1-4244-0059-7/06/$20.00 ©2006 IEEE. 75
often. Our focus group data indicated that monitoring
incidental computer use such as game playing is not seen as
invasive as people perceive these activities as beneficial for
their cognitive well-being.
Following our needs assessment of elderly computer
users, we then tested a research version of the FreeCell
computer game that monitored play performance on a daily
basis. In this pilot study on 12 participants over a 3-week
period, we demonstrated the feasibility of differentiating
cognitively healthy elders from those with mild cognitive
impairment. This work led us to propose testing unobtrusive
measures of computer interactions in a prospective study to
evaluate our ability to provide early predictions of cognitive
decline.
III. METHODS
Our research methodology centered on testing our
prototype software for monitoring computer interactions of
elders in their residential environment. We included both
experienced and inexperienced computer users, as well as
both cognitively healthy participants and those with mild
cognitive impairment (MCI). We tracked the effort and
techniques used for training and maintaining engagement
with using the computer.
A. Development of Monitoring Software
The monitoring program for this pilot project recorded
data about a user’s keyboard, mouse, and application use
using functions from the Windows operating system (for the
purposes of this pilot project, we standardized on the use of
computers with the Windows 98 or 2000 operating systems).
The system registry starts the program when the machine
boots up, thereby avoiding the need for the subject to start
the program. We ensured that no windows from this
software would be displayed, and that it was not represented
on the user’s toolbar or taskbar, thus minimizing the chance
of a subject accidentally terminating the process and
creating gaps in the monitoring record. Whenever a subject
interacted with the keyboard, the program recorded the date,
time, key pressed, and the number of milliseconds (ms)
since the last keystroke (inter-stroke interval). For mouse
movements, it recorded the date, time, mouse location (in X,
Y screen coordinates), and the number of milliseconds since
the last mouse movement. This last measure is usually
determined by the mouse sensitivity and the screen refresh
rate of a given computer, since the program records every
movement of the mouse, rather than just the endpoints of a
given mouse motion. During application use, the software
recorded identifying information about the application (title
and class), as well as the date, time, and milliseconds since
the previous application was active. Applications that were
active for less than 500ms were ignored, to avoid flooding
the record with the large number of applications intended
only for system use; due to their exceedingly short duration,
these applications are unlikely to have been part of a human-
computer interaction.
In this project we developed additional software to
automatically transfer the monitoring data to a secure server
behind our institution’s firewall. A data log program sent
daily reports of attempted data transfers to the researchers on
the project, with the intention of being able to intervene with
the study participants if they were having trouble using the
computer.
B. Subject Recruitment and Descriptive Statistics
We enrolled 15 participants, both men and women, above
the age of 70 years to have their home computer use
monitored for a period of 6 months. The participants were
recruited from senior residential facilities in the Portland,
Oregon metropolitan area and from a research participant
registry at Oregon Health & Science University’s Layton
Aging Alzheimer’s Disease Center). All subjects signed
written informed consent to participate.
The protocol for the study required that on the first visit to
the participant’s home, the research assistant would consent
the participant and then administer the following set of
cognitive tests:
xFinger Tapping Speed (measuring handedness, 3
trials on both right and left hands)
xMini Mental State Examination
xGeriatric Depression Scale
xCERAD Word List Acquisition / Delayed Recall /
Intrusions / Distractors
xDigit Symbol Test
xLetter-Number-Sequencing
xVerbal Fluency Test (Category: Animals, Fruits,
Vegetables)
xBoston Naming
xClinical Dementia Rating
In addition to the cognitive tests, the research assistant
assessed the participants Activities of Daily Living,
Instrumental Activities of Daily Living, and administered a
Computer Use and Attitudes Survey.
The 15 subjects enrolled in this study ranged in age from
71 to 96 years (mean age of 82.8 r 6.3 years). 12 subjects
(75%) were women. From the cognitive assessments, we
determined that 3 of the 15 subjects had mild cognitive
impairment (20%). All subjects were living independently
in their own home or apartment.
Our Computer Use and Attitudes Survey consisted of
questions on computer self-efficacy, computer anxiety,
general technology use, and previous experience with
computers. With regard to computer use, 13 (81%) reported
having used a computer before, 9 (56%) already had a
computer in the home, 9 (56%) had used email before and 8
(50%) had been on the Web. We also asked about their
preferences for computer games. 4 (25%) of the subjects
reported liking computer games, although none of the
subjects were already familiar with FreeCell.
The self-efficacy questions on the computer survey were
of the form “How confident are you in your ability to …”
76
Although some participants were initially confident in their
ability to move a cursor or use email, the vast majority had
little initial confidence on most computer tasks.
We also measured participants’ response to questions
about computer anxiety. In general, there was an even
spread in responses among the participants, with most mean
scores resulting in a “neutral” response. The survey
responses for these questions ranged from 1 = Strongly
Disagree to 5 = Strongly Agree, with 3 being Neutral. The
following is a list of the topics covered by the computer
anxiety portion of the survey, with the mean response at the
end in parentheses.
1. I feel anxious whenever I am using computers. (3.0)
2. I wish that I could be as calm as others appear to be when
they are using computers. (3.6)
3. I am confident in my ability to use computers. (2.5)
4. I feel tense whenever working on a computer. (2.8)
5. I worry about making mistakes on the computer.(2.6)
6. I try to avoid using computers whenever possible. (2.6)
7. I experience anxiety whenever I sit in front of a computer
terminal. (2.6)
8. I enjoy working with computers. (3.3)
9. I would like to continue working with computers in the
future. (4.0)
10. I feel relaxed when I am working on a computer. (2.5)
11. I wish that computers were not as important as they are.
(3.0)
12. I am frightened by computers. (2.0)
13. I feel content when I am working on a computer. (2.8)
14. I feel overwhelmed whenever I am working on a
computer. (2.9)
15. I feel comfortable with computers. (2.7)
16. I feel at ease with computers. (2.6)
Finally, we also collected survey data on each participant’s
use of technology in general. We found that more that 80%
of the participants used a television and answering machine
on a daily basis, but that routine use of other technology,
such as microwave ovens, cell phones, VCRs and CDs were
less frequent.
C. Software Installation and Computer Training
The research assistant for this project was trained to teach
elders to use computers using methods established by the
Spry Learning Company, a company specializing in
teaching seniors to use computers. In their day-long training
of the research assistant, they emphasized effective teaching
strategies for working with seniors and dealing with self-
confidence barriers, as well as hands-on practice preparing
and teaching classes based on Spry Learning’s Computer
and Internet Literacy Curriculum. Our goal in this phase of
the project was to develop methods for efficiently training
elders to use computers and keep them engaged during the
study.
The computer training took place during a second visit by
the research assistant to each elder’s home. Most initial
training took approximately 2 hours. We monitored training
time and emails used to prompt people throughout the study.
Some of the particularly challenging issues in this study had
to do with enrolling 3 subjects with mild cognitive
impairment and not providing subjects with high speed
“always on” connections to the Internet. For this study, we
provided 7 of the subjects (44%) with computers and
monitors (refurbished hardware from OHSU) and newly
purchased modems. Much of the training difficulty and later
questions centered on user interface issues surrounding the
process of dial-up for modem access to the Internet. The
process had substantial delays with no obvious indication of
what was happening during the delay available to the
subject. Our users often kept trying to “make it work”
during the dial-up. An important additional drawback to
using modem access with a single home phone line for this
study is that the computer use poses a safety issue when the
phone line is tied up for long periods of time (users often
forget to log off). However, this would not be a problem if
cable or DSL access were provided, as they are “always on.”
The 3 participants with mild cognitive impairment were very
difficult to train and keep in the study. The participants in
the study were initially classified as cognitively normal, with
a CDR (Clinical Dementia Rating) of 0 during the initial
assessment visit. During repeated home visits over a period
of several months (?) for computer training and
downloading of data it became clear to the research team
that 3 of the participants had MCI, with a CDR of 0.5.
Future studies will need to determine the range of cognitive
impairment that may be present in persons with MCI that
will allow such a subject to follow a computer training
program or for those MCI subjects with prior computer
experience, the degree to which they can continue to
function at the computer.
All of the participants in the study were given hard copy
training materials to remind them of the key concepts. In
general, whereas, current computer users could be brought
up to speed with the requirements of the study and training
on new activities, teaching new users who had never owned
a computer before was challenging and usually took
multiple home visits. There were not enough subjects in
this pilot study to identify other specific subject
characteristics that might influence the capabilities of the
subjects to learn or use the computer.
This pilot project was useful in providing us with a better
understanding of the training challenges involved with new
computer users who live in their homes (as opposed to
residential facilities with several subjects in one location). In
addition, we saw a clear value for remote access (with
permission) to a study participant’s computer for “just-in-
time” training and technical support.
IV. KEYBOARD AND MOUSE ANALYSIS
We looked at keyboard typing speed and mouse
movement indicators of motor activity. For the analysis of
77
keyboard typing speed, our previous work had focused on
using repeated samples of user login typing speed. The
uniform context, presumed steady-state of learning, and
multiple identical measures make this sample of typing
speed as independent of unwanted variation as possible. For
a moving estimate of speed we measured inter-stroke
interval for a known login for each user and averaged each
between key time for an estimate of speed for a single login.
We then used a trimmed mean (excluding the upper and
lower 25% of values) with a moving average over a window
of 1 week for a dynamic estimate of typing speed. We also
used the central 95% of data to measure variability of typing
speed from day to day.
In our analysis of participants’ mouse movements, we
focused on the idea that measuring the efficiency of
trajectories executed by the user may provide useful
information relating to sensory-motor function. The basic
data consist of point-to-point movements, where each move
is represented by samples in time corresponding to the
locations of the cursor on the computer screen. We
developed metrics that are rotation and scale invariant, for
example the ratio of the lengths of the trajectory to the
distance between the starting and ending point. In addition,
we measure Fourier Descriptors in terms of harmonic
functions that capture the various rates of deviation from a
straight line.
In general, there is a possible relation between keyboard
speed (inter-stroke interval) and the variability of
performance and cognitive function. We examined the
means and standard deviations of keyboard inter-stroke
interval hypothesizing from earlier work that the degree of
variability may the best indicator of dementia onset. More
longitudinal data in a larger sample of normal and declining
individuals will be needed to arrive at a more definitive
conclusion with regard to this relationship.
V. RECOMMENDATIONS
Our experiences in this pilot study have provided us with
insight on how to best conduct future studies on monitoring
computer interactions in the home environment. A summary
of recommendations is outlined below:
xIn protocols calling for computer training and ongoing
interaction, careful cognitive assessment at entry and as
the study progresses is necessary to determine the effect
of cognitive decline.
xFor computer interactions monitored in the course of
everyday interactions, cable, DSL or wireless access to
the Internet provides the most seamless interface to the
subject.
xProvide tested user-friendly interfaces to key
applications.
xUpload data nightly with variable access times. Use
Remote Access software to be able to view a
participant’s screen remotely to provide technical
support and “just-in-time” training.
xRecognize the training effort involved for the research
team if new computer users must be each trained in
their homes (as opposed to residential facilities with
several subjects in one location).
xCreate a system for a research assistant to view key data
on subjects on a daily basis. Key variables to record /or
display may include:
xSubject Characteristics
oSubject Identifier
oAge
oPrevious computer experience
oType of computer, type of connection
oOther people in household
oFavorite people to communicate with
oFavorite computer activities
oGoals for future computer use
xCurrent computer activities (plot daily use)
xWeekly “diary” (health issues, vacations, etc.)
xHistory of communications
xLibrary of news items of interest to elders (to use
on Bboards and emails)
xA group email tool that tailors similar emails to
subgroups of users
xCreate a very simple Web Portal for all participants in a
future study to use that includes instant messaging, an
electronic bulletin board for the study, a clickable news
window with frequent updates.
VI. CONCLUSION
We have shown that it is feasible to monitor computer
interactions in the homes of elders. These observations
provide guidelines for future larger in-home or residential
facility-based studies. Several measures, such as computer
use activity, keyboard typing speed, and variability in
performance, are promising measures to include in
algorithms for predicting and monitoring cognitive decline.
This monitoring information may prove to be valuable in
the cognitive health management of elders, potentially
providing indications for early treatment to maintain
function and independence.
REFERENCES
[1] R. N. Anderson, Method for Constructing Complete Annual U.S. Life
Tables. National Center for Health Statistics. Vital and Health Stat,
Series 2(129) (1999).
[2] P. Erickson, R. Wilson, I. Shannon, I.: Years of Healthy Life. US
Department of Health and Human services, CDC, National Center for
Health Statistics, Hyattsville Maryland Statistical Notes 7 (1995).
[3] L. Boise, D. L. Morgan, J. Kaye, R. Camicioli, Delays in the
Diagnosis of Dementia: Perspectives of Family Caregivers. American
J. of Alzheimer's Disease 14(1) (1999) 20-26.
[4] P. Glascock, D. M Kutzik,. Behavioral Telemedicine: A New
Approach to the Continuous Nonintrusive Monitoring of Activities of
Daily Living. Telemedicine, 6(1) (2000) 33-44.
[5] Pew Internet Project. Older Americans and the Internet. Pew Internet
and American Life Project, www.pewinternet.org . March 2004.
78
