Content uploaded by Louise Yarnall
Author content
All content in this area was uploaded by Louise Yarnall on Jan 07, 2017
Content may be subject to copyright.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 1 of 11
PERLS: An Approach to Pervasive Personal Assistance in Adult Learning
Michael Freed, Louise Yarnall, Jason Dinger, Melinda Gervasio, Adam Overholtzer, Mar Pérez-Sanagustin,
Jeremy Roschelle, Aaron Spaulding
SRI International
Menlo Park, CA
freed@AI.SRI.COM, louise.yarnall@sri.com
ABSTRACT
Adult learners in both military and civilian settings increasingly use mobile devices for “Pervasive Learning”
(Banavar et al., 2000; Thomas, 2007), which occurs without classrooms, instructors, and training facilities. By
expanding options for what, when, and how we learn, Pervasive Learning has the potential to remedy stubborn
deficiencies of traditional instruction. The central feature of PERLS is a virtual personal assistant that supports self-
learning by recommending specific content, general topics, and various learning actions based on learners’ interests,
available time, and location. PERLS is intended to guide learners to resources located in both formal (closed corpus)
and informal (open corpus) repositories. In this paper, we present the pedagogical design, user interface, system
architecture, initial concept validation results, and field test goals for PERLS, a prototype PERvasive Learning
System. The concept validation and field-testing take place in one civilian corporate context. The concept validation
indicated that adult learners in the corporate setting favored limited use of “push” reminders to engage in learning
and broader use of adaptive lists of content that have been intelligently informed by contextual data about their
interests and available time for learning. Planned field tests will examine system functionality, usability, and impacts
on self-learning habits around corporate onboarding content for new hires.
ABOUT THE AUTHORS
Michael Freed is a program director in the Artificial Intelligence Center at SRI International. His work focuses on
technology for intelligent personalized assistance in education and training, health care, and interaction with
complex automation.
Louise Yarnall is a senior researcher in the Center for Technology in Learning at SRI International. Her work
focuses on educational technology and program evaluation and design for adult learning and workforce training.
Jason Dinger, Melinda Gervasio, Adam Overholtzer, Mar Pérez-Sanagustin, Jeremy Roschelle, and Aaron
Spaulding are members of the PERLS research and development team, with varied expertise in educational
technology, human-computer interaction, and recommender systems.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 2 of 11
PERLS: A Concept and Formative Evaluation of the Architecture for
Pervasive Personal Assistance in Adult Learning
Michael Freed, Louise Yarnall, Jason Dinger, Melinda Gervasio, Adam Overholtzer, Mar Pérez-Sanagustin,
Jeremy Roschelle, Aaron Spaulding
SRI International
Menlo Park, CA
freed@AI.SRI.COM, louise.yarnall@sri.com
A PERSONAL ASSISTANT FOR LEARNING
As public and corporate budgets decline for formal educational approaches to adult learning, research focuses on
using technology to improve the efficacy of self-directed learning, or “self-learning,” (Collins & Halverson, 2010;
Kay & Kummerfled, 2010; Redecker et al., 2011). Without a human guide, it can be daunting to know which formal
and informal resources and experiences will best address personal knowledge gaps or build confidence in a new
learning domain. In practice, good mentorship that provides such guidance is the exception. Adult learners are
generally on their own.
We are designing and testing an architecture that addresses this “missing mentor” problem by building on advances
in intelligent virtual assistant technology and the widespread adoption of mobile, context-aware devices. The system
offers the possibility of providing self-learning assistance that is always available and scales to large learner
populations. Our system is delivered on a mobile phone through an interface that provides adaptive navigation of a
corpus of learning resources (Brusilovsky & Nejdl, 2005) and a persuasive recommendation framework that fosters
the self-learning skills of planning, reflection, and social learning.
Called PERLS (PERvasive Learning System), our system is a prototype platform developed under the Advanced
Distributed Learning (ADL) Personal Assistant for Learning (PAL) program. PAL aims to offer the highest quality
learning and performance support that can be tailored to individual needs, and delivered cost effectively at the right
time and at the right place. It seeks to develop an architecture that brings together diverse learning resources and
tracks the learner experience intelligently to recommend personalized learning pathways.
PERLS combines anytime/anywhere content delivery with a context-aware personal assistant. The personal assistant
is intended to help typical learners make choices and take actions that strong self-learners use to improve learning
outcomes. In its prototype phase, PERLS is intended to support lifelong (or at least employment-long) learning for
members of large organizations where the need to support learning is high, some capacity to invest in it exists, and
there is a concentration of personnel to support social learning and aggregate data analysis. Ideally, users would be
given access to PERLS at the inception of their employment when the need to learn is especially high and a good
time to engage learners in using a personal learning assistant.
PERLS differs in important ways from content-focused technologies such as cognitive tutors. Cognitive tutors use
technology to automate and personalize formal instruction in academic domains. By contrast, PERLS might
recommend content offered through a cognitive tutoring application. PERLS aims to automate and personalize the
kind of mentorship that supports self-learners’ decision-making about what to study and when in multiple domains.
It is our view that, no matter what the content area, adult learners can benefit from approaching learning in a
disciplined way that is informed by improved awareness about the resources and time they have available.
The focus of this paper will be on identifying the requirements and design of a PERLS virtual personal assistant that
facilitates self-learning. We summarize the literature on self-learning and describe how PERLS’ capabilities for
context-aware, persuasive recommendation are used to provide self-learning assistance. We then discuss the results
of a concept validation study and plans for field-testing. The concept validation has informed the PERLS user
experience design around two key learner behaviors: The decision to engage in learning, and the selection of
appropriate and compelling content. Field testing will inform the PERLS user experience design and engineering,
while testing the impacts on self-learning awareness and engagement.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 3 of 11
SUPPORTING ADULT SELF-LEARNING
Challenges to Self-learning
To enhance adult self-learning, we focus on addressing the most important challenges faced in improving self-
learning goals: finding time and knowing how to assess one’s progress so as to inform next steps in learning.
The time limitations surrounding adult learning are clear in the literature. Adults engage in learning in a range of
contexts—work, community, and classrooms. It involves a mix of both self-directed and expert-directed
experiences, and it occurs relatively regularly, on average about 2 hours a day (Livingstone, 1999; Tough, 1971).
Research indicates that most adult learning is self-directed and occurs in urgent, time-pressed conditions in informal
and incidental ways rather than in extended, mentored conditions similar to school-based learning. Little empirical
research indicates how effective much of this hurried adult learning is. For assessment, adults may seek out and rely
on external authority figures to judge the quality of their self-directed learning or they may cultivate habits of
reflecting on their performance and generating new learning goals (Graves, Rauchfuss, & Wisecarver, 2012;
Marsick & Watkins, 2001). This review indicates that the context of adult learning poses problems of time
constraints and urgency around short-term learning goals, and finding available time to study and reflect to inform
longer-term learning.
In addition to these high-level challenges of adult learning, past research identifies a raft of social and motivational
threats to self-learning. Studies of self-regulation in classroom and workplace settings note that surrounding social
influences strongly impact learners’ decisions to abandon learning (Kuhl, 2000; Marsick & Watkins, 2001).
Finally, studies of schoolchildren (Anderman et al., 2001) indicate that self-learning skill is not a trait that presents
itself consistently; rather, learners may be attentive and focused in one environment or in one subject area, and
distracted and disinterested in others. In sum, it is a malleable skill that may be influenced positively.
Positive Supports for Self-Learning
Meta-analyses of programs that teach schoolchildren how to regulate self-learning indicate they are effective to the
extent that they make learners aware of strategies relevant to specific learning situations (Hattie, Biggs, & Purdie,
1996; Rosenshine, Meister, & Chapman, 1996) and help them apply these strategies as a matter of habit. Studies
identify several activities of special importance in achieving good self-learning outcomes: [1] use of metacognitive
strategies that include orienting oneself before starting on a new task, collecting relevant resources, self-monitoring
comprehension, and self-assessing progress; [2] use of motivation strategies that include setting the scene for
learning, assigning value to a learning activity, getting started, and sustaining effort until task completion; and, [3]
use of volitional strategies that include negotiating local social settings and changes in surrounding learning context
that affect the learning process (for review, see Boekaerts & Cascallar, 2006 and Graves, Rauchfuss, & Wisecarver,
2012).To address the threats to self-regulation, it is key to develop coping strategies to reduce transient arousal to
support the use of volitional strategies to re-focus on the task. Self-learning involves using both “top down” and
“bottom up” self-regulatory systems (Boekaerts & Niemivirta, 2000; Boekaerts & Corno, 2005). There is a top-
down “goal pathway” that serves as a motivating structure and a bottom-up approach for managing one’s well-being
while learning and minimizes distractions that detract from achieving learning goals. This theoretical framework
suggests the complexity of the self-learning process. It is likely to be one that can break down for multiple reasons
and the reasons may vary by learning context and topic.
Studies of learning motivation of schoolchildren indicate that those who have personally meaningful learning goals
are in a better position to learn to self-regulate their own learning (Elliot & Sheldon, 1997). By contrast, for most
adults, learning goals will include those that are externally imposed and those that are intrinsically motivating and
personally meaningful. Externally imposed goals are more likely to be pursued in a short-term, episodic fashion,
while personally meaningful learning goals are more likely to be pursued in the longer term (Ryan & Connell, 1989;
Ryan & Deci, 2000; Sheldon & Elliot, 1998). Support for adult self-learning needs, therefore, to focus on supporting
both intrinsically and extrinsically motivated learning.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 4 of 11
Technology’s Role: Persuasive Recommendation and Self-Learning
Based on this review, pervasive technologies for personal assistance can help self-learners apply a set of
metacognitive strategies, motivational strategies, and volitional strategies. Technology can help by:
• collecting resources for learners and supporting self-monitoring of progress through them (metacognitive
strategy support)
• assigning value to them and helping them get started (motivational strategy support)
• helping to train the learner to persist in learning tasks across different social settings (volitional strategy
support)
Underlying each of these forms of assistance is the need for technology to be able to figure out what actions are
likely to be both valuable and compelling to the learner in a given situation—i.e. to be able to make persuasive
recommendations in context.
Research on persuasive technologies (Fogg, 2003) indicates some effective ways of fostering uptake of
recommendations. All use current and historical context data about the learner to influence behavior around learning
resources (Consolvo, Everitt, Smith, & Landay, 2006; Gasser, Brodbeck, Degen, Luthiger, Wyss, & Reichlin, 2006).
Learner data is most persuasive if it is the following (Consolvo, McDonald, & Landay, 2009): 1) goal-oriented to
support reflection on progress, 2) unobtrusive in presentation to avoid annoyance, 3) public to activate self-
consciousness, 4) aesthetic, meaning that the technology must be attractive, 5) positive in tone, 6) controllable by the
user, 7) historical, showing the users’ past behavior, and 8) comprehensive, meaning it permits inspection of all
behaviors at different levels of granularity. Such technology effectively can create a model of adult learners’
interests and availability for engaging with learning resources, and then make recommendations customized to that
individual schedule and set of interests. Further, a mobile, recommender-based technology can reinforce adult
learners’ awareness of particular long-term goals by creating a user interface that represents desired learning goals
and progress toward attaining them in a motivating, aesthetically pleasing manner.
PERLS
PERLS (PERvasive Learning System) is a personal assistant learning application designed to support adult self-
learners. It provides an extensible platform for diverse instructional technologies, especially including those
developed under the ADL Personal Assistant for Learning (PAL) program. The platform mediates
anytime/anywhere (i.e. pervasive) access to instructional content, allowing users to learn at times and places that suit
their preferences, regulate the pace of learning to suit their schedule, and situate learning to take advantage of
environmental affordances. Personal assistance capabilities in PERLS help users realize potential benefits from
anytime/anywhere access. Instead of relying solely on the learner’s organizational skills, observational skills, and
drive, the assistant encourages and facilitates strategies associated with good self-learning outcomes.
For illustration, consider a new employee given PERLS for the purpose of “enhanced onboarding.” The employee
might start learning compliance content such as how to fill out a timecard or follow safety procedures. But as the
built-in, context-aware assistant learns and adapts to user patterns and preferences, the employee begins to see more
interesting and intrinsically motivating content that helps him/her to operate more effectively in the job role, develop
insider awareness, and make connections. PERLS gently helps the learner establish learning goals, find time and
resources to pursue those goals, clarify and reinforce motivations, and make steady progress. Eventually the learner
may become accustomed to this sort personal assistance and use it to support learning for professional development
throughout their period of employment.
PERLS provides user guidance through Smart Lists and Smart Alerts. With Smart Lists, the user can swipe through
an ordered set of “cards,” quickly deciding whether to select the content or action suggested on the card or to keep
browsing. PERLS uses context-aware recommendation engines (Lonsdale, Baber, Sharples, & Arvanitiss, 2004;
Verbert et al., 2012), to order cards based on context-dependent estimates of the learner’s goals, readiness,
willingness to spend time learning each item and other factors. Each content card (see Figure 1a) displays
contextual “sell points” explaining why PERLS made the recommendation. For example, a sell point might highlight
that the user is near some fleeting learning opportunity, that peers have found the content valuable, or simply that the
user is likely to enjoy the material. Action cards encourage learners to engage in reflection, goal-setting, planning
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 5 of 11
and other metacognitive, motivational, and volitional self-learning strategies. Figure 1b shows an example action
card in which the learner is encouraged to set a learning goal for a topic in which she/he has shown a pattern of
interest. Accepting the recommendation brings the user to the view shown in Figure 1c for specifying topic learning
goals. For example, if the user selects “Explore,” PERLS will tend to recommend additional content on the topic,
preferring items that are interesting but not very challenging. If instead, “Study” is selected, PERLS will recommend
content representing a logical progression of content, and will track progress.
Sell points are based on context information drawn from diverse sources including mobile devices sensors, remote
data services, recent user interactions, analysis of past user behaviors, and analysis of related user populations.
However, not all important context factors can be sensed or inferred. In particular, it is difficult to infer immediate
learning motivation—i.e. why the user is choosing to interact with PERLS right now and what type of learning
experience the user seeks. To compensate, PERLS provides several different types of Smart Lists for a user to
choose among, each corresponding to different kind of interaction motive, and each backed by a separate,
specialized recommendation engine. Different Smart Lists focus on: limited time a user has available for learning
(Quick Pick), a user’s desire for novelty (Surprise Me), a user’s drive to make progress on longer-term goals (To
Do), and a user’s interest in nearby or near term learning opportunities (Here and Now). The system will learn over
time what types of transient learning orientations appeal to the learner.
Smart Alerts address the case where the learner faces challenges related to timing, such as recognizing that a
window of availability for learning has arisen, detecting a specific, transient learning opportunity, or remembering a
prior intention to learn at some time or in some circumstance. PERLS uses the same context awareness and
persuasive recommendation technologies for Smart Alerts as for Smart Lists. However, since mobile device users
are often averse to app alerts, PERLS, filters alert candidates aggressively based on user preference information.
User interface elements of Smart Lists and Smart Alerts together with underlying capabilities for content-aware,
persuasive recommendation, support a range of self-learning strategies associated improved learning outcomes (See
Figure 2).
Figure 1. PERLS user interface showing (a) content card with “sell point”, (b) action card with learning
recommendation, (c) goal setting view
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 6 of 11
Figure 2. PERLS interface self-learning logic model
DESIGNING PERSUASIVE RECOMMENDATION TECHNOLOGY FOR SELF-LEARNING IN THE
WORKPLACE: A CONCEPT VALIDATION STUDY
To address the remaining open questions about how best to implement the PERLS concept of persuasive
recommendation, we have conducted a concept validation study, and are in the process of conducting field tests.
Purpose and Design of Concept Validation
The concept validation aimed to identify key needs and preferences of corporate learners affecting adoption and
continued use of a personal learning assistant. It focused specifically on persuasive recommendations appearing as
content cards in Smart Lists or as Smart Alerts. The study engaged 24 corporate employees (15 females and 9 males;
7 aged under 30, 10 between 31–50 and 7 over 51; 6 working at the company less than 1 year, 5 between 2–4 years
and 13 over 5 years).
First employees responded to a 28-question survey asking them to: (1) rate their level of agreement on a 5-level
Likert scale with 7 possible challenges to self-learning related to finding time and resources, and making and
gauging progress; (2) to estimate the amount of time they devoted each week to urgent and longer-term professional
learning goals and personal learning goals; (3) to describe the types of topics they pursued in each of these three
types of learning goals recently; and (4) to describe the types of resources and information-seeking technologies
they employed in self-learning.
Second, the same employees (n = 23; 1 was unable to attend) participated in one of two 1-hour focus groups rating
the desirability and value of a range of scenarios using the PERLS Smart Lists and Smart Alerts persuasive
recommendation system. Ten storyboards were generated and organized into five pairs with contrasting uses of
Smart Lists and Smart Alerts. Each of these storyboard pairs focused on one type of adult learning goal, from urgent
professional to long-term personal, and a particular type of corporate content, such as a low-stakes general interest
topic (e.g., interesting corporate accomplishment stories) or a high-stakes compliance topic (e.g., completing a
required online course on hazardous materials) (See Figure 3 for storyboard example).
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 7 of 11
Figure 3. Example of storyboard use case for concept validation
In analysis, we examined descriptive frequencies of adult learning challenges and patterns, and we conducted a one-
way ANOVA procedure to examine differences in ratings of Smart Lists and Smart Alerts and a hierarchical linear
model (with survey items nested within person) for looking for general trends overall and by subgroup (e.g., age
bracket, job role, gender).
Findings of Concept Validation
The findings report on the self-learning habits and challenges of the sample of adult corporate learners and their
views of the PERLS concept of Smart Lists and Smart Alerts to develop and improve self-learning habits.
Adult learners in the corporate setting reported self-learning challenges and habits consistent with past research:
“Finding time” posed the greatest challenge, followed by “finding resources to start learning” and “keeping on track
with long-term goals.” We found learning goals vary according to their work schedules. They estimated that most of
the time (74%) they devoted to meeting “urgent” learning goals occurred during work hours, while up to the third of
the time devoted to meeting broader “professional” learning goals occurred outside work hours, and—in a surprising
finding—up to 22% of the time spent meeting “personal” learning goals occurred during work hours. These findings
indicate that a self-learning tool should be useful for both work-related and personal learning purposes—and that the
nature of the needs for self-learning support may vary by time of day.
Learners strongly preferred Smart Lists to Smart Alerts as the technological means of supporting self-learning. They
said they liked the idea of learning resources prioritized by their interests, time availability, and location, and, in
particular, they believed the Smart Lists would give them control around when to choose to use these resources.
They generally disliked Smart Alerts as potentially intrusive, but appreciated them if they “nudged” them to meet
desired learning goals.
We also found learners expressed a strong preference for preserving work content learning for work time and
personal content for off-work time. They particularly disliked having any “push” of high-stakes compliance content
recommended to them outside of work. Such content has low intrinsic interest. They embraced the idea of
persuasive recommendations helping them carve out time during “down periods” for productive self-learning and
finding ways to schedule urgent work learning tasks during the workday.
NEXT STEPS: FIELD TESTING WITH NEW EMPLOYEES
We have begun field testing PERLS with new employees to support and enhance corporate onboarding. Tests will
unfold in two stages: Basic usability/desirability and Initial usage trends. We will briefly outline the design of these
formative design evaluations and the evidence we plan to collect.
During field-testing, we are offering a range of onboarding content accessible through the PERLS system based on
interviews with corporate content providers. We will also test out initial content contribution systems to support
informal, crowd-sourced content generation.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 8 of 11
The initial PERLS content corpus focuses on onboarding content since this is a group known to have high levels of
need for learning. We interviewed content providers to identify the primary sorts of content that new hires need and
to understand how new hires cycle through content during their first year of employment. The types of work content
address a range of urgent and long-term learning goals, as well as goals that we found in the concept validation
corporate learners saw as strictly confined to completion during work hours and types of content that they saw as
having broader personal value and utility during off-work hours.
The learning content addresses a range of onboarding and general employment related goals: Practical Content,
which includes content for learning how to use corporate IT systems and equipment for specific tasks and that
includes “tips” that convey information that is often not formally conveyed but learned the hard way on the job;
Corporate Lore Content builds understanding of corporate culture and history, helping new employees to “become
insiders” in a new environment, a factor associated with improved job satisfaction and retention; Professional
Development Content, which includes learning the corporate marketing strategies, how to write, and how to manage
teams, helps the learner become more effective at core job responsibilities; Compliance Content helps learners
follow internal procedures and meet both corporate and legal requirements. Task support content helps learners
perform task independently rather than seek help from, e.g., administrative and IT personnel. The initial (seed)
corpus will be extended continuously during the field-testing phase. Some of this content, such as the professional
development courses, is formal with sequences and external instructors, but most of this content is informal and
unstructured.
Basic Usability/Desirability Sub-study
Our initial field-testing focus is on usability and desirability, and supporting self-learning. This phase has so far
involved 2 corporate new hires selected by job role (management, non-management). They were selected from those
signing up for weekly orientation sessions and receiving approval from their hiring managers. In addition, we have
involved 2 “controls” who are also starting their new jobs but are not using the PERLS system. Examples of
research questions to be addressed by initial field-testing phase are:
1. How usable and desirable is the PERLS adaptive navigation interface (Smart Lists, Smart Alerts, sell
points, action cards)?
2. Which self-learning strategy supports are most desired and which are typically used without PERLS? To
what extent are these self-learning strategies metacognitive, motivational, or volitional?
3. What PERLS’ learning content is perceived as useful and timely? Do perceptions vary by the job role of
user or the type of content?
At the close of the normal corporate orientation, we have given each PERLS-using participant an iPod loaded with
the PERLS system and a wireless connection to the corporate system. We instructed them on how to use the system
to find content. We configured PERLS to their interests for both urgent and longer-term professional and personal
learning goals. We have instructed them on how to configure the device with wireless servers outside of work and
request that they do so to ensure they have access to PERLS at outside-work locations too. Non-PERLS participants
were introduced to the study too, and told that we want to understand their experiences of starting at the company.
Both groups have been directed to use online resources to support their initial self-learning activities in the first
week of employment, with PERLS users asked to check the app at least 3 times and non-PERLS users asked to
check the company intranet at least 3 times. We interviewed them immediately after hiring about learning needs and
plans, and then followed up a week later.
This phase is also focused on improving functionality and developing an initial framework for understanding how to
interpret user log file data.
We have an initial set of expectations about how much new hires will desire different types of corporate onboarding
content over the first two months on the job (see Table 1). We are refining this model based on feedback from the
first pilot study. In addition, we are using the data from the initial pilot test to refine the PERLS adaptive navigation
tools, log files, and content contribution system as needed.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 9 of 11
Table 1. Sample content interest priorities* for corporate new hires during first year and beyond
*1 is low priority and 5 is high priority
Preliminary findings show positive reactions to the user experience and interface, relatively low need for self-
learning support, and some variation in preferences for types of desired content by job role.
The status of being a new hire is highly motivating for self-learning, and all 4 participants were frequently
monitoring their progress and accessing resources (metacognitive strategies). Self-learning the first week on the job
focused on understanding their job role and responsibilities (9.75 on a rating scale from 1 to10) and company
procedures (8.75/10). These learning goals correspond with PERLS’ Tips content. Usage logs showed that PERLS
users selected a mix of content types: Tips (M time spent = 1m10s), Lore (M time spent = 5m21s), and a particular
content subtype called Perks, which describes special benefits available the workplace (M time spent = 1m78s).
Non-PERLS users reported a similar focus on Tips and Perks on the corporate intranet, but not on Lore.
At the close of the first week, all new hires reported positive levels of satisfaction with their progress, and all
reported turning to multiple resources, with their supervisors, colleagues, and the company intranet as the most
important. On average, they reported turning to each of these resources about 4 times a day. All reported that they
determined they still needed to keep learning by the frequency of needing to ask questions and seek help to do
relatively routine aspects of their work.
These preliminary results indicate that PERLS may be helpful for suggesting different types of content (e.g., Lore
about the new company) from those types typically accessed by new hires (e.g., Tips). One administrative PERLS
user did request a specific type of content—a task support tool to help navigate new corporate systems because her
division lacked such task support tools. PERLS’ utility as a personal assistant to support longer-term self-learning
may come later in the life cycle of coming on board a new company.
Initial Usage Trends Sub-study
We next plan to start the second phase of the field test, we will address a second level of research questions related
to PERLS and self-learning, as follows:
1. How usable is the PERLS system for tracking and supporting self-learning over two months?
2. How effective is the PERLS system in using incoming log data to improve recommendations and address
gaps in content?
3. How accurate are the recommendations made by the PERLS system based on learner interests and
available self-learning times? How may uptake of these recommendations be improved to support self-
learning?
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 10 of 11
For this phase, we will expand the number of participants to up to 30–40. To explore the self-learning question, we
will engage half in using PERLS and half in not using PERLS. To address the log-file question, we will focus on the
PERLS users only. In this case, we will offer PERLS for use on learners’ iPhones (the prototype is available only in
the Apple OS at this time). It is critical that PERLS be made available on the mobile phones that learners typically
use rather than a dedicated phone for the purposes of this field-test. So, for these reasons of ecological validity, we
will make having an iPhone a condition of study participation.
To examine self-learning, we will examine trends in how learners’ goals, task completion, and goal generation
evolve through qualitative data from two online surveys administered to participants and some selected embedded
assessments periodically inserted into the PERLS interface. Further, we will link the log data about how much time
they spend on certain learning content recommended by PERLS and collect data from both periodic queries
embedded in PERLS and follow-up interviews to determine how log data about “time logged with content” and
“type of learner activity” relates to achieving self-learning goals. We will use the findings from these periodic
queries and interviews to improve the design of how log data informs recommendations for content, and check back
with users to see if these recommendations are more accurate and useful.
At the conclusion of the two phases of field-testing, we will have a better model of how the PERLS adaptive
navigation system works to support self-learning and different types of content. We expect that the findings will
make a technical contribution to the field’s knowledge about what types of contextual data can inform self-learning
recommendations and about what types of persuasive recommendations related to planning, reflection, and social
learning may be developed to improve the use and contribution of informal and formal learning content.
ACKNOWLEDGMENTS
We wish to thank Paula Durlach of the Advanced Distributed Learning initiative of the U.S. Department of Defense
for sponsoring this research. This material is supported by the ACC-APG Natick Consulting Division under
Contract Number W911QY-12-C0171. Any opinions, findings and conclusions or recommendations expressed in
this material are those of the author and do not necessarily reflect the views of the ACC-APG Natick Consulting
Division.
REFERENCES
Anderman, E. M., Eccles, J. S., Yoon, K. S., Roeser, R., Wigfield, A., & Blumenfeld, P. (2001). Learning to value mathematics
and reading: Relations to mastery and performance-oriented instructional practices. Contemporary Educational Psychology,
26(1), 76-95.
Banavar, G., Beck, J., Gluzberg, E., Munson, J., Sussman, J., and Zukowski D. (2000).An application model for pervasive
computing. In Proceedings of MobiCOM 2000: The 6th Annual International Conference on Mobile Computing and Networking.
ACM Press, Boston, MA, 266–274.
Boekaerts, M., and Niemivirta, M. (2000). Self-regulated learning: Finding a balance between learning goals
and ego-protective goals. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of Self-Regulation
(pp. 417–451), San Diego, California: Academic.
Boekaerts, M., and Corno, L. (2005). Self-regulation in the classroom: A perspective on assessment and
intervention. Applied Psychology: An International Review, 54(2), 199–231.
Brusilovsky, P. and Nejdl, W. (2005) Adaptive hypermedia and adaptive web. In M. P. Singh (Ed.), Practical Handbook of
Internet Computing. Baton Rouge: Chapman Hall & CRC Press.
Collins, A. and Halverson, R. (2010). The second educational revolution: rethinking education in the age of technology. Journal
of Computer Assisted Learning, 26(1), 18–27.
Consolvo, S., Everitt, K., Smith, I. and J. A. Landay (2006). Design requirements for technologies that encourage physical
activity. Proc. of CHI 2006, 457–66.
Consolvo, S., McDonald, D. W., and Landay, J. A. (2009). Theory-driven design strategies for technologies that support behavior
change in everyday life. Proc. of the SIGCHI Conference on Human Factors in Computing Systems, 405–414.
Fogg, B. J. (2003). Persuasive Technology: Using Computers to Change What We Think and Do. San Francisco, USA: Morgan
Kaufmann Publishers.
Gasser, R., Brodbeck, D. , Degen, M., Luthiger, J., Wyss, R., and Reichlin, S. (2006). Persuasiveness of a mobile lifestyle
coaching application using social facilitation. Proc. of Persuasive 2006, 27–38.
Graves, T. R., Rauchfuss, G., and Wisecarver, M. (2012). Self-learning among Army noncommissioned officers: Experiences,
attitudes, and preferred strategies. Research Report #1961. Ft. Belvoir, VA: U.S. Army Research Institute for the Behavioral and
Social Sciences.
Hattie, J., Biggs, J., and Purdie, N. (1996). Effects of learning skills interventions on student learning: A metaanalysis. Review of
Educational Research, 66, 99–136.
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC) 2014
2014 Paper No. 14335 Page 11 of 11
Kay, J. and Kummerfeld, B. (2010). Tackling HCI challenges of creating personalised, pervasive learning ecosystems. Proc. of
the European Conference on Technology Enhanced Learning, 6383, 1–16.
Knowles, M. S. (1984). The Adult Learner: A Neglected Species. Houston: Gulf.
Knowles, M. S. (1980). The Modern Practice of Adult Education: From Pedagogy to Androgogy (2nd Ed.), New York:
Cambridge Books.
Kuhl, J. (2000). A functional design approach to motivation and self-regulation: The dynamics of personality
systems and interactions. In M. Boekaerts, P. Pintrich, & M. Zeidner (Eds.), Handbook of Self-regulation,
(pp. 111–163), San Diego, California: Academic.
Livingstone, D W. (1999). Exploring the icebergs of adult learning: Findings of the first Canadian survey of informal learning
practices. Canadian Journal for the Study of Adult Education, 13(2), 49–72.
Lonsdale, P., Baber, C., Sharples, M., and Arvanitiss, T. N., (2004). A context awareness architecture for facilitating mobile learning. In J.
Attewell, C. Savill-Smith (Eds.), Learning with mobile devices: Research and development, (pp. 79–85, 2004), London: Learning and Skills
Development Agency.
Marsick, V. J. and Watkins, K. E. (2001). Informal and incidental learning. New Directions for Adult and Continuing Education,
89, 25–34.
Redecker, C., Leis, M., Leendertse, M., Punie, Y., Gijsbers, G., Kirschner, P., Stoyanov, S., and Hoogveld, B. (2011). The future
of learning: preparing for change. JRC Scientific and Technical Reports, European Comission.
Rosenshine, B., Meister, C., and Chapman, S. (1996). Teaching students to generate questions: A review of the
intervention studies. Review of Educational Research, 66, 181–221.
Ryan, R. M., and Connell, J. P. (1989). Perceived locus of causality and internalization. Examining reasons for
acting in two domains. Journal of Personality and Social Psychology, 57, 749–761.
Ryan, R. M., & Deci, E. L. (2000). An overview of self-determination theory: An organistic-dialectical
perspective. In E. L. Deci & R. M. Ryan (Eds.), Handbook of Self-determination Research (pp. 3–34).
Rochester, New York: University of Rochester.
Sheldon, K. M., and Elliot, A. J. (1998). Not all personal goals are personal: Comparing autonomous and
controlled reasons for goals and predictors of effort and attainment. Personality and Social Psychology
Bulletin, 24(5), 546–557.
Thomas, S. (2007). Pervasive scale: a model of pervasive, ubiquitous, and ambient learning. In Proceedings of the Workshop on
Pervasive Learning 2007, Toronto, 10–18.
Tough, A. (1971). The Adult's Learning Projects. Toronto, Ontario, Canada: Ontario Institute for Studies in Education.
Verbert, K., Manouselis, N., Ochoa, X., Wolpers, M. , Drachsler, H., Bosnic, I. and Duval, E. (2012). Context-aware recommender systems for
learning: a survey and future challenges. IEEE Transactions on Learning Technologies, 5(4), 318–335.
