Content uploaded by Peter Pirolli
Author content
All content in this area was uploaded by Peter Pirolli on Oct 16, 2018
Content may be subject to copyright.
Abstract— Computational models were developed in the
ACT-R neurocognitive architecture to address some aspects of
the dynamics of behavior change. The simulations aim to
address the day-to-day goal achievement data available from
mobile health systems. The models refine current psychological
theories of self-efficacy, intended effort, and habit formation,
and provide an account for the mechanisms by which goal
personalization, implementation intentions, and remindings
work.
I. INTRODUCTION
Smartphone platforms provide an opportunity for
projecting behavior change techniques into everyday life and
for collecting rich, fine-grained data necessary for
understanding and predicting day-to-day behavior change
dynamics. The challenge posed by these opportunities for
detailed measurement and intervention is that current theory
about individual health behavior change is not equally fine-
grained and predictive, and does not provide an integrated
account of the many mechanisms involved [1]. In this paper,
I present a theoretical approach and two models that attempt
to provide an integrated account of multiple psychological
mechanisms associated with goal striving and long-term habit
formation.
It is now widely recognized that a major driver of health
care costs are unhealthy behaviors such as physical inactivity,
increased food intake, and unhealthful food choices [2]. An
unhealthy lifestyle can be viewed, in part, as a complex set of
interrelated habits that need to be switched out for healthy
ones [3], a few tiny habits at a time [4]. To illustrate this
complexity, it has been estimated that people mindlessly
make 200 food decisions per day [5], and each of those
habitual decisions needs to be re-learned. Commercial and
health-care provider weight loss programs can often involve
months to years of counseling (and can still be regarded as
too short), which suggests thousands to tens of thousands of
elementary habits being acquired [6]. The working
assumption for our own mobile health (mHealth) research is
that to master the complex fabric of a new healthy lifestyle,
one must master and weave together a new set of elementary
habits.
This paper presents two predictive, computational
models, both of which are implemented in a unified
neurocognitive modeling framework called ACT-R [7, 8].
The first model, presented in detail in Pirolli [9], refines the
psychological constructs of perceived goal difficulty, self-
efficacy [10, 11] and intended effort (a kind of motivation)
*Research supported by National Science Foundation Grant No.
1346066 to Peter Pirolli and Michael Youngblood.
Peter Pirolli is at PARC, 3333 Coyote Hill Rd, Palo Alto, CA 94304; e-
mail: Pirolli@parc.com).
[12] to provide day-by-day predictions of adherence to
exercise goals. The second model is an attempt to provide a
plausible account of how intentions can lead to the initial
effortful striving to carry out goals, how repeated execution
of behaviors can become automated habits, and how
intervention techniques such as implementation intentions
[13, 14] and reminders [15, 16] can support the development
of habits. Together the models map a trajectory from initial
effortful pursuit of a behavior-change goal to new stable
habits. The motivation for developing these ACT-R models is
that they may possibly lead to the development of mHealth
algorithms that can personalize the selection of behavior-
change goals and personalize the selection and intensity of
interventions so that people optimally progress from the
achievement of “easy” habits to more “difficult” ones.
II. ACT-R
ACT-R [7, 8] is a unified theory of how the structure and
dynamics of the brain give rise to the functioning of the
mind. The ACT-R simulation environment is a computational
architecture that supports the development of models.
A. Modules and Buffers
ACT-R is composed of modules, processing different
kinds of content, which are coordinated through a centralized
production module. Each module corresponds to a brain
region. Each module is assumed to access and deposit
information into buffers associated with the module, and the
central production module can only respond to the contents
of the buffers.
The modules and buffers relevant to this paper include:
• Production module (basal ganglia), which matches
the contents of other module buffers and coordinates
their activity. The production module stores
production rules. A production rule is a formal
specification of the flow of information from buffers
in the cortex to the basal ganglia and back again.
Productions have a utility property that is used to
select the single rule that is executed.
• Goal buffer (dorsolateral prefrontal cortex), which
keeps track of the goals and internal state of the
system. The goal buffer stores and retrieves
information that represents the internal intention of
the system and provides local coherence to behavior.
• Declarative module (temporal lobe; hippocampus),
retrieval buffer, and blending buffer (ventrolateral
prefrontal cortex), associated with the retrieval of
knowledge and past experiences from long-term
declarative memory.
From Good Intentions to Healthy Habits: Towards Integrated
Computational Models of Goal Striving and Habit Formation
Peter Pirolli
CONFIDENTIAL. Limited circulation. For review only.
Preprint submitted to 38th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society. Received March 14, 2016.
Knowledge in the declarative and goal modules is
represented formally in terms of chunks [17, 18]. Each
module is limited to placing a single chunk in a buffer.
Chunks have activation levels that determine the probability
and time course of chunk retrieval into a buffer.
B. Subsymbolic Mechanisms
Production utilities and chunk activations are real-valued
quantities produced by subsymbolic mechanisms in ACT-R.
These subsymbolic mechanisms reflect neural-like processes
that determine the time course and probability of cognitive
activity and behavioral performance. The dynamics of
declarative memory retrieval and production selection are
determined by these subsymbolic mechanisms.
Table I presents a subset of the ACT-R subsymbolic
mechanisms relevant to the current models. The first three
equations in Table I define how the level of activation of
chunks in memory relates to the probability of their retrieval
at any given time. The fourth equation defines how activation
levels are increased by repeated experiences, or decay with
time (forgetting). These first four subsymbolic mechanisms
are crucial to the ACT-R model of self-efficacy. The last two
equations in Table I define utility learning, and the relation of
utility to the probabilistic choice of production rules to
execute. These utility mechanisms are crucial to ACT-R
model of habit formation.
C. Production Compilation
Also important in the ACT-R model of habit formation is
the mechanism of production compilation [7, 19, 20], by
which new production rules are acquired. A new production
rule is generated every time two production rules are
executed in sequence. The mechanism works to create new
rules that eliminate internal cognitive processing, such as the
need to retrieve information from the declarative module or
set and maintain sequences of goals.
III. A MODEL OF SELF-EFFICACY, INTENDED EFFORT, AND
GUIDED ENACTIVE MASTERY
Self-efficacy [10] predicts that goals that are perceived as
too difficult are unlikely to be attempted. Self-efficacy is an
individual’s belief that he or she is capable of performing a
behavioral goal. In general, greater levels of self-efficacy
lead to greater likelihoods of achieving a goal. The
Attributional Theory of Performance [12] proposes that the
level of intended effort motivating a performance will
increase with the difference between self-efficacy and the
perceived difficulty of achieving a goal.
Figure 1. Summary data on achievement of exercise goals from the
Konrad et al. mHealth study [21]. In the Easy condition participants were
presented with a fixed schedule of exercise goals that had small daily
increments in difficulty, the Difficult condition presented a fixed schedule
with large increments in difficulty, and the DStress condition was
personalized based on past success or failure.
Pirolli [9] presented a model for a mHealth study [21]
that contrasted a personalization algorithm for daily exercise
goals with non-personalized exercise programs delivered by
the same system (Figure 1). The personalized programs
implemented a form of guided enactive mastery in which
individuals were supported in achieving progressively
difficult goals [10]. The personalization algorithm was indeed
TABLE I. SOME KEY ACT-R SUBSYMBOLIC MECHANISMS
Mechanism
Equation
Description
Blended
Retrieval
𝑉= 𝑚𝑖𝑛 !𝑃
!(1−𝑆𝑖𝑚(𝑉,𝑉
!)
!
!
Pi: Probability of declarative retrieval
Sim(V,Vi) Similarity between compromise value V and retrieved value Vi
Retrieval
Probability 𝑃
!=
𝑒!!!
⁄
∑𝑒!!!
⁄
!
Pi: The probability that chunk i will be recalled
Ai: Activation strength of chunk i
∑Aj: Activation strength of all of eligible chunks j
s: Chunk activation noise
Activation 𝐴!= 𝐵!+ Ɛ!
Bi: Base-level activation reflects the recency and frequency of use of chunk i
ε
i: Random noise value
Base Level
Learning
n: The number of experiences for chunk i
tj: The time since the jth presentation
d: A decay rate
β
i: A constant offset
Utility Learning
Ui(n-1): Utility of production i after its n-1st application
Ri (n): Reward production receives for its nth application
Ui(n): Utility of production i after its nth application
Pi: Probability that production i will be selected
Ui: Expected uti lity of the production determined by the utility equation above
Uj: is the expected utility of the competing productions j
CONFIDENTIAL. Limited circulation. For review only.
Preprint submitted to 38th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society. Received March 14, 2016.
found to increase goal adherence [21] even when exercises
were more difficult: In Figure 1, the estimated difficulty of
exercises in the DStress condition are greater than the
estimated exercise difficulties in the Easy condition.
A. Process Model
The ACT-R model assumes that self-efficacy and
intended effort are fundamentally the result of memory
processes. Past experiences of efficacy at behaviors similar to
a target goal are retrieved and blended together to produce
assessments of self-efficacy and intended effort for the new
goal. Consequently, the dynamics of self-efficacy and
performance exhibit the dynamics of the underlying memory
mechanisms, and exhibit well-known memory phenomena.
In outline, the model involves the following steps:
• A behavioral goal is considered for doing one or
more activities that are believed to have some level
of difficulty to being performed
• Blended memory retrieval to form an assessment of
self-efficacy. Successful experiences are recalled
involving activities similar to the behavioral goal
activities. This process blends the difficulty levels of
those past experiences into a composite assessment
of the difficulty levels achieved in past experiences,
and this is mapped directly to set an assessment of
self-efficacy.
• Blended memory retrieval to form an intended effort
level. Experiences are recalled with similar levels of
self-efficacy and perceived goal difficulty. This
process blends an assessment of intended effort
levels that had been required to achieve success in
those past experiences.
• Predicting success. Based on the goal difficulty,
perceived self-efficacy, and intended effort, the
model makes a prediction about the likelihood of
success.
• Choose to do it (or not). If the expected probability
of success is above a threshold it is attempted.
• Store new experiences. If the activity is attempted,
the experience is stored in memory and influences
future attempts.
B. Impulse-like Dynamics of Self-Efficacy
Figure 2 illustrates the basic dynamics of the ACT-R
model of self-efficacy. The model can be viewed as an
“impulse model” in which each impulse of positive self-
efficacy decays with time, and impulses add to prior ones.
We define stress as the difference between current self-
efficacy level and the perceived difficulty of a new behavior
change goal. Larger degrees of stress produce larger
impulses, and positive impulses at high frequency and low
temporal lags build up rapidly. Positive experiences at
behavior change build up self-efficacy, but those can decay
with time, and substantial achievements produce bigger
boosts in self-efficacy. It is worth noting that this “impulse
model” of self-efficacy is very similar in spirit to the Banister
Impulse-Response Training model [22].
Figure 2. (Bottom) A hypothetical set of successful goal achievements
over days for behaviors at different levels of stress (difference between self-
efficacy and perceived goal difficulty) and different inter-day lags. (Top)
The resultant gains and decays in self-efficacy. All scales in arbitrary units.
Figure 3. Intended effort (motivation) data sampled from simulations
involving different levels of stress (difference between goal difficulty and
self-efficacy). Scales in arbitrary units.
C. Intended Effort v. Goal Difficulty
Figure 3 depicts the general relation in the model between
degree-of-difficulty of a goal relative to the current level of
self-efficacy. Up to a point, increasing goal difficulty leads
the model to assign higher levels of intended effort. This is
consistent with Goal Setting Theory [23]. If, however, the
goal difficulty becomes too difficult, the simulation will
predict such a low expectation of success that it will not
choose to do the goal. In general the model exhibits a
discontinuous nonmonotonic relationship between self-
efficacy and intended effort [24].
IV. A MODEL OF THE GOAL-HABIT INTERFACE
Habits are typically described as behaviors that have been
gradually acquired through repetitive association of
behavioral responses to cues in some set of performance
contexts (e.g., physical settings, preceding actions) [25]. In
the beginning, behaviors are typically directed by goals that
require cognitive effort. After much repetition, the behavior
may come to be automated such that the habit is triggered by
the contextual cues without mediating goals and cognitive
effort. Evidence suggests that goal-driven behaviors involve
different neural circuitry than habitual behaviors [26]. The
-1.5 -1.0 -0.5 0.0 0.5 1.0 1.5 2.0
2.0 2.5 3.0
Goal Stress
Intended Effort
CONFIDENTIAL. Limited circulation. For review only.
Preprint submitted to 38th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society. Received March 14, 2016.
ACT-R model presented here initially involves complex
sequences of goal setting, recall of intentions and plans, and
thinking about those plans to perform actions but, with
repeated practice, mechanisms of production compilation and
utility learning produce efficient productions that are cued by
the environment, require little internal cogitation, and are
durable.
Figure 4. Summary of ACT-R performance and habit formation. Time
progresses from top to bottom. Middle columns depict processing in
different modules or buffers.
A. Illustration: Habit Formation Task with Prospective
Memory, Implementation Intentions, and Reminders
Two examples from the literature illustrate common
approaches to establishing new habits. Tobias [15] describes
a real-world multi-week intervention using reminders
(pamphlets and personal visits) to promote recycling, along
with an agent-based model that provides good fits to the
recycling behavior of the individuals involved. The
intervention and model assume that people have a
prospective memory to recycle. Prospective memory involves
remembering to perform some goal or action in some future
context [27]. Unfortunately, people often forget their
intentions. Tobias’ [15] data and model showed how
reminders strengthened prospective memory and lead to the
execution of behavior, but also how the impact of reminders
could decay over time if behaviors had not been repeated
enough to become durable habits. Prestwich et al. [16] in an
mHealth intervention to promote physical activity used SMS
messaging as reminders, in combination with implementation
intentions [13, 14]. Implementation intentions are declarative
“if-then” plans that are committed to memory that concretely
spell out how to achieve a goal, and using them is a
surprisingly effective technique.
Figure 4 presents a schematic summary of the ACT-R
model’s performance and habit formation for the illustrative
problem of becoming a consistent trash recycler. Time
progresses from top to bottom in Figure 4. The leftmost
column captures cues and events that occur in the world, the
rightmost column captures actions (behaviors) in the world,
and the middle columns each capture processing that occurs
in the core ACT-R modules.
Before intervention (top Figure 4), there are well-
established habits for taking the out the trash “as usual”,
without recycling: “Do Trash” production rules in the
production module are directly triggered by perception of
environmental trigger cues that typically initiate taking out
the trash. The intervention involves reminder events that
strengthen a prospective memory to sort the trash next time,
through base-level learning (Table I). As the intention to sort
is strengthened, it triggers a set of general production rules
that strive to achieve those intentions. In Figure 4, the
“Request Intention” production rule recalls the intention
(indicated by the “?” in the retrieval buffer column), the
intention is recalled, then a “Retrieve Intention” production
rule sets the intention as an active goal in the goal buffer.
Implementation intentions are also stored in declarative
memory as sequences of operators that need to be carried out
to achieve the goal. A sequence of production rules execute
to recall the steps, set new subgoals, and execute actions,
eventually resulting in the external behaviors of sorting the
trash.
Every execution of the new behavior leads to the
compilation of new production rules. Over many repetitions,
there are new production rules that are triggered by the
external cues to take out the trash that lead directly to the
actions to sort the trash first. These new rules compete with
the old “task-out-the-trash-as-usual” productions, but with
repetition and consistent rewards, the new ones will dominate
the old ones.
B. Effects of Remindings on Prospective Memory
Figure 5 reveals some of the underlying dynamics of the
ACT-R habit model. In this simulation, daily remindings
occur on alternate weeks (and are absent otherwise). Figure 5
plots the activation levels of chunks associated with
intentions to take out the trash as usual (old intention), or the
new intention to sort and recycle (new intention). Old
intentions have high activation because they have a high
frequency of past use, however, remindings prime even
higher activation to the new intentions (because of the
recency of the reminding). Unfortunately, new intentions
decay rapidly without the additional priming from the
reminders.
Figure 5. Model’s simulated activation levels of an old intention (trash-
intention) and a new intention (recycle-intention). Reminding of the new
intention occurs on alternate weeks
CONFIDENTIAL. Limited circulation. For review only.
Preprint submitted to 38th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society. Received March 14, 2016.
C. Length of Sustained Remindings and Habit Formation
The general phenomena, then, are that persistent
remindings are required until new production rules are
compiled and gain sufficient utility to dominate old intentions
and production rules. Figure 6 presents data from the same
simulation model illustrating these phenomena. In this case,
the simulation was presented with daily reminders for n
weeks (n = 1...12) then the reminders were absent for 10 days
of sampling. During the daily reminder period, the simulation
always performs the new behaviors (sorting the trash). Figure
6 presents the probability of simulated recycling over the 10-
day sample period following removal of the reminders. Two
weeks of remindings produce no sustained new behaviors,
but somewhere between 6-8 weeks the new habits are being
invoked about 90% of the days.
Figure 6. Model predictions of probability of recycling after different
sustained spans of daily remindings.
V. CONCLUSION
The complexity of the literature on behavior change
techniques and theories is evident in the efforts of the
Behavior Change Techniques and Theory Project at the
University College London [28], which has documented
upwards of 93 intervention techniques, 83 theories of
behavior change, collectively composed of 1725 constructs
[29]. The models in this paper tackle only a tiny fraction of
this space, but the hope is that the approach illustrates a
potential path to some theoretical integration and unification.
ACKNOWLEDGMENT
Thanks to Shiwali Mohan for comments on this paper.
REFERENCES
[1] W. T. Riley, D. E. Rivera, A. A. Atienza, W. Nilsen, S. M. Allison, and
R. Mermelstein, "Health behavior models in the age of mobile
interventions: are our theories up to the task?," Translational
Behavioral Medicine, vol. 1, pp. 53-71, Mar 1 2011.
[2] K. E. Thorpe. (2009, May 28). The future costs of obesity: National and
state estimates of the impact on direct health care expenses.
[3] C. Heath and D. Heath, Switch: How to change things when change is
hard. New York: Broadway Books, 2010.
[4] B. J. Fogg and J. Hreha, "Behavior Wizard: A Method for Matching
Target Behaviors with Solutions," in Persuasive Technology. vol. 6137,
T. Ploug, P. Hasle, and H. Oinas-Kukkonen, Eds., ed: Springer Berlin
Heidelberg, 2010, pp. 117-131.
[5] B. Wansink and J. Sobal, "Mindless Eating: The 200 Daily Food
Decisions We Overlook," Environment and Behavior, vol. 39, pp. 106-
123, January 1, 2007 2007.
[6] P. J. Feltovich, M. J. Prietula, and K. A. Ericsson, "Studies of expertise
from psychological perspectives," in The Cambridge handbook of
Expertise and Expert Performance, K. A. Ericsson, N. Charness, P. J.
Feltovich, and R. R. Hoffman, Eds., ed Cambridge, MA: Cambridge
University Press, 2006, pp. 41-67.
[7] J. R. Anderson, How can the human mind occur in the physical
universe? Oxford, UK: Oxford University Press, 2007.
[8] J. R. Anderson, D. Bothell, M. D. Byrne, S. Douglass, C. Lebiere, and
Y. Qin, "An integrated theory of mind," Psychological Review, vol. 11,
pp. 1036-1060, 2004.
[9] P. Pirolli, "A computational cognitive model of self-efficacy and daily
adherence in mHealth," Translational Behavioral Medicine, pp. 1-13,
2016.
[10] A. Bandura, Self-efficacy: The exercise of control. New York: W.H.
Freeman, 1998.
[11] A. Bandura, "Self-efficacy: Toward a unifying theory of behavioral
change," Psychological Review, vol. 82, pp. 191-215, 1977.
[12] A. Kukla, "Foundations of an attributional theory of performance,"
Psychological Review, vol. 79, pp. 454-470, 1972.
[13] P. M. Gollwitzer, "Implementation intentions: Strong effects of simple
plans," American Psychologist, vol. 54, pp. 493-503, 1999.
[14] P. M. Gollwitzer and P. Sheeran, "Implementation intentions and goal
achievement: A meta-analysis of effects and processes," Advances in
Experimental Social Psychology, vol. 42, pp. 668-675, 2006.
[15] R. Tobias, "Changing behavior by memory aids: a social psychological
model of prospective memory and habit development tested with
dynamic field data," Psychol Rev, vol. 116, pp. 408-38, Apr 2009.
[16] A. Prestwich, M. Perugini, and R. Hurling, "Can implemetation
intentions and text messages promote brisk walking? A randomized
trial," Health Psychology, vol. 29, pp. 40-59, 2010.
[17] G. A. Miller, "The magical number seven plus or minus two: Some
limits on our capacity for processing information," Psychological
Review, vol. 63, pp. 81-97, 1956.
[18] H. A. Simon, "How big is a chunk?," Science, vol. 183, pp. 482-488,
1974.
[19] N. A. Taatgen, "Production compilation: A versatile cognitive
mechanism," AISB Quarterly, p. p. 7, 2004.
[20] P. Dayan, "Goal-directed control and its antipodes," Neural Networks,
vol. 22, pp. 213-219, 2009.
[21] A. Konrad, V. Bellotti, N. Crenshaw, S. Tucker, L. Nelson, H. Du, et
al., "Finding the Adaptive Sweet Spot: Balancing Compliance and
Achievement in Automated Stress Reduction.," presented at the
SIGCHI Conference on Human Factors in Computing Systems (CHI
2015), Seoul, Korea, 2015.
[22] E. W. Banister, T. W. Calvert, M. V. Savage, and T. Bach, "A systems
model for athletic performance," Australian Journal of Sports
Medicine, vol. 7, pp. 57-61, 1975.
[23] E. A. Locke and G. P. Latham, "Building a practically useful theory of
goal setting and task motivation: A 35-year odyssey," American
Psychologist, vol. 57, pp. 705-717, 2002.
[24] J. B. Vancouver, K. M. More, and R. J. Yoder, "Self-efficacy and
resource allocation: Support for a nonmonotonic, discontinuous,
model," Journal of Applied Psychology, vol. 93, pp. 35-47, 2008.
[25] W. Wood and D. T. Neal, "A new look at habits and the habit-goal
interface," Psychol Rev, vol. 114, pp. 843-63, Oct 2007.
[26] A. M. Graybiel, "Habits, rituals, and the evaluative brain," Annua
Review of Neuroscience, vol. 31, pp. 359-387, 2008.
[27] R. K. Dismukes, "Prospective Memory in Workplace and Everyday
Situations," Current Directions in Psychological Science, vol. 21, pp.
215-220, August 1, 2012 2012.
[28] S. Michie, M. Richardson, M. Johnston, C. Abraham, J. Francis, W.
Hardeman, et al., "The Behavior Change Technique Taxonomy (v1) of
93 Hierarchically Clustered Techniques: Building an International
Consensus for the Reporting of Behavior Change Interventions,"
Annals of Behavioral Medicine, vol. 46, pp. 81-95, 2013.
[29] S. Michie, R. West, R. Campbell, J. Brown, and H. Gainforth, ABC of
Behavior Change Theory. Great Britain: Silverback Publishing, 2014.
CONFIDENTIAL. Limited circulation. For review only.
Preprint submitted to 38th Annual International Conference of the IEEE
Engineering in Medicine and Biology Society. Received March 14, 2016.
