Conference PaperPDF Available

Improving Teacher Effectiveness in Schools in Delhi through Behaviorally Informed Lesson Planning Intervention



In this paper we propose a behaviorally informed intervention to address the 'learning crisis' plaguing government schools in Indian. The proposed intervention is aimed at encouraging government school teachers to create and employ lesson plans to increase their effectiveness and improve student learning outcomes. We use behaviorally informed tools such as enhanced active choice, defaults, anchors, and checklists. As well as behavioral insights on social norms and present-bias to inform our intervention design.
Improving Teacher Effectiveness in Schools in Delhi through Behaviorally Informed Lesson
Planning Intervention
Diwakar Kishore
Farah Mallah
Abstract: In this paper, we propose a behaviorally informed intervention to address the 'learning crisis'
plaguing government schools in Indian. The proposed intervention is aimed at encouraging government
school teachers to create and employ lesson plans to increase their effectiveness and improve student
learning outcomes. We use behaviorally informed tools such as enhanced active choice, defaults,
anchors, and checklists. As well as behavioral insights on social norms and present-bias to inform our
intervention design.
Table of Content
I. INTRODUCTION!........................................................................................................................!3!
II. INTERVENTION DESIGN AND BEHAVIORAL INSIGHTS!.................................................!4!
A. ENHANCED ACTIVE CHOICE AND DEFAULT!.....................................................................................!5!
B. INCONSISTENT PREFERENCES: INCENTIVES AND SOFT COMMITMENT!..............................................!6!
C. PLANNING & SOFT COMMITMENT!....................................................................................................!8!
D. ANCHORING IN BEST PRACTICES!......................................................................................................!8!
III. POTENTIAL IMPACT OF IMPLEMENTATION!..................................................................!9!
V. CONCLUSION!..........................................................................................................................!11!
VI. REFERENCES!........................................................................................................................!12!
VII. APPENDIX!............................................................................................................................!16!
APPENDIX A: SAMPLE ACTIVE CHOICE FORM FOR TEACHERS!.......................................................!16!
APPENDIX B: PLANNER FOR TEACHERS WHO CHOSE OPTION 1!.......................................................!17!
APPENDIX C: LESSON PLAN TEMPLATE & CHECKLIST *!..................................................................!18!
I. Introduction
The World Development Report (WDR) (2018) published by the World Bank warned of a
“learning crisis”. The Report notes that millions of children across low and middle-income countries
attend schools for years without learning basic language (reading and writing) and/or arithmetic skills
(World Bank, 2018). Yet, since most governments in developing countries still rely on traditional
economics incentive schemes to address these issues, they have not been very successful.
In recent years, we have witnessed various behaviorally informed experimental interventions
by researchers in an attempt to reform education systems to improve learning outcomes. Duflo et. al
(2012) linked teachers’ attendance to their salaries & discovered that attaching the incentive to a
simple behavior, resulted in a 21% increase in teacher attendance which led to higher student
outcomes. A large-scale intervention to encourage “Teaching at the Right Level” by reorienting
teaching to the level of the student has been showing positive results (Banerjee et al, 2016). In
Uganda, World Bank is conducting research to measure effectiveness of teachers by controlling the
class-size and providing guidance to teachers for improved performance along with non-financial
rewards (World Bank, 2016). In a similar study in New Delhi, researchers are using the feed from the
CCTV cameras installed by the Government to give “timely feedback” to public school teachers on
how to improve classroom instruction (RISE India, 2015). World Bank (2017) is also involved in a
RCT in India to find ways to motivate teachers without using financial incentives. An underlying
principle across these interventions is that they aim to improve teacher performance. This may be
because of two main reasons: teachers “account for about 30% of the variance” in student outcomes
(Hattie, 2003); and in the majority of education systems, especially the Global South, teacher salary
constitutes the highest recurring expenditure (UNESCO, 2016).
We note that in India, governments haven’t devised effective ways to link incentives
(financial or otherwise) to the behavior of teachers to improve education systems, specially with
regards to preparatory work done by teachers prior to class or “lesson planning”. Currently, the
National Council of Educational Research and Training (NCERT) assists “the Central and State
Governments on policies and programs for qualitative improvement in school education” wherein it
“develops and disseminate innovative educational techniques and practices” (NCERT, n.d). The
NCERT does make accessible a 200 page teacher's manual to all public school teachers in India
(Nandrajog et al, n.d). These Teacher’s Manuals typically have tips on “Lesson Planning &
Preparation”. However, neither the Central nor the State Governments have enacted policy to ensure
that teachers prepare and follow lesson plans. Further, in line with the previous studies which showed
that complexity leads to low take-up of a desired policy (Bhargava & Manoli, 2015), on interviewing
several education researchers, we find that not many government school teachers use the complicated
manuals to make lesson plans. Hence, an intervention informed by behavioral economics principles is
warranted to address this government shortcoming.
Our intervention incentivizes group lesson plan adaptations in line with best practices to
improve teaching quality. We propose a pre-existing set of lesson plans to be used by teachers in
India, encourage them to collaborate to improve upon it, to determine their classroom day-to-day
practice. Instead of incentivizing an outcome teachers have less control over (like student test-scores),
we chose to focus on an input teachers have control over: lesson plans. For an incentive to be
successful, it has to be linked to a behavior individuals have control over (Vegas, 2007; Madrian,
2017). Further, it’s important to establish a clear correlation between the input (quality lesson plans)
and the desired output (higher student achievement) (Mizala & Romaguera, 2004). The lesson plan
input we are designing prompts teachers to engage in practices that have been proven to be positively
correlated with student outcomes, such as: communicating clear lesson objectives and expectations to
students, variation in exercises ranging from those requiring lower to higher cognitive abilities,
integrating the content with other subjects, and encouraging metacognition among students (Ko,
Sammons & Bakkum, 2014). The purpose of the intervention are three folds: (1) to insure teachers
have quality lesson plans for each class, (2) to improve communication between novice and
experienced teachers, and (3) nudge teachers to following best classroom practices. We expect the
three elements to contribute to better teacher quality, and ultimately higher student outcomes. We use
four behaviorally informed tools: enhanced active choice, defaults, anchors, and checklists. We also
use behavioral insights on social norms and present-bias to inform our intervention design.
II. Intervention Design and Behavioral Insights
Our intervention design is based on an active choice made by the teachers regarding their
preferred lesson planning method, followed by a lesson plan design that prompts teachers to follow
best practices. Teachers receive a package on the first day they come to work after term-break. The
package includes pre-designed lesson plans and an active choice form. The pre-designed lesson plans
are designed by experts in the Indian curriculum, who are familiar with public school classrooms in
India, as well as in best practices. The active choice form gives teachers two options: (1) To design
their own lesson plans in teams of two, or (2) to follow the pre-designed lesson plans. Teachers who
choose option 2, can depend fully on the pre-designed lesson plans. A sample active choice form is in
Appendix A.
Option 1 requires more time and effort from teachers, but also gives teachers access to a
number of privileges. In this option, teachers are required to form a team of two, one of whom needs
to be a novice teacher (has two years or less experience in teaching) and the other is an experienced
teacher (has more than two years of experience)1. The lesson plans for the first two months of classes
(until midterms) need to be submitted before students resume school. As all the teachers would have
access to the standard lesson plans, the teachers who have opted for Option 1 would be required to
1 Studies have consistently found that teacher effectiveness increases non-linearly with experience during the first two years
of teaching, and later tends to slows down. See Rockoff (2004), Chetty et al (2011), Bau & Das (2017).
submit a brief, justifying how and why their lesson plan differs from the standard lesson. Once they
submit the lesson plans, they receive a bonus equivalent to 4000, with a card thanking them for their
hard work. Teams that submitted the best lesson plans (evaluated by experts) are then invited to attend
a dinner at a nice hotel with State education officials, along with having their names added to the
lottery to potentially win 50,000. Teachers who choose option 1 will receive a planner as well as
lesson planning forms with a checklist. The planner is can be found in Appendix B, and the lesson
planning form along with the checklist can be found in Appendix C.
Teachers are required to sign and submit the active choice form in order to receive their
salaries. The signature does not bind them to any repercussions if they do not follow through with the
plan. It is a soft commitment (Bryan, Karlan & Nelson, 2010). This process will take place over three
years, four times every year: before school starts, mid-semester one, start semester two, and mid-
semester two. Below, we discuss the behavioral insights informing our design and hypothesis.
A. Enhanced Active choice and Default
In our intervention, we present teachers with an active choice between the two options. Active
choice in this case is better fitting compared to a default for a number of reason. First, since designing
lesson plans requires teacher effort, they need to actively choose to do so. If we default teachers in
option 1, they will likely not follow-through the action required. Literature on active-choice suggests
that in the long run, people are more likely to adhere to their choice than in opt-out models because
they feel responsible for their choice (McKenzie, 2013). In addition, teachers know themselves better,
are familiar with their students, the time they can dedicate to lesson planning and their ability, as such
they are better equipped to make the choice (Sunstein, 2015).
We are employing a framing mechanism which “enhances” this active choice architecture
(McKenzie, 2013). Enhanced active choice highlights the losses associated with choosing the non-
preferred option (McKenzie, 2013). In our intervention, the option that we do not prefer is option 2, as
such we highlight what is forgone by choosing the option: “the chance to develop my own lesson plan
and get a bonus of 4000” (Appendix A). This in turn activates individuals’ loss aversion when
making their choice, leading to undervaluing the benefits of the status-quo (McKenzie, 2013). Strong
loss aversion is related to the prospect theory2 which suggests that individuals react more strongly to
losses compared to gains (Kahneman, 2011). In this case, the status-quo is to not design a lesson plan.
In a study that aims to increase the number of patients getting an HIV test, they found that individuals
provided with an active choice were 13% more likely to choose to take an HIV test compared to
individuals who were offered an opt-in option (Montoy, Dow & Kaplan, 2016). We therefore expect
the active choice mechanism to increase adoption of option 1 compared to an opt-in.
2 Prospect theory predicts how people react to risks. It suggests that people’s reaction to risk depends on their reference point
and that they are risk averse (Barberis, 2013).
Further, even if teachers do not make an active choice of drafting lesson plans, they are still
(softly) defaulted3 into receiving a well-curated lesson plan. The likelihood of opting out of a default
is low, as such, defaults tend to have a high impact (Johnson & Goldstein, 2003). In our current
model, they would be required to opt-out if they do not want pre-curated lesson plans. Given that it is
unlikely they will opt-out, by defaulting teachers into receiving a lesson plan, we insure they have a
good guide for their day-to-day practice, which on enquiry with several researchers and Teach for
India fellows, we discovered that currently, most government school teachers across India do not
create and follow.
B. Inconsistent Preferences: Incentives and soft commitment
There are a number of barriers to teachers writing lesson plans, even if they believe in the
benefits of lesson planning. One of which is highly discounting of future benefits (Chabris, Laibson,
& Schuldt, 2010). Based on Chabris, Laibson, and Schuldt (2010), theories of intertemporal choice we
can assume the discounting function of teachers is Quasi-hyperbolic (which tends to better explain
individuals’ behaviors). This leads to dynamically inconsistent preferences4 (refer to Appendix D for
a detailed explanation of how Quasi-hyperbolic discounting leads to dynamically inconsistent
preferences), which leads to procrastinating the act of designing a lesson plan, eventually leading to
either creating it very quickly with little thought or not doing it at all.
In our intervention, we try to decrease the cost ("#)%associated with writing a lesson plan by
simplifying the process. Teachers in both options receive a pre-designing standard lesson plan. This
decreases the cost of designing a lesson plan for teachers who chose option 2, to almost zero, because
the lesson plan is ready all they need to do is look over it and maybe adjust it a little to suit their
students and teaching style. Given the vast ways teachers can approach their lesson, the numerous
choices and the difficulty of aligning them may add cognitive load that leads to making no decision at
all (Gourville & Soman, 2005). The standard lesson plan helps create this alignment for teachers in
options by setting the standard which they can use to evaluate and guide their own design. In addition,
the lesson plan forms we provide teachers in option 1 are intentionally designed to simplify the
process, and help teachers think about the process in sections, decreasing the complexity and
cognitive load. Studies have shown that removing complexity could have large positive effect on the
take-up of the desired policy intervention (Bhargava & Manoli, 2015; Bettinger et al, 2012).
Considering that option 1 requires more work i.e. higher immediate cost than option 2, we
tried to increase the immediate5 and future benefit of designing the lesson plans. We increase the
immediate benefit by giving teachers a financial reward of 4000 in addition to a thank you note.
3 Behavior Economists define a “default” as the pre-set courses of action that take effect, unless the decision maker actively
decides against it (Thaler & Sunstein, 2008).
4 The variation in perceiving the same act in the present and future due to highly discounting the future by a factor of &, such
that an individual sees a particular act as beneficial in the future but their present self perceives it as costly in the present, is
referred to as “dynamically inconsistent preferences” (Chabris, Laibson, & Schuldt, 2010).
5 It is immediate in relative terms compared to the delayed benefit of student outcomes, but it is not “now” as the cost is.
Expression of gratitude can be an effective way to increase effort (Grant & Gino, 2010). In a study by
Grant and Gino (2010) they find that expressing thanks to an employee doubled the likelihood of the
employee providing help a second time, and increased the number of calls made by more than 50%.
We also frame option 1 as a positive identity a teacher can attach to herself in order to
increase the benefit of following through with option 1. In the Active Choice form, in option 1
teachers are asked if they “would like to be a lesson plan designer” (Appendix A). In countries where
being a teacher in itself is not valued, framing the action of designing a lesson plan as not only an
action a teacher takes, but an identity a teacher can attach to herself that has positive connotations
may increase the benefits attached to following through with option 1 and designing a lesson plan. In
a study by Bryan et al (2011) they found that by framing the act of voting as “be a voter” instead of
the action “vote” increased voter turnout by 10.7%.
To increase teachers’ effort at producing high quality lesson plans, we provide additional
social and monetary incentives for teachers who produce the ‘best’ lesson plans. Teachers, like other
individuals, care about how their action is perceived by others (Madrian, 2017). Therefore, teachers
who produce the ‘best’ lesson plans will have their lesson plans shared with the teaching community
and invited to an event that celebrates them. Publicizing individual’s actions was found to have a
positive effect on individuals’ actions, increasing likelihood of voting and decreasing energy
consumption (Gerber et al 2004; Alcott & Rogers, 2014). That said, those studies publicized negative
behavior as well. In our study we will be publicizing positive behavior only. The impact of such
interventions depends on how pro-social norms are defined. If teachers do not perceive producing
lesson plans as a positive thing, this may have a negative impact on participation. In a study on
savings, publicizing savings was found to decrease savings rates (Beshears et al, 2015). Similarly,
sharing peer information on energy consumption among conservatives had two to three times less of
an impact on their energy consumption compared to that on liberals (Costa & Kahn, 2013). In our
intervention, we seek to make great lesson planning a pro-social norm by attaching the creation of
great lesson plans to a prestigious social event where prestige is signaled by the location of the event
(in Leela Palace, a 5-star hotel) and inviting top officials in the community.
We also provide a lottery incentive for teachers to produce great lesson plans which increases
the delayed benefit of lesson planning. According to the prospect theory, individuals overweight low
probabilities of gain (Kahneman, 2011). As such, teachers are likely to expect higher delayed benefits
from writing a great lesson plan that would include their name in the lottery.
In order to increase the impact of delayed benefits in this intervention we try to decrease how
much teachers discount the future by adding a picture of a teacher who won the lottery in the previous
cycle (Appendix A). We expect that by helping teachers imagine themselves winning the lottery, we
can decrease the discount rate of future benefit. In a study that aims to increase retirement savings,
helping individuals connect the decisions they make to their future self by showing an age progressed
image of themselves was found to increase retirement saving rates by 1.8%-1.6% (Hershfield, 2011).
C. Planning & Soft Commitment
Planning and soft commitments both help bridge the gap between the intention of writing a
lesson plan and actually writing one. Since those who chose option 1, chose it on their own accord, we
can assume that they want to complete working on their lesson plans on time, in this case we need to
help them follow-through this intention. In a number of studies, planning was found to increase the
likelihood of following through on good intentions such voting (by 4.1%), and vaccination (by 8%)
(Nickerson & Rogers, 2010; Milkman et al, 2011)
Effective planning prompts encourage individuals to think of how, when and where the act
will be completed, as well as what the tasks are exactly (Rogers et al, 2015). Our prompt encourages
teachers to describe the tasks they need to undertake to complete one lesson plan, how much time
each task will take, what time and day will they work on the tasks, and when will they complete it
(Refer to Appendix B for planner template). Plans are also more effective when shared with others
(Rogers et al, 2015). Since the lesson planning will take place in teams the teachers can be
accountable to one another.
We also intentionally designed the active choice form to sound like a soft commitment.6 That
said, we do have a reward for teachers who follow through on option 1, though there is no financial
penalty associated with not following through. Commitment devices can help individuals who face
dynamically inconsistent preferences (discussed in the previous section) by having them commit their
future self to an action (Bryan, Karlan & Nelson, 2010). In a study, they found that asking individuals
to decide what their future self will watch increases the number of ‘virtuous’ movies chosen compared
to individuals who decided what they will watch sequentially, day-by-day (Read, Loewenstein &
Kalyanaraman, 1999). Similar, providing students with the option to set and commit to their own
deadlines, improved their performance (as cited by Bryan, Karlan & Nelson, 2010). Therefore, we
expect that by having teachers sign at the end of the active choice form following “I commit ...
statement, they will be more likely to follow through with their chosen option, especially in option 1
where there is a reward associated with following through.
D. Anchoring in best practices
By defaulting teachers into receiving standard lesson plans, we anchor7 teachers in high
quality lesson plans. Anchors have a strong impact on people’s estimates. In a study, one group was
asked if the tallest redwood is “more or less than 1,200 feet?” (Kahneman, 2011, p.123), while
another group was asked if it was more or less than 180 feet. The average estimate of the first group
was 844 feet, approximately four times more than that of the second group with an average of 282 feet
(Kahneman, 2011). By providing teachers with a lesson plan we are anchoring teachers in what a
6 A soft commitment is a commitment that does not entail financial reward or penalty, but only entails psychological costs
such as regret (Bryan, Karlan & Nelson, 2010).
7 Anchoring is the impact of considering a certain value before estimating the value of the unknown, as a result the estimated
values tends to be closely linked to the value initially considered (Kahneman, 2011).
good quality, good practice looks like.8 As such, when they deviate they more likely than not will not
deviate significantly. In addition, if teachers are assuming that those who wrote the lesson plans are
experts, the standard lesson plans may result in an endorsement effect. Defaults tend to be effective
because they are perceived as an endorsement of a certain course of action (Beshears, Choi, Laibson
& Madrian, 2009).
We also utilize a checklist that is on the first page of both the standard lesson plans (to all
teachers) and the blank lesson plan forms (to option 1 teachers) to help them move away from their
usual lecture-based method of teaching the status quo (refer to Appendix C). Individuals tend to
make their decisions first, and then come up with reasons to justify it (Shafir, Simonsonb & Tverskyb,
1993). Checklists can help teachers think of reasons they otherwise would have not considered when
making a certain choice (Johnson et al, 2016). In a study looking at the influence of preference
checklists on individual's choice to claim retirement savings, they found by listing reasons to claim
late before reasons to claim early, individuals, on average, chose to claim their retirement saving 18
months later (Johnson et al, 2016). Teachers who are used to a certain way of teaching have available
reasons for why their method is the right one. By having a checklist of reasons for why they may want
to consider another method of teaching, we are making available and salient what was previously
unavailable, encouraging teachers to change their method of teaching. On the checklist, we have
reasons why teachers should endorse student-centered learning methods, before reasons why they
should endorse lecture-based teaching. The order of reasons and choices matters; individuals put more
weight on what comes first (Johnson et al, 2016; Ho & Imai, 2008).
III. Potential Impact of Implementation
We note that similar interventions have not been implemented earlier so it is difficult to
accurately predict the effect size. However, since our policy proposals are layered with behavioral
insights, we anticipate it working on different aspects of teacher performance which impacts learning
outcomes. Further, these proposals will also impact other aspects of education system potentially
leading to a large positive effect on the overall system. When implemented, we would measure the
impact of the proposal on student learning outcomes, teacher subject knowledge & teacher attendance.
Defaulting teachers to get well curated detailed lesson plans will lead to better classroom
teaching by teachers, leading to higher student outcomes. Studies have shown that online access to the
pre-made lessons with supports for teachers increased student’s math test score by about 0.08 standard
deviation (which is considered between small-medium effect size by industry standards (Lipsey et al,
2012) (Jackson & Makarin, 2016). Since our proposal does more than making a pre-designed lesson
plan accessible online, and since defaulting will result in high take up rate (Beshears et al, 2008), we
expect a medium-big effect (by industry standards (Lipsey et al, 2012)) on student outcomes.
8 The standard lesson plan we will provide, as well as the lesson plan form are inspired by best practices and India’s vision
for teacher effectiveness (Ko, Sammons & Bakkum, 2014; Nandrajog et al, n.d.)
For teachers who choose option 1 we expect higher impact on student outcomes and teacher
attendance. Teachers who chose to collaborate to draft their own lesson plans (option 1) would benefit
from working together (sharing knowledge, tips, etc). They are also more likely to spend relatively
more time and thought on how to most effectively teach their class. This should lead to a bigger effect
size on student learning outcomes when compared to option 2, as well as higher teacher subject
knowledge. Lesson planning might also have a small effect on improving teacher attendance.
Planning in advance could lead to a more invested teacher who is more likely to not skip an event for
which she/he has prepared for (due to sunk cost). This is in line with the behavioral insight that people
have irrational need to complete set of things (Carmen, 2017).
We also expect the impact of the intervention to persist with time. Running our intervention
for an appropriate time period could lead to long-term adherence. Current studies show promise of
long term adherence (persistence effect) to behaviorally informed interventions which are run for
longer time period (Alcott and Rogers 2012). We believe our proposed intervention could lead to such
positive effect, since it changes the way people think and prompts an investment in the future that
would lead to habit creation and persistence (Frey & Rogers, 2014). Teachers would see how lesson
planning makes them more effective, potentially reducing stress before the class. In addition, teachers
would have a stock of lesson plans that they can use, making their work on lesson planning easier in
the future (investing in the future). This could lead to a habit formation with teachers continuing to
make and use lesson plans without any financial incentives, but the effect size might be small due to
long-term decay of effect (Frey & Rogers, 2014). Additionally, since repeated cycles of lesson
planning would lead to small further improvement and refinement of lesson plans and this exercise
could also be used to tune up the overall curriculum the States have adopted. Curriculum reforms are
a lengthy and expensive process. A detailed account of what teachers are teaching, how they are
teaching it, and how much time on average are they spending on specific lessons would be very
helpful and informative, while reducing costs, in process of curriculum reforms. In the long run, we
expect our proposals to have a medium positive effect on the education system overall.
IV. Practical implications, potential evaluation and limitations:
Our policy proposal requires significant up-front investment (of resources and time) for large
scale implementation. At scale, our proposal would require standard lesson plans to be made for each
subject from grade 1 to grade 12. Further, to incentivize large number of teachers to actively engage
in collaboration and draft their own lesson plans four times (two each term) a year, the rewards (cash
award, lottery, social recognition etc) could add up into a considerable amount.
That said, the intervention we are proposing is scalable and is politically feasible. The
proposed policy interventions are not contingent on technology and would be able to function by
using well established and functioning Indian Postal Network. However, with increased access to
computer and internet, the proposed intervention can become easier to implement. The proposed
intervention is also politically viable, since all the stakeholders stand to gain from its implementation.
It does not impose any obligation on the teachers. Further, both the Central and State Governments
are investing heavily to improve learning outcomes, with the help of World Bank and other such
players. Given the current high investment and interest in education reform, the costs we propose are
small considering the potential long-term impact.
To evaluate its impact, we envisage running randomized controlled trials for fewer grades at a
smaller scale and/or collaborating with organization currently running RCTs in the field of education
such as the World Bank and RISE. We could measure the effect size by comparing the student
learning outcomes, teacher subject knowledge and attendance on the class/school where the
intervention was applied to the class/school which was in the control group. We acknowledge that any
long term impact of out proposed intervention would be difficult to measure. However, we could
measure long term adherence, by evaluating for persistence effect 5-10 years after the removal of the
incentive model (for creation of lesson plans) from our proposed intervention.
Recent studies have found that in developing countries, well performing teachers have high
levels of intrinsic motivation (Kurniasih & Izati, 2017). Studies have shown that rewards may have an
adverse effect on intrinsic motivation (Jovanovic & Matejevic, 2014). There is a possibility that our
incentive model might drive out the intrinsic motivation that teachers might have to collaborate/make
lesson plans and they might constantly need to be rewarded to carry on this process. While the
effectiveness of using “default” has been documented in various studies, some research indicates that
“active choice” can lead to lower take-up compared to an opt-in (Kessler & Roth, 2014). This is the
reason; we have opted from “enhanced active choice”, which has been found to be more acceptable
and ethical (Keller et al, 2011). That said, we do run the risk of low take-up of option 1.
V. Conclusion
Despite massive investment, Indian Public Education System continues to underperform due,
inter alia, to ineffective government policies based on traditional economic models. By utilizing
insights and tools from behavioral economics we hope to improve learning outcomes by increasing
teacher effectiveness in India. We use four behaviorally informed tools: enhanced active choice,
defaults, anchors, and checklists; along with behavioral insights on social norms and present-bias to
inform our intervention design. Our intervention encourages adoption of best practices by providing
lesson plans that anchor teachers in effective teaching methods, while incentivizing them to make
their own adaptations by increasing immediate and delayed benefits, and decreasing immediate costs
associated with lesson planning. We believe that such policy intervention could have a medium-big
effect on student learning outcomes and a significant positive impact on the education system,
considering the comparatively low cost.
VI. References
Allcott, H., & Rogers, T. (2014). The short-run and long-run effects of behavioral interventions:
Experimental evidence from energy conservation. The American Economic Review, 104(10),
Allcott, Hunt and Rogers, Todd, The Short-Run and Long-Run Effects of Behavioral Interventions:
Experimental Evidence from Energy Conservation (October 2012). NBER Working Paper
No. w18492.
Antepohl, W., & Herzig, S. (1999). Problem,based learning versus lecture,based learning in a course
of basic pharmacology: a controlled, randomized study. Medical education, 33(2), 106-113.
Armbruster, P., Patel, M., Johnson, E., & Weiss, M. (2009). Active learning and student-centered
pedagogy improve student attitudes and performance in introductory biology. CBE-Life
Sciences Education, 8(3), 203-213.
Banerjee, A., Banerji, R., Berry, J. Duflo, E., Kannan, H., Mukerji, S. Shotland, M. & Walton, M.
(2016). Mainstreaming an Effective Intervention: Evidence from Randomized Evaluations of
“Teaching at the Right Level” in India. JPAL.
Bau, N. & Das, J. (2017) The Misallocation of Pay and Productivity in the Public Sector: Evidence
from the Labor Market for Teachers. RISE Working Paper 17/016.
Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2009). The importance of default options for
retirement saving outcomes: Evidence from the United States. In Social security policy in a
changing environment (pp. 167-195). University of Chicago Press.
Beshears, J., Choi, J. J., Laibson, D., Madrian, B. C., & Milkman, K. L. (2015). The effect of
providing peer information on retirement savings decisions. The Journal of finance, 70(3),
Bhargava, S., & Manoli, D. (2015). Psychological frictions and the incomplete take-up of social
benefits: Evidence from an IRS field experiment. The American Economic Review, 105(11),
Bryan CJ, Walton GM, Roger T, Dweck CS. 2011. Motivating voter turnout by invoking the self.
Proc. Natl. Acad. Sci. USA 108:1265356
Bryan, G., Karlan, D., & Nelson, S. (2010). Commitment devices. Annu. Rev. Econ., 2(1), 671-698.
Carmen Nobel (2017). “People Have an Irrational Need to Complete ‘Sets’ of Things.” Harvard
Business School.
Chabris, C. F., Laibson, D. I., & Schuldt, J. P. (2010). Intertemporal choice. In Behavioural and
Experimental Economics (pp. 168-177). Palgrave Macmillan UK.
Chetty, R., J. N. Friedman, N. Hilger, E. Saez, D. W. Schanzenbach, and D. Yagan (2011). How does
your kindergarten classroom affect your earnings? Evidence from project star. Quarterly
Journal of Economics. 126(4).
Costa, D. L., & Kahn, M. E. (2013). Energy conservation “nudges” and environmentalist ideology:
Evidence from a randomized residential electricity field experiment. Journal of the European
Economic Association, 11(3), 680-702.
Eric Bettinger, Bridget Terry Long, Philip Oreoupoulos and Lisa Sanbonmatsu (2012). “The Role of
Application Assistance and Information in College Decisions: Results from the H&R Block
FAFSA Experiment.” Quarterly Journal of Economics 127(3): 1205-1242.
Eric J. Johnson and Daniel Goldstein (2003). “Do Defaults Save Lives?” Science (November 21,
2003): 1138-39.
Erin Frey and Todd Rogers (2014). “Persistence: How Treatment Effects Persist After Interventions
Stop.” Policy Insights from the Brain and Behavioral Sciences 1(1): 172-179.
Frey, E., & Rogers, T. (2014). Persistence: How treatment effects persist after interventions
stop. Policy Insights from the Behavioral and Brain Sciences, 1(1), 172-179.
Gerber, A. S., Green, D. P., & Larimer, C. W. (2008). Social pressure and voter turnout: Evidence
from a large-scale field experiment. American Political Science Review, 102(1), 33-48.
Gourville, J. T., & Soman, D. (2005). Over choice and assortment type: When and why variety
backfires. Marketing science, 24(3), 382-395.
Grant, A. M., & Gino, F. (2010). A little thanks goes a long way: Explaining why gratitude
expressions motivate prosocial behavior. Journal of personality and social psychology, 98(6),
Hattie, J. (2003). Teachers make a difference: What is the research evidence? Paper presented at the
Building Teacher Quality: What does the research tell us. ACER Research Conference,
Melbourne, Australia. Retrieved from
Hershfield, H. E., Goldstein, D. G., Sharpe, W. F., Fox, J., Yeykelis, L., Carstensen, L. L., &
Bailenson, J. N. (2011). Increasing saving behavior through age-progressed renderings of the
future self. Journal of Marketing Research, 48(SPL), S23-S37.
Ho, D. E., & Imai, K. (2008). Estimating causal effects of ballot order from a randomized natural
experiment: The California alphabet lottery, 19782002. Public Opinion Quarterly, 72(2),
Jackson, CK., Makarin, A. (2016) Can Online Off-The-Shelf Lessons Improve Student Outcomes?
Evidence from A Field Experiment. NBER Working Paper No. 22398
John Beshears, James J. Choi, David Laibson and Brigitte C. Madrian (2008). “The Importance of
Default Options for Retirement Savings Outcomes: Evidence from the United States.” In S.J.
Kay and T. Sinha, Lessons from Pension Reform in the Americas (New York: Oxford
University Press), pp. 59-87.
Johnson, E. J., & Goldstein, D. (2003). Do defaults save lives?. Science, 302, Issue 5649, pp. 1338-
Johnson, E., Appelt, K., Knoll, M., Financial, C., Bureau, P., & Westfall, J. (2016). Preference
Checklists: Selective and Effective Choice Architecture for Retirement Decisions.
Jovanovic, D., & Matejevicb, M. (2014). Relationship between Rewards and Intrinsic Motivation for
Learning Researches Review. Procedia - Social and Behavioral Sciences, 149, 456-460.
Juan Carlos Montoy, William H. Dow and Beth C. Kaplan (2016). “Patient Choice in Opt-in, Active
Choice, and Opt-Out HIV Screening: Randomized Clinical Trial.” BMJ 352:h6895.
Keller, P.A., Harlam, B., Loewenstein, G., Volpp K.G. (2011). Enhanced active choice: A new
method to motivate behavior change. Journal of Consumer Psychology, 21, 376383.
Kim, J. S., Burkhauser, M. A., Quinn, D. M., Guryan, J., Kingston, H. C., & Aleman, K. (2017).
Effectiveness of Structured Teacher Adaptations to an Evidence,Based Summer Literacy
Program. Reading Research Quarterly.
Ko, J., Sammons, P., & Bakkum, L. (2014). Effective teaching. Education Development Trust.
Kurniasih, H. & Izati, M. (2017) Can passion make better teachers and cure Indonesia’s poor learning
level? The Conversation. (
Lipsey, M. W., Puzio, K., Yun, C., Hebert, M. A., Steinka-Fry, K., Cole, M. W., Roberts, M., Anothony, k. &
Busick, M. D. (2012). Translating the Statistical Representation of the Effects of Education
Interventions into More Readily Interpretable Forms. National Center for Special Education Research.
Madrian, B. C. (2014). Applying insights from behavioral economics to policy design. Annu. Rev.
Econ., 6(1), 663-688.
McKenzie, D. (2013) Enhanced Active Choice: Utilizing Behavioral Economics to Increase Program
Take-up [blog post]. Retrieved from
Milkman, K. L., Beshears, J., Choi, J. J., Laibson, D., & Madrian, B. C. (2011). Using implementation
intentions prompts to enhance influenza vaccination rates. Proceedings of the National
Academy of Sciences, 108(26), 10415-10420.
Mizala, A., & Romaguera, P. (2004). School and teacher performance incentives: The Latin American
experience. International Journal of Educational Development, 24(6), 739-754.
Nandrajog, M., Vaid, D. K., Kaur, H., Aggarwal, S., Kala, S., Srivastava, S., Kaur, I. (n.d.). In service
teacher education manual for teachers and teacher educators in commerce (high secondary
National Council of Educational Research and Training (NCERT). (n.d.) About us. Retrieved from
Nickerson, D. W., & Rogers, T. (2010). Do you have a voting plan? Implementation intentions, voter
turnout, and organic plan making. Psychological Science, 21, 194199.
Read, D., Loewenstein, G., & Kalyanaraman, S. (1999). Mixing virtue and vice: Combining the
immediacy effect and the diversification heuristic. Journal of Behavioral Decision Making,
12(4), 257.
RISE India. (2015) RISE in India: Research Overview (Technical). Oxford: RISE. Country Research
India (
Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel
data. American Economic Review P & P, 247252.
Rogers, T., Milkman, K. L., John, L. K., & Norton, M. I. (2015). Beyond good intentions: Prompting
people to make plans improves follow-through on important tasks. Behavioral Science &
Policy, 1(2), 33-41.
Saurabh Bhargava and Dayanand Manoli (2015). “Psychological Frictions and the Incomplete Take-
up of Social Benefits: Evidence from an IRS Field Experiment.” American Economic Review
105(11): 3489-3529
Sunstein, C. R. (2015). Active choosing or default rules? : The policymaker's dilemma. Behavioral
Science & Policy, 1(1), 29-33.
Sunstein, C., & Thaler, R. (2008). Nudge. The politics of libertarian paternalism. New Haven.
National Council of Educational Research and Training (NCERT). (n.d.) Teachers Handbook. New
Delhi. Retrieved from
Vegas, E. (2007). Teacher labor markets in developing countries. The future of children, 219-232.
Wijnia, L., Loyens, S. M., & Derous, E. (2011). Investigating effects of problem-based versus lecture-
based learning environments on student motivation. Contemporary Educational
Psychology, 36(2), 101-113.
World Bank. 2016. Does Class-Size Mediate the Effectiveness of Teacher Quality Interventions?
Improving Overcrowded Schools. Washington, DC: World Bank. Brief: What do we do.
Retrieved from
World Bank. 2017. Non-Financial Extrinsic and Intrinsic Teacher Motivation in Government and
Private Schools in India. Washington, DC: World Bank. Brief: What do we do. Retrieved
World Bank. 2018. World Development Report 2018: Learning to Realize Education's. Promise.
Washington, DC: World Bank. doi:10.1596/978-1-4648-1096-1. License: Creative Commons
Attribution CC BY 3.0 IGO.
*Note that the image used here is only to represent how it would look like in the second round of the active
choice form. Instead of it we would have the picture of the lottery winners. The image was taken from the
following link:
VII. Appendix
Appendix A: Sample Active Choice Form for Teachers
Dear [Teacher name],
In order to receive your salary, you need to check the lesson plan option you would like to be enrolled
in for the next 2 months, and submit the form to the principal no later than 1/9/2018. Check the box
on the left of the option you would like to enroll in:
!!Option 1: I would like to be a lesson plan designer and work in a team of two, using the
standard lesson plan provided as a guide.
!I recognize that by choosing this option, one of the team members needs to have less
than 3 years of teaching experience, and another, more than 3 years of teaching
!If I enroll in this option, I will have to complete and submit all my lesson plans for
the next 2 months before students start classes on 12/9/2018.
!If I enroll in this option, I will receive the following privileges:
o!I will receive a bonus of 4000 Rs. I will receive this amount immediately after
submitting the new lesson plans.
o!If my lesson plan is chosen as one of the best lesson plans by a team of expert
"!My lesson plan will proudly be shared with other teachers in my
school and across India.
"!I will be invited to attend a mid-semester dinner in a Leela Palace with
education officials.
"!I will get the chance to win 50 000 in a lottery that includes the
names of the teams that designed the best lesson plans.
!!Option 2: I will follow the pre-designed standard lesson plans I received, forgoing the chance
to develop my own lesson plan and get a bonus of 4000.
I, [teacher name] commit to the option I have chosen above.
Teacher’s Signature and date Principal's Signature and date
If you wish to opt-out of receiving the standard lesson plans, please check the box below and submit
this document to the principal no later than 12/9/2018.
!!I would like to opt-out of receiving standard lesson plans for this year.
Appendix B: Planner for teachers who chose option 1
Dear [teacher name],
Thank you for choosing to enroll in option 1, to help in the process, please use the space below to plan
your submission with your teammate. Remember that you will have to submit the lesson plans for the
first two months of classes by 12/9/2018.
The first row is filled in as an example.
Steps that need to be taken and time each will take
Time and day to
work on each
1.!Go over lesson content and think of potential lesson design
(1.5 h)
2.!Decide lesson objectives and activities together (2 h)
3.!Divide responsibilities among team members (30 mins)
4.!Write up the lesson plan (2 h)
5.!Review draft lesson plan and incorporate feedback (2 h)
Time for tasks:
(1) 8:00-9:30
(2) 9:30-11:30
(4) 12:00-2:00
(5) 2:00-4:00
On Monday,
10:00 am
1The Glossary for Education Reform: Student Centered learning, Retrieved from
2 In reference to Armbruster, Patel, Johnson, & Weiss (2009), Antepohl & Herzig (1999), and Wijna, Loyens & Derous (2010)
*The lesson plan design was inspired by Ko, Sammons & Bakkum (2014) on Effective Teaching, as well as Nandrajog et al (n.d.) lesson planning guide for teachers in India.
Appendix C: Lesson Plan Template & Checklist *
Unit Title:
Lesson objectives (I will communicate the objective to the students)
By the end of the class the student will be able to ….
Date: / /
Student-centered learning approach
places the student at the center of the
learning process, giving them the
opportunity to “lead learning activities,
participate more actively in discussions,
design their own learning projects, explore
topics that interest th em, and generally
contribute to the design of their own course
of study.”1
Reasons to adhere to student-centered
teaching style:2
!Students are more motivated and
engaged to learn
!Students are better able to retain
the information
!Students’ scores are slightly better
on exams
! Students develop better life skills
such as presentation and team
!Allows for teacher creativity,
breaking the school routine for
both teachers and students.
Lecture-based learning approach depends
on the teacher conveying information and
the student listen ing and taking notes. Th e
student is a passive learner in this case.
Reasons to adhere to student-centered
teaching style:
!Easier to manage students’
!Easier for teacher to plan for
Teachers’ names:
Methods of assessment:
I will check student understanding during the class time by ….
Students’ understanding of their thought process (metacognition)
I will develop students’ meta-cognitive skills by ….
!Asking them to voice their thoughts
!Going through the thinking process together as a class
!Having them showcase their thoughts on paper
I will cater to lower than average students' needs by … I will cater to higher than average students’ needs by …
Constructive feedback
I will provide students with constructive, positive, feedback
(1) When they do well, I will … (2) when they do not perform well, I will …
I will encourage cooperation between students by …
Student motivation
I will motivate students to learn by ….
!Relating content to student life
!Having students discover the answer on their own (with guidance)
1The Glossary for Education Reform: Student Centered learning, Retrieved from
2 In reference to Armbruster, Patel, Johnson, & Weiss (2009), Antepohl & Herzig (1999), and Wijna, Loyens & Derous (2010)
*The lesson plan design was inspired by Ko, Sammons & Bakkum (2014) on Effective Teaching, as well as Nandrajog et al (n.d.) lesson planning guide for teachers in India.
Class time
Teacher instructions - In class I will …
Student role - In class the student will …
Appendix D: Quasi-hyperbolic discounting and procrastinating lesson planning
Based on Chabris, Laibson, and Schuldt (2010), theories of intertemporal choice we can
assume the discounting function of teachers is Quasi-hyperbolic (which tends to better explain
individuals’ behaviors). This leads to dynamically inconsistent preferences. Teachers will discount the
future benefit of lesson planning by!an additional factor of ", such that the benefit of writing a lesson
plan in the future is "#$%&where t denotes now, such that t+1 denotes a period that is after now by
“1” time frame. When the benefit of lesson planning is experienced in the future in the form of
positive student outcomes, the cost (C) of writing a lesson plan is experienced in the present '$!.
Evaluated at the present, writing a lesson plan results in a higher cost than benefit, assuming #$= 0, as
shown in the equation below. 9
'$( #$) !"#$%&
At the same time, thinking about the future, teachers see more benefit than cost to writing a lesson
plan as shown in the equation below.
"'$%& * "#$%& ) !"#$%+
The variation in perceiving the same act in the present and future is referred to as “dynamically
inconsistent preferences” (Chabris, Laibson, & Schuldt, 2010). This leads to procrastinating the act of
designing a lesson plan, eventually leading to either doing it very quickly with little thought or not
doing it at all.
9 Note that for the purpose of this exercise we assumed that , - . where , is the exponential discounting of benefits and costs,
such that both benefits and costs are multiplied by a factor of ,t

Supplementary resource (1)

ResearchGate has not been able to resolve any citations for this publication.
Full-text available
The authors conducted a cluster-randomized trial to examine the effectiveness of structured teacher adaptations to the implementation of an evidence-based summer literacy program that provided students with (a) books matched to their reading level and interests and (b) teacher scaffolding for summer reading in the form of end-of-year comprehension lessons and materials sent to students' homes in the summer months. In this study, 27 high-poverty elementary schools (75-100% eligibility for free or reduced-price lunch) were matched by prior reading achievement and poverty level and randomly assigned to one of two implementation conditions: a core treatment condition that directly replicated implementation procedures used in previous experiments, or a core treatment with structured teacher adaptations condition. In the adaptations condition, teachers were organized into grade-level teams around a practical improvement goal and given structured opportunities to use their knowledge, experience, and local data to extend or modify program components for their students and local contexts. Students in the adaptations condition performed 0.12 standard deviation higher on a reading comprehension posttest than students in the core treatment. An implementation analysis suggests that fidelity to core program components was high in both conditions and that teachers in the adaptations condition primarily made changes that extended or modified program procedures and activities in acceptable ways. Adaptations primarily served to increase the level of family engagement and student engagement with summer books. These results suggest that structured teacher adaptations may enhance rather than diminish the effectiveness of an evidence-based summer literacy program.
Full-text available
Study question What is the effect of default test offers—opt-in, opt-out, and active choice—on the likelihood of acceptance of an HIV test among patients receiving care in an emergency department? Methods This was a randomized clinical trial conducted in the emergency department of an urban teaching hospital and regional trauma center. Patients aged 13-64 years were randomized to opt-in, opt-out, and active choice HIV test offers. The primary outcome was HIV test acceptance percentage. The Denver Risk Score was used to categorize patients as being at low, intermediate, or high risk of HIV infection. Study answer and limitations 38.0% (611/1607) of patients in the opt-in testing group accepted an HIV test, compared with 51.3% (815/1628) in the active choice arm (difference 13.3%, 95% confidence interval 9.8% to 16.7%) and 65.9% (1031/1565) in the opt-out arm (difference 27.9%, 24.4% to 31.3%). Compared with active choice testing, opt-out testing led to a 14.6 (11.1 to 18.1) percentage point increase in test acceptance. Patients identified as being at intermediate and high risk were more likely to accept testing than were those at low risk in all arms (difference 6.4% (3.4% to 9.3%) for intermediate and 8.3% (3.3% to 13.4%) for high risk). The opt-out effect was significantly smaller among those reporting high risk behaviors, but the active choice effect did not significantly vary by level of reported risk behavior. Patients consented to inclusion in the study after being offered an HIV test, and inclusion varied slightly by treatment assignment. The study took place at a single county hospital in a city that is somewhat unique with respect to HIV testing; although the test acceptance percentages themselves might vary, a different pattern for opt-in versus active choice versus opt-out test schemes would not be expected. What this paper adds Active choice is a distinct test regimen, with test acceptance patterns that may best approximate patients’ true preferences. Opt-out regimens can substantially increase HIV testing, and opt-in schemes may reduce testing, compared with active choice testing. Funding, competing interests, data sharing This study was supported by grant NIA 1RC4AG039078 from the National Institute on Aging. The full dataset is available from the corresponding author. Consent for data sharing was not obtained, but the data are anonymized and risk of identification is low. Trial registration Clinical trials NCT01377857.
Full-text available
We address the role of "psychological frictions" in the incomplete take-up of EITC benefits with an IRS field experiment. We specifically assess the influence of program confusion, informational complexity, and stigma by evaluating response to experimental mailings distributed to 35,050 tax filers who failed to claim $26 million despite an initial notice. While the mere receipt of the mailing, simplification, and the heightened salience of benefits led to substantial additional claiming, attempts to reduce perceived costs of stigma, application, and audits did not. The study, and accompanying surveys, suggests that low program awareness/understanding and informational complexity contribute to the puzzle of low take-up.
Many websites now warehouse instructional materials designed to be taught by teachers in a traditional classroom. What are the potential benefits of the new resources? We analyze an experiment in which we randomly give middle school math teachers access to existing high-quality, off-the-shelf lessons, and in some cases, support to promote their use. Teachers receiving access alone increased students' math achievement by a marginally significant 0.06 of a standard deviation. Teachers who received access and support increased students' math achievement by 0.09 of a standard deviation. Weaker teachers experience larger gains, suggesting that these lessons substitute for teacher skill or efforts. The online materials are more scalable and cost effective than most policies aimed at improving teacher quality, suggesting that, if search costs can be overcome, there is a real benefit to making high-quality instructional materials available to teachers on the Internet.
Decisions that have consequences in multiple time periods are intertemporal choices. Individuals typically discount delayed rewards much more than can be explained by mortality effects. The most common discount function is exponential in form, but hyperbolic and quasi-hyperbolic functions seem to explain empirical data better. Individual discount rates may be measured in a variety of ways, subject to important methodological caveats. Higher discount rates are empirically associated with a variety of substance abuse and impulsive conditions, including smoking, alcoholism, cocaine and heroin use, gambling, and risky health behaviours. By contrast, low discount rates may be associated with high cognitive ability.
People fail to follow through on all types of important intentions, including staying fit, studying sufficiently, and voting. These failures cost individuals and society by escalating medical costs, shrinking lifetime earnings, and reducing citizen involvement in government. Evidence is mounting, however, that prompting people to make concrete and specific plans makes people more likely to act on their good intentions. Planning prompts seem to work because scheduling tasks makes people more likely to carry them out. They also help people recall in the right circumstances and in the right moment that they need to carry out a task. Prompts to make plans are simple, inexpensive, and powerful interventions that help people do what they intend to get done. They also avoid telling people what to do, allowing people to maintain autonomy over their own decisions.
In Project STAR, 11,571 students in Tennessee and their teachers were randomly assigned to classrooms within their schools from kindergarten to third grade. This article evaluates the long-term impacts of STAR by linking the experimental data to administrative records. We first demonstrate that kindergarten test scores are highly correlated with outcomes such as earnings at age 27, college attendance, home ownership, and retirement savings. We then document four sets of experimental impacts. First, students in small classes are significantly more likely to attend college and exhibit improvements on other outcomes. Class size does not have a significant effect on earnings at age 27, but this effect is imprecisely estimated. Second, students who had a more experienced teacher in kindergarten have higher earnings. Third, an analysis of variance reveals significant classroom effects on earnings. Students who were randomly assigned to higher quality classrooms in grades K-3-as measured by classmates' end-of-class test scores-have higher earnings, college attendance rates, and other outcomes. Finally, the effects of class quality fade out on test scores in later grades, but gains in noncognitive measures persist.
Interventions intended to change people’s behavior are ubiquitous in modern society. Some interventions produce changes in behavior that persist even after the interventions are discontinued, while other interventions generate only short-term behavior changes that disappear once the interventions stop. The framework presented here guides understanding of why and how behavior changes (treatment effects) persist after interventions (treatments) are discontinued. Four persistence pathways explain how persistent treatment effects may arise: building psychological habits, changing what and how people think, changing future costs, and harnessing external reinforcement. Each pathway is illustrated by describing how the pathway may have contributed to the persistent treatment effects produced by a widely used energy-efficiency intervention conducted by the energy-efficiency company OPOWER. Different conditions may make each pathway more or less likely to generate persistent treatment effects in the world. Finally, policymakers might develop more persistent interventions by leveraging each pathway.