Content uploaded by Robert A. Cannon
Author content
All content in this area was uploaded by Robert A. Cannon on Dec 20, 2017
Content may be subject to copyright.
The Sustainability of Benefits
from Educational Development
Projects in Indonesia:
What do the Donors Say?
Sustainability of Educational Development in Indonesia
Project: Working Paper #2
Robert Cannon
cannonra@icloud.com
Version 2
20 December 2017
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
ii
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
iii
Table of Contents
Abbreviations ............................................................................................................... v
Abstract ........................................................................................................................ 1
1 Introduction and purpose ....................................................................................... 3
2 Methodology ......................................................................................................... 5
2.1 Donor reports ................................................................................................. 5
2.2 The donors’ reporting process ......................................................................... 6
2.3 The value of donor reports .............................................................................. 8
3 Analysis ................................................................................................................ 9
3.1 Are the benefits of projects sustainable? ........................................................... 9
3.2 Analysis of sustainability ................................................................................. 9
3.3 Is sustainability of project benefits always a project goal? ............................... 16
3.4 Discussion .................................................................................................... 16
4 Factors contributing to sustainability .................................................................... 21
4.1 Project design ............................................................................................... 21
4.2 Commitment: responsibility and ownership ................................................... 22
4.3 Technical Matters ......................................................................................... 24
4.4 Lessons from Actual Sustainability Analysis .................................................. 29
5 Prospects ............................................................................................................. 31
6 References ........................................................................................................... 33
Appendix: Summary of Evidence from Donor Reports of the Sustainability of Benefits
from Educational Development Projects in Indonesia, 1971 – 2017 .............................. 35
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
iv
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
v
Abbreviations
ACDP
Analytical Capacity and Development Partnership (Australia)
ADAB
Australian Development Assistance Bureau (1977 – 87)
ADB
Asian Development Bank
AIBEP
Australia Indonesia Basic Education Program
AIDAB
Australian International Development Assistance Bureau (1987 –
95)
AusAID
Australian Agency for International Development (1995 – 2013)
BERMUTU
Better Education Through Reformed Management and Universal
Teacher Upgrading (World Bank)
BRIDGE
Building Relationships through Intercultural Dialogue and
Growing Engagement
CIMU
Central Independent Monitoring Unit
CLCC
Creating Learning Communities for Children (UNICEF)
CR
Completion Report (ADB)
CTL
Contextual Teaching and Learning (Indonesian Government)
DBE
Decentralized Basic Education (USAID)
DFAT
Department of Foreign Affairs and Trade (Australia)
DFID
Department for International Development (UK)
DSF
Decentralized Support Facility (World Bank)
EP
Education Partnership (Australia)
ER
Evaluation Report
EU
European Union
GTZ
Gesellschaft für Technische Zusammenarbeit (German Technical
Cooperation Agency)
IAPBE
Indonesia Australia Partnership in Basic Education
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
vi
IASTP
Indonesia Australia Specialised Training Project
IATEP
Indonesia Australia Technical Education Project
ICR
Implementation Completion Report (World Bank)/Independent
Completion Report (AusAID)
ICRR
Independent Completion and Results Report (World Bank)
IDRC
International Development Research Center (Canada)
IEG
Independent Evaluation Group (World Bank)
IPR
Independent Progress Review (AusAID)
JFPR
Japan Fund for Poverty Reduction
JICA
Japan International Cooperation Agency
JSE
Junior Secondary Education
LAPIS
Learning Assistance Program for Islamic Schools (Australia)
LPMP
Lembaga Penjaminan Mutu Pendidikan
(Education Quality Assurance Institution)
MEDP
Madrasah Education Development Project (ADB)
MBE
Managing Basic Education (USAID)
MEQIP
Mathematics Education Quality Improvement Programme (GTZ)
MGPBE
Mainstreaming Good Practices in Basic Education (UNICEF)
MoEC
Ministry of Education and Culture
MTR
Mid Term Review
NTT-PEP
Nusa Tenggara Timur Primary Education Partnership
NZAID
New Zealand Agency for International Development
PCR
Project Completion Report
PEQIP
Primary Education Quality Improvement Project
PER
Performance Evaluation Report (ADB)
PPAR
Project Performance Audit Report (ADB)
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
vii
ProDEP
Professional Development for Education Personnel
PRIORITAS
Prioritizing Reform, Innovation, and Opportunities for Reaching
Indonesia’s Teachers, Administrators, and Students (USAID)
REDIP
Regional Education Development and Improvement Program
(Japan) (REDIP-G = Government; REDIP-P = Provincial)
SBM
School Based Management
SEQIP
Science Education Quality Improvement Project (GTZ)
UNESCO
United Nations Education, Scientific and Cultural Organisation
UNICEF
United Nations Children’s Fund
USAID
United States Agency for International Development
USD
United States Dollar
VR
Validation Report (ADB)
Acknowledgements
Paskal Kleden’s valuable comments and insights on an earlier versions of this paper are
warmly acknowledged. Full responsibility for this version remains, however, with the
author.
The cover artwork by an unknown junior high school student illustrates a student’s
experience of student centred active learning, known by the acronym PAKEM,
(Pembelajaran Aktif, Kreatif, Efektif dan Menyenangkan – learning that is Active, Creative,
Effective, and Joyful).
Project reports analysed for this study were located on donor websites and from libraries.
The following additional sources of evidence for the history of project implementation are
acknowledged with appreciation:
• Dr Sheldon Shaeffer (personal communication).
• The project list contained in the ADB Project Performance Audit Report on the Junior
Secondary Education Project, 1993 – 1998.
• The project list from the report by Cannon, R. and Arlianti, R. (2008). Review of
Education Development Models for Increasing Access to Quality Basic Education in
Indonesia. Jakarta: The World Bank.
All photographs by the author.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
viii
A note on referencing in this Working Paper
References in the text to specific educational development projects are presented in the
form: ‘AusAID’s Australia Indonesia Basic Education Program, 2006 – 2010’. The Appendix
provides details of these projects which are listed in order of project implementation dates.
All other references follow standard referencing conventions.
Record of Working Paper Amendments
Version 1: 10 October 2017
Version 2: 20 December 2017: This version includes two further projects (AusAID’s
BRIDGE and JICA’s Program for Enhancing Quality of Junior Secondary Education) and
related adjustments to the analysis and discussion. A new section 4.4, explores the
findings about sustainability embedded in reports of the 22 projects reporting the actual
sustainability of benefits. More generally, this version discusses recent publications, and
includes several editorial adjustments.
The Sustainability of Educational Development in
Indonesia Project
The Sustainability of Educational Development in Indonesia Project is a privately funded
and implemented research project to assist in the development of quality education in
Indonesia.
Working Paper publications in this series:
Cannon, R. 2017. The Challenging Concept of Sustainability in Educational Development in
Indonesia. Sustainability of Educational Development in Indonesia Project: Working
Paper #1. Adelaide: Author.
Available:
https://www.researchgate.net/publication/316997532_The_Challenging_Concept_of_Su
stainability_in_Educational_Development_in_Indonesia_Sustainability_of_Educational_
Development_in_Indonesia_Project_Working_Paper_1
Ó Robert Cannon. 2017
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
1
The Sustainability of Benefits from
Educational Development Projects in
Indonesia:
What do the Donors Say?
Sustainability of Educational Development in Indonesia Project: Working Paper #2
Abstract
Have the benefits from the development assistance provided to Indonesian education by
the international donor community been sustainable?
This study attempts to provide a basis for answering this question. It is based on an
analysis of 91 end-of-project evaluation reports produced by donors for the period 1971 –
2017. The study also seeks to understand why donors consider benefits to be sustainable.
Donor reports are useful for two purposes. First, in making an overall assessment of the
sustainability of benefits from development assistance and, second, in increasing our
understanding of the factors that lead to sustainable benefits.
Overall, donor evaluation reports show 52% of the 91 projects identified for the present
study are evaluated as ‘likely’ to be sustainable. However, when tested against the time-
dependent criterion of the evaluation of the actual sustainability of project benefits,
conducted two years or more after project completion, a more troubling finding emerges.
Only twenty-two project reports (24% of the 91 projects located for analysis) present an
evaluation of the actual sustainability of benefits two or more years after project
completion. Eleven of these 22 projects, or 12% of all 91 projects, are evaluated as actually
sustainable. For the remaining 76% of projects (n= 69) evaluated at or near to project
completion, the reports tell us that 36 projects, (52% % of these 69 projects), are
considered to be potentially sustainable.
So, to answer the question ‘are benefits sustainable?’ the evidence from 91 donor reports
of educational development projects in Indonesia spanning 46 years, is that only 12% of
all projects show evidence of actually sustainable benefits.
Reports describe strategies to achieve sustainable benefits from donors’ educational
development support. The reports tell us three main things.
First, project design and all activities through to completion should include sustainability
matters. Second, Indonesian society, particularly government, must demonstrate a
commitment to sustainability. Third, a complex web of supporting technical matters must
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
2
be addressed. The continuing financial and technical maintenance of assets of all kinds is
one important requirement.
Donors’ reports show poor sustainability outcomes from projects implemented over 46
years and at a cost exceeding USD 5 billion. These results establish the need to reconsider
missed opportunities for ensuring sustainability and how future project design and
implementation can ensure that the outcomes and impact of these projects are more
sustainable.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
3
“I’m concerned about sustainability. Often when projects end, things return to normal. There should
be an interim period of support but there is no such arrangement beyond this phase”. Advisory Board
Member, Learning Assistance Program for Islamic Schools (LAPIS, AusAID, 2010, 19.)
1 Introduction and purpose
This Working Paper extends the discussion of the sustainability of development in
Indonesian education introduced in Working Paper #1 (Cannon, 2017). That Paper
considered the concept of sustainability and distinguished between the idea of
environmental sustainability and the definition used here for donor-supported projects.
Succinctly enunciated by the former AusAID, the definition is: ‘the continuation of
benefits after major assistance from a donor has been completed’ (AusAID, 2005, 1).
Paper #1 found the principles underpinning donor-supported change and the
sustainability of outcomes from that support to Indonesian education to be unclear,
forgotten over time, and often derived from fields other than education. The Paper noted
donors have mostly failed to seriously enquire into the sustainability of the massive
support they have provided to Indonesian education through their projects and programs.
Donors usually produce an evaluation report of that assistance at or near project
completion. Data analysed in this Working Paper comes from these kinds of reports.
Among other matters, reports present analysis and commentary on the sustainability of
benefits from development assistance. Commonly prepared at about the time of project
completion, reports dating back 46 years have been located for this study. They provide a
large body of evidence from all sectors of education and all Indonesian regions.
There is no rigorous, independent empirical investigation to assess the effectiveness or the
sustainability of benefits from development support to education. What we do have is
donor reports and a smaller number of other studies. A subsequent Working Paper will
analyse these studies.
Analysis of data from donor reports shows that donors have granted or loaned more than
USD 5 billion to Indonesia for at least 101 different projects for educational development
in the 46 years since 1971. The Appendix summarises the evidence of this immense
amount of work.
Have the benefits from the development assistance to education provided by the
international donor community been sustainable? This study sets out to provide a basis
for answering this question from the perspective of the donors. It also seeks to understand
why benefits may or may not be sustainable. That understanding can, in principle,
provide a basis for strengthening policy and practice in achieving more sustainable
benefits from international donor support to educational development in Indonesia.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
4
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
5
2 Methodology
The study relies on the evidence provided by the donors in 91 project evaluation reports1.
Listed in the Appendix, most of the reports were prepared at, or near to, project
completion, however, 22 reports reflect follow up evaluations conducted some years after
project completion. These 22 reports provide insights into the actual sustainability of
benefits after donors have completed their work. The 2014 Australia Indonesia Basic
Education Program, School Survey is a very good example of this reporting. That survey
was conducted when all Program schools had been operational for at least four years.
A few mid-term reports of projects are included in this study where no other evidence of
project completion reporting could be located. Unfortunately, for some projects listed in
the Appendix, no documentary trace could be located at all. Projects include technical
and vocational education projects supported by the Swiss Government in the 1980s and
90s, and Indonesian Government implemented projects such as Contextual Teaching and
Learning.
Donors use a range of different titles to define these reports such as Project Completion
Report, Independent Completion and Results Report, Project Performance Audit Report,
or Final Evaluation, among others. There is neither standard reporting nomenclature nor
standard approaches to end-of-project reporting among donors. Nor do donors have a
uniform conception of sustainability, as Working Paper #1 demonstrated.
Comprehensive documentation such as program design documents, annual reports,
monitoring and evaluation plans and reports, and final evaluations are not readily
available for projects. This gap results in missing information about projects. Detailed
analysis of most projects is difficult given the absence of the availability of these materials.
All these matters contribute to clunky research. However, the large number of
observations made by different evaluators and donors over a long period of 46 years
improves the reliability of conclusions reached in this study.
2.1 Donor reports
End-of-project reports from the World Bank and the Asian Development Bank (ADB) are
generally accessible on the internet. Retrieval of several early Bank reports with restricted
access was not possible. A few reports from the other donors could be located on the web
and in libraries, but there was difficulty in readily accessing these sources of information.
The two Banks’ editorial processes lead to the production of reports that follow a
reasonably clear structure and format. Their reports present comprehensive project data
1 The term ‘project’ used in this Working Paper means a coherent set of interventions, executed over a specified
time and within a defined budget intended to achieve defined purposes. Sometimes the term ‘project’ is used
interchangeably with ‘program’. However, program is more properly applied to a broader range of integrated
activities associated with broader development outcomes.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
6
and demonstrate good professional writing standards. They have much better control of
the presentation of acronyms and technical terms that frequently cripple comprehension
in some other donors’ reporting, leading to confusion for all but a project’s cognoscenti.
Most reports present a broad range of both quantitative and qualitative project data. Few
reports present quantitative data to support their analysis of sustainability. A
distinguished exception is the Australian Department of Foreign Affairs and Trade
(DFAT) Australia Indonesia Basic Education Program Schools Survey, 2014.
Two matters stand out when reading reports’ conclusions. First, a generalised conclusion
about the sustainability of benefits of the kind commonly presented such as ‘Likely’ or
‘Uncertain’ masks the reality that most projects implement several different components
of which overseas fellowships, building construction, management and governance, the
provision of materials and equipment, organisational development, and professional
development are examples. Each component commonly has a different sustainability
outcome.
Second, in some reports, there is a lack of congruence between the generalised
conclusions reached and the data presented and analysed in the report. Evidence of this
weakness is on display in the disparity between some reports’ conclusions about
sustainability that do not accurately reflect the evidence discussed in the body of the
report. One example is the ADB Vocational Education Strengthening Project, 2008 – 2014.
2.2 The donors’ reporting process
Donor reports are not independent sources of information. Interpreting report’s findings
requires caution. Commissioned by the donor, edited by the donor, and sometimes also
by development partners who may add their own evaluation comments, donor reports
risk being positively biased.
There are risks of other weaknesses in the processes leading to the production of these
reports. These weaknesses do not reflect on any specific donor, contractor, or report and
there are many exceptions to the critique presented below. Nor do the weaknesses
described necessarily invalidate reports, but they have an unknown impact on the quality
of the conclusions reached about sustainability. The weaknesses warrant comment for
their possible impact on the reliability and validity of donor reports as a source of
evidence.
• Evaluations are often implemented by contracted small teams either working alone
or in concert with officials from the donor, the recipient government, or both.
Donors sometimes carry out their own evaluations. Evaluation team members
usually work independently of the project team responsible for implementation
although they may depend on them for advice in selecting districts, schools to visit
and respondents, and for support during field visits.
• Often comprising people who may have never worked together previously,
evaluation teams sometimes have mismatched skill sets and interpersonal
difficulties. Moreover, some team members may have limited or no in-country
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
7
experience and may not have the opportunity to participate in the construction of
the evaluation methodology.
• Donors generally determine the evaluation through the questions they pose, a
mandated formulaic structure for reporting, and by the limited time and provision
of resources for the evaluation. Set within strictly limited time frames, donors often
structure the evaluation around common work elements such as briefing and
debriefing, preliminary desk work, field work, analysis, and reporting. Genuine
difficulties arising during an evaluation such as negotiating methodologies, delays,
personnel problems, or ‘Acts of God’ may lead to compromises that weaken the
quality of the evaluation process.
• Given the large scale, complexity, and geographical spread of many projects in
Indonesia, there are usually inadequate resources to undertake the preparation of
sufficiently broadly-based or in-depth studies for data collection and analysis.
Samples of project populations studied are generally small and, having been
selected for the evaluation team, may be skewed in unknown ways. Often,
evaluators must rely on the quality of monitoring and evaluation (M&E) data
provided by the project. As the weakness of M&E is often criticised in project
reports, this practice raises further concerns about data quality.
• Few donor reports explain how they have assessed sustainability. There are
exceptions. A comprehensive analysis of sustainability appears in the Exit
Sustainability Report for the Learning Assistance Program for Islamic Schools, 2005 –
2010 (Sanderson, 2009), in the report on the Indonesia Australia Partnership in Basic
Education, 2004 – 2007, and in the 2014 Schools Survey, effectively a study of the
actual sustainability of the Australia Indonesia Basic Education Program, 2006 – 2011.
These Australian reports present clearly structured and analytical approaches.
Another is the Asian Development Bank (ADB) report of the Junior Secondary
Education Project, 1993 – 1998, where conclusions are based on quantitative field
data. Nevertheless, the reader of many reports will observe that conclusions on
sustainability are mostly global judgements rather being based on careful analysis.
• Finally, donors or beneficiaries may insist on adjustments to drafts of evaluators’
findings, conclusions, and recommendations to meet the needs of officials, the
donor, or the recipient government.
These specific concerns with education reports in Indonesia are reflected in a more
general conclusion about project reports reached by Inder Sud in his 2017 book Reforming
Foreign Aid: Reinvent the World Bank. Sud, formerly a senior World Bank economist,
reports that the Bank’s Independent Evaluation Group’s (IEG) audit of project
completion reports found that these self-assessments were high by about 10 percent. He
also notes in Chapter 15 that sustainability was a key aspect of IEG evaluations until 2000
when it was de-emphasised and reporting on it stopped completely around 2005.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
8
Sud’s claim is only partly true for the evaluation of sustainability in the Bank’s education
projects in Indonesia. Some report on sustainability and some do not and others introduce
a new concept, “risk to development outcomes”.
2.3 The value of donor reports
Given that there are so many concerns about reports, it is reasonable to ask ‘why use
them at all?’ This matter deserves a response.
First, donor reports are the only accessible data we have about most development
projects. Ignoring them leaves little or no evidence to work with at all. On this basis, they
should be considered.
Second, the negative critique above does not apply to all donor evaluations; the worst of
the interferences and difficulties rarely occur together. Care and quality outcomes are
evident in many reports and the quality of project M&E data used in some evaluations are
known to be very good. Examples of quality reports include the Independent Completion
Report of the AusAID Nusa Tenggara Timur Primary Education Partnership, 2002 – 2008, a
project rated independently as among the ‘best of the best’ (Bazeley, 2011). Another is the
Final Evaluation report of the USAID project, Managing Basic Education, 2003 – 2007.
Third, the large body of evidence in the project reports listed in the Appendix and
representing over 46 years of project implementation experience does improve the
reliability of findings compared to relying on only one or a small sample of reports.
Fourth, the perceived risk of a positively skewed approach implied above may not have
led to an improbably positive outcome after all. On the contrary, the evidence for the
sustainability of benefits presented in donor reports is discouraging and bleak.
Fifth, the analysis of donor reports is useful in other ways. They yield insights into design
and implementation factors considered by evaluators likely to help or to hinder
sustainability. Section 4 Factors contributing to sustainability discusses these insights.
Finally, some donors have undertaken follow up evaluations to review the quality of
project completion reports and to validate end-of-project findings. The follow up reports
provide the opportunity to assess the actual sustainability of benefits achieved compared
to the more common prediction of potential sustainability at the time of project
completion. Examples of these validation-type reports are those listed in Appendix for the
ADB Marine Sciences Education Project, 1989 – 1995, the ADB Junior Secondary Education
Project, 1993 – 1998, and the World Bank Third Barefoot Engineers Training Project, 2012 –
2013.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
9
3 Analysis
3.1 Are the benefits of projects sustainable?
If we wish to answer the question ‘are the benefits of projects sustainable?’ then the actual
sustainability of benefits achieved requires evaluation after some time has elapsed after
project completion. Elapsed time is necessary to allow for evidence to emerge of local
responsibilities in maintaining and developing the benefits provided from project support.
Project reports prepared two or more years following project completion can provide
evidence of actual sustainability. Projects evaluated two or more years after completion
are marked with an asterisk in the Appendix.
Logically, project completion is not a sensible time to evaluate the actual sustainability of
benefits from a project. At completion, it is impossible to know whether benefits will be
sustained into the future. At project completion, there can only be an estimate of
sustainability. This estimate is described here as potential sustainability.
Evaluators do not possess a crystal ball; they cannot pass judgement on the actual
sustainability of benefits achieved at the time of project completion. Yet there is an
implication in donors’ reporting practices that this estimate is sufficient to make
judgements about sustainability. It isn’t. The analysis presented here illustrates this
matter.
3.2 Analysis of sustainability
Formal donor reports obtained for 91 projects from the total of 101 educational
development projects listed in the Appendix2 provide the data for analysis. Reports for ten
projects proved impossible to obtain for the study. However, their listing provides
evidence of further development activity in the education sector. The following analysis of
sustainability is based on those 91 project reports. The analysis is in three parts.
Presented first is an overall analysis of the sustainability of all 91 projects. The second
analysis is of the potential sustainability of projects. The last analysis is of the actual
sustainability of benefits evaluated two or more years after project completion. This last
evaluation is for only 24% of projects; there is no evidence of donors evaluating the actual
sustainability of 76% of their educational development projects in Indonesia.
2 There is one exception here. As no official reports for the sequence of Australian technical and vocational
education projects in the 1980s and 1990s could be located, a secondary report written by Maglen and Hopkins,
2000, provided data for analysis.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
10
3.2.1 Overall analysis of sustainability
Reports use a wide variety of terms to describe sustainability outcomes such as yes, likely,
uncertain, no, mixed, partial, or not likely. There is no consistent terminology over time
or between donors to describe conclusions about sustainability. The Tables below group
similar terms reflective of overall evaluative judgements.
Table 1 presents a summary of the evaluation of all 91 projects in this study. Table 1
shows that 47 projects (52%) were thought to be sustainable. For the remaining 49% of
projects, 18% were considered to be unsustainable, 21% were considered to be uncertain,
and for the remaining 10% there was no assessment of sustainability at all.
Given that donor reports may be biased towards reporting positive outcomes, as argued
earlier, this bleak outcome is most likely to be more positive than the reality.
Table 1: Sustainability of benefits from educational development projects (overall
evaluation)3
Project benefits sustainable?
No. of projects
%
Yes, likely, satisfactory, sustainable
47
52
Uncertain, partial, modest, mixed
19
21
No, not likely
16
18
Not assessed/not mentioned
9
10
Total projects
91
100
This overall analysis only gives a partial picture of sustainability. To get closer to
understanding the sustainability of benefits, it is necessary to dig deeper into the data.
When the overall data is disaggregated into that gathered at or near to project completion
and that gathered two or more years following project completion, a different picture
emerges.
3.2.2 Analysis of potential sustainability
Evaluations conducted at project completion can only provide an estimate of
sustainability. This estimate is described here as potential sustainability. The analysis of
potential sustainability is presented in Table 2.
3 In all Tables, percentages shown may not sum to 100 because of rounding.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
11
What do reports tell us about the potential sustainability of the 76% of projects (n= 69)
evaluated at or near to project completion? Thirty-six of these projects, (52% of the 69
projects), are evaluated as likely sustainable. Reports for nine projects have no assessment
of sustainability. Another nine projects are potentially not sustainable, and reports for 15
projects express uncertainty or mixed results.
Table 2: Potential sustainability of benefits from educational development projects
Project benefits potentially sustainable?
No. of projects
%
Yes, likely, satisfactory, sustainable
36
52
Uncertain, partial, modest, mixed
15
22
No, not likely
9
13
Not assessed/not mentioned
9
13
Total projects evaluated at or near project completion
69
100
Total as % of all projects (n=91)
76%
Snapshot of potentially sustainable benefits: AusAID’s Indonesia Australia Partnership
in Basic Education (IAPBE), East Java, 2004 – 2007 p. 11.
‘The design of IAPBE caters well for sustainability of activities, in that mechanisms are in
place to provide necessary skills and knowledge to local stakeholders. There are now
trained personnel within all the targeted schools who are working in teams and going
beyond their localities to disseminate training and information to non-targeted areas with
funding supplied from local budgets. There are also sub-district and district level officials
who have benefited from training and who are able to advise beyond their own
boundaries.
Moreover, many of the local consultants are highly competent, professional people who
had a great deal of experience and expertise before coming into the Partnership. These
Indonesian consultants already advise on as well as plan and implement training. The
Component 3 (school development) consultants have trained trainers who in turn plan
and implement all of the activity’s district training. The strength of the design lies in the
resulting large pool of trainers who can continue the training both in the targeted sub-
districts and their schools and also disseminate beyond to non-targeted areas. All IAPBE
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
12
trainers independently delivered at least 16 training courses before activity completion
with some approaching 30. Their skills are well consolidated.’
Snapshot of mixed results across project components: AusAID/DFAT’s Australia’s
Education Partnership with Indonesia, 2011 – 2017.
‘The performance of the Education Partnership (EP) was weakest in the area of
sustainability. Inevitably, reductions in spend through government systems reduced the
degree to which EP support could prepare those systems to adopt and adapt initiatives
developed by Implementing Partners. Despite the take-up of various EP products –
products that will sustain benefits in particular segments of the education system –
insufficient attention was paid to the political-institutional realities of the sector and, by
consequence, future financing options for systems-level reform were ill-considered (p. ix).
“[Component 1: School Construction] transferred a valuable set of tools and systems for
ongoing GoI programs of school construction and expansion and the high quality of
school construction will contribute to the sustainability of infrastructure” (p.12).
Figure 1: Indonesia Australia Partnership in Basic Education. School principal, centre, a former Partnership facilitator, has
disseminated new teaching and management approaches to adjacent schools in his school’s district as well as ensuring
sustainability of change in his own school.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
13
“Despite [the Ministry’s] support for the extension of Professional Development for
Education Personnel (ProDEP) activities beyond the Education Partnership, Australian
investments in ProDEP are unlikely to produce sustainable outcomes due to the absence
of a clear financing strategy” (p.16).
“[Component 3: Islamic School Accreditation] displayed evidence that it was
strengthening systems that will support improvements to the quality of madrasah service
provision over the longer term (sustainability)” (p.17).
“[Component 4: Analytical Capacity and Development Partnership (ACDP)] … there is
no clear indication of what, if any, institutional benefits the ACDP wishes to sustain and
how that may be achieved during the remaining phase of operations to June 2017. Whilst
institutional sustainability was not foreseen within the design of the ACDP, the lack of
clarity about any expected institutional legacy represents a missed opportunity to generate
a better return on the USD 40-45 million investment. While sustained benefits may accrue
there is a sense in some quarters that the prospects for institutional change have rather
been left to chance and, should that be the case, opportunities may have been missed”
(p.21).
3.2.3 Analysis of actual sustainability
The last analysis is of the actual sustainability of benefits evaluated two or more years after
project completion. Twenty-two project reports (24% of the 91 projects) present an
evaluation of the actual sustainability of benefits two or more years after project
completion. Eleven of these 22 projects (50%) are evaluated as actually sustainable.
Table 3: Actual sustainability of benefits from educational development projects
Project benefits actually sustainable?
No. of projects
%
Yes, likely, satisfactory, sustainable
11
50
Uncertain, partial, modest, mixed
4
18
No, not likely
7
32
Total projects evaluated two or more years after project completion
22
100
Total as % of all projects (n=91)
24%
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
14
So, the answer to the question ‘are benefits sustainable?’ is that from the reports of 91
educational development projects spanning 46 years, we have evidence from the donors
that 11 projects (that is, only 12% of all projects in this study) show evidence of actually
sustainable benefits.
The converse presents this outcome more starkly; 88% of educational development
projects have no clear evidence of the actual sustainability of benefits.
The following snapshots quoted from project reports give a sense of what works and what
does not. They provide insight into the complex processes of success or failure.
Snapshot of actually sustainable benefits: The World Bank, Better Education Through
Reformed Management and Universal Teacher Upgrading (BERMUTU) Project, 2008 –
2013, p. 19.4
‘There is a low to negligible risk for changes to occur that might be detrimental to the
achieved Project Development Objectives because there is ample evidence of the
financial, institutional, and technical sustainability of the key activities financed under
BERMUTU. As mentioned before, the BERMUTU programs are continued (sic) to be
implemented by the GoI and there are some programs financed by other donors, such as
the school principal and supervisor quality improvement funded by the Australian
Department of Foreign and Trade (DFAT) grant.
With respect to the financial sustainability, the district governments and teachers are
automatically bound to implement the regulations under Component 3 of BERMUTU as
these are to be implemented nationally.
With respect to the institutional sustainability, BERMUTU-related activities continue to
be housed within the different units of the Ministry of Education and Culture at the
national and district level and supported by an existing regulatory framework and
instruments. Some districts are enacting their own regulations to complement and
synergize MoEC’s current BERMUTU undertaking. With respect to the technical
sustainability, the internal capacity built during BERMUTU’s implementation period at
both the national, provincial and district level mainstream institutions in terms of trained
staff and procedures’.
Snapshot of actually unsustainable benefits: The ADB Marine Sciences Education
Project (MSEP) 1989 – 1995, p.15.
‘The Marine Sciences Education Project has had successes and produced benefits and
thus has had positive impacts.
4 BERMTU is an example of a complicating factors in evaluating sustainability and determining where
responsibilities for results may lie. This is where, in the case of BERMUTU, a project assists in the implementation
of existing policy and prior analytical work, so adding to the work that the beneficiary, in this case the government,
is carrying out. In addition, other donors (DFAT) have been involved.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
15
However, this must be qualified by negative aspects: the poor condition of some of the
equipment and vessels; underutilization of books, equipment, and many buildings … non
functionality of the Likupang Field Station (North Sulawesi); inactivity at Institut
Pertanian Bogor (West Java); deterioration of the Barang Lompo Field Station (South
Sulawesi); almost universal absence of a fertile academic atmosphere; difficulty of
graduates in finding relevant employment; lack of coordination among project universities
and staff; failure of overseas training to make a meaningful change in the fellows’ roles;
uneven quality of research; meagre interface with the private sector; and failure to have an
effect on the use of marine resources, nutrition or the economy’.
Figure 2: Marine Sciences Education Project. Recently constructed classrooms in advanced stage of deterioration, Universitas Sam
Ratulanggi, Likupang Field Station (North Sulawesi)
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
16
3.3 Is sustainability of project benefits always a project goal?
The analysis so far has proceeded on the assumption that achieving sustainable benefits
was a goal of each project. There is variation among the projects analysed in their
commitment to sustainability.
In the ADB Junior Secondary Education Project (1993 – 1998) one of the four project goals
was to ‘… strengthen the Ministry of National Education and the Ministry of Religious
Affairs to ensure sustainability of improvements achieved under the Project’ (p.2).
Only inferences about sustainability goals are possible for many projects. For example,
this statement of purpose from the ADB Decentralized Basic Education Project simply implies
sustainability: ‘… improved participation, transition, completion, and performance in
basic education among poor children’ (p.xi).
The evaluation of the ADB/JFPR Community Based Basic Education Project, 2002 – 2006,
hints that purposes other than sustainability, such as goodwill and public relations, may
have had salience. These goals seem to be in the minds of implementation personnel if not
in formal policy – as this observation demonstrates: ‘The JFPR program in Indonesia is
considered to be of value by the country director and staff as these showcase ADB’s
concern about eradicating poverty and addressing grassroots problems. JFPR 9000: Street
Children, in particular, generated much goodwill’ (p.55).
Finally, in emergency relief projects such as the Scholarships and Grants Program and School
Improvement Grants Program, 1998 – 2002, sustainability of benefits was not a major goal at
all.
So, the answer to the question posed in this section, ‘Is sustainability of project benefits
always a project goal?’ the short answer is no; it is not. Project documents demonstrate a
range of levels of commitment to sustainability: sometimes explicit, sometimes implicit
and, in a few, none at all.
3.4 Discussion
3.4.1 Overall evaluation
Table 3 shows that 11 projects, only 12% of all 91 projects in the study, present evidence
of actually sustainable benefits.
This finding appears to contradict other donor-sponsored studies of sustainability. A 2010
ADB post-completion evaluation showing 65% of 548 of its projects in all development
sectors in Asia to be most likely or likely to be sustainable (Asian Development Bank,
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
17
2010, ii). A 2011 overall evaluation commissioned for AusAID comments: ‘Sustainability
is deemed to be “satisfactory” in two-thirds of activities (or perhaps more alarmingly
stated as “less than satisfactory” in a third of activities), and “good” or “very high
quality” in only 15% of activities’ (Bazeley, 2011, 11).
The discrepancy between the ADB and AusAID findings and those of the present study
are mostly due to differences in timing of the evaluation. The ADB and AusAID studies
tend to focus on potential sustainability findings whereas this study has sought to tease
out studies that report on actual sustainability. It also likely reflects another important
methodological difference in that both donor studies considered projects in all sectors and
all recipient countries rather than education in Indonesia exclusively as the present study
has done.
Nevertheless, all these findings indicate a substantial task after completion of enhancing
the sustainability of benefits from projects.
Consideration of all 91 projects together, spanning 46 years of development assistance,
provides lessons for future policy and practice.
The absence of sustainability considerations at design, the possible prioritisation of short-
term results over sustainability, and weak donor and beneficiary commitment to
sustainability, should not automatically lead to the assumption that addressing these
matters alone is necessary to solve the problems of sustainability. A much deeper, root
cause analysis5 could locate and assist in understanding the fundamental causes of the
problem to prevent recurrences of this weakness in projects.
Among possible lines of enquiry are the following.
First, recent attention to the cultural and political factors at work in Indonesia will likely
be of benefit in reaching this understanding (Bjork, 2005; Jessyca, 2013; Sopantini, 2014).
Second, the 2017 book about development and the World Bank by Inder Sud lays bare
the challenges of achieving effective project design and implementation from a
bureaucratic perspective. Donor-side bureaucratic, management, and political weaknesses
likely overshadow even the very best cultural, political, and technical perspectives that
educational development specialists might be able to offer to strengthen sustainability.
3.4.2 Sequential approaches to sustainability
Two matters revealed in analysing donor reports deserve specific consideration here.
These are sequential approaches and spatial sustainability.
5 Root cause analysis is a problem-solving method for identifying the fundamental source of a problem. A factor is
considered a root cause if its role in the problem–outcome sequence prevents the ultimate undesirable outcome
from recurring. A causal factor, in contrast, is one that affects an event's outcome, but is not a root cause of
problems. Removing a negative causal factor can benefit an outcome, but does not prevent its recurrence with
certainty. An example of a root cause of weak sustainability outcomes from Indonesian experience is likely to be
an underlying cultural or political factor.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
18
There is evidence in reports of some donors having managed their efforts sequentially
over time to provide on-going support to past project assistance to facilitate sustainability.
A sequential approach is where lessons learned from one project are applied in a
subsequent project and by teams of competent facilitators with relevant experience
moving from a terminating project to the next project.
USAID’s approach in Indonesia to district and school development from 2003 – 2017 has
been to build on the experience of their first project, Managing Basic Education 2003 –
2007, in their subsequent project Decentralized Basic Education, 2005 – 2011, and then on to
their recently completed PRIORITAS project, 2012 – 2017. USAID’s contractors have
also demonstrated sustainability of professional support by continuing to employ effective
project team members from one project to the next.
Another example of sequential sustainability is the World Bank’s Secondary Education and
Management Training Project, 1985 – 1990 and Second Secondary Education and Management
Project, 1990 – 1997. The Australian Education Partnership, 20011 – 2017, sequentially
supported the Islamic school system in support of earlier work in Learning Assistance
Program for Islamic Schools, 2005 – 2010.
By way of contrast, AusAID (now merged within DFAT) abandoned their demonstrably
successful regional models of development implemented in Flores and East Java, through
the Nusa Tenggara Timur Primary Education Partnership, 2002 – 2008, and the Indonesia –
Australia Partnership in Basic Education project, 2004 – 2007, respectively. Instead, they
implemented large, centrally controlled projects, the Australia – Indonesia Basic Education
Project, 2006 – 2010, and the Education Partnership with Indonesia, 2011 – 2017. Compared
with the earlier regional models of development, both big projects received less
satisfactory evaluations for their sustainability outcomes. Ignoring past success in this way
raise questions about priorities and the true purposes of development assistance.
3.4.3 Spatial sustainability: the ‘silo’ mentality in educational development
Donors commonly operate in their own project ‘silos’. This means there is limited or no
evidence in the reports of donors working collaboratively in geographically contiguous
areas, in specific professional areas such as teaching or school management, or in any
other substantive way, in mutual support of each other’s efforts to increase the likelihood
of sustainable benefits.
Donor-supported development on the island of Lombok illustrates this difficulty. The
following summary of concurrent and end-on project activity in the first decade of the
century demonstrates the diversity of donor inputs that took place at provincial, district,
and school levels.
Four donors were implementing six discrete projects in Lombok, and elsewhere in the
province of Nusa Tenggara Barat where Lombok is located. The following projects were
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
19
mostly operating in their own silos yet were overlapping, in the same province and in the
same districts with no spatial, administrative, or professional coordination or cooperation
that may have facilitated sustainable outcomes at any level.
• UNICEF/UNESCO, Creating Learning Communities for Children, 1999 – 2010.
• ADB/JFPR, Community Based Basic Education Project, 2002 – 2006.
• ADB, Decentralized Basic Education Project, 2002 – 2012.
• UNICEF/EU, Mainstreaming Good Practices in Basic Education, 2006 – 2010.
• AusAID, Australia Indonesia Basic Education Program, 2006 – 2010.
• World Bank, Basic Education Capacity, 2008 – 2012.
To what extent this diversity and quantity of inputs assisted – or confused – provincial
and district governments and schools is unknown. But there can be little doubt it
burdened local administrations and made unravelling the sources of success or failure in
achieving educational development impact and sustainability in Lombok nearly
impossible.
For reasons of accountability, donors commonly seek empirical evidence of the outcome
of their specific support to education and could be a major contributor to the silo
mentality.
3.4.4 Responsibility for sustainable outcomes and benefits
Reduced to its simplest structure, the change model assumed in the design and
implementation of many projects seems to be that the donor has the responsibility for
project funding, design, and implementation and, when the implementation phase
concludes, the beneficiaries then assume responsibility for funding, continuing, and
maintaining the outcomes of the project. In other words, it is usually assumed that
beneficiaries are responsible for sustainability and, if this responsibility is not evident, then
that is a deficiency of the beneficiaries.
The responsibility for sustainability goes beyond a simple binary matter of either donors
or beneficiaries. Both have responsibilities. In some projects, donor support is tightly
integrated with government work in implementing current policies and practices.
Examples of policies include decentralisation and the work of the ADB Decentralized Basic
Education Project, 2002 – 2012, and teacher policy issues and the World Bank’s Better
Education through Reformed Management and Universal Teacher Upgrading (BERMUTU)
project, 2008 – 2013.
Identifying responsibility for sustainability outcomes is not a straightforward matter of
looking for evidence from a single project either. Maglen and Hopkin’s (2000) analysis of
the technical education projects implemented in the years between 1980 – 1996 describes
their rare interrelationships with each other and with those of Swiss donors. These
interrelationships indicate the difficulty in attributing change to one project alone.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
20
However, apart from a few formal arrangements among donors, such as the UNICEF and
AusAID cooperation in Papua and the ADB, and European Union and Australian
cooperation in the Analytical Capacity and Development Partnership, donor cooperation is
mostly limited to the project officer/consultant level, rather than through formally
coordinated policies, agreements, or activities.
Thus, the responsibility for sustainability is a shared responsibility involving at least the
donor and the beneficiaries but also, in some contexts, several donors working together.
The lack of clarity about responsibilities and even the early and thorough consideration of
this matter is clearly a neglected matter that has contributed to the kinds of sustainability
results discussed in this Paper.
Figure 3: Sustainable change is evident in the well-maintained computer room in this primary school, East Java, supported by
USAID DBE and then by USAID PRIORITAS.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
21
4 Factors contributing to sustainability
What do the reports tell us about the strategies and activities implemented to achieve
sustainable benefits from donors’ educational development support?
In summary and considered together, the reports tell us three things.
First, project design and activities through to completion should all address sustainability.
Second, Indonesian society, particularly government, must demonstrate a commitment to
sustainability. One clear demonstration of commitment is local financing of ongoing
activities after project completion.
Third, a complex web of supporting technical matters must be addressed. The continuing
maintenance of assets of all kinds – the continuing professional development of human
resources, facilities, equipment, and materials – is a key requirement here.
Technical matters comprise organisation and management on the one hand and specific
educational matters on the other, something the reports’ analyses often neglect. This
neglect is a criticism of donors that allow this to occur; there is little analysis of
curriculum, learning, teaching methods, or assessment. Mostly absent too until recently
are considerations of employment conditions for teachers and their continuing
professional development beyond the life of a project.
These three factors contributing to sustainability do not stand-alone. Instead, they are
complex, overlapping, and mutually reinforcing, as discussed below.
4.1 Project design
Good project design can support the achievement of sustainability benefits. A frequently
occurring finding in reports is that the issue of sustainability missing in project designs.
The implication for policy and practice when providing development assistance is clear:
the design of projects should incorporate sustainability considerations.
Reports present the design consideration of sustainability in both positive and negative
ways. For example, the evaluation report of AusAID’s Indonesia Australia Partnership in
Basic Education, 2004 – 2007, comments positively on the project design for sustainability.
‘The design of IAPBE caters well for sustainability of activities, in that mechanisms are in
place to provide necessary skills and knowledge to local stakeholders. There are now
trained personnel within all the targeted schools who are working in teams and going
beyond their localities to disseminate training and information to non-targeted areas with
funding supplied from local budgets’ (p. 11).
More commonly, reports make negative observations about the absence of evidence for
the consideration of sustainability at design. This is the case with one component, the
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
22
Analytical and Capacity Development Partnership (ACDP), in the recent AusAID/DFAT
project Australia’s Education Partnership with Indonesia, 2011 – 2017.
‘Whilst institutional sustainability was not foreseen within the design of the ACDP, the
lack of clarity about any expected institutional legacy represents a missed opportunity to
generate a better return on the USD 40-45 million investment’ (p.21).
Other reports have also noted the omission of sustainability in the design. Examples
include the Marine Sciences Education Project, 1989 – 1995 and the Mathematics Education
Quality Improvement Programme, 2004 – 2007. The reports of several projects including the
Technological and Professional Skills Development Sector Project, 2001 – 2007 and the Teluk
Bintuni Basic Education Project, 2006 – 2009 omit the concept of sustainability as a
consideration altogether.
4.2 Commitment: responsibility and ownership
The concept of commitment brings together several themes, all having the commonality
of belonging substantially to Indonesian society. ‘Substantially’ is central here. It does not
suggest that donors have no role through activities such as capacity development,
planning and advice. But ultimately, responsibility for demonstrating commitment rests
with Indonesia. The following themes emerge from the analysis of commitment.
4.2.1 Financial commitments in support of sustainable change
One of the most frequently cited sustainability factors discussed in reports is financial
sustainability which relates to district and central governments taking responsibility for
providing continuing financial support for change. Reports also refer to the importance of
local community contributions and to the role of sound financial management in enabling
the sustainability of project benefits.
4.2.2 Government commitment
Government commitment means evidence that the relevant level of government supports
the continuation of benefits after project completion. Support can range from benign
acceptance through to strong leadership and legislative, financial, and regulatory support.
The recent evaluation of USAID PRIORITAS, 20012 – 2017 provides an excellent
example of a project where government commitment has been facilitated and achieved.
Government commitment can be conceptually simple such as when a local government
provides regulatory and funding support for a change. It can also be complex, for
example, when an innovation requires the understanding and active commitment of
several different central government ministries (commonly Education, Religious Affairs,
Home Affairs, and Finance), provincial governments, and district governments as well as
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
23
a range of other formal organisations reporting to them. Examples of these organisations
may be sub-district offices, local planning authorities, universities, and Institutes for the
Improvement of Educational Quality, Lembaga Peningkatan Mutu Pendidikan, or LPMP.
Positive examples are
provided in reports of what
government commitment
looks like. Examples
includes district planning
and funding for
dissemination and
sustainability reported in
the Primary Education
Quality Improvement Project,
1992 – 1999, and the
implementation of new,
government-supported
projects based on
successful donor models.
REDIP-G (Government)
and REDIP-P (Provincial)
projects based on the
original JICA supported Regional Education Development and Improvement Program, 1999 –
2008 are examples of government projects.
Commitment by government to their own projects is often weak. There is evidence of
poor implementation. Dittmer’s report on the government’s Mathematics Education Quality
Improvement Programme (MEQIP) 2004 – 2007 illustrates this matter. Based on the
successful German project, Science Education Quality Improvement Programme 1994 – 2008,
Dittmer notes questionable performance in MEQIP implementation and sustainability
with too much focus on delivery: ‘MEQIP will risk becoming a Mathematics Education
Delivery Programme without quality improvements’ (p.76).
Project reports further mention the critical importance of government commitment to
sustainability through a modicum of administrative discipline in ensuring staffing stability
and continuity in educational organisations and schools following project-led capacity
development activities. Stability is of critical importance in district education offices and
among school principals.
Finally, the reports of the Manpower Development and Training Project, 1986 – 1995 and the
Junior Secondary Education Project, 1993 – 1998 indicate the importance of government
willingness to participate in the strengthening or change of Ministry systems in order to
facilitate other planned project changes.
Figure 4: Securing local government commitment by engaging officials in capacity
analysis prior to full project implementation. UNICEF Mainstreaming Good Practices
in Basic Education project.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
24
4.2.3 Local participation and ownership
Reports refer to the following aspects of involvement and ownership: (i) local government
participation in the development process and ownership through such local institutions as
district parliaments, district education and district planning offices and formal advisory
boards; (ii) community participation, responsibility, and ownership through, for example,
school committees, parents’ direct involvement in school activities, and local business
support, and collaboration with local and regional educational organisations such as
universities and LPMP.
4.3 Technical Matters
Project reports identify numerous technical matters that support or inhibit the
sustainability of benefits arising from projects. Examples of technical matters include
policy development, learning and teaching, professional and organisational development,
school based management, financial management, planning, asset maintenance, and the
management of corruption.
Some projects reflect distinctive technical matters. The need to have high quality science
kits for the Science Education Quality Improvement Project, 1994 – 2008 and the linking of
Minimum Service Standards to existing policy in the Basic Education Sector Capacity Support
Programme Component 1: Minimum Service Standards, 2008 – 2011 are two examples of
technical matters that can support sustainability when present or inhibit sustainability
when absent. Notwithstanding the distinctive nature of the kits and policy to those
projects, both have key lessons for all projects – quality materials and policy support.
4.3.1 The missing technical matter: Where is the ‘education’ in education projects?
There is a dearth of attention to specifically educational matters related to curriculum,
learning, teaching and educational administration in sustainability. Although not entirely
absent, they are rarely at the forefront of thinking about development presented in
evaluation reports. Organisational, management, and financial matters dominate.
There is limited discussion of the place of curriculum in sustainability. This discussion
can be found in the Primary School Teacher Development Project, 1992 – 1999, the Sistem
Pembinaan Professional/Cara Belajar Siswa Aktif, 1980 – 1995 and the Engineering Education
Development Project, 1996 – 2002. Curriculum has been an anchor point in projects
designed to support the implementation of recently introduced curriculum policies such as
the suite of technical education projects implemented between 1980 and 1996 with
Australian assistance, and in the Asian Development Bank’s Junior Secondary Education
Project, 1993 – 1998.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
25
Specific elements of the curriculum can support sustainability of benefits as the report of
the ADB Vocational and Technical Education Project, 1995 – 2001, suggests. In this project,
the income-generating activities embedded in the curriculum contributed to sustainability.
In principle, project activities in support of officially sanctioned curriculum
implementation should produce sustainable benefits for as long as that curriculum
remains in place. Whether this is the case, however, does not seem to have been the focus
of any detailed analysis.
Teaching materials and equipment have received attention in several projects beginning
with the World Bank’s Third Education Project, 1973 – 1982, the ADAB/AIDAB Commerce
Polytechnics Project, 1986 – 1992, the GTZ Science Education Quality Improvement Project,
1994 – 2008, and the World Bank Book and Reading Development Project, 1995 – 2001. The
actual sustainability of benefits from these projects is evaluated as unlikely, however.
Reports regularly comment on
issues such as poor asset
management and control, the
funding of replacement books,
and corruption of
procurement arrangements.
Learning and teaching matters
often missing in the analysis
of sustainability include
teaching methods, student
learning, assessment and
evaluation, access and equity,
grade repetition, and student
transition from home to
school and between one level
of schooling to the next.
Linking these educational
factors to sustainability has not
been considered6. Three examples illustrate the possible directions that analysis might take
in future projects:
• Student learning. Understanding the ways in which students learn and the
impediments to effective learning is steadily improving and should be a point of
reference in educational project design, implementation, and sustainability of
6 This gap in knowledge in Indonesian education projects contrasts with the substantial analysis undertaken in
Papua New Guinea by Gerard Guthrie (2011) in his book The Progressive Education Fallacy in Developing
Countries, In Favour of Formalism. New York: Springer.
Figure 5: Evaluation could explore learning and teaching in greater depth to better
understand the sustainability of educational processes.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
26
change in learning behaviours. Why? Better learning should lead to better
outcomes and sustainable improvements to education. Acting upon evidence-
based strategies to improve student learning outcomes will contribute to
sustainable benefits for learners. Some of these learners will eventually become
teachers so there is the possibility of a sustainability link from learner to teacher
and back to learners in this example.
• Assessment of learning. It is well-understood that assessment of student learning is
one of the most powerful influences on the actual curriculum as implemented by
teachers and on student learning behaviour. Why, then, is such a powerful
influence of behaviour and change overlooked in analyses of sustainability? Apart
from the USAID suite of projects, MBE, DBE, PRIORITAS and UNICEF’s
MGPBE, there is scant evidence of the development and application of valid and
reliable methods of assessing student learning which is the foundation to begin
work on this matter. Relying on poor quality national examinations data is not a
constructive way forward.
• Teaching methods. Welcomed by teachers as a means of making their work more
enjoyable and rewarding, the emphasis placed on student centred active learning in
Indonesia for over 30 years has, among other benefits, created the possibility that
focusing on this method may lead to sustainable teacher behaviours and improved
student learning outcomes. Moreover, project implementation reports from
USAID’s MBE, DBE and PRIORITAS and UNICEF’s MGPBE consistently
show improved test scores in schools using active learning approaches (Cannon,
2010). Yet there is little attention to the analysis of active learning and how this
may, or may not be working to support sustainable quality improvement in
schools.
The report of the Nusa Tenggara Timur Primary Education Partnership, 2002 – 2008 has one
of the most thorough analyses of educational matters, considering among others, bottom
up and top down approaches, breadth versus depth, whole school approaches, and the
place of local languages in education.
4.3.2 Management
Technical issues with educational management usually receive more attention in most
reports than learning and teaching. The following grouping of factors relevant to
sustainability seeks to achieve a sense of cohesive order in the analysis of management
issues identified in reports.
• Institutional management. This focus includes the activities of managing the ‘big
picture’ institutional arrangements in which education occurs (Cannon, 2017, 6).
Examples of institutional arrangements include the formal relationships among
relevant organisations such as Ministries, local and provincial governments, and
the web of subordinate organisations for which they are each accountable. These
organisations include schools, district and sub-district education offices, school
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
27
clusters and teacher/principal working groups, local offices of the Ministry of
Religious Affairs, local planning offices, and universities.
In higher education, the World Bank’s Managing Higher Education for Relevance and
Efficiency Project, 2005 – 2012 is an example of institutional management. In this
case, the legal framework for higher education, the Directorate General of Higher
Education and universities were involved. Numerous projects in basic education
had a strong focus on the institutional environment. USAID’s PRIORITAS and
ADB’s Decentralized Basic Education Project both began with a focus on institutional
management.
Institutional sustainability factors identified in reports include Memoranda of
Understanding established between central government ministries and districts, the
alignment of local activities with national decentralisation policies, and the linking
of a project’s work in schools and government offices with support from local
universities. Linking project work in schools to pre-service teacher education is an
example of a Ministry – university – district – school institutional linkage, a feature
of several projects such as USAID PRIORITAS.
• Organisational management. The reports identify the following organisations, among
others, in their analyses: Ministries, universities, LPMP, schools, school clusters,
teachers’ working groups, and Central, Provincial and District governments.
Focusing on the ways of approaching the development of these organisations is an
important approach to sustainability. One example is by adopting whole-school
and whole-of-district strategies that include all school staff, local organisations,
and the local community. The UNICEF/UNESCO Creating Learning Communities
for Children project, 1999 – 2010 is an important example of this approach. This
project is important because its approach informed so many other basic education
projects7.
The Early Childhood Education and Development Project, 2011 – 2013 uses a similar
concept, a ‘holistic approach’, to building sustainability. Establishing and working
through school clusters and teacher/principal working groups are effective
organisational approaches to achieve sustainability.
• Management of educational personnel. Reports identify a cluster of personnel
management practices that can work for or against sustainability. One very early
observation was the value in linking civil servants’ careers to the success of the
project in the Pendidikan Antara Masyarakat, Orang Tua dan Guru, project, 1974 –
7 This illustrates another and complementary form of sustainability. Whereas analysis commonly focuses on the
benefits accruing to targeted groups and organisations – teachers, supervisors, schools, and district governments
– there is also the idea of the sustainability of approaches to, or models of development. CLCC is an example of a
project whose approaches have informed the work of many subsequent basic education projects of which MBE,
IAPBE, NTT-PEP, MGPBE, and MEDP are examples.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
28
1989 and using ‘high quality human resources’ to support development, noted in
the report on the Learning Assistance Program for Islamic Schools, 2005 – 2010.
Sustainable change at school level requires the active involvement of others from
the wider institutional network of potential beneficiaries including school
supervisors and teacher trainers, according to the report on the World Bank’s
Secondary Education and Management Training Project, 1985 – 1990.
Poor people-management reflected in reports include the practice of allowing
turnover of trained staff, noted in the Book and Reading Development Project, 1995 –
2001, and appointing retired principals and teachers to management roles in the
Secondary Education and Management Training Project, 1985 – 1990.
Constructive management practices include avoiding the concern identified in the
Science Education Quality Improvement Project where management neglect of the
transfer of skills to new teachers in schools weakened the prospect of sustained
change. The implementation of sound professional development strategies
including focusing on new teachers, teachers training their peers, and avoiding a
‘one shot’ approach to providing training (Cannon and Hore, 1997), a key
characteristic of much government-funded training in Indonesia. This issue was
addressed in the Australia Indonesia Basic Education Program, 2006 – 2010.
• Management of physical resources. The Engineering Education Development Project, 1996
– 2002 and the Marine Sciences Education Project, 1989 – 1995, among several others,
note the serious impediment to sustainability through the absence of mechanisms
to ensure the maintenance of physical resources in education such as equipment,
materials, and buildings. This matter represents a design failure and a commitment
failure.
• Project management. Project Implementation Units (PIUs), a special staffing
arrangement used in beneficiaries’ agencies to ‘ring-fence’ a project, have been the
subject of detailed study by the Asian Development Bank (ADB, 2005). The
development implications emerging from the study are complex but one
observation is that their semi-permanent nature can create systemic problems with
the sustainability of capacity creation. Administrative complications, as revealed in
the Marine Sciences Education Project, 1989 – 1995, where it was reported Local PIUs
reporting directly to the Central PIU created operational problems with the
universities where they were located.
• Management of learning and change processes. The professional management of
learning and change means the application of professional, evidence-based
approaches to the design and implementation of project strategies and activities.
This factor has received attention in four reports, all from Australian-supported
projects. For example, school and district development is unlikely to have a
sustained impact when implemented on ‘once-only’ basis according to the report
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
29
on the Australia Indonesia Basic Education Program, 2006 – 2010. This observation
reflects the need for repetition, an established principle of learning.
A second example concerns the widely followed approach to project
implementation known as cascade training 8. Although administratively very
convenient, cascade training has well-known limitations from an educational
perspective. Nevertheless, the successful cascade training model described in the
report on the Indonesia Australia Partnership in Basic Education, 2004 – 2007 received
endorsement, whereas the magnitude and complexity of the cascade approach
used in another Australian initiative, the Bantuan Operasional Sekolah (School
Operational Assistance) Training Program, 2011 – 2012, was less positive.
Finally, the report of the Nusa Tenggara Timur Primary Education Partnership, 2002
– 2008 has a strong analysis of learning and change processes including cascade
training, bottom up and top down approaches, breadth versus depth, whole school
approaches, and the place of local languages in education.
• Cross-cutting management strategies for sustainability. Cross-cutting strategies for
sustainability can be implemented with good effect across the major project phases
of design, implementation, and sustainability. Advocacy for sustainability is one
such strategy. Reports on the Creating Learning Communities for Children, 1999 –
2010 and the World Bank’s Basic Education Capacity, 2008 – 2012 consider
advocacy for several purposes including fostering participation, explanation, and
achieving support for sustainable funding.
Dissemination, scaling up, replication and mainstreaming are further cross-cutting
concepts noted, for example in the Primary Education Quality Improvement Project,
1992 – 1999 and the Regional Education Development and Improvement Program
(REDIP), 1999 – 2008. They were central ideas in the Mainstreaming Good Practices
in Basic Education, 2006 – 2010.
4.4 Lessons from Actual Sustainability Analysis
The discussion so far of the factors contributing to sustainability has been based on
analysis of all project reports, that is, those evaluating potential sustainability and actual
sustainability considered together. Does a re-analysis based on only the 22 reports of
actual sustainability provide different insights into what works or not?
Yes, it does. Analysis of the six projects found to be actually not sustainable emphasise
weakness and practices that provide a clear lesson for development policy and practice.
8 The principle of cascade training is that if you train a trainer to train other trainers who then train others, the
exponential multiplication of learning cuts training time. Although an attractive idea in theory, in practise there are
serious barriers to achieving desirable learning outcomes.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
30
This lesson reflects the misleading message for development outcomes embedded in that
well-worn proverb that has underpinned so much thinking, policy, and practice in
development in the past, “give a man a fish and you feed him for a day, teach a man to
fish and you feed him for a lifetime”. The lesson is common-sense, repeated so frequently
in donor reports, that if you intend the benefits of educational development to be
sustainable, then you must plan for it to happen through careful design and
implementation practices consistent with achieving sustainable benefits.
An essential element of that planning and practice is abandoning the discredited idea that
focusing teaching on “one man” will lead to a “lifetime” of benefits. This is unlikely to
happen. The evidence from Indonesian experience of one-shot training, often
implemented through cascade training, demonstrates this (AusAID’s Independent Review of
the Bantuan Operasional Sekolah (School Operational Assistance) Training Program, 2011 –
2012, Cannon, 1997, Central Independent Monitoring Unit, 1999).
Similarly, in South Asia and Africa, a 2015 DFAT review notes that “This cascade model
of training has, however, been widely shown to be ineffective, with teachers at times
experiencing difficulties sharing their knowledge with colleagues. Such findings point to
the general ineffectiveness of off-site, in-service training. Similarly, providing in-service
training to a sub-group of teachers makes change in teaching practice unlikely across a
school” (DFAT, 2015).
Repeated cycles of teaching or training, working with and through bigger social system of
peers, management, organisations, politics, financial arrangements, and responsiveness to
local culture, are some of the matters that demand constant attention for the sustainability
of benefits to have a chance. Without these, as the World Bank’s Implementation
Completion Report of the Manpower Development and Training Project (1986 – 1995) notes
“In the case of management innovations, the overall conclusion is that few of the
innovations in systems and pilot non-institutional training have survived the project's
implementation phase” (p.9).
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
31
5 Prospects
Analysis of donor reports presents a clear message. This message is that for
sustainability of benefits to occur, project design must include sustainability in all
project activities through to completion.
More specifically, this means planning for the financing of ongoing activities after
project completion. It requires commitment and a demonstrated sense of
responsibility and ownership for the continuation of benefits from Indonesian
society. Finally, attention to a complex web of supporting technical matters,
broadly classified as organisational and management and specific educational
issues focusing on learning and teaching is essential.
Learning and teaching,
however, is a gap in the
evaluation of many
education projects. This
neglect leaves donors,
beneficiaries, and
practitioners little the
wiser about the role of
these domains for
sustainability.
Finally, although donor
reports presented for
analysis cover a period of
project experience from
1971 to 2017, that is 46 years, there is limited evidence that the approaches to
project design and implementation for sustainability have changed much over that
long period. There is no evident improvement in sustainability outcomes, nor in
the quality of the evaluation of sustainability. The continuing use of inconsistent
terms to describe sustainability and sustainability outcomes confuses
understanding and thus acts as a further limitation to progress.
The most concerning results of the analysis are threefold.
First, is the very weak evidence for the sustainability of benefits from an
investment of more than USD 5 billion in educational development over almost 50
years of development assistance to Indonesian education. Second, is the absence of
any rigorous evaluation of the actual sustainability of the benefits from this
investment in educational development in Indonesia provided by any donor.
Finally, there is no evident pattern of progress towards a ‘culture’ of sustainability
of educational development’ that champions its importance. Nor is there an
Figure 6: Sustainability messages from students in North Sumatra!
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
32
emerging ‘technology’ of sustainability that comes anywhere close to reflecting the
knowledge that we do have about it, as limited as that may be.
Perhaps this analysis can make a small contribution to that technology and build
on the constructive work on sustainability produced by AusAID in 2005 in a
document that still has much practical relevance for education today (AusAID,
2005).
The conclusions presented here are, of course, based on donor reports alone. A
following Working Paper will explore other studies to determine if that body of
work contributes new insights and better understanding of the sustainability of
benefits from development assistance to Indonesian education.
The evidence provided by the donors in their reports shows that the sustainability
of the benefits of their assistance has been very limited. As the analysis here shows,
only 12% of donors’ project reports show the benefits to be actually sustainable.
This is a disturbing outcome of failure from so many years of effort and
expenditure, only compounded by the absence of serious attention to addressing
this failure.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
33
6 References
Asian Development Bank (ADB). 2005. The Role of Project Implementation Units.
Manila: Asian Development Bank. Accessed 19 November 2017.
https://www.adb.org/documents/project-implementation-units
Asian Development Bank (ADB). 2010. Post-Completion Sustainability of Asian Development
Bank-assisted Projects. Manila: Asian Development Bank. Accessed 7 May 2017.
http://www.adb.org/sites/default/files/evaluation-document/35410/files/ses-oth-2010-
46.pdf
Australian Agency for International Development (AusAID). 2005. General Guidance 6.4:
Promoting Practical Sustainability. Canberra: AusAID.
Australian Agency for International Development (AusAID). 2010. Independent
Completion Report, Learning Assistance Program for Islamic Schools (LAPIS). Jakarta:
Cardno Emerging Markets Pty Ltd.
Bazeley, P. 2011. Study of Independent Completion Reports. Canberra, Australian Agency for
International Development (AusAID). 2010.
Bjork, C. 2005. Indonesian Education: Teachers Schools and Central Bureaucracy. New York:
Routledge.
Cannon, R. 2010. Impact Evaluation Study: A Study of the Impact of the UNICEF MGP-BE
Project on Participating Districts, Communities, and Schools in Indonesia. Accessed 15 Sep 2017.
https://www.researchgate.net/publication/313990914_Impact_Evaluation_Study_A_Study_of_t
he_Impact_of_the_UNICEF_MGP-
BE_Project_onParticipating_Districts_Communities_and_Schools_in_Indonesi
Cannon, R. 2017. The Challenging Concept of Sustainability in Educational Development in
Indonesia. Sustainability of Educational Development in Indonesia Project: Working
Paper #1. Accessed 15 Sep 2017.
https://www.researchgate.net/publication/316997532_The_Challenging_Concept_of_Su
stainability_in_Educational_Development_in_Indonesia_Sustainability_of_Educational_
Development_in_Indonesia_Project_Working_Paper_1
Cannon, R. and R. Arlianti. 2008. Review of Education Development Models for Increasing
Access to Quality Basic Education in Indonesia. Jakarta: The World Bank. Accessed 6 May
2017.
https://www.researchgate.net/publication/304258318_Review_of_Education_Developm
ent_Models_for_Increasing_Access_To_Quality_Basic_Education_in_Indonesia
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
34
Cannon, R. and T. Hore. 1997. “The Long-Term Effects of ‘One-Shot’ Professional
Development Courses: An Indonesian Case Study.” International Journal for Academic
Development 2 (1): 35–42.
Central Independent Monitoring Unit. 1999. A Study of the 1999/2000 Cascade Training for
Committees. Jakarta: The British Council.
Department of Foreign Affairs and Trade. 2015. Supporting Teacher Development: Literature
Review. Canberra: Office of Development Effectiveness, Department of Foreign Affairs
and Trade.
Guthrie, G. 2011. The Progressive Education Fallacy in Developing Countries, In Favour of
Formalism. New York: Springer.
Inder Sud. 2017. Reforming Foreign Aid: Reinvent the World Bank. Kindle Books.
Jessyca, V. 2013. AusAID and Capacity Building: Sustainability of an Education Aid Project.
M.A. diss., Deakin University.
Maglen, L. and Hopkins, S. 2000. Stepping Offshore: An Examination of Australia’s Bilateral
Program-based Assistance for the Development of Vocational Education and Training in its Region.
Melbourne: The Centre for the Economics of Education and Training, Monash
University.
Sanderson, G. 2009. Exit Sustainability Report for the Learning Assistance Program for Islamic
Schools, 2005 – 2010. Jakarta: Learning Assistance Program for Islamic Schools.
Sopantini 2014. A Case Study of the Implementation of Active Learning in Primary Schools in
North Maluku. D.Ed. diss., University of Tasmania.
Appendix: Summary of Evidence from Donor Reports of the
Sustainability of Benefits from Educational Development Projects in
Indonesia, 1971 – 2017
Notes:
* Denotes a project evaluated two or more years after project completion. These projects are included in the analysis of actual
sustainability. See section 3.2 for a full explanation of actual vs potential sustainability.
# Denotes an evaluation of sustainability based on a specified analytical and/or quantitative approach in contrast to a general or global
judgement expressed in a report.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
36
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
First Education Project, 1971 – n/a.
World Bank
$4.6m
http://projects.worldbank.org/P003715/educa
tion-project?lang=en&tab=overview
(a) Construction and equipping of
technical training centres; (b)
training 330 teachers to staff the
centres; and (c) technical
assistance in the initial operation
and supervision of the centres.
n/a
1. Satisfactory
2. n/a
Second Education (Agricultural Education)
Project, implementation dates n/a.
World Bank
Cost: n/a
http://documents.worldbank.org/curated/en/
docsearch/projects/P003721
Establish Agency for Agricultural
Education and Training; equip
training centres; construct,
rehabilitate and equip agricultural
senior secondary schools and
forestry technician training
centres; provide technical
assistance and fellowships.
n/a
n/a
Third Education Project, 1973 – 1982.
$104m
World Bank
PCR, June 1983.
Improve primary education
through textbooks, teachers’
guides, teacher training and
supervision.
None.
1. Quantitatively, yes;
qualitatively difficult to
determine.
2. Not mentioned.
First Textbook Project, 1974 – 1980 (?)
World Bank
$?
n/a
n/a
n/a
n/a
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
38
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Pendidikan Antara Masyarakat, Orang Tua
dan Guru (PAMONG), 1974 – 1989.
Cost: $3m
IDRC, Canada; USAID; UNICEF.
Study, 1986. Source:
https://archive.org/stream/ERIC_ED312084/
ERIC_ED312084_djvu.txt
Explore the replacement of
conventional approaches with
programed instruction, student
and community volunteers, and
instructional supervisors.
Refers to institutionalisation.
Strong institutional
foundation at inception in
university support and at
Ministry. Civil servants’
careers linked to success of
project.
1. Partly
2. It will be some time before
the full impact can be
determined. Sustainability not
specifically mentioned.
Fourth Education Project, 1976 – 1986 (?)
World Bank
$37m (estimate)
Sources inaccessible from World Bank site.
n/a
n/a
n/a
Teacher Training Project, 1978 – 1983 (?)
World Bank
$43m (estimate)
Sources inaccessible from World Bank site.
n/a
n/a
n/a
Non Formal Education Project, 1978 – 1983 (?)
World Bank
$15m (estimate)
Sources inaccessible from World Bank site.
n/a
n/a
n/a
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
39
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Polytechnic Project, 1979 – 1989 (?)
World Bank
$104m (estimate)
Sources inaccessible from World Bank site.
n/a
n/a
n/a
Second Agricultural Training Project, 1980 –
1987 (?)
World Bank
$71m (estimate)
Sources inaccessible from World Bank site.
n/a
n/a
n/a
* Sistem Pembinaan Professional/Cara Belajar
Siswa Actif (SPP/CBSA), 1980 – 1995.
£2m
DFID
ER, Jan 01 (DFID Evaluation Report EV629).
Enhance the quality of primary
schools through active learning
techniques and teacher support
based on teachers’ groups and
clustered schools.
1. Must be considered at
design. 2. Helped to
implement curriculum and
PEQIP. 3. Institutional
framework did not support
sustainability.
1. Partially successful/largely
unsuccessful
2. Partially successful/largely
unsuccessful
* Indonesian Australian Technical Education
Project, 1980 – 1989.
AUD 33m
ADAB/AIDAB
Source: Maglen, L. and Hopkins, S. 2000.
Contribute to an improvement in
the skills of teaching staff;
increase the number of qualified
teachers available, and to improve
curricula and management
practices.
Indonesia has progressed
with Australia’s assistance.
Changes have strengthened
ability to deliver better
services.
1. Successful
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
40
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
University Development Program, 1981 – 1990
(?).
World Bank
$45m (estimate)
Sources inaccessible from World Bank site.
n/a
n/a
n/a
Second Teacher Training Project, 1982 – 1990.
World Bank
$121m
PCR, December, 1991.
Improve the quality of primary
and secondary teacher education.
Integration of quality
improvement into
Ministry’s activities;
sustainability ensured by
meetings with officials; need
to identify, at appraisal,
resources for sustainability.
1. Successful
2. Uncertain
Second Textbook Project, 1983 – 1989.
World Bank
$25m
PCR, November, 1990.
Improve the quality of book
content and physical standards, as
well as develop institutional
capacity for long-term textbook
publishing and distribution
Sustainability achievements
will depend on formulation
of textbook policies.
1. Partly successful
2. Uncertain
Second Non Formal Education Project, 1984 –
1990.
World Bank
$86m
PCR, September 1991.
Support production and
distribution of learning materials,
institutional development, staff
training, and investments in
equipment and infrastructure,
carry out applied research.
Administrative functions
well established, including
employment of supervisors;
produces teaching materials.
1. Successful
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
41
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Secondary Education and Management
Training Project, 1985 – 1990.
$77m
World Bank
PCR, June1992.
Improve the quality of general
secondary education and
management skills at senior
levels.
1. Ministry changing project
staff a serious challenge;
poor management in
appointing retired staff to
PMUs. 2. Sustainability
requires supervisors and
teacher trainers taking part.
1. Positive impact
2. Likely, especially with
follow-on project.
* University of Sriwijaya Project, 1985 – 1991.
$63m
ADB
PPAR, December 1997.
Improve didactic and scientific
quality in four disciplines.
Two paragraphs only on
financial and physical
sustainability.
1. Generally sustainable
2. Financial, yes; physical
questionable.
Second University Development Project, 1985
– 1993.
$147m
World Bank
PCR, October, 1994.
To develop Indonesia's capacity
to train its own university teachers
and researchers
Complex administrative
arrangements in institution
building make sustainability
unlikely.
1. Partly successful
2. Uncertain
Commerce Polytechnics Project, 1986 – 1992.
ADAB/AIDAB
AUD18m
Source: As for Indonesian Australian Technical
Education Project, above.
Assist in establishment of a
system of commerce polytechnics,
teacher education, curriculum and
materials development,
management at system level and
in polytechnics.
Established business
education in polytechnic
system. At completion, the
nine polytechnics were
operating successfully.
1. Successful
2. Project managers considered
commerce programs did not
have the capacity to sustain
themselves.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
42
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* Manpower Development and Training
Project, 1986 – 1995.
World Bank
$54m
ICR, March, 1996.
To improve the competitiveness
and productivity of industry
through improved skills of
industrial manpower.
Few innovations survived
implementation phase;
Ministry’s procedures
remain same as before
project; poor project
supervision.
1. Unsuccessful
2. No.
Indonesia Australia Technical and Vocational
Education Project A, 1987 – 1996.
AIDAB/AusAID
AUD22m
Source: Maglen, L. and Hopkins, S. 2000.
Contribute to the development of
a network of resource school
clusters for integrated senior
secondary school improvement.
See remarks for Indonesian
Australian Technical
Education Project.
1. IATEP A successful in
improving the quality of
resource schools.
2. Unknown.
Accountancy Development Project, 1988 –
1995.
World Bank
$103m
ICR, January, 1996.
Improve accounting practices and
support the Government's
program to raise the quality of
accounting faculty and teaching
staff, and to prepare for future
expansion of accounting
education and training.
Government commitment;
follow-on second project;
strong demand for
accounting education in
universities; maintenance
issues with school
equipment
1. Satisfactory.
2. Likely
Higher Education Dev. Project, 1988 – 1993.
World Bank
$133m
PCR, June 1995.
Improve planning and
management; increase resources;
improve quality and efficiency;
increase contribution of
universities to development.
Increased resources,
government commitment
uncertain, budgets for
maintenance a serious issue.
1. Satisfactory but “mixed
signals”.
2. Likely (Contradicts body of
the report).
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
43
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* # Marine Sciences Education Project, 1989 –
1995.
$76m
ADB PPAR, July 2000.
Establish marine sciences
program; improve services;
develop research capacity;
upgrade and construct facilities.
1. Neither design nor
implementation gave
sufficient attention to
sustainability. 2. No
evidence of any planning for
sustainability.
1. Partly successful.
2. No, serious limitations.
Professional Human Resource Development
Project, 1990 – 1995.
$159m
World Bank
ICR, December, 1995.
Improve human resource policy
and planning, quality of all staff,
the quality of scientific and
technological research and
development, and the application
of research and technology to
development.
Policy changes will improve
chances of returnees making
an impact and improve the
quality of civil service
performance.
1. Successful.
2. Likely
* Indonesia Australia Technical and Vocational
Education Project B, 1990 – 1996.
AIDAB/AusAID
AUD16m. Source: As for Indonesian
Australian Technical Education Project, above.
Create system-wide human
resource development plan;
improve Directorate’s
management capacity and
capacity for school and regional
in-service activity.
System strengthening to
achieve sustainability; main
strategy was advice and
formal personnel
development, provide
management technology.
1. Very successful
2. Yes.
Second Secondary Education and Management
Project, 1990 – 1997.
World Bank
$137m
ICR, June 1998.
Improve the quality of teachers
focusing on teaching-learning
processes; strengthen education
management capacity.
Government commitment,
and strong human resources
are positives but high
financial risk due to Asian
financial crisis.
1. Satisfactory
2. Yes; reflected in human
resources and translation into
better design for the follow-on
junior secondary education
projects in eleven provinces.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
44
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Six Universities Development and
Rehabilitation Project, 1990 – 1997.
ADB
$114m PCR, December 1998.
Improve quantity and quality of
undergraduates; support 11
private universities adjacent to
project universities.
1. Sustainability will require
universities to address
project weaknesses. 2.
Sustainability requires
attention at design.
1. Partly successful
2. Not likely.
Second Indonesian Polytechnic Project, 1991 –
1997.
AIDAB/AusAID
Source: As for Indonesian Australian Technical
Education Project, above.
Continuation of Commerce
Polytechnics Project. Improve
quality of polytechnic graduates,
extend programs to eastern
Indonesia. Revised to concentrate
on planning and management.
As the Polytechnic Ed. Dev.
Centre had met demand for
trained teachers, most staff
were transferred out.
Implementation problems.
1. Partly
2. Unknown
*Agricultural Technology Schools Project,
1991 – 1997.
ADB
$87m
PPAR, December, 2001.
Improve agricultural technology
education at upper secondary
level; provide equitable access to
quality education, especially on
outer islands; develop
entrepreneurial skills of teachers
and students.
Sustainability of project
schools relies on continued
government budgetary
support; funding for
operations and maintenance
problematic.
1. Successful
2. Likely
Primary School Teacher Development Project,
1992 – 1999.
$37m
World Bank
ICR, June 2000.
Produce an adequate number of
well-trained primary school
teachers adequately distributed.
1. New curriculum
implemented by training
institutions. 2. Government
commitment. 3. Aligned
with policies.
1. Satisfactory
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
45
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Primary Education Quality Improvement
Project, 1992 – 1999.
World Bank
$28m
ICR, April 2000.
Introduce policies and mechanism
to improve primary education
quality aimed at raising student
achievement and completion
rates.
1. Not financially
sustainable 2. Community
participation and school
clusters helps sustainability.
3. Trained staff, physical
facilities. 4. Better design to
ensure sustainability 6. Each
Province has sustainability
and dissemination plans. 7.
Decentralization laws.
1. Satisfactory
2. Uncertain
Third Non Formal Education Project, 1992 –
1999
World Bank
$65m
ICR, May 2000.
Improve the human capital of the
poor, through basic education and
income-generating skills. Assist
Directorate for Community
Education to address literacy and
in restructuring to support
income-generation and vocational
training.
Strong government
commitment; negative
impact of financial crisis;
conflict with
decentralization policies and
expansion of junior
secondary education.
1.Partly successful
2. Uncertain
* # Junior Secondary Education Project, 1993
– 1998.
ADB,
$175m,
PPAR, November 2002.
Implement Curriculum 1994;
improve academic supervision
and student evaluation; strengthen
Ministries to ensure sustainability
of improvements achieved.
1. Strengthening Ministries;
teacher professional
development; budgets;
community participation;
support for next projects.
1.Successful
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
46
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Skills Development Project, 1994 – 1998
World Bank
$15m
ICR, June 1999.
Develop and implement
innovative industrial skills
training schemes within private
enterprises in three provinces,
support a special training program
for unemployed university
graduates in five provinces,
strengthen policy, research, and
management capabilities of the
Ministry of Manpower.
Strong government
commitment, financial risk
to sustainability especially
in Provinces.
1. Satisfactory (from mid-point
only)
2. Ministry component: Likely,
Provincial component:
Uncertain.
Higher Education Project, 1994 – 2000.
ADB
$162m
PCR, December 2001.
Nine regional universities
development; networking;
management development.
1. Beneficiaries should be
involved in project design
and implementation. 2.
Address sustainability in
project design.
1. Successful
2. No conclusion stated on
sustainability.
University Research for Graduate Education,
1994 – 2001.
World Bank
$57m
ICR, December, 2001.
Improve the quality of graduate
education; increase funding for
domestic graduate education and
research; integrate research with
graduate training; strengthen
research capacity and
dissemination of research
findings; attract highly qualified
candidates for domestic graduate
education.
Peer review system well
established; collaborative
research activities positive;
financial difficulties;
requires Directorate to
develop overall strategy for
sustainability.
1. Satisfactory
2. Mixed
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
47
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* Science Education Quality Improvement
Project (SEQIP), 1994 – 2008
GTZ
$40m
Ex-post evaluation 2012/13 Report, 2013.
Modernising science teaching at
primary education levels grades 4
to 6.
1. Good quality science kits.
2. Restocking spare parts a
failure. 3. Transfer of skills
unreliable. 4. Development
of new project based on
SEQIP: MEQIP.
1. Satisfactory
2. Satisfactory
*Junior Secondary School Building
Construction Project, 1995 – 2000.
JICA
¥12m
Junior Secondary School Building
Construction Project Report, January 2003.
Source:
https://www.jica.go.jp/english/our_work/eva
luation/oda_loan/post/2003/pdf/2-15_full.pdf
To construct junior secondary
school buildings, furniture, and
equipment as a model project in
12 provinces.
Policies of decentralization
seek to establish sustainable
school operation and
maintenance organizations
with school-based
management and
community participation.
1. Generally successful
2. Qualitative evaluation in
2002 of sites and quality of
construction and facilities
against five criteria reveal two
thirds of schools in
unsatisfactory condition #.
* Book and Reading Development Project,
1995 – 2001.
World Bank
$133m
ICR, June 2004.
1. Improve basic education
through producing quality texts,
increase availability and use by
teachers. 2. Decentralise and
privatise textbook functions.
1. Book procurement
guidelines not yet in place.
2. Book Centre continues. 3.
Skilled project staff
redeployed to other
functions.
1.Unsatisfactory
2. Modest
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
48
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Vocational and Technical Education Project,
1995 – 2001.
ADB
$92m PCR, June 2002.
Improve the quality, relevance,
and external efficiency of
vocational and technical
education.
Sustainability considered in
project design: promoting
income-generating activities
in schools, local industry
and the private sector are
innovative design features.
1.Successful
2. Likely
Senior Secondary Education Project, 1995 –
2002.
ADB
$116m PCR, March, 2003.
Improve quality of senior
education; address imbalances in
provision of educational facilities
and resources; improve Ministry
capacity.
Government commitment,
enhancing school-based
management effectiveness
to ensure sustainability at
school level.
1.Successful
2. Likely
Indonesia Australia Specialised Training
Project Phase I
AusAID
Source:
http://trove.nla.gov.au/work/32010702?q=In
donesia+Australia+Specialised+Training+Proj
ect+andc=bookandversionId=38872398
n/a
n/a
n/a
Indonesia Australia Specialised Training
Project Phase II, 1995 – 2003.
AusAID
AUD51m
MTR, June 2001.
Improve staff skills in line with
government priorities and
participants’ needs through a
specialised short-term training
program.
1. Some agencies adopting
the IASTP model. 2.
Training alone does not lead
to institutional change;
requires commitment by all
participating agencies.
1. Yes, “impressive”.
2. Indications that the design,
implementation, and M&E are
contributing to sustainable
outcomes.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
49
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Indonesia Australia Specialised Training
Project Phase III.
AusAID
(no data available)
n/a
n/a
n/a
Contextual Teaching and Learning (CTL),
1996 – ?
Ministry of National Education
No records of this project in Ministry archives.
Started as a pilot project in 1996.
Term ‘CTL’ is now adopted as an
approach in teaching and learning
in junior secondary schools.
n/a
n/a
Secondary School Teacher Development
Project, 1996 – 2001.
World Bank
$60m
ICR, Jun 2003.
Enhance teaching in secondary
schools through the improvement
of teacher education.
1. Planning for
sustainability in last two
years. 2. New project
supports institutionalisation.
3. Institutionalisation where
programs covered by
institutions’ budgets.
1. Satisfactory
2. Likely
* Engineering Education Development Project,
1996 – 2002.
ADB
$120m
PER, December 2009.
Improve the quality of
engineering education; enhance
the access of disadvantaged
students.
No budget for maintenance;
civil works design did take
account of earthquake prone
areas; planning for
improving curricula, staff,
and networking missing.
1. Successful
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
50
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Basic Education Project, 1996 – 2002.
ADB
$105m
PCR, September 2003.
Improve standards of Islamic
basic education schools and
management of institutions under
Ministry of Religious Affairs.
Concern about the
sustainability of project
resource centres as routine
budget not institutionalised
and not in design.
1. Successful (notwithstanding
listed delays/problems)
2. Likely
* Private Junior Secondary Education Project,
1996 – 2002.
ADB
$55m
PCR, July 2004.
Improve quality and sustainability
of private junior secondary
education (JSE); enhance access
of disadvantaged groups; upgrade
schools and pesantren; strengthen
institutional framework.
Sustainability undermined
by the financial crisis, which
reduced revenue generation;
optimistic vision for Local
Education Centres
1. Successful
2. Limited sustainability
Higher Education Support Project:
Development of Undergraduate Education,
1996 – 2002.
World Bank
$65m ICR, June, 2003.
Improve the quality of
undergraduate education, the
efficiency of the educational
process, and the relevance of
study programs offered.
Success of the project, the
policy and financial
environment now in place;
universities commitment to
institutionalize innovations
made under the project.
1. Satisfactory
2. Likely
Central Indonesia Secondary Education, 1996 –
2004.
World Bank
$89m
PPAR, June 2005.
Improve access, quality, and
management of junior secondary
education.
Whole-school strategy and
community involvement;
government policies;
uncertain budgets; MoU’s
between Ministry and
districts.
1. Satisfactory
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
51
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
East Java and East Nusa Tenggara Junior
Secondary Education Project, 1996 – 2004.
World Bank
$87m
PPAR, June 2006.
Improve access, quality, and
management of junior secondary
education.
The whole-school strategy
to ensure sustainability of
project inputs through
community involvement
proved effective.
1. Satisfactory
2. Likely
Sumatra Junior Secondary Education Project,
1996 – 2004.
World Bank
$92m
PPAR, June 2006.
Improve access, quality, and
management of junior secondary
education.
The whole-school strategy
to ensure sustainability of
project inputs through
community involvement
proved effective.
1. Satisfactory
2. Likely
Quality of Undergraduate Education, 1997 –
2004.
World Bank
$68m ICR, December, 2004.
Improve the quality of
undergraduate study programs in
national priority fields.
Capacity to replace
equipment; “ownership” of
procurement; project
investments increased
capacity to raise income.
1. Satisfactory
2. Likely
Development of Madrasah Aliyahs Project,
1997 – 2004.
ADB
$150m
CR, December, 2005.
Improve the quality of, and access
to, senior secondary schools,
known as Madrasah Aliyahs
(MA); strengthen the supporting
institutional capacity.
Design needs to address the
sustainability of human
capacity investment;
increased funding for MAs
helpful. Organisational
sustainability of resource
centres a concern.
1. Successful
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
52
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Scholarships and Grants Program (SGP) and
School Improvement Grants Program (SIGP),
1998 – 2002.
Cost: SGP n/a; SIGP USD29m
British Council report, June 2000.
Sources: The Impact of the Scholarships and
Grants Program, June 2000 (British Council);
A Summary of CIMU Monitoring Findings on
the School Improvement Grants Program,
CIMU, Nov 2002.
Monetary crisis relief program to
assist in keeping children in
school.
Nature of this emergency
program makes
sustainability less relevant.
1. Successful
2. Not discussed.
Early Childhood Development Project, 1998 –
2006.
World Bank
$22m ICRR, July 2007.
Protect children six months – two
years against malnutrition;
improve access to and quality of
Borrower’s ECD programs.
Evidence of sustainability
and replication of the
program; districts agree to
continue finance and
maintenance.
1. Moderately satisfactory
2. Risk to development
outcome: negligible.
Second Junior Secondary Education Project,
1998 – 2004.
ADB
$171m
PCR, July 2005.
Improve JSE enrolments and
transition rates from primary
school.
Government funds used to
continue programs;
Recruitment of remote high
school graduates a failure;
SBM now policy; threat
from lack of local funds for
school expenses. 5.
Complied with Loan
Covenant for sustainability.
1. Highly successful
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
53
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
West Java Basic Education Project, 1998 –
2004.
World Bank
$104m
ICR, June 2005.
Improve quality and expand
access to junior secondary
education in districts. Improve
capacity, planning and
management at central, provincial
and district levels.
Project components
designed for sustainability;
Policies and commitment. 3.
Project institutions and
project activities fully
funded from budgets.
1. Satisfactory
2. Likely
Sulawesi and Eastern Islands Basic Education
Project, 1999 – 2006.
World Bank
$61m
ICRR, April 2007.
Mitigate the effects of the
economic crisis on basic
education; support return to
strategy of improving quality and
decentralized management.
Optimistic about potential
sustainability at the central
level: project models now
adopted as national policy
for centrally funded
activities; district potential
for sustainability is mixed.
1. Satisfactory
2. Likely
Sumatra Basic Education Project, 1999 – 2006.
World Bank
$74m
ICRR, January 2007.
Mitigate the effect of the
economic crisis on basic
education enrolments and
operation of primary and junior
secondary schools by:
Maintaining enrolment rates and
transition rates, preventing quality
from deteriorating further.
Government has consistent
commitment to
sustainability in education;
all project institutions and
project activities fully
funded from local or
national budgets; many
project models now adopted
as national policy for
centrally funded activities.
1. Satisfactory
2. Likely
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
54
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* Regional Education Development and
Improvement Program (REDIP), 1999 – 2008.
JICA
$8m
Ex-post Evaluation, December 2012.
Improve junior secondary
education in target districts
quantitatively and qualitatively.
Dissemination to non-target
districts; evolution of two
new projects: REDIP-G
(Government) and REDIP-
P (Provincial); financially
sustainable in districts;
limited participation for
school planning;
sustainability should be
evaluated at design.
1. Partially satisfactory
2. Fair
Creating Learning Communities for Children,
1999 – 2010.
UNICEF/UNESCO/ NZAID/AusAID
$10m
Final Evaluation, May 2010.
Improve the quality of primary
schools through School Based
Management, Active, Joyful and
Effective Learning, Community
Participation.
Training for supervisors;
introduction of SBM into
pre-service teacher
education; advocacy for
sustainability.
1.Successful
2. Sustainable
Capacity Building for Decentralized Social
Services Delivery, 2000 – 2003
ADB
$0.9m
Technical Assistance Completion Report, 2004.
Develop, test, and evaluate model
for school budgeting; test a
training program for school
principals in budget planning and
management; evaluate outcomes
of test runs; inform policy makers
about results.
‘Sustainability’ not
specifically discussed;
however, overall evaluation
of TA points to
sustainability of concepts
and practices in subsequent
project.
1. Successful
2. Yes; input for decisions on
decentralized education and a
positive influence on following
Decentralized Basic Education
Project (DBEP).
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
55
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* Technological and Professional Skills
Development Sector Project, 2001 – 2007.
ADB
$25m
PER, November 2011.
Increased efficiency and
effectiveness of higher education
graduates through better quality,
capacity, and relevance of priority
disciplines in public and private
higher education system.
Sustainability as an
evaluative category omitted;
institutions have sustained
and replicated activities;
sustainability sources:
maintain budgets; strong
capacity to write funding
proposals; ownership;
financial resources from
GoI.
1. Successful
2. Likely sustainable
Library Development Project, 2002 – 2006
$4m
World Bank
ICRR, Jan 2007.
1. Increase use of reading in
community and school libraries;
2. Motivate
students/communities to read. 3.
Develop strategy for community
& school library support in
districts.
Insufficient evidence to that
government has developed
strategy to institutionalise
innovations; third objective
focused on sustainability
strategies.
1. Moderately satisfactory
2. Not known; sustainability
only incidentally discussed.
Nusa Tenggara Timur Primary Education
Partnership (NTT-PEP), 2002 – 2008.
AusAID
AUD 27m
ICR, March 2008. (NB. ICR rated in AusAID
review as among the ‘best of the best’.)
Improve education service
delivery in districts in NTT.
Implemented Sustainability
Strategy; activities ‘owned’
by districts, each has
budgeted Sustainability
Plan; increased funding and
support for
teacher/principal working
groups; political will for
ongoing activities.
1. Yes: significant
improvement in in children’s
achievements in literacy and
numeracy, in school
governance and in gender
mainstreaming.
2. Very good prospects.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
56
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* Decentralized Basic Education Project, 2002
– 2012.
ADB
$168m
PER, November 2014.
Support achieving universal basic
education serving the needs of
poor and marginalized people and
responsive to local needs.
Improved local governance
capacity; teachers train
others; continued
community involvement;
continued local funding;
construction maintenance
likely unsustainable; high
staff turnover a
sustainability issue.
1. Successful
2. Likely sustainable
Community Based Basic Education Project,
2002 – 2006.
ADB/Japan Fund for Poverty Reduction
(JFPR)
$3m
Special Evaluation Study, August 2007.
Intended to develop systems to be
implemented under the
Decentralized Basic Education
Project. However, the two
projects were implemented in
parallel and the pilot function of
JFPR was therefore not relevant.
Processes are integral to
government policy and are
therefore highly sustainable.
Schools renovated are not
maintained.
1. Successful
2. Most likely
Managing Basic Education, 2003 – 2007.
USAID
$11m
Final Evaluation, March 2007.
Strengthen district and school-
based management, community
participation and in-service
teacher training.
Project successfully built in
whole school approach;
strong facilitators; school
planning ensures
sustainability; sustainability
integrated with other issues.
1.Impressive impacts on
district and school
management, community
involvement and teaching and
learning. 2. Sustainability
assured.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
57
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Indonesia Australia Partnership in Basic
Education (IAPBE), 2004 – 2007.
AusAID
AUD 9m
ICR, July 2007.
Improve the capacity of districts
to manage basic education;
Improve the capacity of schools to
deliver basic education services.
Strong ownership; project
‘grown its own people’;
unique cascade model
enhanced sustainability;
designed for sustainability;
sustainability depends on
funding for local support;
local laws support
sustainability.
1. Successful
2. Sustainability assessment
carefully broken down by
eleven project components:
Highly likely = 4 components
Likely = 4 components
Somewhat likely = 3
components.
Mathematics Education Quality Improvement
Programme (MEQIP), 2004 – 2007.
Ministry of National Education.
Total budget not known.
Source: Dittmar, F. 2007. Evaluation of Approach
and Implementation of MEQIP in Comparison to
Approach and Achievements of SEQIP. Deutsche
Gesellschaft für Technische Zusammenarbeit
(GTZ).
To achieve quality improvements
in teaching and learning
mathematics and to make use of
the SEQIP approach as model.
No sustainability criteria
addressed in project;
MEQIP decision makers
and implementers need to
address the issue of
sustainability of project’s
design and implementation.
Questionable performance in
implementation and
sustainability: too much focus
on delivery and not on quality:
‘MEQIP will risk becoming a
Mathematics Education
Delivery Programme without
quality improvements.’ (p.76)
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
58
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Learning Assistance Program for Islamic
Schools, 2005 – 2010.
AusAID
AUD 34m
ICR July 2010.
To contribute to improved quality
of basic education in Islamic
schools.
High quality resources;
quality of human capital;
strengthened structures and
capacities; local ownership
and commitment; Program
positioned outside
institutional structures could
erode sustainability; no
mechanisms to ensure
maintenance and
development of project
inputs; activities likely to
require additional support.
1. Successful
2. Less than adequate.
Decentralized Basic Education (DBE), 2005 –
2011.
USAID
$110m
Final Evaluation, October 2012.
(Project implemented through three semi-
autonomous packages: DBE1, DBE2, DBE3.)
Improve school management and
governance and the quality and
relevance of education in primary
and junior secondary schools.
Sustainability well
integrated with other issues
in this report; project
sustainability – this Report
links forward to next
USAID project:
PRIORITAS.
Sustainability requires equal
GoI commitment,
communication, and
resources.
1. Successful
2. ‘The Fade Factor’ is an early
warning indicator of threat to
sustainability.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
59
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* Managing Higher Education for Relevance
and Efficiency Project, 2005 – 2012.
World Bank
$115m,
PPAR, June 2015.
Enable the evolution of
autonomous and accountable
institutions; Develop mechanisms
for the improvement of quality,
relevance, efficiency, and equity.
The term ‘sustainability’ is
not used in this report.
Risk to development
outcome rated as significant
because of uncertainties in
the political process.
1. Moderately satisfactory
2. Risk to development
outcome: significant
Teluk Bintuni Basic Education Project, 2006 –
2009.
British Council
$n/a
Final Evaluation, November 2009.
Improve governance and quality
in basic education throughout the
District.
Stakeholder workshop
recommended to give
attention to sustaining gains
achieved; next phase to
consider how in-service
training can be made
sustainable.
1. Successful
2. Not known; sustainability
only incidentally discussed.
*# Australia Indonesia Basic Education
Program, 2006 – 2011.
AusAID/DFAT
AUD 395m
AIBEP Schools Survey, 2014.
Improved equitable access to
higher quality and better-governed
basic education services.
Physical infrastructure
maintenance; sustainability
of enrolments in AIBEP
schools, teacher
performance, district
management.
Presents sustainability as
economic concept as in:
“…‘return’ on DFAT’s
investment in Indonesia’s
JSE sector over the medium
to long term” (p.8).
1. Yes, generally positive
2. Mixed: concerns over the
sustainability of physical
infrastructure (esp. toilets);
enrolments improved; school-
based management and
community-participation
continue; concerns with
teacher performance e.g.,
absenteeism), patchy
supervisor-school relationships.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
60
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Mainstreaming Good Practices in Basic
Education, 2006 – 2010. (Basic Education
Sector Capacity Support Programme (BE-
SCSP) Component 2)
EU/UNICEF
$15m
Final Evaluation, December 2010.
Enhanced District, sub-District
and school capacity to govern,
manage and implement basic
education services.
Absence of continuing
support; rapid turnover of
district official and
principals; mainstreaming is
ultimate test of
sustainability; local
universities and LPMP
likely to provide support.
1. Successful
2. Sustainable
Integrated Plan for Junior Secondary
Education Improvement in South Sulawesi
Province in the Republic of Indonesia (PRIMA
Pendidikan), 2007 – 2010.
JICA USD n/a
A model for integrated Junior
Secondary Education
improvement in terms of quantity,
quality and management
developed in target districts.
n/a
n/a
Basic Education Sector Capacity Support
Programme (BE-SCSP) Component 1:
Minimum Service Standards, 2008 – 2011.
ADB
$5m
Final Evaluation, December 2010.
Finalizing basic education
Minimum Service Standards
(MSS) in accordance with policy
guidelines of the Ministry of
Home Affairs.
Sustainable: MSS is an
irreversible process backed
by Ministerial Regulation;
integration of MSS into
current and future training
programs.
1. Successful
2. Sustainable
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
61
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Building Relationships through Intercultural
Dialogue and Growing Engagement
(BRIDGE), 2008 – 2011.
AusAID
$ n/a
Independent Review, Dita Nugroho and
Adrian Beavis, Australian Council for
Educational Research, 2011.
To increase teachers’ and
students’ knowledge and
understanding of contemporary
Australia and Indonesia including
the role of Islam in contemporary
Indonesian society; to support
foreign language learning in
Indonesian and Australian
schools; and to support a small
cohort of schools to acquire
internet technologies.
Not enough opportunity to
sustain relationships; risks
of teacher mobility to
established partnerships;
partnerships strong where
leaders are active;
importance of recurring
face-to-face interactions to
assist in sustaining the
overall partnership;
difficulty of students
maintaining links; ongoing
financial support.
1. Successful
2. Uncertain
Basic Education Capacity, 2008 – 2012.
World Bank
$20m ICR, June 2013.
Support good governance and
delivery of decentralized basic
education services.
80% of districts have post-
project sustainability plans
and budgets; advocacy
achieved funding.
1. Moderately satisfactory
2. Potentially sustainable.
School Operational Assistance (BOS-KITA),
2008 – 2012.
World Bank
$1,100m
ICR, June 2013.
Improve access by strengthening
school management, better
fiduciary arrangements and use of
School Operational Assistance
funds.
Financial management
(transparency and
accountability) will enable
sustainability of project
impacts; program is
financially sustainable.
1. Moderately satisfactory
2. Yes
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
62
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
*Madrasah Education Development Project,
2008 – 2012.
ADB
$71m
VR, January 2015.
Raise standards in 500 madrasah
to benefit 120,000 poor students.
Sustainability listed as
specific project output and
integrated in design and
M&E framework; madrasah
communities will sustain
upgraded facilities because
of ownership.
1.Successful
2. Likely sustainable.
Better Education through Reformed
Management and Universal Teacher
Upgrading (BERMUTU) Project, 2008 – 2013.
World Bank
$138m
ICRR, June 2014.
Contribute to the improvement of
the quality and performance of
teachers through enhancing
teachers’ knowledge of subject
matter and pedagogical skills.
Activities continued with
finance from government;
BERMUTU districts have
an MoU with the Ministry
to continue programs using
their own funds; teacher
working group program
sustainable – accessible to
teachers; financial,
technical, and institutional
sustainability assured by
regulation and structures.
1. Satisfactory
2. Yes
* Vocational Education Strengthening Project,
2008 – 2014.
ADB
$109m VR, December 2016.
Employment opportunities for
vocational school graduates.
Contingent on further
investment in the sector; no
assurance for this as project
downplayed policies to
operate sustainably.
1. Successful
2. Likely to be sustainable.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
63
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Australia-UNICEF Education Assistance to
Papua and Papua Barat, 2009 – 2011.
AusAID
AUD 7m IPR Mar 2012.
To contribute to improved quality
of primary education through
strengthened planning, teaching
practices, and school
management.
Signs of sustainability
planning; universities
institutionalise modules in
teacher training; no
evidence of other
sustainability elements.
1. Rated as adequate quality
2. Some early signs of potential
sustainability.
Support for Education Development in Aceh,
2009 – 2012.
AusAID
AUD 8m IPR, April 2012.
To support the province to
improve the effectiveness and
efficiency of basic education.
Signs of sustainability at
provincial level, none at
district level; sustainability
risks associated with support
of new institutions.
1. Satisfactory.
2. Evidence for the first two
pillars (of three project pillars)
is strong. For the third it is still
to be proven.
Program for Enhancing Quality of Junior
Secondary Education, 2009 – 2013.
JICA USD n/a.
https://www.jica.go.jp/project/english/indon
esia/0800042/index.html
Enhance quality of junior
secondary education through
participatory school-based
management and Lesson Study.
n/a
n/a
Program for Enhancing Quality of Junior
Secondary Education, 2009 – 2013.
JICA Y800m
Terminal Evaluation
https://www.jica.go.jp/english/our_work/eva
luation/tech_and_grant/project/term/asia/c8h
0vm000001rr8t-att/indonesia_2012_01.pdf
Enhance quality of junior
secondary education through
participatory school-based
management and Lesson Study.
Institutionalisation by
government of programs;
human resources for
dissemination have been
developed; dissemination
embedded in the Program,
programs are implemented
or disseminated with
government funding.
1. Yes
2. High
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
64
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Health Professional Education Quality Project,
2009 – 2014.
World Bank $72m
ICRR, August 2015.
To strengthen quality assurance
policies governing the education
of health professionals in
Indonesia.
Requires development of
additional accreditation
instruments, expansion of
training of assessors and
validators, ensure agencies’
sustainability, monitoring of
system’s impact.
1. Satisfactory
2. Moderate
Bantuan Operasional Sekolah (School
Operational Assistance) Training Program,
2011 – 2012.
AusAID
$26m
Independent Review, June 2013.
Delivery of BOS training to
School BOS Management Teams
– approximately 650,000
participants.
Sustainability not discussed;
related concept of impact
assessed as requiring better
implementation, better
cascade training approach,
and greater effort to ensure
beneficiary ‘ownership’ of
the process.
1. Quite successful in terms of
its quantitative goals; less in
terms of other intended
outcomes.
2. Sustainability not discussed.
Early Childhood Education and Development
Project, 2011 – 2013.
World Bank
$93m
ICRR, June 2014.
To improve poor children’s
development and readiness for
further education, within a
sustainable quality Early
Childhood Education and
Development system.
The design included a
holistic approach to building
sustainability; this is evident
in this report by frequent
reference to sustainability.
1.Moderately satisfactory.
2. ‘Risk to development
outcome’ substantial because:
(a) districts and ECED centers
will be unable to sustain
programs; (b) Ministry will not
sustain the teacher training,
national standards, monitoring
and not sustain and expand the
ECED model nationally.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
65
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Australia’s Education Partnership with
Indonesia, 2011 – 2017.
AusAID/DFAT
AUD $369m
ICR, December 2016.
Component 1: Junior secondary
school construction and
expansion.
Component 2: Education
personnel professional
development.
Component 3: Private Islamic
school quality improvement.
Component 4: Analytical and
Capacity Development
Partnership (ACDP).
Performance weakest in
sustainability.
Component 1: Community
based construction model
and good quality control
ensure a high degree of
sustainability for
infrastructure
Component 2: Unlikely to
produce sustainable
outcomes due to lack of
financing strategy.
Component 3: Ministry had
not committed to proposed
sustainability agendas.
Component 4: No
indication of institutional
benefits ACDP wishes to
sustain; sustainability not
foreseen within ACDP
design. While sustained
benefits may accrue, the
prospects for change have
been left to chance.
1. Mixed results
2. The Partnership did not
meet expectations in terms of
its potential impact or with
regards to the likelihood that
key benefits will be sustained.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
66
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
* Poverty Reduction and Millennium
Development Goals Acceleration Program
(PRMAP), 2007 – 2008.
ADB
$400 m
CR, May 2017.
The PRMAP loan supports
commitment towards
achievement of the Millennium
Development Goals (MDG);
poverty reduction (MDG 1),
education (MDG 2), gender
equality (MDG 3), and health
(MDG 4, 5, 6).
Sustainability depends on
continued political and
financial commitment from
government. PRMAP
included embedded
institutionalized policy
actions.
1. Effective
2. Likely
Scholarships Program for Strengthening
Reforming Institutions, 2011 – December 2017.
World Bank
$114m
Implementation Status & Results Report,
December 2016.
To build participating agencies
capacity by (a) strengthening their
human resources and (b)
enhancing their ability to initiate
and manage reforms.
None.
1. Moderately satisfactory.
2. The ‘Systematic Operations
Risk-rating’ evaluates
Institutional Capacity for
Implementation and
Sustainability risk as moderate.
* Third Barefoot Engineers Training Project,
2012 – 2013.
World Bank
$4m
ICRR, June 15.
To provide trained technical
facilitators for rural programs
Papua and West Papua
‘Risk to Development
Outcome’ is substantial:
Lower than expected
retention rates; absence of
long-term strategy to
address shortages of
technical facilitators; limited
alternative employment
opportunities.
1. Moderately satisfactory
2. Risk to development
outcome: substantial.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
67
Project Details
(Title, Dates, Cost, Donor, Source)
Synopsis of Project Goals
Sustainability Elements
Discussed
1. Project successful?
2. Benefits sustainable?
Improving Teacher Performance and
Accountability (Kiat Guru), 2016 – 2017.
World Bank
$4m
Implementation Status & Results Report, April
2017.
Improve the presence and service
quality of primary school teachers
in targeted areas. The objective
will be reached by making
teachers more accountable and by
empowering communities to
participate in education.
A ‘moderate’ risk rating for
this project has been
assigned, accounting for
technical design, political
and governance,
institutional capacity for
implementation, fiduciary
risks, and sustainability.
1. Moderately satisfactory
2. Moderate risk of failure
# Prioritizing Reform, Innovation,
Opportunities for Reaching Indonesia’s
Teachers, Administrators, and Students
(PRIORITAS), 2012 – 2017.
USAID
$ 88m
Final Performance Evaluation, 2017.
Extend the capacity of district and
provincial governments to plan,
manage and finance education.
• Improve education human
resource management in the
provinces and districts
• Upgrade teachers in primary and
junior-secondary schools
• Encourage parental and
community involvement
• Develop participatory and
accountable school management
• Help pre and in-service teachers
by supporting Teacher Training
Institutes.
Districts: commitment of
leaders, activity planning,
regulations, budgets, and
community participation.
School level: teachers
accustomed to new
practices, school planning,
use of teacher and principal
working groups.
National level: aligned with
GoI programs and
regulations, GoI invitation
to train provincial staff,
budget allocations, teacher
training institute facilitators
benefited from training.
1. Yes.
2. Yes, clear indications of
sustainability of PRIORITAS
approaches in districts and
schools.
The Sustainability of Benefits from Educational
Development Projects in Indonesia:
What do the Donors Say?
68
