Google Scholar's Ranking Algorithm: The Impact of Citation Counts (An Empirical Study)

Assignment
PJTM511
A Report about Programme
Management Audits
August, 2008









Author: Timothy Dalton

Lancaster University, UK
Management School
MSc in Project Management
Programme Management







1. Purpose of Programme Management Audits ................................................. 2
2. Content Page of a Programme Management Audit Report ........................... 4
3. Justification .................................................................................................... 6
3.1. Planning Review ............................................................................. 6
3.2. Risk Management Review .............................................................. 8
3.3. Benefit Management Review ......................................................... 9
4. Literature ..................................................................................................... 11

2/11

1. Purpose of Programme Management
Audits

Our company is running multi million dollar programmes but we do not do any
audits to evaluate these programmes. So far we solely relied on the programme
managers opinion and appraisal of our management board. This report will
present you reasons why we should no longer rely on subjective evaluation of
people who are somehow involved in the programmes but conduct objective
and independent audits to evaluate our programmes. There are several kinds of
audits. For instance audits may be undertaken during a programme or after a
programme (Hice 2003). They may be undertaken by internal employees (that
were not be involved in the programme) or by external consultants. In this
document I will focus on audits at the end of a programme undertaken by an
independent third party (whoever that is).

The main goal of a programme management audit is to evaluate how effective
the programme management was (Lycett et al 2004). That means how effective
was the management delivering the programme and have all objectives and
benefits been achieved (Beel 2007)? More importantly the audit pinpoints areas
that need to be improved (McDaniel 1990). An audit is not just a tool that
makes us know how good the evaluated programme was (Mock 1999). It helps
us to deliver better programmes in the future. The focus of a programme
management audit lies on the applied processes and not on single activities
done by the management. That means it will be evaluated if appropriate models

3/11
were used and how good they were applied. Of course also the final outcome
will be evaluated.

A good audit is not cheap. It probably will increase programme costs by a few
percent. However, since the focus of the audit lies on the models and processes,
an evaluation and critique will permanently improve our programme manager’s
best practice and eventually save more money then we spent. Above all audits
will increase the probability of success of future programmes significantly.

In the next section a proposed content page of a programme management audit
report follows.

4/11

2. Content Page of a Programme
Management Audit Report

1. Cover Page

2. Executive Summary

3. Content Page

4. Abbreviations

5. Introduction
The introduction gives an overview of the purpose if this audit, the used
methods in this audit and the people/company involved in this audit

6. Mission
This section gives a brief overview of the mission of the programme
and how that relates to our company’s strategy

7. Background
This section gives an overview of the programme that is evaluated

8. Specifics
Everything special that has to be considered is mentioned in this
section. That relates to the programme (e.g. were there any special laws
that made the programme complicated? Any changes that no one could
expect?) as well as to the audit (were there any special things that made
the audit complicated and may affect the audit’s quality?)

9. Project Portfolio
Overview of the projects of the programme.

10. Reviews
This section includes the important parts. It includes reviews of all
major programme management responsibilities. For every section will
be considered a) if there were models and processes b) if those models
and processes were applied properly c) if there is evidence that those
models and processes that have been applied lead to a beneficial
outcome. Furthermore there will be always asked “why?”. Especially if
something was not good it will be evaluated why it was not good and
how it could be done better in future. “Lessons Learned” may be
provided in each section or at the very end in a separate section.
a. Planning Review

5/11
b. Monitoring Review
c. Controlling Review
d. Configuration Management Review
e. Communication Management Review
f. Change Management Review
g. Resource Management Review
h. Risk Management Review
i. Stakeholder Management Review
j. Quality Management Review
k. Benefit Management Review
Although every section pays attention to the achieved benefits
the Benefit Management Review particularly focus on the
benefits that the programme should have achieved. It is not the
same as the Quality Management Review. The quality of a
programme might be fine but still benefits could be not
achieved. The Benefit Management Review is from a broader
perspective than the Quality Management Review and may
include parts of other reviews.
l. Close Out Management Review
11. Findings and Recommendations
This section summaries the findings and gives additional
recommendations how to improve the management of future
programmes.

12. Appendix 1…n


Depending on the programme there might be more sections included. For
instance if we had a programme to deliver the next Olympic games we might
also have some Cultural Management. In this case a Cultural Management
Review would be part of the audit. This content page relates to content that
should be included in virtually every audit.

6/11

3. Justification
In the following I will analyse three sections closer to demonstrate the
importance of programme management audits. For illustration purposes an
example is used. The example is about a system for road toll collection that has
been introduced in Germany. The example was not officially named as a
programme but as a project. However, the development was a huge project
with several sub-projects all aiming at one particularly benefit. Therefore it can
be seen as a programme. The development of the toll collection has been
carried out by a consortium consisting of three big companies. The system is
called Toll Collect1.


3.1. Planning Review
Planning is the fundament of each programme (Chait 1995). If essential
mistakes are made during the planning stage it is hardly possible to deliver a
beneficial programme. Therefore a review of the planning stage is so
important. The review of the planning stage includes everything before the
execution stage (Radcliff 2003). Some authors distinguish between initation
and planning stage (Waddell 2005). However, the review focuses on the

1 A detailed analyses of Toll Collect is undertaken in my 2nd assignment for the Project
Management in Action module

7/11
question if appropriate models were available and have been used. For instance
it is not possible just to do a plan. Several steps are necessary (Reiss & Leigh
2006): a) Influenced by the company’s strategy a programme mandate is
created; b) then a blueprint is made; c) possible solutions are developed d)
benefits are analysed; e) a concrete project portfolio is created; f) finally the
detailed planning is carried out where again several steps are necessary.

The review also looks at the level of detail of the plan during those stages and
how the plan was validated (Ejigiri 1994). The review also analyses how
feasible the plan is for achieving the goals. This whole review is not only to
discover mistakes. It is there to provide suggestions where and how to improve
the planning process. The review also focuses on used estimation techniques.
Since the programme is already done when the review takes place it is easy to
compare how good the estimations were. But the programme management
audit does not just focus on a comparison of planned and achieved data, it
focus on the processes the management had used to get the estimations (Thiry
2003).

There is one more important task for the programme management during the
planning stage. It has to ensure that all projects fit together to deliver the
benefit(s) (Reiss & Leigh 2006). In the Toll Collect example one company
mainly developed the software, the second company the hardware. Later the
system was not working. The software team said the software would work fine
and it was the hardware’s team fault. The hardware team said the hardware
would work fine and it was the software team’s fault. Probably really both

8/11
hardware and software worked fine separately in their test environments. It
would have been the programme management’s responsibility to ensure that
already during the planning stage (and all other stages too, of course) proper
planning was undertaken that both parts together work fine.

3.2. Risk Management Review
Part of the programme management audit is a review of the risk management.
It has to be evaluated how good the following four processes were done (Reiss
& Leigh 2006), (Waddell 2005).

a) Risk identification process. For good risk management all risks have to
be identified. This can not be done only by the programme manager for
example. For holistic risk identification all stakeholders have to be
involved.
b) Risk analysis process. After the identification risks have to be analysed.
That means a probability and the impact have to be assigned to each
risk.
c) Risk containment planning process. When the risks are analysed and the
critical risks are identified a strategy has to be developed how to
minimise the risks and how to deal with the risks if they occur. This
may include if appropriate what-if scenarios are considered.
d) Risk treatment process. Even the best plan is worthless if people do not
stick to it. Therefore the audit has to review if the programme
management really sticked to the plan and treated the occurring risks as
planned.

The audit has to ensure that especially the programme manager has a deep
understanding of risk management and the four processes. Since the audit takes
place after the end of the programme the audit has to identify risks that did
occur but were not covered in the risk management plan. The audit then has to
analyse why those risks were not covered and if there is a way to improve risk
management in the next programme (McDonald 2002).

9/11

To give an example: Developing the Toll Collect system had many risks. Legal
risks as well as technological. According to Streimelweger (2003) risks had
been identified during the planning stage. Unfortunately no analysis or
containment planning had taken place. When the risks occurred the
management did not know how to cope with the risks. Finally the whole
programme became a completely failure.

3.3. Benefit Management Review
This part is the most important part of the programme management audit. It is
about if the programme delivered benefits to our company or not, that means if
it was successful or not. It is likely that if all other reviews draw a positive
conclusion (good planning, good execution, good risk management, etc.) the
benefits review is positive as well but not necessarily. The audit is about
evaluating if the management kept all the time their focus on achieving the
benefits and if all the single management activities such as planning, risk
management or stakeholder management together could make the programme a
success. The audit also evaluates how good the benefits were defined and how
easy they are measurable. Another important point is how good the benefits
were monitored during the programme. A point that most authors in literature
ignore but that I consider as important is the analysing of unplanned benefits.
The NHS (2004) mentions that an audit may reveal delivered benefits that no
one before had thought about or even realised.

10/11
Toll Collect does not fit perfectly as an example here. As described in the
planning review section it has been realised during the project that not all
project outputs worked fine together. But it is easy imaginably that in other
programmes it is not that easy to see that the project portfolio does not deliver
the desired benefit. Maybe each project manager is very happy with her project
and maybe even the programme management does not notice that the desired
main benefit is not delivered by the projects. Here the audit analyses again if
appropriate steps had been undertaken from the programme management and if
finally the main benefits were achieved.

11/11

4. Literature

Beel, J 2007, ‘Project Team Rewards: Rewarding and Motivating your Project
Team’, Jöran Beel, Createspace, Scotts Valley, ISBN 1434816265.
Chait, EP 1995, ‘Understanding Program-Specific Audits’, Journal of
Accountancy, vol. 180, no. 6, pp. 69-74.
Ejigiri, DD 1994, ‘A Genereic Framework for Programme Management’,
International Journal of Public Sector Management, vol. 7, no. 1, pp. 53-65.
Hice, R 2003, ‘Program Management: Only masochists need apply’,
Computing & Instrumentation, vol. 21, no. 1, pp. 8-52.
Lycett, M & Rassau, A & Danson, J 2004, ‘Programme management: a critical
review’, International Journal of Project Management, vol. 22, pp. 289-
299.
McDaniel, LS 1990, ‘The Effects of Time Pressure and Audit Program
Structure on Audit Performance’, Journal of Accounting Research, vol. 28,
no. 2, pp. 267-285.
McDonald, J 2002, ‘Software project management audits – update and
experience report’, The Journal of Systems and Software, vol. 64, pp. 247-
255.
Mock, TJ & Wright, AM 1999, ‘Are Audit Program Plans Risk Adjusted?’,
Auditing: A Journal of Practice & Theory, vol. 18, no. 1, pp. 55-74.
NHS, 2004, Benefits Management Guidelines, National Health Service, UK.
Radcliff, D 2003, ‘Project Evaluation and Remediation’, Cost Engineering, vol.
45, no. 1, pp. 18-22.
Reiss & Leigh 2006, Effective Programme Management, Supplementary Notes
for the Lecture Programme Management at Lancaster University, UK.
Streimelweger, M 2003, Ausgestaltung und Auswirkungen der LKW-Maut in
Deutschland, Diplomca, Hamburg.
Thiry, M 2003, ‘For DAD: a programme management life-cycle process’,
International Journal of Project Management, vol. 22, pp. 245-252.
Waddell, D 2005, ‘Program Management: The Next Step in the Evolution of
Project Management?’, Problems and Perspectives in Management, vol. 3,
pp. 160-168.