Content uploaded by Christopher C Kelly

Author content

All content in this area was uploaded by Christopher C Kelly on Jan 25, 2020

Content may be subject to copyright.

Content uploaded by Christopher C Kelly

Author content

All content in this area was uploaded by Christopher C Kelly on Oct 30, 2014

Content may be subject to copyright.

OctOber 2012 53

Internal audItor

hy did the auditors cross the road? Because

they did it last year.” The internal auditor’s

comfort zone is analyzing objective histori-

cal data. But when establishing a risk-based

plan of future audits under IIA Performance Standard 2010:

Planning, auditors are obliged to perform an annual risk

assessment, considering future subjective expectations held

by the board, management, and others. In the absence of

data, do auditors just accept those future expectations, or can

they subject them to critical analysis?

Although estimating the likelihood and impact of

future risks is subjective, it will not reﬂect well on the chief

audit executive (CAE) if the organization is sideswiped by

a system failure, signiﬁcant cost overrun, or fraud while

internal audit’s plan was focused elsewhere. The audit plan

Auditors at

an Australian

transport company

use mathematical

analysis to assist

them in their audit

planning.

Christopher Kelly

Prioritizing Risks for the

Future

W

RISK MANAGEMENT

“

OctOber 201254 Internal audItor

Prioritizing risks for the future

should demonstrate that the CAE has

thought broadly and deeply about

“unknown unknowns” and engaged

constructively with those who assess

and manage risk.

Widely used risk matrices pro-

duced using International Organiza-

tion for Standardization (ISO) 31000

often convert discrete risks into

alphanumeric scores. Typically, the

underlying scores of the matrix follow

a geometric progression both in terms

of likelihood and consequence, yet

the matrix layout can have the appear-

ance of an arithmetic progression.

As a result, the seemingly simple risk

matrix can be problematic to inter-

pret, as the built-in scaling may not be

apparent at ﬁrst glance. Furthermore,

VISIT

www.Internal

AuditorOnline.org

to view a ﬁve-year

bubble graph used

by an Australian

transport

company to

identify risks.

MONTE CARLO EXPLAINED

Monte Carlo is a mathematical technique for creating a probabil-

ity distribution showing what might happen in the future based

on what can be foreseen today. It was not designed for internal

audit nor even risk management; instead, World War II weapons scientists

devised it to predict the probable range of uncertain outcomes when

developing atomic weapons.

To illustrate the concept, suppose your spouse asks whether you can

continue paying the mortgage over the next ﬁve years. Although the

future is unknown, you know your future contractual mortgage obliga-

tions, future pay raises are expected to be between -10 percent (worst

case) and +15 percent (best case), income tax will range from 25 percent

to 40 percent, and there could be a 2 percent chance of losing your job

through illness or layoff. Some of the risks are mutually exclusive while

others are interdependent. With inputs like these, Monte Carlo analysis

can help in answering your spouse’s question — not with absolute assur-

ance but within a probable range of certainty. The analysis will accom-

modate any further risks you may need to add: Can you also afford next

year’s European vacation? And your child’s education fees?

Monte Carlo takes those multi-year/multi-risk estimates and creates a

probability distribution from thousands of future pseudo-random scenar-

ios. The scenarios are pseudo-random because they are all bounded by the

probability and impact estimations. Therefore, in the mortgage example

Monte Carlo would not predict a “black swan” event such as a lottery

win, unless it had been built into the estimation inputs. The power of the

method is that intelligent discussions about future risks can be held with

those most knowledgeable about probabilities and impacts.

many organizations’ risk matrices are

designed to grade even insigniﬁcant

and minor risks. If poorly constructed,

risk matrices can derail management

thinking away from the most critical

risks. For instance, at an Australian

transport company the safety depart-

ment had ignored the risks around

fatal accidents in the mistaken, and

untested, belief that preventive con-

trols were infallible. Instead, the

company’s matrix design gave priority

to frequently occurring minor risks.

The matrix would have been more

meaningful if its users had consulted

diligently with the organization’s risk

experts and its design had empha-

sized more important, rather than less

important, risks.

OctOber 2012 55

Internal audItor

Only 45% of executives are comfortable with how well their

organization’s most critical risks are being managed, according to a recent PricewaterhouseCoopers survey.

indicates individual risks by catego-

rizing them into predeﬁned scaled

boxes, the bubble graph — using the

same axes as a risk matrix — plots

each individual risk without scaling.

Had it suited audit purposes, audi-

tors could have substituted other risk

dimensions as the y and z axes, such

as environmental, safety, or compli-

ance impact. But for the ﬁrst version,

ﬁnancial and reputational impacts

were those of highest concern.

This approach enabled a further

analytical step in the form of a prob-

ability distribution to show the likeli-

hood of best- and worst-case scenarios

in aggregate. To understand the distri-

bution of aggregate risk, it was helpful

to consider one risk at a time. One risk

that emerged during the management

consultations was failure to negoti-

ate with the unions future pay raises

ONE RISK AT A TIME

When the transport company’s board

rejected the risk matrix and internal

audit’s plan on the grounds of missing

important risks, auditors turned to

Monte Carlo mathematical analysis to

analyze future risks both individually

and in aggregate (see “Monte Carlo

Explained” on page 54). Getting cred-

ible probability and impact inputs

required a thorough consultation

across the organization’s experts in risk

management, operations, insurance,

treasury, safety, and human resources.

Using the estimates arising from

these meetings, internal audit ﬁrst

generated a bubble graph showing

the worst-case probabilities (x axis)

and worst-case cost impacts of each

individual risk (y axis), as well as

the reputation impact as the size of

each bubble (z axis). While a matrix

commensurate with board expecta-

tions, which management and audit

estimated could cost up to 5 percent

in excess payroll on an existing annual

payroll of AU $200 million. So the

worst-case scenario would be approxi-

mately AU $200 million x 5 percent

x four years (because the current year

costs were already known) = AU $40

million in today’s dollars. Management

considered the cost could be anywhere

between zero and AU $40 million.

This risk was mitigated by the fact

that the company’s most experienced

negotiators had been assigned to the

union discussions. So a negative out-

come was deemed 50 percent likely.

The probability distribution therefore

reﬂected the 50 percent likelihood of a

successful (zero) outcome and 50 per-

cent of scenarios with an unfavorable

outcome of variable cost between zero

800

700

600

500

400

300

200

100

0

120

100

80

60

40

20

0

12 23 60 84 108 132 156 180 204 228 252 276 300 324 348

NUMBER OF SCENARIOS

COST AU $ MILLION

PROBABILITY (%)

ALL RISKS PROBABILITY DISTRIBUTION — MULTI-YEAR PROJECTION

Pre-mitigation excluding reputation

Post-mitigation excluding reputation

Cumulative post-mitigation excluding reputation

Cumulative pre-mitigation excluding reputation

OctOber 201256 Internal audItor

Prioritizing risks for the future

Probability Distribution — Multi-year

Projection” on page 55). Generating

thousands of scenarios is easy once the

model is set up, even with off-the-shelf

spreadsheet software.

As auditors added more risks,

the likelihood of a zero outcome (the

left tail of the distribution) across all

risks diminished. This was because for

each new risk added, the probability

that no risk occurs became less likely.

For instance, in the union negotiation

example the risk of occurrence versus

nonoccurrence was 50/50. By adding a

second risk with a similar 50/50 likeli-

hood, the probability of a zero outcome

in which neither risk occurs was 50 per-

cent x 50 percent = 25 percent. After

adding a third 50/50 risk, the zero risk

likelihood diminished to 50 percent x

50 percent x 50 percent = 12.5 percent.

The more risks added, the less likely

none of them will occur.

Simultaneously, as auditors added

more risks, the right tail grew toward

its upper limit, which was the possible,

albeit unlikely, scenario that all the risks

would occur at their worst estimated

outcome. So, while the left tail of the dis-

tribution shrank, the right tail grew. This

explains why the distribution as shown in

the “All Risks Probability Distribution”

graph stretches out to the right.

Auditors then could give the board

more information about risk than was

available with only a risk matrix. The

probability distribution showed the

worst-case scenarios (i.e., that most of

the risks occurred at their worst cost

estimates), what was likely to occur,

and the best-case scenarios. Because it

and AU $40 million beyond business

plan expectations.

ALL RISKS IN AGGREGATE

As the management consultations had

supplied credible predictive estimates

of future likelihood and impact ranges

across all known signiﬁcant risks,

internal audit used the Monte Carlo

methodology to construct a prob-

ability distribution from thousands of

scenarios across all risks in aggregate.

Each scenario was one version of what

the future might bring. To create each

scenario, random numbers were used

in two ways: ﬁrst, as a random trigger

as to whether or not a risk occurred

based on management’s probability

estimate, and second, as a random

impact between management’s upper

and lower impact estimates. In this

case, impact was described in dollars,

but it just as readily could have been

described as fatalities or injuries. Audi-

tors chose to perform calculations in

today’s dollars with the commitment

to update the analysis at least annually.

For example, one scenario was:

ɅA less-than-ideal union negotia-

tion in year one resulting in AU

$10 million in higher-than-

anticipated labor costs in each of

the subsequent four years.

ɅA labor strike plus adverse cur-

rency movement in year two total-

ing AU $8 million.

ɅA ﬁre with injuries in year three

costing AU $4 million.

ɅNo risk events in year four.

ɅExtreme weather in year ﬁve cost-

ing AU $3 million.

The net cost to the organization of

this scenario after insurance recoveries

would be AU $55 million.

To stabilize the distribution,

internal auditors created 10,000 ran-

dom scenarios describing the future

costs the organization might expect

to incur split down both premitiga-

tion and postmitigation (see “All Risks

showed the distribution from best- to

worst-case scenarios and the most

likely spread in between, it was more

meaningful than merely calculating the

average value (expected cost x probabil-

ity for each individual risk) or slotting

individual risks into a risk matrix. The

shape of the distribution was uniquely

based on the probability and cost esti-

mates generated in the management

discussions. Auditors showed aggre-

gate dollar cost as the x axis, which is

intuitively more straightforward when

talking with management than trying

to describe the distribution in standard

deviations. Once the Monte Carlo

method was understood by all parties,

it helped open up insightful discussions

among board members, management,

and auditors around how the future

could be impacted by various risks.

For instance, the S-curve showed

the cumulative probability, which

enabled management to ascertain the

likely maximum cost at a given prob-

ability level. The mean dollar impact of

all risks was where the x-axis intercept

reached the S-curve at 50 percent,

which could be cross-checked by sum-

ming each individual risk multiplied

by its probability to compute average

aggregate risk. Another useful insight

was to look at the upper and lower

ranges of scenarios to determine the

most likely future impact range. From

the graph it was visually clear that most

of the company’s post-mitigation sce-

narios fell between AU $40 million and

AU $130 million. By viewing the graph

side-by-side with the bubble graph,

auditors could see which individual

Internal audit used the Monte Carlo

methodology to construct a probability

distribution of scenarios across all risks.

OctOber 2012 57

Internal audItor

risks had the most impact on the dis-

tribution, such as the AU $84 million

projected maintenance cost overrun.

These risks in turn generated the most

debate. Discussion was focused on what

action management needed to take to

minimize the impacts. Once the model

was set up, it also was useful to study

how the mean changed over time as

risks were re-estimated and as controls

were implemented.

REORIENTING THE RISK AGENDA

What began as an audit planning exer-

cise developed into something greater.

With insights generated through

Monte Carlo analysis, the transport

company’s internal audit function

was able to have constructive discus-

sions about what management actions,

insurance limits, hedging strategies,

and audits needed priority.

According to one director, it was

the ﬁrst time the board had debated the

organization’s most politically sensitive

risks. Within six months several under-

managed risks were mitigated, labor

negotiation and projected maintenance

cost overrun risks became standalone

board agenda items, management

changes ensured the best executives were

focused on the critical risks, insurance

coverage was reevaluated, probability

and impact estimates were updated as

controls took effect, and management

pointed to new areas of risk in safety and

engineering that also needed internal

audit analysis. Taken together, these

actions strengthened the link between

internal audit’s program and manage-

ment action in shrinking the impact of

risk on the company.

CHRISTOPHER KELLY, FCA, MIIA, is

a partner in consulting ﬁrm Kelly & Yang,

located in Newcastle, Australia.

Auditors then could give the board

more information about risk.

TO COMMENT on this article,

EMAIL the author at christopher.kelly@theiia.org