Content uploaded by Frank Horowitz

Author content

All content in this area was uploaded by Frank Horowitz

Content may be subject to copyright.

ABSTRACT

We have developed a high-performance, ﬂexible

scheme for interpolating multi-dimensional data. The tech-

nique can reproduce exactly the results obtained, for exam-

ple, from Ordinary Kriging and related techniques in 3D, or

from Thin-Plate Splines (Briggs' minimum-curvature algo-

rithm) in 2D. Moreover, compared to traditional implemen-

tations of these algorithms, our method enjoys large

computational-cost savings.

The new approach produces an interpolation that

obeys a Partial Differential Equation (PDE). The PDE may

arise from physically based arguments, but its form can

vary widely. It might be speciﬁed only implicitly (as through

a Model Variogram), or be nonlinear (although a perform-

ance penalty could then apply).

While a formal equivalence between Kriging and

Splines has been known for some time (Matheron (1980)),

the present derivation, from radial basis functions, further

illuminates this connection. Thus, for example, we can

make explicit the PDEs that underlie some of the Model

Variograms most often used in Geostatistics. Besides its

practical utility, the work thereby acquires a theoretical

interest.

BACKGROUND

The aim of this paper is to explore the close connec-

tions between several seemingly different topics: Kriging;

radial basis function (RBF) interpolations; Spline interpola-

tions; and the solutions of Partial Differential Equations

(PDEs). We demonstrate that these techniques can pro-

duce families of identical interpolations.

We start with some notation. Throughout this paper, we

denote vectors in a lower case boldface font, and matrices

in an upper case boldface. Let be the position of a

control point at which some scalar value is known. We

denote by an interpolation which passes through the

set of control points .

KRIGING

Simple Kriging

In the traditional formulation of simple Kriging (and its

derivatives; Isaaks and Srivastava (1989)), one estimates

at some interpolation point from a weighted lin-

ear combination of the

, (1)

where is the vector with components and

denotes the inner (dot) product between two vectors. The

weight vector is found by solving the linear system

, (2)

where the are the covariances between

control points in the assumed model, and

gives the covariances between the control points and .

We choose simple Kriging for its mathematical simplic-

ity, and because most (if not all) of the other Kriging tech-

niques can be derived from simple Kriging via additional

constraints and/or via integrals of the resulting interpolation

(e.g. ordinary Kriging, universal Kriging or Kriging with a

trend, block Kriging).

Dual Kriging

In the dual Kriging formulation (Matheron (1982), or

Royer and Vieira (1984), for an English example), one

forms the estimate from a single set of weights ,

. (3)

The unknown is obtained by solving

. (4)

Details are given in Appendix 1, where dual formulation

analogues of techniques like Kriging with a trend are also

outlined.

xiℜn

∈

ui

ux()

Nux

i

() ui1iN≤≤;=

ux0

()

x0

ui

ux0

() w0u,()≈

uui

ab,()

w0

Cw0b0

=

Cij φxixj

–()=

b0φx0xj

–()=

x0

ux0

()

f

ux() b0f,()≈

f

C†fu=

FAST MULTIDIMENSIONAL INTERPOLATIONS

Franklin G. Horowitz

1,2

, Peter Hornby

1,2

, Don Bone

3

, Maurice Craig

4

1

CSIRO Division of Exploration & Mining, Nedlands, WA 6009 Australia

2

Australian Geodynamics Cooperative Research Centre, Nedlands, WA 6009 Australia

3

CSIRO Division of Information Technology, Canberra, ACT 2601 Australia

4

CSIRO Division of Exploration & Mining, Floreat Park, WA 6014 Australia

Reprinted with corrections from Chapter 9 (pp. 53-56) of the 26th Proceedings of the Application of Computers and

Operations Research in the Mineral Industry (APCOM 26), R.V. Ramani (ed.), Soc. Mining, Metall., and Explor. (SME),

Littleton, Colorado, U.S.A., 538 p., 1996.

Efﬁciency

From the standpoint of the operations count required to

compute an interpolation, a direct implementation of equa-

tions (3) and (4) offers a slight advantage over equations

(1) and (2). Recall that

C

is .

The dual formulation requires solving only the single

linear system (4), which costs operations for practi-

cal algorithms (e.g. Press, et al. (1992) for a readable dis-

cussion). In order to evaluate the interpolation at

different grid points, one needs a further opera-

tions ( evaluations of (3), each of which requires an

dot product between and ). Thus a

direct implementation of dual Kriging requires a grand total

of

(5)

operations.

The simple formulation requires solving the linear sys-

tem (2) at each of the grid points. A direct implementa-

tion requires operations. Evaluation of (1) over

the grid requires a further operations, yielding a

total cost of

(6)

operations. A more sophisticated algorithm might be to

solve (2) by inverting once, costing . Then, over

each of grid points, it would evaluate . Each

matrix multiplication requires operations, resulting

in . The total cost of this algorithm being

, (7)

and it might be more efﬁcient than the direct implementa-

tion (depending on how scales with ). Remember,

sampling is typically pointwise (spot sampling) or linewise

(drill strings, traverses, ﬂight lines), and hence could be

as high as or even , although is

probably more common.

Clearly, the dual formulation is more efﬁcient than

either of the simple formulation algorithms. Equally clearly,

when (common for real world applications),

none

of these algorithms (as stated) is practical on com-

monly available computers. This observation has led to the

development of many localised algorithms, whose primary

computational reason for existence is to force down to

something manageable (like 40 or 50). Of course these

algorithms require large and complex bookkeeping compo-

nents, for such operations as nearest neighbourhood

searches. We will not consider the ramiﬁcations of this

modiﬁcation here.

RBF INTERPOLATION

The theoretical underpinnings of radial basis function

interpolation are described in Powell (1992), and the refer-

ences found therein. Brieﬂy, although the method is not for-

mulated in the language of spatial statistics, the actual

interpolation is computed nearly identically to dual Kriging,

. (8)

Here is as before, and is some polynomial of the

components of . The and the coefﬁcients of the poly-

nomial satisfy an augmented system

. (9)

The only difference between equations (8)-(9) and equa-

tions (3)-(4) is the addition of the polynomial term ,

resulting in the augmentation of the simple Kriging system

of equations (Appendix 1). When we recover

(3), and when , we have the dual Kriging ana-

logue of universal Kriging (also known as Kriging with a

trend).

In RBF parlance, Kriging’s model covariance functions,

, are the radial basis functions upon which the

technique is founded. Just as in Kriging, practitioners of

RBF interpolation choose the form of from a small set of

functions known to be admissible (principally by inducing

the matrix to be invertible). The efﬁciency of RBF inter-

polation is identical to that of dual Kriging (e.g. described

by expression (5)).

PARTIAL DIFFERENTIAL EQUATIONS

Links to RBFs and Kriging

We now motivate the connection between dual Kriging,

radial basis functions, and linear PDEs by relating the PDE

formalism to the RBF formalism. Suppose that, for a given

, there exists an operator such that

, (10)

where is Dirac’s delta, and

. (11)

NN×

ON

3

()

M

OMN()

M

ON()

f

φx0xi

–()

ON

3

()OMN()+

M

OMN

3

()OMN()

OMN

3

()OMN()+

CON

3

()

MC1– b0

ON

2

()

OMN

2

()

ON

3

()OMN

2

()+

MN

M

ON

2

() ON

3

() ON()

N10 000,∼

N

ux0

() fiφx0xi

–()

i

∑Phx()+=

φPhx()

xfi

Phx()

C

ˆ†f

ˆu

ˆ

=

Phx()

Phx() 0=

Phx() 0≠

φx0xi

–()

φ

C

ˆ†

φL

Lφxx

i

–()δxx

i

–()=

δ

LPhx() 0=

For example, if is a differential operator, is the Green’s

function of , and is a homogeneous solution to

the PDE. Admittedly, (10)-(11) are strong assumptions to

make, however, our aim here is to outline an idea, and we

do not wish to cloud the simplicity of that idea with mathe-

matical trickery dealing with difﬁcult cases.

Applying to (8) gives

. (12)

This corresponds to an operator equation, where is the

response of a system (described by ) to a set of point

forces of magnitude at points . The coefﬁcients of

determine, or are determined by, the boundary con-

ditions, depending upon your point of view. Equation (12),

together with adequate BCs, renders the problem positive

deﬁnite. Apart from the question of BCs, this problem is

equivalent to the Kriging (or RBF) problem. In fact, this is

potentially an interesting route to a proof of the existence of

unique and stable solutions to the latter problems (at least

for certain and ). Thus it would seem plausible

that the Green’s functions of invertible PDE problems are

admissible as RBFs. We have

not

shown that an exists

for arbitrary , and indeed such a statement need not be

true. However, the freedom in choosing boundary condi-

tions for the Green’s function problem, together with the

existence of formal inverses deﬁned in terms of Fourier

methods, can take one a lot further in this direction than

one would initially think possible.

Now suppose in the following that

is

a differential

operator. Then we might consider discretising the system

(12) via a Finite Difference approximation of (more gen-

erally, one could include quadrature for an integro-differen-

tial ). In this discretisation, wherever is known,

is unknown and vice versa. So the discretisation of equa-

tion (12) needs re-arrangement such that all unknowns

appear on the left hand side, and all knowns on the right

hand side.

Next, one imposes boundary conditions (BCs) which

implicitly determine . An example, relevant to ore

grade estimation, might be to require that on a

boundary sufﬁciently distant from the region of interest.

Numerics

We have just shown that certain simple Kriging, dual

Kriging and RBF interpolations are equivalent to solving a

PDE subject to boundary conditions. We can now employ

one of the fast PDE solvers in order to compute an interpo-

lation.

We solve the discretised version of (12) using the full

multigrid ﬁnite difference method. As discussed in (for

example) Press, et al. (1992), this technique solves the

problem on a grid of points in

(13)

operations.

This is independent of the number of control points, and

in fact, the method actually converges faster with increas-

ing density of control points. Compare the scaling behav-

iour (13) with those of the more direct formulations of

Kriging or RBF interpolations: expressions (5), (6), and (7).

As a practical matter, one commonly knows only or

only directly. Determining the missing member of the

pair is straightforward upon consideration of the Fourier

Transform of equation (10) (although the resulting convolu-

tion operator may be unstable).

Our current multigrid implementation has not yet

reached the full potential speed of , due to practical

difﬁculties involving the representation of control points on

grids of different scales. The scaling behaviour of our cur-

rent implementation is unquestionably better than

and probably better than . We regard this as a

difﬁculty with our current algorithmic strategy rather than a

fundamental problem with a multigrid approach to interpo-

lation.

Example

As a concrete example of this technique, if ,

is interpreted as displacement, and

as force , then equation (12) would be the PDE

for a two-dimensional thin plate spline (the minimum curva-

ture interpolation of Briggs (1974), common in geophysical

applications). Higher dimensional analogues of thin plate

splines would simply let , and deﬁne appro-

priately. Interpolations which are stiffer than thin plate

splines can be found from considering higher powers of

(e.g. the spherical model variogram).

DISCUSSION

This general approach has the obvious capability to put

smooth interpolation onto a physical basis. If an interpola-

tion is required to satisfy (12) where is known from phys-

ical arguments, a direct application of our technique will

yield a physically realistic solution. However, beyond this, it

allows some Kriging interpolations to be calculated quickly

using PDE solution methods.

Kriging methods form the basis of modern conditional

simulation techniques, which seek to characterize the vari-

ability of the predicted quantity in addition to its mean

value. It is interesting to contemplate the relationship

between these conditional simulations, and the PDE formu-

lation. If there is an equivalent PDE formulation, what might

it look like? A Gibbs spatial process perhaps?

Since multigrid is an iterative scheme, given a closed

Lφ

LP

hx()

L

Lu fiδxx

i

–()

i

∑

=

u

L

fixi

Phx()

φPhx()

L

φ

L

L

Luxi

() fi

Phx()

ux() 0=

M

OM()

φ

L

OM()

OM

2

()

OM Mlog()

L∇4

≡

uux()=ffx()=

xℜ2

∈()

xℜn

∈() ∇

4

∇

L

form expression for (and its ﬁnite difference representa-

tion) we believe that there is no fundamental reason to

restrict application of this technique to linear . For some

classes of nonlinear PDEs, the only challenge to this gen-

eral scheme should be algorithmic details, and a possibility

of scaling performance that is slightly worse than .

We close with the following remark: since some Kriging

problems can clearly be solved by solving PDEs, we sus-

pect that there are many potential synergies to be found in

applications (such as ﬂow ﬁeld simulation in petroleum

engineering) where the output from an interpolation speci-

ﬁes the geometry for further physical simulation. Both

aspects of the problem could now be solved from within the

same software base.

APPENDIX 1

RBF Equivalent of Ordinary Kriging

The linear system for simple Kriging arises from a mini-

mization problem. The conditions that the variance is mini-

mized with respect to the unknown parameters , are

expressed as equation (2). For ordinary Kriging, to impose

an unbiased solution, equation (2) must be solved with the

constraint

,(A1)

which also ensures that a constant function is interpolated

exactly. Together, equations (2) and (A1) are solved via the

method of Lagrange multipliers, yielding a new augmented

system (in partitioned matrix format)

. (A2)

We rewrite equation (A2) as

. (A3)

Now, in the Kriging formulation, the value of the interpo-

lation at a point is expressed as

. (A4)

Here, the inner (dot) product between dimensional vec-

tors is denoted by . The augmented vector

must be constructed such that the equality between the left

two terms of equation (A4) is preserved. The only augmen-

tation that preserves that equality is

. (A5)

Given this, we now derive a radial basis function equiv-

alent to the ordinary Kriging system. Start with the ordinary

Kriging expression for the value at some interpolation point

. (A6)

Since we have assumed is invertible, and given that it is

also symmetric, it follows that there exists an such that

. (A7)

These are the radial basis function equations augmented

by a constant offset. This corresponds to the case of Equa-

tion (8) with . This augmented RBF repre-

sentation can be evaluated at to give

. (A8)

But by (A3) we know that this can be rewritten as

. (A9)

Rewriting (A9) as a matrix manipulation, regrouping, and

transposing, we ﬁnd

. (A10)

Substituting from (A7) it is found

. (A11)

But this is exactly the ordinary Kriging solution for .

Therefore, if there is a solution to the augmented RBF

deﬁned by (A7) then it is equivalent to the ordinary Kriging

solution and vice versa.

Extension for detrending with constrained higher

order moment

Consider an interpolation of a scalar function of .

The augmented RBF system which includes linear func-

tions of the coordinates in , will interpolate any lin-

ear function exactly. The equations determining the and

the three coefﬁcients of are

L

L

OM()

wi

wi

∑1=

C1

1†0

w0

µ

b0

1

=

C

ˆw

ˆ0b

ˆ0

=

x0

ux0

() w0u,()

Nw

ˆ0u

ˆ

,()

N1+

==

N

αβ,

αβ(,)

N

u

ˆ

u

ˆu

0

=

ux0

() w

ˆ0u

ˆ

,()

N1+

=

C

ˆ

f

ˆ

C

ˆ†f

ˆu

ˆ

=

Phx() const=

x0

ux0

() b

ˆ0f

ˆ

,()

N1+

=

ux0

() C

ˆw

ˆ0f

ˆ

,()

N1+

=

ux0

() w

ˆ0C

ˆ†f

ˆ

,()

N1+

=

ux0

() w

ˆ0u

ˆ

,()

N1+

=

x0

ℜ2

Phx()

fi

Phx()

, (A12)

where is a vector of the th coordinates of the set of

control points, . (Here, denotes the th

coordinate at sample point .) Equation (A12) adds further

constraints to the ﬁrst order moments of the “forces” ,

these being and . This is

like linearly detrending the data and kriging simultaneously.

The resulting interpolation is

, (A13)

where . Clearly, higher orders may

be handled similarly, so we have the dual Kriging/RBF ana-

logue to universal Kriging/Kriging with a trend.

ACKNOWLEDGEMENT

This paper is published with the permission of the

Director of the Australian Geodynamics Cooperative

Research Centre.

REFERENCES

Briggs, I.C., 1974 “Machine Contouring Using Minimum

Curvature,” Geophysics, v. 39, no. 1, pp. 39-48.

Isaaks, E.H. and Srivastava, R.M., 1989 “An Introduction to

Applied Geostatistics,” Oxford University Press, New

York.

Matheron, G., 1980 “Splines and Kriging - Their Formal

Equivalence,” in Down-to-Earth Statistics - Solutions

looking for geological problems, ed. D.F. Merriam, pp.

77-95, Syracuse University Geology Contributions.

Matheron, G., 1982 “Pour une analyse krigeante des don-

nées régionalisées,” Note interne, n°732, CGMM, Fon-

tainebleau, 20 p.

Powell, M.J.D., 1992 “The Theory of Radial Basis Function

Approximation in 1990”, in Advances in Numerical Anal-

ysis, Vol. 2: Wavelets, Subdivision Algorithms, and Ra-

dial Basis Functions, ed. W.A. Light, pp. 105-210, Oxford

University Press, New York.

Press, W.H., Teukolsky, S.A., Vetterling, W.T. and Flan-

nery, B.P., 1992 “Numerical Recipes in C; The Art of Sci-

entific Computing,” Cambridge University Press,

Cambridge.

Royer, J.J., and Vieira, P.C., 1984 “Dual Formalism of Krig-

ing,” in Geostatistics for Natural Resources Characteri-

zation, Part 2, ed. G. Verly, M.David, A.G. Journel, and

A. Marechal, NATO ASI Series C: Mathematical and

Physical Sciences, Vol. 122, pp 691-702, D. Reidel Pub.

Co., Dordrecht.

Cc

1c21

c1†000

c2†000

1†000

f

σ1

σ2

µ

u

0

0

0

=

cj

j

xij()

{}

i1…N=xij()

j

i

fi

fixi1()

⋅

i1=

N

∑0= fixi2()

⋅

i1=

N

∑0=

ux0

() b

ˆ0f

ˆ

,()

N1+

=

b

ˆ0b0x01()x02()1†

=