Page 1

IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.4 APRIL 2005

825

PAPER

Path Following Circuits——SPICE-Oriented Numerical Methods

Where Formulas are Described by Circuits——

Special Section on Selected Papers from the 17th Workshop on Circuits and Systems in Karuizawa

Kiyotaka YAMAMURA†a), Member, Wataru KUROKI†, Student Member, Hideaki OKUMA†∗, Nonmember,

and Yasuaki INOUE††, Member

SUMMARY

nonlinear problems on the circuit simulator SPICE. In the method of PFC’s,

formulas of numerical methods are described by circuits, which are solved

by SPICE. Using PFC’s, numerical analysis without programming is possi-

ble, and various techniquesimplemented in SPICE will make the numerical

analysis very efficient. In this paper, we apply the PFC’s of the homotopy

method to various nonlinear problems (excluding circuit analysis) where

the homotopy method is proven to be globally convergent; namely, we ap-

ply the method to fixed-point problems, linear programming problems, and

nonlinear programming problems. This approach may give a new possibil-

ity to the fields of applied mathematics and operations research. Moreover,

this approach makes SPICE applicable to a broader class of scientific prob-

lems.

key words:

circuit simulator, SPICE, nonlinear equation, homotopy

method, optimization problem

Path following circuits (PFC’s) are circuits for solving

1.Introduction

In the computer-aided design of electronic circuits, circuit

simulation is one of the most central tasks. In circuit sim-

ulation, circuits are described by nonlinear equations and

then they are solved by numerical methods implemented in

circuit simulators. As is well-known, SPICE is the most

widely used circuit simulator. In SPICE, various refined

techniques such as sparse matrix techniques, LU decom-

position (for solving linear equations), modified Newton’s

method (for solving nonlinear equations), and stiffly stable

numerical integration methods with efficient time-step con-

trol algorithms (for solving ordinary differential equations)

are implemented. Moreover, there are many variations of

SPICE, and most of them include various know-hows accu-

mulated over many years. Hence, it is a great loss to apply

such an excellent software only to electronic circuits.

Inthe numericalanalysisofnonlinearsystems, systems

(such as circuits) are described by equations, to which nu-

merical methods are applied. In that case, programming of

the numerical methods or using ready-made softwares such

as Mathematica and MATLAB is necessary. However, the

Manuscript received June 17, 2004.

Manuscript revised September 23, 2004.

Final manuscript received November 25, 2004.

†The authors are with the Faculty of Science and Engineering,

Chuo University, Tokyo, 112-8551 Japan.

††The author is with the Graduate School of Information, Pro-

duction and Systems, Waseda University, Kitakyushu-shi, 808-

0135 Japan.

∗Presently, with Panasonic Mobile Communications Co., Ltd.

a)E-mail: yamamura@elect.chuo-u.ac.jp

DOI: 10.1093/ietfec/e88–a.4.825

programming of sophisticated numerical methods (such as

the homotopy method) is often difficult for non-experts or

beginners.

In this paper, we discuss a new approach of numerical

analysis using path following circuits, which is based on the

counter idea: formulas of numerical methods are described

by circuits, and then they are solved by SPICE.

Path following circuits are also called solution curve-

tracing circuits (STC’s).STC’s were first developed

for finding multi-valued characteristic curves easily using

SPICE [1],[2]. Then, this idea was extended to the ho-

motopy method for finding DC operating points of nonlin-

ear circuits with the theoretical guarantee of global conver-

gence [3],[4]. Recently, this idea was further extended so

that general scientific problems (not necessarily circuit anal-

ysis) can be solved using SPICE [5]–[7]. Such methods are

also called SPICE-oriented numerical methods. As a related

study, see [8], where optimization problems are solved by

analog Hopfield neural networks realized on SPICE.

In this paper, we restrict our approach to the homotopy

method, which is known to be a very effective method for

solving nonlinear equations that cannot be solved by New-

ton’s method. We also restrict the application fields only

to those where the homotopy method is proven to be glob-

ally convergent. Namely, we apply the STC’s describing the

formulas of the homotopy method to various problems such

as fixed-point problems, linear programming problems, and

nonlinear programming problems. These problems are typ-

ical application problems of the homotopy method and have

been studied for many years in the fields of applied math-

ematics and operations research. Hence, it is expected that

this approach gives a new possibility to the fields of applied

mathematics and operations research. Moreover, for SPICE

users, the proposed method will be useful because they can

easily solve these problems using SPICE that is familiar to

them.

In the fields of applied mathematics and operations re-

search, tracing solution curves are generally called path fol-

lowing. Hence, in this paper we call solution curve-tracing

circuit (STC) as path following circuit (PFC).

2.Examples of SPICE-Oriented Numerical Methods

In this section, we first show simple examples of the SPICE-

oriented numerical methods [5]–[7].

Consider a system of n nonlinear equations:

Copyright c ? 2005 The Institute of Electronics, Information and Communication Engineers

Page 2

826

IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.4 APRIL 2005

Fig.1

Circuits describing (1).

Fig.2

Circuits describing (2).

f(x) = 0(1)

that does not necessarily describe a nonlinear circuit, where

f = (f1, f2,···, fn)T: Rn→ Rnand x = (x1, x2, ···, xn)T∈

Rn. If we want to solve (1) using SPICE, then we perform

DC analysis to the circuits shown in Fig.1, where xide-

notes the node voltages and Fidenotes the currents of the

controlled sources defined as Fi(x) = fi(x) + Gixi. Since

the circuit equation describing these circuits is f(x) = 0,

the modified Newton’s method implemented in SPICE is

applied to (1) by performing the DC analysis†. Note that

SPICE has many built-in functions (for example, exp, log,

sin, sqrt, abs, etc.) for describing the currents of controlled

sources [9].

Similarly, if we want to solve an initial value problem

of a system of n ordinary differential equations:

˙ x = f(x,t),

x(0) = x0

(2)

using SPICE, then we perform transient analysis to the

circuits shown in Fig.2 where Fi is defined as Fi(x) =

Cifi(x,t). In Fig.2,G is a dummy conductance that is neces-

sary only to avoid the topological restriction of SPICE, and

a negligible small value G ≈ 0 is used so that it is regarded

as virtually open. Since the circuit equation describing these

circuits is ˙ x = f(x,t), the stiffly stable numerical integration

methods using efficient time-step control algorithms imple-

mented in SPICE are applied to (2) by performing the tran-

sient analysis. Ordinary differential equations of more gen-

eral forms:

f(x, ˙ x, ¨ x,···, x(n)) = 0

can also be described by circuits [6] and can be solved using

SPICE.

(3)

3.Path Following Circuits That Describe Homotopy

Methods

Consider a system of n nonlinear equations:

f(x) = 0 (4)

again, where f : Rn→ Rnand x ∈ Rn. Newton’s method is

Fig.3

Circuits describing (9).

commonly used for solving nonlinear equations. However,

Newton’s method often fails to converge unless the initial

point x0∈ Rnis sufficiently close to the solution x∗∈ Rn. To

overcome this convergence problem, the homotopy method

has been studied, and its global convergence property has

been proved for various problems such as fixed-point prob-

lems, optimization problems, and DC analysis of nonlinear

circuits.

In the homotopy method, we consider another equa-

tion:

f0(x) = 0 (5)

that has a known solution x0, where f0: Rn→ Rn, and

define a homotopy function h : Rn+1→ Rnby

h(x,t) = tf(x) + (1 − t)f0(x)

where t ∈ [0,1]. Then, the solution curve (often called the

path) of the homotopy equation:

(6)

h(x,t) = 0 (7)

is followed starting from the trivial solution (x0,0) at t = 0.

If the path reaches the t = 1 hyperplane Rn× {1} at (x∗,1),

then we obtain the solution x∗of (4).

In the method of PFC’s, the path of (7) is followed as

follows [1]–[7]. Let any point (x,t) on the path be repre-

sented as (x(s),t(s)), where s denotes the arc-length of the

path starting from the initial point (x0,0). Then, the arc-

length s is defined mathematically as:

(dx1)2+ (dx2)2+ ··· + (dxn)2+ (dt)2= (ds)2.

Combining (7) with (8), we have a system of the algebraic-

differential equations as follows:

(8)

h(x,t) = 0

?dx1

Hence, by integrating (9) and (10) starting from (x0,0), we

can follow the path of (7). This method is called the arc-

length method [6].

In order to realize the above method on SPICE, we

first describe (9) by the circuits shown in Fig.3, where Hiis

(9)

ds

?2

+

?dx2

ds

?2

+ ··· +

?dxn

ds

?2

+

?dt

ds

?2

= 1.

(10)

†In [6] and [7], Gi = 1 is used, which is simple and easy to

understand. However, in practical application, using the values of

Giwith appropriate order according to the problem often makes

the numerical analysis more stable [5]. For example, if f(x) = 0

is a circuit equation, then using Gi= 10−3will make the algorithm

more stable because actual conductances take the values of order

around 10−3.

Page 3

YAMAMURA et al.: SPICE-ORIENTED NUMERICAL METHODS

827

Fig.4

Circuits describing (10).

defined as:

Hi(x,t) = hi(x,t) +Gixi.

(11)

Then, we describe (10) by the circuits shown in Fig.4 where

G is a dummy conductance with a negligible small value

G ≈ 0. (In our numerical experiments, we used G = 10−12.)

In Fig.4, ˙ xior ˙ t denotes a node voltage that is independent

of xior t but is equal to dxi/ds or dt/ds as a result.

From the first circuits in Fig.4, ˙ xi = dxi/ds holds.

From the second circuit in Fig.4, (˙ x1)2+(˙ x2)2+···+(˙ xn)2+

˙ t2= 1 holds. From the third circuit in Fig.4, ˙ t = dt/ds

holds. Hence, the circuits shown in Fig.4 are described by

(10). Therefore, we can follow the path of (7) by performing

transientanalysistothecircuitsinFigs.3and4startingfrom

(x0,0)†. Note that the arc-length s corresponds to the time in

the transient analysis. The circuits shown in Fig.4 are called

the path following circuits for the homotopy method.

In this method, if we once write the netlist of the cir-

cuits shown in Figs.3 and 4 (such a netlist can be easily

written), then we only need to write Hi(x,t) of the controlled

sources for each problem. Thus, this method can be easily

realized using SPICE.

4.Application to Fixed-Point Problems

From this section, we apply the method described in the pre-

vious section (that will be called the PFC method) to various

problems where the homotopy method is proven to be glob-

ally convergent, and discuss its effectiveness. In our numer-

ical experiments, we used a free software SPICE3f5 and a

Sun Ultra 60 workstation (CPU: UltraSPARC-II 450MHz).

In this section, we first consider fixed-point problems.

The homotopy method has been applied to various fixed-

point problems that occur in the filelds of economic equilib-

rium, game theory, and optimization theory for many years

[10]–[12]. Let us consider the problem of finding fixed-

points of the following functions [10],[11]:

(a)

(b)

(c)

Fig.5

problems.

Result of computation (obtained paths) for the fixed-point

Problem 1:

gk(x) =

1

200

100

?

i=1

x3

i+ k

k = 1,2,···,100

Problem 2:

g1(x) = x1−

100

?

100

?

i=1

xi− 1

gk(x) = xk−

i=1

xi+ xk− 101

k = 2,3,···,100

Problem 3:

gk(x) = exp

cos

k

10

?

i=1

xi

k = 1,2,···,10

†When we determine the initial values of the transient analysis

by .IC command, we set the initial value of ˙ t at s = 0 as ˙ t = 1 so

that the path proceeds to the direction of t > 0.

Page 4

828

IEICE TRANS. FUNDAMENTALS, VOL.E88–A, NO.4 APRIL 2005

by the PFC method. Note that these problems are often used

as test problems. Figures 5(a)–(c) show the result of compu-

tation (obtained paths) where the horizontal line is denoted

as the x1-axis and the vertical line as the t-axis. In Fig.5(b),

the path turns sharply at t = 1. This is because the solu-

tion of this problem is nonsingular. In other words, the PFC

method worked well for such a nonsingular problem. In

Fig.5(c), averycomplicatedpathisobtained. Namely, since

SPICE includes various efficient techniques for solving stiff

circuit equations [9] (such as the variable-step variable-

order implicit integration methods and the time-step control

algorithms), the PFC method is robust for strongly nonlinear

problems where paths become very complicated.

5. Application to Linear Programming Problems

Asnumericalmethodsforsolvinglinearprogramming prob-

lems, the simplex method and the interior-point method are

well-known. Especially, the interior-point method has been

studied very actively since the work of Karmarkar in 1984.

The most efficient and widely used interior-point method is

the infeasible primal-dual interior-point method. Consider a

linear programming problem [13]:

cTx

subject to Ax = b, x ? 0

where x ∈ Rnis a variable vector, A is an m × n matrix, and

b ∈ Rnand c ∈ Rnare constant vectors. Let the dual problem

of (12) be

max

bTy

subject to ATy + s = c, s ? 0

where y ∈ Rmand s ∈ Rnare variable vectors. In [13], it is

shown that the infeasible primal-dual interior-point method

to (12) is equivalent to the homotopy method applied to the

homotopy equation:

Ax − b − (1 − t)(Ax0− b) = 0

ATy + s − c − (1 − t)(ATy0+ s0− c) = 0

Xs − (1 − t)X0s0= 0

min

(12)

(13)

(14)

(a)

(b)

Fig.6

Circuits for generating t that changes from 0 to 1.

where X is an n × n diagonal matrix with the (i,i)th element

xi. Detailed explanation of (14) is omitted here. Notice that

(14) is a system of m+2n equations in m+2n+1 variables.

Linear programming problems can be solved by fol-

lowing the paths of (14) using the circuits shown in Figs.3

and 4. However, since it is also proved [13] that the path of

(a)

(b)

(c)

(d)

(e)

Fig.7

programming problems.

Result of computation (obtained paths) for the linear

Page 5

YAMAMURA et al.: SPICE-ORIENTED NUMERICAL METHODS

829

(14) is monotone in t [namely, the solution of (14) is unique

for t fixed], we can consider another possibility where the

concept of arc-length s is not used. Namely, we can fol-

low the path of (14) by considering t as the node voltage of

the circuit shown in Fig.6(a), where the (piecewise-linear)

independent voltage source Vtis defined by a code:

Vt t 0 PWL(0 0 1 1)

[see the right-hand figure of Fig.6(a)], and performing tran-

sient analysis to the circuits shown in Figs.3 and 6(a). It

is also possible to follow the path by considering t as the

node voltage of the circuit shown in Fig.6(b) and perform-

ing transient analysis to the circuits shown in Figs.3 and

6(b). In both methods, we let t play the role of s because the

solution of (14) can be written as (x(t),t).

In the numerical experiments, we chose more than 30

problems from the standard bench mark problems reposi-

tory Netlib [14] (http://www.netlib.org/lp) and solved them

by the PFC method using the initial points (x0,y0, s0) =

(1,···,1, 0,···,0,1,···,1).

Figures 7(a)–(e) show the result of computation for

AFIRO (28 constraints, 32 variables), SC105 (106 con-

straints, 103 variables), STOCFOR1 (118 constraints, 111

variables), SHIP08S (779 constraints, 2,387 variables), and

SHIP12L (1,152 constraints, 5,427 variables), respectively,

where the left-hand figures showthe movementof the objec-

tive functions and the right-hand figures show the movement

of x1. Although there are several sharp turning points on the

paths, the PFC method worked very well. The computa-

tion time was 3s, 18s, 54s, 617s, and 2,908s, respectively.

Note that the system of nonlinear equations (14) that de-

scribes SHIP12L consists of 12,108 variables. Thus, linear

programming problems of more than five thousand variables

(where the system of nonlinear equations consists of more

than ten thousand variables) could be solved in reasonable

computation time using SPICE. Since SPICE includes effi-

cient time-step control algorithms, it can follow paths with

both almost linear parts and sharply bending parts very ef-

ficiently.

6. Application to Nonlinear Programming Problems

Consider a nonlinear programming problem:

max f(x)

subject to

gj(x) ? 0,

hj(x) = 0,

where f, gj, and hjare concave functions. It is shown [12]

that the optimal solution of (15) can be obtained by follow-

ing the path of the homotopy equation:

−(1 − t)(x − x0) + t∇f(x)T

+

j=1

j=1

α−

j = 1,2,···,r

hj(x) = 0,

j = 1,2,···, s

j = 1,2,···,r

j = 1,2,···, s

(15)

r ?

α+

j∇gj(x)T+

s ?

µj∇hj(x)T= 0

j− gj(x) = 0,

(16)

Fig.8

Circuits for generating α+

jand α−

j.

where α+

jand α−

jare defined by

α+

α−

j= max{0,αj}

j= max{0,−αj}, j = 1,2,···,r

for given α ∈ R1. It is also proved that this homotopy

method is globally convergent [12].

ear programming problem (15) can be solved by the PFC

method. Note that in this case αj and µj are defined as

node voltages of the circuits describing α−

hj(x) = 0, respectively.

However, in order to apply the PFC method, it is neces-

sary to represent α+

not include the function max{·,·}. Since (17) is equivalent to

j=1

2(|αj| + αj)

j=1

2(|αj| − αj), j = 1,2,···,r,

it is sufficient to consider circuits shown in Fig.8, where the

currents of the controlled sources are defined by

(17)

Hence, the nonlin-

j− gj(x) = 0 and

jand α−

jby circuits because SPICE does

α+

α−

(18)

A+

j=1

2(|αj| + αj)

j=1

2(|αj| − αj), j = 1,2,···,r.

We now consider the following example problems.

A−

(19)

Garcia-Zangwill problem [12]:

f(x) = x1− (x2)2− x3

g1(x) = −x1+ (x2)2− 2x2x3+ (x3)2− 3 ? 0

g2(x) = −x1? 0

h1(x) = 2x1− x2+ x3= 0

Rosen-Suzuki problem [15]:

f(x) = −(x1)2− (x2)2− 2(x3)2− (x4)2+ 5x1

+5x2+ 21x3− 7x4

g1(x) = (x1)2+ (x2)2+ (x3)2+ (x4)2+ x1

−x2+ x3− x4− 8 ? 0

g2(x) = (x1)2+ 2(x2)2+ (x3)2+ 2(x4)2− x1

−x4− 10 ? 0

g3(x) = 2(x1)2+ (x2)2+ (x3)2+ 2x1− x2

−x4− 5 ? 0

Figures 9(a) and (b) showthe result of computation for these

problems, respectively. It is seen that the PFC method could