The complexity of analog computation

Article (PDF Available)inMathematics and Computers in Simulation 28(2):91-113 · April 1986with 233 Reads
DOI: 10.1016/0378-4754(86)90105-9
Abstract
We ask if analog computers can solve NP-complete problems efficiently. Regarding this as unlikely, we formulate a strong version of Church's Thesis: that any analog computer can be simulated efficiently (in polynomial time) by a digital computer. From this assumption and the assumption that P ≠ NP we can draw conclusions about the operation of physical devices used for computation.An NP-complete problem, 3-sat, is reduced to the problem of checking whether a feasible point is a local optimum of an optimization problem. A mechanical device is proposed for the solution of this problem. It encodes variables as shaft angles and uses gears and smooth cams. If we grant Strong Church's Thesis, that P ≠ NP, and a certain “Downhill Principle” governing the physical behavior of the machine, we conclude that it cannot operate successfully while using only polynomial resources.We next prove Strong Church's Thesis for a class of analog computers described by well-behaved ordinary differential equations, which we can take as representing part of classical mechanics.We conclude with a comment on the recently discovered connection between spin glasses and combinatorial optimization.
THE COMPLEXITY OF ANALOG COMPUTATION †
Anastasios VERGIS
Department of Computer Science, University of Minnesota, Minneapolis, MN 55455, U.S.A.
Kenneth STEIGLITZ
Department of Computer Science, Princeton University, Princeton, NJ 08544, U.S.A.
Bradley DICKINSON
Department of Electrical Engineering, Princeton University, Princeton, NJ 08544, U.S.A.
We ask if analog computers can solve NP-complete problems efficiently. Regarding this as unlikely, we formulate a
strong version of Church’s Thesis: that any analog computer can be simulated efficiently (in polynomial time) by a digi-
tal computer. From this assumption and the assumption that P NP we can draw conclusions about the operation of
physical devices used for computation.
An NP-complete problem, 3-SAT, is reduced to the problem of checking whether a feasible point is a local optimum
of an optimization problem. A mechanical device is proposed for the solution of this problem. It encodes variables as
shaft angles and uses gears and smooth cams. If we grant Strong Church’s Thesis, that
PNP, and a certain ‘‘Downhill Principle’’ governing the physical behavior of themachine, we conclude that it cannot
operate successfully while using only polynomial resources.
We next prove Strong Church’s Thesis for a class of analog computers described by well-behaved ordinary differen-
tial equations, which we can take as representing part of classical mechanics.
We conclude with a comment on the recently discovered connection between spin glasses and combinatorial optimi-
zation.
1. Introduction
Analog devices have been used, over the years, to solve a variety of problems. Perhaps most widely
known is the Differential Analyzer [4,26], which has been used to solve differential equations. To
mention some other examples, in [25] an electronic analog computer is proposed to implement the
gradient projection method for linear programming. In [18] the problem of finding a minimum-length
interconnection network between given points in the plane is solved with movable and fixed pegs
interconnected by strings; a locally optimal solution is obtained by pulling the strings. Another
method is proposed there for this problem, based on the fact that soap films form minimal-tension sur-
faces. Many other examples can be found in books such as [14] and [16], including electrical and
mechanical machines for solving simultaneous linear equations and differential equations.
Given the large body of work on the complexity of Turing-machine computation, and the recent
interest in the physical foundations of computation, it seems natural to study the complexity of analog
computation. This paper pursues the following line of reasoning: it is generally regarded as likely that
This work was supported in part by ONR Grants N00014-83-K-0275 and N00014-83-K-0577, NSF Grant ECS-8120037,
U. S. Army Research-Durham Grant DAAG29-82-K-0095, and DARPA Contract N00014-82-K-0549. It appeared in
Mathematics & Computers in Simulation 28 (1986) 91-113.
- 2 -
PNP — that certain combinatorial problems cannot be solved efficiently by digital computers.
(Here we use the term efficient to mean that the time used by an ‘‘ideal’’ digital computer is bounded
by a polynomial function of the size of the task description. See [9] for discussion of this criterion.)
We may ask if such problems can be solved efficiently by other means, in particular, by machines of a
nature different from digital computers. We thus come to ask if NP-complete problems can be solved
efficiently by physical devices that do not use binary encoding (or, more generally, encoding with any
fixed radix). We lump such devices together under the term analog computer; in what follows we will
use the term analog computer to mean any deterministic physical device that uses a fixed number of
physical variables to represent each problem variable. This description is admittedly vague and cer-
tainly non-mathematical — we mean it to capture the intuitive notion of a ‘‘non-digital’’ computer.
(More about this in the next section.)
We want to emphasize that the question of whether an analog computer can solve an NP-complete
problem ‘‘efficiently’’ is a question about the physical world, while the P = NP question is a
mathematical one. However, mathematical models of various kinds provide a formalism that is
apparently indispensable for the understanding of physical phenomena. An important connection
between the mathematical world of computation and the physical world of computing hardware was
discussed by Church. In his 1936 paper [6] he equated the intuitive notion of effective calculability
with the two equivalent mathematical characterizations of λ-definability and recursivity. Turing [28]
then showed that this notion is equivalent to computability by what we have come to call a Turing
machine, so that the intuitive notion of effective calculability is now characterized mathematically by
‘‘Turing-computability.’’ This is generally referred to as ‘‘Church’s Thesis,’’ or the ‘‘Church-Turing
Thesis.’’ In our context we express this as follows:
Church’s Thesis (CT): Any analog computer with finite resources can be simulated by a digital
computer.
What we will come to demand is more than that: we are interested in efficient computation, compu-
tation that does not use up resources that grow exponentially with the size of the problem. This
requirement leads us to formulate what we call
Strong Church’s Thesis (SCT): Any finite analog computer can be simulated efficiently by a digital
computer, in the sense that the time required by the digital computer to simulate the analog com-
puter is bounded by a polynomial function of the resources used by the analog computer.
Evidently we will need to give a characterization of analog computers and the resources that they
use. This is discussed in the next section. Following that, we argue that certain numerical problems
are inherently difficult (i.e. not polynomial) for analog computers, even though they are easy for digi-
tal computers.
Something like our Strong Church’s Thesis was discussed recently by Feynman [8] in connection
with the problem of building a (digital) computer that simulates physics. He says:
‘‘The rule of simulation that I would like to have is that the number of computer elements required
to simulate a large physical system is only to be proportional to the space-time volume of the physi-
cal system. I don’t want to have an explosion.’’
- 3 -
We would argue that ‘‘proportional to’’ be replaced by ‘‘bounded by a polynomial function of,’’ in the
spirit of modern computational complexity theory.
A class of mechanical devices is proposed in Section 5. Machines in this class can be used to find
local optima for mathematical programming problems. We formalize the physical operation of these
machines by a certain ‘‘Downhill Principle.’’ Basically, it states that if, in our class of mechanical
devices, there are feasible ‘‘downhill’’ directions, the state vector describing the physical system
moves in such a direction. We also discuss measuring the resources required by these machines.
In Section 6 we reduce 3-SAT (the problem of whether a Boolean expression in
3-conjunctive normal form has a satisfying truth assignment), to the problem of checking whether a
given feasible point is a local optimum of a certain mathematical programming problem. This shows
that merely checking for local optimality is NP-hard.
In Section 7 a mechanical device in the class mentioned above is proposed for the solution of 3-
SAT. Naturally, the efficient operation of this machine is highly suspect. Be careful to notice that the
operation of any machine in practice is a physics question, not a question susceptible of ultimate
mathematical demonstration. Our analysis must necessarily be based on an idealized mathematical
model for the machine. However, we can take the likelihood of
PNP, plus the likelihood of Strong Church’s Thesis, as evidence that in fact such a machine cannot
operate with polynomially bounded resources, whatever the particular laws of physics happen to be.
The paradigm that emerges from this line of reasoning is then the following:
If a strongly NP-complete problem can be solved by an analog computer, and if P NP, and if
Strong Church’s Thesis is true, then the analog computer cannot operate successfully with polyno-
mial resources.
We will then prove a restricted form of Strong Church’s Thesis, for analog computers governed by
well-behaved differential equations. This suggests that any interesting analog computer should rely on
some strongly nonlinear behavior, perhaps arising from quantum-mechanical mechanisms; however,
the problem of establishing Strong Church’s Thesis (or even the Weak Thesis) in the case of
quantum-mechanical or probabilistic laws is an open problem.
2. Some Terminology
We know what a digital computer is; Turing has laid out a model for what a well-defined digital
computation must be: it uses a finite set of symbols (without loss of generality {0,1}) to store informa-
tion, it can be in only one of a finite set of states, and it operates by a finite set of rules for moving
from state to state. Its memory tape is not bounded in length a priori, but only a finite amount of tape
can be used for any one computation. What is fundamental about the idea of a Turing Machine and
digital computation in general, is that there is a perfect correspondence between the mathematical
model and what happens in a reasonable working machine. Being definitely in one of two states is
easily arranged in practice, and the operation of real digital computers can be (and usually is) made
very reliable.
In order to discuss the application of the Turing machine model to solving computational problems,
we need some additional terminology. A problem instance is a finite string of bits, of length L,
together with an interpretation of the bit string that specifies the encoding of a particular computa-
tional problem. The integer Lis termed the size of the input. It is with respect to Lthat the
- 4 -
complexity of computation is measured. If a computation requires no more than Lksteps, for some
fixed k, we say it is polynomial; otherwise we say it is exponential.
We now turn to the task of formulating models for analog computers and to a discussion of how
analog computers are applied to solving computational problems. An analog computer is an indexed
family of physical devices, parametrized by a set of problem instances for which solutions are to be
obtained. Mathematical modeling of the operations of the devices depends on the mathematical
representation for the underlying laws of physics, whatever those laws may be.
Some additional restrictions are assumed to hold. First, for each problem instance, the problem
variables (determined from the interpretation of the bit string) are encoded within the corresponding
physical device as variables whose mathematical descriptions are specifically constrained. Each phy-
sical variable is modeled by a quantity taking values in a normed, finite-dimensional, real space whose
dimension does not depend on the problem instance. As an example, the value of a problem variable
xmay be encoded by the angular displacement of a shaft, by an electric field in 3-space, by a magnetic
field strength, etc. This restriction is to be compared with the use of binary encoding of variables in
digital computers. A ‘‘physical digital computer’’ would allow encoding the value nfor the variable x
with k=O(logn) distinct electric fields, shaft angles, etc.
A second restriction concerns the decoding process whereby the solution of each problem instance
is obtained as a function of the physical variables after operation of the physical device. It is essential
to model the inherent accuracy limitations of physical sensors that must be employed to ‘‘read out’’
the solution to each instance, by assuming that each analog computer has an associated absolute preci-
sion,ε. We require that for any problem instance, the solution obtained from the physical device does
not change when the physical variables range over an ε-neighborhood (defined using the mathematical
model for physical variables) of their nominal values (i.e. the values generated by the mathematical
model of the device).
We want to point out a distinction related to the precision issue. All mathematical models may be
regarded as idealizations of physical reality due to unmodeled and imperfectly modeled effects. In
order to discuss the operation of physical devices using mathematical models, it is important to insure
that the models are robust in the sense that the physical behavior predicted by the mathematical model
is not more sensitive to small changes in the model than is the underlying physical system to small
perturbations. However, it is a difficult task, in general, to come up with suitable quantifications of the
notion of small changes in a model. We would argue that for the purposes of investigating the limita-
tions on analog computation arising from computational complexity theory, the use of ‘‘idealized’’
analog computers whose physical operation corresponds precisely to its mathematical description is
appropriate. In some cases it will be possible to incorporate some robustness in the mathematical
model explicitly through the limited precision property described above.
Finally, we make a general assumption that the physical devices used for analog computation exhi-
bit causal, deterministic behavior: given a complete description of the device (model) at any time
instant t0, the description of the device (model) at times t >t0is uniquely determined by the external
input acting on the device (model) during the interval [t0,t]. We thus rule out quantum-mechanical
systems, although in Section 9 quantum mechanics is discussed by means of an example.
Now that we have a general framework in which to study analog computation through mathematical
models, we need to define our notion of the resources used by an analog computer. Intuitively, we
associate physical resources with the operating costs of the physical device. Thus the physical size, the
mass, the initial stored energy, and the time interval of operation of the device are among the
resources used. In addition, the mathematical model obtained from applying physical laws will
- 5 -
involve physical variables and possibly their time and spatial derivatives. The maximum magnitudes
of all such quantities will also be regarded as resource requirements. As an example, for a particle
described by Newtonian mechanics, the maximum displacement, velocity, acceleration, and applied
force are all resources in addition to the mass and the time of operation of the device.
3. Combinatorial vs. Numerical Problems
An input string of length Lbits can encode a number as large as 2L, and this creates a fundamental
roadblock preventing the efficient solution of certain computational problems with an analog com-
puter. To illustrate the problem, suppose we want to compare two positive integers, n1and n2.We
imagine the following analog comparator. Create two particles with equal charges having masses m1
and m2, respectively. Place them in a uniform electric field. The transit time from a fixed starting
position to another fixed ending position is proportional to mi, so the particle with the smaller mass
arrives at the goal line first. The time complexity of the computation is T(L)=O({i =1,2}
min mi).
We are left to decide how the masses are to encode the numbers. In order to obtain a machine that
does not depend on the specific problem instance, the encoding should be a monotonic function. Sup-
pose we let mi=f(ni). Then if fis a polynomial, the time T(L) is exponential in L. To keep the
time complexity polynomial, therefore, we should choose fto be logarithmic. But this leads to accu-
racy problems: for adjacent large numbers the masses will be so close together that we will have to
make the physical size of the machine exponentially large to discriminate between the arrival times.
(Or what is the same thing, we will need to discriminate between times that are exponentially close
together.)
The difficulty is caused by the fact that the size of a physical quantity (mass in this case) is used to
encode a number that is binary-encoded in the input sequence. We can state this result in general
terms as follows.
Theorem 1. Suppose an analog computer encodes an input variable x that appears in the input string
in binary form by the physical quantity f(x). Suppose that the number of different values that may be
taken on by x is not bounded by any polynomial in L,the size of a problem instance. Then the norm of
the physical variable f(x)is not bounded by any polynomial in L.
Proof. Let εbe the absolute precision associated with the analog computer, and suppose that f(x)
takes its values in p-dimensional space (where pis fixed over all problem instances). If the norm of
f(x) is bounded by the polynomial Lk, the volume of the corresponding sphere in p-dimensional space
is O(Lpk ).However in order to be distinguishable, each possible value of f(x) must be surrounded by
a sphere of diameter ε, and hence volume O(1) with respect to L. Clearly, there can be only polyno-
mially many values taken on by f(x).This proves the result by contraposition.
This result shows the futility of searching for (asymptotically efficient) analog computers to solve
problems involving large numbers. NP-complete problems such as the PARTITION problem and the
INTEGER KNAPSACK problem fall in this class. However, it turns out that there are other NP-
complete problems whose problem instances consist only of input strings corresponding to numbers
that are bounded by some polynomial in L, the size of the instance. This is the class of so-called
strongly NP-complete problems, and it contains such problems as HAMILTON CIRCUIT, 3-SAT, and
others [9].
- 6 -
4. A Polynomial Analog Machine
Our example of comparing two integers shows that some problems that are ‘‘easy’’ for digital com-
puters, e.g. solvable in linear time, are inherently difficult for analog computers because of the nature
of the numerical representation of analog quantities. We now provide an example to show that analog
computers can be found that do in fact solve some (‘‘easy’’) combinatorial problems with polynomial
resources. These problems cannot have numbers encoded in the input string that get exponentially
large. After that we will turn to the more interesting class of seemingly intractable problems, the
strongly NP-complete problems.
Consider the following problem:
GRAPH CONNECTIVITY: Given a graph G=(V,E) and two distinguished vertices s,tV, is
there a path in Gfrom sto t?
Notice that an instance of this problem can be encoded in such a way that the largest number in the
problem description is only polynomially large as a function of the length of the input, L. We will call
such problems combinatorial. This problem can be solved in polynomial time on a Turing machine,
and we say that such problems are in Digital P-time. The amount of tape used by a Turing machine
computation can be no larger than the number of time steps, and it uses no resources other than time
and tape (‘‘space’’). Therefore, a problem in Digital P-time is also guaranteed to use no more than a
polynomial amount of resources on a Turing machine. On the other hand, an analog computer can
conceivably operate successfully in polynomial time but require an exponential amount of some other
resource, such as torque or instantaneous current. We will therefore want to insist that a ‘‘fast’’ and
well behaved analog computation use total resources polynomial in the input description.
It is now easy to show that GRAPH CONNECTIVITY can be solved by an analog machine with
polynomial resources. Make an electrical network out of the graph, as shown in Fig. 1, putting a wire
of constant resistance per unit length wherever there is an edge, and joining the wires at the nodes.
Apply a voltage source of size |V|volts between nodes sand t, and measure the current. If there is a
path between sand tthere will be a resistance of at most |V|ohms between them, and a steady-state
current of at least 1 ampere will flow. If there is no path, the resistance will be high and the current
will ultimately go to zero.
- 7 -
The time required for the operation of this analog computer will depend on the parasitic capacitance
of the circuit, which will determine the effective RC time constant of the circuit. If the wire lengths
grow linearly with the number of edges |E|, the total capacitance seen by the voltage source will be no
worse than proportional to |E|2. Similarly, the total resistance will be no worse than proportional to
the length of the longest wire and the number of edges, and so also O(|E|2). Hence, to distinguish
between the cases where there is and is not a path (in the presence of fixed precision) takes time pro-
portional to the RC time constant, which is O(|E|4). It is also clear that the total size and power con-
sumption of the network are also polynomial in |E|.
We thus have at least one problem where an analog computer operates successfully with polynomial
resources. A key question, then, is whether there is a strongly NP-complete combinatorial problem
(nonnumerical in the sense described above) that can be solved with polynomial resources by some
analog computer. After some preliminaries, we will describe a machine that ostensibly solves such a
problem: 3-SAT. We are able to predict that this machine cannot operate efficiently.
5. A Class of Mathematical Programming Machines
Consider the following instance of a linear programming problem:
max z=2x1+x2
w=x1+x21 (5.1)
x10, x20
The optimum solution of (5.1) is x1=1, x2=0 (see Fig. 2).
We propose the following analog computer for this problem. Each of the variables x1,x2will be
represented by the angular position of a shaft. Shaft positions can be negated by a simple 1:1 gear
coupling and can be multiplied by a constant −| c|by a 1:cgear coupling (see Fig. 3). Two shaft posi-
tions can be subtracted with a differential gear (see Fig. 4). The differential forces the angles p,q,rto
- 8 -
satisfy the equation pq=r. A full description of it can be found in [20]. (Differentials are used in
automobile transmissions.) To preserve symmetry, we shall make the assumption that the differential
adds the angles pand q; this can be accomplished easily by incorporating into it an inverter for the
angle q.
We use the above primitives for multiplication by a constant and summation to solve (5.1), as
shown in Fig. 5. We have four shafts whose angular positions represent the variables x1,x2,wand z.
Their angular positions are not independent; the differentials and gear couplings enforce the relation-
ships: z=2x1+x2,w=x1+x2. Hence we have two degrees of freedom. We can set the angular
positions of any two shafts to any desired values, and this will fix the angular positions of the other
two shafts. The constraint x1+x21 can be imposed by putting a stop at position 1 of the shaft
representing w=x1+x2. The constraints x10, x20 can be imposed similarly, by putting
stops at positions 0 of the shafts representing x1and x2.
Suppose that we start the machine at the feasible state x1=0, x2=0. Then we can maximize
z=2x1+x2(under the constraints of (5.1)), by simply rotating the shaft representing ztowards
increasing values, as far as possible. Since the angular positions x1and x2always satisfy the con-
straints imposed by the stops, the maximum angular position of the zshaft will be the optimum solu-
tion of (5.1).
Now consider the dynamics of the machine. As we start rotating the zshaft towards increasing
values of angular position, 2x1+x2will increase from 0 to 2. Since the x1,x2, and wshafts are left
alone (except for the stops), we are basically using only one degree of freedom. Thus there are many
feasible paths from the initial point (x1,x2)=(0, 0) to the final point (x1,x2)=(1, 0); the one fol-
lowed will be determined by the relative values of the various friction coefficients inside the machine.
For example, assume that the shaft representing x1is much harder to turn than the shaft representing
x2. Then, as we are increasing z, it is possible that x1remains at position 0, and x2=z. That is, the
differential enforcing z=2x1+x2‘‘chooses’’ to distribute the angle zas: x2=z,x1=0. How-
ever, when wreaches 1, wcannot increase further because of the stop at position w=1. At this point,
- 9 -
x2cannot increase any more, but if the force applied to the zshaft is large enough to overcome the
resistance of the x1shaft, x1will start increasing. Since w=x1+x21, x2will decrease until it
reaches the stop at position x2=0. This way, the path p1shown in Fig. 6 will be followed.
In general, a path like pwill be followed; phas the property that it is directed towards increasing
values of z. The actual path pwill be determined by the machine’s preferred direction in the state
space at each state. This is determined by the relative friction coefficients inside the machine. We
ensure that the directions towards increasing values of zare achieved by forcing zforward, with a
force greater than the total frictional resistance.
The above can obviously be extended to the general instance of linear programming:
min z=j=1
Σ
ncjxj
j=1
Σ
naij xjbi,i=1, 2, . . . , m
xj0, j=1, 2, . . . , n.
(Without loss of generality we can assume that cj,aij, and biare integers.)
The summation of more than two variables can be done using a tree of differentials. For example
y=x1+x2+x3+x4can be enforced as implied by y=[(x1+x2)+(x3+x4)]; that is, ywill
be the output of a differential whose two inputs are the outputs of two differentials with inputs x1,x2
and x3,x4, respectively.
The problem of exponentially large numbers appears quite vividly here. For example, the
coefficients aij and cjwill determine the ratios of the gear couplings; we do not want them to be
exponentially large. Also, even if the coefficients are small, a variable (encoded by an angle) may
become exponentially large; this is obviously an undesirable situation. In order to get efficient solu-
tions, we must restrict the inputs of the machine to instances where these phenomena do not occur.
- 10 -
That will be accomplished by assuming that an instance never has numbers greater than some polyno-
mial function of L, the size of the input encoding. That is, from now on we will assume that the
optimization problems we deal with are combinatorial in the sense of Section 3.
An initial feasible solution can be obtained as follows: Assume inductively that we have a feasible
solution x0=(x1
0,...,xn
0) for the first kconstraints
j=1
Σ
naij xjbi,i=1, 2, . . . , k.
If
j=1
Σ
nak+1, jxj
0bk+1
then x0is a feasible solution for the first k+1 constraints; otherwise minimize
j=1
Σ
nak+1, jxj
subject to the first kconstraints with initial feasible solution x0. If its minimum value is bk+1, the
value of x=(x1,...,xn) that does this is a feasible solution for the first k+1 constraints; otherwise
the problem is infeasible.
This technique can be further extended to the mathematical programming problem
min z=f(x)
gi(x)0, i=1, 2, . . . , m(5.2)
assuming, of course, that we have the devices that can enforce the relations yi=gi(x1,...,xn) and
z=f(x1,...,xn). Then a local optimum x*will be found; the machine will move (in the state
space) from its initial state to x*along a path that is directed towards decreasing values of z. In Fig. 7
we depict a device that realizes a smooth function f.
- 11 -
For example, if fis the function of one variable z=x2, this device, called a ‘‘squarer,’’ enforces
z=x2between the angular positions of two shafts. More complex relationships can be enforced. For
instance z=x.ycan be enforced with two squarers, three differentials and a 1:4 gear coupling, as
implied by the formula z=[(x+y)2(xy)2]/4=x.y.
We call such devices ‘‘Mathematical Programming Machines.’’ With them, the feasible space can
be mapped out by simply rotating the x1,...,xnshafts; each combination of angular positions of
these shafts corresponds to a point in the feasible space. When, in our attempt to find a local
optimum, we rotate the zshaft towards decreasing values, we are tracing out some path in the feasible
space.
An important restriction is that these machines can only find local optima. Consider for instance an
optimization problem with a nonconvex feasible space, for example
min z
under the constraint
zh(x).
Assume that x0is a local minimum of h(but not a global minimum). Put z0=h(x0). Then, if we
initialize the machine to (x0,z0) and try to decrease the value of the position of the zshaft, this shaft
will not move. The reason is that there is no way for it to move in the feasible space from point
(x0,z0) to a point (x*,z*), where z*< z0, without first passing through a higher value of z. The
machine will tend to move (in the feasible space) to a new state (x,z) such that z< z0; it tends to
move along a direction such that the projection of the gradient of the objective function on this direc-
tion is negative (since we are trying to decrease the value of the objective function). We formalize
this intuitive notion by the following principle.
Downhill Principle. Let Sbe a Mathematical Programming Machine whose shaft positions (state
variables) xisatisfy the set of relations (5.2). Then if we start it at a feasible state x0and apply a force
to the shaft representing the variable zin the direction of decreasing z, it will move if and only if there
is a feasible direction (in the state space) towards decreasing values of z.
The Downhill Principle simply states that if there exists a feasible ‘‘downhill’’ path (i.e. a feasible
path in the state space such that zdecreases along this path) and ‘‘downhill’’ is a preferred direction
for the state of the system (this is ensured by forcing ztowards decreasing values), then the state will
follow such a path. The Downhill Principle comprises our mathematical model of physics for
Mathematical Programming Machines. It appears to be realistic and adequate for our purposes;
clearly whether or not it holds is a physics question, not a mathematical one that is susceptible to
proof.
When we operate a Mathematical Programming Machine as discussed above, we are relying on this
principle. The resources required for this operation are the size of the machine, the torque required to
move the zshaft if it is going to move, and the time required for significant motion to take place (in
terms of a threshold measurable motion). The size of the machine is clearly polynomial in the size of
the instance, because of our restriction that the instance not encode exponentially large numbers. The
question of whether the other resources required for successful operation are polynomial remains to be
investigated.
- 12 -
6. Checking for Local Optimality is NP-hard
We intend to use the machines described in the previous section to attempt to solve an ‘‘interest-
ing’’ problem, that is a problem that is at least NP-complete. The following decision problem is NP-
hard [9]:
QUADRATIC PROGRAMMING: Given a linear objective function, constraints involving qua-
dratic functions, and a constant K, does there exist a vector that makes the value of the objective
function less than or equal to K, while satisfying the constraints?
We could try to solve QUADRATIC PROGRAMMING with the machines described in the previ-
ous section. However, since these machines can only find local optima, for certain initial conditions
(i.e. certain initial values of the variables), the machine would get stuck at a local optimum. Then we
would have to rotate back the zshaft (i.e. the shaft representing the value of the objective function),
change the values of the other shafts, and try again. Obviously we cannot assert that this procedure
requires polynomial time. (It is interesting to note that an electrical network for solving quadratic pro-
gramming problems is proposed in [5], but it is based on sufficiency of the Kuhn-Tucker conditions,
and depends for its operation on positive definiteness of the quadratic form. At first this might seem to
be another candidate for an analog machine that solves an NP-complete problem, but in fact Quadratic
programming with a positive-definite matrix can be solved in Digital P-time with a variant of the ellip-
soid algorithm [19, Chapter 15, Problem 16].)
If, however, we were asked only to find a local optimum, we would need to try only once, rotating
the zshaft towards decreasing values until it gets stuck. This point must be a local optimum by the
Downhill Principle. There is of course the question of how long we would need to rotate it, which
depends on how far the local optimum is from the initial state. But this problem would also disappear
if we knew what this ‘‘candidate’’ local optimum is; we could initialize the machine to this point and
then just try to rotate the zshaft. If it does not move then this point is a local optimum; if it does
move, then it is not.
According to the above, if we want to show that our machine solves a hard problem, we need to
show that the question, ‘‘Given a feasible point x0of an optimization problem Π, is x0a local
optimum of Π?’’ is at least NP-complete. We are going to prove next that it is NP-hard. First we
define what we mean by local optimum.
Let Πbe the optimization problem: for xRn,
min f(x)
gi(x)0, i=1, 2, . . . , m
We say that a feasible point x0is a local optimum of Πif and only if there exists an ε>0 such that for
every feasible xN(x0,ε), f(x)f(x0), where N(x0,ε) is the neighborhood || xx0|| ≤ ε.
Local Optimality Checking, or LOC for short, is the following decision problem:
LOC: Given an optimization problem: for xRn,
min f(x)
gi(x)0, i=1, 2, . . . , m(6.1)
- 13 -
and a feasible point x0, is x0a local optimum?
For the encoding problem, we can assume that fand each gi,i=1, 2, . . . , m, is restricted to be a
composition of functions taken from a fixed set S={ f1,...,fk}; hence we can have a fixed sym-
bol for each fi. Function composition can be represented as usual, by parentheses.
Theorem 2. LOC is NP-hard for optimization problem (6.1) even if the functions involved are (a)
linear and piecewise linear or (b) linear and quadratic.
Hence the search for local optima in nonlinear optimization is a very hard problem indeed. Even if
we have a candidate point we cannot decide in polynomial time if it is or isn’t a local optimum
(assuming P NP).
Proof. To prove Theorem 2, we reduce 3-SAT to LOC. For reference, 3-SAT is the following prob-
lem, which is one of the earliest known NP-complete problems, and which is strongly NP-complete as
well [9].
3-SAT: Given a set of Boolean variables X1,...,Xn, and given B, a Boolean expression in con-
junctive normal form with exactly 3 literals per clause:
B=(Z11 +Z12 +Z13 )(Z21 +Z22 +Z23 )... (Zm1+Zm2+Zm3)
where each literal Zjk is either some variable Xior its negation Xi, is there a truth assignment for
the variables Xiwhich makes B TRUE ?
For each instance of 3-SAT, we will construct an instance of a problem in real variables xi. By
example, for each clause in Bthat looks like
(X1+X3+X7)
we write an inequality of the form
x1+x3+x7x0(6.2)
on real variables x0,xi,xi,i=0, 1, . . . , n. Also, we add the constraints
xif(xi), i=1, 2, . . . , n(6.3)
where fis the piecewise linear function
f(x)=(4|x| − 5x)/3
In general, if the literal Xiappears in the clause, we include the term xiin the l.h.s. of (6.2); if the
literal Xiappears in the clause, we include the term xi.
The optimization problem is
max x0
subject to the constraints of the form (6.2) (there will be msuch constraints if Bhas mclauses) and
(6.3).
- 14 -
Call this problem REAL 3-SAT, and let Bbe the instance of REAL 3-SAT corresponding to B.
Claim 1. For each satisfying assignment S of B, there exists a direction (d1,...,dn)Rnand a
direction (d1,...,dn)Rn
where
(di,di)=(3, 1) if Xi=TRUE,
and
(di,di)=(1, 3) if Xi=FALSE
in A, such that
x0= θ,
(x1,...,xn)=(d1,...,dn).θ,
(x1,...,xn)=(d1,...,dn).θ,
is a feasible solution of Bfor any θ ≥ 0.
Proof. These values of the real variables satisfy the inequalities (6.2) because each l.h.s. has at least
one term that equals 3θ(at least one literal is TRUE). The sum of the other two terms can be no less
than 2θ, since each of them is either 3θor − θ. Also, the constraints (6.3) are satisfied, since if
xi=3.θthen f(xi)= θ and if xi= θ then f(xi)=3.θ; in both cases xif(xi)
Claim 2. If Bhas a feasible solution with x0>0, then B is satisfiable.
Proof. If x0>0 then each l.h.s. of (6.2) must have a positive term. If xiis positive, put
Xi=TRUE; if xiis positive, put Xi=FALSE. No variable can be set both TRUE and FALSE by
that rule, since we cannot have both xiand xipositive; this follows directly from (6.3). Thus every
clause has a true literal and Bis satisfiable.
We have:
Bis satisfiable ==
>(by claim 1) x0=(0, . . . , 0) is not a local optimum of B==
>
Bhas a feasible solution with x0>0==
>(by claim 2) Bis satisfiable.
Therefore Bis satisfiable if and only if the feasible point x0=(0, . . . , 0) is not a local optimum of
B.
To obtain a reduction to LOC when the functions involved are only linear and quadratic, we write
(6.3) as
xi(5xi+4|xi|)/3, (6.4)
which is equivalent to
xi(5xi+4yi)/3,
- 15 -
yi≤ |xi|.
This is equivalent to
xi(5xi+4yi)/3,
yi
2xi
2(6.5)
yi0.
7. A 3-SAT Machine
We could attempt to solve the ‘‘piecewise-linear’’ version of REAL 3-SAT with a Mathematical
Programming Machine if we could find a machine that realizes the piecewise linear function f(x).
However, such a machine probably cannot be realized, because of the discontinuity of the derivative
of fat x=0. (Consider what happens if we move the shaft representing xwith constant velocity past
the point x=0: the velocity of the shaft representing y=f(x) will be discontinuous.) However, in
the ‘‘quadratic’’ version of REAL 3-SAT, only smooth functions are involved. We can construct a
machine implementing REAL 3-SAT using differentials, gear couplings and a squarer. The squarer
can be implemented by a device like the one shown in Fig. 7, with f(x)=x2.
However, we choose not to try to implement (6.5), in order to avoid the introduction of the nnew
variables yi,i=1, 2, . . . , n. Instead, we write (6.4) as
xi+5xi/34|xi|/3<
==
>(3xi+5xi)/4≤ |xi|.(7.1)
We denote by SQ+the set of continuously differentiable functions F(x) satisfying
F(x)=x2if x0
F(x)0 if x0.
If F(x) is a function in SQ+, it can be implemented with a device similar to the squarer, and (7.1) is
equivalent to
F((3xi+5xi)/4) xi
2.
Hence, we will try to implement the optimization problem
max x0(7.2)
under mconstraints of the form
x1+x3+x7x0(7.3)
and under nconstraints of the form
F((3xi+5xi)/4) xi
2.(7.4)
This optimization problem is equivalent to the two optimization problems discussed in the previous
section.
As discussed in Section 5, we can find out if the point x0=(0, . . . , 0) is a local optimum by ini-
tializing the machine to x0and then applying a torque to the shaft representing x0. If we accept the
Downhill Principle, x0is not a local optimum if and only if the shaft representing x0moves. A way
- 16 -
to interpret this conclusion intuitively is to say that if ‘‘infinitely accurate’’ analog devices could be
built, then they could be used to solve 3-SAT arbitrarily fast.
To summarize the discussion so far, we have arrived at a Mathematical Programming Machine that
solves 3-SAT by testing the local optimality of the origin in the derived problem, REAL 3-SAT. The
machine uses only gears and smooth cams and is only polynomially large. If we grant Strong
Church’s Thesis, that P NP, and the Downhill Principle, we must conclude that such a machine takes
exponential resources.
There are two ways to follow up on this finding. The first is an experimental study to test the
hypothesis that the 3-SAT machine requires exponential resources as predicted by our theory. Alter-
nately, we can replace the Downhill Principle with a more detailed mathematical model for the opera-
tion of the machine, for instance one based on classical Newtonian dynamical equations; this model
can be examined in detail with a view toward an analytical verification of the hypothesis for an
‘‘ideal’’ analog computer. We have carried out a lengthy study to confirm that the prediction of
exponential resource complexity is not altered by taking into account the precision of gear ratios, the
precision of the numbers 3/4 and 5/4 in (7.4), and the precision of the initial shaft positions. One
speculation about the machine is that for polynomially bounded input torque, the time for operation is
exponential, with ergodicity somehow playing a significant role.
In the next section we show how to simulate efficiently analog computers described by a class of
ordinary differential equations. Therefore, we can conclude that no machine that can be modeled by
(7.2) - (7.4) can be described by such differential equations.
8. SCT for Analog Computers Described by a Class of Ordinary Differential Equations
Our interest in this section will be directed towards a class of ‘‘general purpose’’ analog computers,
as opposed to specialized devices designed for solving particular combinatorial problems. We will
introduce a class of differential equations, together with an interpretation of their computational
processes, which corresponds to a model for analog computers of the Bush Differential Analyzer type
[4]. The first steps toward a theory of analog computation were taken by C. Shannon, who showed
that an interconnection of primitive devices — adders, scalar multipliers, and integrators — con-
strained by some natural conditions to ensure well-posedness, generates functions solving ordinary
differential equations of a particular form [26]. Pour-El [22] added some necessary elaborations con-
cerning existence of unique solutions. This work derived the following form for the differential equa-
tions corresponding to a (Bush) analog computer:
A(Y(t)) dt
dY(t)=b(Y(t)) , Y(t0)=Y0(8.1)
where the matrix Aand vector bhave entries composed of linear combinations of the component func-
tions of Y(t)=(1,t,y1(t), ...,yn(t)) .Here tI, a compact interval of the real line.
The following example shows that the notion of analog computation considered by Shannon and
Pour-El does not account for the limitations that are inherent in the general model of analog computer
developed in previous sections. From work of Plaisted [21] it is known that solution of the NP-
complete PARTITION problem is equivalent to the evaluation of a particular definite integral. The
integral may be computed as the solution to a differential equation of the form (8.1). However, PAR-
TITION is not strongly NP-complete, and a specialization of the arguments used in Section 3 shows
that the analog variables associated with the differential equation are not polynomially bounded.
- 17 -
(Certain derivatives grow exponentially, which cannot be remedied by scaling the independent vari-
able tt/τ, while keeping the time interval of interest polynomially bounded.)
From our perspective, the Shannon/Pour-El work, with its emphasis on computability, concerns the
weak form of Church’s Thesis. The PARTITION example suggests that further restrictions and
interpretations of the differential equation model are necessary in order to capture the inherent accu-
racy limitations of analog computation or to make a meaningful statement about complexity.
Our restricted differential equation model is derived from the form
dt
dY(t)=C(Y(t)) , Y(t0)=Y0(8.2)
where C(Y(t)) is a vector of rational functions of the elements of Y(t)=(1,t,y1(t) , ...,yn(t)) .
This form is equivalent to (8.1) when Ais nonsingular. We assume that the analog computer has a
precise (e.g. external) clock so that the first two components of Y(t) are redundant and tmay be used
to parametrize the functions (y1(t) , ...,yn(t)) ′ = y(t).Then (8.2) may be replaced by the
equivalent form
dt
dy(t)=f(y(t),t) , y(t0)=y0(8.3)
We make the usual assumption for well-posedness of the model, namely that fobeys a uniform
Lipschitz condition:
|| f(y1,t)f(y2,t)|| λ|| y1y2|| ,tI,
where λdoes not depend on t.
Our interpretation of the computation carried out by (8.3) is a variation on the standard initial value
problem of computing the value of y(tf) given [t0,tf]Iand y0Rn.Here y0is the ‘‘input’’ to the
analog computer which then operates over a fixed time interval to generate its ‘‘output.’’ We make the
following qualification to incorporate the absolute precision, ε, associated with the use of the differen-
tial equation to represent the operation of an analog computer. The value that is provided as an ‘‘out-
put’’ is any y*that approximates y(tf) in the sense that || y(tf)y*|| ≤ ε.
We adopt as our measure of the resources used by this analog computation
R=t0ttf
max || y
..(t)|| .
This assumes that (8.3) admits a solution whose second derivative exists and is continuous. Since the
derivative y
.cannot grow large in a fixed time interval without y
.. being large, this is a conservative
measure. (In a typical electronic analog computer, for example, the signal y
.appears as a physical
(voltage) signal at the integrating amplifier input. Since a real integrator has finite bandwidth, the time
derivative of its input must be bounded to assure accurate integration.)
We summarize this model for analog computation. It consists of the differential equation (8.3), with
the associated Lipschitz constant λand absolute precision constant ε.The solution at time tfwhen
the initial condition at time t0is y0and the precision constant εdetermine an equivalence class of
‘‘output’’ vectors. The maximum magnitude of the second derivative of the solution vector over the
interval [t0,tf], is used as a measure of the resources required by the computation.
Our result is that these analog computations can be efficiently simulated by a digital computer, the
strong form of Church’s Thesis.
- 18 -
Theorem 3. The differential equation model described above can be simulated by a digital computer
in a number of steps bounded by a polynomial in both R and 1/ε.
Proof We give a constructive proof based on the Euler method of numerical integration for the system
(8.3). (Our approach is based on standard techniques in numerical analysis; see [11].) For 0mN,
set tm=t0+mh where h=(tft0)/N. Then we take γ0=y0and
γm+1= γm+h f(γm,tm) , 0mN1
Supposing that γmis computed without roundoff error, the discretization error, em= γmy(tm) ,
satisfies
em+1=em+h[f(γm,tm)f(y(tm),tm)] h2y
..(ξ)/2
for some ξ,tm<ξ<tm+1.Using the Lipschitz condition gives
|| em+1|| || em|| (1+hλ)+h2R/2
which leads to the bound
|| eN|| h R[exp(tft0)λ − 1]/2λ
When fixed point numerical calculations are used, we have a roundoff error sequence rmdue to
finite precision computation. The numerical approximation is
γm+1
*= γm
*+[hf *(γm
*,tm)]*
where the *denotes rounded value. We define the local roundoff error by
δm=[hf *(γm
*,tm)]*h f(γm
*,tm)
so that we may write
γm+1
*= γm
*+hf(γm
*,tm)+ δm
If we assume that || δm|| <σ, then by an argument similar to the one used for discretization error, we
obtain
|| rN|| ≤ σ[exp(tft0)λ − 1]/hλ
Using the triangle inequality, we may combine these bounds to obtain a total error bound
|| y(tf)− γN
*|| h2
R+h2
σ[exp(tft0)λ − 1]/λ(8.4)
In order to obtain a solution to the same accuracy as the differential equation model for analog com-
putation, we must choose σand hso that this bound is no larger than ε.We have the relationship
h=(tft0)/N and we take σ = 1/N2for convenience. Then it is clear from (8.4) that the number
of discretization steps may be chosen to be proportional to Rand to 1/ε.Since fis rational, each step
involves additions, multiplications, and divisions in order to evaluate the approximate solution value.
To obtain the bound on local roundoff error of σrequires |log2(σ)|bits of accuracy, and so the effort
in evaluating each approximate value is proportional to log2
2(σ).With the choice of σ=1/N2, the
number of steps required by a digital computer to simulate the analog computer is O(Nlog2N), which
is bounded by a polynomial in Rand 1/εas was to be shown. This completes the proof.
- 19 -
What remains is to describe how such an analog computer may be used to solve combinatorial prob-
lems of the type described earlier; we first have the implicit encoding of each combinatorial variable
as an analog variable (possibly vector-valued but with fixed dimension) which appears as part of the
vector y(t).The initial value y(t0) encodes the values of the combinatorial input quantities. Finally,
we assume that the solution of the combinatorial problem may be obtained unambiguously from the
‘‘output,’’ which is to say that function assigning the value of the combinatorial solution is constant
on the sets (quantization bins) Q(y0)={y*:|| y(tf)y*|| ≤ ε} .
In view of the theorem above, this interpretation implies the following result.
Corollary. If a combinatorial problem can be solved on an analog computer of the type described
above, then it can be solved in polynomial time on a Turing machine.
In view of the proof of the theorem, it is possible to allow certain of the parameters of the differen-
tial equation to depend on the input in addition to the initial value y0.This may be important in appli-
cations because the number of variables, and hence the dimension of y(t), n, will generally depend
on the input. We may allow nand the complexity of the rational function components of fto grow
polynomially in the length of the input, and we may allow the corresponding Lipschitz constant, λ, to
grow logarithmically in the length of the input.
Recently, Hopfield and Tank [12] have discussed solving the strongly NP-complete TRAVELING
SALESMAN PROBLEM (TSP) with an analog electrical network whose description is given by a
coupled set of differential equations. Given a problem instance of TSP, they propose to design the
network in such a way that an associated potential (or Lyapunov) function achieves its global
minimum value at an equilibrium point of the network corresponding to the TSP solution path. Then,
if the initial conditions of the analog variables lie in the region of attraction of this particular equili-
brium point, the steady-state solution provides the desired solution to the problem instance. Empirical
studies of 10-city and 30-city problem instances are given in [12]; they indicate that while this analog
approach does not provide a method for obtaining exact (i.e. optimal) solutions to TSP instances, it
does offer a systematic technique for consistently generating good suboptimal solutions.
We view these empirical results as evidence for the validity of SCT in this context, which differs in
some details from that considered in Theorem 3 and its corollary. The analysis and interpretation in
[12] reinforces this view, especially in regard to two difficulties with constructing and operating the
network. First, the network and its associated potential function depend on some free parameters
(amplifier gains, etc.) that must be chosen by empirical means in order to ‘‘tune’’ the analog encoding
of TSP. Second, the choice of unbiased initial conditions is apparently difficult because of the sym-
metry in the network arising from multiple encodings of TSP solution paths without regard to
equivalence of tours (with respect to starting city and orientation).
Other points made by Hopfield and Tank suggest topics for further research in analog complexity
theory. These include a study of the suboptimal solutions obtained for the TSP, a study of the ‘‘fail-
soft’’ fault-tolerance properties of analog computers, a study of the use of penalty function techniques
to obtain constraint satisfaction in the underlying combinatorial problem, and a detailed examination
of proposed analog networks for the solution of other combinatorial problems, including ones in Digi-
tal P-time. Examples of the latter are found in [13,27].
- 20 -
9. The Spin Glass Computer
Recent work on the properties of certain magnetic alloys called spin glasses has led to an interesting
connection between physics and combinatorial optimization [1,15,17], and in fact suggests a physical
device for solving an NP-complete problem. The problem of finding a minimum-energy spin
configuration (the ground state) in a regular lattice model can be expressed as the following problem,
which is NP-complete [15].
GROUND STATE OF A SPIN GLASS: Given an H×L×Wrectangular lattice graph (H,L,Wposi-
tive integers) with edges between vertices that are adjacent in any of the three directions, an integer
interaction weight J(e) for each edge e, and an integer K, the spin energy, is there an assignment of
aspin s(v){1, +1}for each vertex vsuch that
all edges (u,v)
ΣJ(u,v)s(u)s(v)K?
The problem remains NP-complete when Jis restricted to the values {1, 0, +1}, so the problem
is strongly NP-complete. We can then view a piece of spin glass as a candidate (at least in theory) for
a computer that solves a strongly NP-complete problem, just as with our 3-SAT machine. (We leave
aside the problem of initializing the material with the problem ‘‘inputs.’’ If there is any way at all of
setting an interaction weight, the total time for preparing a piece of material should be no more than
polynomial in the number of lattice points. Similarly for reading output spin values.)
The actual operation of such a spin glass computer is similar to the operation of a mathematical pro-
gramming machine; it minimizes a multivariate function. By a natural extension of the point of view
discussed in this paper, suitably formalized to account for quantum mechanical models of systems
such as the spin glass system, we are led to a definite conclusion about the time it takes for our
hypothetical piece of spin glass to reach the ground state: if Strong Church’s Thesis is true, and if P
NP, then it must take an exponential amount of time. Note, however, that this result is worst case over
all inputs (interaction weights J). Thus it may be that almost all pieces of spin glass of a given size
will cool to the ground state fast, and this prospect supports the use of ‘‘simulated annealing’’ as a
heuristic for combinatorial optimization problems [17]. It is worth pointing out that an implementa-
tion of this heuristic requires a source of independent random variables, and so lies outside the realm
of complexity theory as discussed here. Still, a proof of convergence for this kind of stochastic relaxa-
tion algorithm has only been obtained under the assumption of an annealing schedule involving loga-
rithmically decreasing temperature [10]; the resulting algorithm requires exponential time which is in
agreement with our prediction based on assuming Strong Church’s Thesis and P NP.
10. Discussion
The question of how efficiently we can compute with general, non-digital, devices appears to be
difficult indeed. It touches on problems in both mathematics and physics. We have tried in this paper
to establish some link between the mathematical complexity theory of NP-completeness and classical
physics, but we have not dealt with quantum mechanics, or the problem of probabilistic behavior. We
have shown that the likely hypotheses P NP and Strong Church’s Thesis lead to the conclusion that
analog (non-digital) computers are no more efficient than digital computers, at least in the worst-case
over problem inputs, and asymptotically with the problem size. Of course these two hypotheses are
- 21 -
important open questions, but we have been able to prove a restricted form of Strong Church’s Thesis
and perhaps more general results will be forthcoming.
In recent work of Bennett [2,3], there have been discussions that are germane to analog computation
and efficient simulation. He has suggested that efficient simulation of physical systems up to the errors
induced by uncontrollable (environmental) influences should be possible. It is our view that the
effects of uncontrollable influences must be incorporated into the mathematical model of a physical
process, given for example by a system of differential equations, as a fundamental part of the descrip-
tion of the corresponding analog computational process.
As shown by our differential equation model, certain smoothness properties of the mathematical
model can provide a natural measure for the resources used. In particular it is the second derivative of
the analog variables that appears as the natural measure. It is interesting to note that in the work of
Pour-El and Richards [23,24], where it was shown that the three-dimensional wave equation can
transform computable initial data into noncomputable solution values, imposition of continuity condi-
tions on the second derivative will prevent this phenomenon.
Acknowledgments
We thank Karl Lieberherr, Richard Lipton, Jacobo Valdes, and Stephen Wolfram for valuable dis-
cussions about the subject of this paper.
References
[1] F. Barahona, ‘‘On the computational complexity of Ising spin glass models,’’ J. Phys. A 15 (1982) 3241-3253.
[2] C.H. Bennett, ‘‘The thermodynamics of computation - a review,’’ Internat. J. Theoret. Phys. 21 (1982) 905-940.
[3] C.H. Bennett, ‘‘On the logical ‘depth’ of sequences and their reducibilities to random sequences,’’ preprint; to appear in
Inform. and Control.
[4] V. Bush, ‘‘The differential analyzer,’’ J. Franklin Inst. 212 (1931) 447-488.
[5] L.O. Chua and G-N. Lin, ‘‘Nonlinear programming without computation,’’ IEEE Trans. Circuits and Systems CAS-31
(1984) 182-188.
[6] A. Church, ‘‘An unsolvable problem of elementary number theory,’’ Amer. J. Math. 58 (1936) 345-363. (Reprinted in
[7].)
[7] M. Davis, The Undecidable, (Raven Press, Hewlett, NY, 1965).
[8] R.P. Feynman, ‘‘Simulating physics with computers,’’ Internat. J. Theoret. Phys. 21 (1982) 467-488.
[9] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness, (W. H. Free-
man and Company, San Francisco, CA, 1979).
[10] S. Geman and D. Geman, ‘‘Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images,’’ IEEE
Trans. Pattern Analysis Machine Intell. PAMI-6 (1984) 721-741.
[11] P. Henrici, Discrete Variable Methods in Ordinary Differential Equations, (John Wiley, New York, NY, 1962).
[12] J.J. Hopfield and D.W. Tank, ‘‘ ‘Neural’ computation of decisions in optimization problems,’’ Biol. Cybernet. 52 (1985)
1-12.
[13] J.J. Hopfield and D.W. Tank, ‘‘Collective computation with continuous variables,’’ preprint; to appear in Disordered
Systems and Biological Organization.
[14] A. Jackson, Analog Computation, (McGraw-Hill, New York, NY, 1960).
[15] D.S. Johnson, ‘‘The NP-completeness column: an ongoing guide,’’ J. Algorithms 4 (1983) 87-100.
[16] W. Karplus and W. Soroka, Analog Methods, 2nd. Ed., (McGraw-Hill, New York, NY, 1959).
[17] S. Kirkpatrick, C.D. Gelatt, Jr., and M.P. Vecchi, ‘‘Optimization by simulated annealing,’’ Science 220 (1983) 671-680.
- 22 -
[18] W. Miehle, ‘‘Link-length minimization in networks,’’ Oper. Res. 6 (1958) 232-243.
[19] C.H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity, (Prentice-Hall, Engle-
wood Cliffs, NJ, 1982).
[20] R.M. Phelan, Fundamentals of Mechanical Design, 3rd. Ed., (McGraw-Hill, New York, NY, 1970) 433-434.
[21] D. Plaisted, ‘‘Some polynomial and integer divisibility problems are NP-hard,’’ Proc. 17th Ann. Symp. on Foundations
of Computer Science (1976) 264-267.
[22] M.B. Pour-El, ‘‘Abstract computability and its relation to the general purpose analog computer (some connections
between logic, differential equations, and analog computers),’’ Trans. Amer. Math. Soc. 199 (1974) 1-28.
[23] M.B. Pour-El and I. Richards, ‘‘The wave equation with computable initial data such that its unique solution is not com-
putable,’’ Adv. in Math. 39 (1981) 215-239.
[24] M.B. Pour-El and I. Richards, ‘‘Noncomputability in models of physical phenomena,’’ Internat. J. Theoret. Phys. 21
(1982) 553-555.
[25] I.B. Pyne, ‘‘Linear programming on an electronic analogue computer,’’ Trans. AIEE, Part I, 75 (1956) 139-143.
[26] C.E. Shannon, ‘‘Mathematical theory of the differential analyzer,’’ J. Math. Phys. 20 (1941) 337-354.
[27] D.W. Tank and J.J. Hopfield, ‘‘Simple ‘neural’ optimization networks: an A/D converter, signal decision circuit and a
linear programming circuit,’’ preprint, July 8, 1985.
[28] A. M. Turing, ‘‘On computable numbers, with an application to the Entscheidungsproblem,’’ Proc. London Math. Soc.,
Series 2 42 (1936-7) 230-265; corrections ibid 43 (1937) 544-546. (Reprinted in [7].)
  • Preprint
    Full-text available
    We present a survey on analog models of computations. Analog can be understood both as computing by analogy, or as working on the continuum. We consider both approaches, often intertwined, with a point of view mostly oriented by computation theory.
  • Article
    We discuss some claims that certain UCOMP devices can perform hypercomputation (compute Turing-uncomputable functions) or perform super-Turing computation (solve NP-complete problems in polynomial time). We discover that all these claims rely on the provision of one or more unphysical resources.
  • Conference Paper
    Entanglement is not only the key resource for many quantum technologies, but essential in understanding the structure of many-body quantum matter. At the interface of these two crucial areas are simulators, controlled systems capable of mimicking physical models that might escape analytical tractability. Traditionally, these simulations have been performed classically, where recent advancements such as tensor-networks have made explicit the limitation entanglement places on scalability. Increasingly however, analog quantum simulators are expected to yield deep insight into complex systems. This thesis advances the field in across various interconnected fronts. Firstly, we introduce schemes for verifying and distributing entanglement in a quantum dot simulator, tailored to specific experimental constraints. We then confirm that quantum dot simulators would be natural candidates for simulating many-body localization (MBL) - a recently emerged phenomenon that seems to evade traditional statistical mechanics. Following on from that, we investigate MBL from an entanglement perspective, shedding new light on the nature of the transition to it from a ergodic regime. As part of that investigation we make use of the logarithmic negativity, an entanglement measure applicable to many-body mixed states. In order to tie back into quantum simulators, we then propose an experimental scheme to measure the logarithmic negativity in realistic many-body settings. This method uses choice measurements on three or more copies of a mixed state along with machine learning techniques. We also introduce a fast method for computing many-body entanglement in classical simulations that significantly increases the size of system addressable. Finally, we introduce quimb, an open-source library for interactive but efficient quantum information and many-body calculations. It contains general purpose tensor-network support alongside other novel algorithms.
  • Preprint
    There is a widespread view that the human brain is so complex that it cannot be efficiently simulated by universal Turing machines, let alone ordinary classical computers. During the last decades the question has therefore been raised whether we need to consider quantum effects to explain the imagined cognitive power of a conscious mind. This paper presents a personal view of several fields of philosophy and computational neurobiology in an attempt to suggest a realistic picture of how the brain might work as a basis for perception, consciousness and cognition. The purpose is to be able to identify and evaluate instances where quantum effects might play a significant role in cognitive processes. Not surprisingly, the conclusion is that quantum-enhanced cognition and intelligence are very unlikely to be found in biological brains. Quantum effects may certainly influence the functionality of various components and signalling pathways at the molecular level in the brain network, like ion ports, synapses, sensors, and enzymes. This might evidently influence the functionality of some nodes and perhaps even the overall intelligence of the brain network, but hardly give it any dramatically enhanced functionality. So, the conclusion is that biological quantum networks can only approximately solve small instances of NP-hard problems. On the other hand, artificial intelligence and machine learning implemented in complex dynamical systems based on genuine quantum networks can certainly be expected to show enhanced performance and quantum advantage compared with classical networks. Nevertheless, even quantum networks can only be expected to solve NP-hard problems approximately. In the end it is a question of precision - Nature is approximate.
  • Preprint
    In classical computing, analog approaches have sometimes appeared to be more powerful than they really are. This occurs when resources, particularly precision, are not appropriately taken into account. While the same should also hold for analog quantum computing, precision issues are often neglected from the analysis. In this work we present a classical analog algorithm for unstructured search that can be viewed as analogous to the quantum adiabatic unstructured search algorithm devised by Roland and Cerf [Phys. Rev. A 65, 042308 (2002)]. We show that similarly to its quantum counterpart, the classical construction may also provide a quadratic speedup over standard digital unstructured search. We discuss the meaning and the possible implications of this result in the context of adiabatic quantum computing.
  • Preprint
    Full-text available
    We present both an overview and a perspective of recent experimental advances and proposed new approaches to performing computation using memristors. A memristor is a 2-terminal passive component with a dynamic resistance depending on an internal parameter. We provide an brief historical introduction, as well as an overview over the physical mechanism that lead to memristive behavior. This review is meant to guide nonpractitioners in the field of memristive circuits and their connection to machine learning and neural computation.
  • Article
    Full-text available
    Conservation principles, such as conservation of charge, energy, or mass, provide a natural way to couple and constrain spatially separated variables. In this paper, we propose a dynamical system model that exploits these constraints for solving nonconvex and discrete global optimization problems. Unlike the traditional simulated annealing or quantum annealing-based global optimization techniques, the proposed method optimizes a target objective function by continuously evolving a driver functional over a conservation manifold, using a generalized variant of growth transformations. As a result, the driver functional asymptotically converges toward a Dirac-delta function that is centered at the global optimum of the target objective function. In this paper, we provide an outline of the proof of convergence for the dynamical system model and investigate different properties of the model using a benchmark nonlinear optimization problem. Also, we demonstrate how a discrete variant of the proposed dynamical system can be used for implementing decentralized optimization algorithms, where an ensemble of spatially separated entities (for example, biological cells or simple computational units) can collectively implement specific functions, such as winner-take-all and ranking, by exchanging signals only with its immediate substrate or environment. The proposed dynamical system model could potentially be used to implement continuous-time optimizers, annealers, and neural networks.
  • Article
    There is a widespread view that the human brain is so complex that it cannot be efficiently simulated by universal Turing machines, let alone ordinary classical computers. During the last decades the question has therefore been raised whether it is needed to consider quantum effects to explain the imagined cognitive power of a conscious mind. Not surprisingly, the conclusion is that quantum‐enhanced cognition and intelligence are very unlikely to be found in biological brains. Quantum effects may certainly influence signaling pathways at the molecular level in the brain network, like ion ports, synapses, sensors, and enzymes. This might evidently influence the functionality of some nodes and perhaps even the overall intelligence of the brain network, but hardly give it any dramatically enhanced functionality. The conclusion is that biological quantum networks can only approximately solve small instances of nonpolynomial (NP)‐hard problems. On the other hand, artificial intelligence and machine learning implemented in complex dynamical systems based on genuine quantum networks can certainly be expected to show enhanced performance and quantum advantage compared with classical networks. Nevertheless, even quantum networks can only be expected to solve NP‐hard problems approximately. In the end it is a question of precision—Nature is approximate.
  • Article
    As we approach the end of the silicon road map, alternative computing models that can solve at-scale problems in the data-centric world are becoming important. This is accompanied by the realization that binary abstraction and Boolean logic, which have been the foundations of modern computing revolution, fall short of the desired performance and power efficiency. In particular, hard computing problems relevant to pattern matching, image and signal processing, optimizations, and neuromorphic applications require alternative approaches. In this paper, we review recent advances in oscillatory dynamical system-based models of computing and their implementations. We show that simple configurations of oscillators connected using simple electrical circuits can result in interesting phase and frequency dynamics of such coupled oscillatory systems. Such networks can be controlled, programmed, and observed to solve computationally hard problems. Although our discussion in this paper is limited to insulator-to-metal transition devices and spin-torque oscillators, the general philosophy of such a computing paradigm of “let physics do the computing” can be translated to other mediums as well, including micromechanical and optical systems. We present an overview of the mathematical treatments necessary to understand the time evolution of these systems and highlight the recent experimental results in this area that suggest the potential of such computational models.
  • Chapter
    The language of computing to describe physical processes has become popular in a number of scientific fields. However, without a clear definition of computing outside very narrow domains, such usage fails to add content to our understanding of physical reality. In this paper I explore how the theory of these specific engineered devices can possibly help us understand fundamental science, by close consideration of the connection between abstract computational theory and physical computing devices. Using the recently developed formalism of Abstraction/Representation Theory, I show what it means for a physical system to be acting as a computer, and give the conditions for a system to be capable of supporting a computational representation. A computational representation gives nontrivial information about the underlying physical system; but not every system under every physical theory is necessarily capable of supporting such a representation. In the cases where it is possible to represent a system computationally, this then becomes a new language and logic in which to describe, understand, and investigate the fundamental processes of physical reality.