Page 1

Visualizing and Constructing Cycles

in The Simplex Method

David Avis, Bohdan Kaluzny, and David Titley-P´ eloquin

April 4th, 2005

School of Computer Science, McGill University

Montreal, Quebec, Canada

{avis,beezer,dtitle}@cs.mcgill.ca

“Cycling is certainly not completely understood.” - Hoffman, 1951.

Abstract

Most examples of cycling in the simplex method are given without explanation of how they were

constructed. An exception is Beale’s 1955 example built around the geometry of the dual simplex

method in the plane. Using this approach we give a simple geometric explanation for a number of

examples of cycling in the simplex method, including Alan Hoffman’s original example. This gives

rise to a simple method for generating examples with cycles.

La plupart des exemples du cycle dans la m´ ethode simplexe sont donn´ es sans explication de

la fa¸ con dont ils ont ´ et´ e construits. Une exception est l’exemple de Beale (1955) ´ etabli utilisant

de la g´ eom´ etrie de la m´ ethode simplexe duelle dans deux dimensions. En utilisant cette approche

nous donnons une explication g´ eom´ etrique simple pour plusiers exemples du cycle dans la m´ ethode

simplexe, y compris l’exemple original de Alan Hoffman. Ceci provoque une m´ ethode simple pour

produire des exemples avec des cycles.

1Introduction

What is the magic behind cycling in the simplex method? Every textbook on linear programming

dedicates a section to cycling in the simplex method; however, no single textbook provides the reader

1

Page 2

with a geometric intuition behind cycling. This list includes the classic textbooks of Chv´ atal [3]

and Dantzig [4], and recent books such as Sierksma [11]. The examples are usually presented as a

sequence of tableaux or dictionaries - if we follow the terminology and notation used by Chv´ atal. Such

presentations are sure to convince the reader that simplex methods can cycle, but do not provide any

insight behind the construction. Consider the first cycling example, concocted by Hoffman in 1951

[7], [4, p.229], with starting dictionary

x0= 1

x1= −cosϕx3+ ω cosϕx4− cos2ϕx5+ 2ω cos2ϕx6

− cos2ϕx7− 2ω cos2ϕx8− cosϕx9− ω cosϕx10

x2= −[(tanϕsinϕ)/ω]x3− cosϕx4− [(tanϕsin2ϕ)/ω]x5− cos2ϕx6

+ 2(sin2ϕ/ω)x7− cos2ϕx8+ [(tanϕsinϕ)/ω]x9− cosϕx10

z = [(1 − cosϕ)/cos ϕ]x3− ωx4− 2ωx6− 4sin2ϕx7

+ 2ω cos2ϕx8− 4sin2ϕx9− ω(1 − 2cosϕ)x10.

If we set ϕ = 2π/5,ω > (1 − cos2ϕ)/(1 − 2cosϕ) ≈ 4.74 the simplex method using Dantzig’s rule

cycles through a sequence of 10 dictionaries1. The left hand side variables cycle repeatedly through

the sequence {x0,x1,x2}, {x0,x2,x3}, ... , {x0,x9,x10}, {x0,x10,x1}.

How was this example constructed? Strangely, even Hoffman forgot!

“Finally, we regret that we are unable at this date to recall any details of the consid-

erations that led to the construction of the example, beyond the fact that the geometric

meaning of [a pivot rule for the simplex method] was very much in the foreground.” -

Hoffman, 1951.

“Two months after I made up the example, I lost the mental picture which produced

it...” - Hoffman, 1994.

This is a shame, as geometry can certainly help students understand the simplex method through

a visual interpretation. In 1997 Lee [8] successfully untangled the algebraic derivation of Hoffman’s

example, and using column geometry (originally known as the simplex interpretation [4, p.160]) pro-

duced a three-dimensional picture. In Figure 1 we exhibit Hoffman’s example in two-dimensions using

the geometry of the dual simplex method, which was first explained and used to construct cycles by

Beale [2], but not used to analyze Hoffman’s example:

“Unfortunately, the geometric motivation behind Hoffman’s example is not easy to

grasp” - Beale, 1955.

In the next section we see how cycling works in this setting, by giving a simplified version of this

geometrical interpretation. We show that cycling examples obtained from various linear programming

1Note that in [4], [6], [7], and [8] the bound on ω reads ω > (1−cosϕ)/(1−2cosϕ) ≈ 1.81. In fact, the tighter bound

presented here is required for Hoffman’s LP to cycle using Dantzig’s pivot rule. See Appendix A for details.

2

Page 3

texts have a similar structure, and many are combinatorially equivalent. In Section 3 we give a simple

method of constructing examples that contain cycles of arbitrary even length.

1

3

5

7

6

8

4

2

10

9

Figure 1: A picture of Hoffman’s cycle

2Visualizing Cycles

Given a linear program,

maxz =

n

?

j=1

cjxj

n

?

j=1

aijxj≤ bi

i = 1...m

xj≥ 0j = 1...n,

we add nonnegative slack variables xn+1,...,xn+mto obtain a system of linear equations,

n

?

j=1

aijxj+ xn+i= bi

i = 1,...,m. (1)

Solving these m equations for m−n variables we obtain a dictionary. For example, the linear program

max z = 10x1− 57x2− 9x3− 24x4

(2)

0.5x1− 5.5x2− 2.5x3+ 9x4≤ 0

0.5x1− 1.5x2− 0.5x3+ x4≤ 0

x1,x2,x3,x4≥ 0

is encoded by the dictionary

x5= 0 − 0.5x1+ 5.5x2+ 2.5x3− 9x4

(3)

x6= 0 − 0.5x1+ 1.5x2+ 0.5x3− x4

z = 0 + 10x1− 57x2− 9x3− 24x4,

3

Page 4

and each variable xj corresponds an inequality of (2).

the cobasic (right-hand side) variables to zero and evaluating the basic (left-hand side) variables, is

nonnegative then the dictionary represents a basic feasible solution for the linear program, the value of

z being the objective value. Simplex methods attempt to increase the objective value by exchanging

a cobasic variable xswith positive coefficient in the z-row with a basic variable xrwhile maintaining

a basic feasible solution. This is accomplished via a ratio test which chooses xr such that 1) the

coefficient of xsin equation xris negative and 2) solving this equation for xsand substituting for xsin

the other equations then yields another feasible basic solution. This basic operation is called a simplex

pivot. Pivot rules (or refinements) dictate which variables to select; for example, following Dantzig’s

pivot rule [4] we choose the variable in the z−row with largest positive coefficient. We break ties in

the ratio test arbitrarily (usually by selecting the variable with the smallest index). In our example we

would pivot on variables x1and x5, the former chosen as it has the largest positive coefficient in the

z−row and the latter arbitrarily chosen from x5and x6due to a ratio test tie - caused by degeneracy

- yielding the dictionary

If the basic solution, obtained by setting

x1= 11x2+ 5x3− 18x4− 2x5

x6= −4x2− 2x3+ 8x4+ x5

(4)

z = 53x2+ 41x3− 204x4− 20x5.

Degeneracy may cause the simplex method to stall, or pivot from dictionary to dictionary without

increasing the objective value. The simplex method cycles if it stalls indefinitely, revisiting previously

computed dictionaries. Continuing with our example, the simplex method with Dantzig’s rule visits

the dictionaries

x1= −0.5x3+ 4x4+ 0.75x5− 2.75x6

x2= −0.5x3+ 2x4+ 0.25x5− 0.25x6

(5)

z = 14.5x3− 98x4− 6.75x5− 13.25x6,

x2= x1− 2x4− 0.5x5+ 2.5x6

x3= −2x1+ 8x4+ 1.5x5− 5.5x6

(6)

z = −29x1+ 18x4+ 15x5− 93x6,

x3= 2x1− 4x2− 0.5x5+ 4.5x6

x4= 0.5x1− 0.5x2− 0.25x5+ 1.25x6

(7)

z = −20x1− 9x2+ 10.5x5− 70.5x6,

x4= −0.5x1+ 1.5x2+ 0.5x3− x6

x5= 4x1− 8x2− 2x3+ 9x6

(8)

z = 22x1− 93x2− 21x3+ 24x6,

4

Page 5

and returns to dictionary (3), cycling through the sequence of bases {5,6}, {1,6}, {1,2}, {2,3}, {3,4},

{4,5}, {5,6}... The six inequalities of the linear program (2) define a four-dimensional polyhedron,

and the simplex method cycles among the bases of a single vertex. This geometry cannot help us

visualize the cycle. In 1955, Beale [2] constructed a cycle for the simplex method by considering the

geometry of the dual problem. The dual of (2) is

min w = 0y1+ 0y2

(9)

(1) : 0.5y1+ 0.5y2≥ 10

(2) : −5.5y1− 1.5y2≥ −57

(3) : −2.5y1− 0.5y2≥ −9

(4) : 9y1+ y2≥ −24

(5) : y1≥ 0,

(6) : y2≥ 0

which has six inequalities in two dimensions. We can also write the dual problem in dictionary form.

In fact, the dual dictionary is simply a disguised version of the (primal) dictionary for the primal: the

transposed system with signs reversed.

x5= 0 − 0.5x1+ 5.5x2+ 2.5x3− 9x4

x6= 0 − 0.5x1+ 1.5x2+ 0.5x3− x4

z = 0 + 10x1− 57x2− 9x3− 24x4

←→

y3= −10 + 0.5y1+ 0.5y2

y4= 57 − 5.5y1− 1.5y2

y5= 9 − 2.5y1− 0.5y2

y6= 24 + 9y1+ y2

w = 0 + 0y1+ 0y2.

Table 2. Primal and dual dictionaries

In the dual simplex method the w−row is assumed to have all coefficients non-negative. The

leaving basic variable is chosen from the basic variables with negative constant on the right hand side

of the dictionary. A ratio test is used to choose the entering variable so that the w−row remains

non-negative. It is easy to verify that the dual simplex method applied to the dual problem will, with

a suitably chosen pivot rule, follow the same sequence of pivots as the primal simplex method does on

the primal problem. The sequence of bases for the primal problem becomes the sequence of cobases

for the dual problem, and vice versa. Graphing (9), the geometry behind a simplex method cycle

begins to unveil. We obtain an arrangement of lines, each bounding an inequality of (9). Figure 2

depicts this arrangement.

The two-dimensional geometric interpretation of the dual simplex method is simple when the

objective function is zero. For the time being, let’s ignore specific pivot rules and consider simplex

pivots in general. Every line in the arrangement corresponds to a variable in the dual dictionary. The

slack variable yifor i = 3...n corresponds to line i − 2, while y1and y2correspond to lines n − 1 and

n respectively.

5

Page 6

−1010 20

100

5

−20

40

60

6

4

20

80

1

3

2

y

y2

1

Figure 2: Chv´ atal [3]

Each intersection of lines corresponds to a basic dual solution obtained by setting the two corre-

sponding variables to zero. These are the two dual cobasic variables for a corresponding dual dictionary.

For example, the dual dictionary in Table 2 with cobasis {1,2} corresponds to the intersection of lines

labelled 5 and 6. The basic variable chosen in a pivot corresponds to a violated inequality: in our case

the intersection point lies on the wrong side of line 1, so y3is the leaving basic variable. The entering

variable is chosen from those with a positive coefficient in the equation for the leaving variable, so that

that the new basic solution will satisfy the inequality corresponding to the entering variable. In the

example we could choose either y1or y2as the entering variable. However from the cobasis {1,6} we

could pivot to the cobasis {1,2} since this is on the correct side of line 4. We could not pivot to cobasis

{3,6} as this is on the wrong side of line 5. A negative basic variable indicates a degree of infeasibility,

for example y3= −10 + 0.5y1+ 0.5y2indicates that at the intersection of line 5 and line 6, line 1 is

infeasible by

0.5units along line 5 and by

{y1,y2}. The dual simplex method terminates when it reaches a point where either no inequalities

are violated (optimality) or when it cannot pivot to a violated inequality without violating one of the

two intersecting lines (dual infeasible).

It is now routine to check that the cycles shown in Figure 1 for Hoffman’s example, and Figure

2 for our example correspond to valid dual simplex pivots. It follows that the primal problems pivot

through the same cycles, with bases/cobases interchanged.

10

10

0.5units along line 6 with respect to the vector space basis

6

Page 7

Our example is derived from the following example found in Chv´ atal’s book. The first dictionary

of the cycle is presented as

x5= −0.5x1+ 5.5x2+ 2.5x3− 9x4

(10)

x6= −0.5x1+ 1.5x2+ 0.5x3− x4

x7= 1 − x1

z = 10x1− 57x2− 9x3− 24x4,

and consists of seven variables and three equations. However, variable x7never leaves the basis as the

cycle only pivots on the variables x1through x6. The equation x7= 1 − x1only serves to bound the

linear program but can be removed from dictionary (10) without affecting the cycle. It turns out that

many other examples found in literature, including Hoffman’s, can be reduced in this way and then

graphed in two-dimensions. In fact, many are then combinatorially equivalent.

5

3

4

1

2

6

Beale [2]

x5= 0 −1

x6= 0 −1

z = 0 +3

4x1+ 60x2+

2x1+ 90x2+

4x1+ 150x2+1

1

25x3− 9x4

1

50x3− 3x4

50x3− 6x4

5

4

1

6

2

3

Sierksma [11]

x5= 0 − x1+ 32x2+ 4x3− 36x4

x6= 0 − x1+ 24x2+ x3− 6x4

z = 0 + 3x1− 80x2+ 2x3− 24x4

1

2

5

6

3

4

Yudin and Gol’shtein [13]

x1= 0 − 2x3+ 3x4+ 5x5− 6x6

x2= 0 − 6x3+ 5x4+ 3x5− 2x6

z = 0 + x3− x4+ x5− x6

1

2

5

6

3

4

Yudin and Gol’shtein [13]

x1= 0 − x3+ 2x4+ 3x5− 4x6

x2= 0 − 4x3+ 3x4+ 2x5− x6

z = 0 + x3− x4+ x5− x6

7

Page 8

4

3

6

5

2

1

Solow [12]

x1= 0 + 7x3+ 3x4− 7x5− 2x6

x2= 0 − 2x3− x4+ 3x5+ x6

z = 0 + 2x3+ 2x4− 8x5− 2x6

1

2

3

4

5

6

Marshall and Suurballe [9]

x1= 0 −1

x2= 0 −1

z = 0 − x3+ 7x4+ 1x5− 2x6

2x3+11

2x3+3

2x4+5

2x4+1

2x5− 9x6

2x5− x6

−101020

100

5

−20

40

60

6

4

20

80

1

3

2

y

y2

1

Chv´ atal [3]

x5= 0 − 0.5x1+ 5.5x2+ 2.5x3− 9x4

x6= 0 − 0.5x1+ 1.5x2+ 0.5x3− x4

z = 0 + 10x1− 57x2− 9x3− 24x4

Table 2. Some examples of cycling found in literature

1

5

7

6

4

2

Kuhn [1]

x1= 0 + 2x4+ 9x5− x6− 9x7

x2= 0 −1

z = 0 + 2x4+ 3x5− x6− 12x7

3x4− x5+1

3x6+ 2x7

In all eight examples in Table 2, the primal linear program (after reduction to 2 equations) is

unbounded. Thus, by duality theory, all six dual linear programs above are infeasible. Nonetheless,

the simplex method cycles without detecting this fact. (Note that the shaded region in Figure 1

corresponds to the feasible set of Hoffman’s dual linear program).

In no way is the geometry described in this section constrained to cycles with m = 2, as when m = d

oriented lines become d-dimensional oriented hyperplanes and a dual simplex pivot has an analogous

description to its m = 2 counterpart. The geometry can be used to explain cycling exampes with

m ≥ 3 such as [5],[9] and [10]. We choose to restrict ourselves to the two-dimensional interpretation

for simplicity.

8

Page 9

3Constructing Cycles

Every two-dimensional arrangement of k inequalities can be encoded as a primal dictionary with k

equations and k+2 variables. For the time being, let’s ignore specific pivot rules and consider simplex

pivots in general. Only two basic rules need to be followed to build a simplex method cycle:

1. Each pivot must be a simplex pivot: the pivot step from the intersection of lines r and s to

the intersection of lines r and t is a simplex pivot if and only if the inequality bounded by t is

violated at {r,s} and the inequality bounded by s is satisfied at {r,t}.

2. The cycle must be represented by closed loop in the diagram.

The path consists of an alternating series of left and right turns in the line arrangement, as the

next point must always satisfy the two currently intersecting lines.

valid

invalid

L

R

invalid

valid

R

L

Table 3. Left and right turns

We use this fact to construct a cycle from scratch and the reader can follow the construction in

Figure 3. First, using line segments we draw a path consisting of an alternating series of left and right

turns that forms a closed loop. Note that the number of points must be even to form a closed loop,

and the loop can indeed be self-crossing (as in Figure 4). Next, we choose coordinate axes (y1,y2),

and lines bounding y1≥ 0,y2≥ 0 so that the loop lies in the positive quadrant.

Figure 3

→

y

1

2

3

6

8

7

5

4

y

1

2

→

min0y1+ 0y2

(1) : −y1≤ −10

(2) : y1≤ 15

(3) : −y2≤ −10

(4) : y2≤ 15

(5) : 10y1+ y2≤ 175

(6) : −10y1− y2≤ −100

(7) : y1− 10y2≤ −70

(8) : −y1+ 10y2≤ 150

(9),(10) : y1,y2≥ 0

We then fit a line through each line segment in the loop. For each line assign a direction of

feasibility which coincides with the path. Compute these inequalities with respect to (y1,y2), and

add a zero objective function to obtain a dual LP. By duality, we convert to a primal LP, add slack

variables, and compute an initial dictionary:

9

Page 10

x2= 0 + x1− 10x3+ 10x4− 101x7+ 101x8+ x9− 10x10

(11)

x5= 0 + x3− x4+ x6+ 10x7− 10x8+ x10

z = −5x1− 15x3+ 10x4− 75x6− 165x7+ 85x8− 15x9− 25x10.

Now simplex pivots in our diagram correspond to simplex pivots in the primal linear program, cy-

cling repeatedly through the sequence {x2,x5}, {x2,x8}, {x4,x8}, {x4,x6}, {x1,x6}, {x1,x7}, {x3,x7},

{x3,x5}2.

In order for the cycle to follow a specific pivot rule, an additional step is required. Note that for

each inequality, ai1y1+ai2y2≤ bifor i = 1...8, we can scale both sides by a positive constant Ciat will

as ai1y1+ ai2y2≤ bi⇔ Ci(ai1y1+ ai2y2) ≤ Cibi. These constants in turn scale the primal variables,

yielding the dictionary,

x2= 0 +

1

C2x1−10

1

C5x3−

C2x3+10

1

C5x4+

C2x4−101

1

C5x6+10

C2x7+101

C5x7−10

C2x8

(12)

x5= 0 +

C5x8

z = −5C1x1− 15C3x3+ 10C4x4− 75C6x6− 165C7x7+ 85C8x8.

By pivoting along the desired cycle, the corresponding dictionaries constrain the values of Ciwe

can choose so that the pivot rule is followed. For example, for the first pivot to follow Dantzig’s rule we

need 85C8> 10C4in the primal dictionary. We accumulate all the constraints on Ciin this manner,

and only if we find satisfying values for the Ci’s then following the pivot rule yields the cycle. (The

reader can easily check that for any dictionary the coefficient for variable xjin the z−row is strictly

a multiple of Cj). In our example we need 85C8> 10C4, 85C6> 10C1, 80C7> 10C3, 135C5> 15C2,

thus setting Ci= 1 for i = 1...8 results in a simplex method cycle following Dantzig’s pivot rule.

This process has a geometric analogue that can guide us to construct a diagram to ensure that

Dantzig’s rule (or any other coefficient based rule) will follow the loop. The sufficient condition is as

follows: given a diagram, for each point along the loop we list the pivot rule requirements for the cycle

to be followed. For example, at point A in Figure 4 we want to pivot to violated line 1 instead of line

2 or 6 which are also violated at A. This translates into constraints C1> C2and C1> C6. Similar

constraints are obtained from all points B,C,D,E and F along the cycle. If the set of constraints are

satisfiable, as is the case here, then there exist Ci’s large enough such that Dantzig’s rule will cycle

on the resulting LP. Of course the challenge is to draw a diagram with this in mind!

2We can ignore variables x9and x10as they are not involved in the cycle. They correspond, respectively, to inequalities

y1 ≥ 0 and y2 ≥ 0 of the dual LP which are only used to specify the coordinate axes

10

Page 11

6

5

3

2

D

F

E

A

B

C

1

4

Figure 4: Crossing loop.

point A : C4bigger than C2and C6

point B : C6bigger than C2

point C : only line 2 violated

point D : C3bigger than C1and C5

point E : C5bigger than C1

point F : only line 1 violated

References

[1] M. Balinski, A. Tucker, Duality theory of linear programs - a constructive approach with appli-

cations, SIAM Review 11 (3) (1969) 347–77.

[2] E. Beale, Cycling in the dual simplex method, Naval Research Logistics Quarterly 2 (4) (1955)

269–75.

[3] V. Chv´ atal, Linear Programming, Freeman, New York, 1983.

[4] G. Dantzig, Linear Programming and Extensions, Princeton University Press, NJ, 1963

[5] K. Fukuda, H-J L¨ uthi, Optimization Techniques (course notes), 1999.

[6] S. Gass, Linear Programming: Methods and Applications, 5th edition, McGraw-Hill Book Com-

pany, New York, 1985.

[7] A. Hoffman, Cycling in the simplex algorithm, National Bureau of Standards, Washington, 1953.

[8] J. Lee, Hoffman’s circle untangled, SIAM Review 39 (1997) 98–105.

[9] K. Marshall, J. Suurballe, A note on cycling in the simplex method, Naval Research Logistics

Quarterly 16 (1) (1969) 121–37.

[10] E. Nering, A. Tucker, Linear Programs and Related Problems, Academic Press, Boston, 1993.

[11] G. Sierksma, Linear and Integer Programming, 2nd ed. Marcel Dekker Inc., New York, 1996.

[12] D. Solow, Linear Programming: An Introduction to Finite Improvement Algorithms, North-

Holland Press, Amsterdam, 1984.

[13] D. Yudin, E. Gol’shtein, Linear Programming, Israel Program of Scientific Translations,

Jerusalem, 1965.

11

Page 12

AHoffman’s ω: The Correct Bound

All references to Hoffman’s example, for example [4], [6], [7], and [8], state an incorrect lower bound

ω >

1−2·cos(φ)≈ 1.81. If we set the value of ω close to this bound the simplex method with Dantzig’s

rule terminates. We now compute the correct lower bound. Consider the objective function of the

initial dictionary,

1−cos(φ)

z =1 − cos(φ)

cos(φ)

+ 2 · w · cos(2φ) · x8− 4 · sin2(φ) · x9+ ω · [2 · cos(φ) − 1] · x10.

· x3− ω · x4− 2 · w · x6− 4 · sin2(φ) · x7

For positive values of ω, x3 is the only choice to enter the basis since

coefficient in the z-row (cos(2φ) < 0 and [2 · cos(φ) − 1] < 0 becauseπ

We pivot x3with x1and obtain the second dictionary with objective function

1−cos(φ)

cos(φ)

is the only positive

3< φ <π

2).

z =cos(φ) − 1

cos(φ)

+1 − cos(φ)

· x1+ ω ·1 − 2 · cos(φ)

cos(φ)

x4− 2 · sin(φ) · tan(φ) · x5− 2 · ω · cos(φ) · x6

cos(φ)

· x7− 3 · ω · x8− 2 · sin(φ) · tan(φ) · x9− ω · [4 · cos2(φ) − 3] · x10.

For x4to enter the basis, using Dantzig’s rule to pivot, we require both

ω ·1 − 2 · cos(φ)

cos(φ)

≥ 2 · sin(φ) · tan(φ)

and

ω ·1 − 2 · cos(φ)

cos(φ)

≥1 − cos(φ)

cos(φ)

,

together yielding a tighter bound

ω ≥ 2 · sin(φ) · tan(φ) ·

cos(φ)

1 − 2 · cos(φ)=

1 − cos(2φ)

1 − 2 · cos(φ)≈ 4.74.

Proceeding in this manner, analyzing all ten dictionaries of Hoffman’s cycle, the correct lower bound

ω >

1−2·cos(φ)≈ 4.74 is obtained.

1−cos(2φ)

1

3

5

7

6

8

4

2

9

10

Figure 7 A picture of Hoffman’s example with 0 < ω < 4.74

12