Content uploaded by Panchamy Krishnakumari

Author content

All content in this area was uploaded by Panchamy Krishnakumari on Jan 09, 2018

Content may be subject to copyright.

94

Transportation Research Record: Journal of the Transportation Research Board,

No. 2645, 2017, pp. 94–103.

http://dx.doi.org/10.3141/2645-11

Identifying and classifying trafﬁc and congestion patterns are essen-

tial parts of modern trafﬁc management underpinned by the emerging

intelligent transport systems. This paper explores the potential of using

a combination of image processing methods to identify and classify

regions of congestion within spatiotemporal trafﬁc (speed, ﬂow) contour

maps. The underlying idea is to use these regions as (archetype) shapes

that in many combinations can make up a wide variety of larger-scale

trafﬁc patterns. In this paper, use of a so-called statistical shape model

is proposed as a low-dimensional representation of the archetype shape,

and an active shape model algorithm coupled with linear classiﬁcation

is developed to classify the patterns of interest. Application of the pro-

posed method is demonstrated with a preliminary set of speed contour

maps reconstructed from loop detector data in the Netherlands. The

results show that the extended active shape model can be used as a

multiclass classiﬁer. In particular, 70% of the trafﬁc patterns in the test

data were correctly classiﬁed with use of only two archetype shapes and

simple logistic classiﬁers. The results point to the importance of use of

expert knowledge by means of (a priori) manual classiﬁcation of the

training examples. This work opens many research directions, including

semiautomated searches through trafﬁc databases, automatic detection,

and classiﬁcation of new trafﬁc patterns.

In research, in education, and in practice, spatiotemporal contour maps

of speed, density, and ﬂow provide an intuitive means to identify,

study, explain, and illustrate (longitudinal) trafﬁc ﬂow phenomena

on the basis of either real trafﬁc data or data from trafﬁc simulation

models. These phenomena include homogeneous congestion (HC)

patterns at bottlenecks, reduced ﬂows related to blockages, wide

moving jams (WMJs) that propagate over large distances against

the direction of trafﬁc ﬂow, and high-density platoons of heavy

vehicles that form moving bottlenecks. With contour maps these

phenomena become visible, which helps scientists to formulate

hypotheses and derive theories and models to describe the under-

lying dynamics. Figure 1, for example, shows speed contour plots on

the A20 freeway in the Netherlands between Rotterdam and Gouda

(Figure 1a), the A13 between Rotterdam and The Hague (Figure 1b),

and the A16 east of Rotterdam (Figure 1c), which were used in a

study of trafﬁc dynamics related to severe accidents.

Constructing smooth contour maps from (sensor) data is relatively

straightforward with the adaptive smoothing method introduced by

Treiber and Helbing (1) and further reﬁned by others (2, 3). Techni-

cally, contour maps represent matrices of trafﬁc variables (densities,

speeds, ﬂows) on consecutive cells Δx along a route for consecutive

time periods Δt, where neither Δx nor Δt must be of constant size

or duration, respectively. When mapped onto an underlying grid

{xi, tj} with i = 1, . . . , N and j = 1, . . . , M, trafﬁc contour plots

can be understood as images with a color mapping from the trafﬁc

variable of interest to whatever color coding provides the required

visual representation. This image representation opens a wide array

of possibilities for trafﬁc scientists and engineers, such as deriving

distributions of wave speeds from raw trafﬁc data, without making

any prior assumptions (4). These wave speeds and patterns can then

be used for calibration and validation of trafﬁc ﬂow models (5).

This paper explores a different application perspective of image

processing techniques within the trafﬁc domain, that is, the clas-

siﬁcation and identiﬁcation of different trafﬁc patterns. Classifying

congestion patterns has a twofold application—for off-line analysis

and for real-time predictions. For off-line analysis, it can be used to

ﬁnd days and routes in the historical database with similar congestion

patterns. These data can be used by trafﬁc managers to compare two

incidents to gain insight for better trafﬁc control. The classiﬁcation

of partial patterns along with metadata such as incident location and

severity can also be used for short-term predictions.

Classifying congestion patterns is not a new idea. The ASDA/

FOTO method of Kerner et al. is a well-known, patented, theory-

laden approach (6), and many machine learning alternatives are avail-

able (7–10). Whereas the latter studies focused on class labels that

indicate level of service (e.g., light, medium, and heavy congestion),

this study’s aim, like that of Kerner et al., is to classify entire spatio-

temporal congestion patterns. In contrast to that of Kerner et al., this

study does not use an elaborate set of expert rules but instead uses

ﬂexible and data-driven methods. A supervised learning method was

used in an earlier study to classify such patterns with a multiclass

support vector machine (11). That study derived an equal-size fea-

ture vector for all small and larger trafﬁc patterns identiﬁed in trafﬁc

contour maps. In this paper, (geometrical) shapes instead of feature

vectors are used. The method used comprises several image pro-

cessing techniques to break down larger-scale trafﬁc patterns into

smaller regions. The underlying idea is that these regions constitute

base (archetype) shapes that in many combinations can make up a

wide variety of larger-scale trafﬁc patterns. With robust identiﬁca-

tion methods for such archetype shapes, it is possible to dissect,

identify, and classify complex trafﬁc patterns automatically. Despite

Trafﬁc Congestion Pattern Classiﬁcation

Using Multiclass Active Shape Models

Panchamy Krishnakumari, Tin Nguyen, Léonie Heydenrijk-Ottens,

Hai L. Vu, and Hans van Lint

P. Krishnakumari and H. van Lint, Delft University of Technology, Stevinweg 1,

2600 GA Delft, Netherlands. T. Nguyen, Swinburne University of Technology,

Melbourne, Victoria, Australia. Current afﬁliation for T. Nguyen: Delft University

of Technology, Stevinweg 1, 2600 GA Delft, Netherlands. L. Heydenrijk-Ottens,

CGI Nederland B.V., George Hintzenweg 89, 3068 AX Rotterdam, Netherlands.

H. L. Vu, Monash University, 23 College Walk, 3800 Clayton, Victoria, Australia.

Corresponding author: P. Krishnakumari, p.k.krishnakumari@tudelft.nl.

Krishnakumari, Nguyen, Heydenrijk-Ottens, Vu, and van Lint 95

the approach’s simplicity, the classiﬁcation results are encouraging,

with 70% accuracy attained with just two archetype shapes and

simple logistic classiﬁers and without resorting to the use of addi-

tional information (e.g., ﬂow), as in the method of Kerner et al. The

applications for this technique are numerous and range from traf-

ﬁc database searching and indexing to trafﬁc state estimation and

prediction.

The rest of the paper is organized as follows. The next section gives

an overview of the approaches in the multiclass classiﬁer using active

shape models (ASMs). Then the experimental setup is described,

along with the validation method. The results of the method are pre-

sented, and a synthesis of the ﬁndings is then given. The paper closes

with preliminary conclusions and an outlook on further improvement

of the method.

METHOD

In this work, an approach that is fundamentally different from

the state of the art in trafﬁc pattern classiﬁcation is introduced and

developed. Instead of the use of local features to identify charac-

teristics of trafﬁc patterns, (archetype) shapes are used to classify

patterns to capture the global structure of these patterns. This method

works at a higher abstraction level than feature-based methods. The

Space (km)

Space (km)

(b)

(c)

Time

Time

Speed (km/h)Speed (km/h)

Space (km)

(a)

Time

Speed (km/h)

FIGURE 1 Three severe incidents with similar characteristics regarding incident, spatiotemporal

extent of queue, and vehicle loss hours: (a) A20, June 9, 2015; (b) A13, March 24, 2015;

and (c) A16, January 9, 2011.

96 Transportation Research Record 2645

shape-based methods are usually used for shape recognition and

ﬁtting, mainly in image recognition. This is the ﬁrst paper, to the

authors’ knowledge, that introduces shape-based classiﬁcation in

the trafﬁc domain for multiclass classiﬁcation. The overall method

is outlined in Figure 2a.

The idea is to extract contours from speed contour maps and use

these as the basis for building the shape model, the base shape clas-

siﬁer (SC) model, and the pattern classiﬁer model. All these compo-

nents are explained below; a full mathematical explanation is beyond

the scope of this paper. The ﬁnal part of the method explains how

the ﬁtting result from ASM has been used for a multiclass prediction

process.

Contour Extraction

The basic ingredients of the method are contour maps of detector

data generated with the adaptive smoothing method. This method is

extensively described elsewhere (1–3). Raw data for 1 day are con-

sidered in one contour map, which therefore can contain multiple

congestion patterns for a given day. The individual trafﬁc congestion

patterns from each space–time plot are extracted with a naïve contour

extraction.

The naïve contour extraction ﬁnds the outline of a pattern in an

image, as there are multiple patterns in one image (Figure 3a). The ﬁrst

step is to ﬁlter out irrelevant information by assuming a speed thresh-

old, v_thres, that differentiates between congested and freely ﬂowing

trafﬁc. In this paper, v_thres = 65. This crude assumption can be

relaxed, as discussed later in the paper. This step results in a Boolean

mask, as shown in Figure 3b. After thresholding, dilation is used to ﬁll

the holes created to provide a smooth mask. This smoothed mask is

then used to detect the contours in the image by joining the continu-

ous points along the boundary with similar pixel intensity (12). The

detected contours are used to deﬁne the boundary to extract each

pattern in an image (Figure 3c). Resulting from the naïve contour

extraction is a data set of various trafﬁc congestion pattern images.

An additional contour reﬁnement is then performed on the

obtained patterns to extract the congestion shape from each pattern.

The irrelevant information is ﬁltered out from the naïve extracted

pattern image with the same assumption used before; low speed

implies congestion, as shown in Figure 3e. A morphological clos-

ing strategy that consists of two successive binary transformations,

dilation followed by erosion (13), is used to eliminate small and iso-

lated gaps from the relevant regions without destroying the original

shape, as shown in Figure 3f. For both transformations, a 3 × 3 cross-

structuring element is used. This binary smoothed mask is then used

to detect the contours in the image (12), as shown in Figure 3g.

Manual Classiﬁcation

The reﬁned contour extraction results in a data set of various trafﬁc

congestion pattern images. This data set is manually classiﬁed into

ﬁve classes according to the size of images (space and time extent

of trafﬁc jam) and the type of congestion. Table 1 shows the ﬁve

classes with some examples. This classiﬁcation is arbitrary—other

analysts may come up with more or fewer classes and different

criteria. Unsupervised learning also can be used for creating these

classes. However, this would require building feature vectors based

on the application or using complete black-box methods like deep

learning to ﬁnd all the relevant features in the patterns.

Base Shape Identiﬁcation

and Base Shape Predictor

The manually classiﬁed data show that all the patterns in the class

are approximately a combination of two base shapes, isolated WMJs

(IWMJs) and HC. For example, Table 1 shows that low-frequency

WMJ patterns comprise n WMJ shapes, mixed class patterns com-

prise both HC and IWMJ patterns, and so on. The key distinction in

the context of this paper between the two base shapes is that WMJs

are stripe-like shapes, whereas HC patterns form triangular shapes.

This distinction is backed by theoretical research that investigated a

wide range of congestion patterns (1, 14). These works distinguish

two main types of congestion based on ﬁrst-order trafﬁc ﬂow theory,

synchronized and WMJs, which includes characteristics similar to

those in the base archetypes. Hence, it was decided to start with

these two distinct and well-deﬁned base shapes. As more archetypes

emerge from data, the proposed method can be scaled to include these

base shapes.

Speed contour maps

Contour extraction

Multiclass predictor

Class 1 Class 2 Class NIdentify base shapes

Base shape predictor

(a)

Contours of base

shape

ASM training

ASM fitting

Shape classifier

training

Shape predictor

Isolated and Homogeneous

(b)

Pattern classifier

training

For each pattern

For each shape

ASM fitting

Shape predictor

Class predictor

Class 1–5

(c)

FIGURE 2 Approach: (a) overview, (b) base shape predictor,

and (c) multiclass predictor.

Krishnakumari, Nguyen, Heydenrijk-Ottens, Vu, and van Lint 97

The ASM, given a new observed shape, tries to ﬁt this shape to

one of the base shapes. To do this, the originally single-class ASM

algorithm was extended to a multiclass ASM through inclusion of a

linear classiﬁer that predicts whether a given shape is HC or IWMJ.

Figure 2b outlines how to do this step by step. Each step is explained

in the following (a full mathematical explanation of the ASM com-

ponents, including the multiclass extension, is beyond the scope

of this paper). The two main phases in ASM are discussed below:

constructing a statistical shape model (SSM) for each base shape

(model training) and ﬁtting a new shape to this SSM (model ﬁtting).

ASM Training

The ASM is a model-based segmentation method introduced by

Cootes et al. (15). The method is based on the principle that a shape

can be represented by a mean shape and its variations. The mean

shape and the variances constitute the SSM, which contains all the

parameters that are needed to deﬁne that shape. The SSM is used

to ﬁnd potential instances of the shape model in a new image or con-

tour. Initially, a set of landmarks in the new contour is deﬁned, after

which the shape deﬁned by these landmarks is deformed accord-

ing to the SSM to provide the best ﬁt possible within the SSM. The

deformation is based on ﬁnding correspondences between the new

shape and the shape deﬁned by various SSM components and itera-

tive minimization of a cost function for the ﬁt. The allowed degree

of deformation is constrained by the variations deﬁned in the SSM.

If the SSM includes large variances, large deformations are possible,

and vice versa.

An overview of the method is given below. A more detailed

explanation is available elsewhere (15). The steps for building

the SSM model (i.e., the training phase) for a base shape are as

follows:

Step 1. Align the shapes to the ﬁrst shape in the data set, and

generate a mean shape from the aligned shapes. First, all the shapes

must have the same number of landmarks, which is rarely the case.

Therefore, a so-called iterative closest point (ICP) method is used

(a)

(b)

(c)

(d) (e)

(f) (g)

Speed (km/h)

FIGURE 3 Contour extraction, naïve and refined: (a) raw data, multiple patterns;

(b) threshold mask; (c) naïve extracted contour with boundary; (d) naïve extracted

pattern; (e) binary image; (f) performed closing; and (g) refined contours.

98 Transportation Research Record 2645

to register the contours from all classes to a given model contour.

In ICP, a point set is transformed to best match the chosen model,

where the transformations are revised iteratively until the distance

between the point set and the model is minimized (16). The align-

ment itself is achieved with Procrustes analysis because the standard

ASM also uses this method for alignment (17).

Step 2. Realign the shapes to the mean shape and generate a new

mean shape from the newly aligned shapes.

Step 3. Repeat Step 2 (update the mean shape) until convergence.

Step 4. Finally, apply principal component analysis (PCA) to

compute the eigenvectors and eigenvalues of the aligned shapes.

The SSM components are the eigenvectors of the centered shapes in

the training data, and the variances are the eigenvalues of these shapes.

When PCA is applied to the data, any shape (within the training set) x

(an n-dimensional vector of points) can be approximated with

xxPb (1)≈+

where

–

x = SSM mean shape having point correspondences with x and

the same dimension,

P = SSM principal components, and

b = parameters corresponding to the SSM components that deform

the shape.

This is the basis for ﬁtting a new set of landmarks to the SSM.

The ﬁrst four components of the IWMJ base class are shown in

Figure 4a. This ﬁgure explains the shape variations of the class

according to the mean shape.

ASM Fitting

Given a new shape Y′ for testing, the shape is registered with the

mean shape by using ICP to compute Y. With Y and the SSM model

TABLE 1 Class Description for Manually Classified Patterns

Name Remarks Examples

Isolated WMJs Short (1–2 km) high-density trafﬁc jams. Both head

and tail of this queue propagate backward with

virtually constant speeds (typically −18 km/h).

WMJs typically result from large disturbances

(e.g., abrupt braking) farther downstream.

Heterogeneous Congestion 1,

low-frequency WMJs

Large-scale light congestion patterns with a few

WMJs emitting from the congested area.

Heterogeneous Congestion 2,

high-frequency WMJs

Large-scale light congestion patterns with many

WMJs emitting from the congested area.

Homogeneous congestion High-density (low-speed) severe congestion regions,

typically caused by incidents or other lane block-

ages. In terms of shape, the downstream front is

stationary, whereas the upstream front moves with

various shock wave speeds.

Mixed large-scale pattern Combination patterns not falling in either of the

other categories.

Krishnakumari, Nguyen, Heydenrijk-Ottens, Vu, and van Lint 99

of the shape, ASM ﬁtting ﬁnds the model points x that best ﬁt Y. The

steps for ASM ﬁtting are as follows:

Step 1. Initialize the shape parameter b as 0, implying model

points = mean shape, that is, x = –

x.

Step 2. Generate the model points positions with x = –

x + Pb,

where P is the SSM principal component.

Step 3. Find the pose parameters transform that best aligns the

model points x to the new set of landmarks Y by using Procrustes

analysis (17).

Step 4. Project Y into the model coordinate frame Y′ by using the

inverse transform from Equation 1.

Step 5. Update the shape parameters b to match Y′ by ﬁnding the

least squares solution of Ax′ = B, where A is P, x′ is b, and B = Y′ − –

x.

Step 6. Repeat Steps 2 through 5 until convergence.

An example of an ASM ﬁtting result is shown in Figure 4b.

The ﬁtting error metric for the ASM that is used as the convergence

criteria is the Euclidean distance between the mean shape and the

ASM ﬁtted shape. The ASM stops the iteration when there is no

signiﬁcant difference in the error metric result from the previous

iteration and the current iteration. Here, an error difference thresh-

old of 0.0001 was used, which is statistically insigniﬁcant with respect

to the error rate and the shape surface area.

Designing and Training a Base Shape Classiﬁer

Now, with a working ASM model for both base shapes (IWMJ and

HC), a shape classiﬁer is needed that can predict which of these base

shapes provides the best representation for a newly found shape in a

speed contour plot. First, the new shape is ﬁtted with both base shapes,

resulting in ﬁtting error values ei (error on the IWMJ base shape)

and eh (error on the HC base shape), respectively. These errors are

deﬁned as the Euclidean distance between the SSM mean and the

ﬁtted shape. Additionally, to increase prediction accuracy, a third

metric is computed that is based on additional nonshape properties

of IWMJ and HC shapes, respectively. As an example, in this case a

metric is based on the spatiotemporal area covered by the shape a

(in meter × seconds) and the gradient g (the variation in speed in the

given area a). The compound metric is the ratio g/a, which can be

understood as the amount of heterogeneity (variation in speed) per

unit space × time, resulting in a three-dimensional feature vector

(ei, eh, g/a). To build the classiﬁer, a well-known method in linear clas-

siﬁcation, logistic regression, is applied (18). It originally was a con-

ditional probability model that measures the likelihood relationship

between a speciﬁc output and an input by using a logistic function:

pt et

1

1(2)

()

=+−

140

120

100

80

60

40

20

Landmarks (target)

Resulting fit

SSM mean

60 70 80 90 100 110 120 130 140

Result (iteration 11/11, sum of residuals = 28.11)

(b)

(a)

–60

40

30

20

10

0

–10

–20

–30

–40

mean

–40 –20 0

Component 1 ± 3 SD

(explaining 61.54% of original variance)

Component 2 ± 3 SD

(explaining 10.41% of original variance)

Component 3 ± 3 SD

(explaining 5.63% of original variance)

Component 4 ± 3 SD

(explaining 3.78% of original variance)

20 40 60 –40

30

20

10

0

–10

–20

–30

–40 mean

–30 –20 –10 02010 30 40 –40

40

30

20

10

0

–10

–20

–30

–40 mean

–30 –20 –10 02010 30

40

40

30

20

10

0

–10

–20

–30

–40 mean

–40 –20 02040

FIGURE 4 IWMJ: (a) first four PCA components of IWMJ shape model and (b) ASM fitting for IWMJ shape after Nth iterations (N = 11).

100 Transportation Research Record 2645

where t is a linear combination of input feature vector x and a is

the vector of coefﬁcients, which is considered a model character-

istic. In preparing training data, isolated and homogeneous shapes

are labeled as 0 and 1, respectively. These numbers are supposed

to be outputs of logistic function. Coefﬁcient vector aSC is trained

to minimize the cost of matching logistic function to training data,

which is measured by Euclidean distance.

pxy

ii

xi

SC

cost

(3)

2

training set

ai

∑

()

()

=−

∈

xe

eg

a

iih

,, (4)=

yi

0for isolated shapes

1for homogeneous shapes (5)=

The resulting (trained) model is the (base) shape predictor, which

classiﬁes a shape as either IWMJ or HC. The shape classiﬁer is used

to build feature vectors for training pattern classiﬁers for multiclass

classiﬁcation.

Using the Base Shape Classiﬁer

Given a new shape, the shape is ﬁtted to the IWMJ and HC SSM as

described in the section on ASM ﬁtting to create a description vector

xSC = (ei, eh, g/a). This vector is used by the shape classiﬁer model to

make the decision based on the following equation:

a

x

i

shape

0

SC SC

=

>

homogeneous shape

isolated shapeotherwise

(6)

This equation together with the logistic function gives a straight-

forward explanation for the classiﬁer to make the decision. The new

shape is classiﬁed as a homogeneous shape if the probability given by

the logistic function is greater than 0.5. An overview of the method

is shown in Figure 5a.

Multiclass Pattern Classiﬁer and Predictor

The aim of this work is to classify a given trafﬁc (congestion) pattern

into one of ﬁve predeﬁned classes (Table 1). The ﬁnal step now is

to design, train, and test a classiﬁer that can do this. A schematic

overview of the proposed method is shown in Figure 2c.

Since each of these ﬁve patterns can be broken down into com-

binations of two (archetype) base shapes (IWMJ and HC), the ﬁrst

step in classifying a congestion pattern is to break down the pattern

into these base shapes, as shown in Figure 5a. For each of the ﬁve

classes a training set can be built. The output data are the class labels

(1 to 5), and the input data are equal to a simple two-dimensional

feature vector (ni, nh), where ni is the number of occurrences of the

IWMJ shape in the given pattern and nh is the number of occurrences

of the HC shape in the given pattern.

ASM Fitting

Gradient

Area

Shape

Predictor

ei

eh

g/a

Isolated and

Homogeneous

Class

Predictor

Isolated and

Homogeneous

Isolated and

Homogeneous

(ni, nh)Class

(a)

(b)

FIGURE 5 Multiclass predictor: (a) base shape prediction given shape and (b) class prediction

given new pattern.

Krishnakumari, Nguyen, Heydenrijk-Ottens, Vu, and van Lint 101

The general idea of the multiclass pattern predictor is given in

Figure 5b. Multiclass logistic regression is used to implement it.

A simple yet efﬁcient method, the so-called one-versus-all (OVA)

scheme, was used for training; it constructs one binary logistic

regression model for each class and assigns a probability p to the

training sample belonging to this class versus the probability that it

belongs to any of the other classes.

After the classiﬁer is trained, there are ﬁve coefﬁcients, ai=FC1,...,FC5,

corresponding to ﬁve binary logistic regression models of ﬁve classes.

For a given (new) pattern with feature vector xFC = (ni, nh), the class

label is assigned to it, giving rise to the largest probability, that is,

px

i

i

clas

sargmaxFC FC (7)a

i

()

=

EXPERIMENTAL SETUP

The data used in this study come from the National Data Warehouse

for Trafﬁc Information (19), a Dutch organization that archives and

provides real-time access to trafﬁc data from the Dutch agency of the

Ministry of Infrastructure and the Environment (Rijkswaterstaat),

the 12 Dutch provinces, two metropolitan regions, and four of the

largest cities in the Netherlands (Amsterdam, Rotterdam, Utrecht,

and The Hague). The data used for the experiments were collected

from two heavily congested roads in the Netherlands:

• Southbound A13 from Den Haag-Zuid to Rotterdam center and

• Eastbound A15 from Havens 5500–5700 to Rotterdam

Ijsselmonde.

Space–time plots of carriageway speed for these two roads were

constructed with all available loop data for the entire month of

March 2015. Each space–time plot represents a period of 24 h from

00:01 to 23:59 at a resolution of 30 s and 100 m. Each of the plots

contains multiple congestion patterns. After each pattern was iden-

tiﬁed and extracted separately with naïve contour extraction, there

were 140 trafﬁc congestion patterns detected on the ﬁrst road and

160 patterns on the second road. Because only large-scale patterns

with similar space–time ratios are of interest here, 120 patterns were

selected to create the classes. The patterns were manually classiﬁed

into ﬁve classes based on spatiotemporal extent and characteristics

of trafﬁc congestion, as shown in Table 1. The number of patterns

per class is small; the ﬁrst trial was intended to demonstrate the ideas

and learn lessons for larger-scale application. After the classes were

identiﬁed by experts, the naïve contours were reﬁned to construct

better distinctive contours from the patterns. From these reﬁned con-

tours, two base shapes were identiﬁed; 35 contours were manually

chosen to build the SSM models of both homogeneous and isolated

base shapes. These same contours are used to train the shape classiﬁer.

A simple OVA approach was used for the multiclass predictor that

reduced the problem of classifying contours among ﬁve classes into

ﬁve feature vectors, where each model discriminated a given class

from the other four classes (20). For this OVA approach, there were

N = 5 binary classiﬁers, the kth classiﬁer trained with positive exam-

ples belonging to class k and negative examples belonging to the

other four classes. The classiﬁer that produced the maximum output

was considered to be the best ﬁt. Rifkin and Klautau stated that,

provided the binary classiﬁers are tuned well, this OVA approach

is extremely powerful and produces results that often are at least as

accurate as other, more complex approaches (20). The accuracy of the

ASM OVA classiﬁer was measured to judge the overall efﬁciency of

the algorithm with the following formula:

accuracy numberofcorrectly classified images

total numberofimages(8)=

A confusion matrix, or contingency table (21), is constructed to

investigate which class is behaving poorly, and that class is studied

further for a better understanding of the data and to make future

recommendations to improve accuracy.

RESULTS AND DISCUSSION

This section presents the results of the proposed method. The OVA

ensemble of ASM models achieved an average prediction accuracy

of 70%, which is relatively low compared with the state of the art.

However, this method represents ﬁve classes by using only two

archetype shapes, whereas other methods need more degrees of

freedom for deﬁning each class. Thus, the presented method can

constrain the classiﬁcation complexity while providing satisfactory

accuracy for a preliminary study with such a small data set. A more

elaborate confusion matrix is given in the Table 2. The table shows

the percentage of correctly and erroneously classiﬁed patterns in

each class given the ground truth. For example, for the IWMJs class,

the ground truth patterns in that class were correctly classiﬁed with

81% accuracy, but there were some patterns in other classes that

were wrongly classiﬁed as IWMJ. The high-frequency WMJs class

contains the most patterns that were wrongly classiﬁed with 50%

accuracy.

Table 2 shows that two classes have low accuracy compared with

the other classes, namely, high-frequency WMJ and mixed. It is

TABLE 2 Confusion Matrix for OVA Evaluation

Predicted

Known Isolated

Low

Frequency

High

Frequency Homogeneous Mixed

Isolated 0.81 0.03 0.03 0.13 0.00

Low frequency 0.03 0.76 0.09 0.09 0.03

High frequency 0.00 0.28 0.50 0.16 0.06

Homogeneous 0.21 0.00 0.00 0.74 0.05

Mixed 0.05 0.00 0.22 0.17 0.56

102 Transportation Research Record 2645

hypothesized that this result reﬂects the small training data set or

the limited representativeness of the samples in that class. The data

were further investigated to qualitatively test this hypothesis.

Two of the wrongly classiﬁed patterns are shown in Figure 6a;

in this case, both were wrongly classiﬁed as mixed patterns instead

of low-frequency WMJ and HC, respectively. These patterns are

actually mixed congestion patterns. The classiﬁer confused these

because they have a feature vector similar to that of the mixed class.

Successful classiﬁcation depends on the subjective manual classiﬁ-

cation process and the degree to which labeling patterns that can be

distinguished through OVA ASM are successful. This deﬁnition of

success was conﬁrmed by an initial analysis of the wrongly classiﬁed

patterns in which the authors agreed with the classiﬁer’s decision

on 15% of all wrongly classiﬁed patterns rather than with the initial

manual classiﬁcation. To reduce the subjectivity of the manual clas-

siﬁcation, unsupervised clustering and reinforcement learning can be

used as an alternative for building the classes.

A second reason for the low accuracy is the constraints of the

assumptions that have been made for extracting the contours, such

as thresholding at 65 km/h. These are evident in Figure 6b, where

the highlighted shapes consist of combinations of two or more

isolated shapes and combinations of isolated and homogeneous

shapes, respectively. Finally, speed was used exclusively to distin-

guish patterns. Identifying and classifying distinct trafﬁc patterns

also requires information on the ﬂows, particularly when one is dis-

tinguishing between congested patterns. Using just the speed and

the naïve single speed thresholding can uniquely identify only a few

patterns. To extend the repertoire of the classiﬁers, more archetypes

are needed to approximate all the unique shapes that are present

within the patterns, both speed and ﬂow patterns must be combined,

and meta-information (speed limits, geometry, etc.) may be needed

to construct a more dynamic thresholding method to extract the

shapes from the patterns.

Still, given all these limitations, the simplicity and transparency

of the method (a few base shapes + logistic regression) offers great

potential for further research and application development.

CONCLUSION AND FUTURE WORK

This paper proposed a method, consisting of contour detection, shape

models, and a multiclass OVA classiﬁer, to automatically classify

spatiotemporal trafﬁc patterns with network sensor data. The various

components were adequate for labeling complex trafﬁc patterns with

acceptable accuracy, although the data set used was too small to

warrant deﬁnite conclusions. With only two archetype shapes and

simple logistic classiﬁers, 70% accuracy of the classiﬁcation on the

test data was achieved. Furthermore, for the 30% of cases in which

the classiﬁer could not decide on any of the ﬁve designated patterns,

the contour shapes were affected by the assumptions and subjective

manual labeling was used to construct the training data.

There are many future directions of research that can further

improve the accuracy. First, the ensemble scores can be used to

directly put a conﬁdence score to each classiﬁcation. Further sophis-

tication can be reached by ensemble bootstrapping or more modern

Bayesian techniques. Second, an iterative manual–automated classi-

(a)

Low Frequency Ú Mixed

Speed (km/h)

Speed (km/h)

0

20

40

60

80

100

120

Homogeneous Ú Mixed

0

20

40

60

80

100

120

(b)

FIGURE 6 Synthesis of wrongly classified patterns: (a) manually classified and (b) potential base shapes.

Krishnakumari, Nguyen, Heydenrijk-Ottens, Vu, and van Lint 103

ﬁcation procedure is envisaged in which the manual classiﬁcation is

reevaluated after each training round with the classiﬁcation scores

of the OVA ASM. This procedure could yield splitting or combin-

ing classes, hierarchically subdividing classes, or otherwise. Third,

metadata (type of date and time, circumstances, topological charac-

teristics, etc.) can be combined to better classify the patterns. Another

extension will combine speed plots with ﬂow contour plots for better

deﬁnition of classes of congestion and for improved accuracy. Finally,

the database will be further enriched with more congestion patterns,

with a goal of identifying more complex patterns.

ACKNOWLEDGMENTS

The authors acknowledge the Dutch National Data Warehouse for

Trafﬁc Information for sponsoring this project. This work received

support from the SETA Project, funded by the European Union’s

Horizon 2020 Research and Innovation Program, and an Australian

Research Council fellowship grant. The authors thank the anonymous

reviewers for their constructive remarks.

REFERENCES

1. Treiber, M., and D. Helbing. Reconstructing the Spatio-Temporal Trafﬁc

Dynamics from Stationary Detector Data. Cooperative Transportation

Dynamics, Vol. 1, No. 3, 2002, pp. 3.1–3.24.

2. Schreiter, T., H. van Lint, M. Treiber, and S. Hoogendoorn. Two Fast

Implementations of the Adaptive Smoothing Method Used in Highway

Trafﬁc State Estimation. In Proceedings of the 13th International Con-

ference on Intelligent Transportation Systems, 2010, IEEE, New York,

2010, pp. 1202–1208. https://doi.org/10.1109/ITSC.2010.5625139.

3. van Lint, J., and S. P. Hoogendoorn. A Robust and Efﬁcient Method for

Fusing Heterogeneous Data from Trafﬁc Sensors on Freeways. Com-

puter-Aided Civil and Infrastructure Engineering, Vol. 25, No. 8, 2010,

pp. 596–612. https://doi.org/10.1111/j.1467-8667.2009.00617.x.

4. Schreiter, T., H. van Lint, Y. Yuan, and S. Hoogendoorn. Propagation

Wave Speed Estimation of Freeway Trafﬁc with Image Processing

Tools. Presented at 89th Annual Meeting of the Transportation Research

Board, Washington, D.C., 2010.

5. Treiber, M., and A. Kesting. Validation of Trafﬁc Flow Models with

Respect to the Spatiotemporal Evolution of Congested Trafﬁc Patterns.

Transportation Research Part C: Emerging Technologies, Vol. 21, No. 1,

2012, pp. 31–41. https://doi.org/10.1016/j.trc.2011.09.002.

6. Kerner, B. S., H. Rehborn, M. Aleksic, and A. Haug. Recognition and

Tracking of Spatial–Temporal Congested Trafﬁc Patterns on Freeways.

Transportation Research Part C: Emerging Technologies, Vol. 12, No. 5,

2004, pp. 369–400. https://doi.org/10.1016/j.trc.2004.07.015.

7. Andrews Sobral, L. O., L. Schnitman, and F. De Souza. Highway Traf-

ﬁc Congestion Classiﬁcation Using Holistic Properties. Presented at

10th IASTED International Conference on Signal Processing, Pattern

Recognition and Applications, 2013.

8. Jin, X., D. Srinivasan, and R. L. Cheu. Classiﬁcation of Freeway Trafﬁc

Patterns for Incident Detection Using Constructive Probabilistic Neural

Networks. IEEE Transactions on Neural Networks, Vol. 12, No. 5, 2001,

pp. 1173–1187. https://doi.org/10.1109/72.950145.

9. Riaz, A., and S. A. Khan. Trafﬁc Congestion Classiﬁcation Using

Motion Vector Statistical Features. In Proceedings of the 6th Inter-

national Conference on Machine Vision, International Society for Optics

and Photonics, 2013.

10. Thianniwet, T., S. Phosaard, and W. Pattara-Atikom. Classiﬁcation

of Road Trafﬁc Congestion Levels from Vehicle’s Moving Patterns:

A Comparison Between Artiﬁcial Neural Network and Decision Tree

Algorithm. In Electronic Engineering and Computing Technology

(S. L. Ao and L. Gelman, eds.), Springer, New York, 2010, pp. 261–271.

https://doi.org/10.1007/978-90-481-8776-8_23.

11. Nguyen, H. N., P. Krishnakumari, H. L. Vu, and H. van Lint. Trafﬁc

Congestion Pattern Classiﬁcation Using Multi-Class SVM. In Proceed-

ings of the 19th International Conference on Intelligent Transportation

Systems, IEEE, New York, 2016, pp. 1059–1064.

12. Suzuki, S., and K. Abe. Topological Structural Analysis of Digitized

Binary Images by Border Following. Computer Vision Graphics and

Image Processing, Vol. 30, No. 1, 1985, pp. 32–46. https://doi.org/10.1016

/0734-189X(85)90016-7.

13. Vincent, L. Morphological Grayscale Reconstruction in Image Analysis:

Applications and Efﬁcient Algorithms. IEEE Transactions on Image

Processing, Vol. 2, No. 2, 1993, pp. 176–201. https://doi.org/10.1109

/83.217222.

14. Helbing, D., M. Treiber, A. Kesting, and M. Schönhof. Theoretical vs.

Empirical Classiﬁcation and Prediction of Congested Trafﬁc States.

European Physical Journal B, Vol. 69, No. 4, 2009, pp. 583–598.

https://doi.org/10.1140/epjb/e2009-00140-5.

15. Cootes, T. F., C. J. Taylor, D. H. Cooper, and J. Graham. Active Shape

Models—Their Training and Application. Computer Vision and Image

Understanding, Vol. 61, No. 1, 1995, pp. 38–59. https://doi.org/10.1006

/cviu.1995.1004.

16. Chetverikov, D., D. Svirko, D. Stepanov, and P. Krsek. The Trimmed

Iterative Closest Point Algorithm. In Proceedings of the 16th Inter-

national Conference on Pattern Recognition, IEEE, New York, 2002,

pp. 545–548. https://doi.org/10.1109/ICPR.2002.1047997.

17. Goodall, C. Procrustes Methods in the Statistical Analysis of Shape.

Journal of the Royal Statistical Society, Series B: Methodological,

Vol. 53, 1991, pp. 285–339.

18. Hosmer, D. W., S. Lemeshow, and R. X. Sturdivant. Introduction to the

Logistic Regression Model. In Applied Logistic Regression, John Wiley

& Sons, New York, 2000, pp. 1–30.

19. National Data Warehouse of Trafﬁc Information. http://www.ndw.nu/en/.

20. Rifkin, R., and A. Klautau. In Defense of One-Vs-All Classiﬁcation.

Journal of Machine Learning Research, Vol. 5, 2004, pp. 101–141.

21. Fawcett, T. An Introduction to ROC Analysis. Pattern Recognition

Letters, Vol. 27, No. 8, 2006, pp. 861–874. https://doi.org/10.1016

/j.patrec.2005.10.010.

The Standing Committee on Artiﬁcial Intelligence and Advanced Computing

Applications peer-reviewed this paper.