Content uploaded by Xiang Chen

Author content

All content in this area was uploaded by Xiang Chen on Oct 10, 2014

Content may be subject to copyright.

Calibration of Dual Laser-Based Range Cameras

for Reduced Occlusion in 3D Imaging

Aaron Mavrinac, Xiang Chen, Peter Denzinger, and Michael Sirizzotti

Abstract— A robust model-based calibration method for dual

laser line active triangulation range cameras, with the goal of

reducing camera occlusion via data fusion, is presented. The

algorithm is split into two stages: line-based estimation of the

lens distortion parameters in the individual cameras, and com-

putation of the perspective transformation from each image to a

common world frame in the laser plane using correspondences

on a target with known geometry. Experimental results are

presented, evaluating the accuracy of the calibration based on

mean position error as well as the ability of the system to reduce

camera occlusion.

I. INT RO DUC TI O N

Active 3D vision is a popular family of methods for

obtaining robust and accurate three-dimensional digitizations

of real objects [1], [2]. One common paradigm, known as

laser line triangulation, uses a single camera to view a laser

line projected over the surface of an object at some angular

offset from its principal axis, processing the image to obtain

a cross-section proﬁle of the object; this process can be

repeated moving the object in small increments normal to

the laser plane to yield a full surface scan of the object.

A major problem with these devices, particularly in inspec-

tion and metrology applications, is the occlusion of portions

of the target object, resulting in incomplete 3D data. Two

types of occlusion may occur: laser occlusion, in which the

laser is unable to illuminate an object point visible from

the camera, and camera occlusion, in which the camera is

unable to image an object point illuminated by the laser.

Such issues are typically overcome by performing multiple

scans or employing more complex systems, imposing a large

amount of overhead.

Camera occlusion occurs in any case where a portion of

the target surface faces away from the camera at a greater

angle from the horizontal than the camera itself (about the

xaxis, in our convention described in Section II-A). Thus,

it is possible to mitigate this by adding a second camera

to the system at some angle on the opposite side of the

laser plane. In practice, most surface portions occluded from

one direction are visible from the other, with the occlusion

typically being caused by a height discontinuity of some sort;

eliminating this therefore usually yields nearly complete 3D

This research was supported by the MITACS ACCELERATE program

and the Ontario Centres of Excellence Interact initiative in collaboration

between Vista Solutions Inc. and the University of Windsor.

A. Mavrinac and X. Chen are with the Department of Electrical &

Computer Engineering, University of Windsor, 401 Sunset Ave., Windsor,

Ontario, Canada, N9B 3P4. {mavrin1,xchen}@uwindsor.ca

P. Denzinger and M. Sirizzotti are with Vista Solutions,

2835 Kew Dr., Unit #1, Windsor, Ontario, Canada, N8T 3B7.

{pdenzinger,msirizzotti}@vistasolutions.ca

data. We seek to obtain a combined range image with all

information which would otherwise be available from two

separate scans in opposite orientations.

The challenge, then, is to combine the data from both

sources in such a way that existing processes (e.g. for

inspection or metrology) can be applied to the more complete

data unmodiﬁed – in other words, the system should be a

drop-in replacement for the single-camera equivalent. We

present here a robust model-based calibration method for

two laser-based range cameras which allows this data to be

fused into a single 3D point cloud or range image in real

time.

Although model-based calibration of active triangulation

camera systems is essentially the same problem as standard

camera calibration, some techniques taking advantage of

the speciﬁcs of laser line triangulation have been proposed.

Reid [3] presents a method for estimating the projective

homography with the laser plane using correspondences

between the image and a set of orthogonal planes of known

geometry in the scene. Jokinen [4] presents an area-based

matching approach in which multiple proﬁle maps from

different viewpoints are registered to reﬁne an initial target-

based calibration.

Departing from model-based calibration, Trucco et al. [5]

present a direct calibration method which interpolates a

lookup table for the entire ﬁeld of view based on a target of

known geometry, and thus implicitly models all intermediate

parameters; this is further explored in [6].

Vilac¸a et al. [7] present a complete calibration method

for two laser-based range cameras, also with the goal in

mind of reducing camera occlusion. It is similar to our

method in that it constrains lens distortion correction and

the perspective homography to the laser plane – a valid

simpliﬁcation over traditional camera calibration, since sub-

sequent measurements are also constrained there. However,

our approach uses the range data directly for calibration,

which allows for implicit constraint of calibration to the

laser plane, higher accuracy (if range values can be obtained

with subpixel accuracy), a more direct line-based process for

lens distortion correction, and the use of a more practical

calibration apparatus.

In the general case, combining range data from multiple

sources is often achieved via registration algorithms (Salvi

et al. [8] present an excellent overview). While this approach

is well-studied and its various algorithms can be applied in a

diverse range of situations, the calibration approach has clear

advantages: it is completely unaffected by incomplete over-

lap, which in contrast causes severe performance degradation

or imposes additional overhead in registration algorithms;

also, a pre-computed lookup table is far less computationally

expensive than iterative registration, and allows for more

exact results. Laser-based range cameras conveniently lend

themselves to such calibration.

The remainder of this paper is organized thus. Section II

introduces some concepts, conventions, and notation used

subsequently. The proposed calibration method is detailed in

Section III. Experimental results are presented in Section IV.

Finally, some concluding remarks are given in Section V.

II. DE FINI TI O NS

A. Geometry

The plane through which the laser line is projected is

deﬁned as the x−zplane in the world coordinate system

(with xhorizontal and zvertical), and is termed the laser

plane. The direction a target object moves when performing

a scan is termed the transport direction; this must, of course,

have a ycomponent. World coordinates are assumed to be

deﬁned in some real measurement unit; we use millimetres

herein.

We assume a roughly symmetric camera conﬁguration

(Figure 1), in which the transport direction is positive-y

(normal to the laser plane), and the cameras are placed on

opposite sides of the laser plane. Other conﬁgurations are

possible, and our method can be applied to these as well

with minimal modiﬁcation.

Fig. 1. Symmetric Camera Conﬁguration

The raw discrete two-dimensional coordinates of the data

from the camera, denoted (ur, vr), lie in the sensor plane.

A continuous image plane, with coordinates denoted (u, v),

is deﬁned to describe corrected (or ideal) data points.

B. Range Data

We assume that the cameras in question are already able

to perform basic laser line triangulation, either on-board or

in software on a host computer, using well-studied methods

(see Section IV-A for the speciﬁcs of our experimental setup

and imaging hardware).

The range data generated by the camera from a single

image is termed a proﬁle, and consists of an ordered set of

height values – one per camera sensor column – correspond-

ing to the zvalues of object points in the laser plane. Proﬁle

elements are points in the sensor plane, with column index

urand height value vr(height values may be interpolated

and are thus not necessarily actual sensor row indices). An

ordered set of proﬁles, ostensibly taken at regular intervals

along the transport direction, is termed a scan.

Fig. 2. Range Data

III. CAL IB R ATIO N METH OD

Our calibration method is divided into two stages: correc-

tion for lens distortion and determination of a perspective

mapping from the image plane to the laser plane.

A. Lens Distortion

According to Brown’s model of lens distortion [9], image

plane coordinates (u, v)are computed from sensor plane

(raw pixel) coordinates (ur, vr)as follows:

u=ur+u0(K1r2+K2r4+...) + P1(r2+ 2u2

0)

+ 2P2u0v0(1 + P3r2+...)(1)

v=vr+v0(K1r2+K2r4+...) + P2(r2+ 2v2

0)

+ 2P1u0v0(1 + P3r2+...)(2)

u0=ur−ouv0=vr−ovr=qu2

0+v2

0

where (ou, ov)is the optical center on the sensor plane, Ki

are radial distortion coefﬁcients, and Piare tangential (or

decentering) distortion coefﬁcients. For practical purposes,

we limit our model to coefﬁcients K1,K2,P1, and P2;

experiments have shown that higher-order coefﬁcients are

relatively insigniﬁcant in most cases [10].

The parameters can be estimated by exploiting the well-

known fact that, in the absence of distortion, straight lines in

the three-dimensional scene map to straight lines in the two-

dimensional image [11], [12]. Line-based correction lends

itself particularly well to our case, since we have a readily-

available source of straight lines in the scene constrained to

the plane of interest.

1) Line Extraction: Line point sets are obtained by taking

a proﬁle of any ﬂat object. The raw proﬁle may contain

extraneous points; these are eliminated using a two-step

process: ﬁrst, all points with height below a certain threshold

are removed; second, the remaining points are ﬁt to a linear

model using RANSAC [13]. The linear model takes the form:

v=αu +β(3)

with the origin (ou, ov)at the optical center (using the sensor

center as an initial guess). Since, clearly, we cannot yet

obtain uand v, we substitute the raw values urand vr,

respectively, assuming that lens distortion is negligible for

the purpose of line extraction. The RANSAC consensus set

forms the ﬁnal line point set, and the model parameters α

and βare retained to initialize the optimization model in the

following step.

A number of lines, well-distributed over the ﬁeld of view,

should be proﬁled. Although not necessary, it is convenient

to do so with both cameras simultaneously.

2) Parameter Optimization: Given a set of Mline point

sets, each composed of Nmpoints, the objective is to ﬁnd the

distortion parameters such that the total deviation in the line

equations is minimized [11]. This is achieved by minimizing

F, the sum of squared distances from each undistorted point

(u, v)to its corresponding line:

f=αmu−v+βm(4)

F=

M

X

m=1

Nm

X

k=1

f2(5)

where uand vare computed according to Equations 1

and 2, respectively. The set of 2M+ 6 design variables

is {K1, K2, P1, P2, ou, ov, α1. . . αM, β1. . . βM}. The dis-

tortion parameters (Kiand Pi) are initialized to zero, (ou, ov)

is initialized to the sensor center, and the line parameters (αm

and βm) are initialized from the RANSAC model parameters

from the extraction step.

A solution can be found numerically using the Levenberg-

Marquardt algorithm [14] for nonlinear optimization (the

authors of [11] use a simpler gradient descent method,

but note problems with convergence to local minima). This

process requires the Jacobian matrix, consisting of the partial

derivatives of fwith respect to each parameter.

B. Perspective Mapping

1) Homography: A homography between the two-

dimensional image plane and the two-dimensional laser plane

is deﬁned by a 3×3matrix Has:

λ

x

z

1

=H

u

v

1

(6)

where λis a scale factor.

Given a set of laser plane point coordinates and their

corresponding image plane coordinates, this homography can

be solved linearly. A point correspondence pair (x, z)↔

(u, v)yields two linear equations:

x(h31u+h32 v+h33)−(h11 u+h12 v+h13) = 0 (7)

z(h31u+h32 v+h33)−(h21 u+h22 v+h23) = 0 (8)

At least ﬁve such point correspondences are required to

ﬁnd H. An optimal solution to the resulting overdetermined

linear system can be found numerically using singular value

decomposition.

2) Calibration Target: Point correspondences are ob-

tained by taking a single proﬁle of a calibration target of

known structure. The target should have a number (ﬁve or

more) of precise, well-distributed, non-collinear points, such

as sharp corners, which can be easily localized in the image

(see Section IV-A.2 for our design and a brief description of

our point detection method). The detected image plane point

(u, v)is associated with the known laser plane point (x, z )

deﬁned in units of length relative to a common origin.

The proﬁle is taken by both cameras simultaneously, so

that their independent homographies map to a common

coordinate system.

IV. EXP ERI MEN TAL RESU LTS

A. Apparatus

1) Camera System: All experiments were performed us-

ing two SICK-IVP Ranger D industrial 3D smart cameras

with a single laser line source. These were mounted in a ﬁxed

conﬁguration similar to Figure 1 above a small conveyor belt,

with an encoder and photoswitch for controlling the start and

transport direction resolution of scans. Figure 3 shows the

equipment during the calibration procedure.

Fig. 3. Equipment Conﬁguration

2) Calibration Target: We employ a calibration target

with one ﬂat side (for the lens distortion correction stage)

and one side with a number of right-angled steps (for the

perspective mapping stage), as seen in Figure 3. Our target

was manufactured to a tolerance of 0.1 mm. This is attached

to a robotic arm to automate the positioning of the target.

Although accurate positioning is not required, rotation of the

target about the yand zworld coordinate axes introduces

error and should be avoided.

Nine line proﬁles on the ﬂat side are taken for use in

the lens distortion step, in a pattern spanning the ﬁeld of

view. Figure 4 shows a composite of data from all nine scans

for one camera; note the signiﬁcant radial distortion prior to

correction.

Point detection on the stepped side is performed by ﬁnding

the middle (lowest) step, then searching outwards for the

horizontal lines of each step, using RANSAC to extrapolate

the precise corner positions. Figure 5 shows a plot of a

typical set of results, with squares indicating the estimated

corner position.

0

100

200

300

400

500

0 200 400 600 800 1000 1200 1400

Fig. 4. Line Proﬁle Pattern

0

100

200

300

400

500

0 200 400 600 800 1000 1200 1400

Fig. 5. Typical Point Detection Result

3) Software: The software implementation is developed in

ANSI C and compiled with GCC using the MinGW tools.

The SICK iCon API is used to interface with the range cam-

eras over Gigabit Ethernet. For Levenberg-Marquardt opti-

mization, singular value decomposition, and random number

generation for RANSAC, the GNU Scientiﬁc Library [15]

is employed. Range images are output in PGM (P2 ASCII)

format.

B. Accuracy

The accuracy of the sensor calibration is tested by mea-

suring a large number of points of known geometry relative

to a common origin. The mean error is computed separately

for each camera, yielding the calibration accuracy for mea-

surements on each side, and then the mean deviation for each

point from one camera to the other is computed, yielding the

relative accuracy of the pair.

The sensor in the Ranger D camera has 1536 columns and

512 rows. To interpolate the impact position of the laser line

on the sensor, a built-in algorithm using a center-of-gravity

method with a typical resolution of 1/10 pixel is employed;

results are returned in terms of 1/16 pixel, yielding an

effective sensor size of 1536×8192. Our calibration software

generates a lookup table between these sensor coordinates

and world coordinates in the laser plane, by mapping each

discrete coordinate through the distortion model and the

perspective homography.

For our test case with a ﬁeld of view 150 mm high by

200 mm wide and the cameras at an angle of 45◦to the

laser plane, the mean absolute position errors for points in

the individual cameras were measured at 0.1007 mm for the

left and 0.0968 mm for the right (on the same order as the

manufacturing tolerance for our calibration target), while the

mean relative position error between the two cameras was

measured at 0.2123 mm (note that the cameras estimate the

laser plane origin independently, hence the larger relative

error). This corresponds to subpixel accuracy in our output

images, which are sampled at 0.5 mm per pixel. Adjusting for

the relative size of ﬁeld of view and camera resolution, these

results compare favorably with those of Vilac¸ a et al. [7].

C. Occlusion Reduction

Our measurement software generates one row of a range

image by binning and averaging the points in a rectiﬁed

proﬁle – generated by combining the raw proﬁles from each

camera transformed through their respective lookup tables

– according to a given ﬁeld of view width and resolution.

Where no data is available, a value of zero is assigned.

A full range image is generated from a scan, with the

aforementioned resolution typically equal to the spacing

between proﬁles in the scan.

As an approximate quantiﬁcation of camera occlusion, we

deﬁne a closed countour of the scanned object in the range

image by segmentation, and consider the proportion of pixels

within this contour with value zero (black pixels) expessed as

a percentage. We compare the occlusion in the left and right

range images (that is, the range images generated using only

data from the left and right cameras, respectively) to that in

the combined image for a number of objects, with results

shown in Table I.

TABLE I

OCC LU S IO N REDU CT I ON F OR VA RI O US OB J EC TS

Object Occl. Left Occl. Right Occl. Combined

Connecting Rod 8.35% 8.20% 1.22%

Pump Inserts 3.64% 3.80% 0.11%

Transmission Gear 3.10% 3.28% 0.10%

Subﬂoor Panel 8.37% 14.91% 0.82%

Bearing Collar 16.48% 14.62% 2.85%

Toy Bricks 17.14% 13.49% 2.49%

Figures 6 and 7 show the calibrated images from the left

and right cameras as well as the combined image for two

typical objects. The effect of camera occlusion can clearly

be seen in the left and right images: black regions near raised

edges show where no data exists. By contrast, these effects

are almost completely absent from the combined images.

These results highlight the problem faced in many appli-

cations as well as the impact of the solution. Regardless of

how objects are oriented, with single-camera scans, there

is in the general case some degree of camera occlusion.

When the images from two cameras are combined, this

occlusion is dramatically reduced (with minimal computation

as compared to registration).

Left Right Combined

Fig. 6. Occlusion Reduction for Connecting Rod

Left Right Combined

Fig. 7. Occlusion Reduction for Pump Inserts

V. CO NC L US ION S

We have presented a straightforward calibration method

for laser-based range cameras, which adapts the underlying

theory of proven techniques to take advantage of the planar

constraint and readily available straight line data inherent to

these systems. For the individual sensor, the calibration has

been shown to exhibit high absolute accuracy. The method

is used to calibrate dual cameras into a common reference

frame, with which we have achieved a substantial reduction

in camera occlusion with high relative accuracy.

Although we present only dual cameras in a symmetric

conﬁguration here, the principles could easily be extended to

more cameras and different conﬁgurations, potentially result-

ing in further reduction of occlusion for some applications.

The calibration method can also be applied to single-camera

systems.

Our method is applicable to a variety of inspection,

metrology, and other tasks which might beneﬁt from more

complete range data in a single scan. Importantly, it allows

existing methods working with range images to function

unchanged with the fused data, making the system suitable

as a drop-in replacement for a single-camera solution.

REF ER ENC ES

[1] P. J. Besl, “Active, Optical Range Imaging Sensors,” Machine Vision

and Applications, pp. 127–152, 1988.

[2] S. K. Mada, M. L. Smith, L. N. Smith, and S. Midha, “An Overview

of Passive and Active Vision Techniques for Hand-Held 3D Data

Acquisition,” in Opto-Ireland 2002: Optical Metrology, Imaging, and

Machine Vision. SPIE, 2003, pp. 16–27.

[3] I. D. Reid, “Projective Calibration of a Laser-Stripe Range Finder,”

Image and Vision Computing, vol. 14, no. 9, pp. 659–666, 1996.

[4] O. Jokinen, “Self-Calibration of a Light Striping System by Matching

Multiple 3-D Proﬁle Maps,” in Proc. 2nd Intl. Conf. on 3-D Digital

Imaging and Modeling, 1999, pp. 180–190.

[5] E. Trucco, R. B. Fisher, and A. W. Fitzgibbon, “Direct Calibration and

Data Consistency in 3-D Laser Scanning,” in Proc. British Machine

Vision Conf., 1994, pp. 489–498.

[6] E. Trucco, R. B. Fisher, A. W. Fitzgibbon, and D. K. Naidu, “Cal-

ibration, Data Consistency and Model Acquisition with a 3-D Laser

Striper,” Intl. Jrnl. of Computer Integrated Manufacturing, vol. 11,

no. 4, pp. 292–310, 1998.

[7] J. L. Vilac¸a, J. C. Fonseca, and A. M. Pinho, “Calibration Procedure

for 3D Measurement Systems Using Two Cameras and a Laser Line,”

Optics & Laser Technology, vol. 41, no. 2, pp. 112–119, 2009.

[8] J. Salvi, C. Matabosch, D. Foﬁ, and J. Forest, “A Review of Recent

Range Image Registration Methods with Accuracy Evaluation,” Image

and Vision Computing, vol. 25, pp. 578–596, 2007.

[9] D. C. Brown, “Decentering Distortion of Lenses,” Photogrammetric

Engineering, vol. 32, no. 3, pp. 444–462, 1966.

[10] J. Weng, P. Cohen, and M. Herniou, “Camera Calibration with

Distortion Models and Accuracy Evaluation,” IEEE Trans. on Pattern

Analysis and Machine Intelligence, vol. 14, no. 10, pp. 965–980, 1992.

[11] B. Prescott and G. F. McLean, “Line-Based Correction of Radial Lens

Distortion,” Graphical Models and Image Processing, vol. 59, no. 1,

pp. 39–47, 1997.

[12] F. Devernay and O. Faugeras, “Straight Lines Have to be Straight,”

Machine Vision and Applications, vol. 13, pp. 14–24, 2001.

[13] M. A. Fischler and R. C. Bolles, “Random Sample Consensus: A

Paradigm for Model Fitting with Applications to Image Analysis and

Automated Cartography,” Comm. of the ACM, vol. 24, no. 6, pp. 381–

395, 1981.

[14] C. T. Kelley, Iterative Methods for Optimization. SIAM, 1999.

[15] M. Galassi, J. Davies, J. Theiler, B. Gough, G. Jungman, P. Alken,

M. Booth, and F. Rossi, GNU Scientiﬁc Library Reference Manual,

3rd ed. [Online]. Available: http://www.gnu.org/software/gsl/