Available via license: CC BY-NC-ND 4.0

Content may be subject to copyright.

Arch. Min. Sci., Vol. 61 (2016), No 1, p. 15–27

Electronic version (in color) of this paper is available: http://mining.archives.pl

DOI 10.1515/amsc-2016-0002

ADAM HEYDUK*

LASER TRIANGULATION IN 3-DIMENSIONAL GRANULOMETRIC ANALYSIS

TRIANGULACJA LASEROWA W TRÓJWYMIAROWYCH POMIARACH

SKŁADU ZIARNOWEGO

The measurement of the particle size distribution plays an important role in mineral processing.

Due to the high costs and time-consumption of the screening process, modern machine vision methods

based on the acquisition and analysis of recorded photographic images. But the image analysis methods

used so far, do not provide information on the three-dimensional shape of the grain. In the coal industry,

the application scope of these methods is substantially limited by the low reflectivity of the black coal

particle surface. These circumstances hinder proper segmentation of coal stream surface image. The

limited information contained in two-dimensional image of the raw mineral stream surface, makes it

difficult to identify proper size of grains partially overlapped by other particles and skewed particles.

Particle height estimation based on the shadow length measurement becomes very difficult in industrial

environment because of the fast movement of the conveyor belt and because of spatial arrangement

of these particles, usually touching and overlapping. Method of laser triangulation connected with the

movement of the conveyor belt makes it possible to create three-dimensional depth maps. Application

of passive triangulation methods (e.g. stereovision) can be impeded because of the low contrast of the

black coal on the black conveyor belt. This forces the use of active triangulation methods, directly

identifying position of the analyzed image pixel. High contrast of the image can be obtained by a direct

pointwise laser lighting. For the simultaneous identification of the entire section of the raw material

stream it is useful to apply a linear laser (a planar sheet of the laser light). There have been presented

basic formulas for conversion of pixel position on the camera CCD matrix to the real-word coordinates.

A laboratory stand has been described. This stand includes a linear laser, two high-definition (2Mpix)

cameras and stepper motor driver. The triangulation head moves on the rails along the belt conveyor

section. There have been compared acquired depth maps and photographic images. Depth maps much

better describe spatial arrangement of coal particles, and have a much lower noise level resulting from

the specular light reflections from the shiny fragments of the particle surface. This makes possible an

identification of the coal particles partially overlapped by other particles and obliquely arranged particles.

It enables a partial elimination or compensation of image disturbances affecting the final result of the

estimated particle size distribution. Because of the possibility of the reflected laser beam overriding by

other particles it is advantageous to use a system of two cameras. Results of the experimental research

confirmed the usefulness of the described method in spite of low reflectance factor of coal surface.

The fast detection of changes in particle size distribution makes possible an on-line optimization of

* DEPARTMENT OF ELECTRICAL ENGINEERING AND CONTROL IN MINING, FACULTY OF MINING AND GEOLOGY,

SILESIAN UNIVERSITY OF TECHNOLOGY, 44-100 GLIWICE, UL. AKADEMICKA 2A, POLAND.

E-mail: adam.heyduk@polsl.pl

Unauthenticated

Download Date | 5/19/16 7:59 PM

16

complex technological systems – especially those involving coal cleaning in jigs – thus leading to bet-

ter stabilization of quality parameters of the enrichment output products. An additional application of

the described method can be achieved by measuring the total volume of the stream of the transported

materials. Together with the measurement signal from the belt conveyor weight it makes possible to

estimate the bulk density of the raw mineral stream. The low complexity of the signal processing in the

laser triangulation method is associated with the acquisition of high contrast images and analysis based

on simple trigonometric dependencies.

Keywords: laser triangulation, granulometric analysis, depth maps, particle size distribution

Pomiar składu ziarnowego odgrywa istotną role w przeróbce surowców mineralnych. Ze względu

na wysoką czasochłonność procesu przesiewania duże znaczenie nabierają metody wizyjne, oparte

na akwizycji i analizie obrazów fotograficznych. Dotychczas stosowane metody analizy obrazu nie

zapewniają informacji o trójwymiarowym kształcie ziarna. Zakres stosowania tych metod w przemyśle

węglowym ograniczony jest niskim współczynnikiem odbicia powierzchni węgla utrudniającym wła-

ściwą segmentację obrazu. Ograniczenia dwuwymiarowego obrazu powierzchni strumienia materiału

ziarnistego utrudniają identyfikację właściwego rozmiaru ziaren częściowo przesłoniętych przez inne

ziarna oraz ziaren ułożonych ukośnie. Wyznaczanie wysokości ziaren na podstawie pomiaru długości

cienia staje się w warunkach przemysłowych utrudnione przez szybki ruch taśmy przenośnika oraz

przestrzenne ułożenie ziaren, często stykających się ze sobą. Metoda triangulacji laserowej w połącze-

niu z ruchem ta

śmy przenośnikowej umożliwia tworzenie trójwymiarowych map głębi. Zastosowanie

metod triangulacji pasywnej (np. stereowizyjnych) jest utrudnione ze względu na niski kontrast obrazu

czarnego węgla na czarnej taśmie przenośnika. Zmusza to do stosowania metod triangulacji aktywnej,

bezpośrednio identyfikujących analizowany punkt obrazu. Duży kontrast przetwarzanych obrazów

uzyskuje się za pomocą oświetlenia wiązką lasera. Dla jednoczesnej identyfikacji wysokości całego

fragmentu strumienia materiału celowe jest zastosowanie lasera liniowego. Przedstawiono podsta-

wowe zależności umożliwiające przeliczenie położenia punktów obrazu na przetworniku kamery na

współrzędne w układzie rzeczywistym. Opisano stanowisko doświadczalne obejmujące laser liniowy,

dwie kamery o rozdzielczości HD (2Mpix) oraz sterownik silników krokowych, przesuwających po

szynach układ triangulacyjny nad taśmą przenośnika. Porównano uzyskane mapy głębi oraz obrazy

fotograficzne. Mapy głębi znacznie lepiej opisują przestrzenne ułożenie ziaren oraz charakteryzują się

mniejszym szumem wynikającym z odbicia świat

ła od błyszczących fragmentów powierzchni ziaren.

Pozwala to na identyfikację ziaren częściowo przesłoniętych przez inne ziarna oraz ziaren ułożonych

ukośnie. Umożliwia to częściową eliminację lub kompensację zakłóceń wpływających na wynik analizy

składu ziarnowego. Ze względu na możliwość przesłonięcia odbitej wiązki laserowej przez inne ziarna

celowe jest zastosowanie układu dwóch kamer. Wyniki badań doświadczalnych potwierdziły użyteczność

opisywanej metody dla węgla o niskim współczynniku odbicia światła. Szybkie wykrywanie zmian

składu ziarnowego umożliwia optymalizację pracy złożonych układów technologicznych – zwłaszcza

obejmujących wzbogacanie węgla w osadzarkach – prowadząc w ten sposób do lepszej stabilizacji

parametrów jakoś

ciowych otrzymywanych produktów wzbogacania. Dodatkowym zastosowaniem

opisywanej metody może być również pomiar objętości strumienia transportowanego materiału, co

w połączeniu z sygnałami pomiarowymi z wagi taśmociągowej umożliwić może estymację gęstości

nasypowej materiału ziarnistego. Mała złożoność przetwarzania sygnałów w metodzie triangulacji

laserowej związana jest z wysokim kontrastem analizowanych obrazów oraz z wykorzystaniem nie-

skomplikowanych zależności trygonometrycznych.

Słowa kluczowe: triangulacja laserowa, analiza granulometryczna, mapy głębi, skład ziarnowy

Unauthenticated

Download Date | 5/19/16 7:59 PM

17

1. Introduction

Fast measurement of the particle size distribution in particular nodes of the technological

flow-sheet is of great significance in the mineral processing technology. This is related to the

problem of the selection of the operating parameters for respective machines and devices in

order to obtain the best possible operation of the technological process (e.g. crushing, grinding,

gravity concentration, sizing). Information on the current particle size distribution can be used

to ensure proper quality parameters of the commercial product (including a desire size range).

There is also a possibility of significant energy savings, because of very high power consumption

of crushing and grinding processes. Screening – however – is too time consuming-and expensive

process to provide a continuous information stream used as feedback in the crushing or grinding

process, or in order to allow optimization of operating parameter settings of machines and devices

in subsequent stages of the technological flow-sheet. This is due to both the need of periodical

(manual or mechanical) sampling of the material stream and very time consuming sieve analysis

itself. Therefore machine vision methods, based on image analysis are gaining more importance

(Kołakowska-Szponder & Trybalski, 2014). These methods have been originally developed for

surface mining and rock materials (Tosun et al., 2014), but then they have been applied to ore

processing (Trybalski, 2013). But there are currently no practical solution for the coal industry,

because of the difficulties related to the black colour of the coal surface. Extending the application

scope of the methods developed for ore and rock materials is related to the necessity of improve-

ments in image acquisition methods in order to increase a signal-to-noise ratio resulting from

illumination conditions and topographical and textural properties of the analyzed surface. An

additional problem is a reduction of the three-deimensional information to the two-dimensional

one. Third dimension (height) of particle surface is reduced to image intensity changes, which

does not allow to estimate actual height values. This hinders accurate volume (and thereby mass)

estimation of both individual grains and the entire material flow.

2. Errors related to the 2-dimensional approximation

of real 3-D surfaces

Real particles are three-dimensional objects. On the basis of two-dimensional image map-

ping it is possible to estimate the area or the circumference of their projection onto the image

plane. In order to determine their volume (and to estimate their mass) it is essential to know the

third (invisible) dimension. In two-dimensional methods the value of particle height is based

on some simplifying assumptions connecting together all three dimensions in a statistical way.

Direct measurement of the particle height can increase the accuracy of volume/weight estimation

of a single particle – and consequently also the whole particle population. This phenomenon has

been shown schematically in Fig. 1. It should be also noted that two dimensional imaging of

a three-dimensional particle can be subject to errors resulting from the spatial arrangement of

this particle among the whole population of other grains. Some of these cases have been shown

in Fig. 2.

Unauthenticated

Download Date | 5/19/16 7:59 PM

18

Fig. 1. Influence of omitting the third dimension (heights) of particle to uncertainty of its volume and mass

estimation: W

1

,W

2

, L

1

, L

2

– visible dimensions, H

1

, H

2

– invisible dimensions, W

1

= W

2

, L

1

= L

2

, H

1

H

2

, V

1

V

2

Fig. 2. Examples of common measurement errors associated with a two-dimensional representation of the

three-dimensional particle shape: a) the overriding by other particles, b) skewed particle arrangement

3. Methods of three-dimensional surface mapping

An essential problem of three-dimensional surface mapping is a fact that CCD and CMOS

sensors record directly only light intensity value (within a certain range of wavelength – each

corresponding to a particular color) incident on the respective pixel of the converter matrix.

Therefore recorded image depends on both the geometric shape of the surface and on the il-

lumination applied to the system. Consequently, a necessary condition for the efficient use of

Unauthenticated

Download Date | 5/19/16 7:59 PM

19

machine vision is a development of an appropriate illumination system – providing sufficiently

sharp – for further image processing and analysis – projection of the three dimensional shape of

particles onto the flat image plane (Heyduk, 2005). If surfaces of the whole particle population

are characterized by an identical dark colour (e.g. in the case of coal particle stream) the shape

information can be obtained mainly from location of the inter-particle spaces. This is related to

the fact, that a characteristic feature of the fragmented rock is a large contribution of void space

or areas. Under external illumination, these void areas appear as dark or shaded areas. These areas

outline boundaries of individual particles. In the case of coal it is further impeded by the fact that

the black color of the coal surface largely absorbs light, and therefore the intensity differences

between the particle surface and the inter-particle areas are relatively small. For small samples of

separated particles, arranged on a light background, valuable information can be obtained from

the analysis of shadows achieved by unidirectional illumination. A principle of this method is

presented in Fig. 3. The accuracy of this method can be increased using light incident from dif-

ferent directions – but these directions have to be unambiguously distinguished – e.g. multicolor

lightning or a sequential series of flashes (Koh et al., 2007).

Fig. 3. Principle of using the length measurement of a shadow casted by the rock particle

for estimation of the particle height

Application of this method, however, is not possible in an industrial environment. Because

of the conveyor belt movement speed and the tight spatial layout of coal particles on the belt area

it is necessary to apply other methods of three-dimensional image acquisition.

The use of laser technology, generating a beam of light with a very low divergence makes

it possible to focus the light on a very small area, allowing for the precise coordinate identifica-

tion of the highlighted point and therefore enabling development of a new method of 3D image

acquisition.

4. Depth maps and point clouds

In practice, regardless of the measurement method a three-dimensional image acquisition

leads to the formation of the so-called depth map. Such a map can be – and often is – regarded as

Unauthenticated

Download Date | 5/19/16 7:59 PM

20

monochromatic (single-channel) image in which every point stores the distance to the particular

object (part of the whole scene) in the direction indicated by a ray proceeding from an optical

imaging device and passing through a particular pixel on the sensor (Stefańczyk & Kornuta,

2014). A more practical for further image analysis seems to be – accomplished by the appropri-

ate conversion – assignment to each point of the image not a straightforward distance to the

optical measuring device but a distance calculated in the direction perpendicular to the image

sensor plane of the measuring device (or some other reference plane). This can be done by using

appropriate geometric calculations. Then, each pixel of the depth map will store the distance to

the object in the direction perpendicular to the specified reference plane. Both of these methods

have been shown in Fig. 4.

Fig. 4. Methods of distance description for three-dimensional images

a) from the center of the optical system; b) from the plane perpendicular to the axis of the optical system

There can be distinguished two types of depth maps – the so-called dense maps in which

almost every pixel of the image provides information about the depth of a particular part of the

mapped surface and so-called sparse maps in which only some – relatively few – contain such

information. It is related directly to the method of depth map creation. For example, dense maps

are obtained by analyzing the entire image (or set of images) originating from the camera, and

sparse maps are obtained by analyzing only some certain characteristic points e.g. vertices or

edges (Stefańczyk & Kornuta, 2014).

Depth information can also be stored in the form of so-called point clouds. In this method

each pixel is a point in the three-dimensional Cartesian space – it has three spatial coordinates,

and often is described with additional data (e.g. color – i.e three RGB components: red, green,

blue) to facilitate the presentation (visualization) and to join (merge) point clouds from successive

time instants or from different sensors. Due to much more complicated description of the given

point neighbourhood than in the case of the matrix representation, processing and analysis of

objects described as point clouds require new algorithms or adaptation of traditional algorithms

for 2D image processing. These both representations of three-dimensional space are partially

Unauthenticated

Download Date | 5/19/16 7:59 PM

21

compatible, and there are some methods to transform one of these representations to the other

one. Particularly, from each depth map there can be obtained an equivalent point cloud – but in

the opposite direction it cannot always be done losslessly (due to the fixed and limited spatial

resolution of the depth map). Point cloud can be stored as an ordered cloud – e.g. a two dimensional

array, where the points close to each other in the array are also located close to each other in the

3-D space an points spaced apart in the array are far from each other also in reality. Such a point

cloud can be typically created from the transformation of a depth map. However in the case of

merging two or more point clouds such an arrangement is not usually possible. Unfortunately

in the case of a “disordered” point cloud, searching the neighbourhood around a particular point

(this is usually a basic operation used in nearly all image analysis algorithms) is much more dif-

ficult. Hence, often to store this point cloud, there are used much more complex data structures

(Stefańczyk & Kornuta, 2014).

5. The method of laser triangulation

Triangulation – in a geodesic meaning – is a process of determining the length of the sides

of a triangle based on knowledge of the length of one side and two angles of a triangle using

appropriate trigonometric formulas. In geodesy it enables a coordinate determination for all the

points of a triangulation network.

Fig. 5. The principle of a passive triangulation

Passive triangulation used eg. in geodesy or stereo-vision methods is based on the observation

of the same point P from two positions A and B distant from each other by a distance AB of length

b and on the measurement of angles a and b under which that point P is viewed. Angles a and b

are measured respectively between the segments AP and AB and between the segments BP and

BA. Then the geometrical relations depicted in Fig. 5 can be described using an equation system:

Unauthenticated

Download Date | 5/19/16 7:59 PM

22

°

°

°

¯

°

°

°

®

bbb

ȕ

b

d

Į

b

d

21

2

1

tg

tg

(1)

Substituting two first equations into the third one it can be obtained:

¸

¸

¹

·

¨

¨

©

§

ȕĮ

db

tg

1

tg

1

(2)

and finally

ȕĮ

b

d

ctgctg

(3)

Since the observed point P has to be uniquely identified from both points of view A and

B, the observed picture shall be of high contrast (which is in practice very difficult to obtain

in the case of a stream of black coal particles on the black conveyor belt). For more complex

images it must be therefore defined a set of characteristic points (e.g. vertices, edges, etc.), and

these two images have to be compared, using e.g. image correlation methods (Bączek et al.,

2013). Achieving a high unambiguity projection is however difficult, for a rock sample consist-

ing of a large number of particles of similar size, additionally often touching and overlapping

each other.

Unambiguous identification of the observed point is possible with strong marking it with

a concentrated beam of light – e.g. a laser beam, like in laser pointing or sighting devices and

diagnostic devices. This leads to the use of active triangulation methods. The principle of this

method is presented in Fig. 6.

Fig. 6. The principle of an active laser triangulation

Unauthenticated

Download Date | 5/19/16 7:59 PM

23

Using symbols presented in Fig. 6 it is possible to describe a position of the reflected laser

beam on the camera sensor matrix with an equation:

Δx ≈ K · Δz · sin α (4)

where

a — triangulation angle

K — the gain factor dependent on on the parameters of the optical system (lens focal

length, the camera resolution) – its value can be determined theoretically or based

on experimental calibration

Therefore it can be written:

z = Z

0

+ Δz (5)

where Z

0

— base distance, determined by geometric dimensions of the whole system.

D

sin

'

'

K

x

z

(6)

Equations (6) and (7) have been written for the same single point. In the case of triangula-

tion of the entire stream surface it can be used a linear laser (a planar sheet of a laser light) a and

constant speed movement of the observed surface of the conveyor belt. Diagram of such a solu-

tion has been presented in Fig. 7.

Fig. 7. Layout of the laser triangulation system for the material stream transported on the conveyor belt

6. The results of an experimental research

In order to verify the method of laser triangulation described above in the Department of

Electrical Engineering and Control in Mining of the Silesian university of Technology there

has been developed an opto-mechatronic research stand, including a section of the conveyor

Unauthenticated

Download Date | 5/19/16 7:59 PM

24

belt, and the scanning system consisting of the linear laser LC532-5-3-F with a wavelength of

532nm (green) and optical power 5 mW and two HD cameras Logitech Pro C920 (with 2Mpix

resolution). The whole system is moved on the rails along the conveyor belt by a set of stepping

motors for precise positioning. The image analysis software is written in C++ language using

and OpenCV library (Rafajłowicz et al., 2009). An example view of the laboratory stand during

a calibration (including measurements and image rescaling of some simple objects of a known

shape and size) has been shown in Fig. 8.

Fig. 8. A view of the laboratory stand during calibration.

Fig. 9 shows two frames (from left and right-hand cameras) recorded in the same time.

These frames should be symmetrical, but due to the occurrence of reflected beam shadowing in

each of them there can be seen some interruptions and discontinuities. Only a combination of the

information from both frames can lead to a more complete knowledge on the observed portion

of the coal stream. Fig. 10 shows a photograph (a) of the central part (because at the edges of

the image there are greater distortions demanding non-linear correction) of an exemplary coal

stream sample and a corresponding depth map (b), calibrated directly in mm.

Fig. 9. Sample images of the laser lines (intersections of the laser light plane with the coal surface) from the

left and right hand cameras on the basis of which the resultant image is formed

Unauthenticated

Download Date | 5/19/16 7:59 PM

25

a) b)

Fig.10. Sample photograph of a central part of the coal sample (a) and the corresponding depth map (b) (cali-

brated directly in mm)

To illustrate the principle of sequential triangulation scanning Fig. 11 shows a selected (for

clarity) subset of recorded laser lines forming the depth map of Fig. 10b.

Fig. 11. A subset of recorded intersection lines of the laser light plane to the particle surfaces of the depth map

shown in Fig. 10b. (for clarity only every 40th line has been shown)

For the purpose of depth maps and photographic image properties comparison, there have

been selected (for each of these two mappings) two cross sections – one longitudinal and one

transverse, made in places marked with horizontal and vertical lines on Fig. 12. For a photographic

image (Fig. 13 b) there can be seen a high level of noise, related to the reflection of light from

glossy parts of particle surfaces. The depth map (Fig. 13 a) much better reproduces the spatial

arrangement of coal particles.

Unauthenticated

Download Date | 5/19/16 7:59 PM

26

a) b)

Fig. 12. Selected locations of sample cross-sections for the comparative analysis of depth map (a) and the

photographic image (b) properties

Fig. 13. Sample surface cross-sections (longitudinal in the upper subplots, transverse in the lower subplots) of

the laser triangulation-based depth map (a) and a corresponding photographic image (b)

a)

b)

Unauthenticated

Download Date | 5/19/16 7:59 PM

27

7. Summary and conclusions

The three-dimensional image acquisition method for the material flow on the conveyor

belt, based on laser triangulation enables much more precise modeling of structure, shape and

size of each grain than in the case of the analysis based on two-dimensional images. It makes

possible to identify grains partially overlapped by other particles and enables a compensation

of distortions resulting from the oblique particle orientation. Due to the light intensity of the

laser beam, this method is particularly suitable for the analysis of materials with low surface

reflectance(e.g. coal). In addition to more precise description of the shape and size of individual

particles, this method makes possible to measure the overall quantity of material (volume of

the stream) Together with the signal from belt conveyor scales it makes possible to estimate the

bulk density of the material stream. An important advantage of this method is relatively simple

signal processing (thresholding and filtering operations identifying the intersection line of the

laser light sheet and the coal surface are easy due to the high contrast of the resulting image) and

straightforward conversion of laser line image pixel coordinates to real world coordinates based

on simple trigonometric dependences. The fast detection of changes in particle size distribution

enables the optimization of complex technological systems, leading to better stabilization of

quality parameters of output products (Heyduk & Pielot, 2014). This is particularly important in

the case of gravity concentration in jigs (Pielot, 2010).

References

Bączek A., Chudek M., Cierpisz S., Heyduk A., Jendryś. J., Joostberens J., Kleta H., 2013. Visualization-assisted method

of the assessment of the technical condition and safety of the mineshaft lining using a digital image analysis. Mono-

grafia nr 482, Wydawnictwo Politechniki Śląskiej, Gliwice.

Heyduk A., 2005. The influence of lighting conditions on the image segmentation in machine vision system of particle

size analysis. Mechanizacja i Automatyzacja Górnictwa, R. 43, nr 10, s. 21-29.

Heyduk A., Pielot J., 2014. Economical efficiency assessment of an application of on-line feed particle size analysis to

the coal cleaning systems in jigs. Inżynieria Mineralna, R. 15, nr 2, s. 217-228.

Koh T.K, Miles N., Morgan S., Barrie Hayes G., 2007. Image Segmentation of Overlapping Particles in Automatic Size

Analysis Using Multi-Flash Imaging. IEEE Workshop on Applications of Computer Vision, p. 47-52,

Pielot J., 2010. An analysis of effects of coal jigging after changes in the grain composition of a feed. Arch. Min. Sci.,

Vol. 55, No 4, p. 827-846.

Rafajlowicz E, Rafajłowicz W., Rusiecki A., 2009. Image processing algorithms and an introduction to work with an

OpenCV library. Wydawnictwo Politechniki Wrocławskiej, Wrocław.

Stefańczyk M., Kornuta T., 2014. Acquisition of RGB-D images – methods. Pomiary, Automatyka, Robotyka, R. 18,

nr 1, s. 82-90.

Tosun A., Konak G., Toprak T., Karakus D., Onur A., 2014. Development of the Kuz-Ram Model to Blasting in a Lime-

stone Quarry. Arch. Min. Sci., Vol. 59, No 2, p. 477-488.

Trybalski K., 2013. Control, modeling and optimization of technological processes of the ore processing. Wydawnictwa

AGH, Kraków.

Szponder-Kołakowska D.K., Trybalski K., 2014. Modern methods and measuring devices in the study of raw materials

and mineral wastes. Wydawnictwa AGH, Kraków.

Unauthenticated

Download Date | 5/19/16 7:59 PM