Utilization of 3D sensors in robotics

Abstract

Recently, the interest in development of service robots highly increases. The annual turnover in this area is estimated to a number close to 17 milliard Euro in the past years. The annual turnover could rise to 0.1 billion Euro by 2020 by IFR considering 30 % growth every year. Investments are expected to flow into all areas related to service robotics, mainly into the development of assistant robots for seniors and the handicapped people, health monitoring and surgical robots, robots in agriculture, pilotless drones and helicopters and ground vehicles without a driver. Very promising years seems to be coming for all the new and already existing companies focused on this area with their software and sensor engineers together with producers of important accessories.

Full text

* Corresponding author: marcanik@fai.utb.cz
Utilization of 3D sensors in robotics
Miroslav Marcaník1,*, Michal Šustek1, Pavel Tomášek1, and Roman Jašek1
1Tomas Bata University in Zlín, Faculty of Applied Informatics, Nad Stráněmi 4511, 760 05, Zlín, Czech Republic
Abstract. Recently, the interest in development of service robots highly increases. The annual turnover in
this area is estimated to a number close to 17 milliard Euro in the past years. The annual turnover could rise
to 0.1 billion Euro by 2020 by IFR considering 30 % growth every year. Investments are expected to flow
into all areas related to service robotics, mainly into the development of assistant robots for seniors and the
handicapped people, health monitoring and surgical robots, robots in agriculture, pilotless drones and
helicopters and ground vehicles without a driver. Very promising years seems to be coming for all the new
and already existing companies focused on this area with their software and sensor engineers together with
producers of important accessories.
1. Introduction
Monitoring is used in various non-engineering
applications. Tasks not only of this type are operated by
service robots all over the world. Robots are equipped
with a camera system for these purposes. The data from
cameras are recorded and transmitted to the standpoint of
an operator. Service robots can be favourably utilized
wherever the environment is dangerous for a man or
difficult to reach. It could be for example searching for a
dangerous material, security purposes, monitoring
complicated areas, undersea research.
Basic terms related to robotics, taxonomy,
classification of robots and examples of complex robots
are presented in the following section. The next section
is focused on utilization of sensors, sensorial systems
and how they work. In the last section the principle of
3D scanning of objects in free space is described
together with an application in a real project.
2. Robotics
Robotics is a modern multidisciplinary field including
mechanics, electronics, control theory, measurement
techniques, artificial intelligence and other disciplines.
Currently, robotics is mainly connected with the area of
automatization. The first appliance called as a robot was
produced at the beginning of the twentieth century. The
word of robot is known already since 1920. Čapek
brothers (Czech writers) created this word in their book
with the name of R.U.R. A robot represents a non-
biological artificial imitation a man [1].
2.1. Classification of robots
The robots can be divided by several different aspects.
Dividing by mobility:
 Stationary – they cannot move from one place to
another (manipulators in the industry, for instance),
 Mobile – they are able to move (satellites or
vehicles in the aerospace).
Mobile robots can be divided by several other
criteria [2]:
 Remotely controlled – they are controlled by an
operator, who has a visual information about the
space around such a robot,
 Autonomous – they are supposed to solve tasks
independently, for instance line tracking, ability to
react on obstructions in their way, ability to move in
an unknown area.
Moreover, robots can be divided by the environment
in which will the robot operate:
 On the land – wheel and full-track vehicles.
 In the water – explorational submarines, robotic
fish,
 In the air – quadcopters, drones, RC models, robotic
birds,
 In the aerospace (outer space) – service robots,
satellites.
Dividing by the purpose:
 Manipulatory – for manipulations with objects,
 Assembly – as a part of complex assembling lines,
 Service – for repairs and maintenance at places out
of simple reach,
 Inspectional – for controlling of a state of an
observed phenomenon,
 Explorational – for exploring of an unknown area,
 Military – pyrotechnical robots, drones,
 Medical – manipulators, laser scalpels,
 For fun – quadcopters, drones, RC models,
Dividing by the type of a running gear:

DOI: 10.1051/
,02021 (2017) 71250
1
MATEC Web of Conferences
25
matecconf/201
CSCC 2017
2021
© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative
Commons Attribution
License 4.0 (http://creativecommons.org/licenses/by/4.0/).

 Wheel,
 Full-track,
 Walking,
 Crawling,
 Climbing,
 Hybrid,
 Intelligent.
2.2. Examples of robots
ASIMO – Honda (mobile – autonomous – for walking
on the land – medical aims): It is probably the best
publicly known robot. It was introduced in 2000.
ASIMO was constructed to imitate a human being and
thus to replace him in a common human interaction.
Therefore this is a humanoid. ASIMO could server as
companion for elderly or ill people. It can react to a
human behaviour and speech. It is not applicable in
exacting environment or a terrain (compared to robots
from Boston Dynamics). ASIMO is 130 cm high. It
weighs 50 kg. Its maximal speed is 5.7 km/h, 2.7 km/h
normally. ASIMO has 57 degrees of freedom (DOF)
totally. What is very interesting about ASIMO is the fact
that ASIMO can be controlled by a special head cap.
Thoughts (electrical charges) are captured by sensors on
the cap and then the cap transmits the signals. These
signals can control the movement of the robot. Overall,
operation of such a robot is very energetically exhaustive
because there is a need of an actuator for every DOF.
Therefore it can operate only one hour on its internal
battery [2].
Fig. 1. ASIMO [2].
Mini AERCam – NASA (mobile – autonomous – for
outer space – flying – inspectional/service): This robotic
camera was developed by NASA in 2005 for inspections
in the outer space. It is a freely flying robot equipped
with 12 nozzles for compressed gas allowing movements
and rotations in all directions. It is able to orientate in
complex three dimensional space autonomously. It has a
spherical shape. The diameter is 190 mm and it weighs
4.5kg [2].
Fig. 2. Mini AERCam [2].
ATLAS – Boston Dynamics (mobile – autonomous – on
the land – walking – multifunctional): Atlas is highly
mobile humanoid specially constructed for exacting
terrain. It was developed by Boston Dynamics with the
support of Defence Advanced Research Projects Agency
(DARPA). It can climg, run, manipulate and interact
with a surrounding environment. It is 180 cm high. It
was introduced in 2013. Its arms contain joints which
allows ATLAS to use common human tools in its hands.
Atlas is prepared to work in dangerous areas in which a
man cannot survive. Therefore ATLAS is able to use
instruments, manipulate, work with electrical appliances
and perform many other similar tasks. It is equipped with
a high number of sensors, stereo camera, and laser
range-finder. It has implemented 28 DOF. ATLAS
operates on a cable connected to an external power
source [2].
Fig. 3. ATLAS [2].
An Unmanned Aerial Vehicle (UAV) or also a drone is a
plane without crew which can be controlled remotely or
fly autonomously using programmed flying plans or
using more complex dynamic autonomous systems.
These aeroplanes have a common military utilization.
They can explore and also attack. They can also serve in
some specific civil tasks (for example in fire fighting,
monitoring used by police or for exploration of some
terrain) [3].

DOI: 10.1051/
,02021 (2017) 71250
1
MATEC Web of Conferences
25
matecconf/201
CSCC 2017
2021
2

Fig. 4. Model UAV [3].
3. Sensors
Sensors are elements, which produce input block
measuring chain, then this chain is in direct contact with
the measured medium. Other labels of sensors are for
example sensor, detector, converter, and scanner [4].
Sensors could be ordered by usability into the
following categories:
 For measuring quantities,
 For physical principles,
 For the measurement environment,
 For signals according to their transformation,
 For Manufacturing Technology.
Changes of measuring quantities are scan over
sensitive element mostly known as receptor, then
subsequent evaluation in the sensor circuit
(analog/digital converter), where the measured value is
converted mostly into electric impulse, which could be
processed or could be used directly as the output
quantity. Principle of sensor is shown at Figure 5.
Fig. 5. Principle of a sensor [6].
First smart sensor (1978) Beckenbridge and Husson, was
defined as: Smart sensor contains functions for
processing measured data, automatic correction
measured data, and it could automatically detect and
eliminate abnormal and wrong values. It also contains a
set of algorithms, which allows to respond on changes of
external conditions.
Fig. 6. The principle of an intelligent sensors [6].
3D sensor can be defined as a measuring unit which
senses an object in 3D space or occupies angles X, Y and
Z. 3D sensors operate on the principle of two or more
video cameras, or for engaging the third dimension may
be used most of the laser scanner. This makes possible a
whole or only a certain part transform into 3D and 3D
scene then analyses and selects the best way to handle
[5].
From the above description 3D sensors can be
divided into two basic categories by using various
sensors. They are active and passive sensing:
 Active scanning uses laser/camera, additional
information or other auxiliary devices. The concept
of active sensing is very closely linked to the
concept of "creating depth map".
 Passive versus active scan uses only one sensor –
the camera/laser.
Since it is an improved sensor it can be categorized
in a similar 3D sensors. A general allocation can be seen
in Fig. 6 [5].
Fig. 7. Distribution of 3D sensors [5].
4. Principles of sensorial systems
Acquisition data about a space from a sensor and its
processing using some tools is the main purpose of a
sensorial system. Sensors providing the important data
about a space:

DOI: 10.1051/
,02021 (2017) 71250
1
MATEC Web of Conferences
25
matecconf/201
CSCC 2017
2021
3

 Photo sensors – measuring the intensity of a light, or
the sensitivity of an optical device.
 Acoustic sensors – measuring of the sensitivity and
frequency range of a sound.
 Mechanosensor – mechanical interactions.
 Chemosensors – detecting chemicals.
 Thermo sensors – they use thermistors and other
thermoelectric sensors for measuring of the
temperature.
Generally, sensorial systems can be divided into two
basic groups. There are inner and outer systems. The
outer systems provide data about the outer environment
and the inner systems provide data only about inner
status and parts of a robot. The following actions take
place during scanning:
 Localization of objects,
 Studying properties of objects,
 Studying properties of the image of an environment,
 Measuring status of accumulators,
 Orientation in an environment.
5. Principles of 3D scanning
3D sensors are coming to the forefront and are more and
more commonly used in all possible sectors because of
the development of technology and the increasing
demand and popularity of virtual reality. Some examples
include construction, architecture, industrial machinery,
navigation, etc. Perhaps in all applications, it is
necessary to have coordinates in a 3D space. The
scanning is dependent on the position of the object, its
speed, colour, shape, and angle of rotation. The advent
of modern technology and optics, which is still used for
2D scanning, expanded to methods for measuring the
third dimension. Today, there are mainly two methods of
measuring. The first one is a triangulation and the second
is a light interference. The gradual development of
technologies brings about the third method. This method
is called Time of Light (TOF) and uses the knowledge of
the speed of light.
5.1. Triangulation
Triangulation is very often used as a method which
requires a very complicated structure of the measuring
sensor. It is divided into two categories: active
triangulation and passive triangulation.
The principle of the active triangulation involves
photogrammetric reconstruction of the scanned object.
The surface of scanned object is illuminated by the light
source and simultaneously scanned by the CCD sensor
[7].
The principle of passive triangulation involves
photogrammetric reconstruction of the scanned object on
the basis of its projection on the sensor surface device.
One dimension is lost during projection and it is needed
to renew on the basis of a common information from
multiple sources [7].
The light signal is transmitted from the laser source
to the object. The object reflects the light ray to the
camera. The angle of the transmitted ray from the source
is constant, but the CCD sensor depending on the spend
ray of the dimmed sensor. The connector between the
light source and the CCD sensor is called triangulation
base (baseline). Thanks to the knowledge of two angles
and the length of the triangular base, we can calculate
the distance of a point, and then save the coordinates for
later calculations and rendering [5].
To mark the surface, the following are used:
 The light beam (1D triangulation),
 Light (2D triangulation),
 Structured light beam (3D triangulation).
Disadvantages:
 Higher purchase price and better facilities,
 Time demands when evaluating the Record,
 The limiting factor may be the memory size and
especially the quality of the recording,
 More necessary knowledge of issues,
 Treatment of subjects.
5.2. Interferometry
Interferometry is a method suitable for very precise
measurements over a short distance. The principle is
based on light interference. The principle can be seen in
Figure 6. The light source – the laser – is transmitted
through the polarization splitter – a part on the measured
object. The reflected signal is combined with
polarization ray splitter reference called carrier wave and
they may interfere together. The resulting wave
interference is given by equation (1) [5]:
















yxjyxjyxIyxIyxIyxIyxI ,,cos,,2,,, 2121
2
2
2
1 (1)
Disadvantages:
 Technologically intensive production,
 Demanding quantitative interpretation of results,
 Susceptibility to interference.
5.3. Time of light
This is a method that uses the knowledge of the
speed of modulated light signal that is emitted from the
transmitter and subsequently reflected towards the
receiver. The use of this method requires very precise
time value. The principle can be seen in Fig. 7. The
distance of sensing object can be computed from formula
2, where “t” is the total time from sending a signal to the
one more acceptation and “c” is the speed of light (c =
2.998 * 108 m/s) [5].
2
*tc
s (2)
Disadvantages:
 High-accuracy time measurement required,
 Measurement of a light pulse return is inexact, due
to light scattering.
 Difficulty to generate short light pulses with fast rise
and fall times.

DOI: 10.1051/
,02021 (2017) 71250
1
MATEC Web of Conferences
25
matecconf/201
CSCC 2017
2021
4

6. Utilization of triangulation in a real
project
The trajectory of a laser ray (using a laser module M-
18B532-X-GL, for instance) impacts on objects in a
space what is caught by a camera subsystem what is one
of the main functional groups of a robot. A robot can
contain a colour digital camera (like Go Pro Hero4) with
a CMOS sensor positioned in a water and vapour
resistant box or coating. The robot could be controlled
remotely using a radio frequency 2.4 GHz using
approved transmitter and receiver for common
outdoor/indoor utilization. The communication will be
done using a software programmed (in C#, for instance)
providing computations and communication between the
robot and an operator.
7. Conclusion
The qualities of 3D sensors became higher thanks to the
progress in the modern trends in robotics. Grow in this
area increases due to better digitalization, cartography,
security and other aspects.
The main goal of this work lies in outlining
definitions of basic terms and principles of individual
categories of sensors. The authors were focused on
global aspects and coping whole systems as complex
units. Therefore principles of 3D sensors were described
without detailed aspects of low-level technologies.
Alternatives of sensors and their further utilization in
real life were discussed in this work.
However, the progress in application of 3D sensors
and 3D technologies generally is expected to grow more
than people usually consider. This statement is based on
the research data presented in this paper. The utilization
of all the three dimensions (which used to be in
seclusion to date) is the main advantage of 3D sensors.
The authors guess that traditional 2D sensors will almost
entirely vanish and will be replaced with 3D sensors in
several years. New sensors will be more expensive but
they will worth the money due to their possibilities in the
area of visualisation and robotics.
Acknowledgment
This work was supported by the Ministry of Education,
Youth and Sports of the Czech Republic within the
National Sustainability Programme project No. LO1303
(MSMT-7778/2014) and also by the European Regional
Development Fund under the project CEBIA-Tech No.
CZ.1.05/2.1.00/03.0089. This work was also supported
by Internal Grant Agency of Tomas Bata University
under the project No. IGA/FAI/2017/004.
References
[1] L. Kárník, Practical applications of service robots
(2011)
[2] J. Bartušek, Mobile device for movement in
dangerous environment (2015)
[3] Ch. Winkler, Design Technology [online] (2016)
[4] Sensors & Transducers, 206 (2016)
[5] M. Marcaník, J. Dvorak, Use of 3D sensors for the
protection of critical infrastructure elements and
soft targets (2015)
[6] H. Foell, MEMS Sensors and Actuators [online].
(2011)
[7] S.C. Islam, M. Herrmann, R. Beigang, IRMMW-
THz2007 (2007)
[8] P. Skoupý, 3D optical measurement and scanning
systems for engineering (2007)
[9] F. Brunet, Contributions to Parametric Image
Registration and 3D Surface Reconstruction (2010)
[10] W. Schroeder, K. Martin, B. Lorensen,
Visualization Toolkit: An Object-Oriented
Approach to 3D Graphics (2006)
[11] M. Janková, J. Dvořák, Mathematics, Information
Technologies and Applied Sciences, (2014)

DOI: 10.1051/
,02021 (2017) 71250
1
MATEC Web of Conferences
25
matecconf/201
CSCC 2017
2021
5