Automatic Sorting System Using Machine vision
Saurin Sheth1#,Rahul Kher2*, Rushabh Shah3#, Parth Dudhat4*, Pratyush Jani5#
Mechatronics1#, Electronics2*, Mechatronics3#, Mechatronics4*, Mechatronics5# Department, Sardar Patel University
G H Patel College of Engineering & Technology, Vallabh Vidhyanagar, Gujarat, India
email@example.com firstname.lastname@example.org email@example.com firstname.lastname@example.org email@example.com
Abstract: Nowadays various automation techniques are being
adopted & researched on for increase in productivity, for
better accuracy, eliminating the human errors and for safety.
Machine Vision is one such advancement in automatic
systems. Machine vision performs the tasks that are
equivalent to human vision. It helps to automate the systems
where there are limitations of human vision like detecting
various shades of colors or determining high precise
dimensions and thus permitting human employees to serve in
more appropriate positions. Now, what happens when the
questions turn to "Is this part of correct color?" or "Which
parts are blue and which red?” So in our system, colour based
identification of the parts will be done and then it will be
sorted according to different colours. After recognizing the
colour of the object, robotic arm will automatically pick &
place it accordingly. If the colour of the work piece is not
found in accordance to the required one then it will be
rejected. The complete sorting system operates on image
processing using the MATLAB application & microcontroller
which will control different motors in the system. Machine
vision based on color concept has found its wide application in
the pharmaceuticals industry, agriculture industry and
assembly of parts especially in automobile industry. Advances
in both machine vision technology and related software tools
have enabled manufacturers to apply color machine vision on
the factory floor in real time. This technology can solve
problems formerly restricted to manual inspections & sorting.
Keywords- Machine Vision, MATLAB, Microcontroller,
Sorting, Colour Inspection
1.1 Machine Vision
Machine vision is the application of computer vision to industry and
manufacturing.Two important specifications in any vision system are the
sensitivity and the resolution. The better the resolution, the more confined
the field of vision. Sensitivity and resolution are interdependent. All other
factors held constant, increasing the sensitivity reduces the resolution, and
improving the resolution reduces the sensitivity.One of the most common
applications of Machine Vision is the inspection of manufactured goods
such as semiconductor chips, automobiles, food and pharmaceuticals. Just
as human inspectors working on assembly lines visually inspect parts to
judge the quality of workmanship, so machine vision systems use digital
cameras, smart cameras and image processing software to perform similar
inspections. Machine vision systems are programmed to perform narrowly
defined tasks such as counting objects on a conveyor, reading serial
numbers, and searching for surface defects.
Manufacturers favour machine vision systems for visual inspections
that require high-speed, high-magnification, 24-hour operation, and/or
repeatability of measurements. Frequently these tasks extend roles
traditionally occupied by human beings whose degree of failure is
classically high through distraction, illness and circumstance. Human eyes
are sensitive to electromagnetic wavelength s ranging from 390 to 770
nanometres (nm). Video cameras can be sensitive to a range of
wavelengths much wider than this. Some machine-vision systems function
at infrared (IR ), ultraviolet (UV), or X-ray wavelengths.
1.3 Origins of Colour in Machine Vision
Colour in machine vision began in the mid-1980s with the
commercial availability of colour cameras and imaging boards.
The first application for which colour machine vision was
widely deployed was food processing, specifically the high-
speed sorting of fruit where the goal was to detect and remove
flawed, blemished and/or inappropriately sized fruit as well as
extraneous vegetable matter.
In industry applications, automotive fuse block inspection used
RGB color imaging to verify the presence of the correct fuses
was the first use of colour machine vision.
1.4 APPLICATIONS OF THE SYSTEM
Specifically, manufacturers use colour vision to solve three primary
1. Color verification -- Verifying that a certain part's colour matches what
the vision system is programmed to find.
2. Color sorting -- Sorting parts based on colour.
3. Color inspection -- Inspecting coloured parts for defects that grayscale
image processing tools can't detect.
2. DESIGN OF THE SYSTEM
2.1 Block diagram
2.2 Working of the system
Working of our system is very simple. Initially when product is on the
conveyor belt, Sensor will detect the presence of the particular product &
give signal to the microcontroller. Then microcontroller will send this
signal to the computer by serial interfacing. Image processing software
(Matlab) of the system will send the signal to the camera for capturing the
image. Once image is captured, the software will process on the captured
image and will generate signals according to requirement and which in
turn the signals will be send back to microcontroller. Accordingly the
microcontroller will control the conveyer belt & robotic arm. Robotic arm
will pick & place the given component according to the color. If color is
not matched with a given requirement, the product will be rejected. This
cycle will be repeated number of times as per requirement.
3. COMPONENTS OF THE SYSTEM
3.1 Electrical components
Sensor is a device that measures or detects a physical condition.
An electronic sensor converts this measurement or detection into
equivalent analog or digital electrical signal. Here we will be using
proximity sensor. A proximity sensor is a sensor able to detect the
presence of nearby objects without any physical contact. A proximity
sensor often emits an electromagnetic or electrostatic field, or a beam of
electromagnetic radiation (infrared, for instance), and looks for changes in
the field or return signal. The object being sensed is often referred to as
the proximity sensor's target. The maximum distance that this sensor can
detect is defined "nominal range". Some sensors have adjustments of the
nominal range or means to report a graduated detection distance.
Proximity sensors can have a high reliability and long functional life
because of the absence of mechanical parts and lack of physical contact
between sensor and the sensed object.
B. Specification of sensor
Input:10 to 30v
A microcontroller (also microcontroller unit, MCU or µC) is a small
computer on a single integrated circuit consisting of a relatively simple
CPU combined with support functions such as a crystal oscillator,
timers, watchdog timer, serial and analog I/O etc. Program memory is also
often included on chip, as well as a typically small amount of RAM.
Microcontrollers are designed for small or dedicated applications. Thus, in
contrast to the microprocessors used in personal computers and other
high-performance or general purpose applications, simplicity is
Some of our basic requirement for the project from microcontroller:
1. Output port:
4 port pins for each stepper motor with full step sequence
2 port pins for each stepper motor with half step sequence
1 port pin each for servo motor
2. Input port: for proximity sensor
3. Timers: for generating delays for driving stepper motors
4. PWM: for providing delays for the rotation of servo motors
5. USART: for exchanging data with computer
6. Memory (Internal/External): for storing program
220.127.116.11 Selection of microcontroller
For our project purpose, we have selected AVR based microcontroller
named ‘Atmega32’ of the Atmel Corp. which has few of the following
special features to fulfill our above stated requirements:
Up to 16 MIPS Throughput at 16 MHz
32K Bytes of In-System Self-programmable Flash program
1024 Bytes EEPROM
2K Byte Internal SRAM
Two 8-bit Timer/Counters and One 16-bit Timer/Counter
Four PWM Channels
8-channel, 10-bit ADC
External and Internal Interrupt Sources
32 Programmable I/O Lines
Operating voltage of 4.5 - 5.5V
3.1.3 Camera system
It will capture the image of object when signal is given to it by
We are using simple USB web camera. So Interfacing will
become easy with computers.
Image resolution: 160x120,176x144,320x240,352x288
Frame rate:upto30 frames per second
Camera Controls: Color saturation, brightness, sharpness is
Exposure: Auto or manual
Focus Range:4cm to infinity
3.2 Mechanical components
3.2.1 Robot arm
The robot arm is probably the most mathematically complex robot you
could ever build. But here we have made very simple arm for picking or
placing the object according to its colour.
Design of robot arm
A. Degree of freedom
B. Workspace of the robot
C. Torque calculations at different joints
D. Selection of motors
A. Degrees of Freedom (DOF)
The degrees of freedom, or DOF, are a very important term to understand.
Each degree of freedom is a joint on the arm, a place where it can bend or
rotate or translate. You can typically identify the number of degrees of
freedom by the number of actuators on the robot arm.
The robotic arm which we are going to use has 3 DOF.
B. Robot Workspace
The robot workspace (sometimes known as reachable space) is all places
that the gripper can reach. The workspace is dependent on the DOF
angle/translation limitations, the arm link lengths, the angle at which
something must be picked up at, etc. The workspace is highly dependent
on the robot configuration.
The figure shows the free body diagram of robotic arm which we will be
used for our project.
C. Torque calculations
Torque calculation is a very important factor for designing the robot arm.
As our system needs 20cm of robot arm (according to size of conveyor
belt & block), we can calculate motor torque from that.
Here is the FBD of given robot arm.
W1 - Weight of block ( 0.150 kg)
W2 - Weight of gripper ( 0.350kg,motor weight is included)
W3 - Weight of link ( 0.500 kg)
Wm1 – Weight of motor 1 ( .250kg)
L - Length between gripper and motor1 ( 20 cm)
Distance between M1 & M2 is 5cm.
Torque about motor1
= (W1+W2)*L + W3*(L/2)
=(0.150+0.350)*20 + (.500*10)
Torque about motor2
= (W1+W2)*(L+5) + W3*(L/2 + 5) + Wm1*5
= (.150+0.350)*25 + (.500*15) + (.250*5)
= 21.25 kg.cm
We are using stepper & servo motors in arm to get good
accuracy, precision, repeatability.
we are using two servo motors (Motor1 & 2) which have
For gripper we are using dc stepper motor because it needs less
3.2.2 Conveyor belt
Conveyor belt is nothing but the mechanical apparatus consisting of a
continuous moving belt that transports materials or packages from one
place to another. Conveyor belts are often driven by variable speed
electric motors or by other moving parts in a complex system. They are
commonly found in factories, grocery stores, warehouses and public
Length : 900mm
Width :135 mm
Thickness : 5 mm
1. Permanent magnet DC servo motor
1.1. Model : D2415PL20
1.2. SR No : PDD8013
1.3. Torque : 35 kg-cm
1.4. Current Rating: 2.5 Amps
1.5. Voltage : 24 V
1.6. RPM : 80
3.3.1 Image processing software
Image processing software captures the incoming image & processes it
according to its color. & after that it gives signal to microcontroller.
We have used RGB color space. RGB color space uses a rectangular
coordinate system with one coordinate axis assigned to each of three color
components, red, green, and blue.
Here we are using Mat lab software for image processing.
MATLAB is a high-performance language for technical computing. It
integrates computation, visualization, and programming in an easy-to-use
environment where problems and solutions are expressed in familiar
mathematical notation. 
Typical uses include
Math and computation
Modelling, simulation, and prototyping
Data analysis, exploration, and visualization
Scientific and engineering graphics
Application development, including graphical user interface
In Matlab we are concentrating on Image processing toolbox. Image
Processing Toolbox is a collection of functions that extend the capability
of the MATLAB numeric computing environment.
The toolbox supports a wide range of image processing operations,
Spatial image transformations
Neighbourhood and block operations
Linear filtering and filter design
Transforms Image analysis and enhancement
Region of interest operations
18.104.22.168Technique to Set the Threshold Values
For detecting the colour we have to set the threshold values of particular
colour. According to that colour software will give results.
Example: Detection of red colour
The threshold for red component should be the least value of the red
component found in the region of the red ball.
The threshold for green component should be the maximum value of the
green component found in the region of the red ball.
The threshold for blue component should be the maximum value of the
green component found in the region of the red ball.
Algorithm for red colour
fR= rgb_image(:,:,1); fG=rgb_image(:,:,2); fB=rgb_image(:,:,3);
I=((fR>=80) & (fG<=70) & (fB<=70)); (Set the threshold values for red
Fig 1 Fig 2
We have made a program in which when require colour
component comes in the camera, signal (Blue Square) will be
displayed. [fig1].& robot arm will pick & place it according to
If required colour is not there then it will be rejected.[fig 2]
Here we have used GUI (Graphics user interface)for
convenience of user.
Interfacing is a term used in electronics when different electronic
devices are attached. The term is commonly used when attaching memory
chips. Interfacing also often refers to the connection of peripheral devices
to computers. For Micro-controllers it is useful to the extent that they
communicate with other devices, such as sensors, motors, switches,
keypads, displays, memory and even other micro-controllers. Many
interface methods have been developed over the years to solve the
complex problem of balancing circuit design criteria such as features,
cost, size, weight, power consumption, reliability, availability,
In our project we need to do the following interfacing:
A. Computer to Web camera
B. Computer to MAX232 IC
C. MAX232 IC to Microcontroller
D. MICROCONTROLLER to actuators (stepper motor and servo
E. Microcontroller to sensor
A. Computer to Web camera :
Webcam is used for capturing the image, and this image is needed to be
sanded in to the image processing software for processing, this is achieved
by this interfacing.
How it is done:
As webcam used is a plug and play device, it can be easily connected to
computer, via USB port, in which the image processing needs to be done.
B. Computer to MAX232 IC:
The data between the computer and the microcontroller is exchanged via
serial communication (RS-232). Usually all the digital ICs work on TTL
or CMOS voltage levels which cannot be used to communicate over RS-
232 protocol. So a voltage or level converter is needed which can convert
TTL to RS232 and RS232 to TTL voltage levels.
The most commonly used RS-232 level converter is MAX232. This IC
includes charge pump which can generate RS232 voltage levels (-10V and
+10V) from 5V power supply. It also includes two receiver and two
transmitters and is capable of full-duplex UART/USART communication.
How it is done:
To communicate over UART or USART, we just need three basic signals
which are namely, RXD (receive), TXD (transmit), GND (common
ground). So to interface MAX232 with microcontroller we just need the
basic signals. A simple schematic diagram of connections between a RS-
232 port of computer and MAX232 is shown below.
C. MAX232 IC to Microcontroller:
As stated above the need for the data transmission between the computer
and microcontroller, by serial communication over UART or USART,
requires some voltage or level converter IC, which is done via MAX232
How it is done:
The R1OUT and T1IN pins of MAX232 IC are simply connected to RX
and TX pins of microcontroller.
D. MICROCONTROLLER to actuators (stepper
1. Microcontroller to stepper motor
Given circuit is nothing but the interfacing between microcontroller to
stepper motor via ULN2003A.
2. Microcontroller to servo motor
E. Microcontroller to sensor
We have connected the sensor to the port1 which will be used as input
port. So when sensor senses the object, Signal will go to the
Reduce labor cost
Less human interference
Ex: In the white color family, shades might include white, off-
white, champagne, pearl and cream. Human operators often
mismatch these colors, while color vision tools can distinguish
more reliably between them.
7. FUTURE SCOPE & CONCLUSION
Here instead of using a webcam vision camera is to be used. The
machine vision camera obviously will have better resolution, zooming
capacity and clarity and moreover an inbuilt circuitry for external
triggering. So it will save the circuitry used for interfacing the sensor with
computer as the sensor output will be directly made available to the
camera so it will also save the time as computer will not be in the
connection at all.
Here only part sorting is done but in future by some modification we can
also include following things:
Number of parts are sorted
Number of parts passed
Checking label on the part
Check the crack or dent on the part
So the conclusion from the system is we can automate the colour
inspection & sorting of the object with accuracy, good
repeatability & high productivity.
8. 3D MODEL OF THE
An integral automation of industrial fruit and vegetable sorting by
machine vision By F Pla, JM Sanchiz, JS Sanchez - Proc. 8th IEEE
International Conference on Emerging …, 2001
Prospects of computer vision automated sorting systems in agricultural
process operations By AO Raji, AO Alamutu - … Engineering
International: the CIGR Journal of …, 2005 - journals.sfu.ca
Automation, Production System, And Computer-Integrated
manufacturing. By Mikell P. Groover
The 8051 Microcontroller Architecture, Programming, &
Applications. By Kenneth J. Ayala
Digital Image Processing Using MATLA. By Gonzalez, Woods, &