A Low-Cost, Low-Power System for 3D Plant Imaging

Abstract

Photometric Stereo (PS) is a method for the capture of 3D surface data using only a single camera and a number of carefully positioned light sources. By capturing a number of images of a plant, each illuminated from a different direction, we can calculate the surface orientation under each pixel of the image. An advantage of this technique is its low-cost nature, requiring only one camera and a number of off-the-shelf LED light-sources to produce high resolution data that can then be used to reconstruct fine details.

This work comprises of an early feasibility study to assess the possibility of using PS to obtain useful and accurate measurements of plants, above and beyond those possible using traditional 2D imaging techniques. The early-stage prototype hardware is based around a Raspberry Pi 2 (RPi), with the NoIR camera being used to capture images. PS requires a minimum of three light sources to constrain the three degrees of freedom relating to a surface normal; however by incorporating additional light-sources, we can account for the likely situation whereby a pixel is covered by shadow from a particular orientation. As such, we use four 850nm nIR 5V LED clusters, which are triggered through the on-board GPIO connectors of the RPi.

An additional advantage of PS is that, as a matter of course, it separates surface shape from colouring (albedo), which allows us to examine these two aspects in isolation. We use this to our advantage, taking 2D measurements of leaf morphology, such as width and height, from the albedo. We then examine further metrics of the plant using its surface shape, ranging from larger scale aspects such as modelling the 3D geometry of the leaf, down to the isolation of the midrib and individual veins. Initial results are promising, with key leaf features being highlighted through the application of shape metrics.

Whilst the work carried out here is only in its early stages, we believe the technique to be beneficial to the phenotyping community, with potential applications in the augmentation of commonly used imaging methods such as hyperspectral imaging with the rich surface data captured using PS. As the only limitation on the resolution of the images is the camera itself, there is scope to enhance the resolution down to near-microscopic levels, which is a further avenue we hope to explore.

Full text

3DAnalysis2DAnalysis
Calculate
Surface/Albedo
Rectify2D
(flattencurvature)
FeatureSet
CapturePSImages
Geometry
(Volume,Curvature)
SurfaceTexture
(Roughness)
Shape
(ShapeIndex,HK,
Curvedness)
Shape
(Boundaries,Orientation,
Centroid,etc.)
Texture
(GLCM,linedetection,etc)
Size
(Minor/MajorAxis,Area...)
Bristol Robotics Laboratory Centre for Machine Vision
A Low-Cost, Low-Power System for 3D Plant Imaging
Ian J. Hales, Centre for Machine Vision, Bristol Robotics Laboratory
ian.hales@brl.ac.uk
Photometric Stereo [1] for 3D
Reconstruction
Putting Together the Prototype
2D Features
We capture three or more birds-eye images of a
plant, each lit from a different, known direction...
Those images, and their light vectors, are used to
constrain the surface normal at each pixel (x,y):
where, ρ is the surface albedo (reectance map),
L the light vector for this image and N the surface
normal vector.
One of the advantages of PS is the low-cost
nature of the method - we need only one camera
and a few LEDs to build a practical system.
For this project we have endeavoured to keep
equipment costs as low as possible, so have taken
a Raspberry Pi 2 as the basis for the equipment.
By using the RPi NoIR camera, in combination
with the GPIO header, a few low-cost SSR relays
and some off-the-shelf LEDs we have been able
to produce a triggerable, congurable system
for approx. £50 total cost.
I(x, y) = ρ(x, y) (N(x, y) • L)
Because photometric stereo separates out the
2D colour and reectance from the 3D shape,
we obtain high quality, 2D images as well as the
3D reconstructions.
This means can use traditional image processing
techniques to analyse objects of interest.
In the image above, we have calculated the
albedo and found the boundary line, major and
minor lengths and orientation of the leaf.
Having calculated these, nding other metrics
such as the irregularity/dispersion, slimness,
roundness, etc. is trivial.
[1] Woodham, R. J., “Photometric method for determining surface orientation from multiple images,” Optical Engineering 19(1), 139-144 (1980).
[2] Koenderink, J. J. and van Doorn, A. J., “Surface shape and curvature scales," Image and Vision Computing 10(8), 557{564 (1992).
We can reconstruct the 3D volume, simply by
integrating over the surface normal eld. Once
we have this volume, it can be used to rectify
the 2D features as well as nding new ones that
might not otherwise be apparent.
Into 3 Dimensions...
Looking Deeper...
As PS provides us with a high-denition 3D view
of an object, we can exploit the detail to discover
underlying properties.
We can calculate the “roughness” of the surface,
by examining variation in 3D shape across a
neighbourhood for each pixel.
In the above, we see the veins of the leaf stand-
out strongly in the roughness image, as they tend
to form sharp “valleys” in the surface.
Thinking More Semantically...
These metrics are good for looking at ne-
grained details, but lack the semantic meaning
that we, as humans, apply to shape.
The images above and below represent the
“curvedness” and “shape index” [2] respectively.
These assign meaning to variations in shape
such as “Valley”, “Saddle”, Cap”, Cup”, etc.
What’s Next?
Now we’ve got these features, we’re looking
to examine how they can be used for species
differentiation, disease/pest identication and
classication and other related areas.
Ongoing work is also looking to examine how
the high-denition 3D information can augment
existing analysis techniques to correct for errors
caused by orientation differences between the
camera and the object of interest.