Reconstruction of Image Structure
in Presence of Specular Reflections
Martin Gr¨ oger, Wolfgang Sepp, Tobias Ortmaier, and Gerd Hirzinger
German Aerospace Center (DLR)
Institute of Robotics and Mechatronics
D-82234 Wessling, Germany
Abstract. This paper deals with the reconstruction of original image
structure in the presence of local disturbances such as specular reflec-
tions. It presents two novel schemes for their elimination with respect
to the local image structure: an efficient linear interpolation scheme and
an iterative filling-in approach employing anisotropic diffusion. The al-
gorithms are evaluated on images of the heart surface and are suited to
support tracking of natural landmarks on the beating heart.
Glossy surfaces give rise to specular reflection from light sources. Without proper
identification specularities are often mistaken for genuine surface markings by
computer vision applications such as matching models to objects, deriving mo-
tion fields from optical flow or estimating depth from binocular stereo [BB88].
This paper presents two approaches to reconstruct the original image struc-
ture in the presence of local disturbances. The algorithms have been developed
to enable robust tracking of natural landmarks on the heart surface [GOSH01]
as part of the visual servoing component in a minimally invasive robotic surgery
scenario [ORS+01]. There specular reflections of the point light source arise on
the curved and deforming surface of the beating heart. Due to sudden and ir-
regular occurrence these highlights disturb tracking of natural landmarks on
the beating heart considerably [Gr¨ o00]. Reconstruction schemes are sufficiently
general for application in other fields, where disturbances in images should be
eliminated ensuring continuity of local structures.
Previous work mainly investigates specular, together with diffuse, reflection
[BB88, Wol94], which aim to suppress the specular component while enhancing
the diffuse. This work considers local specular reflections with no detectable dif-
fuse components, which cause total loss of information. Therefore reconstruction
can only be guided by surrounding image structures.
The following section introduces robust extraction of image structure by the
structure tensor which two schemes for reconstruction are based on: linear in-
terpolation between boundary pixels and anisotropic diffusion within a filling-in
scheme. The algorithms are evaluated on video images of the heart with specu-
larities (Sect. 3), before concluding with a summary of results and perspectives.
Proc. DAGM 2001, LNCS 2191 c ? 2001 Springer53
The algorithms have been successfully applied to reconstruct the heart sur-
face in the presence of specular reflections. Further, tracking natural landmarks
on the beating heart can be improved greatly, as outliers caused by specularities
are compensated [Gr¨ o00]. Apart from tracking the algorithms are sufficiently
general to reconstruct structured images partially occluded by disturbances.
This work was supported partly by the German Research Foundation (DFG)
in Collaborative Research Centre SFB 453 on “High-Fidelity Telepresence and
Teleaction”. The authors thank Dr. D.H. B¨ ohm from the Department of Car-
diac Surgery at the University Hospital Grosshadern for video sequences of the
[BB88]Gavin Brelstaff and Andrew Blake. Detecting specular reflections using
lambertian constraints. In Proc. 2nd Int’l Conf. on Computer Vision
(ICCV), pages 297–302. IEEE, 1988.
B. Fischl and E.L. Schwarz. Learning an intergral equation approximation
to nonlinear anisotropic diffusion in image processing. Technical Report
CAS/CNS-95-033, Boston Univ. Center for Adaptive Systems, Dec. 1995.
M. Gr¨ oger, T. Ortmaier, W. Sepp, and G. Hirzinger. Tracking local motion
on the beating heart. In FIMH 2001: Int’l Workshop on Functional Imag-
ing and Modeling of the Heart, Helsinki, Finland, Nov. 2001. submitted.
Martin Gr¨ oger. Robust tracking of natural landmarks of the beating heart.
Technical report, German Aerospace Centre (DLR), Institute of Robotics
and Mechatronics, 2000.
Heiko Neumann and Luiz Pessoa. Visual filling-in and surface property
reconstruction. Technical Report 98-04, Universit¨ at Ulm, Fakult¨ at f¨ ur In-
formatik, Feb. 1998.
T. Ortmaier, D. Reintsema, U. Seibold, U. Hagn, and G. Hirzinger. The
DLR minimally invasive robotics surgery scenario.
J. Hoogen, editors, Proceedings of the Workshop on Advances in Interac-
tive Multimodal Telepresence Systems, pages 135–147, Munich, Germany,
William H. Press, Saul A. Teukolsky, William T. Vetterling, and Brian P.
Flannery. Numerical Recipes in C: The Art of Scientific Computing. Cam-
bridge University Press, 2nd edition, 1993.
Wolfgang Sepp and Heiko Neumann. A multi-resolution filling-in model
for brightness perception. In ICANN99: Ninth International Conference on
Artificial Neural Networks, volume 1, pages 461–466. IEE, London, 1999.
Andrey Tikhonov and Vasily Arsenin.
V. H. Winston & Sons, Washington, D.C., 1977.
Joachim Weickert. Anisotropic Diffusion in Image Processing. B.G. Teub-
ner, Stuttgart, 1998.
Laurence B. Wolf. On the relative brightness of specular and diffuse reflec-
tion. In Proceedings Conference Computer Vision and Pattern Recognition,
pages 369–376. IEEE, 1994.
In G. F¨ arber and
[TA77]Solution of Ill-Posed Problems.
60 Proc. DAGM 2001, LNCS 2191 c ? 2001 Springer