Content uploaded by Deric Gray
Author content
All content in this area was uploaded by Deric Gray on Dec 19, 2014
Content may be subject to copyright.
REPORT
DOCUMENTATION
PAGE
1
Form
Approved
The
public
reporting
burden
for
this collection
of
information
is
estimated
to
average
1
hour
per
response,
including
the
time
for reviewing
instructions, searching
xisting
data
sOurces
gathering
and
maintaining
the
data
needed, and
completing
and
reviewing
the
collection
of
information.
Send
comments regarding
this
burdeh
estimate
or
any
other
aspect
of
this
collection
of
information,
including
suggestions
for
reducing
the
burden,
to the Department
of
Defense,
Executive
Services
and
Communications Directorate
(0704-0188).
Respondents
should
be aware
that
notwithstanding
any
other
provision
of
law,
no
person
shall
be
subject
to
any
penalty
for failing
to
comply
with
a
collection
of
information
if
it
does
not display a
currently
valid
OMG
control
number.
PLEASE
DO
NOT
RETURN
YOUR FORM
TO
THE
ABOVE
ORGANIZATION.
1.
REPORT
DATE
(DD-MM-YYYY)
2.
REPORT
TYPE
3. DATES
COVERED
(From
-
To)
24-10-2007
Conference
Proceedings
4.
TITLE
AND
SUBTITLE
5a.
CONTRACT
NUMBER
Automated Underwater
Image
Restorations
and Retrieval
of
Related
Optical
Properties
5b.
GRANT
NUMBER
5c.
PROGRAM
ELEMENT
NUMBER
0602435N
6.
AUTHOR(S)
5d.
PROJECT
NUMBER
Weilin
Hou,
Deric
Gray,
Alan
D.
Weidemann, G.R. Fournier,
J.L.
Forand
5e.
TASK
NUMBER
5f.
WORK
UNIT
NUMBER
73-6867-07-5
7.
PERFORMING
ORGANIZATION NAME(S)
AND
ADDRESS(ES)
8. PERFORMING
ORGANIZATION
Naval
Research
Laboratory
REPORT
NUMBER
Oceanography
Division
NRL/PP/7330-07-7173
Stennis
Space
Center,
MS
39529-5004
9.
SPONSORING/MONITORING
AGENCY NAME(S)
AND
ADDRESS(ES)
10.
SPONSOR/MONITOR'S
ACRONYM(S)
Office
of
Naval
Research
ONR
800
N.
Quincy
St.
Arlington,
VA 22217-5660
11.
SPONSOR/MON
ITOR'S
REPORT
NUMBER(S)
12.
DISTRIBUTION/AVAILABILITY
STATEMENT
Approved
for public
release,
distribution
is
unlimited.
13.
SUPPLEMENTARY
NOTES
14.
ABSTRACT
The
presented
effort
is
aimed
at
establishing
a
framework
in
order
to
restore
underwater imagery
to
the
best
possible
level, working
w/
both
simulated
&
field
measured
data.
Under this
framework
the
traditional
image
restoration
approach
is
extended
by
incorporating
underwater
optical
properties
into
the
system
response function,
specifically
the
point
spread
function
in
spatial
domain
and
modulation
transfer
function
in
frequency
domain.
Due
to
the
intensity
variations
involved
in
underwater
sensing,
denoising
is
carefully
carried
out
by
wavelet decompositions.
This
is
necessary
to
explore
different
effects
of
restoration constraints,
and
especially
their
response
to
underwater
environment where
the
effects
of
scattering
can
be
easily
treated
as
either
signal
or
noise. The
images
are then
restored
using measured
or
modeled
PSFs.
An
objective
image
quality
metric,
tuned
with environmental
optical
properties
is
designed
to
gauge
the
effectiveness
of
the
restoration,
&
serves
to
check
the
optimization approach. This metric
utilized
previous
wavelet
decompositions
to
constrain
the
sharpness
metric
based
on
grayscale
slopes
to
the
edge,
weighted
by
the
ratio
of
the power of
high
frequency components
of
the image
to
the total
power
of
the
image.
Initial results
are
presented,
including
estimation
of
water
optical
properties
from
the
imagery-derived
MTFs,
and
optimization outputs applying
automated restoration
framework.
15.
SUBJECT
TERMS
ocean
optics,
scattering,
image
restoration, modulation
transfer function,
point
spread function,
NIRDD
16.
SECURITY
CLASSIFICATION
OF:
17.
LIMITATION
OF
18.
NUMBER
19a.
NAME
OF
RESPONSIBLE
PERSON
a.
REPORT
b.
ABSTRACT
c.
THIS
PAGE
ABSTRACT
OF
Weilin
Hou
PAGES
Unclassified
Unclassified
Unclassified
UL
4
19b.
TELEPHONE NUMBER
(Include
area
code)
228-688-5257
Standard
Form
298
(Rev.
8/98
Prescribed
by
ANSI
Std.
Z39.18
I
Automated underwater
image
restoration
and
retrieval
of
related
optical
properties
Weilin
Hou,
Deric
J.
Gray,
Alan
D.
Weidemann Georges
R.
Fournier,
J.
L.
Forand
Naval
Research
Laboratory, Code
7333
DRDC
-
Valcartier
Stennis
Space
Center,
MS
39529
2459
Pie
XI
Blvd,
North Quebec
Quebec,
G3J,
IX5,
Canada
Abstractd-
The presented
effort
is
aimed
at
establishing
a
the blurring
caused
by
strong
scattering
due
to
water and
its
framework
in
order
to
restore
underwater
imagery
to
the
best
constituents
which
includes
various
sized
particles.
To
properly
possible
level,
working
with
both
simulated
and
field
measured
address
this
issue,
knowledge
of
in-water
optical
properties
and
data.
Under this framework,
the traditional
image
restoration
their relationship to
the
image
formation can
be
exploited
in
approach
is
extended
by
incorporating
underwater
optical
order to restore the
imagery
to
the
best
possible
level.
This
in
properties
into
the
system
response
function,
specifically
the turn
provides
much
needed
environmental information
via
point
spread
function
(PSF)
in
spatial
domain
and
modulation
through-the-sensor
techniques
and
greatly
enhance
current
transfer
function
(MTF)
in
frequency domain.
Due
to
the
operational capabilities.
intensity
variations
involved
in
underwater
sensing,
denoising
is
carefully
carried
out
by
wavelet
decompositions.
This
is
necessary
to explore
different
effects
of restoration
constrains,
Ii.
FRAMEWORK
COMPONENTS
and
especially
their
response
to
underwater
environment
where
the
effects
of scattering
can
be
easily
treated
as
either
signal
or
A.
Image
Restoration
noise.
The
images
are
then
restored
using
measured
or
modeled
Generally speaking,
a
2-dimentional image
of
an object
is
PSFs.
An
objective
image
quality
metric,
tuned
with
basically
the
combination
of
original
signal,
f(x,y),
convolved
environmental
optical
properties,
is
designed
to
gauge
the
effectiveness
of
the
restoration,
and
serves
to
check
the
by
the
imaging
system
response
of
a
point
source,
the
point
optimization
approach.
This
metric
utilizes
previous
wavelet
spread
function or
PSF
h(x,y),
integrated over
sensor
space
Z:
decompositions
to
constrain
the sharpness
metric
based
on
grayscale
slopes
at
the
edge,
weighted
by
the
ratio of
the
power
of
g(x,y)
=
ff(xi,yi)h(x-xi,y
-
yi)dxidyj
,(!)
high frequency components
of
the
image
to
the
total
power
of
the
image.
Modeled
PSFs,
based
on
Wels'
small angle
approximations,
are
compared
to
those derived
from Monte
The
system
response includes
returns
from
both
the
Carlo
simulation using
measured
scattering properties.
Initial
imaging
system
itself,
as
well
as
the effects
of
the medium.
results
are
presented,
including estimation
of
water
optical
properties
from the
imagery-derived
MTFs,
and optimization
Mathematically,
it is
easier
to
manipulate
the
above
outputs
applying
automated restoration framework.
relationship
in
the frequency
domain
as
the
convolution
operator
becomes
a
simple multiplication. Applying a
Fourier
Keywords-
ocean
optics;
scattering; image
restoration;
transform,
the above
relationship
becomes
moduiation
transfer
function;
point
spreadfunction;
NIRDD
G(u,
v)
=
F(u,
v)H(u,
v),
(2)
1.
INTRODUCTION
where
u,
v
are
spatial frequencies
and
G,
F,
H
are
Fourier
Due to
environmental
conditions
arising
from
different
transforms
of
g,
f
and
h
respectively.
The
Fourier transfer
of
water
types
and
associated in-water
optical
properties,
the
h,
for
example,
takes
on
the following
form:
ability to generally
extend
the
performance
range
as well
as
retrieve
environmental
information
from
underwater
electro-
optical
system is
difficult.
This
capability
however
is
H(u,v)=
[
Jh(x,y)e-J2x(ux+vy)dxdy,
(3)
important
for
many
civilian
and
military
applications,f --
including
target detection (e.g.
mine
detection),
search
and
The
system
response
function
H,
also referred
to
as
the
rescue, and
diver
visibility[I].
Although traditional image
optical
transfer
function
(OTF),
is
the
Fourier
transform
of
the
enhancement techniques
can still be
used
for imagery
obtained
PSF.
The magnitude
of
the
OTF
is
the
modulation
transfer
from
underwater
environments,
without
knowledge
of
any
function
(MTF).
The MTF describes
the
contrast response
of
a
processes
involved or
the
optical
properties,
the
effectiveness
is
considerably
restrained.
The
main challenge
working
with
system
at
different
spatial
frequencies,
and
when
the
phase
underwater imagery
results
from
the
rapid
decay
of
signals
due
information
is
of
little
concern
as
is the
case
for
typical
to
absorption,
which
leads
to
poor
signal
to noise returns, and incoherent systems,
it
is
a
sufficient measure
of
the
power
transfer.
Notice
that the
above
MTF
term
H(u,v)
is
the
total
The
authors
thank NRL
for
continuous
support
through
NRL
project
73-
6867.
20071119063
system
response. Therefore
if
one views
the
complete
path
By
using
a
thin
slab
model with
the
small
angle
scattering
from
target to the
bottom
of
eyes
or
the
recording
CCD
plane,
approximation,
and assuming
a
simple
phase
function,
the MTF
is the effect
of
multiple
individual
components.
b
Because
of
the cascading
nature
of
the
MTF,
in
the
frequency
o,2
=
bO
/
(7)
domain,
it
can be
expressed
by
the
direct
product
of
each
2r(O
.
+02)3/2
component,
for
instance,
the
optical system
itself,
and
the
medium
(plus any
other
factors
when
applicable):
Wells
[4]
showed
that
the DTF
of
the
seawater
can
be
H(u,
v)
=
HSystem
(u,
v)H
.ediu
(u,
v).
(4)
expressed
as
b(1
-e-2 " °
o
)
The
above
formulation,
which
emphasizes
the
validity
of
D(V,)
=
c
-
(8)
the separation
of
the
system and
the
medium, is
important
in
2r
0/
our
analysis.
Usually
the
system
response
H,,,,.(u,v)
can
be
0
is
related
to
the
mean
square
angle
(MSA),
b
is
the
total
pre-determined and
calibrated
to
remove
any
significant
errors,
scattering
coefficient,
and
c
is
the
total
attenuation coefficient
and
in
most
cases,
does
not
vary
with
imaging conditions.
[5].
It
has
been
shown that the exact
shape
of
the
scattering
Furthermore,
one
should
pay
special
attention
to
the band-
phase
function
does
not
affect
the
derived
results
[6].
With
the
limiting
characteristics
imposed
by
H,,,
such
as
a
camera
imaging
range
defined,
the
medium
MTF
can be
obtained
from
system's
field-of-view,
and
Nyquist sampling frequency limits
(6).
imposed
by
the
CCD
resolution
[2].
From
(2),
one
can
see
with
the knowledge
of
system MTF H(p,v)
and
transformed
image
C
output
G(p,v),
the original image
can
be
theoretically
restored
age
Quality
Metric
(IQM)
by
deconvolving
the
effect
in
frequency domain
to
obtain the To
determine
the
quality
of
restored
images,
besides
unblurred
version
after
inverse
transform.
subjective
visual
comparison
which
is
prone
to significant
variations
from
different
viewers,
an
objective
quality
metric
Needless
to
say,
the
presence of
various
noises
(such
as
i
eddfrteesatrn-lre
mgs
hswsciia
scattering
or
surface fluctuations)
complicates
these through-
is
needed
for
these
scattering-blurred images.
This
was
critical
the-sensor techniques.
They
introduce
an
extra
term
in
both
(1)
for
the
development
of
an
automated
restoration
scheme,
since
and
(2).
The
medium
effect
is two
fold:
scattering
would the
computer
needs
to
"know"
which
direction
to
"go" and
contribute
extra blurring
on
top
of
system response,
while
when
to
stop,
on
small
improvement increments.
attenuation
results
in
reduced signal-to-noise
ratio. Different Our approach
is
a
wavelet-decomposed
and denoised,
image restoration approaches exist
to
reduce
and
compensate perceptual
metric
constrained
by
a
power
spectrum
ratio. More
for
the
noise
to
deblur
images,
such
as Wiener,
Lucy-
details
can
be
found
in
[3].
Briefly,
images are
first
Richardson
and blind
deconvolutions[2].
Under our
decomposed by
a
wavelet
transform to remove random
and
framework, these
approaches
arc implemented
and
exploited
to
some
medium noise. This augments
chances
of
true
edge
determine
the
best
approach working
with
underwater
images.
detection. Sharpness
of
each
edge
is
then
determined
by
In
addition
a
denoising technique
based
on
the
wavelet regression
to determine
the slope angles
between grayscale
decomposition
is applied[3].
values
of
edge
pixels
versus
location.
The
overall
sharpness
of
the
image
is the
average
of
measured
grayscale angles
(GSAs),
B.
Modeling
of
System
Response
of
Underwater
weighted
(WGSA)
by
the
ratio
of
the power
of
the
Environments
decomposition
details
to the
total
power
of
the
image.
Adaptive
determination
of
edge
widths
is
facilitated by
values associated
For
circular
symmetrical response
systems,
such
as
the
with
image
noise
variances.
To
further remove
the noise
isotropic
volume scattering
type
found
in
the
seawater,
the
contamination,
edge
widths
less than
corresponding noise
corresponding
2-dimensional
transforms
found
in
(3)
can be
variances or regression
requirements are discarded.
Without
reduced to
a
one-dimensional
Hankel
(Fourier-Bessel)
losing
generality
while
easily
expandable,
only
horizontal
edge
integral,
widths are
used
in
this
study.
H(Vt,r)=2x"
fJo(2T49)h(0,r)OdO.
(5)
D.
Framework
Summary
o=0
The implementation
of
the
framework
is
termed NRL
Wells
[4]
applied
small
angle
approximations
to
the
above
Image
Restoration
via
Denoised Deconvolution
(NIRDD).
The
and derived
a
robust
underwater modulation transfer
model flowchart
in
Fig.
I
shows
the
process involved
in the
which
is
briefly
outlined
below.
By
separating
the
exponential automated
restoration framework.
The
optimization process
is
decay
effect
with
distance due to
the
medium, the
MTF
of
the
based
on
the quality
of
restoration measured
in
WGSAs.
This
medium
in
(4)
can
be
expressed
as
uses
the
Wells'
model
to
derive
the medium
MTF
and
then
the
system PSF
with
knowledge
of
camera/lens
MTF. Both
H
meiu,,-
(v,
r)
=
eDw)r,
(6)
automated and
manual input
(measured
optical
properties)
can
be
incorporated
in
this framework.
This
framework
can
be
where
D(¢V)
is
the
decay
transfer
function (DTF)
and
is
further applied
towards real-time
image
enhancement
in the
independent
of
the range
of
detection. This
provides
a
method
field.
to
compare measurements
at
different ranges
for
consistency.
imew
(medium
+camn)
imr
1
0.7
=op
04,"- -
no ~00A
noW.dsls
Wft
PW
mw lp e
WGSA
W.SA
a2
yes 0.1
optimized
a'
results
Ows
Figure
1.
Sketch
of
the
automated
restoration framework.
Shaded
blocks
Figure
3.
Measured
PSF
via Monte Carlo
(MC)
(solid)
compared to modeled
correspond
to
the
storage
of
optimized
data
during
the
process.
results
(dotted), using
data
from
April
28,
2006 afternoon
experiment.
The
system
response
functions
(PSFs)
of
the
medium
are
Ill.
INITIAL
RESULTS
AND
DISCUSSIONS
derived
from
measurement results
of
the
volume scattering
Test
image
sets
were
obtained using
the
Laser
Underwater functions
and Monte
Carlo
simulations
[7].
Modeled
PSFs
Camera
Imaging
Enhancer or
LUCIE
from
Defense Research
using
(6)
are
compared
to
the measurements
derived.
A
and Development Canada (DRDC), during
an
April-May 2006
comparison
of
the
modeled
PSF
using
(6)
and
the in-situ
NATO
trial
experiment
in
Panama City, Florida.
The amount
measured result
is
shown
in
Fig.
3.
Note
they
are
in
relative
No
catrinand
asorintin
Panaa
cotFlda.
b
e
in
ouing
units.
The
discrepancy
amongst the
two
PSFs might
be
the
of
scattering
and
absorption
were
controlled
by
introducing
result
of
excluding
the
direct
beam contribution
in
Monte
Carlo
Maalox
and
absorption
dye
respectively. Although
the
effects
simulations,
which
inherently reduces
the
peak contribution
of
of
polarizations
are examined
during
the
experiment,
all
non-scattered photons.
The effect
of
multiple
scattering
which
images
used
in
this
study are unpolarized.
In-water
optical
is
accounted
for
in
Monte
Carlo approach
also
helps to
reduce
properties during
the
experiment
were
measured.
These
the
PSF
peak.
In
either
case,
they
are
affected
by
the
sampling
included the absorption and
attenuation
coefficients
(Wetlabs
frequency
limits
imposed
by
detectors
in
spatial
domain.
ac-9),
particle
size
distributions
(Sequoia Scientific
LISST-
100),
and
volume
scattering
functions
(multi-spectral
volume
scattering
meter
or MVSM).
Using the
framework
discussed
above,
image restoration
is
carried
out
and
medium optical
4
properties
are
estimated.
0
The measured
MTFs
of
lens and
LUCIE
camera
system
are
used
to
model
the
combined
system MTF
(Hvim
1
,)in
(4)),
which
is
shown
in
Fig.
2,
modeled
by
a
Gaussian
point
response
(R2
>0.99
in
all fits).
It
is
clear
that
the
camera
is
the
limiting
factor.
o .. ...... --- ,- - I
03
07
0
... . ..
0A.
0 1
On
& o t0 o 20 26 30 36 40
WVdtW
ft
N-m)
Figure
2.
Overall
camera
system
(lens
plus
camera)
MTF.
Meaured
camera
Figure 4.
Sample
original
(top),
restored images
based
on
measured
PSF
and
and
lens
MTF
were used
in
Guassian-typc
fits.
modeled
(bottom),
with
WGSA
values
of
0.05
and
0.14
respectively.
The
images
are
restored
using PSFs
derived from
both the
a=0.27
m",
at
low
angular
frequencies
(Fig.
5).
Clearly
the
modeled and
measured
optical properties,
and
then
quantified
above
result
can
benefit
from measurements
at
increased
by
the
image quality
metric
discussed
earlier.
A
sample
pair
is
spatial
frequencies.
Higher
dynamic ranges
will
also
help
shown
in
Fig.
4,
with
corresponding
WGSA
values
0.05
and
eliminate
probable digitization
errors
0.14
respectively.
The
visual
restoration differences between
measurement derived PSFs and modeled
PSFs
are
small
despite
the
differences
in
PSFs
(Fig.3),
thus
only
one
is
shown.
Further details
can
be
found
in
[3].
An
optimization
approach is
used
to
estimate underwater
optical
properties.
The forward
scattering
and
the mean
square
angles
are
used
for
initial
testing.
A
set
of
individual
images
0A
obtained
under
different
conditions
or
ranges
is
used. Via
the
pathway
shown
in
Table
1,
optimization
on
the image metric
is
carried
out.
Table
I
compares
the
retrieved
optical
properties
02
with
those measured
in-situ.
While
the general
trend matches
well,
significant
deviations
do
exist
(eg
0.35
versus
1.0),
and
is
0A
part
of
ongoing
investigations. Possible causes
include
M, GX
no
*
No
No
',o
deficiencies
in
denoising, the criteria
used
in
the
deconvolution
$,0d
Mew,,
algorithms,
level
of
wavelet depositions,
and the
edge
detection
algorithms.
Automated batch
processing
of
images obtained
Figure
5.
Sample
result
of
retrieved
optical
properties
from
measured
MTFs
within
the
same
time
frame
should
also improve the
retrieval,
based
on
Wells'
small angle
scattering
theory.
Top
and
bottom
curves
correspond
to
turbid
(c-0.95 m")
and
clear
(c=0.35)
conditions respectively.
TABLE
I.
ESTIMATED
OPTICAL
PROPERTIES
BY
OPTIMIZED
IMAGE
RESTORATION
VERSUS
MEASUREMENTS
IV.
SUMMARY
image
range measured
estimated estimated An
automated restoration
framework
for
underwater
ID
r
(m)
b
(m') b
(m')
MSA
imagery
is
implemented, along
with
through-the-sensor
optical
properties
retrieval.
Issues
special
to
underwater
imaging
such
25856
5.5
0.56
0.6
0.01
as
denoising and
image
quality
assessment are addressed.
The
73240
3.9
0.95 0.9
0.02
model includes
the responses
of
the camera as
well
as medium.
63402
5.1
0.95
0.7
0.02
72328
7.5
0.35
1.0
0.01
Analytical
modeling
results
compare
favorably
to Monte
Carlo
simulations
based
on
measured
in-situ
optical
properties.
Lastly
In
addition,
it
is
straightforward
to
obtain medium
optical initial
results
presented
support
the
effectiveness
of
our
properties
from
the
imagery-derived DTF, by applying
the
first
imaging
analysis framework
even
though
further
improvements
order
Taylor
expansion
to
the
exponent
for
under
Wells'
are
needed
to
improve
restoration
quality
and accuracy
of
formulation
(11),
optical property
retrievals.
D(y/
--
0)
=
-
b(1
-
e2
'
K8')
REFERENCES
2,'O0V
[1]
W.
Hou,
Z.
Lee,
and
A.
Weidemann,
"Why does
the
b(l
-1 +
2xlOyfv)
Secchi
disk disappear?
An
imaging perspective,"
Opt.
C
-
Express,
vol.
15,
March
19
2007.
2Or9
0
01
(9)
[2]
H. H.
Barrett
and
K.
J.
Myers,
Foundations
of
image
=
c
-
b
=
a
science.
Hoboken,
NJ:
Wiley-lnterscience,
2004.
[3]
W.
Hou
and
A.
Weidemann,
"Objectively
assessing
D
-
b(1
-0)
=
underwater
image
quality
for
the purpose
of
D(/
-0)
automated restoration,"
in
SPIE
Security
and
Defense
Taking
the
following
regression equation
form
following
(8)
Symposium,
Orlando,
Florida, 2007.
[4]
W.
H.
Wells,
"Theory
of
small
angle
scattering,"
D(X)
= C +
A(I-
ex)
(10)
NATO
1973.
X
[5]
C.
D.
Mobley,
Light
and
Water:
radiative
transfer
in
results
are
shown
in
Fig.
5.
The regression parameters
for
the
natural
waters.
New
York:
Academic
Press,
1994.
clearer water
are A=-33.47
and
C-0.3989.
For
the
turbid
[6]
J.
Jaffe,
"Monte
Carlo modeling
of
underwater
image
setting,
they
are
A=-44.18
and
C--0.7446.
This
approach
would
formation:
validity
of
the
linear
and small-angle
yield
c=0.40
m-
1
for the
clear
water
situation,
inline
with
the
approximations,"
Appl.
Opt.,
vol.
34,
1995.
measurement
(c=-0.35
m)'.
For
the
turbid
situation, the
[7]
D.
Gray,
W.
Hou,
A.
Weidemann,
G.
R.
Fournier,
J.
regression
yields c=0.74
m-
,
which
is
smaller compared
to
the
L.
Forand,
P.
Mathieu,
and
Y.
Rasmussen,
"Through-
measured
value
of
0.95
m-1.
For
absorption
under the
turbid
the-sensor
derived
optical properties
and
image
condition,
the regression
trend
is close
to the
field
measured
enhancement,"
in
Ocean
Optics
XV111,
2006.
