Advances in biomedical image analysis--past, present and future challenges.

Abstract

Starting from raw data files coding eight bits of gray values per image pixel and identified with no more than eight characters to refer to the patient, the study, and technical parameters of the imaging modality, biomedical imaging has undergone manifold and rapid developments. Today, rather complex protocols such as Digital Imaging and Communications in Medicine (DICOM) are used to handle medical images. Most restrictions to image formation, visualization, storage and transfer have basically been solved and image interpretation now sets the focus of research. Currently, a method-driven modeling approach dominates the field of biomedical image processing, as algorithms for registration, segmentation, classification and measurements are developed on a methodological level. However, a further metamorphosis of paradigms has already started. The future of medical image processing is seen in task-oriented solutions integrated into diagnosis, intervention planning, therapy and follow-up studies. This alteration of paradigms is also reflected in the literature. As German activities are strongly tied to the international research, this change of paradigm is demonstrated by selected papers from the German annual workshop on medical image processing collected in this special issue.

Full text

Page 1

© 2004Schattauer GmbH
308
Methods Inf Med 4/2004
Editorial Advances in Biomedical Image Analysis
Past, Present and Future Challenges
T. M. Lehmann1, H. P. Meinzer2, T. Tolxdorff3
1Department of Medical Informatics, Aachen University of Technology (RWTH), Aachen, Germany
2Department of Medical and Biological Informatics, German Cancer Research Center (DKFZ), Heidelberg, Germany
3Institute for Medical Informatics, Biostatistics and Epidemiology, Charité – Universitätsmedizin Berlin,
Campus Benjamin Franklin, Berlin, Germany
1. Introduction
Biomedical image processing is an impor-
tant and continuously growing field of
research in medical informatics. Digital
imaging modalities and the steady increase
of computational power at decreasing costs
enable sophisticated computer assistance in
basic biomedical research,diagnosis,thera-
py, and control of patient treatment. In
1998, several German local activities were
united by a joint workshop on algorithms,
systems, and applications of medical image
processing in Aachen.Since then,the work-
shop is held annually at different locations
in Germany.The last meeting in 2004 took
place in Berlin and was attended by more
than 250 scientists from research, develop-
ment, and application with contributions
from Germany and several other countries.
The papers,poster presentations and indus-
trial showcases present novel techniques
and applications from image acquisition,
analysis,and understanding.
The continuous growth of German ac-
tivities correlates with the international
trends.For instance,the annual symposium
on medical imaging, which is sponsored by
the International Society for Optical Engi-
neering (SPIE), rises both the number of
oral and scientific poster presentations
(Fig. 1) as well as the number of attendees.
For instance in 1984, fewer than 25 papers
were presented on image processing and
analysis algorithms within the conference
that was entitled “Application of Optical
Instrumentation in Medicine” [6]. By 1994
and 2004, there were 88 and 232 papers in
the conference track “Image Processing”
that has been established in 1989, resulting
in a large extention to this track among the
Summary
Starting from raw data files coding eight bits of gray
values per image pixel and identified with no more
than eight characters to refer to the patient, the study,
and technical parameters of the imaging modality,
biomedical imaging has undergone manifold and rapid
developments. Today, rather complex protocols such
as Digital Imaging and Communications in Medicine
(DICOM) are used to handle medical images. Most
restrictions to image formation, visualization, storage
and transfer have basically been solved and image
interpretation now sets the focus of research. Currently,
a method-driven modeling approach dominates the
field of biomedical image processing, as algorithms for
registration, segmentation, classification and measure-
ments are developed on a methodological level.
However, a further metamorphosis of paradigms has
already started. The future of medical image process-
ing is seen in task-oriented solutions integrated into
diagnosis, intervention planning, therapy and follow-
up studies. This alteration of paradigms is also reflect-
ed in the literature. As German activities are strongly
tied to the international research, this change of para-
digm is demonstrated by selected papers from the
German annual workshop on medical image process-
ing collected in this special issue.
Keywords
Computer-assisted image processing, computer-
assisted image interpretation, automatic data
processing, medical informatics applications,
pattern recognition, three-dimensional imaging
Methods Inf Med 2004; 43: 308–14
other conference tracks (Fig. 1). In addi-
tion,a substantial part of the other tracks is
also about image processing technologies
or algorithms. The same tendency is ob-
served for the International Symposium on
Biomedical Imaging (ISBI), which was
founded in 2002 by the Institute of Electri-
cal and Electronics Engineers (IEEE).
However, those figures do not answer the
question which major progress has been
made in the field of biomedical image
analysis and which challenges still need to
be addressed in future work.
Duncan and Ayache have recently de-
scribed the progress of medical image anal-
ysis over two decades [3].They termed pre-
1980 to 1984 the era of two-dimensio-
nal (2D) image analysis, 1985-1991, when
knowledge-based strategies came to the
forefront, 1992-1998, when the analysis of
fully three-dimensional (3D) images be-
came the key goal, and 1999 and beyond,
when advanced imaging and computing
technology facilitate work in image-guided
procedures and more realistic visualization.
But what exactly is meant by advanced
imaging and computing technology? Ac-
cording to Talmon and Hasman, medical
informatics is, in general, a modeling disci-
pline that aims at designing models for
medical processes in order to implement
them in computer systems [13]. Making
these models applicable for daily medical
practice is the applied science or engineer-
ing side of medical informatics. Hasman et
al. have categorized these processes into
biological, communication, decision, engi-
neering, educational, organizational, and
computational processes [7]. Obviously,
biomedical image analysis mainly address-
es biological processes, decision making,

Page 2

and computational aspects.From this point
of view, it is important whether one can
identify the particular processes behind
advanced biomedical imaging.
In this paper, we will try to term such
processes that are assisted by applying
medical image processing techniques. We
will scrutinize the past and present of image
processing and analysis with respect to any
alterations of the paradigms underlying the
modeling process of medical imaging infor-
matics.Based on this appraisal,we will pre-
dict future directions of biomedical image
processing, where the papers contained in
this special issue are dedicated to exem-
plarily emphasize current research and
future developments in the broad field of
medical imaging informatics.
2. Past – Image Acquisition,
Display, and Management
In the following,we will draw a scenario that
could have been at any institution pioneer-
ing biomedical image processing.In the ear-
ly years of research, image generation and
handling was extremely difficult, since the
computers were of less than a thousandth of
today’s computing and storage capacity.
Therefore,reading out the data from the im-
aging device and computing a medical image
that displays valuable information was a
major challenge. Consequently, algorithmic
improvements addressed the speed rather
than the quality or accuracy of processing.
2.1 Image Formation (Digital)
In the beginning, there was a raw data file
carrying a name which was not allowed to be
composed of more than eight characters and
thus degenerated to a cryptic abbreviation
concocted from a patient’s identification,an
examination identifier, a slice number and
all other pertinent information. It consisted
of an unformatted bit-stream that had to be
deciphered.If the decoding did not result in
at least a halfway decent morphological
structure as compared to those displayed
in the anatomic textbook, attempts were
continued with the same suspense that arises
when decoding a secret service message.The
books of Pratt [11], Gonzales and Wintz [4],
which has later turned to Gonzales and
Woods [5], as well as the publications of
Rosenfeld and Kak [12] were piled up
between the terminal’s keyboard and the
impulse-dialing telephone, while one pon-
dered whether the data to be processed
had its origin at the top or bottom left or
whether it consisted of an eight or twelve-bit
depth.If necessary,a phone call was made to
check, but this effort was doomed to failure
with most of the modality producers or
vendors.When this problem was solved, the
look-up tables were programmed manually.
Prominent problems were raised from
simple questions such as how to segment
256 codes into color and gray values or
which role should color play:Should it refer
to the function of tissue or should it clarify
morphological information?
2.2 Print and Display
The one and only mainframe of the clinical
center was extended by some exotic hard-
ware:A loud roaring cabinet now stood in
the computer room, and each delegation
passing by was given the proud explana-
tion: this is a digital image server with the
enormous capacity of one megabyte. It
can even handle color images, which can
be viewed only on a television monitor
that – due to the noise – was placed in a
separated room and connected by four
coaxial cables.
The system halted regularly about every
30 minutes, which required a manual reset
by the operator to continue image process-
ing.Only if one succeeded in distracting the
operator’s attention, the images could be
printed out. With the help of the magical
symbol “1H+”, the line feed of the type-
writer printer was temporarily deactivated
and gray values were simulated by skillfully
printing characters on top of each other.
Frequently, the operators had become
alerted to this acoustic abnormality and
stopped the printing process before the
continuous paper was cut off.
2.3 Pixel Data Manipulation
Those were the days when each image ma-
nipulation had to be painstakingly pro-
Editorial
309
Methods Inf Med 4/2004
Fig. 1
denote the relations between the symposium tracks.
Number of presentations at SPIE’s Medical Imaging according to the conference tracks. The colors are selected to

Page 3

grammed beforehand. The rank growth of
minor private subroutine libraries sprouted
tremendously. Laplace operators, Hough
transformers,Prewitt operators were devel-
oped with a great deal of effort and then
personally tested with as much effort and
diligence.A Fourier transform took so long
that it left you time enough to slowly work
your way through the next chapter of Gon-
zales and Wintz before having to face the
results of your art of programming. It was
an enormous improvement when one of
the first parser generators appeared on the
scene – a generator which allowed a com-
fortable, command-orientated control of
image-processing libraries.It is now hard to
imagine working with processing power of
less than one megahertz or even with non-
quadratic pixels.
2.4 Storage and Transfer
In these early years, standards were non-
existent [9]. This word was not only “not
mentioned”, it was unknown and unex-
plored.The first experiments with databas-
es were made, since the cryptic abbrevia-
tions of patients’ names caused enormous
problems in relocating entire series of im-
ages – even on a hard disk on only 10 MB of
capacity. Of course, there was no integra-
tion of other systems like hospital informa-
tion systems (HIS). Although radiological
information systems (RIS) and picture
archiving and communication systems
(PACS) were already postulated,their end-
less development had not yet begun.
If images were needed for publications,
the light in the lab was turned off, and the
monitor was photographed with a camera
and tripod. When a paper was finally fin-
ished, the color prints were stuck in manu-
ally. Also, the paper had to be mailed at
least ten days before the deadline in order
to arrive in the United States on time.
2.5 Acceleration
If you would have asked the physician
when he would like to receive his image via
the electronic network at his workstation,
he probably would have answered: within
one or two days would be great. However,
this faced remarkable problems to the com-
puter scientist, since he was required to
stray through long corridors at night in
order to press a reset button just to be able
to work undisturbed again for a couple of
minutes. Fortunately, these days have
passed and the underlying infrastructure-
oriented paradigms of image formation,
pixel data manipulation,visualization,stor-
age, and transfer have basically been dis-
solved. Today, the answer of the physician
will be: one or two seconds, which is pos-
sible for many algorithms and procedures
using modern hard- and software.
3. Present – Image Analysis
In the mid-eighties, a first change in the
paradigms of medical image processing
was notable [3, 6]. Digital generation and
handling of medical imagery was extended
to image analysis.Therefore,a priori knowl-
edge had to be integrated into the algo-
rithms that were designed for medical im-
age processing. Consequently, the process-
es modeled by medical imaging informatics
transformed from a structure-driven into a
process-driven scheme (Fig. 2). According
to Hanson, the acceptance of computed
tomography (CT) led to the question: Are
there other ways in which computers could
help radiologists interpret medical images?
[6].With respect to the SPIE’s Internation-
al Symposium on Medical Imaging, the
track “Image Processing” was established
in 1989 to answer this question.Exemplify-
ing prominent categories in this track, de-
formable models are mentioned that have
become prevalent in many areas of image
analysis.Note that such a method-centered
view reflects the process-driven paradigm
presently being dominant in medical image
procession.
This process-driven scheme is of techni-
cal nature.It follows a stepwise sequence of
image data transforms rather than a pa-
tient-centered application of image analy-
sis.Still,image formation is the first module
of this technical processing scheme, which
is followed by registration of pixel data,
segmentation and classification of relevant
objects, as well as quantitative measure-
ments in order to obtain additional infor-
mation supporting the physicians decision
of diagnostics or intervention planning
(Fig. 2). Up to now, this view on medical
imaging informatics is most prominent and
the majority of recently published papers
fits into the scheme.
3.1 Image Formation (Functional)
Since the goal of today’s image analysis is
image interpretation and quantitative
measurements, functional imaging has be-
come the key challenge of biomedical im-
age formation. Functional magnetic reso-
nance imaging (fMRI) is the most popular
example. However, MRI palpation or
ECC-triggered MRI volumetry of the beat-
ing heart are further examples. In general,
today’s technology optimizes the resolution
of established techniques or adds di-
mensionality.As postulated by Duncan and
Ayache, fully 3D images are currently the
goal [3]. Consequently, novel techniques
such as tuned aperture computed tomogra-
phy (TACT®) or limited angle tomography
reconstruct 3D volumes from an incom-
plete number of projections [Linnenbrüg-
ger et al.: Automated Hybrid TACT®
Volume Reconstructions; Weber et al.: A
Linear Programming Approach to Limited
Angle 3D Reconstruction from DSA
Projection (this issue)].The knowledge re-
quired for the reconstruction process is ob-
tained from other modalities, e.g., photog-
raphy in hybrid TACT, or application-spe-
cific assumptions,i.e.modeling the imaging
process with certain respect to the context
of the images.
3.2 Registration
Frequently, one cannot combine different
modalities prior to image acquisition and
formation. Nonetheless, improvement of
measurement accuracy or resolution is
obtained from multimodal combinations.
Consequently,registration of pixel data is a
key module in many image processing tech-
niques in medicine [Fischer and Moder-
sitzki: Intensity-based Image Registra-
tion with a Guaranteed One-to-one Point
Editorial
310
Methods Inf Med 4/2004

Page 4

Match (this issue)]. Unimodal registration
is required for follow-up studies and the
reliable detection of tissue alterations.
Multimodal registration enables the trans-
formation of the high resolution of mor-
phological imaging to functional images,
which are usually acquired with significant
lower resolution. Especially in neuroimag-
ing,digital atlas processing is based on pow-
erful and robust registration techniques. It
enables the transfer of a priori knowledge
on important brain areas onto the structure
of a certain patient’s brain, which might
significantly differ in morphology.
3.3 Segmentation
With respect to the process-driven para-
digm of current image processing in medi-
cine,segmentation is the most essential part
of the processing pipeline. Segmentation
means localization and delineation of rele-
vant structures such as important objects.
While humans can usually perform the
localization part of image segmentation
superior to computer algorithms,the delin-
eation part is the typical domain of biomed-
ical image processing and analysis. Com-
puter-based segmentation yields detailed
and reproducible shapes and/or surfaces
of relevant objects [Bartz et al.: Accurate
Volumetric Measurements of Anatomical
Cavities (this issue)].Again,a considerable
amount of a priori knowledge is modeled
on a rather high level of abstraction.Conse-
quently, automatic segmentation algo-
rithms are specialized to a certain appli-
cation such as the optical disk in retinal
images [Chrástek et al.: Multimodal Reti-
nal Image Registration for Optic Disk
Segmentation (this issue)] or overlapping
cells in digital microscopies [Wittenberg et
al.:A Semantic Approach to Segmentation
of Overlapping Objects (this issue)]. They
are inapplicable to other objects of interest
or other imaging modalities without rede-
signing the procedure,because an adequate
formulation of a priori knowledge on the
new content is required.
3.4 Classification and Measurement
Once the relevant objects have been identi-
fied within the image,image analysis aims at
providing quantitative measurements to the
physicians. According to the processing
scheme that models the technical steps,
features are extracted from the segments
in order to classify them. Usually, these
features reflect the biomedical application.
However, the increasing amount of routine-
ly acquired digital images opens a novel field
of research,where a general classification of
images is the central focus. In particular,
content-based image retrieval in medical
applications requires automatic classifica-
tion of pixel data with respect to the imaging
modality and technical parameters used,the
relative orientation of the imaging device
and the patient, the body region examined,
and the biomedical system under investiga-
tions [Lehmann et al.:Content-based Image
Retrieval in Medical Applications (this
issue)]. Here, the context is unknown pro-
spectively and must be determined automati-
cally in order to optimize successive mod-
ules of the image processing pipeline.
3.5 Evaluation
Although enormous efforts in biomedical
image processing have been made during
the last 15 years, sophisticated algorithms
for registration, segmentation, classifica-
tion, and measurement are infrequently
applied in daily routine of health care.
Obviously, there are two reasons. Firstly,
the process-driven scheme mismatches a
patient-centered application. This fact is
already acknowledged since science and
technology currently changes the general
paradigm of modeling image processing
(see next section).Secondly,the algorithms
simply fail when applied in clinical routine
since the data acquired routinely differs
from those references that were available
during development. Some assumptions
that are inherently used by the algorithms
do not hold for routine data. In other
words, the algorithms have not been suffi-
ciently evaluated for routine use. Conse-
quently, systematic evaluation of medical
image processing techniques is in the fo-
cus of current research [Krüger et al.:
Evaluation of Computer-assisted Image
Editorial
311
Methods Inf Med 4/2004
Fig. 2
The paradigms of medical image processing during the past, at present, and in the future

Page 5

Enhancement in Minimal Invasive Endo-
scopic Surgery; Uhlemann et al.: Objective
Evaluation of Three-dimensional Image
Registration Algorithms.Tools for Optimi-
zation and Evaluation (this issue)].
Over a period of 20 years, the belief in
image processing was that if a result com-
puted on the Lena Image is improved, the
novel algorithm is generally superior to
existing processing schemes. We know to-
day that a sufficient number of reference
data is required for exhaustive evaluation
and statistical analyses need to be applied.
We also have learned that the ground truth
plays an important role for reliable evalua-
tion and that manual references such as
object outlines drawn by a medical expert
are itself unreliable and, therefore, inappli-
cable for reliable evaluation.Consequently,
the generation of valid references and
ground truth is covered by current research
[Uhlemann et al.: Objective Evaluation of
Three-dimensional Image Registration
Algorithms – Tools for Optimization and
Evaluation (this issue)].
4. Future – Routine Application
Up to now, the process-driven view on
medical imaging informatics is dominant
and the majority of recently published
papers match this scheme. Nonetheless, a
second metamorphosis of paradigms al-
ready began a few years ago. In the future,
the patient will be at the center of biomed-
ical image processing and novel algorithms
are primarily designed to fit into routine
application.Integration and routine use are
major challenges of further developments
in a result-driven view of biomedical image
analysis [1].
Therefore,medical imaging will increase
its impact in the next decade,but it will be-
come less apparent than it is now. Classic
radiology still deals with 2D film images.
The end of these times already has come as
the new multi-slice CTs are making much
too many images per patient to be printed
and analyzed in the old fashion way.There-
fore, new challenges arise from handling
and processing the increased data volume.
Also,all effort of the medical imaging com-
munity will be in vain if our support is not
fully integrated into the clinical workflow.
The medical users are not interested in
technical problems and procedures. They
demand help in diagnoses and therapy
planning. Only systems that increase pro-
ductivity and safety while reducing costs
will be accepted.Everything must be easily
accessible, requiring no training time and
being used by intuition. This result-driven
approach forms the underlying paradigm of
future image analysis (Fig.2).
4.1 Image Formation
(Multidimensional)
Continuous image formation is an essen-
tial part of this new paradigm. The times
of 2D images are over. The availability of
serial multi-slice data represents de facto
volume data (3D) and if taken at a number
of times, four-dimensional (4D) data is
obtained. Perhaps it is clearer to refer to
these data as 3D-plus-time data (3D+t).
Either the space changes over time,e.g.if a
patient is scanned several times for follow-
up controls, or the object in the space is
moving, e.g. a beating heart or a moving
diaphragm due to breathing.
The availability of these 3D and 3D+t
data will change the focus in medical imag-
ing from 2D image processing to 3D and 4D
image processing [König and Hesser: Live-
wires Using Path-graphs (this issue)]. Par-
allel to this we will observe an increasing
resolution in space and time.This will permit
us to present ever increasing details which
will improve even more the relevance of
image guided diagnostics and therapy.
4.2 Diagnosis
The old and well known software algo-
rithms will be extended to the additional
dimension.This will demand higher compu-
tational power and more memory. The de-
velopment of the hardware is so fast that
we do not foresee upcoming shortcomings.
The hope of the past that there will be some
kind of automated understanding (i.e.diag-
noses) collapsed due to our failing under-
standing of human perception in general.
Thus, there will be some ‘intelligent’ semi-
automatic (i.e. interactive) procedures, in-
cluding the medical doctors, who by this
will keep their responsibility and control of
the process [Hahn et al.: A Reliable and
Efficient Method for Cerebral Ventricular
Volumetry in Pediatric Neuroimaging;Wa-
genknecht et al.:MRI-based Individual 3D
Region-of-interest Atlases of the Human
Brain.A New Method for Analyzing Func-
tional Data (this issue)]. For instance in
traumatology,there will be devices that can
show 3D/3D+t representations of a patient
within minutes after the patient left a fast
spiral multi-slice CT, putting all interesting
features to the fingertips of the medical
doctors (e.g. F1 is skeleton, F2 is respirato-
ry system,etc.)
4.3 Intervention Planning
Although remarkable efforts have already
been made,the development of detailed at-
lases and powerful mapping tools are still
needed to allow one to locate morphologi-
cal landmarks or functional areas in indi-
vidual patient data are still needed. Com-
puter-assisted intervention planning will be
based on those meta-data in many medical
fields. In future, atlases will not only be fo-
cused on hard tissue to assist orthopedic
surgery [Ehrhardt et al.: Atlas-based Rec-
ognition of Anatomical Structures and
Landmarks and the Automatic Computa-
tion of Orthopedic Parameters (this issue)]
but also on soft tissues such as the brain or
other organs. Furthermore, powerful visu-
alization will strengthen the planning of
micro-invasive surgery [Weichert et al.:
Registration of Biplane Angiography and
Intravascular Ultrasound for 3D Vessel
Reconstruction (this issue)].
4.4 Therapy
The logical next step in the developmental
process is the extension from diagnostic
support and therapy planning to the sup-
port of therapy.One example is the control
of ‘smart’ handheld instruments for cutting,
drilling, ablation etc. [Vogt et al.: Light
Editorial
312
Methods Inf Med 4/2004