Ibn al-Haytham and the origins of computerized image analysis

Abstract

Ibn al-Haytham (Latinized as Alhazen or Alhacen) was born in Basra in 965 A.D. [354 A.H.], but produced nearly all of his work in Cairo’s al-Azhar Mosque, where he wrote nearly one hundred works on topics as diverse as poetry and politics. Al-Haytham is primarily known for his writings on geometrical optics, astronomy, and mathematics, and for nearly four hundred years his treatment of the geometry of reflection from flat and curved surfaces has been known as “Alhazen’s problem.” However, as discussed in this paper, with his landmark seven-volume Kitab al-Manazir [Book of Optics], published sometime between 1028 [418 A.H.] and 1038 [429 A.H.], al-Haytham made intellectual contributions that subsequently were incorporated throughout the core of post-Medieval Western culture. His seminal work on the human vision system initiated an unbroken chain of continuous development that connects 21st century optical scientists with the 11th century Ibn al-Haytham. The noted science historian, David Lindberg, wrote that “Alhazen was undoubtedly the most significant figure in the history of optics between antiquity and the seventeenth century.” Impressive and accurate as that characterization is, it significantly understates the impact that al-Haytham had on areas as wide-ranging as the theology, literature, art, and science of Europe.

Full text

– Invited Paper –
The 2007 International Conference on Computer Engineering & Systems (ICCES'07)
Cairo, Egypt. November 27-29, 2007
Ibn al-Haytham and the Origins of
Computerized Image Analysis
Charles M. Falco†
College of Optical Sciences
Gould-Simpson Building
University of Arizona
Tucson, AZ 85721 USA
† This paper originated from research supported by ARO grant W911NF0610359 and DARPA grant NBCH1050008.
Abstract- Ibn al-Haytham (Latinized as Alhazen or Alhacen)
was born in Basra in 965 A.D. [354 A.H.], but produced nearly
all of his work in Cairo's al-Azhar Mosque, where he wrote
nearly one hundred works on topics as diverse as poetry and
politics. Al-Haytham is primarily known for his writings on
geometrical optics, astronomy, and mathematics, and for nearly
four hundred years his treatment of the geometry of reflection
from flat and curved surfaces has been known as "Alhazen's
problem." However, as discussed in this paper, with his
landmark seven-volume Kitāb al-Manāzir [Book of Optics],
published sometime between 1028 [418 A.H.] and 1038 [429
A.H.], al-Haytham made intellectual contributions that
subsequently were incorporated throughout the core of post-
Medieval Western culture. His seminal work on the human
vision system initiated an unbroken chain of continuous
development that connects 21st century optical scientists with
the 11th century Ibn al-Haytham. The noted science historian,
David Lindberg, wrote that "Alhazen was undoubtedly the most
significant figure in the history of optics between antiquity and
the seventeenth century." Impressive and accurate as that
characterization is, it significantly understates the impact that
al-Haytham had on areas as wide-ranging as the theology,
literature, art, and science of Europe.
I. INTRODUCTION
A. Background
Recently, the artist David Hockney reported visual
discoveries within some of the best-known paintings of
European art that affect long-held understandings of the
development of Western art of the past 600 years[1]. In a
collaboration combining the expertise and visual skills of
one of the world's greatest artists[2,3] with the analytical
skills of an optical physicist, we then developed the
foundations of a new methodology for extracting
information from complex, optics-based images[4,5,6,7].
Briefly, we showed that certain features within very well-
known paintings (e.g. the chandelier in The Arnolfini
Marriage by Jan van Eyck, as shown in Fig. 1) are based on
optical projections. We determined that these optically-
based elements of the paintings are
"photorepresentations"[8]. Our discoveries show that optical
projections were being used by artists over 150 years before
Galileo brought an optical instrument, the telescope, to wide
attention.
B. Human Vision and Computerized Image Analysis
In the context of computerized image analysis, after an
image is captured by a lens-based system, subsequent
processing, including feature extraction, edge detection,
image compression, etc., maintains the original encoding
provided by the lens, as a flat field with an optically-imposed
set of vanishing points. Images of interest can now contain
over ten megapixels, with a continuing drive for even greater
resolution. However, the encoding of these images is a
fundamentally imperfect representation of human vision.
Instead, when images are presented to observers in ways that
mimic the way evolution has programmed our brains to
function[9], humans can recognize images of remarkably
low resolution.
That the mind of a painter is as intrinsically involved in
the creative process as his hand makes paintings intrinsically
complex to analyze. Although tracing projected images is
known to have become a common technique by the 19th
century [10], earlier use of optics has been difficult to
identify and analyze, hindered also by the lack of interaction
between art historians and scientists. In spite of this
difficulty, the painter David Hockney and I recently
identified optical evidence within a number of paintings
demonstrating artists as early as Jan van Eyck (c1425) used
optical projections as aids for producing portions of their
images. While making these discoveries, Hockney and I
developed fundamentally new insights into image analysis
Figure 1. Jan van Eyck, The Arnolfini Marriage, 1434 (detail
showing approximately 25% of the 81.8×59.7 cm painting). A
summary of the evidence that the chandelier in this painting is based
on optical projections is given in References [4,6].

that I am now applying to problems in computerized image
display and analysis.
Although I only briefly address it in this paper, no less
important for understanding the evolution of post-c1425
painting, as well as certain modern applications of image
analysis, is the indirect use of optics. Unlike an image
projected onto film, the human eye constantly adjusts its aim
and focus as the mind constructs the scene it is viewing. As
a consequence, humans do not simultaneously see part of a
scene in focus and part out of focus. Hence, a simple
example of the indirect use of optics is if an artist has
painted a distant portion of a scene as if it were out of focus,
replicating the depth-of-field of an image projected by a lens.
Although modern humans have seen this effect countless
times in the form of photographs, in movies, and on
television, it is not an effect that is part of natural human
vision.
C. Ibn al-Haytham and the Psychology of Vision
The fact that psychology is as intimately involved in
vision as the simple geometrical optics of the eye occupies a
significant part of Ibn al-Haytham's (Latinized as Alhazen or
Alhacen) seven-volume treatise on optics [11, 12], the first
time this topic was addressed in a modern scientific fashion.
However, as discussed below, although he explained images
of the crescent sun projected by a camera obscura in his
treatise, "The Shape of the Eclipse," this optical device did
not play a role in his understanding of vision.
II. IBN AL-HAYTHAM'S INFLUENCE ON EUROPE
A. Background
The noted science historian, David Lindberg, wrote that
"Alhazen was undoubtedly the most significant figure in the
history of optics between antiquity and the seventeenth
century"[12]. Impressive and accurate as this
characterization of Ibn al-Haytham (Alhazen) is, it
significantly understates the impact he had on areas as
diverse as the theology, literature, art, and science of Europe.
Work I am now doing on computerized image analysis can
be rightfully seen as the latest link in an unbroken chain that
connects 21st century optical scientists with our intellectual
progenitor, Ibn al-Haytham; a span of almost 1000 years.
Records indicate al-Haytham wrote nearly one hundred
works, many of which have not survived, and today he is
primarily known for his writings on geometrical optics,
astronomy, and mathematics. However, it is with his
landmark seven-volume Kitāb al-Manāzir [Book of Optics],
first published sometime between 1028 [418 A.H.] and 1038
[429 A.H.] that he made his most important contribution to
the culture as well as the science of Medieval and
Renaissance Europe.
B. Historical Theories of Vision
Al-Haytham's work Kitāb al-Manāzir [Book of Optics]
was translated into Latin in the early thirteenth century[13],
and had a profound influence on European intellectuals,
including figures as diverse as the writer Geoffrey Chaucer,
the theologian John Wyclif[14], and the scientific work on
optics of Bacon, Pecham, and Witelo[15]. Al-Haytham's
work was republished in Latin in 1572, after the advent of
the printing press, and is explicitly referenced in the writings
on optics by Kepler, Snell, and Fermat[12].
Prior to al-Haytham, theories of vision could be classified
into one of three categories: extramission, intromission, or a
combination of the two. Extramission theories required
some sort of illuminating particles be emitted by the eye.
Euclid is one well-known scholar associated with this
category of theory. Although there are obvious flaws with
extramission theories, they do get the geometry right, with a
one-to-one correspondence between points on the object and
points on the eye. To elaborate one important connection
between al-Haytham and western scholarship, Euclidian
geometry, as influenced by al-Haytham's writings, is taught
in every American and European school to this day.
Intromission theories of vision, with Aristotle as a
prominent proponent, had objects continuously sloughing off
replicas of themselves that then traveled to the eye of the
observer. These theories avoided one obvious problem of
extramission, that of near and far objects simultaneously
being visible the moment the eye is opened, but at the
expense of introducing other difficulties. Plato was a
proponent of a combination of these theories, having light
from a source like the sun, along with some sort of short-
range emission from the eye, activate the air to let replicas
travel to the viewer.
The genius of Ibn al-Haytham was not that he recognized
there were problems with all existing theories of vision,
since others before him had realized this as well. His genius
Figure 2. Ibn al-Haytham's description of the human visual system.
From a 1083 [475 A.H.] copy of his Kitāb al-Manāzir in the
Süleymaniye Library, Istanbul.

lay in the fact that he found the solution that had eluded the
best minds of antiquity. As mentioned previously, he
recognized the crucial role of psychology (or how the mind
interprets the world), and realized that to understand vision
we must understand not only the geometrical optics of the
eye, but also the psychological processes that interpret what
the eye collects.
Al-Haytham did get one important aspect of vision wrong.
The fact that an image projected by a lens is upside down
and flipped right-to-left apparently was more than he could
accept in a theory of vision, even though it is contained
within his optical formalism. However, Leonardo da Vinci
also failed to accept this when he approached the problem
much later. Five hundred years later, Kepler directly
followed al-Haytham's formalism to its inevitable and logical
conclusion in developing the theory of the retinal image. The
1572 Latin translation of al-Haytham’s, Opticae thesaurus:
Alhazeni Arabis… is explicitly referenced in the writings on
optics by Descartes and Fermat as well as Kepler.
C. Medieval European Optical Scientists
The Latin translation of the Kitāb al-Manāzir, 'De
Aspectibus', translated sometime prior to the 1230s, and the
proposals contained within it are used in the optics
manuscripts Perspectiva by Roger Bacon (c1265),
Perspectiva by Witelo (c1275), and Perspectiva communis
by John Pecham (c1280). Although today we think of these
scholars as optical scientists, they approached their work as
theologians. In each case their interest in optics was
motivated by their interest in vision, which in turn was
motivated by religious belief. In essence, they hoped that
developing an understanding of physical vision would
provide them with insights into spiritual vision. Hence, the
developments in geometrical optics that came from their
studies were actually incidental to their religious drive to
understand spiritual vision.
D. Medieval European Theologians
The onset of the Protestant Reformation is typically dated
to 1517, when the priest Martin Luther published his '95
Theses' criticizing the Christian 'Catholic' Church. Luther,
however, built directly on the efforts of the 14th century
priest, John Wyclif, who is credited with being the
intellectual progenitor of the Reformation. Like his fellow
priests of the time, Wyclif used optics in his theology. For
example, he classified spiritual vision as direct, refracted,
and reflected, and referred to al-Haytham by name in
discussing the seven deadly sins in terms of the distortions in
the seven types of mirrors analyzed in 'De Aspectibus'. As
shown in Fig. 4, Wyclif even used the Arabic word for
parabolic mirror, mukephi, in his Latin text for 'De
Eucharista', written in 1382.
E. Medieval European Literature
Turning from religion to literature, one of the most widely
read works in the French language for 300 years after its
publication in c1275 was the epic poem Roman de la Rose
[Romance of the Rose] by Guillaume de Lorris and Jean de
Meun. Four pages in this poem describe the properties of
mirrors, with the text exhibiting a surprisingly non-trivial
understanding of optics. One short passage from these four
pages makes its debt to al-Haytham (Alhazen) quite clear:
"Alhazen, the nephew of Hunain, was neither a fool nor a
simpleton, and he wrote the book of 'Optics', which
anyone who wants to know about the rainbow should
know about. The student and observer of nature must
know it and he must also know geometry, the mastery of
which is necessary for the proofs in the book of 'Optics'."
One hundred years later Geoffrey Chaucer produced his
Canterbury Tales (written over the period 1387–1400), the
first major piece of literature in the vernacular English
language. Chaucer, too, was influenced by his
understanding of the content of al-Haytham's works on
vision and optics, as is clear from the following passage:
Figure 4. Excerpt from 'De Eucharistia' by John Wyclif, 1382,
transcribed by Kourim of Bohemia in 1404–05, National Library of the
Czech Republic. I have added a black box to highlight the Arabic
word for parabolic mirror, 'mukephi'.
Figure 3. Excerpt from the first page of al-Haytham’s
Perspectiva, 14th century (Sloane MS 306, fols 1-177, British
Library, London).

Then they referred to many a learned tome
By Aristotle and by Alhazen
And Witelo and other learned men
Who when alive had written down directives
For use of cunning mirrors and perspectives,
As anyone can tell who has explored
These authors.
F. Renaissance European Art
Al-Haytham not only had a direct influence on the
development of European science, theology, and literature,
but it can also be argued that he has had at least an indirect
influence on art theory and art practice.
As Greenstein points out [16], Guerruccio di Cione
Federighian translated al-Haytham into Italian in the 14th
century, and portions of it were incorporated by Ghiberti in
Book 3 of his Commentari. In this book, which was
incomplete at the time of his death, Ghiberti attempted a
theoretical understanding of the arts, relying heavily on
optics.
Recent discoveries reveal al-Haytham’s indirect influence
on Western European art as well. The painter David
Hockney in his book Secret Knowledge [1] made fascinating
observations about some of the best-known paintings of
European art that affect long-held understandings of the
emergence of realism at the dawn of the Renaissance.
Building on these observations, Hockney and I developed
the foundations of a new methodology for extracting
information from complex, optics-based images. Briefly, we
showed that certain features within very well-known
paintings (e.g. the chandelier in The Arnolfini Marriage by
Jan van Eyck) are based on optical projections. In addition
to van Eyck (c1430), we have found evidence of the use of
optical projections within works by later artists, including
Bermejo (c1475), Holbein (c1530), Caravaggio (c1600), de
la Tour (c1650), Chardin (c1750) and Ingres (c1825). These
examples demonstrate a continuum in the use of optics by
artists from c1430, arguably initiated as a result of Ibn al-
Haytham's influence, until today.
III. SUMMARY
Ibn al-Haytham's intellectual contributions are intimately
threaded throughout the core of post-Medieval Western
culture. It is indeed unfortunate that each academic
discipline today is largely unaware of the overall scope of
his influence.
ACKNOWLEDGMENTS
I gratefully acknowledge David Hockney for the many
invaluable insights into imaging gained from him in our
collaboration that investigated paintings from over 1000
years of European art. These insights provided the
foundation for ongoing work subsequently being pursued
with David Graves, resulting in locating documents related
to the early use of optics by artists, the origin of which we
have traced to the writings of Ibn al-Haytham.
REFERENCES
[1] David Hockney, Secret Knowledge: Rediscovering the Lost Techniques
of the Old Masters, Viking Studio, 2001.
[2] Hockney, David (1937– )
"British painter, draughtsman, printmaker, photographer, and designer...
by far the best-known and most critically acclaimed British artist of his
generation."
The Oxford Dictionary of Art and Artists, Ian Childers, Ed.
Oxford University Press, 1996.
[3] "...His drawings and etchings are among the deftest of this century;
posterity may well acclaim him the greatest of modern portraitists."
The Yale Dictionary of Art and Artists, Erika Langmuir and Norbert Lynton,
Eds. Yale University Press, 2000.
[3] David Hockney and Charles M. Falco, “Optical insights into
Renaissance art,” Optics & Photonics News, vol. 11, 52, 2000.
[4] Hockney, David and Charles M. Falco, “The Art of the science of
Renaissance painting,” National Gallery of Ireland, 2004 [Proceedings of
the Symposium on ‘Effective Presentation and Interpretation in Museum’].
[5] Hockney, David and Charles M. Falco, “Optical instruments and
imaging: the use of optics by 15th century master painters,” (Proceedings of
Photonics Asia, SPIE, 5638, 1, 2005).
[6] Hockney, David and Charles M. Falco, “Quantitative analysis of
qualitative images,” (Proceeding of IS&T-SPIE Electronic Imaging, SPIE,
5666, 326, 2005).
[7] Falco, Charles M., “Analysis of qualitative images,” Proceedings of the
Irish Machine Vision and Image Processing Conference. A. Amira, A.
Bouridane, and F. Kurugollu, Eds. Queen’s University Belfast, 2005.
[8] I use the word "photorepresentation" to avoid the reader drawing the
incorrect conclusion that elements within these paintings are effectively
photographs. Even through a projected image is on the surface in front of
him, the artist, unlike a piece of photographic film, is free to use artistic
judgment to trace portions of it exactly as projected, alter other portions to
suit his taste, and ignore yet other parts of that projected image entirely.
Consequently, these paintings are not simply composites of accurate
tracings.
[9] The visual systems of primates have evolved over 30–50 million years to
the point where the human visual cortex occupies over one-third of our
brain mass. As a result, even 20,000-years ago humans were able to
produce remarkably realistic images (e.g. the cave paintings at Lascaux,
France). In contrast, cuneiform tablets from 5000 years ago show that even
by that date our mathematical abilities had yet to advance beyond counting.
[10] W Douglass Paschall et al., Thomas Eakins, exhibition catalogue.
Philadelphia, PA: Philadelphia Museum of Art, 2001.
[11] al-Haytham, Abū 'Alī al-Hasan ibn al-Hasan ibn (Latinized as Alhazen
or Alhacen). Kitāb al-Manāzir [Book of Optics] (Cairo, c1028–38).
[12] Lindberg, David C. Theories of Vision from Al-Kindi to Keppler.
University of Chicago Press, 1976.
[13] Smith, A. Mark. Alhacen's Theory of Visual Perception (American
Philosophical Society, 2001). Smith provides an insightful analysis of the
22 existing Latin manuscripts of the first three books of al-Haytham's text.
It was in Latin, with translation interpretations, omissions and errors, that
the text was studied by Europeans.
[14] al-Haytham and his optics are described in the c1275 epic poem
Romance of the Rose, which for the next 300 years was one of the most
widely-read works in the French language, and in Chaucer's c1400
Canterbury Tales, the first major piece of literature in the vernacular
English language. The 14thC theologian John Wyclif, whose writings were
an important precursor to the Protestant Reformation, used al-Haytham's

writings on physical vision to help explain his own doctrine of spiritual
vision.
[15] Sabra, A. I., "Ibn al-Haytham," in, Dictionary of Scientific Biography.
Charles C. Gillispie, Eds. Charles Scribner's Sons, 1972.
[16] Greenstein, Jack M. “On Alberti’s ‘sign’: vision and composition in
quattrocento painting,” The Art Bulletin, vol. 79, 4, December 1997.