Int J CARS
An interactive 3D framework for anatomical education
Pere-Pau Vázquez · Timo Götzelmann ·
Knut Hartmann · Andreas Nürnberger
Received: 10 January 2008 / Accepted: 17 June 2008
© CARS 2008
Object This paper presents a 3D framework for Anatomy
teaching. We are mainly concerned with the proper unders-
tanding of human anatomical 3D structures.
Materials and methods The main idea of our approach is
taking an electronic book such as Henry Gray’s Anatomy of
constructing the correct linking that allows users to perform
mutual searches between both media.
Results We implemented a system where learners can inter-
actively explore textual descriptions and 3D visualizations.
Conclusion Our approach allows easily performing two
search tasks: first, the user may select a text region and get a
and second, using the interactive exploration of a 3D model
P.-P. Vázquez (B )
Modeling, Visualization and Graphics Interaction Group,
Universitat Politècnica de Catalunya,
Campus Nord, Edifici Omega. C/ Jordi Girona, 1-3,
08034 Barcelona, Spain
Graphics and Interactive Systems Group,
University of Magdeburg, Magdeburg, Germany
Graphics and Interactive Systems Group,
University of Applied Sciences, Flenshburg, Germany
Information Retrieval Group, University of Magdeburg,
the user may automatically search for the textual description
of the structures visible in the current view.
Medical educational materials (W 18.2) ·
Computer graphics (T 385)
Computer-assisted instruction (LB 1028.5) ·
There are many fields in medicine where understanding 3D
spatial relationships is very important. Text books usually
provide information in the form of illustrations and text,
because both of them provide complementary information:
while complicated processes and contextual information are
table to provide information about visual and spatial attri-
number of illustrations is very costly, and usually, illustra-
tors try to minimize the number of pictures by including a
lot of information into each of them. This makes the process
of understanding actual 3D positions of the structures quite
difficult. On the other hand, the simple exploration of a 3D
being observed. Our proposal tries to combine the strengths
of both media: text, and 3D models. On the one hand, we
have a textual description of the anatomy for certain parts of
the body, and on the other we have a 3D model representing
that part. In order to facilitate the exploration and to aid the
model and the text through the use of labels. Our application
allows querying the 3D model through the selection of a cer-
view of the 3D model. As a result, when the student is rea-
ding the text book, the selection of a word or paragraph that
Int J CARS
Fig. 1 Overview of the
contains elements he or she finds interesting leads to auto-
matically loading the corresponding 3D model that contains
the structures of interest, which can be then inspected inter-
actively. Conversely, a 3D view of a model may be chosen
by the user, and, under request, the system looks up the para-
graphs where the structures visible from the selected view
are described. This paper is an extension of papers  and
be interesting for learning Anatomy concepts.
Figure 1 shows an overview of the application, which has
with the use of the mouse. In Results, we give further details
on the architecture of the application and the different ways
to use it.
Illustrations and labels
The key issue of our approach is the use of secondary ele-
support two search tasks: readers may use them in order to
links to more elaborate descriptions for visual objects in cor-
responding text passages (see Fig. 2). To focus the attention
of the user on salient objects, experienced illustrators also
utilize many graphical abstraction techniques.
The creation of expressive illustrations that carefully
reflect the subject matter of the corresponding text is very
expensive. In order to limit the printing costs, illustrators
aim at minimizing the number of illustrations. Nonetheless,
this often results in a high number of labels to be inserted.
But the limited cognitive capacity makes both search tasks
mentioned in the last paragraph more complicated. Further-
more, learners often have to understand complicated spatial
offer new possibilities to support learners: efficient retrieval
mechanisms ease the access to sections containing informa-
interactive 3D visualizations support the mental reconstruc-
tion of complex spatial configurations. Moreover, a dynamic
layout of secondary elements can make illustrations more
This paper proposes a novel technique to coordinate the
content of visual and textual elements in online tutoring sys-
tems. As the majority of the application domains define a
standard terminology, unified retrieval methods are applied
to a large text corpus and annotated 3D models. User inter-
actions on both media—text and graphics—initiate queries
to an information retrieval system; the retrieval results are
searchtasksoftext ↔ imagerelations.Userscanselecttext
passages to obtain corresponding renditions of 3D models.
are considered to determine good views on 3D models. As a
single view might not sufficiently correspond to the content
of text paragraphs, these 3D visualizations can be explored
interactively. User selected points of 3D views are conside-
Int J CARS
Fig. 2 Media coordination in
tutoring material: the most
relevant objects for a learning
task are emphasized in
descriptive texts and illustrations
(Source: Tortora )
text corpus are automatically obtained and presented to the
This section presents related work on text illustration and
viewpoint selection for medical models. Finally, we present
our 3D framework for anatomy learning.
The automatic generation of technical documentation with
coordinated content in several media (e.g., text, images, and
animation) was pioneered by the COMET  and WIP sys-
tems . These research prototypes aim at adopting the
content of online documents to user specific tasks, prefe-
tilingual variants should be generated from a single source.
Therefore, a multitude of planning mechanisms is used in
tations, as well as the need to approve all document variants
according to legal aspects, prevented the application of this
Based on the observation that authors of scientific text-
books, technical documentations, or maintenance manuals
reuse and improve large text databases with a standardi-
zed domain-specific terminology, several researchers deve-
loped (semi-)automatic systems to illustrate online texts. In
order to select appropriate 3D models and to control ren-
der parameters for expressive visualizations, the TextIllus-
trator  employs a database of annotated 3D models. This
system extracts exact matches between terms in the docu-
ment and media descriptors and highlights those objects that
are mentioned in the visible text portion in a 3D visuali-
zation. This illustrative browsing metaphor enables a quick
access to large scientific texts but fails to detect morphologi-
cal and syntactical variants of terms, synonyms, and implicit
semantic associations between domain objects. Therefore,
the Agile system  integrates a morphological analysis and
a shallow syntactic text parsing. Moreover, the Agile system
incorporates a formal domain representation with seman-
tic networks and thesauri for media-and language-specific
realizations of formal concepts. In contrast to multimedia
generation systems, Agile only requires partial formal repre-
sentations. Moreover, for some application domains like
human anatomy, the number of domain concepts is rather
Semantic associations between domain objects can be
inferred without an in-depth analysis of the syntactic or
semantic structure of the text. To implement this, inference
mechanisms based on graph structures were applied to the
same underlying semantic network: Schlechtweg and Stro-
thotte  employed degree-of-interest functions  while
Hartmann etal.usedspreading activation .The Open
Mind Common Sense project by Lieberman et al. 
proved the wealth of these inference mechanisms on large
semantic networks that have been automatically extracted
Recently, Götze  presented a flexible authoring tool
which employs online search engines for images and 3D
models. In this interactive system the domain experts select
those illustrations which are appropriate in the current
context. In order to adjust the retrieved results to another
integrated into the final illustrations. Therefore, Götzelmann
and 3D visualizations with secondary elements. Moreover,
textual labels are considered as an implicit description of
the content of visual elements and might even reflect their
matic, and no user intervention is required for the indexing
process, and (2) to our knowledge it is the first approach that
Int J CARS
Good view selection
There are two main approaches for good viewpoint selec-
tion for medical models: uninformed and informed methods.
or occluded geometric features. The proposed measures
usually work on either the projected areas of visible faces,
or the number of faces visible from a viewpoint. One of the
proposed measures is due to Vázquez et al. , where the
the amount of geometric information captured from a view.
They build this metric using Shannon entropy as a basis. It
is defined as the following sum over all visible faces:
where Aiistherelativesize(solidangle)offacei, Atthetotal
maximum entropy is obtained when all visible faces are seen
0 log 0 is taken to be 0.
Informed approaches take advantage of the knowledge
of the scene. Viola et al. , for example, used context-
depended relevance values to determine appropriate views
on volumetric data sets. Recently, Mühler et al.  presen-
ted a system, which employs an exhaustive pre-process to
determine several view-dependent quality measures and to
weigh their influence during run-time. All these systems as
well as ours incorporate the viewpoint entropy as one para-
meter in a complex view metric.
As these methods were not tailored to link textual and
visible information, none of these approaches are able to
perform text-to-image and image-to-text queries the way we
present in this paper.
(Ai/At) log (Ai/At)
Materials and methods
In this section, we present the different features of our fra-
can interact with elements of both media (text and graphics).
These interactions are transformed into search tasks. When
the user selects a region of text, the words contained in the
selection are transformed to a query to the model database.
As a result, the rendering window shows the model of the
tioned in the text. Moreover, the rendered view of the model
is the one that better shows the structures marked in the text.
In the same way, once the user has finished exploring a 3D
model, he can ask the application to provide textual infor-
mation on the structures appearing on the rendering window.
The application then queries the textual database and issues
a result list of paragraphs where the structures appearing on
where the significance of the structures is higher.
In this section, we present the work space of the application.
In Fig. 1 we already presented a snapshot of our application,
and now we explain its main parts.
The work space is divided into three subparts: the ren-
dering window (right), the text window (left) and the menu
analyzed and permits the interactive exploration through the
use of the mouse. The user can rotate, zoom, and translate
the 3D model easily (see Fig. 3). Labels indicating the parts
of the model may be enabled or disabled in order to facilitate
the understanding of the underlying 3D geometry.
The text window shows the educational text book. In our
case, the application loads the electronic version of the well
known Henry Gray’s Anatomy of the human body . The
user may freely navigate through the book and read the des-
criptions and select the parts of the text he is interested to see
in 3D (see Fig. 4).
The toolbar menu (Fig. 5) is used to perform the different
Fig. 3 Rendering window. The user may freely explore the 3D model
Int J CARS
Fig. 4 The electronic version
of the educational book. The
user may read it and eventually
select the concepts he or she
wants to see in 3D
Fig. 5 Menu toolbar: it allows to load new models or documents and to index them. It is also used to perform document search
models and indexing them in order to make them available
for the search process.
Mutual search and data inspection
Once the application has been started, it automatically opens
the textbook Gray’s anatomy book and a 3D model. At this
moment, the interactive learning support starts. There are
two ways of using our application: the first one consists of
reading the book and looking at a 3D view of the concepts
mentioned in the text, and the second consists of looking for
textual descriptions of certain structures being visualized.
Text to model search
similar to the classical learning steps a student would carry
out. However, there is an important difference: instead of
having the limited number of pictures that usually appear in
the book, we have a set of 3D models that can be inspected
In the first case, we have a process
Int J CARS
from arbitrary points. This means that we are not restricted
to a set of pictures that will usually emphasize just a few ele-
structure, he only has to select it on the text. Our application
chosen structure. Moreover, the model is rendered in such a
way that the structure chosen by the user is shown properly.
The following example shows how this process works.
Whilethestudentsarereadingachapter devoted totheaorta,
they decide if they would like to know where it is placed and
what does look like. The users only need to select the text
segment they are interested in with the use of the mouse (see
Fig. 5). In this case, the user selected Pulmonary artery. If
a matching is achieved, the application automatically deter-
files. Subsequently, it loads the model and shows the best
view that reveals the information related to the concept that
the user highlighted in the text (see Fig. 6).
In order to help the student get the maximum information
contained in our database, the query generated through the
selection process is matched with the complete set of index
files. If we have a large amount of models, it may happen
that the same structure may appear in more than one model.
In this case, the application provides the user not only with
a view of the object that optimally matches the search, but
text (as shown in Fig. 6). On the left we see two snapshots of
the heart model, together with a snapshot of a vascular sys-
tem model. Note that our application searches not only for
a model containing the chosen information, but also the cor-
rect view that optimally shows this information (see the next
section). This is the reason why the search result may origi-
can be used to select one of the alternative models or views.
If the user clicks on one of them, the model is loaded and
the view is shown. On the top left of the rendering window
we see a colored sphere. This sphere shows a temperature
map that indicates which parts of the model are more rela-
ted to the search performed. Warm colors (yellow) indicate
regions that show the information looked for in a better way,
does not show information about the structure indicated by
the user. This means that if we place the camera in any of
the directions indicated by warm colors, we are going to be
able to see part of the information indicated on the text. This
can be seen in Fig. 6, where the user selected “pulmonary
artery”, and the system automatically generates a new view
(see Fig. 7) where the selected information is shown and a
dering window in Fig. 7 also provides a temperature sphere
that indicates with warmer colors the viewing directions that
better show the structures selected in the text.
Model to text search
mation is the following: when the student is inspecting a 3D
model he or she may be interested in the structures being
to the Find corresponding paragraphs button. The applica-
in the current view and compares the vector with the vectors
which describe paragraphs of the documents (see the follo-
wing section). The vectors are reverse ordered according to
their similarity and the result is shown on the left side with
a formatting similar to Google searches’ results, that is, a
link to the paragraph followed by some lines of text of the
referenced paragraph (see Fig. 8).
The second way of searching infor-
The key of our approach is the novel use of Information
retrieval methods and view selection algorithms.
In contrast to query languages that are designed for large
structured databases (e.g., SQL), information retrieval (IR)
has to cope with partial matches between terms in the query
and those used within the documents in a text database. The-
refore, text retrieval techniques are based on similarity mea-
sures for documents and employ different strategies to rank
the retrieved results.For an efficient representation of a large
usually employed. In the vector space model, both the query
and the documents are transformed into a vector representa-
tion. Based on this vector representation IR systems can also
ments in databases. While these systems efficiently handle
terms in natural language, the search for semantic concepts
is still very limited.
In order to reduce the dimension of the vector space,
some standard preprocessing methods can be applied: very
frequent words (stop words) are not considered, and simple
Fig. 6 Text selection of a
concept of interest
Int J CARS
Fig. 7 Result of the searching
transformation rules aiming at normalizing morphological
variants of words are applied to the remaining terms.
Before search engines can retrieve search results, the source
documents have to be indexed. We are using a popular text
In our approach, we extract textual descriptors for the source
documents (text of books and 3D models) which specify
them. We break up the documents into a finer granularity—
models are sampled from a defined number of camera posi-
tions on their bounding sphere. The extraction of the para-
graph descriptors is performed as follows (see also Fig. 9):
first, a simple text parser (1) breaks up the documents into
single paragraphs. Subsequently, each of these paragraphs
cessary stop words (e.g., ‘a’, ‘and’, ‘with’) and writes the
remaining terms into the paragraph descriptor (Fig. 9).
Finally, we sort the set of the remaining terms, and obtain
terms used in the text documents. Then, based on the obtai-
ned dictionaries we compute the document vectors for each
document. Each document vector d specifies weights wdfor
obtain descriptive weight vectors, we apply a standard mea-
sure  that considers both the frequency tfdof the term t
in the current document d as well as its frequency in the res-
pective database, where N denotes the size of the document
collection C and ntrefers to the number of documents in C
that contain term t.
These document vectors are created for each document for
the databases containing the text documents D as well as for
Int J CARS
Fig. 8 Results of the model to
text search. The matching
paragraphs are ranked in
descending priority order, and
the user may view the whole
paragraph and the part of the
book where it is placed by
clicking on the title of the
Fig. 9 The extraction of
Int J CARS
Fig. 10 The extraction of view
3D models M and, if necessary, for a given query vector q.
In contrast to the standard model, we additionally integrate
a boost function, which enables us to increase or decrease
the importance of a specific term during user interaction by
modifying the document weight for a term t, where
boost(t) < 1 → de-emphasized term t;
boost(t) = 1 → normal;
boost(t) > 1 → emphasized term t;
The boost function is fed with the results of the parser that
detects emphasized words in the paragraphs and marks these
words in the paragraph descriptor with a higher importance
value, which is recognized by the text retrieval indexer.
Thus, the original equation of  is extended to:
d= boost(t) · tft
This boost function allows us to describe the content of both
the text documents and 3D models more exactly by their
3D Model indexing
The same procedure is applied to textual annotations asso-
ciated with geometric objects in order to construct the index
for 3D models.
In this case, the different levels of granularity are model,
view, and view descriptor. In order to build the textual des-
of the model are analyzed, and the descriptors of the visible
structures from each view are extracted and stored. This can
easily be achieved by rendering a color-coded view, where
each of the labeled structures is assigned a different color,
and analyzing the view afterwards.
visualization of 3D models. This information is stored in an
annotation table that links unique color codes for geometric
components to technical terms.
For a given 3D model, a number of sample views loca-
ted on an orbit are analyzed with color-coded renditions. We
determine the visibility of their components as well as their
size and position on the projection. Finally, we construct a
view descriptor that contains reference terms for all visible
objects, that specifies their relative size and a measure that
specifies how centered these objects are located on the pro-
jection area (see Fig. 10).
The search process
Our system allows learners to explore comprehensive tuto-
ring materials stored in multi-modal databases in an interac-
tive browser. In order to coordinate the content presented in
text and graphics, user interactions are transformed into que-
ries to an information retrieval system. Two pre-computed
data structures enable us to find corresponding text descrip-
tors and view descriptors in real-time. Subsequently, a text
browser and a 3D browser are used to present the retrieved
results (see Fig. 11).
ted multimodal presentations in our application domain. As
in many scientific or technical areas, students of human ana-
ver, anatomy textbooks focus on descriptions of geometric
properties: chapters on Osteology, for example, contain des-
chapters on Myology employ the features of these bones as
landmarks to describe the course of muscles, and Syndes-
mology explains the direction of movements in joints. These
examples illustrate (1) the wealth of annotated illustrations
needed to learn a domain-specific terminology, (2) the rele-
vance ofbasiclearningtasksinanatomy foralmostallscien-
tific or technical domains, and (3) the need to complement
textual descriptions with expressive illustrations. The inte-
gration of a real-time label layout system and the automatic
Int J CARS
Fig. 11 Search tasks:
Text ↔ Illustration
selection of appropriate 3D models and views from a data-
base in our system aims at supporting each of these learning
Text → view queries
ments in descriptive texts or textual labels in the 3D visua-
lization where they need additional background information
(see Fig. 9). These interactions raise the demand to search
for corresponding textual descriptions and related illustra-
tions. The content of user-selected text fragments are trans-
formed into query vectors. The way we build the descriptors
from paragraphs has no novelty compared to the Informa-
tion Retrieval literature, and is performed as presented in
previous paragraphs. The view index allows retrieving views
on 3D models that correspond to the most relevant terms in
the query. The best fitting 3D model in the best view is loa-
other 3D models are presented in small overview windows
ted by the learner. To support the interactive exploration of
a 3D model, the quality of views with respect to the current
context defined by the query terms is presented on a colored
sphere. Moreover, the label layout within these computer-
only relevant graphical objects are annotated. Therefore, the
3D visualization component of our system integrates a real-
time layout for secondary elements that aims at minimizing
Medical students can select text seg-
lizations to explore complex spatial configurations, our sys-
tem also supports ‘visual’ queries. Contrary to the text
indexing process, building textual descriptors from 3D ren-
derings is a less explored path.
We use the information on visible objects, their relative
size and the viewpoint entropy to construct a query vector as
Our approach uses a set of views placed around an object
and analyzes the structures visible in them (see Fig. 12) by
rendering each structure in a different color.
For each structure, we need an importance factor that
behaves similarly to the boosting functions in text analysis,
that is, in each view descriptor we must give higher impor-
tance to the objects that are better captured, and the overall
information revealed by that image. This is carried out in
two different ways: first, the structure position and size in a
certain view is determined, and we give higher importance
toobjectsthatareplacednear thecenter oftheviewandwith
a larger size (see Fig. 13). Second, we measure the average
amount of information shown in the view by means of the
Viewpoint entropy measure .
Fig. 12 Evaluation of the relative size and centricity of the different
Int J CARS
Fig. 13 3D model analysis
In order to validate the usefulness of our importance mea-
surement method, we have performed a user study.
Using this view descriptor the paragraph index is used to
determine appropriate text segments for user defined views.
The search result is sorted according to the relevance and
presented to the user. User-selected paragraphs are then pre-
sented in their original context of the chosen document.
it to different users and to medical doctors. As mentioned,
the text retrieval system has no novelty compared to current
literature, but the way we index views is original. Therefore,
we studied the validity of the approach by performing a user
study. The details are provided next.
Subjects and design
We designed the test in three different languages (Catalan,
English, and German); the user study was carried out with
115 subjects. The subjects (47 females, 68 males) were sub-
divided into one test group g1(76 participants) without prior
knowledge of 3D software (e.g., 3D games, 3D modeling
Materials and apparatus
The user study consisted of 12 single tests, where the sub-
jects had to choose one out of three alternatives. The test was
presented in a printable document, but could also be perfor-
med on a computer monitor. The users were also asked if
they had problems in understanding the test or recognizing
the images, thus, those results were excluded from the study
to remove distortion by uncertainty.
The first part p1of the study consisted of six single tests
simplified, and presented a cubical arrangement of spheres.
It evaluated which of the criteria (relative size or centricity)
humans consider as superior for a ‘good view’ and to which
extent. To determine the three alternatives, we used the mea-
sures of visible relative size and centricity, as described pre-
viously. The users had to select the view they preferred in
terms of visibility of the green spheres (see Fig. 14).
ral (i.e. in the fields of medical, natural science, engineering,
etc.) we chose a model of a BMW Z3 car, as most subjects
know its interior parts by their names.
a given text (see Fig. 15). In the tests (t10−t12), the users
one out of three alternative textual descriptions in order to
find out if the proposed algorithm correlates with the users
selections (see Fig. 15).
In the tests 1–6, we calculated the value of both measures
both measures corresponded to the preferences of the users.
car components (e.g., steering wheel, leather seats). The top
scored result, together with two lower scored results, were
indexed three different descriptions (each with 2–3 technical
with a specific view of the car, in order to find out which of
the descriptions optimally fits to the image. The top scored
description, as well as two lower scored descriptions were
presented to the user in random order.
Fig. 14 User study question. The users have to decide which of the
distributions of green spheres is better
Int J CARS
Fig. 15 User study question.
The users had to select which of
the cars shows the information
asked for in a better way
Hypothesis 1: Subjects with knowledge of 3D software
select different images than those without.
3D games) have different preferences in the selection of 3D
views than those without. The majority of the subjects of
both groups g1and g2chose the same alternatives. We also
applied Student’s t test to get evidence if the percentages
with a result ≥ 5% (P ≥ 0.05). The t test neither determined
a significant difference for P1(F = 0.062, P = 0.808) nor
for P2(F = 0.278, P = 0.610).Inthefollowing,wemerged
the groups g1and g2in a single group g.
Hypothesis 2: Relative visible size is more important than
relative visible size m1and centricity m2. We used the χ2-
test; since 115 subjects contributed to the user study and
significanceχ2hastobe≥ 5.99(a = 0.05).Figure16shows
the answers to the first group of questions.
objects, while m1differed.
Therewasastrongsignificanceform1(t1: χ2= 169.793
and t1: χ2= 184.976). In t190.43% chose the alternative
with highest value for m1, while 93.04% did so for t2. For
testing m2, the tests t3and t4had equal m1of the important
objects, while m2differed.
There was a significance for m2(t3 : χ2= 17.965 and
t4: χ2= 40.192). In t3only
48.7% chose the alternative with highest value for m2, while
59.13% did for t4. Thus, m2is significant, but seems to be
much less important than m1.
Finally, the t5and t6tested m1and m2against each other.
m1had a higher support (t5 : 73.04% and t6 : 88.70%)
than m2(t5: 19.13% and t6: 9.57%), while the remaining
subjects chose the alternative, where neither m1nor m2was
maximized (t5: 7.83% and t6: 1.74%).
with an unequal weight on both measures m1and m2(5:1).
Hypothesis 3: The approach presents desired views for a
In the tests t7−t9we presented a set of common techni-
cal terms of the car and showed three alternative views of
the car. The user had to select the one where the named
objects were optimally visible. The higher the computed
relevance of our algorithm was, the more users preferred
the view. Most subjects chose the highest ranked alternative
(t7: 86.09%,t8: 84.35%,t9: 89.57%). At a relevancy of
less than 0.8 only a few subjects preferred the view, but with
a relevancy greater than 0.8 the preference increased expo-
nentially. Thus, this seems to be a rough measure for a cutoff
Fig. 16 The answers given by
the users to the six (t1−t6) first
Int J CARS
Fig. 17 Distribution of answers
to questions t7– t12
of retrieval results. The results of these questions are shown
in Fig. 17.
Hypothesis 4: The approach presents desired paragraphs
for a given view.
In the tests t10−t12we presented an image of a car and sho-
of the car. The subjects had to choose the alternative that
optimally describes the image. Like in the previous test, the
ked alternative (t10: 100.00%,t11: 95.65%,t12: 95.65%).
Again, a cutoff of 0.8 for the relevancy of retrieval results
seems to be a reasonable threshold.
First, the study showed that there is no difference in the
computer knowledge. Both the tests for the relative visible
size and the centricity were significant, while the relative
visible size was much preferred over the centricity. In the
testsfor text-image queries as wellas image-text queries,the
relevance determined by our algorithm correlated with user
chose the top ranked view. Since with a relevance value less
than 0.8 only a few subjects chose the view, this value seems
to be a valid cutoff of the retrieval results.
Conclusions and future work
In this paper, we presented a framework for the support of
and the 3D information are linked through labels and there-
fore the user may query a 3D model database in order to be
by the use of the 3D model. This permits a better unders-
tanding of the 3D relationships of the anatomical structures
and eases the learning process by giving the user freedom
to jump from one structure description to another following
what appears on screen.
The text processing presents no novelty compared to the
current literature, but using 3D views to perform searches in
text is one of our contributions. Therefore, we carried out an
experiment that determines that the decisions taken in order
to build the text descriptors of 3D views are correct. We have
shown the application to users and, concretely, we showed it
working to a couple of medical doctors, who found it very
interesting for educational purposes.
Our application currently uses the electronic version of
Gray’s Anatomy book and the 3D models of the Viewpoint
library. We want to extend it by adding labeled versions of
other parts of the body extracted from medical data recently
captured or from the Visible Human Project.
Moreover, some extensions are possible: first, we would
like to improve the text processing by detecting synonyms
or name constructions (i.e., names and adjectives) that may
in order to improve the text processing, the required know-
ledge is already present in Information Retrieval or Automa-
tic Summarization literature.
as visual information. If we had a database of images, we
could also use them to illustrate queries. The only require-
ment would be the adequate annotation of such images.
1. Tortora GJ (1997) Introduction to the human body: the essentials
of anatomy and psychology. Benjamin Cummings, Menlo Park
2. Götzelmann T, Vázquez P, Hartmann K, Germer T, Nürnberger
A, Strothotte T et al (2007) SIGGRAPH proceedings of the 23rd
international spring conference on computer graphics
(2007) Correlating text and images: concept and evaluation. Lec-
ture notes in computer science: 7th international conference on
smart graphics. Springer, Heidelberg
4. Feiner SK, McKewon KR (1993) Automating the generation of
coordinated multimedia explanations. In: Maybury MT (ed) Intel-
5. Wahlster W, André E, Finkler W, Profitlich HJ, Rist T (1993) Plan-
Intell 63:387–427. doi:10.1016/0004-3702(93)90022-4
6. Schlechtweg S, Strothotte T (1999) Illustrative browsing: a new
Int J CARS Download full-text
7. Hartmann K, Strothotte T (2002) A spreading activation approach
to text illustration. In: 2nd internationl symposium on smart gra-
phics, pp 39–46
8. Schlechtweg S, Strothotte T (2000) Generating scientific illustra-
tions in digital books. In: AAAI spring symposium on smart gra-
phics, pp 8–15
9. Furnas GW (1986) Generalized Fisheye views. In: Conference on
human factors in computing systems, pp 16–23
10. Collins AM, Loftus EF (1975) A spreading-activation theory of
semantic processing. Psychol Rev 82(6):407–428. doi:10.1037/
11. Lieberman H, Liu H, Singh P, Barry B (2005) Beating common
sense into interactive applications. AI Mag 25(4):63–76
illustration of text. Simulation und Visualisierung 195–206
13. Götzelmann T, Götze M, Ali K, Hartmann K, Strothotte T (2007)
and illustration framework. J WSCG 15:1
conference, pp 273–280
15. Viola I, Feixas M, Sbert M, Gröller M (2006) Importance-driven
focus of attention. IEEE Trans Vis Comput Graph 12(5):933–940.
16. Mühler K, Neugebauer M, Tietjen C, Preim B (2007) Viewpoint
selection for intervention planning. In: EG/IEEE-VGTC sympo-
sium on visualization
17. GrayH (1918) Anatomyofthehumanbody,20thedn.Lea&Febi-
18. Salton G, Wong A, Yang CS (1975) A vector space model for
theme generation, and summarization of machine-readable texts.
Science 264:1421–1426. doi:10.1126/science.264.5164.1421
20. Götzelmann T, Hartmann K, Strothotte T (2006) Agents-based
annotation of interactive 3D visualizations. In: 6th International
symposium on smart graphics, pp 24–35