3DSEAM: a model for annotating 3D scenes using MPEG-7

Abstract

The progress and the continuous evolution of computer capacities, as well as the emergence of the X3D standard have recently boosted the 3D domain. Associating some semantics with 3D contents becomes a major issue specially for reusing such contents or pieces of content after having extracted them from existing 3D scenes. In this paper, we address this issue by proposing a generic semantic annotation model for 3D, called 3DSEAM (3D semantics annotation model). 3DSEAM aims at indexing 3D contents considering visual, geometric and semantic aspects. 3DSEAM is instantiated using MPEG 7 that we have extended with 3D specific locators. These locators link some visual, geometric and semantic features to the corresponding X3D fragments. These features can then be used for indexing and querying.

Full text

Page 1

1
3DSEAM: a model for annotating 3D scenes using MPEG-7
Ioan Marius Bilasco
LSR IMAG
681 rue de la Passerelle, BP. 72
Saint Martin d’Hères Cedex, France
(+33)476827211
bilasco@imag.fr

Jérôme Gensel
LSR IMAG
681 rue de la Passerelle, BP. 72
Saint Martin d’Hères Cedex, France
(+33)476827280
gensel@imag.fr

Marlène Villanova-Oliver
LSR IMAG
681 rue de la Passerelle, BP. 72
Saint Martin d’Hères Cedex, France
(+33)476827280
villanova@imag.fr

Hervé Martin
LSR IMAG
681 rue de la Passerelle, BP. 72
Saint Martin d’Hères Cedex, France
(+33)476827280
martin@imag.fr
ABSTRACT
The progress and the continuous evolution of computer capacities, as well as the emergence of the X3D standard have
recently boosted the 3D domain. Associating some semantics with 3D contents becomes a major issue specially for reusing
such contents or pieces of content after having extracted them from existing 3D scenes. In this paper, we address this issue
by proposing a generic semantic annotation model for 3D, called 3DSEAM (3D Semantics Annotation Model). 3DSEAM
aims at indexing 3D contents considering visual, geometric and semantic aspects. 3DSEAM is instantiated using MPEG-7
that we have extended with 3D specific locators. These locators link some visual, geometric and semantic features to the
corresponding X3D fragments. These features can then be used for indexing and querying.
Keywords: X3D, MPEG-7, 3D content reuse, 3D locators, indexing, semantic annotations

Page 2

2
1. Introduction
The progress and the evolution of computers capabilities facilitate the deployment of 3D contents. With the
emergence of widely accepted standards such as X3D [1], 3D contents progressively cover more and more
domains of application: from spatial planning, transports, construction, defense to architecture and tourism, to
name a few. What makes the success of 3D is probably the possibility of creating, modifying or running some
simulations on 3D virtual representations of real world environments which are more realistic and more attractive
than to work with linear or 2D data.
Efforts have been made in order to facilitate the work of 3D model designers, providing them with powerful
(commercial) tools (such as Maya by Alias|wavefront or 3D Studio Max by Discreet) for creating complex 3D
objects and scenes. Still, even though the construction of a scene or model requires lot of time and resources the
3D contents are usually designed for specific applications only. The partial reuse and/or the sharing between
several applications of 3D models is not fully supported. Basic means of reusing parts of models are available.
For instance, if the parts of a model are saved in different files, each separate file can be included in some other
scenes. The simplified research criterion for the retrieval and the inclusion in a new model of an existing file is
the exact address (the local address or the address on some server) of that file. However, some interesting
propositions [2] [3] [4] based on some alternate research criterion are available. Models can be automatically
indexed using signal analysis and query by example techniques are applied in order to retrieve 3D contents. But,
these solutions are all concerned with the indexation of the geometry or the appearance of 3D models. Semantic
queries are not available in such approaches. Some high level research criteria (semantics, content, …) increase
the efficiency and the accuracy of the reuse techniques.
The ability of reusing 3D scenes in various application contexts is very important for the multimedia community
as it should reduce the development costs of 3D applications. In order to offer a solution to the reusability
problem of 3D models, our work consists in providing an extensible framework whose final objective is to deal

Page 3

3
with the main 3D model representation formats (3DS, LWO, DXF, COB, X3D…).
By reuse, we mean the reuse of parts of a model independently from a specific application (i.e. the same 3D
model describing an office room could be used in different building models) and the reuse of the same model
within different applications (i.e. the same 3D model can be used in different applications by attaching specific
information to it). In both cases, the reuse process can be decomposed in three steps: 1) the formulation of the
demand in terms of 3D models, 2) the research of the most adequate model according to the demand and finally,
3) the retrieval, the analysis and the integration of the results. In order to support high level research criteria, a
preliminary step consisting of indexing and annotating the 3D models is necessary. These operations enrich the
set of information associated to the basic geometric features of the scene and, hence, richer research criteria can
be formulated.
The attachment of semantic information to multimedia objects is not a recent topic in the multimedia community.
Important research efforts have been made in the characterisation (in terms of low-level features or high-level
semantics) of audios, images or videos. For instance, MPEG-7 [5] descriptors and description schemas are
largely accepted as multimedia description standard tools. Due to its great flexibility and extensibility we have
decided to put the MPEG-7 at the heart of our reusing framework. However, in its current state, MPEG-7, does
not cover 3D media objects. Then, we propose some specific descriptors and description schemas in order to cope
with this problem. The integration of 3D into MPEG-7 description tools opens the way to richer 3D content
annotations.
The paper is organized as follows. The next section gives an insight of the 3D world representation, focusing on
the X3D standard. The section 3 presents issues related to object localization in a 3D scene. In section 4, we
describe 3DSEAM (3D SEmantics Annotation Model), a generic annotation model for 3D objects covering 3D
content management concerns (and especially, the reuse). After a brief introduction to MPEG-7, in section 5, we
describe an MPEG-7-based instantiation of 3DSEAM.

Page 4

4

2. The design of 3D scenes using X3D
The Extensible 3D (X3D) standard, proposed by the Web3D consortium [1], defines a runtime environment and a
delivery mechanism for 3D contents and applications running over a network. It combines geometry descriptions,
runtime behavioural descriptions and control features. It proposes different kinds of encodings, including an
XML encoding.
An X3D document represents a 3D scene as an n-ary tree. Among the most important nodes one find: geometric
primitives (Cube, Box, IndexLineSet, etc), geometric transformations (Transform applying translations and rotations
to primitives), composite objects (Group), alternate contents (Switch), multi level representations (LOD), etc. This
tree also contains environmental elements: lights, viewpoints, etc., and a meta-data node (WorldInfo), which
contains textual information related to the parent node.
a. A real world scene
b. A virtual scene

Figure 1. The White and the Blue buildings modelled using X3D.
In order to illustrate the use of X3D, we are constructing two scenes presenting the building hosting our
laboratory. The first scene (Figure 1a.) is extracted from a 3D scene modelling some building situated on the
University Campus. The (White and the Blue) buildings are positioned following the real-world spatial
organisation. This scene contains a coarse representation of buildings, which are modelled by simple 3D boxes.

Page 5

5
The scene also contains three passages modelled by 3 boxes. The second scene (Figure 1b.) presents a virtual
scene in which the two buildings (the white and the blue one) are organised differently. The representation is
more detailed as floors are modelled. Consequently, a building is represented as a group of boxes. Furthermore,
we have introduced some variations of the main colour for each building in order to better distinguish the floors.
Later on in this paper, we use the different models associated to the real-world objects considered (the White
building and the Blue one) in order to illustrate some issues related to the management and the reutilisation of 3D
objects. Excerpts presenting the general structure of the X3D codes for the both scenes are shown in Figure 2.
1:<X3D profile="Core" ...>
2: <Scene>
3: <Transform>
4: <Transform id="WHITE"
5: translation="-8 4 1" scale="12 4 2">
6: <Shape>
7: <Box/>
8: <Appearance>
9: <Material diffuseColor="0.9 0.9 0.9"/>
10: </Appearance>
11: </Shape>
12: </Transform>
13: <Transform id="BLUE"
14: translation="3 5 13" scale="3 5 6">
15: ...
16: </Transform>
17: <Group id="passages">
18: <Transform
19: translation="3 7 5" scale="0.5 0.25 2">
20: ...
21: </Transform>
22: <Transform
23: translation="3 5 5" scale="0.5 0.25 2">
24: ...
25: </Transform>
26: <Transform
27: translation="3 3 5" scale="0.5 0.25 2">
28: ...
29: </Transform>
30: </Group>
31: </Transform>
32: </Scene>
33:</X3D>33:</X3D>
1:<X3D profile="Core" ...>
2: <Scene>
3: <Transform>
4: <Group id="WHITE">
5: <Transform id="WHITE1st"
6: translation="-23 1 -12" scale="2 1 12">
7: ...
8: </Transform>
9: <Transform id="WHITE2nd"
10: translation="-23 3 -12" scale="2 1 12">
11: ...
12: </Transform>
13 <Transform id="WHITE3rd"
14: translation="-23 5 -12" scale="2 1 12">
15: ...
16: </Transform>
17: <Transform id="WHITE4th"
18: translation="-23 7 -12" scale="2 1 12">
19: ...
20: </Transform>
21: </Group>
22: <Group id="BLUE">
23: ...
24: </Group>
25: <Group id="RED">
26: ...
27: </Group>
28: <Group id="ROADS">
29: ...
30: </Group>
31: </Transform>
32: </Scene>
33:</X3D> 33:</X3D>
a. the real world scene a. the real world scene
b. the virtual sceneb. the virtual scene
1:<X3D profile="Core" ...>
2: <Scene>
3: <Transform>
4: <Transform id="WHITE"
5: translation="-8 4 1" scale="12 4 2">
6: <Shape>
7: <Box/>
8: <Appearance>
9: <Material diffuseColor="0.9 0.9 0.9"/>
10: </Appearance>
11: </Shape>
12: </Transform>
13: <Transform id="BLUE"
14: translation="3 5 13" scale="3 5 6">
15: ...
16: </Transform>
17: <Group id="passages">
18: <Transform
19: translation="3 7 5" scale="0.5 0.25 2">
20: ...
21: </Transform>
22: <Transform
23: translation="3 5 5" scale="0.5 0.25 2">
24: ...
25: </Transform>
26: <Transform
27: translation="3 3 5" scale="0.5 0.25 2">
28: ...
29: </Transform>
30: </Group>
31: </Transform>
32: </Scene>
1:<X3D profile="Core" ...>
2: <Scene>
3: <Transform>
4: <Group id="WHITE">
5: <Transform id="WHITE1st"
6: translation="-23 1 -12" scale="2 1 12">
7: ...
8: </Transform>
9: <Transform id="WHITE2nd"
10: translation="-23 3 -12" scale="2 1 12">
11: ...
12: </Transform>
13 <Transform id="WHITE3rd"
14: translation="-23 5 -12" scale="2 1 12">
15: ...
16: </Transform>
17: <Transform id="WHITE4th"
18: translation="-23 7 -12" scale="2 1 12">
19: ...
20: </Transform>
21: </Group>
22: <Group id="BLUE">
23: ...
24: </Group>
25: <Group id="RED">
26: ...
27: </Group>
28: <Group id="ROADS">
29: ...
30: </Group>
31: </Transform>
32: </Scene>

Figure 2. Excerpts from X3D files.
The boxes that are used for modelling parts of the real world objects are given an adequate size using the scale
attribute of Transform nodes as shown in Figure 2 (line 5 for the White building in 2a or line 6 for the first floor of
the White building in 2b). They are grouped together into Group nodes (line 17 for the passages in 2a, or line 4 in