Content uploaded by Babis Theodoulidis
Author content
All content in this area was uploaded by Babis Theodoulidis on Nov 12, 2014
Content may be subject to copyright.
8 Advanced Uses: Composing
Interactive Spatio-temporal Documents
Isabelle Mirbel1, Barbara Pernici2,
Babis Theodoulidis3, Alex Vakaloudis2, and Michalis Vazirgiannis4
1Laboratoire I3S, Sophia-Antipolis, France
2Politecnico di Milano, Italy
3University of Manchester - Institute of Science and Technology, United Kingdom
4Athens University of Economics and Business, Greece
8.1 Introduction
Lately a new generation of application domains is emerging. Such applications,
heavily dynamic and interactive, include interactive multimedia applications, vir-
tual reality worlds, digital movies and 3D animations. They deal with intensive
spatio-temporal dependencies between the participating objects with motion be-
coming a central issue. Indeed in the aforementioned application domains, much
of the information conveyed and manipulated has spatial and/or temporal as-
pects. Synthetic worlds (like VRML worlds, digital movies, interactive multime-
dia scenarios etc.) can be started at any point in time, and there is a multitude of
events that may occur (e.g., 3D object collisions) which may trigger other actions
(e.g., change of the motion of the objects that collided) provided that some con-
straints hold. The session concept provides a new spatio-temporal context that
has a different temporal origin and perhaps a different evolution according to
the internal/external interaction that takes place. In this case the concept of
scenario is an important entity representing the spatio-temporal course of the
context (i.e., spatio-temporal actions possibly related) in conjunction with the
occurring interaction (in terms of events) and potentially with the validity of
conditions/constraints.
It is therefore evident that there is a strong connection between the topics
studied in the modeling of spatio-temporal information and the issues of concern
in composing interactive presentations. This is the motivation for the work of
this chapter; to apply knowledge earned in the spatio-temporal domain into areas
not directly associated with spatio-temporal databases but still uniform with
them. The result leads to the integrated study of spatio-temporal information
applicable in a range of diverse uses.
In the next section we relate spatio-temporal databases and interactive pre-
sentations, formally termed Interactive Spatio-Temporal Documents (ISTDs).
We then present an approach for modeling the spatio-temporal components of
such presentations. This focuses on the treatment of objects as integrated entities
and thus apart from modeling their geometrical and positional characteristics (in
an application independent format), it also demonstrates how to relate them to
any non-spatial attributes. Next we analyze the method for describing the inter-
active behavior in these applications following the Events Conditions Actions
T. Sellis et al. (Eds.): Spatio-temporal Databases, LNCS 2520, pp. 319–344, 2003.
c
Springer-Verlag Berlin Heidelberg 2003
320 Isabelle Mirbel et al.
paradigm. The behavior of objects includes geometric transformations of trans-
lation, rotation and scaling. Section 8.5 shows how to provide database storage
and querying support. Finally we discuss some examples of applying this work
and we close with references to related and future work.
8.2 Interactive Presentations
and Spatio-temporal Databases
An Interactive Spatio-Temporal Document (ISTD) such as a virtual world is or-
ganized in two parts namely, structural and dynamic. The participating ob jects,
called actors or components comprise the structural part. Similarly to tradi-
tional multimedia objects that have temporal (e.g. sound), spatial (e.g., image)
and spatio-temporal (e.g., video) characteristics, the components of ISTDs are
also managed by their spatio-temporal features. The dynamic part of such pre-
sentations specifies the behavior of components and also the interaction among
them or with the viewer. The viewer is integral part of a presentation as com-
ponents interact either through I/O devices or through the modification of their
properties (e.g., change of position).
Both the structural and the behavioral parts are rich in spatio-temporal
concepts. Components are in fact (possibly 3D) spatio-temporal objects. Their
behavior takes the form of spatial and temporal movement, variation in the
geometrical characteristics and change in the appearance of the objects. As a
result of such changes, further modifications may be triggered as the objects
interact with each other.
The extensive similarities that appear between spatio-temporal data and
ISTDs are reflected on the modeling of these presentations. The requirements
for defining a spatio-temporal data model have been analyzed in Chapters 4
and 5. The same ones (e.g. modeling of object continuous evolution, support
for orthogonal spatial, temporal and spatio-temporal granularity) apply when
defining data models for spatio-temporal scenario components. The following
additional requirements originating from the 3D nature of objects and the need
for interactive support come also into play:
•Viewer inclusion: Incorporate the viewer of an application as part of the data
model and as a query argument.
•Interaction support: Define interaction semantics over 3D-spatio-temporal
objects, the viewer and the application in order to add user-defined behav-
ioral modeling.
•Types of elements: Specify the different cases of interaction elements that
exist in this domain.
•Composite elements: Show how to construct composite structural and inter-
action elements in order to capture complex behavior.
Components are objects with complex spatial and temporal features, which
make their specification a laborious procedure. The storage of components can fa-
cilitate their reusability thus accelerating the process of producing ISTDs. How-
ever, the use of a file system manager for this purpose does not provide enough
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 321
support because components should be retrieved through specialized, context-
based criteria. Therefore, employing a DBMS, relational or object-oriented is
recommended.
In the following subsections, we define concepts that can be implemented
using both relational and object-oriented designs. By providing database support
our goal is not only to store components but also to retrieve them with criteria
that satisfy the developer’s and the application’s requirements.
8.3 Modeling the Components
of Spatio-temporal Interactive Documents
This section presents 3D-STDM, a data model for the components of ISTDs
regarded as 3D-spatio-temporal objects. We specify a mechanism for modeling
them and later we demonstrate how to provide database support in terms of stor-
age and querying. First we discuss the particularities of the 3D-spatio-temporal
domain.
8.3.1 Particularities
of 3D-Spatio-temporal Modeling for Scenario Components
The 3D-spatio-temporal domain derives from the composition of the temporal
and the 3D spatial domains inheriting their characteristics. Towards the com-
bination of spatial and temporal concepts many researchers define a single, in-
tegrated spatio-temporal framework by simply considering time as the fourth
geographical dimension. This simplistic approach is the first and obvious step
in spatio-temporal modeling. However, the spatial and temporal composition is
not simplistic (e.g., space and time are measured differently; we cannot measure
time in feet) and new spatio-temporal-based features which don’t exist in neither
the spatial nor the temporal domain are introduced.
•First of all, the three geometrical dimensions are orthogonal among them-
selves. In contrast they are not independent to time as spatial changes hap-
pen on a temporal basis. Even more, the temporal distinction between the
past and the future is complex and more profound than the analogous be-
tween positive and negative of the spatial fields. Finally research work on
temporal analysis has proved that time is not a simple one-dimensional linear
concept. Modern requirements for temporal functionality are the modeling
of future (and possibly branching) events and the description of the same
fact under different perspectives (version management).
•3D-spatio-temporal worlds are utilized for portraying 3D objects and their
evolution through time. Since these objects are “real-life” graphical objects,
their evolution may only be continuous and never discrete. A real-life graph-
ical object cannot directly just jump from one state into another (as for
example the non-graphical “salary” may at some point get a sharp, crisp
rise). Instead it can only mutate in a continuous way. Only when there is
uncertainty or incomplete knowledge should discrete transitions be recorded.
322 Isabelle Mirbel et al.
•3D-spatio-temporal objects, as they represent actual material, may only
change their formation, but never disappear. For example, steel, glass and
plastic are at one point in time put together to construct a car and twenty
years later some of the parts of this car end up in a scrap yard. The orig-
inal materials remain the same; what is different is their formation (from
amorphous maze, glass becomes part of a car and later part of a scrap yard
belongings). A 3D-spatio-temporal data model should be able to follow the
different formations that a 3D-spatio-temporal object adopts during its life-
time (i.e., as long as the information system requires its modeling).
•Another interesting feature is the role of the viewer in 3D spatial and 3D
spatio-temporal environments. Unlike the case of 2D maps where the viewer
remains external to the presentation (a.k.a. map), in 3D worlds he/she fully
navigates inside the world thus becoming an integral part of the overall
setting. Its position influences the way they conceptualize the relationships
among the different objects. Figure 8.1 exemplifies this phenomenon. The
two boxes have stable positions. Nevertheless, viewer 1 believes that Box 2
is behind Box 1. On the contrary, viewer 2 assumes that Box 2 is in front of
Box 1. A third viewer thinks that Box 1 is on the left of Box 2. Due to this
occurrence, the viewer’s position must be taken into consideration within a
3D-spatio-temporal query language. Under this perspective we should take
into consideration Freksa’s work on conceptual spatial relationships where
qualitative operators (such as left, left-front etc.) are introduced.
•The 3D-spatial domain also introduces the notion of orientation. Some (but
not all) 3D objects have a certain “directioning” in the sense that there is a
consensus on which their top/bottom or left/right side is defined. No matter
where the viewers of a world stand they would all agree for example on the
fact that a glass is on the table. That is because the glass and the table have
universally agreed top and bottom sides.
•Finally the 3D-spatio-temporal domain is particular from a visualization per-
spective. Once again, the realistic features of the participating objects forbid
the use of their visual properties for visualization purposes. The color, po-
sition and geometry in 3D worlds specify the state of an object and their
alteration (for visualization) means that the represented object is not the
right (i.e., exact) one. Fortunately, as it is explained in Chapter 5, the 3D-
spatio-temporal domain has a rich set of properties available for visualiza-
tion.
From all these particularities it becomes evident that 3D-spatio-temporal
modeling, the foundation for specifying ISTDs, cannot be accomplished by nei-
ther extending a 2D-spatio-temporal model nor by attaching a simple times-
tamp to 3D-spatial entities. A 3D-spatio-temporal modeling approach must be
autonomous in order to capture these spatio-temporal-sourced characteristics.
8.3.2 Meta-modeling
The design of 3D-spatio-temporal semantics is characterized by their universal
and elementary nature. “Universability” in design means that they remain inde-
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 323
Viewer Position 1 Viewer Position 2
Viewer Position 3
Box 1 Box 2
Fig. 8.1. Different viewpoints lead to different relationships
Application
Event Application
Condition Application
Action
Application
Viewer
Viewer
Event Viewer
Condition Viewer
Action
1..* 1..*
1..*
1
1
1..* 1..* 1..*
Application
Object Appearance
Position
Geometry
Spatial
Description
1..*
1..*
Object
Event Object
Condition Object
Action
1..* 1..* 1..*
Transaction
Time
Time
Valid
Time
Spatial
Composition
Spatial
Relationship
0..*
0..*
0..*
0..*
0..*
0..* 0..* 0..* 0..* 0..*
compound
component
0..1
0..*
0..1
0..1
1
1
11
1
Fig. 8.2. Structural meta-model for spatio-temporal scenario components
pendent from any application and thus can be used in every domain. Elementary
nature ensures that they are as minimal as possible. They contain information
implied by their types but do not include any context information and do not
make any assumptions about how they are visualized. For example, VGeometry
describes the geometry and therefore does not include the color or a rendering
device.
A relational meta-model is a guide towards creating a framework for the def-
inition of these semantics. A meta-model comes to determine the basic concepts
of a domain and to discover knowledge about the data to be managed. It defines
324 Isabelle Mirbel et al.
Features
Condition
Application
Event
Composition
0..*
Rule
1
1..*
Event Condition Action Action
Composition
Application
Event Object
Event Viewer
Event Application
Action Object
Action Viewer
Action
0..1
0..111
0..*
0..1
Application
Condition Object
Condition Viewer
Condition
Condition
Composition
0..* 0..1
Moving
Action
Setting
Action
Transformation
Action
?
Time
Action
Line
Action
Start/Stop
Event
Temporal
Event
External
Event
Internal
Event
?
Directional
Event
Topological
Condition
?
?
Metrical
Condition
Fig. 8.3. Behavioral meta-model for spatio-temporal scenario components
generic concepts instances of whose are the specific models. Hence, in the design
stage it constitutes a reference base for the development of the various models.
We consider that an ISTD consists of two parts, structural and behavioral,
presented using UML notation in Figures 8.2 and 8.3 respectively. The entities
of the structural part called Application objects have two sides, namely attribute
and spatial (spatio-temporal). The first one contains the attributes with no spa-
tial context (like the salary of an employee). Because such data may change
through time, they acquire an optional bi-temporal timestamp attached on them.
The spatial (spatio-temporal) side describes the viewer and the graphical
properties of an object, which can also vary through time. A spatial object is the
central concept and it can be either primitive or composite. Primitive objects are
the leaf nodes in the object graph and explicitly describe a simple 3D element
by specifying its shape (Geometry), its placement inside the world (Position)
and its display (Appearance). Primitive spatial objects are grouped together to
construct composite ones. The latter consist of primitive and/or complex objects.
They need to have only their position specified as their primitive children already
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 325
describe their geometry and appearance. To generate a spatio-temporal object,
a spatial object is combined with temporal constructs in a form of a composite
timestamp.
The behavioral part is modeled in terms of Event Condition Action (ECA)
rules. Events are the fundamental means of interaction and are raised by user
actions, by participating objects or by the system. They may be simple or com-
plex, and have attached their spatio-temporal signature (i.e., the space and the
time they occurred). Conditions take also as arguments the viewer or objects and
finally actions apply to objects. The scenario stands for the integrated behavior
of a presentation, i.e. what kind of events it will consume and what actions will
be triggered as a result. The different kinds of events, conditions and actions are
summarized in Figure 8.3.
In the next subsection, we define the semantics that build the 3D-spatio-
temporal Data Model (3D-STDM) for implementing the structural part of this
meta-model. These semantics are temporal, spatial and spatio-temporal.
8.3.3 Temporal Semantics
The 3D-STDM temporal semantics provide bi-temporal support to both at-
tribute and spatial data. Even more, they capture the continuous evolution of
spatio-temporal objects. First we give formal definitions for the point in time
and the trend of object evolution:
Definition 1. ApointintimeVTimePoint is defined as follows:
VTimePoint::= [Year],[Month],[Day],[Hour],[Min],[Sec],[msec],[µsec]
All VTimePoint fields are optional in order to accommodate multiple gran-
ularities. Different applications treat time with various details (e.g., year, day,
msec). Furthermore, objects in the same application might require different tem-
poral granularities. To switch from one to another we use the following rules:
•When converting from lower to higher granularity (e.g., sec to year), we just
delete the parts not required by the higher granularity.
•When converting from higher to lower granularity (e.g., year to sec), we
sum all the numbers of lower granularity units that any higher granularity
consists of. For instance to convert 5/3/1989 to days we need to calculate
the number of days for 1989 years and 3 months and 5 days.
Definition 2. A tuple VTrend models the way a 3D-spatio-temporal object
changes from an explicitly recorded state into the next one.
VTrend::= mode, [value]
Where mode ::= Integer||String and value ::= Real.
Tuple VTrend applies to spatio-temporal objects only. It is introduced so as
to satisfy the requirement for the continuous modeling of the state of an object.
326 Isabelle Mirbel et al.
Has spatio-temporal part
VIndex
VTObject
Table X
ID
Other attributes
1
1
Has attribute part
1
1
Fig. 8.4. The use of VTrend
VTrend associates two successive recorded states of an object. Field mode en-
codes the way the object evolved from one state into the next one (e.g., linear,
quadratic, cubic etc.). Value is an optional field that comes along with com-
plex modes and specifies the value of the evolution (like the radius in circular
movement). Using VTrend (Figure 8.4) one can interpolate into the period that
two successive object states form and thus extract the state of an object at any
point. For instance, if a database has recorded the position of a car at 12:00 and
12:30 and we know that the car is moving with steady speed, we may calculate
the car’s position at 12:11, 12:20 etc.
An object may evolve in many and application-dependent ways. It is up to
the application developer to identify these different ways (modes) and try to
simulate them with VTrend.
Both point and period timestamps can be formed. VTimePoint if used alone
is a point timestamp. VTrend in conjunction with two successive instances of
VTimePoint simulates a temporal period. Most important, with the second
method, we not only have the state of the object at the ends of the period
but through VTrend we can calculate the state at any internal point of this pe-
riod. This combination of VTimePoint and VTrend leads us to the definition of
the integrated timestamp, termed VTime:
Definition 3. ATimestampVTime marks the valid and transaction time for a
recording. It also provides the VTrend link to the previous recording.
VTime::= [valid time],[transaction time],[mode, value]
Where:
valid time ::= VTimePoint,
transaction time ::= VTimePoint,
mode ::= Integer||String and
value ::= Real
All the attributes of VTime are optional. Under this perspective, VTime is
optional as well. This approach is adopted in order to give the developer the
option to apply full bi-temporal timestamp or wherever needed to encompass
valid or transaction timestamps only.
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 327
VTime will be used to describe internal changes of 3D-spatio-temporal ob-
jects. We also need to record alterations in the status of the object (e.g., birth,
death) and of the relationships among objects. For this reason we introduce
VLife:
Definition 4. A“Lifestamp”VLife concerns the status of an ob ject. It records
the birth and death of the object and points to the object or objects that gen-
erated the current object. It is time-stamped.
VLife::= birth, death, lif e, previds
Where:
previds ::= VTObjectID,VTObjectID (explained in the next section)
birth, death,
life ::= VTime.
8.3.4 3D-Spatial Semantics
The following 3D spatial semantics are also set according to the guidelines of the
meta-model of Fig 8.2. First we define the three different parts that comprise a
3D-spatial object namely, geometry, appearance and position.
Definition 5. The geometry VGeometry of a primitive 3D-spatial object is either
a Box, or a Cylinder or a Cone or a Sphere:
VGeometry::= VGeometryID,(Box||Cylinder||Cone||Sphere)
Where:
VGeometryID::= Integeridentif ier,
Box ::= width, depth, height,
Cylinder ::= height, radius,
Cone ::= height, radius,
Sphere ::= radius
Where width, depth, height, radius are Real.
Definition 6. The position VPosition of a 3D-spatial ob ject is specified by its
translation, rotation, scale and orientation:
V P osition ::= V P ositionID, T ranslation, Rotation, S cale, Orientation
Where:
V P ositionI D ::= Integer,
T ranslation ::= 3DV ector,
Rotation ::= 3X3Matrix,
Scale ::= 3DV ector,
Orientation ::= 3DV ector.
VPosition is the 3D-Vector abstraction of a 3D-spatial object. It determines
the object’s placement (translation), size (scale) and direction (rotation, orien-
tation). It is to be used in cases such like distance querying where we need to
treat the object as one vector or as one 3D point.
328 Isabelle Mirbel et al.
Definition 7. The appearance VAp pea ra n ce of a primitive 3D-spatial object is
defined by specifying the following parameters:
V Appearance ::= V AppearanceI D, (color||texture)
Where:
V AppearanceID ::= Integer,
color ::=diffuse, specular, emissive, transparency,
texture ::= name of graphics file for the texture diffuse, specular,
emissive ::= 3DV ector,
transparency ::= Real.
By combining these three parts we build the 3D spatial ob ject:
Definition 8. A 3D spatial object VObject is defined as:
VObject::= V ObjectI D, V P osition, [VGeometry][V Appearance]
Where:
VObjectID ::= Integer,
VGeometry,VAppearance,VPositionas defined above.
The above definition covers both primitive and complex objects. In the case
of primitive ones, all three fields of an object must be specified. However, only
VPosition determines complex ob jects. The description of their geometry and
appearance is delegated to their primitive children.
Finally, the 3D-spatial domain introduces the inclusion of the viewer in the
3D environment. To model the viewer’s view, we introduce VViewpoint.
Definition 9. The viewer’s viewpoint is determined by its position, its direction
and its orientation:
VViewpoint::= position, direction, orientation
Where position, direction, orientation ::= 3DV ector
8.3.5 3D-Spatio-temporal Semantics
Up to now, we have listed strictly temporal and 3D-spatial semantics. This
section describes the technique for creating firstly spatio-temporal objects and
secondly complex objects.
Let us begin with the combination of VObject and VTime,whichleadsusto
the definition of VTObject, capturing the semantics for describing a 3D spatio-
temporal object.
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 329
Definition 10. A 3D-spatio-temporal object is a 3D-spatial object VObject with
the particular timestamp VTime bound with its attributes.
VTObject::= VObject X VTime
In order to maintain flexible temporal and spatial support, this time stamping
is optional. It may occur to those attributes that the system developer considers
that they mutate in time. For example the position of the object might need
both valid and transaction time recording while its color may remain static (i.e.,
without any VTime support).
Has spatio-temporal part
VIndex
VTObject
Table X
ID
Other attributes
1
1
Has attribute part
1
1
Fig. 8.5. VIndex links the spatio-temporal and the attribute part
Next we introduce the technique for grouping 3D-spatio-temporal Objects
together.
Definition 11. The pointer VIndex realizes two connections; one between the
spatial object and its parent and the other between the spatial and the attribute
sides of an object. It also records (through a lifeinst) any changes at the status
of the object:
VIndex::= VObjectID,ParentObjectID,TableName,
KeyID,[timeinst],lifeinst
VIndex serves as a bridge between the spatio-temporal and the attribute parts
of an object (Figure 8.5). Attributes VObjectID and ParentObjectID provide the
link to the spatial part and its parent respectively. Hence ParentObjectID is the
notion that groups spatial objects together as it defines a part of relationship. On
the other hand, attributes TableName and KeyID implement the connection to
the attribute part. TableName is the name that the non-spatial part is classified
into while KeyID is its primary key value. By using a VIndex tuple we can
associate the non-spatial attributes of an object (e.g., the constructor of a desk)
to the spatial ones (the graphical display of a desk) and vice versa. Because these
two associations can change in time, the tuple is time stamped with timeinst
(wherever it is needed).
330 Isabelle Mirbel et al.
8.4 Modeling of Spatio-temporal Behavior
In the previous subsection the static part of the ISTD has been defined in terms
of simple and complex objects related to each other with spatial relations. It is
important to define ways that the objects in the world behave. We will further
use the term scenario to represent the behavioral part. A world involves a va-
riety of individual objects presented according to a set of specifications called
Scenario. In our approach a scenario consists of a set of self-standing functional
units (scenario tuples) that include: triggering events (for start and stop), pre-
sentation actions (in terms of spatio-temporal compositions) to be carried out
in the context of the scenario tuple, and related synchronization events. We will
define the behavior of objects in terms of rules that represent the behavior of
the entities that participate in the scenario.
A rule is a description of the action that an entity should carry out (actions),
as a response to a trigger (event), given that a condition is true. This scheme
can be modeled using the well known ECA (Event - Condition - Action) scheme
from active databases. An ECA rule has the form:
WHEN event G occurs
IF condition S holds
THEN trigger the action A
8.4.1 Modeling Interaction with Events
The concept of event is defined in several research areas. In the area of Active
Databases [6] an event is defined as an instantaneous happening of interest. An
event is caused by some action that happens at a specific point in time and may
be atomic or composite. In the multimedia literature, events are not uniformly
defined. In [13] events are defined as a temporal composition of objects, thus they
have a temporal duration. In addition to the temporal aspect of an event, which
is represented by a temporal instance, there are events in ISTDs that convey
spatial information. This is represented by a spatial instance. For example, an
event captures the position of visual objects at a certain point in time. Another
aspect that is also crucial is that, although the number and multitude of events
that are produced both by the user and the system may be huge, we may be
interested only in a small subset of them. Thus, events must be treated in a
different way in this context, as compared to Active Databases. We define an
event in the context of ISTDs as the occurrence of an action, which has attached
a spatial and temporal instance. Some interested human or process recognizes
the event.
In order to assist the authors in the specification of behavior, we have to
provide them with a fundamental repertoire of events. In this framework we
further classify the events into categories. The classification of the events is done
on the basis of the entity that produces the event. The categories are:
Definition 12. User interaction events are the ones generated explicitly by user
interactions. They are mainly input events as the user interacts with the system
via input devices such as mouse, keyboard, touch screen, etc.
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 331
Definition 13. The category of intra-object events includes events that are
related to the internal functionality of an object. This functionality is imple-
mented in object-oriented approaches as method invocation. For instance, the
invocation of a method corresponding to temporal access control such as my-
Object.start(), produces an intra-object event. Another source of intra-object
events is state changes. The viable states of an object as regards its presenta-
tion may be temporal (active, idle, suspended) and/or spatial (show, hidden,
layer classification information, etc.) or related to a geometric transformation
(translation/rotation/scaling). State events occur when there is a state change
of a media object, e.g., image Igets hidden, audio Ais started, etc. Intra-object
events may indicate a discontinuity in the continuous presentation of a media
object.
Definition 14. Inter-object events are events, which occur when two or more
objects are involved in the occurrence of an action of interest. These events
are raised if spatial and/or temporal relationships between two or more objects
hold. In the spatial case, an inter-object event can occur if one object, moving
spatially, meets another media object. A temporal inter-object event can occur
when the deviation between the synchronized presentations of two continuous
media objects exceeds a threshold. Moreover, we may consider spatio-temporal
inter-media synchronization events. Such events occur, for instance, when two
objects are in relative motion during a specified temporal interval (e.g., object
A approaches object B before 2 am).
Definition 15. We classify as complex the events that are defined by the devel-
oper. Developers are provided with an initial set of atomic events, namely the
ones in the aforementioned categories. In many cases, though, they need to define
complex events that are somehow composed of simple ones. Hereafter we present
in summary a model for simple and complex events, based on the event concept
and classification that has been presented above. A detailed presentation can be
found in [15].
Attributes Needed to Represent Generic Events: According to the afore-
mentioned event definition, we need the following attributes to represent a
generic event: The subject and object attributes, that are of type objectList
essentially representing the objects that caused or are affected by the event,
respectively. The attribute spatio temporal signature that takes to the spatial
and temporal instances attached to the event when it actually occurs.
As we mentioned before, it is important to provide the tools to the authors
for the definition of composite events. The composition of events has two aspects:
332 Isabelle Mirbel et al.
i) Algebraic composition is the composition of events according to algebraic
operators, adapted to the needs and features of an ISTD,
ii) Spatio-temporal composition reflects the spatial and temporal relationships
between events.
In this chapter we will elaborate on the former aspect. The reader may refer to
[15] for the latter one.
Algebraic Composition of Events: In many cases the author wants to define
specific events that relate to other existing events. We exploited some of the
operators presented in other proposals on composite events in Active Databases
[6]. We distinguish between the following cases:
Definition 16 - Disjunction.
e:== OR(e1, ..., en): This event occurs when at least one of the events e1, ..., en
occurs. For instance we may be interested in the event eoccurring when button
A(e1) or button B (e2) was pressed.
Definition 17 - Conjunction.
e::= ANY (k, e1, ..., en): This event occurs when at least any kof the events
e1, ..., en occur. The sequence of occurrence is irrelevant. For example, in an
interactive game a user proceeds to the next level when she/he is successful in
two out of three tests that generate the corresponding events e1,e2, and e3.
e::= SEQ(e1, ..., en): This event occurs when all events e1, ..., en occur in the
order appearing in the list. For example, in another interactive game the user
proceeds to the next level when she/he succeeds in three tests causing the events
e1,e2, and e3 one after the other.
e::= TIMES(n, e1): This event occurs when there are noccurrences of event
e1. This implies that other events may occur in-between the occurrences of e1.
In many cases the authors want to apply constraints related to event occur-
rences in specific temporal intervals. To facilitate this requirement we define a
set of operators that are of interest in the context of multimedia applications:
Definition 18 - Inclusion.
e::= IN(e1,t int), event eoccurs when event e1 occurs during the tempo-
ral period tint with start and end Vtime e2,e3 respectively. For example, we
might want to detect three mouse clicks in an interval of 1 sec., so that a
help window appears. If tint =(e2,e3), where e2 corresponds to the start-
ing point of a timer while e3 corresponds to the end of a timer whose dura-
tion is defined as 1 second. The desired event would then be defined as e=
IN(TIMES(3,mouse.click),t int).
Definition 19 - Negation.
e::= NOT(e1,t int): event eoccurs when e1 does not occur during the temporal
interval tint.
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 333
Definition 20 - Strictly Consecutive Events:
In some cases we are interested in whether a series of events of interest is “pure”
or mixed up with other events occurring. The event e::= SCON(e1, ..., en)is
raised when all of e1, ..., en have occurred in the order appearing in the list and
no other event occurred in between them.
Conditions: A condition is a boolean expression that is evaluated each time
a rule is triggered, and if true then the corresponding actions are fired. For
instance, in the sentence “If users clicks on object A, if the object is red, rotate the
object by 90( in the horizontal plane” the words in italics constitute the condition
to be checked. The conditions can be classified in the following categories:
•Existence conditions for objects and groups.
•Object State conditions, such as visibility, invisibility, geometric type, di-
mensions, appearance. As for time dependent media objects, we have the
additional temporal states such as “active”, “suspended”, “idle” etc.
•Group related conditions such as inclusion of an object in a group, number
or type of objects in the group etc.
•Spatial relationship conditions, checking the spatial relationship between
objects. Such can be: “above”, “below”, “near”, “far”, “distance longer than
10 m”.
•Relative motion conditions such as: object “moving” “approaching”, “goes
further”, “speed greater than 5m/sec”.
•We also consider complex conditions on formed by algebraic conjunction or
disjunction operators.
Actions: A rich set of actions is feasible in an ISTD. The interested reader may
refer for further details to [16]. The actions are classified in the following broad
categories:
Definition 21. Object related actions are related to the spatial features of an
on object such as showing/hiding an object or applying a geometric transforma-
tion. Here we have to stress that these actions refer to showing/hiding all the
constituents (children) of complex objects. We also have the actions represent-
ing the set of recognized geometric transformations like: Move, Rotate, Scale.
Each of these actions are applied for a specific time interval using a law f(t)
defining the evolution of the action in time (t). The actions are formally defined
as follows:
HideAction ::= H ideChildrenI D
ShowAction ::= P arentID, C hildrenID, C hildrenI D
MoveAction ::= MoveID,For,WithLaw
RotateAction ::= RotateI D, For, W ithLaw
ScaleAction ::= ScaleID,For, W ithLaw
ActionP arameters ::= ParamID|Params
334 Isabelle Mirbel et al.
Where:
For ::= VTimePoint,ParentID,ChildrenID,MoveID,RotateID,
ScaleID ::= Integer,
WithLaw ::= ActionP arameter s and
Param ::= Float.
Definition 21. The actions that are viewed from one or more potentially moving
viewpoints, are called camera related actions. Cameras that are located in the
local coordinate system characterized by its position and orientation handle the
viewpoints. The translation and rotation transformations that were mentioned
above can be applied to the camera also. Then the related actions are:
MoveCameraAction ::= MoveCameraID, For, WithLaw
RotateCameraAction ::= RotateCameraID, For, WithLaw
Where:
For ::= VTimePoint,MoveCameraID,
RotateCameraID ::= Integer,
WithLaw ::= ActionP arameter s and
Param ::= Float.
These actions imply that the camera moves/rotates for time interval. In many
cases we need to impose a time interval between two actions. This is achieved
with the TimeGapAction defined as:
TimeGapAction ::= Float
Also in many cases we need to include a sound in our document, thus we need
the predicate: SoundAction ::= Sound”URL”SourceID that assigns a sounds
found in a URL to the identifier ID.
Each action can be started, suspended, restarted or stopped (killed) before
its completion. Thus we need the control primitive:
Control ::= play|pause|kill|rewind
The actions can be composed to forms chains of related actions that cor-
respond to sets of grouped actions as perceived by the scenario author. The
complex actions are defined as action sequences as follows:
Sequence ::= Seq uenceID, S equenceItem, S equenceItem
SequenceItem ::= Item, Control, ActionU seAction ::= USEID
Where:
SequenceID,UseID ::= Integer,
Item ::= Float
Rule-Based Scenarios: As we have defined the constituents of the rules, we
now define the rule as constituents of the scenario, which stands for the behav-
ioral part.
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 335
Definition 22. A behavior rule is defined according to the following grammar:
<Rule>::= T rigg eringEvent, Condition, Action
TriggeringEvent ::= anysimpleorcomplexevent
Condition ::= anysimpleorcomplexevent
Action ::= anysimpleorcomplexaction
8.5 Database Support for Scenario Components
As mentioned above, components of ISTD are regarded as objects varying in 3D-
spatial and temporal dimensions. As they are complex objects, their specification
is a laborious and time-consuming procedure. By providing database support we
offer a gateway to reusability where objects can be stored and retrieved possibly
through the use of specialized criteria. In this section we specify how to provide
such support.
3D-STDM is defined under the concept of orthogonality among temporal,
spatial and spatio-temporal semantics. This is because as it is understood, in
an information system there are data that do not have any spatial context (but
might vary in time) along with strictly spatial data (that do not mutate) and
spatio-temporal data. Having kept these semantics orthogonal, we supply the
system designers with control over granularity of support and give them the
opportunity to choose where to provide temporal, spatial or spatio-temporal
support. This task is delivered by applying the following guidelines:
Temporal Support. Use VTime without VTrend and attach the valid time and/or
transaction time fields to any attributes that require valid and/or transaction
time stamping.
Spatial Support. Create a new entry of VIndex without using the timeinst at-
tribute. Acquire the 3D-spatial description of the object and link it to its at-
tribute part using the rest of the VIndex entry attributes. If the 3D-spatial de-
scription contains complex objects, use the ParentObjectID attribute to provide
links inside the spatial description graph.
Spatio-temporal Support. The change in the state of a single object is handled
by connecting the new state to its preceding one using VTrend. In detail, the
types of different changes (from an ontological perspective [9]) are dealt with as:
•Birth: Similarly to spatial support, create a new entry of VIndex,acquire
the initial spatial description and link it to the attribute part. This time
use timeinst.VLife should have birth value equal to as timeinst,deathvalue
null and no previds. Even more, attach VTime (with VTrend) to all spatial
attributes that vary in time. Depending on this variation, it may be either
a full bi-temporal or a valid time or a transaction time timestamp.
•Death: For the object’s entry in VIndex update its “life”. The death field
should acquire the appropriate VTime value.
336 Isabelle Mirbel et al.
•Reincarnation: Follow the same procedure as in Birth but this time, previd
should be the ID of the object(s) that the current object reincarnates.
•Evolution: Mark the new object’s birth, the old object’s death and update
the previd of the new object with the ID of the old one.
•Aggregation: Create a new entry in VIndex including timeinst. Set life’s birth
to the appropriate value and populate the previds field with the identifiers
of the aggregating objects. For each of the latter ones, mark their death.
•Separation: Create as many entries in VIndex as the produced objects. Set
their birth and the death of the producing object.
•Spawning: Follow the same procedure as in aggregation but without marking
the death of the spawned object.
•Mixture: Mark the death of the objects that are to be mixed (except of the
object that incorporates the mixture) and update the previds of the resulting
object.
•Fusion: Follow the same procedure as in separation but without marking the
death of the fused object.
SQL (OQL) statements can handle temporal, spatial and spatio-temporal
support. Finally we briefly introduce special statements for the definition and
the insertion of VObject and VTObject structures (see Figure 8.6).
Operator Signature
CREATE VTObject CREATE VTObject (vtobj, id, parid,table,key,
time, [attName, temSup])
CREATE VObject CREATE VObject (vobj, id, parid, table, key,)
INSERT VNEWVALUE INSERT VNEWVALUE (name, value, time, id)
INSERT ANEWVALUE INSERT ANEWVALUE (name, key, value, time)
INSERT VNEWVINDEX INSERT VNEWINDEX (objid, parid, table, key,
time, life)
Fig. 8.6. Creation and manipulation operators
8.5.1 Querying and Accessing Stored Components
Querying stored objects is delivered with 3D-STQL, which is developed as a set
of functions and predicates that can be incorporated into query statements. This
approach was preferred to that of extending SQL (or OQL) because:
•3D-STQL can be directly implemented in a number of popular platforms
such as Oracle and O2. When providing query language extensions, the use
of such platforms is not straightforward.
•It is not bound to either SQL or OQL but instead it can be used as a
supplement to both of them. The operators are formally defined hence there
is not any question of lacking a formalized foundation.
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 337
•Finally 3D-STQL, as it deals with graphical information, should be accessed
through a graphical user interface. The details of implementation should
be made transparent to the user who would ask queries in a visual, not a
textual way. Therefore, from the user’s perspective it is insignificant how the
operators relate to the query language.
Table of Figure 8.7 lists the operators of 3D-STQL organized as temporal,
spatial and spatio-temporal. Apart from the conventional spatial predicates we
introduce 3D-spatial operators whose result depends on the viewer’s position. For
the evaluation of spatio-temporal topological operators we followed the principle
that a relationship is true (or not) in a period if it is true (or not) for each time
point in that period. Otherwise the result is indefinite.
Temporal Operators VObject At(VTOb ject, VtimePoint)
VTObject Retrieve (VTObject, VTimePoint, VTimePoint)
Boolean Before (VtimePoint, VTimePoint, VTimePoint, VTimePoint)
Boolean After (VtimePoint, VTimePoint, VTimePoint, VTimePoint)
Boolean Begin (VtimePoint, VTimePoint, VTimePoint, VTimePoint)
Boolean Finish (VtimePoint, VTimePoint, VTimePoint, VTimePoint)
Boolean Overlap (VtimePoint, VTimePoint, VTimePoint, VTimePoint)
Boolean Contain (VtimePoint, VTimePoint ,VTimePoint, VTimePoint)
Boolean Equal (VtimePoint, VTimePoint ,VTimePoint, VTimePoint)
Spatial Operators
Topological real Distance((VObject, Vob ject)
realDistance((VObject, plane)
real Volume(VObject)
Boolean Equal(VObject, Vobject)
Boolean Disjoint(VObject, VObject)
Boolean Meet(VObject, Vobject)
Boolean Overlap(VObject, VObject)
Boolean In(VObject, VObject)
Viewpoint Dependent Boolean IsVisible(VObject, plane)
VBoolean Infront(VObject, VObject)
VBoolean Behind(VObject, VObject)
VBoolean Left(VObject, VObject)
VBoolean Right(Vobject, VOb ject)
VBoolean Above(VObject, VObject)
VBoolean Below(VObject, VObject)
Viewpoint Independent VBoolean Upon(Vobject, VObject)
VBoolean Beneath(VObject, VObject)
3D-spatio-temporal Operators
Topological Boolean Equal(VTObject, VTObject, VTimePoint, VTimePoint)
Boolean Disjoint(VTObject, VTObject, VTimePoint, VTimePoint)
Boolean Meet(VTObject, VTObject, VTimePoint, VTimePoint)
Boolean Overlap(VTObject, VTObject, VTimePoint, VTimePoint)
Boolean In(VTObject, VTObject, VTimePoint, VTimePoint)
Viewpoint Dependent VBoolean Infront(VTObject, VTObject, VTimePoint, VTimePoint)
VBoolean Behind(VTObject, VTObject, VTimePoint, VTimePoint)
VBoolean Left(VTObject, VTObject, VTimePoint, VTimePoint)
VBoolean Right(VTObject, VTObject, VTimePoint, VTimePoint)
VBoolean Above(VTObject, VTObject, VTimePoint, VtimePoint)
VBoolean Below(VTObject, VTObject, VTimePoint, VTimePoint)
Viewpoint Independent VBoolean Upon(VTObject, VTObject, VTimePoint, VTimePoint)
VBoolean Beneath(VTObject, VTObject, VTimePoint, VTimePoint)
Behavioral Operators Boolean Grow(VTObject, VTObject)
Boolean Shrink(VTObject, VTimePoint, VTimePoint)
VBoolean Approach(VTObject, VTimePoint, VTimePoint, VViewpoint)
VBoolean Leave(VTObject, VTimePoint, VTimePoint)
Real Differentiation(VTOb ject.(Attribute), VTimePoint, VTimePoint)
Fig. 8.7. 3D-STQL full set of operators
338 Isabelle Mirbel et al.
As our objective was to produce a universal and application-independent
set, these operators are kept generic. Nevertheless, they can be employed for the
definition of complex functions. For example given the VObject v1,v2 and v3 and
the VViewpoint vp1, spatio-temporal predicates between and upon are defined as:
•Between (v1, v2, v3, vp1): Infront(v1,v2,vp1) AND Infront(v2,v3,vp1)
•Upon (v1,v2): Over(v1,v2) AND Meet(v1,v2)
Composite spatio-temporal behavior can be expressed in a similar way as a
combination of spatio-temporal and temporal operators. For example:
•Overtake (v1, v2, vp1, t1, t2, t3, t4): Behind (v1, v2, vp1, t1, t2) AND
Infront (v1, v2, vp1, t3, t4)
8.5.2 A Global Architecture
Figure 8.8 reviews how to provide database support to the construction of ISTD.
A presentation is populated by components retrieved from a database repos-
itory with the use of specialized criteria. This way we increase reusability (a
component can be used in more than one presentations) while ensuring that the
components retrieved meet the user’s requirements. After a customization stage
which arranges components in a presentation and thus produces the scenario
(consisting of structural and behavioral elements), the latter is mapped to a
spatial or spatio-temporal data format. Conversely such data formats are used
for the population of the repository.
Spatiotemporal
Database
Repository
Selected Scenario
Structural
Components
Selected Scenario
Behavioural
Components
3-D
Spatiotemporal
Interactive
Scenario
Spatiotemporal
Data Formats
(VRML or
Java3D)
Customising
Mapping
Querying
Fig. 8.8. Database support for Interactive spatio-temporal documents
8.6 Examples of Applications
Hereafter we present a very simple example leading with a table and a lamp.
In this example we use a relational database and we show how data related to
the table and the lamp are deployed through the different relations. An object
may be built as a composition of basic ones and the table is described as a
composition between four legs (Cylinders with IDs 3, 4, 5, 6) and a plane (Box
with ID 2).
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 339
For the modeling of spatio-temporal data the following tuples of VTIndex
are used (Figure 8.9)
VTObjectID VTime TTime Parent Tab le KeyID LifeInst
ObjectID Name
12000 -1 - - -
22000 1 - - -
32000 1 - - -
42000 1 - - -
52000 1 - - -
62000 1 - - -
Fig. 8.9. VTIndex Tuples referring to the table
After that, one tuple in VGeometry is used to store the geometry of the plane
(Figure 8.10)
VGeometryID BoxID ConeID CylinderID SphereID VTObjectID
1 1 -1 -1 -1 3
Fig. 8.10. VGeometry tuple referring to the pane of the table
The geometry of the table is specified with tuples in VBox (Figure 8.11) and
in detail with tuples in VPoint table (e.g., height in Figure 8.12). All values have
a transaction timestamp.
BoxID WidthID HeightID DepthID VPointID
1 1 2 1 1
Fig. 8.11. VPoint tuples referring to the height of the table
VTime TTime Value
- 1 .3
Fig. 8.12. Timestamps for the plane of the table
The lamp is modeled with a bottom plane, a cylinder and a cone. Two
switches relate to the lamp; when the first is pressed it turns the lamp on and
rotates the lamp around the vertical axis with the rotational speed of 30 de-
grees/sec. The other switch turns off the lamp and stops the rotation.
340 Isabelle Mirbel et al.
ON CLICK (onSwitch)
IF lamp.SwitchLight = FALSE
DO
Lap.SwitchLight = TRUE
Lamp.rotate(vertical_axis,30).start()
END DO
ON CLICK (offSwitch)
IF lamp.SwitchLight = TRUE
DO
Lap.SwitchLight = FALSE
Lamp.rotate(current()).stop()
END DO
8.7 Related Work
There are several tools, which produce enormous code for simple geometry (us-
ing more complex geometries -IndexedFaceSet- to define it with thousands of
3Dcoordinates; like i.e., AC3D). In [8], SCORE, a distributed object oriented
system for 3D real time visualization is presented. It achieves a reusable design
that can be used to develop other virtual 3D buildings where spatial navigation
is required. In this work, emphasis is given to hypermedia and spatial naviga-
tions. Though the system lacks a methodology and a clear model for spatial
navigation.
The specification of 3D-spatio-temporal models can derive form works that
model animation in a 3D environment. MAM/VRS is an object-oriented ap-
proach [5] that defines object ADT for both the spatial (geometrical) and the
temporal attributes of a 3D-spatio-temporal object. Using these ADT the user
can specify interactive animations but though the flow of time is controlled there
is no relationship between the spatial and the temporal constructs.
A continuation of this work is Virtual Reality Modeling Language (VRML),
an attempt to introduce a common and standard language for advanced graphics
and virtual reality. VRML aims to be efficient enough to be used in standalone
applications and in the World Wide Web. Theoretically it can accommodate any-
thing from 3D geometry and MIDI data to JPEG images, links and simple text.
It was created in 1995 as a consortium of academic and commercial partners.
Its first version (VRML 1.0) addressed the modeling of 3D objects in an open
and extensible format. VRML consists (in its latest version) of 55 commands
or nodes. Each node specifies a geometry, an appearance, the background, the
behavior etc. VRML caters for the representation of composite ob jects organized
in a scene graph. The elements of a scene graph have individual positional prop-
erties and appearance. As a whole however, they yield to the transition or the
behavior of the scene graph. If for example an object is placed at point (2, 1, 1)
and the scene graph where it belongs is placed at (-2, -1, -1) then the absolute
position of the object is at (0, 0, 0). VRML follows an orthonormal 3D system of
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 341
coordinates. It supports time thus providing a gateway for spatio-temporal pre-
sentations. Time in VRML starts at January 1st, 1970 (although this absolute
date is rarely important) flows continuously and cannot be stopped.
Applications of VRML vary from scientific visualizations to 3D pictures for
CAD [2] to virtual presentations of stadium, city centers [12] and airports [11].
The advantages from using VRML are interoperability, Internet gateway and
easy 3D navigation.
VRML is a data format and although it provides an interface to programming
languages, the use of programming for advanced interaction it is not always
the best solution in terms of performance and security. Sun Microsystems has
developed Java3D [7], a Java API with classes to model and display 3D data.
Java3D copies the philosophy of VRML with classes that duplicate VRML nodes
and the scene graph technique. The main difference is that 3D modeling is done
with programming and the final result is a Java application or an applet (for
browsers which in the future will support Java3D.
The need to extend VRML to the direction of database support has been
acknowledged by VAG (VRML Architecture Group), the governing body for
VRML [17] which established the VRML Database Working Group [18]. Its
goals are to define a standard set of API for persistence, scalability and security.
Their fulfillment will enable VRML worlds to be stored in multi-user relational
and object databases and be the foundation for building reusable complex and
robust worlds.
Two proposals have been submitted in this direction. Adding regions defines
region node, which has an identifier and fields containing built-in topological
information (distance and visible region). VRML Data Repository [19] defines
C and Java mappings, which enable VRML to be mapped to procedural and
object databases. Although these two works are still in the development stage,
they clearly demonstrate the interest in providing a database interface to VRML.
Another proposal, which relates VRML and databases, is specified in [14]. It
focusses on the issues of reusability and sharing of VRML data. They facilitate
these objectives by specifying semantic associations of VRML files to non-spatial
information and by storing both of them in an object database. The user can
retrieve VRML worlds by querying the associated information. While this effort
provides an answer, it does not constitute a complete solution; firstly, retrieval
is delivered with non-spatial criteria while VRML objects are primarily graphi-
cal objects and secondly its reusability is limited because it is based on object
databases and excludes relational models.
In [4], the concept of virtual environment (VE) is introduced. The main
feature of this system is its navigation means, which is spatial, and the interac-
tivity facilities. VEs are described through three fundamental kinds of element:
context(s) (for structuring the environment), object(s) (which can be passive,
reactive or active and movable or non-movable), and users that can activate ob-
jects, collect or move them. On the contrary of our approach we consider only
one context and we only distinguish the avatar from the other objects. The dy-
namic features of a VE are based on the dynamic features of its elements. All
342 Isabelle Mirbel et al.
elements of a VE are described in terms of their behavior, interaction with the
other components and their dependencies on them. Collaboration between com-
ponents (user-object, object-context, user-context, user-user, context-context) is
defined through constellation’s behavior. Constellations define object instances
at runtime and the relationships among them. In our approach relationships be-
tween objects are described through reactions to events under conditions. Based
on the constellation analysis, the authors present two behavioral patterns: the
compound pattern and the collector pattern. These patterns focus on the way
of propagating the compound’s behavior to the group.
In [3] methods to create complex scenarios for the IOWA-driving simulator
are presented. A scenario specifies the behavior of synthetic vehicles, pedes-
trians, traffic control devices and variations in weather and lighting. To model
basic behaviors of vehicles and other active entities, hierarchical concurrent state
machines are used. The hierarchy of the state machine allows abstraction in be-
havior modeling; the concurrency allows attending to multiple constraints and
goals (through constraints resolution methods). Each state machine encodes a
behavior, each instance of a state machine deals with a vehicle. Diversity is
achieved by varying the attributes of the vehicle (preferred driving speed, de-
sired following distance, etc.). To create complex scenarios (i.e., scenarios that
meet experimental needs while maintaining diversity, reactivity and realism in
object behaviors), the generated sequences of events blend smoothly. In addi-
tion, behavior controller objects are provided (and implemented also through
hierarchical concurrent state machines) to allow interactions among the object
of the virtual worlds through sending of messages. A scene editor and a scenario
editor have been developed.
In [5] an interesting approach is presented for uniform modeling of geometry
and behavior of a 3D graphics and animation framework. More specifically the
authors specify separately geometry and behavior using corresponding nodes
organized in two separate directed acyclic graphs, the geometry and the behavior
graph. Moreover they provide rendering independent graphics objects that can
be visualized by geometry nodes constrained by time and event-rated behavior
nodes and finally provide high-level 3D widget nodes that combine geometry and
behavior nodes.
In [10] an integrated effort for building 3D animations is presented. The sys-
tem provides a small set of modeling primitives, namely the graphical objects,
their properties and the callbacks. The graphical objects are used for construct-
ing scenes, the properties are time-variant and are used for animating various
aspects of a scene while the call backs are the primitives used to handle inter-
action. As for the graphical objects, they basically define the geometric content
of the animation also defining groups that are essentially the scene graphs. The
properties are of several kinds and define the behavior of the objects. The inter-
action in this system is limited to user input (such as selection) and does not
cover the internal animation interaction, which may be very rich.
In [1] a language is presented for specification of virtual agents behavior. The
language presents a higher level behavioral model putting emphasis in temporal
8 Advanced Uses: Composing Interactive Spatio-temporal Documents 343
logic background enriched with qualitative spatial and temporal predicates. The
system uses Petri-nets for rendering the animation.
8.8 Conclusions
We described mechanisms for managing the structural and behavioral parts of
ISTDs. Such applications are rich in spatio-temporal features both in their struc-
ture and behavior and therefore this task was made easier by studying the work
done in the spatio-temporal databases area. We also showed how to store the
components of ISTDs in database repositories in order to reuse them for future,
customized composition of worlds. The need for 3D interactive content in the
areas of cultural, tourism and transportation application is profound. Yet cur-
rent languages do not provide database support and are not high-level enough
and easy to use for specifications.
Particular requirements for modeling the structural part of this domain are:
i) the continuous treatment of time,
ii) the inclusion of the viewer in the data model semantics,
iii) the linking to the descriptive (attribute) data.
Given these requirements we defined 3D-STDM, the metric, object-based
data model of 3D-spatio-temporal data. It follows an open format and hence
can be employed in order to store spatio-temporal components in either object-
oriented or relational databases. Indeed there is a discussion over the suitabil-
ity of the relational model to represent specialized information (such as spatio-
temporal). Nevertheless we chose not to discard the relational model because it is
still a popular database model and is supported by well built and test commercial
database servers (e.g., Oracle, Sybase etc.). The main semantics of 3D-STDM
namely, VTime,VObject and VTObject are orthogonal among themselves thus
giving the developer the opportunity to provide grades of temporal, spatial and
spatio-temporal support.
Towards modeling of the behavioral part, we have presented a model that
enables definition of interactive spatio-temporal scenarios. The fundamental fea-
tures of the language are: (i) complex 3D objects (any complex object can be
defined through a set of elementary ones connected via a set of spatial relation-
ships); (ii) ECA-like rules where E is the triggering event, C the spatio-temporal
conditions that must be fulfilled and A a set of actions connected by temporal
intervals.
The main advantages of our approach are the following:
•It provides a high level generic declarative definition of complex 3D scenarios
- therefore, suitable for data base storage and retrieval (issue of further
research);
•It covers internal and external interaction;
•It handles complex objects specification based on spatial relationships, mak-
ing thus definitions natural and easy;
344 Isabelle Mirbel et al.
•A complete coverage of all three spatial transformations (translation, rota-
tion, scaling) in 3D is provided in addition to a composition scheme (i.e., a
translation simultaneously with a rotation).
The visualization of ISTDs is currently implemented using VRML but it can
also be implemented in any visualization language.
References
1. A. Del Bimbo and E. Vicario. Specification by example of virtual agents behavior
IEEE Transactions on Visualization and Computer Graphics, 1(2), December 1995.
2. VRML Venus pictures, http://www-venus.cern.ch/VENUS/LHC_pict.htm.
3. J.F. Cremer and J.K. Kearney. Scenario authoring for virtual environment. In
Proc. Of IMAGE VII Conf., Tucson, AZ, June 12-17, 1994.
4. A. Diaz and R. Melster. Patterns for modeling behavior in virtual environment
applications. Second Workshop on Hypermedia Development: Design patterns in
hypermedia (in conjunction with Hypertext 99). Darmstadt, Germany, February
1999.
5. J. Dollner and K. Hinrichs, Object-Oriented 3D Modelling, Animation and Inter-
action. J. of Visualisation and Computer Animation, Vol. 8, p33-64, 1997.
6. N.H. Gehani, H.V. Jagadish, and O. Shmueli. Composite Event Specification in an
Active Database: Model and Implementation. In Proceed ing s of VLDB Conference,
1992, pp. 327–338.
7. Java3D Specification. http://java.sun.com/products/java-media/3D/.
8. R. Melster, A. Diaz, and B. Groth. SCORE - The virtual museum, development
of a distributed, object-oriented system for 3D real-time visualization. Technical
report 1998-15, TU Berlin, Germany. October 1998.
9. D. Medak. Lifestyles - An Algebraic Approach to Change in Identity. In
STDBM’99, Edinburgh, UK, September 1999.
10. M. Na jork and M. Brown. Obliq-3d: A high Level, Fast Turnaround 3D Animation
System. IEEE Transactions on Visualization and Computer Graphics, 1(2), June
1995.
11. Schiphol Airport in 3D, http://www.schiphol.nl/maps/3d.htm .
12. VRML world of piazza di Siena
http://www.planetitaly.com/Spazio/VRML/siena.wrl.gz .
13. G. Schloss and M. Wynblatt. Providing definition and temporal structure from
multimedia data. Multimedia Systems Journal, 3:264-277, 1995.
14. M.Kamiura, H. Oiso, K. Tajima, and K. Tanaka. Spatial Views and LOD-Based
Access Control in VRML-object Databases. Springer Lecture Notes in Computer
Science Vol. 1274.
15. M. Vazirgiannis and S. Boll. Events in Interactive Multimedia Applications: Model-
ing and Implementation Design. In Proceedings of IEEE Internat ion al Con fere nce
on Multimedia Computing and Systems (ICMCS’97), June 1997, Ottawa, Canada.
16. M. Vazirgiannis and S. Lazaridis. STEDEL: A language for interactive spatio-
temporal contexts. Technical report, Athens Univerity of Economics & Business.
17. The VRML Architecture Group http://vag.vrml.org.
18. The VRML Database Working Group
http://www.vrml.org/WorkingGroups/dbwork/ .
19. VRML Data Repository API, Oracle proposal, 1997
http://www.olab.com/vrml/overview.html.
