Conference PaperPDF Available

Ontology for Preservation of Interactive Multimedia Performances


Abstract and Figures

Preservation of interactive multimedia performances is becoming important as they are getting more and more popular in performing arts com- munities. A proper preservation does not only require keeping all the necessary components available at the time of reconstruction but also the knowledge about these components are assembled together for in performance. In digital preservation, metadata has been seen as a mean of enabling semantic process- ing, cataloguing and querying on preserved digital objects. However, as a re- cord based approach, metadata is weak at describing interrelationships between digital objects in digital archives, which is one of the key requirements for pres- ervation of interactive multimedia performances. In this paper, a domain ontol- ogy for describing the complex relationships amongst different components of a performance to support its preservation process is introduced.
Content may be subject to copyright.
Ontology for Preservation of Interactive Multimedia
Kia Ng,
Tran Vu Pham,
Bee Ong,
Alexander Mikroyannidis,
and David Giaretta
Interdisciplinary Centre for Scientific Research in Music (ICSRiM),
School of Computing & School of Music,
University of Leeds, Leeds LS2 9JT, UK
STFC, Rutherford Appleton Laboratory, Oxfordshire OX11 0QX, UK
Abstract. Preservation of interactive multimedia performances is becoming
important as they are getting more and more popular in performing arts com-
munities. A proper preservation does not only require keeping all the necessary
components available at the time of reconstruction but also the knowledge
about these components are assembled together for in performance. In digital
preservation, metadata has been seen as a mean of enabling semantic process-
ing, cataloguing and querying on preserved digital objects. However, as a re-
cord based approach, metadata is weak at describing interrelationships between
digital objects in digital archives, which is one of the key requirements for pres-
ervation of interactive multimedia performances. In this paper, a domain ontol-
ogy for describing the complex relationships amongst different components of a
performance to support its preservation process is introduced.
1 Introduction
Interactive multimedia technologies are popularly used in contemporary performing
arts, including musical compositions, installation arts, dance, etc. Typically, an Inter-
active Multimedia Performance (IMP) involves one or more performers who interact
with a computer based multimedia system making use of multimedia contents that
may be prepared as well as generated in real-time including music, manipulated
sound, animation, video, graphics, etc. The interactions between the performer(s) and
the multimedia system can be done in a wide range of different approaches, such as
body motions [20] [17], movements of traditional musical instruments, sounds gener-
ated by these instruments [27] [23], tension of body muscle using bio-feedback [18],
hart beats, sensors systems, and many others
. These “signals” from performers are
captured and processed by the multimedia systems. Depending on specific perform-
ances, the “signals” will be mapped to multimedia contents for generation using a
mapping strategy. Depicted in Fig. 1 is a typical IMP process based on the MvM
See New Interfaces for Musical Expression (NIME) conference series -
(Music via Motion) interactive performance system, in which performer’s motion is
captured in 3D and translated into multimedia contents [20].
IMPs are usually ad hoc. Manipulating multimedia content using computers is an
essential part of a live performance. Using simply performance outputs recorded in
the form of audio and video media will not be sufficient for a proper analysis (e.g. for
studying the effect of a particular performing gesture on the overall quality of the per-
formance) or reconstruction of a performance at a later time. In this context, tradi-
tional music notation as an abstract representation of a performance is also not capa-
ble of storing all the information and data required to reconstruct the performance.
Therefore, in order to keep a performance alive through time, not only its output, but
also the whole production process to create the output needs to be preserved.
Preserving the whole production process of an IMP is a challenging issue. In addi-
tion to the output multimedia contents, related digital contents such as mapping
strategies, processing software and intermediate data created during the production
process (e.g. data translated from “signals” captured) have to be preserved, together
with all the configuration, setting of the software, changes (and time), etc. The most
challenging problem is to preserve the knowledge about the logical and temporal rela-
tionships amongst individual components so that they can be properly assembled into
a performance during the reconstruction process.
Fig. 1.
An example of IMP production process using MvM (Music via Motion) system [20]
This paper introduces an ontology approach to describing an IMP and its internal
relationships to support the preservation process. The proposed ontology is an exten-
sion of the CIDOC Conceptual Reference Model (CIDOC-CRM), which is an ISO
standard for describing cultural heritage [9, 10, 14]. The next section of the paper dis-
cusses the context of requirements for the preservation of IMPs. The discussion in
Section 3 is focused on metadata approaches to digital preservation. The applicability
of CIDOC-CRM for modelling IMPs to support the preservation process and its limi-
tations are analysed in Section 4. In Section 5, the proposed extension from CIDOC-
CRM is presented.
2 Context and Requirements
Preservation of IMP is a part of the Contemporary Arts testbed dealing with preserva-
tion of artistic contents, which is one of the three testbeds of the EU project CASPAR
[2]. The other two are Scientific and Cultural testbeds which are focused on very high
volume and complex scientific data objects and virtual cultural digital objects respec-
2.1 CASPAR Project and OAIS Reference Model
The major goal of CASPAR (Cultural, Artistic and Scientific knowledge for Pres-
ervation, Access and Retrieval) is to build a pioneer preservation environment for a
wide range of digital resources of many different user communities. It is based on the
full use of the OAIS (Open Archival Information System) Reference Model [4] and
the exploitation of the latest developments in knowledge technologies. The target of
preservation in CASPAR is not only the bits of digital resources but also the informa-
tion and knowledge carried by the bits.
The very basic concept defined in the OAIS Reference Model is Information Ob-
ject. As illustrated in the UML diagram of Fig. 2, an Information Object is composed
of a Data Object and one or more layers of Representation Information. A Data Ob-
ject can be a Physical Object (e.g. a painting) or a Digital Object (e.g. a JPEG im-
. Representation Information provides the necessary details for the interpretation
of the bits contained within the digital object into meaningful information. For digital
objects, representation information can be documentation about data formats and
structures, the relationships amongst different data components. Representation in-
formation can also be software applications that are used to render or read the digital
objects. Representation information itself also needs other representation information.
The connections between different layers of representation information will form a
network which is referred to in OAIS as Representation Network.
Together with representation information for interpretation of the digital object to
be preserved, other information about the digital object i.e. Packaging Information,
Preservation Description Information and Descriptive Information that also need to be
documented are preserved. These types of information are described below:
Packaging Information tells how different components of an information package
are packed together.
Preservation Description Information consists of information about Reference In-
formation for identification of the preserved information object, Context Informa-
tion that documents the relationship between the preserved information object and
its environment, Provenance Information that documents the history of the pre-
served information object and Fixity Information for performing integrity checks
on preserved data.
In the context of CASPAR digital preservation, only digital objects are of interest.
Descriptive Information is information that can be used for cataloguing and query
on the preserved information object. It generally can be derived from the preserved
information object and its preservation description information.
In a summary, as specified in OAIS Reference Model and also adopted in
CASPAR, digital preservation is not only dealing with keeping the bits through time
but also other related information for interpretation of the bits and for positioning the
preserved information in a usage context.
Fig. 2. Basic concept of OAIS Reference Model – Information Object [4]
2.2 IMP Object for Preservation
Basically, preserving an IMP is to keep it alive through time. A performance is an ab-
stract entity. It can be sensed, but cannot be physically got hold of and stored directly.
A greedy solution is to preserve everything that is necessary for a performance. In
theory, this solution seems to be a good choice as the whole performance can be au-
thentically recreated using the preserved components. Practically, for long term pres-
ervation, this will not be feasible due to the amount of storage required for all the
hardware used in all the preserved performances. In addition, the lifetime of the
hardware and particularly human beings are limited. A more feasible approach is to
preserve the digital parts of a performance and digitalised versions of all the docu-
mentation about its production process, related equipment and instruments. The per-
formance will be recreated using the preserved digital components and digitally en-
coded knowledge with available functionally equivalent equipment and instruments at
the time of recreation.
With this approach, a performance object for preservation will consists of preserv-
able digital objects. Preserving these digital objects will follow the general conceptual
interpreted using
model defined in the OAIS Reference Model. This means, each digital object will be
accompanied by its representation information (possibly a representation network),
preservation description information (i.e. reference, context, provenance and fixity)
and descriptive information. The information accompanying each digital object is
specific to the object itself. In addition to the digital objects and their accompanying
information, the performance object also needs another layer of information about it-
self as a whole. This layer of information can be seen as representation information,
preservation description information and descriptive information for the performance
object. In particular, this information layer includes:
General information about the original performance, such as, time, data, place,
composers, directors, performers, technicians and its various activities.
Information about various activities necessary for the performance and their tem-
poral relationships. For example, it is necessary to identify what are the activities
required for a performance or the order in which these activities need to be carried
Information about roles required for the performance.
Information about necessary components (including physical and digital compo-
nents) of the performance and how these components can possibly be linked to-
gether to create the performance.
Information about activities actually performed in the original performance (the ac-
tual activities performed during a performance can be different from the necessary
activities mentioned above).
Information about people involved in the original performance and their roles.
Information about specific components used and produced in the original perform-
ance and how these different components were linked together and their usage in
different stages of the original performance. Although it is possible that there can
be similar applications and tools that can deliver the same functionality as the ap-
plications and tools were actually used in the original performance, keeping the
original tools and applications and their configuration as in the original perform-
ance is important, as “look and feel” of the tools and application is also a part of
the artistic content of a performance.
Fig. 3. IMP Object for preservation and its components
Generally, this information layer must provide enough information about the origi-
nal performance (e.g. for keep a history of the performance, for cataloguing and que-
rying, etc.) and the knowledge on how to recreate the original performance from pre-
servable components. These types of information are specific to the performance and
are referred to as Performance Knowledge. The performance object for preservation
and its components are illustrated in Fig. 3.
3 Metadata and Ontology for Digital Preservation
Metadata has been proven to be an important factor in digital preservation regardless
of preservation methods used [6]. Internally within a preservation archive, metadata
are used to describe digital objects, its internal and external relationships, history, ac-
cess rights for managing, processing, cataloguing and querying. Externally, a stan-
dardised metadata set can help to enabling semantic interoperability, which is seen as
an important requirement for digital libraries [24], amongst distributed digital ar-
Current most popular metadata used in digital archives are possibly Dublin Core
[8]. It was developed as a basic metadata set for discovery of digital records on the
Internet. It is now widely used in digital library community for interoperability. How-
ever, it is descriptive and too simple for use in a digital preservation process due to
the lack of mechanism for describing the relationship amongst digital objects. This is
one of the key requirements for digital preservation, such as in the case of preserving
IMPs as described in this paper.
Metadata element sets designed specifically for preservation purposes includes
those developed by RLG Working Group on Preservation Issues of Metadata (RLG)
[1], CURL Exemplars in Digital Archives (CEDARS) project [3, 7], the metadata of
the National Library of Australia (NLA) [19] and the Networked European Deposit
Library (NEDLIB) [16]. The RLG elements are for describing structure and manage-
ment of digital image files in preservation archives. The scope of applicability of
RLG elements is therefore limited. It also lacks the capability to describe the relation-
ships amongst related digital objects. CEDARS, NLA and NEDLIB element sets are
Performance Object
Performance Knowledge
more comprehensive and also compliant to OAIS Reference Model. A comparative
analysis has shown that these three elements sets have a high level of convergence in
their objectives and rationales [22]. In addition to supporting the OAIS Reference
Model, these three sets are aimed at supporting the management of archived digital
objects and can be applicable to a broad range of digital contents. CEDARS and NLA
also have elements for describing the relationships amongst digital objects. Despite
their convergence, as using different set of vocabulary, interoperability between these
metadata sets is a problem. A consensus effort was carried out by the OCLC/RLG
Working Group on Preservation Metadata to develop a common Metadata Framework
to Support the Preservation of Digital Objects, which was based largely on CEDARS,
NEDLIB and NLA element sets [21]. The Preservation Metadata Implementation
Strategies (PREMIS) Working Group later built on this framework a PREMIS data
model and a data dictionary for preservation metadata [25].
Different from its predecessors, the PREMIS data model explicitly defines five ba-
sic types of entities involved in digital preservation activities: Intellectual Entities,
Objects, Events, Rights and Agents. Vocabulary for describing properties (e.g. entity
types, identifiers, characteristics, etc.) of these entities and their interrelationships are
defined in the PREMIS data dictionary. By explicitly separating the different types of
entities involved, it is easier to describe and manage the relationships amongst the en-
tities. However, as using record based approach, the PREMIS is still weak at describ-
ing relationships amongst entities, particularly the temporal relationships. In addition,
extensibility of this core element set for domain specific requirements can also be an
issue, due to its capability of dealing with inheritance.
Relationships and dependencies can be better described in an object oriented way
using ontologies. In the Fedora project, a small simple set of relationship ontologies
was used together with Dublin Core [12]. An ontology has been developed based on
MPEG-7 standard [15] for describing structure and conceptual related aspects of au-
dio-visual contents [26]. The CIDOC Conceptual Reference Model (CRM) is being
proposed as a standard ontology for enabling interoperability amongst digital libraries
[9, 10, 14]. CIDOC-CRM defines a core set concepts for physical as well as temporal
entities. This is very important for describing temporal dependencies amongst differ-
ent objects in a preservation archive. A combination of core concepts defined in
CIDOC-CRM and multimedia content specific concepts of MPEG-7 for describing
multimedia objects in museums has also been introduced. A harmonisation effort has
also been carried out to align the Functional Requirements for Bibliographic Records
(FRBR) [13] to CIDOC-CRM for describing artistic contents. The result is an object
oriented version of FRBR, named FRBRoo [11].
4 CIDOC-CRM for Digital Preservation
Fig. 4. The meta-schema of CIDOC-CRM [9]
CIDOC-CRM was originally designed for describing cultural heritage collections in
museum archives. In museums, CIDOC-CRM is used to describe things and events
from the past. Similarly, in preservation domain, today’s things and events are docu-
mented for the future. Due to this similarity and its wide coverage, CIDOC-CRM has
attracted attention from preservation communities as core ontology for enabling se-
mantic interoperability amongst digital archives.
The meta-schema of CIDOC-CRM is illustrated in Fig. 4. CIDOC-CRM’s concep-
tualisation of the past is centred on Temporal Entities (e.g. events). People (Actors)
and objects (Conceptual Objects and Physical Objects) involved, time (Time-Spans)
and Places are documented via their relationships with the Temporal Entities. Appel-
lations and Types are generally used for identification and classification. For example,
an MvM performance can be described as an event. Participants of the performance
are Actors, tools and instruments used in the performances are Physical Objects, etc.
The mapping of a performance to the meta-schema of CIDOC-CRM is shown in Fig.
5. Appellation and Types are omitted in this diagram. An MvM performance can be
described in more details using entities and properties defined for CIDOC-CRM, as
shown in Fig. 6.
Conceptual Objects
Physical Objects
Temporal Entities
Places Time Spans
ferred/ refine
participate in
within at
refer to
has location
refer to/ identify
Fig. 5. Mapping an MvM performance details to CIDOC-CRM meta-schema
The CIDOC-CRM vocabulary can be used to describe a performance at a high
level. However, more specialised vocabularies are necessary for the interactive per-
forming art domain to precisely describe the relationship amongst the elements of a
performance. For example, it is necessary to model how equipments are connected
together in the performance. The CIDOC-CRM currently also lacks of concepts for
digital objects, which are the target of digital preservation. The relationships amongst
software applications, data, and operating systems need to be documented. Further-
more, CIDOC-CRM is designed primarily for documentation of what has happened,
whereas in digital preservation, it is also required to document the reconstruction of a
past event from preserved components.
Conceptual Objects
Physical Objects
Temporal Entities
Places Time Spans
participate in
within at
refer to
has location
Instruments (e.g. vi
Equipment (e.g. mixer)
Music score
12 Feb 2007:
Leeds, UK
Fig. 6. Simple description of an MvM performance using CIDOC-CRM vocabulary: CIDOC-
CRM entity names are underlined and bold; instances of the entities are listed in below the en-
tity names.
5 Extending CIDOC-CRM for Preservation of IMPs
The CIDOC-CRM ontology is being extended for describing IMP in digital preserva-
tion context. The extended ontology is domain specific. It does not alter any concepts
defined in CIDOC-CRM. Instead, the extended ontology is a specialisation of
CIDOC-CRM concepts for IMP and digital preservation domain. Specifically, the ex-
tension is developed for the following objectives:
To provide a domain specific vocabulary for describing an IMP. The description
includes details on how the archived performance was carried out and how possi-
bly it can be recreated.
To provide vocabulary for describing digital objects, their interrelationships and
operations performed on them in the digital preservation context.
The extension is aimed specially at describing performance knowledge to be archived
with performance objects as discussed in Section 2.2 and illustrated in Fig. 3. It is cur-
rently built on and compatible with CIDOC-CRM version 4.2.1, the latest release
from CIDOC-CRM Special Interest Group [5].
5.1 Describing Performances
The scope of description is the information about the performance, its various activi-
ties, actors, equipment, instruments and digital objects involved in the performance.
Configuration – how different elements are associated with each other – is also of par-
MvM Performance
Kia (Director)
Frank (Performer)
Leeds - UK
E22.Man-Made Object
Sound Mixer
Computer System
E73.Information Object
Music Score
P16F – use specific
P16F – use specific
took place at
has time span
P14F – carried out by
ticular importance. The centre of the description is the “IMP1.Performance” object
“IMP1.Performance” is a specialisation of CIDOC-CRM “E7.Activity” and also of
“E63.Beginning of Existence”. This means that a performance is an activity and it
brings something into existence. Entities and properties for describing the relation-
ships between a performance and its components are shown in Fig. 7.
Fig. 7. “IMP1.Performance” entity and its components: new concepts are attached with the
names (in brackets) of their direct parent from CIDOC-CRM.
In comparison with the description using CIDOC-CRM as shown in Fig. 6, the fol-
lowing concepts have been introduced, in addition to “IMP1.Performance”:
IMP2.Performance Activity”: for describing activities of a performance. For ex-
ample, in an MvM performance, “capturing of 3D motion data” can be modelled as
an activity. “IMP2.Performance Activity” is designed as a specialisation (subclass)
of “IMP1.Performance” as any activity within a performance can be seen as a per-
formance itself. The temporal order in which performance activities were carried
can be modelled using properties inherited from “E7.Activity”.
IMP5.Instrument”: a specialisation of CIDOC-CRM “E22.Man-Made Object”
for modelling musical instruments (e.g. cellos, violins, drums, etc.) used in a per-
IMP6.Equipment”: a specialisation of CIDOC-CRM “E22.Man-Made Object”
for modelling equipment used in a performance. Equipment can be a microphone, a
sound mixer or a computer, etc. Computer related equipment is further classified
into “IMP7.Computer_Hardware” in the extended model to better describe their re-
lationships with computer software.
The extended concepts are prefixed by IMP and an identification number. The original
CIDOC-CRM entity and property names are prefixed by E and P respectively.
Digital Object
sisted of
used specific o
was realisa
used digital o
used instrument
used equipment
carried out by
E63.Beginning of Exi
Made O
Made O
Design or Proc
has time
took place at
IMP8.Performance Procedure”: a specialisation of CIDOC-CRM “E29.Design
or Procedure”, which is a subclass of “E73.Information Object” for describing the
procedure in which a performance should be carried out. A performance procedure
may consist of other performance procedures. Similar to performance activities, the
procedures need to be executed in a particular temporal order in order to achieve
the desired performance. However, as CIDOC-CRM is aimed at describing what
has happened, whereas performance procedures usually tell what is supposed to
happen, CIDOC-CRM does not have precise vocabulary for describing temporal
order of execution of performance procedures. New properties have been intro-
duced for “IMP8.Performance Procedure” to deal with this requirement.
IMP12.Digital Object”: a specialisation of “E73.Information Object”.
“IMP12.Digital Object” and its specialisations are discussed in detail in Section
Configuration of a performance, e.g. assignment of actors to roles, connections of
tools, or association between data and processing applications, can be described using
“IMP29.Performance Attribute Association” and its subclasses: “IMP3.Data Applica-
tion Association”, “IMP4.Actor Role Association” and “IMP30.Equipment Pairing”.
“IMP29.Performance Attribute Association” is a specialisation of “E7.Activity”.
5.2 Describing Digital Objects
Digital objects are not well covered by CIDOC-CRM. In essence, a digital object is a
stream of bits. This bits once interpreted carry some kind of information, e.g. a text
document, a picture or a set of instruction. Strictly under CIDOC-CRM definition, the
stream of bits is an information object (“E73.Information Object”). However, by it-
self, it does not have much meaning to a human being. Only the information it carries,
once interpreted by a computer, is understandable. From this point of view, a digital
object is an information carrier, which is a characteristic of physical objects under
CIDOC-CRM definition. In this extension, digital objects are classified under
“IMP12.Digital Object”, which is a subclass of “E73.Information Object”. Informa-
tion carrying characteristic of digital objects are modelled using properties of
“IMP12.Digital Object” and its specialisations. Hierarchical structure of
“IMP12.Digital Object” and its subclasses is shown in Fig. 8.
“IMP12.Digital Object” has two direct subclasses: “IMP17.Digital Data Container”
and “IMP18.Digital Data Object”. A digital data container (“IMP17.Digital Data Con-
tainer”) is a container of one or more digital data objects (“IMP18.Digital Data Ob-
ject”). An example of digital data container is a file. The bit stream contained within
the file is considered as a digital data object. This separation is necessary to model a
bit stream in memory or in cases where multiple bit streams carrying different infor-
mation carried by a single digital data container. A special type of digital data object
is a computer program (IMP13.Computer Program). In this case, the bit stream is a set
of instructions to be executed by a computer. There are two specialisations of com-
puter programs: “IMP14.Operating System” and “IMP15.Software Application”.
Relationships between digital objects are modelled through their types. An exam-
ple of such a relationship is between software applications and operating systems.
Each class under “IMP12.Digital Object” has a type, which is specified under
CIDOC-CRM “E55.Type”, associated with it. With the above example, suppose that
there is an operating system type “Windows” and a software application type “Micro-
soft Office”, and it is described that software application type “Microsoft Office” “can
run on” operating system type “Windows” then it can be inferred that any application
having type “Microsoft Office” can run on operating system having type “Windows”.
Modelling the relationships between digital objects using type will reduce the com-
plexity when handling individual instances.
Fig. 8. Classification of digital objects
Operations on digital objects can be described using “IMP26.Digital Object Opera-
tion”, which is a specialisation of CIDOC-CRM “E5.Event”. A number of subclasses
of “IMP26.Digital Object Operation” have also been defined to deal with common
operations such as creation, duplication, transformation, modification, access and de-
letion. This is necessary in preservation context, where history of a digital object
(provenance information) needs to be documented.
6 Conclusion
This paper introduces characteristics of IMPs and their requirements for digital pres-
ervation. It notes that metadata and ontology play a very important in digital preserva-
tion process, particular for describing representation information, provenance and
other information related to a digital object as defined by the OAIS Reference Model.
The limitation of metadata approach is that it views an object as a record with a set of
attributes. This makes metadata inefficient for describing the inter-dependencies that
exist amongst digital objects in an archive, particularly in the case of IMPs. An ontol-
ogy approach has been considered, by extending the current concepts defined in
CIDOC-CRM, for preservation of IMPs. A number of concepts for performing art
domain as well as for digital objects have been proposed. This extension is designed
Digital Object
Digital Data Container
Digital Data Object
Operating System
Software Application
Computer Program
specifically for describing an IMP and its internal relationships. It can also be used
together with FRBRoo if there is the need for describing the conception process of the
performance (the process of translating the initial idea of the composer into a per-
formance plan).The new concepts are evaluated using performance data available at
ICSRiM and further in the contemporary arts testbed of the EC IST CAPSAR project
[2]. As a nature of ontology development, it is necessary that the newly proposed
concepts need to be examined and verified by relevant communities for acceptance
and for successful implementations in the future.
Acknowledgements. Work partially supported by European Community under the
Information Society Technologies (IST) programme of the 6th FP for RTD - project
CASPAR. The authors are solely responsible for the content of this paper. It does not
represent the opinion of the European Community, and the European Community is
not responsible for any use that might be made of data appearing therein.
1. Berger, B., et al.: RLG Working Group on Preservation Issues of Metadata - Final Report.
1998, RLG and Preservation.
2. CASPAR. Cultural, Artistic and Scientific knowledge for Preservation, Access and Re-
3. CEDARS. Curl Exemplars in Digital Archives.
4. Consultative Committee for Space Data Systems. Reference Model for An Open Archival
Information System. 2002.
5. Crofts, N., et al.: Definition of the CIDOC Conceptual Reference Model. 2006,
ICOM/CIDOC Documentation Standards Group and CIDOC CRM Special Interest Group.
6. Day, M.: Metadata for digital preservation: an update, in Ariadne. 1999.
7. Day, M.: Metadata for Preservation - CEDARS Project Document AIW01. 1998, UKOLN.
8. DC. Dublin Core Metadata Initiative.
9. Doerr, M.: The CIDOC CRM - an Ontological Approach to Semantic Interoperability of
Metadata. AI Magazine, 2003. 24(3).
10. Doerr, M.: Increasing the Power of Semantic Interoperability for the European Library, in
ERCIM News. 2006.
11. Doerr, M. and P. LeBoeuf. FRBRoo Introduction. 2006.
12. Fedora. Overview: The Fedora Digital Object Model. 2006.
13. FRBR. Functional Requirements for Bibliographic Records - Final Report. 1997, Interna-
tional Federation of Library Associations and Institutions (IFLA): Frankfurt am Main, Ger-
14. Gill, T.: Building semantic bridges between museums, libraries and archives: The CIDOC
Conceptual Reference Model. First Monday, 2004. 9(5).
15. ISO. Information Technology – Multimedia Content Description Interface (MPEG-7).
Standard No. ISO/IEC 15938:2001. 2001, International Organization for Standardiza-
16. Lupovici, C. and J. Masanes. Metadata for long term-preservation. 2000, NEDLIB.
17. Morales-Manzanares, R., et al.: SICIB: An Interactive Music Composition System Using
Body Movements. Computer Music Journal, 2001. 25(2): p. 25-36.
18. Nagashima, Y.: Bio-Sensing Systems and Bio-Feedback Systems for Interactive Media
Arts. in 2003 Conference on New Interfaces for Musical Expression (NIME-03). 2003.
Montreal, Canada.
19. National Library of Australia. Preservation Metadata for Digital Collections. 1999.
20. Ng, K.C.: Music via Motion: Transdomain Mapping of Motion and Sound for Interactive
Performances. Proceedings of the IEEE, 2004. 92(4).
21. OCLC/RLG Working Group on Preservation Metadata. Preservation Metadata and the
OAIS Information Model - A metadata framework to support the Preservation of digital
objects. 2002.
22. OCLC/RLG Working Group on Preservation Metadata. Preservation metadata for digital
objects: A review of the state of the art. 2001, OCLC/RLG.
23. Overholt, D.: The Overtone Violin. in International Conference on New Interfaces for Mu-
sical Expression. 2005. Vancouver, BC, Canada.
24. Patel, M., et al.: Semantic Interoperability in Digital Library Systems. 2005, UKOLN.
25. PREMIS. Data Dictionary for Preservation Metadata - Final Report of the PREMIS Work-
ing Group. 2005, OCLC and RLG.
26. Troncy, R.: Integrating Structure and Semantics into Audio-Visual Documents. in the 2nd
International Semantic Web Conference. 2003. Florida, USA.
27. Young, D., P. Nunn, and A. Vassiliev. Composing for Hyperbow: A Collaboration between
MIT and the Royal Academy of Music. in proceedings of the International Conference on
New Interfaces for Musical Expression (NIME2006). 2006. Paris, France.
... In the context of the CASPAR project, we have adopted an ontology-driven approach [2][3][4] that reuses and extends existing standards, such as the CIDOC Conceptual Reference Model (CIDOC-CRM) [5,6]. The present paper introduces an ontology layering architecture for the preservation of IMPs. ...
Conference Paper
The need to address the fragility of the digital information collected in artistic activities is urgently emerging and increasing. This need is particularly heightened by new multimedia technologies that support the artistic community with many different forms of digital means. The present work concerns an ontology layering architecture for organizing and maintaining a semantic knowledge base for digital preservation of interactive multimedia performances. In addition, the use of certain ISO standardized ontologies is demonstrated in modelling the domain of interactive multimedia performances.
... Another important aspect that needs to be preserved is also the comments and feedbacks that are generated from the users or performer during the production of an IMP and regard the quality of the performance and the used techniques. In the context of the CASPAR project, we have adopted an ontology-driven approach131415 that reuses and extends existing standards, such as the CIDOC Conceptual Reference Model (CIDOC-CRM) [16, 17] for the efficient preservation of an IMP. ...
Conference Paper
Full-text available
Interactive multimedia and human-computer interaction technologies are effecting and contributing towards a wide range of developments in all subject areas including contemporary performing arts. These include augmented instruments for interactive music performance, installation arts and technology-enhanced learning. Consequently, the preservation of interactive multimedia systems and performances is becoming important to ensure future re-performances as well as preserving the artistic style and heritage of the art form. This paper presents two interactive multimedia projects for technology-enhanced learning, and discusses their preservation issues with an approach that is currently being developed by the CASPAR EC IST project.
Conference Paper
Interactive Multimedia Installations (IMIs) are invaluable for the history of art and culture, for media technology and for other disciplines involved in the scientific research and the areas of application. Despite their importance, Interactive Multimedia Installations show some weaknesses that threaten their chance to be preserved long enough to be studied or replicated: (1) they show a complexity that cannot be oversimplified; (2) they are highly refractory to preservation. Moreover, they show an alarmingly short life expectancy compared to other cultural materials, such as audio-video carriers, let alone paper documents, paintings and sculptures. This paper presents the main problems related to the preservation of Interactive Multimedia Installations, and proposes the core of a possible solution. © Institute for Computer Sciences, Social Informatics and Telecommunications Engineering 2013.
Conference Paper
Full-text available
This paper discusses interactive multimedia framework and tools that have been developed under the i-Maestro EC IST project ( for technology-enhanced music education, and focuses on one of the applications which explores 3D motion capture data for real-time analysis and feedback to support the learning processes. The multi-modal data (3D motion, sensor, audio and video) captured with the system offers a more complete and more detailed description of a musical performance. Another project, CASPAR (Cultural, Artistic and Scientific knowledge for Preservation, Access and Retrieval - which is developing a framework for digital preservation uses the multi-modal data for the contemporary arts ?est-bed in order to offer musicologist of the future a better records and description of the playing style and interpretation that may not be easily extracted/understood from a video recording.
Conference Paper
The digital re-recording of analogue material, in particular phonographic discs, can be carried out using different approaches: mechanical, electro-mechanical, opto-mechanical, and opto-digital. In this paper, we investigate the differences among these approaches, using two novel methods that have been developed on purpose: a system for synthesizing audio signals from still images of phonographic discs and a tool for the automatic alignment of audio signals. The methods have been applied to two case studies, taken from a shellac disc. Results point out that this combined approach can be used as an effective tool for the preservation of and access to the audio documents.
Full-text available
This paper presents a framework called Music via Motion (MvM) designed for the transdomain mapping between physical movements of the performer(s) and multimedia events, translating activities from one creative domain to another-for example, from physical gesture to audio output. With a brief background of this domain and prototype designs, the paper describes a number of inter- and multidisciplinary collaborative works for interactive multimedia performances. These include a virtual musical instrument interface, exploring video-based tracking technology to provide an intuitive and nonintrusive musical interface, and sensor-based augmented instrument designs. The paper also describes a distributed multimedia-mapping server which allows multiplatform and multisensory integrations and presents a sample application which integrates a real-time face tracking system. Ongoing developments and plausible future explorations on stage augmentation with virtual and augmented realities as well as gesture analysis on the correlations of musical gesture and physical gesture are also discussed.
Full-text available
This paper presents the methodology that has been successfully employed over the past 7 years by an interdisciplinary team to create the CIDOC Conceptual Reference Model (CRM), a high-level ontology to enable information integration for cultural heritage data and their correlation with library and archive information. The CIDOC CRM is now in the process to become an ISO standard. This paper justifies in detail the methodology and design by functional requirements and gives examples of its contents. The CIDOC CRM analyses the common conceptualizations behind data and metadata structures to support data transformation, mediation and merging. It is argued that such ontologies are property-centric, in contrast to terminological systems, and should be built with different methodologies. It is demonstrated that ontological and epistemological arguments are equally important for an effective design, in particular when dealing with knowledge from the past in any domain. It is assumed that the presented methodology and the upper level of the ontology are applicable in a far wider domain.
Full-text available
Traditionally, music and dance have been comple-mentary arts. However, their integration has not always been entirely satisfactory. In general, a dancer must conform movements to a predefined piece of music, leaving very little room for impro-visational creativity. In this article, a system called SICIB—capable of music composition, improvisa-tion, and performance using body movements—is described. SICIB uses data from sensors attached to dancers and "if-then" rules to couple choreo-graphic gestures with music. The article describes the choreographic elements considered by the sys-tem (such as position, velocity, acceleration, curva-ture, and torsion of movements, jumps, etc.), as well as the musical elements that can be affected by them (e.g., intensity, tone, music sequences, etc.) through two different music composition sys-tems: Escamol and Aura. The choreographic infor-mation obtained from the sensors, the musical capabilities of the music composition systems, and a simple rule-based coupling mechanism offers good opportunities for interaction between chore-ographers and composers. The architecture of SICIB, which allows real-time performance, is also described. SICIB has been used by three different composers and a cho-reographer with very encouraging results. In par-ticular, the dancer has been involved in music dia-logues with live performance musicians. Our experiences with the development of SICIB and our own insights into the relationship that new technologies offer to choreographers and dancers are also discussed.
Conference Paper
Full-text available
Describing audio-visual documents amounts to consider documentary aspects (the structure) as well as conceptual aspects (the content). In this paper, we propose an architecture which describes formally the content of the videos and which constrains the structure of their descriptions. This work is based on languages and technologies underlying the Semantic Web and in particular ontologies. Therefore, we propose to combine emerging Web standards, namely MPEG-7/XML Schema for the structural part and OWL/RDF for the knowledge part of the description. Finally, our work offers reasoning support on both aspects when querying a database of videos.
Conference Paper
Full-text available
In this paper we present progress of an ongoing collaboration between researchers at the MIT Media Laboratory and the Royal Academy of Music (RAM). The aim of this project is to further explore the expressive musical potential of the Hyperbow, a custom music controller first designed for use in violin performance. Through the creation of new repertoire, we hope to stimulate the evolution of this interface, advancing its usability and refining its capabilities. In preparation for this work, the Hyperbow system has been adapted for cello (acoustic and electric) performance. The structure of our collaboration is described, and two of the pieces currently in progress are presented. Feedback from the performers is also discussed, as well as future plans.
Conference Paper
This is a report of research and some experimental applications of human-computer interaction in multi-media performing arts. The human performer and the computer systems perform computer graphic and computer music interactively in real-time. In general, many sensors are used for the interactive communication as interfaces, and the performer receives the output of the system via graphics, sounds and physical reactions of interfaces like musical instruments. I have produced many types of interfaces, not only with physical/electrical sensors but also with biological/physiological sensors. This paper is intended as an investigation of some special approaches: (1) 16-channel electromyogram sensor called "MiniBioMuse-III" and its application work called "BioCosmicStorm-II" performed in Paris, Kassel and Hamburg in 2001, (2) sensing/reacting with "breathing" in performing arts, (3) 8-channel electric-feedback system and its experiments of "body-hearing sounds" and "body-listening to music".
Conference Paper
CASPAR (Cultural, Artistic, and Scientific knowledge for Preservation, Access, and Retrieval) aims at providing secure, reliable and cost-effective access for digitally encoded information for an indefinite time frame. From digital pictures to museum archives, from music to scientific data sets, technology is constantly changing. CASPAR is being developed to address this situation, and provide people and organisations with a user-oriented preservation framework.
This article presents the methodology that has been successfully used over the past seven years by an interdisciplinary team to create the International Committee for Documentation of the International Council of Museums (CIDOC) CONCEPTUAL REFERENCE MODEL (CRM), a high-level ontology to enable information integration for cultural heritage data and their correlation with library and archive information. The CIDOC CRM is now in the process to become an International Organization for Standardization (ISO) standard. This article justifies in detail the methodology and design by functional requirements and gives examples of its contents. The CIDOC CRM analyzes the common conceptualizations behind data and metadata structures to support data transformation, mediation, and merging. It is argued that such ontologies are property-centric, in contrast to terminological systems, and should be built with different methodologies. It is demonstrated that ontological and epistemological arguments are equally important for an effective design, in particular when dealing with knowledge from the past in any domain. It is assumed that the presented methodology and the upper level of the ontology are applicable in a far wider domain.