Applying ontology-based rules to conceptual modeling: a reflection on modeling decision making.

Full text

Page 1

Applying ontology-based rules to conceptual
modeling: a reflection on modeling decision
making
Pnina Soffer1and
Irit Hadar1
1MIS Department, University of Haifa, Israel
Correspondence:
Pnina Soffer, MIS Department, University of
Haifa, Carmel Mountain 31905, Haifa, Israel.
Tel: þ972-48288506;
Fax: þ972-48288522;
E-mail: spnina@is.haifa.ac.il
Received: 29 December 2005
Revised: 7 August 2006
2nd Revision: 29 January 2007
3rd Revision: 16 May 2007
Accepted: 22 July 2007
Abstract
Conceptual modeling represents a domain independently of implementation
considerations for purposes of understanding the problem at hand and
communicating about it. However, different people may construct different
models given the same domain. Variations among correct models, while
known and familiar in practice, have hardly been investigated in the literature.
Their roots are in the decisions made during the modeling process, where
modelers face the need to map reality into modeling constructs. This paper
reports an empirical study whose aim is to explore model variations and in
particular to examine possible directions for reducing them. Specifically, the
study uses a multimethod research paradigm to examine the effect of applying
ontology-based modeling rules on modeling decisions as reflected in resulting
model variations. The findings of the study provide insights into the variations
phenomenon, as well as to the application of ontology-based modeling rules.
European Journal of Information Systems (2007) 16, 599–611.
doi:10.1057/palgrave.ejis.3000683
Keywords: conceptual modeling; ontology; variations; empirical study
Introduction
Conceptual modeling is aimed at reflecting the real world independently
of implementation technology and constraints (Topi & Ramesh, 2002). It
has an important role in defining, analyzing, and communicating about
the requirements for the system to be. Nevertheless, it has been observed
that different people may present different models given the same domain
(Hadar & Soffer, 2006). We term the differences in constructs and relations
between adequately constructed models (see Schuette & Rotthowe, 1998)
model variations.
While model variations are well known to exist and are familiar to
anyone who ever experienced conceptual modeling, it seems that their real
essence has been overlooked so far. Model variations are the result of
different decision paths taken by different modelers through the modeling
process. Since conceptual models are used for understanding and
communication, consistency of these models is of importance. In addition,
when attempting to reuse or to match conceptual models, variations
might reduce the chances of identifying adequate matches (Soffer & Hadar,
2003). Investigating model variations as a reflection of modeling decisions
can lead to an understanding of the modeling process and its embedded
decision making, as well as to possible directions for achieving a higher
uniformity of models.
European Journal of Information Systems (2007) 16, 599–611
& 2007 Operational Research Society Ltd. All rights reserved 0960-085X/07 $30.00
www.palgrave-journals.com/ejis

Page 2

When looking at adequately constructed although
different models, the variations may either reflect
different perceptions of the modeled domain or incon-
clusiveness in the decision making during the modeling
process. In particular, these decisions relate to the
representation of the real world by modeling constructs.
In this paper, model variations are empirically investi-
gated for two main purposes. We wish to examine
possible directions for reducing model variations, and
to gain further understanding of the modeling process
and its decisions. Our underlying assumption is that
model variations can be reduced if better guidance is
given to support transforming information about the real
world into modeling constructs. Such guidance should be
consistent and theoretically anchored. The remaining
variations are expected to be the ones that truly reflect
different perceptions of the domain. These differences
should be addressed when developing an information
system (IS) for that domain.
Several theoretical frameworks for conceptual model-
ing have been suggested in recent years. The strength of
these frameworks is in their explicit guidelines, which
should increase conclusiveness in the modeling decision
making. Hence, applying them can be expected to reduce
variations among models of different modelers. These
frameworks rely on different kinds of theoretical founda-
tions, such as ontology (e.g., Evermann & Wand (2001,
2004, 2005) and Guizzardi et al. (2002, 2004)), speech–act
theory (Agerfalk & Eriksson, 2004), and classification
theory (Parsons, 1996). In particular, ontology-based
frameworks provide clear modeling rules to be applied
when using a specific modeling grammar for conceptual
modeling. Ontologies, as models of the real world, have
been applied for evaluating the expressive power of
modeling grammars (Wand & Weber, 1993) and as a
basis for analyzing modeling constructs and their repre-
sentation of real-world phenomena (Bodart et al., 2001;
Opdahl et al., 2001). Evermann & Wand (2001, 2004,
2005) and Guizzardi et al. (2002, 2004), each relying on a
different ontology, suggested rules for conceptual model-
ing using UML Class Diagram.
In this research, we empirically investigate how apply-
ing an ontology-based set of modeling rules affects model
variations, and in particular, whether such rules can
contribute to the reduction of model variations. The
study reported here examined model variations incurred
with and without applying modeling rules, while explor-
ing the considerations and cognitive processes taking
place when making modeling decisions.
The remainder of the paper is organized as follows: the
next section presents theoretical background including a
framework that explains model variations, a general
introduction to ontology-based modeling rules, and the
research questions. The following sections describe the
methodology and settings of the empirical study and
present its findings. We then discuss these findings and
present our conclusions as well as future research
directions.
Theoretical background and research framework
Ontology-based modeling rules
Ontology-based modeling rules rely on an ontology,
which is a model of the world. The fundamental premise
is that in order to fully represent the world in a
conceptual model, an ontological meaning should be
assigned to the modeling constructs. The use of con-
structs without distinct ontological meaning may lead
to an ontologically meaningless or ambiguous model, or
to multiple model representations of the world.
Hadar & Soffer (2006) indicate that although different
ontologies yield different modeling rules (e.g., Evermann
& Wand, 2001, 2004, 2005; Guizzardi et al., 2002, 2004),
the availability of such rules to rely upon may assist the
decision making in the modeling process. The ontological
framework used in this paper is the Evermann & Wand
(2001, 2004, 2005) framework. Comparing this frame-
work with the one by Guizzardi et al. (2002, 2004), Hadar
& Soffer (2006) found that it provides a broader and easier
to apply set of rules for the purpose of reducing model
variations. The Evermann & Wand (2001, 2004, 2005)
framework is based on Bunge’s (1977, 1979) ontology as
adapted for IS modeling (Wand & Weber, 1993, 2002) and
follows the notion of ontological expressiveness (Wand &
Weber, 1993). Their work analyzes the constructs of UML
Class Diagrams, State Charts, Collaboration, and Se-
quence diagrams, and provides rules that are intended
to assure a distinct ontological meaning of these
constructs. The rules include representation rules, which
define a mapping from the ontological constructs to the
modeling constructs, and interpretation rules, which
map in the opposite direction. In this paper, we relate
to a subset of the rules addressing some of the constructs
of class diagrams, as will be explained in the following
section.
Note that different works have relied on Bunge’s
ontology in order to provide guidance to constructing
UML models (e.g., Burton-Jones & Meso, 2002; Parsons &
Cole, 2004). Nevertheless, these works address specific
constructs and do not provide an overall set of modeling
rules. The Evermann and Wand rules were further
investigated by Lu and Parsons (2005), who tried
to validate them by developing a CASE tool that
incorporates them. Their findings indicate the existence
of some redundancies and inconsistencies with respect
to the entire rule set. These findings justify our
motivation to apply only a subset of the rule base in
our study.
Previous empirical studies addressing ontology-based
modeling guidance related to the effect of such frame-
works on the understanding of models. For example,
Burton-Jones & Meso (2002), Parsons & Cole (2004), and
Poels et al. (2005) evaluated the understanding of models
produced both in compliance and not in compliance
to ontology-based modeling rules. The effect of this
guidance on model construction has not, to the best of
our knowledge, been empirically investigated so far.
Applying ontology-based rules to conceptual modeling
Pnina Soffer and Irit Hadar
600
European Journal of Information Systems

Page 3

Research framework
Soffer & Hadar (2003) proposed a framework for under-
standing the sources of model variations, which was
partially based on theoretical foundations by Topi &
Ramesh (2002). Three classes of factors were identified as
influencing the model produced by an individual for a
given purpose. These include human factors, factors
related to the modeling grammar, and factors related to
the modeling process.
This research addresses factors related to the modeling
process, and particularly the decisions of how to map
real-world phenomena into modeling constructs. It is
primarily aimed at exploring how modeling rules affect
the conclusiveness of modeling decision making, as
reflected in the resulting model variations. Hence, our
main research question is: (1) what is the effect of
applying modeling rules to the modeling process on
model variations?
Our assumption is that model variations occur when-
ever a modeling decision may lead to more than one
legitimate choice of a modeling construct. Considering a
classification of variations to variation types, which can
be related to modeling decisions from which they stem,
we propose a set of modeling rules. These are specifically
aimed at reducing the number of choices for each such
decision type to one possible solution. In the study, we
examine whether the application of the modeling rules
indeed affects the modeling decisions and leads to this
desired result.
In order to do so, the modeling rules should (a) be
practical and simple enough to apply and (b) overcome
factors that contribute to model variations (Soffer &
Hadar, 2003). While testing the applicability of the
modeling rules, we consider it beneficial to characterize
the situations and decision types in which the rules are
indeed applicable and reduce variations and the situa-
tions where they do not. To gain more understanding
about decision-affecting factors and their interaction
with the modeling rules, we formulate a second research
question: (2) what are the factors affecting the modeling
decisions as reflected in model variations, and are their
effects reduced when modeling rules are used?
This question is exploratory, relating to the nature of
the decisions made during the process of mapping reality
into modeling constructs. In the study, we address it as a
secondary research question.
Empirical study
Methodology
Our research questions focus on different aspects of the
phenomenon we intend to study. Hence, we selected a
research methodology incorporating several methods. A
combination of research methods, especially from both
qualitative and quantitative paradigms, was proven
within the IS discipline as effective and contributing for
gaining a wide and deep understanding (cf. Kaplan &
Duchon, 1988; Galliers, 1991; Lee, 1991; Landry &
Banville, 1992; Mingers, 2001).
In this research, we wish to examine and draw
generalizable conclusions about whether applying mod-
eling rules reduces the extensiveness of model variations,
characterize the effects of using modeling rules on the
modeling process and the resulting model variations, and
identify factors affecting modeling decisions made with
and without applying the rules. Throughout our results
presentation and analysis, the data regarding the first
research question will be examined and discussed
through the lens of both qualitative and quantitative
methodologies. The second research question, being fully
exploratory, will be discussed only from the qualitative
perspective.
Setting
In order to study the influence of modeling rules on
model variations, we must first identify both dependent
and independent variables, and plan their manipulation
and control. Types of variables affecting model variations
include human factors, the modeling grammar used, the
purpose of modeling, and the modeling process. Our
intention is to manipulate the modeling process via the
use of modeling rules; hence, the other three categories of
variables should be controlled.
The human factors include several issues such as the
modelers’ experience and knowledge in systems analysis
and development, their prior knowledge regarding the
modeled domain, etc. In order to control these variables,
we conducted our study with the participation of
students, all with similar educational background, and
very limited former experience in industry. As well, we
designed thetwotasks
domains: one was a domain very well known to all
students (university) while the other referred to a domain
we believed, and later verified this in class, to be quite
remote to our students (physical transportation of
goods).
The modeling grammar used throughout this study was
UML. Therefore, no model variations whose source is the
use of different modeling grammars were to be observed.
The purpose of modeling, as explained to the students,
was to understand the problem as a basis for IS
requirements analysis.
The manipulated variable was the modeling process,
influenced by the use of modeling rules. The specific rules
were chosen according to the expected model variation
types. In a previous exploratory study conducted in
industry (Hadar & Soffer, 2006), we had identified and
classified variation types in UML Class Diagram con-
ceptual models, as summarized in Table 1.
Based on this classification, we selected a subset of
modeling rules (Table 2) from the rule base suggested by
Evermann & Wand (2001, 2004, 2005). Two main
considerations led us in this selection. First, to include
the rules relevant to the variation types previously
identified (Table 1). Second, to construct a rule set
concerningtwodifferent
Applying ontology-based rules to conceptual modeling
Pnina Soffer and Irit Hadar
601
European Journal of Information Systems

Page 4

minimal in size, simple, and easy to use. Our concern was
to prevent creating new variations resulting from misuse
of complex rules due to misunderstanding and lack of
experience in applying them. Note that the phrasing of
the rules is somewhat modified with respect to the
original rules for the purpose of simplicity and ease of use
by the students.
The population of the empirical study was senior
undergraduate Management Information Systems (MIS)
students, who took the course ‘Requirements Analysis
Seminar’. The empirical study took place after the
students had learnt about conceptual modeling, its
essence and importance, and had experienced the
construction of such models both in class and as a
homework assignment. As a result, the students’ experi-
ence in conceptual modeling was sufficient so that
significant learning was not expected to occur between
the two modeling tasks performed in the experiment.
The participants in the experiment were first randomly
divided into two groups. The experiment included three
phases:
1. Each group received one of the two tasks presented
below. The students created a conceptual model based
on the textual description they received. Although the
tasks were performed in class, no time limitation was
placed.
2. The students were taught the modeling rules (Table 2).
The teaching process included presentation of the
rules, several examples and class discussion.
3. Each group received the task they had not handled in
phase 1. The students were asked to produce a
conceptual model based on the description they
received while applying the modeling rules.
In addition, we had a control group, which was
handled similarly, but without phase 2 of the experiment.
The experiment group consisted of 37 students, ran-
domly divided to two subgroups of 18 and 19 students.
The control group consisted of 36 students, randomly
divided to two subgroups of 18 students each.
The data collection included the following: (1) the
conceptual models created by the participants; (2)
modeling dilemma documentation – the students were
asked to write down any dilemma they faced while
constructing the models; (3) observations of class discus-
sions; and (4) individual interviews with some of the
Table 1
Initial classification of variation types
Variation typeDescription
Class vs association
Aggregation vs composition
Abstract classes
Association vs aggregation
Association with classes that have
part–whole relationship
Inconsistency in distinguishing between a regular class and an association (or association class).
Inconsistency in distinguishing aggregation from composition.
Generalizing classes that have no instance of their own. May or may not be modeled.
Inconsistency in distinguishing association from aggregation.
Inconsistency in associating a third party class either to the whole or to the part class.
Table 2
Ontology-based modeling rules used in the study
No.RuleVariation types addressed
1Abstract classes, which are classes that do not possess their own
instances, are not to be used in conceptual modeling.
Composition relations are not to be used in conceptual modeling.
Part–whole relationship should always be expressed by an aggre-
gation relation.
An aggregation relation exists when the whole possesses at least one
property which is not possessed by its parts, and is a result of their
aggregation.
In aggregation relation, every property that can be associated either
to the whole or to the part, shall be associated to the parts.
Every association should be represented by an association class.
Association class instances cannot be substantial things.
If a class A is associated to a composite, whose whole is B and part is
C, then if there is at least one property that is mutual to A and the
whole (B), and not related to the part (C), then A is associated to B.
Otherwise A is associated to C.
Abstract classes
2 Aggregation vs composition
3 Association vs aggregation
4 Association with classes that have part–whole relationship
5
6
7
Class vs associationa
Class vs association
Association with classes that have whole–part relationship
aRules number 5 and 6 together are aimed at guiding the modeler in representing classes and associations. First, rule number 5 determines that every
association needs to be modeled as an association class, and then rule number 6 provides a clear criterion for distinguishing a ‘real’ class from an
association class.
Applying ontology-based rules to conceptual modeling
Pnina Soffer and Irit Hadar
602
European Journal of Information Systems

Page 5

participants, for an in-depth understanding of their
thinking process.
The analysis of the written models was aimed at finding
how using the modeling rules affected the models
created, and inparticular,
The quantitative analysis aimed at determining whether
there was indeed such an effect and if so, how extensive
this effect proved to be. It related to the following
hypotheses:
the model variations.
H0:
The difference in the choice of modeling constructs
between the second and first tasks will be the same in
the experiment and control groups.
H1:
The difference in the choice of modeling constructs
between the second and first tasks will be different in
the experiment group (making choices more consistent
with the modeling rules in the second task) as compared
to the control group.
These hypotheses rely on the assumption that in order
to reduce variations the rules are required to change the
way modeling decisions are made. We tested the
hypotheses separately with respect to each variation type
listed in Table 1, using Mann–Whitney U-test. This test
was chosen because normality assumption was not
reasonable due to the relatively small number of possible
modeling constructs in each model.
The qualitative content analysis of the models, docu-
mented dilemmas, class discussions, and interviews was
intended to gain an in-depth understanding of how the
effect of the rules was achieved. The content analysis was
conducted according to the principles of Strauss &
Corbin (1990). The data (i.e., models and text) were
coded and classified according to variation types and
emerging decision-affecting factors.
The models that were identified as clearly inadequate
and could not be justified in the interview as representing
a coherent line of thinking were not included in the final
data analysis. The screening of inadequate models was
initially conducted separately by each of the two
researchers. This was followed by a discussion leading
to an agreement as to the potentially inadequate
constructs. A follow-up interview was conducted with
each of the relevant students, to determine whether the
rationalization supplied by the student was valid. The
main criterion was whether consensus could be achieved
(Schuette & Rotthowe, 1998) regarding the model being a
representation of the domain. After eliminating the
models forwhich no consensus
68 models were included in the data obtained from the
experiment group and 66 from the control group.
A summary of all data collection and analysis methods
and tools, as well as the purpose and expected outcomes
of each of them, is presented in Table 3.
wasestablished,
The tasks
Each task included a textual description of a domain. The
two domains were quite different for three reasons. First,
we wished to avoid a situation where in the second
modeling phase the students would be biased by their
previous solution rather than apply the rules as part of
the modeling process. Second, we did not want our
findings to reflect domain-specific phenomena. Third, we
wished to control the prior domain knowledge variable,
by presenting to the students two domains that differ in
the prior knowledge the students possess about them.
The two tasks are presented in Table 4.
Findings
Being an exploratory study, its findings extend beyond
the initially defined research questions. While the
research framework was designed according to the five
previously identified variation types (Table 1); in this
study, two additional variation types arose from the field.
In this section, we first present these two newly identified
variation types. Then we present the findings related to
the two predefined research questions. We describe the
effect of the rules as observed in the empirical study,
analyzing it for each type of variation, and present
Table 3
Data collection and analysis summary
DataCollection tool Analysis methodPurpose Expected outcome
Models Written models formulated
by students in the class
exercises.
Documented observations.
Codification, classification,
and application of Mann
Whitney U–test.
Text analysis.
Test of the effect of the rules.To what extent and in
which variations are the
rules effective.
Rationale of modeling
decisions.
Class discussions Identify underlying assumptions
and rationale of modeling
decision.
Identify difficulties in decision
making.
Written dilemmas Written dilemmas docu-
mented by students in the
class exercises.
Unstructured interviews
Text analysis.Factors affecting model
variations.
Interviews Text analysis.Clarify specific model constructs
suggested by students
Final elimination of
incorrect models from the
data; rationale of modeling
decisions.
Applying ontology-based rules to conceptual modeling
Pnina Soffer and Irit Hadar
603
European Journal of Information Systems