No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Reliability Analysis of an Evaluation Experiment on Cultural Websites
Abstract
This paper describes the evaluation experiment of the websites of museums’ conservation labs. For this purpose an experiment has been implemented with the participation of 81 subjects and a multi-criteria decision-making model has been used for processing the results and draw conclusions about the electronic presence of the conservation labs of museums. However, the main focus of the paper is on performing a reliability analysis of the whole experiment. This analysis involved two tests, one for examining the reliability of the sample used in the evaluation experiment and another for examining the reliability of the questionnaire provided to the subjects of the evaluation.
ResearchGate has not been able to resolve any citations for this publication.
The museums' conservation labs and the treatments on the artifacts many times are overlooked and are not obvious for the public. Nevertheless, their content, which is more specialized than the content of the main museum, may be of interest to students, researchers, archaeologists, tourists, artists for further education and preservation guidelines purposes. In this paper, we evaluate the electronic presence of museums' conservation labs using both empirical and inspection methods of evaluation. For this purpose, a combination of Analytic Hierarchy Process (AHP) and Fuzzy Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) is used to implement an evaluation experiment that combines inspection and empirical methods of evaluation. The proposed scheme of evaluation that implements a combination of methods and decision making theories for the evaluation of websites with specialized cultural content has been used for evaluating the 29 websites of museum's conservation labs and ranks them taking into account their content, usability, and functionality. Povzetek: Prispevek z metodami umetne intelligence ocenjuje spletne strain muzejev.
The evaluation of cultural websites is a complicated procedure that depends on several criteria. The main contribution of this paper is on showing how this process may be implemented. For this reason, different multi-criteria decision-making methods are combined and compared for processing the results of the evaluation. More specifically, the paper provides the criteria for evaluating a website of cultural heritage and presents the application of Analytical Hierarchy Process for estimating the weights of the criteria. Then, it compares two different multi-criteria decision-making models, fuzzy simple additive weighting and fuzzy weighted product model, for being combined with Analytical Hierarchy Process and processing the results of the evaluation. The evaluation involves 29 websites of the conservation labs of the international museums and useful conclusions are extracted about the application of all methods and the combination of Analytical Hierarchy Process with fuzzy simple additive weighting and fuzzy weighted product model.
Virtual tours in museums are an ideal solution for those that are not able to visit a museum or those who want to have a small taste of what is presented in the museum before their visit. However, these tours often encounter severe problems while users interact with them. In order to check the status of virtual tours of museums, we present the implementation of an evaluation experiment that uses a combination of two multi-criteria decision making theories, namely the analytic hierarchy process (AHP) and the fuzzy technique for order of preference by similarity to ideal solution (TOPSIS). AHP has been used for the estimation of the weights of the heuristics and fuzzy TOPSIS has been used for the evaluation of virtual tours of museums. This paper presents the exact steps that have to be followed in order to implement such an experiment and run an example experiment for virtual tours of Italian museums.
The scope of this paper is on presenting an evaluation model for evaluating and comparing the websites of thematic museums. The particular evaluation model has been used for evaluating the websites of thematic museums on olives and olive oil using a multi-criteria decision making theory called Analytic Hierarchy Process (AHP).
In an effort to extend previous work on developing general recommendations for design of art museum websites, we evaluated four museum websites (the Tate, the Dallas Museum of Art, the Solomon R. Guggenheim Museum, and the Rijksmuseum) and examined the relationships between website features, virtual visitors’ impressions and estimates of repeat visits to the site. The study produced a comparative evaluation of the four museum websites and identified strengths and weaknesses in the sites’ navigation, design, and content features. The study found that website aesthetics was the strongest predictor of the visitor’s overall impressions. Ability to manipulate content (e.g., download, print, and zoom in and out) was also linked to the visitor’s inclination to make repeat visits to the site. This paper describes the methods and results of the study and proposes a set of recommendations for the development and evaluation of art museum websites.
Simple additive weighting is a well-known method for scoring and ranking alternative options based on multiple attributes. However, the pitfalls associated with this approach are not widely appreciated. For example, the apparently innocuous step of normalizing the various attribute data in order to obtain comparable figures leads to markedly different rankings depending on which normalization is chosen. When the criteria are aggregated using multiplication, such difficulties are avoided because normalization is no longer required. This removes an important source of subjectivity in the analysis because the analyst no longer has to make a choice of normalization type. Moreover, it also permits the modelling of more realistic preference behaviour, such as diminishing marginal utility, which simple additive weighting does not provide. The multiplicative approach also has advantages when aggregating the ratings of panel members. This method is not new but has been ignored for too long by both practitioners and teachers. We aim to present it in a nontechnical way and illustrate its use with data on business schools.
Multi-Criteria Decision Making (MCDM) has been one of the fastest growing problem areas in many disciplines. The central problem is how to evaluate a set of alternatives in terms of a number of criteria. Although this problem is very relevant in practice, there are few methods available and their quality is hard to determine. Thus, the question `Which is the best method for a given problem?' has become one of the most important and challenging ones.
This is exactly what this book has as its focus and why it is important. The author extensively compares, both theoretically and empirically, real-life MCDM issues and makes the reader aware of quite a number of surprising `abnormalities' with some of these methods. What makes this book so valuable and different is that even though the analyses are rigorous, the results can be understood even by the non-specialist.
Audience: Researchers, practitioners, and students; it can be used as a textbook for senior undergraduate or graduate courses in business and engineering.
The main objective of this paper is to evaluate and compare the electronic presence of the Protected Areas Managing Bodies with qualitative and quantitative factors. For this reason, an evaluation has been implemented using different methods and theories. The evaluation experiment involved the websites of Protected Areas Managing Boards in Greece. Greece has a high ecotouristic interest as it has rich biodiversity and a lot of tourists during the summer. The paper provides the basic steps of the experiment based on the advantages of the methods used and the steps of the Multi-Criteria Decision Making Theories that are used. The results of this evaluation revealed that the electronic presence of Protected Areas Managing Body in Greece was generally good and promoted environmental public awareness. The implementation of the particular evaluation experiment was very structured and can easily provide a guide for similar experiments by other researchers.
The role of museums is very important in cultural and heritage tourism, and a museum website plays an important role in attracting visitors, both physically and online. Therefore, these websites must be evaluated to ensure that their goals are met. However, the evaluation of a website is complex and is often omitted during the website’s life cycle, despite its importance. To simplify this process, this paper presents a framework for evaluating museum websites. The framework, based on an inspection method, is called the inspection evaluation method (IEM). This method uses criteria proposed by several researchers in a review of the different evaluation experiments for museum websites that have previously used an inspection method. IEM, different from all other methods reviewed, uses a combination of an analytic hierarchy process with a technique for the order of preference by similarity to an ideal solution for combining these criteria. The framework was applied to evaluate and rank the websites of five museums known worldwide. This combination uses the advantages of two multi-criteria decision-making theories and reduces their disadvantages.
Virtual reality attributes to software system characteristics such as intuitiveness, easiness of use, interactivity, and immediacy. Implementing a virtual environment is a complex process prone to usability issues. This article presents a model for usability evaluation of virtual environments deploying FSAW, a fuzzy multi-criteria decision making approach. FSAW has been used for evaluating the usability of 29 museums' websites offering virtual tours using 360-degree panoramic images. The proposed model ranked the virtual tours visited, focusing on the usability problems detected. The implementation issues identified had mainly to do with the naturalness of interaction and the consistency of departure. Another important problem identified was that in many cases the educational role of the museum had been overlooked. Users' interaction was deemed as very good and user-friendly, and the experts deducted a positive general overall impression. © Common Ground Research Networks, Katerina Kabassi, Emmanuel Maravelakis, Antonios Konstantaras, All Rights Reserved.
ICTs and the web have changed the way the users interact with museum exhibits providing alternative interactions and many advantages. Indeed, studies have shown a tremendous increase of online museum visitors. However, for a website to attract more visitors to the museum, it should be evaluated as usable and functional. This paper makes a state of the art on the evaluation experiments of museums’ websites. More specifically, it focuses on the methods used and tries to categorize the experiments found in the literature with respect to the method used. It describes the tools that may be used for implementing an evaluation experiment. Emphasis is given on the evaluation of special aspects of the use of ICTs in museums such as the use of handheld devices as well as VR tours. The review concludes on the most common criteria used for the evaluation of museum websites by real or expert users.
While most research on website has focused on functional tasks, the Internet offers many opportunities for leisure as well as experiential activities. Because of the evolution of developed society towards an experience economy, analyzing the role of technologies in the presence of prior user experiences makes sense. This research identifies variables that play a role and influence online behaviors in a specific experiential environment, namely the high culture museum website. Relying on the literature on experience, we propose a research model tested with two different websites. The results of the free simulation experiment indicates that 1) aesthetics is the most important design criteria for experiential interfaces and 2) that website design influences intentions to visit a physical place.
Museums, galleries and other cultural organisations have been swift in their adoption of the Web and virtual visits to museum Web sites have become popular. For many smaller museums the cost of developing and maintaining a Web site is difficult to justify, particularly where the economic benefit to the museum is hard to demonstrate. Many sites are therefore developed as unofficial, in-house projects—often without financial support. Evaluation is essential for determining whether a Web site is meeting the needs of its users and should be part of an ongoing process, from initial conception to long-term maintenance and development. This contribution focuses on the evaluation of Web sites as a way of addressing some of the limitations inherent in the non-professional development environment, placing it within the context of current museum Web development practice. The paper presents a case study illustrating Web evaluation issues from a computing perspective, using methods appropriate to the non-professional development environment. Four evaluation methods are examined in detail: direct observation, log analysis, online questionnaires, and inspection methods. The experience of applying each method, the benefits and limitations of each method and the relative effectiveness of the methods are analysed.
A previous paper discusses the notion of additive utilities in the expected-utility context when the set of consequences X can be written as the product of n other sets: X = X1 × X2 × ⋯ × Xn. The present paper continues this investigation for cases where X is a subset of the product of n other sets: X ⊂ X1 × X2 × ⋯ × Xn. Under a finiteness restriction, an additivity axiom leads to an additive utility representation for the elements in X. The potential usefulness of the theory in connection with incomplete product sets is demonstrated with priority orderings and assignments.
Counter-examples are given to show that in decision making, different methods of deriving priority vectors may be close for every single pairwise comparison matrix, yet they can lead to different overall rankings. When the judgments are inconsistent, their transitivity affects the final outcome, and must be taken into consideration in the derived vector. It is known that the principal eigenvector captures transitivity uniquely and is the only way to obtain the correct ranking on a ratio scale of the alternatives of a decision. Because of this and of the counter-examples given below, one should only use the eigenvector for ranking in making a decision.
A previous paper discusses the notion of additive utilities in the
expected-utility context when the set of consequences X can be written
as the product of n other sets: X=X1à X2à ⯠à Xn. The present paper
continues this investigation for cases where X is a subset of the
product of n other sets: X\subset X_1\times X_2\times \cdots
\times X_n$ . Under a finiteness restriction, an additivity axiom
leads to an additive utility representation for the elements in X.
The potential usefulness of the theory in connection with incomplete
product sets is demonstrated with priority orderings and assignments.
An Analysis of the Electronic Presence of National Parks in Greece
- A Martinis
- A Papadatou
- K Kabassi
Martinis, A., Papadatou, A., Kabassi, K.: An Analysis of the Electronic Presence of National Parks in Greece, Proceedings of the 5th International Conference on "Innovative
Approaches to Tourism and Leisure: Culture, Places and Narratives in a Sustainability
Context", Athens, Greece, 2018, June 28-30.
The Analytic Hierarchy Process
- T Saaty
Saaty, T. The analytic hierarchy process. New York, McGraw-Hill 1980..