ArticlePDF Available

Virtual restoration of deteriorated religious heritage objects using augmented reality technologies

April 2013
European Journal of Science and Theology 9(2):223-231

Authors:

Girbacia Florin

Transylvania University of Brașov

Silviu Butnariu

Transylvania University of Brașov

Orman Alex Petre

Transylvania University of Brașov

Cristian Postelnicu

Transylvania University of Brașov

In this paper, an approach for the virtual restoration of the religious heritage objects is proposed. The presented method is based on Augmented Reality technologies, which enable virtual restoration of the original heritage object by co-locating the reconstructed 3D virtual model with the real one. This method gives the opportunity to better perceive the damaged object. The proposed method consists in the following actions: obtain documentation about the damaged heritage object, processing of the documentation, 3D reconstruction of the monument, finishing and completing details, registration of the co-located 3D virtual model. The output of the proposed method is a reconstructed 3D model of heritage object, which can be visualized co-located with the real one by using common equipment such as Smartphone or Tablet PC. The application of the proposed methodology, which is used as a case study, was conducted in Brasov, at the Black Church. The proposed methodology may open new possibilities for the restoration of other religious heritage objects.

The 3D virtual reconstruction of the ‘ Black Church’ sculpture using image-based modeling technique.

…

Co-location of the reconstructed virtual model with the ‘ Black Church ’ using the developed AR system.

…

Figures - uploaded by Silviu Butnariu

Content may be subject to copyright.

Content uploaded by Silviu Butnariu

Content may be subject to copyright.

European Journal of Science and Theology, April 2013, Vol.9, No.2, 223-231

_______________________________________________________________________

VIRTUAL RESTORATION OF DETERIORATED

RELIGIOUS HERITAGE OBJECTS USING

AUGMENTED REALITY TECHNOLOGIES

Florin Gîrbacia*, Silviu Butnariu, Alex Petre Orman and

Cristian Cezar Postelnicu

Transilvania University, Department of Automation, Electronics and Computers, 29 Eroilor Blv.,

500036, Brasov, Romania

(Received 25 June 2012, revised 5 February 2013)

Abstract

In this paper, an approach for the virtual restoration of the religious heritage objects is

proposed. The presented method is based on Augmented Reality technologies, which

enable virtual restoration of the original heritage object by co-locating the reconstructed

3D virtual model with the real one. This method gives the opportunity to better perceive

the damaged object. The proposed method consists in the following actions: obtain

documentation about the damaged heritage object, processing of the documentation, 3D

reconstruction of the monument, finishing and completing details, registration of the co-

located 3D virtual model. The output of the proposed method is a reconstructed 3D

model of heritage object, which can be visualized co-located with the real one by using

common equipment such as Smartphone or Tablet PC. The application of the proposed

methodology, which is used as a case study, was conducted in Brasov, at the Black

Church. The proposed methodology may open new possibilities for the restoration of

other religious heritage objects.

Keywords: virtual reconstruction, Augmented Reality, image based modelling,

restoration

1. Introduction

There are many religious heritage objects and old relics, for which the

restoration is not achievable. For an inexperienced visitor, in case of damaged

heritage objects, it will be very difficult to identify or to imagine the original

form and details. Virtual reconstruction is a suitable technology that can be

applied in these cases. Virtual reconstructions should be defined as those

environments where the human operator is transported into a new interactive

environment by means of devices that display signals to the operator’s

sense organs and devices that sense various actions of the operator [1].

* E-mail: garbacia@unitbv.ro, phone: +40 268 418967, fax: +40 268 418967

Gîrbacia et al/European Journal of Science and Theology 9 (2013), 2, 223-231

224

Advances in computer graphics and Virtual Reality (VR) technologies

have allowed creation of virtual 3D model that represent real objects at any scale

and complexity. The key concept is to create from the pictures, paintings or

other historical material of the original object a virtual replica of the damaged

heritage object using computer graphics and VR technologies. The reconstructed

model then can be displayed in an interactive environment by means of devices

like: CAVE-like immersive environments, reality theatres, power walls,

holographic workbenches, individual immersive systems, head mounted

displays, tactile sensing interfaces, haptic feedback devices, multi-sensational

devices, speech interfaces and mixed reality systems [2].

Recently, virtual reconstruction based on VR technologies has been

proposed as an improved interface for ancient architecture reconstruction [1, 3-

10] monumental paintings of the church [11], completion of facial image in

ancient murals [12, 13], restoring content from distorted documents [14] and art

restoration [15]. The VR technologies offer several advantages for virtual

reconstruction: improve the immersion awareness; high-definition stereoscopic

images are obtained; large filed of view; collective visualization; collaboration

between several users. The disadvantages of using VR technologies for virtual

reconstruction are the high complexity and the usage of expensive devices which

limited the applicability of this technology to the consumers.

Augmented Reality (AR) is a relative new research direction that allows

creation of an interactive virtual space embedded into the physical word. In

order to remove presented VR issues, AR technology allows the creation of an

interactive virtual space, embedded into the physical word. Unlike VR systems,

in which users are completely immersed in the virtual environment, AR gives

the users the possibility to see the virtual objects and the real world coexisting in

the same space (co-located). It is the goal of AR to supplement reality rather

than completely replace it as in conventional VR applications. The AR

technology provides useful information about the environment, enhancing the

perception of spatial information and interaction with the real world. The user

can interact with the digital objects in an actual 3D space, which is more natural

and intuitive. In this way, AR offers the possibility to visualize 3D reconstructed

model of the heritage objects co-located with the real environment and provide

an efficient and intuitive communication channel for spatial information.

In this paper, an approach for the virtual restoration of the religious

heritage objects based on AR technologies is proposed. This method enables

virtual restoration of the original heritage object by co-locating the reconstructed

3D virtual model with the real one. The paper is organized as follows: section 2

reviews prior work, section 3 describes the proposed AR-based reconstruction

method, section 4 point up the results and section 5 presents conclusions.

2. The research background

Nowadays, there are presented several applications of using augmented

reality in virtual reconstruction. In [16] AR technologies are used for virtual 3D

Virtual restoration of deteriorated religious heritage objects

225

reconstruction of ancient frescos paintings, fauna and flora and animated

characters from the ancient Pompeii. For the co-location of 3D model in the

virtual environment it was used a mobile AR wearable setup with markerless

tracked camera. In [17] is presented the development of AR-based personalized

electronic guide and tour assistant for the Ancient Olympic Games. The system

uses a position-orientation tracking component to display AR reconstruction of

the temples and animated characters. The site visitors are wearing AR glasses to

see the 3D image display. The disadvantage of this type of display is the low

resolution and autonomy.

AR has also been used for the virtual recovery of the deteriorated art

object [18], where the correction pattern for restoration is generated from non

damaged object’s image scanned in advance and projected on the damaged

object using a calibrated LCD projector. Although their approach has innovative

aspects, this technique can be applied only for the restoration of the discolouring

objects of small dimension, and is not suitable to be used for outdoor

reconstruction of religious monuments. Recently in [19] are presented two

examples of using QR code-based AR system for the three-dimensional

reconfiguration of urban places in order to show evidence of ‘invisible’

architecture.

Considering the virtual reconstruction of the religious heritage objects,

there are only presented applications related to monumental painting [11] and

temple architectural reconstruction [3-5, 8, 9] using a 3D VR technologies. From

the authors knowledge, in the consulted scientific literature there is not presented

a methodology for creation of AR-based virtual reconstruction applications.

Also, in all these reviewed papers, there is not presented AR-based virtual

reconstruction of outdoor religious heritage objects, such as the outdoor

sculpture of Black Church from Brasov.

3. Methodology for creation of AR-based virtual reconstruction

applications

The AR-based system is composed from files which allow visualization of

the co-located 3D model of the reconstructed model and the pattern of the

original heritage monument. In principle, the methodology for creation of AR-

based virtual reconstruction application could be formulated as to cover the

following steps:

(1) 3D reconstruction of the heritage object. This is an important and

elementary step in the process development of AR-based application.

Actual 3D reconstruction techniques for AR which are suitable for Virtual

Heritage applications are: image-based modelling, range-based modelling,

image-based rendering, photogrammetric and combination of image-and

range-based modelling [20]. For reconstruction of outdoor architecture

objects, image based modelling is a technique commonly used. Image-

based modelling refers to the use of images to generate the reconstructed

3D model. It is use a mathematical model to pick up 3D object information

Gîrbacia et al/European Journal of Science and Theology 9 (2013), 2, 223-231

226

from 2D image dimensions or to get 3D data using methods such as shape

from shading, texture, specularity, contour and from 2D edge gradients. The

advantage of image-based representations is the capability to represent

arbitrary geometry. This technique also can handle subtle real-world effects

captured by images, but difficult to reproduce with usual graphics

techniques [21].

(2) Conversion of the 3D reconstructed model data. The virtual model cannot

be loaded in the AR software because there is not standard interoperability

procedure. Therefore this step consists in extracting the entire geometric

data of the 3D reconstructed model and conversion to an appropriate

common exchange file format (for example *.3ds, VRML, X3D, etc.) that

can be loaded by general AR framework.

(3) Registration of the 3D virtual model with the existing monument.

Registration in AR applications represents a precise alignment of real and

virtual objects. This is an essential operation that influences the visual

perception of the co-located model and the efficiency of the developed AR

application. Marker-based tracking is a common technology used for

developing AR applications. But this technology is impractical for outdoor

AR based reconstructions, due to the laborious task of having to place

markers accurately at various predeﬁned locations. An appropriate solution

is to use tracking of textured planar regions using randomized trees based

key point classifier for pose initialization and estimation.

(4) Visualisation of the co-located reconstructed virtual heritage object using a

physical display device (HMD, Smart Phones or Tablet PC).

4. AR-based virtual reconstruction of the Black Church outdoor sculpture

A prototype AR system for the virtual reconstruction of outside sculpture

of the Black Church was developed, to demonstrate the methodology presented

above. The Black Church is the main cathedral in Brasov, a city in south-western

Transylvania, Romania. It stands as the main Gothic style monument in the

country and the largest Gothic church between Vienna and Istanbul. The Black

Church had a turbulent history: built between 1385 and 1477 on the site of an

earlier church (destroyed by Mongol invasions in 1242), the construction of the

Marienkirche was hampered by extensive damage caused by Turkish raids in

1421. The church was given its new name after disaster struck again in 1689,

when the ‘Great Fire’, set by Hapsburg invaders, levelled most of the town,

heavily damaged the church, blackening its walls. Restoration took almost 100

years [http://www.brasovtravelguide.ro/en/brasov/sightseeing/black-church.

php]. Outside sculptures of the Black Church, during the time, where damaged

or totally destroyed (Figure 1). The oldest statue still standing belongs to Saint

Peter and can be seen on the Southern part of the church. The church is built of

friable grit stones and andesite arranged in cubic shape. That is the reason why

the statues placed on the exteriors couldn’t survive the time and some of them

Virtual restoration of deteriorated religious heritage objects

227

are missing or were replaced by new ones. The objective of the developed AR-

based application was to reconstruct the virtual 3D models of these outside

sculptures and co-locate them with the real church.

Figure 1. The ‘Black Church’, Brasov.

Figure 2. Using Tablet PC for perception of co-located reconstructed 3D model.

4.1. Resources

In order to augment human’s visual sense, a physical display device is

used allowing combining real and virtual images and present them to the user.

Many forms of video display can be used: Head Mounted Displays (HMD),

portable displays (like Smart Phones and Tablet PC), monitors and projectors.

HMD is a common choice for AR because it is portable, and it is placed directly

on the users’ visual range. But the use of a HMD is not the best solution to be

used for outdoor activities because of the low autonomy and high cost. Technical

progress in handheld devices and tablet-PC has opened an enormous potential

for development of AR applications due to the. In this research was used a

portable AR system composed from a Tablet PC (Figure 2).

Missing sculptures

Gîrbacia et al/European Journal of Science and Theology 9 (2013), 2, 223-231

228

4.2. 3D reconstruction of the ‘Black Church’ virtual heritage models

To create the 3D model of damaged sculpture it was used the image-based

modeling technique. In a first step, images of the heritage sculpture were

captured. Then these images where processed in order to eliminate the blurry

pictures and to enhance the visibility of the heritage sculpture. In order to apply

the image-based modeling technique we studied several computer programs

available, such as: Autodesk Image Modeler (http://usa.autodesk.com/), Google

SketchUp (sketchup.google.com), ARC3D (www.arc3d.be) [22], MeshLab

(meshlab.sourceforge.net), PhotoModeler (www.photomodeler.com). For the

development of the application, we select the ARC3D program to generate from

a set of images the point cloud of 3D geometry and the MeshLab program to

generate textured 3D mesh VRML model of the reconstructed object. The

specific steps for the reconstruction are: (i) loading the processed images to

ARC3D program, (ii) import the resulted *.v3d file in MeshLab program, (iii)

select and delete the point corresponding to other objects than the focused

outside sculpture, (iv) apply a subsample filter, (v) apply a Poisson

reconstruction filter, (vi) apply the texture and convert the mesh in the neutral

format Virtual Reality Modeling Language (VRML) (Figure 3).

4.3. Registration of the reconstructed 3D models

Architecture software was created for the visualization of the

reconstructed 3D model in the co-located environment (Figure 4). The code

written for the AR-based virtual reconstruction system is based on a library

called Instant Player (www.instantreality.org). The advantage of using this

library is the possibility to integrate various VRML and X3d graphical formats

of virtual objects and possibility to create External Authoring Interface (EAI).

Registration of the reconstructed 3D virtual model with the church building was

developed using the instant reality’s vision Generic Poster Tracker module for

tracking of textured planar regions. The tracking methods used are randomized

trees based key point classifier for pose initialization and a KLT tracker [23]. In

order to create an AR application based on this tracking technology, the user has

to carry out the following steps: (i) acquire a reference image of the real scenario

with the camera used for tracking by using image capturing software (for

example the free software IrfanView www.irfanview.com), (ii) perform an

offline classifier training phase using the function Generate Poster Tracker of

Instant Vision module, (iii) embed the tracker data from the generated *.pm file

into the AR application, (iv) co-locate the virtual model on the original church

building location by modification of scale, 3D position, and orientation, relative

to the camera transformation matrix and save the registration data in a

configuration file. Then the user can visualise the co-located virtual sculpture

object on the original location using the Tablet PC or laptop display device.

Virtual restoration of deteriorated religious heritage objects

229

Figure 3. The 3D virtual reconstruction of the ‘Black Church’ sculpture using

image-based modeling technique.

Figure 4. Co-location of the reconstructed virtual model with the ‘Black Church’

using the developed AR system.

5. Conclusions

In this paper, it was presented the methodology and prototype software

system of an AR-based virtual reconstruction system. This approach helps users

to perceive better damaged sculptures and to learn about the culture by

interacting with a virtual reconstructed environment. Using this system can be

made without the need for expensive equipment, because only common

computing devices are needed. However, out of these positive results the system

has his limitations, since problems can come out when the intensity of the light it

is weak, which affects the level of tracking accuracy and make the registration of

reconstructed 3D model difficult.

Gîrbacia et al/European Journal of Science and Theology 9 (2013), 2, 223-231

230

Acknowledgement

This paper is supported by the Sectoral Operational Programme Human

Resources Development (SOP HRD), financed from the European Social Fund

and by the Romanian Government under the contract number

POSDRU/88/1.5/S/59323 for author (1) and by the Sectoral Operational

Programme Human Resources Development (SOP HRD), financed from the

European Social Fund and by the Romanian Government under the contract

number POSDRU89/1.5/S/59321 for author (4).

References

[1] M. Mehta, Virtual reality applications in the field of architectural reconstructions,

Proc. of 7th International Conference on Virtual Systems and Multimedia, IEEE,

Berkeley, 2001, 183-190.

[2] D. Talaba, I. Horvath and K.H. Lee, Comput. Aided Design, 42 (2010) 361.

[3] J. Yao, H. Zhang and F. She, Research on Method of 3D Reconstruction of Ancient

Architecture (Nanputuo Temple), Proc. of International Conference on

Cyberworlds, IEEE, Hangzhou, 2008, 627-630.

[4] G. Issini, D. Polverini and F. Pugnaloni, Virtual Reconstruction and Real-Time

Interactive Visualization of the Monumental Area between Thien Mu Pagoda and

Van Thanh Temple in Hue City (UNESCO Site), Vietnam, Proc. of 13th

International Conference on Information Visualisation, IEEE, Barcelona, 2009,

561-567.

[5] S.Z. Lu, Virtual Reconstruction of FouGuang Temple Based on Virtual Reality,

Proc. of International Conference on Management of e-Commerce and e-

Government ICMECG '08, IEEE, Barcelona, 2008, 208-211.

[6] D.M. Popovici, R. Querrec, C.M. Bogdan and N. Popovici, International Journal of

Computers, Communications & Control, 5 (2010) 876.

[7] D.M. Popovici, D. Talaba, E. Canciu, V. Voinea, C.M. Bogdan and N. Popovici,

Bulletin of the Transilvania University of Braşov, 3(special issue) (2008) 460.

[8] M. Gaiani, E. Gamberini and G. Tonelli, VR as work tool for architectural &

archaeological restoration: the ancient Appian way 3D web virtual GIS, Proc. of

7th International Conference on Virtual Systems and Multimedia, IEEE, Berkeley,

2001, 86-95.

[9] R.M. Levy, Temple site at Phimai: modeling for the scholar and the tourist, Proc.

of 7th International Conference on Virtual Systems and Multimedia, IEEE,

Berkeley, 2001, 147-158.

[10] M. Callieri, P. Cignoni, F. Ganovelli, G. Impoco, C. Montani, P. Pingi, F. Ponchio

and R. Scopigno, Visualization and 3D data processing in the David restoration,

IEEE Comput. Graph., 24 (2004) 16.

[11] Y.A Petrova, I.V. Tsimbal, T.V. Laska & S.V. Golubkov, Practice of Using Virtual

Reconstruction in the Restoration of Monumental Painting of the Church of the

Transfiguration of Our Saviour on Nereditsa Hill, Proc. of 15th International

Conference on Information Visualisation (IV’2011), IEEE, London, 2011, 389-

394.

[12] Q. Wang, D. Lu and H. Zhang, Virtual Completion of Facial Image in Ancient

Murals, Proc of Workshop on Digital Media and Digital Content Management

(DMDCM 2011), IEEE, Hangzhou, 2011, 203-209.

Virtual restoration of deteriorated religious heritage objects

231

[13] A. Lanitis and G. Stylianou, e-Restoration of Faces Appearing In Cultural

Heritage Artefacts, Proc of 15th International Conference on Virtual Systems and

Multimedia VSMM '09, IEEE, Vienna, 2009, 15-20.

[14] M.S. Brown, S. Mingxuan, Y. Ruigang, Y. Lin and W.B. Seales, IEEE T. Pattern

Anal., 29 (2007) 1904.

[15] L. Bonanni, M. Seracini, X. Xiao, M. Hockenberry, B.C. Costanzo, A. Shum, R.

Teil, A. Speranza and H. Ishii, International Journal of Creative Interfaces and

Computer Graphics, 1 (2010) 54.

[16] G. Papagiannakis, S. Schertenleib, B. O’Kennedy, M. Poizat, N. Magnenat-

Thalmann, A. Stoddart and D. Thalmann, Comput. Animat. Virt. W., 16 (2005) 11.

[17] V. Vlahakis, J. Karigiannis, M. Tsotros, M. Gounaris, L. Almeida, D. Stricker, T.

Gleue, I. Christou, R. Carlucci and N. Ioannidis, ARCHEOGUIDE: First results

of an Augmented Reality, Proc. of Mobile Computing System in Cultural

Heritage Sites, Virtual Reality, Archaeology, and Cultural Heritage International

Symposium (VAST 2001), ACM, New York, 2001, 131-140.

[18] T. Amano and R. Suzuki, Virtual Recovery of the Deteriorated Art Object based on

AR Technology, IEEE Conference on Computer Vision and Pattern Recognition

CVPR '07, IEEE, Minneapolis, 2007, 1-2.

[19] G.M. Girgenti, M. Filippi and F. Marrone, Experiments on the Virtual City: Three-

Dimensional Reconfigurations of Missing, Never Realized or Destroyed Urban

Areas, Proc. of International Conference on Computational Science and Its

Applications (ICCSA 2011), IEEE, Santander, 2011, 275-281.

[20] Z. Noh, M.S. Sunar and Z. Pan, A Review on Augmented Reality for Virtual

Heritage System, Proc. of the 4th International Conference on E-Learning and

Games: Learning by Playing. Game-based Education System Design and

Development (Edutainment '09), Springer-Verlag, Banff, 2009, 50-61.

[21] M.M. Oliveira, IEEE Revista Iberoamericana de Tecnologías del Aprendizaje, 9

(2002) 37.

[22] M. Vergauwen and L. Van Gool, Web-Based 3D Reconstruction Service, Mach.

Vision Appl., 17 (2006) 411.

[23] S. Baker and I. Matthews, Int. J. Comput. Vision, 56 (2004) 221.

Complexity Experts are Task-Discriminative Learners for Any Image Restoration

Preprint

Full-text available

Nov 2024

Recent advancements in all-in-one image restoration models have revolutionized the ability to address diverse degradations through a unified framework. However, parameters tied to specific tasks often remain inactive for other tasks, making mixture-of-experts (MoE) architectures a natural extension. Despite this, MoEs often show inconsistent behavior, with some experts unexpectedly generalizing across tasks while others struggle within their intended scope. This hinders leveraging MoEs' computational benefits by bypassing irrelevant experts during inference. We attribute this undesired behavior to the uniform and rigid architecture of traditional MoEs. To address this, we introduce ``complexity experts" -- flexible expert blocks with varying computational complexity and receptive fields. A key challenge is assigning tasks to each expert, as degradation complexity is unknown in advance. Thus, we execute tasks with a simple bias toward lower complexity. To our surprise, this preference effectively drives task-specific allocation, assigning tasks to experts with the appropriate complexity. Extensive experiments validate our approach, demonstrating the ability to bypass irrelevant experts during inference while maintaining superior performance. The proposed MoCE-IR model outperforms state-of-the-art methods, affirming its efficiency and practical applicability. The source will be publicly made available at \href{https://eduardzamfir.github.io/moceir/}{\texttt{eduardzamfir.github.io/MoCE-IR/}}

HAIR: Hypernetworks-based All-in-One Image Restoration

Preprint

Aug 2024

Image restoration involves recovering a high-quality clean image from its degraded version, which is a fundamental task in computer vision. Recent progress in image restoration has demonstrated the effectiveness of learning models capable of addressing various degradations simultaneously, i.e., the All-in-One image restoration models. However, these existing methods typically utilize the same parameters facing images with different degradation types, which causes the model to be forced to trade off between degradation types, therefore impair the total performance. To solve this problem, we propose HAIR, a Hypernetworks-based plug-in-and-play method that dynamically generated parameters for the corresponding networks based on the contents of input images. HAIR consists of 2 main components: Classifier (Cl) and Hyper Selecting Net (HSN). To be more specific, the Classifier is a simple image classification network which is used to generate a Global Information Vector (GIV) that contains the degradation information of the input image; And the HSNs can be seen as a simple Fully-connected Neural Network that receive the GIV and output parameters for the corresponding modules. Extensive experiments shows that incorporating HAIR into the architectures can significantly improve the performance of different models on image restoration tasks at a low cost, \textbf{although HAIR only generate parameters and haven't change these models' logical structures at all.} With incorporating HAIR into the popular architecture Restormer, our method obtains superior or at least comparable performance to current state-of-the-art methods on a range of image restoration tasks. \href{https://github.com/toummHus/HAIR}{\textcolor{blue}{

\underline{\textbf{Code and pre-trained checkpoints are available here.}}

}}

Sacred Spaces in the Digital Age: Perceptions of Lutheran Christian Priests on Augmented Reality at Holy Sites

Conference Paper

Full-text available

Jun 2024

The concepts of sacred places and spaces appear throughout religions globally. Places such as churches, cathedrals, temples, mosques, synagogues and graveyards are given special meanings, both functionally and spiritually, and separated from the ordinary. Recently location-based augmented reality (AR) technologies and applications have become widespread, and this raises questions regarding how AR content relates to sacred places. In this study, we approached this complex topic by asking clergy of the Lutheran Christian Church (N=47) to reflect on associated phenomena. We approached the data via reflexive thematic analysis and uncovered tensions related to (1) connected vs detached from sacredness; (2) supporting the spiritual purpose of the space vs conflicting with it; and (3) maintaining tradition vs embracing innovation. Overall, our findings suggest that AR technologies and products impact sacred spaces on multiple levels, but currently there is no consensus among the clergy on the impact of these changes

Efficient Degradation-aware Any Image Restoration

Preprint

Full-text available

May 2024

Reconstructing missing details from degraded low-quality inputs poses a significant challenge. Recent progress in image restoration has demonstrated the efficacy of learning large models capable of addressing various degradations simultaneously. Nonetheless, these approaches introduce considerable computational overhead and complex learning paradigms, limiting their practical utility. In response, we propose \textit{DaAIR}, an efficient All-in-One image restorer employing a Degradation-aware Learner (DaLe) in the low-rank regime to collaboratively mine shared aspects and subtle nuances across diverse degradations, generating a degradation-aware embedding. By dynamically allocating model capacity to input degradations, we realize an efficient restorer integrating holistic and specific learning within a unified model. Furthermore, DaAIR introduces a cost-efficient parameter update mechanism that enhances degradation awareness while maintaining computational efficiency. Extensive comparisons across five image degradations demonstrate that our DaAIR outperforms both state-of-the-art All-in-One models and degradation-specific counterparts, affirming our efficacy and practicality. The source will be publicly made available at \url{https://eduardzamfir.github.io/daair/}

Virtual Reconstruction With The Augmented Reality Technology Of The Cultural Heritage Components That Have Disappeared: The Ayazini Virgin Mary Church

Article

Full-text available

May 2023

Virtual Reconstruction With The Augmented Reality Technology Of The Cultural Heritage Components That Have Disappeared: The Ayazini Virgin Mary Church

Article

Full-text available

May 2023

Augmented reality (AR) method could be used in the context of being able to provide for the transfer of the architectural and tectonic attributes related to the period to which a historical building belongs, and under conditions where the reconstruction activities could produce damage to the integrity of the building or in situations where the financial resources remain insufficient for making reconstruction. Ayazini Village was selected as the study area. Ayazini Village is connected to Afyonkarahisar Province and is a region that is an extension of the Phrygian Valley, which has a 6,000-year-old past. The Ayazini Region is composed of layers, which produced the First Degree Archeological, Urban Archaeological and First-Degree Natural site areas. The rock settlements, which form the tectonic structure of Ayazini Village, were produced with the cutting of the tufa rocks that are described as Ayazini rock. The interior spaces and building components of the rock settlements have been damaged or disappeared in time. It is thought from this aspect that there are difficulties in the context of materials, cost and labor for the reconstruction activities of the areas where these interior spaces were damaged. Within the scope of the research, a virtual restoration model was applied in the interior space of the Virgin Mary Church located in Ayazini Village by using the AR technology. The Virgin Mary Church is a unique historical building remaining from the Byzantine Middle Period when Christianity was accepted. The interior space of the Virgin Mary Church was damaged as the result of natural disasters and human influences over time. In this context, within the scope of the virtual restoration model in this research, the sections of the arches set on the columns in the interior space of the church that were damaged were treated in the integration of the interior space with the sign-based AR technology and reconstruction. The surfaces of the lower parts of the columns that were damaged were mapped with a laser scanner. The parts of the Quick Response (QR) code columns, which were damaged, were presented in simulations. These simulations contain the modeling of the columns from the aspect of shape, color and fabric. The columns bearing the arches located in the interior space of the church were simulated in a computer environment by determining the colors, fabric, shape and construction characteristics through the original photographs and based on the data in the literature and with the color pixelization and fabric characteristic mapping of the surfaces. At the conclusion of all these procedures, the data was transferred to the mobile application called Augment. The QR code assigned provided for the aligning under the columns by using the on-site test method and a laser scanner. The smart equipment, through the Augment application, showed the original condition and experienced on-site simultaneously the damaged parts of the column that were completed with the arch by perceiving the QR code.

The Role of Digital Technologies in the Restoration and Conservation of relief Sculptures in the Karnak Temples

Article

Full-text available

Jan 2025

Shrouk Marey

The Karnak temples stand as unique and significant monuments, encapsulating a historical and artistic narrative that forms an integral part of Egypt’s cultural heritage. As one of the major cultural and religious centers of ancient times, these temples reflect a rich historical evolution and diversity in both architecture and the arts. This is especially evident in the relief sculptures that adorn the walls and columns, depicting stories and events experienced by kings and gods throughout the ages. These inscriptions serve as primary records, offering insight into the history of the temple's construction, its development, and the religious and military exploits of its rulers and deities. The importance of these reliefs is immeasurable, and any damage to them poses a serious threat not only to the aesthetic value of the temple but also to the historical narratives they convey. However, international conventions prohibit restorers from making direct interventions on archaeological works, leaving us with the challenge of preserving these invaluable scenes without physically altering them. Technological advancements, which impact a wide array of fields and contribute significantly to humanity, have introduced new techniques to the realm of restoration. One such advancement is the development of digital sculpture, closely tied to the fields of sculpture and restoration, leading to the emergence of virtual restoration. This research aims to explore methods of reviving some of the damaged relief scenes within the Karnak temple by reshaping them digitally through advanced sculpture programs. The goal is to virtually restore these reliefs as closely as possible to their original state, thereby protecting them from deterioration while creating a form of virtual archaeological documentation that endures over time. Additionally, the researcher seeks to propose various solutions and ideas for displaying and presenting the virtually restored models

KUDÜS’ÜN KÜLTÜREL MİRAS ALANLARINDA ARTIRILMIŞ GERÇEKLİK VE SANAL GERÇEKLİK TEKNOLOJİLERİNİN UYGULANABİLİRLİĞİ ÜZERİNE BİR ARAŞTIRMA

Article

Full-text available

Dec 2024

Artırılmış gerçeklik ve sanal gerçeklik uygulamaları kültürel miras alanlarını yorumlama, sınıflandırma ve yapılı çevrenin anlamlandırılmasına yardımcı olmaktadır. Artırılmış gerçeklik ve sanal gerçeklik, kültürel miras yapısının alanı dâhil olmak üzere genişleyen kapsam içinde uygulamaların potansiyelini ortaya çıkarmaktadır. Bu teknolojilerin uygulamaları, diğer bilgi ve iletişim teknolojilerinin gelişmesiyle birlikte bir süreklilik sağladığında kültürel miras alanlarından en iyi şekilde yararlanılabileceğini göstermektedir. Turizm literatüründe bu uygulamaların kullanılması yaşanacak değişim ve rekabet avantajı sağlama açısından önemli olmaktadır. Bundan dolayı, dünya çapında kültürel miras alanlarında birçok artırılmış gerçeklik ve sanal gerçeklik uygulamaları hayata geçirilmeye başlanmıştır. Dolayısıyla bu çalışmada, Kudüs’teki artırılmış gerçeklik ve sanal gerçeklik uygulamalarının uygulanabileceği müzeler, kiliseler, camiler, kaleler ve ören yerleri gibi kültürel miras alanlarının tespit edilmesi amaçlanmıştır. Çalışmada, nitel araştırma yöntemlerinden literatür taraması tekniği kullanılarak bu alandaki ikincil verilerden yararlanılmıştır. Artırılmış gerçeklik ve sanal gerçeklik uygulamalarının kültürel miras alanlarında kullanılması bu değerlerin korunması, gelecek nesilleri aktarımı ve turist memnuniyetini artırmak amacıyla uygulandığı görülmektedir. Bu bağlamda, Kudüs’ün kültürel miras alanlarında artırılmış gerçeklik ve sanal gerçeklik teknolojilerinin uygulanıp uygulanmayacağı sorgulanırken diğer yandan kültürel miras yapılarının gelecek kuşaklara aktarılması konusunda farkındalık yaratması amaçlanarak bu çalışmanın ilgili literatüre önemli bir katkı sağlayacağı düşünülmektedir. Çalışma sonucunda bu teknolojilerin Kudüs’teki kültürel miras alanlarında kullanılmasına dönük önerilerde bulunulmuştur.

EmojiChat: Toward Designing Emoji-Driven Social Interaction in VR Museums

Article

Full-text available

Aug 2024

LagunAR: A City-Scale Mobile Outdoor Augmented Reality Application for Heritage Dissemination

Article

Full-text available

Nov 2023
SENSORS-BASEL

In this paper, we introduce LagunAR, a mobile outdoor Augmented Reality (AR) application for providing heritage information and 3D visualization on a city scale. The LagunAR application was developed to provide historical information about the city of La Laguna in the XVI century, when it was the main city in the Canary Islands. The application provides a reconstructed 3D model of the city at that time that is shown on a mobile phone over-imposed on the actual city using geolocation. The geolocated position is used also for providing information of several points of interest in the city. The paper describes the design and implementation of the application and details the optimization techniques that have been used to manage the full information of the city using a mobile phone as a sensor and visualization tool. We explain the application usability study carried out using a heuristic test; in addition it is probed by users in a qualitative user test developed as preliminary research. Results show that it is possible to develop a real-time application that shows the user a city-scale 3D model and also manages the information of the points of interest.

Lucas-Kanade 20 Years On: A Unifying Framework

Article

Feb 2004

Since the Lucas-Kanade algorithm was proposed in 1981 image alignment has become one of the most widely used techniques in computer vision. Applications range from optical flow and tracking to layered motion, mosaic construction, and face coding. Numerous algorithms have been proposed and a wide variety of extensions have been made to the original formulation. We present an overview of image alignment, describing most of the algorithms and their extensions in a consistent framework. We concentrate on the inverse compositional algorithm, an efficient algorithm that we recently proposed. We examine which of the extensions to Lucas-Kanade can be used with the inverse compositional algorithm without any significant loss of efficiency, and which cannot. In this paper, Part 1 in a series of papers, we cover the quantity approximated, the warp update rule, and the gradient descent approximation. In future papers, we will cover the choice of the error function, how to allow linear appearance variation, and how to impose priors on the parameters.

Virtual Recovery of the Depleted Art Object based on AR Technology

Article

Jan 2007

ARCHEOGUIDE: First results of an augmented reality

Article

Special Issue of Computer-Aided Design on virtual and augmented reality technologies in product design

Article

May 2010
COMPUT AIDED DESIGN

Experiments on the Virtual City: Three-Dimensional Reconfigurations of Missing, Never Realized or Destroyed Urban Areas

Article

Jun 2011

We present two works on the "Virtual City" held at Palermo. They are about the three-dimensional reconfiguration of urban places that show evidence of "invisible" architecture, this is because in one case there were ancient monuments now destroyed, or in another case projects designed for the urban void and never made. The aim is to involve new forms of communication for the development of cultural tourism.

Virtual Reconstruction of FouGuang Temple Based on Virtual Reality

Article

Oct 2008

Shi Zhu Lu

Virtual reality is a combination of image processing, computer graphics, computer vision, human- comp interaction and so on. With virtual reality, we have made a virtual reconstruction to FouGuang Temple. The process of virtual reconstruction consists of data acquiring, modeling, render and real-time browsing system designing. Our system selects single sight of FouGuang Temple as example, supporting the browsing modes including free browsing. It provides a beneficial attempt to applying of virtual reality technology to the protection of Chinese history architecture.

Virtual Recovery of the Deteriorated Art Object based on AR Technology

Conference Paper

Jul 2007
IEEE Comput Soc Conf Comput Vis Pattern Recogn

In this paper, the virtual restoration method of the art piece in the real world is proposed. The correction pattern for restoration is generated from non damaged object's image scanned in advance. As the method to recover the beauty, the marker tracking is used to detect art piece position and the correction image is projected from calibrated LCD projector. In the experiment, the color restoration by a sample image was performed. From the experimental results, the ability of restoration of the discoloring was confirmed in the real world imaginary by the correction image projection.

Research on Method of 3D Reconstruction of Ancient Architecture (Nanputuo Temple)

Conference Paper

Oct 2008

In this paper, regarding the Great Majesty Hall of the NanPutuo Temple as a virtual modeling object, the research on the new technology of 3D reconstruction of the ancient architecture has been done, in which 3DS MAX and MultiGen Creator were applied. The conflict was solved between precision and the amount of data existing in the process of 3D modeling virtual ancient architecture. The practical project has proved that the ancient architecture model made in this way has gotten living effect in roaming system, at the mean time it satisfies the data demand of real-time rendering.

Practice of Using Virtual Reconstruction in the Restoration of Monumental Painting of the Church of the Transfiguration of Our Saviour on Nereditsa Hill

Conference Paper

Aug 2011

St. Petersburg State University has developed a method for restoration of partially or completely lost monumental paintings. As an example and a practical ap-plication of this new technology a virtual reconstruction of the fresco paintings of the Church of the Transfiguration of Our Savior on Nereditsa Hill, a church that was almost completely destroyed during the Second World War was completed. The basic sources of virtual reconstruction were archaeological materials, archival and contemporary historical documents, architectural and art papers, and scientific research in this field. Two main methods were used in the process of virtual reconstruction: Computer graphics technology and analog pictorial reconstruction. The first method makes it possible to accomplish the work of reconstruction with complete fidelity, whereas the second method helps to convey the artist's style, so as to reproduce the form, direction, and strength of the artist's touch and texture of the frescos. The results of the project can be used for further practical work for the restoration of the object. The methodology developed by authors of the project may open new possibilities for the restoration of other fresco ensembles.

Virtual Completion of Facial Image in Ancient Murals

Conference Paper

Jun 2011

Restore damaged face images is an important issue for ancient murals conservation and research. Since the structure and style of facial image varies with identity, dynasties, location and artists, it is a time consuming and challenging job for researchers to achieve objective and precise restoration. In this paper, we propose a novel facial image completion method based on a semantic learning framework. By few user interactions, our approach can fill missing regions with a reconstructed face which is based on candidate images with high semantic similarity. Since most of the ancient murals contain deteriorations, to archive visual pleasant result, we adopt a PCA based region blending method to produce a seamless completion. Moreover, for the criteria of mural restoration are evaluated in the view of both scientific and art, our completion process can provide the researcher an opportunity to preview a diverse set of the final results virtually before they actually start their job. The real applications in Dunhuang mural show the efficiency and effectiveness of our approach. Furthermore, these experiments suggested the potential applications for restoration facial image in ancient line drawing.