Alexey Zalesny’s research while affiliated with Swiss Federal Institute of Technology in Lausanne and other places

This chapter discusses the goals of the Murale project, an Information Society Technologies (IST) project, which is funded by the European Commission in order to advance the use of computer technology in the field of archaeology. The Murale project aims to offer solutions on the basis of photo-realistic modelling tools. The creation of the Murale project allowed archaeologists to solve old tasks with new means. This new technology has been applied to the Sagalassos site in the hopes of creating a convincing impression of how this Turkish province developed over the centuries. In this chapter, the focus is on the work carried out by three of the partners of the Murale: ETH Zurich, Eyetronics, and the University of Leuven. The results of their work predominantly pertain to 3D shape acquisition and image-based texture synthesis.

Working with IBR/BTF data requires a complete calibration. Modern setups speed up recordings by using multiple lamps and cameras. Therefore, the calibration task gets more time consuming and challenging, especially for an accurate calibration of multiple light sources. Most works are dedicated to the calibration of cameras, whereas the light field calibration problem remains as a rule overlooked. Our experiments have shown that the spatial variance of light strength can be vigorous, inducing serious damage to the IBR/BTF data. We propose a novel method based on Helmholtz reciprocity, which derives light field information directly from the target IBR/BTF data rather than from specially dedicated calibration objects. Instead of repeating the recording of a huge number of images, only one additional image is needed.

Many textures require complex models to describe their intricate structures. Their modeling can be simplified if they are considered composites of simpler subtextures. After an initial, unsupervised segmentation of the composite texture into the subtextures, it can be described at two levels. One is a label map texture, which captures the layout of the different subtextures. The other consists of the different subtextures. This scheme has to be refined to also include mutual influences between textures, mainly found near their boundaries. The proposed composite texture model also includes these. The paper describes an improved implementation of this idea. Whereas in a previous implementation subtextures and their interactions were synthesized sequentially, this paper proposes a parallel implementation, which yields results of higher quality.

Textures often result from complicated surface geome-tries. We propose a method that extracts such geometry from raw BTF data. Exploiting the huge amount of input data, only a few and rather weak assumptions about reflectance and geometry suffice. A key element of our approach are viewpoint robustness of reflectance features. We propose a few and compare them for 3D reconstruction.

The purpose of this work is to synthesize textures of rough, real world surfaces under freely chosen viewing and illumination directions. Moreover, such textures are produced for continuously changing directions in such a way that the different textures are mutually consistent, i.e. emulate the same piece of surface. This is necessary for 3D animation. It is assumed that the mesostructure (small-scale) geometry of a surface is not known, and that the only input consists of a set of images, taken under different viewing and illumination directions. These are automatically aligned to build an appropriate bidirectional texture function (BTF). Directly extending 2D synthesis methods for pixels to complete BTF columns has drawbacks which are exposed, and a superior sequential but highly parallelizable algorithm is proposed. Examples demonstrate the quality of the results.

The maximum entropy principle is a cornerstone of FRAME (Filters, RAndom fields, and Maximum Entropy) model considered at times as a first-ever step towards a universal theory of texture modelling or even as "the in-evitable texture model". This paper disputes such opinions. That a wealth of exponential families of probability distri-butions is deduced from the ME principle is well known for decades. The ME property by itself in no way leads to an adequate probabilistic description, and to model a particu-lar texture, specific limitations have to be imposed on signal statistics. Frequency distributions of outputs from a bank of linear filters (the second FRAME's cornerstone) are hardly the only choice outperforming all other alternatives. The paper points also to other hidden drawbacks of FRAME.

In the construction of 3D models of archaeological sites, especially during the anastylosis (piecing together dismembered remains of buildings), much more emphasis has been placed on the creation of the 3D shapes rather than on their textures. Nevertheless, the overall visual impression will often depend more on these textures than on the precision of the underlying geometry. This paper proposes a hierarchical texture modeling and synthesis technique to simulate the intricate appearances of building materials and landscapes. A macrotexture or "label map" prescribes the layout of microtextures or "subtextures". The system takes example images, e.g. of a certain vegetation landscape, as input and generates the corresponding composite texture models. From such models, arbitrary amounts of similar, non-repetitive texture can be generated (i.e. without verbatim copying). The creation of the composite texture models follows a kind of bootstrap procedure, where simple texture features help to generate the label map and then more complicated texture descriptions are called on for the subtextures.

Over the years archaeologists have been swift to embrace new advances in technology that allow them to more comprehensively document the results of their work. Today it is commonplace to find information technologies, in the form MS Office-type tools with some CAD and GIS, deployed for primary data capture, analysis, presentation and publication. While these computing technologies can be used effectively to record and interpret archaeological sites, the radical developments in 3D recording, reconstruction and visualisation tools have had relatively limited impact upon the archaeological community. This is unfortunate as these new technologies have the potential to (a) enable the archaeologists to record their unrepeatable experiments to unprecedented levels of accuracy, (b) enable the archaeologists to reconstruct artefacts such as pottery from sherds, textures and sites from different eras (c) visualise the wealth of excavated information in dynamic new ways away from the archaeological site during post-excavation analysis, (d) make this wealth of detail available to the scholarly community as part of the publication process and secure its digital longevity through its deposition in a trusted digital library/archive and (e) communicate the excitement and importance of their archaeological site and its finds to an interested non-academic audience. This paper describes the overall concept of the EU funded project, 3D Measurement and Virtual Reconstruction of Ancient Lost Worlds of Europe (3D MURALE), that has developed and created a set of low-cost multimedia tools for recording, reconstructing, encoding, and visualising archaeological artefacts and site.

The synthesis of many textures can be simplified if they are first decomposed into simpler subtextures. Such bootstrap procedure allows to first consider a `label texture', that captures the layout of the subtextures, after which the subtextures can be filled in. A companion paper focuses on this latter aspect. This paper describes an approach to arrive at the label texture. Pairwise pixel similarities are computed by matching simple color and texture features histograms in pixel neighbourhoods, using efficient mean-shift search. A graph-based, unsupervised algorithm segments the image into subtextures, based on the similarities.