Jon Yngve Hardeberg

• Norwegian University of Science and Technology

Archaeological textiles are a valuable source for understanding past cultures. However, textiles are vulnerable to decomposition and often found in a fragmentary state. Textile experts rely on their eyes and experience for manually matching similar fragments to reconstruct the original item. This requires much time and effort, as well as physical i...

In spectral imaging, the constraints imposed by hardware often lead to a limited spatial resolution within spectral filter array images. On the other hand, the process of demosaicking is challenging due to intricate filter patterns and a strong spectral cross correlation. Moreover, demosaicking and super resolution are usually approached independen...

Reflectance Transformation Imaging (RTI) is an imaging technique used to analyze objects or surfaces by capturing their appearance under varying illumination directions. This paper proposes two self-supervised learning algorithms to classify surfaces according to their reflectance profiles. The classification problem is addressed using K-means and...

Spectral imaging has revolutionisedvarious fields by capturing detailed spatial and spectral information. However, its high cost and complexity limit the acquisition of a large amount of data to generalise processes and methods, thus limiting widespread adoption. To overcome this issue, a body of the literature investigates how to reconstruct spect...

Scene recognition is the task of identifying the environment shown in an image. Spectral filter array cameras allow for fast capture of multispectral images. Scene recognition in multispectral images is usually performed after demosaicing the raw image. Along with adding latency, this makes the classification algorithm limited by the artifacts prod...

The application of materials with changing visual properties with lighting and observation directions has found broad utility across diverse industries, from architecture and fashion to automotive and film production. The expanding array of applications and appearance reproduction requirements emphasizes the critical role of material appearance mea...

Archaeological artifacts play important role in understanding the past developments of the humanity. However, the artifacts are often highly fragmented and degraded, with many details and parts missing due to centuries’ long degradation. Archaeologists and conservators attempt to reconstruct the original state of the objects either physically or vi...

The value of three-dimensional virtual objects are proven in a great variety of applications; their flexibility allowing for a substantial amount of utilization purposes. In cultural heritage this has been used for many years already, and the amount of users continue to grow as acquisition methods and implementations are becoming more approachable....

This paper investigates the optimization of acquisition in Reflectance Transformation Imaging (RTI). Current methods for RTI acquisition are either computationally expensive or impractical, which leads to continued reliance on conventional classical methods like homogenous equally spaced methods in museums. We propose a methodology that is aimed at...

When characterising a digital camera spectrally or colourimetrically, the camera response to a generally diffusely reflecting colour chart is often employed. The recorded responses to the light incident from each colour patch are typically not linearly related to the power of the irradiance on the chart, and the irradiance varies with position on t...

Reflectance Transformation Imaging (RTI) is a technique that provides an enhanced visualization experience. The current acquisition methods for Reflectance Transformation Imaging (RTI) are time consuming and computationally expensive. This work investigates the idea of getting best light positions for RTI acquisition using surface topography. We pr...

Background Ultrasound-guided laparoscopic common bile duct exploration (LCBDE) is the surgical management of choledocholithiasis. The procedure presents significant benefits to patients but still fails to be generalised because of the complex set of skills it requires. A simulator for ultrasound-guided LCBDE would allow trainee surgeons as well as...

In a previous article, we modelled the spectral and temporal dimensions of the photodegradation behaviour of pigments in the painting “A Japanese Lantern” by Oda Krohg. In particular, we extracted the endmembers and spectral fading rate of pigments by applying tensor decomposition on a time-series of spectroscopic point measurements. Now, we captur...

Over the last decade, hyperspectral imaging has become a popular technique for the non-invasive identification and mapping of painting materials in many typologies of artworks, thanks to the possibility of obtaining spectral information over the spatial region. A few attempts have also been made on stained-glass windows to identify the chromophore...

Photo-sensitive materials tend to change with exposure to light. Often, this change is visible when it affects the reflectance of the material in the visible range of the electromagnetic spectrum. In order to understand the photo-degradation mechanisms and their impact on fugitive materials, high-end scientific analysis is required. In a two-part a...

This paper outlines the use of bidirectional reflectance measurements for the characterisation and evaluation of appearance changes in gilded surfaces caused by varnishing and cleaning. Oil and water gilding mock-ups representative of a 15th-century panel painting were varnished, and a selection of four varnish removal methods was applied. By measu...

Materials with special appearance properties such go-niochromatic materials require complex bidirectional measure-ments to properly characterise their colour and gloss. Normally,these measurements are performed by goniospectrophotometerswhich are expensive and not commonly available. In this papera flexible imaging system composed of a snapshot mul...

In the past years, hyperspectral imaging has become a popular technique for the non-invasive investigation of works of art and has been extensively used for the analysis of pigments in paintings and manuscripts. The application of spectral imaging on stained glass however is very limited. Due to their transparency, imaging of stained glass presents...

The characterization of a painter’s style is useful for a series of applications, such as documenting art history, planning style-aware conservation and restoration, and discarding forgery attempts. In this work, we propose a method to assign paintings to the right artist with two strategies: traditional machine learning and deep learning. In parti...

This article demonstrates a transition in 14th-century Norway to the use of darker inks, visualized with low-cost digital photographs in 950 nm infrared light (IR). In these photographs, the transition manifests itself in increased contrast, on average, in 101 Norwegian dated documents (charters) written immediately after the year 1400, as compared...

Chasing is one of the main decorative techniques found in metal work. The imprints left by different chasing tools are unique not only to the tool, but also to the chaser. The analysis and characterisation of toolmarks provide vast information that is valuable for the understanding of an object. Microtopography is a non-contact imaging technique th...

Complexity is one of the major attributes of the visual perception of texture. However, very little is known about how humans visually interpret texture complexity. A psychophysical experiment was conducted to visually quantify the seven texture attributes of a series of textile fabrics: complexity, color variation, randomness, strongness, regulari...

This paper outlines the use of bidirectional reflectance measurements for thecharacterisation and evaluation of appearance changes in gilded surfaces causedby varnishing and cleaning. Oil and water gilding mock-ups representative of a15th century panel painting were varnished, and a selection of four varnishcleaning methods were applied. By measuri...

In the last several years, deep learning has been introduced to recover a hyperspectral image (HSI) from a single RGB image and demonstrated good performance. In particular, attention mechanisms have further strengthened discriminative features, but most of them are learned by convolutions with limited receptive fields or require much computational...

The work presented in this paper is part of a wider research project, which aims at documenting and analyzing stained glass windows by means of hyperspectral imaging. This technique shares some similarities with UV-VIS-IR spectroscopy, as they both provide spectral information; however, spectral imaging has the additional advantage of providing spa...

A metrological hyperspectral texture descriptor, Relative Spectral Difference Occurrence Matrix (RSDOM) has been proposed in previous work. Due to the probabilistic nature of the feature, Kullback-Leibler divergence (KLD) is used for similarity measure. In the literature, several approaches exist to approximate KLD which is otherwise computationall...

The development of innovative applications of imaging technologies for monitoring change in cultural heritage objects is central to the CHANGE-ITN project. Within this framework, the authors’ ongoing work targets the harnessing of hyperspectral imaging (HSI) for the documentation of cleaning treatments on the monumental University of Oslo Aula unva...

Translucency optically results from subsurface light transport and plays a considerable role in how objects and materials appear. Absorption and scattering coefficients parametrize the distance a photon travels inside the medium before it gets absorbed or scattered, respectively. Stimuli produced by a material for a distinct viewing condition are p...

Multispectral images contain more spectral information of the scene objects compared to color images. The captured information of the scene reflectance is affected by several capture conditions, of which the scene illuminant is dominant. In this work, we implemented an imaging pipeline for a spectral filter array camera, where the focus is the esti...

Patinas are a form of metal polychromy used to decorate metallic artworks. Due to the nature of the metallic surface, their colour and gloss is perceived differently when the illumination and viewing directions vary. Sparkle effect on surfaces is a physical phenomenom caused by micro-facets on the surface coating which are also perceived with chang...

In recent years, smartphone-based colour imaging systems are being increasingly used for Neonatal jaundice detection applications. These systems are based on the estimation of bilirubin concentration levels that correlates with newborns’ skin colour images corresponding to total serum bilirubin (TSB) and transcutaneous bilirubinometry (TcB) measure...

We have developed a system to measure both the optical properties of facial skin and the three-dimensional shape of the face. To measure the three-dimensional facial shape, our system uses a light-field camera to provide a focused image and a depth image simultaneously. The light source uses a projector that produces a high-frequency binary illumin...

Translucency is an optical and a perceptual phenomenon that characterizes subsurface light transport through objects and materials. Translucency as an optical property of a material relates to the radiative transfer inside and through this medium, and translucency as a perceptual phenomenon describes the visual sensation experienced by humans when...

The accuracy of recovered spectra from camera responses mainly depends on the spectral estimation algorithm used, the camera and filters selected, and the light source used to illuminate the object. We present and compare different light source spectrum optimization methods together with different spectral estimation algorithms applied to reflectan...

The digital reproduction of a historical motion picture should resemble as much as possible the analog film projection at the time of the movie release. Nowadays, practices of capturing digital images of films do not properly consider the fundamental elements and conditions of the original film projection. The typical rigid three-band (RGB) capture...

Perception of appearance of different materials and objects is a complex psychophysical phenomenon and its neurophysiological and behavioral mechanisms are far from being fully understood. The various appearance attributes are usually studied separately. In addition, no comprehensive and functional total appearance modelling has been done up-to dat...

This study investigates the potential impact of subsurface light transport on gloss perception for the purposes of broadening our understanding of visual appearance in computer graphics applications. Gloss is an important attribute for characterizing material appearance. We hypothesize that subsurface scattering of light impacts the glossiness perc...

In this study, optical multispectral sensors based on perovskite semiconductors have been proposed, simulated, and characterized. The perovskite material system combined with the 3D vertical integration of the sensor channels allows for the realization of sensors with high sensitivities and a high spectral resolution. The sensors can be applied in...

Texture characterization from the metrological point of view is addressed in order to establish a physically relevant and directly interpretable feature. In this regard, a generic formulation is proposed to simultaneously capture the spectral and spatial complexity in hyperspectral images. The feature, named relative spectral difference occurrence...

In an online OSA Incubator Meeting, top industry and academic researchers explored the importance of accounting for aspects of the human visual system to take augmented and virtual reality to the next step.

The virtual inpainting of artworks provides a nondestructive mode of hypothesis visualization, and it is especially attractive when physical restoration raises too many methodological and ethical concerns. At the same time, in Cultural Heritage applications, the level of details in virtual reconstruction and their accuracy are crucial. We propose a...

Skin cancer is a major public health problem, with millions newly diagnosed cases each year. Melanoma is the deadliest form of skin cancer, responsible for the most over 6500 deaths each year in the US, and the rates have been rising rapidly over years. Because of this, a lot of research is being done in automated image-based systems for skin lesio...

A colour appearance model based on a uniform colour space is proposed. The proposed colour appearance model, ZCAM, comprises of comparatively simple mathematical equations, and plausibly agrees with the psychophysical phenomenon of colour appearance perception. ZCAM consists of ten colour appearance attributes including brightness, lightness, colou...

In this paper, we evaluate the quality of reconstruction i.e. relighting from images obtained by a newly developed multispectral reflectance transformation imaging (MS-RTI) system. The captured MS-RTI images are of objects with different translucency and color. We use the most common methods for relighting the objects: polynomial texture mapping (P...

Color image sensing by a smartphone or digital camera is usually performed by an array of sensor elements covered by color filters. The optical color filters cause color errors like the color aliasing or Color Moiré effect. Furthermore, the quantum efficiency of typical conventional optical color sensor consisted of side-by-side arranged color filt...

Although mixture of LEDs is being considered as a simulator of the CIE daylight series, the performance of the simulations is highly dependent on the SPD of the selected LEDs. An algorithm for selection of the best LEDs for simulation of the CIE daylight series is helpful in this regard. To address this problem, using 200 imaginary light primaries...

Stained-glass windows are very particular artifacts; they not only have an intrinsic artistic and historical meanings, but also a functional role, strictly connected to the buildings where they were originally placed. The investigation of these artifacts is a challenging research opportunity, due to their optical, chemical and physical characterist...

Skin cancer is a major public health problem, with millions newly diagnosed cases each year. Melanoma is the deadliest form of skin cancer, responsible for the most over 6500 deaths each year in the US, and the rates have been rising rapidly over years. Because of this, a lot of research is being done in automated image-based systems for skin lesio...

Color image sensing by a smartphone or digital camera employ sensor elements with an array of color filters for capturing basic blue, green, and red color information. However, the normalized optical efficiency of such color filter-based sensor elements is limited to only 1/3. Optical detectors based on perovskites are described that can overcome t...

A hyperspectral sensor is able to acquire the physical and optical properties of surfaces. On the other hand, applications in climate research, ecology, cultural heritage, and industry require metrological solutions for diagnosis and control quality. Thus there is a direct relationship between the physical properties of materials and their processi...

In this study, the results from a round-robin test of hyperspectral imaging systems are presented and analyzed. Fourteen different pushbroom hyperspectral systems from eight different institutions were used to acquire spectral cubes from the visible, near infra-red and short-wave infra-red regions. Each system was used to acquire a common set of ta...

Multi-instance (MI) learning is a branch of machine learning, where each object (bag) consists of multiple feature vectors (instances)—for example, an image consisting of multiple patches and their corresponding feature vectors. In MI classification, each bag in the training set has a class label, but the instances are unlabeled. The instances are...

Optical color sensors based on perovskites are described. These sensors overcome the limits of conventional image sensors used in smartphones and digital cameras. The sensors allow for detecting the primary colors red, green, and blue without using optical filters. The color sensors consist of vertically stacked diodes using perovskite alloys. The...

Image sensing technology has a great impact to our daily life as well as the entire society, such as health, safety and security, communication systems, and entertainment. The conventional optical color sensors consist of side by side arranged optical filters for three basic colors (blue, green, and red). Hence, the efficiency of such optical color...

Resolution in a projected display is traditionally defined by the number of pixels in the projector's spatial light modulator (SLM). In recent years, different techniques that increase the resolution on the screen above the number of SLM pixels have gained popularity. In one such technique, called pixel‐shifting or shifted‐superimposition, the disp...

This work takes a step towards understanding fundamental aspects of appearance change in cultural heritage. Particularly, we concentrate on the case study of the Hedal Madonna – a polychrome wood sculpture dated to the mid-1200s, and an important object of ecclesiastical art and Norwegian heritage. It is covered with a layered coating that gives ri...

Caustics projected onto the surface carry very interesting information regarding the material they are cast by. It has been observed in previous studies that caustics could be a widely used cue for translucency assessment by human subjects. We hypothesize that changing the reflectance properties of the surface an object is placed on, and removal of...

Previous research investigated the perceptual dimensionality of achromatic reflection of opaque surfaces, by using either simple analytic models of reflection, or measured reflection properties of a limited sample of materials. Here we aim to extend this work to a broader range of simulated materials. In a first experiment, we used sparse multidime...

Comparing and selecting an adequate spectral filter array (SFA) camera is application-specific and usually requires extensive prior measurements. An evaluation framework for SFA cameras is proposed and three cameras are tested in the context of skin analysis. The proposed framework does not require application-specific measurements and spectral sen...

Translucency is one of the major appearance attributes. Apparent translucency is impacted by various factors including object shape and geometry. Despite general proposals that object shape and geometry have a significant effect on apparent translucency, no quantification has been made so far. Quantifying and modeling the impact of geometry, as wel...

Texture analysis and characterization based on human perception has been continuously sought after by psychology and computer vision researchers. However, the fundamental question of how humans truly perceive texture still remains. In the present study, using a series of textile samples, the most important perceptual attributes people use to interp...

Opacity is an important appearance attribute in the textile industry. Obscuring power and the way textile samples block light can define product quality and customer satisfaction in the lingerie, shirting, and curtain industries. While the question whether opacity implies the complete absence of light transmission remains open, various factors can...

A perceptual study was conducted to enhance colour image quality in terms of naturalness and preference using perceptual scales of saturation and vividness. Saturation scale has been extensively used for this purpose while vividness has been little used. We used perceptual scales of a recently developed colour appearance model based on J z a z b z...

We develop a spectral-spatial feature, Relative Spectral Difference Occurrence Matrix (RSDOM) for hyperspectral texture recognition. Inspired by Julesz's conjecture, the proposed feature is based on spectral difference approach and respects the metrological constraints. It simultaneously considers the distribution of spectra and their spatial arran...

Gloss is widely accepted as a surface- and illuminationbased property, both by definition and by means of metrology. However, mechanisms of gloss perception are yet to be fully understood. Potential cues generating gloss perception can be a product of phenomena other than surface reflection and can vary from person to person. While human observers...

Texture analysis and characterization based on human perception has been continuously sought after by psychology and computer vision researchers. However, the fundamental question of how humans truly perceive texture still remains. In the present study, using a series of textile samples, the most important perceptual attributes people use to interp...

Opacity is an important appearance attribute in the textile industry. Obscuring power and the way textile samples block light can define product quality and customer satisfaction in the lingerie, shirting, and curtain industries. While the question whether opacity implies the complete absence of light transmission remains open, various factors can...