Heikki Kälviäinen

• Professor, D.Sc. (Tech.)
• Professor at Lappeenranta – Lahti University of Technology LUT

Figure skating, known as the "Art on Ice," is among the most artistic sports, challenging to understand due to its blend of technical elements (like jumps and spins) and overall artistic expression. Existing figure skating datasets mainly focus on single tasks, such as action recognition or scoring, lacking comprehensive annotations for both techni...

Emotion understanding is a critical yet challenging task. Most existing approaches rely heavily on identity-sensitive information, such as facial expressions and speech, which raises concerns about personal privacy. To address this, we introduce the De-identity Multimodal Emotion Recognition and Reasoning (DEEMO), a novel task designed to enable em...

Facial Action Units (AUs) detection is a cornerstone of objective facial expression analysis and a critical focus in affective computing. Despite its importance, AU detection faces significant challenges, such as the high cost of AU annotation and the limited availability of datasets. These constraints often lead to overfitting in existing methods,...

This paper considers open-set recognition (OSR) of plankton images. Plankton include a diverse range of microscopic aquatic organisms that have an important role in marine ecosystems as primary producers and as a base of food webs. Given their sensitivity to environmental changes, fluctuations in plankton populations offer valuable information abou...

Plankton recognition is an important computer vision problem due to plankton's essential role in ocean food webs and carbon capture, highlighting the need for species-level monitoring. However, this task is challenging due to its fine-grained nature and dataset shifts caused by different imaging instruments and varying species distributions. As new...

Image‐based re‐identification of animal individuals allows gathering of information such as population size and migration patterns of the animals over time. This, together with large image volumes collected using camera traps and crowdsourcing, opens novel possibilities to study animal populations. For many species, the re‐identification can be don...

Face anti-spoofing (FAS) plays a vital role in preventing face recognition (FR) systems from presentation attacks. Nowadays, FAS systems face the challenge of domain shift, impacting the generalization performance of existing FAS methods. In this paper, we rethink about the inherence of domain shift and deconstruct it into two factors: image style...

Recent advancements in the automatic re-identification of animal individuals from images have opened up new possibilities for studying wildlife through camera traps and citizen science projects. Existing methods leverage distinct and permanent visual body markings, such as fur patterns or scars, and typically employ one of two strategies: local fea...

Access to large image volumes through camera traps and crowdsourcing provides novel possibilities for animal monitoring and conservation. It calls for automatic methods for analysis, in particular, when re-identifying individual animals from the images. Most existing re-identification methods rely on either hand-crafted local features or end-to-end...

Planktonic organisms including phyto-, zoo-, and mixoplankton are key components of aquatic ecosystems and respond quickly to changes in the environment, therefore their monitoring is vital to follow and understand these changes. Advances in imaging technology have enabled novel possibilities to study plankton populations, but the manual classifica...

Phytoplankton parasites are largely understudied microbial components with a potentially significant ecological influence on phytoplankton bloom dynamics. To better understand the impact of phytoplankton parasites, improved detection methods are needed to integrate phytoplankton parasite interactions into monitoring of aquatic ecosystems. Automated...

Image-based re-identification of animal individuals allows gathering of information such as migration patterns of the animals over time. This, together with large image volumes collected using camera traps and crowdsourcing, opens novel possibilities to study animal populations. For many species, the re-identification can be done by analyzing the p...

Planktonic organisms are key components of aquatic ecosystems and respond quickly to changes in the environment, therefore their monitoring is vital to understand the changes in the environment. Yet, monitoring plankton at appropriate scales still remains a challenge, limiting our understanding of functioning of aquatic systems and their response t...

Automatic game cameras are commonly used for monitoring wildlife as they allow to document of the activity of animals in a non-invasive manner. By utilizing a large number of cameras and identifying individual animals from the images, it is possible to, for example, estimate the population size and study the migration patterns of the animals. Large...

Phytoplankton parasites are largely understudied microbial components with a potentially significant ecological impact on phytoplankton bloom dynamics. To better understand their impact, we need improved detection methods to integrate phytoplankton parasite interactions in monitoring aquatic ecosystems. Automated imaging devices usually produce hig...

In this paper, pelage pattern matching is considered to solve the individual re-identification of the Saimaa ringed seals. Animal reidentification, together with the access to a large amount of image material through camera traps and crowd-sourcing, provides novel possibilities for animal monitoring and conservation. Image retrieval techniques, suc...

Wildlife camera traps and crowd-sourced image material provide novel possibilities to monitor endangered animal species. The massive data volumes call for automatic methods to solve various tasks related to population monitoring, such as the re-identification of individual animals. The Saimaa ringed seal (Pusa hispida saimensis) is an endangered su...

Plankton communities form the basis of aquatic ecosystems and elucidating their role in increasingly important environmental issues is a persistent research question. Recent technological advances in automated microscopic imaging, together with cloud platforms for high-performance computing, have created possibilities for collecting and processing...

We propose a method for Saimaa ringed seal (Pusa hispida saimensis) re-identification. Access to large image volumes through camera trapping and crowdsourcing provides novel possibilities for animal monitoring and conservation and calls for automatic methods for analysis, in particular, when re-identifying individual animals from the images. The pr...

Wildlife camera traps and crowd-sourced image material provide novel possibilities to monitor endangered animal species. However, massive image volumes that these methods produce are overwhelming for researchers to go through manually which calls for automatic systems to perform the analysis. The analysis task that has gained the most attention is...

Automated wildlife reidentification has attracted increasing attention in recent years as it provides a non-invasive tool to identify and to track individual wild animals over time. In this paper, the first steps are taken towards the automatic photo-identification of the Ladoga ringed seals ( Pusa hispida ladogensis ). A method is proposed that ta...

Monitoring plankton is important as they are an essential part of the aquatic food web as well as producers of oxygen. Modern imaging devices produce a massive amount of plankton image data which calls for automatic solutions. These images are characterized by a very large variation in both the size and the aspect ratio. Convolutional neural networ...

The quality of the end product in the sawmill industry is highly dependent on the distribution of knots. If the internal structure of the logs used as raw material were known it would be possible to optimize the sawing process by controlling the locations of individual knots in the resulting boards. Methods such as Computer Tomography or Magnetic R...

This paper introduces a novel method for segmentation of clustered partially overlapping convex objects in silhouette images. The proposed method involves three main steps: pre-processing, contour evidence extraction, and contour estimation. Contour evidence extraction starts by recovering contour segments from a binarized image by detecting concav...

Plankton communities form the basis of aquatic ecosystems and elucidating their role in increasingly important environmental issues is a constantly present research question. The concealed plankton community dynamics reflect changes in environmental forcing, growth traits of competing species, and multiple food web interactions. Recent technologica...

In order to develop better touch and gesture user interfaces, it is important to be able to measure how humans move their hands while interacting with technical devices. The recent advances in high-speed imaging technology and in image-based object tracking techniques have made it possible to accurately measure the hand movement from videos without...

Pectoral muscle segmentation is a crucial step in various computer-aided applications of breast Magnetic Resonance Imaging (MRI). Due to imaging artifact and homogeneity between the pectoral and breast regions, the pectoral muscle boundary estimation is not a trivial task. In this paper, a fully automatic segmentation method based on deep learning...

Transmission electron microscopy (TEM) provides information about Inorganic nanoparticles that no other method is able to deliver. Yet, a major task when studying Inorganic nanoparticles using TEM is the automated analysis of the images, i.e. segmentation of individual nanoparticles. The current state-of-the-art methods generally rely on binarizati...

Tracking timber in the sawmill environment from the raw material (logs) to the end product (boards) provides various benefits including efficient process control, the optimization of sawing, and the prediction of end-product quality. In practice, the tracking of timber through the sawmilling process requires a methodology for tracing the source of...

Segmentation of overlapping convex objects has various applications, for example, in nanoparticles and cell imaging. Often the segmentation method has to rely purely on edges between the background and foreground making the analyzed images essentially silhouette images. Therefore, to segment the objects, the method needs to be able to resolve the o...

This paper considers the wood species identification from images of boards. The identification using only visual features of the surface is a challenging task even for an expert. The task becomes especially difficult when the wood species are from the same family. We propose a CNN based framework for the fine-grained classification of wood species....

The quality control of timber products is vital for the sawmill industry pursuing more efficient production processes. This paper considers the automatic detection of mechanical damages in wooden board surfaces occurred during the sawing process. Due to the high variation in the appearance of the mechanical damages and the presence of several other...

The conservation efforts of the endangered Saimaa ringed seal depend on the ability to reliably estimate the population size and to track individuals. Wildlife photo-identification has been successfully utilized in monitoring for various species. Traditionally, the collected images have been analyzed by biologists. However, due to the rapid increas...

This paper considers the usability of stereoscopic 3D touch displays. For this purpose extensive subjective experiments were carried out and the hand movements of test subjects were recorded using a two-camera setup consisting of a high-speed camera and a standard RGB video camera with different viewing angles. This produced a large amount of video...

Wildlife photo-identification is a commonly used technique to track animal populations over time. Nowadays, due to large image data sets, automated photo-identification is an emerging research topic. To improve the accuracy of identification methods, it is useful to segment the animal from the background. In this paper we evaluate the suitability o...

In this work, we studied whether coalescence has a direct effect on mass transfer in copper extraction. The coalescence of pendant and sessile droplets containing diluted Acorga M5640 in an aqueous copper sulfate solution was recorded on video. The videos were analyzed to determine droplet rest times, sizes, and concentrations. Concentration measur...

Automatic traffic sign inventory and simultaneous condition analysis can be used to improve road maintenance processes, decrease maintenance costs, and produce up-to-date information for future intelligent driving systems. The goal of this research is to combine automatic traffic sign detection and classification with traffic sign inventory and con...

Three-dimensional human-computer interaction has the potential to form the next generation of user interfaces and to replace the current 2D touch displays. To study and to develop such user interfaces, it is essential to be able to measure how a human behaves while interacting with them. In practice, this can be achieved by accurately measuring han...

Detection, counting and characterization of bubbles, that is, transparent objects in a liquid, is important in many industrial applications. These applications include monitoring of pulp delignification and multiphase dispersion processes common in the chemical, pharmaceutical, and food industries. Typically the aim is to measure the bubble size di...

In order to monitor an animal population and to track individual animals in a non-invasive way, identification of individual animals based on certain distinctive characteristics is necessary. In this study, automatic image-based individual identification of the endangered Saimaa ringed seal (Phoca hispida saimensis) is considered. Ringed seals have...

Segmentation of overlapping convex objects has gained a lot of attention in numerous biomedical and industrial applications. A partial overlap between two or more convex shape objects leads to a shape with concave edge points that correspond to the intersections of the object boundaries. Therefore, it is a common practice to utilize these concave p...

Automatic traffic sign inventory and simultaneous condition analysis can be used to improve road maintenance processes, decrease maintenance costs, and produce up-to-date information for future intelligent driving systems. The goal of this research is to combine automatic traffic sign detection and classification with traffic sign inventory and con...

Mixing performance in continuous rotary drums has been studied. The video analysis method was developed to evaluate different configurations of straight lifters in the rotary drum. The method converts a captured video into a single image, called stack image. The color marker tracking was estimated based on the color saturation of the stack image. C...

This paper presents a novel method for the segmentation of partially overlapping convex shape objects in silhouette images. The proposed method involves two main steps: contour evidence extraction and contour estimation. Contour evidence extraction starts by recovering contour segments from a binarized image using concave contour point detection. T...

The proposed image analysis method allows the measurement of organic phase droplet sizes, velocities, and copper concentrations in single droplet column copper extraction using hydroxyoxime complexation. The method uses image acquisition sequences from video, detection of moving droplets, binarization of background subtracted images, and noise redu...

Powder feeding is one of the single most important parameters that defines a qualitative outcome of the cladding process. This study focuses on how powder feeding angle and powder carrying gas flow rate affect the powder cloud behavior in laser cladding with scanning optics. Focus of this study is to provide the knowledge on how scanned laser beam...

The paper proposes a method for the detection of bubble-like transparent objects in a liquid. The detection problem is non-trivial since bubble appearance varies considerably due to different lighting conditions causing contrast reversal and multiple interreflections. We formulate the problem as the detection of concentric circular arrangements (CC...

Wildlife photo-identification is a commonly used technique to identify and track individuals of wild animal populations over time. It has various applications in behavior and population demography studies. Nowadays, mostly due to large and labor-intensive image data sets, automated photo-identification is an emerging research topic. In this paper ,...

This paper presents a novel method for the segmentation of partially overlapping nanoparticles with a convex shape in silhouette images. The proposed method involves two main steps: contour evidence extraction and contour estimation. Contour evidence extraction starts with contour segmentation where contour segments are recovered from a binarized i...

Understanding how a human behaves while performing human-computer interaction tasks is essential in order to develop better user interfaces. In the case of touch and gesture based interfaces, the main interest is in the characterization of hand movements. The recent developments in imaging technology and computing hardware have made it attractive t...

Progress of a new imaging-based measurement of oxygen bubble size was continued by developing an automated method for bubble detection from image data. The imaging application was found to be able to measure oxygen bubble size in mill conditions when using imaging apparatus connected to the sampling valve after the first oxygen stage mixer in a Fin...

Measuring the visual quality of printed media is important since printed products have an important role in everyday life. Finding ways to automatically predict the image quality has been an active research topic in digital image processing, but adapting those methods to measure the visual quality of printed media has not been studied often or in d...

A bubble size distribution gives relevant insight into mixing processes where gas-liquid phases are present. The distribution estimation is challenging since accurate bubble detection from images captured from industrial processes is a complicated task due to varying lighting conditions which change the appearance of bubbles considerably. In this p...

Bubble detection is a complicated tasks since varying lighting conditions changes considerably the appearance of bubbles in liquid. The two common techniques to detect circular objects such as bubbles, the geometry-based and appearance-based approaches, have their advantages and weaknesses. The geometry-based methods often fail to detect small blob...

The paper presents a framework for the detection of curvilinear objects in images. Such objects are challenging to be described by a geometrical model, and although they appear in a number of applications, the problem of detecting curvilinear objects has drawn limited attention. The proposed approach starts with an edge detection algorithm after wh...

The ultimate challenge of image categorisation is unsupervised object discovery, where the selection of categories and the assignments of given images to these categories are performed automatically. The unsupervised setting prohibits the use of the best discriminative methods, and in Tuytelaars et al. [30] the standard Bag-of-Features (BoF) approa...

One important aspect of assessing the quality in pulp and papermaking is dirt particle counting and classification. Knowing the number and types of dirt particles present in pulp is useful for detecting problems in the production process as early as possible and for fixing them. Since manual quality control is a time-consuming and laborious task, t...

We address the performance evaluation practices for developing medical image analysis methods, in particular, how to establish and share databases of medical images with verified ground truth and solid evaluation protocols. Such databases support the development of better algorithms, execution of profound method comparisons, and, consequently, tech...

Powder feeding is one of the single most important factors which defines the qualitative outcome of the cladding process. If the powder feeding or a powder cloud suffers from any form of interference, it will directly affect the quality of the clad bead. This study thus focuses on the effect of the powder feeding angle and the flow rate of the powd...

Powder feeding is one of the single most important factors which defines the qualitative outcome of the cladding process. If the powder feeding or a powder cloud suffers from any form of interference, it will directly affect the quality of the clad bead. This study thus focuses on the effect of the powder feeding angle and the flow rate of the powd...

Object discovery in visual object categorisation (VOC) is the problem of automatically assigning class labels to objects appearing in given images. To achieve state-of-the-art results in this task, a large set of positive and negative training images from publicly available benchmark data sets have been used to train discriminative classification m...

A method for the detection of bubble-like transparent objects with multiple interfaces in a liquid is proposed. Depending on the lighting conditions, bubble appearance varies significantly, including contrast reversal and multiple inter-reflections. We formulate the bubble detection problem as the detection of Concentric Circular Arrangements (CCA)...

We address performance evaluation practices for developing medical image analysis methods, and contribute to the practice to establish and to share databases of medical images with verified ground truth and solid evaluation protocols. This helps to develop better algorithms, to perform profound method comparisons, including the state-of-the-art met...

This paper considers possibilities which machine vision can provide for quality control along the whole manufacturing line of paper and board products. The scope is from pulping to papermaking, mainly for printing. The motivation of this study comes from the necessity to predict the quality of printing on paper or board, especially in case of image...

Prediction of overall visual quality based on instrumental measurements is a challenging task. Despite the several proposed models and methods, there exists a gap between the instrumental measurements of print and human visual assessment of natural images. In this work, a computational model for representing and quantifying the overall visual quali...

Fine and sparse details appear in many quality inspection applications requiring machine vision. Especially on flat surfaces, such as paper or board, the details can be made detectable by oblique illumination. In this study, a general definition of such details is given by defining sufficient statistical properties from histograms. The statistical...

This invited paper considers the results of the IMAGERET project. The goal of the project is to demonstrate how lesions in a retina caused by diabetic retinopathy can be detected from color fundus images by using machine vision methods. The project consists of the following results: an image annotation tool for medical expert annotation, diabetic r...

In the evaluation of dirt inclusions in paper, the attention is paid not only to the quantity of dirt but also to the type of dirt particles. Automatic classification methods can be designed for the task, but there should also exist proper evaluation data to truthfully compare the methods. For such comprehensive evaluations, reliable ground truth i...

While there is growing interest in in-line measurements of paper making processes, the factory environment often restricts the acquisition of images. The in-line imaging of pulp suspension is often difficult due to constraints to camera and light positioning, resulting in images with uneven illumination and motion blur. This article presents an alg...

In pulping and papermaking, dirt particles significantly affect the quality of paper. Knowledge of the dirt type helps to track the sources of the impurities which would considerably improve the paper making process. Dirt particle classification designed for this purpose should be adaptable because the dirt types are specific to the different proce...

Dirt count and dirt particle characterization have an important role in the quality control of the pulp and paper production. The precision of the existing image analysis systems is mostly limited by methods for only extracting the dirt particles from the images of pulp samples with non-uniform backgrounds. The goal of this study was to develop a m...

Visual object categorisation (VOC) has become one of the most actively investigated topic in computer vision. In the mainstream studies, the topic is considered as a supervised problem, but recently, the ultimate challenge has been posed: Unsupervised visual object categorisation. Hitherto only a few methods have been published, all of them being c...

Visual object categorization is one of the most active research topics in computer vision, and Caltech-101 data set is one of the standard benchmarks for evaluating the method performance. Despite of its wide use, the data set has certain weaknesses: (i) the objects are practically in a standard pose and scale in the middle of the images and (ii) b...

The objective of software development is to develop and modify systems to satisfy customer needs, on schedule and within the budget. The front end activities of software development are most important when customer needs are assessed and software requirements collected. This paper proposes a new group method for the elicitation of software requirem...

Full reference image quality algorithms are standard tools in digital image processing but have not been utilized for printed images due to a “correspondence gap” between the digital domain (a reference) and physical domain (printed sample). In this work, the authors propose a framework for applying full reference image quality algorithms to printe...

Robot control in uncertain and dynamic environments can be greatly improved using sensor-based control. Vision is a versatile low-cost sensory modality, but low sample rate, high sensor delay and uncertain measurements limit its usability, especially in strongly dynamic environments. Vision can be used to estimate a 6-DOF pose of an object by model...

Abstract Interest towards image mosaicing has existed since the dawn,of photography. Many automatic digital mosaicing methods have been developed, but unfortunately their eval- uation has been only qualitative. Lack of generally approved measures,and standard test data sets impedes comparison,of the works by different research groups. For scientifi...

Two problems especially important for supervised learning and classification in medical image processing are addressed in this study: i) how to fuse medical annotations collected from several medical experts and ii) how to form an image-wise overall score for accurate and reliable automatic diagnosis. Both of the problems are addressed by applying...

Measuring visual quality of printed media is important as printed products play an essential role in every day life, and for many “vision applications”, printed products still dominate the market (e.g., newspapers). Measuring visual quality, especially the quality of images when the original is known (full-reference), has been an active research to...

Bag of features is a well established technique for the visual categorisation of objects, categories of objects and textures. One of the most important part of this technique is codebook generation since its within-class and between-class discrimination power is the main factor in the categorisation accuracy. A codebook is generated from regions of...

This paper presents a novel system for the automatic analysis of a hybrid welding process. High-speed imaging and laser illumination are used to measure the regularity of electric arc frequency and flight directions of filler metal droplets. A fuzzy c-means clustering method is used to detect arcs and segment the video sequences. The droplets are l...

Automatic analysis of digital fundus images, where optic disc extraction is an essential part, is an active research topic in retinal image analysis. A simple, fast and robust optic disc localisation method using colour decorrelated templates is proposed which results an accurate location of the optic disc in colour fundus images. In the training s...

Regular patterns, as defined in this study, are found in areas of industry and science, for example, halftone raster patterns used in the printing industry and crystal lattice structures in solid state physics. The need for quality inspection of products containing regular patterns has aroused interest in the application of machine vision for autom...

Several novel and particularly successful object and object category detection and recognition methods based on image features, local descriptions of object appearance, have recently been proposed. The methods are based on a localization of image features and a spatial constellation search over the localized features. The accuracy and reliability o...

Print mottle is one of the most significant defects in modern offset printing influencing overall print quality. Mottling can be defined as undesired unevenness in perceived print density. Previous research in the field considered designing and improving perception models for evaluating print mottle. Mottle has traditionally been evaluated by estim...