Nick Zacharov

• B.Eng (Hons), M.Sc, D.Sc. (Tech.)
• Senior Technologist at Delta

Perceptual evaluation of immersive audiovisual quality is often very labor-intensive and costly, as numerous factors and factor levels are included in the experimental design. Therefore, the present study aims to reduce the required experimental effort by investigating the effectiveness of optimal experimental design (OED) compared to classical ful...

For accurate and detailed perceptual evaluation of compressed omnidirectional multimedia content, it is imperative for assessor panels to be qualified to obtain consistent and high-quality data. This work extends existing procedures for assessor selection in terms of scope (360 videos with high-order ambisonic), time efficiency, and analytical appr...

For accurate and detailed perceptual evaluation of omnidirectional multimedia content, it is imperative for assessor panels to be qualified in order to obtain consistent and high-quality data. This work extends existing procedures for assessor selection in terms of scope of applicability (360 videos with high-order ambisonic), time efficiency, and...

To open up new possibilities to assess the multimodal perceptual quality of omnidirectional media formats, we proposed a novel open source 360 audiovisual (AV) quality dataset. The dataset consists of high-quality 360 video clips in equirectangular (ERP) format and higher-order ambisonic (4th order) along with the subjective scores. Three subjectiv...

In an earlier study, we gathered perceptual evaluations of the audio, video, and audiovisual quality for 360 audiovisual content. This paper investigates perceived audiovisual quality prediction based on objective quality metrics and subjective scores of 360 video and spatial audio content. Thirteen objective video quality metrics and three objecti...

In this paper we study how sound quality is evaluated by different groups of assessors, with different levels of hearing loss. Formal listening tests using the Basic Audio Quality scale were designed using 22 headphones spanning a wide range of qualities and sound quality characteristics. The tests were performed with two formally selected listenin...

In this paper we study the matter of perceptual evaluation data collection for the purposes of machine learning. Well established listening test methods have been developed and standardised in the audio community over many years. This papers looks at the specific needs for machine learning and seeks to establish efficient data collection methods, t...

This paper studies the quality of multimedia content focusing on 360 video and ambisonic spatial audio format reproduced using a head-mounted display and a multichannel loudspeaker setup. Encoding parameters following basic video quality test conditions for 360 videos were selected and a low-bitrate codec was used for the audio encoder. Subjective...

This paper studies the quality of multimedia content focusing on 360 video and ambisonic spatial audio reproduced using a head-mounted display and a multichannel loudspeaker setup. Encoding parameters following basic video quality test conditions for 360 videos were selected and a low-bitrate codec was used for the audio encoder. Three subjective e...

Sensory Evaluation of Sound provides a detailed review of the latest sensory evaluation techniques, specifically applied to the evaluation of sound and audio. This three-part book commences with an introduction to the fundamental role of sound and hearing, which is followed by an overview of sensory evaluation methods and associated univariate and...

The evaluation of object-based audio reproduction methods in a real-world context remains a challenge as it is difficult to separate the effects of the reproduction system from the effects of the audio mix rendered for that system. This is often compounded by the absence of explicitly-defined reference or anchor stimuli. This paper presents a perce...

In this study four rating scales for perceptual evaluation of Preference were compared: 9-point hedonic, Continuous Quality Scale (CQS) (e.g., used in ITU-R BS.1534-3 [1], “MUSHRA”), Labelled Hedonic Scale (LHS) [2], and a modified version of the LHS. The CQS was tested in three configurations to study the role and impact of the reference and ancho...

The benefit of using individual head-related transfer functions (HRTFs) in binaural audio is well documented with regards to improving localization precision. However, with the increased use of binaural audio in more complex scene renderings, cognitive studies, and virtual and augmented reality simulations, the perceptual impact of HRTF selection m...

This study investigates the impact of relaxing presentation methods on listening tests by comparing results from two identical listening experiments carried out on two countries and comprising two presentation methods: the ITU-T P.800 Absolute Category Rating (ACR) recommendation and a modified version of it where assessors had more control on the...

We propose an alternative method for binaural synthesis using the ratio of the HRTFs at the two ears, the interaural transfer function (ITF), for deriving the response at the contralateral ear from the ipsilateral one. This technique allows an optimal binaural synthesis if an accurate ITF model is used but a computationally more efficient binaural...

Sound quality is an important aspect in many sound reproduction applications. In recent years sensory evaluation techniques have been gaining popularity for the detailed perceptual assessment of device sound characteristics. From the literature, 100s of descriptors can be found to describe the nature of sound quality and this often becomes the focu...

improving music perception quality in hearing impaired subjects with hearing aids

In audio quality evaluation, ITU-R BS.1534-1, commonly known as MUSHRA, is widely used for the subjective assessment of intermediate audio quality. Studies have identified limitations of the MUSHRA methodology [1][2], which can influence the robustness to biases and errors introduced during the testing process. Therefore ITU-R BS.1534 was revised t...

The GuineaPig test system has been developed as a generic, software based, multichannel audio system for subjective testing. Based around the SGI hardware platform, the system provides testing with several test categories, including stan-dardised tests, providing currently up to 8-channel output and graphical UIs for testing. The GuineaPig provides...

Multidimensional perceptual unfolding was executed for a set of spatially-processed speech samples reproduced via headphones. This paper describes only the first stage of a two-part study employing the analytic technique termed external unfolding. In this first stage, global dissimilarity ratings were made for all pairwise comparisons of the experi...

The eGauge measure for assessor expertise in listening tests is presented. This ANOVA-based approach aims to objectively qualify the goodness of assessors within an experiment in terms of their discrimination and reliability skill and their overall agreement with the panel. A number of datasets resulting from experiments using different standardize...

This papers explores the perceptual characteristics associated with the application of active noise cancelation (ANC) technologies to headphones. In this popular application, both noise cancelation performance is evaluated in noise & also the impact on the perception of music reproduction in noise. Environmental scenarios were reproduced from B-for...

Sounds may be characterized by objective perceptive attributes (for which there may exist physical metrics) or by subjective (affective or connotative) attributes. This paper will deal with the perceptive attributes. Within product sound quality the metrics for classical the psycho-acoustic attributes (loudness, sharpness, roughness and fluctuation...

As the complexity of mobile phones increases with the evolution of digital convergence, there is increased demand to ensure high audio quality for all applications. VirtualPhone is a graphical user interface based software environment allowing for the rapid prototyping of mobile phone audio and its subsequent calibrated auralisation. This paper des...

Listening speech quality has for many years been considered a key means of evaluating speech quality in telecommunications systems. While traditionally listening tests have been used to study such characteristics, such as in the of form of mean opinion scores (MOS), these methods are known to be quite cumbersome and slow. As a result a number of sp...

As audio and telecommunication technologies develop, there is an increasing need to evaluate the technical and perceptual performance of these innovations. A growing number of new technologies (e.g. low bit-rate coding) are based on specific properties of the auditory system, which are often highly non-linear. This means that the auditory quality o...

Level CalibrationLoudspeaker CalibrationHeadphone Calibration

Listening RoomsListening BoothsOther SpacesListener and Loudspeaker PositioningAccompanying PictureCommonly Encountered ProblemsElectrical Considerations

Statistical Experimental DesignStatistical Analysis

fundamentals of experimentation;research question and hypothesis;auditory evaluation need;expected magnitude of perceptual differences;mean opinion score (MOS) by ITU-R recommendation;definition of initial conditions;statistical experimental design;establishing statistical model;experimental set-up documentation

Response AttributeResponse FormatOverview of Process

Principle of EmpiricismPrinciple of RationalismOther Principles of Scientific ArgumentationSummary

PlanningAdministrationReporting

SignalReproduction SystemListening RoomSubject Considerations

Audio Engineering SocietyAmerican National Standards InstituteEuropean Broadcasting UnionInternational Electrotechnical CommissionThe International Telecommunications Union standards

The results of experiments in which subjects rated the perceived quality of speech and music that had been subjected to various forms of both linear and nonlinear distortion are reported. Experiment 1 made use of artificial distortions (such as ripples in frequency response combined with peak clipping). Experiment 2 included both artificial distort...

In a previous study perceptual experiments were reported in which subjects had to rate the perceived quality of speech and music that had been subjected to various forms of nonlinear distortion. The subjective ratings were compared to a physical measure of distortion, DS, based on the output spectrum of each nonlinear system in response to a 10-com...

The non-linear modeling of an electrodynamic speaker is studied and its use for real-time auralization of an arbitrary sound source is considered. First, the dominant observed non-linear behavior is presented. Then, a black-box approach to the system identification task is used seeking a generalized procedure applicable to any loudspeaker. In this...

The effect of various types of nonlinear distortion on the perceived quality of speech and music signals was examined. In experiments 1 and 2, "artificial" distortions were used, including hard and soft symmetrical and asymmetrical peak clipping of various amounts, center clipping, and full-range waveform distortion produced by raising the instanta...

The design and construction of specialist listening rooms is complex and expensive due to high requirements for sound insulation of structures and noise levels of HVAC devices. Additionally, such rooms require large area and are difficult to post install into existing buildings. The motivation for this study was to design a compact listening room (...

This study presents a method termed Audio Descriptive Analysis & Mapping (ADAM) to a study of spatial sound displays. Sev-eral subjective tasks were performed including a preference exper-iment, descriptive language development and lastly scaling of all stimuli based on developed attribute scales. The process associ-ated with the descriptive langua...

An electret microphone has been implemented into a printed circuit board. Due to novel construction the printed circuit board serves as the microphone backplate. Furthermore, other features of printed circuit board manufacturing technology are applied to minimize the number of components of the microphone. This results in a microphone that can be m...

In an effort to establish a better understanding of multichannel level alignment, a correlation and regression analysis has been performed of objective metrics and subjective data collected and subsequently reported in parts I-III & V. This study considers subjective data acquired with 9 test signals with different set-ups considering loudspeaker p...

The alignment of sound intensity is of great importance particularly in the field of psychometrics. When sounds have different spectral content, loudness alignment becomes a more suitable tool. However, traditional models assume the source either to reside in a free field at 0° azimuth or in a diffuse field. To assess or align sound source loudness...

This study considers research on spatial sound perception in the context of multichannel and other spatial sound reproduction systems. Issues associated with multichannel level alignment are discussed. Additionally aspects relating to the quality of spatial sound reproduction associated with loudspeaker directivity and 3D sound algorithms are consi...

In this paper, the problem of modeling head-related transfer functions (HRTFs) is addressed. Traditionally, HRTFs are approximated in real-time applications using minimumphase reconstruction and various digital filter design techniques, yielding FIR or IIR structures. In this work, binaural auditory modeling has been applied to HRTF analysis and fi...

A group of subjective experiments considering the loudspeaker directivity characteristics for five-channel audiovisual reproduction was examined. A large number of tests were performed with a screened and trained listening panel. The effects of listening position, loudspeaker directivity, and program were examined. Frontal and surround loudspeaker...

The correct level alignment of multichannel reproduction systems is critical to the quality of the reproduction. This, the third paper discusses the results of three experiments employing the nine test signals discussed in paper I. Experiments employing six subjects using the method of adjustment have been conducted in a standard listening rooms us...

As part of the Eureka 1187 (ADTT) project, an audio working group conducted brief assessments of different subwoofer arrangements on the premises of Bang & Olufsen in Struer, Denmark, during 1996 April. The aim was to determine, for the test environment and the loudspeakers used, what constraints there might be on the location and the number of sub...

This paper addresses the topic of subject and listening panel performance in descriptive analysis applications. A summary of the status in the field of audio is provided regarding the selection, development, training and monitoring of subjects and listening panel. Subject categorisation and associated terminologies are discussed, placing emphasis o...

Virtual home theatre systems provide an efficient means of multichannel sound reproduction with two loudspeakers. Six commercial and academically developed systems have been collected and subjectively evaluated under controlled conditions against discrete 5-channel reproduction. Tests have been carried out in two test rooms. The experimental design...

As multichannel sound systems become more and more widespread, the issues to how to obtain optimal sound reproduction become more apparent. Although the matter of level calibration is rather trivial for stereo sound systems, it has been found that it does critically affect perceived sound quality. The more complex and often sub-optimal multichannel...

The correct level alignment of multichannel reproduction systems is critical to the quality of the reproduction. This, the second paper discusses the results of two experiments employing the nine test signals discussed in paper I. Experiments employing the method of adjustment have been conducted in two standard listening rooms, considering both th...