The Effect of Content Desirability on Subjective Video Quality Ratings

ArticleinHuman Factors The Journal of the Human Factors and Ergonomics Society 52(1):105-18 · February 2010with 41 Reads
Cite this publication
Abstract
The purpose of this study was to examine the impact of the desirability of content on viewers' ratings of subjective video quality. Most subjective video quality studies use short-duration clips that are specially designed to exercise the encoding algorithms and do not consider the desirability of the content as a variable. In four studies, we employed a total of 100 participants and 180 movie clips encoded at nine levels from 550 kbps up to DVD quality. Participants viewed clips that were 2 min in length and then were asked about video quality of the clips and desirability of the movie content. The results of these studies show that there is a strong correlation between the desirability of movie content and subjective ratings of video quality. This strong relationship holds across a wide range of encoding levels and movie content when that content is viewed under longer, more naturalistic viewing conditions. The effects of content should be considered when evaluating the subjective quality of encoded video content, as these effects can be as large as those seen between low- and high-quality encodings. Researchers and practitioners trying to determine acceptable levels of video quality for actual consumption by consumers may find that the results and methods described here allow for a more accurate assessment of levels of video quality that are acceptable in a fielded service.

Do you want to read the rest of this article?

Request Full-text Paper PDF
  • Article
    Full-text available
    This paper discusses results from an exploratory study in which Quality of Experience aspects related to mobile video watching were investigated in a semi-living lab setting. More specifically, we zoom in on usage patterns in a natural research context and on the subjective evaluation of high and low-resolution movie trailers that are transferred to a mobile device using two transmission protocols for video (i.e., real-time transport protocol and progressive download using HTTP). User feedback was collected by means of short questionnaires on the mobile device, combined with traditional pen and paper diaries. The subjective evaluations regarding the general technical quality, perceived distortion, fluentness of the video, and loading speed are studied and the influence of the transmission protocol and video resolution on these evaluations is analyzed. Multinomial logistic regression results in a model to estimate the subjective evaluations regarding the perceived distortion and loading speed based on objectively-measured parameters of the video session.
  • Article
    Full-text available
    The Electronic imaging community has devoted a lot of effort to the development of technologies that can predict the visual quality of images and videos, as a basis for the delivery of optimal visual quality to the user. These systems have been based for the most part on a visibility-centric approach, assuming the more artifacts are visible, the higher is the annoyance they provoke, the lower the visual quality. Despite the remarkable results achieved with this approach, recently a number of studies suggested that the visibility-centric approach to visual quality might have limitations, and that other factors might influence the overall quality impression of an image or video, depending on cognitive and affective mechanisms that work on top of perception. In particular, interest in the visual content, engagement and context of usage have been found to impact on the overall quality impression of the image/video. In this paper, we review these studies and explore the impact that affective and cognitive processes have on the visual quality. In addition, as a case study, we present the results of an experiment investigating on the impact of aesthetic appeal on visual quality, and we show that users tend to be more demanding in terms of visual quality judging beautiful images.
  • Article
    Full-text available
    R. Pauliks, I. Slaidins. Impact of Video Content and Technical Specifications on Subjective Quality Assessment // Electronics and Electrical Engineering. - Kaunas: Technologija, 2012. - No. 6(122). - P. 91-96. Replacement of traditional analog TV broadcasting services with various video services based on new advanced digital technologies not always provide high quality of video image if evaluated by viewer. This paper presents an analysis of impact of the video content and technical specifications on the subjective assessment of video picture quality made by non-expert observers. The measurements are performed in experimental setup with 25 observers, using 11 video scenes with wide range of technical specification, various analog and digital interfaces. The results show that high definition analog and digital video interfaces are evaluated by observers as providing nearly the same quality, but temporal activity in a video scene had a significant effect on the subjective quality assessment score. For a very vibrant video scenes with a high level values of temporal and spatial activity there is no significant difference in subjective quality assessment score either using standard definition or high definition, analog or digital interface. Ill. 3, bibl. 20, tabl. 5 (in English; abstracts in English and Lithuanian).
  • Chapter
    Full-text available
    New trends and recent advances in subjective assessment of Quality of Experience - Subjective assessment of Quality of Experience (QoE) is key to understanding us-er preferences with respect to multimedia fruition. As such, it is a necessary step to multimedia delivery optimization, since QoE needs to take into account tech-nology limitations as well as user satisfaction. The study of QoE appreciation dates back to the twentieth century, when it exploded with the advent of CRT first and LCD displays later. For a long time, this branch of research was targeted at determining user sensitivity to impairments induced in the media by suboptimal delivery. The media recipient was considered a passive observer, whose apprecia-tion of the video material was determined primarily by the degree of annoyance due to the impairments affecting it. With the advent of mobile technology and in-ternet-based media delivery, this impairment-centric concept of QoE has shown to be incomplete. The media recipient became an active user who creates content, interacts with the system and selects the media he/she wants to have delivered. As a result, elements such as visual semantics, user personality, preferences and intent, social and environmental context of media fruition also concur to the final experience assessment. The role played by these elements in QoE, and the cogni-tive/affective processes that underlie them are still to be understood, although several models of QoE appreciation have already been proposed. In this paper, we review the evolution of subjective QoE assessment and models from the impairment-centric approach to a more user-centric approach. We analyze relevant features and factors influencing QoE, and point out future directions for subjective QoE assessment research.
  • Article
    Full-text available
    HTTP adaptive streaming facilitates video streaming to mobile devices connected through heterogeneous networks without the need for a dedicated streaming infrastructure. By splitting different encoded versions of the same video into small segments, clients can continuously decide which segments to download based on available network resources and device characteristics. These encoded versions can, for example, differ in terms of bitrate and spatial or temporal resolution. However, as a result of dynamically selecting video segments, perceived video quality can fluctuate during playback which will impact end-users' quality of experience. Subjective studies have already been conducted to assess the influence of video delivery using HTTP Adaptive Streaming to mobile devices. Nevertheless, existing studies are limited to the evaluation of short video sequences in controlled environments. Research has already shown that video duration and assessment environment influence quality perception. Therefore, in this article, we go beyond the traditional ways for subjective quality evaluation by conducting novel experiments on tablet devices in more ecologically valid testing environments using longer duration video sequences. As such, we want to mimic realistic viewing behavior as much as possible. Our results show that both video content and the range of quality switches significantly influence end-users' rating behavior. In general, quality level switches are only perceived in high motion sequences or in case switching occurs between high and low quality video segments. Moreover, we also found that video stallings should be avoided during playback at all times.
  • Conference Paper
    Full-text available
    With the recent increased popularity and high usage of HTTP Adaptive Streaming (HAS) techniques, various studies have been carried out in this area which generally focused on the technical enhancement of HAS technology and applications. However, a lack of common HAS standard led to multiple proprietary approaches which have been developed by major Internet companies. In the emerging MPEG-DASH standard the packagings of the video content and HTTP syntax have been standardized; but all the details of the adaptation behavior are left to the client implementation. Nevertheless, to design an adaptation algorithm which optimizes the viewing experience of the enduser, the multimedia service providers need to know about the Quality of Experience (QoE) of different adaptation schemes. Taking this into account, the objective of this experiment was to study the QoE of a HAS-based video broadcast model. The experiment has been carried out through a subjective study of the end user response to various possible clients' behavior for changing the video quality taking different QoE-influence factors into account. The experimental conclusions have made a good insight into the QoE of different adaptation schemes which can be exploited by HAS clients for designing the adaptation algorithms.
  • Article
    Full-text available
    The next generation of multimedia services have to be optimized in a personalized way, taking user factors into account for the evaluation of individual experience. Previous works have investigated the influence of user factors mostly in a controlled laboratory environment which often includes a limited number of users and fails to reflect real-life environment. Social media, especially Facebook, provide an interesting alternative for Internet-based subjective evaluation. In this article, we develop (and open-source) a Facebook application, named YouQ1, as an experimental platform for studying individual experience for videos. Our results show that subjective experiments based on YouQ can produce reliable results as compared to a controlled laboratory experiment. Additionally, YouQ has the ability to collect user information automatically from Facebook, which can be used for modeling individual experience.
  • Article
    Full-text available
    Nowadays, HTTP adaptive streaming (HAS) has become a reliable distribution technology offering significant advantages in terms of both user perceived Quality of Experience (QoE) and resource utilization for content and network service providers. By trading-off the video quality, HAS is able to adapt to the available bandwidth and display requirements so that it can deliver the video content to a variety of devices over the Internet. However, until now there is not enough knowledge of how the adaptation techniques affect the end user's visual experience. Therefore, this paper presents a comparative analysis of different bitrate adaptation strategies in adaptive streaming of monoscopic and stereoscopic video. This has been done through a subjective experiment of testing the end-user response to the video quality variations, considering the visual comfort issue. The experimental outcomes have made a good insight into the factors that can influence on the QoE of different adaptation strategies.
  • Article
    Full-text available
    In the past decades, a lot of effort has been invested in predicting the users' Quality of Visual Experience (QoVE) in order to optimize online video delivery. So far, the objective approaches to measure QoVE have been mainly based on an estimation of the visibility of artifacts generated by signal impairments at the moment of delivery and on a prediction of how annoying these artifacts are to the end user. Recently, it has been shown that other aspects, such as user interest or viewing context, also have a crucial influence on QoVE. Social context is one of these aspects, but it has been poorly investigated in relation to QoVE so far. In this paper, we report the outcomes of an experiment that aims at unveiling the role that social context, and in particular co-located co-viewing, plays within the visual experience and the annoyance of coding artifacts. The results show that social context significantly influences user's QoVE, whereas the appearance of artifacts doesn't have impact on viewing experience, although users can still notice them. The results suggest that quantifying the impact of social context on user experience is of major importance to accurately predict QoVE towards video delivery optimization.
  • Article
    Full-text available
    Visual quality is a multifaceted quantity that depends on multiple attributes of the image/video. According to Keelan's definition, artifactual attributes concern features of the image that when visible, are annoying and compromise the integrity of the image. Aesthetic attributes instead depend on the observer's personal taste. Both types of attributes have been studied in the literature in relation to visual quality, but never in conjunction with each other. In this paper we perform a psychometric experiment to investigate how artifactual and aesthetic attributes interact, and how they affect the viewing behavior. In particular, we studied to what extent the appearance of artifacts impacts the aesthetic quality of images. Our results indicate that indeed image integrity somehow influences the aesthetic quality scores. By means of an eye-tracker, we also recorded and analyzed the viewing behavior of our participants while scoring aesthetic quality. Results reveal that, when scoring aesthetic quality, viewing behavior significantly departs from the natural free looking, as well as from the viewing behavior observed for integrity scoring.
  • Article
    Full-text available
    There is increasing evidence that the brain possesses mechanisms to integrate incoming sensory information as it unfolds over time-periods of 2-3 seconds. The ubiquity of this mechanism across modalities, tasks, perception and production has led to the proposal that it may underlie our experience of the subjective present. A critical test of this claim is that this phenomenon should be apparent in naturalistic visual experiences. We tested this using movie-clips as a surrogate for our day-to-day experience, temporally scrambling them to require (re-) integration within and beyond the hypothesized 2-3 second interval. Two independent experiments demonstrate a step-wise increase in the difficulty to follow stimuli at the hypothesized 2-3 second scrambling condition. Moreover, only this difference could not be accounted for by low-level visual properties. This provides the first evidence that this 2-3 second integration window extends to complex, naturalistic visual sequences more consistent with our experience of the subjective present.
  • Article
    Full-text available
    Image quality assessment (IQA) enables distortions introduced into an image (e.g., through lossy compression or broadcast) to be measured and evaluated for severity. It is unclear to what degree affective image content may influence this process. In this study, participants (n=25) were found to be unable to disentangle affective image content from objective image quality in a standard IQA procedure (single stimulus numerical categorical scale). We propose that this issue is worthy of consideration, particularly in single stimulus IQA techniques, in which a small number of handpicked images, not necessarily representative of the gamut of affect seen in true broadcasting, and unrated for affective content, serve as stimuli.
  • Article
    Creating algorithms capable of predicting the perceived quality of a visual stimulus defines the field of objective visual quality assessment (QA). The field of objective QA has received tremendous attention in the recent past, with many successful algorithms being proposed for this purpose. Our concern here is not with the past however; in this paper we discuss our vision for the future of visual quality assessment research. We first introduce the area of quality assessment and state its relevance. We describe current standards for gauging algorithmic performance and define terms that we will use through this paper. We then journey through 2D image and video quality assessment. We summarize recent approaches to these problems and discuss in detail our vision for future research on the problems of full-reference and no-reference 2D image and video quality assessment. From there, we move on to the currently popular area of 3D QA. We discuss recent databases, algorithms and 3D quality of experience. This yet-nascent technology provides for tremendous scope in terms of research activities and we summarize each of them. We then move on to more esoteric topics such as algorithmic assessment of aesthetics in natural images and in art. We discuss current research and hypothesize about possible paths to tread. Towards the end of this article, we discuss some other areas of interest including high-definition (HD) quality assessment, immersive environments and so on before summarizing interesting avenues for future work in multimedia (i.e., audio-visual) quality assessment. KeywordsQuality assessment–Objective quality assessment–Subjective quality assessment–Perceived quality
  • Article
    Full-text available
    The efficiency of personal video suggestions generated by recommender systems is highly dependent on the quality of the obtained user feedback. This feedback has to reflect the personal interest in the content of the viewed video, to obtain accurate results. However, user feedback might undesirably be influenced by additional aspects such as the loading speed or the quality of the video. To date, this issue has received very little research attention. Therefore, this study investigates the direct influence of audio-visual quality parameters on explicit user feedback for the first time to our knowledge via a mobile, Living Lab experiment. This paper proposes a feedback model which takes the Quality of Service (QoS) parameters of the mobile network into account. This model can be used as an additional feedback filter for video recommendation systems that could help to eliminate the influences of QoS on explicit user feedback. KeywordsRecommendation–User feedback–Mobile–Quality of experience–User experience–Quality of service
  • Article
    Full-text available
    The field of perceptual quality assessment has gone through a wide range of developments and it is still growing. In particular, the area of no-reference image and video quality assessment has progressed rapidly during the last decade. In this article, we present a classification and review of latest published research work in the area of no-reference (NR) image and video quality assessment. The NR methods of visual quality assessment considered for review are structured into categories and subcategories based on the types of methodologies used for the underlying processing employed for quality estimation. Overall, the classification has been done into three categories, namely, pixel-based methods, bitstream-based methods, and hybrid methods of the aforementioned two categories. We believe that the review presented in this article will be helpful for practitioners as well as for researchers to keep abreast of the recent developments in the area of NR image and video quality assessment. This article can be used for various purposes such as gaining a structured overview of the field and to carry out performance comparisons for the state-of-the-art methods.
  • Popularity of the streaming media content such as videos can be ascribed to the perceptual quality, to some extent, of the content. The traditional methods of audio/video quality assessment lack in provision of the input from higher cognitive of the human perception. Some studies have revealed that liking or disliking of a certain content can bias the human judgement towards video quality. In this paper, we have examined the impact of the use of semantic quality indicators namely audio content, audio quality, video content, and video quality in the assessment of quality of a video. Further, we have proposed a methodology to use these indicators for designing a prediction model for the popularity of streaming videos.
  • Article
    The usage of HTTP adaptive streaming (HAS) has become widely spread in multimedia services. Because it allows the service providers to improve resource utilization and user's Quality of Experience (QoE). Using this technology , the video playback interruption is reduced since the network and server status in addition to capability of user device, all are taken into account by HAS client to adapt the quality to the current condition. Adaptation can be done using different strategies. In order to provide optimal QoE, impact of adaptation strategies from point of view of the user should be studied. However, the time-varying video quality due to the adaptation which usually takes place in a long interval introduces a new type of impairment making the subjective evaluation of HAS challenging. The contribution of this paper is twofold: first, it investigates the testing methodology to evaluate the HAS QoE by comparing the subjective experimental outcomes obtained from ACR standardized method and a semi-continuous method developed to evaluate the long sequences. In addition, influence of using audiovisual stimuli to evaluate video-related impairment is inquired. Second, impact of some of the adaptation technical factors including the quality switching amplitude and chunk size in combination with high range of commercial content type is investigated. The results of this study provide a good insight toward achieving appropriate testing method to evaluate the HAS QoE, in addition to designing switching strategies with optimal visual quality.
  • Thesis
    Full-text available
    Subjective quality assessment of multi-modal services depends on a number of external factors that affect the final judgment, e.g. user expectations, user fatigue, room environment or methodology used in the evaluation process. In order to obtain as accurate as possible measurement of the perceived quality, an experimenter should carefully consider all factors contributing to the overall experience. Particularly important is to choose the right measurement method for the purpose of a specific task. In spite of the fact that a number of standardized test procedures for the quality assessment exist it is not always possible to find the one which suits a certain research purpose. In such case, the development of new assessment techniques is usually necessary, which then needs to be followed by appropriate testing and validation procedures. The lack of an appropriate methodology for instantaneous measurement of user’s audio-visual quality expectations/preferences over extended periods of time, as well as the scarce attention devoted to the topic of temporal development of Quality of Experience, were the main driving factors for this work. This dissertation, composed of five papers, helps to understand the underlying attributes of perceived quality and user cognitive processes used in evaluation of long duration audiovisual content. The work described here is twofold: firstly, a novel methodology for continuous quality evaluation is proposed, and secondly, using the method, the effect of the time dimension on user’s behavioral reaction to the experienced quality is investigated. The momentary-based approach described in this work reflects instantaneous measures of users’ quality judgements. Such measures allow capturing time varying changes of system characteristics and help to contribute to the holistic vision of quality of experience. The results obtained from several experiments described in this work reveal the importance of content duration in the process of quality assessment and its impact on user’s quality requirements. The knowledge gained from those studies can be directly applied to the quality assurance models of multimedia content providers and may serve as a valuable source of information for objective quality metrics development.
  • Article
    Today’s packet-switched networks are subject to bandwidth fluctuations that cause for degradation of user experience of multimedia services. In order to cope with this problem,HTTP Adaptive Streaming (HAS) has been proposed in recent years as a video delivery solution for the future Internet and being adopted by an increasing number of streaming services such as Netflix and Youtube. HAS enables service providers to improve users’ Quality of Experience (QoE) and network resource utilization by adapting the quality of the video stream to the current network conditions. However, the resulting time-varying video quality caused by adaptation introduces a new type of impairment and thus novel QoE research challenges.Despite of various recent attempts to investigate these challenges, many fundamental questions regarding HAS perceptual performance are still open. In this paper, the QoE impact of different technical adaptation parameters including chunk length, switching amplitude, switching frequency and temporal recency are investigated. In addition, the influence of content on perceptual quality of these parameters is analyzed. To this end,a large number of adaptation scenarios have been subjectively evaluated in four laboratory experiments and one crowdsourcing study. Statistical analysis of the combined dataset reveals results that partly contradict widely held assumptions and provide novel insights in perceptual quality of adapted video sequences, e.g.interaction effects between quality switching direction (up/down)and switching strategy (smooth/abrupt). The large variety of experimental configurations across different studies ensures the consistency and external validity of the presented results that can be utilized for enhancing the perceptual performance of adaptive streaming services.
  • Chapter
    Chapter 5 defines the subjective methodologies that relate directly to obtaining meaningful subjective evaluation for still and video image quality. A review of key psychophysical techniques such as category scaling, paired comparisons, acceptability ratings, and mean opinion score will emphasize the strengths and weaknesses of each methodology. The review will also explore the similarities and differences between still and video subjective evaluation techniques. Particular focus will be on the anchor scale method and how that can be used to quantify overall image quality in just noticeable differences for still images.
  • Conference Paper
    Nowadays, Service Providers are increasingly concerned about the concept of Quality of Experience (QoE), even more, when talking about Internet TV or WebTV, where no guarantees of delivery are provided. This paper describes the research and the results on the influence of the level of interest (on a particular sport) in the subjective quality assessment of the corresponding broadcasted media. This analysis is motivated by the work being developed in the European Project My e-Director 2012, which has the capability to cover the London Olympic Games 2011 via the Web. Therefore, a subjective test was prepared and performed where each observer visioned and assessed the perceived video quality of a set of six sports, encoded in four different bitrate/resolution sets. From the analysis of the collected data it is possible to demonstrate that the interest level has a strong influence in the subjective assessment of the video quality. Based on these results, an empiric formula was deduced to estimate the Mean Opinion Score (MOS) as a function of bitrate and interest level.
  • Article
    Nowadays, access to multimedia content is one of the most demanded services on the Internet. However, the transmission of audio and video over these networks is not free of problems that negatively affect user experience. Factors such as low image quality, cuts during playback or losses of audio or video, among others, can occur and there is no clear idea about the level of distortion introduced in the perceived quality. For that reason, different impairments should be evaluated based on user opinions, with the aim of analyzing the impact in the perceived quality. In this work, we carried out a subjective evaluation of different types of impairments with different types of contents, including news, cartoons, sports and action movies. A total of 100 individuals, between the ages of 20 and 68, participated in the subjective study. Results show that short-term rebuffering events negatively affect the quality of experience and that desynchronization between audio and video is the least annoying impairment. Moreover, we found that the content type determines the subjective results according to the impairment present during the playback.
  • Article
    In this paper, two questions related to long duration audiovisual content are addressed: firstly, we investigate whether users' quality expectations/requirements develop differently over extended periods of time for different types of content. We have found indicators suggesting that high spatial and temporal activity indices decrease quality requirements over time. Secondly, we show that for long duration content, viewers' quality requirements are independent of magnitude and appearance (gradual increase vs. spontaneous leap) of quality impairments introduced.
  • Thesis
    Full-text available
    The main objective of the thesis work is to show how image and video quality studies can incorporate cognitive and affective elements when evaluating Quality of Experience (QoE) for audio and video content. The focus of the thesis is on perceived video quality when video is streamed via a wireless network, as this can give an irregular throughput. The latest definition of QoE, as stated in the Qualinet White Paper (2013), “is the degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state”. There are still quite some unknowns in this area, especially around what the exact link is between affective factors such as involvement, sensory perception, and the underlying technical parameters. This thesis therefore sets out a model which aims to clarify the relationship between perceived video quality and involvement. It also aims to supply an operational definition of involvement, as well as a measure which can be used in the bigger framework of Quality of Experience. Assuming that Human Influencing Factors (IFs) are important to determine people’s preferences, a measure which reflects people’s opinion about scene content is necessary. A literature study indicated that related existing measures are not applicable since they assume objects (in marketing), interaction with the content (User Experience, games and immersion) or virtual reality (presence). To develop a fitting measure, a psychometric approach was adopted. First, it was necessary to devise an operational definition of involvement. This was done through concept mapping, and the end result is a model for involvement with audio/video content with 6 attributes: captivation, expressions of involvement, informative interest, relatedness, negative affect and disinterest. Based on the developed model, an involvement questionnaire (inQ) was developed and tested. A first draft version was quickly tested for understanding and wording through cognitive interviews. The second version of the inQ was then placed online, where about 100 participants scored 9 different video scenes with the involvement questionnaire. Results were analysed with the help of exploratory factor analyses, and a third version of the inQ was created. The third version of the InQ was validated through an experiment where 100 participants were shown 15 one-minute scenes, 5 different fragments, with three versions each: a reference version, a version where temporal artifacts (e.g. jerkiness) were induced, and a version where spatial artifacts (e.g. blockiness or blurriness) were induced. The results of this test showed that there was a 2-way relationship between the inQ and the perceived video quality, meaning that scores on inQ can partially predict scores on perceived video quality and vice versa. The results also showed that involvement can be defined by three attributes: positive / negative effect, relatedness and internal captivation. Video content can be judged differently across these factors, which could help differentiating people’s preferences for temporal or spatial artifacts in the future. To conclude, the work presented in this thesis has shown that it is possible to create reliable and valid measures for Human IFs. This was established through both an explorative data analysis and a confirmative analysis. Furthermore, it has also shown that involvement with audio/video content is a salient Human IF and an important characteristic for QoE. Hence, involvement should be taken into account whenever participants are asked to judge video or audio quality, as it may change their quality judgement.
  • Conference Paper
    The streaming of the video contents is currently expanding rapidly as an alternative to traditional TV broadcast. However, because of the variable bandwidth of the networks used to deliver multimedia content, a smooth and high-quality playback experience could not always be guaranteed. Using segments (chunks) in multiple video qualities, HTTP adaptive streaming (HAS) of video content is a relevant advancement with respect to classic progressive download streaming. By recent high usage of adaptive streaming technique, various studies has been carried out in this area which has been generally focused on technical enhancement of HAS technology. Also, all ongoing researches are currently providing the basis of HAS applications. However, still there is a big demand of studying the end-user’s quality perception to optimize the perceived quality of experience. The objective of this experiment was to study the quality of experience (QoE) of a HAS-based video broadcast model by evaluating the end-user’s response to different adaptation scenarios. Considering the experimental factors close to those used in practice, the experimental conclusions have made a good insight into the user´s response which can be used to develop the adaptive streaming algorithms and consequently improving the QoE.
  • Article
    There is a growing interest in the potential value of content-driven requirements for increasing the perceived quality of audiovisual material and optimizing the underlying performance processes. However, the categorization of content and identification of content-driven requirements is still largely based on technical characteristics. There is a gap in the literature when it comes to including viewer and content related aspects. In this paper, we go beyond purely technical features as content classifiers and contribute to the deeper understanding of viewer preferences and requirements. We present results from a qualitative study using semi-structured interviews, aimed at exploring content-driven associations from a bottom-up perspec-tive. The results show that users' associations, requirements and expectations differ across different content types, and that these differences should be taken into account when selecting stimulus material for subjective quality assessments. We also relate these results to previous research on content classification.
  • Article
    The main contribution of this book is offering an overview of current status, challenges, and new trends of visual quality assessment, from subjective assessment models to objective metrics, covering full-reference (FR), reduced-reference (RR), and no-reference (NR), multiply distorted images, contrast-changed images, mobile media, high dynamic range (HDR) images and videos, medical images, stereoscopic/3D videos, retargeted images and videos, computer graphics and animation quality assessment. Figure 10.1 diagrams the content presented in this book.
  • Article
    Full-text available
    In this paper, we examine the implicit aspects of quality activated by the design and storytelling components of the consumer's product and the consumption environment. We describe what affects the consumers' assessment of the product by looking beyond its surface. We review several design and marketing examples including previous research in multimedia and information processing and other fields to understand the how the unconscious realm of thoughts and impressions can create an impact on quality judgement. We also propose a framework in the Quality of Experience that includes the implicit aspects of quality judgement.
  • Conference Paper
    Standard methodologies for subjective video quality testing are based on very short test clips of 10 seconds. But is this duration sufficient for Quality of Experience assessment? In this paper, we present the results of a comparative user study that tests whether quality perception and rating behavior may be different if video clip durations are longer. We did not find strong overall MOS differences between clip durations, but the three longer clips (60, 120 and 240 seconds) were rated slightly more positively than the three shorter durations under comparison (10, 15 and 30 seconds). This difference was most apparent when high quality videos were presented. However, we did not find an interaction between content class and the duration effect itself. Furthermore, methodological implications of these results are discussed.
  • Conference Paper
    Advances in video technologies combined with increased usage of synthetic video artifacts even in natural video scenes may cause problems in subjective perception of quality. In this paper investigation is done on how subjective quality assessment scores depend on technical specifications of video interface (YPbPr, YCbCr and YIQ) and video content containing diverse synthetic video artifacts. Single Stimulus (SS) and Simultaneous Double Stimulus (SDS) subjective quality assessment methods were used and results compared. It was also revealed that SDS method allows better differentiation of subjective quality. This result agrees well with statistics obtained from crowd-based questionnaires used in separate study where most observers preferred SDS as a simpler and more comfortable method compared to SS method.
  • Article
    This paper describes the results of two studies undertaken to better understand the strong positive relationship between the content of a video clip and viewers' subjective ratings of the video quality of that clip. Experiment 1 evaluated the effect of training on video quality ratings. Experiment 2 assessed the impact of attention in these assessments. Experiment 1 showed that training participants on video quality assessment techniques removed viewers' heuristic reliance on affect while increasing accuracy. Experiment 2 demonstrated that participants' video quality ratings could also be improved by directing their focus of attention. Focusing viewers on quality did not show a moderation of the relationship between content immersion and video quality, as participants maintained high levels of content immersion even when focusing on video quality. Because training and attention increased quality rating accuracy, they could be utilized in video quality testing to improve the viewer feedback received. Copyright © 2013 John Wiley & Sons, Ltd.
  • Conference Paper
    Full-text available
    Recently, a lot of effort has been devoted to estimating the Quality of Visual Experience (QoVE) in order to optimize video delivery to the user. For many decades, existing objective metrics mainly focused on estimating the perceived quality of a video, i.e., the extent to which artifacts due to e.g. compression disrupt the appearance of the video. Other aspects of the visual experience, such as enjoyment of the video content, were, however, neglected. In addition, typically Mean Opinion Scores were targeted, deeming the prediction of individual quality preferences too hard of a problem. In this paper, we propose a paradigm shift, and evaluate the opportunity of predicting individual QoVE preferences, in terms of video enjoyment as well as perceived quality. To do so, we explore the potential of features of different nature to be predictive for a user's specific experience with a video. We consider thus not only features related to the perceptual characteristics of a video, but also to its affective content. Furthermore, we also integrate in our framework the information about the user and use context. The results show that effective feature combinations can be identified to estimate the QoVE from the perspective of both the enjoyment and perceived quality.
  • Article
    This paper presents a quality of experience (QoE) conceptual model to the context of unified communications (UC) through summary, classification, and discussion of multiple influence factors (IFs) and dimensions affecting it. A deep and comprehensive understanding of the IFs and their impact on QoE for a given service is an essential precondition for successful QoE management with the overall goal of prominently optimizing end‐user QoE, while making efficient use of network resources and maintaining a satisfied user base. The proposed conceptual model was used to conduct a qualitative meta‐analytical review of selected papers. The results of the qualitative review include various IFs, QoE dimensions, and key findings in the form of research recommendations for QoE in the context of UC. This paper presents a multidimensional quality of experience (QoE) modeling concept to the context of unified communications (UC). The proposed concept was used to conduct a qualitative meta‐analytical review of influence factors (IFs) and perceptual dimensions (PDs) investigated in the studies considering QoE for UC published update. The domain would benefit by placing great emphasis on the complexity of QoE for UC and providing summary of key findings in the form of research recommendations.
  • Article
    Full-text available
    The limited capacity model of television viewing is used to investigate the impact of arousing content and fast paced production of viewers’ information processing of TV messages. Results show that both fast pace and arousing content elicit self‐reported arousal, but they elicit different patterns of physiological arousal. Both fast pace and arousing content increase the allocation of processing resources to messages. The combination of fast pace and arousing content overloads the processing system resulting in less recognition and cued recall for the specific content of the message. Results generally support the limited capacity theory of television viewing.
  • Comparing subjective video quality testing methodologiesVisual commu-nications and image processing (VCIP '03) (pp
    • M H Pinson
    • S Wolf
    Pinson, M. H., & Wolf, S. (2003). Comparing subjective video quality testing methodologies. In Touradj Ebrahimi and Thomas Sikora (Eds.), Proceedings of SPIE: Vol. 5150.Visual commu-nications and image processing (VCIP '03) (pp. 573–582). Bellingham, WA: SPIE (The International Society for Optical Engineering).
  • Single stimulus continuous quality evaluation
    • T Alpert
    • J P Evain
    Alpert, T., & Evain, J. P. (1997). Single stimulus continuous quality evaluation. EBU Technical Review, 271, 12–20.
  • Top Gun [Motion picture]
    • D Simpson
    • J Bruckheimer
    Simpson, D., Bruckheimer, J. (Producers) & Scott, T. (Director). (1986). Top Gun [Motion picture]. United States: Paramount Pictures.
  • YouTube, Sony Pictures in talks over feature films Retrieved from http://news.cnet.com/ 8301-1023_3-10212585-93 Attention to television: Alpha power and its relationship to image motion and emotional content
    • G Sandoval
    Sandoval, G. (2009, April 6). " YouTube, Sony Pictures in talks over feature films. " Retrieved from http://news.cnet.com/ 8301-1023_3-10212585-93.html Simons, R. F., Detenber, B. H., Cuthbert, B. N., Schwartz, D. D., & Reiss, J. E. (2003). Attention to television: Alpha power and its relationship to image motion and emotional content. Media Psychology, 5, 283–301.
  • Using eyes on screen as a measure of attention to television
    • E Thorson
    Thorson, E. (1994). Using eyes on screen as a measure of attention to television. In A. Lang (Ed.), measuring psychological responses to media (pp. 65–84). London, England: Routledge.
  • Cinematic storytelling: The 100 most powerful film conventions every filmmaker must know
    • J Van Sijll
    Van Sijll, J. (2007). Cinematic storytelling: The 100 most powerful film conventions every filmmaker must know. Studio City, CA: Michael Wiese Productions.
  • Article
    Staged 2 different videotaped interviews with the same individual--a college instructor who spoke English with a European accent. In one of the interviews the instructor was warm and friendly, in the other, cold and distant. 118 undergraduates were asked to evaluate the instructor. Ss who saw the warm instructor rated his appearance, mannerisms, and accent as appealing, whereas those who saw the cold instructor rated these attributes as irritating. Results indicate that global evaluations of a person can induce altered evaluations of the person's attributes, even when there is sufficient information to allow for independent assessments of them. Furthermore, Ss were unaware of this influence of global evaluations on ratings of attributes. In fact, Ss who saw the cold instructor actually believed that the direction of influence was opposite to the true direction. They reported that their dislike of the instructor had no effect on their rating of his attributes but that their dislike of his attributes had lowered their global evaluations of him. (PsycINFO Database Record (c) 2006 APA, all rights reserved).
  • Article
    This paper briefly discusses objective and subjective methods for video quality assessment by focusing on the current work plan and schedule of the Video Quality Experts Group (VQEG). Subjective assessment methods have been used reliably, for many yeats to evaluate video image quality, vet, some issues arise with subjective assessment including the cost of conducting rite evaluations and the fact that these methods cannot easily be used to monitor. video image quality in realtime. Traditional analog objective measurements, while still necessary, are not sufficiently to measure the quality of digitally compressed video systems. Thus, there is a need to develop new objective methods utilizing the characteristics of the human visual system. While several new objective methods have been developed, there is no internationally standardized method. The VQEG is comprised of experts from various fields and organizations working on the evaluation and standardization of methods for video quality assessment. The first and current task of VQEG is the evaluation of objective model performance against subjective results for video quality in the bit rate range of 768 kbits/sec to 50 Mbits/sec.
  • Article
    Full-text available
    A laboratory experiment investigated the impact of image motion and emotional content in brief television clips on attention. Using an electroencephalogram (EEG) as a measure of attention, cortical activity in 25 participants was measured while they viewed either still or moving emotion-provoking images. Spectral analysis of the EEG was done to gauge changes in alpha-wave (8-13 Hz) power because reductions in alpha power are thought to indicate greater attention. The analysis revealed that subjective reports of emotional arousal were directly related to cortical activation (i.e., reduced alpha power), and this was particularly true at the parietal recording site. The relationship between alpha power and ratings of valence was curvilinear; cortical activation was associated with both positive and negative images relative to neutral images. Alpha power was also reduced during the viewing of moving compared with still images, and this effect occurred independent of stimulus valence. The association of greater cortical activation with moving than with still images is consistent with previous research using autonomic measures and suggests that image motion modulates emotional responding-and sustains attention-primarily through increasing "gain" in the arousal dimension.
  • Article
    Experiments are described that evaluated a model linking instantaneously perceived quality to overall quality judgments of long video sequences. Subjects evaluated a 3-min MPEG-2 coded video sequence by means of a continuous assessment procedure. Additionally, they rated overall quality of segments of 10, 30, 60, and 180 sec of the same video material. The model describing the relation between the instantaneous and overall quality ratings contains two main ingredients, viz. an exponentially decaying weighting function, simulating the experimentally established recall advantage for the most recently presented material (called the recency effect), and a non-linear averaging procedure stressing the relative importance of strong impairments. The fit of the model to the experimental data resulted in a decay time constant of 26 sec and a power of 3 for the nonlinear weighting. These findings suggest that subjects rely predominantly on the worst events of a sequence when determining their overall quality judgment.
  • This paper describes a study which examined the effect of video content on viewers' perception of the picture and sound quality of that content. 40 participants viewed 20 different clips of major studio release movies that were 2 minutes in duration. Each clip was encoded at five different rates (490 kbps, 771 kbps & 1100 kbps, VHS and DVD). Each participant saw each clip only once and the level at which that clip was encoded was selected from a counter balanced list. Results indicated that desirability of the content played a significant role in a viewer's subjective ratings of the sound and video quality for a particular clip. It was found that the sound and picture quality of highly desirable content (as determined by each viewer) was rated significantly higher than content that was deemed neutral or undesirable. The study has implications for selecting content for subjective video quality studies
  • Executive Producers) & Hackford, T. (Director) Proof of Life [Motion picture].United States
    • T Gilroy
    • S Reuther
    Gilroy, T., Reuther, S. (Executive Producers) & Hackford, T. (Director). (2000). Proof of Life [Motion picture].United States: Castle Rock Entertainment.
  • Article
    Objective video quality measurement has become an important issue, as multimedia services are now widely available over the Internet and other wireless communication media. Traditionally, professional CRT monitors have been used to measure subjective video quality. However, the majority of users have LCD, plasma display panel (PDP), or consumer-graded CRT monitors. We compared the subjective video quality of various TV and LCD PC monitors. Subjective tests were performed with a wide range of video sequences using different monitors, and their correlations were analyzed. Although there were high correlations among the various display monitors, care should be taken in selecting a monitor for certain applications.
  • Article
    Using focus groups the study found viewing to be a complicated mix of both the passive and active theories of programming. While there is a real love for the medium itself, that love is based on specific programs, perceived content and a strong comfort level connected to the feeling that programs will be there when they are wanted. Participants had clear mental images of the schedule and tuned in with specific programs or channels in mind.
  • Article
    Television commercials include elements designed to engage the viewer's attention. Manipulations of the commercial's visual structure that result in rapid pacing or frequent scene changes can be engaging because they require a frequent redirection of visual attention. Manipulations of semantic content through such techniques as the inclusion of humorous or anomalous elements can elicit cognitive engagement. Structural manipulations in videos are known to attenuate the alpha (8 to 13 Hz) rhythm of the electroencephalogram (EEG) recorded over posterior cortical regions involved with visual processes. To examine whether other engaging elements also affect the alpha rhythm, EEG was recorded from 10 participants who viewed television commercials. Principal components analysis was used to decompose the alpha rhythm into underlying factors that varied in spatial topography over the head and in spectral composition. Across commercials, the power of a posterior-distributed alpha component was inversely correlated with the frequency of scene changes. In contrast, a frontal component in the lower frequency portion of the alpha band was attenuated during commercials that elicited high subjective interest, independent of scene change frequency. And, an upper-frequency frontal alpha component was attenuated during commercials for which the item being advertised had a high probability of being subsequently recalled, independent of other factors. These frontal patterns of alpha reactivity are consistent with other neuroimaging results concerning frontal lobe activation by executive processes and episodic memory encoding. With appropriate neurologically guided analysis, EEG signals can provide a unique means of monitoring both perceptual and higher order neurocognitive processes during television viewing.
  • Article
    Full-text available
    Staged 2 different videotaped interviews with the same individual—a college instructor who spoke English with a European accent. In one of the interviews the instructor was warm and friendly, in the other, cold and distant. 118 undergraduates were asked to evaluate the instructor. Ss who saw the warm instructor rated his appearance, mannerisms, and accent as appealing, whereas those who saw the cold instructor rated these attributes as irritating. Results indicate that global evaluations of a person can induce altered evaluations of the person's attributes, even when there is sufficient information to allow for independent assessments of them. Furthermore, Ss were unaware of this influence of global evaluations on ratings of attributes. In fact, Ss who saw the cold instructor actually believed that the direction of influence was opposite to the true direction. They reported that their dislike of the instructor had no effect on their rating of his attributes but that their dislike of his attributes had lowered their global evaluations of him. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
  • Article
    This study empirically tests the congruity between movie-content preference and choice on television, video, and at the cinema; and the impact of consumer involvement on this relationship. The results support the need to treat preference and choice differentially. Media type added little in way of explanation of changes in preference–choice congruence. Involvement with movie choice was found to vary by medium, yet correlational analysis showed that it did not unequivocally change the nature of preference–choice congruence. Nonetheless, further enquiry into the role of involvement is warranted before it is dismissed as irrelevant in the choice of leisure media products. © 2002 Wiley Periodicals, Inc.
  • Article
    This paper presents an information-processing model that is directly applicable to the investigation of how mediated messages are processed. It applies the model to the case of television viewing to demonstrate its applicability. It provides a measure for each part of the model. It presents evidence that supports the model in the television-viewing situation. Finally, it demonstrates how the model may be used to further research and understanding in well-known theoretical traditions. This model is not meant to stand in opposition to any of these theories but, rather, should work well with them by providing hypothesized mechanisms that may underlie well-known effects. This model should prove useful both to researchers and, eventually, to message producers. To the extent that we can better understand how the content and structure of messages interact with a viewer's information-processing system to determine which parts and how much of a communication message is remembered, we will make great strides in understanding how people communicate.
  • Conference Paper
    In this paper, we describe subjective video quality testing using naturally occurring, realistic, video content. The use of realistic content poses a number of challenges that are not encountered when using standard video test clips. The methods that we have developed are targeted to define levels of video quality that will be acceptable to customers of a video service. The methods were developed in the context of home TV viewing, but are applicable to other emerging areas such as mobile video. Some of the key elements of testing with natural content are controlling for natural content variability, measuring content desirability, and using a testing method that encourages natural viewing behaviors.
  • Conference Paper
    International recommendations for subjective video quality assessment (e.g., ITU-R BT.500-11) include specifications for how to perform many different types of subjective tests. Some of these test methods are double stimulus where viewers rate the quality or change in quality between two video streams (reference and impaired). Others are single stimulus where viewers rate the quality of just one video stream (the impaired). Two examples of the former are the double stimulus continuous quality scale (DSCQS) and double stimulus comparison scale (DSCS). An example of the latter is single stimulus continuous quality evaluation (SSCQE). Each subjective test methodology has claimed advantages. For instance, the DSCQS method is claimed to be less sensitive to context (i.e., subjective ratings are less influenced by the severity and ordering of the impairments within the test session). The SSCQE method is claimed to yield more representative quality estimates for quality monitoring applications. This paper considers data from six different subjective video quality experiments, originally performed with SSCQE, DSCQS and DSCS methodologies. A subset of video clips from each of these six experiments were combined and rated in a secondary SSCQE subjective video quality test. We give a method for post- processing the secondary SSCQE data to produce quality scores that are highly correlated to the original DSCQS and DSCS data. We also provide evidence that human memory effects for time-varying quality estimation seem to be limited to about 15 seconds.
  • Article
    Full-text available
    Television viewing distances, visual angles, and viewing angles were calculated for 217 children and 149 adults from 78 families. The data were obtained from time-lapse videotapes automatically recorded in the families' homes over 10-day periods. Viewing distance increases with age, and visual angle decreases with age. Viewers aged 17 years and younger viewed at an average distance of 225.3 em, at an average visual angle of 12.3 deg, and at an average viewing angle of 23.7 deg. Adult viewers watched TV at an average distance of 336.8 cm, an average visual angle of 6.6 deg, and at an average viewing angle of 23.3 deg. The best predictors of viewing location were (1) percentage of time the viewer watched TV from furniture, (2) room area, and (3) screen width.
  • Article
    Full-text available
    Three areas of high-level scene perception research are reviewed. The first concerns the role of eye movements in scene perception, focusing on the influence of ongoing cognitive processing on the position and duration of fixations in a scene. The second concerns the nature of the scene representation that is retained across a saccade and other brief time intervals during ongoing scene perception. Finally, we review research on the relationship between scene and object identification, focusing particularly on whether the meaning of a scene influences the identification of constituent objects.