Article

# VESA Display Stream Compression for Television and Cinema Applications

Authors:
• MacInnis Research
To read the full-text of this research, you can request a copy directly from the authors.

## Abstract

The Video Electronics Standards Association (VESA) Display Stream Compression (DSC) standard is widely accepted as a low-cost, visually lossless codec for use in display links. However, as high dynamic range (HDR) and wide color gamut (WCG) technologies become more prevalent, several new features are needed to address television and cinema applications, including support for greater bit depths and 4:2:2/4:2:0 pixel formats. This paper overviews the DSC standard and outlines some new features that have been standardized in the recent VESA DSC 1.2 standard.

## No full-text available

... VESA Display Stream Compression (DSC) is a lightweight video codec designed for low-impairment, low-latency compression and decompression over display links between computational devices and displays, or internal interfaces [1]. The codec is designed for a variety of display applications but in all cases the goal is to provide modest bandwidth reduction while maintaining sufficient fidelity so that a viewer cannot distinguish the compressed images or image sequences from the uncompressed source (visually lossless) [2]. ...
... In the DSC algorithm slices are independently decodable rectangular regions that can be processed independently. DSC 1.2 is reportedly less prone to horizontal slice boundary artifacts than previous versions of the codec, therefore, slice width in these trials was always at least 2 slices per line to make it more likely that horizontal slice boundaries would be visible [1]. The slice height varied by either 108 lines or 32 lines per slice to test coding quality with respect to slice height due to uneven bit budget allocation in the first line. ...
Conference Paper
Full-text available
VESA Display Stream Compression (DSC) is a light-weight codec designed for visually lossless compression over display links. Such high-performance algorithms must be evaluated subjectively to assess whether the codec meets visually lossless criteria. Here we present the first large-scale evaluation of DSC1.2 according to ISO/IEC 29170-2.
... VESA Display Stream Compression (DSC) is a lightweight video codec designed for low-impairment, low-latency compression and decompression over display links between computational devices and displays, or internal interfaces [1]. The codec is designed for a variety of display applications but in all cases the goal is to provide modest bandwidth reduction while maintaining sufficient fidelity so that a viewer cannot distinguish the compressed images or image sequences from the uncompressed source (visually lossless) [2]. ...
... In the DSC algorithm slices are independently decodable rectangular regions that can be processed independently. DSC 1.2 is reportedly less prone to horizontal slice boundary artifacts than previous versions of the codec, therefore, slice width in these trials was always at least 2 slices per line to make it more likely that horizontal slice boundaries would be visible [1]. The slice height varied by either 108 lines or 32 lines per slice to test coding quality with respect to slice height due to uneven bit budget allocation in the first line. ...
Article
Full-text available
VESA Display Stream Compression (DSC) is a light-weight codec designed for visually lossless compression over display links. Such high-performance algorithms must be evaluated subjectively to assess whether the codec meets visually lossless criteria. Here we present the first large-scale evaluation of DSC1.2 according to ISO/IEC 29170-2.
... VESA Display Stream Compression (DSC) is a lightweight video codec designed for low-impairment, low-latency compression and decompression over display links between computational devices and displays, or internal interfaces [1]. The codec is designed for a variety of display applications but in all cases the goal is to provide modest bandwidth reduction while maintaining sufficient fidelity so that a viewer cannot distinguish the compressed images or image sequences from the uncompressed source (visually lossless) [2]. ...
... In the DSC algorithm slices are independently decodable rectangular regions that can be processed independently. DSC 1.2 is reportedly less prone to horizontal slice boundary artifacts than previous versions of the codec, therefore, slice width in these trials was always at least 2 slices per line to make it more likely that horizontal slice boundaries would be visible [1]. The slice height varied by either 108 lines or 32 lines per slice to test coding quality with respect to slice height due to uneven bit budget allocation in the first line. ...
... The Video Electronics Standard Association (VESA) has proposed a new standard of low-cost, visually lossless video codec for compressing video streams on the host sides and decompressing them on the display sides, thereby resolving the bandwidth limitation; namely, Display Stream Compression (DSC) [9]. The VESA DSC has focused on the environment that the host sides generally have abundant computing resources, such as powerful CPUs and GPUs with ample memory space and enough power resources. ...
... The DSC standard also applies MPP and BP as optional prediction schemes; pixel values are predicted using the neighboring pixels by the algorithms. More details of these prediction modes are explained by Walls and MacInnis [9]. ...
Article
In recent years, new emerging display-based consumer products such as AR/VR headsets and automotive video systems have required higher communication bandwidth due to higher bits per pixel and refresh rates. Thus, many video compression techniques have been actively studied to support the bandwidth in a limited communication environment, and the Video Electronics Standard Association (VESA) has standardized a display stream compression (DSC) that provides visually lossless video quality while preserving low power consumption and implementation cost. In this paper, we describe the detailed design of the DSC decoder and optimize a line buffer size that occupies most of the decoder’s resources in terms of power and area consumption. So our optimization method can save many resources used by the DSC decoder. The proposed decoder was functionally verified on an FPGA-based platform and synthesized with a 65nm standard cell library. The performance analysis showed that the optimized decoder used only 62mW for decoding per frame of FHD video by reducing power consumption by 37.0% and area by 39.9% than the original design; thus, our technology would become an attractive solution for developers of the emerging consumer products.
... Therefore, many studies have been conducted to reduce power consumption by decreasing the size of image data with simple compression method before transmission to use computing processes and memory as small as possible [6][7][8]. Accordingly, low-memory compression methods have been adopted in mobile devices, especially in those that require high-power consumption and bandwidth for high-resolution display [9] and in vehicle safety systems that require very low delays for real-time image transmission [10]. ...
Article
Full-text available
Emerging Internet of things (IoT) technologies have rapidly expanded to multimedia applications, including high-resolution image transmission. However, handling image data in IoT products with limited battery capacity requires low-complexity and small-size solutions such as low-memory compression techniques. The objective of this paper is to propose a line-based compression system based on four-level two-line discrete wavelet transform and adaptive line prediction. Bit stream is generated by multiplexing various frequency components with run-level coding combined with Huffman coding. The proposed system also includes a new bit rate control algorithm that could significantly improve image quality consistency in one frame. The proposed low-memory compression system can retain image quality for visually lossless compression criteria over the whole image frame. It can simultaneously lower total system power consumption in multimedia IoT products better than other existing low-memory compression techniques.
... The complexity of SfM is high, because it requires a separate matrix for each camera view. This problem can be solved by applying a technique called redundant view mining 1 We consider 162°×135°fields of view (FoV), 60 pixels per degree, 96 bits per pixel, 1800 fps refresh rate, and display stream compression [39]. [42], which is able to reduce the processing time by about 36%. ...
Article
Full-text available
Immersive media services, such as augmented and virtual reality (AR/VR)a, 360-degree video, and freeviewpoint video (FVV), are popular today. They require massive data storage, ultra-high computing power, and ultra-low latency. It is hard to fulfill these requirements simultaneously in a conventional communication system using a cloud/centralized radio access network (C-RAN). Specifically, due to centralized processing in such a system, the end-to-end latency as well as the burden on the fronthaul network are expected to be high. Fog computing-based radio access networks (F-RAN), in contrast, have been widely considered as an enabler for immersive media. Our contribution in this paper is threefold: First, we propose various service scenarios reflecting the characteristics of immersive media. Second, we identify the technologies that are required to support the proposed service scenarios under F-RAN, and discuss how they can support the proposed scenarios efficiently. Third, we discuss possible research opportunities. a A list of acronyms can be found in Appendix A.
... Two lightweight algorithms have been developed recently to address this growing requirement. VESA Display Stream Compression standard (DSC) 1.2a is a released, lightweight video codec designed to provide low-impairment, low-latency compression for display applications with greater dynamic range [1]. The recently introduced VESA Display Compression-M Standard v1.0 (VDC-M) is a higher-complexity codec targeting higher rates of compression for use where higher pixel bandwidth is required (e.g. ...
Article
High‐dynamic range displays provide impressive image quality, but require markedly higher bandwidth. Here we report results of the first large‐scale subjective assessment of HDR image compression. We applied the ISO/IEC 29170‐2 flicker paradigm to evaluate two state‐of‐the art VESA codecs (DSC v1.2a, pre‐release version of VDC‐M) at several compression levels.
Article
With the rapid increasing resolution of display devices in computer, television and mobile phone, there is a consensus in the industry to solve the problem of lacking of display link bandwidth using display stream compression (DSC) technologies. Hence, several display stream compression methods continually appear in recent years, for example JPEG-XS, intra-only coding in H.264/AVC and Dirac (VC-2), HEVC-SCC screen contents coding and DSC of VESA, etc. Among these, the VESA DSC is a widely acceptable standard that is used in display link for a low cost, low delay, and visually lossless lightweight codec. A number of high efficient coding technologies are used in DSC including advanced prediction, indexed color history, plain entropy coding and perfect rate control. The paper overviews these new features and key technologies in DSC standard. © 2017, Editorial Department of Journal of Data Acquisition and Processing. All right reserved.
Article
With the recent development of network and video technologies, transporting and sharing of screen content is becoming more and more popular. However, there are numerous challenges in the coding screen content for a mix of computer-generated graphics/text and camera-captured image/video, for example, high compression, lossless quality and low cost, etc. Several crucial techniques responding to these challenges and two state-of-the-art standards are proposed: high efficiency video coding-screen content coding (HEVC-SCC) by JCT-VC and display stream compression (DSC) by VESA. The advance of technologies, standards and applications for screen content video coding are prospected. © 2017, Periodical Office of Nanjing University of Posts and Telecommunications. All right reserved.
Article
Abstract Current video coding standards such as the H.264 or Advanced Video Coding (AVC) and the H.265 or High Efficiency Video Coding (HEVC) are based on the Integer Discrete Cosine Transform (IDCT). However, it has been shown that wavelet-based algorithms have better performance than IDCT-based algorithms for still images. Also, exploiting the human visual system characteristics in stationary point of view image and video transmissions like the fovea aliasing can improve the quality of the reconstructed image. In this paper, two wavelet-based video coding approaches called SPECK-based Codec (SP-Codec) and Adaptive Wavelet/Fovea-based Codec (AWFV-Codec) are proposed. The proposed SP-Codec approach applies the Set Partitioned Embedded Block Codec (SPECK) wavelet-based compression algorithm in order to increase the intra-frame coding quality. The second proposed approach AWFV-Codec uses the Adaptive Fovea Set Partitioned Embedded Block Codec (AFV-SPECK) fovea wavelet-based compression algorithm for intra-frame coding. Fovea based compression allows to increase the quality of the reconstructed frames over regions of interest (ROI). A comparison of the proposed algorithms against the HEVC based on IDCT approach shows how the proposed wavelet-based schemes achieve higher compression ratios and higher reconstruction quality. When fixation point of view is stationary, the proposed wavelet/fovea-based algorithm achieves better quality at lower compression ratios than the wavelet-based only proposed algorithm.
Article
The next generation of wireless networks fosters the adoption of latency-critical applications such as XR, connected industry, or autonomous driving. This survey gathers implementation aspects of different image and video coding schemes and discusses their tradeoffs. Standardized video coding technologies such as HEVC or VVC provide a high compression ratio, but their enormous complexity sets the scene for alternative approaches like still image, mezzanine, or texture compression in scenarios with tight resource or latency constraints. Regardless of the coding scheme, we found inter-device memory transfers and the lack of sub-frame coding as limitations of current full-system and software-programmable implementations.
Chapter
It is well-known that there has been a considerable progress in multimedia technologies during the last decades, namely TV, photography, sound and video recording, communication systems, etc., which came into the world during at least half of the previous century and were developed as analog systems, and nowadays have been almost completely replaced by digital systems. The aforementioned motivates a deep study of multimedia compression and intensive research in this area. Data compression is concerned with minimization of the number of information carrying units used to represent a given data set. Such smaller representation can be achieved by applying coding algorithms. Coding algorithms can be either lossless algorithms that reconstruct the original data set perfectly or lossy algorithms that reconstruct a close representation of the original data set. Both methods can be used together to achieve higher compression ratios. Lossless compression methods can either exploit statistical structure of the data or compress the data by building a dictionary that uses fewer symbols for each string that appears on the data set. Lossy compression, on the other hand, uses a mathematical transform that projects the current data set onto the frequency domain. The coefficients obtained from the transform are quantized and stored. The quantized coefficients require less space to be stored. This chapter is focused on the recently published advances in image and video compression to date considering the use of the integer discrete cosine transform (IDCT), wavelet transforms, and fovea centralis.
Article
This brief presents two software Display Stream Compression (DSC) video decoder designs for many-core processor arrays. The first design exploits fine-grained task-level parallelism and is able to decode pictures configured into one column of slices; it is implemented with 88 processors and 2 shared memory modules. The second design facilitates higher performance by leveraging scalable slice-level parallelism and is tailored for pictures configured into multiple columns of slices; one implementation of this design is mapped to 359 processors and 6 shared memory modules. At 1.75 GHz and 1.1 V, the proposed decoders decode 1080p video sequences in 4:2:0, 4:2:2, and 4:4:4 pixel formats—achieving up to 94.7 frames per second (fps), 95.6 fps, and 47.9 fps, while dissipating 23.9 nJ, 26.7 nJ, and 47.2 nJ per pixel, respectively. Our designs achieve up to $159\times$ higher throughput and $841\times$ lower energy per pixel than a DSC decoder implemented on one core of an Intel i7-7700HQ processor.
Article
We propose a method to reduce the footprint of compressed data by using modified virtual address translation to permit random access to the data. This extends our prior work on using page translation to perform automatic decompression and deswizzling upon accesses to fixed rate lossy or lossless compressed data. Our compaction method allows a virtual address space the size of the uncompressed data to be used to efficiently access variable-size blocks of compressed data. Compression and decompression take place between the first and second level caches, which allows fast access to uncompressed data in the first level cache and provides data compaction at all other levels of the memory hierarchy. This improves performance and reduces power relative to compressed but uncompacted data. An important property of our method is that compression, decompression, and reallocation are automatically managed by the new hardware without operating system intervention and without storing compression data in the page tables. As a result, although some changes are required in the page manager, it does not need to know the specific compression algorithm and can use a single memory allocation unit size. We tested our method with two sample CPU algorithms. When performing depth buffer occlusion tests, our method reduces the memory footprint by 3.1x. When rendering into textures, our method reduces the footprint by 1.69x before rendering and 1.63x after. In both cases, the power and cycle time are better than for uncompacted compressed data, and significantly better than for accessing uncompressed data.
Article
High‐resolution display bandwidth requirements often now exceed the capacity of display link channels necessitating compression. The goal of visually lossless compression codecs such as VESA DSC 1.2 is that viewers perceive no difference between the compressed and uncompressed images, maintaining long‐standing expectations of a lossless display link. Such low impairment performance is difficult to validate as artifacts are at or below sensory threshold. We have developed a 3D version of the ISO/IEC 29170‐2 flicker paradigm and used it to compare the effects of image compression in flat images presented in the plane of the screen (2D) to compression in flat images with a disparity offset from the screen (3D). We hypothesized that differences in the location and size of the compression errors between the disparate images in the 3D case would affect their visibility. The results showed that artifacts were often less visible in 3D compared to 2D viewing. These findings have practical applications with respect to codec performance targets and algorithm development for 3D movie, animation, and virtual reality content. In particular, higher compression should be attainable in stereoscopic compared to equivalent 2D images because of increased tolerance to artifacts that are binocularly unmatched or have disparity relative to the screen.
Article
Full-text available
The Video Electronics Standards Association (VESA®) has standardized on a visually lossless, low-complexity, hardware-based compression algorithm for use over display links. This paper summarizes novel features of this new compression standard and provides some evaluation results.
Article
Full-text available
This paper reviews a set of color spaces that allow reversible mapping between red-green-blue and luma-chroma representations in integer arithmetic. The YCoCg transform and its reversible form YCoCg-R can improve coding gain by over 0.5 dB with respect to the popular YCrCb transform, while achieving much lower computational complexity. We also present extensions of the YCoCg transform for four-channel CMYK pixel data. Thanks to their reversibility under integer arithmetic, these transforms are useful for both lossy and lossless compression. Versions of these transforms are used in the HD Photo image coding technology (which is the basis for the upcoming JPEG XR standard) and in recent editions of the H.264/MPEG-4 AVC video coding standard.
Article
Assessing image quality is an important aspect of developing new display technology. A particularly challenging assessment is determining whether a bitwise lossy operation is visually lossless. We define “visually lossless” and describe a new standard for a subjective procedure to assess whether the image quality meets these criteria. Assessments are made between a reference image and temporally interleaved reference and test images using a forced-choice procedure. In extensive testing, we have validated that this method is suitable for discriminating between subtle differences in image rendering and is free of observer bias or criteria variability. The results of these tests demonstrate the efficacy of using as few as five randomly chosen observers. We have found that the subjective testing is more reliable than several widely available image quality metrics. As part of this work, we release a database of nearly 0.25 million subjective responses collected from 35 observers to 18 different images. The study uses a largely within-subjects design and tested observers from two viewing distances. We encourage the use of this dataset in future research to refine objective image quality metrics to improve predictability of subtle but potentially visible compression-induced image impairments.
Conference Paper
Methods of compressing HDTV (high-definition television) image sequences are investigated. Best results are achieved by a new combination of median adaptive intraframe prediction and arithmetic coding of the resultant prediction error. Fully reversible compression with low computational complexity is possible. The method has been used to compress the simulated progressive scan HDTV image sequence `Kiel Harbor' to a rate less than that of an (uncompressed) interlace source. Interlaced HDTV images have been compressed to approximately 55% of their original rate. Intended applications are image storage and full-quality data exchange
VC-2 Video Compression
• Smpte St
VC-2 Video Compression, SMPTE ST.2042-1, 2012.