Content uploaded by Yo-Sung Ho
Author content
All content in this area was uploaded by Yo-Sung Ho on Jan 04, 2016
Content may be subject to copyright.
Error concealment based on directional interpolation
Abstract
Compressed bitstreams are, in general, very sensitive to channel
errors. For instance, a single bit error in a coded video bitstream may
cause severe degradation on picture quality. When bit errors occur
during transmission and cannot be corrected by an error correction
scheme, error concealment is needed to conceal the corrupted image at
the receiver. Error concealment algorithms attempt to repair damaged
portions of the picture by exploiting both the spatial and the temporal
redundancies in the received and reconstructed video signal. We discuss
several temporal, spatial, and transform-domain error concealment
techniques for MPEG coded pictures, and propose a new scheme based on
directional interpolation. We also compare the performance of these
techniques by computer simulation
... Well-known simple error concealments yield optimal error-concealed results on only a few image contexts. For example, BI is optimal for images with flat characteristics [4], while single direction interpolation (DI) is for image blocks with obvious edges [18]. ...
... Thus, considering the face characteristics and background edges, BI [4] and a multi-directional interpolation can be applied. In background, LB may be divided into several areas taking into account the regularity of image context, and then the lost pixels can be interpolated by using BI [4], DI [18] or MDFI in each area. ...
... In particular, low-resolution of CIF or QCIF images contain more information within a MB compared to high-resolution (SD or HD), which means that more than one edge may be included within the MB. For such LB, the desired error concealment effect may not be achieved, by BI [4] or edge DI [18] as ever. To this end, in [7], the LB is divided into several areas by using image edges with Sobel operator, and interpolates by taking weight interpolation value in each area unit. ...
This paper proposes a human face and image edge-based hybrid spatial error concealment. Though human faces on video sequences are of most interest, the face error concealment is not yet easy in case of loss of face information. And furthermore, when there are bit errors in regular edge shapes in background, it affects more seriously to visual effect than irregular image characteristics. In order to overcome these challenges, the proposed algorithm, at first, classifies the lost block into foreground, boundary and background by using face detection, and then selects adaptively bilinear interpolation(BI) and horizontal symmetrical interpolation(HSI) for foreground, multi-direction filling interpolation(MDFI) for boundary and block division-based interpolation(BDI) for background. HSI, MDFI, Bezier curve-based block division of foreground and background and BDI of background are novel error concealments which are proposed in this paper. Our test reveals that the proposed error concealment can achieve a better PSNR compared with previous works including separate, adaptive or hybrid concealments, in terms of visual effect, PSNR and runtime, etc. The proposed algorithm may be utilized as an effective error resilient tool for real-time video applications, such as telephone conference, mobile telephone conference and wireless multimedia camera networks in which power consumption should be low.
... Some of these methods [9] use the corresponding MB's MV from the previous frame which has a better performance, assuming that the motion in video sequences is smooth. Another simple and common method is the use of the average or median of adjacent MVs of damaged MB [10]. The simulation results show that the use of median is better than average method. ...
With the fast growth of communication networks, the video data transmission from these networks is extremely vulnerable. Error concealment is a technique to estimate the damaged data by employing the correctly received data at the decoder. In this paper, an efficient boundary matching algorithm for estimating damaged motion vectors (MVs) is proposed. The proposed algorithm performs error concealment for each damaged macro block (MB) according to the list of identified priority of each frame. It then uses a classic boundary matching criterion or the proposed boundary matching criterion adaptively to identify matching distortion in each boundary of candidate MB. Finally, the candidate MV with minimum distortion is selected as an MV of damaged MB and the list of priorities is updated. Experimental results show that the proposed algorithm improves both objective and subjective qualities of reconstructed frames without any significant increase in computational cost. The PSNR for test sequences in some frames is increased about 4.7, 4.5, and 4.4 dB compared to the classic boundary matching, directional boundary matching, and directional temporal boundary matching algorithm, respectively.
... In this case, only the first scale can be decoded correctly, leading to more pronounced quality degradation. However, compared to existing approaches, which either require additional bitrate by Forward Error Correction (FEC) [65], re-encoding the intra frame, or performing interpolation at the decoder with minimal gains [66], our hierarchical coding method inherently presents loss resilience, upon which a simple decoder-to-encoder echo indication is sufficient. ...
The enhanced Deep Hierarchical Video Compression-DHVC 2.0-has been introduced. This single-model neural video codec operates across a broad range of bitrates, delivering not only superior compression performance to representative methods but also impressive complexity efficiency, enabling real-time processing with a significantly smaller memory footprint on standard GPUs. These remarkable advancements stem from the use of hierarchical predictive coding. Each video frame is uniformly transformed into multiscale representations through hierarchical variational autoencoders. For a specific scale's feature representation of a frame, its corresponding latent residual variables are generated by referencing lower-scale spatial features from the same frame and then conditionally entropy-encoded using a probabilistic model whose parameters are predicted using same-scale temporal reference from previous frames and lower-scale spatial reference of the current frame. This feature-space processing operates from the lowest to the highest scale of each frame, completely eliminating the need for the complexity-intensive motion estimation and compensation techniques that have been standard in video codecs for decades. The hierarchical approach facilitates parallel processing, accelerating both encoding and decoding, and supports transmission-friendly progressive decoding, making it particularly advantageous for networked video applications in the presence of packet loss. Source codes will be made available.
... The proposed methods in [1,4,9] calculate the gradient using Sobel mask on each reliable pixel within the neighboring area surrounding corrupted block, and direction of the gradient is quantized into one of eight directions from 0°to 180°. The direction of edge within the corrupted block is estimated based on count of pixels assigned to each quantized direction, then directional interpolation is applied along the estimated edge directions. ...
Error concealment can recover the video frame which has been corrupted by packet loss over error-prone channel, however, speeding up of error concealment is very important for various real-time applications such as video conferencing, video chatting, etc. A fast spatial error concealment method using prediction mode of the neighboring blocks is presented in this paper. First, the weighting values of edge prediction direction for sixteen 4 × 4 neighboring blocks are calculated considering prediction mode of the opposite neighboring blocks. Second, the significant edges within a corrupted macroblock(MB) are estimated using the sixteen weighting values of edge prediction direction. Finally, the approximations for each corrupted pixel are calculated along each significant edge, then a weighted average of multiple approximations is computed considering prediction mode of the neighboring blocks. Experimental results show that the proposed algorithm speeds up multi-directional interpolation up to 1.17 times while sacrificing image quality for about 0.01 dB on avaerage compared with the previous method.
... The gray levels of pixels in the EB change slowly with the position. So each pixel in the EB can be concealed by linear interpolation using the nearest pixels from the four neighboring blocks along the block boundaries [22]. But for our method we use neighbor blocks imitation to estimate block pixels intensity and equation 15 is applied to imitate block pixel values. ...
With the advancement of IPTV and HDTV technology, previous subtle errors in videos are now becoming more prominent because of the structure oriented and compression based artifacts. In this paper, we focus towards the development of a real-time video quality check system. Light weighted edge gradient magnitude information is incorporated to acquire the statistical information and the distorted frames are then estimated based on the characteristics of their surrounding frames. Then we apply the prominent texture patterns to classify them in different block errors and analyze them not only in video error detection application but also in error concealment, restoration and retrieval. Finally, evaluating the performance through experiments on prominent datasets and broadcasted videos show that the proposed algorithm is very much efficient to detect errors for video broadcast and surveillance applications in terms of computation time and analysis of distorted frames.
The video sequence restoration described in this article has been extended to many areas of our lives today. People use it to perform various restoration tasks, making it possible for some pictures that were impossible to restore in the past. This paper uses experimental methods to study the error concealment and traditional image restoration in video compression, and mainly studies the CDD model algorithm, TV model algorithm and error mode. This article also has a very important meaning for the related research of video communication error control, that is, the research of digital image restoration technology, in the post-processing of film and television works, the fuzzification of network video, and the restoration of old documentaries.
There have been a lot of error concealment methods proposed in the literature so far. Most of them claim to be better than the other ones, however , there are still no standardized means for the performance evaluation of error concealment methods. To evaluate the quality of reconstruction, typically Peak Signal to Noise Ratio (PSNR) is used. There are some other metrics proposed in literature for estimation of subjective image and video quality. Most of them concentrate on the degradations caused by compression. In this work, some of them will be analyzed in order to determine which one approximates better the user perceived quality with respect to different error concealment methods. Another important performance parameter is the computational complexity , crucial especially for the wireless video due to the size and power limited mobile terminals and also due to the real-time requirements of services. At present there is no 'standard' criteria used to compare the complexity of the error concealment methods. This thesis focuses on the performance indicators for evaluating error concealment methods. To test the performance evaluation methods, H.264/AVC (Advanced Video Coding) video codec is used. A brief introduction of its characteristics is presented in Chapter 2. In the third chapter an analysis of error propagation is explained in order to understand the operation of the error concealment methods. Their implementation to the H.264/AVC decoder is described in Chapter 4. In Chapters 5 and 6, a general approach for measuring the quality and complexity of the methods is proposed. The simulation setup for performing the various tests for comparing the methods is explained in Chapter 7. Using the objective video quality metrics, subjective perceived quality and complexity explained in previous chapters, a comparison of the error concealment methods is made in Chapter 8. Finally, in the last chapter, conclusions are shown.
We describe a motion compensated hybrid DCT/DPCM video compression scheme, which has been proposed for the storage of compressed digital video. The scheme, an extension of a current CCITT/ISO proposal for compressing still-frame images, differs from other hybrid DCT proposals in several ways. (1) Reduction of DCT block-artifacts by implementing inverse AC-quantizers which are of finer granularity than the corresponding encoder quantizers. This process, which we label ‘AC-Correction’, is applied to intraframe coded frames. (2) Entropy coding by means of an adaptive binary arithmetic coder. Arithmetic coding results in better compression efficiency than Huffman coding, thus improving image quality when a fixed bandwidth channel is available. (3) Non-linear loop filtering that preserves edge definition. (4) Tight tolerances in the rate control of groups of frames.
This paper presents an adaptive error concealment technique for MPEG (Moving Picture Experts Group) compressed video. Error concealment algorithms are essential for many practical video transmission scenarios characterized by occasional data loss due to thermal noise, channel impairments, network congestion, etc.. Such scenarios of current importance include terrestrial (simulcast) HDTV, teleconferencing via packet networks, TV/HDTV over fiber-optic ATM (asynchronous transfer mode) systems, etc. In view of the increasing importance of MPEG video for many of these applications, a number of error concealment approaches for MPEG have been developed, and are currently being evaluated in terms of their complexity vs. performance trade-offs. Here, we report the results of recent work on a specific adaptive algorithm that provides excellent robustness properties for MPEG-1 video transmitted on either one- or two-tier transmission media. Receiver error concealment is intended to ameliorate the impact of lost video data by exploiting available redundancy in the decoded picture. The concealment process must be supported by an appropriate transport format which helps to identify the image pixel regions which correspond to lost video data. Once the image region (i.e., macroblocks, slices, etc.) to be concealed are identified, a combination of temporal and spatial replacement techniques may be applied to fill in the lost picture elements. The specific details of the concealment procedure will depend upon the compression algorithm being used, and on the level of algorithmic complexity permissible within the decoder. Simulation results obtained from a detailed end-to-end model that incorporates MPEG compression/decompression and a custom cell-relay (ATM type) transport format are reported briefly.
Broadcasting High Definition Television (HDTV) requires the transmission of an enormous amount of information within a highly restricted bandwidth. Adhering to the transmission constraints, channel errors are inevitable. This paper proposes error concealment techniques to remove subjective effects of transmission errors. Error concealment techniques for several data parameters transmitted for the compressed representation of HDTV are considered. Specifically, we address errors in motion vectors and DCT coefficients. The concealment techniques estimate the true value of the corrupted parameters by exploiting the spatial and temporal correlation within the image sequence. In general, the error concealment techniques are found to be extremely successful in removing degradations from the decoded image.
Broadcasting High Definition Television (HDTV) requires the transmission of an enormous amount of information within a highly restricted bandwidth. Adhering to the transmission constraints, channel errors are inevitable. This paper proposes error concealment techniques to remove subjective effects of transmission errors. Error concealment techniques for several data parameters transmitted for the compressed representation of HDTV are considered. Specifically, we address errors in motion vectors and DCT coefficients. The concealment techniques estimate the true value of the corrupted parameters by exploiting the spatial and temporal correlation within the image sequence. In general, the error concealment techniques are found to be extremely successful in removing degradations from the decoded image.© (1993) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
The MPEG-2 source coding algorithm is very sensitive to the
channel disturbances. For instance a single bit error in the bitstream
will cause a high degradation of picture quality due to error
propagation. Hence, for picture replenishment error concealment
techniques (ECTs) may be required at the receiver. The aim of the
article is to study different ECTs for MPEG-2 hierarchical coded
pictures applied for terrestrial broadcasting. For the base layer
different temporal and spatial ECTs are investigated. Two temporal ECTs
are considered: a simple temporal error concealment (EC) and a temporal
EC with motion compensation. The latter method provides the best results
in inter coded pictures (where motion vectors exist). For the intra
coded pictures, where no motion information exists, two spatial
interpolation techniques are considered. The main problem for spatial EC
in MPEG-2 coded pictures is that only the top and the bottom macroblock
can be used for interpolation, since one error in the bitstream causes
one damaged horizontal stripe of macroblocks in the picture. Different
ECTs for the upper layer of hierarchical coded pictures are also
investigated. The possibility of upsampling the base layer for
concealing the upper layer made by spatial scalability gives best
results
The applications of discrete cosine transform (DCT)-based image and video-coding methods in the asynchronous transfer mode (ATM)
environment are considered. Coding and reconstruction mechanisms are
jointly designed to achieve a good compromise among compression gain,
system complexity, processing delay, error-concealment capability, and
reconstruction quality. The Joint Photographic Experts Group (JPEG) and
Motion Picture Experts Group (MPEG) algorithms for image and video
compression are modified to incorporate block interleaving in the
spatial domain and DCT coefficient segmentation in the frequency domain
to conceal the errors due to packet loss. A new algorithm is developed
that recovers the damaged regions by adaptive interpolation in the
spatial, temporal, and frequency domains. The weights used for spatial
and temporal interpolations are varied according to the motion content
and loss patterns of the damaged regions. When combined with proper
layered transmission, the proposed coding and reconstruction methods can
handle very high packet-loss rates at only a slight cost in compression
gain, system complexity, and processing delay
Methods for the interpolation of lost cells in
asynchronous-transfer-mode (ATM) networks are studied. It is shown that
use of motion-compensated previous frames gives the best results. The
quality of the interpolated pictures improves if the motion vectors
truly represent the actual motion in the scene. This is only possible
with a two-layer coding scheme, where the motion vectors can be
delivered to the decoder through the base-layer guaranteed channel. In
derivation of the motion vectors at the encoder, use of uncoded input
picture frames outperforms the conventional method of motion extraction
from the previous coded pictures, despite the lower bit rate of the
latter to the former. Depending on the quality of the base layer and the
scene activity, the signal-to-noise ratio (SNR) in the
cell-loss-interpolated areas can be improved by up to 10 dB
An algorithm using multidirectional interpolation is presented for
error concealment. In the algorithm, an edge classifier analyzes the
values of pels in the blocks surrounding the missing block and
determines which edge directions cut through the missing block.
Interpolations are performed on a local pel neighborhood along the
directions specified by the edge classifier. Then a mixing operation is
used to restore the missing block by extracting the features obtained
from the different directional interpolations and combining them
together. The method of multidirectional interpolation and image mixing
has demonstrated very good results when a sufficiently large
neighborhood of correlated pels exists. This method of spatial
interpolation can be combined with temporal interpolation to provide for
a powerful error concealment technique for compressed video signal
transmission