-
[show abstract]
[hide abstract]
ABSTRACT: We propose a novel paradigm for interactive video streaming and we coin the term JPEG2000-based scalable interactive video (JSIV) for it. JSIV utilizes JPEG2000 to independently compress the original video sequence frames and provide for quality and spatial resolution scalability. To exploit interframe redundancy, JSIV utilizes prediction and conditional replenishment of code-blocks aided by a server policy that optimally selects the number of quality layer for each code-block transmitted and a client policy that makes most of the received (distorted) frames. It is also possible for JSIV to employ motion compensation; however, we leave this topic to future work. To optimally solve the server transmission problem, a Lagrangian-style rate-distortion optimization procedure is employed. In JSIV, a wide variety of frame prediction arrangements can be employed including hierarchical B-frames of the scalable video coding (SVC) extension of the H.264/AVC standard. JSIV provides considerably better interactivity compared to existing schemes and can adapt immediately to interactive changes in client interests, such as forward or backward playback and zooming into individual frames. Experimental results for surveillance footage, which does not suffer from the absence of motion compensation, show that JSIV's performance is comparable to that of SVC in some usage scenarios while JSIV performs better in others.
IEEE Transactions on Image Processing 06/2011; · 3.04 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: The authors have recently introduced the JPEG2000-Based Scalable Interactive Video (JSIV) paradigm. JSIV relies on JPEG2000 format for providing scalability and accessibility, and on motion compensation and conditional replenishment to exploit temporal redundancy. JSIV can provide considerably better interactivity compared to existing video streaming practices, and can adapt immediately to interactive changes in client interests, such as forward or backward playback and zooming into individual frames. This work extends our previous work by providing server and client policies that can exploit the client's knowledge about the quantization intervals of received samples in selecting a favorable predictor in dyadic hierarchical B-frame arrangement that does not employ motion compensation. We also demonstrate the flexibility of the JSIV paradigm by showing an improved client policy working with a non-improved server policy without any negative impact on reconstructed video.
Image Processing (ICIP), 2010 17th IEEE International Conference on; 10/2010
-
[show abstract]
[hide abstract]
ABSTRACT: The authors have recently proposed a paradigm that can potentially provide for considerably better interactivity compared to existing practices and can adapt immediately to interactive changes in client interests, such as forward or backward playback and zooming into individual frames. The proposed paradigm relies on JPEG2000 format for providing scalability, flexibility, and accessibility; and on transmitting a server-optimized selection of code-blocks and motion side-information. Motion compensation and conditional replenishment are employed to reduce needed bandwidth. This work extends the previous work by providing server and client policies that allow for a realistic implementation and by introducing the use of coarsely quantized code-blocks in improving prediction. This work introduces the concepts, formulates the policies and optimization problems, proposes solutions, and compares the performance to alternate strategies.
Image Processing (ICIP), 2009 16th IEEE International Conference on; 12/2009
-
[show abstract]
[hide abstract]
ABSTRACT: This paper describes a visual optimization strategy for scalable video compression. The challenge scalable coding presents is that truncation of an embedded codestream may induce variable and highly visible distortion. To overcome the deficiencies of visually lossless coding schemes, we propose using an adaptive masking slope to model the perceptual impact of suprathreshold distortion arising from resolution and bit-rate scaling. This allows important scene structures to be better preserved. Following visual masking principles, local sensitivity to distortion is assessed within each frame. To keep the perceptual response uniform against spatiotemporal errors, we mitigate errors compounded by the motion field during temporal synthesis. Visual sensitivity weights are projected into the subband domain along motion trajectories via a process called perceptual mapping. This uses error propagation paths to capture some of the noise-shaping effects attributed to the motion-compensated transform. A key observation is that low contrast regions in the video are generally more susceptible to unmasking of quantization errors. The proposed approach raises the distortion-length slope associated with these critical regions, altering the bitstream embedding order so that visually sensitive sites may be encoded with higher fidelity. Subjective evaluation demonstrates perceptual improvement with respect to bit-rate, spatial and temporal scalability.
IEEE Transactions on Circuits and Systems for Video Technology 04/2009; · 1.65 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Much work has been performed on optimizing the throughput of the block coding system within JPEG2000. However, the question remains as to whether providing parallel simple block coders provides a cheaper method of increasing throughput than complicated optimized block coders. We present the analysis and results for a system on a chip (SoC) software/hardware codesign platform, for parallel coding in JPEG2000 compression standard. We design both a simple and a high performance, optimized peripheral encoder as a hardware accelerator for the JPEG2000 SoC encoding system. The system is implemented on an Altera NIOS II processor with flexible integrated peripheral. We show that there are optimum numbers of parallel block coders and scheduling granularity per row of codeblocks, and that parallel optimized encoders outperform parallel simple encoders. We also demonstrate that the block coding system becomes work starved rather than memory blocked when many parallel coders are present, indicating a discrete wavelet transform bottleneck.
IEEE Transactions on Circuits and Systems for Video Technology 03/2009; · 1.65 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Streaming video as a sequence of JPEG2000 images provides the scalability, flexibility, and accessibility at a wide range of bit-rates that is lacking from the current motion-compensated predictive video coding standards; however, streaming this sequence requires considerably more bandwidth. The authors have recently proposed a novel approach that reduces the required bandwidth; this approach uses motion compensation and conditional replenishment of the JPEG2000 code-blocks, aided by server-optimized selection of these code-blocks. This work extends the previous work to the case of hierarchical arrangement of frames, similar to the hierarchical B-frames of the SVC scalable video coding extension of the H.264/AVC standard. We employ a Lagrangian-style rate-distortion optimization procedure to the server transmission problem and compare the performance to that of streaming individual frames and also to that of predictive video coding. The proposed approach can serve a diverse range of client requirements and can adapt immediately to interactive changes in client interests, such as forward or backward playback and zooming into individual frames. This paper introduces the concepts, formulates the optimization problem, proposes a solution, and compares the performance to alternate strategies.
Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on; 11/2008
-
[show abstract]
[hide abstract]
ABSTRACT: This paper examines the effects of aliasing and investigates the extent to which different transform structures support spatiotemporal scalability. The efficacy of an open-loop spatial pyramid and antialiased DWT schemes are assessed under scalable conditions in terms of their ability to generate highly compressible quality embedded subsets at reduced resolution. The main emphasis is placed on lifting inspired structures with noise suppression or antialiasing properties. As an example, the DWT is augmented with spectral energy exchange lifting steps which disperse aliased content from critical regions of the video sequence at reduced resolution. Finally, we propose alternate ways to characterize a video compression system based on the amount of shift variance and aliasing distortion incurred in half resolution sequences.
Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on; 11/2008
-
[show abstract]
[hide abstract]
ABSTRACT: Oriented wavelets have attracted attention in recent times due to their superior coding performance of images containing diagonal features. We focus on a scheme which effectively aligns the wavelet transform to edges by lifting between shift-interpolated pixels. While compression is improved, poor specification of the orientation can introduce additional aliasing to the lower resolution subband. In this paper, we find that enhanced estimation of the shift field improves both the visual quality of the low resolution subband and compression of the overall image. We also apply an antialiasing transform to the packet decomposition created by the oriented wavelet, which substantially improves the lower resolution at a cost of a reduction in the coding performance.
Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on; 11/2008
-
[show abstract]
[hide abstract]
ABSTRACT: A JPEG2000 compressed video sequence can provide better support for scalability, flexibility, and accessibility at a wider range of bit-rates than the current motion-compensated predictive video coding standards; however, it requires considerably more bandwidth to stream. The authors have recently proposed a novel approach that reduces the required bandwidth; this approach uses motion compensation and conditional replenishment of JPEG2000 code-blocks, aided by server-optimized selection of these code-blocks. The proposed approach can serve a diverse range of client requirements and can adapt immediately to interactive changes in client interests, such as forward or backward playback and zooming into individual frames. This work extends the previous work by approximating the distortion associated with the decisions made by the server without the need to recreate the actual video sequence at the server. The proposed distortion estimation algorithm is general and can be applied to various frames arrangements. Here, we choose to employ it in a hierarchical arrangement of frames, similar to the hierarchical B-frames of the SVC scalable video coding extension of the H.264/AVC standard. We employ a Lagrangian-style rate-distortion optimization procedure to the server transmission problem and compare the performance of both distortion estimation and exact distortion calculation cases against streaming individual frames and SVC. Results obtained suggest that the distortion estimation algorithm considerably reduces the amount of calculation needed by the server without enormously degrading the performance compared to the exact distortion calculation case. This work introduces the concepts, formulates the estimation and optimization problems, proposes a solution, and compares the performance to alternate strategies.
Multimedia Signal Processing, 2008 IEEE 10th Workshop on; 11/2008
-
[show abstract]
[hide abstract]
ABSTRACT: Discrete wavelet transform (DWT) and embedded block coder (BC) are two main modules in JPEG2000 compression system. Data transfer between the DWT and BC modules presents challenges due to difference in data format generated by the DWT and data format required by the BC module. In this paper, we investigate data transfer and storage techniques between the DWT and BC modules. We propose an efficient memory organization and data transfer schemes to reduce the data bandwidth. A VLSI architecture for the proposed data transfer system is proposed and synthesized for TSMC 0.18- process. Simulation results show that our proposed techniques result in approximately four times less bandwidth for the BC module while requiring an extra hardware cost of only 11%.
IEEE Transactions on Circuits and Systems for Video Technology 06/2008; · 1.65 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We present the analysis, design and results for a reduced workload embedded block coding with optimal truncation (EBCOT) computational block, in JPEG2000 compression standard. We propose a new technique of rate distortion (R/D) control technique to reduce the amount of EBCOT computation that is required. In the proposed technique the compression rate requirement is relaxed, and instead we aim at maximizing the image quality while minimizing processing time by reducing the EBCOT workload.
Communications and Information Technologies, 2007. ISCIT '07. International Symposium on; 11/2007
-
[show abstract]
[hide abstract]
ABSTRACT: In this paper we present novel techniques to adapt conventional wavelet transforms to follow locally oriented features found in images. We introduce a shift operator before each step in a lifting implementation of the DWT. The best shifts are estimated by minimizing the high-pass coefficient energy and then used in both the prediction and update lifting steps. To approximate the asymptotically optimal rate-distortion performance of a piece-wise regular function more closely, we adopt a packet wavelet decomposition. Experimental results obtained integrating the proposed transform into the JPEG2000 codec show improvements in both visual and objective tests, allowing for a "better" representation of the edges at very-low rates. Very recently, some related ideas have been presented by other authors. The most distinctive features of this paper include a more flexible packet wavelet decomposition structure and a comparison between subband- and image-domain shifting operators.
Image Analysis and Processing, 2007. ICIAP 2007. 14th International Conference on; 10/2007
-
[show abstract]
[hide abstract]
ABSTRACT: JPEG2000 is a new image compression standard known for its rich set of features, impressive compression performance as well as its complexity for efficient hardware implementation. Discrete wavelet transform (DWT) and embedded block coder (BC) are two main modules in JPEG2000 compression system. Data transfer between the DWT and BC modules presents challenges due to difference in data format generated by the DWT and data format accepted by the BC module. This paper investigates data transfer and storage techniques between the DWT and BC modules. The paper proposed an efficient memory organization and a data transfer scheme to reduce the data bandwidth. A VLSI architecture for the proposed data transfer (DT) system is proposed and synthesized for TSMC 0.18 mum process. Simulation results show that the proposed techniques result in an aggregate reduction in the bandwidth requirement by a factor of four for the BC module while incurring an extra hardware cost of only 5%
Circuits and Systems, 2007. ISCAS 2007. IEEE International Symposium on; 06/2007
-
[show abstract]
[hide abstract]
ABSTRACT: This paper considers the efficacy of a class of intra-channel contrast masking models and proposes a simple extension to increase their effectiveness in capturing perceptual diversity in complex stimuli. "Perceptual diversity" encapsulates the idea that our perceptual response to visual distortion varies, depending not only on where quantization errors occur, but also how they coincide with structural elements that constitute a video frame. We offer a fresh perspective on perceptual modeling and identify factors that limit visual optimization performance. We find that the perceptual impact of suprathreshold visual distortion cannot be fully assessed based on contrast alone, hi this work, we introduce a context-adaptive masking slope to complement the basic functions of our perceptual model. This versatile feature allows visual sensitivity to be emphasized, or suppressed, in a manner which reflects the perceptual significance of local distortion in a video. Finally, we propose a visual optimization strategy for embedded video coding based on a perceptual mapping and distortion scaling approach. This technique mitigates the noise shaping effects due to motion-compensated temporal synthesis
Image Processing, 2006 IEEE International Conference on; 11/2006
-
[show abstract]
[hide abstract]
ABSTRACT: The block coder, which is a key module in the JPEG2000 image compression system, presents challenges for realization of a high-throughput, low-hardware-cost VLSI architecture. Though efficient architectures have been proposed for a block coder operating in specific modes, existing generic block coder architectures have low throughput versus hardware cost performance. In this paper, we present a low-cost, high-throughput VLSI architecture for a generic block coder. Concurrent symbol processing (CSP) is used to improve throughput of the block coder's submodules, the bit plane coder (BPC) and arithmetic coder (AC). The proposed BPC processes one stripe-column/clock-cycle during every coding pass and generates up to 10 context-data (CxD) pairs/clock-cycle. The proposed AC processes two CxD/clock-cycles. Throughput is then further increased by using column speedup and novel run-mode skipping techniques at the BPC module. Hardware cost for the proposed block coder is reduced by using an optimal two-subbank BPC memory architecture. Additionally, image statistics are used to choose efficient configuration parameters for the VLSI architecture. The proposed block coder is implemented on Altera stratix FPGA and TSMC ASIC 0.18-mum platforms. Implementation results show that our block coder has average throughputs of 16.23 and 73.42 Msamples/s, respectively, on the FPGA and ASIC platforms. The block-coder test chip has 22515 gates and 2.33 mm <sup>2</sup> chip area. In comparison with similar existing architectures, it has the highest throughput versus hardware cost performance
IEEE Transactions on Circuits and Systems for Video Technology 08/2006; · 1.65 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: JPEG2000 is a recently standardized image compression algorithm. The heart of this algorithm is the coding scheme known as embedded block coding with optimal truncation (EBCOT). This contributes the majority of processing time to the compression algorithm. The EBCOT scheme consists of a bit-plane coder coupled to a MQ arithmetic coder. Recent bit-plane coder architectures are capable of producing symbols at a higher rate than the existing MQ arithmetic coders can absorb. Thus, there is a requirement for a high throughput MQ arithmetic coder. We examine the existing MQ arithmetic coder architectures and develop novel techniques capable of absorbing the high symbol rate from high performance bit-plane coders, as well as providing flexible design choices
Circuits and Systems I: Regular Papers, IEEE Transactions on 07/2006; · 1.97 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: Optimal rate-control is a very important feature of JPEG2000 which allows simple truncation of compressed bit-stream to achieve best image quality at a given target bit-rate. Accurate distortion estimation with respect to allowed bit-stream truncation points, is essential for rate-control performance. In this paper, we address the issues involved in accurate distortion estimation for hardware oriented implementation of JPEG2000 encoding systems with generic block coding capabilities. We propose a novel hardware friendly distortion estimation technique. Rate control based on the proposed technique results in only an average 0.02 dB PSNR degradation with respect to the optimal distortion estimation approach used in the software implementations of JPEG2000. This is the best performance reported in comparison to existing techniques. The proposed technique requires only an additional 4096 bits per block coder which is 80% less than the memory requirements of optimal approach
Acoustics, Speech and Signal Processing, 2006. ICASSP 2006 Proceedings. 2006 IEEE International Conference on; 06/2006 · 4.63 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: This paper presents a new framework for achieving superior visual quality in scalable video compression. In contrast with adaptive quantization strategies, our approach compensates for motion modeling deficiencies and alleviates temporal artifacts without needing to modify the decoder. The proposed technique works on the principle of post-compression distortion scaling and consists of two key steps. Perceptual analysis identifies regions in the frame, which are visually sensitive. Spatial mappings then incorporate the non-linear effects of the motion field, projecting the sensitivity information back into the subband domain along motion trajectories. The derived sensitivity maps are used to affect the codestream embedding order when quality layers are formed. The overall objective is to raise the distortion-length slope associated with low contrast regions, which are particularly susceptible to the unmasking of artifacts, so that they will be encoded with higher fidelity. This technique substantially improves the visual quality of structural elements that otherwise suffer significant degradation.
Image Processing, 2005. ICIP 2005. IEEE International Conference on; 10/2005
-
[show abstract]
[hide abstract]
ABSTRACT: To achieve high access efficiency in scalable video compression, the embedded codestream should facilitate efficient viewing through a window with compact support in space and time. The ability to interact with a region of interest in any spatio-temporal subband is limited by the granularity of the code blocks chosen during compression, which is linked to coding performance. Previous studies have investigated the optimal code block dimensions with respect to coding efficiency and random accessibility without considering the impact of motion. This paper further examines how the random access cost is exacerbated by motion. The loss in performance due to motion-compensation is considered from a random access perspective, in the context of a non-separable spatio-temporal subband transform. The methods are general and the results are pertinent to a variety of interactive video browsing applications.
Image Processing, 2005. ICIP 2005. IEEE International Conference on; 10/2005
-
[show abstract]
[hide abstract]
ABSTRACT: The JPEG2000 block decoder consists of the bit plane decoder (BPD) tightly coupled with the arithmetic decoder (AD). The BPD generates contexts which are used by the AD to decode sample bits from the compressed bit-stream. The context generation process depends on the decoded bits and thus the BPD has to stall in most cases before it can load a new stripe-column to process. In this paper we present a new selection based sample skipping strategy to restrict the dependence of context generation on decoded bits. This enables the BPD to generate 1 context/clock-cycle for non-empty stripe-columns without stalling to load a new stripe-column. Further we propose speculative technique to improve the operating frequency of our architecture at the expense of extra hardware. The proposed BPD is implemented on an Altera Stratix FPGA. The implementation results show that the proposed architectures yield 50.1-107.1% increase in throughput with only a 27-37% increase in the hardware cost.
Circuits and Systems, 2005. 48th Midwest Symposium on; 09/2005
