Paulo NunesISCTE-Instituto Universitário de Lisboa | ISCTE
Paulo Nunes
PhD
About
64
Publications
4,464
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,055
Citations
Citations since 2017
Publications
Publications (64)
Deep learning has shown promising results in several computer vision applications, such as style transfer applications. Style transfer aims at generating a new image by combining the content of one image with the style and color palette of another image. When applying style transfer to a 4D Light Field (LF) that represents the same scene from diffe...
Automatic image over-segmentation into superpixels has attracted increasing attention from researchers to apply it as a pre-processing step for several computer vision applications. In 4D Light Field (LF) imaging, image over-segmentation aims at achieving not only superpixel compactness and accuracy but also cross-view consistency. Due to the high...
Efficient segmentation is a fundamental problem in computer vision and image processing. Achieving accurate segmentation for 4D light field images is a challenging task due to the huge amount of data involved and the intrinsic redundancy in this type of images. While automatic image segmentation is usually challenging, and because regions of intere...
This paper proposes a novel light field image compression approach with viewpoint scalability and random access functionalities. Although current state-of-the-art image coding algorithms for light fields already achieve high compression ratios, there is a lack of support for such functionalities, which are important for ensuring compatibility with...
This paper proposes a novel efficient light field coding approach based on a hybrid data representation. Current state-of-the-art light field coding solutions either operate on micro-images or sub-aperture images. Consequently, the intrinsic redundancy that exists in light field images is not fully exploited, as is demonstrated. This novel hybrid d...
Light Field (LF) imaging is a promising solution for providing more immersive and closer to reality multimedia experiences to end-users with unprecedented creative freedom and flexibility for applications in different areas, such as virtual and augmented reality. Due to the recent technological advances in optics, sensor manufacturing and available...
Close range photogrammetry (CRP) is a well‐established technique to retrieve quantitative information from objects using photography. CRP is often used in morphology studies when the direct handling of individuals is unpractical or unethical, or to reduce processing costs and time. Although multiple software to extract quantitative information from...
Light field imaging based on a single-tier camera equipped with a micro-lens array has currently risen up as a practical and prospective approach for future visual applications and services. However, successfully deploying actual light field imaging applications and services will require identifying adequate coding solutions to efficiently handle t...
This chapter addresses image and video technologies related to 3D immersive multimedia delivery systems with special emphasis on the most promising digital formats. Besides recent research results and technical challenges associated with multiview image and image, video and lightfield acquisition and processing, the chapter also presents relevant r...
Light field imaging technology has been recently attracting the attention of the research community and the industry. However, to effectively transmit light field content to the end-user over error-prone networks—e.g., wireless networks or the Internet—error resilience techniques are required to mitigate the impact of data impairments in the user q...
![Figure 3-JPEG Pleno reference LF processing chain [21]](profile/Ricardo-Monteiro-8/publication/327711568/figure/fig2/AS:676871834460161@1538390229534/JPEG-Pleno-reference-LF-processing-chain-21_Q320.jpg)
Light field imaging based on microlens arrays - a.k.a. holoscopic, plenoptic, and integral imaging - has currently risen up as a feasible and prospective technology for future image and video applications. However, deploying actual light field applications will require identifying more powerful representations and coding solutions that support aris...
This paper proposes an efficient light field image coding (LFC) solution based on High Efficiency Video Coding (HEVC) and a novel Bi-prediction Self-Similarity (Bi-SS) estimation and compensation approach to efficiently explore the inherent non-local spatial correlation of this type of content, where two predictor blocks are jointly estimated from...
This paper proposes a two-stage high order intra block prediction method for light field image coding. This method exploits the spatial redundancy in lenslet light field images by predicting each image block, through a geometric transformation applied to a region of the causal encoded area. Light field images comprise an array of micro-images that...
Light field imaging based on microlens arrays – also known as plenoptic, holoscopic and integral imaging – has recently risen up as feasible and prospective technology due to its ability to support functionalities not straightforwardly available in conventional imaging systems, such as: post-production refocusing and depth of field changing. Howeve...
Holoscopic imaging, also known as integral, light field, and plenoptic imaging, is an appealing technology for glassless 3D video systems, which has recently emerged as a prospective candidate for future image and video applications, such as 3D television. However, to successfully introduce 3D holoscopic video applications into the market, adequate...
3D holoscopic imaging, also known as integral imaging, light-field imaging, or plenoptic imaging, has been attracting the attention of the research community as a prospective technology for 3D acquisition and visualization. However, to make this technology a commercially viable candidate for threedimensional services, there are several important re...
This paper presents two HEVC compatible methods to encode 3D holoscopic images, which are composed by an array of micro-images, captured by a large number of low resolution cameras. These images have a high spatial redundancy which cannot be straightforwardly exploited by currently available coding standards, due to the lack of specific coding tool...
The technologies which allow an immersive user experience in 3D environments are rapidly evolving and new services have emerged in various fields of application. Most of these services require the use of 3D video, combined with appropriate display systems. As a consequence, research and development in 3D video continues attracting sustained interes...
Holoscopic imaging is a prospective acquisition and display solution for providing true 3D content and fatigue-free 3D visualization. However, efficient coding schemes for this particular type of content are needed to enable proper storage and delivery of the large amount of data involved in these systems. Therefore, this paper proposes an alternat...
One of the main challenges in 3D light-field imaging approaches lies in the massive amount of visual information involved in providing 3D content with sufficient resolution. Consequently, adequate coding tools are essential for efficient transmission and storage of this type of content. In this context, this paper presents and evaluates two coding...
Holoscopic imaging became a prospective glassless 3D technology to provide more natural 3D viewing experiences to the end user. Additionally, holoscopic systems also allow new post-production degrees of freedom, such as controlling the plane of focus or the viewing angle presented to the user. However, to successfully introduce this technology into...
In the continuous effort to develop new multimedia content for-mats able to support more immersive and close to reality user experiences, 3D holoscopic imaging became lately an appealing technology, allowing a more natural and immersive 3D sensation with continuous full motion parallax, opening also new post-processing degrees of freedom, such as r...
Holoscopic imaging, also known as integral imaging, has been recently attracting the attention of the research community, as a promising glassless 3D technology due to its ability to create a more realistic depth illusion than the current stereoscopic or multiview solutions. However, in order to gradually introduce this technology into the consumer...
Holoscopic imaging has recently become a prospective glassless 3-D technology conquering the attention of researchers seeking more realistic depth-illusion approaches. However, backward compatibility with legacy displays is crucial to progressively introduce this technology into the consumer market and to efficiently deliver 3-D holoscopic content...
We demonstrated a 3D holoscopic video system for 3DTV application. We showed that using a field lens and a square aperture significantly reduces the vignetting problem associated with a relay system and achieves over 95 percent fill factor. The main problem for such a relay system is the nonlinear distortion during the 3D image capturing, which can...
Holoscopic imaging, also known as integral imaging, is a promising solution for glasses-free 3D technology since it allows a more natural and immersive 3D sensation with continuous full motion parallax. However, in order to provide 3D holoscopic content with convenient visual quality in terms of resolution and 3D perception, ultra-high resolution a...
Holoscopic imaging is an advantageous solution for glassless 3D video systems, which promises to revolutionize the 3D market in the near future. Besides freeing the user from wearing any viewing device, it supports full motion parallax, improving this way the users' viewing experience. However, in order to provide 3D holoscopic content with conveni...
Holoscopic imaging, also referred to as integral imaging, is an advantageous solution for glassless 3D which promises in the future to change the market for 3D video systems. In order to efficiently transmit this type of 3D video content over current and emerging networks, this paper proposes an improved spatial and temporal prediction scheme which...
Holoscopic imaging, also known as integral imaging, provides a solution for glassless 3D, and is promising to change the market for 3D television. To start, this paper briefly describes the general concepts of holoscopic imaging, focusing mainly on the spatial correlations inherent to this new type of content, which appear due to the micro-lens arr...
Integral imaging, also known as holoscopic imaging, appears to be a promising approach for glassless 3D. This paper presents the general concepts of integral imaging and lenticular lenses, which are used in the image acquisition and displaying step. Special attention is devoted to the analysis of 3D holoscopic video compression considering its intr...
Holoscopic imaging, also known as integral imaging, is promising to change the market for 3D television since it provides a solution for glassless 3D. This paper starts by making a brief presentation of the general concepts behind holoscopic imaging, with a special emphasis on the spatial correlations that are present in this type of content, which...
Sprite coding, as standardized in MPEG-4 Visual, can result in superior performance compared to common hybrid video codecs. We consider sprite coding, which significantly increases the objective as well as the subjective quality of coded video content. The main challenge of this approach is the segmentation of the fore- ground objects in a preproce...
The MPEG-4 audiovisual coding standard introduced the object-based video data representation model where video data is no
longer seen as a sequence of frames or fields, but consists of independent (semantically) relevant video objects that together
build the video scene. This representation approach allows new and improved functionalities, but it h...
In this paper, an automatic and adaptive network-aware macroblock Intra coding refresh method is proposed. It adaptively selects the amount of gracefully forced Intra macroblocks and the amount of cyclic Intra refresh (CIR) macroblocks based on the actual network error conditions, in terms of packet loss rate, an the target encoding bit rate. With...
This paper proposes an improved rate control algorithm for jointly encoding multiple arbitrarily shaped video objects in the context of low-delay MPEG-4 compliant video coding. The algorithm provides adequate mechanisms for dealing with deviations between the ideal and the actual behavior of video scene encoders, notably: 1) compensation mechanisms...
In this paper, an error resilient rate control scheme for the H.264/AVC standard is proposed. This scheme differs from traditional rate control schemes in that macroblock mode decisions are not made only to minimize their rate-distortion cost, but also take into account that the bitstream will have to be transmitted through an error-prone network....
Object-based video coding, as standardized in MPEG-4 Part 2, can result in superior performance in comparison to common hybrid motion-compensated DCT-based approaches. We consider sprite coding which increases significantly the objective as well as the subjective quality of the coded video. The main challenge of this approach is the pre-segmentatio...
![Fig. 1. Error resilient MB Intra/Inter mode decision architecture [6].](profile/Luis-Soares-12/publication/253276507/figure/fig1/AS:655160124665859@1533213754395/Error-resilient-MB-Intra-Inter-mode-decision-architecture-6_Q320.jpg)
This paper proposes a network-aware macroblock (MB) coding mode decision method, which is both error resilient and coding efficient. This method differs from traditional mode decision methods since MB mode decisions are made by simultaneously taking into account: i) their rate-distortion (RD) cost and also ii) their impact on error resilience by co...
This paper proposes new buffer and video object distortion feedback compensation mechanisms for efficiently dealing with deviations between the ideal and the actual behavior of video scene encoders when jointly encoding multiple arbitrarily shaped video objects in the context of compliant low-delay object-based MPEG-4 video coding. The proposed sol...
Rate and distortion models can play a very important role in real-time video encoding, since they can be used to obtain near optimal operation performance in terms of the RD tradeoff without the drawback of having to encode multiple times the same VOP to find the best combination of coding parameters. In the context of object-based video encoding,...
MPEG-4 is the first object-based audiovisual coding standard. To control the minimum decoding complexity resources required at the decoder, the MPEG-4 Visual standard defines the so-called video buffering verifier mechanism, which includes three virtual buffer models, among them the video complexity verifier (VCV). This paper proposes an alternativ...
When using the MPEG-4 standard, the several video objects
composing a scene may vary in size along time and may be encoded at
different temporal rates using different macroblock (MB) coding types.
To limit the decoding complexity of the corresponding bitstreams, it is
then necessary to put some limits on the variability of the number and
type of MB...
MPEG-4 is the first object-based audiovisual coding standard. To
control the minimum decoding complexity resources required at the
decoder, the MPEG-4 visual standard defines the so-called video
complexity verifier (VCV). This paper proposes an alternative VCV model,
based on a set of relative macroblock (MB) complexity weights assigned
to the vari...
Object-based coding approaches, such as the MPEG-4 standard approach,
where a video scene is composed by several video objects, require that
the rate control is performed by using two levels: the scene rate
control and the object rate control. In this context, this paper
presents a new scene level and object level rate control algorithm for
low del...
This paper presents a contour-based approach to efficiently code binary shape information in the context of object-based video coding. This approach meets some of the most important requirements identified for the MPEG-4 standard, notably efficient coding and low delay. The proposed methods support both object-based lossless and quasi-lossless codi...
Any set of MPEG-4 elementary bitstreams building a video scene can only be considered profile@level compliant if it does not violate the MPEG-4 video buffering verifier constraints for the chosen profile@level. This paper analyses this mechanism, discussing its major features and drawbacks, notably in comparison with alternative solutions. Furtherm...
This paper presents a chain code based approach to efficiently
code binary shape information of video objects, in the context of
object-based video coding. The proposed method tries to meet some of the
requirements of the MPEG-4 standard, currently under development,
notably efficient coding, and low delay. This approach allows several
modes of ope...
Very low bit-rate video coding has recently become one of the most important areas of image communication and a large variety of applications have already been identified. Since conventional approaches are reaching a saturation point, in terms of coding efficiency, a new generation of video coding techniques, aiming at a deeper “understanding” of t...
Very low bitrate video coding became in the last years one of the most important areas of image communication due to the identification of several very low bitrate applications such as mobile videotelephony, multimedia mail, electronic newspapers, entertainment, traffic control, and interactive data bases. Since conventional video coding techniques ar...
The advent of widespread mobile communications together with the continuous development of image communication markets led to the idea of offering mobile image communications, particularly mobile videotelephony. Since very low bitrate video coding is still a quite unexplored subject, a large research effort is being put into the study of the possib...
1 — This paper proposes an improved rate control architecture for jointly encoding multiple video objects in compliance with MPEG-4 Visual video profiles. The proposed scheme is capable of efficiently encoding single and multiple arbitrarily shaped video objects under a wide range of bit rates and spatio-temporal resolutions, outperforming the usua...
Audio conferencing is an important aspect of Internet Telephony services. In this article, using a centralized conferencing architecture, we propose to employ application layer multicast for media distribution, by using "agents" responsible for the delivery of streaming media to end-clients, aiming at reducing the traffic in the network and the ser...
This paper describes an MPEG-4 video compliant framework for the creation, encoding and decoding of video scenes composed of multiple video objects. The generated scenes can be compliantly encoded and the bitstreams can be decoded resulting in individual video objects that can be independently accessed in the decoded scene.
This paper addresses the problem of rate and distortion modeling in the context of object-based MPEG-4 video encoding by comparing different rate and distortion models for Intra coding in the form of rate-quantization, distortion-quantization and rate-distortion functions. Rate-distortion modeling is an important tool for achieving proper rate-cont...
Projects
Project (1)
In order to enable light field (LF) content to be presented on various types of displays, such as legacy displays and also newer 3D LF displays, with different characteristics in terms of spatial and view resolutions, an efficient scalable codec has to be developed. New coding approaches, beyond state-of-the-art and standard-based approaches are being investigated.
