About
80
Publications
14,722
Reads
How we measure 'reads'
A 'read' is counted each time someone views a publication summary (such as the title, abstract, and list of authors), clicks on a figure, or views or downloads the full-text. Learn more
1,517
Citations
Introduction
Additional affiliations
January 2007 - present
Publications
Publications (80)
Virtual Production (VP) technologies have continued to improve the flexibility of on-set filming and enhance the live concert experience. The core technology of VP relies on high-resolution, high-brightness LED panels to playback/render video content. There are a number of technical challenges to effective deployment e.g. image tile synchronisation...
Modern video encoders have evolved into sophisticated pieces of software in which various coding tools interact. In the past, single-pass encoding was not considered for video-on-demand (VOD) use cases. In this work, we evaluate production-ready encoders for H.264 (x264), HEVC (x265), AV1(SVT-AV1) along with direct comparisons to the latest AV1 enc...
In recent years, state-of-the-art image and video denoising networks have become increasingly large, requiring millions of trainable parameters to achieve best-in-class performance. Improved denoising quality has come at the cost of denoising speed, where modern transformer networks are far slower to run than smaller denoising networks such as Fast...
With video streaming making up 80% of the global internet bandwidth, the need to deliver high-quality video at low bitrate, combined with the high complexity of modern codecs, has led to the idea of a per-clip optimisation approach in transcoding. In this paper, we revisit the Lagrangian multiplier parameter, which is at the core of rate-distortion...
Since 2015 video dimensionality has expanded to higher spatial and temporal resolutions and a wider colour gamut. This High Dynamic Range (HDR) content has gained traction in the consumer space as it delivers an enhanced quality of experience. At the same time, the complexity of codecs is growing. This has driven the development of tools for conten...
Over the past few years, there has been an increase in the demand and availability of High Dynamic Range (HDR) displays and content. To ensure the production of high-quality materials, human evaluation is required. However, ascertaining whether the full playback pipeline is indeed HDR-compliant can be challenging. In this paper, we present a set of...
Cloud-based deployment of content production and broadcast workflows has continued to disrupt the industry after the pandemic. The key tools required for unlocking cloud workflows, e.g., transcoding, metadata parsing, and streaming playback, are increasingly commoditized. However, as video traffic continues to increase there is a need to consider t...
The complexity of modern codecs along with the increased need of delivering high-quality videos at low bitrates has reinforced the idea of a per-clip tailoring of parameters for optimised rate-distortion performance. While the objective quality metrics used for Standard Dynamic Range (SDR) videos have been well studied, the transitioning of consume...
Video transcoding is an increasingly important application in the streaming media industry. It has become important to investigate the optimisation of transcoder parameters for a single clip simply because of the immense number of playbacks for popular clips. In this paper, we explore the use of a canned optimiser to estimate the optimal RD tradeof...
The majority of internet traffic is video content. This drives the demand for video compression in order to deliver high quality video at low target bitrates. This paper investigates the impact of adjusting the rate distortion equation on compression performance. An constant of proportionality, k, is used to modify the Lagrange multiplier used in H...
This work focuses on reducing the computational cost of repeated video encodes by using a lower resolution clip as a proxy. Features extracted from the low resolution clip are used to learn an optimal lagrange multiplier for rate control on the original resolution clip. In addition to reducing the computational cost and encode time by using lower r...
In the past ten years there have been significant developments in optimization of transcoding parameters on a per-clip rather than per-genre basis. In our recent work we have presented per-clip optimization for the Lagrangian multiplier in Rate controlled compression, which yielded BD-Rate improvements of approximately 2\% across a corpus of videos...
The majority of internet traffic is video content. This drives the demand for video compression to deliver high quality video at low target bitrates. Optimising the parameters of a video codec for a specific video clip (per-clip optimisation) has been shown to yield significant bitrate savings. In previous work we have shown that per-clip optimisat...
Over the past decade, the evolution of video-sharing platforms has attracted a significant amount of investments on contextual advertising. The common contextual advertising platforms utilize the information provided by users to integrate 2D visual ads into videos. The existing platforms face many technical challenges such as ad integration with re...
Motion estimation is a key component of any modern video codec. Our understanding of motion and the estimation of motion from video has come a very long way since 2000. More than 135 different algorithms have been recently reviewed by Scharstein et al http://vision.middlebury.edu/flow/. These new algorithms differ markedly from Block Matching which...
Over the past decade, the evolution of video-sharing platforms has attracted a significant amount of investments on contextual advertising. The common contextual advertising platforms utilize the information provided by users to integrate 2D visual ads into videos. The existing platforms face many technical challenges such as ad integration with re...
Live video commenting systems are an emerging feature of online video sites. Recently the Chinese video sharing platform Bilibili, has popularised a novel captioning system where user comments are displayed as streams of moving subtitles overlaid on the video playback screen and broadcast to all viewers in real-time. LiveBot was recently introduced...
Cutting out an object and estimating its opacity mask, known as image matting, is a key task in many image editing applications. Deep learning approaches have made significant progress by adapting the encoder-decoder architecture of segmentation networks. However, most of the existing networks only predict the alpha matte and post-processing method...
Interactive object cutout tools are the cornerstone of the image editing workflow. Recent deep-learning based interactive segmentation algorithms have made significant progress in handling complex images and rough binary selections can typically be obtained with just a few clicks. Yet, deep learning techniques tend to plateau once this rough select...
The current state of the art alpha matting methods mainly rely on the trimap as the secondary and only guidance to estimate alpha. This paper investigates the effects of utilising the background information as well as trimap in the process of alpha calculation. To achieve this goal, a state of the art method, AlphaGan is adopted and modified to pro...
The majority of internet traffic is video content. This drives the demand for video compression in order to deliver high quality video at low target bitrates. This paper investigates the impact of adjusting the rate distortion equation on compression performance. An constant of proportionality, k, is used to modify the Lagrange multiplier used in H...
Virtual advertising is an important and promising feature in the area of online advertising. It involves integrating adverts onto live or recorded videos for product placements and targeted advertisements. Such integration of adverts is primarily done by video editors in the post-production stage, which is cumbersome and time-consuming. Therefore,...
Online videos have witnessed an unprecedented growth over the last decade, owing to wide range of content creation. This provides the advertisement and marketing agencies plethora of opportunities for targeted advertisements. Such techniques involve replacing an existing advertisement in a video frame, with a new advertisement. However, such post-p...
The rapid increase in the number of online videos provides the marketing and advertising agents ample opportunities to reach out to their audience. One of the most widely used strategies is product placement, or embedded marketing, wherein new advertisements are integrated seamlessly into existing advertisements in videos. Such strategies involve a...
With the advent of faster internet services and growth of multimedia content, we observe a massive growth in the number of online videos. The users generate these video contents at an unprecedented rate, owing to the use of smart-phones and other hand-held video capturing devices. This creates immense potential for the advertising and marketing age...
High efficiency video coding has made it possible to stream video over bandwidth constrained communication networks. Depending on bit rate requirements, a video encoder sacrifices some image details which can then introduce visual artefacts. Due to aggressive encoding a contouring staircase artefact called banding can be observed in image regions w...
This paper has two main contributions. The first is a Bayesian framework for removing two common types of degradations on video known as blotches and line scratches. Most removal techniques assume complete obliteration of the original data at the corrupted sites. This often leads to the introduction of restoration artifacts during removal. Our fram...
Motion-based video segmentation has been studied for many years and remains challenging. Ill-posed problems must be solved when seeking for a fully automated solution, so it is increasingly popular to maintain users in the processing loop by letting them set parameters or draw mattes to guide the segmentation process. When processing multiple-view...
Accurate registration of document recto and verso sides with bleed-through degradation is essential for accurate automatic non-blind bleed-through removal. This paper presents a registration method for documents with bleed-through degradation, and also an objective registration evaluation scheme. In the proposed method the two sides are first globa...
This paper presents recent work on a new framework for non-blind document bleed-through removal. The framework includes image preprocessing to remove local intensity variations, pixel region classification based on a segmentation of the joint recto-verso intensity histogram and connected component analysis on the subsequent image labelling. Finally...
In this paper we present a method to help with the creation of stereoscopic 3D content from a single monoscopic view alone with depth information. It is designed for CGI, where the 3D animation software has generated a high quality depth map. The system allows the Animator or postproduction Artist to define a depth budget, "auto balance" the parall...
This paper introduces a new database of 25 recto/verso image pairs from documents suffering from bleed-through degradation, together with manually created foreground text masks. The structure and creation of the database is described, and three bleed-through restoration methods are compared in two ways; visually, and quantitatively using the ground...
Video reconstruction has become an important tool for rig and artefact removal in cinema postproduction. In this paper we are concerned with reconstructing stereo video material. We propose a method that builds on existing exemplar-based video inpainting techniques and includes a dedicated view consistency constraint. Within a constrained texture s...
Reflections in image sequences consist of several layers superimposed over each other. This phenomenon causes many image processing techniques to fail as they assume the presence of only one layer at each examined site e.g. motion estimation and object recognition. This work presents an automated technique for detecting reflections in image sequenc...
Correcting colour imbalances between stereo views is a major task in the production of high quality 3D stereoscopic content. This paper addresses extreme cases of colour imbalance where the intensity range of one view exceeds the valid range causing image burnout in the degraded view. We propose a comprehensive framework for restoring the degraded...
This paper introduces a new database of freely available stereo-3D content designed to facilitate research in stereo post-production. It describes the structure and content of the database and provides some details about how the material was gathered. The database includes examples of many of the scenarios characteristic to broadcast footage. Mater...
Depth maps are becoming a readily available commodity of the stereo pipeline. We propose to make use of this new free information to improve a key step of postproduction that is matting. We extend the work of Levin et al on closed form matting to introduce two new depth-aware techniques. First we explore how depth can be used as an extra channel in...
Automated blotch removal is important in film restoration and typically involves a detection/interpolation step. Current algorithms model the corruption as a binary mixture between the original, clean images and an opaque (dirt) field. This typically causes incomplete blotch removal that manifests as blotch haloes in reconstruction. This paper prop...
Graph-cut techniques for depth and disparity estimations are known to be powerful but also slow. We propose a graph-cut framework that is able to estimate depth maps from a set of candidate values. By employing a restricted set of candidates for each pixel, rough depth maps can be effectively refined to be accurate, smooth and continuous. The contr...
Large motion displacements in image sequences are still a problem for most motion estimation techniques. Progress in feature matching allows to establish robust correspondences between images for a sparse set of points. Recent works have attempted to use this sparse information to guide the dense motion field estimation. We propose to achieve this...
This paper examines the accuracy of audio-video based localization using multiple cameras and multi-microphones. Covariance mapping theory is used to determine the accuracy of audio and video based localization. Both modalities are compared in terms of their ability to provide accurate location estimates of a moving audio-visual source. Relatively,...
Color is an essential aspect of a picture which conveys to the viewer many emotions and symbolic meanings. Adjusting the color grade of pictures is therefore an important step in professional photography. This process is part of the larger activity of grading in which the color and grain aspects of the photographic material are digitally manipulate...
This paper presents an automatic method to enhance video presentations for distance learning applications. From a material recorded by a fixed, non professional camera, the system matches the slides displayed during the presentation with their electronic versions. The process to achieve slide recognition consists of two phases. In the first phase,...
A common task in image editing is to change the colours of a picture to match the desired colour grade of another picture. Finding the correct colour mapping is tricky because it involves numerous interrelated operations, like balancing the colours, mixing the colour channels or adjusting the contrast. Recently, a number of automated tools have bee...
This article proposes an original method for grading the colours between different images or shots. The first stage of the method is to find a one-to-one colour mapping that transfers the palette of an example target picture to the original picture. This is performed using an original and parameter free algorithm that is able to transform any N -di...
Brightness fluctuations in sequences are a typical impairment in archive and modern footage. These fluctuations are mainly temporal but they may also present spatial variations across each individual frame. Current deflicker techniques consider that these variations are smooth but this paper shows that the flicker localisation can be very fine in s...
Digital media post-production is an industry standard step in media creation. Now that issues of speed and physical storage have largely been rendered less problematic, the emphasis has shifted toward increasing levels of automation. This thesis makes several contributions in the domain of visual post-production in an attempt to bring advanced stat...
This article proposes an original method to estimate a continuous transformation that maps one N-dimensional distribution to another. The method is iterative, non-linear, and is shown to converge. Only 1D marginal distribution is used in the estimation process, hence involving low computation costs. As an illustration this mapping is applied to col...
This paper presents a probabilistic framework for off-line multiple object tracking. At each timestep, a small set of deterministic candidates is generated which is guaranteed to contain the correct solution. Tracking an object within video then becomes possible using the Viterbi algorithm. In contrast with particle filter methods where candidates...
This article proposes an original method for grading the colours between different images or shots. The first stage of the method is to find a one-to-one colour mapping that transfers the palette of an example target picture to the original picture. This is performed using an original and parameter free algorithm that is able to transform any N-dim...
This article proposes an original method to estimate a continuous transformation that maps one N-dimensional distribution to another. The method is iterative, non-linear, and is shown to converge. Only 1D marginal distributions are used in the estimation process, hence involving low computation costs. As an illustration this mapping is applied to c...
This paper presents a new expression of the relationship between integral projections and motion in an image pair. The resulting new multiresolution gradient based approach is used to estimate dominant motion in image sequences degraded by random shake. The paper also describes an implementation using the GPU as a coprocessor for the CPU that allow...
Contour following is a standard activity in rotoscoping in the digital post production domain. An artist might need to cut out or edit an object separately from its background and it is left to the artist to manually create the cut out. Techniques for automatically tracing the edges of the object exist, but these operate with heavy manual intervent...
Temporal random variation of luminance in images can manifest in film and video due to a wide variety of sources. Typical in archived films, it also a#ects scenes recorded simultaneously with di#erent cameras (e.g. for film special e#ect), and scenes a#ected by illumination problems.
Temporal random variation of luminance in images can man- ifest in film and video due to a wide variety of sources. Typical in archived films, it also affects scenes recorded simultaneously with different cameras (e.g. for film special effect), and scenes affected by illumination problems. Many applications in Computer Vision and Image Processing t...
Temporal and spatial random variation of luminance in images, or
'flicker' is a typical degradation observed in archived film and
video. The underlying premise in typical flicker reduction algorithms is that each image must be corrected for a spatially varying gain and
offset. These parameters are estimated in the stationary region of the
image. He...
Temporal and spatial random variation of luminance in images, or 'flicker' is a typical degradation observed in archived film and video. The underlying premise in typical flicker reduction algorithms is that each image must be corrected for a spatially varying gain and offset. These parameters are estimated in the stationary region of the image. He...
- La variation temporelle de la luminance dans les séquences d'images, ou effet de pompage, est une dégradation typique des archives vidéos et cinématographiques. Nous proposons ici un nouveau procédé qui vise à supprimer ces perturbations visuellement désagréables. Plusieurs améliorations sont proposées à la fois sur le modèle de pompage, l'estima...