Loading...
Research Project
Scalable Light Field Representation and Coding for Immersive Systems
Funder
Authors
Publications
Light field image coding with flexible viewpoint scalability and random access
Publication . Monteiro, Ricardo J. S.; Rodrigues, Nuno M. M.; Faria, Sérgio M. M.; Nunes, Paulo J. L.
This paper proposes a novel light field image compression approach with viewpoint scalability and random
access functionalities. Although current state-of-the-art image coding algorithms for light fields already achieve
high compression ratios, there is a lack of support for such functionalities, which are important for ensuring
compatibility with different displays/capturing devices, enhanced user interaction and low decoding delay.
The proposed solution enables various encoding profiles with different flexible viewpoint scalability and
random access capabilities, depending on the application scenario. When compared to other state-of-the-art
methods, the proposed approach consistently presents higher bitrate savings (44% on average), namely when
compared to pseudo-video sequence coding approach based on HEVC. Moreover, the proposed scalable codec
also outperforms MuLE and WaSP verification models, achieving average bitrate saving gains of 37% and
47%, respectively. The various flexible encoding profiles proposed add fine control to the image prediction
dependencies, which allow to exploit the tradeoff between coding efficiency and the viewpoint random access,
consequently, decreasing the maximum random access penalties that range from 0.60 to 0.15, for lenslet and
HDCA light fields.
Light Field Image Coding Based on Hybrid Data Representation
Publication . Monteiro, Ricardo J. S.; Rodrigues, Nuno M. M.; Faria, Sérgio M.M.; Nunes, Paulo J. L.
This paper proposes a novel efficient light field coding approach based on a hybrid data representation. Current state-of-the-art light field coding solutions either operate on micro-images or sub-aperture
images. Consequently, the intrinsic redundancy that exists in light field images is not fully exploited,
as is demonstrated. This novel hybrid data representation approach allows to simultaneously exploit four
types of redundancies: i) sub-aperture image intra spatial redundancy, ii) sub-aperture image inter-view
redundancy, iii) intra-micro-image redundancy, and iv) inter-micro-image redundancy between neighboring
micro-images. The proposed light field coding solution allows flexibility for several types of baselines,
by adaptively exploiting the most predominant type of redundancy on a coding block basis. To demonstrate
the efficiency of using a hybrid representation, this paper proposes a set of efficient pixel prediction methods
combined with a pseudo-video sequence coding approach, based on the HEVC standard. Experimental
results show consistent average bitrate savings when the proposed codec is compared to relevant state-ofthe-art benchmarks. For lenslet light field content, the proposed coding algorithm outperforms the HEVCbased pseudo-video sequence coding benchmark by an average bitrate savings of 23%. It is shown for the
same light field content that the proposed solution outperforms JPEG Pleno verification models MuLE and
WaSP, as these codecs are only able to achieve 11% and −14% bitrate savings over the same HEVC-based
benchmark, respectively. The performance of the proposed coding approach is also validated for light fields
with wider baselines, captured with high-density camera arrays, being able to outperform both the HEVCbased benchmark, as well as MuLE and WaSP.
Optimized Reference Picture Selection for Light Field Image Coding
Publication . Monteiro, J. S. Ricardo; Nunes, J. L. Paulo; M. M. Rodrigues, Nuno; Faria, Sergio
This paper proposes a new reference picture selection method for light field image coding using the pseudovideo sequence (PVS) format. State-of-the-art solutions to encode light field images using the PVS format rely on video coding standards to exploit the inter-view redundancy between each sub-aperture image (SAI) that composes the light field. However, the PVS scanning order is not usually considered by the video codec. The proposed solution signals the PVS scanning order to the decoder, enabling implicit optimized reference picture selection for each specific scanning order. With the proposed method each reference picture is selected by minimizing the Euclidean distance to the current SAI being encoded. Experimental results show that, for the same PVS scanning order, the proposed optimized reference picture selection codec outperforms HEVC video coding standard for light field image coding, up to 50% in terms of bitrate savings
Organizational Units
Description
Keywords
Contributors
Funders
Funding agency
Fundação para a Ciência e a Tecnologia
Funding programme
OE
Funding Award Number
SFRH/BD/136953/2018