Loading...
18 results
Search Results
Now showing 1 - 10 of 18
- Deep Learning-Based Point Cloud Coding and Super-Resolution: a Joint Geometry and Color ApproachPublication . Guarda, André F. R.; Ruivo, Manuel; Coelho, Luís; Seleem, Abdelrahman; M. M. Rodrigues, Nuno; Pereira, FernandoIn this golden age of multimedia, realistic content is in high demand with users seeking more immersive and interactive experiences. As a result, new image modalities for 3D representations have emerged in recent years, among which point clouds have deserved especial attention. Naturally, with this increase in demand, efficient storage and transmission became a must, with standardization groups such as MPEG and JPEG entering the scene, as it happened before with other types of visual media. In a surprising development, JPEG issued a Call for Proposals on point cloud coding targeting exclusively learningbased solutions, in parallel to a similar call for image coding. This is a natural consequence of the growing popularity of deep learning, which due to its excellent performances is currently dominant in the multimedia processing field, including coding. This paper presents the coding solution selected by JPEG as the best-performing response to the Call for Proposals and adopted as the first version of the JPEG Pleno Point Cloud Coding Verification Model, in practice the first step for developing a standard. The proposed solution offers a novel joint geometry and color approach for point cloud coding, in which a single deep learning model processes both geometry and color simultaneously. To maximize the RD performance for a large range of point clouds, the proposed solution uses down-sampling and learningbased super-resolution as pre- and post-processing steps. Compared to the MPEG point cloud coding standards, the proposed coding solution comfortably outperforms G-PCC, for both geometry, color, and joint quality metrics.
- Improving Point Cloud to Surface Reconstruction with Generalized Tikhonov RegularizationPublication . Guarda, André; Bioucas-Dias, José M.; M. M. Rodrigues, Nuno; Pereira, Fernando; Bernardo Pereira, Fernando ManuelPoint cloud rendering has a vital role in the user Quality of Experience for applications adopting point cloud based representations. While this is not a new area, it has recently become more relevant with the recent interest on point cloud coding by major standardization groups, notably JPEG and MPEG. The screened Poisson surface reconstruction is a state-ofthe- art technique for generating a watertight surface mesh from the point cloud samples. While its screening component allows the surface to better fit the cloud points, this fitting may lead to undesired artifacts in the surface, notably when the point cloud is noisy. This paper proposes to improve this reconstruction method by making it more robust to noise by adopting a generalized Tikhonov regularization term. The proposed regularization approach smooths regions that should be flat while keeping the important details in the edges, thus creating more pleasant surface reconstructions.
- Point Cloud Coding: Adopting a Deep Learningbased ApproachPublication . Guarda, André; M. M. Rodrigues, Nuno; Pereira, FernandoPoint clouds have recently become an important visual representation format, especially for virtual and augmented reality applications, thus making point cloud coding a very hot research topic. Deep learning-based coding methods have recently emerged in the field of image coding with increasing success. These coding solutions take advantage of the ability of convolutional neural networks to extract adaptive features from the images to create a latent representation that can be efficiently coded. In this context, this paper extends the deep-learning coding approach to point cloud coding using an autoencoder network design. Performance results are very promising, showing improvements over the Point Cloud Library codec often taken as benchmark, thus suggesting a significant margin of evolution for this new point cloud coding paradigm.
- Report on PCC Core Experiment 1.1: Performance of the Verification Model under ConsiderationPublication . Guarda, André F. R.; Rodrigues, Nuno M. M.; Pereira, FernandoThis document reports the performance results of the first version of the JPEG Pleno Point Cloud Coding Verification Model under Consideration, following the Call for Proposals on JPEG Pleno Point Cloud Coding issued in January 2022 [1].
- Studying the Benefits of a New JPEG AI Profile for the JPEG PCC Verification Model: ISO/IEC JTC 1/SC29/WG1 M100115Publication . Seleem, Abdelrahman; Guarda, André; Rodrigues, Nuno; Pereira, FernandoContext and Objective: The current JPEG PCC VM color coding approach first projects the PC color onto 2D images, then uses JPEG AI to code the 2D images.
- IT/IST/IPLeiria Report on JPEG PCC Core Experiment 4.2: Sparse ConvolutionsPublication . Guarda, André; Rodrigues, Nuno; Pereira, FernandoContext and Objective: The current VM uses a dense 3D representation with dense convolutions Problem: Heavy in computation complexity, cannot encode a full point cloud at once; Underperforms for sparse point cloud Objective: Implement the DL models in the VM with a sparse tensor representation, and verify its performance
- IT/IST/IPLeiria Report on JPEG PCC Core Experiment 4.1: Attention ModelsPublication . Ghafari, Mohammadreza; Guarda, André; Rodrigues, Nuno; Pereira, FernandoContext and Objective: In the JPEG Pleno PC dataset, there are some PCs (e.g., sparse PCs) which are more ‘difficult’ to code and may benefit from improvements in the JPEG PCC VM DL coding model.
- IT/IST/IPLeiria Response to the Call for Evidence on JPEG Pleno Point Cloud CodingPublication . Rodrigues, Nuno M. M.; Pereira, Fernando; Guarda, AndréThis document proposes two scalable point cloud (PC) geometry codecs, submitted to the JPEG Call for Evidence on Point Cloud Coding (PCC).
- Deep Learning-based Compressed Domain Point Cloud ClassificationPublication . Seleem, Abdelrahman; Guarda, André; Rodrigues, Nuno; Pereira, FernandoThe JPEG Pleno PCC scope is a learning-based PC coding standard offering a singlestream, compact, compressed domain representation, targeting both human visualization, with significant compression efficiency improvement over PC coding standards in common use at equivalent subjective quality, as well as effective performance for PC processing and computer vision tasks.
- IT/IST/IPLeiria Response to the Call for Proposals on JPEG Pleno Point Cloud CodingPublication . Guarda, André F. R.; Rodrigues, Nuno M. M.; Ruivo, Manuel; Coelho, Luís; Seleem, Abdelrahman; Pereira, FernandoThis document describes a deep learning (DL)-based point cloud (PC) geometry codec and a DL-based PC joint geometry and colour codec, submitted to the Call for Proposals on JPEG Pleno Point Cloud Coding issued in January 2022 [1]. These proposals have been originated by research developed at Instituto de Telecomunicações (IT), in the context of the project Deep-PCR entitled “Deep learning-based Point Cloud Representation” (PTDC/EEI-COM/1125/2021), financed by Fundação para a Ciência e Tecnologia (FCT).