Browsing by Author "Pereira, Fernando"
Now showing 1 - 10 of 20
Results Per Page
Sort Options
- Adaptive bridge model for compressed domain point cloud classificationPublication . Seleem, Abdelrahman; Guarda, André F. R.; Rodrigues, Nuno M. M.; Pereira, FernandoThe recent adoption of deep learning-based models for the processing and coding of multimedia signals has brought noticeable gains in performance, which have established deep learning-based solutions as the uncontested state-of-the-art both for computer vision tasks, targeting machine consumption, as well as, more recently, coding applications, targeting human visualization. Traditionally, applications requiring both coding and computer vision processing require frst decoding the bitstream and then applying the computer vision methods to the decompressed multimedia signals. However, the adoption of deep learning-based solutions enables the use of compressed domain computer vision processing, with gains in performance and computational complexity over the decompressed domain approach. For point clouds (PCs), these gains have been demonstrated in the single available compressed domain computer vision processing solution, named Compressed Domain PC Classifer, which processes JPEG Pleno PC coding (PCC) compressed streams using a PC classifer largely compatible with the state-of-the-art spatial domain PointGrid classifer. However, the available Compressed Domain PC Classifer presents strong limitations by imposing a single, specifc input size which is associated to specifc JPEG Pleno PCC confgurations; this limits the compression performance as these confgurations are not ideal for all PCs due to their diferent characteristics, notably density. To overcome these limitations, this paper proposes the frst Adaptive Compressed Domain PC Classifer solution which includes a novel adaptive bridge model that allows to process the JPEG Pleno PCC encoded bit streams using diferent coding confgurations, now maximizing the compression efciency. Experimental results show that the novel Adaptive Compressed Domain PC Classifer allows JPEG PCC to achieve better compression performance by not imposing a single, specifc coding confguration for all PCs, regardless of its diferent characteristics. Moreover, the added adaptability power can achieve slightly better PC classifcation performance than the previous Compressed Domain PC Classifer and largely better PC classifcation performance (and lower number of weights) than the PointGrid PC classifer working in the decompressed domain.
- Constant Size Point Cloud Clustering: a Compact, Non-Overlapping SolutionPublication . Guarda, André F. R.; Rodrigues, Nuno M. M.; Pereira, FernandoPoint clouds have recently become a popular 3D representation model for many application domains, notably virtual and augmented reality. Since point cloud data is often very large, processing a point cloud may require that it be segmented into smaller clusters. For example, the input to deep learning-based methods like auto-encoders should be constant size point cloud clusters, which are ideally compact and non-overlapping. However, given the unorganized nature of point clouds, defining the specific data segments to code is not always trivial. This paper proposes a point cloud clustering algorithm which targets five main goals: i) clusters with a constant number of points; ii) compact clusters, i.e. with low dispersion; iii) non-overlapping clusters, i.e. not intersecting each other; iv) ability to scale with the number of points; and v) low complexity. After appropriate initialization, the proposed algorithm transfers points between neighboring clusters as a propagation wave, filling or emptying clusters until they achieve the same size. The proposed algorithm is unique since there is no other point cloud clustering method available in the literature offering the same clustering features for large point clouds at such low complexity
- Deep Learning-Based Compressed Domain Multimedia for Man and Machine: A Taxonomy and Application to Point Cloud ClassificationPublication . Seleem, Abdelrahman; Guarda, André F. R.; M. M. Rodrigues, Nuno; Pereira, Fernando
- Deep Learning-based Compressed Domain Point Cloud ClassificationPublication . Seleem, Abdelrahman; Guarda, André; Rodrigues, Nuno; Pereira, FernandoThe JPEG Pleno PCC scope is a learning-based PC coding standard offering a singlestream, compact, compressed domain representation, targeting both human visualization, with significant compression efficiency improvement over PC coding standards in common use at equivalent subjective quality, as well as effective performance for PC processing and computer vision tasks.
- Deep Learning-Based Point Cloud Coding and Super-Resolution: a Joint Geometry and Color ApproachPublication . Guarda, André F. R.; Ruivo, Manuel; Coelho, Luís; Seleem, Abdelrahman; M. M. Rodrigues, Nuno; Pereira, FernandoIn this golden age of multimedia, realistic content is in high demand with users seeking more immersive and interactive experiences. As a result, new image modalities for 3D representations have emerged in recent years, among which point clouds have deserved especial attention. Naturally, with this increase in demand, efficient storage and transmission became a must, with standardization groups such as MPEG and JPEG entering the scene, as it happened before with other types of visual media. In a surprising development, JPEG issued a Call for Proposals on point cloud coding targeting exclusively learningbased solutions, in parallel to a similar call for image coding. This is a natural consequence of the growing popularity of deep learning, which due to its excellent performances is currently dominant in the multimedia processing field, including coding. This paper presents the coding solution selected by JPEG as the best-performing response to the Call for Proposals and adopted as the first version of the JPEG Pleno Point Cloud Coding Verification Model, in practice the first step for developing a standard. The proposed solution offers a novel joint geometry and color approach for point cloud coding, in which a single deep learning model processes both geometry and color simultaneously. To maximize the RD performance for a large range of point clouds, the proposed solution uses down-sampling and learningbased super-resolution as pre- and post-processing steps. Compared to the MPEG point cloud coding standards, the proposed coding solution comfortably outperforms G-PCC, for both geometry, color, and joint quality metrics.
- Deep Learning-Based Point Cloud Coding: A Behavior and Performance StudyPublication . M. M. Rodrigues, Nuno; Guarda, André; Pereira, FernandoPoint clouds are an emerging 3D visual representation model for immersive and interactive multimedia applications, inparticular for virtual and augmented reality. The huge amount of data associated to point clouds critically asks for efficient point cloud coding technology. While there are already some point cloud coding paradigms in the literature, notably octree, patch and graph-based for geometry data, very recently deep learning emerged in this research domain, offering very promising performances for image coding. While deep learning-based methods often provide interesting results, the understanding of this type of coding solutions is essential to improve their design in order to be used effectively. In this context, this paper presents a study and analysis on the behavior and performance of a deep learning-based point cloud coding solution based on an autoencoder network using only convolutional layers. Beside a promising RD performance, other findings should allow making
- Deep Learning-based Point Cloud Geometry Coding with Resolution ScalabilityPublication . Guarda, André F. R.; Rodrigues, Nuno M. M.; Pereira, FernandoPoint clouds are a 3D visual representation format that has recently become fundamentally important for immersive and interactive multimedia applications. Considering the high number of points of practically relevant point clouds, and their increasing market demand, efficient point cloud coding has become a vital research topic. In addition, scalability is an important feature for point cloud coding, especially for real-time applications, where the fast and rate efficient access to a decoded point cloud is important; however, this issue is still rather unexplored in the literature. In this context, this paper proposes a novel deep learning-based point cloud geometry coding solution with resolution scalability via interlaced sub-sampling. As additional layers are decoded, the number of points in the reconstructed point cloud increases as well as the overall quality. Experimental results show that the proposed scalable point cloud geometry coding solution outperforms the recent MPEG Geometry-based Point Cloud Compression standard which is much less scalable.
- A Double Deep Learning-Based Solution for Efficient Event Data Coding and ClassificationPublication . Seleem, Abdelrahman; Guarda, André; M. M. Rodrigues, Nuno; Pereira, FernandoEvent cameras have the ability to capture asynchronous per-pixel brightness changes, usually called "events", offering advantages over traditional frame-based cameras for computer vision tasks. Efficiently coding event data is critical for practical transmission and storage, given the very significant number of events captured. This paper proposes a novel double deep learning-based solution for efficient event data coding and classification, using a point cloud-based representation for events. Moreover, since the conversions from events to point clouds and back to events are key steps in the proposed solution, novel tools are proposed and their impact is evaluated in terms of compression and classification performance. Experimental results show that it is possible to achieve a classification performance for decompressed events which is similar to the one for original events, even after applying a lossy point cloud codec, notably the recent deep learning-based JPEG Pleno Point Cloud Coding standard, with a clear rate reduction. Experimental results also demonstrate that events coded using the JPEG standard achieve better classification performance than those coded using the conventional lossy MPEG Geometry-based Point Cloud Coding standard for the same rate. Furthermore, the adoption of deep learning-based coding offers future high potential for performing computer vision tasks in the compressed domain, which allows skipping the decoding stage, thus mitigating the impact of compression artifact
- Improving Point Cloud to Surface Reconstruction with Generalized Tikhonov RegularizationPublication . Guarda, André; Bioucas-Dias, José M.; M. M. Rodrigues, Nuno; Pereira, Fernando; Bernardo Pereira, Fernando ManuelPoint cloud rendering has a vital role in the user Quality of Experience for applications adopting point cloud based representations. While this is not a new area, it has recently become more relevant with the recent interest on point cloud coding by major standardization groups, notably JPEG and MPEG. The screened Poisson surface reconstruction is a state-ofthe- art technique for generating a watertight surface mesh from the point cloud samples. While its screening component allows the surface to better fit the cloud points, this fitting may lead to undesired artifacts in the surface, notably when the point cloud is noisy. This paper proposes to improve this reconstruction method by making it more robust to noise by adopting a generalized Tikhonov regularization term. The proposed regularization approach smooths regions that should be flat while keeping the important details in the edges, thus creating more pleasant surface reconstructions.
- IT/IST/IPLeiria Report on JPEG PCC Core Experiment 4.1: Attention ModelsPublication . Ghafari, Mohammadreza; Guarda, André; Rodrigues, Nuno; Pereira, FernandoContext and Objective: In the JPEG Pleno PC dataset, there are some PCs (e.g., sparse PCs) which are more ‘difficult’ to code and may benefit from improvements in the JPEG PCC VM DL coding model.