A carregar...
Projeto de investigação
SCALABLE VIDEO CODING WITH DYNAMIC REGIONS OF INTEREST
Financiador
Autores
Publicações
A generic framework for optimal 2D/3D key-frame extraction driven by aggregated saliency maps
Publication . Ferreira, Lino; Cruz, Luis A. da Silva; Assunção, Pedro
This paper proposes a generic framework for extraction of key-frames from 2D or 3D video sequences, relying on a new method to compute 3D visual saliency. The framework comprises the following novel aspects that distinguish this work from previous ones: (i) the key-frame selection process is driven by an aggregated saliency map, computed from various feature maps, which in turn correspond to different visual attention models; (ii) a method for computing aggregated saliency maps in 3D video is proposed and validated using fixation density maps, obtained from ground-truth eye-tracking data; (iii) 3D video content is processed within the same framework as 2D video, by including a depth feature map into the aggregated saliency. A dynamic programming optimisation algorithm is used to find the best set of K frames that minimises the dissimilarity error (i.e., maximise similarity) between the original video shots of size
and those reconstructed from the key-frames. Using different performance metrics and publicly available databases, the simulation results demonstrate that the proposed framework outperforms similar state-of-art methods and achieves comparable performance as other quite different approaches. Overall, the proposed framework is validated for a wide range of visual content and has the advantage of being independent from any specific visual saliency model or similarity metrics.
Optimal priority MDC video streaming for networks with path diversity
Publication . Correia, Pedro; Ferreira, Lino; Assunção, Pedro; Cruz, Luis; Silva, Vitor
This paper proposes a robust video streaming scheme for priority networks with path diversity, based on a combined approach of multiple description coding (MDC) with optimal picture classification into two priorities. A binary classification algorithm is proposed to define high (HP) and low (LP) priority network abstraction layer units (NALU), which in turn define the packet priorities. An optimisation algorithm is used to find HP pictures, based on dynamic programming and relying on minimisation of the packet loss concealment distortion. The paper shows that the proposed algorithm is able to effectively improve the decoded video without increasing the MDC stream redundancy. The overall performance evaluation, carried out by simulating MDC video streaming over lossy networks with path diversity, demonstrates that the proposed algorithm yields higher video quality for a wide range of packet loss rates (PLR). Comparing with no-priority MDC video streaming schemes, the simulation results show that the proposed algorithm can improve the average PSNR results in 0.7-3.2dB for packet loss rates between 3% and 15%.
3D video shot boundary detection based on clustering of depth-temporal features
Publication . Ferreira, Lino; Assunção, Pedro; Cruz, Luis A. da Silva
This paper proposes an algorithm for automatic detection of 3D video shots with different perceptual features. The proposed algorithm is able to identify distinct three-dimensional visual scenes by detecting 3D video shot boundaries based on clustering of depth-temporal features. A combination of texture variation along the temporal dimension and depth variance is used by K-means clustering to find the stereo frames which comprised the 3D scene boundaries. An important characteristic of the proposed algorithm in comparison with others published in the literature for temporal segmentation of classic 2D video is that no thresholds are used in the decision processes neither training data sets. The experimental results show that the proposed method is capable of achieving high recall (e.g., 0.95) and precision rate (e.g., 1.0) in video sequences with both sharp and smooth 3D scene transitions.
Multiple Description Video Streaming over Asymmetric Channels
Publication . Correia, Pedro; Assunção, Pedro A. A.; Silva, Vitor
his paper proposes an efficient Unbalanced Multiple Description Scalar Quantisation (U-MDSQ) method for video streaming over asymmetric channels. In order to control the asymmetric target rates for each coded descriptions, the U-MDSQ parameters are combined with the rate control method based on the existing linear relationship between rate and percentage of zeros in transform coefficients in MDSQ domain. The simulation results show that the proposed method exhibits high accuracy for a wide range of target bitrates and unbalanced rates between descriptions. Moreover, the obtained results show the effectiveness of the proposed method in order to improve the overall performance over channels with asymmetric rate and packet loss rate (PLR) conditions, when compared with Balanced MDSQ. This method finds application in video streaming with path diversity based on Multiple Description Coding (MDC) with dynamic channel conditions.
Unidades organizacionais
Descrição
Palavras-chave
, Engineering and technology ,Engineering and technology/Electrical engineering, electronic engineering, information engineering
Contribuidores
Financiadores
Entidade financiadora
Fundação para a Ciência e a Tecnologia, I.P.
Fundação para a Ciência e a Tecnologia, I.P.
Fundação para a Ciência e a Tecnologia, I.P.
Programa de financiamento
FARH
Número da atribuição
SFRH/BD/37510/2007