SCALABLE VIDEO CODING WITH DYNAMIC REGIONS OF INTEREST

Publications

A generic framework for optimal 2D/3D key-frame extraction driven by aggregated saliency maps

Publication . Ferreira, Lino; Cruz, Luis A. da Silva; Assunção, Pedro

This paper proposes a generic framework for extraction of key-frames from 2D or 3D video sequences, relying on a new method to compute 3D visual saliency. The framework comprises the following novel aspects that distinguish this work from previous ones: (i) the key-frame selection process is driven by an aggregated saliency map, computed from various feature maps, which in turn correspond to different visual attention models; (ii) a method for computing aggregated saliency maps in 3D video is proposed and validated using fixation density maps, obtained from ground-truth eye-tracking data; (iii) 3D video content is processed within the same framework as 2D video, by including a depth feature map into the aggregated saliency. A dynamic programming optimisation algorithm is used to find the best set of K frames that minimises the dissimilarity error (i.e., maximise similarity) between the original video shots of size and those reconstructed from the key-frames. Using different performance metrics and publicly available databases, the simulation results demonstrate that the proposed framework outperforms similar state-of-art methods and achieves comparable performance as other quite different approaches. Overall, the proposed framework is validated for a wide range of visual content and has the advantage of being independent from any specific visual saliency model or similarity metrics.

2015-11Journal article

Restricted access

Optimal priority MDC video streaming for networks with path diversity

Publication . Correia, Pedro; Ferreira, Lino; Assunção, Pedro; Cruz, Luis; Silva, Vitor

This paper proposes a robust video streaming scheme for priority networks with path diversity, based on a combined approach of multiple description coding (MDC) with optimal picture classification into two priorities. A binary classification algorithm is proposed to define high (HP) and low (LP) priority network abstraction layer units (NALU), which in turn define the packet priorities. An optimisation algorithm is used to find HP pictures, based on dynamic programming and relying on minimisation of the packet loss concealment distortion. The paper shows that the proposed algorithm is able to effectively improve the decoded video without increasing the MDC stream redundancy. The overall performance evaluation, carried out by simulating MDC video streaming over lossy networks with path diversity, demonstrates that the proposed algorithm yields higher video quality for a wide range of packet loss rates (PLR). Comparing with no-priority MDC video streaming schemes, the simulation results show that the proposed algorithm can improve the average PSNR results in 0.7-3.2dB for packet loss rates between 3% and 15%.

2012-07Conference paper

Restricted access