Publicação
Exposing Manipulated Photos and Videos in Digital Forensics Analysis
| datacite.subject.fos | Ciências Médicas::Medicina Clínica | |
| datacite.subject.fos | Ciências Naturais::Ciências da Computação e da Informação | |
| datacite.subject.fos | Engenharia e Tecnologia::Engenharia Eletrotécnica, Eletrónica e Informática | |
| datacite.subject.sdg | 08:Trabalho Digno e Crescimento Económico | |
| datacite.subject.sdg | 09:Indústria, Inovação e Infraestruturas | |
| datacite.subject.sdg | 10:Reduzir as Desigualdades | |
| dc.contributor.author | Ferreira, Sara | |
| dc.contributor.author | Antunes, Mário | |
| dc.contributor.author | Correia, Manuel E. | |
| dc.date.accessioned | 2026-02-23T16:09:52Z | |
| dc.date.available | 2026-02-23T16:09:52Z | |
| dc.date.issued | 2021-06-24 | |
| dc.description.abstract | Tampered multimedia content is being increasingly used in a broad range of cybercrime activities. The spread of fake news, misinformation, digital kidnapping, and ransomware-related crimes are amongst the most recurrent crimes in which manipulated digital photos and videos are the perpetrating and disseminating medium. Criminal investigation has been challenged in applying machine learning techniques to automatically distinguish between fake and genuine seized photos and videos. Despite the pertinent need for manual validation, easy-to-use platforms for digital forensics are essential to automate and facilitate the detection of tampered content and to help criminal investigators with their work. This paper presents a machine learning Support Vector Machines (SVM) based method to distinguish between genuine and fake multimedia files, namely digital photos and videos, which may indicate the presence of deepfake content. The method was implemented in Python and integrated as new modules in the widely used digital forensics application Autopsy. The implemented approach extracts a set of simple features resulting from the application of a Discrete Fourier Transform (DFT) to digital photos and video frames. The model was evaluated with a large dataset of classified multimedia files containing both legitimate and fake photos and frames extracted from videos. Regarding deepfake detection in videos, the Celeb-DFv1 dataset was used, featuring 590 original videos collected from YouTube, and covering different subjects. The results obtained with the 5-fold cross-validation outperformed those SVM-based methods documented in the literature, by achieving an average F1-score of 99.53%, 79.55%, and 89.10%, respectively for photos, videos, and a mixture of both types of content. A benchmark with state-of-the-art methods was also done, by comparing the proposed SVM method with deep learning approaches, namely Convolutional Neural Networks (CNN). Despite CNN having outperformed the proposed DFT-SVM compound method, the competitiveness of the results attained by DFT-SVM and the substantially reduced processing time make it appropriate to be implemented and embedded into Autopsy modules, by predicting the level of fakeness calculated for each analyzed multimedia file. | eng |
| dc.description.sponsorship | The authors acknowledge the facilities provided by INESC TEC, Faculty of Sciences, and University of Porto, for the support to this research. This work is financed by National Funds through the Portuguese funding agency, FCT - Fundação para a Ciência e a Tecnologia, within project UIDB/50014/2020. | |
| dc.identifier.citation | Ferreira, S.; Antunes, M.; Correia, M.E. Exposing Manipulated Photos and Videos in Digital Forensics Analysis. J. Imaging 2021, 7, 102. https://doi.org/10.3390/jimaging7070102. | |
| dc.identifier.doi | 10.3390/jimaging7070102 | |
| dc.identifier.eissn | 2313-433X | |
| dc.identifier.uri | http://hdl.handle.net/10400.8/15699 | |
| dc.language.iso | eng | |
| dc.peerreviewed | yes | |
| dc.publisher | MDPI | |
| dc.relation | INESC TEC- Institute for Systems and Computer Engineering, Technology and Science | |
| dc.relation.hasversion | https://www.mdpi.com/2313-433X/7/7/102 | |
| dc.relation.ispartof | Journal of Imaging | |
| dc.rights.uri | http://creativecommons.org/licenses/by/4.0/ | |
| dc.subject | digital forensics | |
| dc.subject | cybersecurity | |
| dc.subject | multimedia content manipulation | |
| dc.subject | deepfake | |
| dc.subject | convolutional neural networks | |
| dc.subject | support vector machines | |
| dc.subject | discrete fourier transform | |
| dc.title | Exposing Manipulated Photos and Videos in Digital Forensics Analysis | eng |
| dc.type | journal article | |
| dspace.entity.type | Publication | |
| oaire.awardTitle | INESC TEC- Institute for Systems and Computer Engineering, Technology and Science | |
| oaire.awardURI | info:eu-repo/grantAgreement/FCT/6817 - DCRRNI ID/UIDB%2F50014%2F2020/PT | |
| oaire.citation.endPage | 23 | |
| oaire.citation.issue | 7 | |
| oaire.citation.startPage | 1 | |
| oaire.citation.title | Journal of Imaging | |
| oaire.citation.volume | 7 | |
| oaire.fundingStream | 6817 - DCRRNI ID | |
| oaire.version | http://purl.org/coar/version/c_970fb48d4fbd8a85 | |
| person.familyName | Antunes | |
| person.givenName | Mário | |
| person.identifier | R-000-NX4 | |
| person.identifier.ciencia-id | AF10-7EDD-5153 | |
| person.identifier.orcid | 0000-0003-3448-6726 | |
| person.identifier.scopus-author-id | 25930820200 | |
| project.funder.identifier | http://doi.org/10.13039/501100001871 | |
| project.funder.name | Fundação para a Ciência e a Tecnologia | |
| relation.isAuthorOfPublication | e3e87fb0-d1d6-44c3-985d-920a5560f8c1 | |
| relation.isAuthorOfPublication.latestForDiscovery | e3e87fb0-d1d6-44c3-985d-920a5560f8c1 | |
| relation.isProjectOfPublication | 42e9aded-d47f-4d1e-a69d-f5566c0b595d | |
| relation.isProjectOfPublication.latestForDiscovery | 42e9aded-d47f-4d1e-a69d-f5566c0b595d |
Ficheiros
Principais
1 - 1 de 1
A carregar...
- Nome:
- Exposing manipulated photos and videos in digital forensics analysis.pdf
- Tamanho:
- 11.19 MB
- Formato:
- Adobe Portable Document Format
- Descrição:
- Tampered multimedia content is being increasingly used in a broad range of cybercrime activities. The spread of fake news, misinformation, digital kidnapping, and ransomware-related crimes are amongst the most recurrent crimes in which manipulated digital photos and videos are the perpetrating and disseminating medium. Criminal investigation has been challenged in applying machine learning techniques to automatically distinguish between fake and genuine seized photos and videos. Despite the pertinent need for manual validation, easy-to-use platforms for digital forensics are essential to automate and facilitate the detection of tampered content and to help criminal investigators with their work. This paper presents a machine learning Support Vector Machines (SVM) based method to distinguish between genuine and fake multimedia files, namely digital photos and videos, which may indicate the presence of deepfake content. The method was implemented in Python and integrated as new modules in the widely used digital forensics application Autopsy. The implemented approach extracts a set of simple features resulting from the application of a Discrete Fourier Transform (DFT) to digital photos and video frames. The model was evaluated with a large dataset of classified multimedia files containing both legitimate and fake photos and frames extracted from videos. Regarding deepfake detection in videos, the Celeb-DFv1 dataset was used, featuring 590 original videos collected from YouTube, and covering different subjects. The results obtained with the 5-fold cross-validation outperformed those SVM-based methods documented in the literature, by achieving an average F1-score of 99.53%, 79.55%, and 89.10%, respectively for photos, videos, and a mixture of both types of content. A benchmark with state-of-the-art methods was also done, by comparing the proposed SVM method with deep learning approaches, namely Convolutional Neural Networks (CNN). Despite CNN having outperformed the proposed DFT-SVM compound method, the competitiveness of the results attained by DFT-SVM and the substantially reduced processing time make it appropriate to be implemented and embedded into Autopsy modules, by predicting the level of fakeness calculated for each analyzed multimedia file.
Licença
1 - 1 de 1
Miniatura indisponível
- Nome:
- license.txt
- Tamanho:
- 1.32 KB
- Formato:
- Item-specific license agreed upon to submission
- Descrição: