DBoidS - Digital twin Boids fire prevention System

Funder

Organizational Unit

Publications

ADVANCED UAV MONITORING: DEEP LEARNING FOR MULTI -TARGET DETECTION, TRACKING, AND WILDFIRE PREDICTION

Publication . Carreira, Samuel Vitorino de Sousa; Pereira, António Manuel de Jesus; Miragaia, Rolando Lúcio Germano; Ramos, João Pedro Ferreira; Ribeiro, José Carlos Bregieiro

Rising global fire incidents necessitate effective solutions, making forest surveillance crucial. Current methods require substantial investment and labor but are often ineffective. This work proposes a comprehensive monitoring solution utilizing Unmanned Aerial Vehicles (UAVs) to integrate visible and infrared images for real-time detection of people, vehicles, and fires, addressing limitations in low-light conditions, fog, or smoke. We propose a new system architecture for real-time UAV footage transmission, processing, and analysis on a cloud server. For the detection of people and vehicles, we propose a new 4-channel object detection model that significantly improves precision metrics compared to traditional state-of-the-art models that utilize only RGB images. Additionally, our model performs better in conditions unfavorable to RGB images, successfully identifying objects in low light and reduced visibility. To train our model, we present a labeled dataset with aligned thermal and visible images from an aerial perspective. In order to enable object tracking in our solution, which refers to detecting and maintaining a unique identifier for each detection, we propose SAME, a new approach to Multiple Object Tracking (MOT) re-identification. The proposed model is designed to extend the capabilities of existing detectors by using the high-dimensional features they extract as inputs to a transformer-based architecture. This method applies attention and transformers to measure the similarity between tracks across multiple frames, significantly improving re-identification performance. SAME employs transformers to enable past context retrieval, standing out for its modularity while achieving competitive results in known datasets such as MOT17 and BDD100K. Finally, we introduce FireSeq, a novel approach leveraging state-of-the-art deep learning techniques such as VQ-VAEs and Transformers to model wildfire progression in real time. To support this research, we developed the FireSeq dataset, which includes both RGB and infrared (IR) aligned imagery capturing the behavior of wildfires from an aerial perspective. Additionally, the FireSeq dataset includes a labeled multi-class subset designed for early wildfire detection. FireSeq demonstrates a high degree of accuracy in predicting future frames of wildfire footage. The three developed components represent innovative research approaches that together form a comprehensive and robust wildfire monitoring solution. This marks a significant advancement in wildfire prevention and proactive management. By enabling continuous real-time monitoring, detection, and tracking, our solution supports critical applications such as risk analysis, crowd management, and searchand- rescue operations. Furthermore, it introduces a novel method for predicting and detecting wildfire progression, aimed at enhancing early detection capabilities and improving mission planning efficiency.

2024-11-12Master thesis

Restricted access

CLOUD PLATFORM FOR MULTI - UAV MANAGEMENT WITH AI - DRIVEN MODULES

Publication . Marques, Tomás Vieira da Graça; Pereira, António Manuel de Jesus; Ramos, João Pedro Ferreira; Costa, Nuno Alexandre Ribeiro da

Given the increasing severity of wildfires worldwide, there is a pressing need to automate forest fire prevention and surveillance systems, ensuring rapid and efficient responses to potential threats. To prevent and improve response to these global fire incidents, we have developed a solution that integrates visible and infrared spectrum images captured by UAVs, overcoming the limitations of single-sensor imagery in challenging conditions such as low light, fog, or smoke. This dissertation presents a comprehensive solution leveraging Internet-of-Drones (IoD) and advanced machine learning techniques for efficient multi-UAV area mapping and forest surveillance. The IoD concept shifts drone control towards cloud-based multi-UAV solutions, exemplified by this work. We propose IoD CloudStation a cloud multi-UAV management solution that supports high-level command abstraction, modular scalable platform integration, secure, and real-time data exchange. IoD CloudStation incorporates a hybrid communication protocol optimizing connectivity between UAVs, gateway servers, and web clients, facilitating numerous applications including data logging, AR/VR, area mapping, and drone vision projects to work with the groundstation making it AI-driven. To enable computer vision from the UAV perspective, a 4-channel deep-learning computer vision model based on an enhanced version of the YOLOv5 architecture was developed, demonstrating significant improvements in precision and Mean Average Precision (mAP) metrics, especially in low-visibility scenarios. This work also presents an approach using Reinforcement Learning (RL) techniques to enable cooperative multi-UAV area mapping, enhancing mapping efficiency and increasing the overall quality of video feed prompted to the computer vision module. The flexibility of RL allows UAVs to respond on demand to changing mapping parameters, such as the number of UAVs and the size of the area to be mapped. This combination of modules, when integrated into the IoD CloudStation, enables real-time UAV monitoring, forest fire prevention, and surveillance.

2024-11-12Master thesis

Restricted access

DIGITAL TWIN UAV FOR WI LDFIRE PREVENTION AND DETECTION USING MOBILE DEVICES

Publication . Costa, Carlos Eduardo Pereira Martinho da; Gonçalves, Alexandrino José Marques; Rodrigues, Nuno Carlos Sousa; Ribeiro, Roberto Aguiar; Costa, Paulo Manuel Almeida

The escalating severity of wildfires presents a significant menace to forests, air and water quality, and soil health, exacerbating the challenges of climate change. Unmanned Aerial Vehicles (UAVs) have emerged as a practical solution for the surveillance and management of wildfires, characterized by their swift deployment, cost-effectiveness, and minimal environmental impact. This study presents a Digital Twin (DT) system for UAVs that enables users to engage with both virtual and real UAVs through mobile devices for wildfire prevention and detection. The proposed system encompasses four primary services: UAV Flight Data Generator (UAVFDG), Visualization, Control, and Retrieval. Moreover, the system facilitates users in honing their UAV control skills in a virtual environment before transitioning to real-world scenarios, effectively mitigating risks and reducing costs. The significant contributions of this research lie in the conceptualization, implementation, and validation of a DT-based mobile system that leverages Augmented Reality (AR), including features like autonomous flight data attribution and movement. Evaluation results underscore the successful implementation of the proposed system, backed by positive feedback from participants and a System Usability Scale (SUS) score of 79.75.

2024-12-04Master thesis

Embargoed access

INTELLIGENT MULTISPECTRAL UAV IMAGERY FOR FIRE MANAGEMENT AND PREVENTION

Publication . Cruz, Mário Rui Santos; Frazão, Luís Alexandre Lopes; Ramos, João Pedro Ferreira

A crescente ameaça de incêndios florestais na Europa Mediterrânica, particularmente em Portugal, destaca a necessidade urgente de abordagens inovadoras para a previsão de incêndios e a vigilância florestal. O risco de incêndio ainda é predominantemente avaliado com base nas condições atmosféricas, o que limita a precisão das mesmas. Leaf Water Content (LWC) é fundamental para entender a saúde da vegetação, o stress hídrico e a irrigação, o que impacta diretamente a estimativa do risco de incêndio. Esta dissertação apresenta o ForestML, uma arquitetura em tempo real que utiliza Unmanned Aerial Vehicle (UAVs) equipados com câmeras multiespectrais e algoritmos de inteligência artificial para prever e analisar florestas a partir de uma perspectiva aérea. Esta solução, o ForestML, é composta pela Ground Station, a Backstation e uma aplicação de gestão e monitoramento, que facilita a avaliação em tempo real de incêndios e riscos associados, apoiando as tomadas de decisões de especialistas. As principais vantagens desta arquitetura incluem a capacidade de processar dados em tempo real numa máquina externa, em vez de depender de modelos no dispositivo, limitados e menos confiáveis, que comprometeriam o desempenho do UAV, como o tempo de voo e a capacidade de armazenamento. Adicionalmente, um conjunto de dados exclusivo foi produzido especificamente para este trabalho. Anteriormente, não havia nenhum Dataset disponível com essas bandas e com alta precisão, pois inclui valores de LWC validados em laboratório. Além disso, é apresentada uma nova métrica de risco de incêndio baseada nos níveis de humidade no interior das folhas da vegetação, permitindo uma avaliação precisa do risco de incêndio. Criámos também um novo modelo a partir do modelo de segmentação YOLOv8 para lidar com dados multiespectrais com 5 ou 8 canais, permitindo uma extração de características melhorada e maior precisão na análise de imagens multiespectrais. O nosso modelo personalizado com 8 canais superou os modelos comuns de 3 canais. O modelo de 8 canais mostrou um desempenho superior na previsão do teor de água nas folhas, atingindo um box mAP50 e mask mAP50-95 de 90,5% e 58,5%, respetivamente.

2024-12-23Master thesis

Embargoed access