Loading...
Publication
PM4IPLeiria – Implementação de um Data Warehouse e de técnicas de Process Mining para análise de processos do Politécnico de Leiria
| Name: | Description: | Size: | Format: | |
|---|---|---|---|---|
| 2.64 MB | Adobe PDF |
Authors
Advisor(s)
Abstract(s)
presente projeto, desenvolvido no âmbito do Mestrado em Engenharia Informática – Computação Móvel, aborda a análise e otimização dos processos administrativos do Instituto Politécnico de Leiria (IPLeiria). O ponto de partida foi a identificação de significativas ineficiências e não conformidades nos processos atuais da instituição, o que motivou a criação do projeto PM4IPLeiria.
Para responder a este desafio, foi concebida e implementada uma solução completa de Business Intelligence, assente numa arquitetura de Data Warehouse. O objetivo central foi consolidar os dados dispersos de 16 processos de negócio distintos, provenientes de Vistas SQL. Para tal, foi desenvolvido um pipeline de Extração, Transformação e Carga (ETL) automatizado, orquestrado pelo Azure Data Factory. Este pipeline executou uma lógica de transformação complexa em Python, que realizou operações de limpeza, normalização de dados, cálculo de métricas de performance baseadas em horas úteis e de custos associados. Adicionalmente, foram aplicados algoritmos de Process Mining da PM4Py, uma biblioteca open-source de referência nesta área, nomeadamente a descoberta de Directly-Follows Graphs (DFG) e a análise de variantes, e foram carregados os dados processados para o Data Warehouse final. A estrutura de dados resultante foi um Esquema em floco de neve composto por nove tabelas (oito de dimensão e uma de factos), populado através de estratégias de carga híbridas: incremental para dimensões simples e "delete-and-insert" por processo para garantir a consistência das métricas.
Com esta abordagem, pretende-se dotar os dirigentes do IPLeiria de uma ferramenta robusta de suporte à decisão. A solução permite monitorizar indicadores de desempenho, identificar atempadamente anomalias e, consequentemente, fundamentar ações de melhoria contínua dos processos administrativos.
his Project, developed as part of the Master's program in Computer Engineering – Mobile Computing, addresses the analysis and optimization of administrative processes at the Polytechnic University of Leiria (IPLeiria). The project was prompted by the identification of significant inefficiencies and non-conformities within the institution's current workflows, which motivated the creation of the PM4IPLeiria project. To address this challenge, a complete Business Intelligence solution was designed and implemented, based on a Data Warehouse architecture. The central objective was to consolidate scattered data from 16 distinct business processes, sourced from SQL Views. To this end, an automated Extract, Transform, Load (ETL) pipeline was developed, orchestrated by Azure Data Factory. This pipeline executed a complex transformation logic in Python, which performed data cleaning and normalization operations, calculated performance metrics based on business hours, and their associated costs. Additionally, it applied Process Mining algorithms from the PM4Py library—namely the discovery of Directly-Follows Graphs (DFG) and variant analysis—and loaded the processed data into the final Data Warehouse. The resulting data structure was a Snowflake Schema composed of nine tables (eight dimension tables and one fact table), populated through hybrid loading strategies: incremental for simple dimensions and 'delete-and-insert' per process to ensure metric consistency. Data exploration and visualization are performed using interactive dashboards developed in the Microsoft Power BI tool. With this approach, the goal is to provide IPLeiria's managers with a robust decision-support tool. The solution enables the monitoring of performance indicators, the timely identification of anomalies, and consequently, supports data-driven actions for the continuous improvement of administrative processes.
his Project, developed as part of the Master's program in Computer Engineering – Mobile Computing, addresses the analysis and optimization of administrative processes at the Polytechnic University of Leiria (IPLeiria). The project was prompted by the identification of significant inefficiencies and non-conformities within the institution's current workflows, which motivated the creation of the PM4IPLeiria project. To address this challenge, a complete Business Intelligence solution was designed and implemented, based on a Data Warehouse architecture. The central objective was to consolidate scattered data from 16 distinct business processes, sourced from SQL Views. To this end, an automated Extract, Transform, Load (ETL) pipeline was developed, orchestrated by Azure Data Factory. This pipeline executed a complex transformation logic in Python, which performed data cleaning and normalization operations, calculated performance metrics based on business hours, and their associated costs. Additionally, it applied Process Mining algorithms from the PM4Py library—namely the discovery of Directly-Follows Graphs (DFG) and variant analysis—and loaded the processed data into the final Data Warehouse. The resulting data structure was a Snowflake Schema composed of nine tables (eight dimension tables and one fact table), populated through hybrid loading strategies: incremental for simple dimensions and 'delete-and-insert' per process to ensure metric consistency. Data exploration and visualization are performed using interactive dashboards developed in the Microsoft Power BI tool. With this approach, the goal is to provide IPLeiria's managers with a robust decision-support tool. The solution enables the monitoring of performance indicators, the timely identification of anomalies, and consequently, supports data-driven actions for the continuous improvement of administrative processes.
Description
Keywords
Gestão de processos Business intelligence Data warehouse ETL Power BI Apoio à decisão