Automated Generation of Troubleshooting Guides from Incident Data Using Large Language Models (2025)
- Authors:
- Autor USP: CRUZ, BRUNA MAGRINI DA - ICMC
- Unidade: ICMC
- Subjects: APRENDIZADO COMPUTACIONALlarpcal; RESOLUÇÃO DE PROBLEMAS; MUDANÇA CLIMÁTICA; MEIO AMBIENTE
- Language: Inglês
- Abstract: The maintenance phase in large-scale cloud environments is critical as it ensures that incidents reported by users or engineers are effectively mitigated. Troubleshooting Guides (TSGs) play a key role in documenting past incidents, enabling faster resolution by future On-Call Engineers (OCEs); however, generating high-quality TSGs is time-consuming, requiring technical expertise and synthesis of diverse information sources. This study proposes an automated pipeline to generate TSGs by leveraging clustering techniques to group incidents representing the same underlying issue, followed by the application of Large Language Models (LLMs) guided through prompt engineering to extract the cause, symptoms, and mitigation steps. A dataset combining synthetic and confidential cloud incidents was created, totaling 26 incidents across 10 classes. The effectiveness of K-Means clustering was assessed using the Adjusted Rand Index (ARI), yielding a score of 0.71, which indicates a high level of alignment with the ground truth incident classes. Generated TSGs were manually evaluated for correctness and quality, achieving a 67% success rate for correctness and 78% for quality. Incorporating chat histories as additional context improved the completeness of the guides. These results demonstrate the feasibility of automatically generating high-quality TSGs, reducing manual effort and enhancing incident resolution efficiency, while highlighting the potential of combining human expertise with LLM-based automation.
- Imprenta:
- Publisher place: São Carlos
- Date published: 2025
-
ABNT
CRUZ, Bruna Magrini da. Automated Generation of Troubleshooting Guides from Incident Data Using Large Language Models. 2025. Trabalho de Conclusão de Curso (MBA) – Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, São Carlos, 2025. Disponível em: https://bdta.abcd.usp.br/directbitstream/98f65133-017b-4ee3-8b41-15e0d4147433/Bruna_Magrini_da_Cruz_TCC_2025.pdf. Acesso em: 17 maio 2026. -
APA
Cruz, B. M. da. (2025). Automated Generation of Troubleshooting Guides from Incident Data Using Large Language Models (Trabalho de Conclusão de Curso (MBA). Instituto de Ciências Matemáticas e de Computação, Universidade de São Paulo, São Carlos. Recuperado de https://bdta.abcd.usp.br/directbitstream/98f65133-017b-4ee3-8b41-15e0d4147433/Bruna_Magrini_da_Cruz_TCC_2025.pdf -
NLM
Cruz BM da. Automated Generation of Troubleshooting Guides from Incident Data Using Large Language Models [Internet]. 2025 ;[citado 2026 maio 17 ] Available from: https://bdta.abcd.usp.br/directbitstream/98f65133-017b-4ee3-8b41-15e0d4147433/Bruna_Magrini_da_Cruz_TCC_2025.pdf -
Vancouver
Cruz BM da. Automated Generation of Troubleshooting Guides from Incident Data Using Large Language Models [Internet]. 2025 ;[citado 2026 maio 17 ] Available from: https://bdta.abcd.usp.br/directbitstream/98f65133-017b-4ee3-8b41-15e0d4147433/Bruna_Magrini_da_Cruz_TCC_2025.pdf
Download do texto completo
| Tipo | Nome | Link | |
|---|---|---|---|
 |
Bruna_Magrini_da_Cruz_TCC... | Direct link |
How to cite
A citação é gerada automaticamente e pode não estar totalmente de acordo com as normas
