A Multi-Stage AI Framework for Intelligent Incident Detection and Response Automation in Security Operations Centers
DOI:
https://doi.org/10.66279/9zqrqk18Keywords:
Security Operations Center, Automated Incident Response, Anomaly Detection, Machine Learning Security AnalyticsAbstract
Security Operations Centers (SOCs) are essential for safeguarding company infrastructure against increasingly sophisticated cyber threats. The expansion of cloud services, Internet of Things (IoT) devices, and extensive dispersed systems has resulted in a significant increase in security event quantities, burdening analysts and revealing the inadequacies of conventional, manual workflows. Rule-based Security Information and Event Management (SIEM) systems, in particular, experience elevated false-positive rates, leading to alert fatigue and protracted incident response times. This paper introduces IRAS (Incident Response Automation and Management System), an AI-driven platform that automates key stages of the incident response lifecycle within enterprise SOC environments. IRAS integrates a two-stage machine learning pipeline combining Isolation Forest for anomaly detection and Random Forest for multi-class attack classification, coupled with Wazuh for centralized log management and deployed within a Dockerized microservice architecture on Ubuntu Linux. Upon detecting malicious activity, IRAS executes predefined mitigation playbooks including IP blocking, endpoint isolation, and account suspension within seconds of detection. IRAS was tested in a controlled experimental setup using an Ubuntu victim machine and a Kali Linux attack machine on the CIC-DDoS2019 and SSH Brute-Force datasets and got a weighted F1-score of 94.5% and, a false-positive rate of 2.8%, and a mean automated response time of 2.46 seconds, representing a substantial reduction compared to manual SOC baselines of 8–15 minutes. The system achieved a peak throughput of 1,200 incidents per hour and reduced analyst workload by 74%, demonstrating significant improvements in SOC efficiency within the evaluated threat scenarios and providing a foundation for semi-autonomous SOC operations. Generalization to additional attack categories such as ransomware, insider threats, and APTs remains a direction for future validation.
Downloads
References
[1] M. Anisetti, C. Ardagna, M. Cremonini, E. Damiani, J. Sessa, and L. Costa, "Security threat landscape," White Paper Security Threats, 2020.
[2] A. Mohammed, "AI in Cybersecurity: Enhancing Audits and Compliance Automation," Available at SSRN, vol. 5066097, 2021.
[3] N. B. Kilaru, S. K. M. Cheemakurthi, and V. Gunnam, "SOAR Solutions in PCI Compliance: Orchestrating Incident Response for Regulatory Security," ESP Journal of Engineering & Technology Advancements, vol. 1, no. 2, pp. 78-84, 2021.
[4] B. A. Alahmadi, L. Axon, and I. Martinovic, "99% false positives: A qualitative study of {SOC} analysts' perspectives on security alarms," in 31st USENIX Security Symposium (USENIX Security 22), 2022, pp. 2783-2800.
[5] L. N. Kaliyaperumal, "The evolution of security operations and strategies for building an effective SOC," ISACA Journal, vol. 5, no. 1, 2021.
[6] A. L. Buczak and E. Guven, "A survey of data mining and machine learning methods for cyber security intrusion detection," IEEE Communications surveys & tutorials, vol. 18, no. 2, pp. 1153-1176, 2015. DOI: https://doi.org/10.1109/COMST.2015.2494502
[7] J. Ruan et al., "Deep learning for cybersecurity in smart grids: Review and perspectives," Energy Conversion and Economics, vol. 4, no. 4, pp. 233-251, 2023. DOI: https://doi.org/10.1049/enc2.12091
[8] R. O. Andrade and S. G. Yoo, "Cognitive security: A comprehensive study of cognitive science in cybersecurity," Journal of Information Security and Applications, vol. 48, p. 102352, 2019. DOI: https://doi.org/10.1016/j.jisa.2019.06.008
[9] A. Mohammed, "The Paradox of AI in Cybersecurity: Protector and Potential Exploiter," Baltic Journal of Engineering and Technology, vol. 2, no. 1, pp. 70-76, 2023.
[10] A. O. Aljahdali and R. Alsulami, "Streamlining threat response and automating critical use cases with security orchestration, automation and response (SOAR)," Journal of Digital Security and Forensics, vol. 2, no. 1, pp. 36–57-36–57, 2025. DOI: https://doi.org/10.29121/digisecforensics.v2.i1.2025.45
[11] H. Xu et al., "Large language models for cyber security: A systematic literature review," ACM Transactions on Software Engineering and Methodology, 2024. DOI: https://doi.org/10.1145/3769676
[12] J. Kinyua and L. Awuah, "AI/ML in Security Orchestration, Automation and Response: Future Research Directions," Intelligent Automation & Soft Computing, vol. 28, no. 2, 2021. DOI: https://doi.org/10.32604/iasc.2021.016240
[13] D. Stiawan, M. Y. Idris, R. F. Malik, S. Nurmaini, N. Alsharif, and R. Budiarto, "Investigating brute force attack patterns in IoT network," Journal of Electrical and Computer Engineering, vol. 2019, no. 1, p. 4568368, 2019. DOI: https://doi.org/10.1155/2019/4568368
[14] D. Chou and M. Jiang, "A survey on data-driven network intrusion detection," ACM Computing Surveys (CSUR), vol. 54, no. 9, pp. 1-36, 2021. DOI: https://doi.org/10.1145/3472753
[15] F. B. Kokulu et al., "Matched and mismatched SOCs: A qualitative study on security operations center issues," in Proceedings of the 2019 ACM SIGSAC conference on computer and communications security, 2019, pp. 1955-1970. DOI: https://doi.org/10.1145/3319535.3354239
[16] F. S. d. Lima Filho, F. A. Silveira, A. de Medeiros Brito Junior, G. Vargas-Solar, and L. F. Silveira, "Smart detection: an online approach for DoS/DDoS attack detection using machine learning," Security and Communication Networks, vol. 2019, no. 1, p. 1574749, 2019. DOI: https://doi.org/10.1155/2019/1574749
[17] M. A. Ferrag, L. Maglaras, S. Moschoyiannis, and H. Janicke, "Deep learning for cyber security intrusion detection: Approaches, datasets, and comparative study," Journal of Information Security and Applications, vol. 50, p. 102419, 2020. DOI: https://doi.org/10.1016/j.jisa.2019.102419
[18] M. Vielberth, F. Böhm, I. Fichtinger, and G. Pernul, "Security operations center: A systematic study and open challenges," Ieee Access, vol. 8, pp. 227756-227779, 2020. DOI: https://doi.org/10.1109/ACCESS.2020.3045514
[19] G. Pang, A. van den Hengel, C. Shen, and L. Cao, "Deep reinforcement learning for unknown anomaly detection," arXiv preprint arXiv:2009.06847, 2020.
[20] S. Akhtar and T. Javaid, "Machine Learning in SOC Operations: Transforming Incident Detection and Response," 2021.
[21] M. Landauer, F. Skopik, M. Frank, W. Hotwagner, M. Wurzenberger, and A. Rauber, "Maintainable log datasets for evaluation of intrusion detection systems," IEEE Transactions on Dependable and Secure Computing, vol. 20, no. 4, pp. 3466-3482, 2022. DOI: https://doi.org/10.1109/TDSC.2022.3201582
[22] T. T. Bukhari, O. Oladimeji, E. D. Etim, and J. O. Ajayi, "Systematic review of SIEM integration for threat detection and log correlation in AWS-based infrastructure," Shodhshauryam, International Scientific Refereed Research Journal, vol. 6, no. 5, pp. 479-512, 2023.
[23] S. Zhang, Y. Wang, and X. Su, "Threat detection framework based on industrial internet of things logs," IEEE Access, vol. 12, pp. 195642-195657, 2024. DOI: https://doi.org/10.1109/ACCESS.2024.3514097
[24] J. Dorobisz, "Analysis of trends and risks in the field of network security based on statistical data," GIS Odyssey Journal, vol. 4, no. 2, pp. 147-163, 2024.
[25] L. Havi, "Security operations centers in information technology and operational technology environments: A literature review of requirements, differences, issues, and improvements of IT and OT SOC environments," 2025.
[26] D. R. Ankireddy, S. Paria, A. Dasgupta, S. Ray, and S. Bhunia, "Lasa: Enhancing soc security verification with llm-aided property generation," arXiv preprint arXiv:2506.17865, 2025.
[27] J. Manzoor, A. Waleed, A. F. Jamali, and A. Masood, "Cybersecurity on a budget: Evaluating security and performance of open-source SIEM solutions for SMEs," Plos one, vol. 19, no. 3, p. e0301183, 2024. DOI: https://doi.org/10.1371/journal.pone.0301183
[28] R. Amami, M. Charfeddine, and S. Masmoudi, "Exploration of Open Source SIEM Tools and Deployment of an Appropriate Wazuh-Based Solution for Strengthening Cyberdefense," in 2024 10th International Conference on Control, Decision and Information Technologies (CoDIT), 2024, pp. 1-7: IEEE. DOI: https://doi.org/10.1109/CoDIT62066.2024.10708476
[29] M. R. Islam and R. Rafique, "Wazuh SIEM for Cyber Security and Threat Mitigation in Apparel Industries," International Journal of Engineering Materials and Manufacture, vol. 9, no. 4, pp. 136-144, 2024. DOI: https://doi.org/10.26776/ijemm.09.04.2024.02
[30] S. Stanković, S. Gajin, and R. Petrović, "A review of Wazuh tool capabilities for detecting attacks based on log analysis," No Nama Agent Integrity File Added Delete Modified, vol. 1, 2022.
[31] S. M. Tarek, Muthmainnah; Obaid, Ahmed J., "A Cross-Dataset Empirical Evaluation of Adversarial Evasion Attacks and Defenses in Machine Learning-Based Intrusion Detection Systems," Computational Discovery and Intelligent Systems (CDIS), vol. 3, no. 1, pp. 57-70, 2026. DOI: https://doi.org/10.66279/3k5mqs50
[32] F. T. Liu, K. M. Ting, and Z.-H. Zhou, "Isolation forest," in 2008 eighth ieee international conference on data mining, 2008, pp. 413-422: IEEE. DOI: https://doi.org/10.1109/ICDM.2008.17
[33] I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, "Toward generating a new intrusion detection dataset and intrusion traffic characterization," ICISSp, vol. 1, no. 2018, pp. 108-116, 2018.
[34] S. Vethachalam, “Cybersecurity automation: Enhancing incident response and threat mitigation,” World Journal of Advanced Engineering Technology and Sciences, vol. 15, no. 3, pp. 572–585, 2025, doi: 10.30574/wjaets.2025.15.3.0972. DOI: https://doi.org/10.30574/wjaets.2025.15.3.0972
[35] D. Ferraiolo, D. Kuhn, and R. Chandramouli, "Role-based access con-trol," Pr ocee dingsof15th NIST-NCSC National Computer Security Conferenc e, pp. 554-563, 1992.
[36] I. Sharafaldin, A. H. Lashkari, S. Hakak, and A. A. Ghorbani, "Developing realistic distributed denial of service (DDoS) attack dataset and taxonomy," in 2019 international carnahan conference on security technology (ICCST), 2019, pp. 1-8: IEEE. DOI: https://doi.org/10.1109/CCST.2019.8888419
[37] I. Sharafaldin, A. H. Lashkari, and A. A. Ghorbani, "Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization," presented at the 4th International Conference on Information Systems Security and Privacy (ICISSP), 2018. DOI: https://doi.org/10.5220/0006639801080116
[38] J. Luxemburk, K. Hynek, and T. Cejka, "TTPS Brute-force Dataset with Extended Network Flows," ed. Zenodo, 2020.
[39] Mahmoud, H. A., Khalaf, O. I., and Farid, O., “Certainty-Aware Skin Lesion Segmentation with Post-Hoc Reliability Estimation for the Segment Anything Model,” Journal of Smart Algorithms and Applications (JSAA), vol. 3, no. 2, pp. 71–86, Apr. 2026, doi: 10.66279/hzkw5y24. DOI: https://doi.org/10.66279/hzkw5y24
[40] L. Breiman, "Random forests," Machine learning, vol. 45, no. 1, pp. 5-32, 2001. DOI: https://doi.org/10.1023/A:1010933404324
[41] P. Probst, M. N. Wright, and A. L. Boulesteix, "Hyperparameters and tuning strategies for random forest," Wiley Interdisciplinary Reviews: data mining and knowledge discovery, vol. 9, no. 3, p. e1301, 2019. DOI: https://doi.org/10.1002/widm.1301
[42] J. Bergstra and Y. Bengio, "Random search for hyper-parameter optimization," Journal of machine learning research, vol. 13, no. 2, 2012.
[43] M. Akbari Gurabi et al., "Requirements for playbook-assisted cyber incident response, reporting and automation," Digital Threats: Research and Practice, vol. 5, no. 3, pp. 1-11, 2024. DOI: https://doi.org/10.1145/3688810
[44] N. Felix and W. Claudia, "Automated Security Operations: Scaling Threat Response with SOAR and AI-Driven Playbooks," International Journal of Trend in Scientific Research and Development, vol. 5, no. 2, pp. 1317-1323, 2021.
[45] N. Perry, “IP Networks Over Heterogeneous Embedded Serial Links,” 2025. Accessed: Jun. 08, 2026. [Online]. Available: https://hdl.handle.net/1721.1/164271.
[46] M. M. Hasan, “FEDERATED LEARNING MODELS FOR PRIVACY-PRESERVING AI IN ENTERPRISE DECISION SYSTEMS,” International Journal of Business and Economics Insights, vol. 5, no. 3, pp. 238–269, Sep. 2025, doi: 10.63125/RY033286. DOI: https://doi.org/10.63125/ry033286
[47] Saad, A., Ahmed, A. M., Shaban, M., et al., “Explainable machine learning framework for predicting cobalt ion removal by natural hematite,” Scientific Reports, vol. 15, Art. no. 35401, 2025, doi: 10.1038/s41598-025-18981-0. DOI: https://doi.org/10.1038/s41598-025-18981-0
[48] D. El Khaled, R. AlOtaibi, N. Novas, and J. A. Gazquez, “NetworkGuard: An Edge-Based Virtual Network Sensing Architecture for Real-Time Security Monitoring in Smart Home Environments,” Sensors, vol. 26, no. 7, Art. no. 2231, 2026, doi: 10.3390/s26072231. DOI: https://doi.org/10.3390/s26072231
Downloads
Published
Data Availability Statement
IDS 2017 | Datasets | Research | Canadian Institute for Cybersecurity | UNB
Issue
Section
Categories
License
Copyright (c) 2026 Journal of Smart Algorithms and Applications (JSAA)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Journal of Smart Algorithms and Applications (JSAA) content is published under a Creative Commons Attribution License (CCBY). This means that content is freely available to all readers upon publication, and content is published as soon as production is complete.
Journal of Smart Algorithms and Applications (JSAA) seeks to publish the most influential papers that will significantly advance scientific understanding. Selected articles must present new and widely significant data, syntheses, or concepts. They should merit recognition by the wider scientific community and the general public through publication in a reputable scientific journal.
