A Cross-Dataset Empirical Evaluation of Adversarial Evasion Attacks and Defenses in Machine Learning-Based Intrusion Detection Systems
DOI:
https://doi.org/10.66279/3k5mqs50Keywords:
Intrusion Detection Systems (IDS), Adversarial Machine Learning, Network SecurityAbstract
The study aims to assess the adversarial robustness of two intrusion detection systems (IDS), namely XGBoost and Multilayer Perceptron (MLP), using three datasets: NSL-KDD, CIC-IDS2017, and UNSW-NB15. The attacks were carried out using two methods: FGSM and PGD, and transfer-based attacks for non-differentiable models. The results indicate that adversarial attacks significantly affect intrusion detection systems' performance, with attacks reducing the MLP IDS detection rate to 33.19% on the CIC-IDS2017 dataset, to 4.43% with FGSM attacks, and to 0.00% with transfer-based PGD attacks, on the XGBoost IDS. The study also indicates that adversarial training improves the robustness of intrusion detection systems' performance, with the MLP IDS maintaining a 96%+ detection rate even after undergoing adversarial attacks on the CIC-IDS2017 dataset and 68% on the UNSW-NB15 dataset.
Downloads
References
[1] Y. Al-Nashif, A. A. Kumar, S. Hariri, G. Qu, Y. Luo, and F. Szidarovsky, “Multi-level intrusion detection system (ML-IDS),” in Proc. 5th Int. Conf. Autonomic Computing (ICAC), 2008, pp. 131–140, doi: 10.1109/ICAC.2008.25. DOI: https://doi.org/10.1109/ICAC.2008.25
[2] T. Saranya, S. Sridevi, C. Deisy, T. D. Chung, and M. K. A. A. Khan, “Performance analysis of machine learning algorithms in intrusion detection system: A review,” Procedia Comput. Sci., vol. 171, pp. 1251–1260, 2020, doi: 10.1016/j.procs.2020.04.133. DOI: https://doi.org/10.1016/j.procs.2020.04.133
[3] Z. Ahmad, A. S. Khan, C. W. Shiang, J. Abdullah, and F. Ahmad, “Network intrusion detection system: A systematic study of machine learning and deep learning approaches,” Trans. Emerg. Telecommun. Technol., vol. 32, no. 1, 2021, doi: 10.1002/ett.4150. DOI: https://doi.org/10.1002/ett.4150
[4] O. H. Abdulganiyu, T. A. Tchakoucht, and Y. K. Saheed, “A systematic literature review for network intrusion detection system (IDS),” Int. J. Inf. Secur., vol. 22, no. 5, pp. 1125–1162, 2023, doi: 10.1007/s10207-023-00682-2. DOI: https://doi.org/10.1007/s10207-023-00682-2
[5] L. Yang and A. Shami, “IDS-ML: An open-source code for intrusion detection system development using machine learning,” Software Impacts, vol. 14, 2022, doi: 10.1016/j.simpa.2022.100446. DOI: https://doi.org/10.1016/j.simpa.2022.100446
[6] A. H. Azizan et al., “A machine learning approach for improving the performance of network intrusion detection systems,” Ann. Emerg. Technol. Comput., vol. 5, no. 5, pp. 201–208, 2021, doi: 10.33166/AETiC.2021.05.025. DOI: https://doi.org/10.33166/AETiC.2021.05.025
[7] G. Kocher and G. Kumar, “Machine learning and deep learning methods for intrusion detection systems: Recent developments and challenges,” Soft Comput., vol. 25, no. 15, pp. 9731–9763, 2021, doi: 10.1007/s00500-021-05893-0. DOI: https://doi.org/10.1007/s00500-021-05893-0
[8] 2020 54th Annual Conference on Information Sciences and Systems (CISS),
Princeton, NJ, USA, Mar. 2020. IEEE.
[9] V. Heydari and K. Nyarko, “Enhancing adversarial robustness in network intrusion detection: A novel adversarially trained neural network approach,” Electronics, vol. 14, no. 16, 2025, doi: 10.3390/electronics14163249. DOI: https://doi.org/10.3390/electronics14163249
[10] A. Piplai, S. S. L. Chukkapalli, and A. Joshi, “NAttack! Adversarial attacks to bypass a GAN-based classifier trained to detect network intrusion,” arXiv preprint arXiv:2002.08527, 2020. DOI: https://doi.org/10.1109/BigDataSecurity-HPSC-IDS49724.2020.00020
[11] E. Alshahrani, D. Alghazzawi, R. Alotaibi, and O. Rabie, “Adversarial attacks against supervised machine learning based network intrusion detection systems,” PLoS One, vol. 17, no. 10, 2022, doi: 10.1371/journal.pone.0275971. DOI: https://doi.org/10.1371/journal.pone.0275971
[12] S. Sharma and Z. Chen, “A systematic study of adversarial attacks against network intrusion detection systems,” Electronics, vol. 13, no. 24, 2024, doi: 10.3390/electronics13245030. DOI: https://doi.org/10.3390/electronics13245030
[13] Z. Awad, M. Zakaria, and R. Hassan, “An enhanced ensemble defense framework for boosting adversarial robustness of intrusion detection systems,” Sci. Rep., vol. 15, 2025, doi: 10.1038/s41598-025-94023-z. DOI: https://doi.org/10.1038/s41598-025-94023-z
[14] X. Zhang, X. Zheng, and D. D. Wu, “Attacking DNN-based intrusion detection models,” IFAC-PapersOnLine, vol. 53, pp. 415–419, 2020, doi: 10.1016/j.ifacol.2021.04.118. DOI: https://doi.org/10.1016/j.ifacol.2021.04.118
[15] A. Alotaibi and M. A. Rassam, “Adversarial machine learning attacks against intrusion detection systems: A survey on strategies and defense,” Future Internet, vol. 15, no. 2, 2023, doi: 10.3390/fi15020062. DOI: https://doi.org/10.3390/fi15020062
[16] R. Chauhan and S. S. Heydari, “Polymorphic adversarial DDoS attack on IDS using GAN,” in Proc. ISNCC, 2020, doi: 10.1109/ISNCC49221.2020.9297264. DOI: https://doi.org/10.1109/ISNCC49221.2020.9297264
[17] S. Zhao et al., “AttackGAN: Adversarial attack against black-box IDS using generative adversarial networks,” Procedia Comput. Sci., 2021, doi: 10.1016/j.procs.2021.04.118. DOI: https://doi.org/10.1016/j.procs.2021.04.118
[18] H. Liu and B. Lang, “Machine learning and deep learning methods for intrusion detection systems: A survey,” Appl. Sci., vol. 9, no. 20, 2019, doi: 10.3390/app9204396. DOI: https://doi.org/10.3390/app9204396
[19] E. Alhajjar, P. Maxwell, and N. Bastian, “Adversarial machine learning in network intrusion detection systems,” Expert Syst. Appl., vol. 186, 2021, doi: 10.1016/j.eswa.2021.115782. DOI: https://doi.org/10.1016/j.eswa.2021.115782
[20] E. Anthi et al., “Adversarial attacks on machine learning cybersecurity defences in industrial control systems,” J. Inf. Secur. Appl., vol. 58, 2021, doi: 10.1016/j.jisa.2020.102717. DOI: https://doi.org/10.1016/j.jisa.2020.102717
[21] Z. Lin and X. Shi, “IDSGAN: Generative adversarial networks for attack generation against intrusion detection,” in Adv. Knowl. Discov. Data Mining, Springer, 2022, pp. 79–91. DOI: https://doi.org/10.1007/978-3-031-05981-0_7
[22] A. Oprea, A. Singhal, and A. Vassilev, “Poisoning attacks against machine learning: Can machine learning be trustworthy?,” Computer, vol. 55, no. 11, pp. 94–99, 2022, doi: 10.1109/MC.2022.3190787. DOI: https://doi.org/10.1109/MC.2022.3190787
[23] J. H. Metzen et al., “On detecting adversarial perturbations,” arXiv preprint arXiv:1702.04267, 2017.
[24] Y. Song et al., “PixelDefend: Leveraging generative models to understand and defend against adversarial examples,” arXiv preprint arXiv:1710.10766, 2018.
[25] Y. Peng et al., “Detecting adversarial examples for network intrusion detection system with GAN,” in Proc. ICSESS, 2020, doi: 10.1109/ICSESS49938.2020.9237728. DOI: https://doi.org/10.1109/ICSESS49938.2020.9237728
[26] J. Wang et al., “Def-IDS: An ensemble defense mechanism against adversarial attacks for deep learning-based network intrusion detection,” in Proc. ICCCN, 2021, doi: 10.1109/ICCCN52240.2021.9522215. DOI: https://doi.org/10.1109/ICCCN52240.2021.9522215
[27] P. T. Duy et al., “DIGFuPAS: Deceive IDS with GAN and function-preserving adversarial samples in SDN-enabled networks,” Comput. Secur., vol. 109, 2021, doi: 10.1016/j.cose.2021.102367. DOI: https://doi.org/10.1016/j.cose.2021.102367
[28] R. R. Devi and M. Abualkibash, “Intrusion detection system classification using different machine learning algorithms on KDD-99 and NSL-KDD datasets: A review,” Int. J. Comput. Sci. Inf. Technol., vol. 11, no. 3, pp. 65–80, 2019, doi: 10.5121/ijcsit.2019.11306. DOI: https://doi.org/10.5121/ijcsit.2019.11306
[29] A. Yulianto, P. Sukarno, and N. A. Suwastika, “Improving AdaBoost-based intrusion detection system performance on CIC-IDS2017 dataset,” J. Phys.: Conf. Ser., 2019, doi: 10.1088/1742-6596/1192/1/012018. DOI: https://doi.org/10.1088/1742-6596/1192/1/012018
[30] N. Moustafa and J. Slay, “UNSW-NB15: A comprehensive data set for network intrusion detection systems,” in Proc. MilCIS, 2015. DOI: https://doi.org/10.1109/MilCIS.2015.7348942
[31] J. Han, M. Kamber, and J. Pei, Data Mining: Concepts and Techniques, 3rd ed.
Waltham, MA, USA: Morgan Kaufmann, 2012.
[32] A. Kurakin, I. Goodfellow, and S. Bengio, “Adversarial machine learning at scale,” arXiv preprint arXiv:1611.01236, 2017.
[33] A. Madry et al., “Towards deep learning models resistant to adversarial attacks,” arXiv preprint arXiv:1706.06083, 2019.
[34] M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Inf. Process. Manag., vol. 45, no. 4, pp. 427–437, 2009, doi: 10.1016/j.ipm.2009.03.002. DOI: https://doi.org/10.1016/j.ipm.2009.03.002
[35] A. A. Salih and A. M. Abdulazeez, “Evaluation of classification algorithms for intrusion detection system: A review,” J. Soft Comput. Data Min., vol. 2, no. 1, pp. 31–40, 2021.
[36] NSL‑KDD Dataset, Canadian Institute for Cybersecurity, University of New Brunswick, 2009. [Online]. Available: https://www.unb.ca/cic/datasets/nsl.html. [Accessed: 01‑JAN‑2026].
[37] Yasser. Fouad, Asmaa N. Ghareeb and Enas Selem, “Evolution of Routing Protocols in WSN: Challenges, Advances, and Drone-Assisted Innovations”, Computational Discovery and Intelligent Systems(CDIS),vol. 2, no. 2 , pp. 22-41 , 2026 .
Downloads
Published
Data Availability Statement
The datasets used in this study (NSL-KDD, CIC-IDS2017, and UNSW-NB15) are publicly available and can be accessed from their official repositories.
Issue
Section
Categories
License
Copyright (c) 2026 Computational Discovery and Intelligent Systems (CDIS)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Computational Discovery and Intelligent Systems (CDIS) content is published under a Creative Commons Attribution License (CCBY). This means that content is freely available to all readers upon publication, and content is published as soon as production is complete.
Computational Discovery and Intelligent Systems (CDIS) seeks to publish the most influential papers that will significantly advance scientific understanding. Selected articles must present new and widely significant data, syntheses, or concepts. They should merit recognition by the wider scientific community and the general public through publication in a reputable scientific journal.
