Designing Embedded Intelligence through Co-Design Substrates Optimization and System Assurance
DOI:
https://doi.org/10.66279/br8wb683Keywords:
Cross-Layer, Hardware and Software Co-design, Embedded Intelligence, Real-time Embedded Systems, Internet of ThingsAbstract
Embedded intelligence places sensing, inference, adaptation, and decision making inside physical systems limited by energy, memory, latency, cost, and reliability. This narrative review examines the field along four connected axes: hardware and software co-design, computational substrates, optimization at the model, mapping, and system levels, and applications in industrial automation, automotive electronics, healthcare monitoring, and the Internet of Things. Timing, safety, and security are treated as requirements that cut across all four axes. The review separates embedded intelligence from constrained on-device machine learning and from edge artificial intelligence, and it frames deployment as a cross-layer problem in which model quality, hardware mapping, timing, energy, security, and adaptation constrain one another. Two literature-based case studies, one on a field-programmable gate array
and one on distributed microcontrollers, show how a gain at one layer produces a cost at another. A recomputation of the efficiency ratios reported in one of the studies shows why measurement boundaries and metric definitions must be declared. Six research priorities are derived from limitations that recur in the reviewed sources: lifecycle energy accounting, assurance of learned components, security of distributed deployments, reproducible benchmarking, portable deployment toolchains, and concept drift after deployment. Each priority is paired with a research direction and evaluation criteria for later studies. The evidence consists mainly of surveys, analytical studies, and laboratory implementations, so quantitative values are reported as stated in the sources, recomputed values are labeled as such, and approaches are not ranked. The proposed formulations and criteria are conceptual.
Downloads
References
[1] G. De Micheli and R. K. Gupta, “Hardware/software co-design,” Proceedings of the IEEE, vol. 85, no. 3, pp. 349–365, 1997.
[2] W. Wolf, “Hardware-software co-design of embedded systems,” Proceedings of the IEEE, vol. 82, no. 7, pp. 967–989, 1994.
[3] Y. Abadade, A. Temouden, H. Bamoumen, N. Benamar, Y. Chtouki, and A. S. Hafid, “A comprehensive survey on tinyml,” IEEE Access, vol. 11, pp. 96892–96922, 2023.
[4] L. Capogrosso, F. Cunico, D. S. Cheng, F. Fummi, and M. Cristani, “A machine learning-oriented survey on tiny machine learning,” IEEE Access, vol. 12, pp. 23406–23426, 2024.
[5] J. Park, S. Samarakoon, M. Bennis, and M. Debbah, “Wireless network intelligence at the edge,” Proceedings of the IEEE, vol. 107, no. 11, pp. 2204–2239, 2019.
[6] S. A. R. Zaidi, A. M. Hayajneh, M. Hafeez, and Q. Z. Ahmed, “Unlocking edge intelligence through tiny machine learning (tinyml),” IEEE Access, vol. 10, pp. 100867–100877, 2022.
[7] S. S. Saha, S. S. Sandha, and M. Srivastava, “Machine learning for microcontroller-class hardware: A review,” IEEE Sensors Journal, vol. 22, no. 22, pp. 21362–21390, 2022.
[8] S. Deng, H. Zhao, W. Fang, J. Yin, S. Dustdar, and A. Y. Zomaya, “Edge intelligence: The confluence of edge computing and artificial intelligence,” IEEE Internet of Things Journal, vol. 7, no. 8, pp. 7457–7469, 2020.
[9] J. Lin, L. Zhu, W.-M. Chen, W.-C. Wang, and S. Han, “Tiny machine learning: Progress and futures [feature],” IEEE Circuits and Systems Magazine, vol. 23, no. 3, pp. 8–34, 2023.
[10] M. T. Lê, P. Wolinski, and J. Arbel, “Efficient neural networks for tiny machine learning: A comprehensive review,” ACM Transactions on Embedded Computing Systems, vol. 17, Apr. 2026.
[11] P. P. Ray, “A review on tinyml: State-of-the-art and prospects,” Journal of King Saud University Computer and Information Sciences, vol. 34, no. 4, pp. 1595–1623, 2022.
[12] D. Kudithipudi, C. Schuman, C. M. Vineyard, T. Pandit, C. Merkel, R. Kubendran, J. B. Aimone, G. Orchard, C. Mayr, R. Benosman, et al., “Neuromorphic computing at scale,” Nature, vol. 637, no. 8047, pp. 801–812, 2025.
[13] J. Gama, I. Žliobait˙e, A. Bifet, M. Pechenizkiy, and A. Bouchachia, “A survey on concept drift adaptation,” ACM Computing Surveys, vol. 46, Mar. 2014.
[14] S. Prakash, M. Stewart, C. R. Banbury, M. Mazumder, P. Warden, B. Plancher, and V. J. Reddi, “Is TinyML sustainable?,” Communications of the ACM, vol. 66, pp. 68–77, Nov. 2023.
[15] T. Sipola, J. Alatalo, T. Kokkonen, and M. Rantonen, “Artificial intelligence in the iot era: A review of edge ai hardware and software,” in 2022 31st Conference of Open Innovations Association (FRUCT), pp. 320–331, 2022.
[16] R. Wilhelm, J. Engblom, A. Ermedahl, N. Holsti, S. Thesing, D. Whalley, G. Bernat, C. Ferdinand, R. Heckmann, T. Mitra, F. Mueller, I. Puaut, P. Puschner, J. Staschulat, and P. Stenström, “The worst-case execution-time problem—overview of methods and survey of tools,” ACM Transactions on Embedded Computing Systems, vol. 7, no. 3, pp. 1–53, 2008.
[17] A. Rodríguez, J. Valverde, J. Portilla, A. Otero, T. Riesgo, and E. De la Torre, “Fpga-based high-performance embedded systems for adaptive edge computing in cyber-physical systems: The artico3 framework,” Sensors, vol. 18, no. 6, 2018.
[18] F. Manca, F. Ratto, and F. Palumbo, “Onnx-to-hardware design flow for adaptive neural-network inference on fpgas,” in Embedded Computer Systems: Architectures, Modeling, and Simulation (L. Carro, F. Regazzoni, and C. Pilato, eds.), (Cham), pp. 85–96, Springer Nature Switzerland, 2025.
[19] S. Prakash, T. Callahan, J. Bushagour, C. Banbury, A. V. Green, P. Warden, T. Ansell, and V. J. Reddi, “Cfu playground: Full-stack open-source framework for tiny machine learning (tinyml) acceleration on fpgas,” in 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pp. 157–167, 2023.
[20] Y.-H. Chen, T. Krishna, J. S. Emer, and V. Sze, “Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks,” IEEE Journal of Solid-State Circuits, vol. 52, no. 1, pp. 127–138, 2017.
[21] D. Neil and S.-C. Liu, “Minitaur, an event-driven fpga-based spiking network accelerator,” IEEE Transactions on Very Large Scale Integration (VLSI) Systems, vol. 22, no. 12, pp. 2621–2628, 2014.
[22] C. Frenkel, J.-D. Legat, and D. Bol, “A 28-nm convolutional neuromorphic processor enabling online learning with spike-based retinas,” in 2020 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5, 2020.
[23] I. Fedorov, R. P. Adams, M. Mattina, and P. N. Whatmough, “SpArSe: Sparse architecture search for CNNs on resource-constrained microcontrollers,” in Advances in Neural Information Processing Systems, vol. 32, pp. 4977–4989, Curran Associates, Inc., 2019.
[24] B. Darvish Rouhani, A. Mirhoseini, and F. Koushanfar, “Tinydl: Just-in-time deep learning solution for constrained embedded systems,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–4, 2017.
[25] T. L. Hayes, K. Kafle, R. Shrestha, M. Acharya, and C. Kanan, “Remind your neural network to prevent catastrophic forgetting,” in Computer Vision – ECCV 2020 (A. Vedaldi, H. Bischof, T. Brox, and J.-M. Frahm, eds.), (Cham), pp. 466–483, Springer International Publishing, 2020.
[26] I. Hegedűs, G. Danner, and M. Jelasity, “Gossip learning as a decentralized alternative to federated learning,” in Distributed Applications and Interoperable Systems (J. Pereira and L. Ricci, eds.), (Cham), pp. 74–90, Springer International Publishing, 2019.
[27] B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam, and D. Kalenichenko, “Quantization and training of neural networks for efficient integer-arithmetic-only inference,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2704–2713, 2018.
[28] A. Banerjee, K. K. Venkatasubramanian, T. Mukherjee, and S. K. S. Gupta, “Ensuring safety, security, and sustainability of mission-critical cyber–physical systems,” Proceedings of the IEEE, vol. 100, no. 1, pp. 283–299, 2012.
[29] I. Lamrani, A. Banerjee, and S. K. S. Gupta, “Operational data-driven feedback for safety evaluation of agent-based cyber–physical systems,” IEEE Transactions on Industrial Informatics, vol. 17, no. 5, pp. 3367–3378, 2021.
[30] K. Sadeghi, A. Banerjee, and S. K. S. Gupta, “A system-driven taxonomy of attacks and defenses in adversarial machine learning,” IEEE Transactions on Emerging Topics in Computational Intelligence, vol. 4, no. 4, pp. 450–467, 2020.
[31] S. Halder, A. Ghosal, and M. Conti, “Secure over-the-air software updates in connected vehicles: A survey,” Computer Networks, vol. 178, p. 107343, 2020.
[32] F. Tramèr, F. Zhang, A. Juels, M. K. Reiter, and T. Ristenpart, “Stealing machine learning models via prediction {APIs},” in 25th USENIX security symposium (USENIX Security 16), pp. 601–618, 2016.
[33] B. Biggio, B. Nelson, and P. Laskov, “Poisoning attacks against support vector machines,” in Proceedings of the 29th International Conference on Machine Learning (ICML), pp. 1467–1474, Omnipress, 2012.
[34] T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “Badnets: Evaluating backdooring attacks on deep neural networks,” IEEE Access, vol. 7, pp. 47230–47244, 2019.
[35] J. Samuel, N. Mathewson, J. Cappos, and R. Dingledine, “Survivable key compromise in software update systems,” in Proceedings of the 17th ACM Conference on Computer and Communications Security, CCS ’10, (New York, NY, USA), pp. 61–72, Association for Computing Machinery, 2010.
[36] I. Shumailov, Y. Zhao, D. Bates, N. Papernot, R. Mullins, and R. Anderson, “Sponge examples: Energy-latency attacks on neural networks,” in 2021 IEEE European Symposium on Security and Privacy (EuroS&P), pp. 212–231, 2021.
[37] B. Xu, A. Banerjee, M. Urooj, and S. K. Gupta, “Accelerated digital twin learning for edge ai: A comparison of fpga and mobile gpu,” in 2025 IEEE 68th International Midwest Symposium on Circuits and Systems (MWSCAS), pp. 149–153, 2025.
[38] Y. Bolat, I. Murray, Y. Ren, and N. Ferdosian, “Decentralized distributed sequential neural networks inference on low-power microcontrollers in wireless sensor networks: A predictive maintenance case study,” Sensors, vol. 25, no. 15, 2025.
[39] A. G. Frank, L. S. Dalenogare, and N. F. Ayala, “Industry 4.0 technologies: Implementation patterns in manufacturing companies,” International Journal of Production Economics, vol. 210, pp. 15–26, 2019.
[40] A. Banerjee and S. K. S. Gupta, “Your mobility can be injurious to your health: Analyzing pervasive health monitoring systems under dynamic context changes,” in 2012 IEEE International Conference on Pervasive Computing and Communications, pp. 39–47, 2012.
[41] A. Banerjee and S. K. Gupta, “Analysis of smart mobile applications for healthcare under dynamic context changes,” IEEE Transactions on Mobile Computing, vol. 14, no. 5, pp. 904–919, 2015.
[42] A. Banerjee, I. Lamrani, and S. K. S. Gupta, “Synthesizing operationally safe controllers for human-in-the-loop human-in-the-plant hybrid close loop systems,” in Pattern Recognition (A. Antonacopoulos, S. Chaudhuri, R. Chellappa, C.-L. Liu, S. Bhattacharya, and U. Pal, eds.), vol. 15329 of Lecture Notes in Computer Science, (Cham), pp. 17–35, Springer Nature Switzerland, 2025.
[43] C. R. Banbury, V. J. Reddi, P. Torelli, J. Holleman, N. Jeffries, C. Király, P. Montino, D. Kanter, S. Ahmed, D. Pau, U. Thakker, A. Torrini, P. Warden, J. Cordaro, G. Di Guglielmo, J. Duarte, S. Gibellini, V. Parekh, H. Tran, N. Tran, W. Niu, and X. Xu, “MLPerf Tiny benchmark,” in Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks (J. Vanschoren and S. Yeung, eds.), vol. 1, pp. 1–19, 2021.
Downloads
Published
License
Copyright (c) 2026 The Authors
This work is licensed under a Creative Commons Attribution 4.0 International License.
Engineering Systems and Intelligent Technologies (ESIT) seeks to publish the most influential papers that will significantly advance scientific understanding. Selected articles must present new and widely significant data, syntheses, or concepts. They should merit recognition by the wider scientific community and the general public through publication in a reputable scientific journal.
Engineering Systems and Intelligent Technologies (ESIT) content is published under a Creative Commons Attribution License (CC BY). This means that content is freely available to all readers upon publication, and content is published as soon as production is complete.
SIRG Publisher
