APPLICATION OF THE KUBEFLOW TOOL FOR THE INTEGRATION OF MACHINE LEARNING AND ARTIFICIAL INTELLIGENCE IN UNMANNED AERIAL VEHICLE
DOI: 10.31673/2412-4338.2023.036679
Abstract
At the current stage of information technology development, machine learning (ML) and artificial intelligence (AI) are becoming one of the main tools for solving complex applied problems in various fields of activity. Various processes and technologies are used for develop, test, and maintain the infrastructure of the data system. The application of tools for the integration of ML and AI in the management of unmanned aerial vehicles (UAVs) is especially relevant today.
An overview of the ML concept and processes (Machine Learning and Operation, MLOps) was made, which is a set of techniques for implementation and automatic continuous integration, as well as delivery to the product environment and model learning. The concept of MLOps is considered in terms of Kubeflow tools, which work on the Kubernetes platform. The possibilities of using modern MLOps solutions to improve the development processes of ML information systems have investigated. An AI-based information system with the possibility of continuous learning has designed. The concept of using the MLOps pipeline to solve the applied problem of classifying objects from the video of reconnaissance UAVs was presented.
The results of the operation of the model in the Kubeflow arsenal have been checked using such improvement factors as: speed of development, implementation of changes, reduction of time to search for problems, recovery after global interruptions, reduction of the number of errors in the model. A publicly available model was deployed in a Kubeflow cluster using the Seldon Core Serving application manifest for practical analysis.
The conducted research showed that Kubeflow consists of a set of various open source components that have a high level of integration with each other through the Kubernetes platform. At the same time, Kubeflow uses the Kubernetes pattern of operators for ML objects extremely efficiently. It has shown that writing model code is a small part of ML tasks, which affects the need for automation. The concept of a full-fledged information solution based on the continuous integration pipeline, which is the foundation of the implementation of the concept of continuous learning, has formed. Representing abstractions in the form of separate platform resources allows you to reduce the entry threshold for the end user.
Keywords: Kubeflow, MLOps, machine learning, artificial intelligence, continuous learning, unmanned aerial vehicle.
References:
1. Muratahan Aykol, Patrick Herring, and Abraham Anapolsky. Machine learning for continuous innovation in battery technologies. Nat. Rev. Mater. 2020, 5, 10, Р. 725–727.
2. Mahendra Kumar Gourisaria, Rakshit Agrawal, G M Harshvardhan, Manjusha Pandey, and Siddharth Swarup Rautaray. Application of Machine Learning in Industry 4.0. Machine Learning: Theoretical Foundations and Practical Applications. Springer. 2021, Р. 57–87.
3. Ana De Las Heras, Amalia Luque-Sendra, and Francisco Zamora-Polo. Machine learning technologies for sustainability in smart cities in the post-covid era. Sustainability. 2020, 12, 22, Р. 9320.
4. Google Cloud. MLOps: Continuous Delivery and Automation Pipelines in Machine Learning. URL: https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning.
5. Dominik Kreuzberger, Niklas Kühl, Sebastian Hirschl. Machine Learning Operations (MLOps): Overview, Definition, and Architecture. Computer Science: Machine Learning. 2023, Vol. 11, P. 31866.
6. Hidden Technical Debt in Machine Learning Systems. Part of Advances in Neural Information Processing Systems 28. Authors: D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-François Crespo, Dan Dennison. NIPS. 2015, 1486, 9 p.
7. Justin J. Boutilier, Timothy C. Y. Chan. Introducing and Integrating Machine Learning in an Operations Research Curriculum: An Application-Driven Course. INFORMS Transactions on Education. 2023, Vol. 23, 2, P. 64-83.
8. Fiebrink R. Machine Learning Education for Artists, Musicians, and Other Creative Practitioners. ACM Transactions on Computing Education. 2019, Vol. 19, 4, Art. No. 31, P. 1–32.
9. Sulmont E., Patitsas E., Cooperstock J. R. What Is Hard about Teaching Machine Learning to Non-Majors? Insights from Classifying Instructors’ Learning Goals. ACM Transactions on Computing Education, 2019, Vol. 19, 4, Art. No. 33, P. 1–16.
10. Sánchez‐Peña M, Vieira C and Magana A. Data science knowledge integration: Affordances of a computational cognitive apprenticeship on student conceptual understanding. Computer Applications in Engineering Education. 2022, 31, P. 239-259.
11. Elmachtoub A. N., Grigas P. Smart “Predict, then Optimize.” Management Science. 2022, Vol. 68, 1, P. 9–26.
12. Mišić V. V., Perakis G. Data analytics in operations management: A review. Manufacturing Service Operations Management. INFORMS. 2020, 22 (1), P. 158–169.
13. Hazzan O., Mike K. Core Concepts of Machine Learning. Guide to Teaching Data Science. 2023, P. 209-22.
14. Ruibo Chen, Yanjun Pu, Bowen Shi, Wenjun Wu. An automatic model management system and its implementation for AIOps on microservice platforms. The Journal of Supercomputing. 2023, 79, P. 11410–11426.
15. Haoyu Cai, Chao Wang, Xuehai Zhou. Deployment and verification of machine learning tool-chain based on kubernetes distributed clusters. CCF Transactions on High Performance Computing. 2021, 3, P. 157–170.
16. Cong Yang, Wenfeng Wang, Yunhui Zhang, Zhikai Zhang, Lina Shen… MLife: a lite framework for machine learning lifecycle initialization. Machine Learning. 2021, 110, P. 2993–3013.
17. Asharul Islam Khan, Yaseen Al-Mulla. Unmanned Aerial Vehicle in the Machine Learning Environment. Procedia Computer Science. 2019, Vol. 160, P. 46-53.
18. Monfort Samuel S., Ciara M. Sibley, Joseph T. Coyne. Using machine learning and real-time workload assessment in a high-fidelity UAV simulation environment. Proceedings SPIE: Next-Generation Analyst IV. 2016, Vol. 9851, 98510B.
19. Kubeflow. Kubeflow Architecture. URL: https://www.kubeflow.org/docs/started/architecture/.
20. Bondarchuk A., Dibrivniy O., Grebenyk V., Onyshchenko V. Motion Vector Search Algorithm for Motion Compensation in Video Encoding / 2021 IEEE 8th International Conference on Problems of Infocommunications, Science and Technology, PIC S and T 2021 - Proceedings, 2021, p. 345–348.
