A very short introduction to MLOPs for TinyML — Part 1
MLOps Overview
While many organizations recognize that Machine Learning (ML) can drive significant value, successful deployments and effective operations are the main bottleneck for gaining value from AI. As of today, more than 87% of data science projects never make it into production. To support organisations to come up to speed faster in this important domain, it is important to understand ML Operations (MLOps).
A Brief Introduction to MLOps
Teams across all industries and domains find themselves spending an unnecessary amount of time on the development of ML models, without the necessary people, processes, or technology to address these challenges. In the current landscape that we see we find a high degree of manual, one-off work — even in steps in the ML workflow that are known to be iterative (e.g. model training), inflexibility as components are not reusable nor reproducible, and error prone as poorly documented handoffs between Data Science and IT, therefore, falling short of their goals while costing their organizations time and money.
In summary, deploying ML models at scale is very challenging, especially due to:
- Organization Barriers — Managing different ecosystems like programming languages
- Compute Constraints — Continuous and reliable compute resources (dedicated servers or cloud-only option)
- Portability Issues — Huge dependencies on legacy systems
- Seasonality — ML workloads works in patches; this need auto scaling capabilities
Businesses are looking to reduce time from change (via data or code) to deployment, improve deployment frequency, speed up time to value with new ML use cases, lower failure rate of new releases, shorten lead time between fixes, and facilitate better collaboration (reusing features and sharing models).
To address this, organizations need to build the necessary ML engineering culture and capability MLOps aims to unify ML system development (ML) with ML system operations (Ops).
MLOps strongly advocates automation and monitoring at all steps of ML system construction, from integration, testing, and releasing to deployment and infrastructure management. It takes both its name as well as some of the core principles and tooling from DevOps. This makes sense as the goals of MLOps and DevOps are practically the same: to shorten the systems development life cycle and ensure high quality software is continuously developed, delivered, and maintained in production. Its Machine Learning’s unique challenges and needs — managing the lifecycle of Data, Models, and Code — has led MLOps to quickly evolve as a domain of its own. To support the engineering community to realise MLOps in practice, Google made available to the world the practitioners guide to MLOps to help developers implement MLOps and smart practices during the ML workflow.
MLOPs processes and operations
Amongst many processes and operations, MLOps requires:
- Continuous Integration: Is no longer only about testing and validating code and components, but also testing and validating data, data schemas, and models.
- Continuous Delivery: Is no longer about a single software package or a service, but a system (ML training pipeline) that should automatically deploy another service (model prediction service)
- Continuous Training: Is a new property, specific to ML systems, concerning automatically retraining and serving the models.
The maturity of the ML process is defined by the level of integration and automation of these phases shown in the figure below, which reflects the velocity of training new models, quality of the assets generated and reliability of the oversystem.
