Overview of MLOps

Tram Ho

Goals of MLOps

The common goal is to build, deploy, optimize and operate an AI system in a production environment

Lifecycle of an ML . product

image.png Consists of 4 main stages:

  • Scoping: Define the goal, scope and problems to be solved in the project
  • Data : Prepare data, extract features, process input for the model
  • Modeling: Building AI models
  • Deployment: Deploy and operate the product on the production environment.

Stages in MLOps

  • Development & Testing (ML algorithms, new ML models)
  • Build Continuous Integration pipeline (Build source code and run tests)
  • Build Continuous Delivery pipeline : Deploy pipeline to production environment
  • Automated Triggering automatically performs actions on the pruduction environment
  • Model Continuous Delivery (Model serving for prediction)
  • Monitoring Collect data on model performance on real data.

Common phenomena

Data Drift

The training data distribution is DIFFERENT than the actual data distribution

The key word to note is this is CHANGE . The world is constantly changing and we must adapt. Likewise, data is in fact always changing and our model must adapt to those changes.

Main life cycle of ML . products

A question raised at the beginning of the article is, What happens after training the AI ​​model? What are we going to do next? . The answer is to put it on production . And a lot of problems we will face from here on. Always keep in mind that we must not only build a good model on Jupyter Notebook, but also put it into practice, putting it in the hands of as many users as possible.

Phone error check example


Use Computer Vision to detect errors on the phone screen such as cracks, localize the cracks on the phone screen. With the prepared datasets of scratched phone screens and bounding boxes respectively we can train a simple object detection model to solve this problem.

Question asked

How to deploy this service to production?


  • Using a prediction server accept API calls, deployable on cloud or edge devices
  • Get the input image as an image
  • Make predictions with the input image


The problem happened…

  • Training models can work well on test sets. However, with actual data, it is NOT sure .
  • In fact sometimes photos may be darker, scratches longer, shooting conditions are also different. This is called the data Drift phenomenon or concept Drift .


And there are many other phenomena that occur during the deployment process that we do not anticipate.

MLOps execution time

MLOps implementation time is often much longer than model development time. Normally, model development time only accounts for about 5-10%.


Case study: Speech Recognition

Step 1: Scoping


  • Problem definition: Speech recognition
    • Input: X is a voice
    • Output: Y is a text string
  • Evaluation metrics definition:
    • accuracy: the accuracy of the model
    • latency: model latency
    • throughput: number of operations per unit time
  • Evaluate, estimate resources to be used and development timeline

Step 2: Prepare the data


  • Data definition: Ask questions to define data
    • Is the data consistently labeled?
      • For example, with an audio clip containing English, there can be two ways to annotate:
        • This is a question about MLOps
        • This is a question about dimming
        • ….
    • What is the average length of the audio?
    • What are the audio acquisition environments like (multiple voices or not, noisy or not…)
    • Are there any requirements for the data collection device?

Step 3: Modeling


We have the following simple formula:

ML MODEL = A + B + C where

  • A is code
  • B is hyperparams
  • C is data

The specific tasks of each team are as follows:

  • Research / Academia’s Team : Data is fixed, Code and Hyperparams change
  • For Product team , Code is fixed, Data and Hyperparams change.

Step 4: Deployment


This is a very important step and also quite long, so we will discuss it in the next article.


We have covered the main components in an ML system as well as the main tasks in MLOps. See you in the next posts.

Share the news now

Source : Viblo