MLOps with MLflow on Kraken CI

MLOps and MLflow

MLOps is a set of practices that aims to build and maintain machine
learning models in production reliably and efficiently. One of
prominent tools in this area is MLflow.

  • Tracking experiments to record and compare parameters and results
    (MLflow Tracking).
  • Packaging ML code in a reusable, reproducible form to share
    with other data scientists or transfer to production (MLflow
    Projects).
  • Managing and deploying models from various ML libraries to a
    variety of model serving and inference platforms (MLflow Models).
  • Providing a central model store to collaboratively manage the entire
    lifecycle of an MLflow Model, including model versioning, stage
    transitions, and annotations (MLflow Model Registry).

MLflow in Kraken CI

In the following sections, I will describe how to prepare a workflow
in Kraken CI to train an ML model. This is an LSTM model that will
predict stock prices based on historical data.

  1. pulling live stock data and preparing it for training (source 1, source 2)
  2. performing the training (source 3)
  3. storing model metrics in Kraken CI for charting

Workflow Definition

The whole Kraken CI workload is defined here.

  1. Checkout mflow example project sources
  2. Run the mlflow project ie. download data, prepare it, run a
    training and at the end store metrics about the trained model to
    metrics.json
  3. Upload collected metrics together with hyperparameters from
    params.json to Kraken server

Execution and Monitoring

Besides the workflow definition, Kraken UI also shows collected data
and the charts drawn from this data:
https://lab.kraken.ci/test_case_results/595950, the charts tab.

Summary

This article shows how Kraken CI can be used to build an MLOps
pipeline. The pipeline downloads raw data, prepares the data for
training and then executes the training. The trained model
metrics are collected and charted in Kraken UI at the end.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store