Training Pipeline

Overview

A training pipeline in Azure Machine Learning is a multi-step workflow that automates and orchestrates the end-to-end model training process — from data preparation to model training and evaluation. Each step runs as a separate job, and the entire pipeline can be tracked as a single experiment using MLflow.

Example

Description

Submit pipelines, deploys on AI Platform

This notebook offers a clear example of a training pipeline, where each stage is modularized, tracked, and reproducible.

The notebook:

• Defines and runs a 3-step pipeline using Azure ML’s @pipeline() decorator:
  • train_component
  • analysis_component
  • inference_component
• Submits the pipeline job: ml_client.jobs.create_or_update(pipeline_job)
• Extracts output from the training step and registers the resulting model

Secondarily, it does the following:

• After training and registration, the model is deployed to a Kubernetes Online Endpoint using:
  • KubernetesOnlineEndpoint
  • KubernetesOnlineDeployment
• Resource configuration, traffic setting, and test inference (via curl) are all included.

Instructions

1. Go to the repository folder with the notebook example and associated files and folders:

   • equinor/ai-platform-aml-usecases/azureml-tutorials/python/online-deployment-3-stage-pipeline-example

2. Copy the train-on-ai-platform-aks.ipynb to your development environment in AI platform.

3. Before running the notebook to deploy the pipeline, ensure your environment is properly set up and the few configuration values (like workspace and compute names) are filled in.

4. Follow the instructions in the notebook and run the code cells.