Implement Machine Learning Model Lifecycle and Operations · Topic 3 of 5

Model Registration, Responsible AI and Lifecycle

AI-300 lifecycle: register MLflow models, package a feature retrieval spec, check fairness with the Responsible AI dashboard and archive old versions.

5 min

In a nutshell

MLflow models are self describing, which enables no code deployment.
Model versions are immutable; changes mean a new version.
Put feature_retrieval_spec.yaml in the model folder to avoid training/serving skew.

What it is

The model registry in the workspace stores each model as a named, versioned entry with its files, tags and properties. A good registration carries everything needed to use the model later: its MLflow definition and, if it uses a feature store, the feature retrieval specification. Before a model is promoted, you evaluate it with responsible AI tools. When a model is replaced, you archive the old version rather than lose its history.

How it works

Register an MLflow model

An MLflow model is a folder with an MLmodel file (flavors, signature), the serialized model and an environment definition. Because it is self describing, Azure Machine Learning can deploy it with no scoring script and no custom environment.

az ml model create --name churn-model --version 1 \
  --path azureml://jobs/<job-name>/outputs/artifacts/paths/model/ \
  --type mlflow_model

from azure.ai.ml.entities import Model
from azure.ai.ml.constants import AssetTypes

model = Model(
    path=f"azureml://jobs/{job_name}/outputs/artifacts/paths/model/",
    name="churn-model",
    type=AssetTypes.MLFLOW_MODEL,
    description="Gradient boosting churn classifier",
    tags={"stage": "candidate"},
)
ml_client.models.create_or_update(model)

You can also register during training with mlflow.sklearn.log_model(model, "model", registered_model_name="churn-model"), or from a run with mlflow.register_model("runs:/<run-id>/model", "churn-model"). The model type can be mlflow_model, custom_model or triton_model. Registering the same name again creates the next version.

Package a feature retrieval specification

When features come from a managed feature store, the model needs to know which features to look up at inference time. The feature retrieval specification (a YAML file the feature store client creates with generate_feature_retrieval_spec(folder, features)) lists the features the model uses. Name it feature_retrieval_spec.yaml and package it inside the model artifact folder, so the spec is versioned with the model and the system recognises it. In the Learn tutorial, the training pipeline's register step does the packaging.

model/
  MLmodel
  model.pkl
  conda.yaml
  feature_retrieval_spec.yaml

At training time the spec drives point in time joins to build the training set; at inference time the same spec is used to fetch the same features, which prevents training/serving skew. Anyone deploying the model version gets the right feature definitions with it.

Evaluate with responsible AI

Microsoft's responsible AI principles are fairness, reliability and safety, privacy and security, inclusiveness, transparency and accountability. In Azure Machine Learning, the Responsible AI dashboard operationalizes several of them for a model.

Dashboard component	Question it answers
Model overview and fairness assessment	How does performance differ across groups (sensitive features)?
Error analysis	Which cohorts of data fail most often?
Data explorer	What does the data look like and where is it unbalanced?
Model interpretability	Which features drive predictions, globally and per prediction?
Counterfactual what if	What smallest change would flip this prediction?
Causal analysis	What is the real effect of changing a feature?

The dashboard needs a registered MLflow model plus train and test data as mltable. You create it from the model page in the studio, or in a pipeline with the Responsible AI components. A generated scorecard (PDF) summarises results for non technical stakeholders. Treat a failed fairness or error check as a gate before promotion.

Manage the model lifecycle

Model versions are immutable. Use tags and properties to record the stage (for example stage: candidate, stage: production), the metric that justified promotion and the data version used.

az ml model archive --name churn-model --version 1
az ml model restore --name churn-model --version 1

Archiving hides a version from default listings so teams do not pick it by mistake, while keeping it for audit and rollback. You can archive a whole model name or a single version. For multi environment promotion, publish approved versions to a registry and keep dev experiments in the workspace.

A worked example

A retrained churn model is registered as version 4 with a feature_retrieval_spec.yaml in its folder. The Responsible AI dashboard shows lower recall for one age group, so the team adds data and retrains before tagging version 5 as production. Versions 1 to 3 are archived.

How the exam asks about this

"Deploy a model without writing a scoring script." Register it as an MLflow model.
"Make sure inference uses the same features as training." Package the feature retrieval spec with the model artifact.
"Show whether the model treats groups differently." The fairness assessment in the Responsible AI dashboard.

What to remember for the exam

MLflow models are self describing, which enables no code deployment.
Model versions are immutable; changes mean a new version.
Put feature_retrieval_spec.yaml in the model folder to avoid training/serving skew.
Responsible AI dashboard parts: fairness, error analysis, interpretability, counterfactuals, causal analysis.
az ml model archive hides a version without destroying it.

Watch out

The dashboard works from a registered model and test data. A model that was never registered cannot be assessed in the studio.

Common mistakes

Registering only a pickle file. Without the MLflow structure you must supply a scoring script and environment yourself.

Checking only overall accuracy. A good average can hide a poor result for one cohort.

Deleting instead of archiving. You lose the audit trail and the ability to roll back.

Check yourself

Where does the feature retrieval spec go?
In the model artifact folder, so it is registered and versioned with the model.
Which dashboard component shows how performance differs across groups?
Model overview and fairness assessment.
What does archiving a model do?
It hides the version from default lists while keeping it available for audit and rollback.

Topic 3 of 5 in Implement Machine Learning Model Lifecycle and Operations

Test yourself on Implement Machine Learning Model Lifecycle and Operations

The full Microsoft AI-300 question bank has 650 questions, with timed practice exams and an explanation for every answer.

Subscribe Try Sample Exam

This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.