MLflow Runs, Manual Logging and the UI

What it is

An MLflow run is one execution of training code. It records parameters (inputs such as max_depth), metrics (results such as rmse), tags, artifacts (files) and models, and runs are grouped into experiments.

Autologging captures much of this for supported libraries. The exam asks you to log by hand, to find the best run with the client API, and to know what the MLflow UI shows.

How it works

Manual logging inside a run

import mlflow
with mlflow.start_run(run_name="rf_depth_8") as run:
    mlflow.log_param("max_depth", 8)
    mlflow.log_metric("rmse", 4.27)
    mlflow.log_artifact("residuals.png")
    mlflow.sklearn.log_model(model, "model")

Parameters are logged once per run, while a metric can be logged many times with increasing steps. A model logged with a signature and input_example records its expected input and output schema; Unity Catalog registration requires the signature.

Calls outside a with block act on the active run, and mlflow.end_run() closes it. The with form closes the run for you, even on errors.

Finding the best run with the client API

MlflowClient().search_runs filters and sorts runs in one or more experiments.

from mlflow import MlflowClient
client = MlflowClient()
best = client.search_runs(experiment_ids=[exp_id],
    filter_string="metrics.rmse < 5 AND params.max_depth = '8'",
    order_by=["metrics.rmse ASC"], max_results=1)[0]
print(best.info.run_id, best.data.metrics["rmse"])

It returns a list of Run objects; best.info holds the run_id and status, best.data holds metrics, params and tags. mlflow.search_runs takes similar arguments but returns a pandas DataFrame with columns such as metrics.rmse and params.max_depth.

Sort direction depends on the metric: ASC for errors and losses (RMSE, log loss), DESC for scores (accuracy, F1, AUC). Parameter values are strings, so filters quote them.

Worked example

Three runs log rmse of 4.9, 4.3 and 5.2 and r2 of 0.81, 0.86 and 0.78. order_by=["metrics.rmse ASC"] with max_results=1 returns the second run. Ordering by metrics.r2 DESC returns the same run here, because it also has the highest r2.

What the MLflow UI shows

From the run page you can register the logged model directly, and the artifacts tab shows code to load it.

How the exam asks about this

• "Return the run with the highest validation AUC in an experiment." Use search_runs with order_by=["metrics.val_auc DESC"] and max_results=1.
• "Save a confusion matrix image created during training with the run." Use mlflow.log_artifact on the saved file, or mlflow.log_figure on the figure object.
• "Where do you check which Python packages a logged model needs?" The Artifacts tab, in the model's requirements.txt or conda.yaml.