Batch, Streaming and Real Time Inference

What it is

Inference means using a trained model to score new data. On Databricks the same registered MLflow model can be used in three ways: batch (score a whole table on a schedule), streaming (score records as they arrive) and real time (answer one request at a time behind a REST endpoint).

The exam describes a workload and asks which approach fits, then asks how the batch or streaming version is written. Real time endpoints have their own page in this chapter.

How it works

Comparing the three approaches

Batch is the default whenever nobody needs an answer before the next scheduled run. It scores millions of rows in one job, writes them to a Delta table, and downstream reports or apps read that table.

Streaming is driven by data arriving, not by a caller waiting for a reply. Real time needs an always available endpoint, so it costs the most per prediction.

Batch inference with pandas

For data that fits in driver memory, load the model as a generic pyfunc and call predict on a pandas DataFrame.

import mlflow
mlflow.set_registry_uri("databricks-uc")
model = mlflow.pyfunc.load_model("models:/main.ml.churn@champion")
pdf = spark.table("main.ml.customers").toPandas()
pdf["prediction"] = model.predict(pdf[["tenure", "plays", "skips"]])

The @champion alias means the job always scores with whichever version holds that alias, so promoting a new version needs no code change.

Batch inference at scale with spark_udf

When the table is too large for one machine, wrap the model as a Spark UDF so every worker scores its own partitions.

predict = mlflow.pyfunc.spark_udf(spark, "models:/main.ml.churn@champion")
df = spark.table("main.ml.customers")
scored = df.withColumn("prediction", predict("tenure", "plays", "skips"))
scored.write.mode("overwrite").saveAsTable("main.ml.churn_scores")

If the model was logged with a feature engineering training set, FeatureEngineeringClient().score_batch(model_uri=uri, df=df) looks up the features from the feature tables for you, so df only needs the lookup keys and any features not stored in feature tables.

Streaming inference with Delta Live Tables

Delta Live Tables is now called Lakeflow pipelines, built on Apache Spark Declarative Pipelines; existing dlt code still runs. For streaming inference, the model is loaded as a Spark UDF and applied to a streaming source inside a pipeline table definition.

import dlt
import mlflow
predict = mlflow.pyfunc.spark_udf(spark, "models:/main.ml.churn@champion")

@dlt.table
def scored_events():
    df = spark.readStream.table("main.ml.events")
    return df.withColumn("prediction", predict("tenure", "plays", "skips"))

The pipeline processes new records incrementally, manages checkpoints and retries, and can autoscale its compute as event volume changes. The model runs on the pipeline's own cluster, with no network call per record.

Worked example

A retailer wants scores for 2 million customers ready by 06:00. A nightly batch job scores all 2 million rows, then the cluster shuts down.

A podcast app emits tens of thousands of events per second and wants low engagement flagged within seconds. A pipeline that applies the model as a Spark UDF over the event stream fits; calling a serving endpoint once per event would add network overhead for no benefit.

How the exam asks about this

• "Scores are needed once a night for the whole customer table; no low latency requirement." Batch inference on a schedule, the most cost effective choice.
• "A data scientist has a 50,000 row pandas DataFrame and a registered model." Call mlflow.pyfunc.load_model(uri).predict(pdf).
• "Events arrive continuously at high volume and the pipeline compute must resize as volume changes." Apply the model as a Spark UDF in a Delta Live Tables pipeline, not through a serving endpoint.