Snowflake AI Data Cloud Features & Architecture · Topic 16 of 17

Snowpark and Snowflake ML

How Snowpark runs Python, Java and Scala code inside Snowflake, and how Snowflake ML covers features, training, the Model Registry and ML Functions.

  • 5 min
In a nutshell
  • Snowpark supports Python, Java and Scala, with lazy DataFrames pushed down to Snowflake as SQL.
  • UDFs, UDTFs, UDAFs and stored procedures run custom code inside Snowflake.
  • Python packages come from the Snowflake Anaconda channel or a stage.

What it is

Snowpark is Snowflake's developer framework for writing data processing in Python, Java or Scala instead of only SQL. The code runs inside Snowflake, next to the data, so you do not export data to a separate Spark cluster or application server.

Snowflake ML is the set of features for the full machine learning lifecycle in Snowflake: preparing features, training models, registering and versioning them, running inference and monitoring them. It builds on Snowpark and on Snowflake's compute.

Both share one idea that the exam returns to often: move the code to the data, keep governance in one place, and use Snowflake compute.

DeveloperPython, Java, ScalaSnowpark APISnowflakegenerated SQLruns onVirtual warehousenear the datareturnsResultsonly what is needed

How it works

Snowpark DataFrames

The core of Snowpark is the DataFrame API. You build a DataFrame with operations such as filter, select, group_by and join. Nothing runs while you build it; evaluation is lazy.

When you call an action such as collect, show or a write to a table, Snowpark translates the whole chain into SQL and pushes it down to Snowflake. The work runs on a virtual warehouse, and only the final result comes back to the client.

This means Snowpark code gets the same scaling, optimizer and security as SQL, and you can develop it in local tools such as Jupyter, VS Code or IntelliJ.

pandas on Snowflake gives a pandas-style API that runs the same pushdown way, so existing pandas code can work on large tables without pulling them into local memory.

Snowpark functions and procedures

Snowpark also lets you run custom code that SQL cannot express:

ObjectWhat it does
UDF (user-defined function)Returns one value per row, called from SQL or DataFrames
UDTF (user-defined table function)Returns a set of rows per input
UDAF (user-defined aggregate function)Aggregates many rows into one value
Stored procedureRuns multi-step logic, such as a pipeline or a training job

Python code in Snowflake can use third-party packages from the Snowflake Anaconda channel, as well as packages you upload to a stage.

Snowpark-optimized warehouses

Memory-heavy Snowpark work, such as model training inside a stored procedure, can exhaust a standard warehouse's memory. A Snowpark-optimized warehouse gives much more memory per node, by default 16 times that of a standard warehouse, for those workloads.

Small transformDataFrameruns onStandard warehouseML trainingstored procedureruns onSnowpark Optimized warehouselarge memory
Exam tip

If a scenario says a Python stored procedure fails with out-of-memory errors while training a model, the answer is a Snowpark-optimized warehouse, not a larger number of clusters.

Snowflake ML

Snowflake ML has several parts. At the Core level you need to know what each is for:

ComponentPurpose
Container RuntimePrebuilt environment on CPUs or GPUs for distributed training and hyperparameter tuning
Feature StoreDefine, store and serve ML features, with batch and low-latency online retrieval
Model RegistryLog and version models, trained in Snowflake or elsewhere, and run inference at scale
ML JobsDevelop and automate ML pipelines, including from external IDEs
ExperimentsRecord training runs so you can compare models and pick one for production
ML ObservabilityMonitor model performance and drift in production, with alerts on thresholds
ML LineageTrace data from source tables through features and datasets to models
Model servingDeploy registry models to Snowpark Container Services for inference

The Python side is the snowflake-ml-python package, which provides these APIs and preprocessing tools that run in Snowflake.

A model logged in the Model Registry becomes a model object in a schema, with versions. It is governed by RBAC like any other object, and you can call its methods, such as predict, from Python or from SQL.

Raw datatablesFeature StoreFeaturesversionedtrain onContainer RuntimeCPU or GPUlog toModel Registrymodel objectinferencePredictionsSQL or Python

ML Functions

For analysts who do not build models, Snowflake offers ML Functions that work from SQL. Snowflake picks an appropriate model, so you do not need to be an ML expert.

  • Forecasting predicts future values of a time series.
  • Anomaly Detection flags values that differ from what is expected.
  • Classification sorts rows into classes based on their features.
  • Top Insights finds the dimensions that drive changes in a metric.

How the exam asks about this

  • "Data engineers want to write transformations in Python that run in Snowflake without moving data out" -> Snowpark DataFrames.
  • "A model trained outside Snowflake must be versioned and used for inference in Snowflake under RBAC" -> Snowflake Model Registry.
  • "An analyst who knows SQL wants a sales forecast without building a model" -> the Forecasting ML Function.

Common mistakes

Thinking Snowpark pulls data to the client. DataFrame operations are pushed down as SQL and run in a warehouse. Only collect and similar actions bring results back.

Thinking Snowpark is a separate cluster. Snowpark code runs on Snowflake virtual warehouses (and on compute pools for container workloads), not on a separate Spark cluster that you manage.

Scaling out for memory problems. Adding clusters helps concurrency, not memory per query. Out-of-memory Snowpark jobs need a Snowpark-optimized warehouse.

Assuming the Model Registry only accepts models trained in Snowflake. You can register models trained elsewhere and run inference on them in Snowflake.

What to remember for the exam

  • Snowpark supports Python, Java and Scala, with lazy DataFrames pushed down to Snowflake as SQL.
  • UDFs, UDTFs, UDAFs and stored procedures run custom code inside Snowflake.
  • Python packages come from the Snowflake Anaconda channel or a stage.
  • Snowpark-optimized warehouses give 16 times the memory per node by default for memory-heavy work.
  • Snowflake ML covers Feature Store, Model Registry, Container Runtime, ML Jobs, Experiments, observability, lineage and serving.
  • ML Functions such as Forecasting and Anomaly Detection give ML results from SQL without building a model.

Check yourself

  • When does a Snowpark DataFrame actually run?

    When an action such as collect, show or a table write is called; Snowpark then pushes the generated SQL to Snowflake.

  • Which warehouse type fits a memory-heavy Python model training procedure?

    A Snowpark-optimized warehouse, which provides much more memory per node than a standard warehouse.

  • What does the Snowflake Model Registry provide?

    A governed place to log, version and run inference on models, including models trained outside Snowflake.

Topic 16 of 17 in Snowflake AI Data Cloud Features & Architecture

Test yourself on Snowflake AI Data Cloud Features & Architecture

The full Snowflake SnowPro Core COF-C03 question bank has 845 questions, with timed practice exams and an explanation for every answer.

This study guide is independent and is not affiliated with, endorsed by, or sponsored by Snowflake. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.