Snowflake AI Data Cloud Features & Architecture · Topic 16 of 17
Snowpark and Snowflake ML
How Snowpark runs Python, Java and Scala code inside Snowflake, and how Snowflake ML covers features, training, the Model Registry and ML Functions.
- 5 min
- Snowpark supports Python, Java and Scala, with lazy DataFrames pushed down to Snowflake as SQL.
- UDFs, UDTFs, UDAFs and stored procedures run custom code inside Snowflake.
- Python packages come from the Snowflake Anaconda channel or a stage.
What it is
Snowpark is Snowflake's developer framework for writing data processing in Python, Java or Scala instead of only SQL. The code runs inside Snowflake, next to the data, so you do not export data to a separate Spark cluster or application server.
Snowflake ML is the set of features for the full machine learning lifecycle in Snowflake: preparing features, training models, registering and versioning them, running inference and monitoring them. It builds on Snowpark and on Snowflake's compute.
Both share one idea that the exam returns to often: move the code to the data, keep governance in one place, and use Snowflake compute.
How it works
Snowpark DataFrames
The core of Snowpark is the DataFrame API. You build a DataFrame with operations such as filter, select, group_by and join. Nothing runs while you build it; evaluation is lazy.
When you call an action such as collect, show or a write to a table, Snowpark translates the whole chain into SQL and pushes it down to Snowflake. The work runs on a virtual warehouse, and only the final result comes back to the client.
This means Snowpark code gets the same scaling, optimizer and security as SQL, and you can develop it in local tools such as Jupyter, VS Code or IntelliJ.
pandas on Snowflake gives a pandas-style API that runs the same pushdown way, so existing pandas code can work on large tables without pulling them into local memory.
Snowpark functions and procedures
Snowpark also lets you run custom code that SQL cannot express:
| Object | What it does |
|---|---|
| UDF (user-defined function) | Returns one value per row, called from SQL or DataFrames |
| UDTF (user-defined table function) | Returns a set of rows per input |
| UDAF (user-defined aggregate function) | Aggregates many rows into one value |
| Stored procedure | Runs multi-step logic, such as a pipeline or a training job |
Python code in Snowflake can use third-party packages from the Snowflake Anaconda channel, as well as packages you upload to a stage.
Snowpark-optimized warehouses
Memory-heavy Snowpark work, such as model training inside a stored procedure, can exhaust a standard warehouse's memory. A Snowpark-optimized warehouse gives much more memory per node, by default 16 times that of a standard warehouse, for those workloads.
If a scenario says a Python stored procedure fails with out-of-memory errors while training a model, the answer is a Snowpark-optimized warehouse, not a larger number of clusters.
Snowflake ML
Snowflake ML has several parts. At the Core level you need to know what each is for:
| Component | Purpose |
|---|---|
| Container Runtime | Prebuilt environment on CPUs or GPUs for distributed training and hyperparameter tuning |
| Feature Store | Define, store and serve ML features, with batch and low-latency online retrieval |
| Model Registry | Log and version models, trained in Snowflake or elsewhere, and run inference at scale |
| ML Jobs | Develop and automate ML pipelines, including from external IDEs |
| Experiments | Record training runs so you can compare models and pick one for production |
| ML Observability | Monitor model performance and drift in production, with alerts on thresholds |
| ML Lineage | Trace data from source tables through features and datasets to models |
| Model serving | Deploy registry models to Snowpark Container Services for inference |
The Python side is the snowflake-ml-python package, which provides these APIs and preprocessing tools that run in Snowflake.
A model logged in the Model Registry becomes a model object in a schema, with versions. It is governed by RBAC like any other object, and you can call its methods, such as predict, from Python or from SQL.
ML Functions
For analysts who do not build models, Snowflake offers ML Functions that work from SQL. Snowflake picks an appropriate model, so you do not need to be an ML expert.
- Forecasting predicts future values of a time series.
- Anomaly Detection flags values that differ from what is expected.
- Classification sorts rows into classes based on their features.
- Top Insights finds the dimensions that drive changes in a metric.
How the exam asks about this
- "Data engineers want to write transformations in Python that run in Snowflake without moving data out" -> Snowpark DataFrames.
- "A model trained outside Snowflake must be versioned and used for inference in Snowflake under RBAC" -> Snowflake Model Registry.
- "An analyst who knows SQL wants a sales forecast without building a model" -> the Forecasting ML Function.
Common mistakes
Thinking Snowpark pulls data to the client. DataFrame operations are pushed down as SQL and run in a warehouse. Only collect and similar actions bring results back.
Thinking Snowpark is a separate cluster. Snowpark code runs on Snowflake virtual warehouses (and on compute pools for container workloads), not on a separate Spark cluster that you manage.
Scaling out for memory problems. Adding clusters helps concurrency, not memory per query. Out-of-memory Snowpark jobs need a Snowpark-optimized warehouse.
Assuming the Model Registry only accepts models trained in Snowflake. You can register models trained elsewhere and run inference on them in Snowflake.
What to remember for the exam
- Snowpark supports Python, Java and Scala, with lazy DataFrames pushed down to Snowflake as SQL.
- UDFs, UDTFs, UDAFs and stored procedures run custom code inside Snowflake.
- Python packages come from the Snowflake Anaconda channel or a stage.
- Snowpark-optimized warehouses give 16 times the memory per node by default for memory-heavy work.
- Snowflake ML covers Feature Store, Model Registry, Container Runtime, ML Jobs, Experiments, observability, lineage and serving.
- ML Functions such as Forecasting and Anomaly Detection give ML results from SQL without building a model.
Check yourself
When does a Snowpark DataFrame actually run?
When an action such as collect, show or a table write is called; Snowpark then pushes the generated SQL to Snowflake.
Which warehouse type fits a memory-heavy Python model training procedure?
A Snowpark-optimized warehouse, which provides much more memory per node than a standard warehouse.
What does the Snowflake Model Registry provide?
A governed place to log, version and run inference on models, including models trained outside Snowflake.
Topic 16 of 17 in Snowflake AI Data Cloud Features & Architecture
Test yourself on Snowflake AI Data Cloud Features & Architecture
The full Snowflake SnowPro Core COF-C03 question bank has 845 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Snowflake. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.