Implement and Manage an Analytics Solution · Topic 1 of 5
Configuring Fabric Workspace Settings
DP-700 workspace settings: Spark pools, environments and high concurrency, domain assignment, OneLake shortcut caching and Apache Airflow pools.
- 5 min
- Spark settings tabs: Pool, Environment, Jobs, High concurrency, Automatic log.
- Starter pools start in seconds; custom pools give control over node size and autoscale.
- One capacity unit equals two Spark VCores, with bursting up to three times for concurrency.
What it is
Every Fabric workspace has settings that decide how its items run. DP-700 tests four groups: Spark settings for notebooks and Spark job definitions, domain assignment, OneLake settings such as shortcut caching, and Apache Airflow runtime settings for Apache Airflow jobs.
A workspace admin changes these in Workspace settings. Tenant settings, owned by a Fabric administrator, sit above them and can switch a feature off for the whole organization.
How it works
Spark workspace settings
Spark settings live under Workspace settings > Data Engineering/Science > Spark settings.
| Tab | What you set |
|---|---|
| Pool | The default pool (the Starter pool or a custom pool), and whether items may customize their own compute |
| Environment | The default environment and the runtime version |
| Jobs | Job admission, the Spark session timeout (20 minutes by default) and a maximum job lifetime |
| High concurrency | Session sharing for notebooks, and for notebooks run by pipelines |
| Automatic log | MLflow autologging for machine learning experiments |
Starter pools are prewarmed clusters of medium nodes, so a session normally starts in seconds. Custom pools let you choose the node size (Small, Medium, Large, X-Large or XX-Large), autoscale limits and dynamic allocation of executors. A custom pool session starts more slowly because its nodes are created on demand.
Each capacity unit gives two Spark VCores, so an F64 capacity has 128 Spark VCores. Bursting lets Spark use up to three times that number for concurrent jobs, which is why many sessions can run at once before jobs queue.
An environment item bundles libraries (public packages from PyPI or conda, custom .whl or .jar files), Spark properties and compute settings. Set one as the workspace default, or attach a different one to a notebook.
High concurrency mode lets several notebooks share one Spark session. Sharing works only when the notebooks run as the same user, with the same default lakehouse and the same Spark compute settings (in practice, the same environment).
Domain workspace settings
A domain groups workspaces by business area, such as Sales or Finance, so data can be found and governed by the people who own it.
| Role | What it can do |
|---|---|
| Fabric administrator | Create domains, assign domain admins, assign workspaces in bulk |
| Domain admin | Manage the domain, create subdomains, assign domain contributors |
| Domain contributor | Assign workspaces they administer to the domain |
A workspace admin who is a domain contributor assigns the workspace to a domain (or subdomain) in Workspace settings. Domain admins can also apply some settings to every workspace in the domain, such as a default sensitivity label. Users then filter the OneLake catalog by domain.
OneLake workspace settings
The OneLake tab controls shortcut caching. When it is on, files read through shortcuts to Amazon S3, S3-compatible storage, Google Cloud Storage and on-premises gateway sources are cached in OneLake. This cuts egress charges and speeds up repeated reads.
You set a retention period of 1 to 28 days; each read of a cached file resets its timer. Files larger than 1 GB are not cached. Workspace admins can also turn on OneLake diagnostics on the same tab, which records data access events into a lakehouse you choose in the same capacity.
Apache Airflow workspace settings
An Apache Airflow job runs Python DAGs on Airflow managed by Fabric. In the Data Factory section of workspace settings you choose its default runtime pool.
| Pool | Use it for |
|---|---|
| Starter pool | Development and testing; quick to start and pauses when idle |
| Custom pool | Production; you choose node size, autoscale and extra nodes |
Python packages and Airflow configuration overrides are set on the Airflow job's environment, not in the workspace settings.
A worked example
A team of five data engineers complains that notebook sessions take minutes to start, and they all use the same default lakehouse. The workspace uses a custom pool with large nodes that it does not need.
The fix is to set the Starter pool as the workspace default, which removes the start-up wait, and to turn on high concurrency for notebooks so one engineer's notebooks share a session. Shared libraries go into one environment set as the workspace default, so every notebook has the same packages without %pip install in each one.
How the exam asks about this
- "Notebook sessions must start as fast as possible with no custom libraries." Starter pool.
- "Every notebook in the workspace must use the same set of libraries." Create an environment and make it the workspace default.
- "Reduce egress cost when Spark repeatedly reads an S3 shortcut." Turn on shortcut caching in the OneLake workspace settings.
What to remember for the exam
- Spark settings tabs: Pool, Environment, Jobs, High concurrency, Automatic log.
- Starter pools start in seconds; custom pools give control over node size and autoscale.
- One capacity unit equals two Spark VCores, with bursting up to three times for concurrency.
- Domain contributors who are workspace admins assign their workspaces to a domain.
- Shortcut caching covers S3, S3-compatible, Google Cloud Storage and gateway shortcuts, for 1 to 28 days.
Notebooks with different default lakehouses, environments or Spark settings do not share a session even with high concurrency on; each starts its own. Check that they match before blaming the setting.
Common mistakes
Choosing a custom pool for faster start-up. Custom pool nodes are created on demand; the Starter pool is the fast option.
Installing libraries in every notebook. An environment set as the workspace default is the managed, repeatable way to share libraries.
Expecting shortcut caching to help with ADLS Gen2 or OneLake shortcuts. The cache targets external clouds and gateway sources where egress or latency costs apply.
Check yourself
Where do you set the default Spark pool and runtime version for a workspace?
In Workspace settings, under Data Engineering/Science, Spark settings (the Pool and Environment tabs).
Who can assign a workspace to a domain from workspace settings?
A workspace admin who is also a contributor (or admin) of that domain.
What retention range can you set for OneLake shortcut caching?
From 1 to 28 days, and each read of a cached file resets its retention timer.
Topic 1 of 5 in Implement and Manage an Analytics Solution
Test yourself on Implement and Manage an Analytics Solution
The full Microsoft DP-700 question bank has 645 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.