Chapter 5 of 7 · Databricks Associate Developer for Apache Spark
Structured Streaming
Micro batches, exactly once, output modes, windows, watermarks and deduplication.
- 3topics
- 12 minfast track
- 1diagrams and exhibits
- 10%of the exam
Topics in this chapter
Take them in order, or jump to the one you need.
- 1The Structured Streaming Engine, Output Modes and SinksStructured Streaming engine for the Databricks Spark exam: micro batches, exactly once, checkpoints, output modes, sinks…4 min
- 2Operations on Streams - Selection, Windows, Aggregation and WatermarksStreaming operations for the Databricks Spark exam: select and filter, event time windows, aggregations, watermarks, late data…Diagram4 min
- 3Streaming Deduplication With and Without a WatermarkStreaming deduplication for the Databricks Spark exam: dropDuplicates state growth, bounding it with a watermark and…4 min
About this chapter
What this chapter covers
Structured Streaming is 10% of the Databricks Associate Developer for Apache Spark exam. It covers how Spark processes an unbounded stream of data with the same DataFrame API you use for batch work.
The chapter has three parts: how the engine works (micro batches, exactly once semantics, fault tolerance) and how you read and write a stream, the basic operations on a stream including windows and watermarks, and deduplication with and without a watermark.
Questions are single-answer multiple choice. Many show a short PySpark snippet in the stem or the options and ask what it does, which output mode is valid, which line makes the query restartable, or which call meets a requirement. There are no exhibits.
How the topics fit together
A streaming query reads from a source into a streaming DataFrame, applies the same transformations as a batch query, and writes results to a sink in an output mode. The checkpoint makes the query restartable, and the watermark lets Spark discard old state.
- Engine questions ask what a micro batch is and what makes processing fault tolerant and exactly once.
- Write questions ask which output mode, sink or trigger fits a requirement.
- Operation questions ask what a window or aggregation produces and which operations a stream does not allow.
- Deduplication questions ask how state is kept bounded and when a watermark is needed.
Whenever a streaming question mentions "state grows without limit" or "late data", the answer almost always involves a watermark on the event time column.
How to use these pages
Each topic page has the same shape, so you can skim for what you need.
- What it is: the idea in plain words.
- How it works: the mechanism, with tables and examples.
- What to remember for the exam: the facts to memorize and how questions are phrased.
- Common mistakes: the classic traps.
- Check yourself: three questions to test your understanding.
Ready for more practice?
The full Databricks Associate Developer for Apache Spark question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.