Data Preparation

What this chapter covers

Data Preparation is 14% of the Databricks Generative AI Engineer Associate exam. It covers everything that happens to documents before a RAG application can retrieve them, and how you check that retrieval works.

The chapter has two parts: getting good content in (choosing the right source documents, extracting text with the right Python package, filtering noise and writing chunks to Delta tables in Unity Catalog), and retrieving it well (chunking strategies, advanced chunking designs, retrieval metrics and re-ranking).

Questions are mostly short scenarios in which the stated requirement (latency, cost, freshness, who may access what, quality signal) decides the answer. They are single-answer and some multiple-answer multiple choice, and short code is shown as text. There are no exhibits.

How the topics fit together

Bad answers from a RAG application often start here, not in the model. Missing documents, leftover headers and footers, or chunks that cut a table in half all lower retrieval quality, and no prompt can repair a chunk that was never retrieved.

• Source questions ask which documents give the knowledge and quality needed, and whether they may be used.
• Parsing questions ask which Python package fits a file format, such as PDF, HTML or Word.
• Cleaning questions ask what content to remove before chunking.
• Delta questions ask for the steps and order to write chunks to a Unity Catalog table ready for indexing.
• Chunking questions ask which strategy fits a document structure and the embedding model limit.
• Retrieval questions ask how to measure retrieval and what a re-ranker adds.

How to use these pages

Each topic page has the same shape, so you can skim for what you need.

• What it is: the idea in plain words.
• How it works: the mechanism, with tables and examples.
• What to remember for the exam: the facts to memorize and how questions are phrased.
• Common mistakes: the classic traps.
• Check yourself: three questions to test your understanding.