Chapter 2 of 4 · Databricks Machine Learning Associate
Data Processing
Summary statistics, outliers, charts, imputing missing values, one-hot encoding and log scaling.
- 3topics
- 15 minfast track
- 19%of the exam
Topics in this chapter
Take them in order, or jump to the one you need.
- 1Exploring Data: Summary Statistics, Outliers and ChartsDatabricks ML Associate: summary statistics with summary() and dbutils, outlier removal by standard deviation or IQR, charts and…5 min
- 2Imputing Missing ValuesDatabricks ML Associate: when to impute missing values with the mean, median or mode, and how to apply each with Spark ML's…5 min
- 3One-Hot Encoding and Log TransformationDatabricks ML Associate: one-hot encode categorical features in Spark ML, know which models and data sets it does not suit, and…5 min
About this chapter
What this chapter covers
Data Processing is 19% of the Databricks Machine Learning Associate exam. It covers what you do to a Spark DataFrame between loading the raw data and training a model.
The chapter has three parts: exploring the data (summary statistics, outliers, charts and comparing two features), filling in missing values with the mean, median or mode, and transforming features with one-hot encoding and a log scale.
Questions are single-answer and multiple-selection ("Which TWO ...?") multiple choice, mostly short scenarios. Some show a short Python snippet (Spark DataFrame calls, Spark ML feature transformers such as Imputer or OneHotEncoder) and ask what it does or which call meets a requirement. There are no exhibits.
How the topics fit together
You first look at the data to learn each column's type, spread and quality. What you find there decides the rest: a skewed column wants the median and maybe a log, a categorical column wants the mode and maybe one-hot encoding.
- Exploration questions ask which call gives summary statistics, how to drop outliers and which chart or test fits two features.
- Imputation questions ask which of mean, median or mode suits a column and how Spark ML's Imputer applies it.
- Encoding questions ask how to one-hot encode in Spark and when one-hot encoding is the wrong choice.
- Log scale questions ask which column shape calls for a log transformation.
Most questions in this chapter hinge on the shape of the data. Read the scenario for the words "skewed", "outliers", "categorical", "high cardinality" or "orders of magnitude", because each one points to a different answer.
How to use these pages
Each topic page has the same shape, so you can skim for what you need.
- What it is: the idea in plain words.
- How it works: the mechanism, with tables and examples.
- What to remember for the exam: the facts to memorize and how questions are phrased.
- Common mistakes: the classic traps.
- Check yourself: three questions to test your understanding.
Ready for more practice?
The full Databricks Machine Learning Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.