Model Development · Topic 1 of 5
Algorithms, Imbalanced Data and Bias Variance
Databricks ML Associate: pick an algorithm for the scenario, fix imbalanced training data with sampling or weights, read bias and variance in errors.
- 5 min
- The label type decides the problem: number to regression, category to classification, none to clustering.
- Linear models are easy to explain; tree ensembles capture non linear patterns without scaling.
- Fix imbalance with undersampling, oversampling, SMOTE or class weights, applied to training data only.
What it is
Before any tuning, you choose the type of model. The label decides most of it: a number means regression, a category means classification, no label at all means clustering, and user and item ratings point to a recommender.
Two problems then shape the result. Imbalanced data lets a model look accurate while ignoring the rare class, and model complexity decides whether the model underfits (high bias) or overfits (high variance).
How it works
Matching the algorithm to the scenario
| Scenario | Problem type | Typical Spark ML choices |
|---|---|---|
| Predict a house price or demand | Regression | LinearRegression, RandomForestRegressor, GBTRegressor |
| Churn yes or no, fraud or not | Binary classification | LogisticRegression, RandomForestClassifier, GBTClassifier |
| Pick one of several categories | Multiclass classification | LogisticRegression (multinomial), DecisionTreeClassifier, RandomForestClassifier |
| Group customers with no label | Clustering | KMeans |
| Recommend products from ratings | Collaborative filtering | ALS |
Beyond the problem type, the data and the business needs narrow the choice.
| Need | Leaning |
|---|---|
| Explain each feature's effect to auditors | Linear or logistic regression, or a shallow decision tree |
| Non linear effects and feature interactions | Tree ensembles such as random forest or gradient boosted trees |
| Many categorical features, little preprocessing | Tree models, which need no scaling |
| Linear model with regularization | Scale the features and one-hot encode categories first |
| Very fast, simple baseline | Linear or logistic regression |
Random forest trains many trees in parallel on bootstrap samples and averages them, which mainly cuts variance. Gradient boosted trees add trees one after another, each fixing the previous errors, which mainly cuts bias but can overfit if you add too many.
Mitigating imbalanced training data
Picture 10,000 transactions with 100 frauds. A model that always says "not fraud" scores 99% accuracy and catches nothing, so accuracy is the wrong yardstick here.
| Method | What it does | Trade-off |
|---|---|---|
| Undersample the majority | Keep only a fraction of the common class | Throws data away |
| Oversample the minority | Repeat rare rows (sampling with replacement) | Can overfit to repeated rows |
| Synthetic oversampling (SMOTE) | Creates new minority rows between neighbors | Needs a library such as imbalanced-learn |
| Class weights | Make errors on the rare class cost more | Keeps all data, no new rows |
| Better metric and threshold | Use F1, recall or precision recall AUC and tune the cut off | Changes how you judge, not the data |
| Stratified split | Keep the class ratio the same in train and test | Fair evaluation only |
In Spark, sampleBy draws a stratified sample with a fraction per class, and the classifiers that support it take a weightCol column of row weights. In scikit-learn, many classifiers accept class_weight="balanced".
# keep 10% of the majority class (0) and all of the minority class (1)
train_bal = train_df.sampleBy("label", fractions={0: 0.1, 1: 1.0}, seed=42)Resample only the training data. The test set must keep the real class ratio, or the scores will not match production.
Bias, variance and model complexity
| Symptom | Diagnosis | Fixes |
|---|---|---|
| High error on training and validation | High bias, underfitting | More complex model, more or better features, less regularization |
| Low training error, much higher validation error | High variance, overfitting | Simpler model, more data, more regularization, fewer features, early stopping |
| Both errors low and close | Good fit | Keep it |
Worked example: a tree reaches training RMSE 2.1 but validation RMSE 9.8. The gap says overfitting, so reduce maxDepth or raise regularization, not add depth.
As complexity grows, bias falls and variance rises. Total error is lowest somewhere in the middle, and you find that point by tuning complexity hyperparameters against a validation set or cross-validation.
How the exam asks about this
- "Predict the number of support tickets per day." That is a regression problem, so choose a regressor, not a classifier.
- "Only 1% of rows are fraud and the model shows 99% accuracy." The model ignores the rare class; use class weights or resampling and judge with F1 or recall.
- "Training accuracy 99%, validation accuracy 71%." The model overfits; simplify it or regularize it.
What to remember for the exam
- The label type decides the problem: number to regression, category to classification, none to clustering.
- Linear models are easy to explain; tree ensembles capture non linear patterns without scaling.
- Fix imbalance with undersampling, oversampling, SMOTE or class weights, applied to training data only.
- Accuracy hides imbalance; judge with F1, recall or precision recall AUC.
- High bias shows as poor training and validation scores; high variance as a large gap between them.
- More complexity lowers bias and raises variance.
Never resample before the train and test split. Duplicated minority rows would appear in both sets, so the test score measures memory, not generalization.
Common mistakes
Trusting accuracy on skewed data. A model that predicts the majority class every time can score very high accuracy. Look at per class recall or F1 before calling it good.
Making the model bigger to fix overfitting. A large gap between training and validation error is a variance problem. Deeper trees or more features widen the gap; regularization, more data or a simpler model close it.
Choosing a classifier for a numeric target. Predicting an amount, count or price is regression even when the numbers are whole. Turn it into classes only when the business needs bands.
Check yourself
A model has high error on both training and validation data. What is the problem?
High bias, or underfitting, so use a more complex model, better features or less regularization.
Which methods reduce class imbalance without throwing data away?
Oversampling the minority class, synthetic oversampling such as SMOTE, or class weights that make rare class errors cost more.
Which problem type fits grouping customers when no label exists?
Clustering, for example KMeans, because there is no target column to learn from.
Topic 1 of 5 in Model Development
Test yourself on Model Development
The full Databricks Machine Learning Associate question bank has 600 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Databricks. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.