Fundamentals of AI and ML · Topic 3 of 8
Model Fit, Bias and Inference
Understand overfitting, underfitting, bias and variance in models, plus batch versus real time inference and the main ways to host a model on AWS.
- 5 min
- Overfitting means great training results and poor results on new data.
- Underfitting means poor results on both training data and new data.
- High bias tends to cause underfitting, and high variance tends to cause overfitting.
What it is
A model is only useful if it works on data it has never seen. Model fit describes how well a trained model captures real patterns instead of noise or oversimplified rules.
Inference is the stage where the trained model is used to make predictions. How and when you run inference affects cost, speed and architecture.
How it works
Overfitting and underfitting
During training, the model is checked against data it did not learn from. That check reveals one of three situations.
| Situation | What happens | Typical sign |
|---|---|---|
| Underfitting | The model is too simple to capture the pattern | Poor results on training data and on new data |
| Good fit | The model captures the pattern and generalises | Good results on both |
| Overfitting | The model memorises noise in the training data | Great on training data, poor on new data |
Overfitting is the classic trap. A model that scores very well in training but badly in testing has learned the training set instead of the underlying pattern.
Common ways to reduce overfitting include getting more and more varied training data, simplifying the model and stopping training earlier. Underfitting is usually addressed with a more capable model, better features or more training.
Bias and variance
In ML, bias means the model makes strong simplifying assumptions, so it misses real patterns. High bias leads to underfitting.
Variance means the model reacts too strongly to small changes in the training data. High variance leads to overfitting.
There is a tension between the two, called the bias and variance trade-off. Reducing one often raises the other, so the goal is a balance.
This technical meaning of bias is different from social bias in data, which is covered in the responsible AI chapter. Read the question carefully to see which one is meant.
Splitting data
Teams split the dataset into a training set, a validation set and a test set. The training set teaches the model. The validation set is used to compare settings during tuning, and the test set gives a final honest score on unseen data.
Batch and real time inference
| Inference style | How it runs | When to choose it |
|---|---|---|
| Real time | One request in, one answer back quickly | Interactive apps that need an instant reply |
| Batch | A large set of records is processed together | Large jobs where waiting minutes or hours is fine |
Real time inference needs an endpoint that stays available and ready, so it costs more to keep running. Batch inference can start, process a file of records and stop, which is often cheaper per record.
Hosting options on AWS
With Amazon SageMaker AI you can deploy a model to a managed real time endpoint, or run a batch transform job over a dataset. An asynchronous option exists for large payloads or long processing times, and a serverless option exists for traffic that is irregular. The exam stays at the level of knowing that managed options exist and choosing between real time and batch.
A model can also be consumed through a managed API from an AWS AI service, where AWS runs the model for you. That removes the need to train, host or scale anything yourself.
Why models degrade
Real world data changes over time. A model trained on last year's behaviour may become less accurate as customers, prices or language change, which is called drift. This is why deployed models need monitoring and sometimes retraining.
What to remember for the exam
- Overfitting means great training results and poor results on new data.
- Underfitting means poor results on both training data and new data.
- High bias tends to cause underfitting, and high variance tends to cause overfitting.
- More varied data, a simpler model or earlier stopping are common ways to reduce overfitting.
- Training, validation and test sets have different jobs, and the test set is for the final check.
- Real time inference answers requests immediately, and batch inference processes many records together.
- Drift means data in production changes, so accuracy falls over time and monitoring is needed.
- Managed AWS AI services remove the need to host or scale a model yourself.
A very high accuracy on training data is not good news by itself. Always ask how the model performed on data it had not seen.
Common mistakes
Believing a high training score proves the model is good. The score on unseen data is what counts. A large gap between the two is the sign of overfitting.
Mixing up the two meanings of bias. In model fit, bias is about oversimplified assumptions. In responsible AI, bias is about unfair results for groups of people.
Choosing real time inference for every case. If results are needed only once a night for a million records, batch is simpler and cheaper.
Thinking a deployed model stays accurate forever. Data changes, so performance must be watched after deployment.
Using the test set to tune the model. If you tune on the test set, it no longer gives an honest final score.
Check yourself
A model scores very high in training but poorly on new data. What is the likely problem?
It is overfitting, meaning it memorised the training data instead of general patterns.
When is batch inference a better choice than real time inference?
When many records can be processed together and an immediate answer is not required.
What is drift?
Drift is a change in real world data over time that makes a deployed model less accurate.
Topic 3 of 8 in Fundamentals of AI and ML
Test yourself on Fundamentals of AI and ML
The full AWS AIF-C01 question bank has 849 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.