Fundamentals of AI and ML · Topic 6 of 8
The ML Development Lifecycle
Walk through the machine learning lifecycle from business goal and data preparation to training, evaluation, deployment, monitoring and retraining.
- 4 min
- Includes a diagram
- The lifecycle runs from business goal to data, training, evaluation, deployment, monitoring and retraining.
- ML problem framing decides whether ML fits and which task type applies.
- Data collection, preprocessing and feature engineering usually take the most effort.
What it is
Building an ML solution is a repeatable process with clear stages, not a single training run. The same stages apply whether you use a classic model or a foundation model.
The process is a loop. What you learn from monitoring a live model feeds back into earlier stages, so the work is never really finished.
How it works
The stages in order
| Stage | What happens |
|---|---|
| Business goal | State the problem and how success will be measured |
| ML problem framing | Decide if ML fits, and which type of task it is |
| Data collection | Gather data from the right sources |
| Data preprocessing | Clean, fix and format the data |
| Exploratory analysis | Look at the data to find patterns and problems |
| Feature engineering | Create or select the inputs that help the model most |
| Model training | Run the algorithm on training data |
| Hyperparameter tuning | Adjust the settings that control learning |
| Evaluation | Score the model on data it has not seen |
| Deployment | Make the model available for inference |
| Monitoring | Track accuracy and data changes in production |
| Retraining | Update the model when quality drops |
Data work comes first
Most of the effort goes into the data. Data preprocessing handles missing values, duplicates and inconsistent formats. Feature engineering turns raw data into useful inputs, for example turning a date into day of week.
Data is also split into training, validation and test sets, so that the final score reflects real use.
Hyperparameters
Parameters are learned by the model during training. Hyperparameters are set by you before training, such as the learning rate or the number of layers. Hyperparameter tuning tries different settings to find a good combination.
Sources of models and deployment
You do not always train from scratch. A team can call a managed AI service, start from a pre trained model such as those in Amazon SageMaker JumpStart, or train its own model when the task is unique. The SageMaker AI and MLOps page compares these routes in detail.
A finished model can be deployed to a managed endpoint for real time answers, or run in batch over a dataset. Some models run at the edge, close to the device, when low delay or limited connectivity matters.
Monitoring and feedback
After release, teams watch both technical and business results. Technical checks include accuracy and drift in the input data. Business checks include the goal set at the start, such as lower costs or higher sales.
When results fall below an agreed level, the team collects fresh data and retrains. This closes the loop back to the data stages.
Fairness and review
Evaluation should check more than accuracy. It should also look at fairness across groups and at whether the model meets the goal set at the start.
What to remember for the exam
- The lifecycle runs from business goal to data, training, evaluation, deployment, monitoring and retraining.
- ML problem framing decides whether ML fits and which task type applies.
- Data collection, preprocessing and feature engineering usually take the most effort.
- Parameters are learned in training, and hyperparameters are set before training and tuned.
- Evaluation uses data the model has not seen, such as a test set.
- Models can be deployed for real time, batch or edge use.
- Monitoring watches for drift and falling accuracy, and retraining updates the model.
- Pre trained models and managed services can shorten the path to value.
If a question asks what to do first, the answer is almost always to define the business problem and success measure, before collecting data or choosing algorithms.
Common mistakes
Jumping straight to training. Without a clear goal and clean data, even a strong algorithm gives poor results.
Treating deployment as the end. A live model needs monitoring, since data and behaviour change over time.
Mixing up parameters and hyperparameters. The model learns parameters by itself. You choose hyperparameters before training.
Evaluating on the training data. The score will look better than real use. Always test on unseen data.
Forgetting the business metric. A model can have good accuracy and still fail to deliver the business outcome it was built for.
Check yourself
What is the first stage of the ML lifecycle?
Defining the business goal and how success will be measured.
What is the difference between a parameter and a hyperparameter?
A parameter is learned by the model during training, and a hyperparameter is a setting you choose before training.
Why is monitoring needed after deployment?
Data and behaviour change over time, so accuracy can drop, and monitoring shows when retraining is needed.
Topic 6 of 8 in Fundamentals of AI and ML
Test yourself on Fundamentals of AI and ML
The full AWS AIF-C01 question bank has 849 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.