Applications of Foundation Models · Topic 3 of 7
Customisation Trade Offs and Agents
Compare prompt engineering, RAG, fine tuning and training from scratch by cost and effort, and see how Amazon Bedrock Agents handle multi step tasks.
- 4 min
- Includes a diagram
- The customisation ladder from cheapest to costliest is prompting, RAG, fine tuning, continued pre training, training from scratch.
- Prompt engineering and RAG change what you send, while fine tuning and continued pre training change the model.
- Fine tuning uses labeled examples, and continued pre training uses large unlabeled domain data.
What it is
A foundation model out of the box may not know your business. You can customise how it behaves in several ways, which differ a lot in cost, effort and flexibility.
Agents are a different tool. They let a model plan steps and use outside tools, so it can complete tasks and not just answer questions.
How it works
The customisation ladder
| Approach | What changes | Data needed | Cost and effort |
|---|---|---|---|
| Prompt engineering | The instructions you send | Few or no examples | Lowest |
| RAG | The context sent with each prompt | Your documents | Low to medium |
| Fine tuning | The model itself, using labeled examples | Prompt and answer pairs | Medium to high |
| Continued pre training | The model itself, using unlabeled domain text | Large domain collection | High |
| Training from scratch | A new model | Huge broad data | Highest |
Start at the top of the table and move down only if the cheaper option does not meet the need.
Choosing between them
- Need a different tone or format: Try prompt engineering first.
- Need current or private facts: Use RAG.
- Need a specialised style, task skill or vocabulary that prompts cannot achieve: Consider fine tuning.
- Need deep knowledge of a large specialised field: Consider continued pre training.
- Need full ownership of every aspect: Training from scratch, which few organisations do.
Trade offs
Fine tuning needs good quality labeled data and adds training time and cost. A fine tuned model also has to be hosted, and it must be retrained to learn new facts. RAG instead needs a vector store and retrieval pipeline, but updates are as simple as changing the documents.
You can combine approaches. For example, a fine tuned model can be paired with RAG to supply current facts.
Amazon Bedrock Agents
Amazon Bedrock Agents lets a foundation model carry out multi step tasks. The agent breaks a request into steps, decides which tools to use, and calls them to reach the goal.
Typical pieces are:
- Instructions that describe the agent's job.
- Action groups that define the APIs or functions the agent can call, such as looking up an order.
- Knowledge bases that the agent can search for facts.
When to use an agent
Use an agent when a task needs several steps and calls to other systems, such as checking stock, creating a ticket and then replying. For a simple question answered from documents, a RAG setup is enough.
Guardrails and oversight
Agents act, so mistakes can have effects. Limit what the agent is allowed to call, apply Amazon Bedrock Guardrails to filter content, and keep a person in the loop for risky actions.
Other design considerations
Think also about response time, since several steps add delay. Plan for monitoring of agent actions, and decide which data the agent can see.
What to remember for the exam
- The customisation ladder from cheapest to costliest is prompting, RAG, fine tuning, continued pre training, training from scratch.
- Prompt engineering and RAG change what you send, while fine tuning and continued pre training change the model.
- Fine tuning uses labeled examples, and continued pre training uses large unlabeled domain data.
- RAG suits fresh or private facts, and fine tuning suits style and task skill.
- Approaches can be combined, such as fine tuning plus RAG.
- Amazon Bedrock Agents plan steps and call tools through action groups.
- Use an agent for multi step tasks that touch other systems.
- Add Guardrails and human oversight for actions with real effects.
When two answers both work, the exam usually rewards the cheaper and simpler one. Pick prompt engineering or RAG before fine tuning.
Common mistakes
Fine tuning to teach new facts. The knowledge can go stale. RAG handles changing facts better.
Training a model from scratch. It is rarely justified by cost and data needs.
Mixing up fine tuning and continued pre training. Fine tuning uses labeled pairs, and continued pre training uses unlabeled text.
Using an agent for a simple question. A single prompt or RAG call is simpler and cheaper.
Giving an agent unlimited access. Restrict what it may call.
Check yourself
Which is cheaper, fine tuning or RAG?
RAG is usually cheaper and quicker to set up and update than fine tuning.
What do action groups define in an Amazon Bedrock agent?
The APIs or functions that the agent is allowed to call.
Which data does fine tuning use?
Labeled examples, usually prompt and desired answer pairs.
Topic 3 of 7 in Applications of Foundation Models
Test yourself on Applications of Foundation Models
The full AWS AIF-C01 question bank has 849 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.