Fundamentals of GenAI · Topic 1 of 7
Generative AI Core Concepts
Learn the core GenAI terms for AIF-C01: foundation models, LLMs, transformers, tokens, context windows, embeddings, diffusion models and multimodal models.
- 4 min
- GenAI creates new content, while traditional ML mostly predicts labels or numbers.
- A foundation model is large, trained on broad data, and adaptable to many tasks.
- An LLM is a foundation model focused on text, and most use the transformer architecture.
What it is
Generative AI (GenAI) creates new content, such as text, images, audio, video or code, based on patterns it learned from large amounts of data. Earlier ML models mostly predicted a label or a number, while GenAI produces something new.
Most GenAI today is built on foundation models (FMs). A foundation model is a very large model trained on broad data, which can be adapted to many tasks without building a new model each time.
How it works
Foundation models and LLMs
A large language model (LLM) is a foundation model focused on text. It is trained on huge text collections to predict what comes next, and this skill lets it answer questions, summarise, translate and write code.
Foundation models are general. The same model can write a summary in one request and draft an email in the next, which is why they are called foundation models.
Transformers
Most modern LLMs use the transformer architecture, a type of neural network that is good at understanding how the parts of a sequence relate to each other. Its attention mechanism lets the model weigh which words matter most for each other. Transformers also train well on large amounts of data in parallel, which helped models grow in size.
Tokens and the context window
A model does not read whole words. It splits text into tokens, which are small pieces such as words, parts of words or punctuation.
The context window is the maximum number of tokens the model can handle in one request, covering both the input and the output. A long document may not fit if the window is too small. Providers also price many models by the number of tokens processed.
Embeddings and vectors
An embedding is a list of numbers (a vector) that captures the meaning of a piece of content, such as a sentence or an image. Items with similar meaning get vectors that sit close together. This allows search by meaning rather than by exact words, which is the basis of retrieval augmented generation (RAG) described in a later chapter.
Diffusion models
Diffusion models are widely used for image generation. They learn to turn random noise gradually into a clear picture, guided by a text description.
Multimodal models
A multimodal model can work with more than one kind of data, such as text and images together. For example, it can read a chart image and answer a question about it in text.
Related terms
| Term | Meaning |
|---|---|
| Prompt | The instruction or input you give the model |
| Completion or response | What the model returns |
| Inference | Running the model to get a response |
| Parameters | The learned values inside the model, often counted in billions |
| Hallucination | A confident answer that is false or invented |
| Temperature | A setting that makes output more random when raised |
Traditional ML compared with GenAI
| Aspect | Traditional ML | GenAI |
|---|---|---|
| Typical output | Label or number | New content |
| Training | One model per task | One model used for many tasks |
| Data needed | Task specific labeled data | Very large broad data for the base model |
| Example | Predict churn | Write a retention email |
Since FMs are trained once and reused, a company often starts from an existing model instead of training one. That is why selection and customisation, not pre training, are the main skills tested.
What to remember for the exam
- GenAI creates new content, while traditional ML mostly predicts labels or numbers.
- A foundation model is large, trained on broad data, and adaptable to many tasks.
- An LLM is a foundation model focused on text, and most use the transformer architecture.
- Tokens are the pieces of text a model processes, and the context window limits how many fit in one request.
- Embeddings are numeric vectors that capture meaning and enable search by similarity.
- Diffusion models are common for images, and multimodal models handle more than one data type.
- A hallucination is a confident but wrong or invented answer.
- Prompts go in, and responses come out, in a step called inference.
When an option says "creates new text, images or code", it is describing GenAI. When it says "predicts a category or a value", it is describing traditional ML.
Common mistakes
Calling every AI model a foundation model. Only large, broadly trained, adaptable models count. A small model built for one task is not.
Assuming a token equals a word. A token is often a piece of a word, so token counts run higher than word counts.
Confusing the context window with training data. The window is how much the model can read in one request. Training data is what the model learned from earlier.
Expecting an embedding to be readable. It is a list of numbers meant for comparing similarity, not text.
Thinking GenAI always tells the truth. Models predict likely output, so they can produce confident mistakes.
Check yourself
What is a foundation model?
A very large model trained on broad data that can be adapted to many tasks.
What does the context window limit?
The number of tokens the model can process in one request, including the input and the output.
What is an embedding used for?
It turns content into a numeric vector so that items with similar meaning can be found by similarity.
Topic 1 of 7 in Fundamentals of GenAI
Test yourself on Fundamentals of GenAI
The full AWS AIF-C01 question bank has 849 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by AWS. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.