Using Large Models and Prompt Engineering

What it is

A large model is used by sending it a prompt, the text input that tells it what to do. The model answers by generating tokens one at a time.

Prompt engineering is the practice of designing prompts so the model gives accurate, useful and well formatted answers. It changes only the input, never the model's weights, so it is usually the fastest and cheapest way to improve results.

How it works

Ways to use a large model

Elements of a good prompt

Clear, specific prompts beat vague ones. Long tasks work better when split into numbered steps.

Prompting techniques

Chain-of-thought helps most on math and logic problems. Reasoning models such as DeepSeek-R1 are trained with reinforcement learning to produce long reasoning chains on their own.

Generation parameters

Temperature divides the model's scores (logits) before the softmax. For two tokens with logits 2 and 1, temperature 1 gives the first token 1 / (1 + e^-1) = 73.1% probability. Temperature 0.5 doubles the logits to 4 and 2, giving 1 / (1 + e^-2) = 88.1%, so the output becomes more deterministic.

Retrieval augmented generation (RAG)

A model knows only what was in its training data, up to its cutoff date. RAG gives it fresh or private knowledge at question time.

Because the answer is grounded in retrieved text, RAG reduces hallucination and can cite its sources. Updating knowledge means updating the documents, not retraining the model.

Agents

An AI agent uses a large model as its brain and adds planning (breaking a goal into steps), memory (short term conversation and long term stored information) and tools (search, code execution, databases, other APIs). It loops: think, act with a tool, observe the result, then decide the next step. A well known form of this loop is ReAct (reasoning and acting).

Hallucination and risks

Hallucination is fluent output that is false or unsupported, such as an invented citation. Reduce it with RAG, clear instructions to answer only from the given context, lower temperature and human review for important uses.

Prompt injection is an attack where malicious text in the input tells the model to ignore its instructions. Applications must treat user and retrieved content as untrusted.

A worked example

A weak prompt: "Tell me about this ticket." A better prompt adds a role, instruction, context and format:

You are a support analyst for an online bookstore.
Classify the ticket below as billing, delivery or account.
Answer only from the ticket text. If unsure, answer "unknown".
Return JSON: {"category": ..., "reason": ...}
Ticket: """My order arrived two weeks late and the box was damaged."""

The expected output is category "delivery". Adding two labeled example tickets would turn this into a few-shot prompt.

How the exam asks about this

• "Which technique adds a few input and output examples to the prompt?" → Few-shot prompting.
• "A company wants answers based on internal documents that change every week, without retraining. Which approach fits?" → RAG.
• "Raising the temperature makes the model's output more random. True or false?" → True.