System and User Prompts

What it is

A prompt is the input that tells a generative model what to do. In a chat model, the prompt is made of messages, and each message has a role. Good prompts give the model a clear task, the information it needs and the format you want, so that answers are accurate and consistent.

Writing prompts is the cheapest way to improve an AI app, because it needs no training and no new model.

How it works

Message roles

The model keeps no memory between requests. To hold a conversation, the app sends the earlier messages again each time, or uses a service feature that stores the conversation.

messages = [
  {"role": "system", "content": "You are a support assistant for Contoso. Answer in two sentences. Use only the provided policy text."},
  {"role": "user", "content": "Can I return an opened item? Policy: Items may be returned within 30 days if unused."}
]

Qualities of an effective prompt

• Be specific. Say what you want, for whom and in what length. "Summarise this report in three bullet points for a manager" works better than "summarise this".
• Give context. Supply the facts the model needs, because it cannot see your documents on its own.
• State the format. Ask for a table, JSON with named fields, or a fixed number of sentences.
• Set the role or persona. "You are a travel agent" shapes tone and content.
• Add rules and limits. Say what to avoid and what to do when unsure.
• Break complex tasks into steps. Ask the model to work through the steps in order.
• Iterate. Test the prompt on several inputs and refine it.

Examples in the prompt

Examples are an effective way to show the exact format or style you want, without retraining the model.

Grounding and reducing made up answers

When a model lacks the information, it may invent an answer. To reduce this:

• Provide the relevant text in the prompt, and tell the model to answer only from it.
• Tell it to say "I don't know" when the supplied text does not contain the answer.
• Ask it to quote or cite the source passage.
• Keep temperature low for factual tasks.

A longer persona, a higher temperature or a request for longer replies does not add facts, and can increase invention.

Prompt injection

Prompt injection is an attack in which text from a user, or from a document the model reads, tries to override the system instructions, for example "ignore your rules". Mitigations include clear system instructions, treating supplied documents as data, filtering inputs, limiting what the model's tools can do, and using safety features such as prompt shields.

Examples of weak and strong prompts

A worked scenario

A company builds a chatbot that answers staff questions about holiday policy. The first version invents answers when the policy does not cover a case. The team fixes this without changing the model. The system message now says: "You are an HR assistant. Answer only from the policy text provided. If the text does not contain the answer, say 'I don't know' and suggest contacting HR. Reply in no more than three sentences." The app retrieves the relevant policy section for each question and places it in the user message. The invented answers stop, and replies keep a consistent length and tone.