A Chat Client with the Foundry SDK

What it is

A chat client is a small program that sends a prompt to a deployed model and prints the reply. With the Foundry SDK for Python, the pattern is the same each time: create a client for your project, get a client for calling models, make the request and read the answer.

Knowing this short pattern is what the exam tests, rather than long code.

How it works

Packages and sign in

The SDK is in the azure-ai-projects package, and the credential class is in azure-identity.

pip install azure-ai-projects azure-identity
az login

az login signs you in with the Azure CLI, and DefaultAzureCredential then uses that sign in. The same code works later with a managed identity when it runs in Azure, so no key is stored.

A minimal chat call

from azure.identity import DefaultAzureCredential
from azure.ai.projects import AIProjectClient

# Format: https://<resource-name>.services.ai.azure.com/api/projects/<project-name>
FOUNDRY_PROJECT_ENDPOINT = "your_project_endpoint"

project = AIProjectClient(
    endpoint=FOUNDRY_PROJECT_ENDPOINT,
    credential=DefaultAzureCredential(),
)
openai = project.get_openai_client()

response = openai.responses.create(
    model="your-deployment-name",
    input="What is the size of France in square miles?",
)
print(response.output_text)

Reading it part by part:

The code in the quickstart uses the Responses API. Models can also be called with the Chat Completions API, which takes a list of messages with roles and returns the reply in choices[0].message.content.

completion = openai.chat.completions.create(
    model="your-deployment-name",
    messages=[
        {"role": "system", "content": "You answer in one sentence."},
        {"role": "user", "content": "Why is the sky blue?"},
    ],
    max_tokens=200,
)
print(completion.choices[0].message.content)
print(completion.usage.prompt_tokens, completion.usage.completion_tokens, completion.usage.total_tokens)

The usage object reports prompt_tokens, completion_tokens and total_tokens, which you can use to track cost.

Multi turn conversations

A model does not remember earlier calls. There are two ways to keep a conversation going: send the earlier messages again with every request, or use a conversation that the service stores for you and pass its identifier on each call.

Handling secrets

• Do not put keys, tokens or endpoints with secrets in the source file.
• Prefer DefaultAzureCredential with Microsoft Entra ID, so there is no key at all.
• If a key is unavoidable, read it from an environment variable or an Azure Key Vault secret. Do not commit it to the repository, put it in a query string or leave it in a constant.

Handling errors

For long jobs, a client that polls for a result in a tight loop causes throttling. Poll at a sensible interval instead.

Checking the output

Test the client with several prompts, including one that is empty or very long. Check that the reply is not empty before you use it, and log token usage so you notice cost growing.

A worked scenario

A developer builds a command line tool that explains error messages. She installs the two packages, signs in with az login and writes a short script. The script creates the project client, gets the model client and sends the user's error text with a system message that asks for a two sentence explanation. She uses the deployment name, not the catalog name, and reads the reply from the response. The endpoint comes from an environment variable. When the tool is later deployed to Azure, it uses a managed identity, and the code does not change.