Model Deployments, Selection, Versioning and Provisioned Throughput

What it is

A model deployment makes a model from the Foundry catalog callable at an endpoint under a deployment name that your app uses. How the model is hosted, and how you pay, depends on the deployment type. You also choose which model fits the job, keep versions under control, and use provisioned throughput units (PTUs) when traffic is steady and latency must be predictable.

How it works

Serverless versus managed compute

Serverless needs no VM quota and scales for you, but limits and prices are set by the service. Managed compute gives dedicated capacity and control over the VM size, needs core quota for that VM family, and costs money even when idle, so delete deployments you do not use.

Deployment types for Foundry models

Capacity for pay per token is a tokens per minute quota per model, region and subscription. Choose a data zone type when residency matters; choose global for the broadest capacity when it does not.

Select the right model

Use the catalog's model leaderboards and benchmarks to build a shortlist, then run the same evaluation dataset against each candidate. A model router deployment can pick among underlying models per request to balance quality and cost. Use smaller, cheaper models for simple tasks and larger ones only where needed.

Versioning and production strategy

• Pin the model version in production. A deployment has a version upgrade policy (versionUpgradeOption): OnceNewDefaultVersionAvailable upgrades within two weeks of a new default, OnceCurrentVersionExpired upgrades at retirement, and NoAutoUpgrade never upgrades, so the deployment stops working at retirement. Automatic updates apply only to Standard deployment types, and production should avoid surprise upgrades.
• Models are retired on a published schedule, so track retirement dates and plan migrations.
• Apps call the deployment name. To upgrade, create a second deployment with the new version, evaluate it, shift a small share of traffic (through app configuration or a gateway such as Azure API Management), then retire the old one.
• Put deployments in Bicep so versions and capacity are reviewed in pull requests.

resource chat 'Microsoft.CognitiveServices/accounts/deployments@2025-06-01' = {
  parent: foundry
  name: 'chat-v2'
  sku: { name: 'GlobalStandard', capacity: 50 }
  properties: {
    model: { format: 'OpenAI', name: modelName, version: modelVersion }
    versionUpgradeOption: 'NoAutoUpgrade'
  }
}

Provisioned throughput units

A PTU is a unit of reserved model processing capacity. You buy PTUs for a deployment instead of paying per token, which gives stable latency and a fixed hourly cost whatever the volume.

Deployment types are ProvisionedManaged, GlobalProvisionedManaged and DataZoneProvisionedManaged. Size capacity with the capacity calculator in the Foundry portal, using requests per minute and prompt and completion token counts. Each model has minimum PTU and increment sizes. Azure reservations lower the hourly price for long commitments. Watch the provisioned utilization metric for the deployment. For bursts, configure spillover: when the provisioned deployment returns a non 200 response such as 429, requests go to a standard deployment in the same Foundry resource (for all requests, or per request with the x-ms-spillover-deployment header). PTU quota is per subscription, region and deployment type, and having quota does not guarantee capacity, so create the deployment before you buy a reservation.

A worked example

A support bot has steady daytime traffic of thousands of requests per minute and a latency target. The team sizes PTUs with the calculator, buys a reservation, and keeps a smaller standard deployment as the spillover target. A new model version is deployed beside the old one as chat-v3, evaluated, and given 10 percent of traffic before cutover.

How the exam asks about this

• "Steady high volume with predictable latency." Provisioned throughput.
• "Prototype with unpredictable usage and no capacity planning." Standard (pay per token).
• "A new model version must be tested before users see it." A second deployment with the new version and gradual traffic shift.