Design and implement processes and communications · Topic 2 of 3
Metrics, Dashboards and Queries for DevOps
AZ-400 DevOps metrics: cycle time, lead time, time to recovery, DORA measures, Azure DevOps dashboards, Analytics views, work item queries and GitHub Insights.
- 5 min
- Lead time starts at creation. Cycle time starts when work begins. Both end at completion.
- A widening band in a cumulative flow diagram shows a bottleneck.
- Four delivery measures: deployment frequency, lead time for changes, change failure rate, time to restore service.
What it is
You cannot improve a delivery process you do not measure. This topic is about choosing useful metrics for planning, development, testing, security, delivery and operations, and building the dashboards and queries that show them.
A good metric is tied to a decision. "How long does a change wait before it reaches users?" is a flow question with a measurable answer. "How many lines of code did we write?" is not.
How it works
Flow metrics
| Metric | What it measures |
|---|---|
| Lead time | From when the work item is created until it is completed (the customer's wait) |
| Cycle time | From when work starts (the item becomes active) until it is completed (the team's working time) |
| Time to recovery (also time to restore service) | How long it takes to restore service after a failure in production |
| Throughput / velocity | How many items, or how many story points, finish per period |
Lead time is always at least as long as cycle time, because it includes the time the item waited before work began. A cumulative flow diagram (CFD) shows how many items sit in each state over time, and a widening band shows a bottleneck in that state.
DORA style delivery metrics
Four measures are widely used to describe delivery performance.
| Measure | Question it answers |
|---|---|
| Deployment frequency | How often do you deploy to production? |
| Lead time for changes | How long from commit to running in production? |
| Change failure rate | What share of deployments cause a failure or need a fix? |
| Time to restore service | How fast do you recover from a failure? |
Metrics by area
| Area | Examples |
|---|---|
| Project planning | Velocity, sprint burndown, capacity, remaining work, feature progress |
| Development | Pull request age and size, review time, build success rate, code churn |
| Testing | Test pass rate, flaky tests, escaped defects, bug trends, code coverage |
| Security | Open vulnerabilities by severity, time to remediate, secret scanning alerts, dependency alerts |
| Delivery | Deployment frequency, lead time for changes, change failure rate, release duration |
| Operations | Availability, mean time to detect and restore, incident count, alert volume |
Azure DevOps dashboards, queries and Analytics
- Dashboards are made of widgets. Useful widgets include Cycle Time, Lead Time, Velocity, Burndown, Cumulative Flow, Query results, Query tile, Chart for work items and build or deployment status widgets. Dashboards can be team or project dashboards.
- Queries retrieve work items with filters using the Work Item Query Language (WIQL). A query can be a flat list, tree of work items (parent and child) or direct links. Saved queries feed query tiles and charts.
- Analytics is the reporting platform behind the flow widgets and Analytics views. You can connect Power BI to Analytics (through OData) for custom reports.
SELECT [System.Id], [System.Title]
FROM WorkItems
WHERE [System.WorkItemType] = 'Bug'
AND [System.State] <> 'Closed'
AND [Microsoft.VSTS.Common.Severity] = '1 - Critical'GitHub metrics
- The Insights tab of a repository shows Pulse, Contributors, Traffic, Commits, Code frequency, Dependency graph, Network and Forks.
- GitHub Projects can show charts and insights built from item fields, such as items per iteration or by status.
- GitHub Actions shows workflow run history, durations and usage, and the security tab shows alerts for code scanning, secret scanning and Dependabot.
Choosing the right place
| Need | Use |
|---|---|
| Team view of sprint progress | Team dashboard with Burndown and Velocity widgets |
| Find all open critical bugs | A saved work item query and a query tile |
| Custom cross project report | Analytics views with Power BI |
| Repository activity in GitHub | Insights tab |
| Pipeline reliability trends | Pipeline analytics and run history |
A worked scenario
A manager says features take too long to ship, but the team feels busy. The team builds a Cumulative Flow widget and a Lead Time widget. The CFD shows the "In Review" band growing, so the bottleneck is review, not coding. They add a pull request age chart, set a review time goal and watch cycle time fall over the next sprints.
What to remember for the exam
- Lead time starts at creation. Cycle time starts when work begins. Both end at completion.
- A widening band in a cumulative flow diagram shows a bottleneck.
- Four delivery measures: deployment frequency, lead time for changes, change failure rate, time to restore service.
- Azure DevOps uses dashboards, widgets, queries (WIQL) and Analytics. GitHub uses the Insights tab and Projects charts.
- Power BI connects to Azure DevOps through Analytics.
When the choices include "lead time" and "cycle time", look for where the clock starts. Creation means lead time. Start of work means cycle time.
How the exam asks about this
- "Measure how long work waits and is worked on from request to done." Lead time.
- "Find where work piles up." Cumulative flow diagram.
- "Build a custom report across projects." Analytics views and Power BI.
- "See commits and contributors for a repository in GitHub." Insights.
Common mistakes
Mixing up lead and cycle time. The start of the clock is the difference.
Measuring activity, not outcomes. Lines of code and commit counts do not show delivery health.
One dashboard for everyone. Team, project and executive views answer different questions.
Forgetting security and operations. Metrics should cover the whole life cycle, including vulnerabilities and recovery time.
Check yourself
Which metric starts when a work item is created?
Lead time.
What does a widening band in a cumulative flow diagram indicate?
A bottleneck in that state.
Which Azure DevOps component supports Power BI reporting on work data?
Analytics, through its OData endpoint and Analytics views.
Topic 2 of 3 in Design and implement processes and communications
Test yourself on Design and implement processes and communications
The full Microsoft AZ-400 question bank has 766 questions, with timed practice exams and an explanation for every answer.
This study guide is independent and is not affiliated with, endorsed by, or sponsored by Microsoft. Vendor names, certification names and exam codes are trademarks of their respective owners and are used for identification only.