DevOps Research and Assessment (DORA) Metrics and Flow Metrics: Use the Whole Ruler, Not Just Two Inches

Tasktop Technologies
7 min readJul 1, 2021

--

Software delivery is all about delivering more customer value faster to remain competitive in the digital-first world. And like any process, you need meaningful data insights to understand how fast you’re delivering, what’s slowing you down and what you can do to improve. While DevOps Research and Assessment (DORA) metrics help you deliver more quickly, Flow Metrics — as defined by Dr. Mik Kersten as part of his pioneering Value Stream Management paradigm the Flow Framework® — help you deliver the right things more quickly, at the right quality and cost and with the necessary team engagement. In this blog, you’ll learn the difference between DORA and Flow Metrics and how they complement each other.

DORA Metrics: Gold Standard for Releasing Code

DORA metrics are a key set of acceleration metrics for software delivery, helping you to streamline and automate activities from development to deployment:

  • Deployment Frequency-How often an organization successfully releases to production
  • Lead Time for Changes-The amount of time it takes a code commit to get into production
  • Change Failure Rate-The percentage of deployments causing a failure in production
  • Time to Restore Service-How long it takes an organization to recover from a failure in production

These development team metrics set the gold standard for operational efficiency for releasing code rapidly, securely and confidently. They get us off the ground and are valuable for measuring and optimizing development to release. However, they don’t measure and optimize the entire journey from customer request to release (and back through the customer feedback loop to continuously improve value delivery). Nor are they aligned against defined business outcomes (such as revenue, customer responsiveness, customer retention, product team turnover and so on). This is significant because for many organizations, most time and money is wasted work even hits a development team’s backlog:

A real-life example of total end-to-end lead time (Flow Time) from a leading U.S. insurance company

If you’re proficient at DORA metrics and still failing to see the needle move for the business, then it’s probably time to start thinking about end-to-end Flow Metrics.

Value Stream Management and Flow Metrics: Measure The Whole System

“Measuring only one area of the value stream is like only using two inches of a 12-inch ruler.” John Willis, Senior Director of the Global Transformation Office at Red Hat, co-author of The DevOps Handbook, Mik+One: Project to Product Podcast, Episode 17

The 2020 State of DevOps report found that 79% of respondents are mid-level on the DevOps evolution scale (16% high, 5% low). Meaning that in the circles where you compete, everyone is likely to become a DORA high performer in short order.

Your organization, therefore, will remain competitive only if it can deliver business value — not code changes — at an ever-increasing clip. DORA metrics are the first-stage booster rocket that will get you off the ground and into orbit, but they are not enough to get you to the moon.

The meteoric rise of Value Stream Management (VSM) in software delivery reflects the pressing need for end-to-end visibility across the whole value stream. While Agile gets products built and DevOps accelerates delivery, that’s only part of the story. VSM looks at the whole story and Flow Metrics help you to understand the storytellers and the chapters as the story evolves.

These specific value stream metrics help you continuously and systematically to see what’s impeding flow and enable you to remove bottlenecks in a sustainable way to meet business outcomes sooner and better. They form a key part of your continuous improvement journey, identifying areas to investigate while tracking the impact of any changes you make.

Flow Metrics build on Lean manufacturing concepts such as lead time and cycle time associated with DORA over the years, taking it to the next level to provide a top-level view of how value-creating and protecting work travels between all key stages. For instance, while the DORA Lead Time metric measures code commit to code deploy, Flow Time measures the whole system from ideation to production — starting from when work is accepted by the value stream and ending when the value is delivered to the customer.

Flow Metrics: Measuring Value, Not Actions

While DORA and other Agile metrics focus on team activities, we know from systems thinking that we need to take a holistic approach as software delivery is a complex adaptive process. Only with a macro view can you answer critical questions like:

  • Who the customer is (to define value being delivered)
  • What units of value are flowing across the system
  • How the system is delivering value to the customer
  • How quickly the system currently delivering value
  • How much value is the system current delivering
  • Where is the most pressing bottleneck that is slowing value delivery down
  • How to balance resources to understand what to prioritize to accelerate flow where it matters most for the business and your customers

Flow Metrics help you build this macro view for leadership, providing a clear indication of whether value stream flow is sufficient to support targeted business outcomes. They get to the heart of what software delivery is all about, measuring the end-to-end flow of all value-related work:

  • Features: New business value
  • Defects: Quality problems that affect the customer experience
  • Risk: Addressing security, privacy and compliance exposures
  • Debt: Removal of impediments to future delivery (such as technical debt, ways of working, people and process issues)

What are the Flow Metrics?

The Flow Metrics, as set out by Dr. Kersten in the Flow Framework, are:

Flow Velocity

How much customer value are you delivering over time?

Flow Efficiency

Is waste decreasing in our processes? What are the delays and wait times slowing you down?

Flow Time

Is time-to-market getting shorter for faster customer feedback to continuously learn and improve value?

Flow Load

Are we balancing demand vs. capacity to ensure future productivity? High Flow Load is a key indicator that too much work-in-progress is undermining productivity and throughput

As well as the above set of Flow Metrics there is Flow Distribution, which looks at the trade-offs between value creation and protection work, helping you understand what adjustments do you need to make to ensure a sustainable pace of delivery.

The Scaled Agile Framework® Adopts Flow Metrics

Such is the influence and importance of Flow Metrics and their role in data-driven VSM, they’ve just been added as a key measure for the latest SAFe®. In addition to the above Flow Metrics and Flow Distribution, SAFe 5.1 adds another metric — Flow Predictability — which helps you determine when your customers going to get the thing that they asked for.

DORA and Flow Metrics: Working Together

By combining DORA and Flow Metrics, you can ensure your acceleration gains in development and delivery are felt across the whole organization. By using the whole ruler and not just two inches, you can quantify the impact of your DevOps transformation against business results, helping to understand how you can tangibly improve your customer response.

If you want to know more about how you leverage your existing metrics initiatives with Flow Metrics, head over to The Flow Institute, where you can enroll in a range of on-demand and live courses and workshops. Our Flow Experts have worked in and with some of the largest and most impactful organizations on the planet. Whatever your questions about software delivery, VSM or flow, they’ll have the answers to make your daily work life easier and more fulfilling.

A Deeper Drive into Flow Metrics with Dominica DeGrandis

In this on-demand course by Dominica DeGrandis (bestselling author of Making Work Visible) introduces the Flow Metrics, providing a deeper dive into what they are and why you need them. The course explains the theory behind Flow Time, Flow Velocity, Flow Efficiency, Flow Load and Flow Distribution.

Register today.

Written by: Patrick Anderson

Originally published at https://blog.tasktop.com on July 1, 2021.

--

--

Tasktop Technologies

Value Stream Management platform. Experts in moving from Project to Product. Helping enterprises to master software at scale and thrive in the Age of Digital.