An IT Executive’s Guide to Improving your Agile Practice

Your teams have implemented “Agile” — Now what?

Introduction

You’ve introduced Agile practices to your organization. You’ve trained your people, executed the Agile way, and brought in new tools. And perhaps you’ve successfully executed a few Agile projects with a core development and test team.

Now it is time to expand Agile practices to other disciplines in your software development and delivery So you’re faced with the same triumvirate of training people, understanding the process, and supporting them with tools.

But scaling your Agile practice across teams is not about doing the same things you did to implement Agile in a small, core team. In this paper, Dave West covers many of the key issues IT teams face when scaling Agile practices. It also offers IT executives a view on the issues they are likely to face as well as offering guidance on how to plan for them.

We can talk about scaling, talk about enterprise Agile. But honestly, should the focus be on Agility or something else? In other words, is Agile the future?

Over the last 14 years Agile has moved from a nice set of ideas practiced by experts, to mainstream adoption. In 2013, Forrester research reported that 90% of IT organizations were doing some form of Agile — A staggering statistic that highlights the value and importance of Agile. But for the majority of organizations, once the initial sparkle of team-based Agile adoption has worn off, the question becomes “what is next?” How do we scale Agility across the enterprise to introduce it to more teams, and to broaden its reach into other groups? How do we remove the friction points between Agile teams and the rest of enterprise IT?

We have “done” Agile, now what?

Enterprise agility misses the point

When the initial Agile manifesto was created, there was debate between Martin Fowler and the rest of the group about what this work should be called. Fowler wanted to use the term “Adaptive,” others thought the term “Agile” would be more marketable. The term “Agile” won, but it highlights the fundamental premise of Agility. “Agile” describes the ability to adapt to the environment and constraints of the situation in order to deliver software in the most effective way. That means a strong focus on value and effectiveness, which is described in the four fundamentals of the manifesto:

1. Individuals and interactions

2. Working software

3. Customer collaboration

4. Responding to change

In part, these principles described the things the 17 signatories deemed important, but they also highlighted what was wrong with the direction proposed by Big Process and Tools companies. The principles proposed by the signatories work well for core development teams, highlighting the things that matter. But they do not scale to the requirements of the enterprise. In particular they ignore that:

• Large problems call for large solutions. For many endeavors, the luxury of a business aligned team of 7 (plus or minus 2) does not work. Many complexity-removing features of process models like Scrum, such as single product ownership and process ownership, become problematic when trying to coordinate the Teams of Teams that are delivering highly dependent software.
• The Business and IT are on different planning cycles. High-level business planning, budgeting, and reporting tend to be annual in nature, with initiatives starting and stopping based on annual business plans. It’s easy to recognize that this is in conflict with the notions of Agility. But add to this the notion that software, once built, tends to exist for a long time. Many IT organizations are dealing with core systems that were created 30, even 50, years ago. Over this time frame, these applications have evolved to support the ever-changing business, leading to large amounts of technical debt.
• Business is complex and involves MANY stakeholders. The unfortunate truth is that The Business is not a single entity, but a collection of different groups with different agendas and desires. This is fine when human beings are dealing directly with one another to solve a problem, but building software requires clarity, it does not deal well with ambiguity or unknowns. Therefore, it is crucial to manage these stakeholders in order to gain consensus and agreement.

The issues of dependencies, complexity, and large numbers of stakeholders create challenges at a fundamental level for the four principles of Agility. They require us to look into the value of the items on the right in the manifesto — Items such as process and tools, documentation, commitments, and plans:

htttp://agilemanifesto.org

Time for a new check list

I am not going to tell you that SAFe is not the way to go, or that DAD will not help solve your problems. In fact, all these next generation Agile frameworks provide organizations with a great set of ideas and resources to help scale software delivery practices. But, the resulting process and organizations are not enough — software delivery executives need to step back and look holistically at the thing they are trying to fix. And, unfortunately, there is no silver bullet that, if adopted, will make everything easier. That said, I propose a step-by-step process for reevaluating your software delivery practice.

Step 1 — You need to think of software delivery as a system(s)

The act of systems definition requires clear inputs and well-understood outputs. So for many people, the idea that the process of developing and supporting software is “a system” is antithetical to Agile principles. After all, the Agile manifesto taught us that we need to forget old ways of thinking about software delivery and replace them with people-centric approaches. And, the Agile leaders were right — but also wrong. In the same way that quantum physics describes things very differently than Newtonian physics, Software Delivery in the large is very different than Software Delivery in the small. Agile is effective at the team level, but when you start scaling to more people working in the same area, or across the enterprise, the laws that govern good Agile-in-the-small break down. That is why it is key to think about the software delivery process as a system. This way of thinking allows you to:

• A Blackbox approach — What do you want to come out? How frequently? How do you measure success? At some level, the delivery organization has to deliver something of value to the business, based on some level of input from the business. This suggests three different types of systems: 1) bug fixes, 2) major feature changes, and 3) innovation (or something more aligned to the business areas). But understanding the boundary of the system is a key in order to effectively understand the domain of change. Also, thinking about how the system aligns with other systems is important when thinking about delivery in-the-large.
• A Whitebox approach — Once the overall boundary is understood, what are the components that constitute the system? Is it oriented around development, test, PMO, and support? Or is it vertically aligned to applications? Or is it some combination of both? By better understanding the environment that software is delivered in and the objectives or services it provides, you start to build an understanding of how the system behaves at a macro level.
• Ignore the process, disciplines, and organizations and concentrate on flow. It is easy when thinking about the delivery “system” to fall into the rat hole of processes, disciplines, and organizational constructs. But by focusing on the flow of information around the system, it is possible to build a quick and easy understanding without having to immerse yourself in the morass of process documentation and HR details.

At this point you might be wondering why we are applying Systems Engineering to the practice of software delivery. It’s true that systems engineering is often associated with large-scale product development activities undertaken by the Department of Defense. But applying the basic ideas of Systems Engineering to the delivery life cycle provides a great foundation for understanding how to scale Agility. Of course, those ideas need to be applied in a pragmatic way and I am not proposing building a lot of documents or spending six months doing a study, but thinking about the life cycle in a “systems” way can provide a different and useful perspective.

Step 2 — Review the artifacts — one artifact at a time

There are two types of artifacts within the software delivery system. First, there are the artifacts used to manage the Work such as tasks, stories, defects, test plans, enhancement requests, and epics, etc. The other artifact type is the Assets that are being worked on, such as specifications, code, build scripts, deployment scripts, binaries, etc. Artifacts play a key role in efficiency and value because, for many software delivery systems there are huge amounts of waste as artifacts move between the disciplines, departments and organizational boundaries. DevOps initiatives focus on this area of waste. DevOps encourages organizations to employ automation and abstractions (such as Infrastructure As Code) that enable assets to flow from Development to Operations, with limited human intervention. But Assets are not the only artifacts that reduce efficiency and increases cycle time. Work artifacts, such as requirements, defects, tickets, and test results, all transition through different disciplines and attract waste.

Each key artifact should be reviewed for:

• Disconnects, as they flow from one discipline to another. The best way to discover those areas of waste is to examine when artifacts from one system are translated into another. For example defects taken from the test management system and moved into a spreadsheet. Or a ticket might be cut and pasted into email to be sent to development management to be worked on. When an artifact is re-entered, not only does it take time, but it also increases the likelihood of mistakes, which in turn leads to waste and re-work.
• Collaboration on the artifact. How do the different stakeholders work on the same artifact, providing comments and sign off? Often, there is a secondary, costly and time consuming vehicle such as email used to manage this process. Perhaps, the worst example of waste in this area is the large manufacturing company that keeps specifications in a legacy requirements management system. Not only do they print them, scan them, and email them — but also their suppliers then print them out, write notes on them, and re-scan then email them back. They are entered into a change log document, which is then used for discussion in a meeting. This process is expensive and error prone with many cycles invested by both creator and reviewer to ensure their comments are provided.
• Reporting needs. All artifacts are featured in reports that describe the status or communicate dependencies. By reviewing the cost and effort of creating those reports and looking at the expectations of the boundary of the development system, it is possible to understand not only the disconnects, but also the inefficiencies the current system is exhibiting. A great example of this is the monthly report created by the PMO that describes cost burn and project status. That report talks in terms of resources and time, whereas the Agile plan talks in terms of teams, story points, and velocity. The process to take the Agile status and re-cast it into a more traditional status report is both time consuming and difficult, and the resulting status information is misleading. For example, is a story half completed if 10 of the 20 story points are complete?
• Compliance and traceability. If you are building software in the context of a large IT organization then it is important that the software is compliant with many standards. Also, software projects are expensive, so observing that work is being done in a disciplined and controlled manner makes financial controllers feel much happier that the money is being managed wisely. Pharmaceuticals, manufacturing, and financial services organizations also have to comply with external standards. These environmental constraints on the system create requirements that an artifact must support. Also, for many software delivery organizations, dealing with compliance issues is an overhead that must be balanced with delivering new value-creating work.

Step 3 — Look to automation to remove non value added work

Waste is inherent in any complex system, as teams and individuals wait for things; share understanding and status; and report progress. Each time a human is involved in transitioning an artifact, time and effort is expended, and each transition increases errors and the need for rework. DevOps is often described as the next step for Agile and it encourages organizations to automate the flow and reporting for the software assets from development to operations. But I would go one step further and encourage organizations to consider removing as many of the “non value added” steps as possible throughout the life cycle.

Look to automate your life cycle, concentrating on:

• Flow — ensuring that the artifacts move between the different tools used to deliver the software with limited human intervention.
• Collaboration — remove email and external collaboration tools from the process and ensure that the tools used by each discipline and each team connects their activity streams allowing in-context collaboration.
• Reporting — Make sure that each report created does not require a large amount of manual effort to create and communicate.
• Traceability — remove the need for manual traceability and compliance reports by ensuring that compliance and traceability is a natural byproduct of the life cycle.

Step 4 — add analytics and understanding

By automating the creation, management, and transitions of the artifacts that drive the system, it is possible not just to improve reporting, but also take advantage of the meta-data that is captured by the process. For example, by creating a federated, cross-discipline artifact, it is possible to provide in near real time a KANBAN board that shows process bottlenecks. The end goal is that like manufacturing plants that have a KANBAN board that shows the status of the whole process at any one time, software delivery will be able to provide a similar visualization. Adding a data-warehouse component to the tool-stack makes it possible to capture the temporal data associated with each artifact. This data is a record of how the artifacts change over the delivery process, providing the metrics for analytical reports such as:

• Artifact cycle time — Seeing how an artifact cycles through the system and then applying statistical models on the whole set of artifacts, provides a better understanding of how long things really take. As you broaden the life cycle across tools and disciplines, this information will provide insight into how effective your process is.
• Coverage — Seeing how complete an artifact chain is and what is changing within it enables organizations to ensure that the Desire of Focus is actually the Reality of Execution. For example, a continued focus on one part of the requirement stack creating more and more tests might conflict with the management desire of getting just enough coverage and moving onto another high value area of work.
• Value — being able to clearly map the work being done and the value that it is creating is the ultimate end goal of any development organization, and often it is very difficult to obtain. But there are steps towards that goal by providing better visibility into, for example, the connection between customer value metrics held in CRM systems and stories being executed in the development team.

Making it happen

By taking a more holistic view of the software delivery chain (from the PMO, through development, and test, all the way into operations); identifying areas of inherent conflict as work moves from one group to the next; reducing the conflict and waste that’s created in those transitions; and creating an analytics infrastructure that allows you to identify issues and improve your process, you will have created a system that honors the fact that Agility at enterprise-scale is vastly different than Agility in a small, core group.

Rome was not built in a day and scaling your ability to delivery software in an Agile manner can only be improved incrementally. But where do you start?

Looking holistically at your delivery system provides two dimensions to implementation: Projects/ programs and artifacts. For example, start by automating the defect flow for one team. Then add another team. For the first team, help the testers by flowing the requirements from the Business Analysts’ (BA) requirements elicitation and management system to your testers’ test management system. From there, they can build their test strategies and plans.

Next, involve the PMO by flowing their business initiatives to the BAs… and then the requirements from the BAs to the Agile planning tools used by the core development team. The PMO may plan their work and budgets yearly in their PPM tools, but that shouldn’t stop an Agile development team from using their favorite Agile planning tools to build out and implement user stories.

In this way, you will incrementally scale across the entire software development and delivery life cycle, removing conflict, and waste along the way… all while accumulating metrics for a process-improving analytics program.

How Tasktop can help

Tasktop offers two products that can help in this endeavor:

• Tasktop Sync a life cycle integration product that synchronizes artifacts among the tools used in software development and delivery. It flows the artifacts from tool-to-tool, enabling collaboration among stakeholders and reducing wasted time.
• Tasktop Data collects life cycle data from the same tools, and aggregates it into a single database to enable reporting on cross-discipline processes. It enables insights into your life cycle that can’t otherwise be uncovered.

These market-leading products provide organizations with the infrastructure to automate their software delivery life cycle, removing waste and manual steps from the process. These tools can help organizations streamline their life cycle and extend their DevOps approach. For more information email us at info@tasktop.com.

Written by: Dave West