Chances are you do not even know how to answer this question. Few organizations have the infrastructure to properly measure their transformation, even fewer still can use measurements to effectively steer their transformation. Where should you even start?
Navigating this seemingly endless ocean of metrics can easily leave you feeling hopeless, making it really tempting to just move forward without direction hoping you will arrive at your destiny sooner rather than later. However, not everything is hopeless: recent findings in academia can offer some guidance in this frustratingly complex process.
The first thing we can say thanks to these findings is that these transformations do seem to make an overall positive impact. This might seem obvious but in the middle of the chaos of a transformation, it is not uncommon to forget why you even tried such a thing in the first place. Various types of studies all confirm that there is at least some positive impact in terms of productivity, responsiveness, quality, workflow health, employee satisfaction & engagement, and customer satisfaction (Stettina et al. 2021, Olszewska et al. 2016, Laanti et al 2011).
These dimensions of performance are great but how do we go about measuring them? Luckily there are easy ways to get you started with this. What’s more is that you likely already have the infrastructure in place to make some of these measurements, even more so if you are using some kind of backlog tool like Jira. Here are some examples of simple measurements for a few of the dimensions mentioned above:
- Productivity: the number of features you are able to ship to the customer in relation to the amount of money you are spending.
- Responsiveness: the time it takes teams to resolve issues. For the portfolio and program level, you can look at epics and features.
- Quality: track how many of your issues are bug reports as a percentage of your total amount of issues.
- Workflow health: the average story size and the number of issues resolved.
- Employee satisfaction & engagement: the number and distribution of people you are assigning during the flow.
- Customer Satisfaction: although not directly measurable through backlog tooling, it is too important not to mention. Try to ask for your customer’s opinion any time you interact.
Of course, these metrics do not represent your entire transformation but they can give you some initial insight. A large-scale agile transformation is a hugely complicated process, there is no easy solution to guide you through every step. Most importantly, these metrics may provide a starting point for measuring your transformation with a low barrier to entry (or maybe get you thinking about other ways to leverage already existing infrastructure).
From this starting point, you can start having a discussion about your next move as well as give teams transparency on their own performance. If these metrics end up not being right for your context, there is still a lot to be learned from simply initiating that discussion with each other.
If topics like these also fire up your enthusiasm, join us at our workshop: Understanding the impact of scaled agile transformations, May 17th! Here we will talk about new academic findings and share practitioner experiences with the aim of furthering our collective understanding of such transformations.
And if you are interested in this topic, the articles referred to in this column are:
- A case for data-driven agile transformations: Can longitudinal backlog data help guide organizational improvement journeys? (Coming this summer)
- The Impact of Agile Transformations on Organizational Performance: A Survey of Teams, Programs and Portfolios (Stettina et al. 2021)
- Quantitatively measuring a large-scale agile transformation (Olszewska et al. 2016)
- Agile methods rapidly replacing traditional methods at Nokia: A survey of opinions on agile transformation (Laanti et al. 2011)
Een bijdrage van
Christoph Johann Stettina: Christoph is a transformation lead and researcher with experience in a wide range of business agility subjects and a passion for connecting strategy to execution. He is very interested in agile and iterative methodologies, and the human aspects of work. For over 10 years, Christoph has been advising organizations with a strategic IT capability in (co-)designing and (co-)implementing business agility across various industrial sectors, regions, and maturity levels. Focusing on helping organizations to connect effective program delivery to product innovation and strategic portfolios, he helps clients in adopting new ways of working in an effective, non-threatening manner. Being an assistant professor at Leiden University, his work on agile portfolio management, innovation management, governance, and entrepreneurship has been published in more than 30 academic and professional magazines and books.
Gijsbert Boon: Former CTO and owner of the e-learning management company, the market leader in learning solutions in the healthcare industry. Since 2014 Agile transformation coach in various industries; from gaming companies to financial services; SME to large organizations. Real interest in “the million-dollar question”: is your Agile transformation successful? Currently doing Ph.D. research on this topic.
Tim Poot: Tim is a Master of Science student at Leiden University. After completing his Bachelor of Science in Computer Science he is now broadening his knowledge with the master’s program ICT in Business. He is currently writing his thesis at KPMG where he can leverage practitioner knowledge to better understand the impact of large-scale agile transformations.