When a Good Agile Transformation Goes Bad

I was recently looking at Digital.ai’s State of Agile report and many companies are still choosing to adopt agile to help deliver a better quality product to their customers faster. There’s been a steady increase over the years of companies adopting (or adapting) agile methodologies. Obviously, the COVID pandemic has had a significant impact on some traditional models, but not everyone is experiencing success. Why would a good agile transformation go bad?

Looking At Data

The first thing I thought was interesting in the report was that many companies – 43% – reported “cultural clashes” hobbling or even crippling their transformation efforts. And 42% reported “general organizational resistance to change” as a major factor. In my experience, these tend to almost be the same thing. The cultural or organizational resistance is rooted in what Commodore Grace Hopper called “the most dangerous phrase in business”: That’s the way we’ve always done it.

Agile transformations require often significant changes to “how we do it” to be successful. They require transparency; a commitment to finishing, not just starting; moving work to teams, rather than moving people to projects; and how we think about success. Many “advanced” concepts like psychological safety and a “safe to fail” environment are far beyond just adding a daily standup to your calendar (which 87% of companies use). PS – if all you’re doing to “do agile” is adding a daily standup/Scrum meeting, you’re not “doing agile”.

An Agile Transformation Is Hard – And Broader Than Most Think

One challenge that I see with companies that try to “go agile” is that they think that agile is a development-only activity. They envision software development as a giant machine in the basement with a hopper for their requirements documents (and money) and a big handle on a crank. They turn the crank and 6-12 months later, out pops their project. They think agile is just a new handle for a crank. It’s a nice handle – carved wood with mother-of-pearl inlay… It’s a nice handle. But it doesn’t change how they think about it.

But when teams change to agile ways of working, how we interact with the machine changes. When these people come down to the basement with their requirements and their money, the machinery is all different. The cover is off the machine! The crank is gone (along with the handle)! There are these switches and dials and gauges and toggles and displays! AND THERE’S NO HOPPER!

This is because we don’t just want someone dropping off their requirements and going away. We want them to help guide the development. To help us create the most valuable product we can in the time we have by telling us what the next most valuable thing is. But that means engagement throughout the lifecycle and too many people are “too busy” to spend all their time asking us to build things in small chunks.

What this means is that more than just the “dev teams” need to be engaged in an agile transformation. It needs to involve management, the business, sales, marketing, HR, finance – the whole works. If you want a successful transformation and get the most benefit out of “doing agile” (let alone “being agile”), you need to engage more than developers and testers.

Why an Agile Transformation Fails

Instead, working in agile really focuses on the concept of incremental delivery. Working on small pieces, finishing those, and moving on to the next. And when you’re “done” with something, you’re done. It’s not “well, it’s done, but it’s not DONE-done”. There is no “done-done” in agile. When we’re done, it means we never have to look at that again unless we’re asked to (or we choose to for refactoring or some other technical debt items).

So when we look at Digital.ai’s report, the two items that stand out to me are the cultural clashes and the general organizational resistance to change. In my experience, the resistance tends to come down to either believing that agile has a somebody else’s problem field around it or that the people being asked to change don’t know what they’d do if we actually do things the “right way”. One thing I don’t think people tend to realize is that doing some Scrum practices doesn’t mean you’re doing Scrum. You can add a Scrum/daily standup meeting. You can work in two-week increments. You can have retrospectives. But you can do all of that in Waterfall. Yes, you can timebox Waterfall.

Most failed transformations I see tend to fall into the category of “lack of organizational buy-in”. Meaning, that the people in charge (also known as management) think this is an “IT only” thing and doesn’t require them to make any changes. The problem is – it does. Most traditional models use management as the conduit for bringing work to a team. But in Scrum, that’s the job of the Product Owner. Most traditional models rely on the manager telling the team what to do and how to do it. In agile, the team is “self-organizing” and finds solutions on its own. Most traditional models rely on “fixed date/fixed scope” project-based models, especially for funding. In agile shops, teams work on the most important work to the business, regardless of what “project” is funding it, and they work on the most important items to deliver the “most valuable product possible” in the time available, not a fixed scope.

Many folks in management, and other functional areas, don’t know they need to make changes or aren’t willing to make changes. And the transformation falters and is stuck in simple timeboxes trying to “get everything done”. Studies have shown that the idea of “fixed date/fixed scope” projects tend to fail about 9 times out of 10. Meaning you have a 10% chance of success. I’m not sure about you, but I wouldn’t play a game in Vegas with those odds. So why are companies putting millions of dollars on the line for those odds? Because that’s the way we’ve always done it.

So doing the form of agile doesn’t mean you’re actually following agile values and principles. So what can you do?

Getting Your Agile Transformation Back on Track

The first step is to recognize that your agile transformation is off-track.

• Does your organization still budget projects and move people to work?
• Do you have fixed-date/fixed-scope efforts?
• Does your organization list “conflicting priorities” as a major issue?
• Do your teams have to work long hours/nights/weekends to “catch up”?
• Is your software buggy or has low quality?
• Does your organization use annual performance reviews focused on individual contributions?
• Have you been told by every agile coach you’ve hired that you have problems?

Okay, that last one is a little snarky, but many times outsiders can easily see what we can’t because we’re so accustomed to it that we’ve become “organizationally blind” to it. There are steps you can take to start identifying where you want to make investments to start righting your organizational ship.

The first thing is to recognize that to be successful with an agile transformation means that your organization looks beyond just development and IT. To maximize the benefits, you need to engage the whole company. “General organizational resistance” tends to be more than the IT teams – it’s the rest of the organization. So making sure your executives and managers understand what you’re trying to achieve, having them understand the changes they need to make, and getting their buy-in to support those changes is critical. This is why I see almost all transformations fail.

Secondly, our culture drives our results. If you’re not happy with your results, you have to change your culture. And “cultural clashes” are 43% of the challenges with adopting agile, per the Digital.ai report. And our cultures support the way we’ve always done it. But if we want different results, we need a develop a different culture. And that means different behaviors. There’s a great book called Change the Culture, Change the Game that gets into some of this. The Experiences shown at the bottom of the pyramid to the right are the behaviors of our managers and business. And that drives our organization’s beliefs, actions they take, and the results they produce. It’s all about doing things differently – and that means everyone.

Third, you should be looking at if you are structuring your teams and your work properly. Agile recommends long-lived teams with members that can design-build-test all of their work on their own. (Some models recommend adding “deploy” as well, meaning teams should have someone from Operations on them.) In general, cross-functional teams, not siloed, are more successful. And teams that work together for a long time are more successful than those that aren’t (due to Tuckman’s model).

And finally, you should enlist the help of someone who has experience in helping companies with their agile transformation. You’re going to also likely need someone who has experience in executive and management coaching and can look at your organizational problems holistically.

How Can Artemis Help?

Our coaches have years of experience in agile transformation, executive and management coaching, team and agile coaching, and organizational change management. We work from where you are and craft a roadmap for how you can achieve the results you want and your business needs. Please contact us for more information about how we can help you.