What’s an Agile Defect?

One question that should get tossed around by teams and coaches alike is “What is a defect?” It’s a hard question to answer; there are a lot of variables to consider. It’s a little esoteric and has been something I’ve worked with several coaching groups on defining, but it has a significant impact on how we think about development and track issues. From a pure coaching perspective, I think I have a good proposal for “What is an agile defect” and from that, we can figure out what isn’t. Hopefully, this proposal gives you some food for thought and helps inform your discussions with your teams and partners.

First Things First

Back when I first started in Agile as a developer, we would record any problem that we found at any point during development as a defect. We would sometimes fix those issues before our user stories were done and accepted. Other times, we would be unable to fix it and we’d keep track of it in our tracking software (this was long before Jira or Rally…). So any issue we found at any point while we were in development was a defect. But is that a good definition for a bug, an agile defect?

Times Are A-Changing

After I got introduced to Scrum as a methodology (which was more than a decade after I started doing Agile development…), I was talking with a few other coaches about this question. With their teams, they had recommended we only track escaped defects. In other words, only those problems we uncovered after the work met the Definition of Done.

I was a little shocked. How on earth would we track the problems we found during the process? How would we know what was missing?

The reply was succinct and floored me: “It will prevent the story from being accepted or not.” Mind = blown.

For any story, the Product Owner decides if we are going to accept the story as it was built (issues and warts and all) or they won’t. If it’s big enough – either a miss for a requirement or an issue with how it performs (maybe it’s slow), the PO can say, “Nope, you have to fix this before I can accept it.” Or they can accept the story because they determine that the issues with it aren’t that bad. In that case, they may add an additional story to capture the missing functionality or resolve the issue.

So What’s an Agile Defect?

The tl;dr (too long; didn’t read) version is:

An agile defect is any problem we find after a story/feature has met the Definition of Done.

This is a pretty straight-forward delineation. It means that any time an issue is found with functionality after the team is “done” with the work, that’s a defect. Anything we find before the work meets the DoD is not a defect. It’s something that is either not significant enough to prevent the work from being accepted or it is big enough to prevent the work from being accepted. It’s a pretty safe and clean delineation.

Challenges to This Definition

But because everything in Agile tends to follow an “it depends” interpretation, there are going to be some challenges to this approach. The main one I see is around separate teams that do Quality Assurance. Whether they are third-party vendors brought in to validate work done by the teams or they’re a separate organization within a company tasked with finding issues, these groups tend to be tracked on the number of issues they find. These are lazy metrics to track, but someone decided long ago that “issues found per sprint” (or some metric) is a good way to evaluate the effectiveness of these groups. As I said, these are lazy metrics (there are far better ways to do this), but the consequence is that these teams are motivated to find as many issues as possible.

In the short-term, it means that a lot of issues are going to be resolved as “works as designed” or “not a bug” or something similar. In the long term, it creates an antagonistic relationship between the teams and the QA team – something we are constantly striving to avoid!

Even when the “testers” are on the team (which isn’t a Scrum construct – check the Scrum Guide!), where they report in the organization may push for classifying errors or missing functionality in work that’s “in progress” as defects. In this case, we need to educate the teams and the organization. Collaborate with those managers. Find better metrics for tracking effectiveness. Remember the Agile Manifesto – “we are uncovering better ways of delivering software”.

Find those ways.

Experiment all over the place.

Don’t just do better, BE better.

Closing Thoughts

The definition of an agile defect shouldn’t be a difficult thing to define. Far too often, though, the philosophical debates get in the way of experimenting with what works for your organization. I would caution about tracking every issue before work items meet the Definition of Done, however. You may look like you’re finding and fixing a lot of issues, but all you’re likely doing is spending time tracking busy work rather than delivering actual value. And that’s the main goal, isn’t it? Without delivering value, there’s no team and no defects, no matter how you classify them.