Friday, November 13, 2009

It's still a *bug*, and your denial does not change that

UGH!! I've had this frustrating discussion with a developer recently that is really annoying the heck out of me.

In the particular scenario being discussed, this product has a history of not installing correctly. It's a (mostly) .NET desktop suite of programs with a central processing engine. The most common installation errors involve components not registering correctly. The crazy thing about that type of issue is that it manifests itself, for this particular product at least, in fairly inconsistent ways. Sometimes, a user function doesn't work (while sometimes it does!). Sometime, the processing engine locks up unexpectedly.

In the past 4 months, I am certain that I have seen the number wasted man-hours troubleshooting something that ended up being an installation problem climb well into the hundreds. In QA alone, I can cont at least a week, times 3 people. I'm already at 120 -- not counting support, developers, product management ....

There was a customer issue that they had spent the past 2 weeks troubleshooting, only to find out that a file had not been registered properly. So product management asked the developer the following:

Is there an identifiable bug in the installation code that we could track in Bugzilla for inclusion in a future release? Or is this issue a one-off?

And the developer's response was that there was no bug in the installation code, so there was nothing to fix. He went on to explain that the files has just not registered properly or were corrupt. He said that uninstalling the suite should have cleared the problem. (I don't want to copy his response, just because it's fairly product-specific.)

So this was my response to that (I was actually fairly nice in it, because seriously, it drove me *crazy* to see that "it's not a bug" statement!)

Though I can see where there is not a clear bug in the installer, I don’t think that is the same as “we are missing something and should implement some checks”.

It just really seems to me that *any* install that we do should be checking itself, that it installed correctly. I have seen dozens of man-hours wasted in just 4 months, on what came down to things not installed properly.

“It didn’t tell me that it didn’t go right” is a bug, in my book.

Now seriously, let's look at this issue.

If the software does not install properly, that's a bug, right?

If the software fails to install properly, AND does not indicate to the user that *anything* has gone wrong, that's a bug, right?

If this install problem causes the software to not work, that's a bug, right?

Is it considered okay to ask the customer to uninstall and reinstall their product (no trivial task, mind you) when it's not working? Okay, anybody but Microsoft?

Has 10 years of testing led me way far astray from the basics? Am I way off base here?

Lanette Creamer suggested that the developer was right; that it wasn't a "code bug", but rather a requirements failure. This is a valid point, but for some reason, I hate having to "frame" things in a certain way to placate a defensive person.

The bottom line is that this product is doing something it should not be doing, and it is impacting the customers, the developers, the support people, the QA people, the product people ... IT SHOULD BE FIXED. Seriously, don't throw blame, don't deflect attention away from yourself.

Let's instead sit down together and figure out where the problem is and how to solve it.


DT said...

I think you've hit this on the head. It's a (serious) bug to silently fail to install correctly. As you imply by your last paragraph, the semantic argument about how to classify the problem (whether to use the term "bug" or not) isn't really a useful activity.

Even assuming (for example) that the failure was due to a third party installation mechanism, there's a quote that applies here: "Just because it is not your fault does not mean it is not your problem."

Markus Gärtner said...

Michael Bolton taught me to continuously keep on asking "Problem or not a problem?" In your case I would perfectly claim that you got a problem (installation not working), your developers got a problem (need to pinpoint bug to be based on a faulty installation) and your project manager (may we ship this?).

Another thing I noticed during the last paragraph. Weinberg claims to distinguish between the origin of a failure (requirements in your case?) and the place where the error shall be fixed. This is not necessarily the same. For reasons of work-arounds it may be appropriate to change the documentation to be able to mark a bug as a feature - though I never ran into a situation where this was the outcome.

abby said...

ugh, with apologies to Lanette, I really hate the "not a bug, it's a requirements problem" - seriously, this is why we need to stop looking at things from a waterfall view of the world. We are a team that works together to design & deliver software.

That software either works for the customers and makes them delighted, or it does not. If it doesn't, it's a problem, whatever you want to call it.

Sorry to hear your frustration - I feel for you!

Angus said...

Take a bug colored and a feature colored index card, cut them in half, and tape two different colored halves together. Write the task on that card. Don't forget the ':P'

AngryPoodle said...

Arent' we trying to deliver working software? Your problem description just does not fit into the definition of working.

To me it sounds like that there's a plenty of culture of blame. Otherwise there wouldn't be the need for avoiding responsibility.