Bug Tracker Abandoned

Our customer maintains they own trouble ticket system. We used to create tickets in our own system to mimic bug assigned to our project. Then the customer switched trouble ticket systems on us. Our process broke down. Now we no longer maintain a separate trouble ticket in our own system.

We hardly use our own ticket system any more. This is a problem. It is almost impossible to determine the history of how issues have been worked by development now. Some managers seem to be resorting to Microsoft Excel spreadsheets. It is like the stone ages up in here.

I miss the days when I could just fire up my Rational ClearQuest client and figure out everything there is to know about a problem. Now you either need to keep up with all the work going on. Or you need to find some spreadsheet and hope there is good information captured in there.

Looks like we have regressed and moved in the wrong direction regarding capability maturity. Let's hope we don't abandon source code control as well. Then all hell will break loose.

Tracking More Than Bugs

I saw an ad from Alexsys touting the benefits of their Alexsys Team software. It not only tracks bugs. It also tracks team tasks. The thing is supposed to be highly customizable. I guess these tasks can be thought of as action items. You can also track features in addition to bugs.

This software does "trend analysis". I wonder if that means it automatically produces metrics. I used to hate to manually compute monthly metrics on defect closure rates. In the end, I talked my manager out of requiring such metrics gathering.

Looks like Alexsys provides a free trial. My customer is really intent on my team using Clearquest from IBM. So my hands may be tied. However the customer usually does not care what we use internally. Maybe it is time to try something new.

Useless Retests

Testers have been recently tasked with testing the fixes that application development has created. They are finding that most of the bugs are still broken. I found this disturbing. So I took a look at a couple bugs that had been assigned to me. One of them had not been fixed yet. No wonder it still did not work. Another had been fixed. However that fix was not released to the test team.

This conundrum points to some kind of disconnect. I would be unhappy if I were on the test team now. They are wasting their time due to some incompetence. Somewhere in our process there is a severe lack of coordination. This responsibility falls on the shoulders of the team leads.

In theory, our process should prevent such nonsense from happening. One of the problems on our project is that we seem to have very little process. Not a lot of it is written down. If it is written down, nobody seems to know where it is. Or the stuff that is written down is obsolete, incomplete, or downright wrong.

Our bug tracking tool is supposed to help with this mess. You would think that you would change the state of a defect to “ready to test” once it has actually be fixed and delivered to test. I got the feeling that the team leads are just telling the test team to retest everything. A lot of the senior guys are saying we need to get this process under control.

For now I am staying out of this mess. I have found that on this project, you usually get the privilege of fixing problems if you bring them up. When the pain level gets high enough, I will step in. The last time I did this, I ended up serving as team lead for a couple years. That is a thankless and difficult job indeed. So for now I am just writing this rant to all those who read my blog. Enjoy.

Mozilla Bug Reporting

I read an entry on LiveJournal by Zwol entitled “On the Unfriendliness of Bug Reporting”. Zwol wanted to input a new bug he found in Firefox. However there were a lot of confusing points in the bug submission process. You have to decide which component to file the bug under. It is not intuitively obvious which option to choose. Some large categories already have a lot of bugs under them. There are other strange choices on the bug submission screen that have users giving up on the process.

Another part of the problem is the sheer number of fields on the bug report screen. Sure you want to add all the information you have. Zwol recommends a special developer mode for the bug reporting screen. That way the devs can see all the fields they need, and the average user can just enter the simple information to get the bug submission done.

The Mozilla team replied in the comments section of the blog on LiveJournal. They said they are working on the problem. A group of students have been assigned this task. That sounds good given the fact that they are an open source project. Perhaps they were already thinking about improvements to the bug submission screen.

On my project, you also have to go through a lot of hoopla to submit a bug report. New testers have a lot of trouble getting this right. The result is that bugs get lost of channeled to the wrong developers. Our bug submission screen is not only large, it has many screens to boot. This does have some advantages. When you know what you are doing, you can add a lot of important information about a bug. You can attach screen shots of the problem. And you can really specify a lot of characteristics of the bug.

I would think that a product made available to the general public at large should have a much simpler bug submission page. Being an open source project, I bet they roll their own open bug tracking system. And they probably developed their own bug submission screens. Who is the developer going to write those screens for? Unless care is exercised, it is probably not geared toward the casual home user who finds a problem. Maybe it needs to be.

Customer Trouble Tickets

A trouble ticket tracking system may have a short term goal of keeping tabs on software defects. However this is usually a means to another end. You don't track problems for the heck of it. You do it to ensure the users of your product can do their jobs. The Software Maintenance blog has a story about and developer and his trouble tickets.

This developer is staying proactive in making sure customer problems are being solved. Too many times a problem falls between the cracks. It takes an active effort to make sure all problems are prioritized and worked. A bug tracking system can help with this. But you need a process built around it to ensure the best results. You also need some level of diligence in the users of the tracking system.

Microsoft Excel

Normally we have an orderly and well defined process for tracking software defects. If the customer detects the problem, they enter the problem in their proprietary bug tracking tool. Then the help desk team enters this information into ClearQuest for us. A team lead assigns the ClearQuest defect reports to developers. Developers fix the problems. ClearQuest gets updated. Testers and then the customer gets the fix. All during this process, the Help Desk monitors progress by checking ClearQuest.

Currently there is a lot of test activity going on. Our internal testers are trying to complete a smoke test of our application suite. At the same time, our customer is performing a functionality test. The new change is that the customer is not entering problems they find in their bug tracking tool. Our internal test team is supposed to determine when the customer finds a bug, enter the information in ClearQuest, and forward the information to the dev team. I guess our customer does not want to incur the overhead of their own tracking tool for their testing.

I do not know how the deviations in process started. But we seem to be getting a lot of bug lists from our internal test team via Excel spreadsheet. They also provide a summary of smoke and functional test results to a broad audience. The spreadsheet appears to be going to all developers. And it is causing a lot of confusion. For now I am ignoring these spreadsheets and email because I am working on another task for development. But the other developers are going bonkers. Excel by itself is not a bad product. But using it to manage bug lists can be tricky. The new process seems to be adding to the pandemonium.

Home grown bug tracking schemes can work. But a real bug tracking system is better. Yes there is some overhead involved. However you do not have to worry about what version of the spreadsheet you are looking at. You also can avoid getting a bunch of e-mails with Microsoft Excel spreadsheets.

Relative Priority

Our customer has a system to track all problems that their users report. One property of trouble tickets is their priority. This priority ranges from 1 to 5 (1 being highest) based on a strict set of criteria. We are nearing the end of our software maintenance contract on our project. Most of the problems that get reported these days are priority 4.

Recently our customer headquarters group has been thinking about raising the priority of some problems which are crucial for their success. Their goal is to get the trouble tickets fixed quickly. However this will only lead to trouble. High priority tickets require a lot of status reporting overhead. So upping the priority might actually delay the resolution of certain problems.

I think the root cause of the problem is that there is no way for the customer to indicate the relative priority of problems they report. Our process is to work the issues in priority order. And if two or more tickets have the same priority, we work on the oldest ones first. Another part of the problem is that our software development team is shrinking in size due to the end of the contract. Us remaining developers are doing the work of two people these days.

My response to the customer concerns was to have the developers drop whatever they were working on to concentrate on the few high priority issues. It is my sincere hope that we can knock out these problems and leave the customer in good shape. This will also make my life easier as we try to close out this maintenance project.