Bug-finding

[Omega Supreme]

I test software on set-top-boxes. I'm weak at finding problems, issues, and bugs. I mostly run test scripts and procedures, basically spec-testing. How can I become a better end user and ad hoc tester, so I can increase the number and quality of my bugs. I don't want to lose my jobs. Being able to run test procedures is fine, but bugs are where the money and rewards are.

 

[(*steve*)]

Do you have access to any test equipment?

 

[Omega Supreme]

Right now I'm just testing the software, the user interface. So I guess it's just black box testing. I have access to the test procedures a specs, but the vastness and complexity of the code makes finding bugs difficult.

 

[(*steve*)]

So what are you doing? Are you trying to find bugs, or do you have to fix them?

If you are just trying to find them then you need a very good understanding of what the application is supposed to do, the various types of faulty input that it can be subjected to, and an understanding of the correct response to them.

You probably also need to have a methodical approach to testing invalid user inputs, which means making sure that inputs which are unused or incorrect at each point do not result in unexpected behaviour.

 

[Omega Supreme]

I need to find bugs and write detailed reports including steps to reproduce. Software development fixes them. I guess it's hard because the spec documents are so long. It's difficult to memorize and understand everything. Bug-finding seems like looking for a needle in a haystack.

 

[(*steve*)]

Part of your job is to read and understand the specs. The more you do so, the more you will be able to perform testing.

It's only like searching for a needle in a haystack when you don't understand the software, or when the software starts getting really good.

 

[KrisBlueNZ]

In my (limited) experience, most of the probems in set-top box firmware show up when it's being stressed - trying to do lots of things at once - or over a period of time - for example, file system and disk space allocation bugs where the bug causes incorrect information on the disk but the problem doesn't show up until later when the disk gets closer to full.

 

[chopnhack]

In my (limited) experience, most of the probems in set-top box firmware show up when it's being stressed - trying to do lots of things at once - or over a period of time - for example, file system and disk space allocation bugs where the bug causes incorrect information on the disk but the problem doesn't show up until later when the disk gets closer to full.

From an end user perspective, I have noticed this frequently, especially on multi-terminal, distributed databases. The more file access requests occurring from multiple points the slower the system responds. Try accessing the same records from multiple terminals as well... some systems when saving to records at multiple locations causes errors. See it all the time with some of our lousy software. As the system has less swap space, the que becomes full with requests to the point that it seems to the end user that it has bricked!

 

[Omega Supreme]

How do I learn and master a ton of spec documents and test procedures? Is it just a matter of reading them over and over again?

 

[KrisBlueNZ]

Become a user of the product. Try not to consider it a box that you need to test; think of it as a tool to do jobs that you want to do. Real life will create use cases that you would never have dreamed up on your own. If possible, become familiar with similar products from other manufacturers. This will also help you find usability issues that aren't strictly bugs.