Discovering a fact of an application not being tested satisfactorily is probably one of the biggest pains of a Testing Manager and the irony is that, if he spends more time to make product robust, he is labeled as an extravagant Manager. No one likes to understand the fact that releasing the product under-test would result a buggy application, which eventually would cost support and more maintenance. So what is that, which could turn software development a prudential and a quality oriented process. How could a Test Manager achieve optimum quality in lowest possible cost?
I personally experienced, testing a product having 2400 test cases, in 8 different configurations with 3 possible output color spaces and all that on three operating systems…. Huuufff…That makes total of 72 combinations for every test case. I remember, when I informed about it to one of my mates, he sounded cool enough to digest everything and just suggested to implement automation. Was that a real solution? Probably not. We had an excellent tool running day and night, automating most of our test cases. But obviously you cannot leave tool to take all decisions on your part and declare an application good or bad for release. Report analysis certainly had to be manual. Testing one build in every 20 days with automation tool running on two machines simultaneously and report analysis running on third machine, had been a great experience. Now what was that which made all this feasible. Find out below some of those factors which work silently and effectively to let application be tested in time and up to the level of satisfaction of a test manager.
Understand software requirement completely: Most of the failures in software development are caused due to lack of understanding about product requirements. I came across a humorous comic story once which depicted as to what happens if the development team fails to understand the SRS.
Client asked to develop “B”.
Development team understood it to be “B+U”.
They finally created “BU”.
Testers claimed it to be different from SRS and logged an issue to correct it to make “BUY”.
Developers worked hard and created “BUgY”
The final product delivered was “BUGGY”
Understanding the requirements by a tester prevents a lot of futile exercise during testing phase.
Prudential and intelligent decisions are required: A Manager has to be calculative enough to decide as to which feature is most important to be tested and which one could be placed at back foot. There could be certain instances, where testing may not be needed at all. Putting efforts to test static webpages on different OS and database may not be required; though testing them on different browsers cannot be ignored. Based on the decisions taken on what to test and what not to, design a matrix listing all supported platforms on one axis and the features on the other one and then mark as to which features need to be tested on which platform.
Permutation and Combination theory: During my testing experience quoted above, I set permutation and combination in such a manner that my test runs used to cover every configuration to be output in every color space at least once on a particular OS. I could never run all 72 combinations in 20 days. Not even an automation tool could help me in that. So I aimed to run least no. of test runs; covering wider range of my test cases on all three platforms.
Use Historical data: Historical data about the defect types and defects count in a particular kind of projects by a particular developer could help you assess as to what needs more vigilance from testers. Knowing the historical defect trends would enable you to forecast the number of defects that can be uncovered in upcoming projects.
Risk and Mitigation Plan: Identifying risks and to have a mitigation plan ready is very important. What if unexpectedly more defects are found or that a product is released under-test. A well-defined plan helps in calculating the risks efficiently and to redefine your strategies if required.
