Software Testing Case Studies
"This is simply brilliant work. Your team has found the problem where other very competent people have not."
Owner, e-commerce website
Email Testing Case Study
The Brief
A market-leading blue chip company providing an email information service found that 30% of the clients paying for the service could not read the emails. Not only was the company's revenue in danger but more importantly so was its reputation. Test Partners were tasked with resolving the problems.
Our Approach
Customer feedback indicated that a wide range of email clients were in use and that the emails did not display or function correctly with many of them. The emails were multi-part and faults were occurring with both the HTML and text-only versions.
We created a dedicated testing environment including various versions of Domino Server to support Lotus Notes, POP3 mailboxes feeding every version of Outlook and Outlook Express and a range of web-based email services including Hotmail, AOL and Yahoo Mail.
The Outcome
Our previous experience of similar projects meant that several 'quick wins' were possible in the first phase of testing. We created proof-of-concept code to verify the proposed fixes and these were immediately implemented in the live system.
Several regression cycles were required because some faults were masked by others but the final result was a set of email templates that met the company's graphic design requirements whilst being fully compatible with the wide range of email clients used by their customers.
"The level of testing and commitment has been excellent."
e-commerce Manager, high street bank
CD-ROM Testing Case Study
The Brief
A small production company won an order to design and press 1,000,000 CD-ROMs. The rewards were great but so were the risks - the pressing cost was £250,000 and any faults found afterwards could spell financial ruin. When the CD-ROM was complete the company asked Test Partners to verify that it was ready to go to press.
Our Approach
The contents of the CD-ROM were deceptively simple - just a dozen audio and video clips in a Flash-based presentation.
The first step was to set up a range of machines with different hardware and software configurations in order to verify the minimum requirements. The CD-ROM was then run and the key performance parameters such as CPU and memory usage were monitored.
The next step was to verify the quality of the audio and video content and to test the navigation and the functionality of the embedded media player controls.
The Outcome
Almost immediately we identified that the CPU and memory requirements of one of the video clips were excessive, causing even relatively new machines to hang. We then found that the presentation would not even start in Windows 98, which it was required to do. Either of these faults would have caused the CD-ROMs to be rejected by the client.
Several more faults were found with the media player controls and the quality of the media. Four iterations of the CD-ROM were required before it was finally ready to go to press, yet the total cost of testing was still only a fraction of one percent of the project cost.