Introduction:

Quality is important for any product either to obtain or retain the existing customer base. Especially for a software product, it is much more important and tough to maintain the quality statistics, since it gets updated constantly undergoing all possible changes ranging from a block of code to change in the entire framework. In such a dynamic context, QA(Quality Assurance) plays a vital role to find more bugs and identify problems more quickly, while at the same time delivering increased productivity, faster time to market, and decreased risk of brand damage.

The current Case study – QA for xxxxx.com, gives an insight on how we could deliver the product with almost zero bugs in the website, when there was a change in the framework of the application from Perl to PHP. The team size was four, which consisted one Team lead and three Test engineers(including one; an automation engineer who works solely on Selenium Webdriver). The duration of the cycle was 45 days, which included rigorous testing for two different migrations(xxxxx.ua followed by xxxxx.ru) of the site to the new framework.

About xxxxx.com: xxxxxx.com is a Classifieds website that is number one in Russia. Including Russia, it has presence in 10 different countries. Users can register for free and can post advertisements in varied categories available in the site. Site has paid services like Top ad- to show an ad in Top of the respective category listing page, Urgent label- which ads the label ‘Urgent’ against the advert, Pushup- to repost an advert, Highlight- to differentiate advert from others. The site has more than a million registered users with a dense traffic throughout the year.

Types of testing involved:

• Black box – Manual and automated tests
• Functional testing
• Regression testing

Problem description:

There was a change sought in the framework of xxxxx.com from Perl to PHP, as it has been found that the latter is more faster, easy to maintain and more reliable. The development team was completely replaced by new PHP team, where as the QA team remained as Sunfra and the plan was to implement the new framework as soon as it could be done. This made stringent time lines and the task seemed quite a Herculean.

Development team had to understand the complete functionality of the web site, its structure, as it contains a number of categories out of which, some have varied behavior, implement paid services, migrate a huge database of users/classifieds and so forth. As part of QA, Sunfra’s task is to retain the existing functionality, website design, URL structures, user accounts along with their adverts, ensure the paid services are not changed in terms of functionality and importantly pricing, error handling checks etc., which was still tougher as it is quite important not to allow a minute difference, for example, a missing alphabet. Also, QA team is asked to cross check every URL/title, with that off the existing website built on Perl framework and report them to Development team after confirming from the Product manager.

Sunfra’s solution:

Sunfra analysed the problem and made a plan which covered all the aspects to be considered in order to achieve one-hundred percent quality. The plan included the following aspects:

• Defining entry and exit criteria of testing.
• Preliminary discussion with development team to know the new processes in place for reporting defects, clarifications and escalations.
• List out the applications and sub applications that are affected by the new framework.
• Identify methods of testing, that are apt to the context in order to optimize the results.
• Ownership on the affected applications to each of the resources designated in manual testing.
• Automation(using Selenium Webdriver) feasibility check by the QA Manager involving the Automation engineer and assignment of possible automation scenarios.
• Swapping the functionalties among the testers, after a round of tests.
• Schedule internal team discussions, calls with development team and daily stand-up meets to keep everyone on track.

Execution Phase:

With the above plan in everyone’s mind, as soon as the test version of the site is made available, Sunfra’s QA team logged 400 defects in a span of 5 days, with a negligible rejection rate and the defect ratio was excellent. It was a 45 day stretch that included rigorous testing for two different migrations(xxxxx.ua followed by xxxxx.ru) of the site to the new framework. Listed below are Sunfra’s milestones in execution Phase:

• Ensured that the major functionalities in the site remained unaffected.
• No page present in the site that takes the user to error pages(This is important since there are pages with very high page rank).
• Ran automation scripts to crawl the old site which collected 2000 URLs to ensure that they still work(redirect to the new ones or remain same).
• Verified whether emails and messages sent/displayed to the user are same as before.
• In time action taken on fixed defects and provided clarifications on defects, required by development team.
• Regression and adhoc tests as soon as the site went live.
• Repeated above steps for the migration of xxxx.ru site as well, which was followed by migration of xxxxx.ua

Conclusion:
Outsourcing it’s testing activities to Sunfra, xxxx has seen increased customer service levels, ROI and many other important business metrics. Also, Sunfra’s automated software testing efforts have expanded testing coverage and effectiveness while avoiding significant additional expense.