Helping an Indian e-commerce unicorn with performance and scalability testing

QA Process Setup and Manual Testing

After analyzing the objectives, schedules, resources, estimations, and deadlines, a test plan was generated to identify the right approach for testing. A test strategy was prepared to define the roadmap to achieve the desired performance of the application along with on-time delivery. Following the Agile framework for testing, the QA team created a test suite of over 4000 test cases to verify the functional requirement of the application.

To ensure that the progress in development does not impact the stability of the application, regression and sanity test cycles were planned. Regression testing was performed to check any dysfunction in UI and features after every update. Sanity testing, on the other end, checks for the stability of new functionality or code changes in the existing build.

Load Testing and Reporting

Both the platforms- shoppable video hosting and the Q&A were expected to receive a high load of concurrent users during the sale and live expert sessions. For this, the application was load tested for functional behavior and system capacity in a simulated environment.

The load test was performed on the entire workflow to record the server breakdown time. This was done through API load testing using Apache Jmeter (for around 10,000 concurrent users). After multiple rounds of load testing, reports were prepared and shared across stakeholders for further analysis and identification of next action items.

Report Analysis and Load Balancing

Based on the results of load testing, the generated report was shared with the DevOps team so that the right approach could be adopted for load balancing on the server.

Through load balancing, the DevOps team ensured that the concurrent user traffic coming to the network is evenly distributed across multiple servers. This maintained a seamless flow of information between the servers and the endpoint device.

With the right load balancing technique, the resources on the server were optimized to manage the fluctuating demand during peak traffic times. To achieve the most favorable results, server optimization was done several times. Moreover, some of the highly utilized features of both platforms were continuously analyzed for threads/ramp-up time using Jmeter.

Automation Testing

To reduce the time to market and accelerate the regression cycle for frequent releases, the critical features related to testing cases from the regression suite were automated using Selenium. The test scripts were written in Java and were scheduled for auto-execution through Jenkins.

After running these test cases, a report was generated with screenshots having details such as the number of test cases executed, number of cases that passed or failed with reason. Automating the regression suite not only reduced the chances of error leakage while testing but also cut down the testing efforts of critical features by 75-80%.