top of page
Writer's pictureVishaka Warnakulasooriya

Case Study 09: Ensure your software is truly ready for anything through comprehensive Edge Case Testing!




Project Background: Developers will verify the main flows and if the test engineers haven't verified the rest with edge cases, then edge case bugs will be detected by your customers!


90% of edge cases are not brought to the floor via business requirements because it is not possible. Edge cases are those rare, unexpected scenarios. Comprehensive edge case testing ensures that your system is ready for unexpected user behaviors, and flows. Users might input invalid data, navigate through negative paths, or alter the workflow sequence in unintended ways. This can lead to major issues that aren’t visible until after the product is in the hands of the end users.


For this case study, the project was a web application of the Telecom and Finance domains. The key functionality was involved with receiving the customer's payments via various options including online transfers, bill payments via banks, cash payments, and payments via different supermarket outlets. Then all these transactions need to be reconciled daily and monthly. Per week, the daily reconciled amounts should be transferred to the Telecom service provider's bank account. Finally, the monthly transactions should be reconciled with the bank reports.


Reconciliation processes were automated via robotic process automation and it was the system solution of this project.


While the test team ensured that these key functions worked accurately, there was increasing concern about how the software would behave in rare or unexpected scenarios, known as edge cases.


As it was a complex and risky system that was associated with the transactions with Telecom service providers' customers' money, the testing team had to think beyond expectations to identify what would go wrong, which points it would break, which user transactions would make the failures, etc... Generally, that identification comes through the test engineers past experience.


Strategy: We understood that nearly 90% of edge cases are not identified in standard business requirements, as it’s impossible to predict every rare user scenario. Apart from ensuring the main functionalities, our strategy was to prioritize testing these unpredictable edge cases, which included:


  • Unusual user behaviors

  • Extreme input values

  • Uncommon system states


Business requirement identification with the discussion with the respective staff members:

Ensuring that the test team accurately identified the business requirement was the first step of our strategy. Even though this is the first part of every project, here we gave more weight to this step. The team scheduled discussions with the in-charge staff members of the banks, supermarket outlets, and telecom service providers who handled the payment collection and reconciliation processes. Through them, the team identified practical issues they have faced. It was a great support for the team to have a big picture about the business requirements without limiting to the documentation.


Creative Test Design:

We applied advanced test case design techniques like mind maps to systematically explore and identify all potential scenarios that could disrupt the application’s functionality before moving to the Test case designing phase. Through mind maps and decision tables, we were able to visualize a vast number of potential test scenarios that the software could face, far beyond what traditional testing would cover. Those designs were reviewed by the lead engineers and Test cases were derived accordingly. This approach ensured the test coverage for all positive, negative, and exceptional scenarios.


Challenges & Solutions: Apart from verifying invalid user inputs like what should be the system's behavior against payments for invalid telephone numbers, the team was more concerned about the negative and exceptional flows of the reconciliation process's algorithms and logic.

  • Because even though the way of reconciliation process was the same it has been done in different ways according to the customer's payment collection processes' of different banks and supermarket outlets. Therefore, verifying the reconciliation process with different payment collection processes was one key challenge. The test team had to work with many data mappings and it was not an easy task that required an additional effort for test data management.

  • When it comes to the payment handling processes of the banks and other centers, they deposit money to the Telecom service provider's account per 4-5 days. Let's say it is 5. If there were no continuous 5 working days due to holidays, there were many edge cases to consider. Because all of the holidays did not apply to every payment collector. For example, a Bank holiday is not a holiday for a supermarket outlet.


  • The other thing is the same customer had more than one phone number and some of them used to make payments via different methods. When it comes to the data mapping with the phone numbers, customer names, and payment methods, it was a challenge.


  • Tallying the actual money receival with the transactions was another challenge. Therefore, the many challenges involved with numbers and dates.


  • As this was automated processes through Robotic process automation, the test team had to be conscious of the accuracy of the UiPath setup and its flows.


Those were the key challenges of the unpredictable nature of edge cases. These cases are not part of the standard business flow and are often missed during typical development cycles.

To overcome this, the team followed the above test strategies by using test case designing techniques like mind maps to ensure full test coverage. And having discussions with the ones who handled the payment collection and reconciliation processes gave us good confidence about the business requirements.


As this dealt with huge data loads, performance aspects also were important. However, it was handled by another party.



Results: The results of our comprehensive edge case testing were transformative. By identifying and addressing bugs in these extreme conditions before the software was released to users, we:

  • Did not get critical and high UAT bugs and it proved the accuracy of our part we did.

  • Enhanced the overall robustness of the application, ensuring it could handle even the most unlikely user behaviors and inputs.

  • Significantly reduced post-launch maintenance costs and downtime, as fewer emergency patches were needed to fix issues reported by end users.

  • Improved user satisfaction, as the application was now better equipped to manage unpredictable user actions without crashing or malfunctioning.


Conclusion: By focusing on edge case testing, we helped the client move beyond standard testing practices and achieve a higher level of software resilience. Comprehensive edge case testing ensured that their application was ready to handle the unexpected, minimizing risks of post-launch failures. Our systematic approach, using advanced testing techniques and a good understanding of complex business requirements, provided the client with the confidence that their software could operate smoothly even under the most challenging and unforeseen conditions.


Edge case testing isn’t just about finding rare bugs. It’s about ensuring your software is prepared for the real world.



Recent Posts

See All

Comentarios


bottom of page