Today, Jumbo is committed to distinguishing itself through an omnichannel mindset and aims to be an online market leader. To achieve that goal, Jumbo has adopted Agile development practices to deliver new features to customers faster. With a growing number of releases per year, manual testing was insufficient to keep pace with the rate of change and maintain the quality that Jumbo requires. IT leaders at Jumbo knew that implementing test automation was one of the first steps to improving test quality and providing more confidence with each release.
“We have to complete regression testing with every release, which takes two weeks. But in the past, a single regression test cycle took more than two weeks,” says Hanno Koeslag, Development Team Lead.
As a first step, Jumbo’s test specialists got together to develop a list of criteria required in an automation tool to enable fast, high-quality testing for each release across the multitude of technologies in the Jumbo landscape. Their list focused on no-code automation that could be easily facilitated by the existing testers within the Scrum teams, low test maintenance, and the ability to automate multiple technologies (including SAP and non-SAP applications) for complex end-to-end testing.
Faster and more efficient testing solution
A search was initiated to determine which tools on the market met Jumbo’s stringent criteria. After narrowing down the list of tools, the various automation options were put to the test. A Jumbo-wide decision was made to implement Tricentis Tosca as the preferred test automation tool.
SUPERP (Tricentis partner) assisted in the selection of testing tools and was also chosen to accelerate the testing automation efforts. With its in-depth expertise in building and testing powerful business solutions based on SAP software and its knowledge of the Jumbo system landscape, SUPERP is the ideal partner to further support Jumbo in this endeavor.
Jumbo currently runs two SAP systems. The larger, highly customized SAP ECC system is the centerpiece of its larger operation in the Netherlands. SAP S/4HANA is a new instance that controls operations in Belgium. Jumbo is in the process of migrating to a single S/4HANA system for both countries. In the meantime, both systems need to be regularly updated and tested, side by side. Their Tosca tests run on both ECC and S/4HANA, making testing easier to maintain as the migration progresses.
Test automation has helped the Jumbo team to quickly realize a return on investment. “We saw a 46% reduction in the effort required to resolve production errors. The actual cost savings from not having to spend time resolving those issues exceeded the time we invested in building the scripts in the first three months,” says Koeslag.
The unexpected value of test data management
Early on in their test automation journey, Jumbo discovered an unexpected benefit of their test automation. Previously, the master data team regularly received requests for test data from various Scrum teams. This test data had to be generated manually and required a considerable amount of time and effort to produce. With the new test automation scripts, test data was automatically generated that could be used later in the process to populate test scenarios.
“A Tosca article can be created and stored on the test data management server, where another process can use that same data to perform tests and report back to the server. This process can continue and pull the same data through the entire supply chain,” says Peter Groothuis, Scrum Master & Test Analyst.
