fr

Test Automation & Shift Left Testing to Reduce Costs

16.01.2019

Every day, bad software quality costs businesses millions of dollars in lost revenue and additional operational costs.

Bad software quality can impact your company negatively in many ways, including higher development costs, rework, bad publicity for your brand, low team morale, unplanned departures and much more.

The «Shift Left» Testing Principle

We can all agree that excellent software quality is good for your clients, for your team and your business. Several studies have demonstrated that the further along the development team is in the development and deployment process, the costlier it is to fix bugs.

Identifying and fixing errors as soon as possible in the development cycle lower costs in the long run. This approach is called “Shift Left” and it is very like the mantra “Test Early and Often”. Indeed, rather than waiting at the end of the development cycle, testing should occur as early as possible and throughout the entire development cycle.

This approach allows the development team to identify and correct software bugs faster, which over time means better software quality and lower development costs.

The cost of fixing an error (on average) as software development progresses

Figure 1.1:The cost of fixing a software bug (on average) as development progresses

The graphic presented above (Figure 1.1) illustrates the average cost of fixing and correcting an error as software development progresses.

In short, fixing a software bug when the application is being used in production by live users, is on average 1000 times costlier than fixing that same bug at the beginning of the development cycle.

Case Study

This is where automated tests come in, as they go hand-in-hand with the Shift Left principle. Since test scenarios are written as the development progresses, the application can be tested much earlier and faster into the development cycle, resulting in overall test coverage increasing over time.

Numbers don’t lie. For instance, let’s compare the number of bugs in the database of an important application developed by a company that used to test everything manually:

Evolution of number of bugs per month

Figure 1.2: Software bugs in the database of an application developed using manual tests

The number of bugs in the database of an important application developed by a different company that was using automated tests:

Software bugs in the database of an application developed using automated tests

Figure 1.3: Software bugs in the database of an application developed using automated tests

In the first graphic (Figure 1.2), we can see that the first company that used manual testing fell behind early on with over 450 bugs listed in the database, and still had 150 bugs in the database after 24 months of development.

To fix these errors, the first company employed two full-time programmers, but these positions were difficult to fill, as most experts assigned to these tasks ended up leaving or asking to be re-assigned after a few months. As a result, development costs soared and the company lost some key talent.

The second graphic (Figure 1.3) shows the number of bugs in the database of an important application developed by the second company, using automated tests. The number is very high at first, especially in Phase 2, but is almost halved in Phase 3, and eliminated almost entirely in the next phase.

In the end, the second company delivered exceptional software quality. Their client’s acceptance tests only revealed three errors in pre-production and zero in the production phase, which was unheard of for the client with a project that big.

Shift Left and test automation were clearly an important part of this incredible accomplishment.

Automated Testing

So how was this possible? In short, there are many benefits to implementing automated tests because they affect all phases of the development cycle. Not only can automated tests be integrated very early on but they also:

  1. Eliminate testing bottlenecks

    The application mentioned above (Figure 1.3) was composed of more than 800 user screens. To test everything manually, two full-time testers would need at least six weeks. With automated tests, every functionality in the application can be tested in 4.5 hours.

    This massive improvement eliminates the bottleneck that testing often creates. To test the application in depth, all a developer needs to do is start the process, wait until all tests are completed and then look at the results.

    No manual intervention is necessary. This means that your application can be tested overnight or during the weekend, giving developers access to test data that is more reliable and updated more frequently.

  2. Increase test coverage while reducing risks

    As development progresses, automated tests will cover more individual cases and will be able to test more functionalities. As a result, even the more obscure functionalities of your application will be tested in depth each time the automated testing process is launched.

    In other words, unlike manual testers, automated testing doesn’t consider that the functionality was tested hundreds of times before. Your application will be tested thoroughly every single time.

  3. Allow the testing process to be repeated as many times as needed

    Since tests are automated, your team will be able to test your entire application as many times as needed.

    Moreover, using a tool like Askida CT means you will get a detailed report for each test script, allowing you to identify bugs and anomalies more easily and compare the health status of your application over time and as software development progresses.

  4. Prevent software quality regression

    If your development team introduces new errors into existing modules by accident, automated tests will allow you to identity them quickly and help you fix them quickly.

    In the end, automated tests ensure that all the different functionalities of an application work properly, preventing software quality from regressing over time.

Obstacles to Implementing Automated Tests

Why aren’t automated tests more widely used?

If automated tests offer so many advantages, why are different organizations still waiting to implement them? There are many reasons, such as:

  1. Initial costs

    Automated tests allow your company to save money over time as well as improve software quality. However, automated tests can require an important initial investment that some organizations resist, thinking they can live without.

    What they don’t realize is that implementing automated tests today will allow you to recover your initial investment very quickly.

    Figure 1.4: Cost per automated test over time

    Since automated tests validate more functionalities over time, the cost per individual test becomes lower as software development progresses.

    Moreover, the percentage of test coverage can be adjusted according to your company’s internal processes, to your current needs and to your budget.

  2. Fear of the unknown

    Investing in a new technology or a new way of doing things can seem risky. However, automated tests and the methods and technology developed by Askida have already clearly proved their effectiveness. Askida CT has been used since 2010 and has helped deliver several important projects successfully.

    In other words, now is a great time for your organization to implement automated tests as the technology and tools used have now reached technological maturity.

  3. Too Busy or not enough time to implement automated tests

    Some organizations are so busy with manual testing that they have a hard time even considering implementing a different way of doing things.

    If your team is is already overwhelmed and you’re afraid that an important organizational change could bring further complications, you should know that your regular activities don’t have to be interrupted to implement automated tests. It is possible to work in collaboration on an independent pilot project that can prove the feasibility of implementing automated tests and chart a path forward.

    This gradual approach can allow you to start slowly moving towards test automation without affecting your day-to-day operations.

  4. Resistance to change

    Some employees are used to doing things a certain way, which means they may not welcome change, especially if it affects their role in the organization and their daily tasks. It’s important to communicate properly to your team the benefits of adopting automated tests and your long-term vision.

    By automating tests, you can reallocate some resources and assign to more important and more creative tasks. Time is money: Automating a task that was once accomplished manually means time will be saved every instance it is accomplished.

  5. Waiting for automation to be absolutely necessary

    Too often, organizations will look to implement automated tests only after having trouble with quality when developing software. The individuals in charge will then ask how they can we prevent such a situation from happening again and therefore decide to invest in test automation.

    Often, these first initiatives aren’t very successful because companies are looking for a miraculous all-in-one solution after the fact, as opposed to implementing automated test automation progressively throughout their development process.

    Instead of waiting for your very own software development nightmare, why not be proactive and start implementing test automation in your organization today?

  6. Managing test data is too complicated

    We often hear that automated tests are “too complicated” and that data is often hard to obtain and changes constantly. This statment couldn’t be further from the truth. Test automation allows test cases and test data to grow and evolve alongside one another

Frequently Asked Questions

    • How much time will we need to develop and implement automated tests? How long before we can see a positive return on investment?

      This is generally the first question we hear from skeptics. Automation is integrated into Askida’s development projects, which are structured using Agile methodologies right from the start. The direct cost of implementation is rarely higher than 20 to 40% of the total cost of development.

      Since implementing test automation leads to excellent software quality, we consider these costs as a normal part of the development process. A positive return on investment is therefore immediate. In the end, projects are delivered on time and within budget.

    • What percentage of my application can be automated for testing purposes?

      Test automation percentage depends on the total number of written test scripts. Technically, it’s possible to automate testing for 100% the testing of an application’s features. It all depends on your organization’s goals, needs and budget.

    • During which phase of the development cycle should automated tests be implemented?

      The best time to implement automated tests is during the development phase of an application.

      Developers will receive almost immediate feedback on the quality of their code and the features they’ve developed, which will allow them to correct errors faster and earlier in the development process.

Better Software Quality AND Cost Reduction

As we’ve demonstrated, bad software quality can be incredibly costly for your organization. The “Shift Left” approach with test automation can not only help you deliver better software quality, but also lower costs, facilitate development, satisfy users and, ultimately, help you maintain a positive brand reputation.

Even without any statistics, it’s easy to see that manual tests are imperfect, repetitive and demand a lot of time and effort.

Test automation is therefore a crucial investment for your organization. If you’re still not convinced, we recommend investing in a pilot project that will show you the power and benefits of the technology and methods developed by Askida.

Free White Paper: Reduce Your Testing Costs

Fill out this form to start your free 15-day trial!