The Importance of Automated Tests

Having automated tests on the application gives us a lot of benefits, even so, it’s not uncommon to see software that does not have tests ou hear someone saying that tests are not necessary. Tests are used in many kinds of industries, like cars, medicines, and others. The products are constantly tested by the companies to ensure that the product or service that they are offering are safe and work correctly. In software industries, this should not be different.

The software should have automated tests to ensure the good quality and warranty that the software is doing exactly what it was planned to do. Even so, is not rare to see people that don’t like to work with it. On this article, we are going to see the importance of having automated tests and what benefits they can bring.

I was reading a book about tests, which was published in the year 2014, and it says that the United States estimates that the money that is spent on fixing bugs, is almost 60 billion dollars per year. Think about how much of this amount of money could be saved and invest in new features or in other solutions. Besides that, there are some kinds of software that even a small bug can be critical, for examples in cases of medical software.

“Testing is the process of executing a program with the intent of finding errors.”

Excuses about not having automated tests

Some of the most common excuses that are used to explain why the system does not have automated tests are:

• “We do not have time to test”
• “Write test is writing more code”
• “If you write the code correctly, it’s not necessary to test”, or “A good developer must know how to code correctly”
• “Tests should be written by a tester”
• “Write test is expensive”

These are some common excuses that are used to justify why do not have tests on the application, but these are exactly excuses and not reasons, and let’s see why.

“We do not have time to test”

This is one of the more used excuses to justify the absence of tests. It’s true that is it’s not rare to have a tight schedule to deliver the feature that should be implemented, but we need to take into account that the time that will be used on writing the test should be considered when estimating the task.

We can have the impression that if we do not write tests we can delivery the task faster, and that’s partially true, we really can deliver the functionality faster if we do not implement tests for it, but the impact that this functionality without having tests can have in the end, has a big chance to be bigger than if we spend some time writing the test for it. It’s not just a few tasks that are deployed and after some time this same task presents some error or defect and a developer must stop to fix the bug. This situation could be avoided if this functionally had tests. It’s true that having tests does not mean that the software will never have any bug or defect, but at least we can make sure that the probability of some bug happens is much lower.

Think about how many time a developer needs to spend to fix a bug in the code. Fixing bugs is generally not so fast because it’s necessary to search for the issue, and this demands spending some time reading the code and debugging it to check what is happening, and after that, it’s necessary to fix the bug and test a lot of scenarios to warranty that the functionality is not broken and to warranty that the changed that was made did not affect other parts of the code. So is not just about fix the bug itself, but it’s also about the warranty that what was working before, is still working after the bug is fixed.

It’s interesting that someone can say that do not have time to write test, but this same person will find time to fix bugs that were caused because of the lack of tests. So remember that if you say that you do not have time to test, you will need to find time to fix bugs.

“Write test is writing more code”

This is another common excuse used for people who do not want to work with tests. It’s true that writing tests demand that we write more code, but generally what demands more time, is the implementation of the firsts tests, and specific situations where we have a different scenario or some external component. After the firsts tests are implemented in the application, basically what we need to do is create similar tests. And even if we need to write more code to do the test, it’s better to invest time writing tests during the development phase then spent time in fixing bugs after the software is deployed.

“If you write the code correctly, it’s not necessary to test” or “A good developer must know how to code correctly”

This is another common excuse used for people who do not want to work with tests. It’s true that a good developer should know what he/she is doing, even so, can be hard to prevent the impact that some functionality can have on the system, consider all the possible scenarios, and especially when working with software that has complex rules. A developer is a human being, that can make mistakes, can forget about some rule, or maybe the developer not even know about the existence of some specific rule. Sometimes even if the developer writes the code correctly, this code can affect some existent functionality and break something that was working correctly before.

Imagine the situation where we have a complex application, with a lot of business rules, and you need to write a new functionality that will impact other existent functionalities on the system. How can you warranty that what you are doing will not break what is already working correctly? You will need to test all the scenarios, right? But how many time you will need to spend to do it manually? Probably a lot, and in some scenarios we are talking about days of testing. So what generally happens in the real-word is that in this situation, probably the developer will not test all the possible scenarios, and the chance of a bug being introduced is high. And if this happens, the developer will need to spend some extra time to search and fix the bug, and because there is no test for this functionality, the chance of having a bug even after the “fixing”, is still high.

“Tests should be written by a tester”

This is another typical excuse that is used by some people. This affirmation is partially true. There is some kind of tests that should really be written by a tester, for example, the UI tests. But there is some kind of tests that should be written by the developer, for example, the unit test, the integration test, the end-to-end tests, and others. The developer is the one how knows exactly how the software work or should work, so for this kind of tests the developer is the one who should write.

“Write test is expensive”

This is another common excuse that we can hear, but this is not true at all. The 10 Myers rule says that fix a bug or a defect that is founded in the production environment is much more expensive then if it was founded on the development or analyse phase. So it is very worthwhile to invest in automated test, doing that you will be saving time and money.

Benefits of automated tests

Implementing tests on the application has a lot of benefits. Some of the benefits of having automated tests are:

• Allow to test the code frequently and in less time
• Allows to catch bugs before deploy
• Allow refactoring the code with more confidence
• Allow the code to have a better quality
• Provides documentation
• Reduce costs
• Reduce of bugs

Allows to test the code frequently and in less time

With automated tests, it’s possible to test the application in a lot of different scenarios in much less time then if you needed to test it manually. Let’s imagine a financial system, where we have a lot of complex rules, how long does it would take to test all the possible scenarios and situations if we needed to test it manually? Probably some days, and let’s be honest, in a real-world scenario we know that this will not happen, because no one has enough time to stop a developer to test manually all the possible scenarios during some days, so what happens is that the functionality is manually tested on some scenarios and after that, the software is deployed.

With automated tests, after we write the tests, we can just send the command to the tests being executed and we can test all the application, in a lot of different scenarios, and we can do it how many times we want and it will only take some minutes or some hours.

Allows to catch bugs before deploy

When we have automated tests, we can configure many scenarios and run them in a really fast way. Having those scenarios, we can catch bugs during the development, which is less complex to deal with then if the bug was founded in the production environment, and besides that, we can catch bugs before the final user see them. This way the developer can write the functionality, write the test, catch the bugs, fix it, and then deploy the software working correctly.

Allow refactoring the code with more confidence

This is another big benefit of having automated tests in the application. Let’s suppose that we saw a code that is not written in a good way, or could be written in a better way. How we can warranty that after the developer change this code, everything will continue working correctly? The only way to know that for sure is testing. But like we already saw before, testing manually demands a lot of time, so what generally happens in the real-word, is that when the developer sees a code that is not good, if there is no automated test, the developer will not change that code, because this could add bugs in the application and break something that was already working correctly.

Now imagine this situation happening for years, the developer sees that the good is bad, but he can not change anything because he knows that if he changes something, the chance of breaking something is high. What will happen is that after some period of time it will be very difficult to maintain this code. But if the application has automated tests, the developer can make changes, improving the quality of the code, and after that just need to run the tests to warranty that the code is still working correctly.

Allow the code to have a better quality

Having automated tests on the application, also ensures that the code should be written in a better way. For example, let’s suppose that we have unit tests in the application, once that the unit test should test only one thing, this will ensure that the classes and methods follow the Single Responsibility Principle, it means that the methods will not be so big and will be responsible for doing one and only one thing, and do it well. So having tests on the application can also improve the way of coding, making the code being better and well structured.

Provides documentation

The unit tests, for example, can also be used as a documentation of the software. The tests provide a living documentation of the application. Developers can look to the tests to have a better understanding of the functionality.

Reduce costs

Like we already saw in the previous topic, the 10 Myers rule says that fix a bug or a defect that is founded in the production environment is much more expensive then if it was founded on the development or analyse phase. So it is very worthwhile to invest in automated test, doing that you will be saving time and money. So having tests that allow getting the bugs and defects before the software being deployed, can also reduce the development costs. Beyond that, fixing the bug in the development environment avoid that the bug catches the final user.

Reduce of bugs

The tests warranty that the code should workly correctly. When a new feature is implemented in the code, the tests for this functionality will be also written. And if a bug is founded in the code, a new test can be written to cover the specific scenario. This way it will reduce the number of bugs.

The Main Types of tests

There are a lot of types of tests, and for each of them, there is a different cost of writing. The most common types of tests are:

• Unit test — Tests a single unit of an application without its external dependencies. This is the cheaper test to write and also execute fast. They are great to test the logic of the application. A unit test does not care about the whole system, it only cares about if a small part of the system works, it tests only a single unit of the system.
• Integration test — Tests the application with its external dependencies; test the integration between two or more parts of the application. For example, with this kind of test, it’s possible to test scenarios where the test must go to the database or use some external service.
• Loading test — Tests the capacity of the application. It’s also known as a stress test. For example, this kind of tests allows us to know how many users the application supports.
• End-to-end test — Tests the application through its UI. On these tests, the application will be executed, and the tests will be executed simulating a user that is using the system. This kind of tests gives you the biggest confidence when you need to decide if your software is working or not. This kind of tests demands more time to execute.

“Automating your repetitive tests can be a big game changer in your life as a software developer. Automate these tests and you no longer have to mindlessly follow click protocols in order to check if your software still works correctly. Automate your tests and you can change your codebase without batting an eye.” (Martin Fowler)

The Test Pyramid

On the chart below we can see three levels of tests: the unit tests, the integration tests and the UI tests. This chart gives an overview of the level of isolation of the main automated tests, the level of importance and the cost of writing each one of them.

The base of this pyramid is the unit tests, which are the cheaper and the faster (to write and to run) kind of tests that can be implemented. After the unit tests, there are the integration tests, which demands more cost and more time to write then the unit test. On top of this pyramid, there are the UI tests, which demands more time to implement and more time to execute, and it’s the most expensive kind of test.

The unit tests are the base of this pyramid, which means that they are the more important kind of tests that we should have. Followed by the integration tests, and the UI tests. When you start to implement the tests on your application, you should consider starting by the base of the pyramid, so start creating the unit test, and after that go to the integration tests, and after that goes to the UI tests. And of course, if you also can implement the other tests, go ahead.

Conclusion

The use of tests in software development is related to the quality of the code, it can be handy to the prevention of bugs, and help developers to improve the quality of the code, and also changing the existent code without damage what is already working. For each kind of test, it’s necessary to spend some time and money to write it. But as we saw on this article, there are some kind of tests, like the unit test, that are cheaper and do not consume much time to write, so at least the unit test you should consider to have in the project, and if possible of course, also have the other kinds of tests.

If you are using .NET Core and want to start to implement tests on your project, I recommend using the XUnit framework. XUnit allows us to have a better isolation of the tests and is also used by the development team of the .NET Core and ASP.NET.

Thanks for reading!

References

Mastering Software Testing

The Art of Software Testing — Glenford J. Myers

The Practical Test Pyramid — Martin Fowler

Test-Driven Development Test and Design in Real Word — Mauricio Aniche