Tech debt: Flaky Tests and How to fix it

 Are getting Flaky test on your nerves? Here is how you can fix them easily.

 

Here, How to handle flaky tests. Start on top of the list and go down until the problem is fixed. 

1. If it's easy, then fix them.
2. Complain about it. 
3. Disable flaky tests.
4. Delete flaky tests.

Sound easy, yes it is. Now let me explain it.

1. If it's easy, then fix them. But is is in 99.9999% of the cases I seen, it not easy. So you have to try the next idea.
2. Complain about it. This is an underrated or over seen solution. If you complain about a flaky test you trigger maybe an other developer who knows an easy solution. 
3. Disable flaky tests. A flaky test doesn't secure any code behavior, because it's flaky. But it desensitise you, see image above. And a flaky test cost you development time, rerunning the tests. So there are only benefits in disabling them.  
4. Delete flaky tests. If you are ready to disable the flaky tests, you can delete them, because it's dead code. An additional benefit of deleting it, it makes it easier to refactor your code.

 If you you have any comments or other ideas, leave a comment below.

 

Unit Test Anti-Pattern 2

Branches and loops in unit test are tech debt.

General rule: unit test should have a cyclomatic complexity of one.

If of If-Else or Switch branches are a clear indicator that your test is bad one. It's clear that you are not testing one case, maybe your test setup is broken.

If you have catches (exception handling) this also mostly a indicator that you don't understand the code behavior. Don't mix this up with negative tests.

Loops are a little bit different, loops could be a indicator that your test setup is not optimal.

If you have higher cyclomatic complexity test you should try to refactor them. Have a look at parameterized tests, maybe it's the way to refactor your test.

Unit Test Anti-Pattern 1

 Unit test to only increase the coverage metric is an anti-pattern.

Example:

1. testing data object
   
2. testing getter and setter, ... if there is no logic in
   

Why it is an anti-pattern? Here my view.

1. These coverage only unit test hide missing tests because they increasing the coverage metric but not increasing the code quality. No logic part is tested and it makes the coverage metric useless.
   
2. If you have normal unit test then these tests covering all these simple things (getter, setter), if not then you ether missing real unit test or the code is not needed.
   

Improve Code Coverage Metric

 I know, every body knows that Code Coverage is a weak metric (Link). But there are two ways to improve the value of he code coverage metrics:

1. Remove all exclusion pattern
2. Remove integration test from the calculation

Point 1 should avoids that you cheat your self.

Point 2 is not so obviously but it's easy to explain. Integration tests ensure that the whole software or larger parts work. Unit tests are the result of TDD and code coverage could measure the TDD impact. That's the reason why only unit tests should part of the code coverage calculation.

Test Driven Development (TDD) more then software tests

In the practice of software projects and software developers there are great misunderstandings about test-driven development (TDD). Most developers do not see or know the difference between JUnit testing and TDD. They believe that TDD is a theoretical, time-consuming testing approach.

 TDD is more then testing:

1. Proof that your code does the expected thing
   
2. Safety net for (your ongoing) refactorings
3. Helps you to separate concerns in your code
   
4. Last but IMHO most important, TDD is the spec of your code
   

PS: If you think you have to spend more time for the TDD, try it without.

Test Coverage as Software Metric

The measurement of test coverage is often used as indicator of software quality. This is OK but be a aware of following interpretation of this number, if the test coverage is low that means:

1. You have to write tests
2. You are in trouble because your code is not developed in the TDD way and the code is not easy to extend or to maintain.

On the other hand if the code coverage is high, this means nothing. Or with other words: 

High code coverage != High quality code

How to build stable and reliable System Test for Micro Services

Work in Progress, I'm currently overwork this article. Please come back later!

 

 

System tests are testing the orchestration of a bunch of micro services. Most of the test covers external features used by other micro services or by the front end.

If the current micro service is optimized for resilience (Yeah, Great) then system tests on feature endpoints can't discover errors related to (incoming) dependend micro services. The root cause of this lower power system test is the decoupling of micro services eg via the Fail Fast Pattern  or other Circuit Breakers (Thanks to Netflix). Or with other words resilience pattern could reduce the value of system tests.

System Tests improved by Diagnostic Endpoints

One way to avoid this negative side of resilience is to introduce what I called Diagnostic Endpoints. A diagnostic endpoint is only used by system test an not by features. The goal of diagnostic endpoints are to verify that functions that are covered eg by resilience pattern work.

In the real world, we save a huge amount of time after establishing of diagnostic endpoints. The software developers and the Project Owner love them.