How a Service Virtualization approach can save your and others’ time playing with MocksJanuary 2, 2023
JUnit 5 – When to use @ValueSourceApril 4, 2023
Assertions are part of the day-by-day activities of any software engineer. There are numerous ways to apply it to support us in finding issues earlier in the development process. One of the best possible ways to apply it for unit and integration layers, even into another layer, is. the usage of the Soft Assertions approach.
Hard vs Soft Assertions
What’s a Hard Assertion
Hard Assertion is the normal assertion we know: when an expected result is not matching the test framework will halt the test execution and an assertion error is thrown. The main thing is that the test execution stops at the first failure, even though you have more assertions in the test.
Let’s imagine a situation where the
Person object cannot have null values and the age attribute must be equal to or greater than 18.
- On line 8 you can see the
Personobject with the
- The assertions on lines 11 and 12 will fail, but as it’s a hard assertion the test execution will be stopped before it reaches the assertion on line 11
This is the test output:
java.lang.AssertionError: Expecting actual not to be null
Hard Assertions make sense when it stops the execution as, as soon as the test has an error, the execution must stop. But there are cases in which you have an object, and you need to know which assertions have failed as we commonly assert more than one attribute in the tests.
The Soft Assertion approach will solve it.
What’s a Soft Assertion?
Soft Assertion is an approach to verifying numerous assertions (more than one), by storing them temporarily into an object and running the assertions internally, showing the possible test failures without halting the test execution. In short, the assertions will be done before showing its result.
The tools that support the Soft Assertions normally work like the following example in a pseudo-code:
SoftAssertion softAssertion = new SoftAssertion() softAssertion.assertSomething... softAssertion.assertAnotherThing... softAssertion.assertTheLastThing... softAssertion.assertThenAll();
The way of working is that the libraries will have a class to manage the assertions. This class will have as many assertions as you want. Think about this class as an array where each assertion is added to it.
The assertions will be performed as soon. as you call a method that tells the class: run them all!
This class will run and will record all the results, not halting the test execution if the expected result does not match.
Why should I use Soft Assertions?
You should use Soft Assertions when you have more than one assertion to apply to the same object.
Why? Because it’s better to know if all the assertions for that object are matching with the expected result rather than running the test multiple times to know what’s not matching.
I have a golden rule for myself: when I have more than one assertion to do, the soft assertion is in place.
Tools support for Soft Assertions
|AssertJ||Assertion library||✅||The best choice!|
|Truth||Assertion library||❌||There’s an open issue, from 2021 with a proposal to have it|
|Hamcrest||Assertion library||❌||No support 🙁|
|JUnit 5||Testing framework||✅||Supported by the |
|TestNG||Testing framework||✅||Supported by the |
If you are using TestNG as your testing library I have good news: it has the Soft Assertion approach implemented as a feature! If you use JUnit, please jump to the AssertJ 🙂
The TestNG has the
SoftAssert class that does the same as the pseudocode you read previously: it groups the assertions and verifies them as soon as we call a specific method.
In the example below w have the same requirement: the
phoneNumber cannot be
null and the
age must be equal to 25.
- On line 6 you can see a necessary instance of the
- Lines 8 to 10 use the
softAssertionbefore the assertion methods, telling the code that it belongs to the
SoftAssertionclass. You can use any supported assertion from TestNG, as its proxies assertions
- Line 12 calls the
assertAll()method, which will run all the assertions associated with the softAssertion reference
We can see that an assertion error will be shown, and this will be the result:
java.lang.AssertionError: The following asserts failed: Phone number cannot be null Expected :25 Actual :16 <Click to see difference>
Instead of halting the test execution, TestNG ran all the assertions showing all the failures. You can see the
AssertionError describing the failure in the phone number and the difference between the expected and actual result for the age attribute.
It does not have an external specific class to use, it’s already part of the
Assertions class. All you need to do is import it statically.
- Line 12 shows the assertAll() method using two parameters
String, to identify the assertions
Executablecommands, meaning the assertion methods
- Lines 13 to 15 show the usage of the Streams, each for any assertion we must apply
You will see the following exception in the console:
org.opentest4j.AssertionFailedError: Phone number should not be null ==> expected: not <null> at org.example.SoftAssertionJunit5Test.lambda$softAssertionUsingJUnit5$1(SoftAssertionJunit5Test.java:17) org.opentest4j.AssertionFailedError: Age must be 18 ==> Expected :18.0 Actual :16.0 at org.example.SoftAssertionJunit5Test.lambda$softAssertionUsingJUnit5$2(SoftAssertionJunit5Test.java:18) org.opentest4j.MultipleFailuresError: person (2 failures) org.opentest4j.AssertionFailedError: Phone number should not be null ==> expected: not <null> org.opentest4j.AssertionFailedError: Age must be 18 ==> expected: <18.0> but was: <16.0>
The exception is divided into 2 parts:
- The list of failures with the explanation and stack trace
- The summary of failures
- Using a JUnit 4 rule that takes care of calling
assertAll()after each test
- Using the provided JUnit 5 extension which injects a
BDDSoftAssertionsparameter and calls
assertAll()after each test
- Using a
You can use all these different ways to apply it and see all the examples on the AssertJ page. Here you see the assertSoftly static method in use, that’s the most convenient way.
- Line 12 shows the
assertSoftlymethod using a single parameter: the
SoftAssertions, which we need to name it
- Lines 11 to 13 use the consumer reference to call the assertion method
The exception output will look like this:
java.lang.AssertionError: Expecting actual not to be null at SoftAssertionTest.lambda$assertJSoftAssertion$0(SoftAssertionTest.java:16) java.lang.AssertionError: Expecting actual: 16 to be greater than: 18 at SoftAssertionTest.lambda$assertJSoftAssertion$0(SoftAssertionTest.java:17) org.assertj.core.error.AssertJMultipleFailuresError: Multiple Failures (2 failures) -- failure 1 -- Expecting actual not to be null at SoftAssertionTest.lambda$assertJSoftAssertion$0(SoftAssertionTest.java:16) -- failure 2 -- Expecting actual: 16 to be greater than: 18 at SoftAssertionTest.lambda$assertJSoftAssertion$0(SoftAssertionTest.java:17)
- The list of AssertionErrors and its full stack trace (which was omitted in this example)
- The summary of failures with the link for the line of the class that have the assertion error
Which one should I use?
Well, it depends on your preferences. You will have a way to use the Soft Assertions either in JUnit or TestNG.
I would recommend using it from AssertJ. Why? Because it extends the assertions in a lot of different ways not limiting you. For example: did you notice that using AssertJ, instead of
isEqualsTo() (or any
equals variation) we are using
AssertJ will complement the way we can assert the expected results and it provides a lot of different ways to apply the Soft Assertions, even extending them and creating your own if you want.
- you learned what’s the difference between hard and soft assertions
- you learned which most used tools and libraries support it
- you learned how to apply the soft assertion in the supported tools and libraries