Introduction
There are many testing frameworks and tools out there, all of them with different flavors, specifications, pros and cons, but the one that stands out the most, in my opinion, is Spock – a Groovy-based testing Framework.
Most of you are probably familiar with JUnit – the most popular testing framework in the Java world. It is widely adopted, reliable and easy to use, but at the same time not always concise and readable. On the contrary – Spock is all that, as it leverages Groovy programming language features. Groovy is a JVM language, interoperable with Java which provides more dynamic concepts and concise syntax. Spock, on the other hand, is compatible with most of the IDEs and build tools. It allows us to write tests, not only for Groovy but also for Java and Kotlin applications in a more expressive and readable manner. In this article, I will try to introduce you to its most interesting features.
Dependencies
First, we need to add Spock and Groovy dependencies to our project.
<properties>
<spock.version>1.3-groovy-2.4</spock.version>
<groovy.version>2.4.7</groovy.version>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-core</artifactId>
<version>${spock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>${groovy.version}</version>
<scope>test</scope>
</dependency>
</dependencies>
Also, include a Groovy compiler plugin to our build.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.0</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
<encoding>UTF-8</encoding>
<compilerId>groovy-eclipse-compiler</compilerId>
<fork>true</fork>
</configuration>
<dependencies>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-eclipse-compiler</artifactId>
<version>2.9.2-01</version>
</dependency>
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-eclipse-batch</artifactId>
<version>2.4.3-01</version>
</dependency>
</dependencies>
</plugin>
Those dependencies are only in the `test` scope since we will be using Spock and Groovy only for testing.
Specification structure
What is known as a test class in JUnit is called specification in Spock, while test methods are referred to as features. It is a good practice to postfix specification classes with `Spec` word.
First Specification
Let’s write our first Spock specification.
import spock.lang.Specification
class MyFirstSpec extends Specification {
def "should add two numbers"() {
given: 'Two numbers: 2 and 3'
def a = 2
def b = 3
when: 'Numbers are added'
def result = a + b
then: 'The result is equal to 5'
result == 5
}
}
Let’s closely look at `MyFirstSpec` class. Firstly, every Spock specification has to extend `spock.lang.Specification` class. Secondly, feature method names can be written as free text sentences (strings), which makes them easier to read. Lastly, we can see that a feature method body is divided into some labeled code blocks.
Code Blocks
This is Spock’s way of organizing test methods and enabling BDD-style testing. Those code blocks can be labeled with optional string descriptions to increase readability. There are seven basic code block types in Spock:
- `given` (aliased also by `setup`) – where all the feature setup and preparation comes
- `when` – feature stimulus – in other words, this is where the actual method-under-test is being executed
- `then` – where the response is verified and all assertions are made
- `expect` – combines `when` and `then` altogether
- `where` – can be used for parameterized tests
- `cleanup` – where feature cleanup comes
- `and` – helper, used for separating individual parts of any block
The code that is not under any particular block is considered to be under a `given` block.
In the example above we have done our test preparation in a `given` block, then we have executed method-under-test in `when` block and made an assertion in the `then` block. The feature method above could be simplified using `expect` block, in which we combine stimulus and assertions.
def "should add two numbers"() {
expect: '2 plus 3 equals 5'
2 + 3 == 5
}
Assertions
When compared to JUnit, assertions in Spock are easier to write and read because they are leveraging Groovy language mechanisms. We don’t need to use any fancy assertion methods, just simple boolean conditions. Also, non-boolean conditions are allowed and they will be evaluated according to Groovy truth.
def "groovy truth test"() {
expect:
// booleans
2 + 3 == 5
true
!false
// strings
def s1 = ''
def s2 = 'abc'
!s1
s2
// numbers
def n1 = 0
def n2 = 42
!n1
n2
// lists
def l1 = []
def l2 = [1, 3, 6]
!l1
l2
// maps
def m1 = [:]
def m2 = ['key': 'value']
!m1
m2
// objects
def o1
def o2 = new Expando(name: 'groovy')
!o1
assert o2
// matchers:
def ma1 = "groovy" =~ /java/
def ma2 = "groovy" =~ /groovy/
!ma1
ma2
}
As you can see, although `assert` keyword can be used, it is unnecessary to put it inside `then` and `expect` blocks. Nevertheless, it has to be used when you want to make assertions in different test sections. Note also, that groovy provides multiple ways of defining string literals, it could be single quotes, double quotes, triple-single-quotes, triple-double-quotes, slashy and dollar slashy. Differences between them are shown in the table below, which can be found in the groovy docs: https://groovy-lang.org/syntax.html.
Spock also provides nice and informative error messages when assertions fail.
Condition not satisfied: 2 + 3 == 4 | | 5 false <Click to see difference> at MyFirstSpec.groovy truth test(MyFirstSpec.groovy:25)
Asserting exceptions
Spock allows us to assert that tested method thrown an exception, in a very clean manner:
def "should throw an exception on division by zero"() {
when:
3 / 0
then:
thrown(ArithmeticException)
}
Or if we want to check more information about the exception:
def "should throw an exception on division by zero"() {
when:
3 / 0
then:
def e = thrown(ArithmeticException)
e.message == 'Division by zero'
}
For the compiler to know what is the type of the exception, the above can be replaced with:
def "should throw an exception on division by zero"() {
when:
3 / 0
then:
ArithmeticException e = thrown()
e.message == 'Division by zero'
}
Grouping assertions using `with`
Sometimes we need to verify more than one condition for a single object. It may be done in a simple way like this:
def "should find animal by name"() {
when:
def animal = zoo.findAnimal('bono')
then:
animal instanceof Monkey
animal.height == 154
animal.weight == 93
animal.name == 'bono'
}
However, this may seem a bit too verbose. Also, `instanceof` does not suit everybody. Groovy comes with a lot of built-in helper methods, and the one we can utilize in situations like the one above is `with(target, closure)`:
def "should find animal by name - improved"() {
when:
def animal = zoo.findAnimal('bono')
then:
with(animal as Monkey) {
height == 154
weight == 93
name == 'bono'
}
}
As you can see, checking if an animal is an instance of Monkey, is not needed anymore because we do a smart casting. The compiler will know that inside closure of `with` method, it is dealing with an object of type Monkey.
Fixture methods
Similar to JUnit, Spock has a bunch of fixture methods to help us set up and clean up the environment for the test run. There are four fixture methods available:
| Fixture method | Execution | JUnit equivalent |
| `def setupSpec()` | Triggered once, before the first feature method | `@BeforeClass` |
| `def setup()` | Triggered multiple times, before each feature method | `@Before` |
| `def cleanup()` | Triggered multiple times, after each feature method | `@After` |
| `def cleanupSpec()` | Triggered once, after the last feature method | `@AfterClass` |
Parameterized Tests
Sometimes we would like to run the same test/feature method for a couple of different data sets (input + output). This is, of course, possible in JUnit, but has been taken to a whole new level by Spock. To emphasize how good the Spock and Groovy combination works for parameterized tests let’s look at some JUnit examples first.
@RunWith(Parameterized.class)
public class ParameterizedTest {
@Parameterized.Parameters
public static Collection<Object[]> data() {
return Arrays.asList(new Object[][]{
{2, 3, 5}, {3, 11, 14}, {21, 0, 21}
});
}
private int a;
private int b;
private int result;
public ParameterizedTest(int a, int b, int result) {
this.a = a;
this.b = b;
this.result = result;
}
@Test
public void shouldAddTwoNumbers() {
assertEquals(result, a + b);
}
}
When using a Parameterized JUnit runner it’s possible to achieve what we want, but it seems like a lot of hassle to do a simple parameterization. Test results output is not very informative either:
Let’s try out Theories runner instead.
@RunWith(Theories.class)
public class TheoriesTest {
@DataPoints("data")
public static int[][] data =
{{2, 3, 5}, {3, 11, 14}, {21, 0, 21}};
@Theory
public void shouldAddTwoNumbers(
@FromDataPoints("data") int[] data) {
assertEquals(data[2], data[0] + data[1]);
}
}
Fair enough. It’s a bit more concise, but still lacks readability and test results output is even worse than in the previous example.
Here’s where Spock comes to the rescue. Using `where` block, we can specify not only data pipes but also data tables. Everything is clean, clear and less verbose.
def "should add two numbers"() {
expect:
a + b == result
where:
a | b || result
2 | 3 || 5
3 | 11 || 14
21 | 0 || 211
}
Double pipe (`||`) in data tables is just a syntactic sugar coating that visually separates input from output parameters. Test results output, though, is still not perfect:
Condition not satisfied: a + b == result | | | | | 21| 0 | 211 21 false <Click to see difference> at com.kurczyna.ParameterizedSpec.should add two numbers(ParameterizedSpec.groovy:9)
Luckily enough, there is an easy way to improve that. The `@Unroll` annotation on a feature method breaks parameterized test execution into separate runs and allows also, to template method name like following:
@Unroll
def "adding #a to #b should result with #result"() {
expect:
a + b == result
where:
a | b || result
2 | 3 || 5
3 | 11 || 14
21 | 0 || 211
}
The output is pretty neat, isn’t it?
It is also possible to utilize reflection for the parameterizing method name.
@Unroll
def "should return itself on #method"() {
given:
def number = 12.0
expect:
number."$method"() == 12
where:
method << ['toInteger', 'intValue']
}
In the example above we used data pipe to inject method name as String and call it by name. Results are as follows:
Mocking
Sometimes there is a need to test the interaction between objects within the application. It is called interaction-based testing technique and its main purpose is to isolate the logic under test, only to a tested class but not its dependencies, and therefore limit the test scope. It also enables us to verify how the object interacts with its collaborators. JUnit does not come with built-in mocking support. Most of the time we have to use 3rd party libraries like Mockito or EasyMock. Conveniently enough, Spock decided to include mocking capabilities into the framework. Mocking in Spock is lenient, not strict, which means that if we do not specify the behavior of a method, it will respond with a default value.
Verifications
Assume that we have the following model (written in Kotlin):
interface Listener {
fun onBookingConfirmation(id: String)
}
class Booking(private val listener: Listener) {
fun confirm(id: String) = listener.onBookingConfirmation(id)
}
Whenever the `confirm` method is called on a `Booking` object, it `notifies` the `listener` object.
So let’s write a test that verifies that the listener has been notified.
class BookingSpec extends Specification {
Listener listener = Mock()
// or: def listener = Mock(Listener)
Booking booking = new Booking(listener)
def "should notify listener on booking confirmation"() {
when:
booking.confirm('booking-123')
then:
1 * listener.onBookingConfirmation('booking-123')
}
}
We created a mock Listener object in the setup phase of the Specification and then injected it through a constructor as a dependency to Booking object. Now we can to verify, that when `booking.confirm()` method is called, with specific booking `id`, the `listener.onBookingConfirmation()` is called exactly once with the same booking `id`. The syntax is simple and very readable: `cardinality * target.method(arguments)`. There are many possibilities of tweaking all parts of the template above. I’ll give you some examples, but if you want to know more, you should look for it in the official Spock documentation.
Cardinality
- 1 * … -> triggered once
- 0 * … -> triggered 0 times
- (1.._) * … -> triggered at least once
- _ * … -> triggered any number of times
Method matchers
- 1 * listener.onBookingConfirmation() -> onBookingConfirmation called once
- 1 * listener._() -> any method on listener called once
- 1 * listener./on.*/() -> any method starting with `on` called once
Argument matchers
- 1 * listener.onBookingConfirmation(‘booking-123’) -> argument matching specific String
- 1 * listener.onBookingConfirmation(!‘booking-123’) -> argument not matching specific String
- 1 * listener.onBookingConfirmation(_) -> any argument
- 1 * listener.onBookingConfirmation(*_) -> any argument list
- 1 * listener.onBookingConfirmation(_ as String) -> any String non-null argument
- 1 * listener.onBookingConfirmation(startsWith(‘booking’) -> any argument starts with ‘booking’
We can capture the arguments to perform more complicated operations/assertions on them. To illustrate this let’s change our model a little:
interface Listener {
fun onBookingConfirmation(bookingData: BookingData)
}
data class BookingData(val id: String, val paid: Boolean,
val created: LocalDate)
class Booking(private val listener: Listener) {
fun confirm(id: String) {
val data = BookingData(
id = id,
paid = true,
created = LocalDate.now()
)
listener.onBookingConfirmation(data)
}
}
So the booking will notify the listener with some additional data that we would like to verify:
def "should notify listener on booking confirmation"() {
when:
booking.confirm('booking-123')
then:
1 * listener.onBookingConfirmation({ BookingData data ->
data.id == 'booking-123'
data.paid // shorthand for data.paid == true
data.created == LocalDate.now()
})
}
And if there is an error in one of the assertions we get nice and verbose output:
Unmatched invocations (ordered by similarity): 1 * listener.onBookingConfirmation(BookingData(id=booking-123, paid=true, created=2019-07-16)) One or more arguments(s) didn't match: 0: Condition not satisfied: data.created == LocalDate.now().minusDays(1) | | | | | | | | | | | 2019-07-15 | | | | 2019-07-16 | | | class java.time.LocalDate | | false | 2019-07-16 BookingData(id=booking-123, paid=true, created=2019-07-16)
Argument capturing is a powerful technique that allows us to do complex assertions and also manipulate the arguments if needed.
Stubbing
Often we not only want to verify how many times the method is called on a mock but also to emulate that it returns specific value whenever it gets called. This technique is called stubbing.
Consider the following model:
enum class UserState {
VERIFIED, NEW
}
interface UserRepository {
fun getUserState(id: String): UserState
}
class UserService(private val repository: UserRepository) {
fun isUserVerified(id: String) =
repository.getUserState(id) == UserState.VERIFIED
}
We would like to test that `isUserRegistered` method, returns true when the user is verified. So we would like to stub `repository.getUserState` method to return `UserState.VERIFIED`. It can be done this way:
class UserServiceSpec extends Specification {
UserRepository repository
UserService service
def setup() {
repository = Mock()
service = new UserService(repository)
}
def "should return true when user is verified"() {
given: 'user with id user187 is VERIFIED'
def id = 'user187'
repository.getUserState(id) >> UserState.VERIFIED
expect: 'isUserVerified returns true'
service.isUserVerified(id)
and: 'unknown user is not verified'
!service.isUserVerified('unknown')
}
}
`>> UserState.VERIFIED` syntax means that the method on the left hand side will return UserState.VERIFIED any time it’s called. There is a possibility of changing method return value for subsequent invocations, if there is a need for that:
`repository.getUserState(_) >>> [UserState.VERIFIED, UserState.NEW, UserState.VERIFIED]`
This line means that the method will return VERIFIED when it is called for the first and third time, and NEW when it is called for the second time.
We can also emulate the method to throw an exception:
`repository.getUserState(_) >> { throw new RuntimeException(‘bummer’) }`
Combining mocking and stubbing
It is possible to combine mocking and stubbing in Spock, but the most intuitive approach like the one below, will not work:
def "should return true when the user is verified and verify method was called once"() {
given: 'user with id user187 is VERIFIED'
def id = 'user187'
repository.getUserState(id) >> UserState.VERIFIED
when: 'isUserVerified is called'
def result = service.isUserVerified(id)
then: 'result is true'
result
and: 'method was called once'
1 * repository.getUserState(_)
}
Instead, we have to combine mocking and stubbing together:
def "should return true when the user is verified and verify method was called once"() {
given:
def id = 'user187'
when: 'isUserVerified is called'
def result = service.isUserVerified(id)
then: 'result is true'
result
and: 'method was called once and VERIFIED was returned'
1 * repository.getUserState(_) >> UserState.VERIFIED
}
Note that there are also other kinds of Mock objects in Spock: Stub and Spy. The main difference between Mock and Stub is that the latter can only be used for stubbing and not for mocking, while the former for both stubbing and mocking.
Extensions
In addition to what has been written in this article so far, Spock comes with a bunch of powerful built-in extensions. They are activated by annotations, called directives.
The most useful ones are:
- @Ignore – ignores a feature method, or a whole specification
- @IgnoreRest – ignores all feature methods not having this annotation
- @IgnoreIf – ignores a feature method if specific conditions are met
- Example: `@IgnoreIf({ System.getProperty(‘os.name’).contains(‘windows’) })`
- @Timeout – set a timeout for feature execution
- @Retry – for retrying flaky features
- Example: `@Retry(count = 3)`
Custom extensions
To create your custom extension, you can write a class that extends `AbstractAnnotationDrivenExtension` and override any of its methods. In the example below, we are overriding `visitSpec` method and adding a custom listener which extends `AbstractRunListener`. Then we can override `beforeSpec` method of the listener, so we can trigger our custom logic before every specification run.
package com.kurczyna.extension
import java.lang.annotation.ElementType
import java.lang.annotation.Retention
import java.lang.annotation.RetentionPolicy
import java.lang.annotation.Target
import org.spockframework.runtime.AbstractRunListener
import org.spockframework.runtime.extension.AbstractAnnotationDrivenExtension
import org.spockframework.runtime.extension.ExtensionAnnotation
import org.spockframework.runtime.model.SpecInfo
@Retention(RetentionPolicy.RUNTIME)
@Target([ElementType.TYPE, ElementType.METHOD])
@ExtensionAnnotation(EmptyH2Extension)
@interface EmptyH2 {}
class EmptyH2Extension extends AbstractAnnotationDrivenExtension<EmptyH2> {
@Override
void visitSpecAnnotation(EmptyH2 annotation, SpecInfo specInfo) {
}
@Override
void visitSpec(SpecInfo spec) {
spec.addListener(new EmptyH2Listener())
}
}
class EmptyH2Listener extends AbstractRunListener {
@Override
void beforeSpec(SpecInfo spec) {
TruncateH2TablesScript.execute()
}
}
class TruncateH2TablesScript {
static void execute() {
// your logic for truncating H2 tables
}
}
Extension could be applied to specification class as follows:
@EmptyH2
class MyFirstSpec extends Specification {
...
Summary
To sum things up – Spock is a powerful and dynamic testing framework. It works well with Java, Kotlin, and Groovy and has good IDE support. It’s most important traits in my opinion are:
- Enforced test structure which prevents from putting assertions in a random place of the test
- Tests code much more readable, concise and less boiler-plate
- Informative and clean test outputs
- No need to use 3rd party libraries/tools for mocking. Also, stubbing/mocking is very straightforward and doesn’t need much code
- Awesome test parameterization using data tables
- Groovy built-in dynamic concepts can be leveraged
- Extension mechanism
Useful links
- Spock Web Console – https://meetspock.appspot.com/
- Repository with all the examples from this article – https://github.com/patrykkurczyna/spock-demo
- Spock documentation: http://spockframework.org/spock/docs/1.3/all_in_one.html#
- Groovy documentation: https://groovy-lang.org/syntax.html