Parallelize Unit Tests Using Ant (Part 1)

Parallelize Unit Tests Using Ant (Part 1)

Testing your application automatically is important – we all know that. And nowadays most of the developers are used to write unit tests – it’s part of the development process. But the more tests a project holds, the longer the tests need to be processed. Especially when they use external dependencies (e.g. DB, or Web Services), which aren’t mocked yet. And that’s a drawback, and a reason for some developers to not run the tests prior to a commit.

I was pretty excited when Sebastian Bergmann announced that it’s likely that PHPUnit 3.7 is capable of running tests in parallel. That was in 2011 at the PHP Unconference in Hamburg. But it did not happen! And that’s because of the huge efford to realize this feature, which includes a rewrite of the TestRunner, as Volker Dusch mentioned at the PHP Unconference Europe 2013, during his talk about “What did not happen to PHPUnit?”.

Since the PHPUnit itself does not provide the capabilty of parallelizing test execution, we tried to solve this issue by using the parallelism feature of our build tool ANT.

Preconditions

We’ve had to solve the problem that many tests are using the database, and thus they’re somehow not independant. Meaning: One test could override testdata of an other test by accident.

We already had a database setup script, that created test-databases based on the current schema, changed the connection, and also loaded fixtures which were located in the setup dirtectory.

Idea

On every call of a TestSuite the setup script should create its own test-database, distinguished by the current suite-name. So that e.g. 3 parallel running tests are working with their own database, running completely independent.

How / Scenario

1. Using ANT

Ant provides a <parallel> element which encapsulates tasks that should run in parallel.
https://ant.apache.org/manual/Tasks/parallel.html

Example:

2. Use Directories as TestSuites

The directory structure of our tests gave the opportunity to use the top-level directories as starting point, of which each second-level directory acts as a TestSuite, and will run as one thread.

We used a for-loop in combination with a dirset to iterate over the 2nd level directories and hand each of them over to the antcall that does the actual phpunit call:

By adding the attribute “parallel”, set to true, the for-loop starts as many threads as you define by the attribute “threadCount”. When one thread ends, the next will start.

3. Drawback: Console Output

Since PHPUnit outputs the status of the running test directly to the console, we enabled writing the output of PHPUnit to the property “testOutput”, in order to prevent the console to be messed up with multiple threads writing their output. The output of an iteration is written to the console in that moment, when PHPUnit finshed the TestSuite. Additional to that, we had to use the “resultproperty” to determine whether the test failed or not.

What’s next?

Part 2 will explain how to solve the problem with the dependency database, and how we used a TestListener to fix that.