Running your tests in parallel can significantly improve the speed of your test runs, but it could give you occasional problems especially if you're working on static implementations. Also turns out that depending on which testing framework you're using parallelism might be handled differently.

As a friendly heads up I thought I would share something I found out a year ago that caused me a bit of a headache at the time and hopefully by sharing the answer with you I might save you that headache in the future.

Background:
I was working on a project that was using xUnit testing when I was asked to implement a feature I knew I had already built for another project. So a simple copy & paste I thought, perfect! (Turns out not so simple..)

The other project I was copying from was using NUnit and I’d done extensive testing for this specific feature so I also wanted to copy these tests with me of course. Rebuilding a test from NUnit to xUnit and vice versa is usually not very complicated so that’s what I did.

Problem:
This is when my headaches stared! For some reason in my xUnit project random tests would fail only to succeed the next time I ran them and then all of a sudden tests that succeed the first time would fail the second time around. This was driving me nuts and I didn’t get any good clues from reading the failing test messages.

Random tests would fail only to succeed the next time I ran them.

At the same time all tests in my NUnit project was succeeding, every time! After a lot of digging and a few sleepless nights I found the answer and it’s related to how NUnit and xUnit handles parallel test execution by default.

NUnit:
By default, no parallel execution takes place in NUnit, meaning that two or more test will never run at the same time (think of a queue).

You can enable it by using:

[Parallelizable]

This attribute indicates which tests may be run in parallel with other parallel tests.

By default, no parallel execution takes place in NUnit.

xUnit:
In xUnit the parallel feature is enabled by default, meaning that several tests may run at the same time (in parallel with each other).

You can however disable it by using:

[assembly: CollectionBehavior(DisableTestParallelization = true)]

This can be used either on an entire project or a collection of a few specific tests.

In xUnit the parallel feature is enabled by default.

So what should I use?
I guess there’s not really a good answer to “should I run my tests in parallel or not?” which is probably why these two frameworks handle them differently by default.

Personally I prefer it enabled by default and then I can disable it on the few occasions where I don't need it.

Warning (from the NUnit documentation):
"When tests are run in parallel, you are responsible for the thread safety of your tests. Tests that run at the same time and modify instance fields or properties without locks will cause unexpected behaviour as they would in any multi-threaded program."

Conclusion:
So the disappointing answer is: It depends. But it’s good to know that these two frameworks handles parallelism differently by default and depending on which one you use you might need to manually enable or disable it in specific scenarios. 

Hopefully ages from now you might run it to this issue and think “Wait a minute, I’ve read about this in a blog somewhere!”.

You’re welcome, future-you! ❤️