I’ve been using xUnit.net to run some basic acceptance tests. Obviously xUnit is a unit test framework first and foremost but I don’t have a problem with reusing the framework provided you’re really clear about which tests are unit tests and which tests are not.
I created an AcceptanceTest attribute which derives from the xUnit Fact attribute. if nothing else this makes it really clear that what follows is an acceptance (not unit) test. All my unit tests use the StrictFact and StrictTheory attributes to make sure that unit tests don’t take a long time to run (see details here).
While a unit tests should be decoupled from the system environment though the use of mocks and therefore run correctly on any machine the same can’t be said of acceptance tests. I already have a number of acceptance tests which will only run on; NVidia CUDA enabled hardware, x86 processors and processes require access to the interactive desktop (not running as a service).
To this end I added the ability to construct acceptance tests with any number of SkipProviders. Each provider can cause the test to be skipped if certain criteria aren’t met.
This means I can write tests like this:
This test only runs on x86 machines (X86OnlySkipProvider) and where the desktop is accessible (UserInteractiveOnlySkipProvider).
The AcceptanceTestAttribute takes a variable parameter list of SkipProviders each of which can set the test’s Skip property and cause it to be skipped.
Here’s a SkipProvider which will skip tests run when no desktop is available:
Each skip provider implements the ISkipFact interface:
Another approach would be to use xUnit’s test category feature. I prefer dynamic skipping because it places the responsibility of when to run on the test and test author not on the developer running the tests and logs the fact that the test was skipped and why to the build log.
My experience with getting teams to consistently use switches on builds or run specific categories of tests is that people forget and things slip though. Far better to be able to say “run everything all the time” and have the tests figure things out.