Conditional Acceptance Tests with xUnit.net
Tuesday, September 8, 2009 – 9:46 AMI’ve been using xUnit.net to run some basic acceptance tests. Obviously xUnit is a unit test framework first and foremost but I don’t have a problem with reusing the framework provided you’re really clear about which tests are unit tests and which tests are not.
I created an AcceptanceTest attribute which derives from the xUnit Fact attribute. if nothing else this makes it really clear that what follows is an acceptance (not unit) test. All my unit tests use the StrictFact and StrictTheory attributes to make sure that unit tests don’t take a long time to run (see details here).
While a unit tests should be decoupled from the system environment though the use of mocks and therefore run correctly on any machine the same can’t be said of acceptance tests. I already have a number of acceptance tests which will only run on; NVidia CUDA enabled hardware, x86 processors and processes require access to the interactive desktop (not running as a service).
To this end I added the ability to construct acceptance tests with any number of SkipProviders. Each provider can cause the test to be skipped if certain criteria aren’t met.
This means I can write tests like this:
1: [AcceptanceTest(typeof(X86OnlySkipProvider), typeof(UserInteractiveOnlySkipProvider))]
2: public void EndToEndAcceptanceTest()
3: {
4: //...
5: }
This test only runs on x86 machines (X86OnlySkipProvider) and where the desktop is accessible (UserInteractiveOnlySkipProvider).
The AcceptanceTestAttribute takes a variable parameter list of SkipProviders each of which can set the test’s Skip property and cause it to be skipped.
1: using System;
2: using System.Diagnostics.CodeAnalysis;
3: using System.Reflection;
4: using Xunit;
5:
6: namespace NBody.Tools.Xunit
7: {
8: [SuppressMessage("Microsoft.Design", "CA1019:DefineAccessorsForAttributeArguments")]
9: [SuppressMessage("Microsoft.Performance", "CA1813:AvoidUnsealedAttributes")]
10: [AttributeUsage(AttributeTargets.Method, AllowMultiple = false)]
11: [CLSCompliant(false)]
12: public class AcceptanceTestAttribute : FactAttribute
13: {
14: private Type[] _skipProviders;
15:
16: public AcceptanceTestAttribute(params Type[] skipProviders)
17: {
18: SkipProviders = skipProviders;
19: }
20:
21: [SuppressMessage("Microsoft.Performance", "CA1819:PropertiesShouldNotReturnArrays")]
22: public Type[] SkipProviders
23: {
24: get { return _skipProviders; }
25:
26: set
27: {
28: _skipProviders = new Type[] {};
29: foreach (Type t in value)
30: {
31: if (t.GetInterface(typeof(ISkipFact).FullName) == null)
32: throw new ArgumentException("SkipProvider does not implement ISkipFact interface.",
33: "value");
34:
35: ConstructorInfo info = t.GetConstructor(BindingFlags.Public | BindingFlags.Instance,
36: null, new Type[] { }, null);
37: ISkipFact provider = (ISkipFact)info.Invoke(new object[] { });
38:
39: if (provider.Skip != null)
40: Skip += " " + provider.Skip.Trim();
41: }
42:
43: _skipProviders = value;
44: }
45: }
46: }
47: }
Here’s a SkipProvider which will skip tests run when no desktop is available:
1: using System.Windows.Forms;
2:
3: namespace NBody.Tools.Xunit.SkipProviders
4: {
5: public class UserInteractiveOnlySkipProvider : ISkipFact
6: {
7: public string Skip
8: {
9: get
10: {
11: return (!SystemInformation.UserInteractive) ? "Skipping test when not in user interactive mode." : null;
12: }
13: }
14: }
15: }
Each skip provider implements the ISkipFact interface:
1: public interface ISkipFact
2: {
3: string Skip { get; }
4: }
Another approach would be to use xUnit’s test category feature. I prefer dynamic skipping because it places the responsibility of when to run on the test and test author not on the developer running the tests and logs the fact that the test was skipped and why to the build log.
My experience with getting teams to consistently use switches on builds or run specific categories of tests is that people forget and things slip though. Far better to be able to say “run everything all the time” and have the tests figure things out.
Welcome to my blog on software development and probably a whole lot more.
Sorry, comments for this entry are closed at this time.