When writing unit tests or doing TDD, it’s common to find excuses to not test something. It’s not that it may to too hard to test, but it might rely on some outside factor, such as an Internet connection, an Active Directory environment, or relies on some hardware configuration, such as the number of processors.
The common solution to all of these problems is to fake it, using dependency injection and a combination of mocking and stubbing.
But we know all of that. Those are practices that have been preached for a while now. How do we test what we are actually stubbing out – the actual dependencies?
One place where I see this coming up frequently is when a product might call home to information about the environment to provide a better experience, as well as understanding environment configurations, such as needing to know if old platforms still need to be supported.
Here is a little bit of code that might do that:
1: using System;
2:
3: namespace TestingTheUntestable
4: {
5: public class OperatingSystemInfo
6: {
7: public bool Is64Bit { get; set; }
8: public bool IsServerCoreEdition { get; set; }
9: public bool IsClientVersion { get; set; }
10: public Version OperatingSystemVersion { get; set; }
11: }
12:
13: public interface IOperatingSystemInfoProvider
14: {
15: OperatingSystemInfo GetInformation();
16: }
17:
18: public class OperatingSystemInfoProvider
19: : IOperatingSystemInfoProvider
20: {
21: public OperatingSystemInfo GetInformation()
22: {
23: return new OperatingSystemInfo
24: {
25: Is64Bit = Environment.Is64BitProcess,
26: IsClientVersion = false,//Omitted
27: IsServerCoreEdition = false, //Omitted
28: OperatingSystemVersion = Environment.OSVersion.Version
29: };
30: }
31: }
32:
33: public class OperatingSystemInfoSender
34: {
35: private readonly IOperatingSystemInfoProvider
36: _operatingSystemInfoProvider;
37:
38: public OperatingSystemInfoSender
39: (IOperatingSystemInfoProvider operatingSystemInfoProvider)
40: {
41: _operatingSystemInfoProvider = operatingSystemInfoProvider;
42: }
43:
44: public OperatingSystemInfoSender()
45: : this(new OperatingSystemInfoProvider())
46: {
47: }
48:
49: public void SendDetails()
50: {
51: var info = _operatingSystemInfoProvider.GetInformation();
52: //Send
53: }
54: }
55: }
You can probably see where this is going: we can test the OperatingSystemInfoSender by providing a mock IOperatingSystemInfoProvider and returning a stub OperatingSystemInfo. This is common and simple pattern for dependency injection.
So what about OperatingSystemInfoProvider? It’s these units of code that often go untested, because who knows how it will act on another machine. Even worse, some people won’t test the Sender either because they don’t recognize that there is a dependency there that can be mocked out.
Let’s assume we have the dependency separated as above and we can test the info provider independently.
There are different philosophies to this problem, I’ll outline a few of them:
“Don’t test it. The platform is different between workstations, build servers, etc. It’ll be too complicated to maintain these tests.”
This tends to be the more common of the situations. It’s unfortunate, and it doesn’t follow TDD principles.
“We want to test it, but we don’t know how, so we gave up.”
or
“We use to test it, but it was too brittle and failed too often.”
This is also common, but it might be a knowledge limitation.
“Have a dedicated environment that this test passes on. For instance, the build server. Don’t worry if the test fails on dev. stations, just make sure the build server says it’s green.”
This is a better solution, at least the functionality is tested, but it removes some comfort for developers. For XP and some Scrum shops, tests are the documentation, and having documentation that is wrong on a development station might be a bad thing.
“Test the bare minimum. Assert that there are no exceptions raised, and that some form of data is returned that looks correct by checking ranges or regular expressions.”
I prefer this approach. It’s a comfy feeling seeing all tests green on a local development station, as long as the tests are meaningful.
“Use compiler directives to control assertions. Have a symbol defined for “BUILDENV”, and only do assertions if BUILDENV is defined. The build environment will compile it’s code using that symbol.”
It’s still better than nothing, it depends on how much work and precision you are expecting from your tests. It this granularity is required, then there might not be a better alternative.”
We know we want to test it, and we can start with the simplest approach: checking if there is no exception.
1: [TestFixture]
2: public class OperatingSystemInfoProviderTests
3: {
4: [Test]
5: public void ShouldNotThrowException()
6: {
7: var infoProvider = new OperatingSystemInfoProvider();
8: Assert.DoesNotThrow(() => infoProvider.GetInformation());
9: }
10: }
If your testing framework doesn’t have an equivalent for DoesNotThrow, just calling it directly and an exception occurring will also fail the test. However, that lacks clarity of, “What are we really checking with this test?”
This is certainly better than nothing, and a step in the right direction. I think we can do more though, such as the operating system version being greater than 0.0.0.0.
1: [Test]
2: public void ShouldReturnValidVersionNumber()
3: {
4: var infoProvider = new OperatingSystemInfoProvider();
5: var info = infoProvider.GetInformation();
6: Assert.IsNotNull(info);
7: Assert.Greater(info.OperatingSystemVersion,
8: new Version(0, 0, 0, 0));
9: }
You see where this is going. Assertions and tests don’t have to be exact if they still provide meaning. I would be challenged to find code where no tests could be written at all. It just might require a slight change in the details of what a test is.
An interesting topic came up in a discussion today, a phrase I’d never heard before: “Agile Nirvana”. As explained, Agile Nirvana is where Agile is done ‘perfectly’ and following all of the rules by the book. Where there is no room for improvement anymore.
A colleague of mine and I were looking at the test coverage of an existing application. The end result was 65%, to which he replied, “That’s not too awful”. My first thought was, “It was probably lower, something close to 45%”.