Integration Testing Best Practices

We’ve already covered the best practice of Automated Unit Testing. Unit testing has many benefits, but there are times when you need to be able to test how multiple units of code work together.  This is when you need Integration Tests.

Integration Tests (ITs) are automated tests that verify multiple components are working together as expected.  ITs can be used to test end-to-end functionality of an entire system, from the UI controller all the way to the database.  They can also be used to verify smaller pieces of functionality that are difficult to test at the unit level (for instance data access code).  Integration tests have a number of other advantages relative to unit tests:

• Since ITs do not require strict isolation of dependencies, they are easier to integrate into legacy codebases.
• Since ITs are written at a higher level of code, they can more closely reflect the actual software requirements.
• Since ITs cover multiple units of code, a smaller number of ITs can cover a larger amount of code.

The Case for Integration Testing

Consider the following ASP.NET MVC Web API code:

    public class ValuesController : ApiController
    {
        readonly IValuesRepository valuesRepository;

        public ValuesController(IValuesRepository valuesRepository)
        {
            this.valuesRepository = valuesRepository;
        }

        public IEnumerable<string> Get()
        {
            return this.valuesRepository.GetAll();
        }
    }

    public interface IValuesRepository
    {
        IEnumerable<string> GetAll();
    }

    public class ValuesRepository : IValuesRepository
    {
        public IEnumerable<string> GetAll()
        {
            using(var db = new ValuesContext())
            {
                return db.Values.Select(v => v.Data);
            }
        }
    }

    public class ValuesContext : DbContext
    {
        public DbSet<Value> Values { get; set; }
    }

    public class Value
    {
        public int ID { get; set; }
        public string Data { get; set; }
    }

This is about the most simple example possible of an end-to-end Web API action.  But consider the following 4 potential problems:

1. Is ValuesRepository correctly registered in the Dependency Injection container?
2. Does the Get API have the correct access restrictions?  What if it is meant to allow anonymous access, but a global Authorize filter has been added?
3. Is routing configured correctly for the Get API?
4. Do the property names in Value match the column names in the database?

And #5 – a clear defect in the code above:

5. ValuesRepository.GetAll() should be forcing the query evaluation with a ToList() before exiting the using block!

None of these defects could possibly have been found with unit testing.  However, we can find all of them with an integration test.

Integration Testing Example #1 – ASP.NET Web API

ASP.NET Web API allows us to test our server functionality in terms of actual HTTP requests/responses via the HttpServer and HttpClient classes. This is extremely helpful because it catches the whole stack – action filters, routing, app startup, etc.

To implement this sort of test, we need to create an HttpServer instance configured to use our Web API.  This will run a full ASP.NET Web API server in memory and respond to requests just as network-based server would.  Then we can make requests against it with an HttpClient.  The following code illustrates how this is done – if any of the 5 potential defects outlined above are present, this test will fail and alert us to the problem:

// Always include error details to make it easier to debug a failed test caused by a server error.
var config = new HttpConfiguration()
			{
				IncludeErrorDetailPolicy = System.Web.Http.IncludeErrorDetailPolicy.Always
			};

// The following two lines are copied from Global.asax.cs:
// Apply our Web API config (routing, filters, etc.) to the configuration object
WebApiConfig.Register(config);
// Set IoC on the configuration object (if applicable - in this example we're using Unity)
UnityConfig.RegisterComponents(config);
// Create our server and client objects
using (var httpServer = new HttpServer(config))
{

	using(var httpClient = new HttpClient(httpServer))
	{

		// Make the request to our web API.  Since this is a GET we need only a URL; when testing
		// POST or PUT actions we can build a request object and use PostAsync or one of the helper
		// extension methods defined in System.Net.Http in System.Net.HttpFormatting.dll
		// (e.g. PostAsJsonAsync, PutAsXmlAsync).
		//
		// Note that the host of the request URL is completely irrelevant, but one must be specified.
		var response = httpClient.GetAsync(@"http://localhost:49845/api/Values/List").Result;
		if(!response.IsSuccessStatusCode)
		{

			// Since we're expecting a success code, we want to fail the test with the full text of
			// the error response (this is why we set IncludeErrorDetailPolicy.Always on the configuration).
			Assert.Fail("HTTP Server Failure: " + response.Content.ReadAsStringAsync().Result);
		}
		// Read the result as a string[]
		var result = response.Content.ReadAsAsync().Result;
		// Verify it matches an expected condition.
		CollectionAssert.AreEquivalent(new[]
										{
											"TestItem1",
											"TestItem2"
										},
										result);
	}
}

Writing HTTP-level integration tests like this allows us to verify nearly any requirement related to our ASP.NET application.

Integration Testing Example #2 – Entity Framework

Another strong use case for integration tests is for data access code.  It is a common problem for data access code in an application to get out of sync with the actual database.  Fortunately, with code-based data access approaches like Entity Framework Code First, it is easy to verify that everything is correct.  The following test code creates a Value entity, sets its Data property, then saves and reads back the entity.  This ensures that the Data property on the Value entity type is mapped correctly to the database.

using (var context = new ValuesContext())
{
	
	// Create a new Value entity with a known Data value.
	var testValue = new Value() { Data = "Testing" };
	// Insert it into the database
	this.context.Values.Add(testValue);
	this.context.SaveChanges();
	// Read it back (based on the ID, which gets set in the SaveChanges call above)
	var testValueFromDb = this.context.Values.Single(v => v.ID == testValue.ID);
	// Verify that Data value matches.
	Assert.AreEqual("Testing", testValueFromDb.Data);
}

It is good practice to define such tests for any data access code to ensure that the database and the application remain in sync with each other.

Considerations for Writing Integration Tests

As we’ve seen above, integration tests allow us as developers to have automated tests of far more functionality than we can achieve strictly by unit testing.  However, this increased power and flexibility comes at a cost.  Here are some common difficulties with integration tests, and how to overcome them.

ITs often require extra effort to maintain a test environment that reflects real-world scenarios.  For instance, an integration test suite that relies on the state of a database needs to configure that database before running any tests.  This takes development time up front before any integration tests can be written, as well as ongoing maintenance effort as the application evolves.  It’s important to understand that taking time to write ITs is an investment in quality.  When your tests catch a coding error before it ships to production, you’ll be happy you spent that extra time.

ITs can break due to external dependencies.  For instance, if your application relies on a web service, an outage for that web service will likely cause your integration tests to fail.  To avoid this situation, you can create mock implementations of your dependencies to use for the bulk of your tests.  However, it is best to keep at least some tests running against the real thing.

ITs usually take much longer to run than unit tests.  ITs are running lots of code with every test, accessing the database and file system, using the network, etc.  The best way to improve IT performance is to share as much common setup code as possible between tests, and only run this setup once during the entire test run.  Another thing to consider is that since integration tests are running your actual application code, they reflect your actual application performance.  If your integration tests are slow, then your application is slow as well!

Summary

Writing integration tests is a great way to invest in quality for your code.  By having automated tests that run against your application at a high level, you can verify that all the pieces of the system are working together as intended.