.NET 1.1+

NUnit Timed Tests

by Richard Carr, published at http://www.blackwasp.co.uk/NUnitTimedTests.aspx

Sometimes, when performing automated testing, it is necessary to ensure that a test is completed within a given period of time. NUnit helps this type of testing with attributes that fail tests that run for longer than a specified duration.

Timeout Attribute

If you are creating NUnit-based tests that should run within a given duration, you can enforce this using the Timeout attribute. This attribute allows you to specify a number of milliseconds within which the test must complete. If the time limit is exceeded, the test is aborted and flagged as a failure.

This type of timeout is useful when testing code that includes looping or recursion. If there is a logical error and the looping continues indefinitely, the NUnit test runner will abort the test and continue with the next. The attribute can also be useful if you are performing integration testing with a potentially slow dependency.

The sample fixture below shows two tests, each with a two second timeout applied.

[TestFixture]
public class TimeoutTests
{
    [Test, Timeout(2000)]
    public void PassingTest()
    {
        Thread.Sleep(1000);
    }

    [Test, Timeout(2000)]
    public void FailingTest()
    {
        Thread.Sleep(3000);
    }
}

The first test, "PassingTest", includes code that pauses the thread for one second. This is within the timeout value of two seconds and there are no other failures so the test passes. The second test includes a three second pause, which means that two second timeout is exceeded and the test is stopped. The result is a failure with the following message:

The Timeout attribute may also be applied to a test fixture, as shown below. In this situation the timeout duration is individually applied to each test within the fixture.

[TestFixture, Timeout(2000)]
public class FixtureTimeoutTests
{
    [Test]
    public void PassingTest()
    {
        Thread.Sleep(1000);
    }

    [Test]
    public void FailingTest()
    {
        Thread.Sleep(3000);
    }
}

NB: Starting with NUnit version 2.5.5, the Timeout attribute is ignored when the tests are executed in a debugger.

MaxTime Attribute

NUnit includes a second attribute that ensures tests complete within a specified amount of time. MaxTime also accepts a duration in milliseconds and fails the test if this is exceeded. Unlike with Timeout, the test is not aborted when the limit is reached. You can see this with the following tests:

[TestFixture]
public class MaxTimeTests
{
    [Test, MaxTime(2000)]
    public void PassingTest()
    {
        Thread.Sleep(1000);
    }

    [Test, MaxTime(2000)]
    public void FailingTest()
    {
        Thread.Sleep(3000);
    }
}

The second of the above tests fails because it runs for more than the allocated two seconds. The overall running time for the test is the full three seconds of the Sleep method plus any other time required to execute the test. With a Timeout attribute, the test would have aborted after only two seconds. The failure message is similar to the following:

The key benefit of MaxTime is that any assertions within the test will be processed, even when the test runs too slowly. If an assertion causes the test to fail, this failure reason takes precedence over the timing out of the test. To demonstrate, modify the FailingTest method as follows:

[Test, MaxTime(2000)]
public void FailingTest()
{
    Thread.Sleep(3000);
    Assert.AreEqual(1, 2);
}

When the test are run, the test fails due to the assertion, not the running time:

SampleTests.MaxTimeTests.FailingTest:
  Expected: 1
  but was:  2