Testing Failure Cases in jUnit in Java & Groovy

Unit testing expected failure cases is just as important as testing expected positive behaviour. In fact, you could argue that ensuring all failures are happening as expected is more important since it’s one of your best defenses against bad data. If you search for examples of how to test for failures (using old style jUnit 3), you’ll likely come across something like this.

public void testException() { 
    try { 
         Object o = getList().get(0)
         fail("Should raise an ArrayIndexOutOfBoundsException"); 
    } catch (ArrayIndexOutOfBoundsException e) { 
    } 
}

or perhaps

public void testException() { 
    try { 
         Object o = getList().get(0)
    } catch (ArrayIndexOutOfBoundsException e) {
        return;
    } 
    fail("Should raise an ArrayIndexOutOfBoundsException"); 
}

While both of these approaches are ok, they aren’t quite as precise as a good unit test could be. First, you want to be very specific with your verification. Let’s say down the road someone starts throwing an ArrayIndexExceedsSizeException that is a child of ArrayIndexOutOfBoundsException. Your test case will continue to pass. This isn’t ideal since you’ll want to update your test cases to verify all variations of runtime behaviour. And also, neither of these approaches gives you a decent way of making additional assertions against the exception (checking for messages, error codes, etc.)

This takes a bit more code, but the following will always be accurate and will start failing when these types of changes happen and gives you a clear approach for additional assertions.

public void testException() { 
    ArrayIndexOutOfBoundsException actual = null;
    try { 
         Object o = getList().get(0)
    } catch (ArrayIndexOutOfBoundsException e) {
        actual = e;
    } 
    assertExceptionThrown(ArrayIndexOutOfBoundsException.class, actual);    
    assertEquals(actual....
}
protected static void assertExceptionThrown(Class expected, Exception actual) {
	if (actual == null || !actual.getClass().equals(expected)) {
		fail("Exception thrown not of type " + expected.getName() + " but was " + (actual == null ? "null" : actual.getClass().getName()));
	}
}

For groovy tests, you can use the above, or you can use the beauty of closures.

void testException() {
    def ex = expectThrown(ArrayIndexOutOfBoundsException) {
        Object o = getList().get(0)
    }
    assert ex.message....
}
def expectThrown(Class expectedThrowable = Throwable, Closure closure) {
    try {
        closure()
    } catch(Throwable t) {
        if (!expectedThrowable.isInstance(t)) {
            throw t
        }
        return t
    }
    throw new AssertionError("Expected Throwable $expectedThrowable not thrown")
}

You can put these and other exception assertion helpers in a base class that extends your jUnit test class (TestCase or GroovyTestCase). In our next post, we’ll cover how to get around some Grails issues sharing test base classes amongst unit, integration and functional tests.