JUnit exception handling

I was browsing through some test classes lately and saw some ways of handling exceptions in test code. It seems to me that different versions are used randomly; there is no common style, a pattern that everyone follows. So I’d like to share some good and not-so-good solutions.

In order to exemplify exception handling, I’ll use a very simple example: Let’s consider a primitive message sender, that is able to send a text message to a given IP address. The exception we are hunting for is called ExpectedException, and there’s nothing interesting about it. It’s just a simple exception:

package expectedexception;

public class ExpectedException extends Exception{

	private static final long serialVersionUID = 1L;

	public ExpectedException(String message) {
		super(message);
	}
}

Ok, so a system that sends messages to different IP addresses, huh? Then we might need something to check the validity of those addresses, it seems. (Usually I don’t write static methods, but in this case I used one for brevity):

package expectedexception;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

public class IPAddressValidator {

	private static final String IP_ADDRESS_STRUCTURE = "\\d{1,3}+\\.\\d{1,3}+\\.\\d{1,3}+\\.\\d{1,3}+";

	public static boolean isValidIpAddress(String ipAddress) {
		Pattern ipAddressPattern = Pattern.compile(IP_ADDRESS_STRUCTURE);
		Matcher matcher = ipAddressPattern.matcher(ipAddress);
		if (matcher.matches()) {
			return matcher.hitEnd();
		}
		return false;
	}
}

Now let’s see the message sender class. It’s code is not very complicated (note the null pointer exception when adding an element to an uninitialized list!):

package expectedexception;
import java.util.List;

public class MessageSender {

private List sentMessages;

    public void sendMessage(String ipAddress, String message) throws ExpectedException {
        if (IPAddressValidator.isValidIpAddress(ipAddress)) {
            sentMessages.add(message);
            //...
        } else {
            throw new ExpectedException("Hello world!');
        }
    }
}

The first approach we’ll be analyzing looks like:

	@Test(expected = Exception.class)
	public void shouldThrowExceptionWhenIpAddressIsNotValid() throws Exception {
		MessageSender messageSender = new MessageSender();

		messageSender.sendMessage("128.192.1.45", "Hello World");
	}

So we run the test, and get a green bar! If one is not sharp-eyed enough, will just go on and miss the point that the code is not working. The IP address is perfectly valid (it happens at copy-paste programming that someone just forgets to supply the correct parameters), so our message (Hello World) should be sent to that IP address. However, as we pointed out, there is a NullPointerException thrown inside the if statement; NullPointerException is a subclass of Exception, and as such, it perfectly satisfies the test condition. In short, should any kind of exception be thrown inside the method under test, the test case remains green. Never, never expect raw exception as result for a test case.

Ok, let’s suppose we corrected the null pointer from the example above, and so we have (note that “Ip Addres” is misspelled on purpose 🙂 ):

package expectedexception;
import java.util.List;

public class MessageSender {

	public void sendMessage(String ipAddress, String message) throws ExpectedException {
		if (IPAddressValidator.isValidIpAddress(ipAddress)) {
			//...
		} else {
			throw new ExpectedException("Ip Addres not valid");
		}
	}
}

Another approach I saw, is to write a test case that looks like:

	@Test
	public void shouldThrowExceptionWhenIpAddressIsNotValid() throws Exception {
		MessageSender messageSender = new MessageSender();

		try {
			messageSender.sendMessage("128.192.1.45.7.1", "Hello World");
		} catch (ExpectedException e) {
			assertEquals("Ip Addres not valid", e.getMessage());
		}
	}

(We fond out that the test was provided a correct parameter – e.g. valid IP address. As we are handling special cases, we just simply break that value)
Ok, this test case is a little bit better than the one before. However, it is fragile as hell. What if someone comes along and spots the misspelling? In such cases one would correct that message to “IP Address not valid”. There, the test case has just been broken! Test cases should never test exception messages. In TDD, names, messages and other stuff are refactored quite many times. I do mind correcting test cases all the time when such a refactoring is carried out. A test case should test state and/or behavior and not plain text.

Let’s move on to the next approach.

	@Test
	public void shouldThrowExceptionWhenIpAddressIsNotValid() throws Exception {
		MessageSender messageSender = new MessageSender();

		try {
			messageSender.sendMessage("128.192.1.45.7", "Hello World");
			Assert.fail("IP Address validation is broken!");
		} catch (ExpectedException e) {
		}
	}

Well this one is a little bit tricky. It calls the method under test and (as we are testing that an exception is thrown); if everything goes ok, e.g. no exception is thrown, it fails the test case. The static method fail() is part of the Assert class; optionally you can pass in a message as a parameter, so it will be displayed if the test case fails. If an exception is thrown, we are handling it with an empty catch block. The caught exception will not fail the test case, so we get a green bar. This type of construction is much more useful and less fragile then the ones above.

And finally the last one:

	@Test(expected = ExpectedException.class)
	public void shouldThrowExceptionWhenIpAddressIsNotValid() throws Exception {
		MessageSender messageSender = new MessageSender();

		messageSender.sendMessage("128.192.1.45.7", "Hello World");
	}

Simple construction, no empty catch blocks. However, you have to take extra care to expect a special enough exception class. It works exactly as the test case above, but its a much compact solution.

So pick your favorite (the third or the fourth one 🙂 ), and start using it consistently.

Advertisements

Author: tamasgyorfi

Senior software engineer, certified enterprise architect and certified Scrum master. Feel free to connect on Twitter: @tamasgyorfi

5 thoughts on “JUnit exception handling”

  1. Hi tamas,

    My Name is Prasad. I am writing the JUnit test case for my existing project classes. I am getting the following exception like
    java.lang.NullPointerException
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at junit.framework.TestCase.runTest(TestCase.java:154)
    at junit.framework.TestCase.runBare(TestCase.java:127)
    at junit.framework.TestResult$1.protect(TestResult.java:106)
    at junit.framework.TestResult.runProtected(TestResult.java:124)
    at junit.framework.TestResult.run(TestResult.java:109)
    at junit.framework.TestCase.run(TestCase.java:118)
    at junit.framework.TestSuite.runTest(TestSuite.java:208)
    at junit.framework.TestSuite.run(TestSuite.java:203)
    at junit.framework.TestSuite.runTest(TestSuite.java:208)
    at junit.framework.TestSuite.run(TestSuite.java:203)
    at org.eclipse.jdt.internal.junit.runner.junit3.JUnit3TestReference.run(JUnit3TestReference.java:130)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:460)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:673)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:386)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:196)

    so please help or give me the suggestions for this

    1. Hi Prasad,

      Sorry for the late reply, I’ve been really busy lately.
      Unfortunately I can’t really tell from the stack trace what the problem could be. I see that you are using JUnit3 runner. Do you get the same result with JUnit 4?

    1. Hi,

      The thing is that -in different test cases- you have to make JUnit throw different exceptions like this:

      oneOf(mock).doSomething(with(any(Util.class))); 
      will(throwException(new SomeRelevantException());
      

      For each exception thrown you’d put an expected exception annotation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s