It seems like the more I read about different development models, the more I see overlap between the different paradigms. Even if you aren’t using one of them for a given project, just knowing about all of them makes your code better because you realize how to make your code more testable.

One thing that has piqued my interest of late is theories in JUnit 4.4+.

Overview

In JUnit, the still experimental theory functionality is derived from the open-source Popper project. The name comes from Karl Popper’s work on the philosophy of science. He was a major proponent of saying that something isn’t scientific unless it is falsifiable, and that theories cannot be proven correct, merely shown to have exceptions. If someone can provide an exception, it means that the theory does not match all of our observations. Theories are models of the world, so while a model may not be correct, it can still be useful.

So what are theories in software good for? When using test-driven development, the tests serve as a form of code documentation. However, there are certain properties of code that are known or assumed by the developers that are difficult to codify as simple tests. Theories provide a simple and expressive means for at least the following cases:

• identities and data conversion
• globally disallowed behavior
• data that should be preserved after performing an operation or set of operations

For example, you might have a theory that data that is serialized into XML and then unserialized should always result in the original object. This seems to make sense, but it might be time-consuming or error-prone to specify all of the different tests that you have to consider. A single theory can codify this assumption.

In JUnit, theories are a special case of parameterized test. So instead of using the standard test that has no parameters, you add parameters to make testing various combinations of values shorter and easier. You can then take these parameterized tests further by specifying data points to test for by default in an expressive manner.

Example

Alright enough theory. What does a JUnit theory actually look like? Given that you have a theory about two methods on a Currency object that should be inverses of each other, one nice example is:

    @Theory public void multiplyIsInverseOfDivide(
            @TestedOn(ints={0, 5, 10}) int amount, 
            @TestedOn(ints={1, 2, 3})  int multiplier) {
        assertThat(new Currency(amount).times(multiplier).divideBy(multiplier).getAmount(), is(amount));
    }

On the first line, we declare that we are going to postulate a theory, and give it a unique name. This method would typically live in a test suite or test file of some sort. The next two lines are the parameters to the test method. The parameters are prefixed by annotations to put in sample test values. All permutations of these values will be attempted when running the theory, and the values will be reported for any failures so we know what happened when the theory failed. On the last line, we test that the functions are indeed inverses by invoking them and making sure that the original value is the same as the end value.

Going further

So big deal, we could easily just create a helper test method that does something similar and we could pass whatever values to the method that we wanted. Well, hold on a second. One thing I haven’t mentioned yet is that there are some nifty tools that support automatically inserting values in to test theories. The tools automate searching the theory space for members that fit the theory assumptions but cause the theory to fail. Something like this might be useful in detecting an off-by-one error or corner case that you hadn’t considered while implementing. Typically for development you would just run the test cases presented with the method, and then would search the theory parameter space with spare cycles. This enables you to have great power as well as having blazing unit tests, which is key for any test-driven development.

If we allow automated searching of the theory space, there is one small problem with the example theory listed above. Can you see it?

The assert attempts to multiply by the multiplier and then divide by the multiplier, so a zero value for the multiplier will cause a divide by zero error. When we use automated tools for searching the theory space, we want to say that this theory only holds when the multiplier is not zero. We can easily do this:

    @Theory public void multiplyIsInverseOfDivide(
            @TestedOn(ints={0, 5, 10}) int amount, 
            @TestedOn(ints={1, 2, 3})  int multiplier) {
        <b>assumeThat(multiplier, not(0));</b>
        assertThat(new Currency(amount).times(multiplier).divideBy(multiplier).getAmount(), is(amount));
    }

Similar expressions exist for ensuring the value is positive, and so forth. You might create a different test or theory to show the expected behavior when the multiplier is zero.

I really liked the use of Java annotations to make this code considerably more readable. You would probably need a runner of some sort to specify the values if you didn’t do it right there in the method signature.

It seems like there is some overlap between theories and contracts, but I won’t go into that at this time. Theories exert positive design pressures like tests do, in that they tend to lead to leaner code that just does what the theory asks for.

Extensions

In Ruby on Rails, it would be nice to be able to easily test methods with multiple models that you have defined. For example, you might have a theory:

  fixtures :users

  def test_admin_user_can_access_this_page
    login(users(:admin_user_1))
    get :this_page
    assert_response :success
  end

  # the previous test, but better
  def theory_that_admin_users_can_access_this_page(User user)
    assume(user.is_a? AdminUser)
    login(user)
    get :this_page
    assert_response :success
  end

So, instead of using just one user or manually going through a set of users, you could just specify a whole bunch of fixtures and go to town. This might make fixtures less brittle by pointing out subtle interactions between parameters that would cause the theory to fail. I’m not sure how the syntax would look, and I’m not sure how you could use something better than annotations in Ruby.

Another thing that might be nice would be to say: I have a theory that any strings we put to the page from the database will always be escaped. The automated theory explorer could step through the pages that you define and try once with some basic data and then start generating HTML and JavaScript garbage and ensure that nothing gets through to the users. This might be a nice way of preventing cross-site scripting attacks (although I would imagine that there are tools that already exist for just this purpose.)

I looked, but could not find a Ruby plugin to handle theories. Hmmm… :)

Further reading

Some excellent resources are: The Popper tutorial A great paper with examples of use and benefits (see the pdf on that page) General overview of Theories in JUnit [pdf]