Mar 25, 2009

Building Test Data

If your experience with building test data is anything like mine then you've tried a lot of different ways of creating test data - each with their own subtle flaws:

Setup all the data in the test method: going this route you quickly cluttered your tests with data that didn't help the reader understand the test.

Setup all the data in a file (or a database): it made your tests more concise, but you ended up with "magic" numbers that depended on what was in the file or database. This made tests brittle and hard to interpret.

You pulled all of the setup out into setUp (or helper methods or helper classes): much better. You were able to show the significant details, hide the mechanics of setting up test data, and gave helper methods readable names. Well, at first at least. The helper methods were hard to generalize so pretty quickly you ended up with an explosion of helper methods that became hard to manage.

Hmm. This is as far as I'd gotten and I've not been very happy about it.

At this point perhap you'd dumped Java anyway and run off to the Ruby space. If you didn't then did you arrive at something like this? (Edit: the original link referenced the Test Data Builder section).

  Order order1 = anOrder()
.withLine("Deerstalker Hat", 1) // detail important to the test
.withLine("Tweed Cape", 1) // detail important to the test
.withCustomersReference(1234) // detail important to the test
.build();

So this approach appeals to me (read the "Removing Duplication At Point of Use" section to see why). I took the time to code up what this might look like. It feels like a ton of boiler plate:

public class BuilderFactory {
public static OrderBuilder anOrder() {
return new OrderBuilder();
}
// factory methods for each builder...
}

public class OrderBuilder implements Cloneable {
private Customer customer = new CustomerBuilder().build();
private List<OrderLine> lines = new ArrayList<OrderLine>();
private BigDecimal discountRate = BigDecimal.ZERO;
private String voucher = null;

public OrderBuilder withCustomer(Customer customer) {
this.customer = customer;
return this;
}

public OrderBuilder withCustomer(CustomerBuilder customerBuilder) {
this.customer = customerBuilder.build();
return this;
}

public OrderBuilder withOrderLines(List<OrderLine> lines) {
this.lines = lines;
return this;
}

public OrderBuilder withOrderLine(String name, int quantity) {
lines.add(new OrderLineBuilder().withName(name).withQuantity(quantity).build());
return this;
}

public OrderBuilder withDiscount(BigDecimal discountRate) {
this.discountRate = discountRate;
return this;
}

public OrderBuilder withGiftVoucher(String voucher) {
this.voucher = voucher;
return this;
}

public Order build() {
Order order = new Order(customer);
for (OrderLine line : lines) order.addLine(line);
order.setDiscountRate(discountRate);
order.setVoucher(voucher);
return order;
}

public OrderBuilder but() {
try {
return (OrderBuilder) clone();
} catch (CloneNotSupportedException e) {
throw new RuntimeException(e);
}
}

}

// plus all the other builders too...
What's your reaction? Do you have a more elegant solution? [the code]

I like the readability of the code that uses the builders. The real disappointment is that the essence of all of the builder code is in the 6% of the code that defines the sensible defaults for test data, namely:
  private Customer customer = new CustomerBuilder().build();
private List<OrderLine> lines = new ArrayList<OrderLine>();
private BigDecimal discountRate = BigDecimal.ZERO;
private String voucher = null;
The rest is all ceremony. Are there better solutions out there?

7 comments:

Sam Livingston-Gray said...

Don't be fooled -- Rails fixtures suck just as badly as anything else. (Rails 2 dealt with the low-hanging fruit of managing foreign keys, but didn't address the fundamental problem of having many tests that all happen to use the same set of fixtures.) The best I've been able to come up with so far is a combination of nested context blocks in RSpec (so you can refine an inherited scenario) and some helper/factory methods.

(Note, too, that you can replicate the effects of nested context blocks through careful use of inheritance in your test cases. However, being forced to use one class per file can make this unwieldly as well.)

I haven't looked into it, because, well, I'd rather gnaw off a limb than write any actual software in Java, but there's no *technical* reason one couldn't use RSpec via JRuby to test one's Java code...

Of course, RSpec itself is actually way too much code, and its matchers are most of the problem. Something like Shoulda or the new-ish context project might provide a lighter-weight way of getting nested contexts.

Curious Attempt Bunny said...

Thanks, Sam. It helps to hear the grass isn't always greener.

Nested contexts in tests has been bugging me too. Groovy doesn't have the one class per file restriction, and I don't think the great unit testing tool for Groovy has been written yet. No offense to easyb: but we need to keep classes and even JUnit integration.

Scott Willson said...

Some smart people think this is a better builder approach: http://www.thoughtbot.com/projects/factory_girl/

I dunno. I'm still using sparse fixtures + setup helper methods in Test::Unit. I don't like the complexity cost of the alternatives.

I suspect that elaborate test data setup is a code smell; that I need to break down my tested code into smaller chunks. But I'm not smart enough to see how to do it.

Curious Attempt Bunny said...

Thanks for that Scott. In fact, that looks pretty similar to how I'd imagine a Groovy equivalent of the Java code to go.

Anonymous said...

Your prayers have been answered ;-)

http://code.google.com/p/make-it-easy/

Curious Attempt Bunny said...

@ Anonymous:

Looks interesting. This looks like Groovy syntax: a(Apple). Wouldn't it have to be a(Apple.class) in Java?

Curious Attempt Bunny said...

Ah. Tricksy. I've never tried defining a member variable as having the same name as a class..