May 30, 2008

The Long Way Around...

Check out The Long Way Around.... You won't be sorry. Lots of high quality thought provoking posts to be read.

Dorgem + JAI + Groovy = Bliss

I'm back on the Go Vision hobby project of a year ago. This time I'm using Dorgem to capture images for me -- triggered via motion detection (no hassle with JAI and webcam interfaces).

I'm using Groovy instead of Java. So far my code is not all that different from Java code. There are two things that I want to do that will leave Java behind in the dust:

  • Seperate the display of an image operation from the operation itself, so that on focus of the JFrame I reload the operation code and recalculate the image. Hot deploy a la Groovy and Swing!
  • My code is littered with helper methods for add, subtract, and, or, etc. And they all use the same boiler code resulting in a call to a static JAI method passing the name of the operation as a string. This screams out for a "method missing" strategy! My current code bloat would DRY down to hardly any code.
There are also a whole host of useful and not so easy to find JAI pages that I'll blog on at some point.

Perhaps we'll see an improvement on the last version (which required (a) telling it where the corners of the board were and (b) telling it when a move had been played):


If anyone reading this is interested, I'll gladly share datasets and code with them. The more ideas the better!

May 27, 2008

Reviewing January Through March 2007

I thought it'd be good for me to read through my old posts. I've followed up here to say what came from them:

My Technology ToLearn List II

Following on from my previous list:

  • LiquiBase for schema patching that's database independent and ties in with hibernate.
Hmm. I was going to list other items, but I think this is all that I'm committing to for the time being. Wrong title perhaps?

May 24, 2008

Groovy Sparklines

Sparklines are "data-intense, design-simple, word-sized graphics" used as "small, high resolution graphics embedded in a context of words, numbers, images".

The code is here at TestEarly.

On a related note, if you don't mind sharing your data with Google, you might be interested in the Google Chart API.

May 20, 2008

Merb / Rails Anecdote

Eek! Seriously?

"... People pick Merb because a single Merb process consumes half the memory of a Rails process and is able to handle many simultaneous clients with that one Merb process, whereas you must have one 60MB+ Rails process for every single concurrent client. ..."

Waaaah? I'd never have thought Rails was that resource hungry. Yowser. Erm, so 30Mb per concurrent client is then acceptable?! [the rest of the context]

Useful Stand-ups Imply Not Enough Collaboration?

"[C]onsider this: one of the values of Agile is communication and collaboration. The daily stand-up meeting exists to promote communication. But if the team really collaborated well, would it be necessary? Your team really should be sitting together, pairing, and sharing ownership of their work. If they're really doing that... really doing it... the stand-up doesn't add much value."

That's an interesting idea. We get a lot out of stand-ups at work. Oops?

Tapestry 5: New-User Friendly Tutorials


I'm a fan of documentation that helps me jump-start in a new technology. This tutorial does a good job of showing off Tapestry in an informative fashion. And it comes with pictures for short-attention-span me.

SpringSource: OSGi Bundle Libraries

Is this the beginning of the end of the Maven2 repository? I can't say I'd be too upset, assuming SpringSource does a good job. Thanks to Cleve for the tip-off (give his posting a read for a feel for where we're at in the world of Java).

Maintaining That Million Line Groovy Codebase?

"My answer is that with a good dynamic language, you have some seriously elegant features and patterns that allow you to keep the size of your codebase small. Essentially, you don’t maintain a million-line codebase - because it doesn’t take a million lines to write your application." - hahaha.

As an exercise, try coding this problem I had today, but in Java: given a Map<String, Integer>, print all the string keys ordered by their integer values in descending order. It's a one liner in Groovy (or a three liner if you want to spread out the closure). Take that Java:Groovy ratio as an example, and now how big is that million line code base? Actually, I've no motivation to code it up in Java by way of illustration. Does anyone care to work it out for me?

May 19, 2008

I've Been Busy...


I've been busy with a number of things. On the tech front I've been:

On the topic of Groovy, Neal Ford is in town tomorrow (Tuesday May 20th) presenting at PJUG his take on Design Patterns in a dynamic language setting. I'm interested to see whether there will be any fresh insights on offer or not. Come along and decide for yourself.

Also of interest this week: Diana Larsen is organising an Agile User Group meeting over lunch this Friday (May 23rd) 1pm at McMenamins near Lloyd Center on N.E. Broadway & 15th in Portland. Her teaser for a topic is rather fitting:

"The word 'Agile' has moved into the mainstream, the practices have not."

May 6, 2008

Lightning Fast Integration Builds


First, what makes an integration build feel fast?

I think these main factors contribute towards the perception of a fast integration build:

  • avoid doing as much work as possible in the build
  • what it does to it does FAST
  • push logic testing down to the lowest level of test as possible (functional -> integration -> unit)
  • fail the integration build as soon as you know there's a problem
  • work to have unit tests catch 95% of your errors and so that slower tests rarely get to catch errors
  • use faster tools first for faster feedback (i.e. your IDE)


An integration build can have a number of responsibilities. I think it's important to limit the scope of its work to just the steps necessary to support the developer in committing their code. I've organized my thoughts into different areas of responsibility:

Updating From Version Control
  • this should not be a required build step (it's normally done through your IDE anyway)

Compiling
  • do this in your IDE on the whole (don't use the build for this unless you're the CI server)
  • mostly use incremental compilation in place of a clean and compile
  • get better (incremental) compilation performance from a non-Sun compiler such as jikes

Jaring/Waring/Earing
  • this should not be a required build step (developers rarely need them)

Resource Building
  • there should be no file processing in order for resources to be ready for the code (design the architecture so that it does the work at runtime, not the build - or just keep things simple)
  • use the stand src/main/resources and src/test/resources approach to keep resource management simple

Web Services
  • don't generate a WSDL unless it's needed to run tests (consider pushing such tests out to a regression build)
  • don't generate code for a web service unless it's needed to run tests (consider pushing such tests out to a regression build)
  • more ideas?

Web App Sever
  • deploying should not be a required build step (explicitly server restarts, maybe)
  • use a light-weight app server (tomcat, jetty, etc.)
  • use the hot-deploy feature of the app server to avoid starting and stopping the app server
  • structure the project as an exploded war to eliminate the need for preparing the war directory

Database
  • support a variety of SQL dialects for different scenarios: (implies using Hibernate for persistence and Liquibase for schema patching)
  • oracle for a production-like setup
  • mysql (less of a resource hog and cheap) for persistent QA and Dev data (2? times faster than oracle)
  • h2 (an in-memory database) for development and developer testing (3.5 times faster than mysql) - also as it's in-memory it doesn't need truncating

Unit Testing
  • aim for 100+ unit tests per second and treat slow unit tests (>10ms) as broken
  • don't use static initializers in code that do heavy lifting (e.g. spin up threads)
  • don't use static variables that do heavy lifting in their constructors (e.g. hitting the database or the filesystem)
  • don't use the database in unit tests - tests run literally 100x faster when they don't
  • don't read from or write to files in unit tests - increases test speed by 10-50 times
  • don't spin up threads in unit tests - again, this is very costly
  • do not use spring injection in unit tests - limit the scope of the test to just one main class
  • do not open sockets in unit tests - another way to make tests run very slowly
  • for speed, limit the scope of tests to avoid the need for creating large object graphs for each test

Layer/Integration/Component Testing
  • fake or mock dependencies that are in different application layers to the subject of your test - this will speed up the test as well as limit it's scope
  • integration tests should only fail when the integration of objects has failed or the interactions between objects is wrong - save finer grained business logic tests for the unit tests
  • any data that is required in the database must not be able to clash with existing data (to avoid the need for truncating the database)
  • any data that is already present in the database must not change the outcome of a test (to avoid the need for truncating the database)

Functional Testing
  • include only a minimal set of functional tests as part of the integration build (save the rest for a full regression build)
  • functional tests should be very very simple and short so as to avoid replicating the business logic tests of the faster tests - this also makes them less brittle to change (and faster!)
  • bias toward light testing tools for the integration build functional tests (e.g. not selenium)

Tests in General
  • flip-flopping tests (that don't consistently pass/fail) must be treated as broken - in particular to avoid rerunning the integration build

Project Size
  • if the project is large and all the complexity is required, consider splitting the build into smaller modules to reduce the scope of each integration build