How good are your tests?

Does this test have any sense?

public void testMe() {
  String result = concat("Fr", "ank");
  assertNotNull(result);
}

Probably yes, but there's not much use of it, as most of concat(String,String) implementations will pass, e.g. this:

public String concat(String a, String b) {
  return "Not implemented :-)";
}

So, recently I'm working on some smarter testing approach, which you maybe heard of, called mutation testing. It's idea is to rerun tests, modifying the system a little bit every time, to catch those test cases, that don't really care what system returns, or which always pass whatever happens.

Mutation testing is nothing really new, ah actually I think I have seen some articles about it dated back to 1990s..
What nobody liked about it since that old times, is that they're SLOW. Just imagine you have a system of more than 10 classes and you want to make some small changes in it, recompile and rerun whole test suite every time for 100 or 1000 times - you get the idea.

And here I'm getting to the point, why it's worth posting about this on Eclipse blog. After some tiny prototyping with pure junit library, I have switched to jdt.junit and now have ahead of me a pretty nicely working framework. It's really amazing how jdt.junit code is flexible.

Hopefully soon (no later than June - when I'm going to defend my master thesis), my users will get a nasty pinch for every test the tool finds useless :)