Skip to content

Category: Tools

Migrating from SVN to Git

The Spanners demo project is now available on GitHub: https://github.com/hotblac/spanners. Feel free to fork!

I wanted to migrate the Spanners demo from SVN to Git, partly so I could take advantage of GitHub to host it (yes, I know I could host a SVN project on GitHub but where’s the fun in that?). I also wanted to start using a Distributed Source Control Management (DSCM) system and Git is certainly the most popular of these. Even though I’m (currently) the sole contributor to the Spanners demo project, it’s useful to understand how a DSCM system differs from a ‘traditional’ centralized server based SCM such as SVN and CVS.

Maven release plugin

Properly versioning and releasing code can be fiddly if it’s done properly. If you’re not cutting corners, the process involves several steps. If these steps are performed manually the process is error prone and time consuming. I’ve often found it’s easier just to cut a few corners. Why change the version in the Maven pom when you could perform all releases against a single SNAPSHOT version? Why tag the release build when you could probably work out the release version from a Subversion log? Why put release artifacts in a release repo when you can rebuild from a historic version?

I’m not going to argue that you shouldn’t cut corners when creating a release build (all the same, just don’t!). I will argue that the process need not be fiddly. So long as a Maven project is correctly set up, the whole thing can be done with a single command. Wire that single command up to Hudson, Jenkins, CruiseControl or whatever and you can create properly versioned release builds in a single click or (if you’re so inclined) as a scheduled task.

At the heart of this is the powerful – though slightly inflexible – maven release plugin.

DbUnit and Jailer

The easiest way of creating datasets for DbUnit tests is often to script out values from a real database. This can be done with a few lines of code in DbUnit itself (see DbUnit FAQs) but it’s easier to use a database tool that can export to DbUnit dataset files. Jailer is one such tool. In its own words:

Jailer is a tool for database subsetting, schema and data browsing. It exports consistent, referentially intact row-sets from relational databases. It removes obsolete data without violating integrity. It is DBMS agnostic (by using JDBC), platform independent, and generates DbUnit datasets, hierarchically structured XML, and topologically sorted SQL-DML.

It’s pretty easy to set up and has the advantage that it can flexibly script target data as well as all associated data necessary to satisfy foreign key constraints.

Just tell me what broke!

A few years ago, I heard about Glassbox, an automated troubleshooting tool for Java apps. The Google TechTalk seemed interesting (if a little long) and I was reasonably impressed when I plugged it into my own apps and it made (mostly) helpful suggestions on what may be causing bottlenecks. The tagline Just tells you what brokeā„¢ summed up the product perfectly. It didn’t go into unnecessary detail regarding CPU cycles, memory usage, garbage collections, locks, threads and so on. It just showed nice helpful messages like “Slow operation. Cause: Slow database operation”.

Unfortunately, this open source project has ground to a halt and there have been no new releases since 2008. And unfortunately it never quite managed to become completely usable. I’ve tried installing it again recently on a couple of different setups (WebSphere / IBM JDK 1.5 and Tomcat 6 / Oracle JDK 1.6) but never got it doing anything actually useful.

I now have a new project that I suspect is running poorly. There’s probably some problem with the database or nested loops or something like that. I don’t know exactly where the problem is though. I’d like to be able to use a tool like Glassbox to point me in the right direction. I don’t need it to solve my problem for me. I just need to know what to do next. Do I run a heap analysis or do I check my database indexing? So I’ve been looking for replacements for Glassbox.

Web service testing with soapUI

In my previous post regarding Spring-WS and Security I didn’t mention anything about testing the resulting SOAP service. Particularly when it comes to secure services, it’s vitally important to test. First, we want to make sure that the service is functionally correct – that it returns the correct results. Second, we want to make sure it is secure – that it refuses service to any request that does not meet our security requirements.

With regard to how we test, it’s simplest to use some SOAP editor tool that lets us fiddle with the request and press a button to retest instantly. But ideally we want some programmatic test that can be included in the test phase of our build.

This post describes testing the now legendary Spanners WS demo with the following requirements:

  1. Tests must be functional – they test what the webservice does
  2. Security is tested
  3. Tests can be tweaked and rerun instantly
  4. Tests can be included in build process

The updated source of the Spanners WS demo including the tests described here is available to download.

Rounded corners in CSS / IE Tester

Until Internet Explorer 8 is finally retired we still have to dick about with CSS to make IE behave properly. I’m not a CSS hacker but this is one trick that I suspect I’ll need again at least until IE9 becomes standard.

CSS3 includes a property for rounded corners which was (sort of) adopted in Firefox, Chrome and Safari some time ago. I don’t use it on this site – someone else did the hard work there using images for the corners. Presumably because CSS3 support was so poor at the time. This new CSS3 property can however be retrofitted to old browsers with a little work.

Test Coverage

I’ve been looking a lot recently at JUnit (and TestNG) tests on a code base I’m not too familiar with. In many cases I was not convinced that the tests were adequate but it took a fair bit of investigation before I could be satisfied that this was the case. I would need to look at the tests, then look at the code it’s meant to exercise, then try to work out in my head if the test covers everything it should. To make this process a bit easier, I’ve started running code coverage analysis using Emma. While this doesn’t tell me if the test is good or not, it does show me at a glance how much code is covered by the test and exactly which lines, methods and classes are missed. This is usually a good first approximation for the quality of the test case.

I’ve found Emma to be a useful tool to run after I think I’ve written my test cases and got them working. Running the test case tells me if the code being tested works. Running Emma tells me if I’ve tested enough of the code. There’s no point in having 100% test case successes if the tests themselves only exercise 50% of the code.

Memory usage

A year or two back I was working on a web application which was expected to have moderate use – around 50 concurrent users. The product was generally getting thumbs up from our QA guys. It did everything we expected it to do. Then we had a go at testing under load.

Bang!

We found that if we had only a few users hammering the system for any length of time, the memory usage became unacceptable. Simple maths showed that the problem was to do with the number of open sessions. Each session required 20-30MB of memory from the app server. This is a piddly small amount when we have a handful of test users. It went completely unnoticed against the background noise of a typical server’s memory use. However, once just a hundred sessions have been opened (not necessarily at the same time) we’re chewing gigabytes at a time.