Thursday, February 7, 2008

Grails Dataset Loaders, Part 2

About a month ago I posted a few thoughts on Dataset Loaders for Grails. Now that one of my Grails projects is going into beta, I thought it would be a good time to revisit the issue.

Rails History: With rails I used fixtures for test data and had to hand code data loaders for production. I put the loaders in lib/tasks. They were rake tasks that did the initial loading and/or refreshing of data for a specified environment. So data loading for a production or development was a manual task. Not a huge burden, but just one more thing to remember.

Now that I'm in the Grails realm I have created automatic loaders that are environment aware. A DatasetBootStrap class loads data for development, test, and production whenever the server starts. This is in line with what hibernate does when it starts up--verify that the domain models match the current data source, and make changes where necessary. In rails this is done with migrations--a good idea, but yet another manual step.

DatasetBootStrap: The DatasetBootStrap class first determines the current environment, then loads the appropriate dataset classes and invokes the "load" method (or closure if you will). A list of datasets is configured manually for each environment. This approach makes it possible to share loaders for all three environments (or four if you use staging). Some data is just for test. Other data, like a list of US States, is loaded for all three environments. The main point is nothing is repeated, keeping my code as DRY as possible.

Datasets: I currently keep my datasets in src/groovy/datasets, but I think that it's time to move this folder into grails-app/conf/datasets. I also use a package declaration (package datasets) to insure that dataset classes don't conflict with other classes (I've had problems with this in the test structure when trying to create unit tests that have a name conflict with integration tests).

Simulating Rails-Like Test Fixtures: This was the easy part. In each test that requires a specific dataset, the setUp() method is used to load as many datasets as necessary to create a solid test environment. Loading all the data is as easy as calling the DatasetBootStrap's init() method.

Grails Integration: My goal for Datasets is to propose it's use for all grails projects by integrating it into the grails core. I have a bit of refactoring to do before this is practical, but would welcome any comments or suggestions from other grails (or rails) users as to what features a Dataset loader module should include.


Grant said...


Is your Dataset Loader code available online anywhere? I'd like to use it, even its not release-ready.

darryl.west said...

Sorry, I've been busy with two large projects. Both projects use datasets to load development, test, and production data. I plan to have something to release late next month.

vani said...


Is your Dataset Loader code available online anywhere? I'd like to use it, even its not release-ready.

superflav said...


Have you released your Dataset Loader yet? I'd really like to see what you've got cookin'.

Chris Fraser