Grails Dataset Loaders, Part 2

About a month ago I posted a few thoughts on Dataset Loaders for Grails. Now that one of my Grails projects is going into beta, I thought it would be a good time to revisit the issue.

Rails History: With rails I used fixtures for test data and had to hand code data loaders for production. I put the loaders in lib/tasks. They were rake tasks that did the initial loading and/or refreshing of data for a specified environment. So data loading for a production or development was a manual task. Not a huge burden, but just one more thing to remember.

Now that I'm in the Grails realm I have created automatic loaders that are environment aware. A DatasetBootStrap class loads data for development, test, and production whenever the server starts. This is in line with what hibernate does when it starts up--verify that the domain models match the current data source, and make changes where necessary. In rails this is done with migrations--a good idea, but yet another manual step.

DatasetBootStrap: The DatasetBootStrap class first determines the current environment, then loads the appropriate dataset classes and invokes the "load" method (or closure if you will). A list of datasets is configured manually for each environment. This approach makes it possible to share loaders for all three environments (or four if you use staging). Some data is just for test. Other data, like a list of US States, is loaded for all three environments. The main point is nothing is repeated, keeping my code as DRY as possible.

Datasets: I currently keep my datasets in src/groovy/datasets, but I think that it's time to move this folder into grails-app/conf/datasets. I also use a package declaration (package datasets) to insure that dataset classes don't conflict with other classes (I've had problems with this in the test structure when trying to create unit tests that have a name conflict with integration tests).

Simulating Rails-Like Test Fixtures: This was the easy part. In each test that requires a specific dataset, the setUp() method is used to load as many datasets as necessary to create a solid test environment. Loading all the data is as easy as calling the DatasetBootStrap's init() method.

Grails Integration: My goal for Datasets is to propose it's use for all grails projects by integrating it into the grails core. I have a bit of refactoring to do before this is practical, but would welcome any comments or suggestions from other grails (or rails) users as to what features a Dataset loader module should include.

Survey Project--A Grails Implementation

I started small grails application today to enable creating surveys and questions and tracking questions and results. The domain classes are:

• Survey: the primary table that defines a survey project
• SurveyQuestion: object to hold a specific question mapped to a survey
• SurveyResponder: the person that is answering the survey questions
• SurveyResponse: the recorded answers that the reponder enters

And here is the proposed ER diagram, created using dbDesigner.


As the diagram shows, surveys have many questions, questions have many responses. Surveys also have many responders and responders have many responses. You can also see that this is a very simple survey with no branches.

Grails/GORM Implementation:
I began by generating the Survey and Survey question domain classes and basic validation tests. Next was to create the SurveyReponder and SurveyResponse classes. The implementation was straight forward, but I had to fiddle with the domain class definitions to get hibernate to understand my intent. Just a matter of specifically defining class references in dependent models rather than depending on the 'belongs_to' and 'has_many' declarations.

Datasets and Loaders (Fixtures):
Unlike Rails, Grails doesn't use fixtures for test data. Lucky for me I have a Grails enhancement the solves this based on a sandbox proposal from the Grails guys. So I generated the dataset loader classes using: grails create-dataset [domain]. This creates the following:

• a DataLoaderBootStrap class in grails-app/conf/ (unless it exists)
• a DomainDataset class in src/groovy/datasets
• a set of random data to be loaded

An example of what gets created is here:

> grails create-dataset survey
> ...
> file grails-app/conf/DataLoaderBootStrap.groovy exists...
> created file src/groovy/datasets/SurveyDataset.groovy
>

The SurveyDataset class looks like this:

class SurveyDataset {
dataset = {
def set = [
[ name:'879',description:'510',email:'680' ],
]

return set
}

load() {
dataset().each { data ->
obj = new Survey( data )
obj.save()
}
}
}

As you can see, the data is contrived but all of the fields are present and other than the actual data, the script is good to go. I'll show how this data can be loaded from the console, command line and within test scripts in a later post.