Grails Application Deployed to Slicehost

We deployed our beta application on slicehost virtual server. The slice is 1GB using fedora 7, mysql 5.1, jetty 6.1.6 and resin 3.2. Groovy version 1.5.4 is installed on the slice and we used Grails 1.0.1 for development, but it's not installed on the slice.

The application at this point is minimal, but is scheduled to go live later this month. I can't go into to much detail, but application provides a data service between United Parsel Service and a major customer to schedule shipments of large and small freight.

Groovy DateTime Math

DateTime is capable of simple increment (++), decrement (--), and all the add/roll methods provided by Calendar. Here is an example of some of the ops:

refDate = new Date()
dt = new DateTime(date:refDate)
tomorrow = dt + 1
yesterday = dt - 1

assert refDate == dt // insure that the original did not change
assert refDate + 1 == tomorrow
assert refDate - 1 == yesterday
assert refDate + 7 == nextWeek

Groovy Equality: Groovy evaluates '==' as equality, not identity. So it was necessary to override equals() to compare to the millisecond and return true or false. The override makes it possible to return true when the underlying milliseconds match whether from Date, Calendar, Long, or DateTime.

Problems with Milliseconds: When working through the plus and minus methods I made the mistake of using the raw milliseconds to add days. This worked fine on my linux machine, but mysteriously died on OSX. What I neglected to take into consideration was the switch between standard and daylight time (luckily my development cycle is close to the change or I would have missed it). So, a quick fix was to rely on java.util.Date to add days and return a new DateTime copy. A better approach may be to use groovy's Duration classes...

Durations: The groovy library includes a set of duration classes in the groovy.time package. The duration classes are containers for years, months, days, hours, minutes, seconds and millis. They support basic math functions to add and subtract durations--very groovy. DateTime uses the duration classes for basic math with the following methods:

now = new DateTime()
current = now.asDuration()
twodays = new Duration(2, 0, 0, 0, 0)
dt = DateTime.fromDuration( current + todays )

Combined with groovy's TimeCategory, you can do this:

use(TimeCategory) {
  future = new DateTime(date:2.weeks.from.now)
  oneWeek = 1.week  // this is a duration

  // schedule something once each week for the next 5 weeks...
  dates = []
  5.times { dates << it.weeks.from.now }
}

Additional utility methods and parsing to come next...

Groovy DateTime Comparisons

Those of you following my posts know that I'm working on enhanced date/time support for groovy. In a previous post, I discussed a new DateTime class that leverages the power of java's Calendar and updates the interface in a groovy way. Which brings me to after(), before(), and compareTo() methods.

Calendar implements compareTo() that accepts only a calendar object. If you pass it a Date and exception is thrown. How restrictively lame.

DateTime's compareTo() accepts multiple types that can be evaluated to the millisecond level. So, DateTime, Date, Calendar, even long types are accepted. This approach enables this to work:

now = [ new Date(), Calendar.getInstance(), System.currentTimeMillis, new DateTime() ]

yesterday = new DateTime() - 1
tomorrow = new DateTime() + 1
later = new DateTime(hours:23, minutes:59)

now.each {
 assert yesterday.before( it )
 assert tomorrow.after( it )
 assert laster.after( it )
}

DateTime. How groovy is that!

Groovy Temporal Support Continued

One area where ruby outshines groovy is in date and date time support. Ruby's date time support isn't more capable, just easier to use. Java, and groovy have Calendar--very complete, and very clunky. There is also joda time. Very complete, but a bit of an overkill for my tastes.

So here are the objectives:

• ability to add/subtract dates with integers representing days, months, etc.
• ability to subtract two dates to yield intervals representing days, hours, etc.
  
• ability to compare two dates
• ability to parse any type of date input, and most easily parse the standards
• ability to format dates in a wide variety of ways
  
• ability to create dates with integers and words like 10.days.ago or nextweek

Java's calendar class supports the first three requirements, parsing is available through SimpleDateFormat class. Groovy's TimeCategory enables some of the most common integer enhancements to create dates based on common temporal works. So, the pieces are in place, the next step is to wrap this in a groovy way.

DateTime, the Groovy Calendar: The first step is to create an easy (easier) to use, full featured DateTime class. One approach is to simply extend GregorianCalanedar and add the new methods, mostly getters and setters. That's were I started but soon discovered a better solution was to contain the calendar object, and use invokeMethod() to pass methods and arguments. Then, rather than define properties of the DateTime object, define getters and setters in bean style making the variables appear as if they were members. For example, I define these two methods:

int getYear() { return calendar.get(Calendar.YEAR) }
void setYear(int yr) { calendar.set(Calendar.YEAR, yr) }

Now, I can use short hand to access "year" like this:

def dateTime = new DateTime(year:2008)
assert dateTime.year == 2008
dateTime.year = 2020
assert dateTime.year == 2020
dateTime.year++
assert dateTime.year == 2021

Groovy, right? And adding other methods like setSeconds(), clearTime(), isWednesday() were easy one-liners to implement. I also took the liberty of shifting the month to 1..12 rather than 0..11, the calendar default.

Constructors: This was also very groovy. After I added the getter/setters, constructing with the virtual member vars was easy. So the DateTime class can be constructed like this:

dt = new DateTime(year:2020, month:4, day:1)
dt = new DateTime(hours:0, minutes:0, seconds:0)
dt = new DateTime().clearTime()
dt = new DateTime() + 5

Date/Time Formatting: The Rails group enhanced ruby's DateTime.to_s() method by adding formatting like this: to_s(format). This makes a lot of sense. To implement this I use a format hash that stores formatting strings by name. The map is static to enable use application wide (I may regret this later) and there is a default format member variable that controls how toString() is formatted. I also added toString(format) to display a date in multiple formats like this:

// default format is a modified ISO-8601 called 'db'
dt = new DateTime(year:2010, day:25, month:4, hours:15, minutes:35, seconds:29)
assert dt.toString() == '2010-04-25 15:35:29'
assert dt.toString(8601) == '2010-04-25T15:35:29-0800'
assert dt.toString('mdy') == '04/25/2010'
assert dt.toString('dmy') == '25.04.2010'
assert dt.toString('dMony') == '25-Apr-2010'

Other formats are available in the formats hash. The actual formatting is done using SimpleDateFormat that supplies a wide range of date formats and is compatible with Java 1.4, important in the groovy community.

A Groovy Date Parser: I've seen many approaches to this, usually making use of java's SimpleDateFormat class with a boat load of formats. I think there may be a better way. First, extracting the date parts, usually the numbers and constructing date time objects based on the numeric values. I'll discuss this implementation later...

Groovy Temporal Support Classes

Groovy adds a rich set of features to java including closures, ranges, dynamic metaClass, access to existing java code base, etc. What is missing is full featured date processing. One approach to fixing this is to use existing libraries of java date extensions, such as joda time. Today I decided to implement some basic date/time features without a third party dependency. I may end up using joda as the project unfolds, but for now I'm trying to resist the temptation.

HourMinuteSecond: This class acts as a container and formatter of hours, minutes and seconds similar to groovy's TimeDuration class. In native form it's completely mutable, so not thread safe. But it can be made immutable by invoking asImmutable(). I'll explain how groovy makes this easy to implement later on.

The class API is as follows:

static HourMinuteSecond now()
static HourMinuteSecond fromDate(date)

def parse('hh:mm:ss')
def set(hours, minutes, seconds)
def set(Date dt)
def next()  // enables ++
def previous()  // enables --
def add(hours, minutes, seconds)
def sub(hours, minutes, seconds)
boolean isZero()
String toString() // formatted as hh:mm:ss
void addZeroEventListener(closure)
def asImmutable()

The object can be used as a simple up/down timer measured in seconds. When attached to a one second ticker thread, only the next() method needs to be invoked to tally up the seconds. Used as a count down timer, an event is triggered when zero is reached.

The methods are actually closures, so creating the immutable clone was as simple as pointing all the mutators to a single closure that throws an UnsupportedOperationException when a mutating method is called. This mimics how groovy's List.asImmutable() method works.

I will post more as the temporal project unfolds, and supply a link once the library is ready for distribution.

Grails Dataset Loaders, Part 2

About a month ago I posted a few thoughts on Dataset Loaders for Grails. Now that one of my Grails projects is going into beta, I thought it would be a good time to revisit the issue.

Rails History: With rails I used fixtures for test data and had to hand code data loaders for production. I put the loaders in lib/tasks. They were rake tasks that did the initial loading and/or refreshing of data for a specified environment. So data loading for a production or development was a manual task. Not a huge burden, but just one more thing to remember.

Now that I'm in the Grails realm I have created automatic loaders that are environment aware. A DatasetBootStrap class loads data for development, test, and production whenever the server starts. This is in line with what hibernate does when it starts up--verify that the domain models match the current data source, and make changes where necessary. In rails this is done with migrations--a good idea, but yet another manual step.

DatasetBootStrap: The DatasetBootStrap class first determines the current environment, then loads the appropriate dataset classes and invokes the "load" method (or closure if you will). A list of datasets is configured manually for each environment. This approach makes it possible to share loaders for all three environments (or four if you use staging). Some data is just for test. Other data, like a list of US States, is loaded for all three environments. The main point is nothing is repeated, keeping my code as DRY as possible.

Datasets: I currently keep my datasets in src/groovy/datasets, but I think that it's time to move this folder into grails-app/conf/datasets. I also use a package declaration (package datasets) to insure that dataset classes don't conflict with other classes (I've had problems with this in the test structure when trying to create unit tests that have a name conflict with integration tests).

Simulating Rails-Like Test Fixtures: This was the easy part. In each test that requires a specific dataset, the setUp() method is used to load as many datasets as necessary to create a solid test environment. Loading all the data is as easy as calling the DatasetBootStrap's init() method.

Grails Integration: My goal for Datasets is to propose it's use for all grails projects by integrating it into the grails core. I have a bit of refactoring to do before this is practical, but would welcome any comments or suggestions from other grails (or rails) users as to what features a Dataset loader module should include.

UPS Web Services Interface

One of my customers has a requirement to use United Parcel Service to ship large and small packages. UPS has a great set of documentation, but they handle large and small packages with different technologies. Small packages are simple XML request/response transactions that provide methods for requesting a shipment, accepting shipments, and voiding shipments. Large packages allow you to request a shipment, but the request/response is all in SOAP, something of an overkill in my opinion.

UPS requires lots of data concerning who, what and where to establish their pricing. Data is sent to a URL/end point and a response is returned. The response includes pricing and tracking numbers, as well as image data for printing barcoded shipping labels. So, it is mainly a data transfer operation-- a single service.

SOAP is great if you have lots of defined services, like weather in Seattle or Paris or zip code 94705. Or conversion of units from/to. But for simple data transfer, REST, or just plain old XML request/response is much more efficient.

Here is a diagram of the complete data transfer...


...

As you can see, there are lots of communications. The first question is, why is there a server in the middle? Well, here's the deal--the SunReturns client is just that, a client. The data really resides at the DataServer--the client simply persists it's own private session. The UPS server is at their site. All communications are via SOA/services--some simple request/response (ala REST) and others use SOAP.

The technology used in the middle is grails/groovy. The SunReturns server is currently a mixture of JSP (legacy) and groovy/GSP. The groovy components are XML only and include controllers, services, domain objects, etc--similar to grails but without the Hibernate layer. It talks to legacy databases, both Oracle and MySQL--not impossible with grails, but definitely outside the conventions.

Conclusions: The natural loose coupling of SOA provides a new vehicle to solve business problems. At the same time, distributing business processes across diverse servers provides a workable migration path to newer technologies. Groovy, grails, and simple XML transport provide a way to implement SOA with a minimum of effort.