Javascript Asynchronous Operations

Javascript is single threaded.  That's good.  But to take advantage of our ever increasing multi-processor world, we need to adopt functional solutions.  Thats where asynchronous operations come in.

Javascript applications that run in the browser or in node have asynchronous events occurring all the time.  This is especially true in node where most actions return results through asynchronous callbacks rather than direct return values.  This takes some time to get used to.

Consider the following recursive function/snippet:

var list = getList(); 
var nextListItem = function() {
    var item = list.pop();
 
    if (item) {
        // process the item...
        nextListItem();
    }
};
nextListItem();

The recursive list item processor will work fine if the list is a reasonable size. But, if the list is huge, then one has to worry about the size of call stack. With every iteration it grows by one call, so it may exceed memory limits and fail (in a very bad way) if there are too many iterations.

Now consider this small change:

var list = getList();
var nextListItem = function() {
    var item = list.pop();
 
    if (item) {
        setTimeout(function() {
            // process the item...
            nextListItem();
        }, 0);
    }
};
nextListItem();

The stack over-flow problem is now eliminated. Why? Because javascript's event system handles the recursion–not the call stack. When nextListItem runs, if 'item' is true (not null) then the timeout function is defined and pushed to the event queue, then the function exits–and the call stack is now clear. When the event queue runs it's timed' out event, the item is processed and nextListItem is recursively called.

Again, the method is processed from start to finish without a direct recursive call, so the call-stack is again clear–regardless of the number of iterations.

Now consider this example:

var userList,
    cleverProgrammerBonus = 10000;

var userUpdateCallback = function(err) {

    if (err) return updateCompleteCallback( err );
    nextUser();
};

var nextUser = function() {
    var user = userList.pop();
    if (user) {
        user.pay += cleverProgrammerBonus;
        dao.updateUser( user, userUpdateCallback );
    } else {

        updateCompleteCallback();

    }
};

var userFetchCallback = function(err, list) {
    if (err) return updateCompleteCallback( err );
    userList = list;
    nextUser();
};

dao.fetchUsers( userFetchCallback );

As with most functional, asynchronous solutions, it's best to start at the bottom–the dao fetch of users. The fetch callback sets the user list then invokes the recursive nextUser() to get the iteration started. Then the user is updated asynchronously, and when it completes updateUpdateCallback is invoked. This callback continues the recursion by invoking nextUser. This is repeated until the list is empty.

There isn't a need for setInterval ( or nextTick() ) because the dao calls to update the user are asynchronous–always invoked from the event queue, not the call stack. So the call stack remains static no matter how many users are in the list.

Its also important to realize that the function dao.updateUser doesn't really start working until nextUser has exited. It's queued to do some work, but not at the direction of the call stack–just the event queue.

Asynchronous event driven operations.  Another reason to love coding in javascript.

Javascript Coding Standards

I love javascript. I love how it's flexible enough to be as expressive as you want.  The power of closures used in the traditional way or simply as function pointers like in the old C days.

Javascript's flexible nature enables an infinite combination of styles.  From prototypical, IIFE, to classical where functions are actually called "classes" when no actual classes exist.

This flexibility can lead to endless discussions of "what's best" or "what's most efficient" and worse can lead to project code that can resemble a big steamy bowl of pasta with brownish marinara.  As a team leader, the last thing you want is to have to code walk dozens of coding styles.  As a coder and team member the last thing you what to do is traverse hundreds of lines of code that has bad indentation or horrible style.

So, as a manager, you need to adopt a strict coding standard and ruthlessly enforce it.

Coding Standard Objectives

Here is a short bullet list.  The standards should...

• match the team's current experience and talent,
• should closely follow at least one major published standard,
• should have justifications for each decision,
• and include team consensus.

Team Experience

Javascript isn't new, but it's new to many programmers.  If your team has years of java and c++ experience, the wide-open flexibility of javascript will come as a frustrating surprise.  If your team has a ruby influence, then you have a leg up.  Understanding closures, dynamic typing, etc will ease the transition to production ready javascript.  If your team is primarily PHP based, then they know a bit about javascript but probably don't understand its full capability.

For the Java Team

So lets begin with the java programming team--what standards would suite them best?  The choice is easy--use a classical implementation with factory generated components that implement IoC through parameter injection.  Think of it this way, every file is a class defined by a named function variable (the function is still technically anonymous).  Its injection is always through construction (think ruby opts), and always factory built.  Files are organized as pseudo-package folders that may (or may not) have package namespaces.

The coding style is very similar to the java specs.  K&R formatting, title cased classes, mixed case variables, no underscores.  Complete separation of concerns, i.e. MVC plus services, delegates and factories.  And, unit tests--either TDD or BDD for everything.  And not just the normal tests, but also tests to find bugs that the compiler would normally find.  (mocha is probably the best test platform for both client and server javascript today).

Published Standards

Douglas Crockford.  JSLint.  Javascript, the good parts.  If you can only read one javascript book, please read "the good parts".   Coding conclusions aren't drawn on what is best--just what the alternatives are.  But, he does describe what he feels are some best practices for coding style, indentation, curly braces, etc.  And, if you have the time, watch his video series on javascript.

For good published standards, start here:

• google standards.
• yahoo standards.

Standards you probably want stay away from:

• if you are writing applications, the don't follow these standards; utility modules maybe, but...
• M$ standards--known only to a few Seattle east-siders.

Our Standards Document

Here is a link to our current javascript coding standards.  For the most part, this is for client applications, but most parts apply to node/server projects as well.

Conclusions

My best advise is to find a good published standard, modify it for your team's abilities, then stick with it.  You will need to add rules as time goes on--because coders will always find differing ways to interpret the rules.  So be prepared for additions--not so much changes, just clarifications.

Happy coding...

Easy Mac Installs--MySQL & NetBeans

I just went through two very easy installs on my new mac; the first, MySQL 5.1.5. A few years ago, when I installed on my old mac it was the typical tar ball install that linux users are familiar with. Not bad, especially if you have a canned script to remind you of all the steps.

But now there are two dmg package files that do most of the work for you. And, it installs in /usr/local with a dynamic link to mysql--just like it's supposed to be. The only steps left were to remove the anonymous users, supply the root and admin passwords and reboot to start the service. Easy.

Next was NetBeans 6.9. I have been using 6.8 for a few projects, and have loved the support for groovy and especially grails, but I have held off upgrading while in full development mode. But now, with a new grails project in it's infant stages, I decided to make the move.

Installation was easy. I did have to manually replace the 6.8 item with 6.9 on my doc, but other than that all went easy. It found my current projects and I was able to run unit tests from within the IDE. Once again, easy.

Happy coding...

Grails 1.3.4, Groovy 7 and NetBeans 6.8

After about a year of doing nothing but Adobe Action Script projects, I'm finally back to a java/groovy/grails project. So, I downloaded the latest grails and fired up my version of NetBeans (6.8) and generated some code. Here's what I discovered...

First, NetBeans is very groovy/grails friendly. I was able to generate the project inside the IDE. I then dropped to the command line to generate domain classes and tests. I was able to run the full test suite inside the IDE, but I still prefer the command line.

I noticed that the generated code doesn't include an ant script, but just as well. I'm so sick of the XML crap, and the grails command line does everything you need, so...

Next step is to generate controllers, and maybe a few views, but I usually use grails as a JSON or XML server with a Flex/Action Script UI. I'm also creating all the unit tests as I go and running them from inside the IDE and from the command line.

Next post will discuss the controllers and any problems related to getting the application into production. So far everything works as advertised--better than previous versions, and very productive. My new best friend...

UMLet UML Diagraming

Just had to comment on a very useful new(ish) project for editing UML documents called UMLet. No frills or bs* just a good productivity tool in the open source space. Human readable file structure to enable old timers like me to hack in attributes with vi.

Converts your drawings to png, jpg, eps, pdf, svg, etc from inside the UI or even from the command line (a real programmer's tool).

It comes in two flavors 1) eclipse plugin, and 2) standalone Swing application. They have a few skins that help ease the swing pain but who cares about the L&F, I just want a tool that works with me instead of against me.

So, thanks to the UMLet team for a great product!

HTML5, CSS3, Apple, Google and the W3C

About a year ago, I remember responding to a question from a colleague concerning Flex in comparison to HTML5 by saying that HTML5 was not ready yet. For the most part, I still hold that opinion but with additional qualifications.

We are currently bidding on a job that requires iPad as the target device and another that needs a private label application that will work on the desktop, iPhone and eventually Android and Blackberry. After logging many hours researching objective-c and HTML5/CSS3 my first thoughts were that all development needed to be done in objective-c. But, after viewing some of the other videos from Apple, MIT, and Stanford, and doing some small tests, I think that most applications for the iPad/iPhone can be implemented completely in HTML5/CSS3. So here is my revised/updated response...

If you have complete control over the target web browser (i.e., webkit/khtml based) then HTML5 and CSS3 can't be beat. Using the 'canvas' tag/object enables drawing anything that you can draw with flex/flash and also includes drawing text (not possible with flex). Using CSS3 enables all the transitions that are currently available in flex including easing methods. Using WebSocket, provides peer-to-peer real time communications. So if your target includes Safari, iPhone, iPad, iPod/Touch, Google Chrome, Opera, Firefox, as well as the Android, Sprint and Blackberry browsers, the new technology is ready today.

The missing piece is IE, currently at 70% of desktop browser installations. An application that needs to run only in a standard client browser is still better off using Flex (95% penetration) until Microsoft complies with the latest W3C specs. But, the flash player will probably never make it to small devices (it's a hog), so if the requirement is to run on desktop and phone, especially with touch gestures, HTML5/CSS3 is probably a better choice. The down side is waiting for Microsoft to catch up with the fully developed world and hoping they will fully comply with the W3C.

Action Script Threads for Report Generation

My requirement: to create a complex report that may take several seconds to run without blocking the user's interaction with the application. In java the solution is simple: create a thread and invoke start(). In action script, it's a bit more difficult.

Action Script doesn't support threads directly out the the box but it is run in a multi-threaded framework, so I knew there must be a solution. I took some time to research how other flex developers were implementing Action Script threads and concluded that the best solution was to simply use the Timer class to create a new threads.

The timer class has an asynchronous event called TIMER_COMPLETE that is perfect for creating a bloated, heavy weight thread. But, if all you need is a separate thread to run while creating a huge report, this does the trick.

One of the practical considerations is that you will probably want a supporting class to encapsulate run parameters for each thread you create. For reports this works well because there is enough logic in each individual report to warrant its own class. And as long as the report class implements the timer start method then you are in good shape.

The simplest implementation looks like this:

public function start():void {
var timer:Timer = new Timer(1, 1);
timer.addEventListener(TimerEvent.TIMER_COMPLETE, run, false 0, true);
timer.start();
}

// override this base class to implement the run worker
protected function run():void {
// create the report or whatever you need...
}

Implement the run handler and you are set. The class can encapsulate any run time parameters that you require including a common dispatcher to dispatch a 'COMPLETE' event when the process is finished. Easy.

This is an acceptable work-around for a language that does not support threads right out of the box. But for the rare occasion that you really need a thread to enable interaction while a heavy-weight, time consuming task is running, this really does the trick.

Happy coding...