Sacrificing Sleep

I find myself asking this question a lot.  How has this little Javascript library named jQuery managed to so significantly improve the quality of my life?

I find that one of the most complicated aspects of building web applications is the “front-end.”  I define this to include the code that the browser runs - the HTML/CSS and Javascript - as well as the “top layer” of the server-side code that deals with control and view rendering.  In order to build a relatively compelling user experience in a web product these days, you need to use a lot of Javascript. Javascript is a necessity for creating rich dynamic user experiences in web apps, using the popular practices of DHTML and AJAX.  This is great, but it can also require a lot of development time and add a good deal of technical debt if you want to create a lot of slick features.

Every time we’d consider adding a slick UI feature that requires Javascript, we’d have ask ourselves if we have the time to do this, and if we’d be willing to risk all the potential browser compatibility issues.   Well, that is until jQuery came along…

Now, every time we want to add another slick feature, we ask jQuery and typically get an answer we like. And, when we go to test in IE, it almost always just works. Wow. So, just to sum up this ode to jQuery, here are a few of the thing that make it awesome and why you should use it for your projects:

1. It has an amazingly elegant programming model that abstracts DOM manipulation into simple method calls that just “make sense”. Any sort of DHTML logic is now trivial. AJAX is also simple.
2. It ensures that your code will work across the major browsers.
3. It is, as they say on their site, “unobtrusive”. It is. It does not care what technology you’re using on the server side. Use it with anything. For AJAX logic, it just deals with URLs and good old HTTP (standards of “the web”) and there is no associated server side component such as a “tag” library that generates javascript in your pages (ugh).
4. There is a thriving community of Plugins built on top of the core API that run the gamut of things you’ll need to do. Whatever you’re trying to do, look there first!
5. Their site and documentation is a model for open-source projects. Nice work!

I’ll leave it at that. Thanks jQuery, for being awesome.

This sounds a lot easier than it really is. But, I do believe that it mainly comes down to forcing yourself to adhere to this philosophy. Paul Graham says it well here:

Here it is: I like to find (a) simple solutions (b) to overlooked problems (c) that actually need to be solved, and (d) deliver them as informally as possible, (e) starting with a very crude version 1, then (f) iterating rapidly.

I like that. I try to do that. I veer off course at times. But I persevere.

I recently had to figure out a solution to a problem that others may face, so I thought I’d share. The solution turned out to be pretty trivial once it was understood, but it took a little digging to get there.

I wanted to use separate sub-domains for the same web application, in which each sub-domain would point to the same app but would cause slightly different behavior within the app. Specifically, the sub-domains correspond to geographical region names and they cause the app to load content that is geo-coded within the specified region.

This part was easy enough to set up. You can add a wildcard A record into DNS so that all sub-domains map to the same IP, and then build logic into your app that parses the sub-domain and handles things accordingly. The problem was that the user’s session would be lost if the user clicked from one sub-domain to another within the same app. This clearly would not work, as the user’s login state would be lost, etc.

This was happening because of the rules related to which cookies in a browser are allowed to be passed into the header of a request: only those cookies which were set by the same domain, or sub-domain, of the server domain are passed over the http request. After some digging, I realized that the real root issue here was that tomcat was automatically setting the JSESSIONID cookie into the HTTP response header in such a way that it was only associated with the specific sub-domain of the request/response at hand. So, for example, if newyork.sportsvite.com was requested, the browser would only send back the JSESSIONID cookie that was explicitly set by requests to that same sub-domain. What had to be done was to somehow override this behavior and set the Domain property on this cookie to the value of the base domain, sportsvite.com. With this logic in place, the browser would send this cookie back on all requests to any sub-domain under that base domain.

In order to do this, you need to groom the cookies on each incoming request so that this Domain property is set accordingly. Once this is done, all requests coming in for different sub-domains will associate that same cookie to the request and therefore use the JSESSIONID value to match up the same single session within servlet container.

So, it’s really easy in the end. Basically, this cookie grooming logic needs to be placed somewhere along the request processing flow - this could be a Servlet Filter, a Tomcat Valve, or even down in the app framework you’re using. In my case, I chose to keep it down in the app framework, and thus added logic into a custom subclass of the Struts RequestProcessor class. Here is the generic method that you could adapt to your needs:

void performCookieGrooming(HttpServletRequest request, HttpServletResponse response) {

Cookie[] cookies = request.getCookies();
HttpSession session = request.getSession();

String cookieDomain = “sportsvite.com”;
String cookieNameSessionId = “JSESSIONID”;

if (session.isNew()) {
/**
* Create and add a NEW JSESSIONID cookie with the domain set accordingly
*/
Cookie newCookie = new Cookie(cookieNameSessionId, session.getId());
newCookie.setDomain(cookieDomain);
newCookie.setMaxAge(-1);
newCookie.setPath(”/”);
if (request.isSecure()) {
newCookie.setSecure(true);
}
response.addCookie(newCookie);
} else {
/**
* Loop through all cookies in the request, and find the one(s) corresponding to JSESSIONID
*/
if (cookies != null && cookies.length > 0) {
Cookie cookie;
for (int i = 0; i < cookies.length; i++) {
cookie = (Cookie) cookies[i];
/**
* Do the manipulation for session Id cookie(s) only
*/
if (cookieNameSessionId.equals(cookie.getName())) {
String value = cookie.getValue();
cookie.setDomain(cookieDomain);
/**
* Also, to be sure, set the sessionID into the value
* …not sure if this is really needed.
*/
cookie.setValue(session.getId());
}
}
}
}
}

Notice that this method loops through cookies and grooms those that are JSESSIONID cookies. It works fine, and now I’m rolling with multiple sub-domain on the same app. Hope this helps someone!

Building and releasing a software product can be a very draining endeavor. And, those who have done it to some degree of success can attest to how difficult that final 5% of the effort can be - the last part that leads up to the launch. It’s in this final stretch that your list of “to-dos” seems to become regenerative. Every item you knock off the list seems to have another pop up in its place. And, the granularity of these final to-do items gets super specific - even down to the minutia of minute measured tasks. Also, a lot of key realizations occur during the final stretch that were somehow missed along the way. The reality of the imminent release creates a sense of urgency that causes people to think about things in a different light, and thus see new things.

During this final 5%, what is really happening is the intensity level of the team is rising significantly in order to reach optimal productivity when it matters most. It seems like no matter how much forethought and attention is given to making a project go smoothly and follow a predetermined schedule, the final stretch is always an intense crunch time. I think this occurs naturally and even reflects the balance of energy in the universe. The intensity at which a person can work varies over time, leading to variances of productivity. Sometimes you’re able to crank out amazing amounts of work, and sometimes you’re just not in the groove and find yourself “going through the motions”. Writers even have a name for extreme cases of low individual productivity: “writer’s block”. In software development, the best time to have high productivity is during that period leading up to the launch so you can be certain you’re getting your best possible product out there. So, the final 5% of launching gets to be that super intense period on any project. I got another coming up, so I guess I better go get intense now.

One of the borderline ridiculous claims I like to make is that I can actually get work done while I’m sleeping. It sounds crazy, but it’s true. This would be pretty hard to do in some lines of work, but in software development, it can really happen. I imagine it can happen in other of lines of work too, but I can best comment on software development.

I myself have been personally aware of how this works and have experienced it for some time. But, I had never been able to articulate the concept, and had never seen it described by someone else.

Until now. Paul Graham just broke it all down very well in one of his best essays yet, Holding a Program in One’s Head. This one statement sums it up pretty well:

“You hold the whole program in your head, and you can manipulate it at will.”

If you love the software you’re building, then you’ll be more likely to have the code loaded into your head. And while you sleep, you can subconsciously “solve” problems… waking up to see things in a new light. So, if the actual tangible output of software development type work is code - code that is written in a programming language and runs on a computer - then it follows that any progress made on that code equates to actual work progress. You just need to “download” any changes you made to the code while in your head, so you can run it on the computer. Nice.

One of the favorite topics my friend Bob Daly and I like to discuss is the concept of the software “stack”. It’s pretty standard practice by those who build software systems to think about the different parts of a system in terms of their relative positions within the layers of the vertical “stack”.

Starting at the hardware, and working all the way up to the end user, the stack includes all the software components that implement general capabilities. This practice of thinking in stack terms helps one stay mentally organized when making software component selections for a system, and it helps when communicating about and integrating such components.

The stack metaphor makes a lot of sense in the world of software, in which one man’s abstraction is another man’s implementation. One of the most common patterns in software engineering (and many other fields) is to implement generically useful components that other developers can integrate into their systems. And, this integration generally follows a “black-box” approach, in which only the abstraction (or interface) must be understood in order to use the component. The implementation under the covers need not be understood. In this sense, the interface provided by a software component identifies its effective position on the stack, relative to other software components.

Interestingly, the stack metaphor also reflects the history of software in general. We can think about the evolution of software from the early days when the only software a computer had installed was the OS… to the advent of the GUI… to the standardization and success of the database… and all the way to present day in which we have standardized components present at several different levels. These stages of history roughly correspond to the stack layers. As time passes, the stack grows taller and denser.

I think it’s safe to say that most software developers find it useful to think about software components in terms of “stack position”. This way of thinking comes naturally, since it reflects the current state of the evolution of software. The stack provides a frame of reference that enables one to ask questions like “what other options do we have at this layer?”, or “where does this component fit into our stack?”.

So, having given a little rundown of the concept of a software “stack”, this brings me to the point of this post. How do we identify the generic layers of the software stack that could be universally accepted? This is an exercise in high level software categorization. To start, we need to ask the question what is the definition of a layer on the stack? I will propose my definition:

A layer of the software stack represents a generic capability that is commonly present within software systems, and usually drives the creation of community-defined standards.

So, firstly, the stack layer corresponds to some general capability that software systems typically have. Of course, not all systems must employ software components from all layers of the stack… the design of some systems may exclude certain layers. Rather, the stack is a frame of reference that describes the generic layers which a system could employ. Secondly, each layer is centered around a suite of standards. These standards are defined by the community of developers and other stakeholders in an effort to support interoperability across implementations. So, as time passes and the software world evolves, a solid stack layer becomes more mature, standards are defined, various implementation alternatives are created, and ultimately a set of commodity components emerge.

Back to the main question of this post - how do we define a set of stack layers that would loosely represent the breadth of the software world? I’ll take a stab at it. This is the set of canonical stack layers we came up with:

Packaged App

UI

Middleware

Data

System

Yea, I know - this is too simple… What’s the point, right? Well, the point is that you have to keep it this high level in order to make it universally acceptable. And once you have this high level frame of reference, you can then drill down to the next level of granularity. Pretty much any software component could be assigned into this simple stack. It’s generic enough to do this. And even if a component is not exactly matched to a specific layer (as most won’t be), you could assign it above or below a stack layer. In fact, this is the only reliable way a universally accepted stack can work… that is, to think of the stack as a set of “way points” (as Bob puts it), and then assign the software component to some relative position along the stack. This way, you get the benefit of classifying a component into the stack without getting hung up in unending debates about exactly where it goes. The net result is that you can assign a component a stack index value, which would be relative to the way points. For example, I’d assign the java object relation mapping (ORM) framework Hibernate a stack position above the Data layer, but a little below the Middleware layer.

Try it. Think of a software component, and where it would go in this stack. Are there any components that could not be assigned? Probably not. Please shoot me a tip if you can think of any. The goal of this post is to work toward a high level software stack that could be universally accepted. I’ll be coming back to this concept in later posts, and providing more context. Thanks for reading.

Time for me to join this great conversation, and set up my own blog! For so many reasons, I love the “social web” - that being this Internet ecosystem that’s evolving, and enabling so many new ways of social interaction and engagement. I’m interested in the technology that makes it possible, the sociological aspects, and the startups that are emerging from it.

So, this is what I’ll write about in this blog. It’s what I do - check out sportsvite.com. It’s what I love (professionally). It’s work and a hobby. This’ll be fun.