Rhino + Maven for JavaScript compression @ build time

AJAX-based Rich Internet Applications (RIAs) make heavy use of JavaScript. To improve the user experience of RIAs we can minimize the number of (JavaScript) file requests and reduce their size.

These concatenation and compression operations can be done on-the-fly or at build time.
On-the-fly techniques intercept requests and perform merging and/or compression in real-time. They are simple(r) to implement, often involving URL rewriting combined with a compress script, but:

• don't scale, since compression costs grow proportionately to the number and size of the files;
• add latency, by consuming CPU resources during (precious) server response time;
• are harder to test and debug;
• rely heavily on caching for continued performance improvements, when research indicates 40-60% of users browse with an empty cache;
• don't work on all browsers
  

Compression at build time allows the use of more robust, often slower, compression methods (e.g.: a proper JavaScript interpreter instead of a regular expression) and/or combination of different methods (e.g.: run YUI Compressor after Rhino).
At build time we're unconstrained by the "real-time performance pressures".

Here's a step-by-step guide for a Maven/Ant build file using Mozilla's Rhino to serve pre-compressed JavaScript and significantly reduce application load time.

1.Setup

1. Download the rhino.jar from Dojo ShrinkSafe
   
2. Download, install, and configure Maven
3. Copy the rhino.jar to your Maven repository
4. Have a Maven-based Web-application ready
   

2.Add rhino.jar as a dependency to the POM

<dependency>
 <groupId>rhino</groupId>
 <artifactId>custom_rhino</artifactId>
 <version>0.1</version>
 <properties>
   <war.bundle>true</war.bundle>
 </properties>
</dependency>

3.Setup some useful variables on maven.xml

<!-- destination for the compressed JavaScript files -->
<j:set var="js.compression.dir" value="${typically_the_war_src_dir}"/>
<j:set var="js.compression.skip" value="false"/> <!-- enable/disable compression -->
<j:set var="js.compressor.lib.path" value=""/> <!-- path to the compressor jar -->

4.Fetch compressor and make it globally available

    <goal name="get-compressor" description="Set path to JavaScript compressor library">
<!-- get compressor from dependencies, to allow standalone use of goal -->
<j:if test="${empty(js.compressor.lib.path)}">
 <ant:echo message="Fetching JavaScript compressor from repository" />
 <j:forEach var="lib" items="${pom.artifacts}">
   <j:if test="${lib.dependency.artifactId == 'custom_rhino'}">
     <lib dir="${maven.repo.local}">
       <include name="${lib.path}"/>
       <j:set var="js.compressor.lib.path" value="${lib.path}"/>
     </lib>
   </j:if>
 </j:forEach>
</j:if>
<ant:echo>
 Path to JavaScript compressor library is ${js.compressor.lib.path}
</ant:echo>
</goal>

This way, get-compressor can be reused as a pre-requisite of all compression tasks.

5.Aggregate compression sub-tasks

<goal name="compress-js" description="Compress JavaScript across all site components">
<!-- Different sections might have different compression requirements -->
<j:if test="${js.compression.skip == 'false'}">
 <attainGoal name="compress-site-js"/>
 <attainGoal name="compress-forum-js"/>
 ...
</j:if>
</goal>

6.Compression task (to compress a single file)

<goal name="compress-site-js" prereqs="get-compressor" description="Compress JavaScript files for 'site'">
<ant:echo message="Compressing JavaScript files for 'site'" />
<j:set var="stripLinebreaks" value="true" />
<j:set var="js.dir" value="${path_to_javascript_files}"/>
<concat destfile="${js.dir}/site-concat.temp" force="yes">
 <filelist dir="${js.dir}"
   files="file_to_merge.js, another_file_to_merge.js, etc.js"/>
</concat>
<ant:java
  jar="${js.compressor.lib.path}"
  failonerror="true"
  fork="yes"
  output="${js.dir}/site-breaks.temp">
  <ant:arg value="-c"/>
  <ant:arg value="${js.dir}/site-concat.temp"/>
</ant:java>
<!-- move compressed files back from dest to src dir -->
<ant:move file="${js.dir}/site-breaks.temp" tofile="${js.dir}/site_c.js" filtering="true">
 <j:if test="${stripLinebreaks == 'true'}">
   <ant:echo message="Removing line breaks" />
   <filterchain>
      <striplinebreaks/>
   </filterchain>
 </j:if>
</ant:move>
<delete file="${js.dir}/site-concat.temp"/>
</goal>

This goal concatenates the 3 JavaScript files file_to_merge.js, another_file_to_merge.js,
and etc.js into a single (temporary) file site-concat.temp.
It then compresses site-concat.temp into site-breaks.temp (-breaks suffix indicates compressed file still contains line brakes).
Finally, it moves the content of site-breaks.temp into site_c.js (optionally removing line breaks) and cleans-up any temporary files and directories.

7.Compression task (to compress many independent files)

<goal name="compress-forum-js" prereqs="get-compressor" description="Compress JavaScript files for 'forum'">
<j:set var="stripLinebreaks" value="true"/>
<j:set var="src.dir" value="${path_to_javascript_files}"/>
<!-- Delete previously compressed files -->
<ant:delete>
 <ant:fileset dir="${src.dir}" includes="*_c.js"/>
</ant:delete>
<ant:echo message="Compressing 'forum' JavaScript files from ${src.dir}" />
<!-- create temp dir for compressed files -->
<ant:mkdir dir="${src.dir}/_compressedjs"/>
<j:set var="dest.dir" value="${src.dir}/_compressedjs"/>
<!-- compile JavaScript files to compress -->
<ant:fileScanner var="forumJSFiles">
 <ant:fileset dir="${src.dir}" casesensitive="yes">
   <ant:include name="file_to_compress.js"/>
   <ant:include name="another_file_to_compress.js"/>
   <ant:exclude name="*_c.js"/>
 </ant:fileset>
</ant:fileScanner>
<!-- loop through files and compress using compressor set in 'get-compressor' goal -->
<j:forEach var="jsFile" items="${forumJSFiles.iterator()}">
  <ant:echo message="Compressing ${jsFile.name}" />
  <ant:java
      jar="${js.compressor.lib.path}"
      failonerror="true"
      fork="yes"
      output="${dest.dir}/${jsFile.name}">
      <ant:arg value="-c"/>
      <ant:arg value="${src.dir}/${jsFile.name}"/>
  </ant:java>
</j:forEach>
<!-- move compressed files back from dest to src dir -->
<ant:move todir="${src.dir}" filtering="true">
  <ant:fileset dir="${dest.dir}" casesensitive="yes">
    <ant:include name="*.js"/>
  </ant:fileset>
  <j:if test="${stripLinebreaks == 'true'}">
    <ant:echo message="Removing line breaks" />
    <filterchain>
      <striplinebreaks/>
    </filterchain>
  </j:if>
  <ant:mapper type="glob" from="*.js" to="*_c.js"/>
</ant:move>
<!-- delete temp dir -->
<ant:delete dir="${dest.dir}"/>
</goal>

This goal loops through a list of JavaScript files compressing them one by one.

8.Caveats

Rhino compression removes all comments, so beware of IE-specific conditional compilation statements such as the snippet below (taken from http://jibbering.com/2002/4/httprequest.html):

...
var xmlhttp=false;
/*@cc_on @*/
/*@if (@_jscript_version >= 5)
// JScript gives us Conditional compilation, we can cope with old IE versions.
// and security blocked creation of the objects.
try {
 xmlhttp = new ActiveXObject("Msxml2.XMLHTTP");
} catch (e) {
try {
 xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
} catch (E) {
 xmlhttp = false;
}
}
@end @*/
...

There are many possible ways around this issue, namely:

• Use Ant to concatenate the critical code sections with the compressed files after compression
• Move all critical sections to a separate, uncompressed file
• Move all critical sections inline
• Fix the compressor and submit your patch to Dojo :)
  

9.Improvements

When I implemented the solution above, more than a year ago, a common complaint from fellow developers was that the compression task had the undesirable side-effect of slowing down the build -biiig time. A colleague pointed me towards Ant's Uptodate task, which I then used to implement conditional compression. This means files were only compressed if they had been changed. Conditional compression reduced the automated compression time from about 1 minute to 5 seconds on average.

To use conditional compression, replace the ellipsis in the script below with the content of any of the compression goals above.

<goal name="conditional-compress-js" prereqs="get-compressor"
                 description="Conditional compression of JavaScript files">
<j:set var="src.dir" value="${path_to_javascript_files}"/>
<!-- check timestamps to see if compression is required -->
<ant:echo message="Checking timestamps of JavaScript files from ${src.dir}" />
<ant:fileScanner var="jsFiles">
 <ant:fileset dir="${src.dir}" casesensitive="yes">
   <ant:include name="**/*.js"/>
   <ant:exclude name="**/*_c.js"/>
 </ant:fileset>
</ant:fileScanner>
<j:forEach var="jsFile" items="${jsFiles.iterator()}">
 <ant:echo message="Checking last-modified-date of ${jsFile.name}" />
 <uptodate property="js.modified" targetfile="${src.dir}/${jsFile.name}">
   <srcfiles dir="${src.dir}" includes="**/*_c.js" />
 </uptodate>
</j:forEach>
<j:if test="${js.modified}">
 <j:set var="compression.required" value="true"/>
</j:if>
<j:if test="${compression.required}">
 <ant:echo message="Modified JavaScript files. Compression required." />

 <!-- insert compression block here -->
 ...
</j:if>
</goal>

10.Further (potential) Improvements

• Clever use of Caching, to avoid unnecessary downloading of unmodified resources.
• Request parameter trigger to alternate between compressed and uncompressed JavaScript in real-time, a feature that is very useful for testing and debugging.
• Very clever use of Versioning to aid with caching; my friend and former colleague Robert, responsible for an ingenious solution, can write about it in a post of his own.
• In specific and controlled situations JavaScript namespaces can be shortened with tools like Ant, e.g.: <replace file="${file.js}" token="the.long.api.namespace" value="__u._a"/>.
• Clever use of HTTP compression
  

Comments

• I dislike the verbosity (by the very nature of XML) and obtrusiveness in the build configuration file.
• I believe the Maven plugin approach (in combination with Julien Lecomte's excellent YUI Compressor) is a better solution.
• With a few changes the scripts above can also be used to compress CSS resources at build time.
  

(Vaguely) Related Posts:
Measuring client-side performance of Web-apps
CSS clean-up @ build time

References
Custom on-the-fly compression
Make your pages load faster by combining and compressing javascript and css files
Minify
pack:tag

On-the-fly server compression
gzip, where have you been all my life..?

Compression @ build time
YUI compressor
Maven plugin for the YUI Compressor

Relevant Articles
YUI Performance Research - Part 1
YUI Performance Research - Part 2
Minification v Obfuscation
Serving JavaScript Fast
Using The XML HTTP Request
Response Time: Eight Seconds Plus Or Minus Two
Optimizing Page Load Time
Speed Web delivery with HTTP compression