Morning chat I overheard on the tube:
...
Little Girl 1: I wanna get into facebook
Little Boy: yeah, everyone's in it
Little Girl 2: wha' is it?
Little Girl 1: it's kinda like myspace...my sister's in it too. I really want to join
...
From the BBC Business News, 31 August 2007:
Barclays says that a "technical breakdown" in the UK's clearing system forced it to borrow £1.6bn from the Bank of England.
It is the second time this month that the bank has tapped into the central bank's emergency credit line, sparking fears it is facing a cash crisis.
...
On 20 August, Barclays was forced to take out a £314m loan from the Bank of England after HSBC was unable to process a last-minute request for the money.
A few weeks ago I (accidentally) found out that a pet-project of mine, the cross-lingual instant messaging, wasn't working anymore. Debugging showed the "3rd party translation sub-system" was always returning an empty translation.
The fix revealed my development and maintenance process was flawed:
public interface ITranslationEngine {
/**
* Translates 'text' from a given 'fromLanguage' into a given 'toLanguage'
* @param fromLanguage
* @param toLanguage
* @param text
* @return
* @throws Exception
*/
public String translate(String fromLanguage, String toLanguage, String text) throws TranslationException;
}
...
<aop:config>
<aop:aspect id="translationEngineInterceptor" ref="translationEngineAdvice">
<aop:after-returning
method="notify"
returning="translatedString"
pointcut="execution(* translate(String, String, String)) and args(from, to, text)" />
</aop:aspect>
</aop:config>
<bean id="translationEngineAdvice" class="org.jiim.translation.TranslationEngineAdvice">
<property name="mailSender" ref="mailSender"/>
<property name="mailMessage" ref="mailMessage"/>
</bean>
...
public class TranslationEngineAdvice {
protected final Log logger = LogFactory.getLog(getClass());
private MailSender mailSender;
private SimpleMailMessage mailMessage;
public static final String NEW_LINE = System.getProperty("line.separator");
...
public boolean notifyByEmail(JoinPoint jp, Object translatedString, String from, String to, String text) {
String s = (String) translatedString;
// if message string returns empty translation
if (!"".equals(text) && "".equals(s.trim())) {
SimpleMailMessage msg = new SimpleMailMessage(this.mailMessage);
StringBuffer txt = new StringBuffer(msg.getText());
txt.append(NEW_LINE).append("Date [").append(new Date()).append("]");
txt.append(NEW_LINE).append("From language [").append(from).append("]");
txt.append(NEW_LINE).append("To language [").append(to).append("]");
txt.append(NEW_LINE).append("Original Text [").append(text).append("]").append (NEW_LINE);
msg.setText(txt.toString());
try {
this.mailSender.send(msg);
return true;
} catch (MailException me) {
logger.error("Error sending email notification ", me);
}
}
return false;
}
}
...
AJAX-based Rich Internet Applications (RIAs) make heavy use of JavaScript. To improve the user experience of RIAs we can minimize the number of (JavaScript) file requests and reduce their size.
These concatenation and compression operations can be done on-the-fly or at build time.
On-the-fly techniques intercept requests and perform merging and/or compression in real-time. They are simple(r) to implement, often involving URL rewriting combined with a compress script, but:
<dependency>
<groupId>rhino</groupId>
<artifactId>custom_rhino</artifactId>
<version>0.1</version>
<properties>
<war.bundle>true</war.bundle>
</properties>
</dependency>
<!-- destination for the compressed JavaScript files -->
<j:set var="js.compression.dir" value="${typically_the_war_src_dir}"/>
<j:set var="js.compression.skip" value="false"/> <!-- enable/disable compression -->
<j:set var="js.compressor.lib.path" value=""/> <!-- path to the compressor jar -->
<goal name="get-compressor" description="Set path to JavaScript compressor library">
<!-- get compressor from dependencies, to allow standalone use of goal -->
<j:if test="${empty(js.compressor.lib.path)}">
<ant:echo message="Fetching JavaScript compressor from repository" />
<j:forEach var="lib" items="${pom.artifacts}">
<j:if test="${lib.dependency.artifactId == 'custom_rhino'}">
<lib dir="${maven.repo.local}">
<include name="${lib.path}"/>
<j:set var="js.compressor.lib.path" value="${lib.path}"/>
</lib>
</j:if>
</j:forEach>
</j:if>
<ant:echo>
Path to JavaScript compressor library is ${js.compressor.lib.path}
</ant:echo>
</goal>
<goal name="compress-js" description="Compress JavaScript across all site components">
<!-- Different sections might have different compression requirements -->
<j:if test="${js.compression.skip == 'false'}">
<attainGoal name="compress-site-js"/>
<attainGoal name="compress-forum-js"/>
...
</j:if>
</goal>
<goal name="compress-site-js" prereqs="get-compressor" description="Compress JavaScript files for 'site'">
<ant:echo message="Compressing JavaScript files for 'site'" />
<j:set var="stripLinebreaks" value="true" />
<j:set var="js.dir" value="${path_to_javascript_files}"/>
<concat destfile="${js.dir}/site-concat.temp" force="yes">
<filelist dir="${js.dir}"
files="file_to_merge.js, another_file_to_merge.js, etc.js"/>
</concat>
<ant:java
jar="${js.compressor.lib.path}"
failonerror="true"
fork="yes"
output="${js.dir}/site-breaks.temp">
<ant:arg value="-c"/>
<ant:arg value="${js.dir}/site-concat.temp"/>
</ant:java>
<!-- move compressed files back from dest to src dir -->
<ant:move file="${js.dir}/site-breaks.temp" tofile="${js.dir}/site_c.js" filtering="true">
<j:if test="${stripLinebreaks == 'true'}">
<ant:echo message="Removing line breaks" />
<filterchain>
<striplinebreaks/>
</filterchain>
</j:if>
</ant:move>
<delete file="${js.dir}/site-concat.temp"/>
</goal>
<goal name="compress-forum-js" prereqs="get-compressor" description="Compress JavaScript files for 'forum'">
<j:set var="stripLinebreaks" value="true"/>
<j:set var="src.dir" value="${path_to_javascript_files}"/>
<!-- Delete previously compressed files -->
<ant:delete>
<ant:fileset dir="${src.dir}" includes="*_c.js"/>
</ant:delete>
<ant:echo message="Compressing 'forum' JavaScript files from ${src.dir}" />
<!-- create temp dir for compressed files -->
<ant:mkdir dir="${src.dir}/_compressedjs"/>
<j:set var="dest.dir" value="${src.dir}/_compressedjs"/>
<!-- compile JavaScript files to compress -->
<ant:fileScanner var="forumJSFiles">
<ant:fileset dir="${src.dir}" casesensitive="yes">
<ant:include name="file_to_compress.js"/>
<ant:include name="another_file_to_compress.js"/>
<ant:exclude name="*_c.js"/>
</ant:fileset>
</ant:fileScanner>
<!-- loop through files and compress using compressor set in 'get-compressor' goal -->
<j:forEach var="jsFile" items="${forumJSFiles.iterator()}">
<ant:echo message="Compressing ${jsFile.name}" />
<ant:java
jar="${js.compressor.lib.path}"
failonerror="true"
fork="yes"
output="${dest.dir}/${jsFile.name}">
<ant:arg value="-c"/>
<ant:arg value="${src.dir}/${jsFile.name}"/>
</ant:java>
</j:forEach>
<!-- move compressed files back from dest to src dir -->
<ant:move todir="${src.dir}" filtering="true">
<ant:fileset dir="${dest.dir}" casesensitive="yes">
<ant:include name="*.js"/>
</ant:fileset>
<j:if test="${stripLinebreaks == 'true'}">
<ant:echo message="Removing line breaks" />
<filterchain>
<striplinebreaks/>
</filterchain>
</j:if>
<ant:mapper type="glob" from="*.js" to="*_c.js"/>
</ant:move>
<!-- delete temp dir -->
<ant:delete dir="${dest.dir}"/>
</goal>
...
var xmlhttp=false;
/*@cc_on @*/
/*@if (@_jscript_version >= 5)
// JScript gives us Conditional compilation, we can cope with old IE versions.
// and security blocked creation of the objects.
try {
xmlhttp = new ActiveXObject("Msxml2.XMLHTTP");
} catch (e) {
try {
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
} catch (E) {
xmlhttp = false;
}
}
@end @*/
...
<goal name="conditional-compress-js" prereqs="get-compressor"
description="Conditional compression of JavaScript files">
<j:set var="src.dir" value="${path_to_javascript_files}"/>
<!-- check timestamps to see if compression is required -->
<ant:echo message="Checking timestamps of JavaScript files from ${src.dir}" />
<ant:fileScanner var="jsFiles">
<ant:fileset dir="${src.dir}" casesensitive="yes">
<ant:include name="**/*.js"/>
<ant:exclude name="**/*_c.js"/>
</ant:fileset>
</ant:fileScanner>
<j:forEach var="jsFile" items="${jsFiles.iterator()}">
<ant:echo message="Checking last-modified-date of ${jsFile.name}" />
<uptodate property="js.modified" targetfile="${src.dir}/${jsFile.name}">
<srcfiles dir="${src.dir}" includes="**/*_c.js" />
</uptodate>
</j:forEach>
<j:if test="${js.modified}">
<j:set var="compression.required" value="true"/>
</j:if>
<j:if test="${compression.required}">
<ant:echo message="Modified JavaScript files. Compression required." />
<!-- insert compression block here -->
...
</j:if>
</goal>
Besides encapsulating behaviors that affect multiple classes into reusable modules, AOP can also be used for unobtrusive notification tasks.
Instead of browsing application logs looking for exceptions, for example, I like to have the application itself email me the exception stack-trace and the arguments that caused the error when something goes wrong.
Here's an example advice that sends an email after an exception is thrown. The date of occurrence and the exception stack-trace are sent in the body of the email.
The Java code for the advice:
...
public class ThrowableAdvice {
private MailSender mailSender;
private SimpleMailMessage mailMessage;
public static final String NEW_LINE = System.getProperty("line.separator");
// Setters and getters omitted for clarity
...
/**
* Send email notification with exception stacktrace
*
* @param throwable
* @return boolean frue if email message is sent, false otherwise
*/
public boolean notifyByEmail(Throwable throwable) {
SimpleMailMessage msg = new SimpleMailMessage(this.mailMessage);
StringBuffer txt = new StringBuffer(msg.getText());
txt.append(NEW_LINE).append(new Date());
txt.append(NEW_LINE).append(ExceptionUtils.stackTraceToString(throwable));
msg.setText(txt.toString());
try {
this.mailSender.send(msg);
return true;
} catch (MailException me) {
logger.error("Error sending email notification ", me);
}
return false;
}
}
...
...
public static String stackTraceToString(Throwable throwable) {
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw, true);
throwable.printStackTrace(pw);
return sw.getBuffer().toString();
}
...
...
<aop:config>
<aop:aspect id="appThrowableInterceptor" ref="throwableAdvice">
<aop:after-throwing
method="notifyByEmail"
throwing="throwable"
pointcut="execution(* some.package.*.*(..))" />
</aop:aspect>
</aop:config>
<bean id="throwableAdvice" class="ThrowableAdvice">
<property name="mailSender" ref="mailSender"/>
<property name="mailMessage" ref="throwableMessage"/>
</bean>
<bean id="mailSender" class="org.springframework.mail.javamail.JavaMailSenderImpl">
<property name="host" value="localhost"/>
</bean>
<bean id="throwableMessage" class="org.springframework.mail.SimpleMailMessage">
<property name="subject" value="ALERT! Exception thrown @ some app module"/>
<property name="text" value="ALERT! Exception thrown from some module"/>
<property name="from" value="the.notifier@application.org"/>
<property name="to">
<list>
<value>developers@application.org</value>
</list>
</property>
</bean>
...
(Vaguely) Related Posts:
Better Living Through (Unit) Testing and Advice
Reusable Advice - Part I
References:
AOP@Work: Dependency injection with AspectJ and Spring
AOP@Work: Check out library aspects with AspectJ 5
AOP@Work: AOP myths and realities
Improve modularity with aspect-oriented programming
Spring AOP Reference documentation
A few weeks ago, I approached a TfL staff member (let's call him SM) after failing to exit the gate with my Oyster card. Chat transcript below.
Me: "Hi, I can't get out and the machine keeps telling me to seek attention..."
SM (grinning and slowly printing each word in my head): "seek attention? Are you sure the machine told you to seek attention"?
Me (mildly annoyed and a tad confused): "Yes, it displayed the words seek attention in flashing red letters!"
This goes on a few more rounds until SM, visibly amused, opens the gate and let's me through.
...
http://chinesepod.com/
Download free .mp3 podcasts with the basic, free account. Each lesson is a short and entertaining conversation between an English native and a Chinese speaker.
There is also plenty of activity in the discussion panels, where community members help and discuss spoken and writing details about the lessons.
http://fsi-language-courses.org/
Very thorough and authoritative course material developed by the United States government. Each course module contains audio (.mp3), a student textbook (in .PDF format), and a workbook. The lessons are impeccable and demanding.
http://www.pinyin.info/
"This website is aimed at contributing to a better understanding of the Chinese languages and how romanization can be used to write languages traditionally associated with Chinese characters (such as Japanese, Korean, and especially Mandarin Chinese)."
Very academic with plenty of excellent scholarly essays, such as:
I know I'll be 'intermediate material' the day I am able to karaoke to this song...
I'm writing a unit-test code generator for CRUD operations to keep me from repeating myself. In preparation for this, I'm compiling a comprehensive set of CRUD unit-tests beyond the eponymous ones.
So, for a given business object Foo I may want to test:
Create
public void TestCreate_NullFoo {
try {
PersistenceServices.create(Null);
} catch(PersistenceServiceException pse){
return;
}
fail("Expected exception not thrown");
}
// possible test, depends on requirements
public void TestCreate_EmptyFoo {
Foo foo = new Foo(); // no properties set
try {
PersistenceServices.create(foo);
} catch(PersistenceServiceException pse){
return;
}
fail("Expected exception not thrown");
}
// depends on requirements; validation shouldn't happen here
public void TestCreate_InvalidFoo {
Foo foo = new Foo();
foo.set...; // populate with known invalid values
try {
PersistenceServices.create(foo);
} catch(PersistenceServiceException pse){
return;
}
fail("Expected exception not thrown");
}
// depends on requirements; validation shouldn't happen here
public void TestCreate_IncompleteFoo {
Foo foo = new Foo();
foo.set...; // miss required values
try {
PersistenceServices.create(foo);
} catch(PersistenceServiceException pse){
return;
}
fail("Expected exception not thrown");
}
public void TestCreate_IncrementsCount {
int beforeCreate = PersistenceServices.countFoos();
Foo foo = new Foo();
foo.set...; // populate with required values
PersistenceServices.create(foo);
int afterCreate = PersistenceServices.countFoos();
assertTrue(afterCreate > beforeCreate);
}
public void TestCreate_IsNotIdempotent {
Foo foo = new Foo();
foo.set...; // populate with required values
Foo aCreated = PersistenceServices.create(foo);
Foo bCreated = PersistenceServices.create(foo);
assertNotEquals(aCreated, bCreated);
}
public void TestCreate_Foo {
Foo foo = new Foo();
foo.set...; // populate with required & validated values
Foo created = PersistenceServices.create(foo);
assertEquals(foo, created);
}
public void TestRead_NullFoo {
Foo read = PersistenceServices.getFoo(Null);
assertNull(read);
}
// depends on requirements
public void TestRead_EmptyFooId {
Foo read = PersistenceServices.getFoo("");
assertNull(read);
}
public void TestRead_DoesNotIncrementCount {
int beforeRead = PersistenceServices.countFoos();
PersistenceServices.getFoo(aFoo); // aFoo is known, existing Foo
int afterRead = PersistenceServices.countFoos();
assertTrue(beforeRead == afterRead);
}
public void TestRead_DuplicateFoo { // see IsIdempotent below
}
public void TestRead_IsIdempotent {
Foo aFoo = PersistenceServices.getFoo(aFoo);
Foo bFoo = PersistenceServices.getFoo(aFoo);
assertEquals(aRead, bRead);
}
// depends on requirements; could throw friendly exception
public void TestRead_DeletedFoo {
Foo deleted = PersistenceServices.delete(aFoo);
Foo read = PersistenceServices.getFoo(deleted);
assertNull(read);
}
public void TestRead_FooById {
Foo foo = new Foo();
foo.set...; // populate with required & validated values
Foo created = PersistenceServices.create(foo);
Foo read = PersistenceServices.getById(created.id);
assertEquals(foo, read);
}
public void TestUpdate_NullFoo {}
public void TestUpdate_EmptyFoo {}
public void TestUpdate_InvalidFoo {}
public void TestUpdate_IncompleteFoo {}
public void TestUpdate_DuplicateFoo {}
public void TestUpdate_UnchangedFoo {}
public void TestUpdate_DoesNotIncrementCount {}
public void TestUpdate_IsIdempotent {}
public void TestUpdate_UndoFoo {}
public void TestUpdate_Foo {}
public void TestDelete_NullFoo {
Foo deleted = PersistenceServices.delete(Null);
assertNull(deleted);
}
public void TestDelete_EmptyFoo { // dependent of business requirements
Foo foo = new Foo(); // no properties set
Foo deleted = PersistenceServices.delete(foo);
assertNull(deleted);
}
public void TestDelete_InvalidFoo {
}
public void TestDelete_IncompleteFoo {
}
public void TestDelete_DuplicateFoo {
}
public void TestDelete_DecrementsCount {
Foo aFoo = PersistenceServices.getFoo(aFoo);
int beforeDelete = PersistenceServices.countFoos();
PersistenceServices.delete(aFoo);
int afterDelete = PersistenceServices.countFoos();
assertTrue(beforeDelete > afterDelete);
}
public void TestDelete_IsIdempotent {
Foo aFoo = PersistenceServices.delete(aFoo);
Foo bFoo = PersistenceServices.getFoo(aFoo);
assertNull(bFoo);
aFoo = PersistenceServices.delete(aFoo);
bFoo = PersistenceServices.getFoo(aFoo);
assertNull(bFoo);
}
public void TestDelete_ExistingFoo {
Foo aFoo = PersistenceServices.delete(aFoo);
assertNotNull(aFoo);
Foo bFoo = PersistenceServices.getFoo(aFoo);
assertNull(bFoo);
}
public void TestDelete_MissingFoo {
// aFoo does not exist
Foo deleted = PersistenceServices.delete(aFoo);
assertNull(deleted); // could expect exception
}
// depends on requirements
public void TestDelete_ReturnsDeleted {
Foo aFoo = PersistenceServices.getById(aFoo.id);
Foo deleted = PersistenceServices.delete(aFoo);
assertEquals(aFoo, deleted);
}
I've been watching the videos from the Seattle Conference on Scalability talks. These are my notes from the YouTube Scalability talk by Cuong Do, YouTube Engineering Manager. It's pretty amazing how a team of only 9, including 2 developers, 2 sysadmins, 2 scalability architects, 2 network engineers, and 1 DBA, grew YouTube from Null to delivering over 100 million videos/day (prior to the Google acquisition).
Here is a simple Maven goal to optimize CSS delivery at build time and low cost.
<!-- set path to CSS files -->
<set var="css.cleanup.src.dir" value="path_to_css_files" />
<goal name="cleanup-css" description="Remove comments and blank lines from CSS files">
<echo message="Removing comments and blank lines from files in ${css.cleanup.src.dir}" />
<!--
remove single+multiline comments,
see docs@http://ant.apache.org/manual/OptionalTasks/replaceregexp.html
-->
<replaceregexp byline="false" flags="gm">
<regexp pattern="\/\*(.\s)*?\*\/" />
<substitution expression="" />
<fileset dir="${css.cleanup.src.dir}">
<include name="*.css" />
</fileset>
</replaceregexp>
<!-- remove multiple blank lines -->
<replaceregexp byline="false" flags="gm">
<regexp pattern="^\r\n^\n" />
<substitution expression="" />
<fileset dir="${css.cleanup.src.dir}">
<include name="*.css" />
</fileset>
</replaceregexp>
</goal>
Is it really worth it?
I tested it on 5 CSS files (test1.css, test2.css,test3.css, test4.css, test5.css) of different sizes and obtained a respectable file size reduction. In addition, dev comments are not shipped and exposed to the outside world.
Below is the output of the Python stats scripts I wrote to analyse situations like this (testX_c.css is the compressed version of the original file):
>>> describe(cmpfilesizelist(d, listfiles(d, "*.css")))
file test1.css is 4.57% larger than test1_c.css
file test2.css is 33.40% larger than test2_c.css
file test3.css is 21.93% larger than test3_c.css
file test4.css is 34.55% larger than test4_c.css
file test5.css is 36.54% larger than test5_c.css
('Sample Size: 5', 'Min/Max: ', (4.5725935634192512, 36.543124350536885), 'Mean: 26.20', 'Median: 33.40', 'Std. Dev: 13.37', 'Std. Error: 5.98')
The compressed files also validated on the w3c CSS validator
About 80% of Web-application loading time is spent on the browser. Since users prefer faster web sites, the user experience is affected by what happens on the browser.
Thus, it is possible to improve the user experience by breaking down, measuring, and optimizing the activities on the browser.
Enter Page Detailer, a graphical tool that measures client side performance of Web pages.
Page Detailer assesses performance from the client's perspective by showing how the page was delivered to the browser. It provides detailed graphical and tabular views about the timing, size, data flow, and identity of each item in a page, shown in the order started by the browser.
It has a very simple interface and is very easy to use -it will automatically capture, time, and plot calls made by the browser.
It is very useful to:
Caveats
The captured data does not include the browser's rendering time, and does not time separately any intelligent use of lazy loading techniques. Thus, it is not an exact mirror of the end user experience.
There are trade-offs between client-side performance optimization and improvements to the (holistic) user experience, e.g.: uncompressed items 'travel slower' but render faster on the browser and vice-versa.
Download from IBM alphaWorks @ http://www.alphaworks.ibm.com/tech/pagedetailer
Similar tools
This made my day week, gotta love the flying special effects :D
Daniel Gilbert's fantastic book won the Royal Society prize for science books.
Two other books in the same league for me were:
Watch it @ the YUI Theater or download for Quicktime
5 lessons to retain from the talk:
I often hear "predictions" about the inevitability of a Google downfall because they are stretching themselves far beyond search. The Economist claims:
"A leapfrog may seem unlikely, given Google's reputation as an innovator, but its diversification into so many fields beyond web-searching might yet cause it to stumble"from Out-googled, May 10th 2007.
"Google's mission is to organize the world's information and make it universally accessible and useful."The word 'search' is not mentioned in the mission statement. Thus, the core search service as we know it is simply a means to an end -an entry point to a much more ambitious goal: "organize the world's information and make it universally accessible and useful".
[click on image to enlarge]
