The latest version of quickSub has been released, with a couple of minor changes. My Yahoo! has been added to the list of aggregators, and the stylesheet has been updated to address a rather nasty window size issue on IE for Mac.
I’ve just realized that it has been 4 months, to the day, since I last posted to this blog. Now, some may put it down to laziness, lack of enthusiam, or the inability to muster together something interesting to talk about. However, in my case, it is none of the above. Back in March, my wife and I decided to join the league of homeowners. After a number of years of renting, we had decided that we’d funded enough of our landlords’ vacations, and it was time to gain some equity.
So, amongst looking at houses, sorting out mortgages, tracking down insurance, finding out about taxes, and the thousands of other things that happen during the house buying process, this poor blog went on the back-burner.
Aside from home life, things have been very busy at Envoy. We’re working on some very interesting stuff right now, and sometime in the near future I’ll go into more detail about the solutions we’re building.
So, I’m back in the saddle again. Some interesting things have happened over the past few months. On the Java side of things, Eclipse v3 was finally released. I’ve been using it for the past few weeks and have been very pleased. The CVS Integration is great, and makes handling configuration management tasks very easy – in fact, it’s even been known for an IntelliJ-using colleague of mine to ask me to do some CVS tasks through Eclipse!
The recent news about IBM contributing Cloudscape to the Apache group is also interesting. The choice of production-quality Open Source (non-GPL) embeddable Java databases has been rather limited. Projects looking for embedded database capabilities now have the additional choice of a proven technology.
A posting on Brian Duff’s blog about creating native Windows launcher executables for Java applications inspired me to write a short posting about my experience with the Nullsoft Scriptable Install System (NSIS).
I will not go into much detail about NSIS – I’ll leave the review for a future post. Suffice to say, it is an extremely flexible Windows installer builder, exposing a powerful scripting language. Not only does the functionality offered by this scripting language allow the creation of installers, it also allows the creation of custom Java application launchers.
For my nntp//rss project I created a custom Java launcher in NSIS, based upon the Jelude and Java Launcher scripts. The script searches for a local install of the Java Runtime Environment, and, if it finds one, uses it to start launch nntprss-start.jar. You can find the code in the nntp//rss CVS repository. The nice thing is that you can easily display a dialog box if the user does not have a JRE installed, and then open a browser to Sun’s Java Download page. Once you’ve created the launcher script, NSIS has a compiler that quickly turns the script into a small (~40K) executable.
NSIS is definitely worth checking out. I’ve also used it to build a .NET application installer that checks for the appropriate version of the .NET framework, and found it pretty easy to work with the scripting language. The product is supported by a great user community and, best of all, it’s completely open source. You can check out the project at http://nsis.sourceforge.net/
… otherwise known as when is a sync() not a sync()?
Recently I ran some performance tests on disk I/O, from both Java and C-based applications. The nature of the applications is such that they require transactional logging for reliability, and therefore need a guarantee that data has been written to disk. After running some simple write tests, I noticed an order of magnitude difference in performance between a couple of machines. This got me thinking about the impact of disk write synchronization, and what kind of differences would lead to this delta in performance.
Java makes explicit synching a relatively simple task. For the standard I/O library, you need to obtain a FileDescriptor instance, and from there you can invoke sync(). If you’re using NIO, subclasses of FileChannel and the MappedByteBuffer expose a force() method. These methods map down to a Operating System call (e.g. fsync()) which will force all outstanding I/O for the file to be written to disk.
Or so, based upon the documentation, you would be led to believe.
However, there’s one little thing missing from this description. It is the fact that modern hard drives commonly have an on-disk write cache. This helpful little cache, when enabled, provides a considerable performance gain for disk writes. It also helps most hard drive manufacturers boast some pretty impressive performance figures, but I’ll leave that point to another discussion. The downside to this cache is that in the case of a system failure (power, OS crash, etc), there’s a fair chance that there’ll be some data in the cache which is not on the disk. For a transaction processing system, this could be fatal – there’s a chance that data may have been lost. Now, I must mention that there are some drives that have battery-backed write cache, and higher end RAID controllers also have equivalent battery-backed stores.
Back to the performance tests: What I haven’t mentioned so far is that there is a good chance that the write cache has been enabled by default. How do you find out if it is enabled? On Windows, it is as simple as opening up Device Manager, drilling down to your Hard Drive, bringing up its properties and selecting the ‘Disk Properties’ tab. You should see a checkbox indicating whether the write cache is enabled. If it is not enabled, you’ll get the following helpful message when you enable it:
“By enabling write caching, file system corruption and/or data loss could occur if the machine experiences a power, device or system failure and cannot be shutdown properly.”
Some good advice from Microsoft!
If you’re running Linux, the hdparm utility allows to enable or disable write caching for IDE-based drives. Be very careful with hdparm, as it can do a lot of nasty stuff to your hard drive.
/sbin/hdparm -W 0 /dev/hda 0 Disable write caching
/sbin/hdparm -W 1 /dev/hda 1 Enable write caching
For my Linux tests, changing this setting instantly clarified the difference in performance. On one Linux system, configured with a 40GB 5400RPM IDE drive, the write cache had been enabled by default. This system had shown 10x the performance of the other system, which was configured with an 18GB 10000RPM SCSI drive. Disabling the write cache on the 40GB drive brought the performance back down below that of the 18GB drive, as expected.
Are there any alternatives to write caching? There’s a concept called Tagged Command Queuing, which uses intelligent algorithms to map disk commands to the rotational and seeking characteristics of the drive. This is commonly supported across SCSI drives, and was introduced as part of the ATA-4 IDE spec. It is also supported within newer Serial-ATA devices. This requires support from the disk I/O drivers, and I’ve yet to investigate implementations of this feature.
What’s the key take-away from this post? Well, writing reliable code is only half the challenge. The hardware platform and its constituent devices need to be carefully tuned to ensure that integrity is maintained and optimal reliability is achieved.
Check out the following links for more reading on this topic:
Microsoft Support How To: Manually Enable/Disable Disk Write Caching
Apple: Technote discussing Write Cache Flushing
EXT3 and disk write back cache
Experiments on Disk Write Back Caches
IDE write ordering
Questions regarding Journalling-FSes and w-cache recording
I’ve been working somewhat intensively with XDoclet over the past week, and, as it has proven to be quite a valuable tool, I thought I’d share my experience.
The specific reason for my foray into this particular technology was to automate the creation of JMX MBean descriptor metadata for Commons Modeler. Commons Modeler consumes an XML file describing the exposed operations and attributes of managed components. Having to maintain this XML document separately from the code is very burdensome and, of course, and easily lead to disconnects between the actual code and what is described within the XML doc.
Therefore, as a means to resolve this issue, I decided to look into XDoclet. I’d briefly investigated it in the past when doing some EJB development, but did not have time to pursue it further. So, this was an educational exercise that had long been overdue. XDoclet enables Attribute-Oriented Programming. Basically this means leveraging metadata within existing source files to automatically generate code. This metadata can either be the actual characteristics of Java classes and their intrinsic methods, parameters and attributes, or annotations in the form of JavaDoc tags. The beauty of it is that it allows you to specify and extract metadata from within the code, rather than having to be separately maintained.
XDoclet contains two core components. First, an optimized Java source parser, capable of exposing JavaDoc tags and class metadata through an object model. Second, a template engine, similar in ways to other popular Java-based template technology. XDoclet uses template (.xdt) files while iterating over Java source files in order to generate new content – in my case, the mbeans-descriptors.xml
Overall I found using XDoclet a positive experience. There’s an immediate ROI in the form of saved time and reduced scope for error just from not having to separately maintain metadata from the actual code. Creating a custom template was pretty easy, and even creating a custom tag handler was a relatively intuitive process, especially if you have previously written JSP custom tags. That said, I have to give thanks to the authors at Vecna for the Customizing XDoclet page, which provided a great jumpstart on the process.
If you’re looking for ways to exploit the metadata richness available from your Java application code – whether for automatically generating descriptors for JMX or EJB deployment, creating Struts configuration files, or producing other types of content – you should spend some time checking out XDoclet. They’re actually in the midst of working on version 2 (XDoclet v2 Wiki), which now uses QDox as the Java source parser engine and supports both Jelly and Velocity as templating engines.
Hot on the tails of SIPphone, Skype today announced the introduction of conferencing support into its P2P-based VoIP telephony platform. I heard about this from an article in The Register, and went straight to download the 0.97 release to try it out.
First impressions: the UI needs some work, but you’ll soon forget about that when you start a conference call. I tried a four-way call (Skype currently supports up to five conference participants), and had hardly any complaints about audio quality. Some minor jitter, but overall a very pleasant experience – certainly good enough to give traditional POTS-based conference providers a run for their money.
All I can now wonder is how long will it last before Skype starts turning some of their offerings into value-added services? Certainly the End User License Agreement has, since the outset, indicated that “certain functions in the Skype Software are only available to paid subscribers after a free trial period of the Skype Software and Services (the “Free Trial Period”) ends.”
I’m very intrigued to see what will be Skype’s next steps…
Mike Spille recently posted a three-part series covering XA and the joys of 2 Phase Commit. He provides a thorough description of the topic, with some great examples and good coverage to areas such as failure scenarios and recovery techniques. The final post provides a great insight into various logging techniques that could be adopted by the Transaction Manager developer, as well as other implementation concerns. Well worth a read. Even if you’re not building transaction-oriented systems, or are depending on the transaction services of your Application Server, these posts will give you a good appreciation of what is going on behind the scenes.
XA Exposed Part I
XA Exposed Part II
XA Exposed Part III: The Implementor’s Notebook
When I initially released quickSub, back in July of last year, there were to drivers behind the project:
1. Provide a mechanism by which to alleviate the end-user from the current issued faced by content providers for offering a consistent subscription interface
2. Generate sufficient stimulus within the development community to drive convergence on a standardized subscription model for newsreaders and aggregators.
Driver one seems to have been reasonably successful. quickSub is now in use at a number of major blog-related sites, and used across a range of individual blogs.
Driver two seems to still be a sticking point. In a way, the day quickSub has been successful on the second point is the day I lay the project to rest. In recent months there has been some activity focused around a ‘feed’ URI scheme. This has been adopted by a few of the more popular aggregators and newsreaders, and allows one-click subscription. However, it doesn’t really provide a solution for the web-based aggregators (Bloglines, myFeedster et. al.), for whom a URI scheme-based solution would require the deployment of a subscription agent on the end user’s machine.
… has just been released. Now featuring support for NetNewsWire.
Wishing you all a safe, healthy and prosperous 2004!
