Fixing package retrieval quirks in Ubuntu Gutsy Gibbon Server

In the default installation of Ubuntu Server (Gutsy Gibbon), the default source of packages is the CD-ROM drive. That's fine if you make the assumption that there will always be somebody physically present in order to maintain the servers, but for most server's these days, that's a fallacious assumption. The really bad part about this default, is that there's no way I know of at the time of this writing to override this on the update command. In our company, we have our servers hosted off-site like so many other companies, so everything must be done remotely. Therefore, the CD-ROM as a source for Ubuntu packages is a bad idea. To remedy this, you have to go into the apt-get configuration stored at

/etc/apt/sources.list

, and you'll have to find a line similar to :

deb cdrom:[Ubuntu-Server 7.10 _Gutsy Gibbon_ - Release i386 (20071016)]/ gutsy main restricted

Comment out this line, try your update again and you should be good to go.

Fixing Messenger when you've accidentally screwed it up

If you're like me, you like to have your Windows Live Messenger conversation windows be as simple and clean as possible, with no toolbars, and no contact pictures. To that end, I removed the toolbars from my messenger contact windows so that I wouldn't have to see them. However, this became a problem one day when I dragged a picture that I intended to send to someone into a conversation window, and instead of sending it to them, Messenger permanently changed the background of my chat windows with that person to that picture. To resolve this, I had to open a chat window with person X, press the Alt key to get the old-school windows standard toolbar to pop-up at the top, and then go to Tools -> Show toolbars -> Standard. Then from the standard toolbar, you can click on the 'Show Menu' button on the top right of the conversation window from the standard toolbar, and go to Tools -> Backgrounds ... . This wil take you to a screen where you can select a blank default background. However, dragging the picture may have changed the default background colour for the window, as it did in my case. To remedy this, you have to change the background colour to match the rest of the Messenger colour scheme, and you can do so by clicking on the paintbrush icon on the standard toolbar, and selecting a default background colour from there to make things consistent with the rest of the Messenger colour scheme.

How to recurlively delete a folder and its contents in PowerShell

The command :

Remove-Item -recurse -force [directory name]

The really sad thing is that I googled this and there are no sites that explicitly state how to do this. The comparable command in bash :

rm -rf [directory name]

Yet one more reason I hate Microsoft and PowerShell in particular.

RESTful web services with JAXB 2.0

In case you're not familiar, JAXB (Java API for Xml Binding) is an API developed by Sun for binding between XML and Java beans. The first incarnation, 1.x, was interface+implementation based like XMLBeans from the Apache Project. The second incarnation, 2.x, is annotation+pojo based. It's a very useful API, especially when combining it with Hibernate (and the Java Persistence API) to make restful web services. But, that's not really the point of this post. The point is that I've been using JAXB 2.0 for web services much in the past, but this is the first time where I've had to map the "class hierarchy per-table" pattern used by Hibernate into XML. As it turns out, JAXB 2 has very good support for this pattern. You can read more on this at the java.net project page for JAXB.

Logging for tests using the Spring Framework

It's been a while since I've posted to the blog 'cause I've been so busy, and it seems fitting that this be a good way to resume posting, as this issue has pissed me off quite a bit and been a major thorn in my side for the longest time.

The Spring Framework has some pretty good support for creating test classes for your application, however it by default does not properly initialize log4j logging when doing tests, and I found out today why. When running your application in a Servlet container, you'd configure Spring logging in web.xml. However, when running in a standalone context, Spring has no way of knowing how you want logging configured, so it leaves it up to log4j to configure itself. On that front, you have to realize what log4j's default configuration strategy is : reading a 'log4j.properties' file from the root of the classpath. Once this hits you (and it took me a while), getting logging running for your test cases becomes a simple matter of placing a valid 'log4j.properties' config file in the root of your test classpath, and logging starts working properly, so now you can read those pesky hibernate generated queries off your test log .

Determining which version of Ubuntu you're running

At our company, we use Ubuntu to run our servers. Sure, there's other distributions with potentially better performance, but none so easy to setup, and that was the deciding factor in choosing which distro was going to run our systems because I had to set it up, and since I'm both developer and system administrator (*very* small company), I really don't have time to fuck around and I need to make things as easy on myself as possible. To that end, I've decided to start writing tutorials on everything I learn how to do, and I needed to make sure I note which version of Ubuntu / Linux I'm using. To that end, in order to determine which version of Ubuntu you're running, run this :

cat /etc/issue

I learned of this little trick from here.

*Edit (2008-10-29)*
A much better command for determining the current Ubuntu version you're running :

lsb_release -a

More random MySQL glitches

I ran into the following exception recently when moving an application from one server to another :

java.lang.NullPointerException
 at com.mysql.jdbc.StringUtils.indexOfIgnoreCaseRespectQuotes(StringUtils.java:959)
 at com.mysql.jdbc.DatabaseMetaData.getCallStmtParameterTypes(DatabaseMetaData.java:1296)
 at com.mysql.jdbc.DatabaseMetaData.getProcedureColumns(DatabaseMetaData.java:3670)
 at com.mysql.jdbc.CallableStatement.determineParameterTypes(CallableStatement.java:702)
 at com.mysql.jdbc.CallableStatement.(CallableStatement.java:513)
 at com.mysql.jdbc.Connection.parseCallableStatement(Connection.java:4422)
 at com.mysql.jdbc.Connection.prepareCall(Connection.java:4496)
 at com.mysql.jdbc.Connection.prepareCall(Connection.java:4470)
 at org.apache.commons.dbcp.DelegatingConnection.prepareCall(DelegatingConnection.java:275)
 at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareCall(PoolingDataSource.java:292)

As it turns out, there was a permissions issue whereby the function was created with a line specifying the definer and this causes the 'SHOW CREATE FUNCTION' call to fail and return null unless the user calling the statement has permissions for select on the 'mysql' database. Some recommended solutions (that I didn't get around to trying) were changing the call to 'prepareStatement' as opposed to 'prepareCall'. The issue is logged as MySQL Bug #24065. Lately it just seems as if I keep hitting every bug out there, and can't get anything done. I hope this helps somebody.

Commons VFS (FTP) problems

I recently had a problem with Commons VFS whereby I could log into an FTP server and list the contents of the initial directory, but no contents were shown, nor could I otherwise resolve any of the contents or subdirectories. As it turns out, there's a problem with FTPClient which is part of the Commons Net package, and the basis for the FTP module of Commons VFS. After logging in, if the VFS client tries to set passive mode and the server doesn't respond that it allows passive mode (ie PureFTP specifically), then the client just continues as if there's no problem instead of throwing an Exception. The result was getting an error message like "The file root [ftp://myuser:*****@localhost/my_sub_folder] is not a folder". Since I wasn't able to get PureFTP successfully configured to allow passive mode, I was able to resolve the problem by forcing the code to use active mode instead. Fortunately this wasn't a problem since the FTP server in question was on the same machine as the code trying to access it (ie not across the boundary of a router).

Software developers are arrogant assholes

Like the title of this post says, software developers are arrogant assholes. I can say that because I am one. The reason I say this is that I've been Googling like a mofo for articles related to RESTful development and RESTful web services servers using Ruby on Rails, and I must say that it's a real pain in the ass. The vast majority of the articles I'm able to find are in blog format, and are (as is typical with blog-style articles / tutorials) very vague, and don't provide enough detail for a learner to get going. Essentially, they make the assumption that you already know something about Ruby on Rails, and it's a rather hypocritical assumption for a tutorial / article to make. I will admit that I've been somewhat guilty of doing this same thing in my own blog whenever I've provided code examples, so it stops now. Any future "tutorials" on this blog are going to be as complete as possible, time constraints at work be damned. Vagueness cannot be allowed to continue.

Setting up your own domain in (Ubuntu) Linux

...is easier than you actually think. I managed to get a working domain name for our company's servers going using tutorials and information collected from here, here, here and here.
I'd love to be able to write a coherent guide based off of the information from these pages, but I'm just too busy at work to do it at the moment.

A Quick note on SFTP jails

SFTP jails are uber useful and I've blogged about them before, but one crucial piece of information that I've been missing has been where it all started. This thread on the Ubuntu forums was the base for all of the work that I've done and it has made it super easy to setup jails here at work, which is a blessing due to the frequency with which we move files and make changes. I'm finding more and more that I owe an increasing amount of thanks to the people and posts on forums, both Ubuntu and others, so thanks.

Understanding Linux ACLs

Ok, so for a while, I've had an SFTP jail setup within our company for our clients to connect and dump batch files to our systems. It later became necessary to have an administrator user for our staff to be able to go in and read any of the files from any of the clients without having to use a bunch of different logins to login as each individual client. This makes sense as doing so would be far too cumbersome. This is where ACLs came in. I learned what I had to so that I could get going, but that wasn't very much (forunately at the time, unfortunately later). I've since learned a couple of interesting things since then:

There are two types of ACLs in Linux :

1. The access ACL which controls access to files and directories


2. The default ACL which applies to directories only and acts like a template for newly created files and subdirectories within that directory.

The former I knew about right from the beginning (pretty obvious), but I didn't really know the proper name for what I was manipulating. The latter I learned about today, and had previously assumed that Access and Default ACLs were one and the same (I didn't know about the distinction because it wasn't mentioned in the fucking

man

page.

Today's karmic balance

It's pretty rare that I have something to blog about within a couple days of each other, never mind on the same day, so I suppose that this post is the karmic balance of the other for today. I just found out yesterday that because one of our processing partners had their douchebag of a contractor ditch on them, I now have to maintain two systems and integrate them both together. That in and of itself is kinda stupid. But at least this'll be my first production interaction with Ruby on Rails, and I'm actually really looking forward to the experience. I've been meaning to get myself up to speed with Rails, but between feeling tired after I get home and working so damn much that I don't have the time when I get home, I haven't really had the chance. It's going to be an interesting coming couple of weeks.

Use DTDs, they're there for a reason

<rant>
As the title of this post states, if you're a web developer, use a fucking DTD. They exist for a reason, that reason being to allow web browsers to have a reasonable expectation of the content and layout of your website so they can deliver the desired experience to your user. If you're a web developer and you don't know what a DTD is, then you're in serious fucking trouble. In the event that you don't know, DTD stands for Document Type Definition, and it's a single line of XML that goes at the very beginning of HTML documents and almost at the very beginning of XML documents (if they're validated by DTDs). An example of a DTD :

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

That DTD is for a strictly validated XHTML 1.0 compliant HTML document. This means that it's strongly validated and web browsers can expect everything after it to be well-formed quality. That's right bitches, I use a good version of HTML and I'm standards compliant like a mofo.
</rant>

Sorry, but I just had to get that out. Lately, I'm being inundated with pages from our partners that don't use DTDs, don't validate to anything, and are just generally horrible markup. Now, normally, a person wouldn't need to care, but when your partners are lame and you have to scrape their pages for data, the lack of developing to a DTD becomes a giant pain in the ass.

A letter to Apple

Yes, these things are cliche, but whatever, screw it.

Dear Apple,

In case you haven't figured it out from my buying patterns, I've stopped buying music from you that isn't DRM-free. This is because, as Steve Jobs has admitted himself, DRM is archaic, and frankly doesn't work. What your company doesn't seem to realize is that if you want to continue your high level of sales, (especially to technophiles / audiophiles such as myself) you're going to need to get going on casting off the DRM that exists on your considerable library of music. I steadfastly refuse to purchase any music that is not DRM-free. I want to be able to play it anywhere on whatever device I may happen to own, and I'm not going to cave into unreasonable demands of an already clearly greedy music industry that refuses to evolve and adapt to technological reality. If I can't purchase music through large corporations such as yourself, I'm going to look elsewhere : smaller sites, independent labels, and independent artists who publish their music for free. So, until you get your head out of your ass : screw you, you're not getting one thin dime from my pocket.

Beginning with SQL Server

Given changes in the environment in which our business does it's business, I've had to start learning to use SQL Server and various other Microsoft technologies lately. A quick note on connecting to a foreign SQL Server instance (ie another company's ...) :

Create an alias for the database so that Enterprise Manager has something to work with

Open up the Client Network Utility which should be included in the Start menu programs group for your SQL Server installation.

Start -> Programs -> Microsoft SQL Server -> Client Network Utility -> Alias (tab) -> Add... (button).

Under 'Server alias' enter an easy to remember name for the connection. Under 'Network libraries' select the TCP/IP option. Under 'Connection parameters' enter the DNS name or IP address under 'Server name' of the server you wish to connect to. Change the option for 'Dynamically determine port' if the server you're attempting to connect to doesn't run under the standard port of 1433 for SQL Server. Hit 'Ok' to save your settings, then 'Ok' to close out of the Client Network Utility.

Create a new SQL Server Registration to add your SQL Server instance to the Enterprise Manager

Open Enterprise Manager. It should be in the same programs group in the start menu as the Client Network Utility. Once open, create a new SQL Server Group if you haven't already got one (or haven't got one that you want to add your new SQL Server instance to). Then add a new SQL Server Registration. You should be prompted by a Wizard at this point. (If you're not, then you've probably already disabled the Wizard and don't need to be reading this tutorial.) In the wizard, click on 'Next' to proceed to the SQL Server selection screen. Under the list of available servers, you should see the alias you just created in the previous step. Select it and click on 'Add >' to add it to the list of added servers, and then click on 'Next'. After this, you should be prompted for credentials, and the rest is pretty self explanatory.

Cool MySQL tricks

It's been a while since I've written a blog post, and I figured this was a good topic to write about. For the most part, I don't use SQL. There's numerous ORM (Object-Relational Mapping) tools and frameworks out there (ie Hibernate) that abstract away all the boiler-plate CRUD code. When I do have to use SQL (be it on the command line or with a query browser), I'm not afraid to dive right in, and it's interesting to learn new things about SQL because there's so much to learn and it's such a ridiculously powerful query language.

I had to update the configuration in one of my databases, but the configuration required inserting numerous new lines into a table for user roles. I could've written a stored procedure to do it, but given my (admittedly somewhat limited) knowledge of SQL, I figured this would take too much time and I didn't want to do it. (This is what I like to call constructive laziness). I kept Googling and found MySQL's INSERT INTO ... SELECT syntax, which allows you to conveniently insert the results of one query into another table. I'm sure that most other database implementations out there have a similar convenience syntax, but I happen to work with MySQL so there you go. It took what would've been a numerous line script with looping and separate SELECT and INSERT statements down to a one liner. If you use MySQL (especially with LAMP / AJAX) I recommend that you read about it.

Deleting .svn folders

Subversion is a great tool, but unfortunately it has to store metadata somewhere, and like all other code repository tools, it stores them in metadata directories ... for each directory in whatever project you're working with. I recently found that I have to copy parts of my project documentation out for some of our clients to access our APIs, and I wanted to quickly delete all the SVN metadata and trim certain other files where necessary. I figured the quickest way to do with would be a simple command. Windows PowerShell provides a good (though not great, and certainly not as good as bash) way of doing it :

Get-ChildItem -Recurse -force |where {$_.PSIsContainer -AND $_.Name -match "svn"} | foreach ($_) {remove-item -force -recurse $_.fullname}

Two Google queries yielded the above command from this blog. Happy reading.

Resolving logging issues with Tomcat

An idea for resolving the logging issues in Tomcat just occurred to me. I should probably be putting the logging jars in the individual lib directories for each of the webapps. I'll have to give this a try and post my results.

Onward to Eclipse Europa ... and then right back

Ok, so after Eclipse Europa (3.3) has been out for quite some time, I decided to give it a try. I was shocked to discover a lot of the changes that had happend :
it seems that the Eclipse foundation has become Red Hat's bitch. There's a ton of new Red Hat sponsored / produced editors (none of which are very good.) There's a ton of useless JBoss integration. The editor overall is slower. All my hotkeys (save one or two, literally) were standard with installation in Eclipse 3.2, and are now broken in Eclipse 3.3. I'm severely disapponted after using Europa for three weeks. The only good part about it is that access to my SVN repository was considerably faster. And it seems I'm not the only one who feels this way about Europa. Get it together Eclipse.

Oh, and the thing that brought me to writing this post in the first place : the XML schema editor is flat out broken. What the fuck ?!

Spring, Tomcat and memory

As my application has been growing more and more in functionality, it has also been growing in its memory footprint. To that end, I've started getting OutOfMemoryErrors because the heap has started overflowing. That's really not a problem if you have access to the external Tomcat server, you can just increase the heap size using "-Xms128M -Xmx512M" (or other applicable sizes) on the command line, or do this via the GUI if you're running Tomcat in Windows. However, it's not as obvious if you're using Tomcat for debugging within the Web Tools Platform plugins for Eclipse. I started running into this problem recently and it ground my development to a halt until I was able to fix the problem for WTP in Eclipse. WTP passes its arguments to the Tomcat server via the Launch Configuration. To get to it, right click on the Tomcat server in the Servers view -> 'Open' -> 'Open launch configuration' -> 'Arguments' tab -> 'VM arguments:' text box, then add the "-Xms128M -Xmx512M" segment to the end of the parameter list.

Actual plugin development for Maven 2

Ok, so after Googling around and hours of patient command line testing (yeah, I know, it's horrible, you don't have to tell me) I managed to complete my ToLDya plugin for generating TLD files. Hopefully this plugin will be a great aid to people developing JSP tag libraries other than myself. But along the way, I learned a lot about plugin development in Maven, and I'm quite certain that I've got a shitload more to learn.

1. Starting out with a barebones Mojo from the Maven provided archetype literally doesn't get you much. It gets you just enough to plug into the Maven framework so that maven can actually run your Mojo, but that's about it.


2. The AbstractMojo provided by Maven is pathetic. It gives you a logger, and that's about it. By default, it does not give you many of the things a plugin is quite likely to want (more about that later)


3. Despite the fact that Maven 2 was supposed to be the "lessons learned" version of Maven, I don't think that the Maven developers learned much at all. Maven is grossly behind the times, still relying on XDoclet annotations and pre-JDK 5.0 compatibility. Yes, there is something to be said for keeping things backward compatible (especially in a corporate environment, I know), but at some point you have to move on and do better, in this case : getting up to date with the latest JDK (1.6.03 at the time of this writing).


4. If you need anything (ie from Maven) while writing a Maven plugin, you have to specifically request that it be injected for you. (See the Maven documentation, this is the one area where they're actually good about documenting things and helping out developers)


5. If your plugin needs to access any of the classes in the project on which it's run, you need to load them yourself with your own classloader. Maven will not give you one (which is pretty ridiculous to my mind).

Here are some important bits of knowledge for doing anything with a Maven plugin:

• If you need access to anything from the project (ie any information stored in the POM), you'll need to include the following dependency :

  
  

  <dependency>
   <groupId>org.apache.maven</groupId>
   <artifactId>maven-project</artifactId>
   <version>${maven.version}</version>
  </dependency>

  
  

  In my current POM, the property 'maven.version' is set to 2.0.7. You'll then need to have a property in your plugin Mojo called 'property' (or whatever else you find suitable) and annotate it like so (from within a Javadoc comment of course):

  
  

  
   * @parameter expression="${project}"
   * @required
  

  
  



• As mentioned previously, if you want to load any of the classes that are in the project on which your plugin is to execute, you have to load them yourself. The same goes for any of the project's dependencies. In order to do this, you'll have to get the list of dependencies (compile, test, runtime) from the MavenProject object ('property', remember?) The MavenProject object has a property called 'runtimeClasspathElements'. This gives you a list of strings that are fully qualified file system paths to the classes in the ${project.build.outputDirectory} as well as all of the dependency JARs on which the client project depends. You'll then have to load them yourself. I did so with a URLClassLoader (part of the JDK). I used the following function for creating the classloader :

  
  

  private static URLClassLoader getDependencyClassloader(List dependencies) throws MalformedURLException {
   List classpathUrls = new Vector();
    
   URL url = null;
   for(int index = dependencies.size() - 1; index >= 0; index--) {
    url = new File(dependencies.get(index)).toURI().toURL();
    
    classpathUrls.add(
     url.toExternalForm().endsWith(".jar") ?
      url :
       new URL(url.getProtocol(),url.getHost(),url.getPort(),url.getFile() + "/") //add the '/' o indicate a directory   );
    }  
    URLClassLoader ucl = new URLClassLoader(classpathUrls.toArray(new RL[] {}), Thread.currentThread().getContextClassLoader());
   
   return ucl;
  }

  
  

  Once you have this class loader, you can use it in the long version of Class.forName() to load any classes you may need, as well as perform any loading logic you need with the .getResources() functions on the classloader.

  
  

I'll be doing more plugin development in the coming months I'm sure, so I'll try to post what I learn here, but that's about it for now.

Plugin development with Maven, obsessed

Ok, I let my obsessions get the better of me and I couldn't drop it : I kept searching the interwebs until I had found a solution for my problem. Surprise, surprise, the maven-plugin-testing-harness is a broken ass piece of shit. Enough editorializing though, here's how things went :

1) I had to make sure all of this sh*t was in my POM :
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-core</artifactId>
<version>2.0.7</version>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-project</artifactId>
<version>2.0.7</version>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-descriptor</artifactId>
<version>2.0.7</version>
</dependency>
<dependency>
<groupId>org.apache.maven</groupId>
<artifactId>maven-plugin-api</artifactId>
<version>2.0</version>
</dependency>
<dependency>
<groupId>org.apache.maven.shared</groupId>
<artifactId>maven-plugin-testing-harness</artifactId>
<version>1.0</version>
<scope>test</scope>
</dependency>

2) I then had to change from using 'lookupMojo' to instantiating my own mojo and using 'configureMojo' on the test harness. With the former, you'd have to put the /META-INF/maven/plugin.xml file somewhere on the classpath so that the MavenPluginDiscoverer module can detected it.

Why didn't they just tell you helpful things like these on the wikis for the plugins ? God dammit, I'm quickly hating Maven more and more. And to boot, it doesn't properly evaluate the required expressions and inject stubs for them, but maybe I've just missed something (easy, given how poor the documentation for the project is).

The anniversary

I didn't even notice it until today that it's the one year anniversary of this blog. Actually, the anniversary was two days ago, but close enough. Considering my general dislike of (non-development) blogs, that's actually quite impressive. I didn't think it'd last this long. Yay blog.

Maven ... OMG

My recent foray into developing Maven plugins has led me to a horrible discovery : the Maven developers ... suck. They're just not professional. There's no javadoc for the core classes. There's no javadoc for the base plugin classes and interfaces. And the AbstractMojo class which is supposed to be a good starting base class for all other plugins is, to say the least, horribly designed : it doesn't contain access to the MavenProject that's currently being operated on, you have to add it in manually Ironically (or appropriately, take your pick) they made that a plugin too. The supposed "tutorial" on the Maven main site doesn't show any mention of this. The plugin "cookbook" on the site has been "coming soon" for so long that you have to wonder if it's ever going to get written. And, on top of all this, there's no generics and no direct support for JDK 5 annotations and enums. (See previous post). I've now reached a point of severe disappointment with the Maven project, and I'm fast approaching the point where I'm genuinely sorry that I've made this much of a time investment in it.

Maven 2 quirks and the "bandwagon"

I recently found myself developing (and using) a lot of small tag libraries to make my life easier when it comes to developing my web applications. The only thing that really gets me is that when I write these tag libraries, I have to manually update all of the TLD (tag library descriptor) files that are necessary for using the tags in JSPs. This irks me and it occurs to me that there's no reason I can't have Maven generate the TLDs for me based on annotations I place on the tag classes. The only problem with this is that no such plugin exists, so I figured that as a foray into Maven plugin development, I'd try making such a plugin as my first try. It is then that I ran across a rather large and bothersome quirk with Maven : it's not entirely ready for Java 5 and up. In fact, rather than using proper Java 5 annotations, it uses XDoclet comment annotations to provide metadata for building plugins, but that's not the quirk. The real quirk is that it can't handle having Java 5 annotations in the same project as the plugin. It can handle Java 5 enums just fine, but not annotations, which seems rather strange to me.

This is all part of the bigger problem of companies not keeping themselves up to date with the latest Java technology and staying with Java 1.4.2 and earlier. Sun has done a great job of maintaining backward compatibility, and there's (almost) no reason that companies shouldn't be upgrading the JVMs on their servers to the latest and greatest versions. The only real reason I can think of is that somewhere in their code, they've used variables named 'enum' which becomes a Java keyword in Java 5 and up (aka Java 1.5, 1.6, etc), and this can be mitigated by refactoring the code. Hell, Eclipse makes that job quick and easy, especially when you really know your keyboard shortcuts. I'm fortunate enough to be able to use Java 6 (and very quickly upgrade to Java 7 as soon as it's released and stable). As much as this is going to make me a snob, I'm getting really tired of being dragged down by other people's need for backward compatibility (and hence also tool developers' appeasement of these people which then affects me). Seriously people, get your act together.

Java enums are even cooler than I ever knew !

A quick note on Java enums : I discovered today that not only can enums have methods (which is exceedingly useful to begin with), but that you can individually override methods on a per-enum-value basis! That's so cool. I read it on this blog.

An interesting Spring Framework quirk

Recently, I discovered the joy of the PropertyEditorRegistrar interface in the Springframework, for conveniently registering property editors to bind between objects and text when rendering forms. This joy led me to discover an interesting quirk : if you subclass a Form Controller and register property editors that way, the object of a field will be bound to the BindStatus.value property, but if you use a PropertyEditorRegistrar, that value gets edited and the spring representation of it bound to BindStatus.value. This is a small distinction that makes a huge difference when writing (and rendering) your pages.

RSS and Me

I love RSS, I think it's a great way to read the news and generally stay up to date on any sites you read and any (legit) torrents you may want to download, or podcasts if that's your thing. I've been kicking around the idea of having an RSS notifier for our system at work, so that I wouldn't have to be sending out extraneous emails all over the place. According to this guy, creating an RSS feed is stupid simple. Maybe when I get the chance I'll give it a try.

*EDIT* : Ok, me being me, I was fascinated with the idea of dealing with something new and couldn't let it go. I followed one breadcrumb after another and found, fortunately for me, that Spring already has an (Abstract)RSS view class in the Spring Modules library that uses the Java Rome RSS library in the background. I can't wait to slog through the crap I have to deal with at the moment so that I can fool around with RSS and have RSS feeds supplying the notifications for our production system. This is going to save so many headaches.

*EDIT 2* : Heh, this guy's page is awesome. It shows that you can use security with RSS feeds, which will be perfect for my company.

Moar Hibernate !!!

Yet again, I'm posting because of fucking hibernate. One of the old quirks I had run across was that if you had a setter for a collection, ie :

public void setItems(List items)
{
this.items = items;
}

...this would replace the hibernate-backed collection that already existed (if one did) and could replace it with a non-hibernate implementation such as java.util.Vector, and any cached items would not get properly dealt with, and the collection would not get properly persisted, even to the point of throwing an exception. This resulted in me having to change my setters to this :

public void setItems(List items)
{
if(this.items == null)
{
this.items = items;
} else
{
this.items.clear();
this.items.addAll(items);
}
}

The only problem with this is that depending on the scenario, hibernate gets the collection, and then sets exactly the same list object back into the persistent entity we're dealing with, which would result in clearing exactly the same list we're trying to assign. The remedy :

public void setItems(List items)
{
if(this.items == null)
{
this.items = items;
} else if(this.items != items) //fix: identity check the two lists!
{
this.items.clear();
this.items.addAll(items);
}
}

That really should have been there anyway, but hibernate inspired it. Fuck you hibernate.

Spring and PropertyEditors, important details

As mentioned in the documentation, the Spring Framework uses property editors in two places:
1) When parsing ApplicationContexts from XML configuration files
2) When binding beans from HTTP requests.

It's all well and good if the only classes you need to bind are already covered by the editors built in with the Spring Framework, but if you need property editors for other kinds of classes, here's how to get Spring to include them, for both situations :
1) Use a CustomEditorConfigurer like so :

<bean class="org.springframework.beans.factory.config.CustomEditorConfigurer">
<property name="customEditors">
<map>
<entry key="java.text.DateFormat">
<bean class="com.mypackage.CustomDateFormatEditor"/>
</entry>
<entry key="java.util.TimeZone">
<bean class="com.mypackage.TimeZoneEditor"/>
</entry>
<entry key="java.text.MessageFormat">
<bean class="com.mypackage.CustomMessageFormatEditor"/>
</entry>
<entry key="java.text.NumberFormat">
<bean class="com.mypackage.CustomNumberFormatEditor"/>
</entry>
</map>
</property>
</bean>

* Note * : The editors can be singletons because they're going to be the only instances needed by the application context to parse string values.

2) Implement and instantiate (or otherwise use) an implementation of PropertyEditorRegistrar. This will be used with your form controllers to register property editors for individual fields or whole types.

Ubuntu Linux is finally in the right place

I love Linux. Now, at this point, you're probably thinking that I'm just one of these fanboy nerds that loves to sit on a computer all day and hack on code. You're partially right : I do love to sit on computers and hack on code, but certainly not all day and certainly not at the expense of other fun activities like going to hockey games and being with friends. That said, I like Linux because it's good for doing development and it's ridiculously stable. It's even quite performant as well, which is
quite nice. I like it to the point that I'd want to put it on all of the computers here in the office, and as per the title of this post, I think that Ubuntu Linux is at the point where that's a practical possibility. In the past few days, I've installed Ubuntu 7.04 (Feisty Fawn) on two machines here in the office, set them up to do networked printing, networked file sharing, and everything else that may be needed by office workers. Hell, they can even use the built in Terminal Services client to remote into our server here in the office if they need to do any centralized work. They even have all their networked drives mounted for them, and they can see the Windows domain on which this place runs. The only thing remaining is for the systems to pass the Boss Test : sit the big boss of the company down on one, and if he can print his stuff, access his stuff and access our server exactly the same as he could in Windows, then it stays.

There's a great tutorial on setting up Samba with Windows Shares in Ubuntu here. Printing was
ridiculously easy to set up : I went to System -> Administration -> Printing , it autodetected the printers on our LAN and setting them up was merely a matter of following on screen directions.

MySQL character sets, the end

Many times in the past I've tried to get our foreign MySQL server to properly store and handle unicode character sets. Despite the fact that I repeatedly set 'default-character-set=utf8' all over the config files and set the server and client character sets and collations in my.cnf, it still wouldn't handle them properly. Here's the kicker : to make absolutely sure that the server uses only server settings for handling character set (and sets 'fuck you' to whatever the client requests), you have to use the '--skip-character-set-client-handshake' argument when starting the MySQL daemon (server). This is what finally got it going for me. I hope this post helps somebody out someday. (Given my rate of forgetting things, it's likely to be me)

<rage>MOAR HIBERNATE! :@</rage>

God damn, I'm getting so fucking tired of Hibernate and its quirks. Here's a new one regarding Criteria queries :

When you create a criteria query, you almost invariably have to specify a result transformer of Criterion.DISTINCT_ROOT_ENTITY (that's a quirk, but not the topic of this post). When you have an entity on which you want to build a criteria query and you want to limit the number of search results (ie distinct root entities), things get really tricky. Specifying 'setMaxResults' on the query affects the number of rows returned from the database that are actually inspected. Therefore, if there are any joins on your entity that have collections, this will cause a fetch with an outer join strategy to generate an excessive number of rows and affect the results when using a maxResults setting. In the case of using embedded properties, specifying a fetch mode of SELECT will not override these (this is a glitch in hibernate). You'll have to specify the fetch mode manually in the metadata (be it XML or Annotations) permanently for the embedded class. I fucking hate Hibernate sometimes, I really do.

Spring AOP rage ...subsiding

Spring AOP is breaking my heart. Spring overall is a great framework and I love working with it. The idea of using aspects to interweave code and keep concerns separated and code clean is a wonderful though to somebody who loves to architect software, such as myself. But I swear, it feels like I'm hitting every bug in the book when it comes to using AOP in Spring. Certain pointcuts don't get matched properly, regexp based pointcuts get loaded when they shouldn't, the list goes on. I really want to use AOP to design the next big phase of my project that's coming up, but they're making it really hard to justify the decision to do so. You're making my heart cry, Spring.

Google Disappointment

...do it. You just might find a page with me ... pointing at Google on my screen.

Today I was messing around with some of the CSS styling in my API documentation for one of our company's merchant partners, and I found that some of the elements weren't quite right. I then remembered an article I had read on Digg regarding CSS reset stylesheets, and how Google and Yahoo both use them in their free APIs to provide consistent styling results across browsers, so I went to investigate using one of said reset stylesheets to help me out. I figured I'd Google it first (no pun intended), and upon not finding anything relevant quickly, figured Google would be smart enough to use their own reset stylesheets in their own pages. I pulled up the source on one of my query pages, only to discover a developer's horror : inline styling all over the place, and google didn't even use their own stylesheets anywhere in their main site! The code was horrible spaghetti. I ran the page through the W3C web site validator, and it didn't pass a single standard, ie HTML 4.01 / XHTML (any flavour). As one of the big Web 2.0 sites, I would have expected you to have higher regard for international web standards. I'm very disappointed in you Google.

A quick note on versioning

I've come to really admire the way the open source community has been versioning their products over the last couple years, especially in the java open source community, which I've found to be very bright and vibrant. Generally, the projects adhere to the following conventions (where 'x' is an integer) :

Version :
x.x.x - Version with major version, minor version, patch increment
0.x.x - Beta software, not to be considered ready for production use (generally, some projects have very odd development cycles and version conventions)
1.x.x - Version 1 (good for looking at, but you may want to wait for version 2, especially Apache projects *cough*maven*cough*struts*cough*)
x.x.x-Mx - Milestone beta version - has certain promised features according to the milestone version, but not the final version with that number
x.x.x-RCx - Release candidate beta version - has all the promised features according to the release plan for that version for the project, but is not considered to have been sufficiently tested

If anybody feels differently about my descriptions, please, by all means, correct me.

Weak Java

I rarely (if ever) use switch statements, mainly because if their use is required, generally a very poor design decision has been made. But sometimes they're the right choice. That said, I was forced to use a switch statement in some of my code today, and found that Java switch statements can only switch primitive ints and enums in the java language. Not even longs or shorts. That is the weakest shit I've ever encountered. Are you fucking retarded Sun ?

Sorry, I just had to bitch about it, because it's just that stupid.

I love Eclipse now more than ever

Eclipse has always been a very useful platform, with tons of little developer driven quirks that make the programmer's job so muche easier. I found this out especially lately when I've been trying to get into developing with C# in Visual Studio 2005, and I've been missing features that are just there in Eclipse, but you have to buy a plugin for with VS2k5. And here's the great thing about that whole situation : you can develop C# in Eclipse on Windows with a free plugin. But that's really all just a side note. The main point of this post is this : I just now observed an exceedingly useful feature that's been in Eclipse for several versions. Ever press Ctrl + Shift + T to open the quick loader for classes ? It has a text bar at the top so you can type in the simple name of the class you're looking for and it'll filter out the results. In the filtering options up top, there's the usual ? and * filters, but beside that there's also a spot that says "TZ - TimeZone" which I've never really noticed before, and I don't know what made me notice it today. I typed in TZ, and much to my surprise, a whole list of classes remained in the window, filtered, and they all had TZ capital letters in them, following the general naming conventions in Java. Intrigued by this, I typed in MPREF (ManualPaymentRequestEntryFlow) which is a class in my project that I just finished working on, and sure enough, it filtered the list to have that in it (it was the only class not filtered out) . I thought that was just the greatest filter I'd ever seen. And to build on it, if you use the content assist hotkey (Ctrl + Space) with the same scheme, it automatically fills in those classes for you. How cool is that !?

Moar (sic) hibernate !

Yet more hibernate fun facts that I wish I'd had before I started any projects : Hibernate has the ability to persist instances of java.util.Locale, java.util.Currency, and java.util.TimeZone. A sprinkling of a few bits of these pieces of code and their corresponding hibernate annotations could have saved me an assload of time spent on doing my own design, development and testing.

On a small side note, I found I've actually managed to come up with a much better and more appropriate solution for dealing with currencies than the java.util.Currency class, as it's woefully inadequate to the task of dealing with currencies in an enterprise system. I refer specifically to the way it deals with floating point arithmetic in its number of decimal places, rather than adering more closely to the ISO 4217 specification whereby currencies should have a base and an exponent when calculating a multiplier to go between base currency units and major currency units.

Rampaging bull software design

If you're into the software scene, then you most likely know that some of the big buzz terms in the last few years have been RAD (Rapid Application Development) and Agile development (keeping your code clean and maintainable, and incrementing your design and development process in very small steps). With ever changing requirements and very little chance to get the big picture ahead of time given the small amounts of information I get from my bosses, I have no choice but to follow an agile development process. As best as I can tell, I've been doing so ever since I've started here.

Unfortunately, this style of development has caught up with me : I'm finding that even though my code is fairly clean and very easily maintainable, there are memory leaks (I'm using Java, so this means that something's holding onto references and preventing the automatic garbage collector from doing its job). This means that I now get to go back with a profiler, analyze the running of my software, and find out where the leaks are and correct them.

This is going to be interesting.

Moar (sic) Spring Batch !

I received a pleasant surprise in my inbox today (which doesn't happen often), and that surprise was an email from the Spring Batch team notifying me that they'd just released Spring Batch 1.0-M2 to the subversion repo and made it public. This was great because it gave me a chance to download the code and play around with it, look at the samples, and generally get an idea of what's coming for Spring Batch.

In case you weren't aware, I've been looking forward to the release of Spring Batch for quite a while because it can help me a great deal at work, where I'm already using the Spring Framework for my web applications. Unfortunately, the M2 release doesn't appear nearly ready for a production environment, but it's a start. I want more ! faster ! now !

Why you should write good software

Today at the company where I work, our remotely hosted server went down suddenly. Nobody know why, and then the shit started to hit the fan, with emails flying back and forth, and phones ringing off the hook in search of an answer to that one timeless question, WTF ? Of course, everything filtered towards me eventually because I'm considered responsible for the server (even though I'm no where near it and if I don't have a network connection to it, I can't do shit). After several phone calls back and forth between our former partners (who happen to be in the same building as our ISP and still maintain our server), and our ISP, and our firewall technician (who, oddly, is outsourced by our ISP and doesn't technically work for them), it was discovered that the machine was alive, but not responding to any network traffic, so the quickest and simplest solution was to just get somebody to reboot the machine. Eventually, one of our former partners' employees went down to the cage and rebooted the server. Later, in an email explaining what had happened, that same employee (who happened to be their sysadmin) explained that their DVD backup had frozen the whole box and that this situation had never happened before. (This during our peak period for the day).

Now, I can believe that (I've had weird shit like that happen to me, so I can't really bitch at the guy.) But it brings to mind the need for good software and why good software should be written : if you write software, and it fucks up, then the managers of those using your software don't see it as your fuckup, they see it as their employees' fuckup, and hold them responsible. I hate the thought of being one of those employees (and very nearly was one today). This has brought forth some small amount of Karmic inspiration to put more time into testing and finding those odd failure modes for my own software that you wouldn't think can happen, but can because you eventually hit an odd and unexpected set of circumstances. Just something to think about if you're a serious developer and you're reading this.

Linux : the more you know ...

I'm somewhat embarrassed to admit this, but I've just discovered Linux ACLs, which are apparently in every kernel 2.6+, and has plugins for 2.4-. I guess a really linux savvy person should know these, especially since they're incredibly useful and give so so much more power over the default User - Group - World permissions built into the file system.

JSP Tags

They're oh so useful for encapsulating small bits of reusable logic, but, here's the kicker : when they're compiled, only one instance of a tag is generated for a given page, so you have to design them to be stateful and ensure that you override the 'release' method when inheriting from TagSupport or BodyTagSupport. It took me a couple of hours to figure this out, and sadly, it was for the second time. I guarantee there won't be a third after this.

Moar (sic) Hibernate

OK, so once again in my (seemingly never-ending) struggle with Hibernate, I have stuff to report that's mainly being posted here for my own reading so that I can reference this shit later. This time, I've decided to abandon Xdoclet and just go with Annotations. They're so much easier and I don't have to deal with Xdoclet's quirks any more (like ridiculous parsing exceptions between versions).

If you want to use annotations to do a bidirectional OneToMany assocation with list-based semantics, here's the annotations you use :

Parent.java (equals, hashCode, getters/setters removed for clarity):
@Entity
@Table(name = "parent")
public class Parent implements Serializable {

private static final long serialVersionUID = -1989884660562516228L;

@Id
@Column(name = "id")
@GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;

@Column(name = "name")
private String name;

@OneToMany(mappedBy = "parent", fetch = FetchType.EAGER, cascade = {CascadeType.ALL})
@Cascade({org.hibernate.annotations.CascadeType.ALL,org.hibernate.annotations.CascadeType.DELETE_ORPHAN})
@IndexColumn(name = "idx", nullable = false)
private List children;
}


Child.java (equals, hashCode, getters/setters removed for clarity):
@Entity
@Table(name = "child")
public class Child implements Serializable {
private static final long serialVersionUID = -6414526385302360120L;

@Id
@Column(name = "id")
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;

@Column(name = "name")
private String name;

@ManyToOne
@JoinColumn(name = "parent_id")
private Parent parent;

@Column(name = "idx")
private int index;

public Child() {
super();
}

public int getIndex() {
return this.parent.getChildren().indexOf(this);
}

public void setIndex(int index) {
}
}

Note that you need the 'index' pseudo-property in the child class in order for hibernate to properly persist the ordering of the elements in the collection. The setter for the index property should do nothing, the getter should determine the object's placement in its parent, and you'll need the field with annotations in order to get hibernate to read everything properly. The name of the column in the annotation for the index property MUST be the same as that specified in the parent in the @IndexColumn property. They aren't very clear about this in the Hibernate Documentation (once again.) They mention a similar structure for the .hbm.xml mappings in a faq on the main site for hibernate (not the hibernate annotations faq), but they don't explicitly mention this anywhere for Annotations. You can read the mention of it for .hbm.xml files here. I'm sure I'll have to post more about annotation configurations later, but that's all for now. And if you're wondering, the misspelling in the title of this post is an inside joke.

PS. Note that in addition to the OneToMany and IndexColumn annotations on the child collection in the parent, you also need to have a @Cascade annotation in order to properly remove orphans from the database when deleting children from the child collection in the parent. You can also find the table that gave me my final clues on Darren Hicks' blog

More Maven2

Today I had the chance to build a new project right from scratch for a quick and dirty app that needs to be spat out right quick. I figured that this would be a good chance for me to start really getting on the band wagon with Maven2 and using it full force in my development environment.

Building on what I've mentioned about Maven2 previously, here's something else that needs to be mentioned:
1. Adding mvnrepository.com to your list of external repositories :
<project>
|
|
<repositories>
<repository>
<id>mvn-repository</id>
<name>www.mvnrepository.com</name>
<url>http://www.mvnrepository.com</url>
</repository>
</repositories>
|
|
</project>

Interesting geography lessons

This probably sounds pretty off-topic for a tech blog, but it's not that far off. Recently I had to populate our database with ISO data about other countries so that we could accurately ship to them, and in the process learned a great deal about foreign geography and postal lessons. The biggest of these lessons was that by comparison, Canada and the US have dead simple postal systems that nobody should screw up. For example, France's geographical (and therefore also postal) subsystem is broken down into 102 departments, which may be further broken down into communes and 'arondissements'. The first 2 digits of a France postal code indicate the department (synonymous with state or province in US/CA) in which the postal code is located, so that even though France's postal structure is more complicated than the US or Canada, that complexity is well managed and actually very easy to deal with (especially for the French since they're the ones that have to move the mail in that country.) Like France, Great Britain has a far more complicated system than either the US or CA.

For comparision, here's what happens in the US/CA postal systems. Each address is broken down as follows :
123 Any St
My Town ABC123 CA

Or, in a more grammar specific way (s+ indicates there may be whitespace, ? indicates optional):
[Street number][s+][Street address,s+]
[Municipality/Town/City,s+][s+][Postal code][s+][Two digit state / province code]
Now, in the US/CA, Postal codes are \d{5} for the US, [A-Z0-9]{6} for Canada, where every CA postal code follows the pattern A1A1A1 (ie alternating letters and numbers, starting with a letter). Very specific and easy to understand.

Now, in Great Britain, a standard sample address is as follows :
[Street number][s+][Street address,s+]
([Locality][s+])?[Municipality/Town/City,s+]
[Postal Code,s+]
[County,s+]?

And to boot, GB also has Boroughts, Metropolitan Districts, and Unitary Authorities in addition to the counties. And in GB, a postal code matches [A-Z0-9]{6,7}, with no specified interleaving whatsoever, so you can have any possible combination of letters and numbers.

On top of that, there's also the issues of cultural differences within the country at hand. In Canada, there's pretty much English and French addresses, and if an address is French, it stays French (same for English), the point being that it doesn't get Anglicized or Francocized at any point within the postal system. In GB however, there are English and Welsh versions of all the addresses in Wales.

Now, this much complexity in GB (as compared to US/CA) is fine if you live in GB, but if you live outside the country and you're trying to ship to it, and be reliable, GB really blows.

It's there for a reason

I think that one of the things people realize when they start a new job (I haven't) is that the way things are done currently are done that way for a reason (most of the time, though definitely not always). Some past experience has been the basis for changing the way things were done before, and so now the new way of doing things is done to avoid some past (most likely bad) experience.
People then proceed with their work, and if they're not a fan of the way things are done, they do it their own way. In many cases, they'll encounter a previously encountered bad experience and realize why things are done they way they are. This has happened to me several times. And this happened to me right now while using Hibernate. I finally realized why they have the "saveOrUpdate" method on Session objects. Love those epiphanies.

Branching out

I recently stumbled across a glowing review of the Wicket framework that made me want to research it and try it out. The author's main argument for Wicket was that it's pretty much just HTML and Wicket, as opposed to the combination of Spring, Spring Webflow, JSP, and tons of other stuff thrown into the mix. Unfortunately I really don't have the time to mess around with it right now, but I'll try to be back on this topic later.

*EDIT* : To add to the list of things I should really learn, Seam's going on the pile. It looks very cool and it's made by Gavin King, the maker of Hibernate. (I know I've bitched about how bad the Hibernate documentation is on numerous occasions, but really, I can't blame him since I hate doing documentation just about as much as if not more than the next guy)

Better idea for multi-versioned webapps

A while ago at our company, a request was made to me to create a separate instance of one of our webapplications for a select group of customers. In doing this, I had to create a separate instance of the backing database and copy over much of a preexisting version of the application's database which was simply configuration data.

It occurred to me today while working on another part of the application that I really should have separated the configuration from the data. The hard part about that is keeping security intermingled between data (for audit trailing) and keeping security intermingled with the configuration data (as part of configuration). I'm still searching for an answer to that one.

Excel and its annoying quirks

Excel has this annoying quirk of propagating unwanted hyperlinks in the background of spreadsheets, and in our organization, it gets to the point where it infests every cell of the sheet. But, I did find a fix (sorta). Press Alt+F11 to go into the VB editor in Excel, go to Insert -> Module, and paste this in:

Sub RemoveHyperlinks()

'Remove all hyperlinks from the active sheet
ActiveSheet.Hyperlinks.Delete

End Sub

Save it, return to the worksheet, press Alt+F8 to open the macros menu (Tools -> Macro -> Macros otherwise) and run the Macro of the same name as the subroutine above. This should delete all your hyperlinks. Unfortunately, this will most likely remove all other special formatting in the cells as well. If you have a really large worksheet, it will most likely cause Excel (XP and earlier at least) to crash, so you should really save your work immediately before you do this (or before you add the Macro to the sheet).

Taking a REST

Lately I've started to get back into research mode in the hopes of making my development cycles shorter, especially since the end of a large sprint is in view. With that in mind, I find myself being continually reminded of the increasing demand for REST-ful applications (ie web applications that make use of the REST HTTP method as opposed to POST or GET). I'm finding it's at least something I shoud be familiar with as a web developer, even if I don't use it.
*EDIT* : Look up UML with Rational Rose

Spring Batch Processing

No, it's not batch processing that takes place in the springtime (although that does happen at our company). It's actually a new subproject of the Spring Framework meant to provide a convenient , common framework for batch processing, and it might be just the thing I need to help me out. Oddly, I found out about it while I was searching for the documentation for the Spring-binding package.

Quirks with Maven2 and m2eclipse

If you're using m2eclipse (which I've found to be quite useful), you should be aware that it doesn't always check the settings.xml file in the local repository or in the maven home dir, and so that pesky command-that-you're-trying-to-get-working-that-should-be-working-after-you've -updated-settings.xml-but-still-isn't-working may have to be run from the command line using 'mvn' rather than from within eclipse.

Logs and RSSH

I've learned the hard why today why logs are uberuseful (once again) and that RSSH is a pain to debug if you're having trouble with it. For future reference, check '/var/log/messages' as the first thing you do when having trouble with SSH, especially RSSH.

For those not in the know, RSSH stands for Restricted Secure SHell, and is very useful if you want to create jails for people who have to transfer files between their boxes and your server. However, it's very hard to debug because when you log in, you don't get any messages, you either get the "This is a an account restricted by RSSH" message or you get the ever so friendly and useful "Connection closed" message. As it turned out, the passwords for several of my users had expired today and this created a huge issue when they weren't able to upload batch files to our server. After spending just over an hour finding that the problem was related to SSH login, I received a very helpful message from one of the people who operates our server showing me a grepped slice of the /var/log/messages log saying that the password had expired for several of the users : coincidentally, the only users that I had tried troubleshooting the problem with.
The point of all this: check '/var/log/messages'.

*EDIT:* Today is officially known as Bad Monday for me.

Fucking Hibernate

God f*ck*ng dammit, I'm so freaking tired of fucking Hibernate's shitty ass documentation wasting my time:

In the event that you ever have to query in HQL on a component and that component contains ManyToOne elements (ie entities, rather than strings or numbers), you have to query on properties of that element rather than the element itself and have hibernate query for the ID.

...which is lame, considering Hibernate is so good about that kind of thing everywhere else.

Maven ... and why you should use it (2, not 1)

For the longest time during my developer's journey of discovery while learning how to build web applications, I've come across references to Maven and how it's a build system like Ant, but supposedly better. I've also heard untold numbers of references to it being frustrating and very difficult to use, largely due to shitty documentation. It was for this reason that I avoided learning it and just stuck with my current build process, which is just using Ant, exporting everything with eclipse, then manually uploading things to the server. Or in the case of dealing with a library I've made that I share among various web apps, compiling everything, going through Eclipse's JAR export wizard and copying files around. No more.

I must have came into Maven at the right time because I've found there to be lots of documentation (or at least a sufficient amount for myself) to learn it and get going with it. I've also converted my library from just using Ant to using Maven to build it, test it, package it and deploy it to a central server within my company that will now be the new Maven repository for the library. From now on, my applications can retrieve it from a central repository for me automatically instead of me having to manually export the library, copy it around, and maintain various versions of it in CVS. And I'm thrilled that this is going to make my life easier.

I did however run into a few quirks when getting Maven going. They're as follows:
1) You should really align your project to have the same layout structure as the one recommended by the Maven documentation. It'll get things going faster and you'll have far fewer build problems.

2) If you use Java (1.)5 or greater, you may need to manually insert configuration settings in order to get your (tests in my case, and possibly other) code to compile properly since Maven strives to achieve compatibility by default and will likely try to compile your code for older VMs such as 1.3. To configure Maven to use 1.5 (or whatever) by default, put the following snippets in your pom.xml file :

<project ... >
<!--
|
|
-->

<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
</build>

<!--
|
|
-->
</project>

3) Deploying your library (or other code) to a Maven repository is wicked easy, especially since Maven doesn't require any special software to set one up: a Maven repository is just a folder containing libraries and metadata with a prescribed folder layout and naming convention. The code for deploying to a server is as simple as this :

<distributionManagement>
<!-- This repository depends on the following settings.xml file being in the local maven2 repository
and a .ssh folder being created under ~ (or %HOMEPATH%) in Windows:
<settings xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">

<servers>
<server>
<id>[REPOSITORY NAME]</id>
<username>[REPOSITORY USER]</username>
<password>[REPOSITORY PASSWORD]</password>
</server>
</servers>

</settings>
-->
<repository>
<id>[REPOSITORY NAME]</id>
<name>My Maven2 Repository</name>
<url>scp://mybox.mydomain.com/path/to/repository</url>
</repository>
</distributionManagement>

Maven has been pretty simple to set up and use so far, once you've read the documentation and you have at least some idea of what you're doing. I also recommend getting the m2eclipse plugin if you're going to be using Maven with Eclipse. My next steps are going to be getting Maven to compile my XMLBeans for me and deploying those as well.

The head is venturing out of the ass

For the longest time, I've had my head up my ass: I've been content to build my projects with ant and the (very few) ant targets I've really needed: reports, xmlbeans and hibernate. (And fiddling around with jspc to precompile my JSPs). But now I've found that using Xdoclet2 for Hibernate is no longer acceptable since a certain bug has forced me away from it, and now I'm finding that it's time to move on to real build systems like Maven2. I've been scared away from it before by the fact that I've heard that it's exceedingly poorly documented and that it's been a total pain and has load of incompatibilities, but apparently it's been getting better and now I'm forced to learn it in the hopes of saving myself some trouble. I'm also going to have to learn EJB3 Annotations and their Hibernate extensions, but that's another matter. Let's see how it goes.

My list of stuff to learn

Frequently I find that there are dozens of technologies I want to learn, but really just don't have the time. As a list to myself (and anybody else who might care) of the stuff that I want to learn, but just haven't had the time, here's a short enumerated list:

• FreeMarker
• Scheme
• Groovy
• Grails
• Ruby
• Rails
• Python
• JPA (Java Persistence API)
• EJB Annotations (primarily for Hibernate)
• XSLT

And that's just the stuff I can remember off the top of my head as I write this. I'm sure that I'll add more later.

Reading blogs pays off

I've recently been trying to read more and more developer blogs in the hopes of improving myself as a developer. One of the blogs I came across is Brian Burridge's, and he recently had a good post about a little utility called Denim. It's a storyboarding/diagramming tool that was originally meant for web developers to quickly model their sites, but I see no reason why it can't be used for user interfaces in general. The program's made in Java so not unexpectedly) its Swing UI is ugly as shit, but it's so functional it's insane. This is how I'd make an interface (though I'd try to pretty mine up), but they got the basics right: functionality before beauty. If you're a serious developer, I think you should at least give it a shot and play around with it. It's free, open-source, and made at an academic institution. BTW, I jacked the latter link right from his page to save anybody reading this some time, not to take away from his blog, which I think you should check out.

The difference between links and forms ... at least for Spring

This post is really just a mental note to myself: the difference between links and forms is that links do not submit form variables. (Duh!) Therefore, you can't bind varibles in forms to Form Backing Objects in spring if you're going to use a link to do the transition rather than a form button.

(Web)Harvesting the web

I was recently assigned a task whereby I had to obtain data from the website of one of our commercial services, but the website is completely Web 1.0 and has no APIs of any kind that
we could hook into. To give you a bit of context, the site performs transactions on our behalf and we need information about those transactions. As my boss saw it, there were only two solutions really available :
1) Have somebody sit at a computer and download the transactions every 15 minutes
2) Have a computer set up to run a screen macro to log in and download the transactions file every 15 minutes (better, but still no where near ideal)

I think without even realizing it, my boss gave me the idea for option three. He continually mentioned the idea of screen-scraping the page to retrieve the information. Traditionally this means taking a visual representation of something and extracting information from what's essentially a picture. After doing some reading, I interpreted his suggestions to mean that I should find a way to do a web-scrape on the page (subtle difference). After doing some research, I found the perfect library for doing it in Java.

It's a project called WebHarvest, and it's fairly simple yet really powerful. You start off by writing a Scraper configuration in XML, load it in code, run the scraper, and it will store your data for you in the Scraper to retrieve when need it. The library itself works by doing either POSTs or GETs to a page, taking the response data, (almost certainly) doing a transform to convert the HTML to well-formed XHTML, and then running XPath queries and regular expressions on the result to get the data you need (ie rows of a table). It's incredibly powerful and if you need a solution where you want to automate logging into a website and retrieving data, then this is a great way to do it.

UTF-8 ... and why you should use it, part 2

I've sadly discovered that my previous post was lacking some bits of information. For starters,
when using tomcat, you should place this directive ( URIEncoding="UTF-8" ) in your connector definition. Then, you should use this attribute ( accept-charset="utf-8" ) with all your forms; this isn't really necessary, just good for practice. Then, you should ensure that you have a filter defined within your web.xml document to coerce incoming requests into UTF-8 format. The easy way of doing this if you're using Spring is to place the following definitions into your web.xml document :


<filter>
<filter-name>charsetFilter</filter-name>
<filter-class>
org.springframework.web.filter.CharacterEncodingFilter
</filter-class>
<init-param>
<param-name>encoding</param-name>
<param-value>UTF-8</param-value>
</init-param>
</filter>

<filter-mapping>
<filter-name>charsetFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>

This will filter all your incoming requests for you. *Edit* : As one additional side note, if you're moving an existing database to UTF-8 (which is fairly likely since new production databases aren't started that often), then you'll also need to run an ALTER DATABASE statement on the database in MySQL to set its default character set to UTF-8.

Lousy documentation at its finest

This post's going to have to be short because I desperately want to get out of work to go play poker. The short version: I just spent the last three hours in a rage at MySQL because, for a new person following the documentation on their site, it's hard to realize that despite example after example of calling a stored *FUNCTION* with CALL [function name](); syntax, said syntax applies only to stored *PROCEDURES* and you should call a stored function like so : SELECT [function name]();

Fuck, sometimes I hate my job as a developer and I wish I could just go work at a bar, like certain friends. And it's been one of those days.

Intricacies of Hibernate

If you're a serious developer / software engineer, then you know how important it is to keep a system you're building as modular as possible for all the standard reasons: separation of concerns, testing, etc . To this end, I've been developing my systems at work to use Data Access Object interfaces, rather than giving them direct exposure to hibernate. One of the benefits of this is that it abstracts my business and presentation layers from my persistence layer. One of the detractors is that it makes it harder to do certain other things:

1. Maintain efficiency when performing database operations
2. Use Hibernate effectively when loading / storing objects.

Case in point: for the last two days I've been trying to figure out why my detached objects keep having exceptions thrown regarding dirty collection references when I try to update them. As it turns out, the Session.update() function is really best used within the context of a transaction, where you modify your business objects within the transaction, so the persistent objects never become detached (this last portion is a lot more meaningful if you're familiar with Hibernate parlance) However, in a well modularized Web application, your business, presentation and persistence layers are detached so one never knows that your business objects are being persisted with Hibernate, and as such you must 'detach' your business objects from their sessions. Updating them in a new session won't work if you want to use Session.update. Instead, you need to use Session.merge . The subtle difference between these two is that .update checks things like dirty member collections and caching, whereas .merge verbatim overwrites the existing state of your business objects' members in the database. It took me two days to realize this and I'm very pissed off. I really wish it had been mentioned in the Hibernate documentation.

I still don't know enough, but I know something

For quite a while I've wanted an easier way of using Jasper Reports in my web applications. My first go around with Jasper Reports involved implementing custom servlets for Jasper reports and using (what I now realize is) a giant kludge to add to and maintain those reports and compile and run them on the fly. I've clearly learned a lot since that first try. My second go around with Jasper Reports involved precompiling them with an Ant task and using report views in Spring after I moved to the Spring framework. This was a lot better than what I had been doing, but I still made mistakes when I didn't quite know the best way to get parameters into my report.

Now I'm about to start with round three. This round started with an epiphane the other day: why not use a tag library to insert a Jasper Report into a page rather than dealing with entire views ? This would surely be far more compact and would even look far better. After googling for several hours and finding nothing even close to satisfactory in this regard, I decided to make my own tag library. I whipped up the tag library tutorial on Sun's website, and within a couple of hours, I had the very beginnings of a rudimentary tag library going, and I had successfully embedded a Jasper report within a page of one of my web apps. (It was a piece of shit test report embedded within a div, but it was more than enough for proof-of-concept).

This short snippet:
<div style="overflow: auto;">
<jr:report printerName="${printerName}" reportName="test" reportParameters="${sessionScope.mymap}"/>
</div>

Showed just how powerful and useful tag libraries can be. Combining this with Spring's dependency injection made things a lot easier too. I thought to myself: this could be really useful. And I was really surprised that nobody else had done this already. In light of my epiphane, I think I'm going to start up a source forge project for this, probably call it JasperTags. Who knows, it might even be useful. But there's a bunch of other stuff I need to learn first, like Maven 2, JIRA, and numerous other little useful developer tools so I can at least put on the facade of being a competent open source developer.

RAD, Spring, and thoomp!

Ok, so only the first two of the title trio have anything to do with each other. The latter's just the sound an air cannon makes and I thought it went well here. Moving on:


One of the best things about the Spring Framework for Java is that thanks to the fact that it's an Inversion of Control (IoC) container, you can very easily add to projects and update them with new code. The latest thing I did with one of my Spring-based projects was to add notifiers when certain batches are uploaded to the system. The first notifier I did was an email notifier, so nothing special. But immediately after I finished implementing it, my "Wouldn't it be cool if ..." sense kicked in and I considered implementing a notifier for MSN Messenger so that certain people in our organization could receive instant notification for batch uploads. After two minutes of google searching, I came upon the JML (Java MSN Messenger Library) posted on SourceForge.net . After a few minutes of looking at the example code posted with the project (still fairly sparse), I came up with the following to inject an ApplicationListener into my Spring context:

<bean id="msnMessenger" class="net.sf.jml.impl.MsnMessengerFactory" factory-method="createMsnMessenger" init-method="login" destroy-method="logout">
<constructor-arg value="my-email@hotmail.com"/>
<constructor-arg value="mypass"/>

<property name="logIncoming" value="true"/>
<property name="logOutgoing" value="true"/>
</bean>

<bean id="msnMessengerNotifier" class="my.package.ApplicationListenerImpl">
<constructor-arg ref="msnMessenger"/>

<property name="recipients">
<map>
<entry key="email_of_our_batch_processing_guy@ourcompany.com" value="His Name"/>
</map>
</property>
</bean>

...and the relevant implementation code :

this.msnMessenger.sendText(Email.parseStr(emailAddress), messageString);

...and in under 20 minutes I had an instant messaging notifier implemented thanks to the ease of Spring and the JML library. Please go see that site if you need instant messaging in Java with MSN Messenger, it's actually a great little project and they deserve to be supported. It seems however, that the one little caveat of this (and it's not even JML's fault) is that the notifier and the person receiving the notification must have each other on their contact lists.

I've lost all meaning for the term "day off"

I have. Really. I'm sitting around researching screen scrapers (because it'll be horribly useful for work, sadly) and while Googling and reading blogs, I've come across a very useful looking tool called Firebug (http://getfirebug.com/) that lets you inspect and modify a page's DOM in real time. The original thread that prompted this was of course searching for existing methods of web scraping. I found a pretty good looking thread on this guy's blog which I can't wait to try out at work on certain pages that I probably can't mention in a personal blog. Suffice it to say that the content of that blog entry has me convinced, now more than ever, that Firefox is hands down the best browser ever.

<edit> This page is super useful for learning screen scraping with Ruby </edit>

Staying on the ball

If you know me, then you've seen my analogy with two fists to staying on the ball. If not, just know that it's a great analogy. The real topic of this post is that as a software developer, during the course of your education (if you have one beyond high school) you can't help but have this one idea beaten into you: you have to stay current with trends in software development and you have to be on the ball with learning new tools to get the job done quicker, otherwise you will find yourself out of a job.

As I read through blogs, articles, newsfeeds, etc, I'm learning more and more how behind the times I really am. I think part of that is due to my education, ironically enough: there's enough out there that you need to know to be truly effective in the workplace that they can't possibly keep up in universities and colleges, which makes it all that much harder to get a really good job that's going to pay you a lot of money. As a result, you have to find a way to keep up and stay current on your own or risk being left behind in a heap of a job with shit for pay. I don't want to do the latter. It's with that in mind that I realize there are some buzzwords / buzztechnologies that I should at least be familiar with if not know intimately and use on a day-to-day basis:

1. .NET / C# / ASP.NET : There's an assload of jobs out there that use this rather than Java or any of the popular open-source technologies. And unless you go a technical institute, they don't teach you this; it's something you have to learn on your own. The need to learn .NET and other Microsoft technologies has recently been burnt into my own mind after I went for a job interview and they went with somebody else, not because I didn't know .NET (but could learn it very rapidly given my existing experience) but because they could get a technical institute (*cough* NAIT *cough*) grad with maybe a tenth the software development experience far cheaper than a university-educated developer.

<interlude>
My experience is mainly with free and open source (FOSS) technologies like Java, Spring, Hibernate, Javascript, XML, HTML etc with a fair bit of Ruby and some Perl / Python / BASH scripting thrown in. While certainly very interesting and academically stimulating areas of software development, they unfortunately don't look nearly as good on a resume as does Microsoft technologies, which a majority of companies are very much locked into.
</interlude>

2. XQuery, XML databases, XSLT, Atom/APP, XForms, AJAX: All XML related technologies, their goal is to make web development easier and make web-based applications more robust and usable. I think it's important I at least get familiar with these technologies, if not use them on a day-to-day basis for my projects at work.

<interlude2>
As somebody who's graduated from a Canadian university with a degree in Engineering, I have the chance (and desire) to get my PEng designation. However, due to certain restrictions of my local engineering association, I have to meet certain strict requirements with regard to the work that I do. Some of those restrictions include having to work on engineering-related software (ie essentially engineering software for other engineers and engineering projects).
</interlude2>

3. Embedded devices / Device drivers / System-level development: this is something that has long garnered my interest, but that I've never had the time to devote to learning. I'll especially need it if I'm to accomplish certain career objectives as mentioned above.

There's a number of other technologies / etc, whatever that I can go on about that I need to learn, but it's really too numerous to mention in a single blog post. I've enumerated three balls that I want to be on, and to accomplish that, I'm going to have to jump from ball to ball on a constant basis and hope I don't fall flat on my ass.

Internet Explorer quirks

What I'm about to say is neither a new thought nor a new sentiment in the least: Internet Explorer sucks.

I don't just say that because I'm a huge fan of many other browsers, but because as somebody who develops applications that have to be completely browser-agnostic, dealing with Internet Explorer is a fucking pain. The quirk that's got me today is this: Internet Explorer doesn't properly handle XHTML. The consequence of this is that it doesn't recognize self-closed script tags as valid (ie <script src="blah.js" type="text/javascript"> ) and as a result will biff if you try to use them. Firefox, Opera and SeaMonkey all have no problem with it. Unfortunately I don't have a Mac with which to try out Safari and Camino, but the point is that using the above code snippet will cause Internet Explorer to NOT EVEN RENDER THE PAGE. It strikes me that this is a horrible bug, especially for a large corporation like Microsoft. It's this kind of BS that drives people to better solutions. I feel bad for the people who haven't realized there's better shit out there than IE. Sorry for the harsh language, but yes, I'm bitter.

The things every user should know about Linux / Unix

There's a ton of things every user should know about Linux / Unix, and I'll be the first to admit that I don't know nearly as much as I should. Here's some basic stuff though (I'll enumerate it as much to remind myself as to inform others) :
user[add|mod|del] - Administrate users. As with anything else, read the man pages and learn the arguments they take inside and out.
group[add|mod|del] - ...same thing, just for groups.
chsh - Change your shell
echo $0 - Get the name of the shell you're currently running
whoami - Lets you know what user you're running under
users - Lists all the users currently logged into the system.
echo "stuff" /dev/pts/[some integer here] - Echoes text to the running pts, good for command line communication with other users.

Some important locations:
/etc/groups - Stores all of the system's user groups
/etc/passwd - Stores all of the system's user information

There's a ton more that should go in here, but I need to finish this post and start a new one. So yeah ... done.

Fucking quirks

In an attempt to make one of my web applications more user friendly and bulletproof it against silly people at the same time, I've been adding Javascript to a lot of the pages. The basic functionality and validation is already there, but I've decided to use Javascript to help make a lot of the repetitive tasks go quicker and generally make the app easier to use:

This means automatically selecting certain options when other options are selected, putting in "Select All" and "Select None" buttons for large lists of items, etc. It's during these times that I've come across (1) one of Javascript's (in particular) little quirks, and additionally (2) a larger issue which affects all scripting languages.

1. In Javascript, when you want to select all the checkboxes (or radio buttons) in a form (or even a particular set), Javascript becomes quite inconsistent: if there's only one element in the group, it returns the element itself, rather than an array. I'll let you mull that over for a couple seconds, then keep reading.


Done ?

Ok. Speaking to any developers out there (or anybody else who's written code) who may be reading this, is that not the most retarded idea ever ? Seriously. That's just asking for trouble and wasting large amounts of time unnecessarily on debugging, especially for somebody unfamiliar with (and trying to learn) the language. Not to mention the fact that it leads to further code cruft due to (in my mind) unnecessary case checking that could be eliminated if something like form.checkboxes or form.radios returned a collection in any case, regardless of how many elements would be in the set. I mean, come on! Admittedly, I've had far less experience than the people who designed and implemented the Javascript language, but I think that kind of design decision is retarded on any level. I'm sorry guys, but it is.

2. While trying to debug the above mess, I came across the larger issue that always affects scripting languages: the lack of strongly-typed objects. By this very nature of scripting languages, this makes debugging harder than any given strongly-typed language, and it sucks. I suppose though, in all fairness to the Javascript language, it's the implementors that are responsible for providing help with debugging. On that note, Microsoft (and Internet Explorer X.X) are horrible for that sort of thing, displaying only "[object]" when trying to put an object to a string. Firefox is a bit better, in fact, it was Firefox that got me through my little ordeal above when I was expecting to see something like "[array]" and it gave me "[object HTMLInputObject]". Thank you Firefox and the developers working on the Mozilla project, it seems like you actually know something and understand what programmers have to go through when debugging things. To the people at Microsoft, how have you not thought of shit like this ? You suck.

Google and Firefox are officially fucking scary

To give you a little bit of background, I recently upgraded to Firefox 2.0 and I frequently use the hotkeys and search bar. A lot. More than you'll ever know. The Firefox search bar has had the ability to bring up entries that you've searched for in the past for quite a while, so I didn't notice this little trick until just recently :
When you have Google selected as your current search engine (I frequently use others as well, such as Wiki), it will automatically search for possible matches for what you're typing in and suggest them to even if you've never searched for it before. That in itself is a neat little trick, but the relevance of the results today astonished me when I wanted to search for the lyrics to "I believe in a thing called love" by The Darkness. I started by typing "the darkn" and right then and there "the darkness lyrics" was the third entry in the suggestion list. When I got to "the darkness i ", "the darkness i believe in a thing called love lyrics" was the first entry in the list. Is it just me, or has Google's seeming omnipotence combined with its usefulness and relevance gotten downright fucking disturbing ? I'm going to revel in this realization the rest of the day.

UTF-8 ... and why you should use it.

I've recently discovered that for the sake of compatibility with various languages and internationalization, that UTF-8 should be used for everything. That might sound like
a very vanilla statement with nothing to back it up, but I really don't have time to go into
the reasons I've discovered for using UTF-8. Also, suffice it to say that it has now been
forced on me as a requirement, one which I don't mind adhering to, but it takes some
effort to convert. Case in point is using Hibernate as your persistence framework. If you're
using XDoclet2 to generate all your mapping files from annotations, the conversion is very simple. In your component definitions in your ant task, make the the following changes:
<component classname="org.xdoclet.plugin.hibernate.HibernateMappingPlugin" destdir="${basedir}/src" version="3.0">

and :
<component destdir="${src.dir}" classname="org.xdoclet.plugin.hibernate.HibernateConfigPlugin" jdbcdriver="${jdbc.driver}" jdbcpassword="${jdbc.password}" jdbcurl="${jdbc.url}" jdbcusername="${jdbc.username}" dialect="${hibernate.dialect}" cacheprovider="${hibernate.cache.provider_class}" cacheusequerycache="${hibernate.cache.use_query_cache}" jdbcpool="" jdbcisolation="${hibernate.connection.isolation}" showsql="${hibernate.show_sql}" version="${hibernate.version}" style=""/>

... and that's all there is to updating Hibernate to generate your files with UTF-8 encoding. However, if you're making a web app, that's not the only thing that you're going to have to change. All your JSP(X) files should start with the following :

<jsp:root jsp="http://java.sun.com/JSP/Page" version="2.0" c="http://java.sun.com/jsp/jstl/core" fmt="http://java.sun.com/jsp/jstl/fmt" spring="http://www.springframework.org/tags" display="urn:jsptld:http://displaytag.sf.net" authz="http://acegisecurity.org/authz">

<jsp:directive.page language="java" contenttype="text/html; charset=UTF-8" pageencoding="UTF-8">

<jsp:output declaration="false" element="html" public="-//W3C//DTD XHTML 1.0 Transitional//EN" system="http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">

... or something very similar, the key point being that the encoding(s) specified are in UTF-8.
Also, since you're most likely using a database if you're using a web application, you'll need to update your database settings. One of the most common databases out there (and the one that I use) is MySQL. To permanently change the settings of MySQL to use the UTF-8 collation, you'll have to find your appropriate 'my.cnf' file and put this line:
default-character-set=utf8

...under the [client] and [mysqld] headings.