Reflection in Ruby

I've recently gotten started going on a Ruby on Rails project for my company, that's meant to be a simple app that we'll throw up and let run for any prospective customers. But I ran into a problem with copying property values between objects, since I'm not as familiar with the language as I'd like to be. The Object type defines a method called 'send' which lets you invoke a method with given arguments. However, I wanted to be able to assign property values via this method, and just sending the name of the property as the name of the method to invoke wasn't working. (Works just fine if you want to get the property). As it turns out, the method for invoking the setter of a property is brain-dead simple : use the name of the method as you'd define it if you were overriding the setter for the property. For example, if you have a property called 'name', you'd define the setter for it as follows :

def name= (name)
  @name = name
end

So if you wanted to set the value of this property reflectively, you'd do something like :

 @person_instance.send "name=", "John Smith"

A moment of developer clarity and zen

I had a moment today where I was just letting my mind wander. It was one of those moments where you have a brilliant idea that just seems so common sense that you can only ask yourself "Why the hell didn't I think of that before ?" This moment of clarity happened to be concerning my horribly slow laptop, with it's horribly old ATA 100, 5400 rpm hard drive. Then it occurred to me : just like with video encoding that I had setup months ago, I could setup a RAM disk to speed up Eclipse by storing my workspace on the RAM disk to speed up access to all the (ridiculous number of) temp files that Eclipse uses. On Ubuntu Linux, it turns out this was ridiculously easy : http://ubuntuforums.org/showthread.php?t=182764

The really amazing part was just how much this sped things up : it took my project build time from 6 mins, 30 seconds down to 35 seconds. And those are real numbers (plus or minus a couple seconds, due to slight variations in repeated builds.)

More JAXB quirks

Today I ran across yet more quirks of JAXB 2.0. I'm getting really tired of running across stupid errors that are difficult to debug because the programmers who wrote JAXB are lazy. Today, I was receiving a java.lang.InternalError when I tried to compile a schema from beans I'd recently refactored. The error I received was something along the lines of "could not escape schema namespace". As it turns out, it was caused by the lack of an @XmlType annotation on the enums I had added to one of my XML serializable beans. I hope this helps anybody who has the same problem.

A quick note about JAXB

JAXB (the Java API for XML Binding) is a great piece of technology (2.x, not 1.x). However, if you're starting an application and your JAXBContext is throwing an exception saying that it's unaware of a given class, there's a quick fix. At this point, I'm guessing you've configured your JAXB context to use a package, and the class that the JAXBContext is unaware of is in the given package, and you're scratching your head as to why it can't seem to find a class that's in the proper package. Well, here's the most probable answer : if you're using package-configured JAXBContexts, you need to have factory methods for all top level elements, similar to the following :

    /**
     * Create an instance of {@link PurchaseOrder }
     * 
     */
    public PurchaseOrder createPurchaseOrder() {
        return new PurchaseOrder();
    }

... assuming that mypackage.PurchaseOrder is the class the JAXBContext is complaining that it's unaware of. This little quirk took me a couple hours to debug because I had just regenerated some web services code using Apache CXF 2.1.1 from an updated WSDL document for an updated Web Service that our company uses. Little did I know, that in comparing the two ObjectFactory files in Eclipse, I had missed copying over certain important functions. Hopefully this helps you.

Pure FTP on Ubuntu

Pure FTP is a great FTP server and I love it because it's ridiculously fast, reliable and works well for our company. The only problem is that it's tricky to configure on Ubuntu because just like with numerous other programs, the people at Ubuntu, in their infinite wisdom, felt the need to fuck with the program's default way of doing things. Specifically, in Ubuntu, command line arguments for starting the FTP service are done through individual files for each command line option you want the service to be started with, with values for the options placed inside each of the files. This is a stupid way of doing things, but that's not the topic of this post.

The problem I've been having lately is that I recently deployed new projects within our production network, and these new projects required access to the FTP server, and aren't on the same box as the server, as is the sole other project that has been using the FTP server. I'd been getting problems trying to connect to the server from any LAN box, but not any external boxes nor the machine itself (localhost). The error I got back was "421 Service not available". I googled around for hours and found nothing useful, until I started realizing that other people were getting 421 errors when their PureFTP instance was misconfigured, but with different messages, and then it got me thinking that maybe my instance was somehow misconfigured.

I re-read the documentation for PureFTP and after an hour or so, it hit me that the server does reverse lookups to resolve fully qualified names, and such resolution doesn't work properly on our network (for good reasons that I'm not going to go into). After disabling reverse DNS resolution with the -H startup option (`echo "yes" >> DontResolve` in the configuration directory in Ubuntu), the problem went away.

I hope this helps anybody else who runs into the same problem.

More Tomcat quirks

I recently ran across a problem with receiving a LinkageError in one of my web applications. The problem occurred when Acegi security tried to use libraries that were not yet loaded and tried to load them itself. Strangely, the error only occurred while using my RESTful API with a Spring Framework controller. If I logged in through my web interface which uses Acegi and Spring MVC, then the linkage error didn't occur.

After numerous hours of Googling around, I found a small company's JIRA issue tracker that reported the same bug, but also that it only occurred when using the security manager. After disabling the security manager on my own Tomcat instance, I found that the problem went away. I'd very much like to have a better resolution, but it's going to have to do for the time being.

Tomcat quirks

In recent weeks, we've had problems with our company's servers going down because of power problems at our colocation provider. As a result, the fact that Tomcat hadn't been properly configured to boot at startup had really burned us. Consequentially, I've been spending the past few days troubleshooting why the servers wouldn't startup, and ensuring that they do come up properly at boot time in future. The following are some of the things I've run across.

• 
  The JAVA_HOME and JRE_HOME variables must be set. If they're not, then Tomcat won't start, at least on Ubuntu Gutsy Gibbon Server. This is a problem I encountered with the default installation of JIRA on one of our production servers.
  


• 
  When using EhCache with Hibernate, the tmp directories don't get deleted, regardless of whether or not Tomcat is shutdown properly. This will block the container from starting up again, so these temp files must be deleted. I encountered this problem with my own applications.
  

Finally! The trick to proper Java Date and Calendar formatting

It's well documented in the Java API as well as numerous forums and blogs, that the Java API is not properly implemented, and in some respects flat out broken, hence why Java 7 is going to have a brand spanking new Date/Time API based on the Joda Time library hosted at sourceforge.net. In implementing Dates / TimeZones in Java, the developers of Java saw fit to separate time zones from date implementations because, really, a date is just a number of milliseconds from Jan 1, 1970 at 00:00:00.000 GMT in the morning. This is a noble goal which I whole heartedly agree with : dates should be represented separately from their timezones because the printed representation of a date will change depending on time zone, whereas no matter where you are in the world, the number of milliseconds from the Epoch has not changed. However, the Java developers implemented this concept poorly. They chose to attach TimeZones to Calendars and then not have them affect any of the time fields, which is fine. But the TimeZone attached to a calendar is not used when using a DateFormat directly. Here's the key : the DateFormat object uses its own internal TimeZone during formatting (which it takes from the currently running JVM) by default. In order to format using the TimeZone attached to a Calendar, you have to explicitly set the TimeZone of the DateFormat to that of the Calendar instance you want to format.

An interesting 'accesskey's quirk

Since I learned about accesskey support in HTML, I've been including them where appropriate in my web applications to speed up my access and usage of said applications. However, when trying to use the accesskey for a password field, P, I ran across an interesting quirk. The accesskey modifier is different for various browsers. In order to use access keys in IE, the modifier is Alt, for Firefox it's Alt+Shift, for Opera you press Shift+Esc, then the accesskey. I primarily use Firefox (go open source), so when I pressed Alt+Shift+P to access a password field, it opened Windows Media Player. As it turns out, this is the hotkey for accessing WMP from the toolbar (ie the miniplayer). To prevent this hotkey combination from opening WMP, you'll have to disable the toolbar by going to the taskbar, Right-Click -> Toolbars and uncheck Windows Media Player (I don't like the mini-player toolbar that much anyway).

Spring Framework URL mapping quirks

My recent foray into developing using RESTful principles and resources has lead me to an interesting quirk in the Spring Framework, which I use for our company's primary application. As it turns out, the URL mapping scheme requires that you set 'alwaysUseFullPath' to true so that the entire path of a URL that gets submitted to the framework gets inspected to determine the proper controller mapping. Otherwise, it can lead to some quirks, like not submitting the request to the proper controller. I had to learn this the hard way through several hours of debugging.

Why you should always export your sources with your Java projects

With the advent of popular and widespread Maven 2 adoption, there's no reason you shouldn't be using it to build your open-source or closed source Java projects. With that in mind, it's dead easy to use Maven's 'source' plugin to automatically generate a separate JAR with your project's sources in it when you package and deploy your project. Posting the sources with your project not only allows other people to easily debug their code while using your project, but also serves as a backup in the event of a catastrophic failure of a computer hosting your code for which you did not have proper backup procedures already in place. In the event that you couldn't tell, that happened to me tonight.

JAXB schemagen - Finally got it working

For those not in the know, schemagen is a program that comes as part of the Java Web Services Developer Pack and it's used to generated XML schemas from JAXB annotations on Java beans. It so happens that the creators of the JAXB reference implementation also made a Maven 2 plugin for this program. I had previously attempted to use it to generate schemas in my initial attempts with JAXB, but had been presented with a slough of unfriendly exceptions being thrown at me whenever I ran the plugin. Not having the time back then to really play around with it and being considerably less experienced with JAXB, I had to shelve it and find other, less satisfying solutions to my problems with generating documentation.

Recently, I've had to come back to using JAXB because I'm making a RESTful Web Services API that leverages the power of both JAXB and Hibernate to do all my heavy lifting for me. This time around, I wasn't willing to tolerate a lack of schema to give to our users of the API, because it would mean a lot more work for them, and a lot more work for me. This time, I decided to be persistent and dig around the jaxb-schemagen plugin to make sure I could get it working. I'm proud to say that my perseverance paid off, and I now have my RESTful API schema being automatically generated for my application. Here are the problems and the solutions I ran into when I was trying to get going on this:

NullPointerException

I came across persistent NullPointerExceptions while trying to run the plugin at first. The stack trace indicated that the problem was with apt, the Java annotation processing tool. I later discovered that in order to properly map your classes into XSD schema types, you must specify an @XmlType annotation on *all* of the classes you wish to map (at least with JAXB 2.0, which is the version I'm currently using)

ClassCastException

After I resolved the first problem, I came across another pesky exception that likewise did not provide any useful debugging information. The plugin output indicated that it was finding annotations that it didn't know how to deal with, specifically the Java Persistence API annotations that I was using for Hibernate. After much digging around on Google and some detective work, I discovered that any annotations encountered by the jaxb-schemagen plugin had to be on the Maven runtime classpath. I double checked my POM to find that I had specified the scope of the JPA / Hibernate annotations as 'provided', which instructs Maven not to load them into either the compile or runtime classpaths. The initial reason I had for specifying the dependency scope as 'provided' was so that they would not needlessly get included with my project WAR files. I changed their scope to 'compile' (which also includes 'runtime') and voila, apt could now properly detect the unknown annotations, and get past them to deal with the JAXB annotations it needs to create the XML schema.

If anybody else encounters similar errors with JAXB-schemagen, I hope they find this page and find it useful.

Fixing package retrieval quirks in Ubuntu Gutsy Gibbon Server

In the default installation of Ubuntu Server (Gutsy Gibbon), the default source of packages is the CD-ROM drive. That's fine if you make the assumption that there will always be somebody physically present in order to maintain the servers, but for most server's these days, that's a fallacious assumption. The really bad part about this default, is that there's no way I know of at the time of this writing to override this on the update command. In our company, we have our servers hosted off-site like so many other companies, so everything must be done remotely. Therefore, the CD-ROM as a source for Ubuntu packages is a bad idea. To remedy this, you have to go into the apt-get configuration stored at

/etc/apt/sources.list

, and you'll have to find a line similar to :

deb cdrom:[Ubuntu-Server 7.10 _Gutsy Gibbon_ - Release i386 (20071016)]/ gutsy main restricted

Comment out this line, try your update again and you should be good to go.

Fixing Messenger when you've accidentally screwed it up

If you're like me, you like to have your Windows Live Messenger conversation windows be as simple and clean as possible, with no toolbars, and no contact pictures. To that end, I removed the toolbars from my messenger contact windows so that I wouldn't have to see them. However, this became a problem one day when I dragged a picture that I intended to send to someone into a conversation window, and instead of sending it to them, Messenger permanently changed the background of my chat windows with that person to that picture. To resolve this, I had to open a chat window with person X, press the Alt key to get the old-school windows standard toolbar to pop-up at the top, and then go to Tools -> Show toolbars -> Standard. Then from the standard toolbar, you can click on the 'Show Menu' button on the top right of the conversation window from the standard toolbar, and go to Tools -> Backgrounds ... . This wil take you to a screen where you can select a blank default background. However, dragging the picture may have changed the default background colour for the window, as it did in my case. To remedy this, you have to change the background colour to match the rest of the Messenger colour scheme, and you can do so by clicking on the paintbrush icon on the standard toolbar, and selecting a default background colour from there to make things consistent with the rest of the Messenger colour scheme.

How to recurlively delete a folder and its contents in PowerShell

The command :

Remove-Item -recurse -force [directory name]

The really sad thing is that I googled this and there are no sites that explicitly state how to do this. The comparable command in bash :

rm -rf [directory name]

Yet one more reason I hate Microsoft and PowerShell in particular.

RESTful web services with JAXB 2.0

In case you're not familiar, JAXB (Java API for Xml Binding) is an API developed by Sun for binding between XML and Java beans. The first incarnation, 1.x, was interface+implementation based like XMLBeans from the Apache Project. The second incarnation, 2.x, is annotation+pojo based. It's a very useful API, especially when combining it with Hibernate (and the Java Persistence API) to make restful web services. But, that's not really the point of this post. The point is that I've been using JAXB 2.0 for web services much in the past, but this is the first time where I've had to map the "class hierarchy per-table" pattern used by Hibernate into XML. As it turns out, JAXB 2 has very good support for this pattern. You can read more on this at the java.net project page for JAXB.

Logging for tests using the Spring Framework

It's been a while since I've posted to the blog 'cause I've been so busy, and it seems fitting that this be a good way to resume posting, as this issue has pissed me off quite a bit and been a major thorn in my side for the longest time.

The Spring Framework has some pretty good support for creating test classes for your application, however it by default does not properly initialize log4j logging when doing tests, and I found out today why. When running your application in a Servlet container, you'd configure Spring logging in web.xml. However, when running in a standalone context, Spring has no way of knowing how you want logging configured, so it leaves it up to log4j to configure itself. On that front, you have to realize what log4j's default configuration strategy is : reading a 'log4j.properties' file from the root of the classpath. Once this hits you (and it took me a while), getting logging running for your test cases becomes a simple matter of placing a valid 'log4j.properties' config file in the root of your test classpath, and logging starts working properly, so now you can read those pesky hibernate generated queries off your test log .

Determining which version of Ubuntu you're running

At our company, we use Ubuntu to run our servers. Sure, there's other distributions with potentially better performance, but none so easy to setup, and that was the deciding factor in choosing which distro was going to run our systems because I had to set it up, and since I'm both developer and system administrator (*very* small company), I really don't have time to fuck around and I need to make things as easy on myself as possible. To that end, I've decided to start writing tutorials on everything I learn how to do, and I needed to make sure I note which version of Ubuntu / Linux I'm using. To that end, in order to determine which version of Ubuntu you're running, run this :

cat /etc/issue

I learned of this little trick from here.

*Edit (2008-10-29)*
A much better command for determining the current Ubuntu version you're running :

lsb_release -a

More random MySQL glitches

I ran into the following exception recently when moving an application from one server to another :

java.lang.NullPointerException
 at com.mysql.jdbc.StringUtils.indexOfIgnoreCaseRespectQuotes(StringUtils.java:959)
 at com.mysql.jdbc.DatabaseMetaData.getCallStmtParameterTypes(DatabaseMetaData.java:1296)
 at com.mysql.jdbc.DatabaseMetaData.getProcedureColumns(DatabaseMetaData.java:3670)
 at com.mysql.jdbc.CallableStatement.determineParameterTypes(CallableStatement.java:702)
 at com.mysql.jdbc.CallableStatement.(CallableStatement.java:513)
 at com.mysql.jdbc.Connection.parseCallableStatement(Connection.java:4422)
 at com.mysql.jdbc.Connection.prepareCall(Connection.java:4496)
 at com.mysql.jdbc.Connection.prepareCall(Connection.java:4470)
 at org.apache.commons.dbcp.DelegatingConnection.prepareCall(DelegatingConnection.java:275)
 at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareCall(PoolingDataSource.java:292)

As it turns out, there was a permissions issue whereby the function was created with a line specifying the definer and this causes the 'SHOW CREATE FUNCTION' call to fail and return null unless the user calling the statement has permissions for select on the 'mysql' database. Some recommended solutions (that I didn't get around to trying) were changing the call to 'prepareStatement' as opposed to 'prepareCall'. The issue is logged as MySQL Bug #24065. Lately it just seems as if I keep hitting every bug out there, and can't get anything done. I hope this helps somebody.

Commons VFS (FTP) problems

I recently had a problem with Commons VFS whereby I could log into an FTP server and list the contents of the initial directory, but no contents were shown, nor could I otherwise resolve any of the contents or subdirectories. As it turns out, there's a problem with FTPClient which is part of the Commons Net package, and the basis for the FTP module of Commons VFS. After logging in, if the VFS client tries to set passive mode and the server doesn't respond that it allows passive mode (ie PureFTP specifically), then the client just continues as if there's no problem instead of throwing an Exception. The result was getting an error message like "The file root [ftp://myuser:*****@localhost/my_sub_folder] is not a folder". Since I wasn't able to get PureFTP successfully configured to allow passive mode, I was able to resolve the problem by forcing the code to use active mode instead. Fortunately this wasn't a problem since the FTP server in question was on the same machine as the code trying to access it (ie not across the boundary of a router).