Mmm, software

Deleting more Subversion folders

2011-06-26T10:08:00.000-07:00

Further to my previous post on deleting subversion folders with powershell, I recently had the need to do the same in bash on linux. Here's a useful little command :

find . -name .svn -exec rm -rf {} \;

Determining what ports Tomcat is running on

2010-10-13T17:54:00.000-07:00

I've recently had problems connecting to a Tomcat server setup by another developer. In order to troubleshoot these problems, I wanted to use netstat to see what ports were being bound to, but apparently server sockets don't come up by default. If you really want to see what server sockets are in use on your machine, use the following :

netstat -lntp

The undocumented 't' option will cause netstat to show server sockets.

Event models in Silverlight vs WPF

2010-10-01T15:44:00.001-07:00

To really understand the event model in Microsoft's Silverlight / WPF frameworks, you need to start off with a proper mental model. You can think of the root XML element in a XAML object as being at ground level. Each successive child XML element goes deeper into the ground. With that in mind, there's two terms that are used in both of these frameworks to describe how event handlers propagate between objects : Bubbling and Tunneling. With Tunneling, event handlers start at the root XML element, and "tunnel" deeper into the "earth" (your control stack) until they get to the original source of the element, which is your controls. With Bubbling, event handlers start at the original source of the element (your control which initiated the event), and then "bubble up" to the root control. We use the earth analogy because the terminology goes hand in hand with gravity : tunnelling follows gravity, like digging into the earth, and bubbling goes against gravity, like a bubble coming up to the surface from the bottom of the ocean. I never really had a clear mental model of the Silverlight / WPF event models until right now.

With all that said, there are some differences between Silverlight and WPF that turn out to be very important when it comes to implementing your event handlers, and maintaining compatibility between the two. The biggest difference is that WPF supports both Bubbling and Tunneling events, whereas Silverlights supports only Bubbling events. Keep this in mind if you're designing desktop applications that you might want to port over to web applications at some point

Holy crap, a Windows ramdisk program that works

2010-09-30T08:40:00.000-07:00

In the past, I've had a bit of experience trying to install and run ramdisks in Windows XP, but I never found anything really good. Microsoft provided a ramdisk driver for Windows 2000 that one could get working in XP, but it wasn't really useful because it only provided a maximum of 32 MB of storage (Why ?!). I decided to revisit the issue of ramdisks at work today for Windows 7 because I wanted a way to speed up Visual Studio's caching and other operations (and Eclipse, but that's another matter). After searching around some more, I found a program called ImDisk. The installation is dead easy (if a little lacking in the notification department to tell you it's successfully installed). What's even better, you can make disks of arbitrary size, have it simulate various kinds of devices, and you can setup multiple ramdisks. The only catch is that you have to start the service in Administrator mode, and it's a bit more than trivial, though it is easy using the following steps :

1. Start -> All Programs -> Accessories -> Command Prompt -> Right-click -> Run as administrator
2. sc config imdisk start= auto (note the space between start= and auto, this got me the first time)
3. net start imdisk
4. Open up Control Panel -> ImDisk Virtual Disk Driver

... and have at it!

Editing your file system mappings for TFS paths

2010-09-17T08:41:00.001-07:00

So, as you may or may not know, when using Microsoft Team Foundation Server for version control, TFS maps remote project paths into local file system patmhs for checkout, etc. As I learned today, there are times when you check out the wrong path, and/or map it to the wrong path in the file system. If you ever need to modify or just nuke your file system mappings in TFS, here's how you go about doing it :


Team Explorer -> Source Control (double click) -> Workspaces (dropdown) -> Workspaces ...

Once you've selected your workspace in the dialog that comes up (you'll likely only have one anyway), click on :


Edit ... -> Working Folders

... and then select the folder to file system mappings that you want to remove, or create whatever new file system mappings you want right there.

Starting a new job ... and a new philosophy

2010-09-17T08:38:00.000-07:00

Ok, so I've left my previous employer and started at a new job. This means new domains of knowledge, new tools, and new people. My new employer is a Microsoft-exclusive shop, for almost every aspect of their software. If you've read this blog in any significant amount in the past, you'll know that I'm really not a Microsoft fan. In fact, I hate almost everything that's ever come out of Redmond, because for the most part it's deficient in how it's been engineered, and not as usable as other products out on the market (or even a lot of open source products). Therefore, the tone of this blog is probably going to change somewhat, and I'll be ranting and raving like a lunatic on things I'm learning about dealing with Microsoft products more often. It's going to be interesting.

Finally ... MySQL workbench sucks less

2010-09-02T16:13:00.001-07:00

MySQL workbench has been out for quite a while, under the guise of the people at MySQL. For the longest time, I stuck to using just the individual MySQL Query Browser and Administrator because they weren't too bad, and there really wasn't anything out there that I liked much better for query browsing alternatives. I tried out the old Workbench back when MySQL was standalone, but it really wasn't a very positive experience, so I just dropped it.

However, lately, something drove me to search for better alternatives to the MySQL query browser again, and I don't even know why. In my Google search, the MySQL Workbench came up, and I saw that it was a very recent version that was a good .2 versions up from the last one I had used, so I figured I'd give it a try. The difference was startling. Not only did they completely revamp the interface (at least for the Mac) but the workbench was just generally much more reliable and performant than the old query browser. If you get the chance, give it a shot. The new integrated interface is much more user friendly, and there's a bunch of new "Copy to clipboard" snippets that I personally find incredibly convenient and useful.

Running multiple Tomcat 6 instances in Ubuntu, the quick and dirty way

2010-08-16T17:57:00.000-07:00

Here's a quick list for running multiple Tomcat 6 instances on Ubuntu 10.04 :

Copy the /etc/init.d/tomcat script with a new name in the same directory

Update the NAME variable in the startup script with a name for the new instance that you want to run.

Copy /usr/share/$(old)NAME to the (new) NAME you've just created, along with /var/lib/$(old)NAME and /etc/default/$(old)NAME

Edit the server.xml file under /var/lib/$(new)NAME/conf/ and change all the ports (ie for shutdown, and all your Connectors) so that they don't conflict with the old instance

Run /etc/init.d/$(new tomcat script name) start

You should now have a running, fully functional instance of Tomcat on the server, using a different port.

Quickly dropping all the tables in a MySQL database without dropping the database itself

2010-08-04T18:01:00.000-07:00

I've recently come across a case where I need to drop all the tables in my database (ie effectively truncate the database) but MySQL has no built in command for doing so. This is where the magic of command lines becomes very useful. I found a great little trick here that will very quickly let you get rid of all that annoying data so you can load in new test data into your database :


mysqldump -u[USERNAME] -p[PASSWORD] --add-drop-table --no-data [DATABASE] | grep ^DROP | mysql -u[USERNAME] -p[PASSWORD] [DATABASE]

If you've got GnuWin32 or another set of GNU programs installed on your Windows box, you can even do this in Windows without even changing the syntax !

Quickly dumping a MySQL database out to a file

2010-08-03T18:54:00.001-07:00

I've found that sometimes, I just need a quick and dirty copy of a database to test changes against, and it doesn't matter if the data is recent, or even consistent for that matter. That's where mysqldump comes in handy with the --single-transaction option. It can be used on a live database because it doesn't lock the tables and prevent your web application from continuing to insert, modify and delete new records. One example would be :


mysqldump -u myusername -h myhost -p --single-transaction mydbname | gzip > mybackupfile.20100803.sql.gz &

This can be made even quicker by combining this dumping into an SSH transfer to copy the output data to another machine.

Piping a MySQL database from one server to another

2010-08-03T18:17:00.000-07:00

I know that there's a lot of great, wonderful things that can be done on-the-fly through SSH. It's one of the greatest tools out there for moving data or communicating between two machines. So I thought "Why not try to move my database in the fastest way possible via SSH?", and here's the command I found :


mysqldump -ux -px database | ssh me@newhost "mysql -ux -px database"

This is of course a very basic, stripped down version of the command, which I haven't tested yet, but it's a good start to what seems to be a very common problem among developers.

Converting Unix timestamps to something readable in Excel

2010-08-03T10:20:00.000-07:00

Our company uses log files in our machines to log raw data in real-time so that we have a history of what's happened on jobs that can be analyzed and, if necessary, used to prove to inspectors that our machines are doing what they say they are. We have our own log file parsing and analysis package for performing various analysis, but sometimes we need to look at the raw signals themselves in the logs, and this is where Excel comes in handy. The timestamps that are put on the signals are Unix timestamps, so seconds since the Epoch (Jan. 1, 1970 at 00:00:00 hours), which are not very useful in Excel, because Excel has its own Epoch (Jan. 1, 1900). To make things just a little bit more complex, we have machines in different timezones, which we need to account for as well. Thanks to a post at this site, I managed to come up with a modified version of their formula that will convert the unix timestamps to Excel format, and adjust them for timezone offsets :


=(ROUNDDOWN(A10116,0) / 86400) + 25569 - TIME(6,0,0)

Coding for Effect vs Coding a Model

2010-07-15T16:25:00.000-07:00

One of the software engineers I've worked with in the past has a very different way of doing things. He will take the most expedient way of implementing a particular piece of software in order to get the job done, always. Every time. (Admittedly, there is the odd time where this is the most effective means of doing software in order to get a job done, and potentially avoid getting fired, but that's for another discussion). The person in question takes no pride in his own work (by his own admission) and doesn't care about the quality of his work. Essentially, he codes with only the desired effect of his work in his mind. I've found that this way of doing things (while being faster) is a great way to introduce bugs and to cause problems later that arise as a lack of planning and forethought, ie lack of maintainability, inability to add features to the software.

Whenever I write my own software, I write it to model what's going on in a business or scientific process. (Really, isn't that what software's supposed to do ? ;P) I think about what's going on, and I try to model it in the software, taking into account all possible factors (or at least everything that reasonably occurs to me) that may influence the process. This typically results in a reliable system that very rarely breaks down. When a breakdown does occur, it's always been the result of something that I hadn't anticipated. I then go back to the software, and re-evaluate the model to see if there's something I need to change, and change it. This way of implementing software results in software that's easy to maintain, is (virtually) free of side effects, and can virtually eliminate undesired behaviour.

Working with CSV files in Excel the way you want ... rather than the way Microsoft tells you to by default

2010-07-05T15:33:00.001-07:00

In both my current job and my past jobs, I've worked with CSV files in Microsoft Excel as a matter of necessity. Excel is just too useful not to use (OpenOffice has its own quirks, but that's for another post.) The only problem with Excel is that it formats fields automatically when you open a CSV file by double clicking on it or by going through the File -> Open option (or Ribbon -> Open in Office 2007 and later). Sometimes this is nice, but most times it's a pain in the ass, especially when you want to be able to save that CSV data right back out to CSV again. What happens is Excel converts the values to particular data types after introspecting the data in the cells. This is annoying and stupid, especially when you have financial, scientific and engineering data that's in a format that doesn't fit will into Microsoft's algorithm for dealing with numbers, dates, times, and currencies.

There is a way around this giant annoyance. Instead of taking the easy way of opening the file, you can open Excel directly, with a fresh spreadsheet. Then, on the ribbon, go to the Data tab, and click on the 'From Text' button. This will let you open a delimited text file, and treat it as a data source, and Excel will give you options for opening the file and how you want to deal with its contents. This is much better for dealing with the data, especially when dealing with tab-delimited files.

Background on the home search page ? WTF Google ?

2010-06-10T08:25:00.000-07:00

Today I went to Google (as I do most days) to search for some stuff, but today when I got there, I was greeted with a horrible looking background on the search. My very first thought was "Did I just go to Bing by mistake ?", then I looked closer at the page and realized it was indeed Google. Needless to say, I was incredibly pissed. Why did Google change their signature search page ? It was great the way it was ! That was specifically why I went to Google. If I wanted to use a search engine that looked like a piece of shit, I'd use Bing. In my mind, this is nothing short of an epic fail on Google's part. And apparently, I'm not alone in this feeling.

My hatred of Microsoft is justified ... yet again

2010-05-05T14:44:00.000-07:00

There are two different schools of thought when it comes to being a vendor of very large software used by millions of people :

Move your software forward as often as possible, even if a few customers have to suffer lack of backward compatibility

Move your software forward as little as possible, even if a few customers have to suffer lack of innovation

I've noticed from my own personal experiences that Apple tends to take the former attitude, whereas Microsoft tends to take the latter attitude. In the end, someone (probably you) is going to get screwed over at some point, it's just a matter of how you want it to happen. As a consumer, I generally like Apple products, so on my personal technology front, I choose the former. As a software developer, I'm forced by business constraints to accept the latter. The specific circumstances that bring me to mention this :

Today I was working with version 4.0 of the .NET framework, the very latest (and supposedly greatest) from Microsoft, along with the very latest version of their Visual Studio (2010) software. I'm also using these with WPF (the Windows Presentation Foundation) to create an application for my employer. For various reasons, I had to go back and refactor some old code, part of which required opening log files for viewing within the application. In order to open a file in WPF, you must use the OpenFileDialog class that comes with the Windows Forms library, which has been around since Windows 2000 (if not earlier, I'm admittedly not as familiar with the lifecycle of this technology as I could be). If you're a developer on the Windows platform, or even just an observant user, you'll have noticed that on the OpenFileDialog there's a Places bar on the left hand side of the dialog that provides common places for storing files, some of which are very general and will require some drilling down into subfolders to find what you actually want. I wanted to be able to add a place to this bar so that users of the application (company employees who work as technicians out on job sites) could have a direct link to the typical directory used to store log files on their machine.

It turns out that adding a folder to the places bar is obscenely difficult, and must be done through registry hacks. My question is this (directed squarely at the developers at Microsoft responsible for writing the GUI controls, specifically these dialogs) : what the hell was going through your head that would make you think that the way you've implemented these dialogs is a good thing ? It's obscenely hard on developers to customize the generalized tools that you've given them, and this in turn makes it hard on users of that software to use the software made by developers bound by these stupid and seemingly arbitrary constraints. I find myself growing increasingly frustrated by stupid decisions of the Microsoft developers that seem obviously poor when you look at them from the perspective of a framework user. These poor decisions are costing me a lot in terms of time I have to spent developing around these decisions and researching alternatives. It's now no wonder to me how other companies can make absurd sums of money off of controls designed as work arounds for the short-sightedness of Microsoft's developers, and frankly, it's quite depressing.

Fixing telnet disconnects

2010-05-04T11:20:00.000-07:00

I've recently been working with a lot of embedded systems via telnet, and I'll frequently forget to disconnect my session before physically disconnecting the machine from my local LAN. Unfortunately, this results in telnet hanging the session, and me being unable to do anything else. Until now, I've just closed the window and opened another, but that became a pain in the ass, especially when I had to discard my (very useful) terminal history. Apparently, you can rescue a hung telnet session and avoid having to kill your terminal session by pressing Ctrl+5. This sends a signal to telnet to end the current session and return you to a prompt. Incredibly useful. Looks like there's a ton of other stuff where I found this useful tidbit.

Resolving slow SSH login times

2010-04-21T11:56:00.000-07:00

At work, like most other organizations, we have a Linux server that we access via SSH. However, lately, I've found my use of it skyrocketing in order to test software, and log-ins and file copying have been very slow. In my efforts to find out why, I found out that sshd has in its configuration file a setting called 'UseDNS'. The default is 'yes', so even if this setting is commented out, it will try to perform reverse DNS lookups on the IP addresses of users logging in. This would be a bad thing for us, especially since we have this box locked down and unable to contact any DNS servers. After disabling DNS lookups, my login and file copy time dropped to near 0. I hope this helps somebody.

Two bash functions that will make your software development life 10% easier

2010-04-20T11:50:00.000-07:00

... if you use SSH to move between machines a lot. Which I do. I have several machines that I work on regularly, and a centralized development machine shared with others that I frequently have to exchange files with. The following two functions are the most useful two bash functions that you can put in your .bashrc file if you use SSH as much as I do :

function ssh-create-keys {
ssh-keygen -t rsa
}

function ssh-setup-on {
cat ~/.ssh/id_rsa.pub | ssh $1 "cat - >>.ssh/authorized_keys"
}

The first sets up an SSH key file for you in your home directory, This key file will be used to identify you on other machines, and can be used for various purposes on your local machine as well. The second function logs you into the machine determined by the username@anothermachine given as the first argument to the function, and adds your public key to the list of authorized keys for that machine.

Once you've run the first function, and run the second function to setup your public key on an account on another machine, you'll now be able to use SSH to log in and copy files freely with the account on the other machine without having to enter a password. While this is incredibly convenience, it also comes with a caveat : it's dangerous. If somebody other than yourself gains physical access to your machine and can log on as you, (or use your already logged on account), they can move to those same machines freely and perform possibly malicious actions, as you. Keep that in mind.

Copying files in linux

2010-04-17T13:38:00.000-07:00

Lately I've been doing a lot of embedded development with Linux, and copying files between systems has been a bit of a pain. Fortunately, a combination of RSync and SSH solved my problems, with a command that lets me copy files from a directory on one system to a directory on another system, recursively with symlink preservation (and even duplication !) :

rsync -azuv -e ssh  user@systemaddress:~/path/to/dir/* .

WTF ?! Windows (as of Vista) no longer supports multiple different monitors!? WHY ????

2010-04-04T17:10:00.000-07:00

Apparently, as of Windows Vista and later, Windows no longer supports multiple video cards that don't use the same driver!! Why ? I use a Radeon and a Matrox QID to drive 6 displays on my development box, and this just entirely fucks me over. At best, I can only use four of my displays now, and that's only going to be when Matrox gets off their lazy ass and releases a Windows 7 driver for their QID LP PCIe video cards. I'm so disappointed with what's a complete step backward for Microsoft. Epic fail, Microsoft.

Seeing where Perl looks for its modules on a system

2010-01-23T10:07:00.001-08:00

Because we have very limited space on some embedded devices that we use that run Perl, we can only store a very few modules on these systems. This is a consequence of the fact that we run Busybox on these things, so by definition everything on these boxes is limited, if present at all. Therefore, we have to check and see if a module is available before we can use it in our code. Fortunately, there's a quick one-liner to see where Perl is looking for modules on a system :

perl -e'print join "\n", @INC'

I got this off a forum post from somewhere, and I'd post it here if I could find the link again. My apologies to the author of that forum post if they ever happen to run across this blog.

Embarking on a Perl journey

2010-01-23T09:07:00.000-08:00

A long time ago, in a job far, far away, I had to deal with some Perl. I learned just enough to get me by for the duration of the task at hand, and then pretty much forgot everything I had learned. Now, at my latest job, I'm having to deal extensively with legacy systems which have a considerable amount of logic written in Perl that needs to be either ported over to other languages (for various reasons) or updated and new things written because Perl is the only language that's both abstract enough and not too processor intensive to run on the embedded systems we deal with. Therefore, you're going to start seeing a lot more Perl posts on this blog.

Setting up Tomcat (5.5) on Ubuntu Server 8.10

2010-01-21T21:53:00.000-08:00

I recently ran into some old quirks when provisioning a new server for our company's web applications on Ubuntu 8.10 (Intrepid Ibex). Because the manager apps are no longer installed by default, you need to add extra packages to the list to install when installing Tomcat :

sudo apt-get install -y tomcat5.5 tomcat5.5-admin tomcat5.5-webapps

If you're copying configuration over from a previous Tomcat / Ubuntu installation, you need to make sure the permissions on all the files you copy are set correctly. In most cases, you'll have to run :

chown -R tomcat55:adm [file and folder list here]

If you're securing the applications with a certificate, try to make sure it's valid for your location and ensure that you've set it properly in your server.xml configuration file. If you want useful logging, you'll also have to place a log4j.properties file in

$CATALINA_HOME/common/classes

Hope this helps.

Dumping just your schema with MySQL dump

2010-01-21T21:39:00.000-08:00

A simple one-liner :

mysqldump -u root -p mydatabasename --no-data=true --add-drop-table=false > test_dump.sql

With this command, you'll be prompted for your root password. I got this from here. Simple

The Curious Case of Damned DataIntegrityViolationException

2010-01-12T19:35:00.001-08:00

In one of the projects on which I contract, we recently started encountering a problem importing and parsing text record files into our system which previously had no problem. My first thought on hearing this was that the partner from which we obtain the files had changed the file format (again). Upon closer inspection, nothing had changed in the files. My next step was to try importing them into a development system and seeing what was going on. As it turned out, the application was catching Spring's DataIntegrityViolationException. I was floored as soon as I saw this because our application was supposed to be catching this exception behind one of the business interfaces and converting it to an internal exception which is used in business logic. After some more poking around to confirm what was really going on, I threw the problem into google, and on the second result was a post in the Spring forum made by a user having exactly the same problem I was.

To summarize their problem quickly, they were using a transaction manager, and they were intercepting their business methods (via interfaces) with Aspects, which we're also doing. The problem was this : as soon as the internal Aspects were applied to the business interface, this changed the ordering of advice applied to the interface implementor, so now the Hibernate session underneath was getting flushed later by the transaction manager, instead of in the business method where it had been flushed previously. The result of this was that now DataIntegrityViolationExceptions were being thrown outside of the interecepted method, instead of inside where it was expected. A manual session.flush() inside of a HibernateCallback within the business method fixed this :


 /**
  * @see AchPaymentNoticeOfChangeService#registerNoc(AchPaymentNoticeOfChange)
  */
 @Override
 public void registerNoc(final AchPaymentNoticeOfChange changeNotification) throws IllegalArgumentException, NoticeOfChangeAlreadyExistsException, Exception {
  try {
   getHibernateTemplate().execute(new HibernateCallback() {
    @Override
    public Object doInHibernate(Session session) throws HibernateException, SQLException {
     
     session.save (changeNotification);
     
     // flush the session to ensure that the database gets synchronized
     // the end of this call, rather than waiting for any transaction
     // managers to handle it and risk letting a DataIntegrityViolationException
     // occur outside of this method's handling
     session.flush();
     
     return null;
    }
   });
  } catch (DataIntegrityViolationException dive) {
   throw new NoticeOfChangeAlreadyExistsException("A notice of change already exists for payment ["+changeNotification.getAchPayment().getId()+"]", dive, changeNotification);
  } catch (Exception e) {
   throw e;
  }
 }

I hope this post finds somebody else who runs into this problem.

Composing XML in C# 2.0 or later

2009-12-30T08:06:00.000-08:00

When testing some custom XML serialization I was writing for a project, I ran across a quirk of the XmlWriter created by XmlWriter#Create() : The writer that gets produced by this method doesn't actually write out XML to whatever stream or file you've given it in the Create method until Close() is called on the generated writer. This means that if you've been using the XmlWriter along with a using() statement, you're fine, but if you haven't been using it, as I have in the unit tests I've been trying to write, you're going to get some unexpected results. Something to pay attention to.

Visual Studio (2010?) Debugging Gotchas - Part I

2009-12-22T13:29:00.000-08:00

I recently ran across a problem where I was trying to debug some code and the Visual Studio debugger wouldn't stop in my class. It would stop in the test class from which I was debugging, but not the class I really wanted to debug. I looked on the breakpoint and noticed that it was transparent, so I hovered over it to view the tooltip, and the tooltip informed me that my class had a DebuggerStepThroughAttribute assigned to it. My first thought was "what ? that's bullshit", and then it occurred to me that I hadn't looked at the rest of the (partial) class which was defined in another file that was generated from an XML schema by xsd.exe. Sure enough, xsd.exe places DebuggerStepThroughAttribute attributes on the classes it generates. Why is this default behaviour ? This was an annoying as hell bug that took time out of my day that's better devoted to other things, like being productive so that I don't get shit from my boss. It also took more time than I'd care to admit to, largely due to my inexperience with Visual Studio and .NET. I've had yet to see anything like this coming from a Java / scripting background.

Tricky styling issues in WPF

2009-12-18T08:14:00.000-08:00

For about the last 8 or 9 hours (spread across two days) I've been troubleshooting a styling issue in WPF. I have a list of items that I'm displaying in a ListBox that have an 'enabled' property, which is an enum with values OFF and ON. Understand that this type was generated from an XML schema which is very tightly controlled, hence why the type is not a boolean (as would make sense with a name like 'enabled'). I've been trying to create a DataTemplate for the ListBoxItems so that they'll each have an icon associated with them depending on whether they're enabled or not. My original list looked like this :

<ListBox x:Name="ProcedureList" ItemsSource="{Binding Path=procedure}" HorizontalAlignment="Stretch" VerticalAlignment="Stretch">
  <ListBox.Resources>
    <Style TargetType="{x:Type ListBoxItem}">
      <EventSetter Event="MouseDoubleClick" Handler="WeldProcedure_MouseDoubleClick" />
      <Setter Property="HorizontalContentAlignment" Value="Stretch" />
    </Style>
    <Style TargetType="{x:Type Image}">
      <Style.Triggers>
        <DataTrigger Binding="{Binding}" Value="OFF">
          <Setter Property="Image.Source" Value="Resources/Error_16x16_72.png"/>
        </DataTrigger>
        <DataTrigger Binding="{Binding}" Value="ON">
          <Setter Property="Image.Source" Value="Resources/Success_16x16_72.png"/>
        </DataTrigger>
      </Style.Triggers>
    </Style>
  </ListBox.Resources>
  <ListBox.ItemTemplate>
    <DataTemplate DataType="{x:Type model:WeldingProcedure}">
      <DockPanel HorizontalAlignment="Stretch" LastChildFill="True">
        <Image DockPanel.Dock="Right" Height="16" Width="16" DataContext="{Binding Path=enabled}" ToolTip="Disabled"/>
        <TextBlock DockPanel.Dock="Left" VerticalAlignment="Center" TextAlignment="Left" Text="{Binding Path=procedureName}"/>
      </DockPanel>
    </DataTemplate>
  </ListBox.ItemTemplate>
</ListBox>

However, the images in the DataTemplate were not getting styled with the images as they should have been. As it turns out, if you're referencing default styles in Templates as above, the style resolution doesn't go outside of the template, so the default styling for all Image elements as above needs to go *inside* the DataTemplate resources :

<ListBox x:Name="ProcedureList" ItemsSource="{Binding Path=procedure}" HorizontalAlignment="Stretch" VerticalAlignment="Stretch">
  <ListBox.Resources>
    <Style TargetType="{x:Type ListBoxItem}">
      <EventSetter Event="MouseDoubleClick" Handler="WeldProcedure_MouseDoubleClick" />
      <Setter Property="HorizontalContentAlignment" Value="Stretch" />
    </Style>
  </ListBox.Resources>
  <ListBox.ItemTemplate>
    <DataTemplate DataType="{x:Type model:WeldingProcedure}">
      <DataTemplate.Resources>
        <Style TargetType="{x:Type Image}">
          <Style.Triggers>
            <DataTrigger Binding="{Binding}" Value="OFF">
              <Setter Property="Image.Source" Value="Resources/Error_16x16_72.png"/>
            </DataTrigger>
            <DataTrigger Binding="{Binding}" Value="ON">
              <Setter Property="Image.Source" Value="Resources/Success_16x16_72.png"/>
            </DataTrigger>
          </Style.Triggers>
        </Style>
      </DataTemplate.Resources>
      <DockPanel HorizontalAlignment="Stretch" LastChildFill="True">
        <Image DockPanel.Dock="Right" Height="16" Width="16" DataContext="{Binding Path=enabled}" ToolTip="Disabled"/>
        <TextBlock DockPanel.Dock="Left" VerticalAlignment="Center" TextAlignment="Left" Text="{Binding Path=procedureName}"/>
      </DockPanel>
    </DataTemplate>
  </ListBox.ItemTemplate>
</ListBox>

Generating XML objects from a schema in .NET

2009-12-11T13:15:00.000-08:00

In my last job, I frequently made use of the JAXB library provided with the Java SDK for object de/serialization. I chose to generate my schemata from objects back then because I started off with the business objects and I knew how I wanted to serialize them to XML. Now I have the reverse situation, and I'm using a new language. As it turns out, generating classes from XML schema is dead easy : use the xsd.exe tool. It comes with .NET.

Useful GIMP tricks - Batch file conversion

2009-12-10T10:17:00.000-08:00

Now that I've started having to do a lot more documenting for my job, I've found myself importing a lot of pictures taken as steps in tutorials and for providing figures. Since the camera gives us a nice, high-res version that's not really suitable for documentation, I have to scale the images down so that they're in a nice, readable size. When your tutorial has 30 steps, and an image (or even two or three) is required for *each step*, this can mean a lot of manual clicking, dragging, opening and saving in an image editor. Fortunately, GIMP has a plugin called Davids Batch Processor that allows you to easily do fairly simple transforms to file and output them in the format (that's most likely) of your choice. For the simple tasks of rotating, resizing (scaling) and exporting the images in a new format, this plugin's great. There's a tutorial on the site and the plugin itself has already proven to be exceedingly useful for me. I hope it will be for you too.

Subtleties of Perl - Reading files

2009-12-03T13:05:00.001-08:00

I've recently begun a new job, and with it has come a whole new segment of the software development universe. The new job uses a lot of Perl, C, Bash and various other languages to get stuff done. Today, I ran afoul of a Perl idiosyncrasy that's worth making a note of, because I'm sure I'll stumble across this problem again and I'm going to need to refer to this in the future. I should also note that I'm writing this as I'm waiting for a significantly large file to parse.

We have large log files that we parse on a daily basis to extract summary information from them about mechanical systems. We read the files, and then output a summary on a secondly basis, one line at a time. Recently, I ran afoul of Perl's file reading mechanisms. When reading files in Perl, there's any number of ways to do so, and it turns out that for the longest time, we've been using the wrong one. Previously, we had been using :


foreach my $line_of_log (<LOG>)
{
    // DO STUFF WITH $line_of_log
}

We thought that this was reading the file in one line of the log file at a time, processing it, and then moving on. What it was actually doing was reading (or "slurping") the whole file into memory, and giving us an array of strings, which we processed one line at a time. After 10 minutes of cursory Googling, I ran across a tutorial which presented this :


while (<LOG>)
{
    my $line_of_log = $_;
    // DO STUFF WITH $line_of_log
}

The 'while' version of the file read actually does what we thought we where doing all along: reading one line from the file, and then doing stuff with it. The difference between the two methods is that in the 'foreach' version, the entire input file gets read into memory, whereas in the 'while' version, only a single line gets read into memory at any given time. As it turns out, another difference is that reading in a 7 MB file resulted in Perl grabbing 34 MB of memory with the 'foreach' version, but only 2.2 MB with the 'while' version. That's an ENTIRE ORDER OF MAGNITUDE in difference!. This also makes a huge difference when running Perl on memory-limited systems, as we are.

Tweaking visual studio

2009-11-10T11:08:00.001-08:00

I've recently begun using Visual Studio 2010 Beta 2 more and more as I have increasing amounts of work that require .NET. In order to keep things nicely tabbed the both I and my boss like it, there's a setting that can be set according to instructions founds here : http://mhinze.com/tabs-whitespace-visual-studio/

Using the MySQL EXPLAIN statement

2009-09-29T06:58:00.000-07:00

Recently I had been having trouble with queries on a certain table in a system that I've been maintaining. I had gone through just about every excuse for why queries on the table could be performing so slowly : the machine was slow (DB running on a VM), the webserver was slow (also running on a VM), I wasn't using the native libraries (webserver was Tomcat), I had other processes running in the background (I didn't). Then I ran across a tip on a forum suggesting usage of the MySQL EXPLAIN statement. I had known all about it for the longest time, but it never occurred to me to actually use it (I think I'm that good at writing queries, turns out : I'm wrong). After using the EXPLAIN statement, I found out that the query processor was using a suboptimal query plan which utilized an index I had added with the intent of improving performance (the index had a fairly high arity, so choosing to use it was sketchy at best in the first place). Most DBMSs should have a similar functionality built in. I think that'll be the first place I go in future.

Ubuntu 9.04 servers slow to login

2009-09-18T12:16:00.000-07:00

Recently our production system's servers have been getting progressively slower, and it has finally gotten to the point where it's merited my full attention to remedy the situation. In my research on the problem, one of the things I came across was how incredibly slow the logins were (among other problems). After watching top during numerous logins, I saw that console-kit-daemon was going horribly slow and shooting the sshd CPU usage through the roof. After some time googling for issues related to console-kit-daemon, I found that many people were getting errors in their /var/log/daemon.log file regarding console-kit-daemon being unable to initialize policykit. After installing policykit, my logins to my production servers are now lightning fast

Keeping your servers up to date

2009-07-14T01:31:00.000-07:00

As I've previously mentioned on this blog, our company uses Ubuntu for our servers. I won't re-iterate the reasons why in this post, you can search this blog using an Ubuntu tag if you're interested in them. Our servers have uptimes of months and months, and as a result, the clocks on the machines tend to drift over time, which has inspired me to start using NTP to keep the clocks synchronized. A quick Google search yielded the desired results. In order to manually synchronize with NTP, you can run the following commands :

sudo /etc/network/if-up.d/ntpdate

sudo ntpdate pool.ntp.org

Obviously, entering this command every day or every week gets tiresome and stupid, so you can get cron to schedule this daily for you by running the following commands :

echo "sudo ntpdate ntp.ubuntu.com" >> /etc/cron.daily/ntpdate
chmod 755 /etc/cron.daily/ntpdate

Tomcat keystore : too many open files - Continued

2009-07-14T01:24:00.000-07:00

Further to my last post, it seems there was another, larger underlaying problem that was causing the exceptionally high number of connections to our servers. One of our clients "required" (I use that term loosely because they really didn't the information) extra information for transactions that was not included in the optimized change metadata that they had been instructed to query for in order to update their transactions. So instead of just querying for the update metadata, they would do that, and then they'd query for each individual transaction. Assume we page our metadata at 100 transactions / page. For example, if we were to have a batch of 800 transactions submitted by the client, instead of making 800 / 100 = 8 calls to our server to update the transactions in the batch, they'd have 8 + (800 * 1) = 808 calls to update their system. They were effectively launching a Denial of Service attack on our servers each time they wanted to update a batch of transactions. Needless to say I consulted with them on the issue and updated our change metadata to include the information they "need" (which they've already got in their system), and they've updated their system to take the number of requests down to the proper level to update their transactions. So let this be a lesson to anybody reading this blog post that has to develop systems that deal with external clients : ensure your clients fully understand the purpose and intent of all the features of our system before they start developing for it and using it.

Too many open sockets error in Tomcat 5.5

2009-07-08T01:16:00.000-07:00

Our company recently started a new project. Our primary client is the first one to use it, and use they have. I've suspected they've been putting an unusually high load on our servers, and tonight it was confirmed when our server stopped handling requests, refusing them and then generating "Too many open files" errors in the Tomcat log files referring to my Tomcat SSL keystore. After doing some brief research on the error, I've discovered that this can happen when the Tomcat element in server.xml is configured with too few 'maxThreads' and a too low an 'acceptCount'. I've since tripled the number of threads and acceptable connections. Hopefully this will resolve things.

XML namespace error with Spring WS

2009-06-28T00:53:00.000-07:00

Today I ran into a strange error trying to access a webservice via Spring WS (Spring Web Services). The exception looked like the following :

org.springframework.ws.soap.saaj.SaajSoapEnvelopeException: Could not access envelope: Unable to create envelope from given source: ; nested exception is com.sun.xml.messaging.saaj.SOAPExceptionImpl: Unable to create envelope from given source: 
Caused by: 
com.sun.xml.messaging.saaj.SOAPExceptionImpl: Unable to create envelope from given source: 
 at com.sun.xml.messaging.saaj.soap.EnvelopeFactory.createEnvelope(EnvelopeFactory.java:95)
 at com.sun.xml.messaging.saaj.soap.ver1_1.SOAPPart1_1Impl.createEnvelopeFromSource(SOAPPart1_1Impl.java:51)
 at com.sun.xml.messaging.saaj.soap.SOAPPartImpl.getEnvelope(SOAPPartImpl.java:106)
 at org.springframework.ws.soap.saaj.Saaj13Implementation.getEnvelope(Saaj13Implementation.java:145)
 at org.springframework.ws.soap.saaj.SaajSoapMessage.getEnvelope(SaajSoapMessage.java:84)
 at org.springframework.ws.soap.AbstractSoapMessage.getSoapHeader(AbstractSoapMessage.java:42)
 at org.springframework.ws.soap.server.SoapMessageDispatcher.handleRequest(SoapMessageDispatcher.java:91)
 at org.springframework.ws.server.MessageDispatcher.dispatch(MessageDispatcher.java:189)
 at org.springframework.ws.server.MessageDispatcher.receive(MessageDispatcher.java:166)
 at org.springframework.ws.transport.support.WebServiceMessageReceiverObjectSupport.handle(WebServiceMessageReceiverObjectSupport.java:78)
 at org.springframework.ws.transport.http.WebServiceMessageReceiverHandlerAdapter.handle(WebServiceMessageReceiverHandlerAdapter.java:60)
 at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:819)
 at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:754)
 at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:399)
 at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:364)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:709)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:802)
 at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:252)
 at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:173)
 at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:213)
 at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:178)
 at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:126)
 at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:105)
 at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:107)
 at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:148)
 at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
 at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
 at org.apache.tomcat.util.net.PoolTcpEndpoint.processSocket(PoolTcpEndpoint.java:527)
 at org.apache.tomcat.util.net.LeaderFollowerWorkerThread.runIt(LeaderFollowerWorkerThread.java:80)
 at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:684)
 at java.lang.Thread.run(Unknown Source)
Caused by: javax.xml.transform.TransformerException: org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces.
 at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(Unknown Source)
 at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transform(Unknown Source)
 at com.sun.xml.messaging.saaj.util.transform.EfficientStreamingTransformer.transform(EfficientStreamingTransformer.java:371)
 at com.sun.xml.messaging.saaj.soap.EnvelopeFactory.createEnvelope(EnvelopeFactory.java:83)
 ... 30 more
Caused by: org.w3c.dom.DOMException: NAMESPACE_ERR: An attempt is made to create or change an object in a way which is incorrect with regard to namespaces.
 at com.sun.org.apache.xerces.internal.dom.AttrNSImpl.setName(Unknown Source)
 at com.sun.org.apache.xerces.internal.dom.AttrNSImpl.(Unknown Source)
 at com.sun.org.apache.xerces.internal.dom.CoreDocumentImpl.createAttributeNS(Unknown Source)
 at com.sun.xml.messaging.saaj.soap.SOAPDocumentImpl.createAttributeNS(SOAPDocumentImpl.java:142)
 at com.sun.org.apache.xerces.internal.dom.ElementImpl.setAttributeNS(Unknown Source)
 at com.sun.xml.messaging.saaj.soap.impl.ElementImpl.setAttributeNS(ElementImpl.java:1190)
 at com.sun.org.apache.xalan.internal.xsltc.trax.SAX2DOM.startElement(Unknown Source)
 at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.closeStartTag(Unknown Source)
 at com.sun.org.apache.xml.internal.serializer.ToSAXHandler.flushPending(Unknown Source)
 at com.sun.org.apache.xml.internal.serializer.ToXMLSAXHandler.startElement(Unknown Source)
 at org.xml.sax.helpers.XMLFilterImpl.startElement(Unknown Source)
 at com.sun.xml.messaging.saaj.util.RejectDoctypeSaxFilter.startElement(RejectDoctypeSaxFilter.java:157)
 at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(Unknown Source)
 at com.sun.org.apache.xerces.internal.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown Source)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)
 at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
 at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(Unknown Source)
 at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(Unknown Source)
 at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(Unknown Source)
 at org.xml.sax.helpers.XMLFilterImpl.parse(Unknown Source)
 at com.sun.org.apache.xalan.internal.xsltc.trax.TransformerImpl.transformIdentity(Unknown Source)
 ... 34 more

As it turns out, the Java 6 JDK comes with its own integrated versions of Apache Xerces and Apache Xalan. The problem is that the integrated versions are old. Updating your project's POM to ensure versions of Apache Xerces >= 2.8.1 and Apache Xalan >= 2.7.0 are in your classpath will remedy the problem.

Some helpful troubleshooting tips for SFTP jails

2009-05-20T15:45:00.001-07:00

Further to my previous post on SFTP jails, I've run across a few issues. When logging in via command line after first setting up the jail, you may encounter immediate disconnects with the response "Connection Closed". One possible cause of this is missing files, specifically "/lib64/ld-linux-x86-64.so.2" under the jail. There are other files that can possibly be missing, but that's a biggie if you're running a 64-bit kernel. If you're running WinSCP, you may also encounter the error message "Is the host running a SFTP server?"

A quick note on intercepting annotations with Spring AOP

2009-05-11T15:29:00.000-07:00

Spring AOP is incredibly powerful. One of the most powerful features of the framework is that it lets you intercept methods that are decorated with particular annotations using the following AOP expression :

@annotation(my.package.MyAnnotation)

This can be used in place of :

execution(public * my.package.MyService.serviceMethod(*))

...or in combination with it. But one important point : the annotation being intercepted must be in service method implementations (ie concrete class implementations), not on the service method interface declarations! Otherwise Spring AOP will not be able to read and intercept the annotated methods / classes

Setting up Microsoft Outlook 2007 with Microsoft Exchange

2009-04-17T14:31:00.000-07:00

...via RPC over HTTPS :

1. Ensure you have your root certificate in your trusted authority store in the Microsoft Management Console (mmc.exe)
2. Ensure you have your personal certificate in your personal key store in the Microsoft Management Console (mmc.exe)
3. Ensure you've got the correct server settings

Setting up a VPN connection to an Astaro security applicance on Mac OS X

2009-04-13T16:45:00.000-07:00

So I finally got a working VPN configuration to be able to connect my shiny new MacBook Pro to our VPN on our production system which uses an Astaro firewall appliance. Turns out it's fairly easy too :

Go to your Astaro firewall's user profile page, ie login as a user, and download the key / OpenVPN configuration package for your router. The great thing about these appliances is that they provide the package for you right from the device

Download Tunnelblick. It's an (apparently very easy to use) GUI front end for OpenVPN on Mac OS X. The latest version (as of the time of this writing) is 3.0 beta 10. Tunnelblick is hosted on Google Code. The installation's even easier. Load up the disk image, and drag the sole application icon into your applications folder.

Open the VPN package you downloaded from your Astaro firewall appliance, and copy everything in it to the ~/Library/openvpn folder created by the Tunnelblick installation

Tunnelblick should automatically detect the additional files and provide a listing for the Astaro configuration in its connection list. Once you've got the item in the list, just click on it and it should automagically connect to the Astaro firewall appliance and your VPN connection should be working.

Oh, and make sure you've actually got Tunnelblick started, or else you won't be able to connect ;)

Playing with my new Mac

2009-04-07T14:09:00.000-07:00

I recently managed to convince my employer to provide me with a MacBook Pro so that I could start developing in-house iPhone applications for the company. I'm not yet at the point where I'm writing applications, but I am installing tons of stuff on the laptop so that I can use it as my mobile computer for when I'm on the road for work. It certainly has its quirks that I have to get used to over running everything in Windows. This will be a very brief post, but suffice it to say that I've learned a few things :

1) When it comes to Maven, ensure that JAVA_HOME is set in your environment variables.
2) Make sure you're choosing the right JVM. MacOSX comes with a bunch of default JVMs installed, and you can configure them through the Java control panel in Applications -> Utilities

More Hibernate glitches, part 2

2009-03-25T08:26:00.000-07:00

I had previously written on an Expected positional parameter glitch in Hibernate when joining between entities using OneToOne and PrimaryKey + ForeignKey columns. It turns out, Hibernate has this problem no matter what field you query on the joined entity if you're quering by another entity. A way around this is to use the ID property of the joined entities and add another association to your HQL / Criteria query.

ie So if you have Person and Address having a OneToOne relationship


public class Person {
  @OneToOne
  private Address address;
}

... and you want to query for a Person by Address, you would probably write something like :

from Person p where p.address = ?

... in HQL. Well, this is where you start running into the Expected positional parameter glitch. Instead, write your query like this :

from Person p where p.address.id = ?

... and pass in a long / int / whatever type you use for an identifier for Person / Address instead and this will work around the glitch until it can be fixed.

Stupid MySQL tricks

2009-03-15T02:50:00.000-07:00

Recently, I found that our secondary server's MySQL partition was starting to get full. After running du -h --max-depth=1 several times recursively, I found that the source was the MySQL binary logs. As it turns out, MySQL comes with a very handy command for purging all log files. If you run :

show master logs

, you'll be given a list of the logs that the MySQL server is currently using. You can clear out old log files (some of which can be very large) by running the command

purge master logs to 'log-name-here -from-display'

. This will remove the old log files from the hard disk and save up some potentially much needed space.

Spring Webflow 2 Quirks

2009-03-13T02:50:00.001-07:00

As I recently found out after a marathon debugging session, if you have a <transition> element that executes actions, and any beans involved in an expression evaluation don't exist in your application context, SWF2 will spin it's wheels instead of properly resolving things, leaving your flow in the lurch. At the time of this writing, I'm using Spring Webflow 2.

More Hibernate glitches

2009-03-12T22:39:00.001-07:00

I've recently discovered the joy of actually getting OneToOne primary key columns on objects working, so I've started using them more often (where appropriate). Naturally, increased use means increased chances of finding a bug, which I have. It seems there's a parser problem in HQL when it comes to querying on ID columns that are also objects. If you try to query on the object type (ie Person) rather than the ID type (ie Long), you'll get an exception saying something along the lines of :


Expected positional parameter count: 1, actual parameters: [Parent@bec357b] [from Child this where this.id.parent = ?]

The above line is from HHH-2254, but it's representative of the problem I ran across. The problem is that hibernate doesn't properly parse and fill in the parameters in the query. You can get around this by changing the query to use a simple type for ID instead of the object type.

Stupid Hibernate Tricks

2009-03-11T11:27:00.000-07:00

I've recently been finding myself looking for a way to get Hibernate to override the default type for strings (varchar) and use char instead in order to boost performance of the database for fields that I know are going to have a fixed length. During my search, I came upon a couple of solutions.

Using annotations: You can use the 'columnDefinition' attribute of the @Column annotation to specify the SQL type directly. This has the unfortunate side effective of possibly reducing the database portability of your application.
Using SQL Dialects: You can override one of Hibernate's built in SQL Dialects and make char() types the default for strings. One other interesting possibility this opens up is that it allows you to only override the definition for certain lengths, ie, you could make any strings less than or equal to 4 characters in length use a char() definition, and the rest could use varchar definitions. This is done using the 'registerColumnType' method of Dialect.

I love Dojo

2009-02-25T11:30:00.000-08:00

I have recently discovered the magic that is Dojo. I love it. The widgets that it provides are uber-useful and have really boosted the functionality of our site. And once I integrated it with Spring Webflow 2, using some custom tags I wrote, my development speed shot right up.

Dojo Home Page
Dojo Campus - Great tutorial site

Solved! Finally fixed classpath issues with Maven, Eclipse and JUnit

2009-02-04T12:27:00.000-08:00

I have several pieces of code in one of our company's projects that dynamically reads the classpath and scans for classes that have certain annotations and values in those annotations. For the longest time, I was able to unit test these successfully. Then one day, I started getting strange ClassNotFoundExceptions when trying to execute the tests, and I could never figure out why. The project still ran just fine on our servers, so I let the issue slide at the time because I was pressed for time and needed to get other shit done. Overtime, I continued to let the problem bounce up and down over rocks as I tried to unit test certain things and couldn't because of this problem. Today I finally got fed up and wound up investing 3 hours troubleshooting the problem, and here's what happened:

Right around the same time I started getting this problem, I had upgraded my version of the M2Eclipse plugin (which I use to integrate the Maven build system and Eclipse) to a new version. As it turns out, in the intervening versions, the Maven developers changed how Maven integrates with the Eclipse classpath, and if you got the plugin to update your project's Eclipse configuration, it would specify separated output folders for your projects code and your project's test classes, and that's what caused the problem. Separating the two meant I could no longer dynamically resolve my project's model classes in a test environment, because the classpath visitor wouldn't receive the correct source folder from the test environment's classloader. So, the solution as it turned out, was to have Eclipse's output go into the same folder for source and test classes.

Allowing external root access to your MySQL database

2009-01-06T17:21:00.000-08:00

...is recommended against in the MySQL documentation, but it can be oh so handy for running scripts and adding databases, tables and views when you can't (or don't want to) log into that remote system on the command line to update it. However, when you try it (at least on a fresh Ubuntu install), you may be greeted with an Error 2003. This happens when your system does not allow root external access. To enable external access, you can run this query (as root) on your server from the command line (must be localhost) :

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY ('myrootpassword');

* EDIT * : The above statement does not necessarily copy all privileges over to the new entry for 'root'@'%' in the mysql.user table. A better way to accomplish full external root access is the following :


DELETE FROM `mysql`.`user` WHERE User = 'root' AND Host = '%';
INSERT INTO `mysql`.`user` (Host, User, Password, Select_priv, Insert_priv, Update_priv, Delete_priv, Create_priv, Drop_priv, Reload_priv, Shutdown_priv, Process_priv, File_priv, Grant_priv, References_priv, Index_priv, Alter_priv, Show_db_priv, Super_priv, Create_tmp_table_priv, Lock_tables_priv, Execute_priv, Repl_slave_priv, Repl_client_priv, Create_view_priv, Show_view_priv, Create_routine_priv, Alter_routine_priv, Create_user_priv, ssl_type, ssl_cipher, x509_issuer, x509_subject, max_questions, max_updates, max_connections, max_user_connections) (SELECT '%', User, Password, Select_priv, Insert_priv, Update_priv, Delete_priv, Create_priv, Drop_priv, Reload_priv, Shutdown_priv, Process_priv, File_priv, Grant_priv, References_priv, Index_priv, Alter_priv, Show_db_priv, Super_priv, Create_tmp_table_priv, Lock_tables_priv, Execute_priv, Repl_slave_priv, Repl_client_priv, Create_view_priv, Show_view_priv, Create_routine_priv, Alter_routine_priv, Create_user_priv, ssl_type, ssl_cipher, x509_issuer, x509_subject, max_questions, max_updates, max_connections, max_user_connections FROM `mysql`.`user` WHERE User = 'root' AND Host = 'localhost');
FLUSH PRIVILEGES;

Setting up MySQL server on Ubuntu

2009-01-06T13:13:00.000-08:00

...is really easy. Run the command :

apt-get install -y mysql-server

Once that's done, you'll need to perform some additional configuration and setup the root password. My added configuration involves making table name comparisons all lower case (convenient for queries) and making the server run entirely in utf8. In order to configure the server as such, you'll need to make the following changes :

Add the following lines to the [mysqld] section of the my.cnf configuration :


default-character-set=utf8
default-collation=utf8_general_ci
lower_case_table_names=1

Add the following line to the [mysql] section of my.cnf (note that this is not the same as the [mysqld] section, the difference being the 'd' on the end) :


default-character-set=utf8

In order to have the Ubuntu MySQL installation properly run on external hosts, you'll have to comment out the line that looks like :


bind-address     127.0.0.1

After you've configured the my.cnf file, you'll have to stop the server, and restart it with --skip-grant-tables enabled so that you can have free, unfettered access to the system with privileges disabled. This is required so that you can set the initial root password. Run the following commands as root :


/etc/init.d/mysql stop
mysqld_safe --skip-grant-tables --skip-networking &

The latter command will start the MySQL server with privileges disabled.
Then log in with the command

mysql -u root -h localhost

Once logged in, run the command :


UPDATE `mysql`.`user` SET Password = PASSWORD('thepassword') WHERE User = 'root';

This will set the user password to 'thepassword' (though I recommend you don't use that exact password). Once you've set the root password, exit mysql and run the command


/etc/init.d/mysql restart

This will restart the server as usual, so that you'll no longer have unlimited privileges (and neither will anyone else).

Setting up an SSH Server on Ubuntu

2009-01-06T12:49:00.001-08:00

Ubuntu is usually pretty good about coming pre-configured with everything you need for your system, but one thing I've found that it's lacking (at least the server distribution is, anyway) is a pre-setup SSH server. Fortunately though, it's quite easy to set one up. All you need to do is issue the command:

apt-get install openssh-server

...as root. If you get an error to the effect of :

Package does not exist

...then you'll have to run :

apt-get update

...to update your system's packages first.

Configuring MySQL for table case-insensitivity

2008-12-30T17:01:00.000-08:00

While trying to add a recent update to our systems, I noticed that our demo system wouldn't start, which was very strange considering I had no problems on my local test system. I eventually traced the problem down to MySQL not being able to find certain tables. This was because I had created the tables in a different case than the Quartz library was expecting. My local testing box is a Windows box whereas our demo and production systems run Linux, the difference being that my Windows MySQL installation has case-insensitivity on by default, whereas the Linux installations do not. In order to remedy this, I had to explicitly add a line to my my.ini / my.cnf configurations :

lower_case_table_names=1

You can determine the case-sensitivity of your own MySQL installation by logging in with the command line client or running the query browser and entering the following command :

SHOW VARIABLES LIKE 'lower_case%'

Integrating Quartz and Spring using persistent jobs

2008-12-11T13:19:00.000-08:00

I've recently found myself having a lot of jobs running in the background of our Java applications, often requiring changes to the scheduling based on my boss' whims. Combine this with the fact that our system's API is now seeing heavy use, and having the triggers configured in the Spring application context (and thus requiring a restart in order to change them) just doesn't cut it any more. Thus, I now find myself needing persistent jobs. However, there's not a lot of documentation out there for dealing with Quartz' persistent job feature in Spring, so I'm going to hopefully provide some help. To start, you need to create the quartz tables in whatever database you're going to be using. There are scripts for creating these tables in the Quartz distribution available on their website for whatever database you may be using. For convenience, I've copied the MySQL Inno DB script here :


DROP TABLE IF EXISTS QRTZ_JOB_LISTENERS;
DROP TABLE IF EXISTS QRTZ_TRIGGER_LISTENERS;
DROP TABLE IF EXISTS QRTZ_FIRED_TRIGGERS;
DROP TABLE IF EXISTS QRTZ_PAUSED_TRIGGER_GRPS;
DROP TABLE IF EXISTS QRTZ_SCHEDULER_STATE;
DROP TABLE IF EXISTS QRTZ_LOCKS;
DROP TABLE IF EXISTS QRTZ_SIMPLE_TRIGGERS;
DROP TABLE IF EXISTS QRTZ_CRON_TRIGGERS;
DROP TABLE IF EXISTS QRTZ_BLOB_TRIGGERS;
DROP TABLE IF EXISTS QRTZ_TRIGGERS;
DROP TABLE IF EXISTS QRTZ_JOB_DETAILS;
DROP TABLE IF EXISTS QRTZ_CALENDARS;
CREATE TABLE QRTZ_JOB_DETAILS(
JOB_NAME VARCHAR(200) NOT NULL,
JOB_GROUP VARCHAR(200) NOT NULL,
DESCRIPTION VARCHAR(250) NULL,
JOB_CLASS_NAME VARCHAR(250) NOT NULL,
IS_DURABLE VARCHAR(1) NOT NULL,
IS_VOLATILE VARCHAR(1) NOT NULL,
IS_STATEFUL VARCHAR(1) NOT NULL,
REQUESTS_RECOVERY VARCHAR(1) NOT NULL,
JOB_DATA BLOB NULL,
PRIMARY KEY (JOB_NAME,JOB_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_JOB_LISTENERS (
JOB_NAME VARCHAR(200) NOT NULL,
JOB_GROUP VARCHAR(200) NOT NULL,
JOB_LISTENER VARCHAR(200) NOT NULL,
PRIMARY KEY (JOB_NAME,JOB_GROUP,JOB_LISTENER),
INDEX (JOB_NAME, JOB_GROUP),
FOREIGN KEY (JOB_NAME,JOB_GROUP)
REFERENCES QRTZ_JOB_DETAILS(JOB_NAME,JOB_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_TRIGGERS (
TRIGGER_NAME VARCHAR(200) NOT NULL,
TRIGGER_GROUP VARCHAR(200) NOT NULL,
JOB_NAME VARCHAR(200) NOT NULL,
JOB_GROUP VARCHAR(200) NOT NULL,
IS_VOLATILE VARCHAR(1) NOT NULL,
DESCRIPTION VARCHAR(250) NULL,
NEXT_FIRE_TIME BIGINT(13) NULL,
PREV_FIRE_TIME BIGINT(13) NULL,
PRIORITY INTEGER NULL,
TRIGGER_STATE VARCHAR(16) NOT NULL,
TRIGGER_TYPE VARCHAR(8) NOT NULL,
START_TIME BIGINT(13) NOT NULL,
END_TIME BIGINT(13) NULL,
CALENDAR_NAME VARCHAR(200) NULL,
MISFIRE_INSTR SMALLINT(2) NULL,
JOB_DATA BLOB NULL,
PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP),
INDEX (JOB_NAME, JOB_GROUP),
FOREIGN KEY (JOB_NAME,JOB_GROUP)
REFERENCES QRTZ_JOB_DETAILS(JOB_NAME,JOB_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_SIMPLE_TRIGGERS (
TRIGGER_NAME VARCHAR(200) NOT NULL,
TRIGGER_GROUP VARCHAR(200) NOT NULL,
REPEAT_COUNT BIGINT(7) NOT NULL,
REPEAT_INTERVAL BIGINT(12) NOT NULL,
TIMES_TRIGGERED BIGINT(7) NOT NULL,
PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP),
INDEX (TRIGGER_NAME, TRIGGER_GROUP),
FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_CRON_TRIGGERS (
TRIGGER_NAME VARCHAR(200) NOT NULL,
TRIGGER_GROUP VARCHAR(200) NOT NULL,
CRON_EXPRESSION VARCHAR(120) NOT NULL,
TIME_ZONE_ID VARCHAR(80),
PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP),
INDEX (TRIGGER_NAME, TRIGGER_GROUP),
FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_BLOB_TRIGGERS (
TRIGGER_NAME VARCHAR(200) NOT NULL,
TRIGGER_GROUP VARCHAR(200) NOT NULL,
BLOB_DATA BLOB NULL,
PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP),
INDEX (TRIGGER_NAME, TRIGGER_GROUP),
FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_TRIGGER_LISTENERS (
TRIGGER_NAME VARCHAR(200) NOT NULL,
TRIGGER_GROUP VARCHAR(200) NOT NULL,
TRIGGER_LISTENER VARCHAR(200) NOT NULL,
PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP,TRIGGER_LISTENER),
INDEX (TRIGGER_NAME, TRIGGER_GROUP),
FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP)
REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_CALENDARS (
CALENDAR_NAME VARCHAR(200) NOT NULL,
CALENDAR BLOB NOT NULL,
PRIMARY KEY (CALENDAR_NAME))
TYPE=InnoDB;

CREATE TABLE QRTZ_PAUSED_TRIGGER_GRPS (
TRIGGER_GROUP VARCHAR(200) NOT NULL,
PRIMARY KEY (TRIGGER_GROUP))
TYPE=InnoDB;

CREATE TABLE QRTZ_FIRED_TRIGGERS (
ENTRY_ID VARCHAR(95) NOT NULL,
TRIGGER_NAME VARCHAR(200) NOT NULL,
TRIGGER_GROUP VARCHAR(200) NOT NULL,
IS_VOLATILE VARCHAR(1) NOT NULL,
INSTANCE_NAME VARCHAR(200) NOT NULL,
FIRED_TIME BIGINT(13) NOT NULL,
PRIORITY INTEGER NOT NULL,
STATE VARCHAR(16) NOT NULL,
JOB_NAME VARCHAR(200) NULL,
JOB_GROUP VARCHAR(200) NULL,
IS_STATEFUL VARCHAR(1) NULL,
REQUESTS_RECOVERY VARCHAR(1) NULL,
PRIMARY KEY (ENTRY_ID))
TYPE=InnoDB;

CREATE TABLE QRTZ_SCHEDULER_STATE (
INSTANCE_NAME VARCHAR(200) NOT NULL,
LAST_CHECKIN_TIME BIGINT(13) NOT NULL,
CHECKIN_INTERVAL BIGINT(13) NOT NULL,
PRIMARY KEY (INSTANCE_NAME))
TYPE=InnoDB;

CREATE TABLE QRTZ_LOCKS (
LOCK_NAME VARCHAR(40) NOT NULL,
PRIMARY KEY (LOCK_NAME))
TYPE=InnoDB;

INSERT INTO QRTZ_LOCKS values('TRIGGER_ACCESS');
INSERT INTO QRTZ_LOCKS values('JOB_ACCESS');
INSERT INTO QRTZ_LOCKS values('CALENDAR_ACCESS');
INSERT INTO QRTZ_LOCKS values('STATE_ACCESS');
INSERT INTO QRTZ_LOCKS values('MISFIRE_ACCESS');
commit;

Once this is done, you'll need to integrate Quartz with Spring. If you're using Maven 2, getting Quartz into your project is as simple as adding the following snippet to the <dependencies> section of your pom.xml configuration file.


<dependency>
  <groupId>opensymphony</groupId>
  <artifactId>quartz</artifactId>
  <version>1.6.0</version>
  <scope>provided</scope>
</dependency>

Note that you're going to need Quartz version 1.6.0 to work with Spring 2.5.3 or greater, otherwise you can use Quartz 1.5.x. After you've got Quartz into your project, you'll need to define a Scheduler that accesses your database to schedule your jobs. Fortunately, Spring happens to come with a handy FactoryBean for defining Quartz schedulers. I've used the following :


<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
  <property name="jobFactory">
    <bean class="org.springframework.scheduling.quartz.SpringBeanJobFactory"/>
  </property>
  <property name="dataSource" ref="mainDataSource" />
  <property name="transactionManager" ref="mainDataSourceTransactionManager" />
  <property name="quartzProperties">
    <util:properties location="/WEB-INF/config/quartz.properties"/>
  </property>
  <property name="applicationContextSchedulerContextKey" value="applicationContext"/>
  <property name="waitForJobsToCompleteOnShutdown" value="true" />
</bean>

The above Spring beans configuration snippet requires some explanation, so let's go through it.

The property 'jobFactory' is a Spring implementation of the Quartz JobFactory interface. When the scheduler encounters a trigger in the database that's to be fired, it loads the corresponding JobDetails from the database, loads it into a JobDetail object, places that JobDetail object into a TriggerFiredBundle, and passes the TriggerFiredBundle to the JobFactory interface to obtain a Job that's to be run. Now, you might think that because of the name of Spring's JobFactory implementation, it's going to do something with a bean defined in the Spring ApplicationContext, right ? Wrong. What this implementation does is get the Job class provided by JobDetail.getClass(), instantiate a new instance of it with a default no-arg constructor, possibly populate it with properties, and then run the Job. Which sucks, and is kind of useless, because no Jobs I need can do so without getting something from Spring's ApplicationContext. More on this later.

The 'dataSource' property should be the javax.sql.DataSource used by your application to connect to whatever database you're using that stores the persistent Jobs in the tables used by Quartz.

The 'transactionManager' property should be set to a org.springframework.transaction.PlatformTransactionManager implementation that's used to demarcate transactions in the database. If you don't have one for your dataSource already defined, then it's fortunate that Spring comes with one that you can define and use, like so :
```
<bean id="mainDataSourceTransactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager" p:dataSource-ref="mainDataSource" />
```

The 'quartzProperties' property should be configured with a java.util.Properties instance containing configuration values for Quartz. A sample is posted here for you :
```
org.quartz.threadPool.class=org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount=5
org.quartz.threadPool.threadPriority=4

org.quartz.jobStore.tablePrefix=qrtz_
org.quartz.jobStore.isClustered=false
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
```
This sample contains a few simple properties that should be enough to get you started. There are numerous others that you should look into in the Quartz documentation. The first three properties are for configuring Quartz' thread-pooling capabilities. The latter three are for configuring the database (and access thereto-) that Quartz uses for storing persistent jobs. The table prefix is used to prefix Quartz' tables so that they don't conflict with any pre-existing tables in your database. The 'isClustered' property is used to determine whether Quartz is acting in a cluster. Clustered Quartz configurations are beyond the scope of this article. The 'driverDelegateClass' property is used to determine the class Quartz will use for dealing with your specific dialect of SQL (ie MySQL, MS-SQL, PostgreSQL, etc).

The 'applicationContextSchedulerContextKey' property sets a key that's used to access the Spring ApplicationContext in the JobExecutionContext given to a Job implementation at run time after the SpringBeanJobFactory has populated it for you. This is important as this property does not have a default value and will not put the ApplicationContext into the JobExecutionContext for you unless you specify one.

The 'waitForJobsToCompleteOnShutdown' property specifies, well, I think you can figure that one out.

At this point in time, you've got the tables which Quartz requires for persistent jobs defined (and presumably accessible) in your database, you've got the Quartz libraries present in your classpath, and you've got Quartz configured within Spring so that Quartz will run and check the database for jobs to be run. However, we're still missing a few more important pieces of information.

JobDetails stored in the database describing jobs to be run

Triggers defined in the database for describing when JobDetails will be executed

For the rest of this article, lets assume that you've set your Quartz table prefix as 'qrtz_'. In your database, lets look at the definition of the 'qrtz_job_details' table. It includes columns for defining job_name, job_group, description, job_class_name, is_durable, is_volatile, is_stateful, requests_recovery, and job_data.

The 'job_name' field is, as you've almost certainly guessed, the name of the job that you want to define. The job must also have a 'job_group' specified because triggers can be used to signal the execution of entire groups of jobs at once, not just single jobs. The two of these fields together form a unique key for the table. If you really don't give a shit about grouping jobs, you can just use Quartz' default group name, which is 'DEFAULT'. You also need to set a 'job_class_name'. This value is a fully qualified Java class name that must have a default, no-arg constructor (ie it's a bean, according to the Java Beans specification). This class will get instantiated at run time by the SpringBeanJobFactory and run as a Quartz job. Note that this class, obviously, must implement the org.quartz.Job interface. A decription of the job can be put in the 'description' field if you want to have a log-friendly message available to you. The is_durable, is_volatile, is_stateful, and requests_recovery fields are all (essentially) boolean fields that Quartz defines as being single characters. These fields can have the values 'Y' or 'N'. See the Quartz API for org.quartz.JobDetail for further elaboration on the meaning of these fields. The 'job_data' field is a BLOB object that's used to serialize the 'jobDataMap' (java.util.Map) associated with the JobDetail. In order to populate this field, you'll obviously have to write some JDBC code.

The 'qrtz_job_details' table is, obviously, where you store the information for the org.quartz.Jobs you want to run. Anytime you need to define a new job, you'll need to insert a new row into this table, properly configured of course and paying particular attention to the 'job_class_name' field.

Ok, so assuming you've got a couple rows in this table (ie defined a couple of jobs), now you'll need to schedule them so that, at some point, they'll actually run. This is where triggers come in. Now, in Quartz, there are a few different methods of triggering off Jobs, by far the most common being org.quartz.SimpleTriggers (qrtz_simple_triggers table) and org.quartz.CronTriggers (qrtz_cron_triggers table). SimpleTriggers have their use, but our company uses some pretty complex scheduling at times, so I'm going to do an example of using the cron triggers.

The 'qrtz_cron_triggers' table is pretty simple, it only has four fields : 'trigger_name', 'trigger_group', 'cron_expression' and 'time_zone_id', all of which should be pretty indicative of their content. The 'time_zone_id' field should be a valid id for a java.util.TimeZone, ie 'America/Denver'. See the documentation for that class for more on valid IDs.

Once you've set up a trigger with a valid name, group (ie DEFAULT), cron expression and TimeZone ID, you're pretty much ready to go. When Quartz starts up with your application, it'll consult the 'qrtz_triggers' table and ensure that entries exist for all of your corresponding cron / simple / blob triggers. It'll then proceed to fire off triggers and the Scheduler will pass off jobs to the SpringBeanJobFactory for execution. Now, if you haven't figured it out already, if you've got a lot of jobs, so far from what this article has shown you, you'll have to create a Quartz Job implementation for each job you want to run off, and even then, it'll have to be pretty simple, since you'll have to be able to instantiate it with a no-arg constructor, and so far you've got no way of injecting it with anything from Spring, aside from simple properties that are defined by the 'schedulerContextAsMap' property on the Spring SchedulerFactoryBean. So, what I did was create a class that uses its JobDetail#name as a convention for getting a bean from the application context, assumes it's an org.quartz.Job implementation, and executes it. The code is attached :


public class SpringBeanDelegatingJob implements Job {
 
 private static final Log LOGGER = LogFactory.getLog(SpringBeanDelegatingJob.class);
 
 public static final String APPLICATION_CONTEXT_KEY = "applicationContext";

 @SuppressWarnings("unchecked")
 public void execute(JobExecutionContext arg0) throws JobExecutionException {

  JobDetail jobDetail = arg0.getJobDetail();
  
  String beanName = substringBefore(jobDetail.getName(), "Detail");
  
  if (LOGGER.isInfoEnabled()) {
   LOGGER.info ("Running SpringBeanDelegatingJob - Job Name ["+jobDetail.getName()+"], Group Name ["+jobDetail.getGroup()+"]");
   LOGGER.info ("Delegating to bean ["+beanName+"]");
  }
  
  ApplicationContext applicationContext = null;
  
  try {
   applicationContext = (ApplicationContext) arg0.getScheduler().getContext().get(APPLICATION_CONTEXT_KEY);
  } catch (SchedulerException e2) {
   throw new JobExecutionException("Holy fuck, there was some kind of god-damned problem with the fucking Scheduler", e2);
  }
  
  Job bean = null;
  
  try {
   bean = (Job) applicationContext.getBean (beanName, Job.class);
  } catch (BeansException e1) {
   throw new JobExecutionException("Unable to retrieve target bean that is to be used as a job source", e1);
  }
  
  bean.execute (arg0);
  
  return;
 }

}

Now, this makes the persisted jobs a lot more capable : we can now run org.quartz.Job beans that are declared in our Spring ApplicationContext. What if we want to be able to run arbitrary methods on arbitrary beans as jobs ? Well, we can write an adapter class for that too, quite similar (though not as functional as) Spring's MethodInvokingJobDetailFactoryBean :


public class SpringBeanMethodInvokingJob implements InitializingBean, Job {
 
 private Object targetBean;
 private String targetMethod;

 //Constructors

 public SpringBeanMethodInvokingJob() {
  super();
 }
 
 //Behaviour Methods

 public void execute(JobExecutionContext arg0) throws JobExecutionException {
  
  Method method = null;
  
  try {
   method = targetBean.getClass().getMethod(targetMethod);
  } catch (Exception e) {
   throw new JobExecutionException("Unable to get targetMethod ["+targetMethod+
    "] on bean with class ["+targetBean.getClass().getName()+"]");
  }
  
  try {
   method.invoke(targetBean);
  } catch (Exception e) {
   throw new JobExecutionException("Unable to invoke method ["+method.getName()+"] on bean ["+targetBean.toString()+"]");
  }
  
  return; //done
 }

 public void afterPropertiesSet() throws Exception {
  Assert.notNull(targetBean, "'targetBean' cannot be null");
  Assert.isTrue(isNotBlank(targetMethod), "'targetMethod' cannot be blank");
 }

 //Property Accessors
 
 @Required
 public final void setTargetBean(Object targetBean) {
  this.targetBean = targetBean;
 }

 @Required
 public final void setTargetMethod(String targetMethod) {
  this.targetMethod = targetMethod;
 }

}

Note that the class above is implements org.quartz.Job, not org.quartz.JobDetail as is the product of MethodInvokingJobDetailFactoryBean. I hope this article has been useful for anybody reading it, feel free to comment on it.

The greatest MySQL companion ever

2008-12-05T10:39:00.000-08:00

I just recently found out about the 'show processlist' command in MySQL that lets you get a resultset of all the processes MySQL is currently using and what query they're running. Today I was working in top monitoring the performance of our servers as I restarted our production server. MySQL popped up briefly on the top list of running processes and it occurred to me, "Wouldn't it be great of there was something like this for MySQL?". So while I was waiting, I popped open google, and ran a query for a MySQL and top. To my surprise, up came mytop. It's literally top for MySQL and it's so ridiculously useful you wouldn't believe it. Even better, it's got a .deb made for it and it's in the Ubuntu repositories, so installing it is as simple as running :

apt-get install mytop

It's a good day today :)

Setting a user's password in MySQL

2008-12-05T09:29:00.000-08:00

I've recently found the need to change a bunch of passwords in MySQL. The command to do it is as follows :

SET PASSWORD FOR 'myuser'@'%.wherever.com' = PASSWORD('newpass');

You can also read the documentation on the MySQL website.

Finding out what MySQL is doing

2008-12-03T11:19:00.000-08:00

I was recently updating our production system when I noticed that our main MySQL database was going unusually hard. On a quad-core processor system, it was using up three processors simaltaneously. This prompted me to find out what MySQL is doing, and after being unable to read the logs, I did a Google search and ran across this useful little command :

show processlist

This runs a query (can be run from either the command line or the query browser) and shows you a summary of all the processes that MySQL is currently using.

Migrating from Spring Webflow 1 to Webflow 2

2008-11-26T07:12:00.001-08:00

...was relatively painless. At least in the very small app that I'm currently working on as a helper application for dealing with one of our partners' systems. The flow definition migration assistant that comes with Webflow 2 is a huge help. There are, however, a few things to take note of when it comes to differences between Webflow 1 and Webflow 2. They are significantly different beasts to say the least :

My configuration went from this (in Webflow 1) :


 <flow:registry id="flowRegistry">
  <flow:location path="/WEB-INF/flows/**/*-flow.xml"/>
 </flow:registry>

 <flow:executor id="flowExecutor" registry-ref="flowRegistry">
  <!--flow:execution-listeners>
   <flow:listener ref="webflowDebugListener"/>
  </flow:execution-listeners-->
 </flow:executor>
 
 <!--bean id="webflowDebugListener" class="org.springframework.webflow.execution.DebuggingListener"/-->
 
 <bean name="flowController" class="org.springframework.webflow.executor.mvc.FlowController">
  <property name="flowExecutor" ref="flowExecutor" />
  <property name="argumentHandler">
   <bean class="org.springframework.webflow.executor.support.RequestParameterFlowExecutorArgumentHandler" />
  </property>
 </bean>

To this (in Webflow 2) :


 <webflow:flow-registry id="flowRegistry" flow-builder-services="flowBuilderServices">
  <webflow:flow-location-pattern value="/WEB-INF/flows/**/*-flow.xml"/>
 </webflow:flow-registry>

 <webflow:flow-executor id="flowExecutor" flow-registry="flowRegistry">
  <webflow:flow-execution-listeners>
   <!-- webflow:listener ref="webflowDebugListener"/ -->
   <webflow:listener ref="securityFlowExecutionListener"/>
  </webflow:flow-execution-listeners>
 </webflow:flow-executor>
 
 <bean id="securityFlowExecutionListener" class="org.springframework.webflow.security.SecurityFlowExecutionListener" />
 
 <webflow:flow-builder-services id="flowBuilderServices" view-factory-creator="viewFactoryCreator" conversion-service="webflowConversionService"/>
 
 <bean id="webflowConversionService" class="com.mypackage.modules.springframework.webflow2.ConversionServiceFactoryBean">
  <property name="editorMappings">
   <map>
    <entry key="java.util.Calendar"><idref bean="sqlDateCalendarEditor"/></entry>
   </map>
  </property>
 </bean>

 <bean id="viewFactoryCreator" class="org.springframework.webflow.mvc.builder.MvcViewFactoryCreator">
  <property name="viewResolvers">
   <list>
    <ref local="xmlViewResolver"/>
    <ref local="decoratedJstlViewResolver"/>
    <ref local="urlBasedViewResolver"/>
   </list>
  </property>
 </bean>

 <bean id="webflowDebugListener" class="org.springframework.webflow.execution.DebuggingListener"/>
 
 <bean name="flowController" class="org.springframework.webflow.mvc.servlet.FlowController">
  <property name="flowExecutor" ref="flowExecutor" />
  <property name="flowUrlHandler">
   <bean class="org.springframework.webflow.context.servlet.DefaultFlowUrlHandler" />
  </property>
 </bean>

Now, you may be asking yourself, "Why would there be more configuration in Spring Webflow 2 ? Doesn't Spring generally improve on things like configuration syntax between major versions ? The answer is, of course, yes. However, there are a number of new features that come with Webflow 2 that need to be configured, hence the extra beans for configuration. Note also, that the syntax has changed. I direct your attention to the added 'flow-' prefixes on element names of the previously existing Webflow elements.

Webflow 2 has much better, much closer integration with Spring Security, hence the need for SecurityFlowExecutionListener listed above. I haven't had a chance to play around with the new integration between Spring Security 2 and Spring Webflow 2, but I'm quickly getting there.

Thanks to the built in model binding and validation that comes with Spring Webflow 2, there's much less syntax for views that simply bind to a model object on transitions. Instead of having to add in FormActions and adding action elements to the view-state you want them applied to, you can now simply specify a model name for the 'model' attribute of the view state, and Webflow 2 will automatically bind the form object for you (which leads us into our next point). Of course, the Spring developers maintained their usual style of keeping things as configurable as possible, so you can not only choose not to bind the model object for particular state transitions, but you can disable validation as well, using the new 'bind' and 'validate' attributes of Webflow 2 'transition' elements.

Because Webflow 2 can now do binding and validation for us right in the flow definition without having to define FormAction beans in our Spring ApplicationContext, Webflow 2 comes with its own updates to the spring-binding framework, hence the need for the 'webflowConversionService' bean in the configuration above. Now, you're probably saying to yourself right now, "Well what the fuck am I supposed to do with the custom PropertyEditors I wrote for my objects when dealing with Spring / Webflow 1 ?". Don't worry, all is not lost. There's a PropertyEditorConverter class that adapts PropertyEditor implementations to the new Webflow 2 Converter interface. You can use this to interface your editors with Webflow 2, and in fact that's what I did.

Webflow 1 left it up to the Spring framework to resolve and render views. Some people found this inconvenient, so Webflow 2 now allows you to have Webflow directly resolve and render views itself. However, since I'm already using the Spring framework and don't need Webflow's abstraction away from the underlaying framework, I chose to keep the Webflow 1 style of resolution and view rendering, hence the need for the 'viewFactoryCreator' bean above.

Once all the new configuration was done and out of the way, it was simply a matter of upgrading my existing flows to the new Webflow 2 syntax, which was almost entirely taken care of by the migrator that comes with Webflow 2. Previously, I used the 'var' element heavily to make use of beans in my Spring ApplicationContext, but the 'var' element has since had its usage completely reworked for Webflow 2, so I'm going to have to refactor a couple of my flows and change my style of design with Webflow 2.

One other feature that I'm learning about as I write this post is the Webflow 2 PersistenceContext. It's a solution for the use case where you want to be able to load objects from a database, and only allow a single user to be able to access them at once, ie you wouldn't want multiple people logging into a database and being able to access data on a particular user at the same time and potentially overwrite each others' changes. I'll post more about that later.

Migrating from Acegi Security to Spring Security 2

2008-11-21T12:23:00.000-08:00

In an effort to keep our applications current, I've been looking at upgrading from Acegi Security to Spring Security 2 in one of our smaller applications as a test, to see how well it'd go over for our larger applications. So far, the tests are pretty positive. The main advantage to the migration that I can see is improved Web Service security (which I'll definitely be needing), as well as a considerably simplified configuration syntax. The following is a small migration guide :

0. If you haven't already, update your project to use Spring 2.5.4 or greater. This is necessary because of changes to Spring that Spring Security (as well as other Spring subprojects) requires. If you're using Maven 2, this is dead easy, you need only update the <version> elements of the appropriate dependencies.

1. Change all package references starting with 'org.acegisecurity' to 'org.springframework.security' in your code. If you've architected your system right, this should mean very few changes.

2. In your POM, remove all mention of Acegi security and replace it with the following :

<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-core-tiger</artifactId>
    <version>${spring.security.version}</version>

    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-core</artifactId>
        </exclusion>

        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-support</artifactId>
        </exclusion>
    </exclusions>

</dependency>
...
<dependency>
    <groupId>org.springframework.security</groupId>
    <artifactId>spring-security-taglibs</artifactId>
    <version>${spring.security.version}</version>

    <exclusions>
        <exclusion>
            <groupId>org.springframework</groupId>
            <artifactId>spring-web</artifactId>
        </exclusion>

    </exclusions>
</dependency>

The exclusions are necessary to prevent Spring 2.0.x dependencies from being unnecessarily pulled in.

3. Go through your JSPs and remove all xmlns:authz="..." declarations and replace them with :


    xmlns:security="http://www.springframework.org/security/tags"

4. Go through your applicationContext*.xml files and replace any constants you may have referring to Acegi security with their proper Spring Security 2 counterparts. This means replacing the package prefixes 'org.acegisecurity' with 'org.springframework.security' and in some cases replacing ACEGI with SPRING_SECURITY in some static constant names.

This migration guide was based on (and expanded from) the one provided by Matt Raible. If you get the time, I recommend you check out his site, he has a lot of good wisdom and examples to share.

On Hibernate UserTypes

2008-11-21T08:16:00.000-08:00

Hibernate UserTypes are a great way for persisting complex classes which may require more than one column in a table in order to be properly represented in a database. Many people have used them successfully in order to be able to simplify storage of complex objects and thus simplify their applications at the same time. One simpler use case of them was to create Enum types before support for Java 5+ enums came along.
Now, you'd think that once you've got Java 5 support with Hibernate, you'd never have to use such a simple use case again because you've got the Enums themselves, along with the @Enumerated annotation you can place on fields and properties, right ? Wrong. I've recently started migrating my projects to Spring Security from Acegi Security, and I've found that in the process, they've made the GrantedAuthority interface extend Comparable. This conflicts with the fact that Java 5+ Enums already implemented the Comparable interface with a generic parameter, and as such your code won't compile if you have an Enum that also implements GrantedAuthority, such as the following :


public enum Role implements GrantedAuthority, ConfigAttribute
{
 ROLE_ADMINISTRATOR;

 public String getAuthority() {
  return name();
 }

 public String getAttribute() {
  return name();
 }

 public String getMessageKey() {
  return "enum.Role.".concat(name());
 }
}

In order to remedy the conflict, I've had to convert my nice, pretty, simple Enum into a class, and simulate Java 5+'s enum behaviour, like so :


public final class Role implements GrantedAuthority, ConfigAttribute, Serializable, Cloneable {

 private static final long serialVersionUID = 1L;

 public static final Role ROLE_ADMINISTRATOR = new Role(0, "ROLE_ADMINISTRATOR");

 private static final Role[] VALUES = new Role[] {
  ROLE_ADMINISTRATOR
 };
 
 private static final Map NAME_MAPPINGS = new HashMap();
 
 static {
  Arrays.sort(VALUES, new Comparator() {
   public int compare(Role o1, Role o2) { 
    return o1.ordinal - o2.ordinal;
   }
  });
  
  for (Role r : VALUES) {
   NAME_MAPPINGS.put (r.getName(), r);
  }
 }
 
 private int ordinal;
 private String name;

 //Constructors

 /*
  * DO NOT EVER USE THIS! It exists only for serialization purposes.
  */
 public Role() {
  super();
 }
 
 private Role(Role r) {
  this(r.ordinal, r.name);
 }

 private Role(int ordinal, String name) {
  super();
  this.ordinal = ordinal;
  this.name = name;
 }

 //Behaviour Methods

 public int compareTo(Object o) {

  if (o == null || o.getClass() != Role.class) {
   throw new IllegalArgumentException(
     "Comparison object may not be null, and must be a Role");
  }

  return this.ordinal - ((Role) o).ordinal;
 }
 
 public String getAuthority() {
  return getName();
 }

 public String getAttribute() {
  return getName();
 }

 //Pseudo-properties

 public String getMessageKey() {
  return "enum.Role.".concat(getName());
 }
 
 //Property Accessors

 public final int getOrdinal() {
  return this.ordinal;
 }

 public final String getName() {
  return this.name;
 }

 //Helper Methods
 
 public static final int hashCode(Role r) {
  return r == null ? 0 : r.hashCode();
 }
 
 public static final boolean equals(Role x, Role y) {
  return x == null ? (y == null) : x.equals(y);
 }
 
 public static final Role clone(Role r) {
  return r == null ? null : r.clone();
 }
 
 public static final Role[] values() {
  return VALUES;
 }
 
 public static final Role valueOf(String s) throws IllegalArgumentException {
  
  String key = defaultString(s).toUpperCase();
  
  if (NAME_MAPPINGS.containsKey(key)) {
   return NAME_MAPPINGS.get(key);
  } else {
   throw new IllegalArgumentException("No role by the name ["+s+"] exists");
  }
 }
 
 //Object Overrides
 
 @Override
 public String toString() {
  return new StringBuffer().append(this.name)
   .append("(").append(this.ordinal).append(")")
   .toString();
 }

 @Override
 public Role clone() {
  return new Role(this);
 }

 @Override
 public int hashCode() {
  final int prime = 31;
  int result = 1;
  result = prime * result + ((this.name == null) ? 0 : this.name.hashCode());
  result = prime * result + this.ordinal;
  return result;
 }

 @Override
 public boolean equals(Object obj) {
  if (this == obj)
   return true;
  if (obj == null)
   return false;
  if (getClass() != obj.getClass())
   return false;
  final Role other = (Role) obj;
  if (this.name == null) {
   if (other.name != null)
    return false;
  } else if (!this.name.equals(other.name))
   return false;
  if (this.ordinal != other.ordinal)
   return false;
  return true;
 }

}

Once this was done, I created a Hibernate UserType implementation for Role, so that Hibernate could persist Roles as Integers, just as was already being done for the Enum version of Role:


public class RoleUserType implements UserType {

 private static final int[] SQL_TYPES = new int[] { Types.INTEGER };
 
 public static final String HIBERNATE_TYPE_NAME = "RoleUserType";

 public Object deepCopy(Object value) throws HibernateException {
  return value;
 }

 public Object assemble(Serializable cached, Object owner) throws HibernateException {
  return (Role)cached;
 }

 public Serializable disassemble(Object value) throws HibernateException {
  return (Role)value;
 }

 public boolean equals(Object x, Object y) throws HibernateException {
  return Role.equals((Role)x, (Role)y);
 }

 public int hashCode(Object x) throws HibernateException {
  return Role.hashCode((Role)x);
 }

 public boolean isMutable() {
  return false;
 }

 public Object nullSafeGet(ResultSet resultSet, String[] names, Object owner) throws HibernateException, SQLException {
  int roleOrdinal = resultSet.getInt(names[0]);
  
  return resultSet.wasNull() ? null : Role.values()[roleOrdinal]; 
 }

 public void nullSafeSet(PreparedStatement statement, Object value, int index)
   throws HibernateException, SQLException {
  if (value == null) {
   statement.setNull(index, Types.INTEGER);
  } else {
   statement.setInt(index, ((Role)value).getOrdinal());
  }
 }

 public Object replace(Object original, Object target, Object owner) throws HibernateException {
  return original;
 }

 @SuppressWarnings("unchecked")
 public Class returnedClass() {
  return Role.class;
 }

 public int[] sqlTypes() {
  return SQL_TYPES;
 }

}

Once the Hibernate UserType implementation was written, it was just a matter of adding my model package to Hibernate's list of annotated packages (using Java 5 after all) :


@TypeDef(name = RoleUserType.HIBERNATE_TYPE_NAME, typeClass = RoleUserType.class)
package com.mypackage.model;

import org.hibernate.annotations.TypeDef;
import eu.alenislimited.acshelper.support.hibernate.RoleUserType;

After defining the Hibernate UserType type, it was then just a matter of going around to all my fields that used Role, and replacing ...


@Enumerated(EnumType.ORDINAL)

... with ...


@Type(type = RoleUserType.HIBERNATE_TYPE_NAME)

Fortunately, doing the conversion was just extra work, and didn't require any special changes to any of my Spring configuration, or even my Hibernate configuration (beyond adding the @TypeDef to my model package and adding it to my Hibernate SessionFactory's list of annotatedPackages).

A first time gem creator's addendum

2008-11-04T10:27:00.001-08:00

I recently started researching how to create Ruby Gems because I was working on on a Ruby application that is a reference application for accessing our production system at work, and I wanted to make the specific access code for our production system a Gem in Ruby so that I could just hand it off to future clients who want to integrate with Ruby. Further to that end, I read about the Newgem gem, and so I decided to use it, since it had recently hit 1.0.1. You can read the documentation there, so I won't repeat it here, but there's a couple of things you should do, prior to following their instructions.

Set a 'HOME' environment variable if you're using Windows (as I am)

gem install cucumber newgem

rake config_hoe

The first step is to install newgem, but to also ensure that cucumber is installed as well because it's an unlisted dependency. The second step is to ensure that there's a basic .hoerc file created in your home directory that's ready to go whenever you want to run any tasks on your gem while testing / packaging it. I'm guessing it's automatically created in Linux, but I had to manually set the 'HOME' environment variable and run the configuration task myself in order to have the proper configuration file generated for me to run the tasks for creating and testing my gem.

A quick note on ActiveResource in Ruby on Rails

2008-10-29T12:26:00.000-07:00

I've learned a bunch of new things about Ruby on Rails in the past few days, specifically when it comes to RESTful web services.

ActiveResource uses the built in Ruby XML::Mapping library for its OXM (Object to XML mapping) layer

Apparently, no Ruby library out there properly handles namespaces. Not a one : REXML, ROXML, XML::Mapping

ActiveResource can be highly customized to adjust to other RESTful web service schemes, just not when it comes to the generated XML.

Unfortunately this makes it very hard to integrate (in ruby) with the RESTful web service that I've written for our company's production web site. Ironically, I wanted to make the service very Ruby friendly. I really wish there was more in depth documentation out there for Ruby APIs. It would have made things so much easier when I was designing our API, which runs in Java, using JAXB.

A quick note on Maven plugins

2008-10-13T08:51:00.000-07:00

I've recently had to go abroad to various other countries for work, and I've found that porting my projects around to various workspaces hasn't been as easy as it otherwise should have been, and here's why : I use Maven, especially the plugins. I've used others, and I've even written my own. I've also learned that one should specify versions of all plugins used, because the behaviour of a plugin may change between versions and if you're unaware of that and you start getting strange behaviour from your project build, this can lead to hours of wasted time trying to figure out what the hell is going on. I'm referring specifically to the maven-war-plugin for building web application WAR files. I had previously left the version unspecified, and when I ported my workspace to my laptop, Maven chose to use a different version of the plugin than that of my desktop, and as a result was including files I didn't want in my WAR file. This led to classpath errors, and me spending several hours trying to find out where the problem was. So in short, always specify versions of the plugins you use to ensure consistent behaviour in your build, wherever you may be working on your project.

Reflection in Ruby

2008-08-29T11:43:00.001-07:00

I've recently gotten started going on a Ruby on Rails project for my company, that's meant to be a simple app that we'll throw up and let run for any prospective customers. But I ran into a problem with copying property values between objects, since I'm not as familiar with the language as I'd like to be. The Object type defines a method called 'send' which lets you invoke a method with given arguments. However, I wanted to be able to assign property values via this method, and just sending the name of the property as the name of the method to invoke wasn't working. (Works just fine if you want to get the property). As it turns out, the method for invoking the setter of a property is brain-dead simple : use the name of the method as you'd define it if you were overriding the setter for the property. For example, if you have a property called 'name', you'd define the setter for it as follows :


def name= (name)
  @name = name
end

So if you wanted to set the value of this property reflectively, you'd do something like :


 @person_instance.send "name=", "John Smith"

A moment of developer clarity and zen

2008-08-16T20:19:00.001-07:00

I had a moment today where I was just letting my mind wander. It was one of those moments where you have a brilliant idea that just seems so common sense that you can only ask yourself "Why the hell didn't I think of that before ?" This moment of clarity happened to be concerning my horribly slow laptop, with it's horribly old ATA 100, 5400 rpm hard drive. Then it occurred to me : just like with video encoding that I had setup months ago, I could setup a RAM disk to speed up Eclipse by storing my workspace on the RAM disk to speed up access to all the (ridiculous number of) temp files that Eclipse uses. On Ubuntu Linux, it turns out this was ridiculously easy : http://ubuntuforums.org/showthread.php?t=182764

The really amazing part was just how much this sped things up : it took my project build time from 6 mins, 30 seconds down to 35 seconds. And those are real numbers (plus or minus a couple seconds, due to slight variations in repeated builds.)

More JAXB quirks

2008-08-11T23:58:00.001-07:00

Today I ran across yet more quirks of JAXB 2.0. I'm getting really tired of running across stupid errors that are difficult to debug because the programmers who wrote JAXB are lazy. Today, I was receiving a java.lang.InternalError when I tried to compile a schema from beans I'd recently refactored. The error I received was something along the lines of "could not escape schema namespace". As it turns out, it was caused by the lack of an @XmlType annotation on the enums I had added to one of my XML serializable beans. I hope this helps anybody who has the same problem.

A quick note about JAXB

2008-07-30T15:45:00.001-07:00

JAXB (the Java API for XML Binding) is a great piece of technology (2.x, not 1.x). However, if you're starting an application and your JAXBContext is throwing an exception saying that it's unaware of a given class, there's a quick fix. At this point, I'm guessing you've configured your JAXB context to use a package, and the class that the JAXBContext is unaware of is in the given package, and you're scratching your head as to why it can't seem to find a class that's in the proper package. Well, here's the most probable answer : if you're using package-configured JAXBContexts, you need to have factory methods for all top level elements, similar to the following :



    /**
     * Create an instance of {@link PurchaseOrder }
     * 
     */
    public PurchaseOrder createPurchaseOrder() {
        return new PurchaseOrder();
    }

... assuming that mypackage.PurchaseOrder is the class the JAXBContext is complaining that it's unaware of. This little quirk took me a couple hours to debug because I had just regenerated some web services code using Apache CXF 2.1.1 from an updated WSDL document for an updated Web Service that our company uses. Little did I know, that in comparing the two ObjectFactory files in Eclipse, I had missed copying over certain important functions. Hopefully this helps you.

Pure FTP on Ubuntu

2008-07-11T09:39:00.000-07:00

Pure FTP is a great FTP server and I love it because it's ridiculously fast, reliable and works well for our company. The only problem is that it's tricky to configure on Ubuntu because just like with numerous other programs, the people at Ubuntu, in their infinite wisdom, felt the need to fuck with the program's default way of doing things. Specifically, in Ubuntu, command line arguments for starting the FTP service are done through individual files for each command line option you want the service to be started with, with values for the options placed inside each of the files. This is a stupid way of doing things, but that's not the topic of this post.

The problem I've been having lately is that I recently deployed new projects within our production network, and these new projects required access to the FTP server, and aren't on the same box as the server, as is the sole other project that has been using the FTP server. I'd been getting problems trying to connect to the server from any LAN box, but not any external boxes nor the machine itself (localhost). The error I got back was "421 Service not available". I googled around for hours and found nothing useful, until I started realizing that other people were getting 421 errors when their PureFTP instance was misconfigured, but with different messages, and then it got me thinking that maybe my instance was somehow misconfigured.

I re-read the documentation for PureFTP and after an hour or so, it hit me that the server does reverse lookups to resolve fully qualified names, and such resolution doesn't work properly on our network (for good reasons that I'm not going to go into). After disabling reverse DNS resolution with the -H startup option (`echo "yes" >> DontResolve` in the configuration directory in Ubuntu), the problem went away.

I hope this helps anybody else who runs into the same problem.

More Tomcat quirks

2008-07-08T09:56:00.000-07:00

I recently ran across a problem with receiving a LinkageError in one of my web applications. The problem occurred when Acegi security tried to use libraries that were not yet loaded and tried to load them itself. Strangely, the error only occurred while using my RESTful API with a Spring Framework controller. If I logged in through my web interface which uses Acegi and Spring MVC, then the linkage error didn't occur.

After numerous hours of Googling around, I found a small company's JIRA issue tracker that reported the same bug, but also that it only occurred when using the security manager. After disabling the security manager on my own Tomcat instance, I found that the problem went away. I'd very much like to have a better resolution, but it's going to have to do for the time being.

Tomcat quirks

2008-07-07T11:33:00.000-07:00

In recent weeks, we've had problems with our company's servers going down because of power problems at our colocation provider. As a result, the fact that Tomcat hadn't been properly configured to boot at startup had really burned us. Consequentially, I've been spending the past few days troubleshooting why the servers wouldn't startup, and ensuring that they do come up properly at boot time in future. The following are some of the things I've run across.

The JAVA_HOME and JRE_HOME variables must be set. If they're not, then Tomcat won't start, at least on Ubuntu Gutsy Gibbon Server. This is a problem I encountered with the default installation of JIRA on one of our production servers.

When using EhCache with Hibernate, the tmp directories don't get deleted, regardless of whether or not Tomcat is shutdown properly. This will block the container from starting up again, so these temp files must be deleted. I encountered this problem with my own applications.

Finally! The trick to proper Java Date and Calendar formatting

2008-07-02T15:59:00.000-07:00

It's well documented in the Java API as well as numerous forums and blogs, that the Java API is not properly implemented, and in some respects flat out broken, hence why Java 7 is going to have a brand spanking new Date/Time API based on the Joda Time library hosted at sourceforge.net. In implementing Dates / TimeZones in Java, the developers of Java saw fit to separate time zones from date implementations because, really, a date is just a number of milliseconds from Jan 1, 1970 at 00:00:00.000 GMT in the morning. This is a noble goal which I whole heartedly agree with : dates should be represented separately from their timezones because the printed representation of a date will change depending on time zone, whereas no matter where you are in the world, the number of milliseconds from the Epoch has not changed. However, the Java developers implemented this concept poorly. They chose to attach TimeZones to Calendars and then not have them affect any of the time fields, which is fine. But the TimeZone attached to a calendar is not used when using a DateFormat directly. Here's the key : the DateFormat object uses its own internal TimeZone during formatting (which it takes from the currently running JVM) by default. In order to format using the TimeZone attached to a Calendar, you have to explicitly set the TimeZone of the DateFormat to that of the Calendar instance you want to format.

An interesting 'accesskey's quirk

2008-06-13T15:04:00.000-07:00

Since I learned about accesskey support in HTML, I've been including them where appropriate in my web applications to speed up my access and usage of said applications. However, when trying to use the accesskey for a password field, P, I ran across an interesting quirk. The accesskey modifier is different for various browsers. In order to use access keys in IE, the modifier is Alt, for Firefox it's Alt+Shift, for Opera you press Shift+Esc, then the accesskey. I primarily use Firefox (go open source), so when I pressed Alt+Shift+P to access a password field, it opened Windows Media Player. As it turns out, this is the hotkey for accessing WMP from the toolbar (ie the miniplayer). To prevent this hotkey combination from opening WMP, you'll have to disable the toolbar by going to the taskbar, Right-Click -> Toolbars and uncheck Windows Media Player (I don't like the mini-player toolbar that much anyway).

Spring Framework URL mapping quirks

2008-06-11T17:17:00.000-07:00

My recent foray into developing using RESTful principles and resources has lead me to an interesting quirk in the Spring Framework, which I use for our company's primary application. As it turns out, the URL mapping scheme requires that you set 'alwaysUseFullPath' to true so that the entire path of a URL that gets submitted to the framework gets inspected to determine the proper controller mapping. Otherwise, it can lead to some quirks, like not submitting the request to the proper controller. I had to learn this the hard way through several hours of debugging.

Why you should always export your sources with your Java projects

2008-06-05T07:09:00.000-07:00

With the advent of popular and widespread Maven 2 adoption, there's no reason you shouldn't be using it to build your open-source or closed source Java projects. With that in mind, it's dead easy to use Maven's 'source' plugin to automatically generate a separate JAR with your project's sources in it when you package and deploy your project. Posting the sources with your project not only allows other people to easily debug their code while using your project, but also serves as a backup in the event of a catastrophic failure of a computer hosting your code for which you did not have proper backup procedures already in place. In the event that you couldn't tell, that happened to me tonight.

JAXB schemagen - Finally got it working

2008-06-05T06:47:00.000-07:00

For those not in the know, schemagen is a program that comes as part of the Java Web Services Developer Pack and it's used to generated XML schemas from JAXB annotations on Java beans. It so happens that the creators of the JAXB reference implementation also made a Maven 2 plugin for this program. I had previously attempted to use it to generate schemas in my initial attempts with JAXB, but had been presented with a slough of unfriendly exceptions being thrown at me whenever I ran the plugin. Not having the time back then to really play around with it and being considerably less experienced with JAXB, I had to shelve it and find other, less satisfying solutions to my problems with generating documentation.

Recently, I've had to come back to using JAXB because I'm making a RESTful Web Services API that leverages the power of both JAXB and Hibernate to do all my heavy lifting for me. This time around, I wasn't willing to tolerate a lack of schema to give to our users of the API, because it would mean a lot more work for them, and a lot more work for me. This time, I decided to be persistent and dig around the jaxb-schemagen plugin to make sure I could get it working. I'm proud to say that my perseverance paid off, and I now have my RESTful API schema being automatically generated for my application. Here are the problems and the solutions I ran into when I was trying to get going on this:

NullPointerException: I came across persistent NullPointerExceptions while trying to run the plugin at first. The stack trace indicated that the problem was with apt, the Java annotation processing tool. I later discovered that in order to properly map your classes into XSD schema types, you must specify an @XmlType annotation on *all* of the classes you wish to map (at least with JAXB 2.0, which is the version I'm currently using)
ClassCastException: After I resolved the first problem, I came across another pesky exception that likewise did not provide any useful debugging information. The plugin output indicated that it was finding annotations that it didn't know how to deal with, specifically the Java Persistence API annotations that I was using for Hibernate. After much digging around on Google and some detective work, I discovered that any annotations encountered by the jaxb-schemagen plugin had to be on the Maven runtime classpath. I double checked my POM to find that I had specified the scope of the JPA / Hibernate annotations as 'provided', which instructs Maven not to load them into either the compile or runtime classpaths. The initial reason I had for specifying the dependency scope as 'provided' was so that they would not needlessly get included with my project WAR files. I changed their scope to 'compile' (which also includes 'runtime') and voila, apt could now properly detect the unknown annotations, and get past them to deal with the JAXB annotations it needs to create the XML schema.

If anybody else encounters similar errors with JAXB-schemagen, I hope they find this page and find it useful.

Fixing package retrieval quirks in Ubuntu Gutsy Gibbon Server

2008-06-03T13:25:00.000-07:00

In the default installation of Ubuntu Server (Gutsy Gibbon), the default source of packages is the CD-ROM drive. That's fine if you make the assumption that there will always be somebody physically present in order to maintain the servers, but for most server's these days, that's a fallacious assumption. The really bad part about this default, is that there's no way I know of at the time of this writing to override this on the update command. In our company, we have our servers hosted off-site like so many other companies, so everything must be done remotely. Therefore, the CD-ROM as a source for Ubuntu packages is a bad idea. To remedy this, you have to go into the apt-get configuration stored at

/etc/apt/sources.list

, and you'll have to find a line similar to :

deb cdrom:[Ubuntu-Server 7.10 _Gutsy Gibbon_ - Release i386 (20071016)]/ gutsy main restricted

Comment out this line, try your update again and you should be good to go.

Fixing Messenger when you've accidentally screwed it up

2008-05-27T12:41:00.000-07:00

If you're like me, you like to have your Windows Live Messenger conversation windows be as simple and clean as possible, with no toolbars, and no contact pictures. To that end, I removed the toolbars from my messenger contact windows so that I wouldn't have to see them. However, this became a problem one day when I dragged a picture that I intended to send to someone into a conversation window, and instead of sending it to them, Messenger permanently changed the background of my chat windows with that person to that picture. To resolve this, I had to open a chat window with person X, press the Alt key to get the old-school windows standard toolbar to pop-up at the top, and then go to Tools -> Show toolbars -> Standard. Then from the standard toolbar, you can click on the 'Show Menu' button on the top right of the conversation window from the standard toolbar, and go to Tools -> Backgrounds ... . This wil take you to a screen where you can select a blank default background. However, dragging the picture may have changed the default background colour for the window, as it did in my case. To remedy this, you have to change the background colour to match the rest of the Messenger colour scheme, and you can do so by clicking on the paintbrush icon on the standard toolbar, and selecting a default background colour from there to make things consistent with the rest of the Messenger colour scheme.

How to recurlively delete a folder and its contents in PowerShell

2008-05-26T20:27:00.001-07:00

The command :

Remove-Item -recurse -force [directory name]

The really sad thing is that I googled this and there are no sites that explicitly state how to do this. The comparable command in bash :

rm -rf [directory name]

Yet one more reason I hate Microsoft and PowerShell in particular.

RESTful web services with JAXB 2.0

2008-05-26T19:04:00.001-07:00

In case you're not familiar, JAXB (Java API for Xml Binding) is an API developed by Sun for binding between XML and Java beans. The first incarnation, 1.x, was interface+implementation based like XMLBeans from the Apache Project. The second incarnation, 2.x, is annotation+pojo based. It's a very useful API, especially when combining it with Hibernate (and the Java Persistence API) to make restful web services. But, that's not really the point of this post. The point is that I've been using JAXB 2.0 for web services much in the past, but this is the first time where I've had to map the "class hierarchy per-table" pattern used by Hibernate into XML. As it turns out, JAXB 2 has very good support for this pattern. You can read more on this at the java.net project page for JAXB.

Logging for tests using the Spring Framework

2008-05-08T12:02:00.000-07:00

It's been a while since I've posted to the blog 'cause I've been so busy, and it seems fitting that this be a good way to resume posting, as this issue has pissed me off quite a bit and been a major thorn in my side for the longest time.

The Spring Framework has some pretty good support for creating test classes for your application, however it by default does not properly initialize log4j logging when doing tests, and I found out today why. When running your application in a Servlet container, you'd configure Spring logging in web.xml. However, when running in a standalone context, Spring has no way of knowing how you want logging configured, so it leaves it up to log4j to configure itself. On that front, you have to realize what log4j's default configuration strategy is : reading a 'log4j.properties' file from the root of the classpath. Once this hits you (and it took me a while), getting logging running for your test cases becomes a simple matter of placing a valid 'log4j.properties' config file in the root of your test classpath, and logging starts working properly, so now you can read those pesky hibernate generated queries off your test log .

Determining which version of Ubuntu you're running

2008-03-31T09:16:00.000-07:00

At our company, we use Ubuntu to run our servers. Sure, there's other distributions with potentially better performance, but none so easy to setup, and that was the deciding factor in choosing which distro was going to run our systems because I had to set it up, and since I'm both developer and system administrator (*very* small company), I really don't have time to fuck around and I need to make things as easy on myself as possible. To that end, I've decided to start writing tutorials on everything I learn how to do, and I needed to make sure I note which version of Ubuntu / Linux I'm using. To that end, in order to determine which version of Ubuntu you're running, run this :

cat /etc/issue

I learned of this little trick from here.

*Edit (2008-10-29)*
A much better command for determining the current Ubuntu version you're running :

lsb_release -a

More random MySQL glitches

2008-03-30T15:30:00.000-07:00

I ran into the following exception recently when moving an application from one server to another :


java.lang.NullPointerException
 at com.mysql.jdbc.StringUtils.indexOfIgnoreCaseRespectQuotes(StringUtils.java:959)
 at com.mysql.jdbc.DatabaseMetaData.getCallStmtParameterTypes(DatabaseMetaData.java:1296)
 at com.mysql.jdbc.DatabaseMetaData.getProcedureColumns(DatabaseMetaData.java:3670)
 at com.mysql.jdbc.CallableStatement.determineParameterTypes(CallableStatement.java:702)
 at com.mysql.jdbc.CallableStatement.(CallableStatement.java:513)
 at com.mysql.jdbc.Connection.parseCallableStatement(Connection.java:4422)
 at com.mysql.jdbc.Connection.prepareCall(Connection.java:4496)
 at com.mysql.jdbc.Connection.prepareCall(Connection.java:4470)
 at org.apache.commons.dbcp.DelegatingConnection.prepareCall(DelegatingConnection.java:275)
 at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.prepareCall(PoolingDataSource.java:292)

As it turns out, there was a permissions issue whereby the function was created with a line specifying the definer and this causes the 'SHOW CREATE FUNCTION' call to fail and return null unless the user calling the statement has permissions for select on the 'mysql' database. Some recommended solutions (that I didn't get around to trying) were changing the call to 'prepareStatement' as opposed to 'prepareCall'. The issue is logged as MySQL Bug #24065. Lately it just seems as if I keep hitting every bug out there, and can't get anything done. I hope this helps somebody.

Commons VFS (FTP) problems

2008-03-30T14:28:00.000-07:00

I recently had a problem with Commons VFS whereby I could log into an FTP server and list the contents of the initial directory, but no contents were shown, nor could I otherwise resolve any of the contents or subdirectories. As it turns out, there's a problem with FTPClient which is part of the Commons Net package, and the basis for the FTP module of Commons VFS. After logging in, if the VFS client tries to set passive mode and the server doesn't respond that it allows passive mode (ie PureFTP specifically), then the client just continues as if there's no problem instead of throwing an Exception. The result was getting an error message like "The file root [ftp://myuser:*****@localhost/my_sub_folder] is not a folder". Since I wasn't able to get PureFTP successfully configured to allow passive mode, I was able to resolve the problem by forcing the code to use active mode instead. Fortunately this wasn't a problem since the FTP server in question was on the same machine as the code trying to access it (ie not across the boundary of a router).

Software developers are arrogant assholes

2008-03-03T08:32:00.000-08:00

Like the title of this post says, software developers are arrogant assholes. I can say that because I am one. The reason I say this is that I've been Googling like a mofo for articles related to RESTful development and RESTful web services servers using Ruby on Rails, and I must say that it's a real pain in the ass. The vast majority of the articles I'm able to find are in blog format, and are (as is typical with blog-style articles / tutorials) very vague, and don't provide enough detail for a learner to get going. Essentially, they make the assumption that you already know something about Ruby on Rails, and it's a rather hypocritical assumption for a tutorial / article to make. I will admit that I've been somewhat guilty of doing this same thing in my own blog whenever I've provided code examples, so it stops now. Any future "tutorials" on this blog are going to be as complete as possible, time constraints at work be damned. Vagueness cannot be allowed to continue.

Setting up your own domain in (Ubuntu) Linux

2008-02-26T18:39:00.000-08:00

...is easier than you actually think. I managed to get a working domain name for our company's servers going using tutorials and information collected from here, here, here and here.
I'd love to be able to write a coherent guide based off of the information from these pages, but I'm just too busy at work to do it at the moment.

A Quick note on SFTP jails

2008-02-25T18:07:00.000-08:00

SFTP jails are uber useful and I've blogged about them before, but one crucial piece of information that I've been missing has been where it all started. This thread on the Ubuntu forums was the base for all of the work that I've done and it has made it super easy to setup jails here at work, which is a blessing due to the frequency with which we move files and make changes. I'm finding more and more that I owe an increasing amount of thanks to the people and posts on forums, both Ubuntu and others, so thanks.

Understanding Linux ACLs

2008-02-14T09:32:00.000-08:00

Ok, so for a while, I've had an SFTP jail setup within our company for our clients to connect and dump batch files to our systems. It later became necessary to have an administrator user for our staff to be able to go in and read any of the files from any of the clients without having to use a bunch of different logins to login as each individual client. This makes sense as doing so would be far too cumbersome. This is where ACLs came in. I learned what I had to so that I could get going, but that wasn't very much (forunately at the time, unfortunately later). I've since learned a couple of interesting things since then:

There are two types of ACLs in Linux :

The access ACL which controls access to files and directories

The default ACL which applies to directories only and acts like a template for newly created files and subdirectories within that directory.

The former I knew about right from the beginning (pretty obvious), but I didn't really know the proper name for what I was manipulating. The latter I learned about today, and had previously assumed that Access and Default ACLs were one and the same (I didn't know about the distinction because it wasn't mentioned in the fucking
man
page.

Today's karmic balance

2008-02-13T14:20:00.001-08:00

It's pretty rare that I have something to blog about within a couple days of each other, never mind on the same day, so I suppose that this post is the karmic balance of the other for today. I just found out yesterday that because one of our processing partners had their douchebag of a contractor ditch on them, I now have to maintain two systems and integrate them both together. That in and of itself is kinda stupid. But at least this'll be my first production interaction with Ruby on Rails, and I'm actually really looking forward to the experience. I've been meaning to get myself up to speed with Rails, but between feeling tired after I get home and working so damn much that I don't have the time when I get home, I haven't really had the chance. It's going to be an interesting coming couple of weeks.

Use DTDs, they're there for a reason

2008-02-13T09:11:00.001-08:00

<rant>
As the title of this post states, if you're a web developer, use a fucking DTD. They exist for a reason, that reason being to allow web browsers to have a reasonable expectation of the content and layout of your website so they can deliver the desired experience to your user. If you're a web developer and you don't know what a DTD is, then you're in serious fucking trouble. In the event that you don't know, DTD stands for Document Type Definition, and it's a single line of XML that goes at the very beginning of HTML documents and almost at the very beginning of XML documents (if they're validated by DTDs). An example of a DTD :


<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

That DTD is for a strictly validated XHTML 1.0 compliant HTML document. This means that it's strongly validated and web browsers can expect everything after it to be well-formed quality. That's right bitches, I use a good version of HTML and I'm standards compliant like a mofo.
</rant>

Sorry, but I just had to get that out. Lately, I'm being inundated with pages from our partners that don't use DTDs, don't validate to anything, and are just generally horrible markup. Now, normally, a person wouldn't need to care, but when your partners are lame and you have to scrape their pages for data, the lack of developing to a DTD becomes a giant pain in the ass.

A letter to Apple

2008-02-12T09:42:00.000-08:00

Yes, these things are cliche, but whatever, screw it.

Dear Apple,

In case you haven't figured it out from my buying patterns, I've stopped buying music from you that isn't DRM-free. This is because, as Steve Jobs has admitted himself, DRM is archaic, and frankly doesn't work. What your company doesn't seem to realize is that if you want to continue your high level of sales, (especially to technophiles / audiophiles such as myself) you're going to need to get going on casting off the DRM that exists on your considerable library of music. I steadfastly refuse to purchase any music that is not DRM-free. I want to be able to play it anywhere on whatever device I may happen to own, and I'm not going to cave into unreasonable demands of an already clearly greedy music industry that refuses to evolve and adapt to technological reality. If I can't purchase music through large corporations such as yourself, I'm going to look elsewhere : smaller sites, independent labels, and independent artists who publish their music for free. So, until you get your head out of your ass : screw you, you're not getting one thin dime from my pocket.

Beginning with SQL Server

2008-01-30T09:58:00.000-08:00

Given changes in the environment in which our business does it's business, I've had to start learning to use SQL Server and various other Microsoft technologies lately. A quick note on connecting to a foreign SQL Server instance (ie another company's ...) :

Create an alias for the database so that Enterprise Manager has something to work with

Open up the Client Network Utility which should be included in the Start menu programs group for your SQL Server installation.

Start -> Programs -> Microsoft SQL Server -> Client Network Utility -> Alias (tab) -> Add... (button).

Under 'Server alias' enter an easy to remember name for the connection. Under 'Network libraries' select the TCP/IP option. Under 'Connection parameters' enter the DNS name or IP address under 'Server name' of the server you wish to connect to. Change the option for 'Dynamically determine port' if the server you're attempting to connect to doesn't run under the standard port of 1433 for SQL Server. Hit 'Ok' to save your settings, then 'Ok' to close out of the Client Network Utility.

Create a new SQL Server Registration to add your SQL Server instance to the Enterprise Manager

Open Enterprise Manager. It should be in the same programs group in the start menu as the Client Network Utility. Once open, create a new SQL Server Group if you haven't already got one (or haven't got one that you want to add your new SQL Server instance to). Then add a new SQL Server Registration. You should be prompted by a Wizard at this point. (If you're not, then you've probably already disabled the Wizard and don't need to be reading this tutorial.) In the wizard, click on 'Next' to proceed to the SQL Server selection screen. Under the list of available servers, you should see the alias you just created in the previous step. Select it and click on 'Add >' to add it to the list of added servers, and then click on 'Next'. After this, you should be prompted for credentials, and the rest is pretty self explanatory.

Hibernate !!!

2008-01-29T09:48:00.000-08:00

So, this is the first post of the new year, and regrettably, it's one written in anger over Hibernate ... again.

This time, it's regarding the Criteria API. It seems that if you want to be able to use the Criteria API for querying entities with composite properties (or ids), you have to use a special notation. Say you have the following entities :


public class Cat implements java.io.Serializable {
  @Embedded
  private CatMetadata metadata;

  @Id
  @Column(name = "id")
  @GeneratedValue(strategy = GenerationType.IDENTITY)
  private long        id;

  //constructors, getters, setters, etc omitted for clarity
}

@Embeddable
public class CatMetadata implements java.io.Serializable {

  @Column(name = "name")
  private String name;

  @Column(name = "age")
  private int    age;

  @Column(name = "colour")
  private int    colour;

  //constructors, getters, setters, etc omitted for clarity
}

If you wanted to get all cats with the name "Spunky", your first instinct would likely be to do something like this (given what's in the Hibernate documentation):


Session s = sessionFactory.openSession();
List cats = s.createCriteria(Cat.class)
                  .createCriteria("metadata")
                  .add(Restrictions.eq("name","Spunky"))
                  .list();

s.close();

for(Cat c : cats) {
  System.out.println(c);
}

...and your first instinct would be quite wrong. The above code would work just fine if

CatMetadata

were an assocation rather than embedded. Instead, if

CatMetadata

is embedded, you must do the following :


List cats = s.createCriteria(Cat.class)
                  .add(Restrictions.eq("metadata.name","Spunky"))
                  .list();

Note the difference in the restriction that's added to the criteria. The desired end property must be expressed in dotted notation relative to the embedded object. This little inconsistency tripped me up for hours trying to Google a fix for it. Why Hibernate couldn't just stick with a consistent scheme for dealing with nested properties is beyond me (and given time constraints at work, I really don't have the time to go figure it out).

Cool MySQL tricks

2007-12-20T10:13:00.000-08:00

It's been a while since I've written a blog post, and I figured this was a good topic to write about. For the most part, I don't use SQL. There's numerous ORM (Object-Relational Mapping) tools and frameworks out there (ie Hibernate) that abstract away all the boiler-plate CRUD code. When I do have to use SQL (be it on the command line or with a query browser), I'm not afraid to dive right in, and it's interesting to learn new things about SQL because there's so much to learn and it's such a ridiculously powerful query language.

I had to update the configuration in one of my databases, but the configuration required inserting numerous new lines into a table for user roles. I could've written a stored procedure to do it, but given my (admittedly somewhat limited) knowledge of SQL, I figured this would take too much time and I didn't want to do it. (This is what I like to call constructive laziness). I kept Googling and found MySQL's INSERT INTO ... SELECT syntax, which allows you to conveniently insert the results of one query into another table. I'm sure that most other database implementations out there have a similar convenience syntax, but I happen to work with MySQL so there you go. It took what would've been a numerous line script with looping and separate SELECT and INSERT statements down to a one liner. If you use MySQL (especially with LAMP / AJAX) I recommend that you read about it.

Deleting .svn folders

2007-12-03T14:16:00.000-08:00

Subversion is a great tool, but unfortunately it has to store metadata somewhere, and like all other code repository tools, it stores them in metadata directories ... for each directory in whatever project you're working with. I recently found that I have to copy parts of my project documentation out for some of our clients to access our APIs, and I wanted to quickly delete all the SVN metadata and trim certain other files where necessary. I figured the quickest way to do with would be a simple command. Windows PowerShell provides a good (though not great, and certainly not as good as bash) way of doing it :


Get-ChildItem -Recurse -force |where {$_.PSIsContainer -AND $_.Name -match "svn"} | foreach ($_) {remove-item -force -recurse $_.fullname}

Two Google queries yielded the above command from this blog. Happy reading.

Resolving logging issues with Tomcat

2007-12-02T12:50:00.000-08:00

An idea for resolving the logging issues in Tomcat just occurred to me. I should probably be putting the logging jars in the individual lib directories for each of the webapps. I'll have to give this a try and post my results.

Onward to Eclipse Europa ... and then right back

2007-11-29T14:07:00.000-08:00

Ok, so after Eclipse Europa (3.3) has been out for quite some time, I decided to give it a try. I was shocked to discover a lot of the changes that had happend :
it seems that the Eclipse foundation has become Red Hat's bitch. There's a ton of new Red Hat sponsored / produced editors (none of which are very good.) There's a ton of useless JBoss integration. The editor overall is slower. All my hotkeys (save one or two, literally) were standard with installation in Eclipse 3.2, and are now broken in Eclipse 3.3. I'm severely disapponted after using Europa for three weeks. The only good part about it is that access to my SVN repository was considerably faster. And it seems I'm not the only one who feels this way about Europa. Get it together Eclipse.

Oh, and the thing that brought me to writing this post in the first place : the XML schema editor is flat out broken. What the fuck ?!

Spring, Tomcat and memory

2007-11-15T12:29:00.000-08:00

As my application has been growing more and more in functionality, it has also been growing in its memory footprint. To that end, I've started getting OutOfMemoryErrors because the heap has started overflowing. That's really not a problem if you have access to the external Tomcat server, you can just increase the heap size using "-Xms128M -Xmx512M" (or other applicable sizes) on the command line, or do this via the GUI if you're running Tomcat in Windows. However, it's not as obvious if you're using Tomcat for debugging within the Web Tools Platform plugins for Eclipse. I started running into this problem recently and it ground my development to a halt until I was able to fix the problem for WTP in Eclipse. WTP passes its arguments to the Tomcat server via the Launch Configuration. To get to it, right click on the Tomcat server in the Servers view -> 'Open' -> 'Open launch configuration' -> 'Arguments' tab -> 'VM arguments:' text box, then add the "-Xms128M -Xmx512M" segment to the end of the parameter list.

Actual plugin development for Maven 2

2007-11-03T14:23:00.000-07:00

Ok, so after Googling around and hours of patient command line testing (yeah, I know, it's horrible, you don't have to tell me) I managed to complete my ToLDya plugin for generating TLD files. Hopefully this plugin will be a great aid to people developing JSP tag libraries other than myself. But along the way, I learned a lot about plugin development in Maven, and I'm quite certain that I've got a shitload more to learn.

Starting out with a barebones Mojo from the Maven provided archetype literally doesn't get you much. It gets you just enough to plug into the Maven framework so that maven can actually run your Mojo, but that's about it.

The AbstractMojo provided by Maven is pathetic. It gives you a logger, and that's about it. By default, it does not give you many of the things a plugin is quite likely to want (more about that later)

Despite the fact that Maven 2 was supposed to be the "lessons learned" version of Maven, I don't think that the Maven developers learned much at all. Maven is grossly behind the times, still relying on XDoclet annotations and pre-JDK 5.0 compatibility. Yes, there is something to be said for keeping things backward compatible (especially in a corporate environment, I know), but at some point you have to move on and do better, in this case : getting up to date with the latest JDK (1.6.03 at the time of this writing).

If you need anything (ie from Maven) while writing a Maven plugin, you have to specifically request that it be injected for you. (See the Maven documentation, this is the one area where they're actually good about documenting things and helping out developers)

If your plugin needs to access any of the classes in the project on which it's run, you need to load them yourself with your own classloader. Maven will not give you one (which is pretty ridiculous to my mind).

Here are some important bits of knowledge for doing anything with a Maven plugin:

If you need access to anything from the project (ie any information stored in the POM), you'll need to include the following dependency :
```
<dependency>
 <groupId>org.apache.maven</groupId>
 <artifactId>maven-project</artifactId>
 <version>${maven.version}</version>
</dependency>
```
In my current POM, the property 'maven.version' is set to 2.0.7. You'll then need to have a property in your plugin Mojo called 'property' (or whatever else you find suitable) and annotate it like so (from within a Javadoc comment of course):
```
 * @parameter expression="${project}"
 * @required
```

As mentioned previously, if you want to load any of the classes that are in the project on which your plugin is to execute, you have to load them yourself. The same goes for any of the project's dependencies. In order to do this, you'll have to get the list of dependencies (compile, test, runtime) from the MavenProject object ('property', remember?) The MavenProject object has a property called 'runtimeClasspathElements'. This gives you a list of strings that are fully qualified file system paths to the classes in the ${project.build.outputDirectory} as well as all of the dependency JARs on which the client project depends. You'll then have to load them yourself. I did so with a URLClassLoader (part of the JDK). I used the following function for creating the classloader :
```
private static URLClassLoader getDependencyClassloader(List dependencies) throws MalformedURLException {
 List classpathUrls = new Vector();
  
 URL url = null;
 for(int index = dependencies.size() - 1; index >= 0; index--) {
  url = new File(dependencies.get(index)).toURI().toURL();
  
  classpathUrls.add(
   url.toExternalForm().endsWith(".jar") ?
    url :
     new URL(url.getProtocol(),url.getHost(),url.getPort(),url.getFile() + "/") //add the '/' o indicate a directory   );
  }  
  URLClassLoader ucl = new URLClassLoader(classpathUrls.toArray(new RL[] {}), Thread.currentThread().getContextClassLoader());
 
 return ucl;
}
```
Once you have this class loader, you can use it in the long version of Class.forName() to load any classes you may need, as well as perform any loading logic you need with the .getResources() functions on the classloader.

I'll be doing more plugin development in the coming months I'm sure, so I'll try to post what I learn here, but that's about it for now.