Linux : the more you know ...

I'm somewhat embarrassed to admit this, but I've just discovered Linux ACLs, which are apparently in every kernel 2.6+, and has plugins for 2.4-. I guess a really linux savvy person should know these, especially since they're incredibly useful and give so so much more power over the default User - Group - World permissions built into the file system.

JSP Tags

They're oh so useful for encapsulating small bits of reusable logic, but, here's the kicker : when they're compiled, only one instance of a tag is generated for a given page, so you have to design them to be stateful and ensure that you override the 'release' method when inheriting from TagSupport or BodyTagSupport. It took me a couple of hours to figure this out, and sadly, it was for the second time. I guarantee there won't be a third after this.

Moar (sic) Hibernate

OK, so once again in my (seemingly never-ending) struggle with Hibernate, I have stuff to report that's mainly being posted here for my own reading so that I can reference this shit later. This time, I've decided to abandon Xdoclet and just go with Annotations. They're so much easier and I don't have to deal with Xdoclet's quirks any more (like ridiculous parsing exceptions between versions).

If you want to use annotations to do a bidirectional OneToMany assocation with list-based semantics, here's the annotations you use :

Parent.java (equals, hashCode, getters/setters removed for clarity):
@Entity
@Table(name = "parent")
public class Parent implements Serializable {

private static final long serialVersionUID = -1989884660562516228L;

@Id
@Column(name = "id")
@GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;

@Column(name = "name")
private String name;

@OneToMany(mappedBy = "parent", fetch = FetchType.EAGER, cascade = {CascadeType.ALL})
@Cascade({org.hibernate.annotations.CascadeType.ALL,org.hibernate.annotations.CascadeType.DELETE_ORPHAN})
@IndexColumn(name = "idx", nullable = false)
private List children;
}


Child.java (equals, hashCode, getters/setters removed for clarity):
@Entity
@Table(name = "child")
public class Child implements Serializable {
private static final long serialVersionUID = -6414526385302360120L;

@Id
@Column(name = "id")
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;

@Column(name = "name")
private String name;

@ManyToOne
@JoinColumn(name = "parent_id")
private Parent parent;

@Column(name = "idx")
private int index;

public Child() {
super();
}

public int getIndex() {
return this.parent.getChildren().indexOf(this);
}

public void setIndex(int index) {
}
}

Note that you need the 'index' pseudo-property in the child class in order for hibernate to properly persist the ordering of the elements in the collection. The setter for the index property should do nothing, the getter should determine the object's placement in its parent, and you'll need the field with annotations in order to get hibernate to read everything properly. The name of the column in the annotation for the index property MUST be the same as that specified in the parent in the @IndexColumn property. They aren't very clear about this in the Hibernate Documentation (once again.) They mention a similar structure for the .hbm.xml mappings in a faq on the main site for hibernate (not the hibernate annotations faq), but they don't explicitly mention this anywhere for Annotations. You can read the mention of it for .hbm.xml files here. I'm sure I'll have to post more about annotation configurations later, but that's all for now. And if you're wondering, the misspelling in the title of this post is an inside joke.

PS. Note that in addition to the OneToMany and IndexColumn annotations on the child collection in the parent, you also need to have a @Cascade annotation in order to properly remove orphans from the database when deleting children from the child collection in the parent. You can also find the table that gave me my final clues on Darren Hicks' blog

More Maven2

Today I had the chance to build a new project right from scratch for a quick and dirty app that needs to be spat out right quick. I figured that this would be a good chance for me to start really getting on the band wagon with Maven2 and using it full force in my development environment.

Building on what I've mentioned about Maven2 previously, here's something else that needs to be mentioned:
1. Adding mvnrepository.com to your list of external repositories :
<project>
|
|
<repositories>
<repository>
<id>mvn-repository</id>
<name>www.mvnrepository.com</name>
<url>http://www.mvnrepository.com</url>
</repository>
</repositories>
|
|
</project>

Interesting geography lessons

This probably sounds pretty off-topic for a tech blog, but it's not that far off. Recently I had to populate our database with ISO data about other countries so that we could accurately ship to them, and in the process learned a great deal about foreign geography and postal lessons. The biggest of these lessons was that by comparison, Canada and the US have dead simple postal systems that nobody should screw up. For example, France's geographical (and therefore also postal) subsystem is broken down into 102 departments, which may be further broken down into communes and 'arondissements'. The first 2 digits of a France postal code indicate the department (synonymous with state or province in US/CA) in which the postal code is located, so that even though France's postal structure is more complicated than the US or Canada, that complexity is well managed and actually very easy to deal with (especially for the French since they're the ones that have to move the mail in that country.) Like France, Great Britain has a far more complicated system than either the US or CA.

For comparision, here's what happens in the US/CA postal systems. Each address is broken down as follows :
123 Any St
My Town ABC123 CA

Or, in a more grammar specific way (s+ indicates there may be whitespace, ? indicates optional):
[Street number][s+][Street address,s+]
[Municipality/Town/City,s+][s+][Postal code][s+][Two digit state / province code]
Now, in the US/CA, Postal codes are \d{5} for the US, [A-Z0-9]{6} for Canada, where every CA postal code follows the pattern A1A1A1 (ie alternating letters and numbers, starting with a letter). Very specific and easy to understand.

Now, in Great Britain, a standard sample address is as follows :
[Street number][s+][Street address,s+]
([Locality][s+])?[Municipality/Town/City,s+]
[Postal Code,s+]
[County,s+]?

And to boot, GB also has Boroughts, Metropolitan Districts, and Unitary Authorities in addition to the counties. And in GB, a postal code matches [A-Z0-9]{6,7}, with no specified interleaving whatsoever, so you can have any possible combination of letters and numbers.

On top of that, there's also the issues of cultural differences within the country at hand. In Canada, there's pretty much English and French addresses, and if an address is French, it stays French (same for English), the point being that it doesn't get Anglicized or Francocized at any point within the postal system. In GB however, there are English and Welsh versions of all the addresses in Wales.

Now, this much complexity in GB (as compared to US/CA) is fine if you live in GB, but if you live outside the country and you're trying to ship to it, and be reliable, GB really blows.

It's there for a reason

I think that one of the things people realize when they start a new job (I haven't) is that the way things are done currently are done that way for a reason (most of the time, though definitely not always). Some past experience has been the basis for changing the way things were done before, and so now the new way of doing things is done to avoid some past (most likely bad) experience.
People then proceed with their work, and if they're not a fan of the way things are done, they do it their own way. In many cases, they'll encounter a previously encountered bad experience and realize why things are done they way they are. This has happened to me several times. And this happened to me right now while using Hibernate. I finally realized why they have the "saveOrUpdate" method on Session objects. Love those epiphanies.

Branching out

I recently stumbled across a glowing review of the Wicket framework that made me want to research it and try it out. The author's main argument for Wicket was that it's pretty much just HTML and Wicket, as opposed to the combination of Spring, Spring Webflow, JSP, and tons of other stuff thrown into the mix. Unfortunately I really don't have the time to mess around with it right now, but I'll try to be back on this topic later.

*EDIT* : To add to the list of things I should really learn, Seam's going on the pile. It looks very cool and it's made by Gavin King, the maker of Hibernate. (I know I've bitched about how bad the Hibernate documentation is on numerous occasions, but really, I can't blame him since I hate doing documentation just about as much as if not more than the next guy)

Better idea for multi-versioned webapps

A while ago at our company, a request was made to me to create a separate instance of one of our webapplications for a select group of customers. In doing this, I had to create a separate instance of the backing database and copy over much of a preexisting version of the application's database which was simply configuration data.

It occurred to me today while working on another part of the application that I really should have separated the configuration from the data. The hard part about that is keeping security intermingled between data (for audit trailing) and keeping security intermingled with the configuration data (as part of configuration). I'm still searching for an answer to that one.

Excel and its annoying quirks

Excel has this annoying quirk of propagating unwanted hyperlinks in the background of spreadsheets, and in our organization, it gets to the point where it infests every cell of the sheet. But, I did find a fix (sorta). Press Alt+F11 to go into the VB editor in Excel, go to Insert -> Module, and paste this in:

Sub RemoveHyperlinks()

'Remove all hyperlinks from the active sheet
ActiveSheet.Hyperlinks.Delete

End Sub

Save it, return to the worksheet, press Alt+F8 to open the macros menu (Tools -> Macro -> Macros otherwise) and run the Macro of the same name as the subroutine above. This should delete all your hyperlinks. Unfortunately, this will most likely remove all other special formatting in the cells as well. If you have a really large worksheet, it will most likely cause Excel (XP and earlier at least) to crash, so you should really save your work immediately before you do this (or before you add the Macro to the sheet).

Taking a REST

Lately I've started to get back into research mode in the hopes of making my development cycles shorter, especially since the end of a large sprint is in view. With that in mind, I find myself being continually reminded of the increasing demand for REST-ful applications (ie web applications that make use of the REST HTTP method as opposed to POST or GET). I'm finding it's at least something I shoud be familiar with as a web developer, even if I don't use it.
*EDIT* : Look up UML with Rational Rose

Spring Batch Processing

No, it's not batch processing that takes place in the springtime (although that does happen at our company). It's actually a new subproject of the Spring Framework meant to provide a convenient , common framework for batch processing, and it might be just the thing I need to help me out. Oddly, I found out about it while I was searching for the documentation for the Spring-binding package.

Quirks with Maven2 and m2eclipse

If you're using m2eclipse (which I've found to be quite useful), you should be aware that it doesn't always check the settings.xml file in the local repository or in the maven home dir, and so that pesky command-that-you're-trying-to-get-working-that-should-be-working-after-you've -updated-settings.xml-but-still-isn't-working may have to be run from the command line using 'mvn' rather than from within eclipse.

Logs and RSSH

I've learned the hard why today why logs are uberuseful (once again) and that RSSH is a pain to debug if you're having trouble with it. For future reference, check '/var/log/messages' as the first thing you do when having trouble with SSH, especially RSSH.

For those not in the know, RSSH stands for Restricted Secure SHell, and is very useful if you want to create jails for people who have to transfer files between their boxes and your server. However, it's very hard to debug because when you log in, you don't get any messages, you either get the "This is a an account restricted by RSSH" message or you get the ever so friendly and useful "Connection closed" message. As it turned out, the passwords for several of my users had expired today and this created a huge issue when they weren't able to upload batch files to our server. After spending just over an hour finding that the problem was related to SSH login, I received a very helpful message from one of the people who operates our server showing me a grepped slice of the /var/log/messages log saying that the password had expired for several of the users : coincidentally, the only users that I had tried troubleshooting the problem with.
The point of all this: check '/var/log/messages'.

*EDIT:* Today is officially known as Bad Monday for me.

Fucking Hibernate

God f*ck*ng dammit, I'm so freaking tired of fucking Hibernate's shitty ass documentation wasting my time:

In the event that you ever have to query in HQL on a component and that component contains ManyToOne elements (ie entities, rather than strings or numbers), you have to query on properties of that element rather than the element itself and have hibernate query for the ID.

...which is lame, considering Hibernate is so good about that kind of thing everywhere else.

Maven ... and why you should use it (2, not 1)

For the longest time during my developer's journey of discovery while learning how to build web applications, I've come across references to Maven and how it's a build system like Ant, but supposedly better. I've also heard untold numbers of references to it being frustrating and very difficult to use, largely due to shitty documentation. It was for this reason that I avoided learning it and just stuck with my current build process, which is just using Ant, exporting everything with eclipse, then manually uploading things to the server. Or in the case of dealing with a library I've made that I share among various web apps, compiling everything, going through Eclipse's JAR export wizard and copying files around. No more.

I must have came into Maven at the right time because I've found there to be lots of documentation (or at least a sufficient amount for myself) to learn it and get going with it. I've also converted my library from just using Ant to using Maven to build it, test it, package it and deploy it to a central server within my company that will now be the new Maven repository for the library. From now on, my applications can retrieve it from a central repository for me automatically instead of me having to manually export the library, copy it around, and maintain various versions of it in CVS. And I'm thrilled that this is going to make my life easier.

I did however run into a few quirks when getting Maven going. They're as follows:
1) You should really align your project to have the same layout structure as the one recommended by the Maven documentation. It'll get things going faster and you'll have far fewer build problems.

2) If you use Java (1.)5 or greater, you may need to manually insert configuration settings in order to get your (tests in my case, and possibly other) code to compile properly since Maven strives to achieve compatibility by default and will likely try to compile your code for older VMs such as 1.3. To configure Maven to use 1.5 (or whatever) by default, put the following snippets in your pom.xml file :

<project ... >
<!--
|
|
-->

<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
</plugins>
</build>

<!--
|
|
-->
</project>

3) Deploying your library (or other code) to a Maven repository is wicked easy, especially since Maven doesn't require any special software to set one up: a Maven repository is just a folder containing libraries and metadata with a prescribed folder layout and naming convention. The code for deploying to a server is as simple as this :

<distributionManagement>
<!-- This repository depends on the following settings.xml file being in the local maven2 repository
and a .ssh folder being created under ~ (or %HOMEPATH%) in Windows:
<settings xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">

<servers>
<server>
<id>[REPOSITORY NAME]</id>
<username>[REPOSITORY USER]</username>
<password>[REPOSITORY PASSWORD]</password>
</server>
</servers>

</settings>
-->
<repository>
<id>[REPOSITORY NAME]</id>
<name>My Maven2 Repository</name>
<url>scp://mybox.mydomain.com/path/to/repository</url>
</repository>
</distributionManagement>

Maven has been pretty simple to set up and use so far, once you've read the documentation and you have at least some idea of what you're doing. I also recommend getting the m2eclipse plugin if you're going to be using Maven with Eclipse. My next steps are going to be getting Maven to compile my XMLBeans for me and deploying those as well.

The head is venturing out of the ass

For the longest time, I've had my head up my ass: I've been content to build my projects with ant and the (very few) ant targets I've really needed: reports, xmlbeans and hibernate. (And fiddling around with jspc to precompile my JSPs). But now I've found that using Xdoclet2 for Hibernate is no longer acceptable since a certain bug has forced me away from it, and now I'm finding that it's time to move on to real build systems like Maven2. I've been scared away from it before by the fact that I've heard that it's exceedingly poorly documented and that it's been a total pain and has load of incompatibilities, but apparently it's been getting better and now I'm forced to learn it in the hopes of saving myself some trouble. I'm also going to have to learn EJB3 Annotations and their Hibernate extensions, but that's another matter. Let's see how it goes.

My list of stuff to learn

Frequently I find that there are dozens of technologies I want to learn, but really just don't have the time. As a list to myself (and anybody else who might care) of the stuff that I want to learn, but just haven't had the time, here's a short enumerated list:

• FreeMarker
• Scheme
• Groovy
• Grails
• Ruby
• Rails
• Python
• JPA (Java Persistence API)
• EJB Annotations (primarily for Hibernate)
• XSLT

And that's just the stuff I can remember off the top of my head as I write this. I'm sure that I'll add more later.

Reading blogs pays off

I've recently been trying to read more and more developer blogs in the hopes of improving myself as a developer. One of the blogs I came across is Brian Burridge's, and he recently had a good post about a little utility called Denim. It's a storyboarding/diagramming tool that was originally meant for web developers to quickly model their sites, but I see no reason why it can't be used for user interfaces in general. The program's made in Java so not unexpectedly) its Swing UI is ugly as shit, but it's so functional it's insane. This is how I'd make an interface (though I'd try to pretty mine up), but they got the basics right: functionality before beauty. If you're a serious developer, I think you should at least give it a shot and play around with it. It's free, open-source, and made at an academic institution. BTW, I jacked the latter link right from his page to save anybody reading this some time, not to take away from his blog, which I think you should check out.

The difference between links and forms ... at least for Spring

This post is really just a mental note to myself: the difference between links and forms is that links do not submit form variables. (Duh!) Therefore, you can't bind varibles in forms to Form Backing Objects in spring if you're going to use a link to do the transition rather than a form button.

(Web)Harvesting the web

I was recently assigned a task whereby I had to obtain data from the website of one of our commercial services, but the website is completely Web 1.0 and has no APIs of any kind that
we could hook into. To give you a bit of context, the site performs transactions on our behalf and we need information about those transactions. As my boss saw it, there were only two solutions really available :
1) Have somebody sit at a computer and download the transactions every 15 minutes
2) Have a computer set up to run a screen macro to log in and download the transactions file every 15 minutes (better, but still no where near ideal)

I think without even realizing it, my boss gave me the idea for option three. He continually mentioned the idea of screen-scraping the page to retrieve the information. Traditionally this means taking a visual representation of something and extracting information from what's essentially a picture. After doing some reading, I interpreted his suggestions to mean that I should find a way to do a web-scrape on the page (subtle difference). After doing some research, I found the perfect library for doing it in Java.

It's a project called WebHarvest, and it's fairly simple yet really powerful. You start off by writing a Scraper configuration in XML, load it in code, run the scraper, and it will store your data for you in the Scraper to retrieve when need it. The library itself works by doing either POSTs or GETs to a page, taking the response data, (almost certainly) doing a transform to convert the HTML to well-formed XHTML, and then running XPath queries and regular expressions on the result to get the data you need (ie rows of a table). It's incredibly powerful and if you need a solution where you want to automate logging into a website and retrieving data, then this is a great way to do it.