MySQL character sets, the end

Many times in the past I've tried to get our foreign MySQL server to properly store and handle unicode character sets. Despite the fact that I repeatedly set 'default-character-set=utf8' all over the config files and set the server and client character sets and collations in my.cnf, it still wouldn't handle them properly. Here's the kicker : to make absolutely sure that the server uses only server settings for handling character set (and sets 'fuck you' to whatever the client requests), you have to use the '--skip-character-set-client-handshake' argument when starting the MySQL daemon (server). This is what finally got it going for me. I hope this post helps somebody out someday. (Given my rate of forgetting things, it's likely to be me)

<rage>MOAR HIBERNATE! :@</rage>

God damn, I'm getting so fucking tired of Hibernate and its quirks. Here's a new one regarding Criteria queries :

When you create a criteria query, you almost invariably have to specify a result transformer of Criterion.DISTINCT_ROOT_ENTITY (that's a quirk, but not the topic of this post). When you have an entity on which you want to build a criteria query and you want to limit the number of search results (ie distinct root entities), things get really tricky. Specifying 'setMaxResults' on the query affects the number of rows returned from the database that are actually inspected. Therefore, if there are any joins on your entity that have collections, this will cause a fetch with an outer join strategy to generate an excessive number of rows and affect the results when using a maxResults setting. In the case of using embedded properties, specifying a fetch mode of SELECT will not override these (this is a glitch in hibernate). You'll have to specify the fetch mode manually in the metadata (be it XML or Annotations) permanently for the embedded class. I fucking hate Hibernate sometimes, I really do.

Spring AOP rage ...subsiding

Spring AOP is breaking my heart. Spring overall is a great framework and I love working with it. The idea of using aspects to interweave code and keep concerns separated and code clean is a wonderful though to somebody who loves to architect software, such as myself. But I swear, it feels like I'm hitting every bug in the book when it comes to using AOP in Spring. Certain pointcuts don't get matched properly, regexp based pointcuts get loaded when they shouldn't, the list goes on. I really want to use AOP to design the next big phase of my project that's coming up, but they're making it really hard to justify the decision to do so. You're making my heart cry, Spring.

Google Disappointment

...do it. You just might find a page with me ... pointing at Google on my screen.

Today I was messing around with some of the CSS styling in my API documentation for one of our company's merchant partners, and I found that some of the elements weren't quite right. I then remembered an article I had read on Digg regarding CSS reset stylesheets, and how Google and Yahoo both use them in their free APIs to provide consistent styling results across browsers, so I went to investigate using one of said reset stylesheets to help me out. I figured I'd Google it first (no pun intended), and upon not finding anything relevant quickly, figured Google would be smart enough to use their own reset stylesheets in their own pages. I pulled up the source on one of my query pages, only to discover a developer's horror : inline styling all over the place, and google didn't even use their own stylesheets anywhere in their main site! The code was horrible spaghetti. I ran the page through the W3C web site validator, and it didn't pass a single standard, ie HTML 4.01 / XHTML (any flavour). As one of the big Web 2.0 sites, I would have expected you to have higher regard for international web standards. I'm very disappointed in you Google.

A quick note on versioning

I've come to really admire the way the open source community has been versioning their products over the last couple years, especially in the java open source community, which I've found to be very bright and vibrant. Generally, the projects adhere to the following conventions (where 'x' is an integer) :

Version :
x.x.x - Version with major version, minor version, patch increment
0.x.x - Beta software, not to be considered ready for production use (generally, some projects have very odd development cycles and version conventions)
1.x.x - Version 1 (good for looking at, but you may want to wait for version 2, especially Apache projects *cough*maven*cough*struts*cough*)
x.x.x-Mx - Milestone beta version - has certain promised features according to the milestone version, but not the final version with that number
x.x.x-RCx - Release candidate beta version - has all the promised features according to the release plan for that version for the project, but is not considered to have been sufficiently tested

If anybody feels differently about my descriptions, please, by all means, correct me.

Weak Java

I rarely (if ever) use switch statements, mainly because if their use is required, generally a very poor design decision has been made. But sometimes they're the right choice. That said, I was forced to use a switch statement in some of my code today, and found that Java switch statements can only switch primitive ints and enums in the java language. Not even longs or shorts. That is the weakest shit I've ever encountered. Are you fucking retarded Sun ?

Sorry, I just had to bitch about it, because it's just that stupid.

I love Eclipse now more than ever

Eclipse has always been a very useful platform, with tons of little developer driven quirks that make the programmer's job so muche easier. I found this out especially lately when I've been trying to get into developing with C# in Visual Studio 2005, and I've been missing features that are just there in Eclipse, but you have to buy a plugin for with VS2k5. And here's the great thing about that whole situation : you can develop C# in Eclipse on Windows with a free plugin. But that's really all just a side note. The main point of this post is this : I just now observed an exceedingly useful feature that's been in Eclipse for several versions. Ever press Ctrl + Shift + T to open the quick loader for classes ? It has a text bar at the top so you can type in the simple name of the class you're looking for and it'll filter out the results. In the filtering options up top, there's the usual ? and * filters, but beside that there's also a spot that says "TZ - TimeZone" which I've never really noticed before, and I don't know what made me notice it today. I typed in TZ, and much to my surprise, a whole list of classes remained in the window, filtered, and they all had TZ capital letters in them, following the general naming conventions in Java. Intrigued by this, I typed in MPREF (ManualPaymentRequestEntryFlow) which is a class in my project that I just finished working on, and sure enough, it filtered the list to have that in it (it was the only class not filtered out) . I thought that was just the greatest filter I'd ever seen. And to build on it, if you use the content assist hotkey (Ctrl + Space) with the same scheme, it automatically fills in those classes for you. How cool is that !?

Moar (sic) hibernate !

Yet more hibernate fun facts that I wish I'd had before I started any projects : Hibernate has the ability to persist instances of java.util.Locale, java.util.Currency, and java.util.TimeZone. A sprinkling of a few bits of these pieces of code and their corresponding hibernate annotations could have saved me an assload of time spent on doing my own design, development and testing.

On a small side note, I found I've actually managed to come up with a much better and more appropriate solution for dealing with currencies than the java.util.Currency class, as it's woefully inadequate to the task of dealing with currencies in an enterprise system. I refer specifically to the way it deals with floating point arithmetic in its number of decimal places, rather than adering more closely to the ISO 4217 specification whereby currencies should have a base and an exponent when calculating a multiplier to go between base currency units and major currency units.

Rampaging bull software design

If you're into the software scene, then you most likely know that some of the big buzz terms in the last few years have been RAD (Rapid Application Development) and Agile development (keeping your code clean and maintainable, and incrementing your design and development process in very small steps). With ever changing requirements and very little chance to get the big picture ahead of time given the small amounts of information I get from my bosses, I have no choice but to follow an agile development process. As best as I can tell, I've been doing so ever since I've started here.

Unfortunately, this style of development has caught up with me : I'm finding that even though my code is fairly clean and very easily maintainable, there are memory leaks (I'm using Java, so this means that something's holding onto references and preventing the automatic garbage collector from doing its job). This means that I now get to go back with a profiler, analyze the running of my software, and find out where the leaks are and correct them.

This is going to be interesting.

Moar (sic) Spring Batch !

I received a pleasant surprise in my inbox today (which doesn't happen often), and that surprise was an email from the Spring Batch team notifying me that they'd just released Spring Batch 1.0-M2 to the subversion repo and made it public. This was great because it gave me a chance to download the code and play around with it, look at the samples, and generally get an idea of what's coming for Spring Batch.

In case you weren't aware, I've been looking forward to the release of Spring Batch for quite a while because it can help me a great deal at work, where I'm already using the Spring Framework for my web applications. Unfortunately, the M2 release doesn't appear nearly ready for a production environment, but it's a start. I want more ! faster ! now !

Why you should write good software

Today at the company where I work, our remotely hosted server went down suddenly. Nobody know why, and then the shit started to hit the fan, with emails flying back and forth, and phones ringing off the hook in search of an answer to that one timeless question, WTF ? Of course, everything filtered towards me eventually because I'm considered responsible for the server (even though I'm no where near it and if I don't have a network connection to it, I can't do shit). After several phone calls back and forth between our former partners (who happen to be in the same building as our ISP and still maintain our server), and our ISP, and our firewall technician (who, oddly, is outsourced by our ISP and doesn't technically work for them), it was discovered that the machine was alive, but not responding to any network traffic, so the quickest and simplest solution was to just get somebody to reboot the machine. Eventually, one of our former partners' employees went down to the cage and rebooted the server. Later, in an email explaining what had happened, that same employee (who happened to be their sysadmin) explained that their DVD backup had frozen the whole box and that this situation had never happened before. (This during our peak period for the day).

Now, I can believe that (I've had weird shit like that happen to me, so I can't really bitch at the guy.) But it brings to mind the need for good software and why good software should be written : if you write software, and it fucks up, then the managers of those using your software don't see it as your fuckup, they see it as their employees' fuckup, and hold them responsible. I hate the thought of being one of those employees (and very nearly was one today). This has brought forth some small amount of Karmic inspiration to put more time into testing and finding those odd failure modes for my own software that you wouldn't think can happen, but can because you eventually hit an odd and unexpected set of circumstances. Just something to think about if you're a serious developer and you're reading this.

Linux : the more you know ...

I'm somewhat embarrassed to admit this, but I've just discovered Linux ACLs, which are apparently in every kernel 2.6+, and has plugins for 2.4-. I guess a really linux savvy person should know these, especially since they're incredibly useful and give so so much more power over the default User - Group - World permissions built into the file system.

JSP Tags

They're oh so useful for encapsulating small bits of reusable logic, but, here's the kicker : when they're compiled, only one instance of a tag is generated for a given page, so you have to design them to be stateful and ensure that you override the 'release' method when inheriting from TagSupport or BodyTagSupport. It took me a couple of hours to figure this out, and sadly, it was for the second time. I guarantee there won't be a third after this.

Moar (sic) Hibernate

OK, so once again in my (seemingly never-ending) struggle with Hibernate, I have stuff to report that's mainly being posted here for my own reading so that I can reference this shit later. This time, I've decided to abandon Xdoclet and just go with Annotations. They're so much easier and I don't have to deal with Xdoclet's quirks any more (like ridiculous parsing exceptions between versions).

If you want to use annotations to do a bidirectional OneToMany assocation with list-based semantics, here's the annotations you use :

Parent.java (equals, hashCode, getters/setters removed for clarity):
@Entity
@Table(name = "parent")
public class Parent implements Serializable {

private static final long serialVersionUID = -1989884660562516228L;

@Id
@Column(name = "id")
@GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;

@Column(name = "name")
private String name;

@OneToMany(mappedBy = "parent", fetch = FetchType.EAGER, cascade = {CascadeType.ALL})
@Cascade({org.hibernate.annotations.CascadeType.ALL,org.hibernate.annotations.CascadeType.DELETE_ORPHAN})
@IndexColumn(name = "idx", nullable = false)
private List children;
}


Child.java (equals, hashCode, getters/setters removed for clarity):
@Entity
@Table(name = "child")
public class Child implements Serializable {
private static final long serialVersionUID = -6414526385302360120L;

@Id
@Column(name = "id")
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;

@Column(name = "name")
private String name;

@ManyToOne
@JoinColumn(name = "parent_id")
private Parent parent;

@Column(name = "idx")
private int index;

public Child() {
super();
}

public int getIndex() {
return this.parent.getChildren().indexOf(this);
}

public void setIndex(int index) {
}
}

Note that you need the 'index' pseudo-property in the child class in order for hibernate to properly persist the ordering of the elements in the collection. The setter for the index property should do nothing, the getter should determine the object's placement in its parent, and you'll need the field with annotations in order to get hibernate to read everything properly. The name of the column in the annotation for the index property MUST be the same as that specified in the parent in the @IndexColumn property. They aren't very clear about this in the Hibernate Documentation (once again.) They mention a similar structure for the .hbm.xml mappings in a faq on the main site for hibernate (not the hibernate annotations faq), but they don't explicitly mention this anywhere for Annotations. You can read the mention of it for .hbm.xml files here. I'm sure I'll have to post more about annotation configurations later, but that's all for now. And if you're wondering, the misspelling in the title of this post is an inside joke.

PS. Note that in addition to the OneToMany and IndexColumn annotations on the child collection in the parent, you also need to have a @Cascade annotation in order to properly remove orphans from the database when deleting children from the child collection in the parent. You can also find the table that gave me my final clues on Darren Hicks' blog

More Maven2

Today I had the chance to build a new project right from scratch for a quick and dirty app that needs to be spat out right quick. I figured that this would be a good chance for me to start really getting on the band wagon with Maven2 and using it full force in my development environment.

Building on what I've mentioned about Maven2 previously, here's something else that needs to be mentioned:
1. Adding mvnrepository.com to your list of external repositories :
<project>
|
|
<repositories>
<repository>
<id>mvn-repository</id>
<name>www.mvnrepository.com</name>
<url>http://www.mvnrepository.com</url>
</repository>
</repositories>
|
|
</project>

Interesting geography lessons

This probably sounds pretty off-topic for a tech blog, but it's not that far off. Recently I had to populate our database with ISO data about other countries so that we could accurately ship to them, and in the process learned a great deal about foreign geography and postal lessons. The biggest of these lessons was that by comparison, Canada and the US have dead simple postal systems that nobody should screw up. For example, France's geographical (and therefore also postal) subsystem is broken down into 102 departments, which may be further broken down into communes and 'arondissements'. The first 2 digits of a France postal code indicate the department (synonymous with state or province in US/CA) in which the postal code is located, so that even though France's postal structure is more complicated than the US or Canada, that complexity is well managed and actually very easy to deal with (especially for the French since they're the ones that have to move the mail in that country.) Like France, Great Britain has a far more complicated system than either the US or CA.

For comparision, here's what happens in the US/CA postal systems. Each address is broken down as follows :
123 Any St
My Town ABC123 CA

Or, in a more grammar specific way (s+ indicates there may be whitespace, ? indicates optional):
[Street number][s+][Street address,s+]
[Municipality/Town/City,s+][s+][Postal code][s+][Two digit state / province code]
Now, in the US/CA, Postal codes are \d{5} for the US, [A-Z0-9]{6} for Canada, where every CA postal code follows the pattern A1A1A1 (ie alternating letters and numbers, starting with a letter). Very specific and easy to understand.

Now, in Great Britain, a standard sample address is as follows :
[Street number][s+][Street address,s+]
([Locality][s+])?[Municipality/Town/City,s+]
[Postal Code,s+]
[County,s+]?

And to boot, GB also has Boroughts, Metropolitan Districts, and Unitary Authorities in addition to the counties. And in GB, a postal code matches [A-Z0-9]{6,7}, with no specified interleaving whatsoever, so you can have any possible combination of letters and numbers.

On top of that, there's also the issues of cultural differences within the country at hand. In Canada, there's pretty much English and French addresses, and if an address is French, it stays French (same for English), the point being that it doesn't get Anglicized or Francocized at any point within the postal system. In GB however, there are English and Welsh versions of all the addresses in Wales.

Now, this much complexity in GB (as compared to US/CA) is fine if you live in GB, but if you live outside the country and you're trying to ship to it, and be reliable, GB really blows.

It's there for a reason

I think that one of the things people realize when they start a new job (I haven't) is that the way things are done currently are done that way for a reason (most of the time, though definitely not always). Some past experience has been the basis for changing the way things were done before, and so now the new way of doing things is done to avoid some past (most likely bad) experience.
People then proceed with their work, and if they're not a fan of the way things are done, they do it their own way. In many cases, they'll encounter a previously encountered bad experience and realize why things are done they way they are. This has happened to me several times. And this happened to me right now while using Hibernate. I finally realized why they have the "saveOrUpdate" method on Session objects. Love those epiphanies.

Branching out

I recently stumbled across a glowing review of the Wicket framework that made me want to research it and try it out. The author's main argument for Wicket was that it's pretty much just HTML and Wicket, as opposed to the combination of Spring, Spring Webflow, JSP, and tons of other stuff thrown into the mix. Unfortunately I really don't have the time to mess around with it right now, but I'll try to be back on this topic later.

*EDIT* : To add to the list of things I should really learn, Seam's going on the pile. It looks very cool and it's made by Gavin King, the maker of Hibernate. (I know I've bitched about how bad the Hibernate documentation is on numerous occasions, but really, I can't blame him since I hate doing documentation just about as much as if not more than the next guy)

Better idea for multi-versioned webapps

A while ago at our company, a request was made to me to create a separate instance of one of our webapplications for a select group of customers. In doing this, I had to create a separate instance of the backing database and copy over much of a preexisting version of the application's database which was simply configuration data.

It occurred to me today while working on another part of the application that I really should have separated the configuration from the data. The hard part about that is keeping security intermingled between data (for audit trailing) and keeping security intermingled with the configuration data (as part of configuration). I'm still searching for an answer to that one.

Excel and its annoying quirks

Excel has this annoying quirk of propagating unwanted hyperlinks in the background of spreadsheets, and in our organization, it gets to the point where it infests every cell of the sheet. But, I did find a fix (sorta). Press Alt+F11 to go into the VB editor in Excel, go to Insert -> Module, and paste this in:

Sub RemoveHyperlinks()

'Remove all hyperlinks from the active sheet
ActiveSheet.Hyperlinks.Delete

End Sub

Save it, return to the worksheet, press Alt+F8 to open the macros menu (Tools -> Macro -> Macros otherwise) and run the Macro of the same name as the subroutine above. This should delete all your hyperlinks. Unfortunately, this will most likely remove all other special formatting in the cells as well. If you have a really large worksheet, it will most likely cause Excel (XP and earlier at least) to crash, so you should really save your work immediately before you do this (or before you add the Macro to the sheet).