Background on the home search page ? WTF Google ?

Today I went to Google (as I do most days) to search for some stuff, but today when I got there, I was greeted with a horrible looking background on the search. My very first thought was "Did I just go to Bing by mistake ?", then I looked closer at the page and realized it was indeed Google. Needless to say, I was incredibly pissed. Why did Google change their signature search page ? It was great the way it was ! That was specifically why I went to Google. If I wanted to use a search engine that looked like a piece of shit, I'd use Bing. In my mind, this is nothing short of an epic fail on Google's part. And apparently, I'm not alone in this feeling.

My hatred of Microsoft is justified ... yet again

There are two different schools of thought when it comes to being a vendor of very large software used by millions of people :

1. Move your software forward as often as possible, even if a few customers have to suffer lack of backward compatibility


2. Move your software forward as little as possible, even if a few customers have to suffer lack of innovation

I've noticed from my own personal experiences that Apple tends to take the former attitude, whereas Microsoft tends to take the latter attitude. In the end, someone (probably you) is going to get screwed over at some point, it's just a matter of how you want it to happen. As a consumer, I generally like Apple products, so on my personal technology front, I choose the former. As a software developer, I'm forced by business constraints to accept the latter. The specific circumstances that bring me to mention this :

Today I was working with version 4.0 of the .NET framework, the very latest (and supposedly greatest) from Microsoft, along with the very latest version of their Visual Studio (2010) software. I'm also using these with WPF (the Windows Presentation Foundation) to create an application for my employer. For various reasons, I had to go back and refactor some old code, part of which required opening log files for viewing within the application. In order to open a file in WPF, you must use the OpenFileDialog class that comes with the Windows Forms library, which has been around since Windows 2000 (if not earlier, I'm admittedly not as familiar with the lifecycle of this technology as I could be). If you're a developer on the Windows platform, or even just an observant user, you'll have noticed that on the OpenFileDialog there's a Places bar on the left hand side of the dialog that provides common places for storing files, some of which are very general and will require some drilling down into subfolders to find what you actually want. I wanted to be able to add a place to this bar so that users of the application (company employees who work as technicians out on job sites) could have a direct link to the typical directory used to store log files on their machine.

It turns out that adding a folder to the places bar is obscenely difficult, and must be done through registry hacks. My question is this (directed squarely at the developers at Microsoft responsible for writing the GUI controls, specifically these dialogs) : what the hell was going through your head that would make you think that the way you've implemented these dialogs is a good thing ? It's obscenely hard on developers to customize the generalized tools that you've given them, and this in turn makes it hard on users of that software to use the software made by developers bound by these stupid and seemingly arbitrary constraints. I find myself growing increasingly frustrated by stupid decisions of the Microsoft developers that seem obviously poor when you look at them from the perspective of a framework user. These poor decisions are costing me a lot in terms of time I have to spent developing around these decisions and researching alternatives. It's now no wonder to me how other companies can make absurd sums of money off of controls designed as work arounds for the short-sightedness of Microsoft's developers, and frankly, it's quite depressing.

Fixing telnet disconnects

I've recently been working with a lot of embedded systems via telnet, and I'll frequently forget to disconnect my session before physically disconnecting the machine from my local LAN. Unfortunately, this results in telnet hanging the session, and me being unable to do anything else. Until now, I've just closed the window and opened another, but that became a pain in the ass, especially when I had to discard my (very useful) terminal history. Apparently, you can rescue a hung telnet session and avoid having to kill your terminal session by pressing Ctrl+5. This sends a signal to telnet to end the current session and return you to a prompt. Incredibly useful. Looks like there's a ton of other stuff where I found this useful tidbit.

Resolving slow SSH login times

At work, like most other organizations, we have a Linux server that we access via SSH. However, lately, I've found my use of it skyrocketing in order to test software, and log-ins and file copying have been very slow. In my efforts to find out why, I found out that sshd has in its configuration file a setting called 'UseDNS'. The default is 'yes', so even if this setting is commented out, it will try to perform reverse DNS lookups on the IP addresses of users logging in. This would be a bad thing for us, especially since we have this box locked down and unable to contact any DNS servers. After disabling DNS lookups, my login and file copy time dropped to near 0. I hope this helps somebody.

*EDIT:* I've also encountered this problem on the BeagleBone Black boards which I've recently acquired for use in a project.

Two bash functions that will make your software development life 10% easier

... if you use SSH to move between machines a lot. Which I do. I have several machines that I work on regularly, and a centralized development machine shared with others that I frequently have to exchange files with. The following two functions are the most useful two bash functions that you can put in your .bashrc file if you use SSH as much as I do :

function ssh-create-keys {
ssh-keygen -t rsa
}

function ssh-setup-on {
cat ~/.ssh/id_rsa.pub | ssh $1 "cat - >>.ssh/authorized_keys"
}

The first sets up an SSH key file for you in your home directory, This key file will be used to identify you on other machines, and can be used for various purposes on your local machine as well. The second function logs you into the machine determined by the username@anothermachine given as the first argument to the function, and adds your public key to the list of authorized keys for that machine.

Once you've run the first function, and run the second function to setup your public key on an account on another machine, you'll now be able to use SSH to log in and copy files freely with the account on the other machine without having to enter a password. While this is incredibly convenience, it also comes with a caveat : it's dangerous. If somebody other than yourself gains physical access to your machine and can log on as you, (or use your already logged on account), they can move to those same machines freely and perform possibly malicious actions, as you. Keep that in mind.

Copying files in linux

Lately I've been doing a lot of embedded development with Linux, and copying files between systems has been a bit of a pain. Fortunately, a combination of RSync and SSH solved my problems, with a command that lets me copy files from a directory on one system to a directory on another system, recursively with symlink preservation (and even duplication !) :

rsync -azuv -e ssh  user@systemaddress:~/path/to/dir/* .

WTF ?! Windows (as of Vista) no longer supports multiple different monitors!? WHY ????

Apparently, as of Windows Vista and later, Windows no longer supports multiple video cards that don't use the same driver!! Why ? I use a Radeon and a Matrox QID to drive 6 displays on my development box, and this just entirely fucks me over. At best, I can only use four of my displays now, and that's only going to be when Matrox gets off their lazy ass and releases a Windows 7 driver for their QID LP PCIe video cards. I'm so disappointed with what's a complete step backward for Microsoft. Epic fail, Microsoft.

Seeing where Perl looks for its modules on a system

Because we have very limited space on some embedded devices that we use that run Perl, we can only store a very few modules on these systems. This is a consequence of the fact that we run Busybox on these things, so by definition everything on these boxes is limited, if present at all. Therefore, we have to check and see if a module is available before we can use it in our code. Fortunately, there's a quick one-liner to see where Perl is looking for modules on a system :

perl -e'print join "\n", @INC'

I got this off a forum post from somewhere, and I'd post it here if I could find the link again. My apologies to the author of that forum post if they ever happen to run across this blog.

Embarking on a Perl journey

A long time ago, in a job far, far away, I had to deal with some Perl. I learned just enough to get me by for the duration of the task at hand, and then pretty much forgot everything I had learned. Now, at my latest job, I'm having to deal extensively with legacy systems which have a considerable amount of logic written in Perl that needs to be either ported over to other languages (for various reasons) or updated and new things written because Perl is the only language that's both abstract enough and not too processor intensive to run on the embedded systems we deal with. Therefore, you're going to start seeing a lot more Perl posts on this blog.

Setting up Tomcat (5.5) on Ubuntu Server 8.10

I recently ran into some old quirks when provisioning a new server for our company's web applications on Ubuntu 8.10 (Intrepid Ibex). Because the manager apps are no longer installed by default, you need to add extra packages to the list to install when installing Tomcat :

sudo apt-get install -y tomcat5.5 tomcat5.5-admin tomcat5.5-webapps

If you're copying configuration over from a previous Tomcat / Ubuntu installation, you need to make sure the permissions on all the files you copy are set correctly. In most cases, you'll have to run :

chown -R tomcat55:adm [file and folder list here]

If you're securing the applications with a certificate, try to make sure it's valid for your location and ensure that you've set it properly in your server.xml configuration file. If you want useful logging, you'll also have to place a log4j.properties file in

$CATALINA_HOME/common/classes

Hope this helps.

Dumping just your schema with MySQL dump

A simple one-liner :

mysqldump -u root -p mydatabasename --no-data=true --add-drop-table=false > test_dump.sql

With this command, you'll be prompted for your root password. I got this from here. Simple

The Curious Case of Damned DataIntegrityViolationException

In one of the projects on which I contract, we recently started encountering a problem importing and parsing text record files into our system which previously had no problem. My first thought on hearing this was that the partner from which we obtain the files had changed the file format (again). Upon closer inspection, nothing had changed in the files. My next step was to try importing them into a development system and seeing what was going on. As it turned out, the application was catching Spring's DataIntegrityViolationException. I was floored as soon as I saw this because our application was supposed to be catching this exception behind one of the business interfaces and converting it to an internal exception which is used in business logic. After some more poking around to confirm what was really going on, I threw the problem into google, and on the second result was a post in the Spring forum made by a user having exactly the same problem I was.


To summarize their problem quickly, they were using a transaction manager, and they were intercepting their business methods (via interfaces) with Aspects, which we're also doing. The problem was this : as soon as the internal Aspects were applied to the business interface, this changed the ordering of advice applied to the interface implementor, so now the Hibernate session underneath was getting flushed later by the transaction manager, instead of in the business method where it had been flushed previously. The result of this was that now DataIntegrityViolationExceptions were being thrown outside of the interecepted method, instead of inside where it was expected. A manual session.flush() inside of a HibernateCallback within the business method fixed this :

 /**
  * @see AchPaymentNoticeOfChangeService#registerNoc(AchPaymentNoticeOfChange)
  */
 @Override
 public void registerNoc(final AchPaymentNoticeOfChange changeNotification) throws IllegalArgumentException, NoticeOfChangeAlreadyExistsException, Exception {
  try {
   getHibernateTemplate().execute(new HibernateCallback() {
    @Override
    public Object doInHibernate(Session session) throws HibernateException, SQLException {
     
     session.save (changeNotification);
     
     // flush the session to ensure that the database gets synchronized
     // the end of this call, rather than waiting for any transaction
     // managers to handle it and risk letting a DataIntegrityViolationException
     // occur outside of this method's handling
     session.flush();
     
     return null;
    }
   });
  } catch (DataIntegrityViolationException dive) {
   throw new NoticeOfChangeAlreadyExistsException("A notice of change already exists for payment ["+changeNotification.getAchPayment().getId()+"]", dive, changeNotification);
  } catch (Exception e) {
   throw e;
  }
 }

I hope this post finds somebody else who runs into this problem.

Composing XML in C# 2.0 or later

When testing some custom XML serialization I was writing for a project, I ran across a quirk of the XmlWriter created by XmlWriter#Create() : The writer that gets produced by this method doesn't actually write out XML to whatever stream or file you've given it in the Create method until Close() is called on the generated writer. This means that if you've been using the XmlWriter along with a using() statement, you're fine, but if you haven't been using it, as I have in the unit tests I've been trying to write, you're going to get some unexpected results. Something to pay attention to.

Visual Studio (2010?) Debugging Gotchas - Part I

I recently ran across a problem where I was trying to debug some code and the Visual Studio debugger wouldn't stop in my class. It would stop in the test class from which I was debugging, but not the class I really wanted to debug. I looked on the breakpoint and noticed that it was transparent, so I hovered over it to view the tooltip, and the tooltip informed me that my class had a DebuggerStepThroughAttribute assigned to it. My first thought was "what ? that's bullshit", and then it occurred to me that I hadn't looked at the rest of the (partial) class which was defined in another file that was generated from an XML schema by xsd.exe. Sure enough, xsd.exe places DebuggerStepThroughAttribute attributes on the classes it generates. Why is this default behaviour ? This was an annoying as hell bug that took time out of my day that's better devoted to other things, like being productive so that I don't get shit from my boss. It also took more time than I'd care to admit to, largely due to my inexperience with Visual Studio and .NET. I've had yet to see anything like this coming from a Java / scripting background.

Tricky styling issues in WPF

For about the last 8 or 9 hours (spread across two days) I've been troubleshooting a styling issue in WPF. I have a list of items that I'm displaying in a ListBox that have an 'enabled' property, which is an enum with values OFF and ON. Understand that this type was generated from an XML schema which is very tightly controlled, hence why the type is not a boolean (as would make sense with a name like 'enabled'). I've been trying to create a DataTemplate for the ListBoxItems so that they'll each have an icon associated with them depending on whether they're enabled or not. My original list looked like this :

<ListBox x:Name="ProcedureList" ItemsSource="{Binding Path=procedure}" HorizontalAlignment="Stretch" VerticalAlignment="Stretch">
  <ListBox.Resources>
    <Style TargetType="{x:Type ListBoxItem}">
      <EventSetter Event="MouseDoubleClick" Handler="WeldProcedure_MouseDoubleClick" />
      <Setter Property="HorizontalContentAlignment" Value="Stretch" />
    </Style>
    <Style TargetType="{x:Type Image}">
      <Style.Triggers>
        <DataTrigger Binding="{Binding}" Value="OFF">
          <Setter Property="Image.Source" Value="Resources/Error_16x16_72.png"/>
        </DataTrigger>
        <DataTrigger Binding="{Binding}" Value="ON">
          <Setter Property="Image.Source" Value="Resources/Success_16x16_72.png"/>
        </DataTrigger>
      </Style.Triggers>
    </Style>
  </ListBox.Resources>
  <ListBox.ItemTemplate>
    <DataTemplate DataType="{x:Type model:WeldingProcedure}">
      <DockPanel HorizontalAlignment="Stretch" LastChildFill="True">
        <Image DockPanel.Dock="Right" Height="16" Width="16" DataContext="{Binding Path=enabled}" ToolTip="Disabled"/>
        <TextBlock DockPanel.Dock="Left" VerticalAlignment="Center" TextAlignment="Left" Text="{Binding Path=procedureName}"/>
      </DockPanel>
    </DataTemplate>
  </ListBox.ItemTemplate>
</ListBox>

However, the images in the DataTemplate were not getting styled with the images as they should have been. As it turns out, if you're referencing default styles in Templates as above, the style resolution doesn't go outside of the template, so the default styling for all Image elements as above needs to go *inside* the DataTemplate resources :

<ListBox x:Name="ProcedureList" ItemsSource="{Binding Path=procedure}" HorizontalAlignment="Stretch" VerticalAlignment="Stretch">
  <ListBox.Resources>
    <Style TargetType="{x:Type ListBoxItem}">
      <EventSetter Event="MouseDoubleClick" Handler="WeldProcedure_MouseDoubleClick" />
      <Setter Property="HorizontalContentAlignment" Value="Stretch" />
    </Style>
  </ListBox.Resources>
  <ListBox.ItemTemplate>
    <DataTemplate DataType="{x:Type model:WeldingProcedure}">
      <DataTemplate.Resources>
        <Style TargetType="{x:Type Image}">
          <Style.Triggers>
            <DataTrigger Binding="{Binding}" Value="OFF">
              <Setter Property="Image.Source" Value="Resources/Error_16x16_72.png"/>
            </DataTrigger>
            <DataTrigger Binding="{Binding}" Value="ON">
              <Setter Property="Image.Source" Value="Resources/Success_16x16_72.png"/>
            </DataTrigger>
          </Style.Triggers>
        </Style>
      </DataTemplate.Resources>
      <DockPanel HorizontalAlignment="Stretch" LastChildFill="True">
        <Image DockPanel.Dock="Right" Height="16" Width="16" DataContext="{Binding Path=enabled}" ToolTip="Disabled"/>
        <TextBlock DockPanel.Dock="Left" VerticalAlignment="Center" TextAlignment="Left" Text="{Binding Path=procedureName}"/>
      </DockPanel>
    </DataTemplate>
  </ListBox.ItemTemplate>
</ListBox>

Generating XML objects from a schema in .NET

In my last job, I frequently made use of the JAXB library provided with the Java SDK for object de/serialization. I chose to generate my schemata from objects back then because I started off with the business objects and I knew how I wanted to serialize them to XML. Now I have the reverse situation, and I'm using a new language. As it turns out, generating classes from XML schema is dead easy : use the xsd.exe tool. It comes with .NET.

Useful GIMP tricks - Batch file conversion

Now that I've started having to do a lot more documenting for my job, I've found myself importing a lot of pictures taken as steps in tutorials and for providing figures. Since the camera gives us a nice, high-res version that's not really suitable for documentation, I have to scale the images down so that they're in a nice, readable size. When your tutorial has 30 steps, and an image (or even two or three) is required for *each step*, this can mean a lot of manual clicking, dragging, opening and saving in an image editor. Fortunately, GIMP has a plugin called Davids Batch Processor that allows you to easily do fairly simple transforms to file and output them in the format (that's most likely) of your choice. For the simple tasks of rotating, resizing (scaling) and exporting the images in a new format, this plugin's great. There's a tutorial on the site and the plugin itself has already proven to be exceedingly useful for me. I hope it will be for you too.

Subtleties of Perl - Reading files

I've recently begun a new job, and with it has come a whole new segment of the software development universe. The new job uses a lot of Perl, C, Bash and various other languages to get stuff done. Today, I ran afoul of a Perl idiosyncrasy that's worth making a note of, because I'm sure I'll stumble across this problem again and I'm going to need to refer to this in the future. I should also note that I'm writing this as I'm waiting for a significantly large file to parse.

We have large log files that we parse on a daily basis to extract summary information from them about mechanical systems. We read the files, and then output a summary on a secondly basis, one line at a time. Recently, I ran afoul of Perl's file reading mechanisms. When reading files in Perl, there's any number of ways to do so, and it turns out that for the longest time, we've been using the wrong one. Previously, we had been using :

foreach my $line_of_log (<LOG>)
{
    // DO STUFF WITH $line_of_log
}

We thought that this was reading the file in one line of the log file at a time, processing it, and then moving on. What it was actually doing was reading (or "slurping") the whole file into memory, and giving us an array of strings, which we processed one line at a time. After 10 minutes of cursory Googling, I ran across a tutorial which presented this :

while (<LOG>)
{
    my $line_of_log = $_;
    // DO STUFF WITH $line_of_log
}

The 'while' version of the file read actually does what we thought we where doing all along: reading one line from the file, and then doing stuff with it. The difference between the two methods is that in the 'foreach' version, the entire input file gets read into memory, whereas in the 'while' version, only a single line gets read into memory at any given time. As it turns out, another difference is that reading in a 7 MB file resulted in Perl grabbing 34 MB of memory with the 'foreach' version, but only 2.2 MB with the 'while' version. That's an ENTIRE ORDER OF MAGNITUDE in difference!. This also makes a huge difference when running Perl on memory-limited systems, as we are.

Tweaking visual studio

I've recently begun using Visual Studio 2010 Beta 2 more and more as I have increasing amounts of work that require .NET. In order to keep things nicely tabbed the both I and my boss like it, there's a setting that can be set according to instructions founds here : http://mhinze.com/tabs-whitespace-visual-studio/

Using the MySQL EXPLAIN statement

Recently I had been having trouble with queries on a certain table in a system that I've been maintaining. I had gone through just about every excuse for why queries on the table could be performing so slowly : the machine was slow (DB running on a VM), the webserver was slow (also running on a VM), I wasn't using the native libraries (webserver was Tomcat), I had other processes running in the background (I didn't). Then I ran across a tip on a forum suggesting usage of the MySQL EXPLAIN statement. I had known all about it for the longest time, but it never occurred to me to actually use it (I think I'm that good at writing queries, turns out : I'm wrong). After using the EXPLAIN statement, I found out that the query processor was using a suboptimal query plan which utilized an index I had added with the intent of improving performance (the index had a fairly high arity, so choosing to use it was sketchy at best in the first place). Most DBMSs should have a similar functionality built in. I think that'll be the first place I go in future.