I’m sure you remember those books from when you were a kid. The choose your own adventure books. You started reading, came to a decision point from where the book gave you a goto-statement to another page from where the story continued. Often to horrific deaths as I recall. Sometimes I get the feeling that I’m on such a page, choose if you’re going to go through the cellar or sneak across the garden to get to rescuing the princess. The decision you make is going to have an impact on you for a long time, potentially irreversible.
Oh yeah, and by the way, when I was a kid I immediately recognized it as a rudimentary programming book. And I converted it into BASIC on my Amiga so I could run the book as a computer game. I was that kid.
In a development context, these decision points often are when you decide on technology choice. It becomes especially interesting when it used to be a no-brainer but there has recently appeared new contenders. I’m going to tell a tale here about such a choice. Unfortunately, or fortunately as we shall see, the choice wasn’t really made by me – I just had to deal with the fallout.
Once upon a time, years ago, it was decided that a new subsystem was going to be built in the system I would work on later on. The subsystem in question was a hierarchical inventory of network equipment, basically sorting everything into a huge tree. When embarking on this quest, the brave adventurers of old had to choose a strategy for data storage. Remember kids, this is probably back in -05 or -06. Being adventurers the old proven broadsword of SQL seemed dull and boring and someone had heard about db4o. Without really consulting with anyone as this team had all the self governance in the world, they decided that this looked exciting and off they went!
The system got built and everything looked in order for nearly two years. People came and went and the actual whys of why db4o was chosen was lost. Also lost was the knowledge of how you really were supposed to use db4o. Then it started to break. None of the original programmers were there any longer, and at this time at least googling the problem turned up little help. We had ended up with a technology island which had become unmaintainable.
You see, when these sorts of databases break, they do so in weird ways. I’m sure db4o is a good and stable product these days, it might even have been so back then – the cause of the problems might have been us abusing it. But we got data loss. And not the normal kind where things go away. We had things where if you followed a connection one way, you’d find something. But if you went the other way you’d end up empty handed. Also, it might work if you tried to do it a few more times. The exact same call. In the end we wrote an interesting piece of software to convert the data from db4o into a normal MS-SQL database. It contained such features as “try to get the data five times before giving up” and “wait a little before trying again to let things cool down”. We ran it in the middle of night, since the product was up and running and some of the data in the database were high traffic in both reads and writes. It took hours to run. In the end, we rescued almost every piece of data.
What I’m getting at here is the lack of analysis when it came to technology choice. In my mind, it really boils down to this. Do you want to be cool and use the latest tech but at high risk, or use something proven, boring and low risk.
It’s a tough choice on many levels. If you go with the latest stuff, you’ll potentially attract great talent since they want to use cool tech, you might get better performance, you might get enthusiastic support from the open source community if you’re lucky. Or you might end up in the middle of the night running arcane rescue tools.
The well trodden path is full of search results in google. Its full of help on stack overflow. It’s also pretty dull and I understand the feeling of frustration when you’re still writing the same stored procedures you did five years ago. No-one likes stored procedures.
In my mind the choice isn’t about technology really. It’s about people. And it’s about sharing code ownership. If you introduce a strange new technology you have to make sure that: Everyone can understand it and the code and maintenance procedures must be completely shared. If you have those two things, you can pretty much choose whatever weird shit you want, a team of superstars that share the code can handle it. But if this is one guys pet project, you’re pretty screwed. He’ll be happy at first, you’ll let him run “his” part of the project. No-one will care about it. When he quits it’ll go into disrepair and you’ll write blog posts about it years down the line.