mini-cases: things take time

My manager uttered a phrase that, to a software developer, was simultaneously thrilling and terrifying: “It’s time for an upgrade.”

I was working for a company in the financial services sector; their core business involved a proprietary system for predicting the likely performance of publicly-traded companies. It was a good company, and a solid system.

And the technology behind it was…functional. Yes, it was cobbled together with the programming equivalent of duct tape and string; some PHP language here, Javascript language there, a few different databases, half the code in one version-control system and the other half on somebody’s hard drive. Not a textbook example of a best-in-class enterprise-level application, but it was functional, and it made money.

However, with a new CEO came new expectations of growth. We realized that, if our sales projections came remotely true, our little web application would be quickly overwhelmed. So it was time to search for a replacement.

Nobody was really against this plan; that was the “thrilling” part of the project. We recognized that the current system had its shortcomings, and we welcomed the chance to have formal approval to build something better.

We decided that we’d try not to reinvent the wheel, and management evaluated various “enterprise” platforms with the built-in features, functionality and scalability we expected to rapidly require. The one they picked wasn’t a black horse candidate by any means, and we were all pretty excited by the sales demos.

There was one very important caveat: this particular platform was written in a language that, although popular, was not one that any of the developers (including myself) were experts in. But we would have plenty of training, we were told, and consultants would be on-hand to help us build the application.

That was the “terrifying” part.

We commandeered an empty office, moved our desktop PCs into our new bullpen, scheduled training sessions, met our consultants, and immersed ourselves deeply in our new development environment. And so it went, for a number of months. Code, test, refactor, repeat.

I’m not sure exactly when we realized we were not going to meet our deadline.

Maybe it was after we discovered that the process of deploying code on our new platform was significantly more involved than we’d previously believed. Maybe it was when we found that code that deployed fine in development mysteriously failed on the not-yet-live production server. Or when we realized that we couldn’t debug the production code by just outputting errors to standard output—we needed specialized debugging subroutines.

Which also didn’t work.

Plus, we were understaffed. Well, to be more specific, we didn’t have too few staff, we had too few staff who knew what they were doing. As I recall (though I could be wrong) I was one of the only ones who had exposure to this particular language, and my experience involved reading most of a “For Dummies” book and following some online tutorials. It was the blind leading the blind, deaf and dumb.

Personally, I realized we were in trouble I discovered that our new platform’s vendor had recommended that a group of programmers—with multiple years of experience in the language the platform used—would probably take about 18 months to deploy the sort of application we were attempting to build.

The application we were attempting to build in 6 months.

Was the project a failure? Well, we did code a stopgap solution in a language I was far more experienced with, which only took a few weeks. Marketing had to scale back a bit on some of their promises. Sales had been…optimistic…in their estimates of new customers, so that ended up being okay.

But did we build the application we had hoped to build, using our fairly expensive new enterprise-level framework?

Not even close.

We spent a lot of money. We met our needs, but not in the way we had hoped…sunk costs, in the end. I left that company not long afterward—certainly not as a result of this project. I had personal reasons; specifically, I was trying to woo my now-wife, which worked, so, good decision. Career isn’t always everything.

I like to think that some lessons were learned (besides the above exhortation that your job sometimes has to take a backseat to your personal life). First, while I strongly, strongly recommend using frameworks rather than solving every problem from scratch, there’s nothing like having a backup in your back pocket. Embrace your chosen new language, but if it all hits the fan, there’s nothing like falling back on a time-tested classic—PHP, Ruby on Rails, Excel, Access, or a shoebox full of index cards.

You can carve the following on my tombstone: Technology does not make money. Solving problems makes money.

Second: VARs (Value-Added Resellers) or vendors or providers or whatever are in the sales business. While they will (hopefully) not lie, they have no vested interest in saying anything contrary to that which will get you to adopt their solution, thus increasing their sales numbers.

If they tell you that doing X with their solution will take time period Y, they are erring on the low side. Think about it: if a person selling a house tells you it’s “10 minutes from downtown” they have absolutely no reason to believe that it only takes 5 minutes; if they thought that, they’d say “the house is 5 minutes from downtown.” Most likely, getting to downtown in “10 minutes” means either you’re weaving in-between buses on a motorcycle at 70 mph, or it’s 5 a.m. on New Year’s Day. 10 minutes is the absolute lowest value they felt comfortable quoting.

So if the vendor says “18 months” and you have a 6-month deadline, one of three things needs to happen: you need to hire a bunch of developers with the recommended level of expertise, or you need to change your deadline, or you need to scale back your project.

The second and third options are, in my experience, the ones most likely to be successful, and the least likely to be followed (see “Mini-Cases: Taking Stock”, on the importance of being absolutely sure you need a given feature).

The first option is the third lesson I learned: you’ve got to plan ahead.
I remember being in a meeting, hearing a manager talk about an upcoming staff augmentation, designed to help us meet our rapidly-approaching deadline. I (in a serious Career-Limiting Move) raised my objection that, even if a new hire arrived the next day, it would still take them three to six months to get up to speed, that hiring someone would likely take at least a month even if we started looking right after the meeting was over, and that, regardless, our deadline would be long-past when by the time our alleged new people were actually adding any value.

This was not well-received. Lesson 3.5: calling your superiors on their questionable logic, no matter how well-intentioned, is not a good thing to do.

But my intentions were pure, and, frankly, make sense: if your strategy involves employees having requisite skills, you need to recognize that this takes time.

There are lots of people without jobs right now. And there are lots of jobs available. Unfortunately, the skills of the available workforce may well not match with the requirements of the available jobs. So that complicates things: you have 500 .NET developers in your backyard, but you use PHP.

Second, a new employee is probably going to need, at a minimum, 2 weeks to give notice at a current job. And don’t forget that scheduling interviews takes time as well. If you don’t think ahead, you’re not going to have the resources you need in time for them to do any good.

One final lesson: a smart, capable person was in charge of web application development. It’s my understanding that the above-mentioned CEO asked him if he felt we were capable of meeting the goals of the business in the requisite timeframe, to which he gave a realistic reply. He was promptly demoted below another individual who responded in the affirmative to the impossible task.

Wishful thinking.

Don’t mistake caution for a bad attitude. Don’t mistake enthusiasm for idiocy.

Or plain old…questionable logic.