Sunday, October 11, 2009

The failure of IT projects and the Y2K hoax

I've blogged about the Y2K hoax in the past and I think it provides valuable insights into people and how they operate. I listened to Stephen Fry's "In the beginning was the nerd" (still on the iPlayer) - a potted history of IT with particular emphasis on hardware and software designers. The interview with Ross Anderson was by far the most interesting part and I managed to get hold of the unedited interview which I've stuck on my server here.
One of the most interesting points he makes is the high degree of failure of IT projects - approximately 31% of computing projects in the private sector fail (that's a long understood number) - when a project goes so over-budget or when completed radically under-performs. In a sense, for privately funded projects you expect a degree of failure because if business leaders aren't taking risks they aren't finding new and profitable ways of doing things.
The horror I discovered is the rate of failure of IT projects in the public sector - more than 70% - and this from civil servants who should be more risk-averse than businessmen.
Anyway - he covers all of this in what is a very insightful piece.
It's a good job that traditional engineering projects don't have such a high rate of attrition.


Ovid said...

Y2K was absolutely not a hoax and anyone who thinks it was is abysmally ignorant of what happened. As one of the people who slogged hard on dealing with those issues and was intimately involved in it, it amuses me that those who know so little about it still make this silly claim. It's not the case that hundreds of thousands of IT workers the world over could possibly conspire on such a thing.

Phil Crawley said...

Well maybe hundreds of thousands of IT worker in this country were involved but in other countries of similar demographic/technical development (Italy, South Korea) spent less than 1% of what the UK spent and suffered a similar (i.e. almost non-existent) level if disruption.
Anyway - to paraphrase from an earlier blog entry; I was chief engineer at a large TV facility at the time and suffered a daily pestering by various "Y2K consultants" from the BBC an others. To a man (no ladies were stupid enough to be sucked in to that degree!) they were clearly riding a gravy train that they were going to suck dry! I'd upgraded every OS and bit of application software as per manufacturer recommendations but that wasn't enough - no sir!

"Is that combination door lock Y2k compliant?"
"No - it's a mechanical gadget that has no notion of the date"
"That's not good enough - if you guys are going to get BBC/Channel 4/etc etc millennium certification you're going to have to provide me with manufacturer documentation to show it is compliant"
Suffice to say we sailed through the 1999-2000 junction with no problems - I'd simulated it twice at that company (and once at my previous firm - another large TV company) - but that kind of level-headed thinking cut no mustard with the folks from the Y2k industry!

Anyhow - I noticed some interesting trends in the run up to 2000 - aside from the hundreds of books by doom-mongers (have these people ever been taken seriously since?) there were well defined attitudes in all the people who wanted the world to end by the close of 1999

* Once people had woken up to the fact that programmers had been addressing the issue for most of the nineties and databases are used to badly formatted input the danger turned to "embedded systems" - everything from your car's electronic ignition to nuclear missiles were going to go wrong. All the people I spoke to wouldn't accept the fact that most microcontrollers know nothing about large time periods - your engine management operates over hundreds of milliseconds (or at most months for the service interval). The view of engineers who dealt with embedded controllers on a daily basis counted for nothing over the views of the doom merchants.
* By mid-1998 it was fashionable for the Y2k industry to imply that although the USA would be OK all those "foreign countries" would cause the trouble - racism was everywhere! In the end Italy spent less than 1% of what the UK spent on Y2k preparedness and they suffered less than our modest amount of pain!
* In the aftermath of new year 2000 I read lots of online comments from people like Gary North claiming that the US administration was covering up the nature of the problem and we'd see the collapse of society etc etc soon - perhaps he is still holed up in a cave in Montana living on beef jerky and listening to his clockwork radio!

Now I did a maths & programming degree in the eighties and back then we were talking about it and being warned about 'garbage checking' database input etc.

I say again - it was a hoax, a chance for people to make lots of money. Although it might not have been a conspiracy in the classic sense there was a definite conspiracy of silence because nobody wanted to look foolish in the face of a (false) orthodoxy.