Andy, ITGuy's post reminded me of a topic I've been meaning to weigh in on this since it popped up during the security blogosphere's last iteration of the security ROI debate. If you aren't familiar with the debate, it's about justifying the cost of security investments. Investing in security acts to prevent losses, so in the pure financial sense there is no measurable ROI. Security investments pay for themselves through cost avoidance. For example, by preventing breaches that would be more costly to the company than what the investment costs. There's no doubt that without investing in security the company would face tremendous risk of loss, but quantifying that risk into dollars and cents is a very inexact science making ROI an inadequate metric for justifying security investments to business leaders. Every so often someone in the security industry brings this up, starting another round of debate with various folks jumping into the mix with their own metrics for justifying security purchases.
In response to all of the debate, Andy and Jack Daniels (the blogger not the whiskey) coined the acronym "FOI" for Failure of Investment. The premise is that a security investment's failure or success at its purpose should be a central metric. Personally, I believe this is something that often gets overlooked or minimized when companies evaluate products. How often is there honest discussion to answer the question, "What is the risk that this product is going to fail at what we want it to do?"
Everything else about a product is irrelevant if it doesn't end up working the way you need it to. But you see it happen all the time. The vast majority of network IPS appliances that are deployed out there are only actively blocking on a subset of attack signatures. A large number of SIEM tools out there are relegated to being expensive log collectors if they're even deployed. Examples in the security industry are all over the place.
Are these investments made with the understanding that they wouldn't end up working the way they're supposed to? Of course not. The risk of failure wasn't fully understood and adequately managed, leading to Failure of Investment.