I let myself get caught up in a pointless twitwar yesterday, during which I took much abuse from my proponent for basically disagreeing with the assertion that you can just walk into an organization and "know" what is and is not important without doing some degree of assessment. His later point is that you don't need to do a "full" assessment, which is correct and not my point.
My point, quite simply was this: dowsing (or "divining") is no way to assess or manage enterprise risk. Dowsing is the ancient mystical practice of using a dowsing rod to find water hidden underground. To this day, well water is a very important commodity. In olden dayes, technology did not exist for finding sources, and so divining came into practice. Using the divining rod (or dowsing rod), a skilled individual could walk around an area, feeling mild tremors through the rod. The skilled individual would then move around until these tremors were maximized and the rod pointed down to the source of water.
In many ways, risk assessment today is exactly like dowsing. We walk into organizations with some mystical methodology that assesses pseudo-risk and then we act is if we've done something that is in fact truly legitimate and well-founded. The problem, of course, is one of repeatability. The INOFSEC Assurance Methodology (IAM) tries to specifically address this concern by setting up the System Criticality Matrix, but there are potential weaknesses in this approach. Similarly, FAIR leverages Bayes for providing reasonable modeling in the absence of real data. [6/4: correction - Bayes requires data, just provides model based on knowledge-state instead of nature-state]
Both cases are challenged, however, and at best "science" in the way of the "social sciences" (or so-called "soft" sciences). The problem, quite simply, is that there is no reliable way (today, anyway) to quantify a qualitative value. As such, we're stuck with gut instinct in assessing risk ratings, challenged in trying to come up with a consistent, reliable, and accurate method. If the method cannot withstand rigor, then it's not particularly sound or scientific.
This problem is one that is being actively researched. Notable figures like Alex Hutton (formerly of RMI and currently of Verizon Business) talk about this frequently; that enterprise risk management is a broken field that lacks scientific rigor. In my mind, this is spot on, and fully analogous to the state of the security industry. Gunnar Peterson, I think, captures this perfectly in his comment that "Its too bad but assumptions of yesteryear lead to building things on shaky foundations." His notable chart tells the story:
Similar to the lack of innovation and growth in infosec, where the world still revolves around firewalls and SSL, so does risk management revolve around pseudo-quantitative risk assessment that is based on qualitative assessments of varying degrees of reliability that are then converted to numbers, or otherwise averaged out. Dowsing risk in the enterprise is no way to live, and a good way to get completely off-track. Let's hope the future reveals a better way to exist.