Monday, January 19, 2009

Estimates, Johnny Carson, and the meaning of life.

How long will it take to install a new server? How many file restores will you do next week? How long will it take to code a new web page with data from an RDBMS? How many application updates will the line of business request this year?



As a young adult I loved watching The Tonight Show with Johnny Carson. If you've never seen his Carnac the Magnificent routine, then you have missed a truly precious moment in the history of comedy. Ed McMahon, Johnny's sidekick would hand Johnny (dressed as a carnival mystic) a sealed envelope. Carnac would hold the envelope to his forehead and divine the answer to a question. Upon opening the envelop, we would hear the question. Some examples follow:

Carnac, holding the sealed envelop, "The answer is Frat House"
Ed would repeat it, "Frat House"
Carnac opens the envelope and reads the question, "What do you call a Japanese home after Godzilla steps on it?"

Carnac, "The answer is Siss, Boom, Baa"
Ed, "Siss, Boom, Baa"
Carnac, opens and reads, "What is the sound of an exploding sheep?"

Carnac, "The answer is Catch-22"
Ed, "Catch-22"
Carnac, "What would the [Pittsburgh Pirates] do if you hit 100 pop flies?"

I.T. is a tough business. Like Carnac, there are days when I feel like we are supposed to know the answers to questions we haven't seen. According to the Hitchhikers Guide to the Galaxy, 42 is the number from which all meaning derives (the meaning of life, the universe, everything). Legions of loyal readers have sought to discover all of the places 42 shows up:
  • The angle at which light reflects off of water to create a rainbow is 42 degrees.
  • Two physical constants in the universe are the speed of light and the diameter of a proton. It takes light 10 to the minus 42nd power seconds to cross the diameter of a proton.
  • The sum of the ordinal alphabetic positions of the initials (SPG) for Stan (P.) Gibilisco, an oft-published science and technology writer, is equal to 42 (S=19, P=16, G=7).
  • A barrel holds 42 gallons.
So maybe the next time you're asked for an estimate, you should reply with 42. "It'll take 42 hours to fix the bug that is causing that error we cannot reproduce." "It'll take 42 weeks to develop the application for which we have no requirements." "Since you cannot tell me how many records will be in the database, I estimate we'll need 42 gigs of space." 42. I like it.

Of course, there may be a better, more practical, factually-based way of estimating future needs. Think about your field, domain, or discipline. Maybe you're a server-build professional, maybe you support databases, create applications, establish networks; whatever. Now consider any four years in your past; maybe '97, '98, '99, and 2000, or '02, '03, '04. and '05. I'll bet that your deliverable in the fourth year followed a fairly predictable linear trend when compare with the previous three. A study was done comparing our advanced weather predicting techniques which use Doppler Radar, Satellite Imagery, weather balloons, reporting stations, and historical data. They found that on average we can predict tomorrow's weather to an accuracy of 70%. If however, you simply predicted that tomorrow would be like today, you'd only be wrong 30% of the time (do the math).

What is needed in our world is a "zero-intelligence" model, a formula based on well-defined inputs that can predict behaviours such as user requests. We are all well aware of the volatility of the stock market, yet J. Doyne Farmer a researcher with the Sante Fe Institute, has developed a formula that accurately predicts market busts and boons based on three variables:
  • Market Orders
  • Limit Orders
  • Cancellation Orders

I'll not go into the math here; you can read about it in Jefferey Kluger's book Simplexity, but suffice to say the complexity of the stock market can be understood to a great degree by focusing on these three variables. Likewise, Dr. Lee Goldman found a credible, substantial correlation between three data points and the probability that someone was having a heart attack. Turns out that hospitals are inundated with potential myocardial infarctions and the amount of data that doctors attempt to correlate only make a critical situation worse. Dr. Goldman worked out his theories in the 1970's but they were pretty much ignored, because a "simple formulas couldn't possibly predict what required years of medical training to understand." Eventually, he was validated when the U. S. Navy began to use his approach to avoid having to bring submarines to the surface to get attention to an ailing sailor, only to find out, he wasn't really all that sick.

Predicting how many servers to build out this year, or how long the next web app will take, may seem like a very complicated thing. We tend to find complexity in our life's work. But if you analyze the data, look at historical trends, and stop trying to be perfect - just close; you will find a way to predict, accurately-enough, what your near term future holds. In many cases you can create a growth or capacity prediction without ever talking to your customers - just look at your own records, and by the way - you have the records.... somewhere. In large measure, we'll be judged on improving our time to market numbers this year, and that can be appreciably enhanced by reasonably predicting the workset.

So let me leave you with the challenge of completing Carnac's last prediction (the winner gets a sealed mayonnaise jar!):

Carnac, "The answer is 42."
Ed, "42"
Carnac, ?

Follow by Email