Jay Cross helps people work and live smarter. Jay is the Johnny Appleseed of informal learning. He wrote the book on it. He was the first person to use the term eLearning on the web. He has challenged conventional wisdom about how adults learn since designing the first business degree program offered by the University of Phoenix.
Norm Kamilkow, Mike Prokopeak, and the CLO staff must have a crystal ball back at the office in Chicago. How else would they have known, six months in advance, that the U.S. economy would be going down the toilet? The CLO Symposium that just ended at the Hotel Del Coronado focused on the perfect topic for times of economic meltdown — Measuring Success: Learning’s Positive Impact on Business.
Norm kicked things off by drawing an analogy between training and baseball. Baseball is easy to quantify but statistics don’t tell the entire story The Chicago Cubs hold the record for the most wins in a row… but that was in 1906 or so. Ballpark figures can lead you astray.
Baseball scouts once judged new recruits primarily by gut feel. Now it’s a science. Scouts live by metrics.
Similarly, training departments once took chances on things because they looked good. These days it’s imperative that learning leaders make their case with relevant numbers.
I’m going to cherry-pick topics from the Symposium here, and in doing so I’ll call Norm’s that last sentence into question.
Following a keynote by Ed Cohen, former CLO at Booz and now head of Satyam Computer Services’ School of Leadership, we launched into a ThinkTank session chaired by Cedric Coco of Lowe’s with observations from Kent Barnett (Knowledge Advisors), Michael Echols (Bellevue University’s Human Capital Lab), and Patti Phillips (of The ROI Institute).
Issue: Human Capital is the largest unmanaged cost in business. Who should be in charge?
Issue: What are the biggest mistakes in installing HR metrics?
My note-taking took a hit at this point because I could not sit still. I was handed the microphone and sputtered out a diatribe, saying, among other things, that:
Learning professionals have been mouthing the same words about this stuff for the last quarter century, and it’s time for us to wake up and smell the coffee. All the data in the world is not going to do any good if the decision-maker doesn’t buy it. You have to start by getting the manager holding the purse strings to agree on what measures satisfy her that the training intervention has been effective. That’s it. You don’t need to measure everything that moves.
Measuring business impact is an inexact science. What the boss takes as proof is proof enough.
Patti told the audience that business decisions are micro. ROI is the answer. You build up the numbers and that’s your case. I think that if the world worked like that, CFOs could replace CEOs. It ain’t gonna happen. Bold executives are driven by intuition and qualitative factors, not arithmetic. Geez. No wonder learning professionals in most companies are not invited to the table with the senior execs.
Kent: Nobel laureate Gary Becker says just start measuring something. Put a stake in the ground.
Chris Hardy chimed in that DAU has hundreds of courses. They don’t do the micro number. He asked how you measure the performance of the learning organization. The Think Tank session didn’t address Chris’s issue. At heart, Chris was asking about scalability. The conversation in the room implied there was one best way to measure ROI. Obviously, Chris needs something more.
Making strategic decisions is fundamentally different from making operating decisions. Senior leadership uses gut feel, informed judgment, and vision to set direction. Managers at lower levels decide what projects to fund by describing the logic of how they will help carry out the strategy; this is where running the numbers is useful. ROI hurdles help identify the projects with the greatest potential return. They don’t address the big picture.
In a subsequent session, David Vance, former president of Caterpillar University, offered practical advice on metrics. Click the slides to make them legible; these are choice.
Jayne Johnson, director of Leader Development for GE at Crotonville, delivered the final keynote presentation. Someone asked how she measured the on-going effectiveness of Crotonville; she doesn’t. As for cost-justification in advance, no, GE believes in “launch and learn.” Experiment a lot, and keep what works.
This month’s CLO magazine has a wealth of viewpoints on measurement and ROI. Look at the excellent article Value Creation with Human Capital by Bellevue University’s Mike Echols. Read Norm’s Editor’s Letter for caution on spewing out too many metrics.
I may put together a round-up site for information on learning metrics. Here are video interviews with three of the wisest voices on this topic:
Kent Barnett, KnowledgeAdvisors
David Vance, former president, Caterpillar University
Greg Brisendine, Intrepid Learning Systems
Unfortunately, my interview with Mike Echols disappeared into the ether; do read his article; it’s worth your while.