Not Your Father's ROI


The July issue of Chief Learning Officer is now available online. It features an article in which Jon Husband and I delve into how to measure the impact of learning in the network era.


Productivity in a Networked era: Not Your Father’s ROI

Today’s networked era requires a new way to make investment decisions that incorporates intangible assets and more accurately depicts how value is created.

The industrial age has run out of steam. Look at General Motors. Look at Chrysler. We are witnessing the death throes of management models that have outlived their usefulness.

The network era now replacing the industrial age holds great promise. Networked organizations are reaping rewards for connecting people, know-how and ideas at an ever-faster pace. Value creation has migrated from what we can see (physical assets) to intangibles (ideas). Look at Google and Cisco.

Understandably, seasoned executives, chief learning officers among them, are having a devil of a time shifting from the industrial age mindset of logic, certainty and bounded constraints to the network gestalt of interaction, self-organization, unpredictability and fewer limits to potential. The pressure is constantly on to meet quarter-to-quarter revenue and earnings targets that in turn accentuate the need to take decisions that support achieving those targets. At the same time, we are shifting into an era in which knowledge work and learning occur where re-engineered business processes collide with a participative and interactive ecology of information flows.

How can a chief learning officer hope to make informed judgments in this continually expanding networked environment that’s flowing ever faster, spreading power among its members and producing outsized impacts in unpredictable ways? What to do?

One cherished industrial age concept that is proving particularly difficult to let go of is return on investment (ROI). But like Pontiacs and Oldsmobiles, old-school ROI’s day in the sun is waning. In an environment of continuous flow and interaction, there’s a need to consider an emerging metric: return on investment in interaction (ROII). The working definition of ROII is the observable development of capacity and capability to create economic values out of intangibles.

Consultants and smart-aleck MBAs will tell you if you want to sell a big project internally, you’ve got to talk ROI. It’s the language senior managers understand. Being fluent in ROI talk addresses the “hard” tangible returns stemming from an investment in a specific project or capacity. It is supposedly the secret handshake that gets you to the inner circle of those who control budget dollars.

Let’s look at what ROI was, how it needs to be changed and how to recapture its original intent in the network era, in which continuous learning and knowledge work are becoming inseparable. As Steven Forth of the LeveragePoint division of the Monitor Group puts it, “Too many people who talk about the ROI of learning are focused on being precisely wrong rather than directionally correct.”

Traditional ROI
ROI is an accounting and financial management concept businesses use to decide where to make investments and to assess the success of investment decisions after the fact. ROI reduces both return — R, what you expect back — and investment — I, what you expect to put in to numbers — making it possible to compare one investment opportunity to another. The numbers tie back to categories on the balance sheet and income statement, (i.e. tangible assets and hard-dollar returns).

ROI is what you get for your money, divided by what you spent to get it. It’s R/I expressed as a percentage. In a business culture that is skeptical of nonnumerical reasoning, ROI implies disciplined, mathematical rigor. It ties actions to intended results. It shows the logic of how results will be achieved.

Companies set up ROI hurdle rates to gauge whether there will be sufficient payback over a reasonable and defined period of time to justify the capital invested to acquire additional capacity or produce a defined result. Companies also use ROI to evaluate past performance. In retrospect, what was spent and what benefits were received? This simplifies making the case for similar projects in the future.

What You Can’t See
In the network era, things you can’t see are more valuable than things you can. Thomas Stewart sounded a clarion call in his book The Wealth of Knowledge with his exhortation that building the capacity to create economic value through things such as innovating and enhancing brand reputation is as important, or more important, than generating specific results from a specific initiative. Twenty-five years ago, intangibles accounted for less than a third of the value of the S&P 500. Today, intangibles can make up more than 80 percent of that value.

On paper, Google’s net worth was about $30 billion at the end of 2008. That’s what it paid for computers, buildings and stuff you can see, minus debts and the expense of wear and tear. Stock market investors value Google at $125 billion. Where does the extra $95 billion come from? Intangibles.

“Intangible assets — a skilled workforce, patents and know-how, software, strong customer relationships, brands, unique organizational designs and processes, and the like — generate most of corporate growth and shareholder value,” wrote NYU Professor Baruch Lev in Harvard Business Review in June 2004.

Corporate decision makers say their goal is to increase shareholder value. In a networked, information-based environment, shareholders value brand, reputation, ideas, relationships and know-how. These assets don’t appear on the balance sheet, but more and more often they provide a corporation’s competitive edge. These most important aspects of the business aren’t recognized by old-school accounting and therefore aren’t factored into ROI calculations.

Organizations that make decisions based solely on things that are sufficiently tangible to be counted directly might as well consult a Ouija board to set their goals. Leaving the most important sources of value out of the ROI equation is not conservative — it’s foolish.

Measuring intangibles involves making judgment calls, so managers often exclude intangibles from their ROI calculations. Several purported authorities on calculating ROI suggest taking intangibles into account by putting them on a list but refusing to estimate their value. This leads you to comparing numbers to words, apples to oranges.

You Must Manage What You Can’t Measure

“You can’t manage what you can’t measure” was a mantra of industrial age management. Adopting F.W. Taylor’s brilliant research and models, generations of managers have carried stopwatches and pored over measurements in a continual quest to make things work better. Efficiency was the road to riches in the slower-moving, predictable industrial age, and measurement was the proof of the pudding.

While the measurement meme works when your goal is to tweak the way you’ve been doing things and other operational decisions, it doesn’t apply to making judgment calls, strategic choices or disruptive innovations.

Executives manage immeasurable things all the time. The more powerful the executive, the more likely he or she is involved in effectiveness — doing the right things rather than doing things right. Intuition, judgment and gut feelings guide these more important decisions. Qualitative assessment often can make up for a concrete numeric result.

Make a hypothesis of cause and effect. Interview a statistically significant sample of the workforce to see if the hypothesis holds up. Often, results obtained from social science research methods will produce more meaningful feedback than solid counts of the wrong thing.

The old “can’t measure, can’t manage” dodge doesn’t free businesspeople from making decisions under conditions of uncertainty, and the network era ushers in uncertainty in spades.

Making Decisions in the Era of Networks
A business network is a group of individuals or organizations that are linked together by factors such as values, visions, ideas, financial exchange and collaboration to further the ends of the corporation. Business networks share common characteristics with all networks:

• They multiply like rabbits because the value of a network increases exponentially with each additional connection.
• They naturally become faster and faster because the denser the interconnections, the faster its cycle time.
• They subvert hierarchy because previously scarce resources such as information are available to all.
• Network interactions yield volatile results because echo effects amplify signals.
• Networks connect with other networks to form complex adaptive systems whose outcomes are inherently unpredictable.

Intangibles travel via networks, and networks are the infrastructure for doing business in the future. An overarching caveat here: Strategist and practitioner Stuart Henshall said trust is critical. “It’s the one qualitative factor all networks depend upon.”

ROI, the tool we once used to evaluate projects in stable times, clearly is not up to the task. The impacts of collaboration-based knowledge work are accelerating. However, the Western world is lurching from crisis to crisis, and executives are under constant pressure to perform. It’s difficult for them to give up models they understand well.

In the future, organizational effectiveness will be defined by the interaction of workers in a networked environment. Exchanges of information and knowledge are what make peoples’ brains work on a purpose and what gets the imagination going to formulate pertinent responses. However, the return on networked collaboration is less tangible than the results generated from stable and ordered sequential tasks that dominate the efficiency-oriented industrial era.

So we face the problem of convincing managers to adopt new mental models that incorporate the intangibles generated by a whole system, the organization and its interconnected networks. Making a business decision to invest in new ways of working is a complex process involving many factors and intricate tradeoffs, such as:
• Risks must be weighed against rewards.
• Short-term vs. long-term aims.
• Alignment with strategic initiatives.
• Scarce resources call for shrewd horse trading.

Identifying and Measuring ROII
The focus in this new world of work is to do what’s important and involve those who know what’s important, why it’s important and what they know (or know how to find out) about a problem or issue. To begin measuring increases in productivity and value in a networked social computing environment, we propose return on investment in interaction (ROII), derived from the principles of Metcalfe’s law of networks.

Some core assumptions about ROII :
• Continuous flows of information are the raw material of an organization’s value creation and overall performance.
• Information flows are carried by links, alerts, RSS feeds, search engines, aggregation and filtering of content.
• All leading vendors’ productivity platforms now feature collaborative social networking and computing.
• These platforms’ architectures facilitate purposeful cross-silo communications and exchange.

In a June 2008 “The Network Thinker” blog post, social networking pioneer Valdis Krebs outlined four generic metrics that are becoming widely accepted as leading to observable, tangible measurable outputs:
• Increase in size of network.
• Increase in internal network connectivity.
• Increase in connection to valuable third parties.
• Increase in number of projects formed from all three factors above.

It’s important to note here that we are not proposing a definitive answer, but rather the need to debate and clarify the issues. Each of the principles outlined above proposed by Krebs addresses the productivity of network activity. Unpacking them can help us understand how to begin to assess ROII.

Increase in Network Size
If we follow the logic of two heads are better than one, and therefore X heads are better than two, in social- and knowledge-building networks, we can expect to find:
• More engagement with an issue.
• More analysis by more people.
• More input from more people.
• More possibilities that may have been overlooked.
• Quicker and more comprehensive analysis.

CapGemini’s relaunch of its knowledge management initiatives offers a great example. Its initial program wasn’t working: 20 percent year-on-year usage decline, three and a half year average document age and an average of seven years to refresh current knowledge. It relaunched informally via word of mouth and within six months had 27,000 of 83,000 employees using it, involved in 900 communities exchanging information and pertinent knowledge on a daily basis. All that activity came without spending a single dollar on formal internal communications or training.

Increase in Internal Network Connectivity
Increases in network connectivity involve the degree, frequency, density and concentration of information flows between nodes in a social network. The organization is able to define better business and market intelligence, more frequent and tangible customer centricity and responsiveness, and clear instances in which cross-silo knowledge exchanges lead to tangible results.

At CapGemini, six months after the informal launch, the 900 communities of practice were using 500 forums, 500 wikis and more than 250 expertise- or project-focused blogs. Business results as defined in the previous paragraph are not long behind.

Increase in Connection to Valuable Third Parties
In today’s increasingly interconnected environment, ignoring external parties that have an interest in products or services is a guarantee for trouble. These interested parties talk about brands or offer up opportunities, and organizations that respond rapidly and effectively to issues gain competitive advantage.

Ford Motor Co. opened up its launch of the new Sync service to customer input and conversation. With 1 million page views in less than 12 months, the company experienced a significant reduction in customer-service support costs as 10,000 customers began to offer each other tips, pointers and answers. Further, it began to receive significant tangible market intelligence as engaged users began to share product integration and compatibility experiences, tips and tricks.

Increase in Number of Projects
ROII is obvious when the scope, degree and intensity of interaction increase due to implementation of the three above principles. An increase in the number of projects creates value as people learn to work together effectively in networks, putting informal learning to work on resolving issues, creating opportunities and generating activity that enhances an organization’s reputation for listening and responding effectively.

Fast Company recently published an article on Cisco Systems’ large-scale adoption of social computing as the main means of working with information and knowledge. CEO John Chambers said that as a result, Cisco has gone from being able to focus on three to five strategic initiatives at a time, to now working on 26-27 strategic initiatives in parallel.

Informed Judgment
The heart of the matter is providing decision makers with an informed business case that ties investment to the results that it brings. A solid case describes results in business terms, such as increased revenue, better customer service, reduced cost or speedier time to performance.

Network returns are asymmetric, so simplistic count-’em-up approaches are no longer viable. But how can one make a solid network-era case to an executive who is still playing by yesterday’s rules?

The answer is to improve the corporate network as a continuous process, not as a project with a hurdle rate. Improving network performance need not be all-or-nothing. It can be implemented in small stages. Break major decisions into numerous low-risk incremental decisions. Instead of making one major decision a year, CLOs might look at boosting network results as a series of monthly decisions. Continuous monitoring of the statistics of ROII would guide mid-course corrections.

Life was simpler when you could measure performance by counting the number of widgets produced, shipped or sold. Given that the networked workplace and markets are here to stay, how can managers begin to adapt and refocus long-standing mental models about what and where to invest precious energy and time? An effective response to this conundrum is qualitative assessment.

Create a hypothesis and use existing techniques — surveys, focus groups, facilitated brainstorming — to find out what employees and customers are doing and how they want to work together. Then, check it out with a wider sample of the workforce to see if it holds up. It’s clear we are moving rapidly into a networked world in which responsiveness, innovation, gaining competitive advantage through learning faster and embedding knowledge into products and services are all important.

In a world of intangibles, we need to contribute to the productivity, viability and profitability of any given enterprise. We should rethink and expand our methods for making judgments about where, when and how we invest in the ongoing interaction between our employees and customers. That is the return on investment in interaction.

12 thoughts on “Not Your Father's ROI

  1. Thomas Stone

    Much thanks Jay and Jon — this is a great article! I hope ROII catches on because I think you are defining an important concept here.

    I have a couple of quick comments to add. For several years now we’ve been using wikis internally at Element K, and one of the striking things I hear from people and see myself is the increased amount of “informal learning enablement” that is provided. What is enabled is the capturing of conversations and other informal learning that was going on already, because now people have a simple place to document: “Hey, good thoughts Joe — when you get back to your desk, please document that in the wiki.” Then overtime the networking effects you describe multiply these benefits as more people have access to that content, actually interact with it and improve it, and this leads to a virtuous cycle. And all of this process is visible to management — indeed to everyone — instead of the (less frequent) informal learning and knowledge sharing being completely unseen prior.

    Another tidbit to toss in is that while I think blogs, wikis, and forums can provide a lot of value when used well in organizations, the technologies that are easiest to understand in the light of your new ROII concept are internal social networking and internal micro-messaging. These involve quantifying relationships — whether two-way “friend” or “connection” relationships, or one-way “follower” relationships. I think the latter in particular, the model used by Twitter, will be very powerful for organizations, as I’m reading more positive results from the use of Yammer and the like practically every day.

    The only caveat with all of this of course is to avoid “spinning one’s wheels”, for instance by engendering more conversations and idea sharing as an end in itself, rather than as a means to greater productivity, greater innovation, etc. Not everyone needs to be connected to everyone else in an organization, nor talking with everyone else about everything. At some point, some invididuals will be wasting time, could be flooded with too many tangential facts or conversations, and so on. So those tasked with ROII analysis need to be careful to avoid perverse incentives to increase the numbers (e.g., the number of network connections and conversations occuring) for their own sake, and make sure that the growth is in fact productive and meaningful. In most cases, this will occur naturally and won’t be an issue (and taking this concern to far would obviously kill the goose that lays the golden egg, such as “locking down pages in a wiki” can). But it is worth mentioning because anytime you introduce something new — in this case, paying more attention to your people network and rewarding its growth — negative unintended consequences and side effects can result to get you side-tracked. Positive unintended consequences, of course, are much desired!

  2. Pingback: We Need Appropriate Measurement for Appropriate Management | A Chronicle of a Learning Journey

  3. Howard

    I believe your conclusion (expand the methods for making judgements) and your basic premise (current ROI method are inadequate) is spot on. However, I would disagree with the suggestion to abandon measurement for gut instincts. I elaborate at my blog,

  4. Jay Cross Post author

    Howard, I would try to respond but I cannot find your entry at that URL. jay

  5. Mary Adams

    Jay- Thank you for this wonderful discussion. I am a great believer in looking at knowledge assets using the approach of the intellectual capital movement (of which Stewart was one of the early thought leaders). This looks at human, structural and relationship capital as an integrated system. Measuring the system requires looking at all the pieces together. I think this is what you are saying with your ROII.

    One of the ways I have been playing with is creating models of the systems to help facilitate a discussion of how to manage it–including cost to build, cost to maintain, capacity and, yes, return.

    I have some slides here that model Google’s search business as an example. I would love to hear what you think.

  6. Pingback: Your favorite 2009 posts on Internet Time Blog

  7. Holly Chun

    Great post! I think I’m witnessing humanization of technology. People’s minds open. It is like a new age of enlightenment. Technology is not without feelings or flesh. Technology is based on peoples’ feelings and real people with real lives. Technology is only the means. Human beings are once again acknowledged as the most important resources, and this direction is correct. It was wrong to think that machine can replace human beings. The human minds and human feelings, the way socialization evolve and augmented by technology revolutionizes the world’s direction and ways of thinking.

  8. Pingback: Harold Jarche » CCLD KSEN Workshop follow-up

  9. Pingback: Daily 02/09/2011 | LINKING ONLINE TO OFFLINE |

  10. Pingback: Harold Jarche » Training Evaluation: a mug’s game

  11. Pingback: August 2011 « S.A. E-Learning Newsletter

Leave a Reply

Your email address will not be published. Required fields are marked *

four × = 32

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>