Author: Andy Zynga

It is a profound irony that the more you know about a particular industry, and the more experience you gain in it, the more difficult it can be to move it forward with truly meaningful innovation. But it’s true, thanks to something known as “the curse of knowledge” — one of the most vexing cognitive biases identified by psychologists and behavioral economists. (Another big one is “functional fixedness” — a topic I will save for another day). Cognitive biases are very human and arise from our need to make sense of a situation before deciding on a course of action. As we acquire, retain, and process relevant information, we filter it through the context of our own past experience, likes, and dislikes. Not surprisingly, with every subsequent challenge, our response is increasingly shaped by our knowledge of “how we’ve always done it.”

This is part of why open innovation is so powerful. By definition, it sources valuable ideas and inventions from outside the walls of an organization. That not only brings more brainpower to bear on a problem to be solved, it brings minds that are not constrained by industry conventions.

But if you think that by merely opting for open innovation you will escape the curse of knowledge, you may be wrong. Assumptions based on convention can still undermine the effort because, at the outset of any open innovation, someone has to communicate what is being sought.

Made to Stick authors Chip and Dan Heath share a vivid illustration of how the curse of knowledge leads to communication failures. In an experiment, psychologist Elizabeth Newton asked subjects to choose among 120 well-known songs and then tap out the melody with their finger on a table for a listener to try to identify their choice. When she asked the tappers to guess how likely listeners were to recognize the songs, they predicted a 50% success rate. As it turned out, the listeners correctly identified only 2.5% of the melodies they heard tapped. (Newton’s 1990 PhD dissertation, “Overconfidence in Communication and Intent: Heard and Unheard Melodies,” gives full details.)

Try it yourself — tap away while a familiar tune plays in your head — and you will understand why the answers seemed so obvious to the tappers. In the same way, the knowledge in an engineer’s or technologist’s head (or a group of them) causes them to make assumptions about what should be clear to anyone, while failing to give outsiders the understanding of a problem that would allow them to solve it in a new way.

Sometimes the curse of knowledge leads experts to communicate what they’re looking for at too low a level. Recently, for example, my colleagues and I assisted a large consumer products company attempting to improve its packaging. It sells a perishable product that consumers don’t use all at once, so its engineers had identified the need for a better re-sealing solution. But when we articulated the need, we went beyond describing what would constitute an ideal sealing technology; we specified how much freshness and taste quality had to be maintained over what length of time. (Other packaging performance factors such as ease of use and cost were also stipulated.) Being clear that the need was to preserve food quality, not just to seal a package, affected how solution providers approached the problem. The overall set of submissions was of high quality, as judged by how well each met the criteria for an ideal solution. Most important, the search resulted in a new package innovation -creatively combining different approaches to achieve the goal — which was promptly patented by the client.

In another search, the curse of knowledge made an organization communicate its need at too high a level. This was the International AIDS Vaccine Initiative, which decided to sponsor an open challenge to the scientific community to come up with an effective inoculation against that terrible disease. Unfortunately, defining the Request for Proposals as a vaccine challenge did not yield many high-quality responses. Our advice was to break down the need to a level where scientists who did not think of themselves as vaccine creators would engage. More fundamentally, this was a protein stabilization challenge. Once it was refocused on that critical stumbling block, the technology search brought back 34 proposals from highly qualified scientists in 14 countries. Three of these were sufficiently promising that IAVI funded their further development with $875K in research grants. Even for an organization full of smart scientists — indeed, especially for such an organization — it can take a third party, unencumbered by presupposition, to overcome the curse of knowledge.

The vaccine example underscores that if you define a challenge and its ideal solution’s qualities and characteristics in an application-agnostic way, you defeat the curse of knowledge in two ways. First, the knowledge of the expert sourcing the solution doesn’t translate to limiting assumptions about what form it will take. Potential solution providers are given an understanding of the challenge that doesn’t constrain their ideation. Second, the potential solution providers are less likely to self-select themselves and their ideas out of contention because they don’t think they’re relevant.

I regularly see companies’ open innovation efforts being undermined by the curse of knowledge. They write detailed specifications for the technology they are seeking based on what they have seen work in the past. They draw up exclusion lists that automatically remove certain companies or industries, and the science they have mastered, from their consideration. Without even recognizing that they are making assumptions, they contract their universe and discourage viable submissions.

The only way to avoid these missteps is to place a lot of emphasis on how the need for a solution is communicated up front. In open calls for innovation, we need to clearly communicate to others the real problem to be solved and the benefits the solution must deliver, as well as our own understanding of the chief stumbling blocks and the features a solution will offer.

In our role as innovation facilitators, we have to remember that a little knowledge can be a dangerous thing — and a lot of knowledge can be a curse.

When I urge managers in a company to make open innovation part of their innovation strategy, they get it conceptually. They see that a solution coming from outside is more likely to be a dramatic leap versus an incremental one—simply because it came from a mind not narrowed by conventional thinking. They understand that the infusion of fresh ideas can also open their own R&D engineers and technologists to new and different possibilities and cause them to push their thinking further. At a minimum, they agree that better solutions tend to emerge when there are more options to consider in the first place.

A recent project we did for FVA, a branch of the German trade association VDMA for companies that provide drive-train systems and components, bears this out. Every year, FVA conducts an industry-wide technology search to identify new and emerging solutions, and it shares the results as a service to its member companies. Although the search has been productive, members within the FVA leadership circle wanted to see if there was a better search process that would unearth more solutions and potentially create more value. A vice president of the group had been to a presentation on Open Innovation and was intrigued by the concept. And although he seriously doubted it could work in a field as specialized as drive-train technology, he was willing to find out. FVA commissioned Aachen University to manage a selection process and trial study, and the selection committee found us at NineSigma most suitable to conduct a series of technology searches.

These technology searches took the form of requests for proposals (RFPs) seeking solutions to four different technological needs: 1) durable, non-lubricated gear materials; 2) extra-fine micro particle removal from lubricated mechanisms; 3) translational research in bio-similar particle capture and 4) low-friction hard surfaces. We distributed the RFPs worldwide, within and outside the industry. The results were measured in two ways: How many solutions were submitted by new sources versus known sources? And how many of the solutions submitted were new to the industry versus already familiar to industry engineers?

In all four areas, the new solution providers far outnumbered the known ones. For example, 33 new providers and only 2 known ones addressed the low-friction hard surfaces problem. The new solutions swamped the familiar ones, as well. For example, for the durable, non-lubricated gear materials problem, the search yielded 16 new solutions versus 6 known.

For the kind of manager who believes it’s a waste of money to go looking for smart ideas from “the crowd,” this is the kind of evidence that challenges the skepticism. Moreover, when we talk to companies about how they should connect with innovators outside their business, we encourage them to solicit ideas both from unrelated AND adjacent industries.

I’m convinced this focus on both unrelated and adjacent industries produces the best yield of viable solutions. At the same time, however, I’ve come to recognize a danger in the approach. When it comes time to harvest the actual value of an innovation, everything depends on how well the solution has been implemented across all steps of the value chain—and the more ideal a solution appears to be right out of the box, the more its discoverers might be inclined to think value will materialize automatically. In other words, if a solution seems like it isn’t too much of a stretch, you might be fooled into thinking it will be simple to put into place and start benefitting from immediately.

Think again. Even the most obviously applicable solution has to clear three big barriers to uptake:

It wasn’t invented here.
First, whatever solution your open innovation search yields, it is something that by definition was “not invented here.” And just as biological organisms have antibodies whose job it is to combat intruding organisms, every company has human versions of antibodies who make it their business to fend off foreign ideas. Rather than being expressed directly, this usually comes in the form of challenges like, “Do they really understand our issue?” Or: “the technology is close, but not quite developed enough.” Or, my personal favorite: “Yes, it’s exciting and I think it might actually be more applicable to so and so’s program…”

It might not actually work.
Consider, however, that not all the antibodies are acting out of mindless prejudice; some may be reacting reasonably to heightened risk. Before you completely discount their worries, think about whether there might be something to them. Because solutions found in other industries are often more mature, they can look like a convenient shortcut. They’re more fully developed, further along the R&D timeline, and often easier to modify for your application. However, even mature solutions should undergo rigorous evaluation and testing. This sounds obvious. The common pitfall, however, is that the company doesn’t invest the time and effort into creating and performing the kind of test that will show whether or not the solution will really hold up when it goes into production.
.
The change is disruptive.
Even when testing does prove the solution’s viability, integrating it won’t be painless because the organization will need to adjust its current processes to accommodate it. It is basic change management wisdom that any alteration of work routine is more naturally cursed for the disruption it causes than celebrated for the better outcome it will eventually yield. Managers know this means they must manage expectations and shift mindsets to appreciate the case for change. But again, all this is more likely to be forgotten in the excitement of discovering an already existing solution in an adjacent space.

None of this is insurmountable, of course. It doesn’t even require much beyond the usual carrots and sticks that managers use to bring about any kind of change. Engineers and technologists are as eager for fame and reward as anyone and there’s nothing like the recognition from above to affect behavior. So recognition should be given equally to people who unearth solutions that succeed in market, whether they attracted them from outside or developed them in-house. (The days of celebrating only the inventor-genius who gets the patent should really be over.) And most everyone responds when they are made more aware of competitive pressures. Open innovation offers the appealing prospect of finding ways to leapfrog over competitors still relying on solutions produced entirely within the industry.

Open innovation is an approach every company should consider. Having more technologies to choose from always increases the chances of finding one that has the potential to not only fit the bill, but truly advance the end-product above and beyond the competition’s capabilities. But here’s what most people don’t understand, or they don’t understand soon enough: A plan to switch to any new solution that doesn’t include tactics for overcoming internal barriers to adoption is no better than a plan for failure.