4. Why the invention rate exceeds the innovation rate

In this Section we shall consider two topics. First, the institutional innovations that have resulted in an acceleration of discovery, invention, and innovation in the late 20th century. Second, why the rate of innovation needs to be increased to absorb the higher rate of invention.

4.1 Acceleration of Discovery, Invention, and Innovation

Most inventors in the 18th century English industrial revolution were intelligent artisans with little technical training. In the early 19th century countries industrializing could adapt English technology to their economic conditions. For example, during the early development of manufacturing in the US, most manufactured items were adaptations of British technology rather than developments from new discoveries. Slater, a British mechanic, in conjunction with Brown, a Providence businessman, set up the first textile mill in the United States in 1793, but it wasn't until the 1820s, when Lawrence made improvements to the designs copied from the mills around Manchester that textiles became consistently profitable in the United States24. Furthermore, the mechanical skills used in developing textile machinery were transferable to the development of machinery for the factory production of other goods25.

The development of new products in the first hundred years generally required very little resources and very few people. Inventors created most inventions using only an empirical trial-and-error development procedure without extensive applied research. The classic example of this method is the folklore surrounding Edison's development of the electric light bulb. With this methodology, inventors during the first hundred years created a high rate of adaptations, inventions, and improvements simply by intuitively exploiting surface knowledge.

In the 19th century firms also made numerous innovations using an improvisatory strategy. In the United States, labor was more expensive than in England; hence, successful adaptation of technology mandated using less labor than the original English technology. One American contribution to manufacturing in this period was the concept of interchangeable parts in mechanical equipment. This innovation reduced labor both in assembly and in subsequent repair. A second major American innovation in manufacturing was assembly-line production, which increases efficiency of labor by greater specialization. Cyrus McCormick had employed both concepts in the production of reapers by 185026.

In addition, private innovators improved the organization of manufacturing firms by creating the manufacturing corporation organized by functions27. The corporation was a desirable institution for raising capital, as it limited the liability of the stockholders to the value of the corporate assets. The functions of the corporation, such as production, marketing, and finance were departmentalized, and each department was headed by a vice president managing a specialized staff. The methodology of innovation with respect to business organizations was an intuitive, improvisatory strategy with imitation. The corporate form for manufacturing is, in fact, an adaptation of the corporate form developed to promote railroads, and this organization, in turn, was an adaptation of the joint stock trading companies.

In the early 19th century in order to promote a rapid economic advance, a large investment to promote basic research was not necessary. As long as manufacturing could rapidly advance through intuitive trial and error experimentation to adapt English technology to American conditions, there was little need to consider creating new ideas that would lead to new inventions. But as the US caught up to England technologically and the focus of inventions expanded into chemistry and electricity, the factors needed to promote manufacturing and invention changed.

First, both the workforce and inventors needed a higher level of education. As they industrialized, states developed a system of primary and secondary public education. At the federal level, the Morrill Act in 1862 promoted the creation of public universities. Up until this time the only basic research conducted in this country was exploration such as the Lewis and Clark expedition. But in the latter part of the nineteenth century, the major universities began to emulate the German model of a university by emphasizing research as an important university goal28. Also, engineering became a university degree program. Moreover, a number of individuals founded new universities, such as John Hopkins, specifically for the purpose of promoting research. Thus, the creation of public education and the research university in the second half of the 19th century laid the foundation for the subsequent acceleration in discovery. By 1900 there were eight research universities


By the end of the 19th century major US corporations imitated the German innovation of corporate research and development organizations. Corporate research and development organizations developed a more systematic approach to invention by separating the activity of invention was separated from the activity of production. This lead to greater inventive efficiency as the focus of invention expanded to chemistry such as improving the steel making process and making improvements on electric inventions such as the light and telephone. Such inventions generally required considerable applied research to reap their full economic potential29. Scientifically trained engineers and scientists were more effective than trial and error tinkers because they could eliminate numerous alternatives from theoretical considerations without having to test them in experiments30. Consequently, research and development organizations increasingly hired trained engineers and scientists. By 1900 a few large corporations such as At&T and General Electric had initiated R&D programs.

Public funding of research was initiated by the the Hatch Act of 1887, which began the public funding for university research in agriculture. The impetus for expanding the scope of public funding to all types of research was the success of applied research in World War II in producing such technology as the atomic bomb. The National Science Foundation was created to publicly fund basic research in most fields through a peer group review system of grant proposals31. Most of the basic research funded by the US government was performed in research universities and government laboratories. Public funding increased in response to the Soviet cold war threat and the specter of Soviet technological domination raised by Sputnik. Up to the present, high levels of expenditures for basic discovery have been sustained by a growing realization of its importance in military and economic competition.

Much applied research is now publicly funded by government department through government laboratories, universities and private corporations. For example, the Department of Energy through its Office of Industrial Technologies funds non-nuclear energy projects of small companies, individual researchers and university researchers through competitive reviews in nine energy industries. To promote commercialization the firms hold the intellectual property rights. Since the creation of OPEC the increased prices of energy have created incentives for private firms to become much more energy efficient. For political reasons, US policy wants the US to promote technological advance that reduces our dependence on foreign energy supplies. Over the years technology developed through the Department of Defense has had major commercial applications. For example, Boeing developed the Boeing 707, one of the first successful commercial jet airplanes from its knowledge in developing a jet used to refuel fighter planes in the air. The development of the IC had some DARPA funding in its early stages because the military want to build smaller guidance systems for military rockets.

Public funding of all types of research created the research university as the institutional arrangement to accelerate the rate of discovery. Political support for the creation of the research university with a teaching role was a coincidence of the need to educate the baby boomers and the need to dominate science. Major state-supported universities were accordingly transformed from teaching institutions to research institutions through the expansion of PhD programs. The PhD student became both a junior researcher and the ubiquitous TA or teaching assistant, an arrangement which enabled the faculty to spend more time on research and at the same time teach more students by using TAs. With generous public funding the number of research universities increased in number to about one hundred today.

The magnitude of this expansion in research can be indicated by the fact that in the second hundred years the number of universities conducting research has increased by an order of magnitude, and there are more scientists living today than have lived in all previous times. In addition to universities, government laboratories and some private laboratories are engaged in basic discovery. While the expansion has occurred in all fields of study, funding has been highest in fields corresponding to the perceived potential for advances in military weapons, economic competition and social value. The research university is now common in advanced countries and is also being imitated in developing countries.

Besides the expansion of research universities, there has been a great expansion of corporate research and development laboratories staffed by university-trained specialists in the 20th century. By World War I, over fifty corporations had laboratories32, and today most corporations, intent on improving existing products and inventing new ones, conduct organized research and development at various levels, from basic knowledge research for major breakthroughs to the most mundane forms of applied research to create minor improvements in existing products. After WW II the US government also began to fund firms to perform applied research through agencies such as the Department of Defense. This funding is much greater than funding for basic research. Unlike basic research, most of this research is directed at achieving specific objectives. Firms have had powerful incentives to perform this research because they capture the benefits of any resulting intellectual property. Corporate research and development is a characteristic of firms in advanced countries and is increasing the developing world.

A US innovation in the creation of new industries which evolved from the 19th century is the startup. In the US new industries are usually created by the competition of numerous small firms (startups), a few of which become giants absorbing most of the smaller firms in the growth process. Examples are automobiles and each new type of computer. In the 20th century this process has been fostered research universities creating new ideas by new institutions of venture capital to finance the growth of the startups and incubators to promote their initiation. In the US and other areas the pragmatic interest in promoting research universities is to create an active business culture of startups and venture capital. Starting with the natural growth of Silicon Valley and Route 128 near Boston, state governments have actively tried to create such a business culture around their research universities. Texas has had considerable success in Austin. This process is being imitated throughout the world.

This institutional structure of new product development is in constant flux. In recognition of the need to improve the transfer of knowledge from public government research laboratories into private economic products, the federal government has create agencies such as the National Technology Transfer Center, NASA Commercial Technology Network, and the National Technical Information Service to aid the transfer of technology from government laboratories to the private sector.33 States are creating institutional structures to speed the technological transfer from state research universities to private firms. For example, in Texas the Institute for Creative Capitalism at the University of Texas as Austin has a program in technological transfer and the nonprofit Texas Technological Transfer Association at Texas A&M University coordinates technological transfer throughout the state.34

The greatest improvement in innovation has come about by the creation of a consulting industry for promoting innovation and imitation in large firms and government. Some of these firms have arisen as new activities of accounting firms such as Andersen Consulting and Price Waterhouse and Coopers consulting and others have always been consulting firms such as McKinsey and Bain. By contracting a consulting firm to implement an innovation or imitation or an innovation, the contracting firm does not need to hire a large staff for a one time activity and in case of failure the contracting manager can blame the consulting firm. If a consulting firm implements a successful innovation in one firm, they frequently acquire considerable additional business in implementing imitations in other rival firms. Given the large overhead of current consulting firms they focus their attention on large firms. Ernst and Young has developed an online consulting service called Ernie to develop a consulting business with smaller firms.

In addition to better evaluation technology, a very small number of industries have made improvements in the intuitive improvisatory strategy for innovation. In agriculture, for instance, the adoption of a better innovation implementation strategy, the separation strategy, has accelerated not only the process of discovery and invention but also the process of innovation itself. Starting with the Morrill Act of 1862, legislation created a system of agricultural experimental stations for research and a system of county agents to transmit their successes to farmers. Agricultural research stations now experiment with production techniques such as fertilizer application as will as inventions such as new seeds and equipment. This means that innovation itself is accelerated by the adoption of a separation strategy wherein innovation as well as invention benefits from good, statistically designed experiments35.

Recently, an institutional arrangement for a separation strategy has been created for the 381,000 small manufacturing firms. In 1988, Congress directed the National Institute of Standards and Technology (NIST) to create an institutional framework called the Manufacturing Extension Partnership (MEP) similar to the agricultural framework to promote innovations in performance enhancing technologies in small manufacturing businesses. The incentive for this public program was that the productivity advance in small manufacturing has been much less than that of large manufacturing. The cost of consulting services has been too high for small firms to make the most innovative use of advancing technology.

The MEP program has three parts. The first are federal- state funded, non-profit Manufacturing Extension Centers that are designed help small and mid-sized firms innovate using new manufacturing technology. Each center is expected to become focused to the needs of local industry. There are also two national centers. One focuses on improving the production of the 52,000 printing establishments. These centers and the Manufacturing Technology Centers created by NIST have test facilities to experiment with new technology.

The second part of the MEP program are the State Technology Extension Programs (STEP) that provided the agents that transmit the new knowledge to small and mid-sized firms. Each state specializes in the industries of its state, For example, New Jersey Manufacturing Extension Partnership has initiated its efforts in promoting production in metal working & Machinery, Rubber & Plastics, and Electronics & Instrumentation. The state extension programs provide the field agents to analyze particular small and mid-sized businesses and make recommendations.

The third aspect of MEP is creating Links in computer networks such as the WEB to provide small and mid-sized manufacturers accurate, up to date information concerning MEP programs and technological services plus databases of useful information. The network will link all the offices of MEP with partnership organizations such as federal laboratories and universities.

Linked to the WEB site of MEP are numerous success stories. Nevertheless, it will probably take several decades to determine a good estimate of social rate of return of the MEP. To the extent that this new institutional arrangement demonstrates good social performance, it will be adapted to other types of innovation.

For large firms there are less economies of scale in a program such as MEP. To adopt a separation strategy, private firms can create an institutional arrangement, such as a consortium, to perform the research as a separate task. One such successful consortium is Sematech, which performs research in production techniques in semiconductor manufacturing for large semiconductor firms.

4.3 Evaluation of Discovery, Invention and Innovation

A central thesis of this chapter is that in the 20th century the rate of invention has accelerated faster than the rate of innovation. One contributing factor is that the rate of discovery which promotes invention has accelerated relative to the rate of discovery which promotes innovation. That is scientists working in the natural and biological sciences have made much larger gains in understanding natural and biological processes than scientists working in the social sciences have progressed in their efforts to produce knowledge about social behavior. The accumulation of knowledge promoting inventions in many cases has advanced to state that inventors are able to make quantitative predictions, whereas the accumulation of knowledge promoting innovations has only advanced to state in most cases that innovators can make qualitative predictions.

The advances in scientific knowledge have led to the creation of simulation programs which enable inventors to evaluate the performance of alternatives without experiments. The advances in the social sciences, however, have provided fewer tools for analyzing alternatives, and the poor forecasting performance of most social science simulation programs limit their usefulness. This means that to evaluate an alternative for innovation, an empirical implementation is generally required. This is true even for many innovations in production because current knowledge in social sciences is insufficient to simulate accurately the impact of alternative organizations and incentive systems on worker and manager performance.

One reason for greater discoveries promoting invention than innovation is the disparity in funding. Although the government funds considerable research, most of its resources go toward creating a public feedstock of ideas for the private development of inventions. As the benefits of social sciences research are general and frequently controversial, the social sciences lack powerful interest groups promoting their interests and consequently, current funding for the social sciences is only 5% of the NSF budget36.

A second reason is that investigators in the innovation sciences have much more difficulty obtaining systematic observations than investigators in the invention sciences. For social scientists and business researchers systematic observation generally conflicts with proprietary considerations and privacy. Firms generally desire to restrict the flow of such information to prevent competitors from understanding their competitive positions. Between private parties, of course, information policy depends on voluntary release of information as modified by required disclosures. For example, producers of food products must disclose the ingredients, including additives, on the label. Private parties typically filter voluntarily released information in such a way as to promote their own interests; consequently, such information does not constitute a representative sample. Investigators in most areas of discovery promoting innovation are limited in their ability to obtain representative samples of behavior they wish to study. In a few cases this is also true in the natural sciences. For example, hydrologists can not obtain data on oil flows in private reservoirs from the oil firms without their permission.

Given the severe restrictions on obtaining representative samples, most empirical research on political economic behavior is frequently carried out using data samples collected for oblique reasons: hypothesis testing is restricted to those hypotheses for which data is available. Naturally, this problem inhibits the development of the business disciplines as well as the social sciences. For example, an empirical management scientist can rarely obtain representative samples of data on the relationship between incentives and performance of middle managers across companies in an industry.


Even if observations are improved nothing will be learned if the variables under study do not if fact vary. Individuals, firms, and the government generally pursue their respective goals with the best current knowledge. This means that for many variables of interest, goal-seeking behavior can result in very little variation. For example, rival firms in an industry can each use very similar production techniques, organizations and incentives. This is especially true for the federal government, where equal treatment before the law and an aversion to experimentation inhibit variation. Private firms have conducted experiments in incentives such as the famous Hawthorne experiments37, which were conducted by General Electric to determine what factors influenced worker productivity. Since World War II, the government has conducted a limited number of social experiments in such issues as peak load pricing and guaranteed income38. Currently the amount of variation in public and private policy (especially systematic variation from experiments) is far to low to promote discovery which in turn promotes innovation.

A second factor contributing to the disparity between the invention rate and the innovation rate is that inventors use a more advanced learning strategy than innovators. Currently most invention experimentation has advanced from trial-and-error to more systematic applied science in research and development laboratories. Invention is a separate activity from production so that inventors employ a good experimental design based on cost-benefit considerations.

In contrast, in almost all cases the strategy for innovation implementation remains an intuitive, improvisatory strategy. This means that most business, innovation remains largely the creation of the heroic figure of the entrepreneur. As this is an intuitive estimation and control problem, the ability to experiment is subordinate to the overall profit objective of the firm. One of the few examples of a separation strategy for innovation is agricultural production innovation where applied research is performed at agricultural research stations and the results are transmitted to the farmers by agricultural agents. For this latter strategy to be effective there must be a large number of individuals, firms, or governments with similar tasks and incentives to fund joint experimentation. To some extent this condition has been met in the joint research consortia.