[BETA] Fortnightly

Theory and experience teach that commercial market research

can be of very poor quality. What does that mean

for regulators and utility managers?

How can regulators and utility managers know whether and to what extent to trust commercially prepared market research? After all, the task of disentangling market information from background noise demands a fair amount of institutional knowledge and technical expertise. Moreover, the nature of market research is such that faulty work product easily can be passed off undetected.

The answer lies in an understanding of the nature of market research and the pecuniary incentives faced by commercial market research firms. This understanding provides clues with which to determine the likely worth of research work product.

The Theory of Credence, Experience, and Lemons

Some years ago, a study of peer-reviewed research papers uncovered reluctance on the part of academics to disclose the research data that formed the basis for their work. Of those research authors who were surveyed, several

reported that their data sets had been destroyed "inadvertently" through accidental erasure, hard disk problems, and the like. These responses were interpreted as the academic equivalent of "the dog eating the homework," raising suspicions not only that the authors' data might have been manipulated and falsified in some way, but also that interpretations given to otherwise valid data were knowingly faulty. These suspicions could be neither confirmed or dismissed from the data and analysis presented in the papers. However, they reasonably could be inferred from the signals given off by the authors. Such is the nature of primary research, whether conducted by academics or by commercial research firms.

The world of research is driven by two irrevocable truths. One truth is that information and knowledge are not free but, rather, must be produced at positive (em and sometimes substantial (em cost. The other truth is that informa-tion about the worth of any piece of research is distributed

asymmetrically between producers and consumers.

Intuition suggests that commercial market research firms can capitalize on the "hidden" nature of competitive market information, both by supplying "research" that panders to the implicit desires of clients and by an attitude of indifference toward the quality of research work product. Theory and observation give substance to this intuition.

Market research can be distinguished from other goods and services by the degree to which it is difficult (i.e., costly) to ascertain the quality and value of work product. In the parlance of economics, market research is a "credence" good whose quality and value cannot be ascertained readily. This characteristic distinguishes market research from other goods whose quality and value can be determined either prior to purchase ("search goods") or subsequent to purchase and use ("experience goods").1

Economic theory demonstrates that suppliers of all goods and services have an incentive to provide quality and value only where purchasers are able to detect it.2 Accordingly, the "credence" nature of market research naturally raises the suspicion that research work product might be of poor quality. "Quality" in this context is defined in terms of "validity" and "reliability." A study is valid if