EDITOR’S NOTE: Goizueta Newsroom regularly shares news on the school, its faculty, staff, students and alumni. But we also take pride in sharing vital pieces of research from the world-class business school faculty. Look for regular updates on published pieces of research like this one.
Title: Evaluative Schemas and the Mediating Role of Critics
Authors: Greta Hsu (UC-Davis); Peter W. Roberts, Professor of Organization and Management, Goizueta Business School; Anand Swaminathan, Goizueta Chair and Professor of Organization and Management, Goizueta Business School
Journal: Organizational Science (2012)
Critics are influential. When it comes to restaurant meals, movies, books and wines, critics can be a key source of information for consumers who can’t otherwise know a product’s quality prior to purchase. But critics also give guidance to producers by spelling out – to greater and lesser extents – the evaluative schemas that are used to define quality. In a new research paper, Peter W. Roberts and Anand Swaminathan, of the Goizueta Business School, along with Greta Hsu, a colleague at the University of California-Davis, propose that when such evaluative schemas are made clear, producers can better anticipate the judgments of critics and set prices accordingly. Taking the example of the United States’ winemaking industry, the researchers, conducted an exhaustive analysis of over three decades worth of wine reviews, ratings and prices.. They found evidence that prices are less erratic when critics use clearer schema for evaluation.
When producers try to win the favor of the consuming public, they often must first subject themselves to evaluation by market mediators, such as critics. The schemas or quality standards used by mediators are transparent in some cases and opaque in others. Consider the case of diamonds, where stone characteristics are appraised according to a clear and common schema developed by professional societies; i.e., the four C’s. It’s quite a different story with the Motion Picture Association of America, which doesn’t provide much in the way of rationales for the parental advisory ratings issued for films. In intermediate cases such as that of the winemaking industry, critics publicly issue detailed reviews along with their wine ratings, but the underlying evaluation structure isn’t quickly discerned.
The researchers focused their analysis on wine reviews published by Wine Spectator, a magazine launched in 1976, as the United States became a serious wine-producing country. Reviews in Wine Spectator have a narrative component that analyzes the characteristics of a wine as well as a numerical rating on a 100-point scale. This idea of scoring wines was first introduced by influential wine critic Robert Parker and these measures now exert a strong sway over both public perceptions of quality and the price winemakers can charge. To analyze the degree to which words used in reviews clearly indicate the evaluative schemas used in the rating system, the researchers conducted a textual analysis of reviews published in the magazine since the early 1980s. Theanalysis focused on 320 keywords. Across 23 wine varietals, there existed significant variation in the extent to which critics used similar language to describe wines of similar quality. In some varietals, there was no correspondence. But in others, reviews of similar-quality wines had approximately one-quarter of the key words in common.
The common language is indicative of clearer evaluative schemas, the researchers wrote, a conclusion reinforced by supplementary analysis of 674 reviews that noted a wine’s high value proposition. In most cases, when critics reported being surprised about a wine’s low price at the end of a blind tasting, it was in a varietal category where the evaluative schema was less clear. This supplementary finding suggests producers of those wines were less clear about the prices they might charge because they were less clear about what the critics were actually looking for.
The final step in the researchers’ analysis brought together the findings about schema clarity with data on list prices for 20,019 wines. When evaluative schemas were clearer, list prices (which were set before the producers knew what ratings the wines would receive) were less variable around expected levels. A supplementary analysis showed that this effect wasn’t so pronounced for producers who had not previously received any critical coverage, as well as for producers who were less specialized and those with lesser reputations. The findings fit with the notion that some producers are less concerned about what the critics think because they use different marketing rationales when setting prices.
When critics stick to clearer schemas over time, producers have a firmer foundation for their pricing decisions, the researchers concluded. There’s a tension in this relationship, however, since the power of critics often stems from their status as market leaders rather than simply scorekeepers. But at the same time, critics who try to distinguish themselves by promoting evaluative schema that clash could undermine the legitimacy of all criticism.
“Further exploration of the issues we raise above will allow organization theorists and economic sociologists to move past repeated demonstrations that critics have important effects on markets and market order and begin to address important questions about how these effects are realized,” the researchers concluded.
