A study of literary critics was recently conducted and the results have been posted at Harvard Business School’s The Working Knowledge journal.
Executive Summary:
The professional critic has long been heralded as the gold standard for evaluating products and services such as books, movies, and restaurants. Analyzing hundreds of book reviews from 40 different newspapers and magazines, Professor Michael Luca and coauthors Loretti Dobrescu and Alberto Motta investigate the determinants of professional reviews and then compare these to consumer reviews from Amazon.com.
Key concepts include:
- The data suggest that media outlets do not simply seek to isolate high-quality books, but also to find books that are a good fit for their readers. This is a potential advantage for professional critics, one that cannot be easily replicated by consumer reviews.
- Expert ratings are correlated with Amazon ratings, suggesting that experts and consumers tend to agree in aggregate about the quality of a book. However, there are systematic differences between these sets of reviews.
- Relative to consumer reviews, professional critics are less favorable to first-time authors. This suggests that one potential advantage of consumer reviews is that they are quicker to identify new and unknown books.
- Relative to consumer reviews, professional critics are more favorable to authors who have garnered other attention in the press (as measured by number of media mentions outside of the review) and who have won book prizes.
Author Abstract
This paper investigates the determinants of expert reviews in the book industry. Reviews are determined not only by the quality of the product, but also by the incentives of the media outlet providing the review. For example, a media outlet may have the incentive to provide favorable coverage to certain authors or to slant reviews toward the horizontal preferences of certain readers.
Empirically, we find that an author’s connection to the media outlet is related to the outcome of the review decision. When a book’s author also writes for a media outlet, that outlet is 25% more likely to review the book relative to other media outlets, and the resulting ratings are roughly 5% higher. Prima facie, it is unclear whether media outlets are favoring their own authors because these are the authors that their readers prefer or simply because they are trying to collude.
We provide a test to distinguish between these two potential mechanisms, and present evidence that this is because of tastes rather than collusion — the effect of connections is present both for authors who began writing for a media outlet before and after the book release. We then investigate other determinants of expert reviews. Relative to consumer reviews, we find that professional critics are less favorable to first time authors and more favorable to authors who have garnered other attention in the press (as measured by number of media mentions outside of the review) and who have won book prizes.