Party science: You can’t judge a wine by its bottle/price tag
PROLOGUE: FRUGAL WEDDINGS
When planning our wedding a few years ago, a guidebook recommended that we provide modestly priced wine, citing blind tasting experiments that have found no correlation between people’s enjoyment of wine and its price (at least in people with no training in wine). This matches my subjective experience, but ever the scientist and looking for an excuse to have people over, I organized a blind tasting with some friends to see if we could replicate the results. I purchased four bottles of red wine ranging in price from $3 to $45 (Figure 1). The most expensive and cheapest bottles were cabernet sauvignons, with a zinfandel and pinot noir in the middle to provide some variety.
The most expensive was from Stag’s Leap, a Napa Valley winery that is famous for beating French wines in a highly publicized “Judgment of Paris” tasting in 1976. The victory established California wines as world class and earned the winning bottle of wine a place at the Smithsonian National Museum of American History. The cheapest was Trader Joe’s Charles Shaw Blend, also known as “Two-Buck Chuck,” which is famous for being extremely cheap. All four bottles were purchased at the one Trader Joe’s in Manhattan that is allowed to sell wine.
My six volunteer tasters were asked to rate each wine’s quality on a scale of 1 to 10, with 10 being best. They were also asked match the wine to its grape type (Cab, Pinot, or Zin) and to its description on the bottle. The descriptions were as follows:
- Stag’s Leap: “…a wine with lush fruit flavors balanced by structure and elegance.”
- Layer Cake: “…like a great layer cake, fruit, mocha and chocolate, hints of spice and rich, always rich.”
- Trader Joe’s Grower’s Reserve: “…mouth-filling flavors of ripe berry and plum. Full-bodied with peppery overtones, this excellent red wine compliments full flavored spicy dishes.”
- Charles Shaw: “I cost $2.99.” (I actually had to make this up as the label didn’t have a description on it.)
SUBJECTIVE QUALITY VS. COST
Figure 2 shows each taster’s rating of the four wines. Indeed there is no clear correlation between cost and subjective quality. One way to quantify this is to measure the rank correlation between each individual’s rating and the wine costs and then to compare the mean correlation against a null hypothesis of 0. Using the rank correlation metric Kendall’s tau, there is actually a tendency for a positive correlation (mean tau=0.23, t(5)=1.64, p=0.16, d=0.67, 95% CI: -0.13 to 0.58). A possible confound of this analysis is that different types of grapes may be significantly affecting subjective preference. To control for this, I compared just the two cabernets (Figure 3). Again, there is a tendency for the more expensive wine to be rated better, but the difference is small and far from reliable (t(5)=0.34, p=0.75, d=0.14).
IDENTIFYING TYPE OF GRAPE AND WINE DESCRIPTION
Figure 3 shows how well each taster could identify the grape used to make each wine. Although all but one taster scored better than the median accuracy expected by chance, a two-tailed sign test shows that this not clearly reliable (p=0.22). As you might expect, identifying which flowery label description goes with which wine appears to be hopeless (Figure 4). Here only one taster does better than a chance. The labels might as well have been randomly assigned (two-tailed sign test, p=1.00).
In general, the subjective preferences of our blind tasters (whom I take to be representative of neuroscientists and neuropsychologists, if not the general population) did not clearly correlate with the wine’s price tag. Moreover, our tasters could not clearly discriminate between the different types of wines and the wine’s descriptions were as good as random. Granted, this is a very small sample of tasters and my analyses don’t attempt to make inferences beyond the four specific wines we tasted. Thus, we can’t conclude from this that there is no correspondence between wine price/type and taste. However, I was struck by the lack of subjective difference between the quality of the Charles Shaw and Stag’s Leap given that their prices spanned an order of magnitude. Is a tendency for such a subjective difference enough to justify spending 10 times as much on a bottle of wine at your next dinner party?
After seeing our results, my answer is “no” and this is backed up by a much larger and careful 2008 study by Goldstein and colleagues who found:
Individuals who are unaware of the price do not derive more enjoyment from more expensive wine. In a sample of more than 6,000 blind tastings, we find that the correlation between price and overall rating is small and negative, suggesting that individuals on average enjoy more expensive wines slightly less. For individuals with wine training, however, we find indications of a positive relationship between price and enjoyment. Our results are robust to the inclusion of individual fixed effects, and are not driven by outliers: when omitting the top and bottom deciles of the price distribution, our qualitative results are strengthened, and the statistical significance is improved further. Our results indicate that both the prices of wines and wine recommendations by experts may be poor guides for non-expert wine consumers.
So find wines you like without emptying your wallet. And if you have guests who are less rational and turn up their noses at a bottle of Two-Buck Chuck because of the price, serve them using a decanter (or beaker). 🙂
-David M. Groppe
P.S. If you’re looking for evidenced based tips on wine, Consumer Reports provides some recommendations. You have to subscribe or go through a library to read them though. However, some general tips (e.g., wine myths vs. reality and food pairings) are available for free. For what it’s worth, Steven Levitt of Freakonomics fame recommends the book The Wine Trials as a buying guide.
Goldstein,R., Almenberg,J., Dreber,A., Emerson,J.W., Herschkowitsch,A., Katz,J. (2008). Do more expensive wines taste better? Evidence from a large sample of blind tasting. Journal of Wine Economics, 3 (1), 1-9 DOI: 10.1017/S1931436100000523
~ by eeging on February 26, 2014.