Judging Wine is Tricky for Somms
In an experiment, wine experts were blindfolded and given the same wine to taste and evaluate three times. In each tasting, the tasters rated the wine on a scale of 80-100, and their ratings varied by as much as four points from one tasting to the next. A wine rated 91 in one tasting might be rated as low as 87 or as high as 95 on the next.
Not a single judge in this study was a certified sommelier, nor were any of the judges given sensory wine training. When wine judges were required to have at least a Level 3 Certification, these issue disappeared.
Red or White Wine?
Another study found that those judges with consistent ratings in one year were far less consistent in other years, indicating that consistent performance by a judge is simply luck. It’s worth repeating that these weren’t random people off the street, but expert judges at the wine competition at the California State Fair, a one of the most well-established and prestigious competitions in the country.
Expert wine judges even have trouble distinguishing red wines from white. In 2001, a study was conducted in which 54 wine experts were asked their opinions on two very different wines, a red wine and a white wine. In reality, the two wines were actually the same white wine, one with food coloring added.
Not a single one of the experts noticed that the wines were the same, and described the red wine with words like “jammy” and other words typically reserved for red wines. The experts also remarked on flavors yielded by the “crushed red fruit.
However, this study was repeated with graduates of a sommelier program that offered sensory training. The results were completely reversed, with over 86% able to identify the wines were identical.
Fine Wine or Not?
Another study, and one that has been repeated with different wines, involved a bottle of wine labeled as a fancy grand cru and another labeled as an average vin de table. As in the earlier study, the two wines were really the same wine. The experts in this study characterized the wines in nearly opposite ways. The grand cru was described as “complex,” “balanced,” “woody,” “rounded,” and “agreeable.” The vin de table was portrayed as “weak,” “light,” “flat,” “short,” and “faulty.”
Wine ratings can be influenced by a variety of factors unrelated to the wine, including time of day, other wines tasted, and how long it’s been since the taster has eaten. Based on these factors, a taster’s rating of a given wine might vary several points from one tasting to the next.
Again, when this study was repeated with Level 3 Graduates with sensory training, the results were quite different. Over 73% of the participants identified that the wines were the same. Furthermore, only 2% of the participants were influenced by factors other than the wine.
A study published in a 1996 issue of the Journal of Experimental Psychology found that wine experts were unable to reliably recognize more than about four of the flavor components of a wine, though most critics typically claim to recognize at least six.
Further tests have shown that only well trained sommeliers can reliably recognize more than five flavors in a wine.
Who Loves Cheap Wine?
A recent meta-analysis of wine reviews conducted by a behavioral economist found that reviewers use “cheap” and “expensive” terms in very specific ways.l Cheap terms are used often, while expensive terms are used infrequently. The study found that it is possible to estimate the price of a wine based simply on the language used in the review. The researcher concluded that words used to describe expensive wines fell into three categories, specifically:
- Dark words: Words like intense, smoky, velvety, or smoky
- Exclusive sounding words: Elegant, old, or cuvee
- Single, defined flavors: Tobacco or chocolate
Less expensive wines were typically described with terms opposite to these, specifically:
- Lighter words: Pleasant or refreshing
- More common words: Value, good, clean or enjoy
- Less defined flavors: Fruity or juicy
Not only are cheap and expensive wines described differently, they are also paired with different foods. Cheaper wines are more often paired with foods like chicken, pizza, or burgers, while more expensive wines might be paired with shellfish, pork, or steak.
A 2008 study of more than 6,000 blind tastings found that those with wine training were more likely to enjoy expensive wines more than cheaper wines. In other words, if you aren’t attending a wine school –especially one with a sensory training element– you may not appreciate all the complexities of fine wine.
Education is Everything
Bottom line: if you love wine, you owe it to yourself to become educated. In this day and age where wine programs are available across the country, there isn’t any excuse to not sign up for a sommelier course at your local wine school.