The Reilability of Wine Judging

Brad Post:

Visit most any winery today are you likely to find a showcase of their best wines shrouded in medals and ribbons. The more notable places, if lucky enough, will be adorned not only with traditional accoutrement but the highly sought after acknowledgment and praise of the upper stratum of wine critics, such as likes of Robert Parker and other wino bigwigs.

Of course, I must confess, that I lack a certain sophistication in my personal wine consumption and have also been known to enter a wine contest or two (yes, won some awards too)...but, at the same time my scientific mind says: Hmmm, I wonder about the consistency of wine judges. In other words, can wine judges replicate their own judging? For example, if our judge was evaluating a small sample of wines (10 Chambourcin) at our wine competition, we should reasonably expect they would be able to judge the same wine the same (or at least closely) if we slyly entered two of the exact same wines (from the same bottle) in this round. And if they couldn't, should we not be at least a little suspect of their abilities and question their sway over us?

Funny I should ask...in a recent study in the Journal of Wine Economics, Dr. Robert Hodgson investigated 65 judging panels and concluded only 30 panels came close to consistency in evaluation. By definition, when measurements of the same item are inconsistent over time, then we say the measure is unreliable. So, what is going on here? Can judges distinguish between the noted bell pepper, asparagus, and pineapple sensations when judging or are they simply deluding themselves or are they engaged in an elaborate group-think, self-reinforcing process?

I will avoid the social psychological considerations and briefly focus on the sensation and perception of taste to at least touch on the subject. It is well known that taste and smell are intricately related, such that if you have a bad cold, your ability to detect scent decreases. For example, without scent the potato tastes much like an apple. There are four main types of tastes: salt, sweet, sour and bitter (yes, I haven't forgotten savory umami, so make that five) each with their own distinct taste thresholds.

Some Taste Facts (courtesy of Schaffin (2001) Sensation and Perception):

1. All taste stimuli must be dissolved or soluble solutions.
2. Sour comes from acid compounds; Bitter, from alkaloids; sweets, from nutrients such as organic substances.
3. Temperature effects taste thresholds (e.g., salty foods taste saltier when cooler than 22C or warmer than 32C)
4. Taste sensitivity (ability to distinguish) decreases for most after age 60.
5. Adaptation: Prolonged exposure to a taste solution results in a decrease or complete loss of sensitivity. How many wines can someone taste and still detect their qualities?
   
6. Adaptation-Produced Potentiation: A fancy way of saying that our ability to taste can shift because of adaptation. For example, if you are drinking a substance, such as lemonade and follow that with a glass of cold water the resultant taste is sweet. Here are how the four tastes are effected by the adaptative potentiation of water: A) Bitter - can elicit a sweet taste with water; B) Sweet - can elicit a sour and bitter response from water; C) Salts - can elicit sour, sweet and some bitter water taste; D) Sour - can elicit a sweet water taste.
   

Without going into the anatomy of taste I think it is fair to conclude taste is a fairly subjective and complex phenomenon. Furthermore, I believe wine judging, even when it is done well, is relatively inconsistent and is likely a long ways from anything terribly scientific. So on one hand we have consumers who frequently base their decision to purchase on a particular wine scoring system (generally, a 100 point scale); likewise, we have winemakers who use this information to market their wines accordingly and take pride in a well-made wine because of the evaluation.

I guess the challenge is to decide for yourself, perhaps with the guiding evaluations of the experts, to make a wine purchase decision - or to take a little extra pride in making a good wine for the winemakers. Frankly, I do think the experts know the difference between a very great wine and a great wine...but for me, it just seems like another layer of complication keeping good people away from good wines. And like Andrew Zimern, I say "if it tastes good, drink it".

Hoping his wines will win at the next competition,
Brad