Imagine sitting down at a table with 2 glasses of tequila (or any high-proof spirit) in front of you. Your job is to smell and taste both, and then decide which you like better. Sounds like a fun time, right?

Now imaging having to taste 50 tequilas (or any high-proof spirit) in a single sitting and within a few hours, and choose which you like better. Do you think you could do it? I was given that opportunity recently as a judge for the Spirits of Mexico competition in San Diego, the biggest and most prestigious tequila event of every year.

Together with about 10 other judges, sitting in the same hotel conference room, I went about the task of evaluating 99 different agave-based distillates over the course of 2 days. This task was not easy. I found that after about 6 tequilas, my abilities were diminished. By the 12th tequila, I felt like I was no longer able to trust my judgment. Not because I was drunk (we were wisely instructed not to swallow), but because of palate fatigue.

Some of the others in the group were ‘professional tasters’ who seemed to be able to get through all of these spirits without issue. When things started getting really rough (especially when we were tasting the flavor-saturated extra añejo tequilas), they would swish vodka around in their mouths as a way to reset their palate between glasses. (Hardcore!)

I’m definitely not a professional taster. I love tequila, but I’m just an “end user”, fan, loyal customer, and an advocate for the spirit. I never claimed to be an expert taster, so I can only assume that I was selected to participate as a contest judge because my only experience is that of a consumer of tequila.

My respect goes out to those unique individuals who can taste and evaluate at this level. Their abilities are impressive, and they clearly have special talents. The experience was indeed fascinating, and I don’t regret any of it. In those two days, I learned a lot from them, and I am grateful.

In the past several years, I’d heard from other contest judges that palate fatigue was their biggest challenge, too. When asked if they would judge another spirits competition again, their answer is always “no, I don’t think so.” Now I can relate.

I couldn’t help but start to think that maybe there was a better way, and that our app could help.

A year ago, we met Crystal Murphy and Omar Mercado who run the Monterey Tequila & Mezcal Expo. They had just completed their contest judging, and it followed the traditional process with all of the judges in the same room at the same time. I heard talk of palate fatigue among the judges once again, so I offered to help. Crystal and Omar immediately accepted my invitation.

The Tequila Matchmaker app has always had the ability for a user to rate a tequila blind, but this feature didn’t work for more than one person at a time, so we added a new feature that would make “group blind ratings” possible.

No more palate fatigue.

With this tool, the Monterey Tequila & Mezcal Expo broke new ground. The judges were sent 50ml bottles, labeled “A”, “B”, “C”, etc., which corresponded to an item inside of the app. They could review the samples over the span of month, from the comfort of their own home, and use the app to step them through the process and provide a common set of criteria.

Since I knew all of the tequilas in the contest, I wasn’t able to participate as a judge. However, Scarlet was one of the eight judges so I had a front-row seat right from my own home. She had no idea which tequilas she was tasting. She followed the judging instructions carefully. She took her time, tasted 3 at a time, and sometimes went back again if scores were close to refine her ratings. She rated everything in the same room, at the same time of day, using the same glassware.

I was also able to watch what all the other judges were doing through the back-end administration console used to run the group blind rating function. This way I could identify any judge who may be having technical problems, or needed extra time.

Another issue of concern with traditional competitions, especially among brands who entered the contest, was the issue of transparency. This was yet another area we could address with our app simply by disclosing all of the scores from all of the judges, and making this information public.

By displaying this information in graphical form, you can actually see trends among the judges, their own flavor preferences, and any biases they may have. It provides the justification to the awards.

No more mysterious ratings.

Although there are many spirit competitions in the world, many brands refuse to enter them. Faith in the fairness of the process is usually the reason. Brands that already have an established reputation don’t stand to gain from contests. A win is expected, anything else can only hurt, so why bother?

In a traditional competition, the goal is to award a medal. That’s basically it. If you win, or if you don’t, you aren’t going to learn anything from the experience. There is very little a brand can take away from the experience that will help them improve their product or marketing.

I can’t help but wonder if brands knew they would get useful data from the experience, would more of them enter?

[ View complete results: Monterey 2013 ]

Using the app, we are able to extract the ratings information and show a brand exactly why they rated they way they did. Perhaps they scored high on aroma, but low on finish. Perhaps their tequila is priced too high, or too low? Maybe their tequila is too sweet for several of the judges. By sharing the data, brands can find out.

Contests should be more than medals and awards. They should help improve the category be providing actionable data and objective feedback.

— Grover