FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 


Dearest KCBS Judges - I have a bit of a complaint

 
Post new topic   Reply to topic    The Smoke Ring Forum Index -> Competition BBQ
View previous topic :: View next topic  
Author Message
I_use_the_force



Joined: 10 Oct 2012
Posts: 19

PostPosted: Mon May 11 15 4:11 am    Post subject: Dearest KCBS Judges - I have a bit of a complaint Reply with quote

I've been finishing mid pack for the past few years, and am sick of it. The absolute most frustrating thing in the world of BBQ has to be not knowing what you're doing wrong. I try new things, perfect old techniques, produce great entries, and can get a perfect 9-9-9 from one judge and a 6-7-7 from the judge right next to them (it happened last year at the Greer, SC competition)

IF YOU ARE A KCBS JUDGE:

A) FOR THE LOVE OF GOD AND ALL THAT IS HOLY IN BBQ, FILL OUT A #*$*()@#$() COMMENT CARD!!! It doesn't matter whether you're the most experienced judge who knows everything or a complete noob. Your scores as judges NEVER line up with judges sitting beside you, and I need to know why!! If you score a person with anything less than a perfect 9 you need to fill out a comment card as to why!!! If the KCBS organizers had an ounce of sense, COMMENT CARDS SHOULD BE REQUIRED! (Are you listening over there at the KCBS?! If not, I'll be happy to join an organization that does require comment cards - hint, hint Memphis).

I've been doing this over 5 years now, and I STILL have no better than an educated guess as to what I do wrong on my entries. Over that time period, I've received a total of about 5 comment cards. We do 8 contests a year, that's 40 contests so far, with each contest having 6 judges. That's 5 comment cards out of 240 possible comment cards. IT'S PATHETIC!!!! STOP BEING LAZY. I DON'T CARE IF YOU WANT TO GO HOME. I DON'T CARE IF IT TAKES LONGER. FILL OUT THE COMMENT CARD! I WAIT FOUR HOURS AFTER I'M DONE WITH TURN IN'S JUST TO READ HOW WE'VE SCORED. I STAYED UP ALL NIGHT AND SPENT HOURS PREPARING THAT PRODUCT FOR YOU - YOU CAN AT LEAST GIVE ME A FEW SECONDS AND TELL ME WHY YOU GAVE ME A 6,7, OR 8!!!!!

B) While a comment card that tells us something is better than no comment card, and trust me, it's very much appreciated, if you ding me on an entry, please try to make your comments more objective than subjective. Details that consist of more than one 2 syllable word would also be appreciated. What I'm trying to say is if you ding my boston butt for being over cooked, try to use the KCBS guidelines as your guide for wording your response. Write a comment that says, "-1 pt for saltiness, -1 pt for overcooked" or "-2pt for saltiness". Try to tie the comment in with the judging standards given by the KCBS rulebook for each category - if necessary, keep a rulebook at your judging table as a reference.

C) TAKE IT FROM ONE JUDGE TO ANOTHER
Here are my scores from the first two competitions we did this year based on category.

Contest #1
Appearance: 9,8,7,6,8,8 Range: 4 points Variation: 44.4%
Taste: 6,8,8,7,8,7 Range: 3 points Variation: 33.3%
Tenderness: 6,7,9,7,8,9 Range: 4 points Variation: 44.4%
Comments Cards: 1 - said "Too Salty"

Contest #2
Appearance: 9,8,9,8,7,8 Range: 3 points Variation: 33.3%
Taste: 8,9,8,8,8,7 Range: 3 points Variation: 33.3%
Tenderness: 7,9,9,7,9,7 Range: 3 points Variation: 33.3%
Comment Cards: None

The key here is the variation in scores. In every single category, there's a huge difference of opinion on each turn in. Now, which piece they got could explain some of that, but not the frequency with how it happens. Besides, KCBS drops the lowest score, but that doesn't really seem to help matters. I won't even touch the 'table of death' conversation as that is an entirely different matter, but judges shouldn't always be off 33-44% from one another in their scoring every single time!

I am a BJCP craft beer judge - the BJCP is the homebrew equivalent to the KCBS. We reference the beer category before judging each beer whether it is required or not. By reading the guidelines it helps us become more familiarized with the categories at each contest we judge. We also give them comments on how they can improve their results - which is priceless. I know most beer categories like the back of my hand now, and there are more than 27 categories and 70 styles. KCBS has 4 - FOUR - and the judging results have never been consistent. If I can walk up to a judge and say, "Give me the criteria that makes up the perfect pulled pork entry?" and I get 10 different answers from 10 diffrent judges - there's a big problem there. I highly recommend following the BJCP judge certification standards and contest programs as a model. With the BJCP, multiple judges add up their scores to give the entrant a total score out of 50. All in all, judges being more than 2-3 points off in their assessment of a beer is an extreme rarity - and that's the OVERALL score out of 50, not the category difference like we see in KCBS. That's a variation in judging of less than 10%, and probably closer to 5% - not like the 33-44% that I've seen from the KCBS since we joined 5 years ago. So YES! EVEN THOUGH PEOPLE'S PALLETS ARE DIFFERENT, AS LONG AS THEY ARE TRAINED PROPERLY AND REMINDED ON A CONSISTENT BASIS OF THEIR JUDGING GUIDELINES, THERE CAN BE A NEAR CONSENSUS AMONG JUDGES.

I have a lot of judges that are friends of mine on the circuit. They've routinely told me that certain peers will often judge by their personal preferences (such as tenderness in ribs) rather than by guidelines given by KCBS. I hate to break this to you, but WE DON'T COOK FOR YOUR PERSONAL OPINION, we cook by the guidelines set out by the KCBS. Don't ding my entry because you don't agree with the KCBS guidelines and then not tell me about it. Judges like that are absolutely worthless, and are qualified to judge a tiny tot beauty contest, much less BBQ.

D) Know that proper comment cards mean that ABSOLUTE WORLD to us backyard warriors. IT IS THE SINGLE GREATEST WAY TO HELP US IMPROVE OUR PRODUCT! . I have often thought of leaving the KCBS circuit because of the shear number range I get back from judges on the scorecard and lack of explanation. It is, by far, one of the most frustrating things I've ever seen any type of competitive organization put members through. It's the equivalent of a baseball player hitting a ball and thinking that by reaching first you should get a run - only to find out you actually have to reach home plate.

CONCLUSION
I apologize for my negative tone, but this is stuff that needs to be said. This is FIVE YEARS of built up frustration from wasted time, money, and finishing mid pack or worse. No one should be this disappointed when they pour their heart and soul into their craft, and it has zero to do with where we finish. It has 110% to do with improving as BBQ chefs, and the craft we love. So I'm begging each and every judge, please fill out the comment cards every chance you get from now on!
Back to top
View user's profile Send private message
SoEzzy
Site Admin


Joined: 13 Oct 2006
Posts: 13184
Location: SLC, UT

PostPosted: Mon May 11 15 6:28 am    Post subject: Reply with quote

I've been judging for 9 years and competing for 7, KCBS or similar judging criteria.

Judging has always been and probably always will be contentious!

But here are some observations from both sides of the divide.

KCBS runs on 30 minute between turn in's.

It can take 5 - 10 minutes to get the box's to the table, then 4 or 5 minutes to get the appearance scores written down, another 4 or 5 minutes to get everyone of the 6 judges at the table to have meat on the plate, (in the correct square), this is complicated stuff you know? So that 30 minutes is possibly down to 12 minutes, so then cleanse the palette to start and take your first bite, 30 - 45 seconds, assess, judge, now 20 seconds to write it down for the scores. Cleanse the palette again 20 seconds, repeat with second entry, 1:25 x 6 is 8.5 minutes, so with everything "running right" there are 3.5 minutes left before the next meat category.

If people are going to have to write a comment, they should do that as they put down the important scores. But most tend to wait and put them down when reminded, at the end in that 3.5 minutes, and they really don't pay attention, so who knows if the comment is for you or for anyone else.

You can then get confusion with competitors, I witnessed a competitor having to turn in ribs from two different racks, one rack was sweeter and the other more spicy, I was there to help on the turn in table, and having competed for a couple of years at that point, knew what he'd done and why. After the contest, he got comment cards for his ribs, out of 6 judges he got 2 cards, one said too sweet, the other said too spicy, the competitor went ballistic, chewing on the promoter, the reps and on me. After being up since Friday morning, preparing, cooking, turning in the best they could, he'd forgotten about the two styles of ribs. When I pointed out what he'd done, and asked if the racks were the same, he admitted they were not and went back to apologize to the promoter and the reps. So even when the judges do their best and fill out the comment cards, sometimes there are circumstances that make those comments irrelevant.

This is an organization problem, because with volunteer judges who already give up 3 - 4 hours, you still need good table captains, who know what they are doing, to make the system work, too often table captains are one off volunteers, dragged in to fill out the judging spots, sometimes they are first time judges, who took the class the day before that contest, and don't arrive early enough to take a judging spot.

A good table captain can make a big difference to the judging abilities of new judges, reinforcing, guiding and helping to adjust attitudes of new judges! The other side of their job, is to check the scoring and bring strange scores to the attention of the contest rep, if there is a difference of more than 2 points across the score cards. So if you get an honest 7 score, the spread is a possible 4 points, from 9 - 5, there being no 10, only a score of 4 or lower would need to be checked by the table captain. If you should get an 8, it's 5 and below, and with 9's its 6's and below that would be queried! But the table captain may not be experienced, may not get to sample each box, (if there are only 6 pieces in the box, for example), and may not be paying attention to what's going on.

Good table captains have eyes in the back of their heads, and are paying attention to facial expressions, body language etc. To keep an eye on the subliminal information they could pick up, (if only they knew it was there)!

There is generally no room on the tables for each judge to keep their judging book on hand, and they will soon end up covered in sauce and become unusable.

The judging meeting is meant to remind the judges of their responsibilities, and with a good table captain, or an experienced judge who can do a quick reminder / critique of the chicken category, it does help to bring judging more into line around the table.

Your scores.

Quote:

Contest #1
Appearance: 9,8,7,6,8,8 Range: 4 points Variation: 44.4%
Taste: 6,8,8,7,8,7 Range: 3 points Variation: 33.3%
Tenderness: 6,7,9,7,8,9 Range: 4 points Variation: 44.4%
Comments Cards: 1 - said "Too Salty"

Contest #2
Appearance: 9,8,9,8,7,8 Range: 3 points Variation: 33.3%
Taste: 8,9,8,8,8,7 Range: 3 points Variation: 33.3%
Tenderness: 7,9,9,7,9,7 Range: 3 points Variation: 33.3%
Comment Cards: None


Contest #1
Appearance 1 x 9, 3 x 8, 1 x 7, 1 x 6, the 1 x 6 is dropped, so you now have 3 x 8, 1 x 9, 1 x 7 average 5 x 8.
Taste 3 x 8, 2 x 7, 1 x 6, the 1 x 6 is dropped, so you now have 3 x 8, 2 x 7, average 5 x 7.6.
Tenderness 2 x 9, 1 x 8, 2 x 7, 1 x 6, the 1 x 6 is dropped, so you now have 2 x 9, 1 x 8, 2 x 7 average 5 x 8.

Other sanctioning bodies run different scoring systems, some score to 10, some have a decimal point and a 9 is a 9 but it could be a 9.7 or a 9.8 with the different sanctioning body.

If comment cards are compulsory, but judges don't care, (because they've never been a competitor and don't understand the usefulness of honest answers), you are still stuck with their written words, accurate or not, true or not.

I personally would rather have the judges able and willing to write down the scores they give each team, and for teams to be able to connect with judges during those 3 - 4 hours, and give a run down on their judging for those teams that want it.

Having friends, family, other competitors who are brutally honest, and whose opinions you trust, can be used along with the score cards, to gain some additional knowledge about what's happening with your appearance, taste, & tenderness scores.

At the end of the day, I don't think there will be any method that will satisfy competitors, unless they do it like BBQ Pitmasters, where you actually get to see the judges critique each of 3 or 4 teams. But again that's only a possible 7 x 3, (21 total) critiques for a 30 minute show! Think how long that's going to run for 4 meats, 6 judges, and 24 teams trying to get information from those 6 judges! Then the teams moving on to another table of judges for another 4 meats, 6 judges, 23 other teams... x 4, you'll never get to the end of it.

If you ask those that take calls and walks on a regular basis, those teams that do the same things week after week, the same way, and sometimes they take a grand championship, and the next week don't get a call. That's the frustrating nature of the game! Bring your own judges, competitors or friends to eat you food and comment honesty on it, and IMO you'll know more about what you are doing and where you are going.

The other thing to do is to take the KCBS judging class, (if you haven't done it already), and go and judge a few contests. I recommend it to all competitors, just to see that other side of the turn in table.
_________________
Here's a change Robert.

I still work here!
Back to top
View user's profile Send private message Send e-mail
I_use_the_force



Joined: 10 Oct 2012
Posts: 19

PostPosted: Fri May 15 15 3:59 am    Post subject: Reply with quote

I appreciate seeing a judges side of the equation ezzy, and had a few additional thoughts I'd like to share (so we can chat about the 'table of death' now Very Happy).

1) I really think KCBS should review their turn in process. Instead of a staggered category (ie the 12, 12:30, 1, 1:30 turn in times), they should stager the teams. Based on the number of judges available, you would schedule the team turn in times. Teams A-F (or however you want to categorize the teams) would turn their butts in at 12, Teams G-R 12:30, Teams S-Z 1:00 - that way, by staggering the teams, you can have the exact same judges judge all the entries in a category. It has a ton of benefits:

*The table from death and table from heaven would be eliminated.
*Not that it happens, but rigging any contest would be almost impossible since each judge would only have one category.
*The teams would still be able to stagger their turn ins, so you wouldn't have a team having to turn in say butts & ribs at the same time.
*I'm a programmer, so a simple algorithm could be made to assign turn in times, and assign judges to a particular category.
*Since you have judges assigned by meat instead of by a table number, you can cut down on the 5-10 minutes it takes to get the boxes to the table. You could even further cut down this time by preassigning boxes to a particular table. i.e. You have a judging category for Butts, but there are too many entrants, when teams pick up their boxes, it could be Butt1-TeamX, or Butt2-TeamX...there'd be absolutely no confusion. Box table assignments could also be done by a computer algorithm almost instantaneously.
*You could be very, very precise with turn in times, so the judges could possibly enjoy hot food. Instead of a turn in time at 12 and 12:30, you could actually do team turn ins based on 5 or 10 minute increments.

Sticking with the current time schedule, there wouldn't be any additional time requirements placed on the judges, and we could eliminate the table of death almost entirely.

Chicken Turn In:
Teams 1 - 12: 12p
Teams 13-25: 12:30p
Teams 26-37: 1p
Teams 38-50: 1:30p

Ribs Turn In:
Teams 38-50: 12p
Teams 1 - 12: 12:30p
Teams 13 - 25: 1p
Teams 26-37: 1:30p

Boston Butt Turn In:
Teams 26-37: 12p
Teams 38-50: 12:30p
Teams 1 - 12: 1p
Teams 13-25: 1:30p

Brisket Turn In
Teams 13-25: 12p
Teams 26-37: 12:30p
Teams 38 - 50: 1p
Teams 1 - 12: 1:30p

BAM! NO MORE TABLE OF DEATH!!! The only downside is judges probably would prefer to taste more than one category. However, as far as fairness is concerned, I'd think that would be a small price to pay.
Back to top
View user's profile Send private message
SoEzzy
Site Admin


Joined: 13 Oct 2006
Posts: 13184
Location: SLC, UT

PostPosted: Fri May 15 15 5:30 am    Post subject: Reply with quote

The difficulties as I see that are two fold!

1) You have to have judges that want to eat 24 bites of chicken, or ribs, or pork shoulder or brisket... put me down for ribs or brisket and I'd have a blast, put me down for 24 bites of chicken or pulled pork, and I'll need extra time and a bucket to heave into,

2) Judges attend at their leisure and not at their cost, no matter what you do, you'll never get a situation where the table of death does not exist. Many times at competitions I've heard s--t from judges that made me want to throw up. I can't tell you the number of times I've heard, "I marked it down... because I can cook better than that at home"! or "I marked it down because I don't like pork"! You are a BBQ judge at a BBQ contest and 1/2 the categories are Pork, what sort of judge do you really think you are even being there!

When I hear that and I'm a judge or I hear that and I'm a competitor, I will report them to the rep, and the contest organizer, to see if we can all avoid that thought process the next year round.

The table of death only really needs 2 bad judges, there are often more on "table 3", but you only need 2 bad judges to kill a score on any team.

If there are only 24 teams then everyone get the table of death and it evens out, but as soon as a contest has 5 tables and 30 teams, then 6 teams could miss the table of death, and now there is inequality for the other 24 teams that did get the table of death.

To me the real secret to the whole thing is training, and banning, the old carrot and the stick, bring in the judges, have good table captains that have judged contests and are willing to have additional training, to make sure that the newbie judges are split among all the tables evenly, the same way they try and split up family members. Once seated and going through chicken, there could be a designated table guide judge, I try and do this if no one else steps up, before the score cards and handed in and before the platters are cleared, I'll ask the table what they thought of team 1, team 2 etc. I ask them for their comments and for their scores, (some quibble at that, but the scores are written down already and I'm not asking them to change their scores, just to think about them).

Only after everyone else at the table has had their say about each entry, would I offer my opinion, (not that my opinion is always right), but I've judged / cooked a lot of bbq and I try and tell it as I see it. Often you will see a difference in scores between chicken and ribs off the tables that have a chance to actually review their scores. Unfortunately I've been told not to discuss scores with teams or judges. If you take the KCBS judging class, you will see in the Rules, that rule 17 states you are not allowed to talk to the teams about any scores you may have given, because it upsets the teams if you get the information wrong, (but if you were able to record your scores and the team numbers, you could honestly answer questions from teams, if you were allowed to hang around and speak to the teams once the score cards were removed and the teams could be told their turn in # as different from their team box number).

Keep thinking on things, speak with reps and promoters when you can, talk to other competitors and judges, and I'll bet that in 10 or 15 years, we may see some improvements!

The other side that may help is to stay involved with the KCBS and the members of the board, I let my membership lasp, because we were down to 1 contest a year in state that was KCBS, (now with the Sam's Club event, we are at 3 contests a year), I was cooking them, but did not see the need for $40.00 membership to do that. There are a whole lot of folks that know the system is not perfect, and they need support of the ordinary members of the KCBS to be involved and to help push for issues to be resolved, it's always sounded like a whole load of politics, I got into BBQ to get away from all that stuff, not to plunge into it more.
_________________
Here's a change Robert.

I still work here!
Back to top
View user's profile Send private message Send e-mail
BRBBQ
BBQ Super Pro


Joined: 01 Mar 2008
Posts: 1294
Location: Council Bluffs, IA

PostPosted: Sat May 23 15 10:40 pm    Post subject: Reply with quote

I can feel your pain and I don't compete, I've judged around 10 contest and at 2 different contest I judged, their wasn't enough judges, so they've grabbed joe blow off the street to be a judge, this person has never judged in his life, so how is that fair to you, I say NO! As far as contest, I'd say every contest has a different judge eating your food, so each person has a different taste, Sucks No doubt. I don't judge to my personal taste, I judge the meat.. Years ago I emailed Merl (KCBS head honcho) about eating bloody chicken and his response was, if I can find one good piece of meat on the turn in, then judge it on that one good spot Confused Went off topic sorry.. I'd say KCBS needs judges that go to more class's thru out the year, the class is shotgun blast training, when I was a new judge even though I went to kcbs class, I was still a green horn and didn't understand how much effort the Teams put into the contest, and my judging skills weren't really that good..These days I know different and pay attention to details,,Good luck in the future.
_________________
XL Baby Stumps
Back to top
View user's profile Send private message
Old Dave
BBQ Pro


Joined: 04 Nov 2005
Posts: 686
Location: Coatesville, Indiana

PostPosted: Sun May 24 15 8:32 pm    Post subject: Competition Pictures Reply with quote

Here are the pictures this gentleman posted to another forum with the same complaints.









As a judge and cook with over 17 years experience, I would tell you that rather than pounding the judges or system, you need to work on your presentation to start. I would then advise you to take a competition class to help you with your competition cooking.
_________________
Old Dave
Ribs & Bibs BBQ Team
http://olddavespo-farm.blogspot.com/
Back to top
View user's profile Send private message
k.a.m.
BBQ Mega Star


Joined: 12 Dec 2007
Posts: 25938
Location: Southeast Texas.

PostPosted: Sun May 24 15 11:02 pm    Post subject: Reply with quote

The reality is no two people have the same taste in food. Trained or not moist pork in my mouth may be dry in yours.
Judges are human not robots all the classes in the world will not produce two identical folks that can say the same thing.
Today's judges may favor your food while next week or month you flop.
I cook mainly IBCA where the judges are everyday folks off the street with no more training than what is instructed by the head judge before you start.
Debbie and I also judge final tables when we are not cooking and sometimes I will ask myself how a certain tray made it to the final table because of the quality of food handed me.

This year a new rule was implemented into the IBCA that states all final table food must be discarded at awards, some folks were pissed about this because they want to taste what won that day. My opinion is I could care less! It is water under the bridge as that same 1st place brisket next week may not place at all.

My opinion on comment cards are the same they are useless to me. I know if my turn ins are not hitting consistently I need to adjust and that is what I do.
_________________
Always remember slow and steady wins the race.

Hybrid Cooker
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    The Smoke Ring Forum Index -> Competition BBQ All times are GMT + 8 Hours
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group