An efficient method to evaluate and enhance sport judges’ performance during competition: a case study in acrobatic gymnastics

Abstract

Who judges the gymnastics judges? How do we measure their accuracy and concordance? How do we know if the judging process is fair? The superior jury has this responsibility. However, they normally lack time to provide effective feedback during competitions.

Using macro functions to process statistical and mathematical statements, we designed and validated an automated Excel-based tool called the Automatic Acrobatic Gymnastics Judges Individual Report Tool to evaluate judges’ performances quickly and easily during competition, automatically creating and exporting individual reports showing each judge’s accuracy and concordance performance on a daily basis, rather than after the competition is ended.

We present empirical data for 76 experienced international judges evaluating acrobatic gymnastics routines in four major official events. A total of 1240 individual reports were analyzed and sent confidentially to the judges during the competition, and 952 were analyzed to evaluate whether this feedback was effective in improving judges’ performance during the competition.

The tool provides efficient and easily understood evidence-based feedback on acrobatic gymnastics judges’ performance during competition, quickly and automatically creating, analyzing and sending individualized information to judges, thus helping with specific Technical Committee scoring control tasks during competitions. We suggest that judges' performances remain high or are enhanced after receiving daily evaluation during major competition events.