Steps to Improve Your Athlete Evaluations
Are you the person who gets handed the pile of paper after an evaluation and gets the prized of doing countless hours of mind-numbing data entry? What about the coach that gets handed a team, and after seeing the kids listed wondering “How did that kid make this team?”
This blog is dedicated to all the coaches who give selflessly to their sport, to make their lives easier, and improve what to do with information we as coaches spend hours gathering.
As a coach, the 3 biggest areas that I saw with the evaluation process of young athletes are:
- Athletes almost never received any formal feedback on their performance after an evaluation
- Coaches spend countless hours doing data entry into spreadsheets to try and make sense of the data
- Parents are often unsatisfied with justification for the placement decision of their kids kids and very seldom receive information to help understand what their child needs to work on and develop
It all comes down to information. A coaches’ time is already spread thinly that they rarely have the time to reformat information in a spreadsheet. Let alone to a “form” consumable for each athlete. As a result, the athlete and his/her parents never understand WHY they didn’t make the top team.
Technology today can solve all that. Having the evaluators gather information through the SkillShark app or website simplifies everything to the point that the young athlete can now receive a report card on their performance before they even have their cleats/skates off. A coach can spend time analyzing information instead of doing days of data entry. Lastly, parents can finally have a clear indication what their child needs to work on in order to be considered for the top team next year.
What Are The Biggest Steps to Ensure a Smooth Evaluation Process?
As coaches we know the importance of planning out everything we do, and a good evaluation is no different. The key things every organization should know well in advance of an evaluation are:
Assigning Player Numbers
The three most common methods for assigning athletes a number so they can be easily identifiable to evaluators are:
- Preassign numbers (e.g. 1-100) and have that athlete come prepared with that number pinned to a shirt or written on their leg. Using hockey tape to create numbers usually does not work well as the tape falls off leaving no way to identify them. The slightest bit of rain can make any papers pinned to their shirt become fragile and tear off.
- Hand out uni-size numbered bibs preassigned to kids
- Handing out unique sized bib/numbers upon arrival at the event. This requires a registration station where someone records the number/color of the bib provided and records that on the website.
The Check In Process
Think about how many people you need working the desk and whether you need power cords to power laptops or charging stations.
Make sure to understand how much time each evaluator will have to measure each athlete when the drill is being performed. You need to know how many stations will be required and if you have enough space. You might need to split into multiple sessions if you aren’t able to run them all at once.
Do I have enough people/equipment to run the drills at each station? Now with technology like SkillShark, having more evaluators doesn’t mean increased data entry time into spreadsheets. You can have as many helpers as you like and you have instant access to the data once the evaluation is over.
How many evaluators will I have at each station? If measuring objective data (ie pitch velocities, 60Yd Dash, etc) you may only need one evaluator but if measuring subjective data on athletes, you’ll want to have more than 1 set of eyes.
How much time will the evaluators require to accurately measure (performance, number of metrics, time of each drill)? It’s important to understand that each added metric will require more allotted time, and that measuring performance may take a more a detailed approach. However you decide to build your evaluation, just be aware of the time each drill will take. Ask yourself, with the method that I plan to run each drill, will the evaluators have enough time to record a score, then see the next athlete fully? People running the drills should communicate with the evaluators to get the right cadence.
Is there a reliable network connection at the location of the event? This determines whether you’ll need to download the app ahead of time, sync before they leave the house, gather data, then sync data to the SkillShark website when they get home or near a cell tower. Knowing this will also help you understand how you’ll handle the registration process or walk-ups.
The traditional evaluation process is riddled with problems when using pen/paper, so why not let SkillShark remove ALL those problems?
The second the evaluation is done, rather than spending 50+ hours doing data entry in Excel only to be left with nothing that can be then sent to each athlete as a report card, coaches and organizers can start sorting kids onto teams.
Users can generate different types of reports:
SkillShark’s Individual Reports
SkillShark’s Weighted Reports
Run multiple reports for the data you gathered. In the cases of Baseball and Softball, run reports for your Top Players, (example: Top Pitchers Top Catchers, Top Outfielders, Top Infielders, etc.) and quickly draft your teams using SkillShark’s interface.
If you have one spot on the top team and are trying to decide who gets it, you can compare multiple players against each other.
Player Progress Tracking
You can track player development over time on objective measurements to show the athlete the results of all the extra effort they’ve put into their development.
One of the biggest issues with pen/paper evaluations
We haven’t even discussed this but here is one of the biggest issues with pen and paper that many who run evaluations know about but may not know how to solve. It all surrounds EVALUATOR BIAS. The bias I’m talking about here is not someone who intentionally gives higher or lower scores, it’s the unconscious bias we are talking about.
Here’s a situation:
We have Bob and Tim who are evaluators at a soccer tryout. Tim is that ‘nice guy’ who doesn’t want any conflicts with parents and doesn’t want to hurt a young kid’s feelings so when measuring subjective scores from 0-10, Tim tends to stay on the high side giving a low score in the group of 7 and a high score of 9 from all 8 athletes.
Bob is more forward and ‘calls it like he sees it’. If he thinks a kid performs poorly then he will give a score as low as a ‘1’ whereas if a kid does well, he will give a high score of a ‘7’.
99.9% of the youth sport organizations if using pen/paper/spreadsheets, will take the numbers from both evaluators, average it then rank the kids from best to worst. (See Diagram 1 below)
Diagram 1. Bob and Tim gathering soccer scores using pen and paper, then ranking the kids.
Unfortunately, in the end, Bob ends up having a much greater effect on a kid’s outcome relative to Tim only because he used a larger range of scores. Let’s take a look at Player 8. Bob thinks Player 8 is one of the worst, and Tim thinks Player 8 is one of the best, but because of Bob’s larger effect on the average, Player 8’s ranking is near the bottom.
SkillShark solves this unconscious evaluator bias by a process called Data Normalization. SkillShark will take the lowest and highest scores for every metric from each evaluator and stretch it out to the full 0-10 range.
See Diagram 2.
By doing this we actually can determine a much more accurate ranking of the 8 athletes. Look at Player 3 who was considered to be the 3rd ranked player when using pen/paper but who actually should have been identified as #1 in the group.
Solving this biggest problem with pen and paper evaluations will remove many of the situations where athletes get assigned to teams they should not be on. The larger the number of athletes in the group in this situation, then statistically, the higher number of athletes will be ranked incorrectly.
Going through trial runs of evaluations and visualizing how things will run the day of the event in advance will solve many of your stressful issues during evaluations. The trend of all sports is definitely leaning towards increasing communication from the sports organization to the athletes/parents so utilizing tools like SkillShark allow for your organization to have data transparency and save 50+ hours per event on average.