How accurate are the stats when recorded?

[jefffff34]

I have started a large project on a player that played the majority of his career in the 1990s. It is very statistical based, as he is someone that I believe was thought of as being extremely high skilled, and my project is trying to prove that, and to see how he would be compared to the elite of today's game.

I just completed the first game of the project's analysis. The project recorded all the usual stats, and also involves a few stats that I have thought of, that are very good in analysing things such as a players influence on the game. Anyway, I thought I would check on AFL Tables, to check the stats with what was recorded back in the day, as I was unsure whether or not a soccer off the ground to his own advantage (that went about 20 mts) would actually be counted in the stats.

The difference astounded me...
* I had him with 3 more kicks (inc. the kick off the ground, my analysis was very clear and he definitely had all those kicks)
* I had him with 1 more mark
* I had him with 1 more tackle
* I had him with 2 less frees against

It may not look like much there, but when I (at least think) am doing a very sophisticated and careful analysis, I reckon it is a lot. I also reckon I am very harsh with what should constitute a tackle (if a player is brought to ground, but get a handball out, that is not a tackle in my books.

I don't think Paul is wrong as his job is just data input from the old papers. I'm just wondering on the thoughts of how accurate the initial raw data is, 20 years ago, and also now.

(FWIW, in the future games I am analysing, player X would want to improve his disposal ratings a lot...very overrated at the moment!)

 

[Belnakor]

its not accurate because it is not put through proper scurinity

 

[RogersResults]

I have started a large project on a player that played the majority of his career in the 1990s. It is very statistical based, as he is someone that I believe was thought of as being extremely high skilled, and my project is trying to prove that, and to see how he would be compared to the elite of today's game.

I just completed the first game of the project's analysis. The project recorded all the usual stats, and also involves a few stats that I have thought of, that are very good in analysing things such as a players influence on the game. Anyway, I thought I would check on AFL Tables, to check the stats with what was recorded back in the day, as I was unsure whether or not a soccer off the ground to his own advantage (that went about 20 mts) would actually be counted in the stats.

The difference astounded me...
* I had him with 3 more kicks (inc. the kick off the ground, my analysis was very clear and he definitely had all those kicks)
* I had him with 1 more mark
* I had him with 1 more tackle
* I had him with 2 less frees against

It may not look like much there, but when I (at least think) am doing a very sophisticated and careful analysis, I reckon it is a lot. I also reckon I am very harsh with what should constitute a tackle (if a player is brought to ground, but get a handball out, that is not a tackle in my books.

I don't think Paul is wrong as his job is just data input from the old papers. I'm just wondering on the thoughts of how accurate the initial raw data is, 20 years ago, and also now.

(FWIW, in the future games I am analysing, player X would want to improve his disposal ratings a lot...very overrated at the moment!)

If you want to pay for the stats to be collected by multiple observers and audited by watching multiple angle replays from the businesses that now offer the service in recent times, then you may get some guarantee of accuracy.

Stats published in newspapers etc, the further you go back the more discrepancies you will find from one account to another. Think about how and who might have collected them. There are discrepancies in the scores let alone how many handballs somebody had.

Also as you note yourself, many of the 'stats' can be subjective depending on the observer.

The best old-time coaches could tell which players were doing the right thing at the right time most often without counting things.

 

[Ron The Bear]

If you want to pay for the stats to be collected by multiple observers and audited by watching multiple angle replays from the businesses that now offer the service in recent times, then you may get some guarantee of accuracy.

Stats published in newspapers etc, the further you go back the more discrepancies you will find from one account to another. Think about how and who might have collected them. There are discrepancies in the scores let alone how many handballs somebody had.

Also as you note yourself, many of the 'stats' can be subjective depending on the observer.

The best old-time coaches could tell which players were doing the right thing at the right time most often without counting things.

Well summed up. Prior to Champion Data, stats (other than goals and behinds) were incidental to the game itself. When I was a kid, two Sunday papers in Melbourne - the Observer and the Press - carried the stats when no-one else did. If a player had a lot of possessions, say 30 or more, the sources occasionally differed on possession count by as many as 5 or 6.

There wasn't nearly the same television coverage. Typically the public would get to see replays of one full quarter of two games, maybe 15 minutes of a third, plus additional highlights from the three games on World of Sport on the Sunday. Grainy slow motion footage - perhaps 60 or 90 seconds' worth in each case - was all that was shown from the other three games. In addition, the weaker clubs usually missed out on the better coverage. All of this meant player recognition/identification wasn't as good.

 

[AstroboyUK]

agree: data from the past is very unreliable- i remember the days when possession stats varied wildly between papers too. by necessity there is a bit of smoke and mirrors in all of this. i have a friend who works for Hawkeye for example.. he says at least 20% of what they show is either manufa tured or 'fixed' manually in the van outside the ground!

 

[BuddyMarn]

Hey guys, long time reader, first time poster.

I have worked collecting the statistics in Sydney, but while we didn't cover AFL games, I did do the NRL, Super Rugby and A-League (AFL done in Melbourne by a separate company, or they may have started doing it there) so I'll give you a breakdown and let you know where and why errors happen.

The statistical input is broken into two sessions, the first being the "live call", and the second being the "second call".

The stats are done from inside the building, with only the television footage to go by (unless you're doing NRL U/20s which is done at the game).

The first call is what is published in the newspaper currently, so I would assume that they would have done the one call back in the day. What this involves is one person saying out loud what is going on in a set way, and another putting it into a computer. I mainly do NRL so I'll let you know how it works and try to guess how they would do it. The capitalised are the players, the others are actions, and the arrows (>) indicate the next set of buttons that would come up.

Caller: SANDOW > Whistle > Kickoff > REYNOLDS > in goal > KASIANO > kick return > 23 metres > tackled (then the players numbers, for example 8-9-10) > play the ball > TOLMAN > missed tackle > 10 > no more (missed tacklers) > hit up > 9 metres > tackled (9, 12) > play the ball > PRITCHARD > hit up > 9 metres > tackled 7, 10, 12 > error.

What you just read will happen in around maybe, 20 seconds. Once you have made an error in the input, that's it, you move on or you will make way more. The papers are sent this info at the end of the first call, so there will be discrepancies in everything from metres, to tackles, tackle busts, etc.

I doubt that in the 90s people would have done an in depth second call of a game where you:
1. Iron out mistakes
2. Add everything else you don't have time to add (exactly where all of this occured, whethere the tacklers came in 1st, 2nd or 3rd).
3. Flag things which could be disputed. In RL it is often "line breaks", as sometimes there is no set line, or perhaps he makes a half break, busts the line, but may have a player holding on to his jersey. These disputes are often not decided until Monday, and often come down to a vote.

In AR, I can think of a few common situations where there might be differences of opinion. Say a player gathers the ball, and is being brought to ground. The player has a bit of a slap at the ball in a half hearted pass, but the ball could be seen as going free, is that a pass, or releasing the ball (you cannot go against the referee even if you think it is a throw).

You also only have the televised footage to go off (a replay could block what happened so you have to guess, as someone took a mark, but who kicked it? There were two players scrambling for the ball and the commentators were gibbering on about something else). You might also have a scramble for a ball and there could be tackles going on that can't be seen, but someone else might make an assumption that the tackle happened.

I find it highly unlikely that any huge amount of care was taken back in the day to record this, as one of the main payments for this info are clubs. You give them this info which contains all the stats, by player, by stat type, but importantly time coded to a tape (the coaches get multiple angles).

Then for example, Clarkson notices that Rioli has handballed to ground a lot recently, so he goes to his computer and can bring up all Rioli's handballs from the last 4 months and can see what he is doing wrong.

Sorry to waffle on but I hope this helps.

 

[RogersResults]

Hey guys, long time reader, first time poster.

I have worked collecting the statistics in Sydney, but while we didn't cover AFL games, I did do the NRL, Super Rugby and A-League (AFL done in Melbourne by a separate company, or they may have started doing it there) so I'll give you a breakdown and let you know where and why errors happen.

Excellent and informative.

No doubt the most successful coaching staff also rely on their own perceptions and not just the figures, but the figures can obviously point their attention in the right direction.

I wonder if you think that Australian Football would be easier/harder or just different from collecting the stats from the other codes?

 

[Ron The Bear]

No doubt the most successful coaching staff also rely on their own perceptions and not just the figures, but the figures can obviously point their attention in the right direction.

Generally speaking, it's unwise to start with the data and work backwards to formulate maxims. The observation/idea/theory should come first, followed by data analysis to verify/refute.

 

[BuddyMarn]

Excellent and informative.

No doubt the most successful coaching staff also rely on their own perceptions and not just the figures, but the figures can obviously point their attention in the right direction.

I wonder if you think that Australian Football would be easier/harder or just different from collecting the stats from the other codes?

From looking at it, and seeing some of the stats (a huge amount aren't given out) I would say it would probably be in the same category as possibly RL which is the second hardest after Union. Union is really difficult because the play is very hard to predict for longer periods than most sports (they kick when you think they'll run, run when you think they'll kick) and the ruck makes it hard to pick out players sometimes (you give the individual tackles the second time round, unlike RL).

There's a thing called a "pick and drive" where a player grabs the ball from the back and pushed through the middle. Now imagine watching a pile of blokes, then one of them grabs the ball, and without seeing his number just leaps forward only to be engulfed in players. You might never see that person again in that frame if they pass it wide so you have to go back and forward and look at the relative positions of players and have a stab.

AR can be easy to predict the overall flow (like league), but when there's a loose ball (especially in and around the goal) it would be a nightmare. Add in that there are so many players on the field, and kick metres (how do you judge that with next to no markings? And the MCG is bigger than other stadiums, etc), and the positional changes, it could get messy, very quickly.

Imagine if there's a free for all and old mate comes in off screen and hoofs the thing up field and you see a pair of white legs before a frame change. Bad form old mate .

 

[Wallaby]

The clubs have several dedicated stats men at each game and each is usually only assigned a few stats to record. These are done by pushing buttons on a PC - eg - put a player name 'map' over the keyboard (so Judd equates to the 'Q" key for example). If he does, say, a shepherd, the recorder presses the 'shepherd' button and the Q button - result, Judd gets a shepherd recorded with a timestamp, so they can later check it out on the replay. This enables the club to correlate types of related stats against times and players - eg, if Judd gives a handpass, how often do Carlton score in the next 30 secs?

This is a very simple example, but it illustrates how complex analysis is becoming for clubs (they can and do also measure position on ground, time player has been on the field since last interchange, wind strength - everything you can think of).

I had a mate who did some stats for Melbourne a few years ago - he recorded WISPS - Weak As Piss efforts. He says Travis Johnston used to feature heavily..............

 

[Fudge]

I can fill in what happens for each AFL game. We have 10 people per game capturing various things like calling, interchange and matchups (at the ground), pressure and graphics at the office. When the game is hot, the caller just calls and it's up to the capture person to keep up. There's a back caller in place to make note of what's going on and when a quiet moment arrives (plenty of those), edits are made of things missed previously. During period breaks, we use vision to replay stuff we weren't sure about. We also do post match editing which takes about 10-15 minutes. We think what we capture is as accurate as can be. It's very rare for us to go back and change something. Accuracy is highly valued by coaches and their staff who use what we do. If there were any problems with the accuracy, they'd let us know. Also, it's not possible to do accurate capture from TV. Too much stuff is missed. You do need people at the ground.

 

[Ron The Bear]

I can fill in what happens for each AFL game. We have 10 people per game capturing various things like calling, interchange and matchups (at the ground), pressure and graphics at the office. When the game is hot, the caller just calls and it's up to the capture person to keep up. There's a back caller in place to make note of what's going on and when a quiet moment arrives (plenty of those), edits are made of things missed previously. During period breaks, we use vision to replay stuff we weren't sure about. We also do post match editing which takes about 10-15 minutes. We think what we capture is as accurate as can be. It's very rare for us to go back and change something. Accuracy is highly valued by coaches and their staff who use what we do. If there were any problems with the accuracy, they'd let us know. Also, it's not possible to do accurate capture from TV. Too much stuff is missed. You do need people at the ground.

Thanks, explains a few anomalies I've noticed.

 

[jefffff34]

Just studied a player from the Claremont vs East Perth game that is on ABC iView. The WAFL site has him as 3 kicks, 8 handballs. I have him as 6 kicks and 12 handballs...there was one tenuous off the ground kick which I didn't put down as a stat as well.

What is going on here?