Team Rating System, update after Finals, Week Three

[blitzer]

I'll be back to have a look at how this system goes in future weeks. Its always interesting to read this sort of thing and see the ideas are put into this kind of work. Thanks for the effort and the openness with your model.

 

[Tas]

I did run it last year. It worked fine. I don't know what you're calling "reliable results". If you like, I can tally the win-loss predictions and compare with bookie's favourites, and compare the margin predictions with bookie's lines.

Yeah, I'd be interested in how accurate it was on last year's results, both in total and looking at how well it predicted harder to predict results, ie between two teams that are going okay.

 

[Hobbes]

Sounds a lot like the squiggle

Yes, there are similarities. One thing that i do differently is that the Squiggle rates on results against the current rating of the team you're playing, whereas I recalibrate retrospectively based on the subsequent results of your opponents. (So, if you beat a team just before their nine match winning streak, the result is re-evaluated as a very strong result). The squiggle separates offence and defence, but I think my margin calculations are roughly similar in effect. I'm sure the squiggle graph looks way cooler than anything my non-existent graphic design skills could manage.

I rate recent results as more significant than older results, and I don't think the Squiggle does this.

Several things which the Squiggle doesn't take into account, i don't do either.

I'd be interested to compare my tipping success rate with the squiggle throughout the season. I suspect that our tips will be similar a lot of the time - I'd be interested to see when they tend to differ.

 

[Eshay187]

Yes, there are similarities. One thing that i do differently is that the Squiggle rates on results against the current rating of the team you're playing, whereas I recalibrate retrospectively based on the subsequent results of your opponents. (So, if you beat a team just before their nine match winning streak, the result is re-evaluated as a very strong result). The squiggle separates offence and defence, but I think my margin calculations are roughly similar in effect. I'm sure the squiggle graph looks way cooler than anything my non-existent graphic design skills could manage.

I rate recent results as more significant than older results, and I don't think the Squiggle does this.

Several things which the Squiggle doesn't take into account, i don't do either.

I'd be interested to compare my tipping success rate with the squiggle throughout the season. I suspect that our tips will be similar a lot of the time - I'd be interested to see when they tend to differ.

Sounds good - I like the sound of those differences

 

[sonnywalters]

So, basically its the ladder?

Not really as the season goes on it will differ from the ladder by giving more weight to recent form. So it is more like a 'form ladder'.

I am interested to know how Hobbes will make adjustments for variance in the draw v the quality of opposition adjuster. e.g If a side has a seemingly easier draw and wins lots of games and are say sitting in the top 4 on the actual ladder - will this be adjusted down because of team's they played. Also how will the team with the easier draw who is on a roll be 'rated' as an opposition?

Anyway. I am impressed with Hobbes model and will look fwd to seeing it 'roll out' over the season.

Edit: I think Hobbes answers my query - he will re-calibrate ratings retrospectively.

I think this approach has real merit. Should get a clearer picture at the halfway point of the season.

 

[Final Siren]

I recalibrate retrospectively based on the subsequent results of your opponents. (So, if you beat a team just before their nine match winning streak, the result is re-evaluated as a very strong result).

Love to know how this works out! It's something I've thought about. I suspect its usefulness is greatest around right now, i.e. a few weeks into the season, where you want to figure out who's changed over the off-season without much exposed form.

I rate recent results as more significant than older results, and I don't think the Squiggle does this.

It does but only a little. It's effectively a weighted average, so last week's result is 8% of the total, the previous week is 7.4% (92% of 8%), the week before that is 6.8% (92% of 92% of 8%), and so on.

You should definitely run lots of older years through your model! The vast majority of things I try to improve forecasting don't work, and I only know that because I can test against thousands of games.

 

[Hobbes]

Not really as the season goes on it will differ from the ladder by giving more weight to recent form. So it is more like a 'form ladder'.

I am interested to know how Hobbes will make adjustments for variance in the draw v the quality of opposition adjuster. e.g If a side has a seemingly easier draw and wins lots of games and are say sitting in the top 4 on the actual ladder - will this be adjusted down because of team's they played. Also how will the team with the easier draw who is on a roll be 'rated' as an opposition?

Anyway. I am impressed with Hobbes model and will look fwd to seeing it 'roll out' over the season.

Edit: I think Hobbes answers my query - he will re-calibrate ratings retrospectively.

I think this approach has real merit. Should get a clearer picture at the halfway point of the season.

Actually, I only sort-of recalibrate it retrospectively. It

Love to know how this works out! It's something I've thought about. I suspect its usefulness is greatest around right now, i.e. a few weeks into the season, where you want to figure out who's changed over the off-season without much exposed form.


It does but only a little. It's effectively a weighted average, so last week's result is 8% of the total, the previous week is 7.4% (92% of 8%), the week before that is 6.8% (92% of 92% of 8%), and so on.

You should definitely run lots of older years through your model! The vast majority of things I try to improve forecasting don't work, and I only know that because I can test against thousands of games.

So, another difference in how I operate is that I weight recent results more heavily - I have the most recent match weighted at roughly 17%.

I'm working on a formal record of exactly how it works through the 2015 season - I tried it as a hobby last year but didn't keep records. I'll post the outcomes here when it's done.

 

[BEaston]

Good work Hobbes

 

[Hobbes]

In the 2015 season, the algorithm tipped at 139/196, or roughly 70%, in the minor round, and managed 7/9 finals (failing on both of North's wins). It boycotted the round 23 match between Port and Fremantle because it was only the Fremantle seconds team which turned up. Results from 2014 were included to make up the ratings.

 

[Kerleys Ghost]

Please rename thread 'Power Rankings'.

Can we call you 'Hoby' for short?

 

[Hobbes]

Please rename thread 'Power Rankings'.

Can we call you 'Hoby' for short?

I was hoping that laying bare all of my methodology (I've probably provided enough information for somebody to duplicate everything I've done, and virtually none of it is subjective) might provide a point of difference to Roby.

From a personal perspective, I'm probably doing this as an intellectual exercise rather than a product of ego (or marketing), so I don't mind if somebody borrows part of what I'm doing for their own model. (But if you're doing something cool with it, please credit me).

 

[KrissCross]

Except his multiple factors combined to over rate the team he just happens to support, call me suspicious but I call bullshit.

For example even of the two win teams beating West Coast and The Bulldogs is surely better than beating Port and Richmond.

The problem is his system seems to favor beltings over the quality of the opposition, beating Richmond by 36 away is better than beating the Bulldogs by 3 at Etihad except does anybody actually think that?

The sample size is also way to small it probably rates port as an impressive win but they really haven't beaten anyone of note at this point.

Perhaps you should return to the Adelaide boards if you don't like people question just how an Adelaide poster manage to get his team so high in the rankings.

He has tried to explain how it works...You think he has sat there and deliberately designed the system to pump Adelaide up? That deserves a
Like I said in previous post...just stick to the official AFL ladder bro

 

[JaxsonB]

The tips produced look fairly solid for an algorithm output, so looking forward to seeing how well they match up with the actual results. Good luck!

 

[HoneyBadger35]

The trick is to not tell us your model and to claim you profit every week in the face of overwhelming evidence that says otherwise.

You're doing this all wrong.

 

[jamaican_eagle]

West Coast +19 v Richmond
Geelong +19 v Essendon
Hawthorn +13 v St Kilda
Brisbane v Gold Coast +33
Carlton v Western Bulldogs +26
Adelaide +6 v Sydney
GWS +6 v Port Adelaide
Melbourne +8 v Collingwood
North Melbourne +30 v Fremantle

So far, these have almost been spot on apart from West Coast defeating Richmond by 68 points.

Nice work on getting the Geelong and Hawthorn wins within 2 goals. Most would've tipped those sides to win by a lot more.

 

[Hobbes]

MARK 1 algorithms (based on this year's results only) and predictions

West Coast +19 v Richmond
Geelong +19 v Essendon
Hawthorn +13 v St Kilda
Brisbane v Gold Coast +33
Carlton v Western Bulldogs +26
Adelaide +6 v Sydney
GWS +6 v Port Adelaide
Melbourne +8 v Collingwood
North Melbourne +30 v Fremantle

Well, that's 8/9 correct on win/loss, which I'm quite sure I won't be able to maintain as a long-term average. I doubt anybody but the keenest fan would have tipped Brisbane, so I don't feel bad about that. Margins were nearly spot on (North, Adelaide), close (Geelong, Hawthorn, Bulldogs) a bit wrong (Melbourne) and way off (GWS, West Coast).

1. Sydney
2. Adelaide
3. Western Bulldogs (+1)
4. GWS (+6)
5. Geelong
6. North Melbourne
7. West Coast (+2)
8. Gold Coast (-5)
9. Hawthorn (-2)
10. Melbourne (+1)
11. Port Adelaide (-3)
12. St Kilda
13. Brisbane (+5)
14. Fremantle (+2)
15. Essendon (-1)
16. Richmond(-3)
17. Collingwood(-2)
18. Carlton(-1)

And predictions for round 5

Hawthorn v Adelaide +2
Sydney +22 v West Coast
Gold Coast +4 v North Melbourne
Western Bulldogs +33 v Brisbane
Port Adelaide v Geelong +10
St Kilda v GWS +12
Fremantle +14 v Carlton
Melbourne +15 v Richmond
Collingwood v Essendon +2

 

[Hobbes]

MARK 2 Algorithm, using last year's results to fill out the data set

West Coast +27 v Richmond
Geelong +40 v Essendon
Hawthorn +52 v St Kilda
Brisbane v Gold Coast +17
Carlton v Western Bulldogs +58
Adelaide +17 v Sydney
GWS v Port Adelaide +10
Melbourne v Collingwood DRAW (I calculate an advantage to Collingwood of 0.2)
North Melbourne +33 v Fremantle

This time, 6.5/9 correct, which is about the same as tipping the bookie's favourite every time.

1. Sydney (+3)
2. Adelaide
3. Hawthorn (-2)
4. West Coast (+1)
5. Western Bulldogs (-2)
6. North Melbourne
7. Geelong
8. GWS (+4)
9. Port Adelaide (-1)
10. Melbourne (+4)
11. Richmond (-2)
12. Fremantle (-1)
13. Gold Coast (-3)
14. St Kilda (+2)
15. Brisbane
16. Collingwood (-3)
17. Essendon
18. Carlton

(The movement at the top looks weird, but the top 3 are almost tied, Sydney's loss to Adelaide almost counted as a draw when home-ground advantage was taken into account, and Sydney's win against GWS looks like a much stronger result now).

And Round 5 tips

Hawthorn +6 v Adelaide
Sydney +15 v West Coast
Gold Coast v North Melbourne +20
Western Bulldogs +43 v Brisbane
Port Adelaide v Geelong +4
St Kilda v GWS +18
Fremantle +35 v Carlton
Melbourne +1 v Richmond
Collingwood +4 v Essendon

 

[Crankyhawk]

Is there value in displaying the numeric ratings of each team? So we can see if the top 3 are tightly bunched and then there's a big gap in rating? Currently it is hard to tell whether 4th is a lot better than 5th or the difference is marginal.

 

[Hobbes]

Is there value in displaying the numeric ratings of each team? So we can see if the top 3 are tightly bunched and then there's a big gap in rating? Currently it is hard to tell whether 4th is a lot better than 5th or the difference is marginal.

I don't know. I can post the ratings anyway.

(MARK 1)
1. Sydney 18.2
2. Adelaide 14.2
3. Western Bulldogs (+1) 13.7
4. GWS (+6) 9.9
5. Geelong 9.71
6. North Melbourne 8.7
7. West Coast (+2) 8.4
8. Gold Coast (-5) 7.5
9. Hawthorn (-2) 6.0
10. Melbourne (+1) 0.3
11. Port Adelaide (-3) -5.9
12. St Kilda -7.5
13. Brisbane (+5) -13.2
14. Fremantle (+2) -14.6
15. Essendon (-1) -15.2
16. Richmond(-3) -15.3
17. Collingwood(-2) -17.5
18. Carlton(-1) -18.5

(MARK 2)
1. Sydney (+3) 23.03
2. Adelaide 23.01
3. Hawthorn (-2) 22.9
4. West Coast (+1) 20.4
5. Western Bulldogs (-2) 18.2
6. North Melbourne 17.5
7. Geelong 10.2
8. GWS (+4) 5.9
9. Port Adelaide (-1) 0.7
10. Melbourne (+4) -4.1
11. Richmond (-2) -5.0
12. Fremantle (-1) -8.1
13. Gold Coast (-3) -8.5
14. St Kilda (+2) -17.9
15. Brisbane -18.5
16. Collingwood (-3) -23.5
17. Essendon -27.5
18. Carlton -34.6

 

[JaxsonB]

From a non-mathematical point of view, mark 1 seems eerily accurate. In particular, most fans and even Squiggle would be tipping Richmond and Collingwood next weekend, but quite frankly, Essendon are a decent chance at snagging a win here, and Melbourne has been playing better than Richmond - and your mark 1 tips reflect both of these. The Adelaide tip also accurately represents form, though I'm personally tipping Hawthorn for a few reasons.

Incidentally, I got the exact same 8/9 tips as it did, so perhaps I'm just biased.

 

[Hobbes]

From a non-mathematical point of view, mark 1 seems eerily accurate. In particular, most fans and even Squiggle would be tipping Richmond and Collingwood next weekend, but quite frankly, Essendon are a decent chance at snagging a win here, and Melbourne has been playing better than Richmond - and your mark 1 tips reflect both of these. The Adelaide tip also accurately represents form, though I'm personally tipping Hawthorn for a few reasons.

Incidentally, I got the exact same 8/9 tips as it did, so perhaps I'm just biased.

Speaking personally, I'd tip Hawthorn over Adelaide. Aside from the home ground advantage, I think Adelaide must be exhausted after Saturday's slugfest.

 

[JaxsonB]

Speaking personally, I'd tip Hawthorn over Adelaide. Aside from the home ground advantage, I think Adelaide must be exhausted after Saturday's slugfest.

Exactly the two reasons I'm tipping them.

 

[blueiris]

MARK 1 algorithms (based on this year's results only) and predictions

West Coast +19 v Richmond
Geelong +19 v Essendon
Hawthorn +13 v St Kilda
Brisbane v Gold Coast +33
Carlton v Western Bulldogs +26
Adelaide +6 v Sydney
GWS +6 v Port Adelaide
Melbourne +8 v Collingwood
North Melbourne +30 v Fremantle

Well, that's 8/9 correct on win/loss, which I'm quite sure I won't be able to maintain as a long-term average. I doubt anybody but the keenest fan would have tipped Brisbane, so I don't feel bad about that. Margins were nearly spot on (North, Adelaide), close (Geelong, Hawthorn, Bulldogs) a bit wrong (Melbourne) and way off (GWS, West Coast).

1. Sydney
2. Adelaide
3. Western Bulldogs (+1)
4. GWS (+6)
5. Geelong
6. North Melbourne
7. West Coast (+2)
8. Gold Coast (-5)
9. Hawthorn (-2)
10. Melbourne (+1)
11. Port Adelaide (-3)
12. St Kilda
13. Brisbane (+5)
14. Fremantle (+2)
15. Essendon (-1)
16. Richmond(-3)
17. Collingwood(-2)
18. Carlton(-1)

And predictions for round 5

Hawthorn v Adelaide +2
Sydney +22 v West Coast
Gold Coast +4 v North Melbourne
Western Bulldogs +33 v Brisbane
Port Adelaide v Geelong +10
St Kilda v GWS +12
Fremantle +14 v Carlton
Melbourne +15 v Richmond
Collingwood v Essendon +2

You obviously didn't take into account the Lions record against the Suns especially at the Gabba I also picked 8/9 without even using your system

 

[Hobbes]

You obviously didn't take into account the Lions record against the Suns especially at the Gabba I also picked 8/9 without even using your system

No, I don't take those sorts of things into account. If I did, I probably would have tipped Sydney.

 

[Crankyhawk]

Speaking personally, I'd tip Hawthorn over Adelaide. Aside from the home ground advantage, I think Adelaide must be exhausted after Saturday's slugfest.

As a Hawks man I have Adelaide favourites we are getting little out of the middle apart from Sam Mitchell and we have only one big key defender - against 3 key forwards. Jenkins will kick a bag.