Wednesday, 2 July 2014

Tipping Performance Analysis

This little Tipping League had its origins in the FMI blog that tried to tip game outcomes and margins
And while it used a regulation modern computer and excel spreadsheets, we played up the angle of a big, lumbering 'old skool' computer 'machine'. It was eventually named "Ol' Bessie" by John @theholyboot, and adopted by us as her name.

In 2012, we happily tipped games and margins for our own fun and no profit, to a then weekly blog readership numbering in the tens, if that.
Somewhere along the way, the notion of a 'machine' dominating a tipping competition was borne by Dave @davpope, who challenged us to match his wits to our 'machine'.

And so back in mid 2012, the first ramshackle tipping competition began.  Through the rest of 2012, Dave fought the rise of the machine, and was joined by others, some consistently, and others intermittently. And so, some alleged fun was had by all (none of it able to be proven in the Federal Court though).

In the lead-up to 2013, the foundation of the FMITL (FootyMaths Institute Tipping League) was laid, rules written, re-written, added too, washed, rinsed, and finally codified, The website/blog you see today began as the true home of the FMITL, with 10 'foundation partners', who have barely missed a beat for almost 2 seasons now.

We at the Institute campus are greatly encouraged by how this little competition has been received and the good nature that you all play with. "You have floored me, totally floored me" with your comments about this competition. And the way the new players this season have embraced it is even more encouragement. For us at FMITL HQ... 'This is dizzy stuff, folks!'.


Which brings us to the point of this post.

A new player this year is Michael @Carlo_Monty_AFL is also an avid spreadsheet manipulator and tipping aficionado. He has spent this year loading data from the FMITL, and all the other tipping competitions in Melbourne's newspapers.

In his logging of data he has compared our tipping efforts (restricted to 5 games tipped, as they are) to those published by journos, experts and former players. And while travelling through the northern hemisphere, he composed this first for the FMITL - a guest post on correlations of tipping.





GUEST POST
Correlating Your Tips With Other Published Tippers

The excitement of anticipation of a football season of pitting my tipping model against the collective might of 'the only margin tipping competition in the western hemisphere' has quickly evaporated over a handful of rounds.  The expectation of a rapid rise through Division 2 and asserting my rightful place amongst the Foundation Club members in Division 1 has been replaced by frantic outbreaks of sweat and anxiously checking mid round results in the hope of avoiding the indignation of
- last place;
- the worst MAPE;
- low tipping accuracy, and, in general;
- the scorn of my fellow FMITipsters.
In order to deflect some of the scorn I should obviously find some statistics to quote and generate many newspaper inches of commentary which will in turn dilute the focus on my poor tipping ability.

The secondary goal of my recent analysis of some margin tipping statistics is to develop a deeper understanding of my competition and to turn that to my advantage in future rounds – sort of applying 'The Art of War' to FMI Tipping.


So where to start?
Well the majority of the Carlo Monty excel spreadsheet is used to rate and rank the teams and massage a game day prediction for each match. Rather than just predicting a winner and a loser, individual teams scores are predicted and the margin used for the FMIT predictions.

A couple of sheets in the excel file are dedicated to keeping an eye on the published margins in the newspapers (The Age and The Herald Sun) and a handful of published internet margin tips (Matter of Stats: Chips; Footy Forecaster; TipBetPro and FootyForecaster).
The Village Idiot and current footballers who do not tip a margin are excluded (with the exception of the Bulldogs Murphy who we assign a week in and week out prediction of a ten point win for the doggies).

All of the published margin predictions were filtered to only include official FMITL games (presented in the standard format of –ve for an away win and +ve for a home win).  The actual margin for all of the relevant games is used for correlation purposes:

FMITL Tipping Data with Other Published Data
Click to Enlarge

We then run good old excel correlation to look for simplistic correlations between the actual margin and all of the published tipsters.  Sounds simple but it took the old work computer just under an hour to crunch the numbers.  But oh the nuggets of gold uncovered by this process!  Questions such as who correlates the closest to Tony Abbott or Jason Dunstall are answered via a quick scan of the correlation table.


FMITL Margin Correlation Ladder (as at end Rnd 12)
 Unfortunately for me, the first statistic I looked at was who of the FMI tipsters has the lowest correlation to the actual margin as at the start of Round 13.

Yep coming in in 21st place with a correlation of 0.413 is the good ol' Carlo model.  TheT and Kevs are helping me hold up the ladder, but there is a healthy buffer between 21 and 20!

At the other end Wal1 is a clear leader on 0.683 which is also the highest correlation when considering the papers and is only bettered by TipBetPro on a massive 0.724






Given there was not much joy for me in looking at the league table of correlations to actual margins I turned my attention to which of the “celebrity” tipsters do I have the most in common.

The analysis of the FMITipsters compared to the published celebrity tipsters was interesting:

Correlating FMITL Tipsters with 'Celebrity' Tipsters (as at end Rnd 12)
The correlation leader Wal13Freo correlates strongly with Jake Niall, SgtButane with Jay Clark even more strongly, LaurenceRosen with David King and TheTipsGuru with Gerard Whateley.

The remaining  FMI tipsters surprisingly (to me) share strong correlations with other FMI tipsters:
J_Foreigner and A_OKeefe must follow Sam Lane closely.  While Footy_Maths and LucasGarth follow Jesse Hogan (hence the FMI Atari  is a boy and Ol Bessie is a misnomer!!).

Ethan_Meldrum and Kevs_View must be dyed in the wool Hawks fans as they follow Jason Dunstall closely – Although Ethan has another strong correlation, being as he is the FMI tipster with the strongest correlation with Tony Abbott: we will need to get some security guards around to protect our fellow tipper? Dunstall correlates more strongly to Abbott.

The largest groupings of commonality correlations are left for Gary Lyon and Dennis Commetti (DavPope correlates strongly with both) Supermercado99, Amul82 and TheHolyBoot side with Lyon.  While Coldogs, SJHRoss and myself align with Commetti.

Commetti!  How could I?

Tim Lane I could cope with or Crawford.  Even Dermie.  But Comemtti!?!
Commetti!


Below I publish the full tipster correlation ladder.  The conclusions I draw from this analysis?
- The Murph and I should have a couple of beers at the Plough Hotel in Footscray.
- My games against TheTipsGuru and Kev look set to be the grudge matches.
- Wal113Freo sits atop Div 1 for a reason – the powerhouse foundation club tipper that everybody else is gunning for!  Does Wal play more Friday night and Anzac day games than anybody else?
If TipBetPro published their tips earlier I should use them as a default – either that or I have got to spend my number crunching time to try and work out their system. Then I might be able to climb the ladder and pass Tony Abbott who notoriously picks GWS and still manages a better margin correlation than my ten year old model.
Click to Enlarge

3 comments:

  1. This is brilliant, thanks Michael.

    Question, in this correlation treatment, does the pair +1,-1 have the same weight as (say) +3,+1? I'd have thought yes, whereas the FMITL treats them very differently. It's for this reason that, just between you and me, since the most recent coaching staff upheaval here at the Isotopes about halfway into the season, for FMITL purposes I have made an upward adjustment to all my 'true' margin tips (which can be seen at http://www.csse.monash.edu.au/~footy/).

    Seeing all this laid in addition to the tipping DNA chart has engendered a thought. In a variety of realms, some a bit more serious than footy tipping, over the years I've noted a tendency for an averaged 'basket' of predictor indices to perform better collectively than any one of them on its own (is there a name for this principle?). It then occurs to me - could the performance of FMITL expressed as a hive mind be good enough to beat the bookies on a regular basis?

    Just speaking hypothetically, of course.

    ReplyDelete
  2. Thanks Carlo_Monty, I enjoyed the blog.

    Just regarding Mark's reply. I found it interesting that you 'upward adjust' your FMI tips. I've been aware of the problem with my tips for a while where I would be constantly closer to the pin over the 5 games but lose or draw the FMI match. I didn't want to..ummm. for lack of a better term...'bastardize' my system. I really enjoy letting my computer go into battle against other people's systems or gut predictions even though I know picking a team by less than around 8pts is a bad FMI tactic. I take more pleasure from the MAPE ladder but enjoy taking a few scalps along the way.

    As for your second point on averaging out FMITL predictions(Wisdom of the Crowds). This will achieve an extremely competitive MAPE compared with FMI individual MAPE's as well as compared to the bookies. I have a list of lines from one bookie taken from when the ball bounces that I'd be happy to send to @Footy_Maths if he'd like to run this experiment. But I can guarantee that it won't have a better MAPE than the bookies. This is due to lots of reasons - firstly, bookies have the power of 1000's of people's predictions, they also have access to (assumingly) better technology and well paid professionals. I would also say that money plays a role huge in getting the most accurate margin. By that I mean if 500 people think -15pts to Geel is good value and 500 people think +15 Ess is good value but the -15 Geel side are throwing down double the amount than Geel should probably be more than -15 even though the average of the individual 1000 punters is -15. Of course the bookies move the line to balance the money so they win either way but the number of punters on each side isn't balanced.

    Apologises if all of this is very obvious but I'm going somewhere with this ahah. The longer the season goes on the more I see my computer model getting closer to the bookies. That makes sense, the vast majority of a prediction is essentially 'for and against'. Doesn't matter if you use shots on target, final score, inside 50's or anything like that. You can add and subtract and spin it all upside down but we're all getting the same information in the end - EVEN if we're just betting with our heads, we weigh up past results and come to a conclusion.

    And there's the catch. If the bookies and us are using the same information to make a prediction and the bookies are taking a 2.5%-5% than we're going to have to be more accurate than the bookies to even breakeven over the long term. No system that relies on using the same information will be successful long term unless you're extremely selective of the games you bet on. Key is, use information that the vast majority of people wouldn't use.

    The irony of this is that those that are the best predictors of margins using detailed information are the least likely to make any money from punting. Maybe one day when I'm not derailing peoples blog posts I'll discuss this further. All of what I've written is things I think I've learnt over the past 2 years. I'd love to hear from somebody that actually understands maths properly(I only work in finance :/) or the punting industry.

    Cheers guys,

    Sarge - @SgtButane

    ReplyDelete
  3. Thanks for the comments:

    I too resist altering my models tips and have suffered after submitting the dreaded 1, 2 or 3 point margin tip.

    My MAPE is ghastly this year - favouring the home teams too much.

    I do not have Village Idiot data - there are a couple of published candidates I could recommend...vote 1...?

    Average of the Newspaper Crowd plus FMI is sitting on an average MAPE of 31.5 and 89 wins compared to bookies 90 (differ in only three cases)

    Two models based (Courtesy Tony Corke) on a select group of published tipsters are sitting on 89 and 93 correct tips and MAPES of 28.7 and 29 (but hindsight model: based on first 8 rounds is a luxury)

    All of this is really just building a many MB spreadsheet to keep me occupied over summer : )
    Oh and to try and win a couple of burgers (still to do this year : ()

    ReplyDelete