Final Thoughts on the HoF and the Skill Paradox

A final thought on the HoF.

PT Top 8s in the modern era are worth less than PT Top 8s from earlier in magic’s history. This is in spite of the average modern player being much more skilled.

**What I am writing about is an adaptation of well known theory in investing, Sabremetrics and Poker. For those interested in a more detailed analysis I suggest Mauboussin and googling the ‘Skill Paradox.’.

The Paradox of Skill

We start with the fairly simple assumption that

Performance = Skill + Luck

We also assume each person’s luck is drawn each tournament from some distribution that is equal to all players. E.g. LSV might have been be luckier in PT Kyoto than Nassif because he opened Nicol Bolas at that specific tournament, however they had equal chances to open it.

Because a person’s skill and luck are uncorrelated, we arrive at the Paradox of Skill:

Variance of Population Performance =

Variance of Skill in population + Variance of Luck in Population

As the Variance in skill gets smaller, the variance associated with luck starts to dominate in determining the overall outcome of tournaments.

Consider the following example:

A)    A PT in 1999, where Jon Finkel is far and away the best player. The 100^th best player barely knows how to draft and the rest of population is somewhere in between.

B)     A PT in 2013 where, the top 100 players are all equal to skill Jon Finkel @1999 skill level.

It should be clear that someone’s final position in PT A is strongly correlated with skill. In other words we can be confident that the person in 8^th was better than the person in 16^th.

In PT B, the opposite would be true. The only difference between someone who gets 8^th and 15^th was the amount of luck they had in that specific tournament.

Assumptions I am using:

1)      The skill dispersion (especially at the top of the game) is much lower today than it was historically. In other words, the top 50 players in the game are much closer today (even if they are all much better) than they were historically.

And that’s it. Everything I have read from Kai, Finkel and Kibler on the topic would seem to support the view, but I haven’t bothered to try and prove the above assumption.

Just to reinforce that this situation isn’t completely impossible. In a world where the top 50 players attend 3 PTs a year and each have 10% chance to top8 a PT: we would still expect one person to top 8 two PTs a year. In other words the fact that some players do consistently well isn't enough to disprove assumption 1. If you have ever heard or read about the birthday paradox, the same principles apply.

Practical Implications

We are seriously overweighting T8s and wins in the modern era competitors. Instead we should focus on a looser metric (e.g. 32s/64s etc…). Rate metrics and consistency become much more important. For older players, top 8s are more likely to imply that they were one of the best 8 players in the tournament. And a top 32 is more likely to imply that they were NOT one of the top 8.

Recently in his SCG article Reid Duke made the point we shouldn’t punish anyone for having a few bad initial years on the PT. And I really wanted this to be true (Because me obv). But if we now know that luck is the major determinant in people’s short term success rates, things like 3 Yr medians should mean less for modern competitors. Forgiving a few “bad years” makes it more likely you select someone who's results are variance driven (as opposed to skill).

Putting this together for HoF implications I think should go as follows. Suppose someone has 2 PT Top 8s and 6 PT top 16s. In the Modern Age: I think “Hes unlucky”. If he is old school: I think “he probably wasn’t that good”.

Focusing on results through this lens I think we could argue:
Underrated (in no particular order):

1)      Shouta Yasooka

2)      Hoaen (do we consider him “modern”?)

3)      Osyp

Overrated
      1)   Edel.

2)      Saito (if “Modern”)

3)      Ikeda

4)      Gary

Final Unrelated HoF Thoughts:

Stats I used in my previous formula driven HoF Ballot:
Longevity = # of PTs, # of Pts

Consistency = PT Median, 3 Yr Median, Difference in Medians, T16s, GP Top 8s

Best in World = 3 Yr Median, POYs

Place in History = These are indicator variables (e.g. are you in the top 20%). In other words having 4 PT Top 8s is the same as zero because 80% of ballotees had 4 or less.

Top 8s, Money List, GP Top 8s, Pro Points.

Skill = T16s per PT, Median Finish, POYs per years played.

My Ballot (which I don’t have):

1. LSV

2. Edel + Ikeda

These are the only two who are not top 5 stat wise. I think pioneers in a field deserve credit. I am willing to go beyond the stats if there is proof they did something truly unique. I feel the case for Ikeda is weaker than Edel (he has more similar analogues in Fujita, Oishi etc..). I could be convinced to vote for Osyp (easily the most underrated candidate on the ballot) instead.

3. Shota Yasooka.

Stats + Skill Paradox already implied he was one of the best players skill wise on the ballot. Juza’s interview on cfb was a nice (if unnecessary) confirmation.

4. Saito.

1.      He was (or at least top 3) the best deckbuilder in the world for a long period of time. Still seems like he might be.

2.      He is one of the best players I have seen play. I can sometimes remember individual matches where I was blow away by the play I saw. Saito in TSP block is amongst those. Ditto San Juan. Most players I have talked to feel that he was easily amongst the best when he played.

3.      He was an angle shooter, but a lot of people on the PT are. Stalling in particular seems like one of the most hypocritical things for many players to call someone out on (based on my PT experience). So while he might be the scummiest of successful players (which I doubt), other players are close to that level. This might be too much apologizing for someone who is arguably a cheater (I differentiate between rules lawyers/cheaters/angle shooters), but I don’t believe (based on 1 and 2) that his results were significantly impacted by his angel shooting.

The honourable mentions: BenS, Efro, Gary.