The Ultimate Ranking System?

[The Bofa on the Sofa]

OK, here we go. How about this for the "Ultimate Ranking System": the ranking that best describes who won and lost

This should satisfy everyone who thinks that winning and losing is all that matters. And teams that beat others should be ranked ahead of them as much as possible.

So I've done that, as best I can. Almost. It's still cranking along, but it's cranking on it. And the results globally look awesome. For example, while normal Pablo rankings this year correctly reflect (with home court corrections) 82.6% of the outcomes, this new ranking system is right now sitting about 87.6%, and still creeping up. It's probably too late today to get the final values done, but I will have some preliminary results before 5:30.

I mentioned this in another thread, but when yo do this type of optimization, you can get some strange results. For example, the best example would be a team that loses to #80 but beats #25 and #5. How do you rank them? Well, most people would say, ok, their results from #5 and #30 are a wash, so they are somewhere in-between, and their win over #25 is good, so let's put them in the 10 - 20 range, right?

However, if you want to maximize correct results, the proper approach is to put them at #4, because that way you have 2 correct outcomes, whereas at #15, you only have one.

Now, the consequence of that is that teams that lost to that team can now be ranked anywhere below 4, instead of 15, so it pulls up their competition.

I've done this exercise before, and I've seen that happen. I'm suggesting don't be surprised to see it again. Hint hint

Currently up to 88.1% correct. Out of 4500 matches, that means that it's like 3950 correct. That's HUGE. I'll post the top 60 in a bit.

[The Bofa on the Sofa]

Right now, the rankings correctly reflect the outcomes of 88.2% of the matches that have been played.

Here's the top 50 or so
HCA 161


1 Stanford stan 7315
2 Wisconsin wi 6855
3 Illinois il 6830
4 Washington wa 6820
5 Texas tx 6735
6 North Carolina nc 6720
7 Florida State fs 6715
8 Penn State ps 6670
9 Colorado co 6665
10 Oregon or 6530
11 Nebraska nebr 6510
12 Arizona az 6485
12 Florida fl 6485
12 UCLA ucla 6485
15 Colorado State csu 6460
16 BYU byu 6390
17 Loyola Marymount lm 6335
18 Oregon State ors 6290
19 Texas A&M txam 6270
20 Kansas State ks 6240
21 Kentucky ky 6235
22 USC usc 6225
23 Arizona State asu 6205
24 San Diego sd 6190
25 Utah utah 6185
26 Creighton crei 6180
26 Miami-FL miam 6180
26 Seton Hall seth 6180
29 Illinois State ils 6170
29 Long Beach State lbs 6170
29 Duke duke 6170
32 Iowa State is 6155
33 Ohio State osu 6145
34 Ohio ou 6105
35 Wyoming wy 6090
36 Kansas k 6075
37 Purdue purd 6070
38 Minnesota mn 6045
39 Santa Clara sanc 6030
39 Pacific pac 6030
41 Marquette marq 6015
41 Northern Illinois nil 6015
43 LIU Brooklyn li 6010
43 Michigan mi 6010
43 Hawaii hi 6010
46 Alabama al 6005
47 LSU lsu 5985
48 Northwestern nw 5955
49 Cal State Northridge csn 5940
49 Arkansas-Little Rock alr 5940
49 Oklahoma ok 5940

[bayarea]

I like this ranking system, with 5 WCC teams in the top 40.

[The Bofa on the Sofa]

I haven't had a chance to evaluate anything other than how well it reflects who won

[badgerbreath]

Is this just a fitting procedure...in other words, do you test against a set of data not used to generate the rankings? And are you simply searching through rankings to see which results in the best fit to outcomes, in terms of who beats whom? Or are their other inputs, like points scored, home-away etc?

[pogoball]

Wow, 88% is crazy!

I'm looking forward to reading about the overall methodology. Is this a completely new approach for you, something you've considered in the past but never implemented, or what?

Great stuff as usual.

[nik12]

Very cool. 88% is impressive.

[The Bofa on the Sofa]

Nov 20, 2014 19:18:27 GMT -5 nik12 said:

Very cool. 88% is impressive.

90% is not uncommon in naia or d3 where matchups are more lopsided on average

[BeachbytheBay]

what about early season W/L's vs. recent W/Ls - treated equally? does it evaluate 'trending' at all


how does it treat a team like Davis that was lousy (150+ Massey) and is now good (80ish Massey) ?

[BeachbytheBay]

Nov 20, 2014 17:16:20 GMT -5 bayarea said:

I like this ranking system, with 5 WCC teams in the top 40.

I hate it - lol. The Big West took a dump with this.

and here I thought the Big West and WCC suffered from the same disease - RPI!!!

[The Bofa on the Sofa]

Nov 20, 2014 17:36:59 GMT -5 badgerbreath said:

Is this just a fitting procedure...in other words, do you test against a set of data not used to generate the rankings? And are you simply searching through rankings to see which results in the best fit to outcomes, in terms of who beats whom? Or are their other inputs, like points scored, home-away etc?

It's not a forecasting method, so there's no training set. It is just set up to get the most number of w/l right.

So it's kind of like the flip side of Pablo, which is designed to maximize in predicting wins in matches to be played, this maximizes wins in matches that have been played.

The algorithm of this is tough, though, and I do it brute force. As I mentioned elsewhere, the problem is that the function is not continuous, so there are some things you have to do to maximize the chance that you explore all the space. I've got a visual basic script that I use for an excel macro, and it's not optimal yet (it probably takes twice as long as it could, even using this approach).

There are some things that I want to explore with this, though. For example, adding a time component is pretty easy, because I've got the time function done. Another idea I want to explore is to give the range of ratings for each team. While these are the best ratings, there is room to move the ratings without losing W/L accuracy, and that can be good info. So if you can a team is rated X, but could also be X+100. Not hard, but will take a bit of programming. I haven't done visual basic in in 10 years, so I have to remind myself on that.

[vbprisoner]

In your description you said it ranks teams based on won/loss, I like the ranking outcome, but why would Kentucky be one spot in front of USC when USC beat Kentucky? Is it because USC had several other losses or it was early in season?

[The Bofa on the Sofa]

Nov 20, 2014 20:56:54 GMT -5 vbprisoner said:

In your description you said it ranks teams based on won/loss, I like the ranking outcome, but why would Kentucky be one spot in front of USC when USC beat Kentucky? Is it because USC had several other losses or it was early in season?

Because it's still preliminary. It has NOTHING to do with early in the season or anything like that.

If it ends up that way, then it is because moving Kentucky up or moving USC down would leads to less fewer correct results.

As I said, this is a challenge because you have to go back and recheck and recheck and recheck, and after getting everything converged, you start over and do it again

This version was only about 5 iterations in. I've got probably 5 more to go before declaring it done.

For example, I know that it has improved again. Currently up to 88.34%

[gnu2vball]

Is it like the Cyberdyne Systems in the Terminator movies? Has it become sentient?

[The Bofa on the Sofa]

Nov 20, 2014 17:49:59 GMT -5 pogoball said:

Wow, 88% is crazy!

I'm looking forward to reading about the overall methodology. Is this a completely new approach for you, something you've considered in the past but never implemented, or what?

Great stuff as usual.

I actually did something like this back 10+ years ago, but never pursued it. As I said, and you will see, there are sometimes some bizarre results that will make people uncomfortable. However, they make sense.

However, without focusing on results, I'd rather talk about the concept. Right idea? Too simplistic of a concept? Other?