Bracketology (11/6)

[tablealgebra]

Nov 11, 2024 23:54:09 GMT -5 n00b said:

THIS IS THE WAY

So I want to make sure I'm interpreting this the correct way, because I certainly didn't at first glance.

The metric (after power ranking all the teams)...
1) looks at a team's schedule
2) creates probabilities of how many games an "average AVCA Top 25 team" would win against that schedule (wondering if they pick a power ranking in the middle, or run separate results for each top 25 team)
3) uses the team's record and assigns a probability of winning at least as many games to that team
4) ranks the teams based on that, lowest probability being the highest rank.

Which seems very solid. As always, not seeing the engine behind the statistic means a leap of faith but I like this a lot

[bbg95]

Nov 12, 2024 10:02:16 GMT -5 bluepenquin said:

Nov 12, 2024 9:32:59 GMT -5 bbg95 said:

My point is that Massey uses RAT as his actual rating. PWR is just a component and not equivalent to Pablo, which is the actual rating for that system. I don't think PWR is as refined as Pablo is, so I don't really see the point of using it when its creator uses RAT as the primary ranking.

In terms of win probabilities - which was the point of the discussion, who is the better team and who is more likely to win a hypothetical match - PWR is clearly what Massey uses and not RAT.  And when wanting Massey's comparable to Pablo - it would have to be PWR.  Pablo directly creates win probabilities.  Massey also directly creates win probabilities from its PWR rating and not the RAT rating.

Again, though, the default ranking is RAT. If Massey wanted PWR to be the primary consideration, he would use that as the default.

[bluepenquin]

Nov 12, 2024 10:34:39 GMT -5 bbg95 said:

Nov 12, 2024 10:02:16 GMT -5 bluepenquin said:

In terms of win probabilities - which was the point of the discussion, who is the better team and who is more likely to win a hypothetical match - PWR is clearly what Massey uses and not RAT.  And when wanting Massey's comparable to Pablo - it would have to be PWR.  Pablo directly creates win probabilities.  Massey also directly creates win probabilities from its PWR rating and not the RAT rating.

Again, though, the default ranking is RAT. If Massey wanted PWR to be the primary consideration, he would use that as the default.

Yes - default ranking it is.  But that wasn't what the conversation was about.  It was about what Massey thinks in terms of who the better team is.


RAT is more about who is more deserving.  RAT would be what RPI is trying to do.  PWR is what Pablo is trying to do.  And in terms of who is more likely to win a future match - Massey is using PWR and not RAT.

[bluepenquin]

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

[bbg95]

Nov 12, 2024 10:54:11 GMT -5 bluepenquin said:

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

Who are you talking about? I think RPI is terrible. My analysis of them is based on who they have beaten and who they have lost to. Penn State is also sixth in adjusted RPI right now, so it's not even like RPI favors them that much. It's just not laughably low like Pablo is at 11.

Edit: Purdue, a five-loss team that Penn State beat by 20 points on the road, is ranked higher (9) in Pablo than Penn State (11) right now.

[trianglevolleyball]

Nov 12, 2024 10:54:11 GMT -5 bluepenquin said:

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

Has someone really said Pablo should be used to determine seeding? That sounds purely idiotic, why even play matches at that point? Just have teams play 100 points against each other and don’t mark down W or L.

[c4ndlelight]

Nov 12, 2024 10:54:11 GMT -5 bluepenquin said:

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

Strawman. No one is saying to rate Penn State high because of their RPI.

[boxcariii]

Nov 8, 2024 22:34:13 GMT -5 bbg95 said:

Gaming RPI in basketball no longer works. Gaming NET in basketball still does, at least until they hopefully tweak it. I think most agree that RPI is a bad metric that should be abandoned in favor of something better. But until that happens, you can't blame teams and conferences for gaming it.

Sorry about this late response, but you raised a very good point about basketball coaches gaming the NET now vs the RPI previously. You will never ever create a mathematical model that won't eventually by gamed by very good coaches (if it's used as a basis for selection). Right now, it's NET for basketball and RPI for most other sports.

I like the idea of using multiple models as "tools" to inform decisions, but not necessarily as official criteria. I understand why the NCAA wants a baseline metric though. Having one assists in speeding up the selection process (for better or for worse). The fact that coaches will always game whatever system is created does not preclude them from using a better different metric or making RPI better (in the way that two things can be true at the same time).

[n00b]

Nov 12, 2024 11:54:56 GMT -5 boxcariii said:

Nov 8, 2024 22:34:13 GMT -5 bbg95 said:

Gaming RPI in basketball no longer works. Gaming NET in basketball still does, at least until they hopefully tweak it. I think most agree that RPI is a bad metric that should be abandoned in favor of something better. But until that happens, you can't blame teams and conferences for gaming it.

Sorry about this late response, but you raised a very good point about basketball coaches gaming the NET now vs the RPI previously. You will never ever create a mathematical model that won't eventually by gamed by very good coaches (if it's used as a basis for selection). Right now, it's NET for basketball and RPI for most other sports.

I like the idea of using multiple models as "tools" to inform decisions, but not necessarily as official criteria. I understand why the NCAA wants a baseline metric though. Having one assists in speeding up the selection process (for better or for worse). The fact that coaches will always game whatever system is created does not preclude them from using a better different metric or making RPI better (in the way that two things can be true at the same time).

Strength of Record is basically impossible to game. You’d need to (a) schedule matches you have a low probability of winning, and (b) win those matches.

[bbg95]

Nov 12, 2024 12:17:55 GMT -5 n00b said:

Nov 12, 2024 11:54:56 GMT -5 boxcariii said:

Sorry about this late response, but you raised a very good point about basketball coaches gaming the NET now vs the RPI previously. You will never ever create a mathematical model that won't eventually by gamed by very good coaches (if it's used as a basis for selection). Right now, it's NET for basketball and RPI for most other sports.

I like the idea of using multiple models as "tools" to inform decisions, but not necessarily as official criteria. I understand why the NCAA wants a baseline metric though. Having one assists in speeding up the selection process (for better or for worse). The fact that coaches will always game whatever system is created does not preclude them from using a better different metric or making RPI better (in the way that two things can be true at the same time).

Strength of Record is basically impossible to game. You’d need to (a) schedule matches you have a low probability of winning, and (b) win those matches.

Correct me if I'm wrong, but isn't KPI a similar concept to Strength of Record? Relevant because KPI is now an official criteria.

[staticb]

Nov 12, 2024 10:54:11 GMT -5 bluepenquin said:

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

Isn't Penn State often rated lower in Pablo than i the RPI?  When I think of reasons for Pablo, Penn State and it's historical regional RPI advantage is one of the teams that come to mind.

[mikegarrison]

Nov 12, 2024 11:54:56 GMT -5 boxcariii said:

You will never ever create a mathematical model that won't eventually by gamed by very good coaches (if it's used as a basis for selection). Right now, it's NET for basketball and RPI for most other sports.

Metrics stop being metrics when they start being targets.

en.wikipedia.org/wiki/Goodhart%27s_law

[bluepenquin]

Nov 12, 2024 12:49:38 GMT -5 staticb said:

Nov 12, 2024 10:54:11 GMT -5 bluepenquin said:

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

Isn't Penn State often rated lower in Pablo than i the RPI?  When I think of reasons for Pablo, Penn State and it's historical regional RPI advantage is one of the teams that come to mind.

No - Pablo has usually rated PSU very well and often times better than RPI.  My opinion in the Russ Rose days - he didn't give a crap about RPI and did nothing to 'manipulate' his RPI.  As such - his RPI was often times worse than Pablo and sometimes a lot worse.  But when his teams were really great - RPI and Pablo were likely to be pretty similar.

PSU did take advantage of the RPI system (in a small way) - by having weaker RPI opponents (vs. Pablo) in the tournament early rounds.  Geography was an even bigger factor.

[bluepenquin]

Nov 12, 2024 11:06:51 GMT -5 trianglevolleyball said:

Nov 12, 2024 10:54:11 GMT -5 bluepenquin said:

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

Has someone really said Pablo should be used to determine seeding? That sounds purely idiotic, why even play matches at that point? Just have teams play 100 points against each other and don’t mark down W or L.

Yes.  There has been a large group of people on this board that believes;

1) Pablo is greatly superior to RPI
2) Pablo should be used as one of the key metrics for determining seeding.

[bluepenquin]

Nov 12, 2024 11:43:57 GMT -5 c4ndlelight said:

Nov 12, 2024 10:54:11 GMT -5 bluepenquin said:

Interesting to see some of those that have been most critical of RPI in the past - and calling out for a better system such as Pablo - are the ones that want to so easily trash Pablo and embrace RPI when it comes to Penn State this year.

Strawman. No one is saying to rate Penn State high because of their RPI.

You have been among the strongest proponents of Pablo for 10+ years (along with among the strongest critics of RPI).  Interesting that you are now so easily able to toss Pablo aside with regards to Penn State this year.  Interesting that RPI for Penn State is much closer to what believe than Pablo.

If Pablo was the key metric for seeding instead of RPI - Penn State isn't sniffing a top 4 seed.  With RPI as the key metric - they are 85% likely to be a top 4 seed.