|
Post by The Bofa on the Sofa on Dec 2, 2015 14:41:51 GMT -5
I will say, I can sympathize with the OP. I get tired of hearing whining about RPI, especially in cases where it is off topic. Save your rants for the appropriate places. But discussions about who is getting in the tournament are not the places to complain about RPI. Save it for threads like this. The pattern is persistent. How do you fix it? I think you covered it in detail a year or two ago, but would weighting the 3 factors differently help RPI. Say 40-20-40, just as an example. The opp opp % differences are minute, correct? So maybe a 40-40-20 would help? Or 40-50-10? Just spitballin' here. Whenever I've tried to improve RPI in terms of reflecting winners, it goes the other way - up the contribution from Opp-Opp. However, the problem with that is that it is a correlation, not a cause. Teams aren't better because they have bigger Opp-opp records, but better teams have schedules that have higher opp-opp records.
|
|
|
Post by jake on Dec 2, 2015 16:03:19 GMT -5
I will say, I can sympathize with the OP. I get tired of hearing whining about RPI, especially in cases where it is off topic. Save your rants for the appropriate places. But discussions about who is getting in the tournament are not the places to complain about RPI. Save it for threads like this. For example, you want to know why people complain about RPI? It's for reasons like these.... 1) RPI is supposed to be a reflection of what a team has done, taking SOS into account, and is not supposed to be a predictive tool. OK, let's see how it does then, in terms of reflecting "what a team has done." For example, how well does it reflect outcomes of matches that has been played? This year it was 81.4%. That's historically consistent. Is it good? Well, the Ultimate Rankings gets something on the order of 87% (ignoring HCA). Pablo rankings this week have 83.3% with HCA, and if I just ignore the HCA, it's still 82.2%, and that's not optimized. So in terms of just reflecting who won, RPI doesn't do as good as a job as Pablo. That's a difference of about 60 matches that Pablo gets right that RPI doesn't. And that is just looking at the thing that RPI is supposed to be doing best! 2) And then there are the regional issues. If we break down the correct outcomes over inter-regional matchups, the pattern is terrible. For example, when teams from the (NCAA) east region play matches outside the region, when they are favored by RPI they win 78% of the time, but when the opponent is favored, they win 84% of the time. What that means is that RPI is overrating teams from the east region. The midwest is worse, winning only 77% of the time when favored, but non-regional opponents are winning 87% of the time when they are favored. At the other end, when teams from the Pacific or West regions are favored in inter-regional matchups, they win 86-87% of the time, but when the opponent is favored, they only win 72% against the Pacific team and 78% against the West region team. You can group them up by trends. East/Midwest/Northeast are the regions overrated, Central/Mideast/South are pretty neutral, and Pacific/West are underrated. Add them up, and you get the following comparisons E-MW-NE vs C-ME-S When the EMWNE teams are favored they win 79.2% of the time When the CMES teams are favored, they win 84.9% of the time E-MW-NE vs P-W When the EMWNE teams are favored, they only win 61.7% of the time (yikes!) When the PW teams are favored, they win 90.8% of the time RPI is massively overrating the eastern teams here C-ME-S vs P-W When the CMES team is favored, they win 75.9% of the time When the PW team is favored, they win 88.1% of the time Again, the western teams are massively underrated. And the problem is, this happens every. Stinking. Year. I've done this analysis many years, and it is always the same. The Pacific and (to a lesser extent) West teams are getting hammered, compared to what they have done. They beat the teams in the east, but, ultimately, are more likely to be ranked behind them in RPI than when the teams in the east beat those in the west. The pattern is persistent. How do you fix it? This is too convoluted for me,...just a simple guy.
The RPI would be fine...if,... 1. it would drop the % figures of the opponent's opponent's record. 2. teams cannot lose RPI points for beating another team,...OOC or conference matches.
Make these adjustments and at least the playing field is level.
|
|
|
Post by The Bofa on the Sofa on Dec 2, 2015 16:15:40 GMT -5
I will say, I can sympathize with the OP. I get tired of hearing whining about RPI, especially in cases where it is off topic. Save your rants for the appropriate places. But discussions about who is getting in the tournament are not the places to complain about RPI. Save it for threads like this. For example, you want to know why people complain about RPI? It's for reasons like these.... 1) RPI is supposed to be a reflection of what a team has done, taking SOS into account, and is not supposed to be a predictive tool. OK, let's see how it does then, in terms of reflecting "what a team has done." For example, how well does it reflect outcomes of matches that has been played? This year it was 81.4%. That's historically consistent. Is it good? Well, the Ultimate Rankings gets something on the order of 87% (ignoring HCA). Pablo rankings this week have 83.3% with HCA, and if I just ignore the HCA, it's still 82.2%, and that's not optimized. So in terms of just reflecting who won, RPI doesn't do as good as a job as Pablo. That's a difference of about 60 matches that Pablo gets right that RPI doesn't. And that is just looking at the thing that RPI is supposed to be doing best! 2) And then there are the regional issues. If we break down the correct outcomes over inter-regional matchups, the pattern is terrible. For example, when teams from the (NCAA) east region play matches outside the region, when they are favored by RPI they win 78% of the time, but when the opponent is favored, they win 84% of the time. What that means is that RPI is overrating teams from the east region. The midwest is worse, winning only 77% of the time when favored, but non-regional opponents are winning 87% of the time when they are favored. At the other end, when teams from the Pacific or West regions are favored in inter-regional matchups, they win 86-87% of the time, but when the opponent is favored, they only win 72% against the Pacific team and 78% against the West region team. You can group them up by trends. East/Midwest/Northeast are the regions overrated, Central/Mideast/South are pretty neutral, and Pacific/West are underrated. Add them up, and you get the following comparisons E-MW-NE vs C-ME-S When the EMWNE teams are favored they win 79.2% of the time When the CMES teams are favored, they win 84.9% of the time E-MW-NE vs P-W When the EMWNE teams are favored, they only win 61.7% of the time (yikes!) When the PW teams are favored, they win 90.8% of the time RPI is massively overrating the eastern teams here C-ME-S vs P-W When the CMES team is favored, they win 75.9% of the time When the PW team is favored, they win 88.1% of the time Again, the western teams are massively underrated. And the problem is, this happens every. Stinking. Year. I've done this analysis many years, and it is always the same. The Pacific and (to a lesser extent) West teams are getting hammered, compared to what they have done. They beat the teams in the east, but, ultimately, are more likely to be ranked behind them in RPI than when the teams in the east beat those in the west. The pattern is persistent. How do you fix it? This is too convoluted for me,...just a simple guy.
The RPI would be fine...if,... 1. it would drop the % figures of the opponent's opponent's record. Makes it worse. If anything, RPI underweights Opp-Opp Why not? I don't see anything wrong with it, given the motivation. You want to encourage teams to play opponents up to their level. That's the first thing. The second thing is that you are thinking about it wrong. RPI is really only designed to be considered after the schedule is complete. In fact, the NCAA used to not even release it until afterward, because they didn't want people to think of it like that. Given that the RPI only should be applied after all your matches are played, there is no "drop" in RPI due to anything. There is only one value that has any meaning. This is why I'm such a strong advocate for BluePenguin's RPI futures. It's trying to deal with that issue, the end of the season RPI. We are already seeing this crap with basketball, with people posting the current RPI rankings, and mocking them as being silly. Which they are. But the problem is, no one says otherwise at this point. No one ever suggested that RPI has any meaning 3 weeks into the basketball season, and people who are trying to point to it as anything are making one big straw man. No, it's not "a good thing that the NCAA committee doesn't rely on RPI." They do, in fact, to a pretty large extent. Not as much as other sports, but they still do. What they don't do is pay any attention to the RPI rankings 3 weeks into the season.
|
|
bluepenquin
Hall of Fame
4-Time VolleyTalk Poster of the Year (2019, 2018, 2017, 2016), All-VolleyTalk 1st Team (2021, 2020, 2019, 2018, 2017, 2016)
Posts: 12,423
|
Post by bluepenquin on Dec 2, 2015 16:18:17 GMT -5
You don't need to get rid of RPI, just use it sanely... like the Men's Basketball Committee, that leaves teams out with an RPI of 22 when they recognize that rating is bunk from a conference-wide RPI inflation, and will take teams with high-60s RPIs when they have a good resume.
Seniors from one of the snubbed teams (Colorado? Da Beach?) should file a Title IX suit against the NCAA, arguing for disparate treatment (not in the RPI being used, but HOW it's being used) from MBB. I don't even think they'd need to win to get the NCAA to settle and agree to some different procedures. I agree with this in the sense that they are not getting rid of the RPI, so how can they improve the selection process within the framework of RPI?
They have been making 'adjustments' on the selection for years - usually favoring PAC and B1G schools below the cut line. For this year, where I think the committee failed (and for the most part I think this bracket is the best one in years) is why would they move the 4th team from the Missouri Valley ahead of the 2nd and 3rd team in the Mountain West who had a better RPI? I have no problem with leaving Colorado out - or essentially the 8th team from the PAC 12 - but why not include the 2nd team from the Big West who had an RPI better than a team that did get a bid?
IMO, the error is not that they may on occasion exclude Colorado or Michigan State this year or Purdue/Minnesota last year - but they are excluding Long Beach, Wyoming/Boise this year and St. Mary's from a couple years ago. I am not saying that the Mtn West should always get at least 2 teams in the tournament - but I think they should have this year and particularly in a year when they actually had a better RPI then a team from the MVC that gets in.
|
|
|
Post by vbphilsdad on Dec 3, 2015 7:27:40 GMT -5
Maybe the RPI should be "pre-populated" with ALL scheduled opponents and opp opps. Right from the beginning of the season, for example, this year Florida is impacted by LIU and their opp opp %, and LIU is impacted by Florida's -- since they WILL play. Then there is neither a hit or jump when they actually play, except for who wins or loses that game. From day one, a teams RPI would be impacted by the same "basket" of teams. Then only conference tourneys (or cancellations) would change how a team's RPI is calculated.
|
|
|
Post by The Bofa on the Sofa on Dec 3, 2015 7:44:03 GMT -5
Maybe the RPI should be "pre-populated" with ALL scheduled opponents and opp opps.. That's basically what bluepenguin's RPI futures does. It's a good approach.
|
|
|
Post by dd2000 on Dec 3, 2015 9:28:51 GMT -5
This is probably the best discussion of RPI I've seen. Thanks to the OP for kicking this off
|
|
|
Post by vbphilsdad on Dec 3, 2015 10:56:10 GMT -5
Maybe the RPI should be "pre-populated" with ALL scheduled opponents and opp opps.. That's basically what bluepenguin's RPI futures does. It's a good approach. Yup. Except, that there is a bit of speculation as to who will win (I believe), and predicts next week's RPI (likely based on the current RPI). Kinda self-feeding. I guess I'm wanting non-predictive RPI futures (maybe he is doing this).
|
|
|
Post by The Bofa on the Sofa on Dec 3, 2015 11:06:14 GMT -5
That's basically what bluepenguin's RPI futures does. It's a good approach. Yup. Except, that there is a bit of speculation as to who will win (I believe),
No. There is not "quite a bit of speculation." There is prediction based on Pablo, which has some variation for an individual team wins and losses, but works VERY well for opp and opp, because the sample sizes are vary large. This is the whole reason BluePenguin started doing it, because he realized that the opp and opp opp records are actually pretty easily determined once you get into the conference season. ? No, it predicts the end of the season RPI. And it does a great job at doing that. The variation you see is mostly based on teams performing above or below expectations. Win matches you should, you stay in place. Upset someone, you move up. Get upset, you move down. It's beautiful.
|
|
bluepenquin
Hall of Fame
4-Time VolleyTalk Poster of the Year (2019, 2018, 2017, 2016), All-VolleyTalk 1st Team (2021, 2020, 2019, 2018, 2017, 2016)
Posts: 12,423
|
Post by bluepenquin on Dec 3, 2015 12:42:51 GMT -5
Yup. Except, that there is a bit of speculation as to who will win (I believe),
No. There is not "quite a bit of speculation." There is prediction based on Pablo, which has some variation for an individual team wins and losses, but works VERY well for opp and opp, because the sample sizes are vary large. This is the whole reason BluePenguin started doing it, because he realized that the opp and opp opp records are actually pretty easily determined once you get into the conference season.
Having never really thought about this before 3 years ago - the reality of this hit me like a ton of bricks. The startling thing is this becomes way more determined/predictable after as few as 2 weeks or at least 3 weeks than I would have ever thought and I assume that most realize(d).
Example - it was obvious after week 2 or 3 that Hawaii had no chance of getting an RPI under 20 - no matter what they did. At the same time, it was obvious that Arkansas State and Western Kentucky could achieve a sub 16 RPI.
|
|
|
Post by BeachbytheBay on Dec 3, 2015 13:03:06 GMT -5
I know this will be too complicated for the NCAA to consider, but consider this,
1) Consider the OOC SOS as a factor and give it weight - that way mid-majors (who can't control the records of many conference members) may not get penalized as much for poor SOS brought on by a down conference - not saying overall SOS isn't important, but when I see the comment of a team having a poor SOS, there is only so much a team can do.
2) If you are gonna really weigh heavily having a top 25 win (and so many of the teams below 40 basically end up in the range of 0-2 wins - and not a lot with 2, that magnifies perhaps incorrect the chance of a fluke top 25 win getting weighted way too much - while I agree in the seeding and top 16 evaluation, top 25 is a much more important statistic - but it really shouldn't carry a disproportionate weight when you get below the top 40 (or maybe top 30) teams. The bubble practically starts at 30. Granted if a team on the bubble has 3 top 25 wins, that's really significant.
I hate to keep using Oregon as the example because they are a tournament quality team - but it really boggles my mind that a team that had only FOUR top 100 wins with ONE of those against the depleted NCAA tourney team ASU - and with only ONE out of conference (under the NCAA criteria of 'quality' using RPI) gets an at-large. again, they are a good team, and a tournament quality team, but it really makes me wonder how they applied their criteria in Oregon's case - it reeks of them applying selective criteria to get them in the tourney. Oregon is a team that was defined by who they lost to more than who they beat, and that would never happen in Men's basketball.
|
|
|
Post by c4ndlelight on Dec 3, 2015 13:12:08 GMT -5
I know this will be too complicated for the NCAA to consider, but consider this, 1) Consider the OOC SOS as a factor and give it weight - that way mid-majors (who can't control the records of many conference members) may not get penalized as much for poor SOS brought on by a down conference - not saying overall SOS isn't important, but when I see the comment of a team having a poor SOS, there is only so much a team can do. 2) If you are gonna really weigh heavily having a top 25 win (and so many of the teams below 40 basically end up in the range of 0-2 wins - and not a lot with 2, that magnifies perhaps incorrect the chance of a fluke top 25 win getting weighted way too much - while I agree in the seeding and top 16 evaluation, top 25 is a much more important statistic - but it really shouldn't carry a disproportionate weight when you get below the top 40 (or maybe top 30) teams. The bubble practically starts at 30. Granted if a team on the bubble has 3 top 25 wins, that's really significant. I hate to keep using Oregon as the example because they are a tournament quality team - but it really boggles my mind that a team that had only FOUR top 100 wins with ONE of those against the depleted NCAA tourney team ASU - and with only ONE out of conference (under the NCAA criteria of 'quality' using RPI) gets an at-large. again, they are a good team, and a tournament quality team, but it really makes me wonder how they applied their criteria in Oregon's case - it reeks of them applying selective criteria to get them in the tourney. Oregon is a team that was defined by who they lost to more than who they beat, and that would never happen in Men's basketball. Oregon had a SIGNIFICANTLY better OOC SOS than Boise, Wyoming, LBSU and Pitt and had a Pablo and Massey rating than all of the others. The Beach's OOC SOS was 93, fared worse in common opponents than Oregon, have nastier bad losses... yet because the Beach went 4-0 against teams ranked between 73-98 in RPI (in other words, teams not close to good enough to being in conversation for the tourney and who you don't prove tournament caliber by beating) you want to say the Beach has a better resume? OK. Boise didn't beat any non-AQ tournament teams - how are they defined by who they beat? Pitt's a good team but they scheduled worst of all (163 OOC SOS!) and lost to two non-tourney teams in the last week of the season. Wyoming I am somewhat sympathetic too, but then I also note that they have a huge HCA and didn't beat a Top 100 team away from home (and have 2 absolutely atrocious road losses that call the HCA into question),
|
|
|
Post by aztecbuff on Dec 3, 2015 13:19:10 GMT -5
I have no problem with leaving Colorado out - or essentially the 8th team from the PAC 12 - but why not include the 2nd team from the Big West who had an RPI better than a team that did get a bid?
5th best team in the Pac 12.
|
|
|
Post by The Bofa on the Sofa on Dec 3, 2015 13:33:36 GMT -5
No. There is not "quite a bit of speculation." There is prediction based on Pablo, which has some variation for an individual team wins and losses, but works VERY well for opp and opp, because the sample sizes are vary large. This is the whole reason BluePenguin started doing it, because he realized that the opp and opp opp records are actually pretty easily determined once you get into the conference season.
Having never really thought about this before 3 years ago - the reality of this hit me like a ton of bricks. I love those things, where you get that major insight that you didn't see coming. The key to getting hit by the ton of bricks is that it has to be something that you really didn't see coming, and that it has to be significant. I will admit to not having thought or paid attention to it previously, but I consider this one really significant in what you've uncovered. I don't know to what extent it is appreciated or understood by the RPI guys in other sports. This is kind of the same impact as the "pitcher has no control over batting on balls in play" discovery in baseball. Once you understand that, it changes completely how you view things. In my own experience, that happened to me when I did the plot of transformed winning percentage vs point percentage. I was just hoping to get an empirical relationship that I could use to turn Pablo rating differences into point probability, so I could simulate individual matches. Instead, I got a straight line, and I was like, whoa, that is bizarre? But that led to a completely revised view of how to think about Pablo rankings.
|
|
|
Post by BeachbytheBay on Dec 3, 2015 13:35:49 GMT -5
"Oregon had a SIGNIFICANTLY better OOC SOS than Boise, Wyoming, LBSU and Pitt and had a Pablo and Massey rating than all of the others. The Beach's OOC SOS was 93, fared worse in common opponents than Oregon, have nastier bad losses... yet because the Beach went 4-0 against teams ranked between 73-98 in RPI (in other words, teams not close to good enough to being in conversation for the tourney and who you don't prove tournament caliber by beating) you want to say the Beach has a better resume? OK. Boise didn't beat any non-AQ tournament teams - how are they defined by who they beat? Pitt's a good team but they scheduled worst of all (163 OOC SOS!) and lost to two non-tourney teams in the last week of the season. Wyoming I am somewhat sympathetic too, but then I also note that they have a huge HCA and didn't beat a Top 100 team away from home (and have 2 absolutely atrocious road losses that call the HCA into question)," Read more: volleytalk.net/thread/61174/stop-rpi-complaints#ixzz3tHgrsDiRnone of what you've said changes the fact that using the NCAA's criteria - Oregon had a total of FOUR top 100 wins, 3 less than LB (and we can look at a number of other teams), that is the danger of giving too much weight to SOS Oregon went 2-3 against teams 51-100 - really? all I am saying is their resume is really suspect based on RPI and they got rewarded for losing to a LOT of good teams and if we want to use Massey, LB had 12 top 100 wins, and 3 top 50 wins, so it actually would enhance their breadth of quality wins if we want to parse wins, then LB had 2 wins from RPI 26-30, so in relation to Oregon, it's BETTER than their 1 'fluke' win against UCLA again, I'm not trying to bag on Oregon cause they are a good team - but their inclusion (and they had a rep on the committee) makes the analysis interesting it points to the issues of using a system (like Pablo) as well that 'rewards' points scored - because Pablo is really a predictive based model that ranks based on a goal of predicting, not necessarily intended to be the end-all for selection. my point wasn't to suggest Boise was tournament quality
|
|