SB Nation - Login for mobile commenting

Burnt Orange Nation

Pundit Roundup Thinks Doug Lesmerises is a National Treasure

Punditroundup

Highlights this week include: a discussion of human polls generally and the Doug Lesmerises' controversy specifically; new installments of The Philosophers Club and Twitter Tracker; plus this week's Undulating Curve of Media Hype.  Ready, set, go.

Star-divide

Doug Lesmerises is a sportswriter for the Cleveland Plain Dealer.  I had no idea who he was until about a week ago, and I'm betting you didn't either.  Turns out he's an AP voter and he decided to do a novel thing with his ballot starting last week, which was (i) throw away his ballot from the previous week, (ii) take into account the games that had been played already, (iii) not  take into account his personal belief how good he thinks teams probably are based on things other than the games that had been played, (iv) not take into account predictions about where teams will end up ranked at the end of the year.  In a nutshell, this is resume-ranking and this is apparently what AP voters have been told to do the entire time.  According to Lesmerises, the AP guideline is: "Base your vote on performance, not reputation or preseason speculation."

Philosopher

After Ole Miss lost to South Carolina on Thursday, Rebel Bradley Sowell had this to say about his team's previously lofty ranking: "I'm glad it's gone so we can get back to working and win some ballgames.  You can't really pay attention to [the ranking] anyway. I'm glad it's over with so everyone can just stop talking about it and play ball." That speaks more to the state of that program under Houston Nutt than anything else I could possibly imagine.  Ole Miss fans agree.

So there you go.  Lesmerises read the AP guidelines for voting, adhered to them and came up with a top 25 ranking reflecting his assessment of the resumes of teams based solely on the actual results of the actual games that had already been played.  So of course he was feted throughout the land for doing his job correctly.....what's that?  No?  Everyone called him an idiot and called for his ballot to be taken away?  Well, that's completely irrational isn't it?

The answer is yes.  Completely and utterly irrational.  First of all, the AP guidelines for ranking teams tell everyone to do exactly what he did.  Second, the reason they have guidelines for ranking teams is that, in order for a poll of numerous people to be a valid measure of anything, the pollsters have to be in agreement about what exactly they are ranking.  Third, the reason the AP chose resume ranking as its guideline rather than anything else is that it's the only completely rational way of ranking teams.  Fourth, as evidenced by Lemerises' explanation of this week's rankings, he's thinking long and hard about what he's doing.

Unfortunately, a lot of people out there are arguing with Lesmerises about the wrong things.  They're saying that he's stupid or just being provocative because he's ranking teams differently than other pollsters.  But in doing so, they are ridiculing his method, not his results, even though the method is a rational system prescribed by the organization administering the poll.  They are arguing with points 1-4 in the previous paragraph rather than the results of points 1-4.  You may disagree with the way in which Doug Lesmerise ranks certain teams, and that's fine, but in doing so, you should, for example, be arguing that Florida's wins are better than Houston's wins rather than that Florida deserves to be #1 because you and everyone else obviously knows that they're the best team in the country.  In short, the debate should be framed the way that Chris Littman at The Sporting News framed it: I understand what you're doing and it's legitimate, but based on this resume standard, I disagree with you for the following reasons. Let's talk about rankings in way that's meaningful rather than counterproductive.

Twitter_tracker_medium

Bruce Feldman: "RT @MoveTheSticks Did Jack Youngblood get as much credit for playing with a broken leg as Tebow is getting for playing with a cold?"

Bruce Feldman: "Wife's observation: why are there six cops surrounding Bobby Petrino during this interview?"
Bruce Feldman: "Well done. RSFarley@BFeldmanESPN. Arkansas State Troopers are assigned to make sure Petrino doesn't leave for another job during halftime."

Now, if I may speak about polls a bit more esoterically (as if this entire weekly column isn't a bit esoteric to begin with), I have gone on record several times stating my general support for the BCS, while laying out a plan to modify the system to help it achieve its own goals better.  But my one caveat to this general support has been that the ranking systems have to get better.  For example, as I said last December, "(1) there should be nothing that computers can't take into account except for perhaps margin of victory over a certain point (really, beating a team by 42 points isn't much different from beating them by 28 points), (2) there should be more computers to get a meaningful average, and (3) human voters shouldn't be idiots."  I should also add to this list the following: (4) the coaches should have nothing whatsoever to do with anything related to polls.

Caveats one, two and four are easily fixable by a rule change.  Caveat three is much more difficult to fix because it hinges on the vagaries of human minds (some of which don't operate at optimal capacity), but this Doug Lesmerises incident has brought me around to thinking that we're on a good path towards fixing that as well.  While resume ranking is nothing new, and pollsters have been claiming to do it for years, absolute pure resume ranking is really a rather recent phenomenon, at least as far as I'm aware.  And more importantly than it being new is that it has operated largely in the realm of bloggers like Sunday Morning Quarterback.  But then something happened and mainstream printing-press-by-god-sportswriters like Doug Lesmerises started to take note and realize that what they had been doing for all these years didn't really make sense.

I think some of it has to do with things like SMQ becoming Dr. Saturday (while remaining Matt Hinton) and writing for a fairly mainstream outlet like Yahoo Sports (while still pounding the drum of resume ranking), but I think that, more than that, it's a symptom of a larger trend: we live in a golden age of college football discourse.  Sure, there's a lot of nonsense out there, but if you know where to look and who's worth reading, you can find serious (and funny) discussions about aspects of college football that has been largely ignored for years.  When I was growing up, the AP poll was pretty much gospel.  While there were debates about whether one team really should have been ranked x spots higher than another, no one seriously asked fundamental questions about the poll like what it was trying to measure.  Now, everyone is debating resume-ranking methods, established scribes are beginning to take notice, and those with votes are actually thinking about the "why" and "how" of their ballots.

For instance, here is a quote from Doug Lesmerises about the poll generally, as told to Pat Forde of ESPN:

"What people think in August [when filling out preseason Top 25s] is at least partly based on what Phil Steele and Athlon thought in April [when their preview magazines are written]," Lesmerises said. "A multi-gazillion-dollar industry based on people putting 15 minutes into a poll that is based on a magazine that did its rankings in April is not a good system."

The voters have become self-aware!  Seriously though, I can't imagine hearing an AP voter say this 5-10 years ago. 

You might think it a coincidence that this new awareness arises at the same time as the proliferation of other, more open sources of media (i.e. blogs), but I don't.  Most mainstream writers read sites like BON and Dr. Saturday and EDSBS.  We've been contacted numerous times by ESPN.com authors about things that we've written.  That story about Chip Kelly giving a refund to a fan for his trip to BSU?  The one that you saw on SportsCenter?  That was originally reported on EDSBS.  Sportswriters read these things now and there's a new openness to the discourse on the college football polls.  And that's most definitely a good thing.  

I don't mean to say that they're stealing our ideas or don't have original thoughts themselves.  I just mean to say that the echo chamber is gone.  There are dissenting thoughts out there and some members of the media are starting to pay attention to them and re-evaluate their own ballots.  And while other members of the media don't pay attention to the dissent when it comes from people like me or Peter or Matt Hinton, they might when it comes from someone like Doug Lesmerises or Jon Wilner of the San Jose Mercury News.

Right now, those two AP voters are outliers.  But if the mainstream media debate over Lesmerises' ballot this past week is any indication (there were even multiple television segments about it!), more and more mainstream writers and AP voters are going to start to actually think about what they're doing and why they're doing it, rather than just doing what's always been done.  And the more the media and AP poll drive talk about this "novel idea" of polling, the more Harris Poll voters catch on and, perhaps, the better this whole BCS thing might get.  But seriously though, get rid of the coaches' poll.

And now, without further ado, but with apologies to New York Magazine and Adam Sternbergh, here is your weekly Undulating Curve of Media Hype.

Undulating_week_4

1 recs  |  150 comments

Comments

final rankings

I’d be interested to see how much different a “pure” resume ranker’s ballot like Lesmerises’ looks at the end of the regular season vs. the run-of-the-mill voter’s ballot at the end of the year. After one or two games, “pure” resume rankings are going to look pretty crazy, so I can see the temptation to try to project into the future.

By the end of the year, when the polls really matter, and when there aren’t any games left to play, I would think that most voters do revert to something close to pure resume rankings.

His methodology...

Is fine. But when you have a clear common opponent like Tech with Houston & Texas… You can’t ignore it.

Texas beat Tech by 10. Houston beat Tech by 1 pt on a last minute drive.

Based on that you put Houston at #3 and Texas @ #7???

Agreed that UH’s SOS is marginally better, but given that clear common opponent and given that we both have the same record that should be the trump card.

Houston beat Oklahoma State on the road...

And Texas beat, uh, Wyoming on the road…. Yes, Texas played Tech better than Houston played Tech and that certainly matters, but common opponents can’t be the be-all, end-all of the discussion.

Regardless, we can disagree on this point, but it’s a good conversation to have, which was one of the main points of the article.

again...

Common opponent shouldn’t be the end-all be all. But it SHOULD be the trump card in a case where we know very little about either team.

Again, it took Houston winning with 30 seconds left… and if Tech had made a chip shot earlier they wouldn’t have even won the game.

If onfield performance is the decider… then LET IT DECIDE.

I don't think anything should be a trump card at any point.

To say that one thing should absolutely trump everything else is to say, in effect, that nothing other than this one thing matters. And to do that is to disallow us as humans to have differing thoughts about the importance of various things. If we have trumps, then why not just have a set of rules for how teams should be ranked? Why include humans at all?

Look, I understand what you’re saying, but every human mind has the ability to think differently than you do about the relative importance of various things. For instance, I disagree with you here. The human polls are an amalgamation of different humans thinking in their own separate ways about the same topic. There should be no absolutes.

Oy.

Take LSU for example. Even though they’ve beaten more quality teams than most, they won them in a not-so impressive fashion. It shouldn’t just be THAT UH won, but HOW they won. Winning on a last second drive should be part of the equation. that is HOW they played ON THE FIELD.

For so many scenarios you have to look at other things… for example… who should be ahead… LSU or Texas. In that case we don’t have head to head OR a common opponent… But when you DO have one of those two they should be the primary factors when doing rankings…. Not the only ones…but primary.

If Texas had beaten Wyoming and UTEP by less than 10, then UH over Texas wouldn’t be so problematic for me. But given that both teams have taken care of business versus their lesser opponents means that common opponents should be given more sway.

This weekend UH will play UTEP. If they win 35-14 would you still say UH should be ahead of Texas? If so, then its pointless having this conversation. :)

Absolutes

There definitely should be those.

UH should ABSOLUTELY be ahead of Oklahoma State right now.

USC should ABSOLUTELY be ahead of Ohio State right now. To argue otherwise is to make a mockery of what happens on the field.

Texas Tech ABSOLUTELY should have been ahead of UT last year?
Nope.

That was more complicated because you had a 3 team circular. No such problem exists in the 2 examples I cited.

Given new information it's no longer an absolute?

So where does Washington fall? They beat USC and lost to Stanford. Are they above or below Ohio State?

Washington lost to Stanford… they have 2 losses. So USC over Washington is ok.

#1. Losses
#2. Head to head

Got it?

Our problem isn't that you think this is the way it should be.

It’s that you’re insisting that everyone should rank the exact same way that you do.

You have every right to look at what happens on the field and rank teams however you want. And so does everyone else in the world.

Maybe...you just added a new rule for the universe. I need time to adjust.

BYU is absolutely above OU right? Even if they both go undefeated the rest of the way?

That would have a one loss UT below both BYU and OU by necessity. Orrin Hatch would be happy in this world.

If USF goes undefeated then they have to be above BYU and OU. Orrin Hatch may be happy.

Sen. Hatch may not be happy.

Then he can bite the bullet. nt

Just as long as he doesn’t sing.

Lord help us. We might have to make that bullet a little less pristine. nt
In your hypothetical...

UT would have one loss to OU
OU would have one loss to BYU
USF would have 0 losses.

In my opinion USF would be ahead of all of us. Furthermore, USF beat Florida State who throttled BYU. That would be further proof that they belong in the discussion.

Why is Florida State important?

They’re a two loss team, at least, that’s the first priority of your system.

Unless we want to bring Washington back into the picture.

So you're saying that no reasonable person could possibly disagree with you?

That’s arrogant and absurd.

You act like I’m not talking about things that happened on the field. I’m talking about ONLY things that happened on the field. I’m talking about MULTIPLE games that happened on the field that should together determine a ranking. You’re insisting that ONE game that happened on the field should by itself determine a ranking notwithstanding anything else that happened on the field in any other game! Which one is more of a mockery of what happened on the field?

no no no...

But there are rules & standards.

If you ignore head to head than what’s the point?

No one's ignoring head to head!

We’re taking it into consideration along with everything else that’s happened! How is that wrong?

Again… i think there is a pecking order.

Maybe it would be good for everyone to agree on what the pecking order should be. That would be a fun article… :)

#1. losses. It doesn’t matter that Cal beat USC in 04. Cal had 6 losses and therefore should not have been ahead of a one or two-loss USC.

#2. Head to head. Of course there are exceptions like the UT-Tech-OU trio from last year. IN that case you go to the next several items in the pecking order.

#3. Common opponents.

#4. Nature of your wins/margin of victory. LSU and TX are both 4-0. But LSU has looked crappier than we have and won by the skin of their teeth once. UH beating OSU on the road is very important. Just not as important as #3 IMHO. Last year this is why I think TX should’ve been ahead of OU & TECH. We beat OU on a neutral field. Whereas we lost on the road and Tech won at home. We lost on a last second play…

Obviously we will all disagree to various extents… but lets not say that what happens on the field is of UTTERMOST importance and then ignore the clear relationship of a common opponent.

Hi. Every single game that is played "happens on the field," Not just head-to-head games.

Thanks.

Nature of wins isn't as important as head to head?

So when comparing two teams, their performance against a common opponent is more important than every other game they played? Texas beating Tech by 10 rather than 1 shouldn’t be more important than the fact that Houston beat a ranked team on the road. At this point in the season, in all fairness UH should be ranked ahead of UT because of their body of work.

These become meaningless as the season goes on

Back in 2003, I demonstrated on one of the other Texas boards that Wofford (for some reason, treated as a mock rival of Texas) would have beaten USC, LSU and Auburn under an “A beat B, B beat C therefore A would beat C” sort of methodology without taking too many steps (8-10) in each case.

No doubt...

But that doesn’t mean its not a helpful tool!

Again… ya’ll are missing my point. Head to head isn’t the ONLY factor but it should be considered over all others. Weighted more so to speak.

Just because you can give an example or two where it isn’t helpful doesn’t mean its not helpful in every scenario.

It's the last sentence you typed that convinced me

Usually, I find that when people use words like “absolute” and “every” they’re “always” correct.

Argh!!!!

Off-topic, but a personal pet peeve. It’s “y’all,” not “ya’ll”!

slang has a grammar nazi?
"Y'all" is a perfectly good English word!
It is a perfectly cromulent word.
My mind has been embiggened by this entire thread.
I'm fixin' to go look that one up
It's Texas colloquial. Slang is such a slut. nt
And by the end

of the year our resume will be stronger than UH so it all works out in the end.

The curve

Is Mike Leach’s whole schtick wearing thin on anybody else? Maybe next week’s curve should track how Leach went from being a refreshing novelty (with a high point the publication of the Michael Lewis article) to being rather annoying with his carping about bad calls (starting in 2007 after the Texas game), questionable decisions that alienate his team, etc. I guess when you associate with freaks like Carter and make them captains on your team, there’s bound to be a problem.

Same reaction

to the point that my first thought was that, if we’re talking about Leach and Brandon Carter specifically, they ought to reverse positions.

Eh.

Mike Leach is at least entertaining to me, even as he completely loses control of that team and his public image. Brandon Carter’s schtick wore thin the second I learned who Brandon Carter was.

On a scale of 1-10, how happy do you think the UW athletic department is right now?

I think Scipio over BC summed him up best:

Leach is like an elephant. I love elephants. They’re amazing. Fun to watch. They do things that make you laugh. They do things that are awe inducing. They’re smart, pecuilar, and capable of great mischief and mayhem.

All of that said….

I have no desire to OWN an elephant.
Leach seems to have become a compulsive gambler.

When he can’t seem to comprehend that insuring a win with field goal like at UH or even stopping OU’s big Mo last year, he is either arrogant or blind to the possibilities. Don’t know which. His arrogance could be honestly come by, with hm thinking there was plenty of time to get more scores against UH; unfortunately, his team was not as good as he thought. Or he perhaps feels that things will balance out in the long run (with fourth downs he is generally correct but in any given situation can be wrong, depending on the circumstances).

However, his comment “that even a blind man could see that it was helmet-to-helmet contact” wrt Kindle hit on Potts was a little cheap and oblivious and rendered in a dead-pan completely devoid of emotion, Just weird shit. I no longer find him entertaining, more of a serious head case, like Mackovic in the Brackens induced haze period. Not connecting.

Oh what the hell...

I’ll stick up for Mike Leach.

He’s taking a lot of flack for that 4th down call, but I agreed with it at the time. In retrospect, we can all say it likely cost Texas Tech the game, but the field goal would have put Tech up by 8, which is only one score, and Leach knew his defense wasn’t going to stop Houston on the final drive.

In his head, he sees that a field goal results in a likely overtime (lets say 50% likely in his estimation), while a 1-yd touchdown ends the game right then and there (a touchdown Tech is far likelier than 50% to convert).

I agree with his logic. The odds say he was correct. But even if his strategy were to pay off the majority of the time, then he has to live with the minority in which it doesn’t, which is ultimately what happened. But that doesn’t mean he was wrong.

You mean "wrong at the time" of the decision.

Good defense. I can understand most of his gambles…hell, every play is a gamble of sorts. The pros don’t make those bad decisions because in the long run they do lose; the pros virtually always choose to improve their position and give their opponent as little as possible. It’s not that Leach gambles, it is that he gambles too much, too often. He can usually cover, but when he can’t – like against UH – it really burns his ass.

I just think he is over the line at this point and his short term considerations are at odds with the long term ones. If he had any MNC considerations, even with the loss to Texas, he just threw them away. Maybe to him, it’s just a play and football is just a game. Nothing more or less.

Correct...

“wrong at the time,” strictly speaking.

I still think his strategy pays off in the long-term. If he makes that call in 10 similar situations, I bet he scores from the 1 yardline in 8 of them.

Perhaps this criticism is evidence that his program has fostered more respect of late and that the expectations are appropriately higher. Leach has been gambling at Tech for 10 years, usually winning, and occasionally losing. But now his program is more visible, and the gambles that don’t pay off more scrutinized. In other words, when you’re ranked #22 and people expect an 8-4 season, nobody notices these infrequent and statistically unlikely negative occurrences. But when you make an appearance (however brief) in the 10+ win club, and have outside championship potential, then the very things that you’ve been doing all along, the very things that got you this far, are going to be more vocally criticized when they don’t work out.

I didn't agree with the call

It’s a whole different scenario when the potential lead is 8 rather than 7 (as Houston subsequently showed by failing at the two point conversion). While on pure statistics, it may look like the right choice, but Leach also has to consider if he can really afford to fail. Even though 8 points is only one score, it is an insurmountable score; Houston cannot beat Tech on one drive and would need a 2 pointer to tie, which is not guaranteed.

That being said, I understand why he did it and I’m more critical of other decisions throughout the game. The use of timeouts and the horrendous clock management on Tech’s last drive was a pretty sad display.

That's a strong point about more scrutiny, BrooklynHorn.nt
At the end of the year...

…Dougie’s week-to-week ranking will look like work of a schizo in a mental asylum, while all those bad, bad, BAD AP voters will look practically Nostradamus-like by comparison. Yeah, follow the letter of the rules and look like a total retard. Way to defend the man, billyzane.

Dougie's poll will become more "coherent" in the future

when conference play starts and team’s resume’s start building. I don’t have any problem with his top 25 because at the beginning of the season you can’t really rank these teams adequately, so you either have to go by a combination of performance and preseason expectations (everybody), or just performance (Doug). In just looking at performance without any preseason expectations, the first few weeks of polls is going to be completely populated at the top by teams who have beaten good teams, and the actual “top teams” (Florida, Texas, etc.) are going to be punished (justly) for playing cream puffs. But as the season gets into conference play, and the mediocre teams start to lose, and the mid-majors are only playing weaker opponents, the two types of polls (performance vs. expectations) are going to slowly combine into one.

Not enough data

The problem with pure resume voiting this early on is that there is simple not enough data to make any meaningful conclusions. There is not enough overlap with common opponents, sparse matrix, etc. etc.

The other problem is that while resume voters don’t consider the reputation of the team in question, they seem to happily apply the reputation to their opponents to determine good and bad wins. If VT beats Miami, we call it a good win, but we really don’t know if Miami is any good and therefore don’t know if it was a good win and if VT should be ranked higher compared to a Iowa beating Penn State. There simply is not enough data for pure resume voting this early on.

I agree with everything you said.

And I should have made that more clear in the column. I certainly said as much in this column that i wrote last year and linked to above.

Polls shouldn’t exist until at least the midpoint of the season.

Another issue that would come from this

Would be the effect of OOC scheduling if teams were truly rated on resume.

As you say, and the logical conclusion is that rankings could not come out until enough data points are established, meaning, no preseason polls. Then to establish yourself in that first poll, do you want Div2 schools on your resume, or BCS teams? Hell even a “good” loss to a BCS team could look better than a drubbing of Idaho State.

If the players didnt automatically start at the top due to reputation, would they be more likely to at least schedule one marquee game to help their rankings? Probably so.

Then if there’s more inter-conference play, we could also be happier about the final BCS rankings with more crossing data.

Absolutely.

I covered that in the “Preseason Polls Hate America” post that I keep linking to. Maybe I should have just reproduced that piece. But then what would iamjackburton have done??? (my guess? called me a reTARD.)

To make that happen

Polls and fans have to be okay with lowering a team even if they win. The idea that if you are in a spot it is yours until you lose goes against the idea of resume rankings

Correct me if I'm wrong

If all pollsters used a pure resume-ranking throughout the entire season it shouldn’t matter where you started the polling. The idea to start the polling at a midpoint is basically to help the pollsters save face when they have unordinary looking ballots at the beginning of the season.

I don’t mind seeing a ballot like Lesmerises in week 3, because I know if he holds true to form and Texas remains undefeated they will slowly crawl up his poll. It’s pretty simple to me, criticizing an early season resume-ranked ballot is much like criticizing the look of a house in middle of construction. If people are patient, the end result will probably be what they envisioned at the beginning.

True

If you can trust pollsters to not be affected by the first poll.

Ideally you could, realistically, probably not. But at least delaying that first poll to get it more accurate would help lessen the effects.

Good in theory, impossible in practice
Polls shouldn’t exist until at least the midpoint of the season.

The Harris Poll demonstrates why this doesn’t work. This is the fifth year of the Harris Poll. The one unique characteristic of the Harris Poll is the delay, until several weeks into the season of the release of the first poll. This was designed to counter the problem of preseason biases being set into stone as the season progresses.

Yet, for the fifth straight year, the initial Harris Poll more or less mimics the AP and coaches’ poll, which of course more or less mimic each other. Conventional wisdom has already been established, and the Harris Poll voters demonstrate that they are aware of this conventional wisdom when they vote.

I don’t think this problem could be solved other than placing each Harris Poll voter in Family Feud-like isolation chambers.

Even if the initial release of the AP and coaches polls were delayed until mid-season, I think the problem would still exist since conventional wisdom has many other places to manifest itself in this day and age (ESPN power poll, SI power poll, etc.) To use this season as an example, if this week featured the first AP poll of the season, does anyone think it would look any different from how it looks now with preseason bias built in?

In other words, I think preseason bias is an unavoidable problem in a system which relies on polls.

Wouldn't fix everything.

But would be a step in the right direction and certainly wouldn’t hurt anything.

In fact, any pre-season poll is basically a racing form. And just as reliable.

If resume ranking became the real means, the early season polls would be hilarious, a lot of fun, where even a Troy that upsets an LSU could be Numero Uno, at least for a short period. No different than a horse race where a sprinter jumps in front for a while before fading into the pack.

However, I suspect the racing form mentality has a strong hold and will be hard to eradicate. A better strategy might be to formalize it as a separate entity and start anew with the season with resume ranking. Everyone at true zero.

As the blogosphere influenced the traditional media...

I wonder if the BCS formula, more specifically the computer portion, had any influence on the blogosphere.

Computer rankings were obviously around for some time before the introduction of the BCS but were somewhat marginally known and didn’t hold much credence. The BCS was introduced with them as a component and suddenly they were in the limelight and, perhaps, suggested the alternative to the commonly accepted “AP methodology”. It would amuse me to no end if the much maligned computer polls ended up influencing future polling.

That being said, I have no idea if anyone was blogging about resume ranking prior to the BCS. Can someone please check the History of the Internet?

Interesting point that I hadn't thought of.

It’s certainly plausible.

Had the same thought

the descriptions in this piece regarding resume ranking raise the quesiton of what really would then be the practical distinction between the human polls and the computer algorithms. To me, the human polls were kept in the equation to allow for a sort of emotional common sense to override strange results that a literal interpretation of data might produce.

You may argue that such a human role is vague and undefined, and I won’t argue against you. My point is that the humans were kept in the mix for SOME reason, and my question then becomes why continue to have both components if the humans begin to adopt similar behaviors and standards as the computer polls.

not much to add

Except really nice write-up, as always. You made my head hurt during lunch. Congratulations.

Here's my problem with Lesmerises

It has absolutely nothing to do with the fact that he thinks independently enough to submit ballots which look different from those of other voters. That is a Good Thing. My favorite example of independent thinking goes all the way back to 1992, when some idiot sportswriter, not from Alabama, kept voting the Crimson Tide #1 all season long despite the well-established fact that Miami was without question the best team in the country. Now, deviation from the conventional wisdom doesn’t justify bias (hello, Briles and Leach), but merely going against the flow on its own doesn’t merit condemnation.

I assume, though, that that 1992 sportswriter was not only looking at results on the field but also using his own subjective analysis, based on watching the teams with his own eyes and then applying his knowledge as a college football expert, to justify ranking Alabama ahead of Miami. And that is where my issue with Lesmerises arises.

My specific problem with Lesmerises is that he is mimicking a computer. Yes, the guidelines state “Base your vote on performance, not reputation or preseason speculation.” But there needs to be room for a human subjective analysis of that performance. If there is no room for a subjective analysis, what is Lesmerises doing that is any different from what a computer does when it takes a random assortment of games at the beginning of the season without taking into account anything else about the teams? And if not, why is there a human voter component at all?

Under Lesmerises’ methodology, there could be a three-time defending national champion returning all 22 starters, but if that teams starts the season against a couple of lightweights, and some Appalachian State-type team registers an impressive upset in week one, that ASU team would be ranked ahead of the three-time defending champs even though anyone with even the slightest bit of college football knowledge would know that the defending champs were the better team. I think it’s entire appropriate for a voter to look at the results and still rank the clearly-better team, from a subjective analysis, ahead of the team that got lucky once.

Id be all for ditching humans all together and going with computers

After all, humans can program the computers to consider all the factors they find most important, the computers will just be more effective at applying those factors without additional outside bias, which is a good thing.

I also see nothing at all wrong with your scenario in the final paragraph. Its not a power poll, its a resume poll. Or at least its supposed to be. The only clearly better team is the one that has proved it on the field. To quote a great NFL cliche, you are what you are. Would ‘05 USC really beat ’05 Texas 9 out of 10 times? Who cares, they didn’t, and thats all that matters.

The problem with going with just computers is that they have biases too

Not in terms of what they spit out but in terms of how they’re programmed.

For example, since “running up the score” is frowned upon, computers are not allowed to take into consideration the margin of victory. So a computer can’t distinguish an ugly, turnover-plagued 13-12 victory from the solid 27-0 win. Allowing humans into the equation helps correct this programming bias.

That's not the fault of computer polls

That’s the fault of human created rules to place on top of the computer polls that are used in the BCS.

Presumably, BiC’s all-computers idea would remove any and all restrictions from the computer polls, and allow them to be programmed as the programmers want. Thus, they could take margin of victory into account.

A few points without hopefully going too far into it…

1. No one computer algorithm to rank teams is perfect obviously. If it was, that would be all there is to it. In my perfect polling world, Id have every BCS school submit an independently created computer poll, and drop some 30-50% of outliers. You try to weed out the bias by dropping the extremes. If you have enough polls, and drop enough, you get a reasonably fair assessment of the teams imo.

2. I think its absurd that no margin of victory is to be accounted for, the reason its not is not for accuracy, but to discourage unsportsmanlike conduct. Since I’m projecting my way, Id let games within 21 points have an effect, but disallow anything over that for a similar effect, but allow for more accuracy in the ranking.

Good points, Meekrob and BiC

I think we can agree that the computer component, whether it is 33%, 100%, or somewhere in between, should be fixed to allow for many more computer polls programmed pretty much any way the programmers want to program them.

Preseason ballot

Wondering under what methodology Lesmerises fills out his preseason ballot.

Resume Ranking

I totally agree that resume ranking is the only rational way to rate teams. However, it must be clearly understood that the ranking concerns only the standing of teams at the time it is made. It does not predict the eventual champion. It can’t, because it only looks at what has already been accomplished. (And who cares who the best team was at the end of September?)

As I have said so many times already, the best solution is a selection committee — like they use in basketball. Have the committee decide, armed with all the data the computer systems generate, but ultimately rely on their expert judgment. Teams can try to sway the expert opinion by scheduling top opponents, posting high scores, winning conferences, or whatever. And when the rankings are made, there will be people able to explain how they were made.

I'm not sure a pure resume ranking is even possible, especially at this point

He’s just pushing back his preconceived notions from those of the team he’s ranking to those of the teams that team has played.

For instance, why should Iowa be number 5? Because they beat Penn State? But why is that impressive? Penn State hasn’t done anything this year to warrant being a big victory for anyone.

Agreed.

Similar point that drycreek made above. Holding out having a poll until midseason would actually give pollsters some basis on which to judge how impressive a victory actually is.

To add some fuel to the fire, here are the current Sagarin elo chess rankings:

These are what will contribute to the BCS rankings, if I understand them right:

1. Iowa
2. Boise State
3. Alabama
4. LSU
5. Virginia Tech
6. Oregon
7. Florida
8. TCU
9. USC
10. Miami
11. Georgia
12. Texas

Sample set isn’t large enough yet for these ratings to be independent of preseason rankings, but it shows you how computers may view the current state of things.

Discuss…

Easy. Sagarin hates children and puppies.
Best idea I've heard:

1. Before the season, obtain 1000s of different computer polls. That is, obtain a set N (with N very large, like in the 1000s) methodologies of ranking teams based on results of football games. How each poll performs its evaluation is immaterial, just so long as there isn’t anything like “Oregon State is ranked number 1 no matter what…” — there has to be some causality between real world events. Secondly, the metrics that each poll uses to determine ranking cannot change during the season.

2. Take the average of all of these computer polls to determine weekly ranking. Statistically, each of these polls is pretty highly correlated with the other, but overall you’ll get a reduction in noise by taking an average.

3. After each season, eliminate those polls that were outliers and replace them with new polls (which may be worse or better).

Rinse and repeat.

I don't understand the rush to throw out outliers.

We can’t punish people or computers for coming to an end result that is different from the general consensus. We should make sure that their methods are sound and only take into account things that happen on the field that season (which is why Richard Billingsley’s computer poll should be thrown out), but once that’s assured, results are results.

I suppose it depends on how you define outliers.

But I agree with you in that step 3 isn’t critical. In general, the way I would define an outlier is a computer poll that lies more than 3 standard deviations away from the mean rankings. This isn’t so provocative, and likely most seasons very few polls wouldn’t be thrown out. What am I basing that on? Pure conjecture.

Here’s one reason that I would want to remove outliers: because I want to use the mean of these polls to determine what we’ll call “consensus” #1 and #2. Clearly not all the computers will select the same #1 / #2, but its important not to have a situation where the distribution of rankings is so widely spread that the mean value for the #1 ranked team is totally diluted. You don’t want to select a #1 based on 20% of polls selecting #1. One way (and maybe not the best way) to reduce the uncertainty (if you want to call it that) in the rankings is to reduce the standard deviation by excising the outliers.

Is it a polling methodology issue?

I thought it was common practice when doing polls or other analysis similar to this to disregard outliers to come up with more consistent results. If that’s the case, there’s not really a football-centric reason why it’s done.

Here’s a hypothetical situation: 9 out of 10 computers have Team A .01 ahead of Team B. But the 10th computer has Team B 1.1 points better than Team A. Since 9 out of 10 agree, is it a better or worse result to let the 10th computer skew the average to such a degree?

I meant that the 10th computer has Team B .11 better

A fun thing to do would be to over time accumulate data concerning how good of a predictor each poll is. That is, if a poll ranks team A higher than team B, and it turns out that team A beats team B, then the poll was a good predictor in that instance. By accumulating data over several years you can ostensibly get a good handle on how good of a predictor the poll is. Then, when you’re doing your average, do a weighted average, giving more weight to those polls that are historically good predictors.

So in the example you provided, the 10th computer may have a big impact on the mean or not, depending on how well it historically predicted outcomes in the past.

The problem is that polls shouldn't be about predicting future results.

That’s completely counter to resume ranking. If that was the case, I’d put Florida 1 and Texas 2 and forget about it.

I would argue

That under a good model the two are one and the same. A good computer model draw information about future success from the resume of a team. The problem with humans is that we tend to draw information about future success from ethereal intangibles in addition to resume.

I understand your distinction, but still think I disagree.

I think polls should be about ranking who has had the best season, period. I think the resume is the end result, not a means to get to the end result.

I see your point.

The really interesting question is what people think a “Championship Game” ought to be. Its why I find statements like Matt Leinart’s post ‘06 Rose Bowl a little odd, when he said that he thought that they were still the better team. I don’t think there is any reliable way to determine who the “best” team is — we wouldn’t even know how to begin describing what that means. What we do have is a way to determine who wins a particular game — we play the games.

That was me agreeing with you, by the way.
Wow

This thread exploded in the brief time I was in class. I feel left out now.

First, great write-up, as always, billyzane. I look forward to Flex System 2.0.

I will say this about resume ranking. As I noted in a comment here, I think there will always be a speculative element in the polls. Consider:

-The very nature of evaluating losses and wins on the field is fraught with qualitative, and debatable, analysis. Is barely beating Texas Tech at home better than thrashing Kentucky on the road? If Houston played Tech in Lubbock, how likely would a loss be? If Florida played Kentucky at home, is the result any different? Furthermore, what reasons were there for a loss or win? Do we take into account Sam Bradford getting knocked out? What if Florida barely wins against LSU or even loses and Tim Tebow was held out of the game. How much do we take the result “on the field?”

-Some will argue that a team’s “resume” is also the talent level they put on the field. With so many different teams, we can argue that it is impossible to avoid projecting who would win if we are arguing between a few teams. If Boise State goes undefeated and Florida ends up with one loss, not only would “strength of schedule,” which is debatable in itself, most likely keep the Gators ahead, so would an honest assessment of the quality of players Florida possesses compared to Boise.

-Related to the first two points, upsets do happen and better teams on a bad day can lose to inferior opponents. Honestly, I did not think Washington should have been even ranked in the Top 25 after defeating USC, much less be ranked ahead of the Trojans as some argued. After all, both Taylor Mays and Matt Barkley (who, if you haven’t heard, is exactly like Joe Montana) did not play, among others. Furthermore, when you watch the game, USC shot themselves in the foot numerous times, and the Trojans clearly have a massive edge as far as quality players. It’s a game USC simply blew and had key injuries in, which is not unlike what we’ve seen in previous years (I’ve never heard a soul argue that Stanford was better than USC when they beat them at the Coliseum).

-If a team beats a highly ranked team, how much credit is given if that team goes on and tanks? Example: Alabama whips #9 Clemson last year and the Tigers fall off the map. Do we credit Alabama and explain Clemson’s drop as an aftermath of the beating they took? Was Clemson simply overrated? On the flip side, if a team kills an unranked team early who ends up having big wins late in the season to vault to the Top 15, did they really beat top 15 team? Or did that team suck before but simply show a lot more improvement than others over the course of the season?

I’ll stop now. I agree wholeheartedly that preseason rankings and voter inertia are out of hand. However, even if we could eliminate those, we would still have the problem of speculative projection and qualitative analysis of everything that happened on the field. That makes it difficult for even knowledgeable people, much less the ignorant voters of the Harris poll and the careless coaches who vote in the coaches poll. Thus, while I appreciate Lesmerises’ willingness to break from the stupidity of most AP voters, I still don’t think he did a very good job with his own ballot.

All the points are good.

Resume ranking isn’t just about wins and losses, it’s about how you played in those wins and losses and the circumstances surrounding them. Road/home matters; blowouts/close games matter; etc.

And, if you hold off on ranking teams until later in the season, then you actually have a basis on which to value certain wins and losses of each team.

You point about Alabama and Clemson is exactly the problem...

We can throw accolades or firebombs at this AP voter all we want, but the problem is that even if he strictly followed what he thought were the AP’s guidelines that is still a biased judgment, especially early in the year. How can Mr. Lesmerises be so certain that beating OSU and Tech are that impressive after just a few games? How much credit is he giving Houston for playing two teams after they just finished playing a high profile opponent?

He may be savvy enough to work out the kinks and end up with a vote similar to the majority of voters at the end of the year, but he could also get trapped into other biases that plague voters, such as winning means you can’t drop. So if Houston or Boise State wins out how will Mr. Lesmerises decide at what point they should drop and to whom should they make way for as punishment for playing their patsy conference slate? That is critical and it is ultimately a problem that most AP voters probably can’t reconcile which is why the work on the power poll model.

If you're going to hold off for half of the season to have polls

Why not just hold off until after the season is over and then conduct a poll, which will decide the championship game? Half way through the season you have better data than after 3 weeks, but still not enough data to really resume rank.

But then how would ESPN advertise their games?

I think the polls are more fodder for bloggers and advertising for television until the last week of the season.

Exactly

There’s a reason the AP conducts a poll in the first place… to sell papers / justify advertising income. But let’s not forget that advertising dollars are based on demand. I think the college football fandom in general would lose their shit if they weren’t told their team is X number of spots better than the next big team on their schedule.

Whatever system it is that determines a national champion will ultimately be as much concerned with “getting it right” as it is with satisfying the general populace. If the determining poll(s) weren’t released until the end of the season, I think there’d be a much bigger dissatisfaction with the results. Homers nationwide need time to adjust to the fact that others don’t see their team in the same light that they do.

I've considered that as well

but I see some potential problems with it. Most notably, if pollsters have only once to put their voice out, then you have neither an evolution to their thought processes, nor any check against their potential corruption.

Toward the first point, I think its good for people to evaluate on a weekly basis, hone their evaluative process, look at their mistakes, and refine their ballot the following week. Its the idea of a rough draft, if you will.

Second, if voters put forth only one final ballot, the likelihood that someone politically places teams in a biased order is too high. You’d have something akin to the Oscars, in which voters are not scrutinized throughout the season, but rather make one statement, and then can just bail out. In my personal estimation, the AP poll is FAR more accurate/meaningful than is the Academy. Although I will concede the difference is more in degree, probably, than principle.

Texas Would Have Overtaken Both UF and OU Last Season If Not For BCS Computer SOS Flaw

According to the rules the computer rankings use, the results of games against lower division schools are not counted. So, when Florida beat the Citadel (4-8) last season and Oklahoma beat Tennessee-Chattanooga (1-11) those games did not help or hurt the Strength-of-Schedule component in the computers.

But, when Texas beat UTEP (4-8) last season that result counted against UT’s SoS component. With this rule in place amazingly there is an incentive to play FCS teams over low-rated FBS teams.

I know, I know, voters are supposedly taking into account the weak scheduling, but in conferences with strong weekly opponents like the SEC and Big 12 teams play, having any FCS team scheduled makes much more sense than a terrible FBS team – if your goal is to get an edge in the SoS component of the BCS system.

And as we learned last year, every little edge you can give for yourself and your team to be ranked 1-2 in the final BCS poll you should take. From running up the score on any team you can to scheduling any FCS team over the worst FBS team you would otherwise have faced makes the difference between National Championship teams and teams that are gypped.

Let's have 1000 computers pick the best AP poll submissions

The computers come up with a list of the top 10 AP pollsters, then those pollsters will be seeded in a tournament bracket against eachother, victories being decided by hand to hand combat in an octagon coated with nacho cheese sauce.

The winner’s AP poll represents the poll for the AP that week, and if any of that winner’s top 10 picks lose the next week to an unranked team, then they should be apprehended and branded with the mascot of said unranked team while that team’s fight song plays. Fans of that team should be invited to attend and throw rotten garbage and hold up home-made signs of ridicule.

Once those ceremonies are finished, TV voters would text in their favorite pollsters, with the bottom 10 pollsters recieving votes being removed from the polls alltogether.

It’s really the only system that makes sense.

Epic thread

I guess this is what happens on an open week.

this probably fits in somewhere above...

but like a coaches’ poll voter, I’m too lazy or unmotivated to fit it in precisely where it makes the most sense…

The big issue with Lesmerises’ ballot, and with all resume ballots early on, is that they come as such a culture shock. They absolutely do not fit in with the “Narrative” of each season. And they’re therefore very uncomfortable for both pundits and fans – witness the uproar over some heretofore anonymous Ohio voter. As some have noted above, the “resume” ballot will start to more closely resemble the standard “no criteria whatsoever” ballot as the season continues and the narrative begins to reflect actual data.

But to get to the point where everybody kind of settles on the resume as the preferred measuring stick, the CFB world is going to have to grow comfortable with the increased volatility of that method, especially early in the season. Which would be a good thing; if voters had no great fear of drastically changing their ballots week to week, they’d certainly be no less likely to come up with better results. And there’d at least be some semblance of intellectual consistency, rather than the current hodge-podge of some voters building a predictive/power poll, some basing theirs more on actual results, and most just spicing the two up with some bias and mixing up into some unholy stew…

You must Login with your SB Nation account and be a member of Burnt Orange Nation to post a comment.