While I believe that Johan Santana and Mariano Rivera were clearly the two best pitchers in the league that year, I'm not going to argue that one of them should have won the award. I recognize that each of them faced a significant challenge to winning the award, challenges that the BBWAA rarely works hard enough to overlook. In Santana's case, his team's general mediocrity suppressed his win total, and in Rivera's case, his role as a reliever kept him from throwing very many innings. I could go chapter and verse on why either man was a better pitcher in 2005 than anyone else in the league, but that's been done before by many, and I'm not in the mood to do it again.
Instead, I want to have a discussion of how the BBWAA evaluates candidates for the Cy Young in general, and I think I've identified the perfect candidates for framing this discussion.
Bartolo Colon was the winner of the American League Cy Young Award in 2005, receiving 17 of the 28 possible first-place votes. This award was based upon Colon's 21-8 record and 3.48 ERA for the AL West champion Los Angeles Angels of Anaheim. His teammate, John Lackey, posted a record of 14-5 and a similar ERA of 3.44. He didn't receive a single Cy Young vote, but when his performance for the season is compared to Colon's, I think the similarities are noteworthy.
That said, there are quite a few factors in Lackey's favor that narrow the gap between them significantly:
When all of these factors are combined, and the respective records of the two pitchers are neutralized (according to baseball-reference.com) to account for home-road discrepancies, run support, etc., we find that their marks aren't terribly different:
Colon: 14-10, 3.33 ERA, 224.3 innings, 158 strikeouts, 42 walks
Lackey: 14-9, 3.21 ERA, 210.3 innings, 200 strikeouts, 70 walks
In other words, Lackey's neutralized record is very close to the record he actual posted, while Colon's makes it clear that his final numbers were largely the result of fortunate circumstances beyond his control.
Now, even presuming that Colon should get an extra couple of wins due to durability issues, the question becomes this; Why did Bartolo Colon win the Cy Young Award in 2005 when a teammate with an extremely similar performance didn't receive a single vote? Was Colon so much better than Lackey that he deserved to be labeled the best pitcher in the league while Lackey received no consideration for that title at all? I don't think so.
I think the BBWAA saw that league-leading win total and stopped asking questions. Colon won more games than anyone and he did it for a division winner, so the question of whether or not his performance was really indicative of the best pitching in the league became an open and shut case for most voters. At the same time, Lackey's comparatively pedestrian win total, particularly in the absence of a league leading ERA or strikeout total, eliminated him from consideration for most, if not all, of the writers before they even filled out their ballots.
This is pretty typical for the BBWAA. Time and again, we've seen them focus on one or two key statistics - wins for starters and saves for relievers in the Cy Young voting, RBI for hitters in the MVP voting - and essentially halt their performance analysis at that point. I don't agree with it, but I've come to expect it.
Still, when you've got a situation like Colon and Lackey in 2005, you'd hope the voters would take a bit more time to examine just a couple of basics. If any one of them had bothered to look at them side-by-side, they would have seen two teammates with nearly identical ERAs who started the same number of games for the same team, and the team posted identical 22-11 records in their respective starts. On the field, where all of this is supposed to matter, John Lackey and Bartolo Colon provided essentially equal value to the Angels, with Colon providing a touch more due to his added durability. Wouldn't the voters, if they were really doing their jobs, takes pains to ensure that the voting results reflected this reality? If you really think Colon was the best pitcher in the league, shouldn't Lackey be somewhere down your ballot, probably just one slot below Colon?
Instead, we get the writers sending the tacit message that they really only consider one or two numbers when they cast their votes, whether those are the proper numbers to consider or not. Silly results ensue, leaving anyone who cares about this stuff either scratching their head or, even worse, expressing no surprise at all considering how frequently this kind of thing happens.
For me, I can't help but think that it's a pretty sad commentary on the voting process when it's no longer surprising that the results of the writers' ballots don't match the reality we witness on the field.