Alabama head coach Nick Saban works with Alabama defensive back Marlon Humphrey (29) works through drills during Alabama's sixth preseason football practice of the 2014 season on Aug. 5 at the Thomas-Drew Practice Facility in Tuscaloosa, Ala.VASHA HUNT/The Associated Press
Might as well call it Mission: Impossible.
Starting in mid-October, a panel of 13 football experts will begin meeting once a week to determine the four teams who will compete for the national championship in the very first College Football Playoff. Their goal is to choose the four best teams in the nation at the end of the regular season.
But how does someone answer a question for which there will almost always will be more than one correct answer?
"We need both faith and reason," said committee member and West Virginia athletic director Oliver Luck. "We have to use our logical skills and the deductive skills that the people in that room have. But the college football world needs a little bit of faith that we all check our pasts at the door and we go about this in a very straightforward and honest manner."
Where the College Football Playoff has the Bowl Championship Series beat is simply the number four. Twice as many teams will now enter the post-season with a chance to win the national championship. Most fans agree this is progress over an old system were deserving teams were left out.
How will the new panel decide on top teams? Will so-called great teams still be left out? As its first season unfolds, the group could create an all-new set of questions about whether the process works.
THE HUMAN ELEMENT
The biggest change in the process is that people — not computer rankings — are solely in charge of teams' fates. So the panel will have to learn how to effectively debate, compromise and even account for bias.
"This small number of human beings sitting across the table from each other can evaluate nuances to a much greater degree than the old system could," said Bill Hancock, the executive director of the College Football Playoff. "There can be give and take. There can be questions and answers. That element just wasn't a part of the old system."
The down side of that is with so few people, if there is bias in the room it can have a greater impact on the final decision. If someone consistently overrates the importance of offence over defence, for example, that can affect the process more so than when the teams were chosen by more than 100 voters in Harris and coaches' polls.
"With 13 people, if somebody really has it in their head that no matter what South Carolina is not that good, if one person votes them No. 15 even if everybody else has them around the top four, they're probably not going to get in," said Bill Connelly, a writer for SB Nation whose statistical rating system called F+ can be found at www.footballoutsiders.com .
On Oct. 21, the committee will release its first weekly rankings, a top 25 that will allow fans to see what the panel is collectively thinking down the stretch of the season.
Hancock has said the weekly rankings were a nod to college football tradition, and transparency. The ranking give fans a way to assess to some degree where teams stand in the championship race and they shine light on the process.
Maybe a little too much light?
"The obsession with transparency drives me crazy," Connelly said. "I get the idea that it's good to know how somebody came up with what they come up with, but they have gone overboard. It's going to make it a far more painful process for them."
Dave Bartoo, a data analyst, consultant and founder of www.cfbmatrix.com , said he anticipates the selection committee will rank teams similarly to other poll panels. Traditionally, college football poll voters tend to create tiers based on the number of loses and wins and adjust the teams within those tiers, making some alterations to account for perceived schedule strength.
"I feel that's what should happen because they (the committee) don't have enough information to do any better than the people who have been doing it," Bartoo said.
STRENGTH OF SCHEDULE
The committee has been instructed to place an emphasis on strength of schedule, head-to-head matchups and conference championships.
Strength of schedule has become a particular buzz phrase this off-season, with coaches and conferences already touting the virtues of their lineup.
Bartoo said it's misguided to emphasize strength of schedule because calculating it accurately is not possible, no matter how it's done.
"A lot of people want to believe that strength of schedule is a valid stat for separating teams," he said. "It's not the best one or the worst one. They are all about a coin flip."
There are not enough data points in college football to make a good sample size to study strength of schedule because teams play so few games and the best teams rarely play each other, Bartoo said.
ANALYZING THE DATA
The committee will have a virtual bank vault full of data at their fingertips, provided by a company called SportsSource Analytics, to help them judge teams.
Bartoo is concerned about information overload and the committee using numbers that have not been properly vetted.
"Who is helping with oversight on that?" he said. "Can the playoff committee call me? 'We think this metric is valuable, can you back test it for that?"'
The BCS taught college football fans there is no perfect way of picking the best teams. That won't change with the College Football Playoff.
"They will do the best they can and I'm confident they'll pick the best four teams," Hancock said. "Some other group of 13 people might come up with different teams, but our group will spend a season evaluating teams unlike any other group with fall. More time watching video and analyzing data than any other group.
"But I certainly know that another group of 13 might come up with something different."