‘How do you have Eli Manning scrambling for his life and throw that ball in the Super Bowl? … How do you capture that kind of playmaking in a test?’ Cyrus Mehri, a lawyer who leads the Fritz Pollard Alliance, says of the New York Giants quarterback’s clutch throw in the 2008 title game.Julie Jacobson/The Associated Press
For decades, hundreds of college players have gathered each year at the NFL's scouting combine, where their strength is tested, their speed is timed and, in an effort to measure their intelligence, they are asked questions like: "When a rope is selling 20 cents per 2 feet, how many feet can you buy for 30 dollars?"
That query is part of the Wonderlic Personnel Test, a 12-minute, 50-item quiz that has been used by NFL teams since the 1970s. It is, however, famously unreliable in predicting football success – forgettable players have scored high, stars low – and there have been quiet concerns that its reliance on knowledge taught in school might result in a racial bias.
So the players at this week's combine here are facing a new segment in their extended job interviews: an hour-long psychological assessment designed to determine and quantify the nebulous qualities coaches have long believed make the most successful players – motivation, competitiveness, passion and mental toughness – and to divine how each player learns best.
The new test, like the Wonderlic, is mandatory for the more than 300 players who attend and it was given for the first time last Thursday.
While the Wonderlic is thought by coaches and general managers to be most useful in evaluating quarterbacks and offensive linemen, positions that are believed to demand the greatest intellect because of the need to decipher complex defences, the hope is the new test, called the Player Assessment Tool, will give teams clearer insight into a broader range of players.
"I knew players who didn't score well on the Wonderlic, but had great instincts," said Ernie Accorsi, the former New York Giants general manager who was consulted during the creation of the new test. "I had a player once, this guy played in a good league in college, but the psychological testing indicated he didn't handle pressure well. You know what? He didn't, as it turned out. The Wonderlic can't tell you that."
The new test was devised by Harold Goldstein, a professor of industrial/organizational psychology at Baruch College in New York. He worked with Cyrus Mehri, a lawyer in Washington who leads the Fritz Pollard Alliance, which monitors the NFL's minority hiring practices.
Personality tests have been a staple in other industries and some NFL teams have used them during their own scouting efforts, which often take months. But last fall, Goldstein and Mehri began the process of producing the first such test for the entire league.
They asked a group of GMs what qualities they wanted in a player. They came up with 16 aspects thought to be predictors of NFL success, including learning agility and conscientiousness. The test most closely mirrors those given to firefighters, Mehri said, because they, like football players, have to be able to quickly assess a situation and make a decision about how to proceed under stress.
The goal was to eliminate the impact of prior knowledge – subjects that are taught in school, such as math, where racial and socioeconomic factors may have an influence. To determine their personalities, players are asked a series of questions about their preferences and behaviour. To evaluate their cognitive abilities, they might be told to look at four different diagrams and figure out how they relate. Then, to measure how quickly they are able to adjust their thinking, the items they are comparing might change, forcing the players to determine their relationships anew. And to see how they learn best, questions are presented in verbal and graphic form.
"How do you have Eli Manning scrambling for his life and throw that ball in the Super Bowl?" Mehri said, referring to the Giants quarterback's clutch throw in the 2008 title game. "Aptitude tests suggest to me you're testing how smart you are. It's so much more than that."
He added: "How do you capture that kind of playmaking in a test? You can't figure that out the way the combine is now. How you handle pressure, your mental toughness. At least this can be a window into it."
It is not easy designing a test that accurately predicts if a cornerback will be able to cover a wide receiver, or if a quarterback can learn how to quickly find the hole in a defence with a pass rusher in his face.
"These tests will never capture or perfectly predict performance; they are always limited," said Karen Blackmon, a clinical psychologist and research scientist at New York University. "Things like motivation are more difficult to capture than what we understand, like verbal and nonverbal learning. Motivation is nebulous and it fluctuates over time. How do you test for mental toughness? I don't really know."
The league did not allow anyone to see the test before it was administered, angering some agents. The NFL's goal was to minimize the kind of preparation players do for the Wonderlic in an attempt to boost their scores.
"This is the Super Bowl of their college career, the culmination of everything they have worked for," agent David Canter said. "You don't want them to be prepared for it?"
The plan is to give the results to teams in March, less than two months before the NFL draft in late April. So the information they get about players will not influence their own interviews, individual workouts and film study. Coaches have indicated they are most intrigued by the information about learning styles.
After the draft, teams will be asked for feedback about how useful the test was in the scheme of the vast information collected about players before the draft. The plan is to further hone it for coming years.
Still, the test's ability to predict whether a player is the next Peyton Manning or the next Ryan Leaf probably will not be known for at least a few years, after this class of players has had time to start their careers in a sport in which determination might have at least as much to do with success as a 40-yard dash.
"That's the unlocked mystery," Accorsi said. "But in our game, more than any other because of the physical nature, there's a key you try to unlock."