Success in the NFL and other sports consists of four core factors:
- physical prowess (e.g. Julius Peppers‘ power, Calvin Johnson’s quickness),
- technique (Drew Brees’ throwing precision),
- attitudinal disposition (Ray Lewis’ competitive intensity), and
- cognitive functioning (e.g. Aaron Rodgers‘ offensive vision and Ed Reed‘s defensive vision).
At EXACT we often get asked about the merits of the “Wonderlic” in measuring football IQ. As experts in modeling & predicting human athletic performance, we have extensive insight about ‘mental’ evaluations. EXACT’s own psychological/neurocognitive toolkit is relied on by over 60 professional sports teams including the Minnesota Twins and the Pittsburgh Penguins. Every prospect entering the NHL draft has undergone EXACT’s behavioral and cognitive screening. US National teams rely on our tools to assist coaches in training decisions and over 100 colleges rely on our expertise to support on and off-field success. EXACT’s Mental Achievement Program (MAP) is the most widely used sports psych tool in the United States.
The Wonderlic Cognitive Ability Test (“Wonderlic”) is a questionnaire known throughout sports as the test that measures intelligence and the cognitive prowess of football players. It was one of the first tools introduced in the sports community and has become a common metric for prospective National Football League (“NFL”) players.
Every year, the Wonderlic draws attention for both high and low score results. Recently, Morris Claiborne’s score was put under the microscope. In year’s past, players such as Hakeem Nicks, Sebastian Janikowski, Vince Young & Jeff George have taken heat for their unimpressive (low) results. Good results get gawked at as well. Those of Ryan Fitzpatrick, Darrell Hackney, Charlie Frye, and Omar Jacobs are discussed frequently in the blogosphere. The big question that GMs, scouts, coaches, players and even fans often ask is, “Does the Wonderlic matter?“.
Before I evaluate the merit and flaws of this instrument, a little history.
Background
The Wonderlic was developed in the 1930s by E.F. Wonderlic, a Ph.D. candidate at Northwestern University. By the late 1930s, Wonderlic had marketed and sold his instrument as a pre-employment screening survey.1 The Wonderlic was first used in the NFL in the early 1970s. Dallas Cowboys head coach Tom Landry believed that NFL players required both cognitive skills and a desire to learn to succeed in the game. He required testing of all Dallas prospects. By the end of the decade, the NFL agreed with Landry’s approach and began the annual ritual of Wonderlic testing.
Good for General Employment Screening
Where does the Wonderlic succeed? The Wonderlic has shown utility as a measure of intelligence and has demonstrated good psychometric properties. Research conducted by C. B. Dodrill has shown that Wonderlic-rendered IQ scores closely resemble the WAIS Full Scale IQ (FSIQ).3 The two tests were similar in terms of reliability of clinical classification (both were good), but the Wonderlic demonstrated fewer practice effects than the WAIS.4 The Wonderlic has been successfully relied upon in industries that require math and literacy skills.
Bad for Athletics
The Wonderlic was designed for corporate use — not for athletes. It completely fails to provide value in the measurement of an athlete’s cognitive ability. There are 3 primary concerns that have been observed in its use by the National Football League:
1) The Wonderlic is not used by NFL teams in their decision-making process.
2) The Wonderlic provides a “stereotype threat” to players.
3) The Wonderlic results are unrelated to NFL performance.
Unused by NFL Teams
While each team receives the results, NFL franchises do not select “smarter” quarterbacks sooner or compensate them better than their peers.5 Other measured factors like level of past competition (as observed through scouting), and 40 yard dash time (as measured at the combine & pro days) do contribute to draft order. Prospects seem to agree as 30.8% said the Wonderlic is the most usless combine test. “It’s a brainteaser,” shared one pass-rusher with ESPN. “And I don’t remember a time that a brainteaser helped me sack the quarterback.”
It is a waste of player time & NFL resources to administer a measurement protocol that has no impact on the selection process.
“Stereotype Threat” Apparent Among Players
Stereotype threat reflects the behavioral effects that result from an individual’s fear that his actions will confirm a negative stereotype of a group to which he belongs.6 Stereotype threat can manifest in anxiety, which can impair performance and trigger a self-fulfilling prophecy. The stress related to his or her group membership causes the individual to underperform, thereby unintentionally corroborating the underlying group stereotype.
This topic has been researched by social psychologist, Claude Steele and is considered an issue during standardized testing. It is possible that groups that are predicted to underperform (e.g. African American players) face an unfair disadvantage due the stereotype threat.
Unrelated to Football Intelligence
The most important issue that the Wonderlic faces is that the instrument does not have relevance to the decision making required for football. General Intelligence (often referred to as ‘g’) encompasses 10 broad abilities including fluid intelligence, crystallized intelligence, quantitative reasoning, reading & writing ability, short-term memory, long-term storage and retrieval, visual processing, auditory processing, processing speed, and decision/reaction time/speed.
Below are two example Wonderlic questions:
1) When a rope is selling 20 cents per 2 feet, how many feet can you buy for for 30 dollars?
2) Which of the numbers in this group represents the smallest amount? a) 0.3 b) 0.08 c) 1 d) 0.33
As both of these questions demonstrate, the Wonderlic is measuring crystallized intelligence (i.e. the person’s knowledge of currency, mathematics, and language), quantitative reasoning, as well as reading & writing ability. These components of intelligence, while relevant to many occupations, have very low importance to football.
Research has consistently confirmed that the Wonderlic does not predict for success in the NFL. Three different studies examined factors of “success” — salary, number of games, and positional game results (e.g yards per carry) — and results showed there is no correlation.7 McDonald Mirabile’s research also confirmed the lack of relevance in a study that looked at intelligence and college performance. His results revealed no statistically significant relationship between “intelligence” and collegiate passing performance. Likewise, there is no evidence of higher compensation in the NFL for players with higher Wonderlic scores.
Summary: There is a Better Approach
The Wonderlic provides no utility to the NFL, to the teams, or to its players. The NFL must either stop cognitive testing completely or do cognitive assessment the right way. In ‘scrapping’ the Wonderlic, the NFL will save resources and allow decision-makers (the teams) to focus on useful, relevant information. The optimal solution is to use appropriate tools to measure football ‘IQ’. EXACT Sports’ own neurocognitive assessment, the Assessment of Mental Performance (AMP) was designed to measure athlete’s decision-making accuracy, processing speed and visuospatial awareness. These components are building blocks for successful NFL players (as well as other sports) and are minimally related to an athlete’s non-sports intelligence (ie. the assessment is written at the elementary school level to avoid education biases). EXACT’s AMP has been successfully used by the National Hockey League since 2007 and has been administered to thousands of athletes in sports such as baseball, hockey, soccer, tennis and football.
If you agree or disagree about the merits of the Wonderlic, let us know your comments.
——————————————
1 Tom Silverstein, Wonderlic Personnel Test Raises Eyebrow, MIL. J. SENT., Apr. 18, 2001
6 Claude M. Steele & Joshua Aronson, Stereotype Threat and the Intellectual Test Performance of African Americans, 69 J. PERSONALITY & SOC. PSYCHOL. 797 (1995);
7 Lyons, Hoffman, and Michel (Human Performance, 2009), Kuzmits and Adams (Journal of Strength and Conditioning Research, 2008), Berri and Schmidt (2009).
1
