PSYCHOLOGY is drowning in a sea of meaningless statistics, warns a
committee convened by the American Psychological Association (APA). It wants
psychologists to abandon the fancy statistics that have become the vogue in
recent years. Many researchers don鈥檛 understand the methods they are using, the
committee warns.
Statistical software packages now allow researchers to use tests that would
be impractical to compute by hand. But many have been lulled into an uncritical
acceptance, says Robert Rosenthal of Harvard University, co-chair of the APA鈥檚
Task Force on Statistical Inference in Psychological Research. Many use the
tests wrongly and some fail even to check that they have entered their data
correctly.
In a new report, the task force urges psychologists to return to simpler
statistical tests. Errors would then be much easier to spot. 鈥淧eople should not
simply take a bunch of data, stuff it into a computer and publish whatever comes
out the other end,鈥� says Rosenthal.
Advertisement
Some psychologists go even further. One group wants the APA to ban even
simpler versions of the most common type of statistical test, called the
null-hypothesis test. A null-hypothesis test starts by assuming that two groups
have a characteristic which is identical鈥攆or example, that men and women
have the same average IQ鈥攁nd calculates the probability that an apparent
difference between the groups revealed in experimental data could have arisen by
chance. If this probability turns out to be less than an arbitrarily chosen
level鈥攆or instance, one in 20鈥攔esearchers label their result
鈥渟tatistically significant鈥� and conclude that the groups are different.
John Hunter, a psychologist at Michigan State University, claims that up to
60 per cent of psychologists use null-hypothesis tests wrongly. The tests
are only valid if the initial assumption of no difference between groups is
almost always true. But that is often not the case, as scientists tend to pick
groups that are likely to be different.
The APA task force argues that banning particular types of test would be
wrong. But Rosenthal says that psychologists must become more discerning about
using statistics of all kinds.