Just another site

Monthly Archives: December 2011

There’s more to scientific judgement than statistical significance

Interesting piece in The Guardian by Philip Ball in the Saturday Critical Scientist slot (replacing Ben Goldacre, and still worth reading):

He’s discussing the value of statistical analysis of the results in the Higgs Boson and ‘faster-than-light neutrinos’ studies.

In any experiment, all sorts of complications can influence results. So if you see something interesting, you need to make sure it’s not just a random fluctuation. That depends on how widely spread out your results are: the bigger the fluctuations, the more you’re apt to be misled by them. The spread is measured by a quantity called sigma. The bigger your “interesting” signal is relative to sigma, the more “statistically significant” it is: the more likely it is worth heeding.

In psychology, we use p to express the likelihood of a result occurring by chance, rather than sigma, the number of standard deviations from the mean of a chance distribution, but the basic principle is the same.

…these statistics don’t put numbers on the probability of a particular hypothesis being right or wrong, because experiments don’t care a hoot about your hypothesis. They just show the universe doing its thing.
And to interpret what the universe just did requires that we take into account what we know already: as evidence changes, so do the degrees of belief we may hold in a theory. This is commonly called Bayesian reasoning, after the 18th-century mathematician Thomas Bayes.

Ball’s argument is that he’s pretty well prepared to accept Higgs Boson results with low statistical significance, but even high levels of unlikeliness and statistical significance won’t be very convincing for the ‘faster than light’ results. The one result is in line with what we know about the universe: the other isn’t. As he says:

You could put it crudely this way: the real question about the faster-than-light neutrinos experiment is not “what is the chance it disproves relativity?” but “what is the chance that it disproves relativity given that your GPS system (which relies on relativity) works?”

What’s that got to do with psychology and Schools of Thought? Well, it fits with the fact that many well-known effects in social psychology are demonstrated by a small number of classic experiments with rather low levels of statistical significance, and could fit in the category of comfortable myths – but they fit with other stuff we know, and probably with our non-scientific expectations as well. So it’s not that unreasonable to accept that fairly low-grade evidence. On the other hand, as I said in the lecture about ‘unacceptable ideas’, there are bodies of research with some pretty impressive reported significance levels which I’m not going to believe in, whatever the stats: telekinesis and precognition, for instance. I presented my beliefs there as being in some way unscientific, but Ball (and Bayes) show how there’s scientific sense as well as prejudice in my judgement.

Relying too much on statistical significance is complicated because very unlikely things do happen by chance all the time. There’s line in a Paul Simon song about that. After all, 14+ million to one is pretty low odds, but a 14m: 1 chance comes off most weeks, when someone matches the lottery numbers and wins the jackpot. If people buy 20m+ tickets each week, that’s not surprising. Just knowing that people do win doesn’t make it any more likely that you will, but the fact that it seems to be vanishingly unlikely doesn’t make it any less real for those who do win.

So I’m with Ball when he says:

Which is why I’m only being scientific when I say screw the sigmas: I’d place a tenner (but not a ton) on the Higgs, while offering to join Jim Al-Khalili in eating my shorts if neutrinos defy relativity.


Abuse and changes in adolescents’ and children’s brains

Two studies reported recently on changes in the brains of adolescents and children who have suffered abuse. Despite my prejudice against ‘we’ve found some kind of brain activity, so that explains everything’ research, this does look interesting, and maybe meaningful.
First ‘past abuse leads to loss of gray matter in the brains of adolescents’, reported in both Medical News Today: and PsyPost: (you probably don’t need both links: they say very much the same things, being lifted from the same Yale University press release). The study was on ‘forty-two adolescents without psychiatric diagnoses’. Hilary Blumberg, one of the authors, has published quite a bit on brain changes in people with bipolar disorder (and so is looking for Szasz’ ‘bad brains’: for all the criticism there is of strictly medical models of mental illness, it’s quite possible that some problems do have physical origins or physical accompaniments).

The brain areas impacted by maltreatment may differ between boys and girls, may depend on whether the youths had been exposed to abuse or neglect, and may be linked to whether the neglect was physical or emotional.
[…]The reduction of gray matter was seen in prefrontal areas, no matter whether the adolescent had been physically abused or emotionally neglected. However, in other areas of the brain the reductions depended upon the type of maltreatment the youth had experienced. For example, emotional neglect was associated with decreases in areas that regulate emotions.
The researchers also found gender differences in patterns of gray matter decreases. In boys, the reduction tended to be concentrated in areas of the brain associated with impulse control or substance abuse. In girls, the reduction seemed to be in areas of the brain linked to depression.

The original paper is Erin E. Edmiston; Fei Wang; Carolyn M. Mazure; Joanne Guiney; Rajita Sinha; Linda C. Mayes; Hilary P. Blumberg (2011) Corticostriatal-Limbic Gray Matter Morphology in Adolescents With Self-reported Exposure to Childhood Maltreatment Arch Pediatr Adolesc Med.;165(12):1069-1077.
The abstract is here:

Blumberg points out that adolescents’ brains are still pretty malleable, so these changes may not have long-term significance

Here’s another related finding (

When children have been exposed to family violence, their brains become increasingly “tuned” for processing possible sources of threat, a new study reports. The findings, reported in the December 6th issue of Current Biology, a Cell Press publication, reveal the same pattern of brain activity in these children as seen previously in soldiers exposed to combat.
The study is the first to apply functional brain imaging to explore the impact of physical abuse or domestic violence on the emotional development of children, according to the researchers.
“Enhanced reactivity to a biologically salient threat cue such as anger may represent an adaptive response for these children in the short-term, helping keep them out of danger,” said Eamon McCrory of University College London. “However, it may also constitute an underlying neurobiological risk factor increasing their vulnerability to later mental health problems, and particularly anxiety.

The stimuli used were pictures of angry, neutral and sad women’s faces. The heightened response was shown to angry faces, but not sad faces. The children had been ‘exposed to documented violence in home’ and were matched with controls. In the .pdf version, I can’t see any information about the age of the children, but there were 20 in the experimental sample.
The reference is McCrory, De Brito, Sebastian, Mechelli, Bird,  Kelly and Viding (2011) Heightened neural reactivity to threat in child victims of family violence Current Biology, 21 (23), R947-R948, and the full article is at

Again, this looks as though it might be saying something useful, though the ‘long-term’ claims would maybe depend on plasticity again.

Both news releases on PsyPost have the same old useless ‘brain’ picture on them.

British Psychological Society rubbishes DSM-5, backs Miller

The British Psychological Society* has posted a response to the American Psychiatric Association’s (APA) invitation to comment on the Development of the DSM-5 (the latest revision of the Diagnostic and Statistical Manual of Mental Disorders):

There are two big points here:

  • The BPS is effectively trashing the whole idea of a multiple medical categorisation of mental illness problems.
  • What the BPS say brilliantly backs up what I said in this Tuesday’s ‘Ways of Being Mad’ lecture – and no, I hadn’t read their response before doing the lecture.

The BPS group that reviewed the DSM-5 proposals did include Richard Bentall, who I highlighted in the lecture as throwing doubt on the whole Kraepelian schizophrenia/bipolar model, so perhaps it’s not surprising that they were doubtful about the DSM’s systematic classification, but they go further than that.

Their central statement (repeated over and over in their analysis of specific categories):

We believe that classifying these problems as ‘illnesses’ misses the relational context of problems and the undeniable social causation of many such problems. For psychologists, our well-being and mental health stem from our frameworks of understanding of the world, frameworks which are themselves the product of the experiences and learning through our lives.(emphasis added)

I’ve ended up quoting the BPs document very extensively, because they do make a lot of (what I think are) good points.
rst of all, they’re speaking up for a more psychological and less medical approach:

The Society is concerned that clients and the general public are negatively affected by the continued and continuous medicalisation of their natural and normal responses to their experiences; responses which undoubtedly have distressing consequences which demand helping responses, but which do not reflect illnesses so much as normal individual variation.

They point out, that although the overall model is medical, the criteria that are used aren’t medical/biological ones:

The putative diagnoses presented in DSM-V are clearly based largely on social norms, with ‘symptoms’ that all rely on subjective judgements, with little confirmatory physical ‘signs’ or evidence of biological causation. The criteria are not value-free, but rather reflect current normative social expectations. Many researchers have pointed out that psychiatric diagnoses are plagued by problems of reliability, validity, prognostic value, and co-morbidity.

This echoes Szasz’s (admittedly simplistic and over-the-top) distinction between ‘bad brains’ (illness) and ‘bad behaviours’ (expressions of problems in living). I think you can see Szasz’ (‘there is no such thing as mental illness’) and Laing’s (‘schizophrenia is a way of trying to coping with unbearable social relationships’) criticisms as extreme and over-the-top extensions of the BPS position, and not quite as weird and irrational as they appear at first.

The BPS response is politely phrased, but the underlying message seems to be to be ‘this is a load of rubbish, and is based on a mistaken, and over-inclusive, idea of mental illness’. The model the APA is using fits well with the ’Neo-Kraepelinian Manifesto’ that I showed you in the lecture.
The BPS ends up by saying:

Diagnostic systems such as these therefore fall short of the criteria for legitimate medical diagnoses. They certainly identify troubling or troubled people, but do not meet the criteria for categorisation demanded for a field of science or medicine (with a very few exceptions such as dementia.) We are also concerned that systems such as this are based on identifying problems as located within individuals. This misses the relational context of problems and the undeniable social causation of many such problems. For psychologists, our wellbeing and mental health stem from our frameworks of understanding of the world, frameworks which are themselves the product of the experiences and learning through our lives.

The Society recommends a revision of the way mental distress is thought about, starting with recognition of the overwhelming evidence that it is on a spectrum with ‘normal’ experience, and that psychosocial factors such as poverty, unemployment and trauma are the most strongly-evidenced causal factors. Rather than applying preordained diagnostic categories to clinical populations, we believe that any classification system should begin from the bottom up – starting with specific experiences, problems or ‘symptoms’ or ‘complaints’. Statistical analyses of problems from community samples show that they do not map onto past or current categories (Mirowsky, 1990, Mirowsky & Ross, 2003). We would like to see the base unit of measurement as specific problems (e.g. hearing voices, feelings of anxiety etc)? These would be more helpful too in terms of epidemiology.

I think it’s unlikely that the BPS views will affect the new version of the DSM much. As I said in the lecture, psychiatric care is an industry and, particularly in the US, the insurance industry is a large and powerful part of the system, and such industries need to be able to apply industrial standards, which is what the DSM does.

Read on if you want a bit more detail about the BPS response (and how it fits with the lecture):

They point out the overall lack of specificity of the apparently very specific diagnostic approach used in the DSM:

Finally, disorders categorised as ‘not otherwise specified’ are huge (running at 30% of all personality disorder diagnoses for example).

They make Bentall’s point that the basic categorisation system doesn’t hold up well:

Since – for example – two people with a diagnosis of ‘schizophrenia’ or ‘personality disorder’ may possess no two symptoms in common, it is difficult to see what communicative benefit is served by using these diagnoses. We believe that a description of a person’s real problems would suffice. Moncrieff and others have shown that diagnostic labels are less useful than a description of a person’s problems for predicting treatment response, so again diagnoses seem positively unhelpful compared to the alternatives.

They point out that ‘showing some symptoms’ isn’t really an indication of mental illness, but needs to be taken in context:

Personality disorder and psychoses are particularly troublesome as they are not adequately normed on the general population, where community surveys regularly report much higher prevalence and incidence than would be expected. This problem – as well as threatening the validity of the approach – has significant implications. If community samples show high levels of ‘prevalence’, social factors are minimised, and the continuum with normality is ignored. Then many of the people who describe normal forms of distress like feeling bereaved after three months, or traumatised by military conflict for more than a month, will meet diagnostic criteria.

This fits with the (brief) discussion in the lecture of statements like “1 in 4 British adults experience at least one diagnosable mental health problem in any one year, and one in six experiences this at any given time.” Perhaps all this means is that lots of people feel depressed/anxious/confused/persecuted from time to time, and there’s often a good reason for that, and these are not necessarily ‘symptoms of mental illness’ in themselves, even though people who do have serious mental problems may show the same responses.

They’re also dubious about the vagueness and ‘borderline-ness’ of some categories:

In this context, we have significant concerns over consideration of inclusion of both “at-risk mental state” (prodrome) and “attenuated psychosis syndrome”. We recognise that the first proposal has now been dropped – and we welcome this. But the concept of “attenuated psychosis system” appears very worrying; it could be seen as an opportunity to stigmatize eccentric people, and to lower the threshold for achieving a diagnosis of psychosis.

This parallels the point I made in the lecture about Blueler’s (1911) worryingly general category of ‘latent schizophrenia:

“There is also a latent schizophrenia, and I am convinced that it is the most frequent form, although admittedly these people hardly ever come for treatment. It is not necessary to give a detailed description of the various manifestations of latent schizophrenia …irritable, odd , moody, withdrawn or exaggeratingly punctual people arouse, among other things, the suspicion of being schizophrenic.” Blueler, 1911, quoted by Bentall in Understanding Madness

“irritable, odd , moody, withdrawn or exaggeratingly punctual”? Yes, that’s me.

They’re particularly concerned about ADHD diagnoses and medication with children:

We have particular concerns about the inclusion of Attention Deficit/Hyperactivity Disorder in this categorisation. Many of the concerns about the scientific validity and utility of diagnoses per se (articulated above) apply to ADHD. We are very concerned at the increasing use of this diagnosis and of the increasing use of medication for children, and would be very concerned to see these increase further.

*I have often rubbished the BPS in the past, and compared it unfavourably with the American Psychological Association, which I think is more socially aware and constructively self-critical than the BPS (one of the reasons I recommend so many articles from American Psychologist), but maybe I should apologise: I think this is well done.

Do you have to be mad to become a mass killer?

Interesting piece in Friday’s The Guardian by Simon Baron Cohen (a big name in research on autism) discussing the sanity or otherwise of Anders Breivik who shot and killed 69 people, mainly teenagers, who were at a left-wing summer camp on Utoya island in Norway.

My comments here are related to next Tuesday’s ‘Ways of Being Mad’ lecture, though I’ll say near the beginning that cases like Breivik’s aren’t the main point of the lecture. For those interested in ‘serial killers’, I guess Breivik isn’t one – ‘mass murderer’, maybe – but do you have to be mad to do such a crazy, appalling, heartless thing?

Baron Cohen reckons not, even though:

…if we could ask the court psychiatrists why Breivik murdered children, they would, according to this week’s reports, say it is because he had paranoid schizophrenia. This diagnosis, if confirmed by independent clinicians, has surprised some people following the case because the 1,518 pages of Breivik’s manifesto do not appear to be the incoherent output of “thought disorder”, but instead read like a rather linear, carefully crafted tome. It is the work of a man with a single vision, a single belief that he wishes to prove to the world in exhaustive detail, and in a logical fashion.

[….] If we had asked Breivik why he murdered all those young people, he would have said it was to draw attention to his manifesto aimed at saving Europe from the Muslims. Indeed he emailed his deeply disturbing “2083: A European Declaration of Independence”, to more than a thousand people 90 minutes before he bombed the government buildings in Oslo and just before he went out and shot all those people on the island camp.

He reckons the issue is in lack of empathy:

….those with antisocial personality disorder (including psychopaths) typically have [….] no trouble reading other people’s thoughts and feelings (intact cognitive empathy) but other people’s suffering is of no concern to them

and he goes on to discuss his ideas about the genetic and experiential origins of this lack of affective empathy, but:

….low affective empathy is not sufficient to explain such cruelty, because there are people with low affective empathy who do not go on to commit such acts.
Low affective empathy is the precondition for cruelty, interacting with other factors. In Breivik’s case, his deeply held (and frightening) ideological convictions may have been one extra ingredient in the deadly mix.

He ends by comparing Breivik with the young Hitler, who started his revolutionary career with an ‘irrational’ act:

At 8.30pm on 8 November 1923, Hitler (then aged 34) burst into the largest beer hall in Munich, fired a shot into the ceiling and jumped on a chair, yelling: “The national revolution has broken out!” Breivik also thought he was starting a revolution. When arrested, Hitler wanted to use the trial to make political speeches, just as Breivik hoped to do.

So, a psychopath, lacking in empathy, driven by racist intolerance (against Jews or Muslims, or whoever you can find) – but was the difference that Hitler was a shrewd political operator, and knew that he needed to save the heartless mass murder for later, while Breivik was a bit thick and unrealistic in thinking that the Norwegian people would rise up behind him? Or maybe Hitler seized the right time – economic collapse and chaos in Germany in the 20s – while Breivik should have waited until next year, for this century’s economic collapse and chaos?

Mmmm.. not sure. I go along with most of Baron Cohen’s analysis, but how dim/obsessed would you need to be to think that killing a load of children (presumably to intimidate their lefty parents and other ‘multiculturalists’) would serve to rally the forces of ethnocentric hatred behind you? There has to be some distortion of reality, surely – but does that count as being mad (or madder than Hitler?).