Monday, 15 March 2010

Late-breaking FAIL: webcomic creator in cissupremacist quiz error

I decided recently that I should apply for membership of the British Psychological Society. I have a valid degree to apply for graduate membership, and the recent business at the APA  over the DSM-V categorisation of Gender Incongruence shows that psychology, as a science, needs people who can think beyond the binary. Ironically, one of my favourite webcomics has recently provided an indication of why this is the case.

XKCD is a webcomic produced by Randall Munroe. Munroe comes from a scientific background, having worked as a contractor for NASA, and often includes geeky, science-based humour in the strip. He's also currently conducting some kind of research into colour blindness, and has included a survey on colour-blindness on his site.

If you've been following this blog awhile, you'll have worked out what's annoyed me. It's this question:

Do you have a Y chromosome?
If unsure, select "Yes" if you are physically male and "No" if you are physically female. If you have had SRS, please respond for your sex at birth. This question is relevant to the genetics of colorblindness.
 
What's particularly annoying about this is the fact that Munroe is clearly trying to be gender-inclusive, bless him. And I can sort of see why you might want to know the birth gender of people who are colour-blind, if you're studying the genetics of the condition. But...the fail. It burns.
 
It burns for a lot of reasons. Mainly because it's more complicated than that. Munroe doesn't allow for intersex people: by reducing 'physical maleness' to the matter of having a Y-chromosome, he excludes men with Klinefelter's syndrome (on which note check out Helen from Bird of Paradox's post about KS Awareness Week), who have a Y chromosome, it's true, but also have an extra X chromosome, and are less 'physically male' than the generic-model XY-guy. He excludes those people with tetragametic chimerism who can have XX and XY chromosome structures in different parts of their bodies. And he is of course tremendously hurtful to trans people by reducing the issue of their gender to what chromosomes they were born with.
 
I don't think that this was deliberate on Munroe's part. He's mentioned people who've had SRS, he's tried to frame the question not as 'are you male or female?' but 'do you have a Y-chromosome?' He's tried. This isn't the kind of Cisfail the Guardian, say, engage in when they run columns about trans people and queerness by folk like Julie Bindel or Bea Campbell. But he's got the whole thing bloody wrong. Not just ethically, in fact, but methodologically.
 
Because the thing is, doing a survey on the internet, in which anyone can take part, is a lousy way to carry out research. You'll get lots of responses, but how do you know those responses are the same people? How do you know which participants make up your sample? The data at the start of the survey are meaningless because - and I'm gonna rock you in your socks here, people - the internet lies. Anyone (i.e. me) could go on Munroe's survey, and claim to be, say, a colourblind Frenchwoman in possession of a genuine Y-chromosome, and then proceed to answer the survey by, say, giving the colours increasingly ridiculous names. This happens when you do research. One of my old psych lecturers said that psychology experiments are tainted, for the most part, because the samples they usually use are made up of psychology students, and usually most of those students will either (a) be nice students trying to 'help' you get the result you 'want' or (b) evil little feckers (i.e. me, again) deliberately trying to give answers which will give you the result you don't 'want.' So, y'know, doing a survey on the internet in which anyone can participate is methodologically unsound from the get-go. So why ask an offensive question in the first place?
 
It'll get lots more responses, that's certain. But a more tightly controlled research project carried out among a smaller participant population would yield better quality data from participants who could be much better described. As it is, Munroe's survey is basically an open invitation to people to lie about their gender, about whether or not they're colourblind, about what country they're from (and we've noticed the annoying 'Tell us your native language, but answer questions in English' question, haven't we?) and so on, and then to 'answer' the survey by calling the colours things like 'Jan Vermeer's Pannetone Cyclotron', 'Grrr-nommy-nomminy' and 'the mongspoddler'. Not that I would endorse such behaviour (which may or may not have been carried out by me).
 
The fact is that, frankly, trans people, as much as we might wish it otherwise,  are such a statistically small section of the population (yes, even on the internet) that, unless you're actually doing research on the trans community (and research in such a sensitive area should come with very specialised ethical requirements), it isn't worth controlling for the possibility of trans men and women answering your survey. Munroe could easily get away with asking the question 'Are you male or female?' safe in the knowledge that most of his results will come from cis people, and that any statistical patterns relating to biological gender will be easily apparent in the data. To get into discussion of chromosomes, to talk of being 'physically male' or 'physically female' and to ask trans people to give their birth gender (therefore reminding them of their status in some bigots' eyes as not real women or men, with all the traumatic memories that will trigger) is both unnecessary and irresponsible. It's bad science, in both senses of the word. And, coming from the creator of a strip who's so often found humour in mocking other peoples' scientific errors, it's a depressing thing to see.
 
As any scientist will tell you, the biggest part of the job is asking the right questions. Munroe has tried to ask the right question in his survey, but he's tried too hard: and when he finds himself toiling through page after page of deliberately buggered-up results, he'll only have himself to blame.
 

1 comment:

  1. This is mostly in response to your twitter comments today that suggest you've gone too easy on Munroe, and I'm sorry I've got to disagree, strongly.

    Intent is so important when it comes to an error such us this, its clear that this had an intent of inclusiveness, regardless of how wrongly it was carried out. This intent should then inform the reaction. It's clear that ignorance has caused the problem, but its also clear that the problem hasn't resulted from maliciousness. The benefit of the doubt MUST be given, the error politely pointed out and the ignorance removed through education, their reaction is then critical, if they do nothing then by all means unleash the dogs, but if they do something to rectify the error then another victory against ignorance has been won.

    It's a highly complex issue, that even those who live and breath it get wrong. How can the rest of the population be expected to get it right all the time, and to jump down their throats when they inevitably make a mistake is absolutely the worst thing to do, which will only result in further alienation. We'll never make any progress in understanding if we attack every slight, we need to choose the right battles to fight but far more importantly take the opportunity and generate the right circumstances to provide education, THAT is the only answer to ignorance.

    Sorry Adam, but your was right first time round.

    ReplyDelete