Return of the Deathfatz!
Hello, internet! I’ve handed in all of my assignments on time and am now somewhat smug. And because I’ve now handed in all of my assignments, I can talk about deathfatz!
Our project brief was to “give a presentation that is interesting, both from a statistical and general perspective, on a matter where statistical ideas are critical and taking as a starting point some contemporary (2010) source.” The deadline was December, and we got given the brief at the start of November, so of course, my immediate reaction was to ignore it for a while. Eventually, with two weeks to go, I realised that I should probably just write the bloody presentation now. I’d had a few ideas, but despite my reservations, my thoughts kept returning to the frankly nonsensical story that appeared on the BBC website recently, titled: Obesity’s link to sense of smell.
This has turned into a bit of a rant, so I’m hiding the rest under the cut. Keep reading if you have an interest in dodgy statistics and media misrepresentation of science. Probably don’t keep reading if you don’t want to think about the fatz today.
The story links to an article that’s available to read on the online version of the journal Chemical Senses. (Incidentally, there is at least one journal – the International Journal of Obesity – that focusses, as the title suggests, on obesity. The article wasn’t submitted to that journal, for reasons that will become clear.) It’s an account of two small studies that were conducted at the University of Portsmouth by Dr. Lorenzo Stafford, a psychologist, and Kimberley Wellbeck, who appears to be a current student at the university. The first study took 24 volunteers and tested their sensitivity to a “neutral” – read: “not food-related” – smell. All of the participants were asked to refrain from eating after 11pm the evening before the experiment, and only half of them were given lunch. The results of this study concluded that the participants were more sensitive to the neutral smell if they hadn’t eaten. The second study was slightly larger, with 40 volunteers. They were also asked not to eat before the test, and were tested for their sensitivity first to the neutral smell, then to a food-related smell (some kind of herb flavouring). This study replicated the results of the first (that participants were more sensitive to the neutral smell whilst hungry) but found the opposite result for the food-related smell – the participants were more sensitive to the smell if they had eaten.
So far, so simple. These are results that might indeed be interesting if you were making a study of peoples’ sense of smell. I should point out here that I know bugger all (scientifically speaking) about how sense of smell works, and so I can’t tell whether the tests that these researchers conducted were meaningful in any way. After all, you can make numbers dance on a page as much as you like, but if the method that you used to gain those numbers was flawed, the results tell you nothing. Remember that, it’ll come in useful.
What you’ll notice about the results I’ve reported above is that they’ve got nothing to do with obesity. You might even find that a little peculiar. Why, you might be asking yourself, would the BBC print an article claiming a link between obesity and sense of smell based on a study that didn’t even mention it?
The answer is: they did mention obesity. Kind of. In the second study, the one with 40 volunteers, the researchers also took the participants’ heights and weights. They used that information to get the BMI of each participant, and they took the median of those. The median – for people who don’t routinely read maths textbooks for fun – is the average that you get by arranging the numbers by size and picking the middle one. It’s not a bad one to use – it ignores the very large or very small numbers at the ends, which can be useful. Having found that number, they basically drew a line down the middle of the participants. Anybody with a BMI smaller than the median got put into one group (“low BMI”), and anybody with a BMI above the median got put into another group (“high BMI”). The idea was that this would give them groups of equal sizes, but through some quirk of the BMI numbers, they ended up with 21 in the “low” group and 18 in the “high” group (they’d already discounted one person, who apparently couldn’t smell anything!). And then they did some analysis of all the numbers they had, based on whether the participant was in the “low” or the “high” BMI group, and they found that when participants in the “high BMI” group were hungry, they were more sensitive to the food-related smell than the people in the “low BMI” group.
This probably sounds very convoluted, and that’s because it is. Sorry about that. When I actually gave my presentation, I did it with the aid of this stick-figure drawing:
Here’s three lines of stick-figures. They’re meant to represent the participants being arranged in order of BMI. People with smaller BMIs are to the left, people with larger BMIs are to the right. Right through the middle of all three lines of stick-figures is a vertical arrow to show where the median is. (For simplicity, I’ve made it look like the median is in between two people. It doesn’t have to be that way.) The groups on the left are the “low BMI” groups; the groups on the right are the “high BMI” groups. The stick-figures are also colour coded. The reason for that, and the reason for there being three lines to begin with, is that the article didn’t report what the median value was. In fact, it didn’t report anything about the BMIs of the participants. So I colour coded my little stick figures to show a few of the possibilities that this threw up. In the top line, they’re all coloured green, to represent a “normal” BMI. It’s perfectly possible that every single one of the participants had a “normal” BMI, and in that case, there was nobody obese to measure, and so certainly no link with obesity. In the middle line, all but two of the stick figures are green. One on the left is black, to represent an “underweight” BMI, and one on the right is red, to represent an “obese” BMI. (Incidentally, I’m sorry about the way these colours sound. But when I was choosing colours, I had to go with ones that were a) visible on a slideshow and b) easy to distinguish – although with hindsight, I probably shouldn’t have put red next to green anyway. Argh. The point is, I was going for colours that didn’t make my eyes hurt, rather going for some kind of good/bad traffic-light colour scheme.) If that were true, the results might say something about obesity, but probably not much. One person is not a very good sample. Or, like the third line, they could have ended up in a situation where absolutely everybody in the “low BMI” group were “underweight”, and everybody in the “high BMI” group were “overweight”.
There are other possibilities – lots of them, in fact. But I thought those three were enough to make my point, which was this: as it is currently written, that article says precisely nothing about the role obesity may or may not play in the detection of food-related odours. The fact that the researchers used a categorization scheme of their own, rather than using the existing BMI categories, raises a massive red flag for me, and the fact that they chose not to report anything about the BMIs that they collected makes me think that they did so because they knew that there was actually nothing to report. They made those numbers dance on the page, but since the method they used to gain those numbers was so flawed, that whole portion of their article is, in fact, meaningless.
I should say, too, that I actually contacted the lead researcher about this from my university email account (making sure to avoid making any of the infuriating mistakes that students often make when writing emails, and also being a lot more polite than I’m being here!). Sadly, but unsurprisingly, he never responded.
Still, it was just a small portion of their research. It wasn’t even mentioned in the title of the article, and the article wasn’t submitted to a journal that focussed on obesity. Were it not for the BBC, the study might never have been brought to my attention. And the BBC decided to interpret the study as saying that “overweight people have a greater sense of smell for food”. Except, of course, that the study showed nothing of the kind. It did show that the participants in the “high BMI” category were more sensitive to the smell of one herb when they were not hungry, but that was all.
This long, extended rant was suitably edited by me to fit into an 8-minute presentation, during which time I managed to name check Kate Harding, and mention the BMI project, because sometimes you just have to. I suspect that my classmates were somewhat confused by the slide featuring Moxie (the “morbidly obese” cat from the project), but they were suitably entertained by my mini-rant on why the words “obesity” and “epidemic” don’t go together. (Answer: 1] because I don’t think that word means what you think it means, and 2] because if the media repeatedly implies that obesity is a transferrable, fast-spreading disease, eventually somebody will believe them.)