Reporting Done Right

Every now and then, I go on a rant about bad science reporting. Most often, it's about geology, the science I know best: I can spot the weaknesses in the study, I often know what's being left out or misrepresented, and I'm sensitive to hype- overstating the meaningfulness of the results.

I think medical reporting is the worst, but I don't often comment on it because I don't know much about medicine, or the particulars of research design in that field. But on any given day, you can be assured that there will be some medical story in almost any given news source you follow thoroughly, stating that this new discovery "may" lead to a cure for cancer, or aids, or acne or whatever, with the stern implication that you should eat more broccoli steamed in red wine and covered with chocolate sauce. If you want to live.

I don't read a whole lot of those articles anymore.

Research is always going to be an iffy thing. The complexity (and I would argue, beauty) of the real world is due to many, many interacting factors. One of the goals of scientific research is to reduce those factors, or variables, to a number that can actually be managed. So a bit of research that says "x" under conditions "1,2,3" may be contradicted by research that says "y" under conditions "1,2,4," even though "x" and "y" seem mutually exclusive. And that doesn't even take into account unrecognized variables "33, 47" that may or may not have existed in one or both studies.

And the fact is that the vast majority of us simply don't have the background, patience and perseverance to actually read the professional literature outside of our own discipline(s).

All that said, I just finished a brilliantly written article in The Guardian that lays out the methodolgy, results, and conclusions of a recent journal article on smoking cessation. The author starts with a template of questions, then writes one to a few paragraphs answering each of them.

• What do we know already?
• What does the new study say?
• How reliable are the findings?
• Where does the study come from?
• What does this mean for me?
• What should I do now?

I encourage other journalists to use this form, and not just in medicine, but to report any research that might be expected to have implications for the general public. Quit copying press releases. They're not about research. Some of the relevant information might be in the press release, but part of what we used to expect from journalism was that it would do the hard work of making sense of something, then explain it in a way that was mostly accurate, reasonably objective, and palatable to consume for the average high schooler.

I'm not demanding perfection, I'm just asking that you do your jobs.

Heh. I was going to end by applauding the author. Here's the citation at the end:

Piper ME, Smith SS, Schlam TR, et al. A Randomized Placebo-Controlled Clinical Trial of 5 Smoking Cessation Pharmacotherapies. Archives of General Psychiatry. 2009;66(11):1253-1262.

© BMJ Publishing Group Limited ("BMJ Group") 2009

BMJ is the British Medical Journal. It was written by professional medical journalists and doctors.

See what happens when you have journalists who actually know something about the subject on which they're reporting?