Monday, June 30, 2008

But the Study Said!

A while ago I put up a posting on being wary of polls and statistics. That should also have included being wary of studies. When an article starts out "studies say" liberally sprinkle your credulity with skepticism.

For years now the medical research community has been railing away about the evils of caffeine. They have also been warning us about the possible causative relationship of caffeine and cancer. And then the tide sort of started turning. And some studies showed that there was no or little causative relationship. And some studies said there never had been. Yes indeed, "some studies."

Today our local paper reprinted an article written by Linda Searing of The Washington Post. She writes a series entitled "Quick Study," in which she gives just the bare bone facts about various health studies. Bare bones is truly an understatement. There would seem to be a truly positive side to her reporting, if only we had more of the facts.

The headline was "Coffee, tea not linked to breast cancer." This title was printed in inch-high letters. No way that readers were going to miss this. And some readers, working on the assumption that newspapers can't/don't print lies aren't going to read any further than the headline for the article. And they will now be convinced that they know all they need to know. But will those readers who read the whole article have any better information?

Let me reprint for you here the entire article so you can see why I am sometimes such a skeptic.

Linda Searing
The Washington Post

Does drinking coffee or tea affect the likelihood that a woman will develop breast cancer?

This study analyzed data on 85,987 cancer-free women who, on average, were in their mid-40s at the start of the study. In a 22-year span, 5,272 of them were diagnosed with breast cancer.
Women who reported drinking the most tea or coffee (four or more cups a day, caffeinated or decaffeinated) had no greater or less risk of developing breast cancer than those who drank virutally no coffee or tea.
Also, no association was found between consumption of caffeinated sodas or chocolate (which contains caffeine) and the risk for breast cancer.

Women. U.S. women face a one-in-eight chance of developing breast cancer.

The authors wrote that further study was needed on the possibility that caffeine "may modestly reduce risk of post-menopausal breast cancer."

Find this study in the May issue of the International Journal of Cancer.

Got to love Searing's reference to "This study," especially since she never names the study. She says that "the authors wrote," but she doesn't name the authors either. She says the study covered a"22-year-span" but doesn't give the calendar years for the study. She sends us to "the May issue" but there is more than one issue in May.

Then there is her "one-in-eight chance of developing breast cancer" statement. According to whom? Where did she get this number from? She never tells us that either. Go to the figures she does present us with and you will see that 5,272 of the 85,987 of the women involved in the study got breast cancer. I'm not a mathematician but 1/8 of the women in this study should be around 10,748 women. The study, or at least the figures that Searing gave us, shows that far fewer women got cancer than the one-in-eight figure she gives us. The study is showing us a one-in-seventeen chance, a huge difference.

Then there is her use of the word "caveats." A caveat, singular, is "a warning or caution; admonition; qualification." A caveat tells us to be ware in how we look at information just presented to us. Normally we use the word to tell readers that what they are reading may have a problem, that they should take what was read with a grain of salt. First, she doesn't give us "caveats"--her word, plural, but one single caveat. Next, she is not warning or cautioning us about anything that she has said before. She brings up something totally new. She tells us that further study will be needed on whether or not caffeine "may modestly reduce risk of post-menopausal breast cancer." "Clearly," and there is nothing clear about it, something in the study she mentions must have pointed to a possibility of caffeine as a risk reducer. And if it is, that is a highly positive thing, and doesn't need a "caveat."

The Washington Post is supposed to be a "respectable" newspaper, with high journalistic standards. And yet there is Searing's little piece. What is respectable about a piece that makes a loudly definitive statement and then backs up that statement with almost no information?

We get information from a lot of different sources. One thing we need to do carefully is evaluate that information. I guarantee you that there are thousands of people out there, perhaps hundreds of thousands, perhaps millions, who are going to joyfully return to caffeinated drinks, all on the basis of Searing's article's appearing in papers around the country. It is possible that the headline is correct, and we can rejoice in one less thing that causes breast cancer. And it is entirely possible that the headline is not correct.

Would you base your caffeine intake on what you read in the article?


Anonymous said...

I can't imagine why the Post would print garbage science reporting like this. But why would your paper reprint it? You'd think that between 2 newspapers someone would have noticed all the problems.

Anonymous said...

I admit that I only skimmed the post at first and skimmed the article too. When you started pointing out what was missing I slowed down and read every word. That's part of the problem I think. At least for me I haven'e got that much time so I skim things and I guess I think that all the important stuff was actually there even when it so obviously wasn't.

Anonymous said...

Thanks for this post. My students so often use sources in their papers that aren't much better then the one you dissected. Had a cup of caffeinated coffee while reading this. I'm living in hope that just maybe this study will have shown what it is reported to have shown.

Anonymous said...

The author of the article wasn't worried that anyone would find out how little information she was giving. The Journal she sent readers to is not available for free anywhere online, it is not in the free databases of the public library and the cost of buying the copy of the Journal was about $60.