Musings on Methods (or Frustrating Facebook Feuds)

I seem to become most inspired to write after I've had some sort of run-in with somebody on the internet. Lessons re-learned: you don't have to respond to challenges, particularly from people you don't know and if you know you're right, that's enough.

Or it should be.

In general, if I take the time to comment on something, I feel fairly strong about what I'm saying. If I then get challenged, I generally come back and politely say why I said what I said, in more detail than the first time.

I always feel I have to defend myself.

Honestly, I gotta get over that.

All that happened today -- in a discussion of a friend's link to a report about a recent study in the journal Cancer Epidemiology, Biomarkers and Prevention, linking pancreatic cancer to relatively low soda consumption (two drinks per WEEK). I had posted the same study to my friends on Facebook two days earlier, which I think is damn good evidence that I totally AGREE that soda consumption is bad for you and that the main finding in the study is extremely provocative. The odds ratio of developing pancreatic cancer, when comparing those who drank two or more sodas per week to those who drank no soda, was 1.87 -- that means nearly TWICE the risk.

Provocative to say the least!

However, the limitations of the study have been discussed fairly widely by other researchers who've read the article. First, the incidence of pancreatic cancer is low to begin with (take a look at the American Cancer Association's list of top cancer sites, and it isn't even in the top 10), so making a big to-do about soda consumption and pancreatic cancer *might* be leading people to believe that pancreatic cancer is the most likely outcome of soda consumption. Hardly!

Second, other confounding factors (such as overall sugar consumption or race) were not adequately controlled for. The study was done in Singapore with an entirely Chinese sample -- probably doesn't matter, but we don't know until we have a racially diverse, randomly chosen sample where we can look at racial differences in soda consumption and pancreatic cancer rates.

In addition, the majority of the pancreatic cancer victims claimed NOT to be soda consumers. So, that tells me that it's those people in the pancreatic cancer group who consumed LARGE quantities of soda that drove the finding. In other words, it's the "or more" part of "two or more" that likely made the finding statistically significant.

There's also the issue of relying on self-reports. Most people don't intentionally lie, but heck, most of us want to look good, especially to doctors, and so most researchers admit that relying on self-reported data has serious reliability and validity issues. (This means that some people may have UNDERREPORTED their soda consumption, which would likely have changed the finding!!)

Self-reports are often the best data a researcher has, so it's what a researcher uses. But in an IDEAL world, to test the effects of diet on health, you'd want to control -- and WATCH -- the diets VERY carefully, so carefully that very few people would consent to be in the study. Do you want me to tell you what to eat for the next 14 years? (That's the timeline of the study, which is also impressive.)

While it's totally believable that soda consumption, like overall sugar consumption, is linked to rising rates of diabetes, obesity and pancreatic diseases, this study doesn't prove that two sodas a week will give somebody pancreatic cancer.

And yet, that's what some people want all of us to believe.

There are other potential issues with the study -- all, by the way, very common in ANY study. I'm not writing this to rag on this particular study, but rather to point out that the criticisms of the study do not necessarily represent the interests of the big bad soda and sugar industry (which the man I was arguing with accused me of siding with).

As fun as it is to think of conspiracies against medical research and interests of corporations, not all criticisms of a study originate from a desire to side with the "bad guy".

I have no desire to be on any "side"; it just so happens that the criticisms I've read of this study conveniently play into the hands of the sugar industry, which of course wants you all to continue chugging that Pepsi or Coke and eating those candy bars and that packaged crap sitting at the bottom of your purse or backpack.

Don't do it. Eat right.

But also remember some facts of methodology. One, correlation doesn't equal causation, and two, a statistically significant finding does not equal a meaningful finding.

I thought saying all this in a Facebook post was showing quite a bit of attention to detail.

I was called "kneejerk".

Yeah, I was pissed and inspired to write.