From The Desk of...The Chief Scientist

"P-hacking the System"

Written by Jaclyn Reynolds on Monday, 12 February 2018. Posted in From The Desk of...The Chief Scientist

Science is hard. Scientists have to stare at mountains of data and try to figure out what secrets nature is whispering to them. There are innumerable blind alleys, dead ends, and false starts in academic research. That's life, and that's why over the centuries we've developed sophisticated statistical techniques to help lead us to understanding. But if you're not careful, you can fool yourself into thinking there's a signal when really you've found nothing but noise.

The problem is in correlations, or when two variables in your experiment or observation seem to be related to each other. Uncovering a correlation is usually the first step in "hey I think I found something," and so many researchers report a connection as soon as their experiment reveals one.

But experiments are often exceedingly complex, with many variables constantly changing - sometimes under your control and sometimes not. If you have, say, twenty variables that are all totally random, then by pure chance at least two of those variables will be correlated.

So when scientists fail to spot the correlation they were looking for, sometimes they start digging through the data until something pops up. And when it inevitably does - publish! But it was just a statistical fluke all along.

This practice is called "p-hacking", for reasons I'll get into another time, and it's a prime source of juicy headlines but faulty results.


Written by Paul Sutter on Wednesday, 07 February 2018. Posted in From The Desk of...The Chief Scientist

Once again there's a fresh round of blaring headlines and nervous chatter about potential links between cell phone use and cancer, this time based on a recent study purporting to show that a group of rats exposed to radio waves had greater incidents of tumors compared to a control group.

The radiation emitted by cell phones is not at the right frequency to ionize, excite, or even vibrate the stuff you're made of, so there's little to provide a causal link to cancer. So the standards of evidence for a study like this ought to be incredibly high. And then you read the study and find that the radiation-exposed rats lived longer, as a group, than the control rats. Since cancer rates correlate with age, something tells us that the research was not conducted well.

Sometimes scientists are their own worst enemies. There are the genuinely unscrupulous ones, willing to lie and cheat to advance their careers. And there are scientists who are, let's face it, inept at experimental design and statistical analysis. While we'll always have to be vigilant against those characters to make progress, they are thankfully in the minority and we can assume that most researchers, including the ones behind this study, have good intentions and are decent at their jobs.

But there are flaws in the modern scientific system that we must acknowledge. The unrelenting pressure to publish and the exhausting lifelong chase for funding create incentives for poor research to make it into journals and into the public discourse, muddying the waters and hurting science in the long run. This is especially harmful in fields that study extremely complex systems, like epidemiology, where good statistics are naturally hard to come by.

Scientists are just fighting for their careers, but when we need another round of discussions to (re)explain how to spot mistakes in metholodogy, something needs to be fixed.

"A Horoscopic Perspective"

Written by Paul Sutter on Monday, 29 January 2018. Posted in From The Desk of...The Chief Scientist

I feel incredibly sympathetic towards our ancestors for looking to the sky for answers. The dependable, regular movements of the heavens stand in stark contrast to the chaotic, unpredictable, and often violent Earthly world. It's not a huge stretch to imagine that perhaps the positions of the planets would tell us something important. Indeed, many of the early pioneers of what would become science were motivated not by understanding nature for the sake of it, but to build better tools for astrology.

Alas, the more we learned about the celestial realm the more we realized that it's just as chaotic, unpredictable, and often violent as it is down here. And while one of the great triumphs of scientific understanding was the realization that physics is universal throughout the cosmos, that same understanding places strict limits on what can influence what across the vast expanses of time and space that we call the universe. And it's very clear that the positions of the planets bear no relation to human activities.

So why does astrology persist even today? It's part confirmation bias - horoscopes and attributes are so vaguely written that you can give the exact same "prediction" to a hundred people and most will agree it applies to them, despite their sign. It's part cultural tradition - when humanity has been doing something for at least thousands of years it's hard to shake it off. And it's part comfort - who doesn't desire some form of control or knowledge over their lives?

It's this last part that puts science communicators in a tough spot. Obviously horoscopes fill a need in the lives of some people. If we're to (rightly) claim that they're bogus, what do we have to offer in replacement?

<<  2 3 4 5 6 [78 9 10 11  >>