"The Hard Truth About Data"

I've had a pithy saying that I like to toss around. I don't know where I got it, and I can't find a source for it, so I'll go ahead and take credit for it: the first thing that lies to you is the data.*

If we have a question about the world, or about best communication practices, or what audiences prefer, or whatever, it's tempting to just "get some data" and call it a day. We'll do some measurements or make some observations, and that will tell us how to move forward. Easy-peasy!

Except, it usually won't.

People lie in surveys. Bias sneaks through in unexpected ways. Instruments can be miscalibrated or used incorrectly. And perhaps worst of all, uncertainties are miscalculated or left out altogether. The classic book "How to Lie with Statistics" by Darrell Huff goes into this last point in depressingly hilarious detail, and should be required reading for...well, let's just go with the whole human race.

Data can easily mislead you, and if you're not careful it's even easier for unscrupulous people to abuse the "results" for their own purposes. While some good questions to ask will come in another piece, it's most important to remember that most data-collection efforts are far more inconclusive and nuanced than they may appear at first blush. Never be afraid to challenge results and arguments - this is exactly what happens in the halls of academia every single day.

It's tempting to put a blind trust in data, but that's a rookie mistake. The extensive training that it takes to be a scientist starts with data collection techniques, but quickly moves to data interpretation, and pretty much occupies the rest of their professional careers.

*The second thing that lies to you is yourself, but that's the subject of another piece. And yes, "data" is usually plural, but it sounds better as a singular here so I'm leaving it.--