Faulty Research

Alissa Grosso

Last week I wasted my time on the internet reading an article about which U.S. states had the highest number of “porch piracy” incidents, aka had the most delivered packages stolen. The number one state for porch piracy according to this article was Vermont, but I would caution you to take that with a grain of salt. What data was used to arrive at this conclusion? Did the researchers analyze police reports? Did the researchers track the number of claims received by the post office and delivery companies like UPS? Nope. Instead they analyzed search volume for specific terms. In this case “package stolen,” “package lost,” and “package theft.”

The first issue I see with this is that these are a narrow set of search terms. For one thing, they all rely on the use of the word package. What about box, delivery or order? Those could all be used, and might even be more common than the term “package” in some places. Maybe they just say “package” more in Vermont. (Get your mind out of the gutter!) What if there’s some bar band in Burlington that goes by the name Package Stolen? That could throw off the results entirely.

I mean, I get why researchers might rely on this strategy. It’s pretty easy to do. You could employ AI or some other software program and gather your data in no time. The problem is that this lazy research method, doesn’t necessarily return the best results.

Not long ago I read a clickbait style article that attempted to determine once and for all what the most popular style of pizza is. For those of you reading this who are wondering if they mean regular or Sicilian, they do not. I’ve heard of Chicago deep dish pizza before, but the fact that there’s something called Detroit style pizza was news to me. Sicilian apparently didn’t even make the cut. But what I think of as “regular pizza” is technically “New York style pizza.”

So, here’s why this particular article was especially useless. To determine what the most popular pizza style was they analyzed Yelp reviews for the terms “New York style,” “Detroit style” and “Chicago style” pizza. Now, it’s no surprise that “New York style” ranked as the most popular, but even if it hadn’t the results would be meaningless. Because let me assure you that no one that grew up in the greater New York metropolitan area refers to it as “New York style pizza.” It’s just pizza. I think the first time I’ve ever used the phrase “New York style pizza” was in writing this post.

It’s like when I see football-themed graphic t-shirts that were clearly designed by folks in other countries because they use the term “American football” which is not something any American ever says. To us it’s just football. The rest of you call it “American football” because you insist on calling soccer football and have to make the distinction.

Anyway this rant is a reminder to be wary of articles or other claims that may be backed up by dubious research. Stay skeptical, friends, and check their sources!

— Alissa