Hmm: Facebook stats used to tout ‘fake news’ problem turn out to be erroneous

Hmm: Facebook stats used to tout ‘fake news’ problem turn out to be erroneous

“Fake news” seems to be the new “climate consensus.”

“Fake news” basically means whatever you need it to mean, for the purpose of a given debate.  (It rarely refers to actual “fake news,” as in “totally made up stuff that never happened that is being reported as if it did.”)

“Fake news” is a rallying standard for people to range themselves on the right side of, if they know what’s good for them.  If you’re in the “in” group, you think all the approved things about fake news.  You blame it for all the bad things that you, as a member of the “in” group, don’t like.

Nothing can disprove “fake news.”  No matter what happens, no matter how counterintuitive events are for the “fake news” cult belief, it all just proves that fake news is bigger, more pervasive, more powerful, and more worser than we thought.

And sure enough, the methodology and data of “fake news” fall apart at the lightest touch.  Any day now, there’s going to be a fake news hockey-stick graph.  That’s the kind of thing “fake news” is.

As if on cue, a social-media-savvy data sleuth, Tim Peterson, has unearthed an inconvenient fact that undercuts the fake-news narrative.

The narrative made its biggest public splash a few weeks ago, when BuzzFeed posted an article claiming that “fake news” outperformed legitimate news on Facebook, in the days before the 8 November election.  With this article, BuzzFeed was refining a theme developed in October about social media being flooded with “fake news,” which a shadowy new organization calling itself PropOrNot suggested was being generated by the Russians.

People have gotten bogged down in things like debating what “fake news” is, and who really engages in it.  But the main point of the BuzzFeed article from November was actually to show that fake news had taken over Facebook, and was influencing voters’ perceptions.

This theme immediately went high-order in the mainstream media, and among Democratic politicians.  “Fake news” was influencing the vote!  People voted based on the false perceptions created in their minds by “fake news”!  And since Russians were involved, it somehow all got mashed together with the completely unrelated concept of hacking by the Russians, and quickly became a whole new branch of cosmology overnight.

The big problem?  As Tim Peterson discovered, the Facebook engagement numbers used to demonstrate that “fake news” outperformed legitimate news, in terms of user engagement, were invalid.

Peterson reports that Facebook has acknowledged this, after being challenged on the numbers.  They don’t add up.  It should be possible to parse out subsets of engagement numbers from the most comprehensive set of engagement numbers – but when Peterson tried, he found that at least one subset was actually larger than the comprehensive set he wanted to extract it from.  It was significantly larger, in fact.  The error couldn’t just be incidental.

And the analysis done by BuzzFeed, of Facebook engagement relating to “fake news” posts versus legitimate news posts, involved just such data sets.

facebook_linkengagements-1920
Tim Peterson graphic, marketingland.com. Peterson’s caption: “Facebook’s Graph API and mobile app report mismatched engagement rates for outside links.”

There are presumably correct numbers that can answer data-slice questions about Facebook engagement.  But Peterson’s investigation has shown that, as things stand, querying for them doesn’t return a result that can be cross-checked as valid.

I’m sure BuzzFeed used the numbers they did come up with from Facebook in good faith.  It may be that they got lucky, and the numbers that went into their analysis were valid.  (Or, at least, the relative pattern between “fake news” and legitimate news was valid, even if the exact figures weren’t.)

But we don’t know that, at this point.

Bottom line: wholly aside from whether BuzzFeed identified “fake news” accurately, we don’t know what BuzzFeed suggests we know about which posts were getting more user engagement.  All the support posts of the “fake news” narrative have crumbled.

Of course, we have never known either if the engagement measurements BuzzFeed chose had any causal relationship with how people voted.  It’s not even clear how that could be proven.

We’ve just had a lot of pundits and politicians jumping on the narrative and hammering it, regardless of how evidence-free it is.

Sounds an awful lot like “climate consensus,” doesn’t it?

J.E. Dyer

J.E. Dyer

J.E. Dyer is a retired Naval Intelligence officer who lives in Southern California, blogging as The Optimistic Conservative for domestic tranquility and world peace. Her articles have appeared at Hot Air, Commentary’s Contentions, Patheos, The Daily Caller, The Jewish Press, and The Weekly Standard.


Commenting Policy

We have no tolerance for comments containing violence, racism, vulgarity, profanity, all caps, or discourteous behavior. Thank you for partnering with us to maintain a courteous and useful public environment where we can engage in reasonable discourse.

You may use HTML in your comments. Feel free to review the full list of allowed HTML here.