My brother is a professor of political science, and during a recent holiday we got to talking about how the word "truthiness" has burst onto the scene in recent years, and how it has gummed up the works of tried and true practices of fact-checking. Perhaps what has troubled me most about hearing this word bandied about is that it seems to ignore the rich history of statistics - the hundreds of years of discovery that our non-partisan ancestors developed to help us appraise the likelihood of events.
As someone who has spent years in research labs, I know first hand that the field of statistics (like its big sister mathematics) can seem like a vast ocean one couldn't even dream of circumnavigating in a lifetime. Anyone who claims to know everything there is to know about statistics is lying to you. So proposing that we apply tools from statistics to the problem of determining the "truthiness" of a statement (and can't we just say veracity?) may seem like we are moving the conversation out of the realm of public debate and onto a chalkboard in the basement at MIT. And yet, I don't believe we have to venture too far into statistics to surface some useful frameworks for this conversation.
Typically when we want to verify a statement, we test how likely the statement is. Certainly this could be framed as, "how likely is it that this person would say this?", but what's most relevant is, "how likely is this statement true given the evidence at hand?". We are looking for the point, or a range, on the distribution of probabilities of the statement's veracity. If the evidence stacks up that the statement is not very true, we'll get a number close to 0. If the evidence stacks up showing the statement is very likely to be true, our number will be closer to 100%.
And there you have it. It seems our work is done. It's true we could back to that MIT basement and hash out approaches for defining a bespoke probability distribution for the problem at hand, but for most political fact-checking we can use one of the go-to distributions out of the box (Gaussian, binomial, geometric, Poisson), and arrive at a suitable result. Furthermore, I think you would find that creating a chart that tells you how probable an event is is something you would arrive at if left alone to ponder a problem you believe is important. The mind bends towards making meaning, and that includes charting patterns of events.
So if it's not statistical sophistication that we're stuck on, where's the challenge? How do we have conversations about truth when the very concept of truth (as verified by evidence) is called into question? And what do us scientists and mathematicians do in the meantime?