One of the things I hate most about the current AI craze is the apparently sanctioned ignorance in business and computer science about how the rest of the world works. Canada’s “Leaders Prize” just announced a $1 million prize for an AI application to automate fact checking. This assumes that a “fact” exists in the wild and can be quantified. It also assumes that fact checking involves dealing with written information, rather than talking with people. Among other things, actual fact checking for actual journalism requires calling people up and running quotes or statements by them and getting their reactions. In its current state, machine learning systems are simply not equipped to do this work.
So here are my hot takes. AI systems currently have no reliable method for detecting facticity, and any measures of verification they come up with will be too limited to be useful for a real life situation. I’ll be quite surprised if at the end of a year, whatever they come up with will be able to tell the difference between the Jeffrey Epstein story and Pizzagate, or any other meaningful example of fake news in the wild as opposed to whatever controlled conditions they come up with. Second, lack of fact checking is not the main problem with news right now. Third, the problem with current fact-checking is that outlets like Facebook don’t actually want to do it and for-profit media corporations have eviscerated commercial journalism. Someone please contact me when a $1 million prize is announced for an AI-based solution to rapacious capitalism.