In plain words: Fake news claims are judged like a logic check: does the story follow from the evidence, contradict it, or neither? Several such checkers plus a large language model are combined and retrained on possibly-wrong labels, reaching 88.063% accuracy for third place.
Abstract
This report describes the entry by the Intelligent Knowledge Management (IKM) Lab in the WSDM 2019 Fake News Classification challenge. We treat the task as natural language inference (NLI). We individually train a number of the strongest NLI models as well as BERT. We ensemble these results and retrain with noisy labels in two stages. We analyze transitivity relations in the train and test sets and determine a set of test cases that can be reliably classified on this basis. The remainder of test cases are classified by our ensemble. Our entry achieves test set accuracy of 88.063% for 3rd place in the competition.
Kai-Chou Yang, Timothy Niven, Hung-Yu Kao
arXiv:1907.07347 · cs.CL · submitted Jul 17, 2019
abstract · pdf · html