about
NewsEdits 2.0: Learning the Intentions Behind Updating News (arxiv.org)
1 point by PaulHoule on Dec 17, 2024 | hide | past | pdf | discuss on HN

In plain words: A system reads a news draft and predicts which facts will change, trained to tell factual edits apart from style or storytelling ones. Letting a question-answering model skip answers likely to go stale, it nearly matched an ideal system that knows the future.

Abstract

As events progress, news articles often update with new information: if we are not cautious, we risk propagating outdated facts. In this work, we hypothesize that linguistic features indicate factual fluidity, and that we can predict which facts in a news article will update using solely the text of a news article (i.e. not external resources like search engines). We test this hypothesis, first, by isolating fact-updates in large news revisions corpora. News articles may update for many reasons (e.g. factual, stylistic, narrative). We introduce the NewsEdits 2.0 taxonomy, an edit-intentions schema that separates fact updates from stylistic and narrative updates in news writing. We annotate over 9,200 pairs of sentence revisions and train high-scoring ensemble models to apply this schema. Then, taking a large dataset of silver-labeled pairs, we show that we can predict when facts will update in older article drafts with high precision. Finally, to demonstrate the usefulness of these findings, we construct a language model question asking (LLM-QA) abstention task. We wish the LLM to abstain from answering questions when information is likely to become outdated. Using our predictions, we show, LLM absention reaches near oracle levels of accuracy.

Alexander Spangher, Kung-Hsiang Huang, Hyundong Cho, Jonathan May
arXiv:2411.18811 · cs.CL, cs.AI, cs.DL · submitted Nov 27, 2024
abstract · pdf · html · 9 pages main body, 11 pages appendix

add comment on HN