about
Diagnosing Representation Dynamics in NER Model Extension (arxiv.org)
1 point by PaulHoule 328 days ago | hide | past | pdf | discuss on HN

In plain words: By tracking how a name-finding model's features shift while adding email and phone detection, the study explains why old skills survive: semantic and pattern-based features stay separate. Only locations suffered, and the background 'no entity' tag had to be retrained to stop blocking patterns.

Abstract

Extending Named Entity Recognition (NER) models to new PII entities in noisy spoken-language data is a common need. We find that jointly fine-tuning a BERT model on standard semantic entities (PER, LOC, ORG) and new pattern-based PII (EMAIL, PHONE) results in minimal degradation for original classes. We investigate this "peaceful coexistence," hypothesizing that the model uses independent semantic vs. morphological feature mechanisms. Using an incremental learning setup as a diagnostic tool, we measure semantic drift and find two key insights. First, the LOC (location) entity is uniquely vulnerable due to a representation overlap with new PII, as it shares pattern-like features (e.g., postal codes). Second, we identify a "reverse O-tag representation drift." The model, initially trained to map PII patterns to 'O', blocks new learning. This is resolved only by unfreezing the 'O' tag's classifier, allowing the background class to adapt and "release" these patterns. This work provides a mechanistic diagnosis of NER model adaptation, highlighting feature independence, representation overlap, and 'O' tag plasticity. Work done based on data gathered by https://www.papernest.com

Xirui Zhang, Philippe de La Chevasnerie, Benoit Fabre
arXiv:2510.17930 · cs.CL, cs.AI · submitted Oct 20, 2025 · updated Oct 23, 2025
abstract · pdf

add comment on HN