In plain words: By tracking how a name-finding model's features shift while adding email and phone detection, the study explains why old skills survive: semantic and pattern-based features stay separate. Only locations suffered, and the background 'no entity' tag had to be retrained to stop blocking patterns.
Abstract
Extending Named Entity Recognition (NER) models to new PII entities in noisy spoken-language data is a common need. We find that jointly fine-tuning a BERT model on standard semantic entities (PER, LOC, ORG) and new pattern-based PII (EMAIL, PHONE) results in minimal degradation for original classes. We investigate this "peaceful coexistence," hypothesizing that the model uses independent semantic vs. morphological feature mechanisms. Using an incremental learning setup as a diagnostic tool, we measure semantic drift and find two key insights. First, the LOC (location) entity is uniquely vulnerable due to a representation overlap with new PII, as it shares pattern-like features (e.g., postal codes). Second, we identify a "reverse O-tag representation drift." The model, initially trained to map PII patterns to 'O', blocks new learning. This is resolved only by unfreezing the 'O' tag's classifier, allowing the background class to adapt and "release" these patterns. This work provides a mechanistic diagnosis of NER model adaptation, highlighting feature independence, representation overlap, and 'O' tag plasticity. Work done based on data gathered by https://www.papernest.com
Xirui Zhang, Philippe de La Chevasnerie, Benoit Fabre
arXiv:2510.17930 · cs.CL, cs.AI · submitted Oct 20, 2025 · updated Oct 23, 2025
abstract · pdf