about
Cross-Lingual Alignment Without Joint Training (arxiv.org)
1 point by codelion 22 days ago | hide | past | pdf | discuss on HN

In plain words: Separately trained monolingual models were compared to see if their representations line up without shared data or training. They do: one rotation maps internal states between models, and swapping in a rotated English state makes a German model answer with the English model's capital.

Abstract · Cross-Lingual Alignment Without Joint Training: Do Monolingual Language Models Converge on Universal Representations?

Cross-lingual alignment in multilingual language models is typically attributed to joint training: shared parameters, mixed-language batches, or explicit alignment objectives. We ask whether monolingual models trained on non-parallel data learn alignable representations without joint training. By testing on strictly monolingual language models, such as the Goldfish model families and independently developed models from different research labs, we find three results. Correlation: these models develop alignable representational geometry across layers, with alignment strengthening as data scale, model scale, or linguistic proximity increases. Construction: a single Procrustes rotation fit on parallel sentences maps hidden states between models. Causation: the same rotation transfers functional content; patching a rotated English residual into a German model on a factual cloze flips the prediction to the donor's capital in most cases. We confirm that cross-lingual alignment can emerge from the structure of language and the information it carries rather than from joint training, and this points to practical future directions including model stitching, merging, and modular multilingual systems built from monolingual components.

Ej Zhou, Suchir Salhan, Catherine Arnett, Anna Korhonen
arXiv:2608.27115 · cs.CL · submitted Aug 27, 2026
abstract · pdf · html

add comment on HN