 |
ML Newsnew | past | best | rss | submit | about |
| Bert Has a Mouth, and It Must Speak:Bert as a Markov Random Field Language Model (arxiv.org) | |
| 3 points by iron0013 on Feb 21, 2019 | hide | past | pdf | 1 comment on HN | |
|
In plain words: Treating BERT as a web of word-to-word constraints, rather than a left-to-right writer, makes it possible to sample whole sentences from it. The sentences it produces are more varied than those from a traditional left-to-right model, though slightly lower in quality. Abstract · BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language ModelWe show that BERT (Devlin et al., 2018) is a Markov random field language model. This formulation gives way to a natural procedure to sample sentences from BERT. We generate from BERT and find that it can produce high-quality, fluent generations. Compared to the generations of a traditional left-to-right language model, BERT generates sentences that are more diverse but of slightly worse quality. | |
| add comment on HN |
via https://twitter.com/W4ngatang