 |
ML Newsnew | past | best | rss | submit | about |
| The Daunting Task of Real-World Textual Style Transfer Auto-Evaluation (arxiv.org) | |
| 2 points by sel1 on Oct 11, 2019 | hide | past | pdf | discuss on HN | |
|
In plain words: Style rewriting is usually scored on three things: whether the style changed, whether the meaning stayed, and whether it reads smoothly. This review argues those automatic scores are misleading because the test sentences don't match real uses of style rewriting. AbstractThe difficulty of textual style transfer lies in the lack of parallel corpora. Numerous advances have been proposed for the unsupervised generation. However, significant problems remain with the auto-evaluation of style transfer tasks. Based on the summary of Pang and Gimpel (2018) and Mir et al. (2019), style transfer evaluations rely on three criteria: style accuracy of transferred sentences, content similarity between original and transferred sentences, and fluency of transferred sentences. We elucidate the problematic current state of style transfer research. Given that current tasks do not represent real use cases of style transfer, current auto-evaluation approach is flawed. This discussion aims to bring researchers to think about the future of style transfer and style transfer evaluation research. | |
| add comment on HN |