about
Towards Universal Dialogue Act Tagging for Task-Oriented Dialogues (arxiv.org)
2 points by sel1 on Jul 9, 2019 | hide | past | pdf | discuss on HN

In plain words: Different dialogue datasets label conversation moves in different ways, so their labels were merged into one shared set to train a tagger for unlabeled human-human customer chats. With a little labeled target-domain data it scored 57.7% on the standard tagging score, saving over a thousand hand-labeled turns.

Abstract

Machine learning approaches for building task-oriented dialogue systems require large conversational datasets with labels to train on. We are interested in building task-oriented dialogue systems from human-human conversations, which may be available in ample amounts in existing customer care center logs or can be collected from crowd workers. Annotating these datasets can be prohibitively expensive. Recently multiple annotated task-oriented human-machine dialogue datasets have been released, however their annotation schema varies across different collections, even for well-defined categories such as dialogue acts (DAs). We propose a Universal DA schema for task-oriented dialogues and align existing annotated datasets with our schema. Our aim is to train a Universal DA tagger (U-DAT) for task-oriented dialogues and use it for tagging human-human conversations. We investigate multiple datasets, propose manual and automated approaches for aligning the different schema, and present results on a target corpus of human-human dialogues. In unsupervised learning experiments we achieve an F1 score of 54.1% on system turns in human-human dialogues. In a semi-supervised setup, the F1 score increases to 57.7% which would otherwise require at least 1.7K manually annotated turns. For new domains, we show further improvements when unlabeled or labeled target domain data is available.

Shachi Paul, Rahul Goel, Dilek Hakkani-Tür
arXiv:1907.03020 · cs.CL, cs.AI · submitted Jul 5, 2019
abstract · pdf · html · Interspeech 2019

add comment on HN