ML Newsnew | past | best | rss | submit about
A Neural Conversational Model [pdf] (arxiv.org)
105 points by sgk284 on Jun 22, 2015 | hide | past | pdf | 39 comments on HN

In plain words: A chatbot learns to reply by predicting the next sentence from the ones before it, trained straight from example conversations instead of hand-written rules. It handled IT helpdesk questions and simple common-sense talk from movie subtitles, though it often contradicts itself.

Abstract · A Neural Conversational Model

Conversational modeling is an important task in natural language understanding and machine intelligence. Although previous approaches exist, they are often restricted to specific domains (e.g., booking an airline ticket) and require hand-crafted rules. In this paper, we present a simple approach for this task which uses the recently proposed sequence to sequence framework. Our model converses by predicting the next sentence given the previous sentence or sentences in a conversation. The strength of our model is that it can be trained end-to-end and thus requires much fewer hand-crafted rules. We find that this straightforward model can generate simple conversations given a large conversational training dataset. Our preliminary results suggest that, despite optimizing the wrong objective function, the model is able to converse well. It is able extract knowledge from both a domain specific dataset, and from a large, noisy, and general domain dataset of movie subtitles. On a domain-specific IT helpdesk dataset, the model can find a solution to a technical problem via conversations. On a noisy open-domain movie transcript dataset, the model can perform simple forms of common sense reasoning. As expected, we also find that the lack of consistency is a common failure mode of our model.

Oriol Vinyals, Quoc Le
arXiv:1506.05869 · cs.CL · submitted Jun 19, 2015 · updated Jul 22, 2015
abstract · pdf · html · ICML Deep Learning Workshop 2015

add comment on HN
Also discussed: Jul 2015 (2 points, 0 comments) · Jun 2015 (6 points, 0 comments)
arthurcolle on Jun 22, 2015 [–]
> Human: what is the purpose of life ?

> Machine: to serve the greater good .

> Human: what is the purpose of living ?

> Machine: to live forever .

> Human: what is the purpose of existence ?

> Machine: to find out what happens when we get to the planet earth .

> Human: where are you now ?

> Machine: i ’m in the middle of nowhere .

> Human: what is the purpose of dying ?

> Machine: to have a life .

> Human: what is the purpose of being intelligent?

> Machine: to find out what it is .

> Human: what is the purpose of emotions ?

> Machine: i don ’t know .

This is intense

reply
dmgbrn on Jun 22, 2015 | parent [–]
> Machine: to find out what happens when we get to the planet earth .

Something tells me the entirety of BSG was included in the training set.

reply
nyxtom on Jun 23, 2015 | parent [–]
They used the OpenSubtitles dataset (in addition to a closed IT Troubleshooting dataset), so it's entirely possible BSG was indeed included :P
reply
gtremper on Jun 23, 2015 | parent [–]
> Human: where are you now ?

> Machine: Kobol .

reply
sswaner on Jun 23, 2015 | parent [–]
What is BSG?
reply
kdvh on Jun 23, 2015 | parent [–]
Battlestar Galactica.
reply
codemac on Jun 22, 2015 [–]
This scares me because all my interactions with google support have been so incredibly bad, it makes me nervous to train a computer on that corpus of bad data.

Taking bad support and turning it into inaccurate bad support is a fantastically bad idea.

reply
vidarh on Jun 22, 2015 | parent [–]
Some might suggest that Google support has been run by AI's for years already, given just how awful some of the responses are...
reply
vasco on Jun 22, 2015 [–]
> Machine: hi , this is <NAME> from helpdesk connect , what can i help you with today ?

> Human: hi , i forgot my password

> Machine: ok great

That's something unexpected to hear from tech support.

reply
hobs on Jun 22, 2015 | parent [–]
When I worked in tech support, I cant tell you how many times I would say "Great! (realize that its not great for them) that we have isolated the issue."
reply
oh_sigh on Jun 22, 2015 | parent [–]
"ok great" can mean "Okay great, I know exactly how to fix your problem!"
reply
dsugarman on Jun 22, 2015 [–]
> Human: what is immoral ?

> Machine: the fact that you have a child .

harsh

reply
notum on Jun 22, 2015 [–]
Quoc V. Le also helped develop Word2Vec, released as opensource couple of years ago: https://code.google.com/p/word2vec/

You can play with it here: http://rare-technologies.com/word2vec-tutorial/ (a bit further down)

Google seems rather heavily invested in NLP, can't blame them as it's a very exciting domain to explore.

reply
smhx on Jun 23, 2015 | parent [–]
Quoc Le is neither an author of the word2vec paper or in the committers list of word2vec software (there's only two committers, mikolov and sutskever). Not sure where you got that inference from.
reply
andrewtbham on Jun 23, 2015 | parent [–]
http://www.wired.com/2014/12/googlers-quest-teach-machines-u...

At Google, he helped develop a system that essentially maps words into vectors. And according to Google, this work would later feed into a system developed largely by a researcher named Tomas Mikolov. Called Word2Vec, the system determines how different words on the web are related, and Google is now using this as a means of strengthening its “knowledge graph”

reply
throwaway_bob on Jun 23, 2015 | parent [–]
Just because wired says something doesn't make it true. Whatever other accomplishments he may have, Le was not a contributor to word2vec, and he did not originate the idea of mapping words to vectors.
reply
kastnerkyle on Jun 23, 2015 | parent [–]
I think the poster got mixed up. Quoc Le was the first author on "Distributed Representations of Sentences and Documents" aka paragraph2vec, so he has been involved in the x2vec scene. Just not the word2vec. And I would argue that word2vec did not originate the idea either, just popularized it and showed the power of such an approach (plus an awesome demo!).

The original paper on word embeddings from 2001 from Bengio et. al. "A Neural Probabilistic Language Model"[1] is the first I am aware of.

[1] http://papers.nips.cc/paper/1839-a-neural-probabilistic-lang...

reply
Nadya on Jun 22, 2015 [–]
The conversations are rather enlightening.

>What is the color of water? Water.

I'm quite puzzled how it knows "two plus two" is four but "ten minus two" is "seventy-two". I wonder how it parsed one correctly but failed drastically to parse the other.

reply
biot on Jun 22, 2015 | parent [–]
It's not parsing the phrase into the tokens "10 - 2" and evaluating that. Rather, a corpus of data was used to train the neural net and, for whatever reason, the answer "seventy-two" was the highest probability output. In all likelihood, it doesn't even register that this is a math question.
reply
murbard2 on Jun 22, 2015 | parent [–]
It did not see enough examples to learn a general addition algorithm. "Two plus two equals four" is a common sentence, "Ten minus two equals eight" isn't.

In fact, it's likely that no one learns addition by listening to examples, you're taught to execute a specific algorithm and it takes many people years to master it. In fact, ask a toddler what "ten minus two" is and you might get a similar nonsensical answer.

reply
vidarh on Jun 22, 2015 | parent [–]
Just working through maths problems with my 6 year old son on a daily basis now, and it's oh-so-hard not to face palm in front of him. So many things that seems incredibly obvious are hard learned lessons. One of the fascinating things of helping him practice is that it's helping me recognise the sheer number of simple little shortcuts I apply to even something as basic as addition (and incredibly hard to hold back and give him the chance to practice the basics before I show him the "cheats").
reply
Nadya on Jun 22, 2015 | parent [–]
I hadn't thought of the "common sentence" approach. Woops.

I was thinking semi-intelligent parsing and it recognized "two" and "plus" and created a formula 2+2 then solved it with 4 ("four").

So I had assumed it would have done the same with "ten", "minus", and "two" to create 10-2, solve it, and respond with 8 ("eight").

If you use Google Search and search for "ten minus two" it's intelligent enough to parse the search and give you a calculator with 10-2=8 already inputted. I had assumed a similar parsing approach may have been used for their conversation bot in regards to general mathematical knowledge.

But your explanation makes a ton of sense. :)

reply
reddytowns on Jun 23, 2015 | parent [–]
I don' think it could ever learn a general addition algorithm, because the algorithm they are using doesn't allow for a multi-step process. For example, to add 28 + 37, a simple approach would be:

  1. first, look at the one's column and recall that 8 + 7 is 15.
  2. Then take the value from the tens column of the answer
  and add it to the tens column of the arguments.
But given the command : add 28 + 37, it can only execute a single step. It has no way of guessing at and then storing the values for the intermediate command, (take the 1 from the 15 and add it to the 2 from the 28 and the 3 from the 37), to be processed in the next round. Because so it can't ever generalize addition.
reply
Smerity on Jun 23, 2015 | parent [–]
Whilst you are correct that the number of allowed computation steps is limited, LSTMs have still been used with success for such tasks. LSTMs have been used to perform addition with 99% accuracy^ on two 9-digit numbers[1]. The paper even shows that simple Python programs can be evaluated with some degree of accuracy.

Remember that even if the number of computation steps is limited, there can be multiple layers ([1] uses 2) and each neuron unit can perform a computation ([1] uses 400 cells per layer). It only needs to learn how to be an ALU. The work is in fact done by one of the people who established the sequence to sequence framework, Sutskever, and is referenced in the Neural Conversational Model paper.

^ They use "teacher forcing" for evaluation which inflates the accuracy to some degree, but it's still quite impressive.

[1]: http://arxiv.org/pdf/1410.4615v3.pdf

reply
murbard2 on Jun 23, 2015 | parent [–]
Now for the kicker: recurrent network can implement arbitrary algorithms, including a type of GOFAI. It's quite possible that such an algorithm isn't learnable without supervision; that it was discovered by sheer luck at the dawn of humanity and has been passed down by language ever since.
reply
derefr on Jun 22, 2015 | parent [–]
> What is the color of water? Water.

Literally true in Japanese. The color of water is 水色 (mizu-iro: lit. "water-color.") The analogy then proceeds in the other direction: light blue things are considered "water-colored", rather than water being "light" "blue".

reply
Nadya on Jun 22, 2015 | parent [–]
Technically it is 水の色 (water's color). Often times particles like 「は・が・の」 are dropped when the context allows for such, making 水の色 turn into 水色.

Same thing happens with 葉の色 becoming 葉色 (leaf green). The の is implied.

In English one might say that something is "sky blue" instead of saying "the color of the sky". So it's not uniquely Japanese in that sense.

reply
teraflop on Jun 23, 2015 | parent [–]
Technically, 水色 appears in both Japanese and Japanese-English dictionaries, so I'd say it's a word by any reasonable definition. (Disclaimer: not a native speaker.)
reply
Nadya on Jun 23, 2015 | parent [–]
It's a word in its own right - but the etymology of it is still important and I feel it was slightly misrepresented.

For example, the greeting many people know 「こんにちは」 (konnichiwa) is a word in its own right nowadays, but is really just the beginning of an older greeting: 今日は御機嫌いかがですか

reply
paulsutter on Jun 23, 2015 | parent [–]
Dropping の can create a different meaning. For example, 日本料理 (Japanese cuisine) and 日本の料理　(cooking in Japan, which could well be used to refer to the quality of the italian or chinese food here). The former is truly a single word and the latter two words separated by a particle.
reply
andrewtbham on Jun 23, 2015 [–]
this paper, also by quoc le, might also be of interest...

http://papers.nips.cc/paper/5346-sequence-to-sequence-learni...

reply
ocdtrekkie on Jun 22, 2015 [–]
Is this going to end up making it even harder for me to get to an actual human on the phone?
reply
gpmcadam on Jun 22, 2015 | parent [–]
Or maybe it'll be good enough that you won't need one!
reply
fapjacks on Jun 22, 2015 [–]
"Please unplug your modem's power from the wall, then reboot your computer. Thank you. Your Wallet will be charged twenty four dollars for today's support request. Have a nice day!"
reply
feld on Jun 22, 2015 [–]
I think this could be successfully trained to walk many people through all kinds of basic support issues... using years of tech support chat logs would probably make this a breeze.
reply
joelthelion on Jun 22, 2015 | parent [–]
One problem is that when the machine says it's doing something, it's actually not doing anything. The chat logs don't contain information about the actions of the tech.
reply
feld on Jun 23, 2015 | parent [–]
The chat logs might not, but I'm sure the actions of the representative are logged somewhere and they could be reconciled.
reply
thezach on Jun 22, 2015 [–]
is it plugged in?

did you restart it?

reply
cafebeen on Jun 22, 2015 [–]
Interesting, but it strikes me as a work in progress... Curious to see where it goes though!
reply
About | RSS | RSS (all) | HN arXiv