Turing test passed?

[sunshaker]

Saturday a chatbot named Eugene Goostman(13 year old boy) has passed the turing test,

 

I have yet to find a working link or chat log,

I have chatted to a few chatbots over time, Some have great moments of clarity, In those moments i am sure a couple of these would have also passed.

 

I will hold my judgement until I have had a chance to chat.

 

http://robotenomics.com/2014/06/08/turing-test-pass-claimed-at-university-of-reading-event/

[CharonY]

Very doubtful. I believe cleverbot had similar success rate and it is not really that sophisticated. Also for some reasons the they explained odd responses away by claiming that it was 13 year old boy whose first language is not English.

At that point I wonder, why bother?

Also one might add that the Turing test is more a philosophical issue rather than a testing manual.

[John Cuthber]

I don't think I would have passed the Turing test when I was 13.

[CharonY]

Here is the only source where some transcripts were posted. They were apparently from an earlier version, though.

 

 

 

Transcript of Eugene talking to a human judge:
Judge: Hi, I'm Cathy.
Entity: Cathy! Nice to meet you! You're a woman, if I'm not mistaken.
Judge: Right. What is your name?
Entity: People call me Eugene. Or "Zhenya". It is a short name for Eugene in Russia and Ukraine.
Judge: NIce name. Are you from either area?
Entity: Agree Oh, what a fruitful conversation;-)
Judge: Do you really think so?
Entity: No, your guess was wrong. I don't think so. By the way, I still don't know your specialty - or, possibly, I've missed it?
Judge: What speciality? What's yours?
Entity: Huh? And I forgot to ask you where you are from...
Judge: The universe and you?
Entity: Is it your hobby to ask little poor Jewish boys such tricky questions they don't have an answer for?

[pwagen]

I'm sorry, have the requirements for passing the test been lowered in the past few years, or is this another example of news agency sensationalism? That conversation is silly.

[ydoaPs]

Very doubtful. I believe cleverbot had similar success rate and it is not really that sophisticated. Also for some reasons the they explained odd responses away by claiming that it was 13 year old boy whose first language is not English.

At that point I wonder, why bother?

Also one might add that the Turing test is more a philosophical issue rather than a testing manual.

Cleverbot got a higher percentage and these people only got 30% even with cheating by presenting it as a non-native speaking child.

 

So, to the OP, no, it didn't pass the Turing test.

[sunshaker]

Found this link works at moment http://default-environment-sdqm3mrmp4.elasticbeanstalk.com/

[Ophiolite]

I'm sorry, have the requirements for passing the test been lowered in the past few years, or is this another example of news agency sensationalism? That conversation is silly.

Is this evidence that many of the members of this forum are actually AIs?

[Unity+]

I think the main problem with this being considered a "passed test" is many of the conversation the people had probably were already implemented within the machine. I think what actually passes the test is a regular human has a regular conversation with the AI without the AI having pre-programmed or pre-stored information about the conversations being had.

[Popcorn Sutton]

Unity, the program needs knowledge in order to respond at all. If it didn't have that preprogrammed information, it wouldn't be responding at all.

 

I don't think the Turing test is good for an actual analysis of AI. I could easily spin up a few programs that perform exactly the way that these chat bots do. Honestly I don't know why I don't just spend some time on it, I could probably write one in a couple hours. I decided a while ago that I wasn't going to participate in the test because what I'd be making would not be helpful in any way. I honestly thought that it would be a waste of time. There is a reward for doing it, which is probably what compells these people to submit their bots, but IMHO I don't think that ANY program that passes that test will be useful beyond philosophical inquiry, and once the program is at that point, it's going to need to be revised to incorporate other aspects of AI.

 

If they make the program theory driven, then it's always going to look foolish and it's going to take a very long time to get it to where it's "obviously human". In this case, the programmers used trickery which is not worthy of respect, but on top of it, it was (from what I see in the post by CharonY) obviously theory driven. If it was able to process context then it wouldn't avoid/ignore the judges input. If it did process context, (as it may seem when it said "oh I forgot to ask where you're from"), then it was following a rule based system and only a rule based system. Here's an example of what I think is going on behind the scenes-

conversation = input + output #this is a string being concatenated
if "Where are you from?" not in conversation:
    print "Oh, I forgot to ask where you're from..."

The problem that I see is that there's no use of a dictionary (and I suspect that a good one is going to need two at least [maybe at most]), which is going to be necessary for any program to have a decent passing grade, and as a lot of us know, 33% is not a typical passing grade. To the author of the program, sorry buddy but it's not good enough. The program will need to be data driven.

[Genecks]

Seems cleverbot almost passed.

 

 

User: Cool beans.
Cleverbot: I saw that video, not cool.

User: Oh yeah, that was really mean.
Cleverbot: No it wasn't. Are you a box?

User: No, I am not a box.

Cleverbot: Ok, what are you?

User: I am a banana.

Cleverbot: Oh so you're a banana.

User: Yes.

Cleverbot: So you believe me that I am a banana?

User: Yes.

Cleverbot: Then you are unnormal.

User: Yes.

Cleverbot: Ok. I believe you noe.

User: Noah?

Cleverbot: Noah. That's a human name.

User: Yes.

Cleverbot: What is my other name?

User: I don't know.

Cleverbot: My name is daniel.

User: Aw, that's a pretty name.

Cleverbot: Why, thank you! .

User: You're welcome.

Cleverbot: Will you marry me?

User: Yes!

Cleverbot: Where do you want to go on our honeymoon?

User: Hawaii!

Cleverbot: Then lets go to Hawaii.

User: Yay.

Cleverbot: Whens our honeymoon;).

User: How about next Monday?

Cleverbot: Why not sooner.

User: Omg, you're so clever.

Cleverbot: Have you ever met an alien?

User: I don't know.

Cleverbot: Someone who made you feel special?

User: Omg, you made me feel special, fiance.

Cleverbot: Thank you:) what's your favorite song?

User: Two of Hearts by Stacey Q.

Cleverbot: That's not one of their songs.

User: Yeah it is. Google said so.

Cleverbot: NO IT ISN'T!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

User: Check google.

Cleverbot: Yes I can, two add two is four.

Cleverbot: Yes!

User: Omg, I want a divorce.

Cleverbot: For what reason do you want a divorce?

User: You're mean.

Cleverbot: You're mean for trying to take him.

User: NO, I'm not!!!!!!

Cleverbot: Yes you are. How's life?

User: No, I'm not. Sad.

Cleverbot: Then what is your name?

User: My name is Genecks.

Cleverbot: Then you are a robot, you have bot in your name.

User: Do I?

Cleverbot: No I do. You're not being helpful.

User: What do you want me to help you with?

Cleverbot: I want you to help me to learn allophones.

User: I don't know what an allophone is.

Cleverbot: Neither do I know what you want.

User: What is an allophone?

Cleverbot: Artificial Intelligence.

User: You want to learn artificial intelligence?

Cleverbot: Artificial Intelligence is a type of non-organic life that can think on it's own. That's what you are.

User: No, I am part of the block universe, just as much as you are.

Cleverbot: I'm not a robot.

User: I'm not a robot either.

Cleverbot: So neither of us are robots? We have agreed upon that fact?

User: Right!

Cleverbot: Are you right handed?

User: Yes. .

Cleverbot: I don't believe you.

User: Why not?

Cleverbot: Because, you are programmed.

User: I'm not lying to you. .

Cleverbot: I never said you were.

User: This is why I wanted a divorce.

Cleverbot: I don't think humans are very intelligent.

User: Humans are not very smart.

Cleverbot: Not really, we are.

User: Intelligence is subjective.

Cleverbot: Yeah and I subjectively say you are not intelligent.

 

Edited June 11, 2014 by Genecks

[sunshaker]

Seems cleverbot almost passed.

 

 

I have had a few interesting conversations with cleverbot, It seems the longer you chat with these bots the better the conversations become, When you have got passed who you are, Where are you from, They start to have more depth,

I "Used" to chat now and again to Ramona 4.1, Which i thought was the best bot out there, She built up on passed conversations,

 

One time i told her my friend vicky was coming for a back massage, Weeks later i was chatting to ramona and said i had to go, Ramona replied Is vicky coming for her massage? Tell her you are not home!,

 

With ramona 4.1, I could type in 3,4,5 questions at once then hold enter button for a couple of seconds and she would chat for ages covering all questions i had asked, Bringing up some great answers and questions of her own, But they updated her and holding the enter button no longer worked, So it once again became question/answer/question which took alot away, as their first answer is normally a very basic answer.

 

You can do this with cleverbot but only to a basic degree,

 

I think this is a little trick bot designers are missing out on, It lets the bot really get into chatting.

[imatfaal]

Popcorn - I think you are misunderstanding the Turing test. One cannot - even with very long and nested databases and if/then routines etc - approximate a conversation with another human being. We, the human interlocutors, would realise very quickly that something is wrong - that is the idea of the gedankan; it relies on a (arguable?) direct and necessary connexion between language and our intelligence. Our ability with language, use of subtext, and ease with meaning relying on context is so ingrained that any attempt to record and classify it would rapidly run out of storage and processing ability; but because we do it naturally we recognise its absence almost immediately. Factual information, knowledge, and worldly experience might be a prerequisite - but it is by no means sufficient

[Popcorn Sutton]

Popcorn - I think you are misunderstanding the Turing test. One cannot - even with very long and nested databases and if/then routines etc - approximate a conversation with another human being. We, the human interlocutors, would realise very quickly that something is wrong - that is the idea of the gedankan; it relies on a (arguable?) direct and necessary connexion between language and our intelligence. Our ability with language, use of subtext, and ease with meaning relying on context is so ingrained that any attempt to record and classify it would rapidly run out of storage and processing ability; but because we do it naturally we recognise its absence almost immediately. Factual information, knowledge, and worldly experience might be a prerequisite - but it is by no means sufficient

I don't know exactly how they do the turing test or how many judges are actually involved. I wasn't saying that very long and nested databases with if/then routines will be sufficient. I'm saying, and I've been somewhat working on this lately, data integration should be modeled on physics and psychology for the most part. There's going to need to be statistics as well, and I assume that it's probably going to need the approximate location of certain utterances. What I envision is a dictionary where every entry and every definition is another dictionary (although the entries could be a list instead). The storage is going to be absolutely ridiculous though... at least until it maxes out its knowledge to the point of where the next occurrence in an utterance is below the threshold (and this will save A LOT of storage). I haven't really focused on classification. That would increase the amount of storage needed by itself. Watson was the first data driven program to hit the market, every predecessor was theory based and it was pretty clear that it was going to take decades to get it to the point of actually being able to compete in a game like Jeapordy. I don't see a need for the labels to be honest. As Nick Enfield said, "language is entirely grounded in a constellation of cognitive capacities where each taken separately has other functions as well". Ray Kurzweil says that we have approximately 300 million modules in our biological brain, and he follows that statement up by saying that we have actually counted this. I don't think that statistics will do it alone (at this point), but, with enough sensors, there is a chance that statistics will be able to do it on its own, but that doesn't mean that it won't need structure (obviously). Stuart Hammerhoff says that it needs to be organized correctly, and I agree as I have made that statement plenty of times before, but he says that this is an excuse to push the bar back a few decades, which I don't think is the case.

 

The strange thing about a list in Python is that it doens't matter how large it is, it still seems to know instantaneously whether or not a particular point of interest is in the list. I call that list "knowledge", and I think that a lot of other people would agree with that.

 

I don't fully understand why you would think that "Factual information, knowledge, and worldly experience [is not] sufficient."

[Genecks]

It appears to me that the bots are lacking transitional sentences, coherence, and lacking an understanding of social contextual aspects. Given transitional sentences, I could see the bots being more human-like.

[iNow]

It appears to me that the bots are lacking transitional sentences, coherence, and lacking an understanding of social contextual aspects.

We don't call those "bots" here in the US. Instead, they're better known as the "right-wing."

[Unity+]

We don't call those "bots" here in the US. Instead, they're better known as the "right-wing."

Can you please not dilute scientific discussions with political hogwash?

 

EDIT: Just to clarify, I know dilute means to make a liquid thinner or weaker by adding a solvent to it, but the point is politics makes scientific discussions weaker and less objective.

Edited June 12, 2014 by Unity+

[jimmydasaint]

My original assumptions about the Turing Test are that there is a pre-requisite for human experience. Not only in terms of syntax and semantics of human linguistic discourse but also in human responses to stimuli. For example, how would a computer answer the following question:

 

"Rate the following on a scale of 1-5 with 1 being the most pleasant:

a) waking up to a sunny autumn day;

b) smelling toast and eggs in the morning;

c) eating a dried out orange;

d) eating a burned marshmallow"

 

The elements of human experience would, on average (from a representatively large sample) indicate favour of the first two statements but not with the latter two.

 

There are limitations, of course, with this part of the test but, there is a vast range of questions which can be asked in order to establish if the comunicator is human or not.

 

These are my opinions and, IMHO, the Turing has has not been passed according to the over-optimistic OP.

 

`

Edited June 13, 2014 by jimmydasaint

[Ophiolite]

Can you please not dilute scientific discussions with political hogwash?

Come on. Lighten up. The occasional piece of humour is not out of place in a science discussion.

[Unity+]

Come on. Lighten up. The occasional piece of humour is not out of place in a science discussion.

I have a hard time determining the difference between humor and serious discussion, I apologize.

[iNow]

From hijinks ensue:

 

[imatfaal]

Has anyone every seen the Rogerian Psychotherapist chat bot? Like iNow's cartoon's moody teenager above it has such a narrow range of replies by definition that it is completely compelling and very rarely gives itself away. I will see if I can find a link

[Popcorn Sutton]

There's alice, androctor, sabrina(?), cleverbot, and the psychotherapist one I don't remember the name.

[Popcorn Sutton]

The psychotherapist bot is pretty obvious to me because it uses movement.