Article
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

In this paper, we look at a specific issue with practical Turing tests, namely the right of the machine to remain silent during interrogation. In particular, we consider the possibility of a machine passing the Turing test simply by not saying anything. We include a number of transcripts from practical Turing tests in which silence has actually occurred on the part of a hidden entity. Each of the transcripts considered here resulted in a judge being unable to make the ‘right identification’, i.e., they could not say for certain which hidden entity was the machine.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... In this section we explain briefly how it is quite possible for a machine to pass the Turing test not by its apparent skill at human conversation but rather by simply remaining silent throughout [40]. Rather than being a mere theoretical or philosophical quirk it turns out that in fact passing the Turing test in this way also has an underlying practical basis to support it with numerous examples to boot. ...
... Essentially the machine makes no utterances which give the game away that they are a machine and hence the judges involved have no evidence to use against them. This whole issue of the strategy of silence is discussed at length in Warwick and Shah [40]. ...
... The example given here is just that, an example, as there are numerous other cases reported on in Warwick and Shah [40]. An interesting feature is the response of the interrogators involved with those particular transcripts. ...
Article
Full-text available
In this paper we look at the phenomenon that is the Turing test. We consider how Turing originally introduced his imitation game and discuss what this means in a practical scenario. Due to its popular appeal we also look into different representations of the test as indicated by numerous reviewers. The main emphasis here, however, is to consider what it actually means for a machine to pass the Turing test and what importance this has, if any. In particular does it mean that, as Turing put it, a machine can “think”. Specifically we consider claims that passing the Turing test means that machines will have achieved human-like intelligence and as a consequence the singularity will be upon us in the blink of an eye.
... Alan Turing (Turing, 1950) believed that a machine might pretend to be intelligent like a human if it provides appropriate and sustained responses to any questions in a similar to human manner. Much later, Warwick & Shah (2017) demonstrate that, actually, the truly intelligent machine is the one that knows also when and why be silent. Their experiments show that a machine sometimes passes the Turing test simply by not saying anything. ...
Preprint
Full-text available
Operating with ignorance is an important concern of the Machine Learning research, especially when the objective is to discover knowledge from the imperfect data. Data mining (driven by appropriate knowledge discovery tools) is about processing available (observed, known and understood) samples of data aiming to build a model (e.g., a classifier) to handle data samples, which are not yet observed, known or understood. These tools traditionally take samples of the available data (known facts) as an input for learning. We want to challenge the indispensability of this approach and we suggest considering the things the other way around. What if the task would be as follows: how to learn a model based on our ignorance, i.e. by processing the shape of 'voids' within the available data space? Can we improve traditional classification by modeling also the ignorance? In this paper, we provide some algorithms for the discovery and visualizing of the ignorance zones in two-dimensional data spaces and design two ignorance-aware smart prototype selection techniques (incremental and adversarial) to improve the performance of the nearest neighbor classifiers. We present experiments with artificial and real datasets to test the concept of the usefulness of ignorance discovery in machine learning.
... Turing himself addressed several of these objections and a good summary of these arguments can be found in (Oppy and Dowe 2016). One important limitation of imitation games is also that it might be possible to pass them merely by remaining silent (Warwick and Shah 2016d). A limited version of the Turing test has recently been passed by a computing machine named 'Eugene Goostman' (Warwick and Shah 2016a). ...
Article
Full-text available
Since Turing proposed the first test of intelligence, several modifications have been proposed with the aim of making Turing’s proposal more realistic and applicable in the search for artificial intelligence. In the modern context, it turns out that some of these definitions of intelligence and the corresponding tests merely measure computational power. Furthermore, in the framework of the original Turing test, for a system to prove itself to be intelligent, a certain amount of deceit is implicitly required which can have serious security implications for future human societies. In this article, we propose a unified framework for developing intelligence tests which takes care of important ethical and practical issues. Our proposed framework has several important consequences. Firstly, it results in the suggestion that it is not possible to construct a single, context independent, intelligence test. Secondly, any measure of intelligence must have access to the process by which a problem is solved by the system under consideration and not merely the final solution. Finally, it requires an intelligent agent to be evolutionary in nature with the flexibility to explore new algorithms on its own.
... As has been noted this is not enough to give the chatbot legitimacy, but it may be enough to stop it from being identified. It is interesting to compare this analysis to that of Shah [15] where the strategy of silence for a chatbot is considered, and certainly in creating wildcard responses for the bot (and the sampling of exchanges that was taking place) then having a silence response rather than risking making an erroneous responses was certainly one of the strategies employed. ...
Conference Paper
Full-text available
It has been suggested that a covert Turing Test, possibly in a virtual world, provides a more level playing field for a chatbot, and hence an earlier opportunity to pass the Turing Test (or equivalent) in its overt, declared form. This paper looks at two recent covert Turing Tests in order to test this hypothesis. In one test (at Loyola Marymount) run as a covert-singleton test, of 50 subjects who talked to the chatbot avatar 39 (78 % deception) did not identify that the avatar was being driven by a chatbot. In a more recent experiment at the University of Worcester groups of students took part in a set of problem-based learning chat sessions, each group having an undeclared chatbot. Not one participant volunteered the fact that a chatbot was present (a 100 % deception rate). However the chatbot character was generally seen as being the least engaged participant—highlighting that a chatbot needs to concentrate on achieving legitimacy once it can successfully escape detection.
Article
Full-text available
Operating with ignorance is an important concern of geographical information science when the objective is to discover knowledge from the imperfect spatial data. Data mining (driven by knowledge discovery tools) is about processing available (observed, known, and understood) samples of data aiming to build a model (e.g., a classifier) to handle data samples that are not yet observed, known, or understood. These tools traditionally take semantically labeled samples of the available data (known facts) as an input for learning. We want to challenge the indispensability of this approach, and we suggest considering the things the other way around. What if the task would be as follows: how to build a model based on the semantics of our ignorance, i.e., by processing the shape of “voids” within the available data space? Can we improve traditional classification by also modeling the ignorance? In this paper, we provide some algorithms for the discovery and visualization of the ignorance zones in two-dimensional data spaces and design two ignorance-aware smart prototype selection techniques (incremental and adversarial) to improve the performance of the nearest neighbor classifiers. We present experiments with artificial and real datasets to test the concept of the usefulness of ignorance semantics discovery.
Thesis
Full-text available
Deception-detection is the crux of Turing’s experiment to examine machine thinking conveyed through a capacity to respond with sustained and satisfactory answers to unrestricted questions put by a human interrogator. However, in 60 years to the month since the publication of Computing Machinery and Intelligence little agreement exists for a canonical format for Turing’s textual game of imitation, deception and machine intelligence. This research raises from the trapped mine of philosophical claims, counter-claims and rebuttals Turing’s own distinct five minutes question-answer imitation game, which he envisioned practicalised in two different ways: a) A two-participant, interrogator-witness viva voce, b) A three-participant, comparison of a machine with a human both questioned simultaneously by a human interrogator. Using Loebner’s 18th Prize for Artificial Intelligence contest, and Colby et al.’s 1972 transcript analysis paradigm, this research practicalised Turing’s imitation game across three original experiments with over 400 human participants and 13 machines. Results show the current state of artificial dialogue can recall information and share personal interests presenting an illusion of personality. The technology achieved a deception rate of 8.33% in 60 machine-human simultaneous comparison tests. Results also show that more than 1 in 3 Transcript Analysts reviewing five transcripts involving Elbot, winner of the 2008 Loebner Prize bronze award for ‘most human-like’ machine, were unable to correctly identify artificial dialogue. Deception-detection is essential to uncover the increasing number of malfeasant programmes, such as CyberLover, developed to steal identity and financially defraud users in chatrooms across the Internet. Practicalising Turing’s two tests can assist in raising awareness of this risk and preventing cybercrime.
Article
Full-text available
In this article we consider transcripts that originated from a practical series of Turing's Imitation Game that was held on 6 and 7 June 2014 at the Royal Society London. In all cases the tests involved a three-participant simultaneous comparison by an interrogator of two hidden entities, one being a human and the other a machine. Each of the transcripts considered here resulted in a human interrogator being fooled such that they could not make the ‘right identification’, that is, they could not say for certain which was the machine and which was the human. The transcripts presented all involve one machine only, namely ‘Eugene Goostman’, the result being that the machine became the first to pass the Turing test, as set out by Alan Turing, on unrestricted conversation. This is the first time that results from the Royal Society tests have been disclosed and discussed in a paper.
Article
Full-text available
Manuscript ID: TETA-2012-0146.R2 This paper presents some important issues on misidentification of human interlocutors in text-based communication during practical Turing Tests. The study here presents transcripts in which human judges succumbed to the confederate effect, misidentifying hidden human foils for machines. An attempt is made to assess the reasons for this. The practical Turing tests in question were held on 23rd June 2012 at Bletchley Park, England. A selection of actual full transcripts from the tests is shown and an analysis is given in each case. As a result of these tests, conclusions are drawn with regard to the sort of strategies which can perhaps lead to erroneous conclusions when one is involved as an interrogator. Such results also serve to indicate conversational directions to avoid for those machine designers who wish to create a conversational entity that performs well on the Turing Test. Free to download, as at 15 June 2014 from here: http://www.tandfonline.com/action/.U53BzGfjhWA#.U53DV9SwXMU
Conference Paper
Full-text available
This paper presents an analysis of three major contests for machine intelligence. We conclude that a new era for Turing's test requires a fillip in the guise of a committed sponsor, not unlike DARPA, funders of the successful 2007 Urban Challenge. 12
Conference Paper
Full-text available
This paper presents a pilot study involving 99 participants analysing conversations between Judges 2, 4 and 7, hidden humans (Confederates), and Jabberwock, bronze prize winner for most human-like machine from Loebner's 2003 Contest, instantiation of Turing's Test for machine intelligence. The transcripts from these conversations were given to children (aged between 8 and 12), and adults (aged between 18-35). The machine was identified in its conversation with Judge 7, but the Confederate Effect featured in the decisions regarding the nature of the Judges and Confederates, who were both sometimes considered machine-like from their textual discourse. Creators of Jabberwock-type systems may find results presented here useful in their human-machine textual interaction designs.
Conference Paper
Full-text available
One of the differences between natural conversational entities – NCE (humans) and artificial conversational entities – ACE (such as Carpenter’s Jabberwacky), is the ability the former have to constrain random output during dialogue. When humans want to participate in, and pursue conversation with each other they maintain coherent dialogue through contextual relevance, create metaphors – fusing seemingly unrelated ideas to ensure abstract points are understood, and indicate topic change at mutually acceptable junctures.
Conference Paper
Full-text available
One feature of 'humanness' that Turing did not factor into his imitation game for machine thinking and intelligence is that mistakes will be made by some of the human interrogators, and others are easily fooled. Those that err and those susceptible to deception include 'experts' who cannot be described as unintelligent as a result of their judgement. Turing described two imitation game scenarios: i) a 3-participant test involving a human interrogator questioning two hidden interlocutors simultaneously (STT) and decides which is human and which is machine; ii) 2-participant viva voce test (VVTT) involving the interrogator questioning one hidden interlocutor deciding whether it is human or machine. In this paper we summarise errors from 120 STTs conducted by Reading University, UK. 276 Turing tests have been conducted so far by the authors (96 tests at Reading University, 2008 i ii ; 180 tests at Bletchley Park, 2012 iii iv). Human interrogators and hidden humans in those tests included members of the public, experts, males/females, adults/ teenagers, and native/non-native UK English speakers. What we focus on here are the findings from the 120 machine-human simultaneous Turing tests and four types of errors that occurred in them (see Table 1):
Conference Paper
Full-text available
Passing the Turing Test is not a sensible goal for Artificial Intelligence. Adherence to Turing's vision from 1950 is now actively harmful to our field. We review problems with Turing's idea, and suggest that, ironically, the very cognitive science that he tried to create must reject his research goal. 1
Article
The Turing Test has captured the imagination of the general public due to fundamental questions about the nature of the mind. But Stevan Harnad argues the hype over the supposed passing of the Turing Test is misplaced. Alan Turing’s idea for cognitive science was simple: Stop worrying about what the mind “is” and explain instead what the mind does. But we are nowhere near having designed a system that can do everything a person with a mind can do.
Article
Artificial Intelligence (AI) is the study of enabling computers to reason, make judgements, and learn. Timesharing, networking, window systems, and the mouse all resulted from AI research. Throughout the western world, classes and seminars in AI have filled to capacity. Interest in AI has reached fad-like proportions. Current interest in AI reflects a widespread realization that making computers think like people is the next logical conquest in that revolution.
Article
When judging the capabilities of technology, different humans can have very different perspectives and come to quite diverse conclusions over the same data set. In this paper we consider the capabilities of humans when it comes to judging conversational abilities, as to whether they are conversing with a human or a machine. In particular the issue in question is the importance of human judges interrogating in practical Turing tests. As supportive evidence for this we make use of transcripts which originated from a series of practical Turing’s tests held 6-7 June 2014 at the Royal Society London. Each of the tests involved a 3-participant simultaneous comparison by a judge of two hidden entities, one being a human and the other a machine. Thirty different judges took part in total. Each of the transcripts considered in the paper resulted in a judge being unable to say for certain which was the machine and which was the human. The main point we consider here is the fallibility of humans in deciding whether they are conversing with a machine or a human, hence we are concerned specifically with the decision-making process
Article
A series of imitation games involving 3-participant (simultaneous comparison of two hidden entities) and 2-participant (direct interrogation of a hidden entity) were conducted at Bletchley Park on the 100th anniversary of Alan Turing’s birth: 23 June 2012. From the ongoing analysis of over 150 games involving (expert and non-expert, males and females, adults and child) judges, machines and hidden humans (foils for the machines), we present six particular conversations that took place between human judges and a hidden entity that produced unexpected results. From this sample we focus on features of Turing’s machine intelligence test that the mathematician/code breaker did not consider in his examination for machine thinking: the subjective nature of attributing intelligence to another mind.
Article
Bien qu'importante, la question: L'ordinateur, pense-t-il? n'est pas la seule question sur la nature de la pensee posee, consciemment ou inconsciemment, par Turing dans son texte, extremement complexe et riche « Computing machinery and intelligence » (1950). Moins apparentes, mais non pas moins importantes d'apres l'A., sont les questions sur la nature du sexe, le naturel et l'artificiel, l'analogue et le discret, le biologique et le culturel
Article
The test Turing proposed for machine intelligence is usually understood to be a test of whether a computer can fool a human into thinking that the computer is a human. This standard interpretation is rejected in favor of a test based on the Imitation Game introduced by Turing at the beginning of "Computing Machinery and Intelligence."
Book
In 1872, a woman known only as 'An Earnest Englishwoman', published an open letter entitled 'Are women animals?' She protested that women were not treated as fully human; their status was worse than that of animals. What does it mean to be 'human' rather than 'animal'? If the Earnest Englishwoman had turned her gaze to the previous century, her critique could have applied to slaves. Exploring the legacy of more than two centuries, this meticulously researched and illuminating book of history examines the ever shifting line drawn between the human and the animal.
Chapter
I propose to consider the question, “Can machines think?”♣ This should begin with definitions of the meaning of the terms “machine” and “think”. The definitions might be framed so as to reflect so far as possible the normal use of the words, but this attitude is dangerous. If the meaning of the words “machine” and “think” are to be found by examining how they are commonly used it is difficult to escape the conclusion that the meaning and the answer to the question, “Can machines think?” is to be sought in a statistical survey such as a Gallup poll.
Article
Based on insufficient evidence, and inadequate research, Floridi and his students report inaccuracies and draw false conclusions in their Minds and Machines evaluation, which this paper aims to clarify. Acting as invited judges, Floridi et al. participated in nine, of the ninety-six, Turing tests staged in the finals of the 18th Loebner Prize for Artificial Intelligence in October 2008. From the transcripts it appears that they used power over solidarity as an interrogation technique. As a result, they were fooled on several occasions into believing that a machine was a human and that a human was a machine. Worse still, they did not realise their mistake. This resulted in a combined correct identification rate of less than 56%. In their paper they assumed that they had made correct identifications when they in fact had been incorrect.
Article
Purpose The purpose of this paper is to consider Alan Turing's philosophical paper on “Computing machinery and intelligence”, in which he defined the “imitation game”, now usually known as the Turing test. Design/methodology/approach The paper shows that Turing's paper contained more than the test; it contained a wide survey of what a computer could do and its relation to human thought. Findings This paper discusses how Turing's paper represented the outcome of many years in which Turing has both developed the concept and the design of the digital computer, and considered how its action could be related to human thought. Originality/value Analysis of Turing's paper provides an understanding and appreciation of Turing's contributions and the significance of the Turing test.
Article
Purpose – The purpose of this paper is to consider Turing's two tests for machine intelligence: the parallel‐paired, three‐participants game presented in his 1950 paper, and the “jury‐service” one‐to‐one measure described two years later in a radio broadcast. Both versions were instantiated in practical Turing tests during the 18th Loebner Prize for artificial intelligence hosted at the University of Reading, UK, in October 2008. This involved jury‐service tests in the preliminary phase and parallel‐paired in the final phase. Design/methodology/approach – Almost 100 test results from the final have been evaluated and this paper reports some intriguing nuances which arose as a result of the unique contest. Findings – In the 2008 competition, Turing's 30 per cent pass rate is not achieved by any machine in the parallel‐paired tests but Turing's modified prediction: “at least in a hundred years time” is remembered. Originality/value – The paper presents actual responses from “modern Elizas” to human interrogators during contest dialogues that show considerable improvement in artificial conversational entities (ACE). Unlike their ancestor – Weizenbaum's natural language understanding system – ACE are now able to recall, share information and disclose personal interests.
Book
Decades of scientific research on neurophysiology have proven emotions are not simply a minor aspect of human activity, but rather a fundamental one. The Handbook of Research on Synthetic Emotions and Sociable Robotics: New Applications in Affective Computing and Artificial Intelligence focuses on the integration of emotions into artificial environments such as computers and robotics. Written by an international collaboration of experts within the field, this Handbook of Research covers topics such as emotion simulation and emotion synthetic development.
The essential Turing-The ideas that gave birth to the computer age
  • B J Copeland
Emotion in the Turing test: A downward trend for machines in recent Loebner prizes
  • H Shah
  • K Warwick
The status and future of the Turing test The Turing test-The elusive standard of artificial intelligence
  • J H Moor
What kind of Turing test did Turing have in mind? Tekhnema 3/A Touch of memory, Spring. Retrieved from http
  • J Lassègue
Conversation, deception and intelligence: Turing’s question-answer game Alan Turing: His life and impact. Part III building a brain: Intelligent machines, practice and theory
  • H Shah
Turing’s two tests for intelligence The Turing test-The elusive standard of artificial intelligence
  • S G Sterrett
Lecture on the automatic computing engine The essential Turing: The ideas that gave birth to the computer age
  • A M Turing