Learn About The Turing Test: Who’s a Bot?

The concept of Artificial Intelligence is not a new one. In fact, it has rattled around our collective imaginations for decades. Even the famous French philosopher, René Descartes, discussed the idea of machines being able to think in his Discourse on the Method.

While we still have a ways to go before a sentient AI like HAL 9000 becomes a part of our daily reality, there is no doubt that our devices are becoming smarter every day.

Personal assistants like Siri and Google Assistant may not be a true AI but they can make our lives easier, either by reminding us of our next appointment or showing us the shortest route to our chosen destination. On the other end of the spectrum, there are actual chatbots that are capable of fooling people into falling in love with them.

Does that mean that Alan Turing’s prediction of computers becoming intelligent and fooling humans is finally coming true?

Who Was Alan Turing

Alan Turing is nowadays considered the father of theoretical computer science and artificial intelligence. He was an English mathematician, logician, computer scientist, cryptanalyst, and theoretical biologist.

His first notable invention was the Turing Machine which was able to simulate the logic of a given algorithm. This in turn led to the idea of the “Universal Turing Machine” a single machine that would be able to compute any algorithm.

During the Second World War, Turing worked for the Government Code and Cypher School (GC&CS) at Bletchley Park. While there, he devised a number of techniques for breaking the German encrypted messages which helped the Allies to defeat the Nazis in many crucial battles.

After the war was over, he worked at the National Physical Laboratory before joining Max Newman’s Computing Machine Laboratory at the Victoria University of Manchester. It was there that he became interested in mathematical biology and morphogenesis which led to a paper on the chemical basis of morphogenesis. The paper also predicted oscillating chemical reactions which were first observed in the 1960s.

He is also responsible for the first chess program written for a computer as well as the LU Decomposition Method which is used today for solving matrix equations.

In 1952, he was prosecuted for homosexual acts and accepted the chemical castration treatment. He died two years later from cyanide poisoning which was ruled a suicide with a note that it might have been accidental poisoning. In 2009, British Prime Minister Gordon Brown made an official public apology on behalf of the British government for “the appalling way he was treated” which was followed by a posthumous pardon by Queen Elizabeth in 2013.

More Information:

• Alan Turing: The Enigma: the official homepage for Alan Turing and an online companion for the biography by the same name written and maintained by Andrew Hodges.
• Royal pardon for codebreaker Alan Turing: this article describes the Internet campaign which led to the official pardon for Turing which came into effect on December 24, 2013.
• Alan Turing: Inquest’s suicide verdict “not supportable”: a BBC article which explains why Jack Copeland, a leading expert on Turing, believes the evidence in Turing’s case would not be accepted as sufficient to establish a suicide verdict.
• Alan Turing: The Enigma: The Book That Inspired the Film “The Imitation Game” (2014) by Andrew Hodges: the official written biography of Alan Turing, the book tells a story of how his novel idea of a universal machine gave rise to modern computer science along with insight into breaking the German cypher and Turing’s personal life.
• The Man Who Knew Too Much: Alan Turing and the Invention of the Computer (2006) by Leavitt David: this book provides an in-depth look at Turing’s life, his work, and its implications.

The Origins of the Turing Test

The Turing test was developed in 1950. The basis for it is described in Turing’s paper, Computing Machinery and Intelligence which proposes the idea of machines being able to think.

Turing first suggested that in order to consider this question, a definition needs to be provided for both the terms “machine” and “thinking.” However, since it would be hard to define “thinking” Turing chose not to answer the original question, but rather replace it with a new question “which is closely related to it and is expressed in relatively unambiguous words.”

He described this new question as a variation of a simple imitation game in which there are three participants: a man (A), a woman (B), and an interrogator (C). The interrogator is in a separate room and has to determine which of the participants is the man and which one is the woman.

The interrogator can ask questions and the objective of person A is to try and confuse the interrogator while person B is allowed to provide answers that would help the interrogator. The answers are sent to the interrogator via a teleprinting machine.

Turing’s version of the game would replace person A with a machine while the role of person B is still performed by a woman. If the machine could convince the interrogator that it’s a woman, it would pass the test.

Turing originally predicted that this would happen in about 30% of the cases. The test relies heavily on natural language processing and it does not check the machine’s ability to give correct answers to questions, only how closely answers resemble those of a human.

His test was subject to a lot of criticism which Turing addressed in his paper and gave his own opinion on why he considered the objections invalid. Nevertheless, his paper became an important contribution in artificial intelligence research.

Later Additions and Variations

Later on in the article, Turing suggests replacing person A with a computer while the role of person B is performed by a man. In this version of the test, both the computer and the man are trying to trick the interrogator into making the wrong decision.

A number of modifications appeared throughout the years. The most famous one is the Reverse Turing Test, where the roles are reversed and a human has to be able to convince the computer that it’s interacting with a human rather than another computer. The best example is the use of CAPTCHA where a person is presented with alphanumerical characters within an image that’s been distorted and asked to type them out as a means of preventing a bot from gaining access. The idea behind the test is that there is no such software that could accurately read and reproduce the distorted image.

The Total Turing Test variation was proposed by Stevan Harnad and it added two further requirements to the traditional Turing test, where the interrogator can also test the perceptual abilities of the subject as well as its ability to manipulate objects.

The Minimum Intelligent Signal Test proposed by Chris McKinstry allows only true/false or yes/no responses.

Another notable variation is the Hutter Prize test which tests the machine’s ability to compress natural language text. The results of the test can be used to compare which of the two machines is more intelligent and the test itself doesn’t require the computer to lie to the judge.

More Information:

• Computing Machinery and Intelligence: Turing’s original paper is available on Loebner’s website and describes his idea in its entirety as well as the objections and Turing’s own beliefs and consideration of principle objections to his test.
• The Alan Turing Internet Scrapbook: this part of Alan Turing’s website contains links and resources along with quotes and extra comments which give additional insight into Turing’s work.
• Harnad and The Turing Test: Stevan Harnad is a big proponent of the Turing Test and in his paper he argues that Turing’s Imitation game is a valid scientific criterion.

Attempts to Pass the Turing Test

Since the publication of Turing’s paper, there have been various attempts to create programs that would pass the test. One of the first was a program called ELIZA, created by Joseph Weizenbaum. The program would examine user’s typed comments and look for keywords. In the event that a keyword is found, the program would supply a sentence that included a transformed comment submitted by the user. In the event no keyword was found, the program would supply a generic answer or revert to repeating an earlier comment.

Another attempt was created by Kenneth Colby. He created PARRY, a program which was tested on a group of psychiatrists who analyzed real patients and computers running PARRY through teleprinters. These answers were shown to another group of psychiatrists. Both groups had to identify which patients were human and which were computer programs. The psychiatrists were able to make the correct identification only 48 percent of the time.

Since then, a number of bots and computer programs have been created for attempting to pass the Turing test. These programs participate in the annual Loebner Prize competition. While a gold and silver medal have never been won, the competition has awarded the bronze medal every year for the computer system that demonstrates the “most human” conversational behavior among that year’s entries. A notable winner is the Artificial Linguistic Internet Computer Entity (A.L.I.C.E.) that won the bronze award on three occasions.

More Information:

• ELIZA: a real example of a Turing test: this article gives a brief history of the Turing test and ELIZA as well as an opportunity to actually interact with the program itself.
• When PARRY Met ELIZA: A Ridiculous Chatbot Conversation From 1972: an insight into an actual conversation between ELIZA and PARRY with actual transcripts.
• PARRY: An Artificial Intelligence Program with “Attitude”: a historical overview of PARRY and its encounter with ELIZA.
• Loebner Prize: the homepage of The Loebner Prize for artificial intelligence (AI) which is the first formal instance of a Turing Test. It has details on the development of the Loebner Prize and the reasons for its existence.

Turing Test Alternatives

Many consider the Turing test fundamentally flawed. As such, several alternatives have been proposed.

Winograd Schema Challenge

Authored by Hector Levesque, a professor of Computer Science at the University of Toronto, the Winograd Schema Challenge (WSC), presents a number of multiple-choice questions within a very specific format.

The Marcus Test

The test devised by Gary Marcus suggests building a computer that can watch any arbitrary TV program or a show and answer questions about the content in a meaningful way.

The Lovelace Test 2.0

This test was developed in 2001 by Selmer Bringsjord and colleagues. The test aims to detect artificial intelligence by examining its capacity to create a true work of art.

The Construction Challenge

Also known as the IKEA challenge, the test takes into consideration perception and physical action supposing that computers don’t have eyes or hands and by using robots, this challenge could easily be overcome. A similar version of the Construction Challenge is the Visual Turing Test that challenges a machine to mimic the visual abilities of humans.

More Information

• Proposing an Alternative to the Turing Test for AI: an explanation of the Winograd Schema Challenge which agrees with the notion that passing the Turing Test doesn’t actually make a computer intelligent.
• The Winograd Schema Challenge: the original paper published by Levesque.
• Lovelace 2.0 Test of Artificial Creativity and Intelligence: this PDF document contains the original proposal for the Lovelace 2.0 test.
• What Comes After The Turing Test?: this article published in New Yorker has an in-depth explanation of the aforementioned Marcus Test.
• Visual Turing Test: this website lets you take the interactive Visual Turing Test and try it out yourself.

The Impact of the Turing Test Lives On

The Turing Test may be considered flawed by today’s standards but there is no doubt that it paved the way toward making computers more intelligent and has influenced artificial intelligence research. Use the resources above to learn more about the Turing Test. Who knows, you might discover a way to improve it or a fool-proof way to beat it one day.

Further Reading and Resources

We have more guides, tutorials, and infographics related to computers:

• Top 10 Artificial Intelligence Experts: learn about the movers and shakers in the modern world of AI research.
• Composing Good HTML: this is a solid introduction to writing well-formed HTML and using HTML validator software.
• CSS3 — Intro, Guides and Resources: this is a great place to start learning webpage layout.

Web Design Trends You’ll Never Forget

Check out our infographic on the history of chatbots, How to Avoid Falling in Love with a Chatbot.

How to Avoid Falling in Love with a Chatbot