What is the Turing test, and what would passing it show?

The Turing test, proposed by the mathematician Alan Turing in 1950, measures whether a machine can show behavior indistinguishable from a human's in text conversation. A human evaluator talks with both a human and a machine without knowing which is which. If the evaluator cannot reliably tell them apart, the machine is said to have passed. What passing demonstrates is contested: some see it as good evidence of intelligence, others as a measure of imitation only. [1] [2]

Turing described a game with three parties: a person, a machine and an interrogator who sits in a separate room. The interrogator knows the two hidden parties only as X and Y, asks them questions, and at the end says which is the person and which is the machine. The machine tries to make the interrogator wrongly pick it as the person, while the human tries to help the interrogator get it right. Turing thought the question of whether machines can think was too meaningless to discuss directly, so he replaced it with the question of how well a computer can do in this game. [3]

The result does not depend on whether the machine answers correctly, only on how closely its answers resemble a human's. [2]

Turing predicted that in about fifty years computers could play the game so well that an average interrogator would have no more than a 70 percent chance of identifying correctly after five minutes of questioning. The Stanford Encyclopedia of Philosophy stresses that this prediction is separate from the test itself, since the percentage, the time and the number of exchanges are adjustable parameters. It says claims that the program Eugene Goostman passed in 2014, after fooling 33 percent of judges, rested on very small trials that were not reliably repeatable. [3]

In June 2024, researchers reported that GPT-4 was judged human 54 percent of the time within five minutes. Live Science notes that this study used two players rather than Turing's three-player setup, so GPT-4 did not pass under the exact conditions he defined. [4]

One position holds that passing is meaningful evidence. The Stanford Encyclopedia argues that a program must survive questioning by an interrogator who knows one party is a machine, and do so across many trials. If it did, there would be at least prima facie reason to think intelligence is present. Turing's claim, on this reading, is only that conversational ability gives good grounds for attributing intelligence, not that intelligence requires conversation. Failing the test would not prove a machine unintelligent either, since a truthful machine that refused to pretend could be spotted at once. [3]

A second position holds the test is too weak. Simple programs such as ELIZA have seemed intelligent to ordinary observers for extended periods, and in a short test cleverly designed but unintelligent programs might fool most people. Stanford HAI adds that modern chatbots can mimic human conversation without genuine understanding. [3] [1]

A third position is that the test does not indicate consciousness or intelligence at all. Live Science describes it as a critique of what counts as thought, dependent on the interrogator's judgment, on comparison with humans, and on behavior alone. The results do not settle which view most specialists hold, but the sources agree the test is influential and controversial as a measure of true intelligence. [4] [1]