This year marks the 70th anniversary of the publication of Alan Turing’s concept of evaluating the ability of a machine to support human conversation. The scientist predicted that by 2020 the chances of distinguishing AI from humans in a simulation game, where the evaluator will not know the respondent, will be less than 70%.
However, 20 years later, this goal has not been achieved. Amazon’s chief scientist, Rohit Prasad, says this criterion is no longer relevant. The purpose of AI has changed. Now his task is to help a person and expand the boundaries of his capabilities.
Turing test is deprecated. It’s time to change the criteria for evaluating AI abilities
Since its inception, the Turing test has become the guiding star of the AI sphere. In the 1960s and 1970s, the first chatbots ELIZA and PARRY were developed with the main goal of passing this test. In 2014, chatbot Eugene Goostman managed to cope with the test – 33% of judges mistook him for a person. However, even at that time, some considered the test obsolete, and the minimum pass threshold of 30% was too arbitrary.
Nevertheless, this concept still attracts public attention. The developers of the GPT-3 (Generative Pre-trained Transformer 3) language model from OpenAI are already claiming that it can handle the test. Journalists and other observers also often wonder if Amazon’s Alexa virtual assistant will pass this test. And although this is one of the ways to measure the intelligence of a system, is it relevant to use it today?
Advantages and disadvantages of the Turing test
Let’s go back to the time when Turing first presented his thesis. In 1950, the first commercial computer had not yet been sold, materials on fiber optic cables would not be published until four years later, and the field of AI did not formally exist – this happened only in 1956.
A modern smartphone has 100,000 times more computing power than the Apollo 11 spacecraft, which made its first moon landing. Plus, thanks to the cloud and high-speed connectivity, AI can now make decisions based on massive amounts of data in seconds.
While Turing’s original concept is still inspiring, his test is still limited by the era in which he was introduced. Several factors point to this:
- It does not take into account one of the most effective features of modern AI – the high speed of computing and information retrieval. To trick a person into passing this test, the AI needs to pause while answering difficult questions. For example, “What is the cube root of 3434756?” or “What’s the distance between Seattle and Boston?” In reality, the AI already knows these answers and is ready to provide them instantly. Adding a pause to allow him to humanize is not the best way to use his power.
- It is limited to text. The test does not take into account the growing ability of AI to use sensors to hear, see and feel the outside world.
What makes modern AI systems useful is the ability to efficiently perform everyday tasks. To turn off the lights in the garage, a person does not need to have a conversation with a virtual assistant. It is enough for him to execute the request and a simple notification in the “Turn off” or “Done” style.
Even if you talk to an AI assistant about popular topics or ask him to read a story for a child, you still want to know that you are interacting with a machine, not a person. In fact, such deception presents some risks. Already, you can see bots sending out disinformation and a growing number of deepfake technologies.
New ways to measure AI abilities
Instead of striving to create AI indistinguishable from humans, it is worth focusing on developing systems that will expand the boundaries of human intelligence and improve our daily lives by creating an equitable and inclusive environment.
The main goal should be to create an algorithm that combines human qualities such as common sense, self-control and language skills, with machine efficiency – fast search, memory reproducibility and everyday tasks. This will allow you to go beyond what the average person is capable of.
Current research focuses on the really important areas of AI:
understanding with sensors;
maintaining a conversation;
expansion and deepening of knowledge;
thinking for decision making;
elimination of any undue bias and prejudice in order to achieve objectivity.
The development of these areas can be measured in several ways.
- Divide the task into several component parts. For example, in Kaggle’s Abstraction and Reasoning Challenge, AI systems compete to solve problems they haven’t faced before.
- Design a large-scale task in which man and machine will interact. Among these are the Alexa Prize Socialbot Grand Challenge, a student competition dedicated to advancing conversational AI technologies. Smart bots need to communicate consistently and engagingly with people for 20 minutes on popular topics such as entertainment, sports, politics, and technology. Users need to rate how much they would like to communicate with the bot again. In the final, independent judges will award the participants a score on a five-point scale. Those with 4.0 and higher must complete the Grand Challenge, but no one has yet managed to reach this stage.
This technique is driving the development of AI with the ability to support conversation using deep learning-based neural techniques. They allow the machine to show empathy and give it a sense of humor, without forcing it to pretend to be human.
Widespread adoption of virtual assistants
The proliferation of assistants in human daily life is another opportunity to change the progress of AI development.
At their core, these services rely on human conversational skills to accomplish both simple tasks (setting an alarm) and complex tasks (planning a weekend). However, to maximize their benefits, we need to move towards the Ambient AI concept, which will allow the system to respond to requests when needed and anticipate needs. For example, Alexa can detect the sound of breaking glass and warn about it. If you set an alarm before bed, she recommends turning off the lights in other rooms.
Among other things, such systems must also cover an ever-increasing number of tasks. This is only possible with a more generalized learning ability. Thus, in the next decade and in the future, the level of practicality of AI, capable of communicating and actively providing assistance, will help to assess progress in this area.
All of these facts do not diminish Alan Turing’s original vision. Its concept was developed as a thought experiment, not the ultimate measure of AI’s usefulness. It’s time to leave the Turing test in the past and be inspired by the bold concept of its creator to accelerate progress in creating AI systems that will help people.
*The article has been translated based on the content of Rusbase by rb.ru. If there is any problem regarding the content, copyright, please leave a report below the article. We will try to process as quickly as possible to protect the rights of the author. Thank you very much!
*We just want readers to access information more quickly and easily with other multilingual content, instead of information only available in a certain language.
*We always respect the copyright of the content of the author and always include the original link of the source article.If the author disagrees, just leave the report below the article, the article will be edited or deleted at the request of the author. Thanks very much! Best regards!