The fact that the paradigms that are useful for learning are so different than paradigms for reliable systems that it is really hard to
AI has this funny history where nothing works as well as people hope. Instead, there is a pattern where people get really excited about the potential of a new technology or idea, give massive funding to researchers in hopes of some exciting technologies, and then eventually give up, taking home some small and seemingly unimportant bits of technology. But the researchers continue working, technology progresses, and eventually someone come up with a new exciting idea, starting a new hype train.
Some respectable people, think that this time around is going to be different, that this time, we are not going to give up, that AI is on an unbridled path to unending society as we know it in the next 10-20 years.
But there are two problems that ultimately stands in the way of progress. First, like all technological bubbles, there is the problem of inertia, that hype trains have a hard time making the necessary step back in order to take two steps forward. But the great researchers already more or less understand this, and a few, like the legendary Geoffrey Hinton are taking that step back and placing criticism on the successful ideas of today, so that tomorrow’s ideas have even more potential.
However, there is an even more difficult problem standing in the way of progress. Confounding that is an even more difficult problem, which is what I call the hard problem of AI.
good at AI promises to be part of future technologies. Part of technologies which make you have to worry less about how to get to work, or how to make good decisions based of the data.
The first thoughts about AI were some of the most insightful thoughts written to date. in the fascinating and playful paper “Computing Machinery and Intelligence” Alan Turing lays out the famous Turing Test for general intelligence, arguing that an conversation in English can convey about every human intellectual experience.
Since then, people have expressed significant dissatisfaction with the Turing test, but no one has been able to many flaws in the Turing
He also mentions the first ideas on how a machine can be programmed to pass as a human, including elements of evolutionary process, teacher-student learning, and random search, all critical to modern AI.
How was this paper, despite its flaws, so able to hit upon so many of the points of AI? I can’t say for sure, but in these days, computers were still in their infancy, used only for numerical problems like brute forcing cryptographic codes. The general public still didn’t really understand what a computer was. So Turing felt a need to argue how powerful computers were in theory, and really think about their true capabilities.
After the hype generated in large part by Turing’s paper, there was a lot of money thrown at tasks like
In the fascinating and comprehensive paper “Computing Machinery and Intelligence” Alan Turing lays out the Turing Test, the famous test for a generally intelligent machine that can converse with a human. I recommend reading the whole paper, because while some parts are certainly dated, other parts are remarkably modern, relevant, and you will immediately become optimistic about relative ease of AI.
He goes over some of the first ideas of how a machine can learn. These ideas roughly remain the same today, so I will go over them in detail.
How would a machine learn go copy an adult’s conversational patterns? You could try to program it to directly copy an adult’s behavior, but this would require an absurd level of effort. So why not try to have a machine learn like a human does? This means that the information the computer uses comes from three sources
Programming a initial state of a machine will be difficult. We do not know what a brain does exactly, or how it is structured at birth exactly, and we can’t replicate all of the senses, so what can you do? Well, success is how quickly the machine learns. Then, Turing frames it as an evolutionary problem where you have:
Structure of the child machine = Hereditary material Changes „ „ = Mutations Natural selection = Judgment of the experimenter
To solve the first problem, he hopes that the infant’s brain is simple enough in structure that it can be easily simulated. To solve the second problem
In particular, he goes on at length of an idea of a teacher-student setup where a human instructor teaches a computer how to do things, while the teacher does not necessarily know what is going on. The teacher could point to things, talk, make gestures, direct, etc, as if they were instructing a child.
He even makes a concrete estimate of how much computation capacity will be needed for an AI
Estimates of the storage capacity of the brain vary from 10 10 to 10 15 binary digits. I incline to the lower values and believe that only a very small fraction is used for the higher types of thinking. Most of it is probably used for the retention of visual impressions. I should be surprised if more than 10 9 was required for satisfactory playing of the imitation game, at any rate against a blind man…. It is probably not necessary to increase the speed of operations of the machines at all.