Foresight Nanotech Institute Logo
Image of nano

A brief history of AI

  • 40s: Cybernetics, the notion the brain did logic in circuits, feedback
  • 50s: the computer, stored programs, Logic Theorist
  • 60s: LISP, semantic nets, GOFAI
  • 70s: SHRDLU, AM
  • 80s: AI winter, expert systems, neural nets
  • 90s: robots, machine learning
  • 00s: DARPA grand challenge level of competence

The main point of this post is to answer any objections of the form: you’ve been working on this so long, why don’t you have it yet?  (Or perhaps, AI is the technology of the future and always will be. :-) )

One key thing to note is that cybernetics was the original line of inquiry that was going to let us understand how the brain worked and allow us to build smart machines.  Many people assume that cybernetics failed since it more or less disappeared as a discipline.  But in fact it learned some very key and useful insights, forming the basis of control theory and neuroscience; but it fell apart due to personalities in its cadre (a veritable soap opera between Wiener and McCulloch and Pitts involving Wiener’s daughter) and political disfavor in the US involving Wiener’s antiauthoritarian stances.

So GOFAI was born with a built-in bias against some of the insights of cybernetics.  That has now been repaired; it was forced by the reintegration of control theory and the growing use of knowledge from neuroscience in the 90s, when AI robotics began to get serious.  There are reasons AI floundered in the 80s, and that’s one — another is a diversion from basic research to applications before it was really ready.

Another point that is rarely made is that AI, the small sub-discipline of CS, isn’t the real major part of the work in the 20th century that will have led to intelligent machines.  It’s the invention of the computer itself and all the work that’s been done to bring us the processing power we need to do the job, and the software to manage it and the complexity of human-comparable systems.  And nobody could reasonably claim that that effort has been standing still, or has come to nothing, or anything even vaguely similar.

An AI will be a hardware/software network and system so complex and powerful that it will make the entire ARPANET of the 70s look like a toy — and it will have to manage its own internals completely automatically.  I personally think that it will need the internal robustness that can only come from incorporating feedback and automatic resource management into the basic fabric of its computing platform.  But that’s the kind of thing that can easily be done in a decade, once someone decides to do it.  And it will be useful for a lot of other applications as well!

4 Responses to “A brief history of AI”

  1. Tim Tyler Says:

    Re: that can only come from incorporating feedback and automatic resource management into the basic fabric of its computing platform

    What does that mean? In which senses do modern computer systems not do these things already? You mean you want data-centre robots to replace failed hard drives? That kind of thing is probably not a pre-requisite for machine intelligence.

  2. J. Storrs Hall Says:

    Certainly you can be an intelligent human and still need to go to the doctor from time to time. So intelligence doesn’t require strict long-term autonomy.
    However, I was thinking more along the lines of software. In the brain, as I’m sure you know, there’s actually more retrograde traffic than feedforward along many of the pathways — e.g. more proprioceptive and sensory feedback along motor pathways than motor signals going out.
    I would claim that this phenomenon is not limited to distal control and sensing but to every aspect of internal computation. One of the reason that symbolic AI kept hitting a glass ceiling is that there was no feedback inside the programs’ logic, and thus every part of them was always operating by dead reckoning.
    Feedback can’t be built into the lowest level of the standard sequential model of computation because the model arranges operations in time — and there can’t be signals going backwards in time. Conceiving of the same computation as being as parallel as possible — which is how the brain does things — allows for a lot more feedback to be built into the very lowest level primitives. (simple example: for every forward signal, there is a feedback signal saying how much precision and bandwidth is needed downstream.)
    That doesn’t mean that can’t be emulated in sequential software — indeed, that’s exactly how it will be implemented in any systems that will be built any time soon. But doing so requires an extra level of design discipline above the standard feedforward-only, waterfall, fire-and-forget, sequential paradigm.

  3. TheRadicalModerate Says:

    When it comes to the neural back channels, I tend to favor the idea I first learned from the Jeff Hawkins book (I won’t pretend to understand its proper academic ontogeny): The back-channels are used more for prediction than they are for classical control. If you model the brain as a forward-directed graph of pattern recognizers, the back-channels allow the higher-level entities to tell the lower-level entities things like, “I recognize the stuff you’ve just sent me, things are going well, and based on that recognition, the next thing you should recognize is probably this.” From an evolutionary standpoint, this is probably a huge energy savings in the brain, since the amount of signaling, and consequent neural activation, between entities is vastly reduced when most of them are merely reporting up the chain, “Yup, what you expected is indeed what I recognized next,” instead of, “Whoa! something’s way different! Time to start learning and engaging high-level cognitive functions to figure out what’s going on!”

    This isn’t to say that stable control isn’t essential; it obviously is. However, the control feedback is merely more sensory information that can get mixed into the feed-forward pathways, at every level in the directed graph. In other words, higher-level areas direct motor functions (essentially a feed-forward operation) while at the same time they watch proprioceptive sensory input (also a feed-forward operation). The back-channel is then reserved for things like, “Uh-oh! The arm is out of position and I’m not going to hit the saber-tooth tiger I’m aiming at with the rock that I’m throwing.” It won’t do you much good for the current rock, but it’ll help you throw the next rock better. Of course, as you get lower and lower in the cognitive hierarchy, the neural entities become less cognitive and more like real control systems. But that’s the difference between intelligence and complex control.

    I don’t hold out much hope for all the legacies of GOFAI, even for higher-level cognitive functions. The development that we need to make major progress is the ability to scale our current neural network models from software-driven simulators that can handle 10^5 or 10^6 virtual synapses to hardware-driven systems with massive connectivity, able to handle 10^11 to 10^14 synapses. This kind of connectivity gives you the opportunity to solve a lot of the control and perceptual problems using genetic algorithms that wire stuff up in progressively more interesting ways, while using learning and plasticity to fine-tune the connectivity. What we’ll wind up with are pretty smart machines that work, even though we can’t tell you exactly why they work.

  4. J. Storrs Hall Says:

    Radical: have a look at this:
    http://mind.ucsd.edu/papers/intro-emulation/intro-em.pdf

Leave a Reply