the Foresight Institute

In response to my Robo Habilis post, Tim Tyler replied:

An intelligence challenge should not involve building mechanical robot controllers – IMO. That’s a bit of a different problem – and a rather difficult one – because of the long build-test cycle involved in such
projects.
There are plenty of purer tests of intelligence that use more abstract ideas – games, puzzles, and other classical intelligence test fodder.
If you want to measure the abilities of mechanical robots, then fine, but let’s not pretend that it’s the same thing as measuring intelligence.

This is a fairly widely held view — there were a couple of researchers at the AGI Roadmap meeting expressing the same idea.  If I understand him correctly, Minsky feels the same way.  I believe, however, that it is not true.

To begin with, that was the reigning paradigm of the entire “golden age” of AI from the 50s through the 70s. Even Shakey the Robot had a bicameral control architecture: a body control program written in SAIL, and a cognitive engine written in LISP.  It was strongly believed that the parts of thought that were hard for humans would be the hard ones to program, and that once we got those licked, building the lower-level body-controller stuff (or vision, or speech-to-text for the input) would be an afterthought, or at most a clean-up engineering exercise.

Over the course of the 60s, classic AI had a tremendous run of success, which is pretty neatly summed up by the work in Minsky’s “Semantic Information Processing.”  They had programs that did games, puzzles, intelligence tests, arithmetic word problems, freshman calculus. The hard stuff. They were full of optimism, and predicted that AI would run to a successful conclusion, creating an artificial mind, in another decade or two. They had done the college student; how much more effort should it take to do a toddler?

They were wrong.  The greatest lesson that came out of the Golden Age was that “the hard stuff is easy, and the easy stuff is hard.”  Any toddler could recognize a dog in a picture; it would be three more decades before AI could get even close (and it’s still not really there yet).

The mind, it turns out, is like an iceberg — most of it is unseen to consciousness, below the waterline.  Perhaps a better analogy would be that consciousness is like the legislature of a country, or the head office of a company. What they perceive is in reality only an executive summary of what’s really happening. What the early AI researchers had done was to build a “company” consisting only of the board of directors and secretaries, but no factories, no sales force, no middle managers, no shop foremen, and no labor force.

The brain was evolved as a body controller. Evolution typically takes a structure that works and copies and adapts it to the next task. Consider the increasing intelligence of animals as we work ourselves up the evolutionary tree towards the human: insects, reptiles, mammals, primates.  At every level new and improved kinds of control, feedback, discrimination, planning, and learning are built into the structure — and it’s all still there forming the part below the iceberg, the real company outside the boardroom, of human intelligence.

The classic AIers at the Roadmap asked me, “Isn’t a blind paraplegic still intelligent?” and of course he is — but only because his brain still contains all the mechanism that was evolved to to the control and interpretation he now lacks.

The buzzword in current AI for the reason bodies are important is “symbol grounding.”  This refers to philosophical theories of meaning among symbols in symbol-processing machinery, and a simplistic reading of it is that whereas SHRDLU doesn’t “really know” what a red block is, a physical robot that plays with them really does.  Unfortunately, the term in common use is often taken as implying that there is some magical transubstantiation of meaning into symbols by virtue of having a physical body, and this isn’t right and obscures the real issue. The paraplegic still has meaning in his mind.

What has to be there is not the actual body, but the mental mechanism for controlling it — that allows the mind to imagine, predict, describe, and relate other concepts to the one said to be understood. Most of our higher-level concepts are drawn from, by analogy and blending, the basic (very large) set of concepts we have learned, by experience, on the shop floors of our minds as we interact with the real world over the course of our lives.

Could that interpretive, predictive, concept-building, etc, cognitive machinery be built another way than working up a controller for a humanoid robot body? Certainly. But there are two reasons to do it with a body: first, it’s most likely easiest that way.  There are a lot of things we don’t know yet about how the mind works.  There’s no reason to think that we have no more blind spots like the classic AIers did. Working with real robots will show us the gaps fastest.

The second reason is that once we get the brain built, if we’ve put it together in a rough semblance of the phylogenetic/ontogenetic sequence that the human mind is built, there’ll be a much better chance that its meanings will match ours. It will understand things the way we do (of course humans vary a lot in the way we understand things), and do things the way we do, and thus appreciate the way we do them, and vice versa.  For example, the parts of the brain that control language and manual manipulation are strongly overlapped. Try to teach your robot sign language without a similar structure and it will never get the “accent” right.  Nor, unless it has the same kind of manipulation control to borrow, will it ever be as fluent in English as a human.

Separating “intelligence” from the rest of cognitive function is a false dichotomy, and one that has led AI astray — in a big way — before.

There are at least 4 stages of intelligence levels that AI will have to get through to get to the take-over-the-world level. In Beyond AI I refered to them as hypohuman, diahuman, epihuman, and hyperhuman; but just for fun let’s use fake species names:

• Robo insectis: rote, mechanical gadgets (or thinkers) with hand-coded skills, such as Roomba or industrial robots or automated call-center systems or dictation programs.
• Robo habilis: Rosie the housemaid robot level intelligence, able to handle service level jobs in the real world but not a rocket scientist.
• Robo sapiens: up to and including rocket scientists, AI researchers, corporate executives, any human capability.
• Robo googolis: a collection of top R. sapiens wired together in a box running at accelerated speed, equivalent to, say, Google (the company and the search engine together).

First point: One R. googolis can’t take over the world, any more than Google could. You’d have to get to the next stage (R. unclesammus).  Any AI in the earlier stages of development that acted antisocial gets stomped on fast (and in early days, they’ll have no rights — so they’ll basically be exterminated).

Second point: As Robin Hanson and many economists point out, the complementary effect of machines up through the R. insectis stage has generally been much stronger than the substitution effect, so that improving technology has had a general beneficial effect on incomes even though it put specific people, buggy-whip makers for example, out of work. Complementarity is seen when comparative advantage holds, substitution when it doesn’t:

So far, machines have displaced relatively few human workers, and when they have done so, they have in most cases greatly raised the incomes of other workers. That is, the complementary effect has outweighed the substitution effect–but this trend need not continue.
In our graph of machines and humans, imagine that the ocean of machine tasks reached a wide plateau. This would happen if, for instance, machines were almost capable enough to take on a vast array of human jobs. For example, it might occur if machines were on the very cusp of human-level cognition. In this situation, a small additional rise in sea level would flood that plateau and push the shoreline so far inland that a huge number of important tasks formerly in the human realm were now achievable with machines. We’d expect such a wide plateau if the cheapest smart machines were whole-brain emulations whose relative abilities on most tasks should be close to those of human beings.

I don’t think that the “plateau” is really flat, though. There are two reasons. The first is that human capability is a range, with R. habilis at one end and R. sapiens at the other. It’ll take some time to get through — at least a decade, maybe two.

The other reason is that the comparative advantage we saw in the Industrial Revolution may just get turned on its head.  Right now we have a Moore’s Law for the robot’s brain but not for its body.  In other words, we may enter a strange period where white-collar workers are replaced by beige boxes but blue-collar ones are still cheaper — for a little while — than a fully-capable humanoid robot body.  (That will disappear soon enough after nanotech manufacturing takes hold, but at the moment, it looks like AI may be a decade earlier than real nanotech.)

The key thing to remember when thinking about the economic AI takeover is that it is not something we should be trying to prevent. Why shouldn’t we, the human race as a whole, build machines to do the hard work we need done, and spend our time enjoying the resulting wealth?  Why shouldn’t we spend our efforts deciding what needs to be done, and let the machines do it?

Questions like unemployment are the result of taking a system that is well-adapted for one economic situation and applying it to a totally different one. What should the economic system look like when robots do all the work? And once we get that figured out, how do we get there from here?

My Robo Habilis post was picked up on by Michael Anissimov who wrote:

(me:) It seems to me that one obvious way to ameliorate the impact of the AI/robotics revolution in the economic world, then, is simple: build robots whose cognitive architectures are enough different from humans that their relative skillfullness at various tasks will differ from ours. Then, even after they are actually better at everything than we are, the law of comparative advantage will still hold.

Boom, friendliness problem solved. Build robots with different cognitive architectures than us, and they will be forced to keep us around, due to Ricardo’s law of comparative advantage. Sounds wildly naive to me.

All I can say is thanks for noticing I’ve solved the most important problem of the 21st century with a single paragraph! I’m confidently expecting my Nobel Peace Prize.

But seriously, I would like to argue that the concept of the “friendliness problem” is a dangerous misreading of the real difficulties and problems we will face as a result of the development of artificial intellegence over the next few decades. It seems to me that one could characterize the people working on “Friendly AI” as essentially trying to redo moral philosophy, from scratch, and get it right this time.  There’s nothing wrong with this; moral philosophy is a valuable intellectual tradition and worthwhile human activity.  But the notion that the whole business, with the addition of the new insight that there can be intelligent machines as well as humans among the class of moral agents, could be solved in any useful sense, just strikes me as silly. Indeed, the new insight makes moral philosophy a lot harder, rather than bringing it any closer to any kind of closure.

Instead let’s look at the kind of problems we’re really going to face. There is not — I guarantee it — going to be any single overarching solution to them; there will be a host of minor things we can do to ameliorate the problems as they arise, and we’ll just have to keep coming up with them as problems arise.

We know what it will be like should we manage to invent and implement a giant, powerful decision-making system that takes over the world. We know because we’ve already done it. Some people have observed this system in action and seem to think that it has a “friendliness problem”:

We’re Governed by Callous Children

…

When I see those in government, both locally and in Washington, spend and tax and come up each day with new ways to spend and tax—health care, cap and trade, etc.—I think: Why aren’t they worried about the impact of what they’re doing? Why do they think America is so strong it can take endless abuse?
I think I know part of the answer. It is that they’ve never seen things go dark. They came of age during the great abundance, circa 1980-2008 (or 1950-2008, take your pick), and they don’t have the habit of worry. They talk about their “concerns”—they’re big on that word. But they’re not really concerned. They think America is the goose that lays the golden egg. Why not? She laid it in their laps. She laid it in grandpa’s lap.
They don’t feel anxious, because they never had anything to be anxious about.

Peggy Noonan thinks the government is screwing us up because it’s made of people who don’t care. But I beg to differ.  There’s a classic fallacy in the philosophy of mind that shows up in places ranging from Leibniz’ story of the “magnified mill” to Searle’s Chinese Room, which is that for a system to have some property, the property must be present among the parts. This is just as false for caring as it is for understanding or consciousness. In fact the existing system is a perfect example, although in reverse — it’s composed of people who do care, but they interact in a structure that results in an evil bureaucracy.

Instead, what’s happened is that we made a blunder in designing the system that is exactly equivalent to a favorite example of Eliezer Yudkowsky: instead of building a paperclip-maximizing machine, we built a vote-maximizing machine.

I claim that the problem is much more productively looked at from another point of view: the system as a whole is incompetent. It doesn’t do what it was built to do (“… promote the general welfare, secure the blessings of liberty …”). The designers simply assumed a vote-maximizer would do the things they wanted, but they were wrong.  Similarly, no human wants the universe to be converted into paperclips, so if he built a machine with that goal, he would have designed incompetently.  I claim we should be spending our time on is figuring out how to build competent AI.

First principle of competent AI design: Build a machine that understands what you want.  The paperclip maximizer is a study in amazing contrasts — presumably an intelligence powerful enough to take over the world would be capable of understanding human motivations even better than we do, so as to manipulate us effectively. Yet it’s built with a complete cognitive deficit of appropriate motivations, goals, and values for itself. Incompetent.

Second principle: build machines that know their limitations.  This basically means that it should confine its activities to those areas where it does understand the effects of its actions.

But in order to do that, we first have to be able to build a machine that can actually understand something — anything — in the full human-level meaning of understanding. And that is the necessary first step to a future of useful and beneficial AI, and it’s what anyone concerned about such things should be working on.