Foresight Institute Logo
Image of nano

Richard Smalley comments on runaway replicators

from the sufficient-unto-the-day dept.
Richard Smalley, Nobel-laureate researcher into carbon nanotubes at Rice University, recently appeared as a panelist on National Public Radio's "Talk of the Nation – Science Friday" program (11 August 2000). In response to a question about the concerns that nanotechnology might get out of control, Smalley responded:

"Most of the discussion, at least in my experience, ends up coming back to the self replicating out of control robots, and I think that that, at the moment, is a silly concern."

In response to a question about the concerns that nanotechnology might get out of control, Smalley responded:

"Most of the discussion, at least in my experience, ends up coming back to the self replicating out of control robots, and I think that that, at the moment, is a silly concern.

"There are more substantive concerns that almost certainly will come out, pretty much in direct proportion to the power of the technologies that are evolving.

"It is true that it seems as though almost anything can be done if one can position atoms in the right place, but it's not going to be simple and overnight. So I think there will be time to become alerted to dangers as they arise and to deal with them, in many ways much like this issue has come up in genetic engineering and earlier aspects in biotech."

17 Responses to “Richard Smalley comments on runaway replicators”

  1. Kadamose Says:

    Silly Concern??

    Don't you think believing that 'runaway assemblers' is a problem and a worry that needs to be addressed before the problem DOES happen? Creating something first and then addressing the problems later is negligent and could be very disasterous.

    I believe addressing the problems and dangers of Nanotechnology should come FIRST even if we are 10-20 years away from a working assembler. If the problems aren't addressed, then the people will not be educated enough to stop the problem after it starts.

    Educating the people now about the problems (and yes, runaway assemblers/nanites IS one of those problems) could very well stop the problem BEFORE it ever even has a chance to start.

  2. redbird Says:

    It is silly

    I've written this before deep down in some previous threads, but self replication concerns are pretty silly. No matter how out of control the situation gets, those nanobots can only replicate so quickly. Let's say they can double themselves every hour. We will have many hours before there are enough of them to pose a threat, and almost as much time before they are a taking over. Thus, in those hours we would be able to develop anti grey goo 'bots.

    Also, I would align nanobot development more with programming than engineering. After all, once the assembler knows how to build nanobots, we just need to tell those nanobots what to do.

  3. RichardTerra Says:

    First order calculations on global ecophagy

    Rob Freitas has done an initial study on scenarios of destructive replicators. It's available at: "Some Limits to Global Ecophagy by Biovorous Nanoreplicators, with Public Policy Recommendations", on the Foresight web site.

  4. RichardTerra Says:

    Re:Silly Concern??

    Well, Smalley's words are his own.

    I don't think any thoughtful person would disagree with your point that the problem of controlling replicators does need to be addressed well in advance of the development of the hardware.

    I think the present concern is that over-reaction to potential problems may lead some people to advocate short-sighted policies that regulate, restrict, or even ban ("relinquish" is one term being bandied about . . .) the development of replicating nanosystems in such a way as to actually hinder the development of effective solutions, without doing anything to prevent the advent of the very problem itself (i.e., runaway destructive replicators).

  5. Malcalyps Says:

    Re:It is silly

    The senario stating that "Let's say they can double themselves every hour. We will have many hours before there are enough of them to pose a threat, and almost as much time before they are a taking over. Thus, in those hours we would be able to develop anti grey goo 'bots." is dangerously understating a potential catastrophe. Unless the uncontrolled replicators can be deprived of 1)energy or 2)building materials, the only good controlled defense agaist them would probably be defensive assemblers. ( I'm sure some thought may yeild some other defenses, microwaves? liquid nitrogen to slow it down? etc ) How long until these uncontrolled replicators are detected by someone capable of responding to it? (Anybody out there up to calculating something like this? I'm assuming the "grey goo" is a weapon and not an unfortunate accident) How long would it take for the "grey goo" to grow from say the size of a football to the the West Coast area? (probably this would not be a simple growth problem, new material boundry surface and available energy would limit the growth rate) Any nanotech response would be manufactured using the same exponential or geometric growth rate and therefore be consistantly unstrength to suppress the "grey goo". I do believe this was the primary reason for the "active shield" defense idea. A little foresight into methods of defense should be at the top of everybody's list when developing this technology. p.s. Counter to the above doom and gloom, I think nanotechnology has enormous potential to be beneficial for us all al long as we don't exterminate ourselves first.

  6. MarkGubrud Says:

    Yeah, it's not just goo

    I would direct more attention to the second paragraph of the Smalley quote: "There are more substantive concerns that almost certainly will come out, pretty much in direct proportion to the power of the technologies that are evolving." Note that the potential power of nanotech stems in large part from the possibility of self-replicating systems. So it is possible that some of the concerns that have been expressed have been misunderstood as simply variants of the grey goo scenario. I would agree with Smalley that too much of the discussion has focused on this rather simple-minded view of the range of possible dangers. But, for example, I read Bill Joy's famous article as identifying "GNR" technologies as dangerous in particular because of their potential for large-scale impacts through self-replication, not only as out-of-control replicators but possibly very much under the control of evil or misguided actors.

  7. Kadamose Says:

    Re:Yeah, it's not just goo

    Very true – however, I don't think the 'grey goo' discussion should stop, mainly because it is one of those dangers that can wipe us all out before we even know what hit us. Granted, there are countless dangers with Nanotech (stupid people running the technology, etc etc), and I feel these issues need to be addressed as well.

  8. planetp Says:

    Not Silly: Biosphere Immune System?

    A potential run-away grey-goo scenario is anything but silly.

    Assuming somebody actaully creates a general purpose environmentally powered self-replicator, the results are likely to be catastrophic. I know fo at least two reasons:

    1) The power of doubling can be deceptively misleading. Assuming a doubling time of one hour, and an intial 1 kg. of grey-goo, that would yield over 4000 kilos in 12 hours. The weight of a large truck. In another 12 hours, you will have the wieght of 4000 trucks; aprox: 1.7 million kilos. 36 hours after intitial release: 7 billion kilos, 48 hours: 3 Trillion Kilos!!

    So in less than 2 days after intial release we running rampant in the bioshpere, and doubling every hour. In less than a week, assuming these replicators can operate in cold and hot environments and are powered by the sun, the entire earth including the crust would be consumed by grey goo.

    2) Engineering a response to a grey-goo outbreak would be next to impossible. The anti-grey goo would have to replicate faster than the grey goo in order to catch up. This is unlikely since grey-goo is not picky in what matter it decides to digest, where anti-grey-goo would have to be in order not to further the destruction. So what matter will anti-grey goo go use? It can't consume people, grass, trees or anything, unless you feel the threat od grey goo is greater than the potential loss of millions of people.

    So as far as I can see, the only adequate defense against grey goo, is creating super-sophisticated anti-grey goo assemblers and release them into the bioshpere as a pre-cautionary measure. You could see this as bossting the biosphere's immune system. The equivalernt of biospheric 'white blood cells'.

  9. Jeffrey Soreff Says:

    Re:Not Silly: Biosphere Immune System?

    I think that it is useful to distinguish a couple of different scenarios:

    1. grey goo today:

      Smalley is quite right. No one can build an MNT replicator yet.

    2. grey goo accident during first steps of MNT replicator research:

      I bet that Smalley is right, because I'll bet that the first MNT replicator looks like a molecule in an intensive care unit, strapped to the equivalent of a ventilator, IVs, etc.

    3. mature MNT, grey goo industrial accident

      It takes a pretty stupid design to get this to happen, the equivalent of building self-replicating vinyl siding and managing to disable all the "off" switches. I suspect that this will happen on rare occasions, but I also suspect that something which is originally intended to produce a useful product will probably be sufficiently slow growing that most of these accidents will look more like bad jokes than like catastrophes. I think that the Foresight guidelines are a reasonable approach to avoiding this.

    4. mature MNT, intentional grey goo from an individual

      I think that this will be a problem, and will require something like active shields. We've got a while till this is a problem.

    5. mature MNT, intentional grey goo from a State or competent terrorists

      See the whole arms control discussion. This is one of a very broad class of MNT weapons (others are likely to be more useful…). A lot of these are hard to control.

  10. Kadamose Says:

    The Grey Goo Already Exists!

    Well, I do agree with most of your comments, but the dreaded 'grey goo' already exists in the world today, and therefore, I believe this is an important issue to address. What is the 'grey goo' which exists today in the modern world, you ask? The answer is: PEOPLE. Currently there are 6.3 billion people alive in a world that can only support half that – in contrast to this, the population in 2003 is expected to be 8.1 billion, and by the year 2012, an estimated 12 billion people. Is this not a problem? From what it looks like to me, people are acting just like nanites would, except they don't reproduce as fast.

    (a cure for this would simply be to release nanites in the atmosphere, making every woman/man/child involutarily inhale these little machines, which, upon consumption, would go to the reproductive organs and either 1) Shut them off OR 2) Limit them to 1 Reproduction period.)

    This method would EASILY fix the very big population problem we have now, though I am quite sure it would cause quite a controversy, but who cares – we are no longer in the Dark Ages afterall, and anyone who thinks that it is mandatory to reproduce is a nutcase who should be disposed of.

    I believe the same method can be used for Nanites, as well, thus halting the Grey Goo Effect.

  11. BryanBruns Says:

    Global carrying capacity

    You assert: " Currently there are 6.3 billion people alive in a world that can only support half that." Perhaps you'd like to clarify what you mean by "support"?

    If you are interested in empirical data and theoretical analysis about what level of population the world could support, I highly recommend Julian Simon's book, The Ultimate Resource 2, which thoroughly refutes common misconceptions about "limits to growth." More generally, most current estimates suggest that improvements in education, health, wealth and other factors will lead to population stabilizing through a voluntary process at some point in the next century or so.

  12. jstsumschmuck Says:

    Re:It is silly

    I agree. "In those hours, we would develop anti grey goo" just doesn't hold water. A period of hours isn't an acceptable design window for *ANY* assemblers, let alone defensive self-replicators. We're talking about a self-replicating machine to seek out and alter *ALL* instances of some molecular profile, anything less is only a delay. It would have to repoduce, saturate an environment, *stop reproducing at some point*(so as not to become just an alternative grey goo), determine which matter it encounters needs to be dissasembled or pacified, perform the defensive action, and then just loiter in the environment with all it's inherently-menacing complexity unless disposed of somehow. It would have to be several orders of complexity greater than the simple doubling-machine it's pitted against. That means it would require more time to reproduce, more complex building blocks, and hopefully a great deal of peer review in it's design process. It seems pretty likely that even if the rushed cure isn't unintentionally more damaging than the disease, it would be unable to compete for the resources available even without the offensive nanites having a headstart. I know, I'm jstsumschmuck, and not an authority on anything I've just written about, but I live here too. And I'd like to survive to see mnt's potential payoffs.

  13. redbird Says:

    Re:Global carrying capacity

    This is very much true. For you Americans, consider how many of the educated people you know are obsesed with having children. Not many is likely to be the answer. Many people who have been educated have come to see children as a burden (they saw themsleves as burdens, if that helps), so don't have them and leave it to someone else. Of course, once we're post human, we can have as many kids as we want! ;-)

    Warning: there were some pretty broad generalizations above. So, if you don't find them to be true, that's okay, but they have been for me.

  14. BarryM Says:

    Not silly, but also not overly concerning

    Assemblers running amuck are a concern in the same way alien abductions are a concern. It is trivially simple to design fail-safe systems to prevent such disasters; I suspect that is why Richard Smalley refers to such concerns as "silly". This is perhaps the wrong word, since it is also trivially simple to design fail-safe systems to prevent Chernobyl-style meltdowns, and yet they happen – in every instance to date not because there was a problem with the carefully designed procedures, but outside (e.g. human) factors. Since human sloppiness, incompetence and unwillingness to RTFM will not really enter into the nanofabrication process, it is much less of a concern, IMHO. That is, good fail-safe systems could form part of the design of the nanomachines themselves, as well as the fabrication unit making them. We can, for example, prevent a hegemonising nano-swarm using principles similar to those which Nature uses to prevent the entire globe being covered in E.Coli…

  15. TomMcKendree Says:

    URLs that Work

    The URLs in the main message didn't work for me. works for Science Friday, and http://www.sciencefri works for the particular episode.

  16. RichardTerra Says:

    Re:URLs that Work [Corrected]

    The link in the main message has been corrected – Thanks, Tom, for catching this.

  17. ChrisPhoenix Says:

    Blue goo need not self-replicate.

    Blue goo can be manufactured ahead of time and stored inactive until it's needed. This might be a useful thing to do with all the megatons of extra carbon in the atmosphere. Then while the gray goo is trying to replicate and avoid the blue goo, the blue has only one thing on its mind: killing gray.

    Robert Freitas has shown that a gray goo trying to convert the biosphere would have a very hard time of it: it couldn't run at anything near full speed because of the heat it would generate.

    A blue goo design that self-replicated in the wild would in fact face the problem of how to grow without doing damage. But what if we converted the U.S. Strategic Petroleum Reserve into blue goo, and distributed caches worldwide for fast response?

    Also, goo is not very good as a military weapon, or even a terrorist weapon. I'm mainly worried about script kiddies, and a pre-made blue goo would probably be adequate to deal with that.

    Of course, today we don't deal with computer infections nearly as well as we could.


Leave a Reply