Foresight Institute Logo
Image of nano

Defending Against Replicating Weapons

Insanely Destructive Devices: Trying to defend against self-replicating weapons of mass destruction, by Foresight advisor Lawrence Lessig, about a class he's teaching with Steve Jurvetson, Senior Associate and tutorial/Gathering speaker at our May meeting.

Marking the 4th anniversary of Bill Joy's Wired article "Why the Future Doesn't Need Us," Lessig writes about a course he and Steve Jurvetson put together on the challenges raised by Joy in his article. Using the example of engineered mousepox viruses that are 100% lethal and even 60% lethal to vaccinated mice, they asked their students "how a society should protect itself from innovations that lead to pox viruses with 100-percent kill rates." Initial student responses favored government regulation, decreased freedoms, and bulky bio-isolation suits.

"Then one student suggested a very different approach. If we can't defend against an attack, perhaps the rational response is to reduce the incentives to attack. Rather than designing space suits, maybe we should focus on ways to eliminate the reasons to annihilate us. Rather than stirring up a hornet's nest and then hiding behind a bush, maybe the solution is to avoid the causes of rage. Crazies, of course, can't be reasoned with. But we can reduce the incentives to become a crazy. We could reduce the reasonableness – from a certain perspective – for finding ways to destroy us.

"The point produced a depressing recognition. There's a logic to P2P threats that we as a society don't yet get. Like the record companies against the Internet, our first response is war. But like the record companies, that response will be either futile or self-destructive. If you can't control the supply of IDDs, then the right response is to reduce the demand for IDDs. Yet as everyone in the class understood, in the four years since Joy wrote his Wired piece, we've done precisely the opposite. Our present course of unilateral cowboyism will continue to produce generations of angry souls seeking revenge on us.

"We've not yet fully understood Joy. In the future there most certainly will be IDDs. Abolishing freedom, issuing space suits, and launching wars only increases the danger that they will be used. We had better learn that soon."

7 Responses to “Defending Against Replicating Weapons”

  1. ChrisPhoenix Says:

    This isn't nanotech…

    Lessig is talking about biotech here, *not* nanotech.

    Self-replicating nanoweapons are a favorite of science fiction writers. But a non-replicating prebuilt weapon will be far more effective, since it won't have to stop and forage.


  2. genda Says:

    Picking your problem space wisely…

    We have met the enemy and he is us…

    As long as we function, unconscious of the underlying drives, the compulsive behavior buried in our reptilion and mammilian lower brain, the drives to complete, the drives to control our environment to the detrement of ourselves and those around us, the endless constant focus on self benefit at the expense of the world we live in… As long as we function this way… we are on rails to our own doom.

    We must as a race, put down the childish fears and prejudices that have shaped our history. We must gather ourselves up, and insure to a person, that we build a future of peace and prosperity for all, acceptance for the diversity of culture and race, and the constant and unerring promotion of wisdom and knowledge. That is our only salvation. To do other is to cast ourselves and our posterity into the unthinkable horror of third millenium technology as weapon. We can ill afford to tolerate our own ignorance and arrogance any longer.


  3. WillWare Says:

    Let's recall Joy advocated relinquishment

    We've not yet fully understood Joy.

    Understanding the dangers Joy identified is different from understanding his proposed "solution". The uproar in the nanotech community was because his "solution" was precisely what Lessig has pointed out as hopelessly ineffective, and which is now failing in both Iraq and the music industry.

    Abolishing freedom … and launching wars only increases the danger that [dangerous weapons] will be used.

    Joy didn't understand this point at all. The dangerous thing about his proposal of relinquishment is that (assuming we Western democracy types are the "good guys") it guarantees that dangerous weapons will be developed first by the "bad guys". Hmm, grant an overwhelming military advantage to people who prefer might-makes-right to our concepts of liberty, rule of law, trial by jury, etc. That's a pretty effective recipe for everything reasonable people would hope to avoid.

    Lessig has thought this stuff through with much deeper insight than Joy ever did. It may be impolite for him to point that out himself, but he should at least be aware that it's true.

  4. RobertBradbury Says:

    Re:This isn't nanotech…

    Biotech *IS* nanotech. To argue that it isn't you only have a single argument — that biotech doesn't have the covalent bond density per unit volume that diamondoid has. If you study Table 16.1 in Nanosystems carefully one can understand there are multiple stages of nanotechnology. Stages 1&2 are very close to biotechnology. Stages 3&4 are "classical" nanotechnology. So when one says this "is" or "isn't" nanotech is very arbitrary.

    I've already pointed out to Steve Jurvetson that viruses (e.g. mousepox, smallpox, etc.) are not really self-replicating and only bacteria (e.g. anthrax) are.


  5. RobertBradbury Says:

    Re:Let's recall Joy advocated relinquishment

    Relinquishment clearly doesn't work unless we want to send humanity back to the dark ages. But we do have to be very clever about seeing what the problems might be. That includes everything from radioactivity detectors for shipping containers to robots that scan train tracks ahead of trains to image systems that recognize people leaving behind bags to stockpiles of antibodies against various viruses, etc.

    There is a lot that can be done with an emphasis on defensive strategies.

    In addition to this one has to use biotech and nanotech to improve conditions in the world to eliminate incentives for people to expose themselves to self-sacrifice/imprisonment because they are part of the class of "have-nots" rather than "haves".

    But you are still going to have a fraction of people in the world who are "crazy". You can cite everything from the Aum Shinrikyo to Jim Jones as examples. Against those like them one needs robust defenses. As both 911 and recent events in Spain show one will never have perfect intelligence to know when they may do something crazy. The *only* way to deal with this is to anticipate what they might attempt and be prepared for it.


  6. WillWare Says:

    Re:Let's recall Joy advocated relinquishment

    Relinquishment clearly doesn't work… But we do have to be very clever about seeing what the problems might be… There is a lot that can be done with an emphasis on defensive strategies.

    Certainly, I wouldn't advocate complacency about this. Hopefully we can take some qualitative lessons from has been learned in recent years with computer virii/trojans/worms.

    I remember thinking about some of this stuff a long time ago, and found a Usenet thread about it. Rob Freitas was working out some of the ideas that later became his paper on ecophagy. There is some interesting thinking there about the threshold of what is detectable, assuming the threat is something self-replicating.

    [avoid big disparity between haves and have-nots]

    Absolutely. Give people vastly more wealth and self-determination than they've had in the past, and a lot of the motivations for mischief will go away. If we could end our dependence on oil and refrain from further mucking around with the Middle East, they might eventually get bored with killing Americans.

    one will never have perfect intelligence to know when they may do something crazy.

    The Smart Dust idea may go a long way toward giving us nearly perfect intelligence, if we can get a shrink of another linear factor of ten from the Berkeley prototype. The Berkeley folks are working aggressively on commercializing this stuff (alas, they've beat my alma mater to the punch). I'm sure espionage would be an application they'd be giving a great deal of thought too, as their initial grant money was from DARPA.

  7. WillWare Says:

    Re:Let's recall Joy advocated relinquishment

    There is a lot that can be done with an emphasis on defensive strategies.

    I almost forgot about another interesting Usenet thread on this topic. The best idea I could come up with at the time (and I haven't improved on it since) was that there should be red teams and blue teams, with the red teams taking the role of terrorist weapon designers, and the blue teams designing defenses. The design output of the blue teams would be released publicly whenever it was deemed safe to do so, for example whenever the offensive weapon couldn't be inferred from the defense, or when the defense had already been put in place.

Leave a Reply