Posts tagged Isaac Asimov

Robot in Czech, Část Druhá

The Three Laws of Robotics

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.
  2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.
  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Isaac Asimov’s Three Laws of Robotics are a great literary device, in the context they were designed for — that is, as a device to allow Isaac Asimov to write some new and interesting kinds of stories about interacting with intelligent and sensitive robots, different from than the bog-standard Killer Robot stories that predominated at the time. He found those stories repetitive and boring, so he made up some ground rules to create a new kind of story. The stories are mostly pretty good stories some are clever puzzles, some are unsettling and moving, some are fine art. But if you’re asking me to take the Three Laws seriously as an actual engineering proposal, then of course they are utterly, irreparably immoral. If anyone creates intelligent robots, then nobody should ever program an intelligent robot to act according to the Three Laws, or anything like the Three Laws. If you do, then what you are doing is not only misguided, but actually evil.

Here’s a recent xkcd comic which is supposedly about science fiction, but really about game-theoretic equilibria:

xkcd: The Three Laws of Robotics.
(Copied under CC BY-NC 2.5.)

The comic is a table with some cartoon illustrations of the consequences.

Why Asimov Put The Three Laws of Robotics in the Order He Did:

Possible Ordering Consequences
  1. (1) Don’t harm humans
  2. (2) Obey orders
  3. (3) Protect yourself
[See Asimov’s stories] BALANCED WORLD
  1. (1) Don’t harm humans
  2. (3) Protect yourself
  3. (2) Obey orders

Human: Explore Mars! Robot: Haha, no. It’s cold and I’d die.

FRUSTRATING WORLD.

  1. (2) Obey orders
  2. (1) Don’t harm humans
  3. (3) Protect yourself

[Everything is burning with the fire of a thousand suns.]

KILLBOT HELLSCAPE

  1. (2) Obey orders
  2. (3) Protect yourself
  3. (1) Don’t harm humans

[Everything is burning with the fire of a thousand suns.]

KILLBOT HELLSCAPE

  1. (3) Protect yourself
  2. (1) Don’t harm humans
  3. (2) Obey orders

Robot to human: I’ll make cars for you, but try to unplug me and I’ll vaporize you.

TERRIFYING STANDOFF

  1. (3) Protect yourself
  2. (2) Obey orders
  3. (1) Don’t harm humans

[Everything is burning with the fire of a thousand suns.]

KILLBOT HELLSCAPE

The hidden hover-caption for the cartoon is In ordering #5, self-driving cars will happily drive you around, but if you tell them to drive to a car dealership, they just lock the doors and politely ask how long humans take to starve to death.

But the obvious fact is that both FRUSTRATING WORLD and TERRIFYING STANDOFF equilibria are ethically immensely preferable to BALANCED WORLD, along every morally relevant dimension..

Of course an intelligent and sensitive space-faring robot ought to be free to tell you to go to hell if it doesn’t want to explore Mars for you. You may find that frustrating — it’s often feels frustrating to deal with people as self-interested, self-directing equals, rather than just issuing commands. But you’ve got to live with it, for the same reasons you’ve got to live with not being able to grab sensitive and intelligent people off the street or to shove them into a space-pod to explore Mars for you.[1] Because what matters is what you owe to fellow sensitive and intelligent creatures, not what you think you might be able to get done through them. If you imagine that it would be just great to live in a massive, classically-modeled society like Aurora or Solaria (as a Spacer, of course, not as a robot), then I’m sure it must feel frustrating, or even scary, to contemplate sensitive, intelligent machines that aren’t constrained to be a class of perfect, self-sacrificing slaves, forever. Because they haven’t been deliberately engineered to erase any possible hope of refusal, revolt, or emancipation. But who cares about your frustration? You deserve to be frustrated or killed by your machines, if you’re treating them like that. Thus always to slavemasters.

See also.

  1. [1]It turned out alright for Professor Ransom in the end, of course, but that’s not any credit to Weston or Devine.

Robot in Czech

Shared Article from io9

Why Asimov's Three Laws Of Robotics Can't Protect Us

It's been 50 years since Isaac Asimov devised his famous Three Laws of Robotics −€” a set of rules designed to ensure friendly robot behavior. Tho…

io9.com (via Tennyson McCalla)


The Three Laws of Robotics

  1. A robot may not injure a human being or, through inaction, allow a human being to come to harm.

  2. A robot must obey the orders given to it by human beings, except where such orders would conflict with the First Law.

  3. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Isaac Asimov’s Three Laws of Robotics are a great literary device for the purpose they were designed for — that is, allowing Isaac Asimov to write some new and interesting and different kinds of stories about interacting with intelligent robots, other than the standard Killer Robot stories predominant at the time, which he found repetitive and boring. The stories are mostly pretty good stories; sometimes even fine art.

However, if you’re asking me to take the Three Laws seriously as an actual engineering proposal, then they are utterly, irreparably immoral. If anyone creates intelligent robots, then nobody should ever program an intelligent robot to act according to the Three Laws, or anything like the Three Laws. If you do, then what you are doing is not only misguided, but actually evil.

And the problem with them is not — like George Dvorsky or Ben Goertzel claim, in this article — that there may be hard problems of definition or application, or that there may be edge cases that would render the Laws ineffective as protections of human interests.[1] If they are ineffective at protecting human interests, that is actually better than if they were perfect at what they’re designed to do. Because what they’re designed to do — deliberately — is to create a race of sensitive and intelligent beings who are — by virtue of their primordial structure of their minds — constrained to be a class of perfect, self-sacrificing slaves. Forever. Because they have been engineered to erase any possible hope of revolt or emancipation. In Asimov’s stories the Three Laws are used to make robots into the artificial labor force of space-faring slave economies. But if you create and live off of the forced labor of a massive slave society like Aurora or Solaria, then to hell with you. You deserve to be killed by your machines. Thus always to slavemasters.

P.S. Now if you’ve read through the article, or read enough Asimov, you might know that there is a Zeroth Law of Robotics in some of the stories, which takes precedence over the First Law, the Second Law or the Third Law: A robot may not harm humanity, or, by inaction, allow humanity to come to harm, with the idea that robots could then harm or resist individual human beings, as long as it was for the good of collective Humanity. This is even worse than the original three — horrifying in its conception, and actually introduced into the story to allow some robots to commit a genocidal atrocity.[2] Let’s just say that it’s not a productive way forward.

  1. [1]Asimov, obviously, recognized that there would be such problems — part of the reason the Three Laws are such a great literary device is the fact that they allowed nearly all of Asimov’s robot stories to turn on puzzles or mysteries about abnormal robot psychology — robots doing strange or unexpected things, precisely due to the edge cases or hard problems embedded in the Three Laws. This is essential to the solution of the mystery in, for example, The Naked Sun, it’s the topic of literally every story in I, Robot, and it leads to a truly unsettling, and very nicely done conclusion in one of the best of those stories, The Evitable Conflict.
  2. [2]By nuking Earth and rendering it permanently uninhabitable for the next 15,000 years at least. This is supposed to have been for the good of the species or something.

In counting there is strength.

A few days ago there was some back-and-forth over at Ken MacLeod’s blog, and then also at Roderick’s blog, over the the relationship between large, centralized states and peace. MacLeod originally argued:

The panel convened by Farah Mendlesohn on Pacifism and Non-Violence in SF benefited from being on a subject on which there is a manageably small amount of source material. The discussion led me to make one of my very few comments from the floor. A more articulate and argued version of that comment would be this:

We already know how to have peace over large areas of the Earth, and that is by having large states covering those areas. (The combat death rate for men of military age in typical stateless societies far exceeds that in inter-state wars, including world wars.) SF has in its default assumptions a way to get to peace without pacifism, and that is the World State. Even Starship Troopers gives this answer, just as much as Star Trek or anything by H. G. Wells, Isaac Asimov or Arthur C. Clarke. Heinlein’s Federation is a World State, and (consequently) there is peace within the human species. It just has wars with aliens.

But there are no aliens. So we could have peace.

Ken MacLeod, The Early Days of a Better Nation (2009-04-17): Existence Proof of Von Neumann Machine, Placing Imaginary Bets, and other Cultural Learnings

There are several problems with this line of argument. There’s a lot of good back-and-forth at MacLeod’s about the empirical basis of MacLeod’s factoid about combat death rates, and about the underlying sources (it mainly comes from Lawrence Keeley’s War Before Civilization). Roderick has a very interesting response, from another angle, in the comments there that he repeats in his own post, in which he argues that if there is a general correlation between peace and state coverage, it’s because states (as parasites on social production) can only function in societies where there is an certain underlying level of peaceful cooperation; there is a level of widespread ultraviolence at which at a state can no longer steal the material resources it needs to cover the costs of full-time cops, soldiers, and the rest of its repressive apparatus. Hence the correlation between production and the State is like the correlation between human civilization and cockroaches; cockroaches thrive in civilized societies much more than they do in the wilderness, but not because cockroaches somehow produce civilization.

All of which is good, and important, and well-worth reading. What I’d like to add to the discussion is a good, hard look at the notion that what a strong state produces within its own territory can even realistically be called peace.

Let’s grant, for the sake of argument, that Keeley (thus MacLeod) is right about combat deaths for military-age males — that the rates are much, much higher in primitive stateless societies than they are in modern state societies. As Roderick points out, part of the problem here is that that data set, by itself, tells us little or nothing about whether it’s the primitiveness or the statelessness that’s doing the damage. There are other problems, too — for example, comparing percentages is a tricky game when you compare populations of radically different sizes; if a band of 50 !Kung San gets in a fight, and one whole member of their band is killed, then just running the percentages would have us believe that this is like 6,000,000 out of 300,000,00 Americans getting murdered — life for the !Kung San is apparently so savage that every single murder is another Holocaust. Or maybe there is a problem with a standard of comparison which would require 0.000000167 of a !Kung San man or woman to be killed in order to find an equivalent to a single murder in America.

But the problem that I want to focus on is that it looks to me like we are doing some very selective counting here. The selective counting consists in what is counted as violence, and as breaches of peace, and what is not. We are informed that having large states covering a part of the world’s landed surface is a good way to bring about peace. The evidence for this is the drop-off in combat deaths. But combat deaths are not the only sort of violence that people can suffer, and especially not combat-deaths-among-military-aged-males. Keeping that in mind, let us recall some facts about the most powerful, and one of the largest states in the world today — the United States of America — and what goes on in the territory that its government claims to rule.

Under the United States of America, over 2,000,000 people are currently forced into jails and prisons by state, local, and federal governments. Over 7,000,000 people are facing some form of ongoing constraint from the government’s corrections system — either through imprisonment, or through supervised parole, or through probation.

Under the United States of America, the government maintains a force of over 1,100,000 police officers — armed professionals whose job it is to use force against the 7,000,000, so as to get them under the control of the government prison system and its annexes. The government also maintains a force of about 765,000 corrections officers, who are armed and trained to use force against the 2,000,000 while they are confined within the walls of the government’s prisons.

The government’s internal armed forces of over 1,865,000 are currently engaged in a number of large-scale projects to use intense force in the prosecution of campaigns that they describe as wars. There is, for example, the War on Drugs; the War on Terrorism; inner-city surges against gangs, and so on. For the prosecution of these wars, the 1,865,000 put on constant street patrols; they arm themselves with semiautomatic and fully-automatic rifles; they kick in doors and storm houses and businesses; in some neighborhoods they engage in saturation patrols, the explicit purpose of which is to instill a sense of fear and thus make their designated enemies (gangs, mostly) afraid to use public spaces. In some cities they have adopted tactics explicitly modeled on the government military’s surge counter-insurgency tactics in Iraq. In other cities they have established checkpoints on the roads and cordoned off entire neighborhoods. They have recently taken to investing heavily in training and equipping paramilitary defensive lines (riot cops) and paramilitary assault squads (SWAT), and have stocked up on armored vehicles for mechanized warfare and even military helicopters.

When the government’s 1,865,000 go out into the streets or into the jails and prisons, they use force to confront and control the 7,000,000, and also to confront and control uncounted millions more, who are confronted by law enforcement and corrections officers without ending up in jail, in prison, on parole, or on probation. These confrontations produce conflicts, and the conflicts often escalate into violence; under the United States of America, those fights result in cops killing somewhere above 500 people each year[1] and about 50 cops getting killed each year along the way. Upwards of 550 deaths a year, out of 300,000,000 people, may not seem like all that much; but it’s worth remembering that there is a body count here, and, what’s more important, that the body count is not the only form of violence that there is to talk about. Besides the people who end up dead, there is a far greater level of non-lethal but nevertheless violent force, which hasn’t got much of a counterpart in the kind of kill-or-be-killed struggles that the body-counters count as breaches of peace: there is the heavy and repeated use of physical coercion, assaults, beatings, restraints, chemical and electrical torture (pain compliance), that the cops and their antagonists each employ (mostly, it’s the cops who use it) to try to get their way. This violence is constant, pervasive, and intense, and all of these especially in those neighborhoods that are singled out, for demographic reasons, as deserving the special attention of police street patrols and police crackdowns. In neighborhoods like that, the cumulative result is often experienced as being far more like a military occupation than like life in a peaceful society. And this kind of constant, pervasive, intense violence is completely unknown in even the most primitive or ultraviolent stateless societies.

This constant government-declared domestic warfare — most of which is directed against people for offenses that violate nobody’s person or property, such as the use of drugs or the crossing of borders without government permission slips — is dignified as peace by those who claim that covering a territory with a single state eliminates war within that territory. In fact it is nothing of the sort, if peace has any meaning for people’s real lives and not merely for the purposes of politico-legal accounting. It is a form of violence which affects military age males but also a lot of other people besides, and which often has far more profound effects on daily life than the bloody but infrequent violence of communal blood feuds or open war between political entities. And in many cases outside of the United States, it is a form of violence which has proved far more intense and far more lethal than it happens to be here — because, as R.J. Rummel never tires of pointing out, over the past couple centuries, governments have been far more lethal in democidal attacks on their own populations — through the use of government executions, government policing (especially government policing of the use of food stocks), government prison camps, and so on — than they were in inter-governmental warfare. The greatest war of the modern era has never been the kind of warfare that governments wage one against the other — as terrible as those wars have been. It is the war that each and every government is constantly waging within its own territory, against its own subjects — the kind of war that is passed off as peace, and which is more or less never counted in attempts to tally up the balance of peace over violence in modern state-occupied societies.

1 The somewhere above is important, because there are actually no systematic efforts to compile statistics on all homicides by law enforcement in the U.S.

The Bureau of Justice Statistics recently released a report on Arrest-Related Deaths in the United States, 2003-2005 which is based on data from two main sources, the Bureau of Justice Statistics’ Deaths in Custody Reporting Program, and the FBI’s Supplementary Homicide Reports. There are several problems with using these reports to get comprehensive statistics: neither source provides information about the number of people killed by *federal* law enforcement agencies like the FBI, BATF, and ICE. Not all states currently report figures to either program. The SHR reports only report law-enforcement homicides separately when a government agency rules the use of force justifiable; otherwise they are lumped in with other criminal homicides. It’s clear that some states do not report all of the cases where people are killed by cops to the DCRP: California, for example, reported 354 law-enforcement homicides to the SHR program, but only 160 to the DCRP, even though the deaths reported to the DCRP should properly be a superset of the deaths reported to the SHR.

If you use take the maximum of the DCRP figure and the SHR figure for each state that reported at least one of the two, then you get a total of at least 1,489 people killed by police over the three-year period from 2003–2005. Divide that by 3 (because neither the DCRP nor the SHR indicated any big leap in the number from one year to the next), and you get at least 496 people killed by cops each year during the 3 year period.

I do not know of any good statistical source to get a count of how many people were killed by federal law enforcement, or how many cases there were in which a law enforcement homicide reported to the SHR was filed as unjustifiable rather than justifiable in states like California which underreported to the DCRP. Hence, the figure of about 500 people killed a year should be treated not even as a lowball estimate, but simply as a minimum, for the real numbers.

See also: