Wendell Wallach

Wendell Wallach

In April, the United Nations will hold meetings in Geneva for the third year in a row on the call to ban lethal autonomous weapons systems (AWS). This meeting and follow-up activities offer a unique opportunity for Canada and its new government to internationally assert moral leadership. While there is an emerging consensus that any short-term benefits of AWS systems will be dwarfed by detrimental longer-term impacts, forging an arms-control agreement to restrict their use will be difficult. Thus, even States favorable to a ban have held back from making any commitment. What is needed is one or two major countries with the courage to step forward and affirm that a ban is a good idea, presuming consensus can be garnered on means to ensure it will be effective. Given Canada’s past leadership in development of the ban on landmines, known as the Ottawa Treaty, this could be seen as a natural role for the new Liberal government, and a statement that Canada is again ready to be a moral leader on the world stage. While Canada made a considerable investment in the landmine ban, little or no investmentwould be necessary at this time to move forward a ban on AWS.

The April meeting in Geneva will be conducted by the Convention on Certain Conventional Weapons (CCW). Currently 115 States are parties to the convention and an additional five states have signed but not ratified the three Protocols, which designate prohibitions on the use of specific weapon systems. AWS refers to robotic weapons in the relatively near future that both select and destroy their own targets. Debate focuses largely upon whether such weapon systems should be prohibited from killing people unless they are under real time meaningful human control. Definitions for terms such as “autonomy” and “meaningful human control” are contested, but the basic idea is that a robotic weapon must not kill humans without a clear real-time go ahead from an authorized military leader who can be held responsible for the act, and particularly the death of non-combatants.

A ban on AWS is uniquely problematic for a number of reasons:

• Unlike other weapons that have been banned, some uses of AWS are perceived as morally acceptable. For example, they are used defensively to quickly counteract an incoming attack. More importantly, in offensive combat they can be substituted for, and thus save the lives of, one’s own soldiers.

• It is unclear whether armed military robots should be viewed as weapon systems or weapon platforms.

• Arms control usually focuses on working out modes of verification and inspection regimes to determine whether adversaries are honoring the ban. The difference between a lethal and non-lethal robotic system may be little more than a few lines of code, which would be difficult to detect and could be deleted before or added after an inspection. In other words, if a party elects to cheat, there is no means to enforce a ban in advance.

• In the future, AWS systems will be relatively easy to assemble using technologies developed for civilian applications.

Nevertheless, there is strong sentiment among both civilians and a large portion of the military that AWS are a game changer and should be banned. Of course there are some within military planning circles, and within the military industrial complex, who either favor the development of AWS, or do not want their options limited by “well-meaning” but “naïve do-gooders.” They contend that either the sentiment against robotic weaponry is based upon science fiction or that it would be impossible to put in place an effective ban, and therefore we must develop advanced robotic weaponry before our adversaries do. After all, no military planner wants to surrender the strategic advantage gained from having the most advanced technology. The difficulty with “we must do it before our adversaries do” logic is that it drives a never-ending and rapidly escalating development of more and more advanced weaponry. Furthermore, the countries that develop new weapons first in order to have technological superiority speed up the process and lower the cost to countries that follow suit by riding upon the proof of concept and lessons learned from the initial research. Furthermore, espionage, poor export controls, and backward engineering also speed-up and lower cost to countries who follow the leaders.

There is presently a window of opportunity to enact a ban, but that window could close quickly once countries begin deploying even relatively dumb systems. Delay plays into the hands of those who desire such weaponry. Precipitating confusion as to whether a ban can be forged is a powerful tool for those who want to delay any action. Indeed, some opponents of a ban argue there is no rush, because such systems do not yet exist. However, U.S. Defense Department officials claim they do not want to take humans out of the decision-making loop, but then go on to suggest that Russia, and perhaps China, are moving forward in the development of autonomous weapons systems.

Given the unique challenges posed by AWS, arms negotiators will need to be very creative in forging a new approach to banning their use. I have suggested that we first agree that machines making life and death decisions are mala in se (a thing that is bad or evil in and of itself) because they are unpredictable and can’t be fully controlled, and their use would make attribution and affixing responsibility for wrongful death difficult if not impossible. Once this principle is in place, negotiators can move on to what will be a never-ending debate as to whether or when AWS are extensions of human will and intention and under meaningful human control. It will be impossible to define when that line has been crossed, but with a strong moral principle in place it will be possible to condemn egregious acts.

The lack of discrimination in present-day artificial intelligence means that any AWS would be a violation of international humanitarian law, also known as the laws of armed conflict. Simply clarifying this point could be seen as a first step in declaring that machines making life and death decisions are mala in se. If, for example, Prime Minister Trudeau, declared that Canada views AWS as violating existing international humanitarian law, his government would convey a strong message that would force other countries to agree or disagree, but in either case they would stop hiding behind definitional ambiguities as a delaying tactic.

Wendell Wallach is the ISSP's 2015-2016 Visiting Fulbright Research Chair in Science and Society. He is also a scholar of Yale University’s Interdisciplinary Center for Bioethics, senior advisor to the Hastings Center), and author of “A Dangerous Master: How to Keep Technology from Slipping Beyond Our Control.

Back to top