Lethal Autonomous Weapon Systems and the Future of Warfare

Lethal Autonomous Weapon Systems and the Future of Warfare
Adriano Iaria*

The enormous advances made in the field of artificial intelligence and deep machine learning over the past years have led to mounting fears and debates on the future of warfare in the digital and autonomous era. Last August, 117 leading experts in the fields of robotics and artificial intelligence, including Elon Musk, the CEO of Tesla, called on the UN to ban the development and use of “killer robots”, warning of the dangers of a “third revolution in warfare” if legal norms and treaties are not updated to keep the pace of technological advancement.[1]

Like previous technological revolutions, artificial intelligence poses new challenges for international humanitarian law, particularly regarding accountability and the legal regulations governing the use of force. In fact, after an initial interference by a human operator, or automatically for defense purposes,[2] lethal autonomous weapon systems (LAWS) are capable of selecting – using sensors, image recognition and software – both the target and means of attack.

International actors are today seeking answers to certain ethical and legal questions concerning the relationship between LAWS and international humanitarian law (IHL). For instance, what happens if an autonomous weapon system commits a grave breach of international humanitarian law? Who is responsible for the action, the operator, the software programmer or the system itself?

When states engage in conflict, they are under obligations to guarantee the full respect of IHL and, inter alia, five customary rules apply: states shall distinguish between combatants and civilians, an attack must be proportional and take all the necessary measures to avoid superfluous injury, unnecessary suffering or serious damage to the natural environment.

According to the definition of LAWS provided by the International Committee of the Red Cross (ICRC), an autonomous weapons system is defined as: “any weapon system with autonomy in its critical functions. That is, a weapon system that can select (i.e. search for or detect, identify, track, select) and attack (i.e. use force against, neutralize, damage or destroy) targets without human intervention”.[3]

The act of war has always spurred legal and ethical debates, whether limits to war-waging exist and what these limits are. At the end of the 19th century, ethical and legal aspects found a strong link with the so-called “Martens Clause”, which established that “in cases not covered by existing treaties, civilians and combatants remain protected by customary IHL, the principles of humanity and the dictates of the public conscience”.[4] The Martens Clause has been integrated in several treaties and is considered a rule of customary law. Appling this to artificial intelligence and autonomous weapon systems poses some problems however. How can autonomous weapon systems apply the principles of humanity and the dictates of public conscience if they are not humans?

Such issues are particularly important for those states who have ratified the 1977 Additional Protocol I to the Geneva Conventions, which stipulates that states do not have complete autonomy to choose their preferred “methods or means of warfare” (Article 35). Moreover, Article 36 states that “in the study, development, acquisition or adoption of a new weapon, means or method of warfare”, states party to the protocol are “under an obligation to determine whether its employment would […] be prohibited by this Protocol or by any other rule of international law” applicable to states parties.[5] This wording clearly applies to the development and use of LAWS.

In December 2016, states party to the Convention on Certain Conventional Weapons (CCW) agreed to establish an open-ended Group of Governmental Experts (GGE) to address issues concerning such autonomous technologies. Eighty-six countries, over 60 civil society groups, the United Nations Institute for Disarmament Research (UNIDIR) and the ICRC attended the first meeting of the group, held between 13-17 November 2017, to develop legal rationales prohibiting the development, production and use of lethal autonomous weapons systems.[6]

While a majority of countries, particularly among the Non-Aligned Movement (NAM), are calling for a legally-binding instrument to prohibit LAWS,[7] a handful of states, including the US and Russia, are reluctant to start negotiations on weapons that do not exist and have not yet been deployed.[8] Such arguments are quite weak however, considering that the 1995 Protocol on Blinding Laser Weapons (Protocol IV to the CCW) marked a milestone in the field of disarmament. In fact, as argued in the aftermath of the approval by the ICRC, “The prohibition, in advance, of the use of an abhorrent new weapon the production and proliferation of which appeared imminent is an historic step for humanity. It represents the first time since 1868, when the use of exploding bullets was banned, that a weapon of military interest has been banned before its use on the battlefield and before a stream of victims gave visible proof of its tragic effects.”[9]

Nevertheless, and largely resulting from the objections of large influential states, the first meeting of the GGE ended without a working definition of autonomous weapon systems, a major impediment to future progress on the issue. Any international instrument – a moratorium, a treaty or declaration – will be affected by the working definition of LAWS.

New meetings of the group are planned for 2018.[10] The role of international civil society will be key to progress in this domain. Such actors have recently secured some important victories in the field of international disarmament. The 2013 Arms Trade Treaty and the 2016 Treaty on the Non-Proliferation of Nuclear Weapons would not have been adopted without the active support and advocacy played by civil society actors.[11] The decision to assign the 2017 Nobel Peace Prize to the International Campaign to Abolish Nuclear Weapons (ICANW) is a further recognition of the work made by civil society in addressing disarmament issues.

While the implications of a potential proliferation of LAWS technology may appear distant and remote, limited for the time being to the domain of science fiction and celebrated futuristic movies, technological advances are progressing quickly and could pose concrete threats to both international peace and security and the present legal regimes and norms governing warfare and the use of force. Awareness about the risks posed by such technology will therefore be key to an effective prevention campaign. One recently publicized case in which engineers working with Facebook apparently conducted an experiment where two AI robots began talking to each other in an unknown language provides an example of the unknown territory lies ahead.[12] In this respect, one should heed the advice provided by leading experts in the field, including Elon Musk, Stephen Hawking and Bill Gates, all of which have called for a complete ban on “killer robots” before such technology crosses the Rubicon becoming almost impossible to limit and control.[13] A strong legal framework governing the development and use of lethal autonomous weapon systems is therefore a key priority for international disarmament debates.

* Adriano Iaria collaborates with the Rome-based Institute of International Research Archive Disarmament (IRIAD) and is an instructor of international humanitarian law for the Red Cross. In 2012, he supported the Permanent Mission of Italy to the United Nations in the context of negotiations for the Arms Trade Treaty (ATT).

[1] Samuel Gibbs, “Elon Musk Leads 116 Experts Calling for Outright Ban of Killer Robots”, in The Guardian, 20 August 2017, https://gu.com/p/73hdq; Open Letter to the United Nations Convention on Certain Conventional Weapons, 2 August 2017, https://futureoflife.org/?p=13940.

[2] Examples of these systems for defense purposes are the Dutch Goalkeeper Close-in Weapon System, the Israeli Iron Dome, the Russian Kashtan Close-in Weapon System and the German Nächstbereichschutzsystem (NBS) MANTIS.

[3] International Committee of the Red Cross (ICRC), Views of the ICRC on Autonomous Weapon Systems, 11 April 2016, p. 1, https://www.icrc.org/en/document/views-icrc-autonomous-weapon-system.

[4] Neil Davison, “A Legal Perspective: Autonomous Weapon Systems under International Humanitarian Law”, in UNODA Occasional Papers, No. 30 (November 2017), p. 8, https://www.un.org/disarmament/?p=182797. See Preamble of the Convention (IV) respecting the Laws and Customs of War on Land and its annex: Regulations concerning the Laws and Customs of War on Land, The Hague, 1907; Preamble of the Convention (II) with Respect to the Laws and Customs of War on Land and its annex: Regulations concerning the Laws and Customs of War on Land, The Hague, 1899; and article 1 of the Additional Protocol (I) of 1977 to the Geneva Conventions.

[5] See Additional Protocol I to the Geneva Conventions, 1977, https://ihl-databases.icrc.org/applic/ihl/ihl.nsf/INTRO/470.

[6] Campaign to Stop Killer Robots, Support Grows for New International Law on Killer Robots, 17 November 2017, https://www.stopkillerrobots.org/?p=6579.

[7] See General Principles on Lethal Autonomous Weapons Systems. Submitted by the Bolivarian Republic of Venezuela on behalf of the Non-Aligned Movement (NAM) and Other States Parties to the Convention on Certain Conventional Weapons (CCW) (CCW/GGE.1/2017/WP.10), 13 November 2017, https://www.unog.ch/80256EE600585943/(httpPages)/F027DAA4966EB9C7C12580CD0039D7B5.

[8] Campaign to Stop Killer Robots, Support Grows for New International Law on Killer Robots, cit.

[9] ICRC, Vienna Diplomatic Conference Achieves New Prohibition on Blinding Laser Weapons and Deadlock on Landmines, 13 October 1995, p. 3 [emphasis added], https://www.icrc.org/eng/resources/documents/misc/57jmlc.htm.

[10] United Nations, Final Report of the 2017 Meeting of the High Contracting Parties to the CCW. Advance version, 29 November 2017, p. 9, https://www.unog.ch/80256EE600585943/(httpPages)/A0A0A3470E40345CC12580CD003D7927.

[11] Adriano Iaria, “Consensus, Expertise and Lobbying: How Civil Society Has Influenced the Arms Trade Treaty (ATT)”, in Sistema Informativo a Schede, No. 10/2015 (October 2015), http://www.archiviodisarmo.it/index.php/it/2013-05-08-17-44-50/sistema-informativo-a-schede-sis/sistema-a-schede/finish/132/3972.

[12] Adrew Griffin, “Facebook’s Artificial Intelligence Robots Shut Down After They Start Talking to Each Other in Their Own Language”, in The Independent, 31 July 2017, http://www.independent.co.uk/life-style/gadgets-and-tech/news/-a7869706.html.

[13] Samuel Gibbs, “Musk, Wozniak and Hawking Urge Ban on Warfare AI and Autonomous Weapons”, in The Guardian, 27 July 2015, https://gu.com/p/4b2em; Open Letter on Autonomous Weapons, 27 July 2015, https://futureoflife.org/?p=1793.