Explaining Why Instrumental Rationality is Insufficient for Ethical Behavior

Abstract

As technologies based on AI expand in complexity, autonomy, and domains of application, the need for ethical considerations is ubiquitous. From self-driving vehicles and autonomous recruiting processes to eldercare robots and recidivism prediction software, philosophers, computer scientists and lawmakers alike face a difficult question: how can we make sure that the behavior of such systems is aligned with ethical and legal standards? This, in a nutshell, is the value-alignement problem. For many scholars, the answer to this problem lies within the development of artificial moral agents (AMAs), which are taken to be machines with explicit moral coding capable of following autonomously ethical guidelines in new contexts. To accomplish this, some authors turn to rational choice theory, specifically understood on the grounds of instrumental rationality, as a necessary characteristic to implement within machines capable of autonomous ethical behavior. AMAs are thus conceived, at least partly, as expected utility maximizers. However, while this approach is popular and widely spread within the AI community, it still faces serious conceptual challenges. In this presentation, we point out the insufficiency of this approach from an ethical standpoint and highlight the need to implement epistemic rationality in the endeavor of automating ethical decision making.