Life was quiet in the small town of Wanzer close to the former GDR border until shortly after the turn of the millennium. However, a few years later, everything changed: On week days, the remote country road that winds through the residential area of the town became filled with trucks. The introduction of an HGV toll on German motorways meant lorry drivers were re-routing through Wanzer. The drivers were using GPS navigation systems and smartphones with routing algorithms that sent them to the small town. The algorithms suggested that this section of previously quiet road would help drivers avoid paying a few Euros at the toll, while at the same time keeping traveling time between destinations in Eastern and Western Europe to a minimum. The former serenity and joy in nature behind the dike along the Elbe river in Wanzer was replaced by traffic noise and the scent of exhaust fumes.
The inhabitants of the town experienced first hand how automated decisions – that they have no influence over – can touch daily life in unexpected ways. We placed this example at the beginning of our report because we do not want to merely cover the familiar use cases of automated decision-making, such as Predictive Policing, in our Atlas of Automation. We also want to show that our entire everyday life is interfused with small or big automated decisions that we might not even register as such – and yet have consequences.
WHAT IS THE SCOPE OF THIS ATLAS?
A comprehensive inventory of automation across all sectors of society would be a massive undertaking. To present the perennial transformation of industry alone would need a separate and extensive deliberation. With this Atlas, we therefore chose to concentrate on one specific aspect which we consider to be particularly relevant: The relevance for social participation. Other than the term discrimination, the term participation does not only examine the systematic disadvantage of specific groups in an abstract way, it also brings our attention to the field in which this disadvantage can take place. Participation means access to public goods and services as well as the ability to claim one’s rights. Examining systems of automated decision-making, in respect to their relevance for social participation, means looking at ways in which these systems could impede access to public goods and services as well as the ability to exercise one’s rights – especially for persons who are already destitute or can be described as being disadvantaged. We consider the extent to which such groups of people are enabled or refused from participating socially to be a decisive indicator for the state of democracy in a society.
Despite all the criticism, we do not want to give the impression that automation should be rejected. In its 200 year long history, automation has brought about enormous social progress and improved quality of life to such an extent that no-one would want to roll back the clock. Hence, the Atlas not only points out the potential for discrimination that can accompany automated decisions, it also illustrates the opportunities and benefits that the use of automated decisions make possible or imaginable.
WHY NOT SPEAK OF ARTIFICIAL INTELLIGENCE?
The term “Artificial Intelligence” (AI) has had something of a renaissance over the past two years. It was first coined more than sixty years ago, and ever since there have been repeated periods during which it was en vogue for some time to use the term “Artificial Intelligence”. We have our doubts as to whether the current excitement for this term, which is mainly based on progress in “Machine Learning” and “Neural Networks”, will last very long. Outside of the world of experts, the use of the term AI is rather unclearly defined and some people award it almost magic capabilities. In the German government’s AI strategy, almost anything in connection to digitalization is summed up under “AI”. To which extent machines will ever be able to show “intelligence” that equals human autonomy and intentionality has long been disputed. However, it is agreed that the AI of today is still a very long way from human capability.
Mixing up automated decision-making with the term artificial intelligence rather leads us down the wrong path. When it comes to automated decision-making, what is essential is the fact that humans delegate machines to prepare decision-making or even to implement decisions. These machines can consist of highly complex neuronal networks, but they can also be made up of fairly simple software which calculates, weighs and sorts data according to a plain set of rules. In this Atlas, we therefore consistently speak of automated decision-making (ADM – see box) instead of artificial intelligence – even though many of the phenomena that we group under the term ADM could also be called AI applications.
In the past ten years, we have seen an unprecedented increase in automation through software. The amount of available data that helped make ADM possible in the first place, together with the proliferation of devices, equipment and infrastructure that are essential for ADM, have grown exponentially. With this, the influence of ADM on various aspects of society has inevitably expanded. As a result, we want our Atlas of Automation to initiate debate in society on the consequences of ADM. This is something that we believe to be an urgent necessity.
THE ELEMENTS OF THIS ATLAS
The current Atlas is a collection, not of geographical maps and diagrams, but of topics that in our view are particularly relevant in regard to the issue of ADM in connection with participation. Following this introduction, we present a set of recommendations which we developed based on our research and analysis. In the chapter “Methodology” we describe how we determined and defined the elements of this Atlas. The chapter “Actors” presents which authorities, research institutes, interest groups and NGOs decisively shape the discourse on ADM. This is followed by an overview of existing approaches to regulation of ADM that are relevant to participation. Individual chapters on social infrastructure, health and medicine, labour, consumer protection, the Internet, and security and control deal with the use of ADM in specific sectors of society.
Part of the Atlas project consists of a free digital database in which we describe about 150 actors and regulations, software systems and technologies. We will continue to expand this database. At the end of the report you can read more in the chapter entitled “Database” which also includes a selected bibliography.
We do not claim that our Atlas is exhaustive. We mainly want to illustrate ways in which ADM shapes society in the age of digitalization. In this we follow the Mission Statement which AlgorithmWatch set out in 2016: “It is not a law of nature that ADM processes are concealed from those affected by them. This has to change.” The statement goes on to say: “We have to decide how much freedom we want to transfer to ADM.” In this sense, this current Atlas of Automation is meant to help with taking the right human decisions.
ADM – Systems of automated decision-making
Systems of automated decision-making (ADM) are always a combination of the following social and technological parts:
- A decision-making model
- Algorithms that make this model applicable in the form of software code
- Data sets that are entered into this software, be it for the purpose of training via Machine learning or for analysis by the software
- The whole of the political and economic ecosystems that ADM systems are embedded in (elements of these ecosystems include: the development of ADM systems by public authorities or commercial actors, the procurement of ADM systems, and their specific use).
Participation means actively and passively using or exercising social opportunities and rights. People who are hindered in or denied access to public goods and services, and exercising their rights solely based on:
- their sex
- their sexual orientation
- their age
- their religion and/or their world view
- their origin (geographic, social)
- their health status
- their social status (education, occupation)
are limited in their participation in society. (In the German report we use the term „Teilhabe“, which has a slightly different connotation than ”Partizipation” in German language.)
We consider ADM-systems, technologies, regulations and actors as relevant to participation if their actions or their application hinders or enhances participation.
To preserve freedom of expression and of information within the framework of a common public sphere is, in this sense, an integral part of participation. Similarly, a functioning ecosystem is relevant to participation because, by providing a common livelihood for all of us, it is an important public good.