Labor
In the labor market, the potential for discrimination is particularly high. When systems of automated decision-making enter into the administration authorities of labor and unemployment, high vigilance is necessary. The same is true for recruitment processes, internal personnel management and staff performance monitoring.
RECRUITMENT PROCESS
Algorithm-based selection processes (“Robo-Recruiting”) [LINK] search applicant profiles for specific qualifications and keywords. When there are many applicants for a position, a preselection can thus be made. However, humans are usually involved in the recruiting process at a more advanced stage. Apart from the financial savings, the advantages of automation can consist in the selection process proceeding along clear cut criteria and in the elimination of sympathies or aversions of the personnel manager which are irrelevant to the job placement.
In a further step beyond the selection according to preset criteria, recruiting algorithms can be allowed to independently evaluate the qualifications of candidates, potentially aided by processes of Machine Learning. In 2014, the online giant Amazon began developing such software, but the tool was never employed and the project was cancelled in early 2017. The data that formed the foundation of the training of the algorithm-based software was based on Amazon’s recruitment practices over the previous ten years. During this time, mostly men were recruited into the tech sector. The software concluded that men should be preferred over women when it comes to filling job vacancies. Furthermore, the software reproduced other discriminating selection criteria [LINK].
Other procedures for quality assessment count on the evaluation of the personality based on psychometric properties such as voice [LINK]. IBM’s Watson technology “Personality Insights“ evaluates personal communication on social media or other digital formats in its analysis of personality traits.
Apart from it being questionable whether the processes applied are functional and efficient, it is a problem that the applicants are rarely informed when they are evaluated automatically. In fact, individual job applicants can only take action when their job application is rejected by refering to the Anti-discrimination Law (Antidiskriminierungsgesetz – AGG). The extent to which labelling or certification requirements could regulate the use of ADM systems in recruitment processes needs to be discussed.
PERSONNEL MANAGEMENT
Larger companies in particular use software in their personnel management, for example in payroll accounting, holiday planning or in the registration of sick days. Furthermore, external service providers offer software that helps to identify employees who are likely to look for jobs elsewhere. This is important for companies that are particularly concerned about talent retention. Other applications focus on employee performance monitoring based on various data samples that are generated in the course of everyday processes within the company. Other products offer procedures for continuous staff surveys in order to analyze team dynamics and the job satisfaction of individual employees.
Operators of digital platforms like Uber (taxi service), Foodora (food delivery service) or Helpling (placement service for cleaning personnel) also use software in order to replace middle management, customer service and accounting when acting as an agent for their self-employed members of staff. Planning of shifts, order allocation and performance control are automatically performed via a smartphone app. It would need further examination to find out which legal means already exist against the use of ADM or could be created for such freelance workers (keywords: “Gig-Economy“ and “Platform Economy”) who offer their services to digital platforms.
Since the beginning of 2018, AlgorithmWatch has been engaged in a research project on automated personnel management and Corporate Co-determination (Betriebliche Mitbestimmung) [LINK]. Apart from determining how common ADM systems are in Germany, the project also looks at the benefits and downsides these systems bring to employees. In addition, the project examines who has access to data and whether or not aspects of Corporate Co-determination are affected.
UNEMPLOYMENT
In Germany, the institutions that make up the employment agency (Arbeitsagentur – ARGE) and its so-called “Job Centers” employ a number of software systems. As some of the Job Centers are run solely by the local administrations (optional local management – Optionskommunen), it is hard to gain a full overview. In regard to the employment agency, the answer to a Minor Interpellation of the parliamentary group “Die Linke” in the autumn of 2018 clearly showed that some processes that contain ADM components are already in use or are in the planning stages. [LINK]
Among these are:
- PP-Tools: These serve as the basis for “calculation aid for labor market opportunities” (Berechnungshilfe Arbeitsmarktchancen – BAC) which is used by a wider circle of people (at least 12,500) and “calculates the labor market opportunities of the client”. So far it has been impossible to find out how this was done. The kind of problems that such a system may exhibit are demonstrated by the job market opportunities model of the Public Employment Service Austria (AMS-Arbeitsmarkt-Chancen-Modell, see box).
- DELTA-NT: This application is used by the ARGE’s occupational psychology service. It is a computer supported psychological assessment tool that is part of the career orientation process (“psychological suitability diagnostic”). This procedure is also called “Computer Assisted Testing” CAT and was developed by the German army. [LINK]
- VERBIS: The central information system for placement and consultation at the ARGE is linked to many other systems and processes. It contains, for example, features that automatically match the profiles of employment seekers stored at the job agency with job vacancies and training programs.
- 3A1: “Automated Application Processing of Unemployment Benefit” (Automatisierte Antragsbearbeitung Arbeitslosengeld). This project has been in development since the beginning of 2019 and is supposed to reach “process maturity” in a first step by the middle of 2020. According to the German federal government, the necessary processes and data flows for an “automated preparation of decision-making” were tested beforehand with a prototype. However, the decisions concerned were “circumscribed powers”. The automation of “discretionary administrative practices” was not part of the project.
Some questions remain open: Which software systems are in use to prepare for “discretionary administrative practices” directly or indirectly? How are these software processes monitored and checked for potentially discriminating effects? Every year tens of thousands of decisions are taken to court. What role does the Job Center software play in these contested decisions?
AMS
In Austria at the end of 2018, an announcement by the Public Employment Service Austria (Arbeitsmarktservice – AMS) – which is similar to the German employment agency (Arbeitsagentur – ARGE) – stirred controversy. From 2019, a software extension started automatically evaluating the chances of a job placement on the basis of a statistical model. According to media reports, grouping into one of the three categories “high, medium and low” was not intended to have consequences for the time being. However, from 2020 it might have an impact on the granting or denial of funds [LINK]. Criticism was sparked by the fact that the weighing of the different factors was solely based on the analysis of the recent labor market, and thus mirrors existing discrimination within the automated system. This means that specific groups (women, older persons, non-nationals) are generally assessed less positively than others. [LINK] Austrian researchers called the approach of this software solution commissioned by the Public Employment Service a “prime example for discrimination“ [LINK].