Atlas der Automatisierung

Automatisierte Entscheidungen
und Teilhabe in Deutschland

Der Atlas der Automatisierung wird aktuell nicht mehr aktualisiert.
Die Daten sind daher nicht mehr auf dem neuesten Stand.


Health & Medicine

ADM is used as an instrument in diagnostics, for therapy decisions, and for the allocation of resources in the health sector.

Not only are clinics, doctors’ surgeries and health insurance companies using ADM in the medical sphere, so are individual citizens. Private users tend to use ADM in the form of apps that go beyond simply looking at the medical angle and enter the realm of well-being and self-optimization. ADM in clinics and surgeries is often embedded in complex software solutions and adjusted to the prescriptions of the Ordinance on Medical Devices and to the interaction with diagnostic tools, e.g. in radiology.


Apps can support medical professionals in diagnosing illnesses. Algorithms that evaluate image data such as computed tomography (CT), magnetic resonance imaging (MRI) or other medical data that help detect cancer or a need for prenatal medicine, for example, are particularly advanced. Scientific studies show that ADM systems can detect and interpret abnormalities in images more reliably than the trained human eye. Further uses of ADM include helping interpret genetic tests, aiding in robotic surgery and assisting in analyzing databases of scientific literature. [LINK]

Other ADM-supported applications address patients as end users. These aim to assist patients in the treatment of their disease. Most of these applications are smartphone apps which can be of great use, especially with chronic or long term illnesses. Some of these apps include simple reminders to take medica tion, while more complex systems provide close monitoring, such as blood sugar levels, to simplify the management of diabetic patients.

While on the one hand ADM-supported apps and other therapy aids significantly increase therapy success and allow chronically ill patients in particular to become more independent from constant therapy and control by doctors, they also bring with them a range of problems and risks. From a patient’s perspective it can seem problematic that the ADM-systems employed for diagnosis and therapy recommendations are not necessarily geared towards optimizing the patient’s benefit, but can also aim at increasing profits for those who use or market the ADM-system. This could happen through recommendations for tests, drugs and medical devices which in turn could be connected with additional costs or with excessive side effects for the patients. [LINK]

Regarding aspects of participation, the use of ADM in diagnosis is problematic for a number of reasons. Groups of patients who represent a minority in terms of some biological traits might find themselves systematically disadvantaged because the database used is insufficient for the respective group or leads to misjudgements. For example, in one study assessing the risk of heart disease, algorithm-based diagnosis led to both over and underestimation for patients who did not belong to the white majority of the population.


ADM-Systems are not only of use when optimizing diagnosis and therapy, they are also helpful in enabling a more efficient distribution of resources in the health sector.

One example: During the decision on the allocation of donor organs in the context of an organ transplant, patients are already prioritized according to specific parameters, including urgency and the chances of success. Employing ADM-based allocation systems could lead to self-fulfilling prophecies and thus to systematic disadvantages for some groups of patients. This could, in turn, also impact other medical intervention decisions. For example, patients with brain damage or infants who are born premature could virtually be excluded from some treatments due to the supposedly limited prospects of success. It is possible that an ADM system could follow such a practice. In this manner, decision-making pathways are reinforced even if they are not legitimized by reliable data on treatment results.

Private health insurance companies are another area where ADM is used. Here, it can help to calculate the individual risk for specific diseases which in turn can be used to adapt insurance policies. To a greater extent than ever before, policy holders could be divided into different risk groups that then have to pay different rates for their insurance. In regard to social fairness and participation, this could exacerbate existing inequalities. Health and lifestyle apps on smartphones could play an important role in this context (think of “quantified self”). Some German health insurance policies already make the use of such apps a prerequisite for receiving favorable insurance tariffs. This could discriminate against insurance holders who cannot use such an app or who would not benefit from it in regard to their tariff. In addition, the way in which the data is gathered and then used to adapt the tariff could also lead to discrimination.


Regulation of Digital Medical Products

ADM-based software systems that are employed in a clinical or outpatient setting have to be registered as Medical Devices. The approval and quality control of Medical Devices is regulated by the Federal Institute for Drugs and Medical Devices according to the Europe-wide Ordinance on Medical Devices.

Currently, health apps for smartphones that address individual consumers are rarely registered as Medical Devices and thus do not undergo any quality control. However, with the new version of the EU Ordinance on Medical Devices coming into force in 2020 the registration requirements for apps will be extended. Apps that take a distinctive diagnostic or therapeutic approach will be placed into a higher risk category. Whether the changes in the revised version of the ordinance are sufficient to remedy presumed deficiencies is currently being debated.

Next chapter: Topic: The Internet

Previous chapter: Labor