Fairness and discrimination
The GDPR places fairness among the fundamental principles governing the processing of personal data in Article 5 and is mentioned along with transparency concerning the data subject and the legality of the processing.
A similar distinction is used in the EU expert group’s ethical guidelines for trustworthy AI and is reflected in the National AI Strategy with the key principles legal, ethical and secure.
There is still little practice that can clarify the specific content of the principle of fairness in more detail, but there is some guidance in the recitals of the GDPR. It is also worth mentioning that the principle of fair processing of personal data is meant to be a flexible legal standard which can be adapted to the specific processing situation.
The specific situation of the data subject must be taken into consideration when assessing whether the processing is compatible with the requirements in the Regulation. One must take into what reasonable expectations the data subject has for protection of their personal data, as well as any power imbalance between the data subject and the data controller.
In its guidelines of 4/2019 on embedded privacy, the European Data Protection Board highlights several aspects included in the principle of fairness, including non-discrimination, the expectations of the data subject, the broader ethical issues of the processing and respect for rights and freedoms. In order to ensure that the solution was also assessed in a broader perspective in terms of fairness, the project involved the Gender Equality and Anti-Discrimination Ombud (LDO).
In a separate workshop, the LDO presented links between privacy regulations and the Equality and Anti-Discrimination Act. The Ombud also gave an introduction to how an actor can assess whether unlawful discrimination is taking place in accordance with Sections 6-9 of the Equality and Anti-Discrimination Act.
The LDO emphasised the following concerning Secure Practice’s service:
In an early phase of the project, Secure Practice launched an option where those with the best scores in the tool could work as ambassadors in their company. The LDO pointed out the risk that the ambassadors would have a more positive career trajectory than others if employers had access to each employee’s score. The LDO also showed that if the AI tool rewarded characteristics and interests which for example are most common in individual groups, there would be a risk of indirect discrimination in the model. Indirect discrimination is generally unlawful in accordance with Section 8 of the Equality and Anti-Discrimination Act.
Secure Practice is now taking further steps to ensure that employers do not have access to individuals’ scores, which the LDO believes is a good measure to reduce the risk of the employer being able to use the information concerning employees’ knowledge of cyber security for purposes other than intended. At the same time, the LDO believes that going forward it is important for Secure Practice to specify what demographic data is to be collected, how this data will be used as well as the justification for and factuality of the relevant data use.
The LDO encourages Secure Practice to ensure that the training and course adaptation work equally as well for all groups; women and men, different age groups and people with disabilities, etc. In order to achieve this, Secure Practice should avoid playing on stereotypical ideas about various groups when the training is adapted to the different employees. Playing on stereotypes is not necessarily discrimination but can contribute to reinforcing traditional ideas about people who belong to specific groups. Such ideas can be less accurate which will reduce the value of the training adaptation. Other measures encourage Secure Practice to periodically test, and where necessary, adjust their own service to ensure that the customization features work equally as well for all users.
Under these conditions the LDO believes that the discrimination the employees are exposed to by the training being adapted to the individual's skill level, is an objective form of discrimination, see Sections 6 and 9 of the Equality and Anti-Discrimination Act.