The Norwegian Police University College (PHS) wishes to utilise chat logs from criminal cases, in which the judiciary has concluded that grooming has taken place, in order to train an AI tool to identify grooming conversations. The aim is for the tool to be able to identify online social arenas that require extra vigilance and – not least – that it is has the ability to flag specific ongoing conversations, allowing the police to stop grooming incidents and prevent abuse.

This is currently a research project at the concept stage. In this sandbox project, we have therefore primarily focused on whether, for research purposes, it is possible to develop PrevBOT within the framework of current legislation and ethical guidelines.

Summary of results

• Legal: The sandbox project first addressed some general but fundamental issues relating to the police’s use of personal data for the development of artificial intelligence. However, the focus of the assessment has been on whether personal data may be processed in connection with the PrevBOT project. These assessments may also be relevant to other projects and areas of the policing.
• Ethical: PrevBOT is an example of a project that has such honourable intentions that there is a danger of the ends justifying the means as the project progresses. The PHS wants PrevBOT to be not only a tool that is legal, but that also meets the requirements for responsible AI. The sandbox project has made a preliminary analysis and considers it ethical to commence research based on criminal case data. We also point out key ethical dilemmas and important values when it comes to development.
• Technical: The PHS would like to build PrevBOT on a Tsetlin machine (TM). TM is a relatively new method of machine learning, and its strength lies in categorisation and explainability. TM is expected to offer a higher level of explainability than neural networks, meaning it is easier to receive an answer as to why the tool reaches its conclusions. A high level of explainability will be important for transparency and trust in a tool that could potentially be perceived as invasive in personal conversations. The pivotal question in the research project is whether the technological features the PHS would like to integrate into PrevBOT – each of which has been proven possible in neural networks – can actually be transferred to and combined in a Tsetlin machine. We have not been able to see how a Tsetlin machine functions in practice in the sandbox project, but we have made a theoretical assessment and are generally hopeful that in a few years, it will be able to contribute to more sustainable AI tools that incorporate good explainability.

The way forward

The sandbox project has assessed and outlined how the PHS can legally conduct research into such an AI tool. However, a green light for PrevBOT research may be of little value if the tool being researched and developed will not be lawful to use in practice. Once in use, such a tool will inevitably need to process (sensitive) personal data. Depending on how it is implemented, its use could be perceived as rather intrusive to the privacy of victims and perpetrators, as well as to random individuals whose conversations are analysed by PrevBOT while they are online.

It would probably be wise to establish a plan early on for assessing the legality of using such a tool in practice, and that could definitely be the topic of a new sandbox project.

The PrevBOT project is still at an early stage, and the way forward depends on many decisions yet to be made. From a data protection perspective, it will be particularly interesting if the ambition is maintained that it will be a tool of prevention used to intercept attempts at grooming. The PrevBOT project is now clear that this is the goal. However, during the transition from idea to ready-to-use AI tool, there are forces that may seek to influence the project, giving the tool the capability to collect evidence against and pursue perpetrators. The Norwegian Data Protection Authority recommends that the project identifies at an early stage the uses of PrevBOT it considers unethical and undesirable, and strive during the development phase to prevent such uses from being pursued.

The desire for freedom and the desire for security are often presented as conflicting goals. The PrevBOT project is an excellent example of freedom, security and privacy being interdependent – and that it is all about finding the right balance. Minors have a right to autonomy and a private life , but without a certain level of internet security, they would not be able to exercise their autonomy and freedoms. As the tool is gradually designed in more detail, an important part of the project will be to find this equilibrium.

The sandbox project has identified several ways in which PrevBOT can help make the internet and everyday life safer for vulnerable groups. PrevBOT may end up being not just one tool, but the basis for a number of different measures, which together provide effective protection against online grooming.

See the final chapter for more information about the way forward.