UTOPIA aims to make artificial intelligence safer and more trustworthy
Artificial intelligence (AI) is rapidly becoming part of the health service. It can support medical decision-making, improve public services and identify patterns in large volumes of data. But if we are to realise its potential, one condition is essential: we must be able to trust the technology.
This is at the heart of the UTOPIA project at the Norwegian Centre for E-health Research, which held its kick-off meeting in Tromsø on 27 May. For the first time, all the project partners came together to begin work towards a shared goal: developing knowledge, methods and tools that make AI systems safer, more robust and better able to protect individuals' privacy.
UTOPIA stands for Understanding Threats and Optimizing mitigation and enabling Privacy-by-design in AI. The project will investigate when and why AI systems become vulnerable, how risks arise throughout the lifecycle of an AI model, and which measures can genuinely reduce risk without compromising usefulness.
From enthusiasm to responsibility
AI can contribute to better services and more precise decision-support systems. At the same time, the technology can introduce new types of risk. Models can be manipulated into producing inaccurate or unfair results. They may also unintentionally disclose sensitive information, such as details from the data on which they have been trained.
In the health sector, this is particularly serious. Patient records, clinical texts and health data are among the most sensitive types of data there are. Health is therefore an important arena for UTOPIA. The project will test and validate its work in realistic health settings, where privacy, security and trust are all critical.
“This is not just about developing better AI. It is about developing AI that deserves to be trusted,” says project manager Kassaye Yitbarek Yigzaw at the Norwegian Centre for E-health Research.
Threats, synthetic data and Norwegian language models
The kick-off day was combined with the seminar “Privacy and Security in AI: From Threats to Solutions”, where participants immediately delved into some of the most pressing issues related to safe AI.
The programme opened with an introduction to the UTOPIA project by project manager Kassaye Yitbarek Yigzaw.
Dr Thomas Vakili, teacher at Stockholm University, then presented the privacy threat landscape in natural language processing, while Professor Cristina Soguero-Ruiz from Rey Juan Carlos University in Madrid discussed techniques and evaluation metrics for generating synthetic tabular and time-series data. Later, Professor Hercules Dalianis and Department Director Phuong Dinh Ngo from the Norwegian Centre for E-health Research presented work on privacy-preserving Norwegian clinical language models, from de-identification to differentially private synthetic text generation. The day concluded with Mohsen Toorani, Associate Professor in Cybersecurity at the University of South-Eastern Norway, who spoke about threat modelling for trustworthy AI.
Together, the programme demonstrated the breadth of UTOPIA, from concrete technical attacks and vulnerabilities to practical solutions such as synthetic data, de-identification, differential privacy and model training without sharing raw data.
Creating a common language for risk
A central ambition of UTOPIA is to develop a common way of describing security and privacy risks. Today, different research communities, sectors and technology providers may use different concepts and measurement methods. This makes it difficult to compare results and assess how robust an AI system actually is.
UTOPIA will therefore develop standardised tests to measure how well AI systems withstand different types of risk. The project will examine the entire AI lifecycle: from data collection and model training to real-world use.
The work is closely linked to European requirements and expectations. Both the EU AI Act and the General Data Protection Regulation, GDPR, emphasise that security and privacy must be built in from the outset, not added afterwards.
Protecting privacy without losing precision
One of the major challenges is balancing security, privacy and performance. If data is protected too strongly, the AI model may become less useful. If protection is too weak, the risk of privacy breaches, discrimination or incorrect decisions may become too high.
UTOPIA will therefore investigate how different measures can be used and combined. This may involve removing or replacing identifying information, creating synthetic data that can be used more safely, limiting how much sensitive information a model can learn, or training models without sharing raw data.
The project will also assess how such measures affect accuracy, fairness and computing power requirements. The aim is not merely to make AI safer in theory, but to identify solutions that can be used in practice.
Results for multiple sectors
Although UTOPIA will be validated in the health sector, its ambition is broader. Many of the challenges linked to AI, privacy and security also apply to other parts of the public sector and society more widely.
By sharing tools, results and practical guidance, the project will help organisations build AI systems that are more robust, better protect privacy and enjoy greater legitimacy among the public.
The kick-off meeting in Tromsø therefore marked more than the start of a research project. It marked the beginning of a collaboration on one of the most important questions in the digitalisation of the health service: How can we use artificial intelligence in a way that is innovative, secure and trustworthy?