UTOPIA - Understanding Threats and Optimizing mitigation, and enabling Privacy-by-design in AI
Description
Artificial Intelligence (AI) is increasingly used to support medical decisions, improve services, and identify patterns in large datasets. At the same time, it introduces new risks that can undermine public trust. Some AI systems can be manipulated to make wrong or unfair decisions, while others may unintentionally reveal sensitive information, such as details from the data used for training. These risks can arise at different stages of the AI lifecycle, including data collection, model training, and real-world use. European regulations such as the AI Act and GDPR highlight the need to build security and privacy into AI systems from the outset.
UTOPIA aims to strengthen the knowledge, tools, and methods needed to develop and use secure, privacy-preserving AI systems. The project focuses on understanding when and why AI becomes vulnerable, and how such risks can be evaluated and reduced in practice.
Goals
UTOPIA studies realistic risk scenarios and tests how AI systems behave under different conditions. The project develops shared approaches for analysis of security and privacy threats, together with standardized methods to assess how robust AI systems are. This enables clear and transparent comparison across different models, data types, and training approaches.
UTOPIA also develops practical ways to reduce risks without harming performance. These include removing or replacing identifying information in data, generating synthetic data, limiting how much sensitive information models can learn, and training models without sharing raw data. The project further studies how such approaches can be combined while maintaining accuracy, fairness, and manageable computing needs.
UTOPIA validates its work in healthcare, where privacy and trust are critical, and aims for results that can also be applied in other sectors. By sharing tools, results, and practical guidance, UTOPIA supports organizations in building AI systems that are secure, protect privacy, and deserve public trust.