Data Sciences
Nicolas Papernot is combining privacy protection with the performance of machine learning algorithms
Date:
Changed on 22/10/2025
Nicolas Papernot, a Professor at the University of Toronto and a member of the Vector Institute, is an Inria International Chair holder and works alongside the PreMeDiCal joint project team from the Inria branch in Montpellier. He plays a key role in developing machine learning approaches for precision medicine, with a focus on causal and federated learning, while ensuring both the confidentiality of medical data and the performance of the algorithms.
Verbatim
I am interested in privacy and security in the development of machine learning algorithms.
Auteur
Poste
Professor at the University of Toronto, holder of an Inria international chair
Tell us about your background. ?
« I began my engineering studies at Centrale Lyon engineering school in France, before going on to do a PhD at Pennsylvania State University where I focused on computer security. That’s when I came across the topic that still drives my work today: ensuring privacy and security in the development of machine learning algorithms. After completing my thesis, I did an internship at Google Brain, Google’s artificial intelligence research laboratory. I stayed there for a year following my thesis defence, before enrolling at the University of Toronto in 2019. »
What were your very first research topics ?
« My initial research focused on what we call ‘adversarial examples’. This is about exposing the weaknesses and vulnerabilities of an algorithm, by showing it can be misled by details that are imperceptible to the human eye. A classic example is a STOP sign altered with a small sticker. The human would still recognise the stop sign, but the AI model might misinterpret it, because the sticker confuses its analysis. This is a very crucial issue in the case of autonomous vehicles or more broadly, in the field of security. »
Then at Google Brain, the issue of personal data came up...
« Yes, we began looking into developing algorithms capable of analysing data that may contain personal information, without actually retaining this information. Our datasets include conversations and exchanges between individuals, but our algorithms must not remember specific data about these individuals, such as their names or address for example. We have managed to develop several approaches based on the concept of ‘differential privacy’. »
You now have your own research team.
« I have had my own laboratory and my own team at the University of Toronto since 2019. I particularly enjoy working with students. They bring fresh ideas and new perspectives to the table. Their creativity is a real driving force behind some of our most interesting discoveries ! »
Verbatim
Protecting privacy can limit the accuracy of machine learning algorithm predictions, especially for rare cases.
What is your research mainly focused on today ?
« Most of our work focuses on examining machine learning models. When you're a regulator or a legislator, how do you go about analysing an algorithm or AI model that has been deployed by the private sector or in a hospital for example? It is challenging to model a population based on its data while still making accurate predictions about individuals. Protecting privacy can limit the accuracy of machine learning algorithm predictions, especially when it comes to rare cases. In medical datasets for instance, it's easier to model healthy individuals than those with an illness. And privacy-preserving algorithms can exacerbate this issue. We are particular interested in understanding and addressing these specific details. »
How does the PreMeDiCal project-team in Montpellier come into this ?
« We are assessing the performance of algorithms trained on highly academic datasets, which pose few security issues. But in reality, things are very different, particularly in the medical sector. The data is far more complex and more difficult to work with, particularly when the goal is to develop high-performing algorithms that also preserve privacy. In such cases, we need to work on the machine learning models in advance. This is where our partnership with the PreMeDiCal project team comes into play. A student has already come to work with us and I have written articles with Aurélien Bellet, Inria’s Director of Research. »
Why choose to collaborate with French research teams ?
« During my last visits to France, I was struck by the exceptional quality of scientific discussions. Research teams in France are organised somewhat differently compared to ours. Our laboratories are more independent, and we have significant responsibilities to take on, related to management and administration. In France, researchers seem more able to devote all their time to scientific projects, including the smaller details. »
Machine learning algorithm
A machine learning algorithm is a procedure that allows a computer system to learn patterns and improve its performance using data. It has not been explicitly programmed to perform the tasks has been given. It identifies patterns and trends in the data to make predictions and decisions. These algorithms are used in various fields to perform complex tasks or to improve processes such as diagnosing a disease, and simulating the impact of a decision or a phenomenon.
Photo by Matthew Tierney - Illustration : By K6ka - Own work, CC BY-SA 4.0, Link.