Supervision in the Age of AI
Delft University of Technology
Delft University of Technology
November 4, 2026
“[…] unified front of our expertise in this area formed to help the financial industry solve the growingly complex challenges.”
What about topics specific to supervision?
Augmenting human experts with AI.
Born out of the need for explanations …
Counterfactual Explanation (CE) explain how inputs into a model need to change for it to produce different outputs (Wachter et al. 2017).
Provided the changes are realistic and actionable, they can be used for Algorithmic Recourse (AR) to help individuals who face adverse outcomes.
From ‘loan denied’ to ‘loan supplied’: CounterfactualExplanations.jl 📦.
In the EU, individuals have the right “[…] to obtain an explanation of the decision reached after such assessment and to challenge the decision.” (Recital 71 of the General Data Protection Regulation (GDPR))
In our example, who do you think is most likely to ask for an explanation?
The starting point for most counterfactual generators is as follows,
\[ \begin{aligned} \mathbf{Z}^\prime =& \arg \min_{\mathbf{Z}^\prime \in \mathcal{Z}^L} \{ {\text{yloss}(M_{\theta}(f(\mathbf{Z}^\prime)),\mathbf{y}^+)} \\ &+ \lambda {\text{cost}(f(\mathbf{Z}^\prime)) } \} \end{aligned} \tag{1}\]
where \(\mathbf{Z}^\prime\) is a counterfactual, \(M_{\theta}\) is the black-box model and \(\mathbf{y}^+\) is the desired output.
Equation 1 looks a lot like an adversarial attack (Goodfellow et al. 2015), doesn’t it?
In both settings, we take gradients with respect to features \(\nabla_{\mathbf{Z}^\prime}\text{yloss}(M_{\theta}(f(\mathbf{Z}^\prime)),\mathbf{y}^+)\) in order to trigger changes in the model’s output.
There’s no consensus on the exact definition of plausibility but we think about it as follows:
Definition 1 (Plausible Counterfactuals) Let \(\mathcal{X}|\mathbf{y}^+= p(\mathbf{x}|\mathbf{y}^+)\) denote the true conditional distribution of samples in the target class \(\mathbf{y}^+\). Then for \(\mathbf{x}^{\prime}\) to be considered a plausible counterfactual, we need: \(\mathbf{x}^{\prime} \sim \mathcal{X}|\mathbf{y}^+\).
But: Higher plausibility seems to require larger changes and hence increase costs to individuals.
Moving just across the decision boundary may minimize costs to individuals but it may also generate external costs for other stakeholders (Altmeyer, Angela, et al. 2023).
Is plausibility really all we need?
All of these counterfactuals are valid explanations for the model’s prediction. Which one would you pick?
These images are sampled from the posterior distribution learned by the model. Looks different, no?
We propose a way to generate counterfactuals that are as plausible as the underlying model permits (under review).
Definition 2 (Faithful Counterfactuals) Let \(\mathcal{X}_{\theta}|\mathbf{y}^+ = p_{\theta}(\mathbf{x}|\mathbf{y}^+)\) denote the conditional distribution of \(\mathbf{x}\) in the target class \(\mathbf{y}^+\), where \(\theta\) denotes the parameters of model \(M_{\theta}\). Then for \(\mathbf{x}^{\prime}\) to be considered a faithful counterfactual, we need: \(\mathbf{x}^{\prime} \sim \mathcal{X}_{\theta}|\mathbf{y}^+\).
Now that we have a tool to faithfully explain models we may ask: how do models learn plausible explanations? Initial evidence:
JuliaResearch informs development, development informs research.
Trustworthy Artificial Intelligence in Julia.
Taija is a collection of open-source packages for Trustworthy AI in Julia. Our goal is to help researchers and practitioners assess the trustworthiness of predictive models.
Our work has been presented online for JuliaCon 2022, at MIT in Boston for JuliaCon 2023 and hopefully beyond.
All the work presented today is powered by CounterfactualExplanations.jl 📦.
There is also a corresponding paper, Explaining Black-Box Models through Counterfactuals, which has been published in JuliaCon Proceedings.
If you decide to use this package in your work, please consider citing the paper:
Conformal Prediction is a model-agnostic, distribution-free approach to Predictive Uncertainty Quantification: ConformalPrediction.jl 📦.
Effortless Bayesian Deep Learning through Laplace Approximation Daxberger et al. (2021): LaplaceRedux.jl 📦.
Joint Energy Models (JEMs) are hybrid models trained to learn the conditional output and input distribution (Grathwohl et al. 2020): JointEnergyModels.jl 📦.
Includes joint work with Cynthia C. S. Liem, Arie van Deursen, Mojtaba Farmanbar, Aleksander Buszydlik, Karol Dobiczek, Giovan Angela and many other students at TU Delft.
Slides power by Quarto.