Séminaire de Thomas Schatz

Thomas Schatz
AMU
https://www.ilcb.fr/thomas-schatz/

Date(s) : 10/03/2025 iCal
14h00 - 15h00

Title

Unbiased Estimation from Unbalanced Hierarchical Samples, with Applications to the Evaluation of Representation Learning Algorithms

Abstract

Classical U-statistics’ theory generalize statistical estimation theory from averages of functions of one i.i.d. sample (for example the Euclidean norm is a function of one sample, if samples are real vectors) to averages of function of several i.i.d. samples (for example, the Euclidean distance is a function of two samples if samples are real vectors). It notably provides Uniformly Minimum Variance Unbiased Estimators (UMVUEs) for many statistical models of practical relevance. But what happens if we are not working with simple i.i.d. samples?

In this talk, motivated by applications to representational geometry analysis in machine learning and cognitive (neuro)science [1-6], we will consider the case of possibly heavily unbalanced hierarchical samples. We will start with the non-trivial issue of defining hierarchical samples with enough generality to cover the application cases of interest. We will then see that i.i.d. samples can be represented as graphs with the special property that all graph isomorphisms are also graph automorphisms and that the simplicity of the classical theory depends on this property. In the case of unbalanced hierarchical samples, this property no longer holds and we will discuss what are the proper ways to generalize and what can be salvaged of the classical theory in this case. We will conclude with some computational considerations.

References

[1] Aarre Laakso and Garrison Cottrell. “Content and cluster analysis: assessing representational similarity in neural systems”. Philosophical psychology 13.1 (2000), pp. 47–76.

[2] James V Haxby, M Ida Gobbini, Maura L Furey, Alumit Ishai, Jennifer L Schouten, and Pietro Pietrini. “Distributed and overlapping representations of faces and objects in ventral temporal cortex”. Science 293.5539 (2001), pp. 2425–2430.

[3] Nikolaus Kriegeskorte, Marieke Mur, and Peter A Bandettini. “Representational similarity analysis-connecting the branches of systems neuroscience”. Frontiers in systems neuroscience 2 (2008), p. 249.

[4] Thomas Schatz, Vijayaditya Peddinti, Francis Bach, Aren Jansen, Hynek Hermansky and Emmanuel Dupoux. “Evaluating speech features with the minimal-pair ABX task: Analysis of the classical MFC/PLP pipeline”. Proceeding of INTERSPEECH 2013.

[5] Corinna Cortes, Mehryar Mohri, and Afshin Rostamizadeh. “Algorithms for learning kernels based on centered alignment”. The Journal of Machine Learning Research 13 (2012), pp. 795–828.

[6] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. “Similarity of Neural Network Representations Revisited”. Proceedings of ICML 2019.

Emplacement
I2M Saint-Charles - Salle séminaire

Catégories