LADE Seminar

Abstract

Feature selection is a common practice in many applications and tends to come with attached uncertainties such as: What is the best dimensionality of a reduced feature space in order to retain maximum information? How can one correct for different units of measure? What is the optimal relative scaling of importance between features? To solve these questions, we extend Information Imbalance, an effective statistic to rank information content between feature spaces, to its differentiable form. This Differentiable Information Imbalance is used as a loss function to optimize relative feature weights, simultaneously per- forming unit alignment and relative importance scaling. The same method can generate sparse solutions. The optimal size of a reduced space is conveniently found by considering the change in the Differentiable Information Imbalance as a function of the number of non-zero features.