You are here
Fisher consistency of supervised learning methods.
Many of the classification algorithms developed in the machine learning literature, including the support vector machine and boosting, can be viewed as methods that minimize a convex surrogate of the 0-1 loss. The convexity makes these algorithms computationally efficient. The use of a surrogate, however, has statistical consequences that must be balanced against the computational virtues of convexity. One property that establishes the optimality (under assumptions) for a classifier obtained by a minimization procedure such as support vector machine is that of Fisher consistency. In this talk I will describe first the notion of Fisher consistency on this setting and then describe the main results that have been emerged in recent years. In particular, I will review results for binary, multiclass classification and ranking algorithms. I will finish by stating novel results obtained for ordinal regression methods.