Abstract:
A binary classification game is being played between a learner and an adversary. At each round, the adversary selects a natural number x and the learner has to predict its label h(x). Is there a bound on the number of mistakes made by the learner, assuming that h comes from some class H? Littlestone gave a learning algorithm which achieves an optimal mistake bound d, provided H has a certain combinatorial property. However, Littlestone’s algorithm is believed to be computationally demanding. Can we have a feasible strategy for the learner and if so, how many mistakes would it make? I will introduce such a strategy, which is fast (for finite classes) and achieves an exponential (in d) mistake bound. The algorithm was devised together with Sasha Kozachinskiy.