The optimal learning rate for adaptive gradient methods applied to { \lambda}-strongly convex functions relies on the parameters { \lambda} and learning rate { \eta}. In this paper, we adapt a universal algorithm along the lines of Metagrad, to get rid of this dependence on { \lambda} and { \eta}. The main idea is to concurrently run multiple experts and combine their predictions to a master algorithm. This master enjoys O(d log T) regret bounds.
Categories: cs.LG math.OC