Empirical Bayes via ERM and Rademacher complexities: the Poisson model

We consider a problem of estimating means of n multivariate Poisson distributed random variables. Our methodological contribution is a new empirical Bayes approach that directly optimizes a regression function from observations to predicted means by solving an empirical risk minimization (ERM) for a special loss and subject to the monotonicity constraint. Classically, regression function is computed by first estimating the prior from the data (e.g., via NPMLE) and then computing posterior mean via Bayes rule. The ERM procedure suggested here is much more computationally efficient (1000x in our experiments even in dimension 1) and scales easily to higher dimensions. Furthermore, our shape constrained ERM procedure enjoys rigorous statistical guarantees: it attains optimal minimax regret over two popular classes of priors (compactly supported and subexponential). In order to show these (``fast'') near-parametric rates, we utilize modern ideas of offset Rademacher complexity, whose adaptation to non-standard loss and class of functions is our main technical contribution.