In this paper we study the support recovery problem for single index models $Y=f(\boldsymbol{ X}^{ \intercal} \boldsymbol{ \beta},\varepsilon)$, where $f$ is an unknown link function, $\boldsymbol{ X}\sim N_p(0,\mathbb{ I}_{ p})$ and $\boldsymbol{ \beta}$ is an $s$-sparse unit vector such that $\boldsymbol{ \beta}_{ i}\in \{ \pm\frac{ 1}{ \sqrt{ s}},0\}$. In particular, we look into the performance of two computationally inexpensive algorithms: (a) the diagonal thresholding sliced inverse regression (DT-SIR) introduced by Lin et al. (2015); and (b) a semi-definite programming (SDP) approach inspired by Amini & Wainwright (2008). When $s=O(p^{ 1-\delta})$ for some $\delta>0$, we demonstrate that both procedures can succeed in recovering the support of $\boldsymbol{ \beta}$ as long as the rescaled sample size $\kappa=\frac{ n}{ s\log(p-s)}$ is larger than a certain critical threshold. On the other hand, when $\kappa$ is smaller than a critical value, any algorithm fails to recover the support with probability at least $\frac{ 1}{ 2}$ asymptotically. In other words, we demonstrate that both DT-SIR and the SDP approach are optimal (up to a scalar) for recovering the support of $\boldsymbol{ \beta}$ in terms of sample size. We provide extensive simulations, as well as a real dataset application to help verify our theoretical observations.
Categories: math.ST stat.ML stat.TH