For sufficiently smooth targets of product form it is known that the variance of a single coordinate of the proposal in RWM (Random walk Metropolis) and MALA (Metropolis adjusted Langevin algorithm) should optimally scale as $n^{ -1}$ and as $n^{ -\frac{ 1}{ 3}}$ with dimension $n$, and that the acceptance rates should be tuned to $0.234$ and $0.574$. We establish counterexamples to demonstrate that smoothness assumptions of the order of $\mathcal{ C}^1(\mathbb{ R})$ for RWM and $\mathcal{ C}^3(\mathbb{ R})$ for MALA are indeed required if these scaling rates are to hold. The counterexamples identify classes of marginal targets for which these guidelines are violated, obtained by perturbing a standard Normal density (at the level of the potential for RWM and the second derivative of the potential for MALA) using roughness generated by a path of fractional Brownian motion with Hurst exponent $H$. For such targets there is strong evidence that RWM and MALA proposal variances should optimally be scaled as $n^{ -\frac{ 1}{ H}}$ and as $n^{ -\frac{ 1}{ 2+H}}$ and will then obey anomalous acceptance rate guidelines. Useful heuristics resulting from this theory are discussed. The paper develops a framework capable of tackling optimal scaling results for quite general Metropolis-Hastings algorithms (possibly depending on a random environment).
Categories: math.PR math.ST stat.TH