To our knowledge, this paper is the first attempt to consider the existence issue for fractional $p$-Laplacian equation: $(-\Delta)_p^s u= \lambda f(u),\; u> 0 ~\text{ in}~\Omega;\; u=0\;\text{ in}~ \mathbb{ R}^N\setminus\Omega$, where $p>1$, $s\in (0,1)$, $\lambda>0$ and $\Omega$ is a bounded domain with $C^{ 1, 1}$ boundary. We first propose a notion of stable solution, then we prove that when $f$ is of class $C^1$, nondecreasing and satisfying $f(0)>0$ and $\underset{ t\to \infty}{ \lim}\frac{ f(t)}{ t^{ p-1}}=\infty$, there exists an extremal parameter $\lambda^*\in (0, \infty)$ such that a bounded minimal solution $u_\lambda \in W_0^{ s,p}(\Omega)$ exists if $\lambda\in (0, \lambda^*)$, and no bounded solution exists if $\lambda>\lambda^*$. Moreover, no $W_0^{ s,p}(\Omega)$ solution exists for $\lambda > \lambda^*$ if in addition $f(t)^{ \frac{ 1}{ p-1}}$ is convex. To handle our problems, we show a Kato-type inequality for fractional $p$-Laplacian. We show also $L^r$ estimates for the equation $(-\Delta)_p^su=g$ with $g\in W_0^{ s, p}(\Omega)^*\cap L^q(\Omega)$ for $q \geq 1$, especially for $q \le \frac{ N}{ sp}$. We believe that these general results have their own interests. Finally, using the stability of minimal solutions $u_\lambda$, under the polynomial growth or convexity assumption on $f$, we show that the extremal function $u_* =\lim_{ \lambda\to\lambda^*}u_\lambda \in W_0^{ s,p}(\Omega)$ in all dimensions, and $u^*\in L^{ \infty}(\Omega)$ in some low dimensional cases.
Categories: math.AP