In this paper, we consider a data matrix $X\in\mathbb{ C}^{ N\times M}$ where all the columns are i.i.d. samples being $N$ dimensional complex Gaussian of mean zero and covariance $\Sigma\in\mathbb{ C}^{ N\times N}$. Here the population matrix $\Sigma$ is of finite rank perturbation of the identity matrix. This is the "spiked population model" first proposed by Johnstone in \cite{ 21}. As $M, N\to\infty$ but $M/N = \gamma\in(1, \infty)$, we first establish in this paper the asymptotic distribution of the smallest eigenvalue of the sample covariance matrix $S:= XX^*/M$. It also exhibits a phase transition phenomenon proposed in \cite{ 1} --- the local fluctuation will be the generalized Tracy-Widom or the generalized Gaussian to be defined in the paper. Moreover we prove that the largest and the smallest eigenvalue are asymptotically independent as $M, N\to\infty$.
Categories: math.PR