Strong consistency of lasso estimators

In this paper, we study the strong consistency and rates of convergence of the Lasso estimator. It is shown that when the error variables have a finite mean, the Lasso estimator is strongly consistent, provided the penalty parameter (say, λ_n) is of smaller order than the sample size (say n). We also show that this condition on λ_n cannot be relaxed. More specifically, we show that consistency of the Lasso estimators fail in the cases where λ_n/n → a for some a ∈ {0; ∞}. For error variables with a finite αth moment, 1 <α <2, we also obtain convergence rates of the Lasso estimator to the true parameter. It is noted that the convergence rates of the Lasso estimators of the non-zero components of the regression parameter vector can be worse than the corresponding least squares estimators. However, when the design matrix satisfies some orthogonality conditions, the Lasso estimators of the zero components are surprisingly accurate; The Lasso recovers the zero components exactly, for large n, almost surely.