In this thesis, we propose a preconditioning algorithm for least squares problems. The preconditioner is constructed to be a sparse approximation to the Moore-Penrose inverse of the coefficient matrix A. For this preconditioner, we provide theoretical analysis to show that under our assumption, the least squares problem preconditioned by this preconditioner is equivalent to the original problem, and the GMRES method can determine a solution to the preconditioned problem before break down happens. In the end of this thesis, we also give some numerical examples showing the performance of the method.