Ahmad Shirzadi

‎In the era of big data‎, ‎classification problems involving sparse data with high dimensionality have become a fundamental challenge in statistical analysis‎. ‎Classical methods such as standard logistic regression fail to perform adequately in such settings due to numerical instability‎, ‎high variance‎, ‎and overfitting‎. ‎This thesis investigates and compares three important penalized logistic regression methods (namely Ridge‎, ‎Lasso‎, ‎and Elastic Net) in dealing with sparse data‎, ‎and evaluates their capability to control model complexity‎, ‎select relevant variables‎, ‎and improve predictive performance‎. ‎The performance of these methods is first examined through simulation studies under various scenarios with different sample-to-feature ratios‎, ‎and then implemented on two real-world datasets‎: ‎a spam email collection with a limited number of features‎, ‎and a high-dimensional genomic dataset with an imbalanced class distribution‎. ‎The findings indicate that in simulated sparse data with numerous noisy features‎, ‎sparsity-inducing methods such as Lasso and Elastic Net perform better by effectively eliminating irrelevant variables; however‎, ‎in real-world data with more complex underlying structures‎, ‎the Ridge model provides more accurate predictions by leveraging all features and reducing variance‎. ‎Overall‎, ‎the choice of an appropriate penalization method depends strongly on the nature of the data and the analysis objective—whether predictive accuracy or interpretability through variable selection‎.