Research Info

Learning meaningful representations of high-dimensional data is a foundational challenge in modern statistical learning. While classical linear methods like Principal Component Analysis (PCA) provide computationally efficient and interpretable solutions, deep nonlinear architectures such as Autoencoders (AEs) and Variational Autoencoders (VAEs) offer greater modeling flexibility. This paper presents a unified statistical learning perspective on these three paradigms, interpreting them as successive generalizations within a common empirical risk minimization framework with varying structural constraints and latent regularizations. To evaluate the practical utility of their learned representations, we investigate a two-stage pipeline where low-dimensional latent features are first learned in an unsupervised manner and subsequently fixed for downstream supervised learning tasks. Through extensive empirical evaluation on the MNIST dataset across various latent dimensions and distinct classification heads, we analyze the trade-offs between reconstruction, latent regularization, and predictive performance. Our results demonstrate that while nonlinear models (AE and VAE) significantly outperform PCA in low-dimensional regimes, the deterministic AE consistently yields superior downstream classification accuracy compared to VAE, highlighting the impact of probabilistic latent regularization on feature discriminability