Hamid Karamikabir

This study introduces a novel shrinkage regularization approach for semi-parametric regression trees‎, ‎combining the strengths of parametric and non-parametric methods to enhance predictive performance and adaptability‎. ‎The proposed method integrates weighted regression with regularization penalties‎, ‎leveraging prior estimations and data-driven weighting schemes to mitigate multicollinearity and overfitting‎. ‎We evaluate the model alongside classical techniques (Ridge‎, ‎LASSO‎, ‎Elastic Net) in both parametric and semi-parametric frameworks‎, ‎incorporating regression trees for non-parametric components‎. ‎Simulation studies and real-world datasets (Liver Disorders and Computer Hardware) demonstrate the superior performance of the proposed method‎, ‎particularly in handling multicollinearity and capturing non-linear relationships‎. ‎Results show significant improvements in predictive accuracy‎, ‎as measured by R^2‎, ‎MSE‎, ‎AIC‎, ‎and BIC‎, ‎with tree-augmented variants (e.g.‎, ‎NR-Trees) achieving the highest performance‎. ‎This work bridges the gap between interpretability and flexibility‎, ‎offering a robust tool for complex data structures in fields like econometrics‎, ‎epidemiology‎, ‎and machine learning‎.