|
Keywords
|
Semi-parametric regression, Regularization, Regression trees, Predictive modeling, Multicollinearity.
|
|
Abstract
|
This study introduces a novel shrinkage regularization approach for semi-parametric regression trees, combining the strengths of parametric and non-parametric methods to enhance predictive performance and adaptability. The proposed method integrates weighted regression with regularization penalties, leveraging prior estimations and data-driven weighting schemes to mitigate multicollinearity and overfitting. We evaluate the model alongside classical techniques (Ridge, LASSO, Elastic Net) in both parametric and semi-parametric frameworks, incorporating regression trees for non-parametric components. Simulation studies and real-world datasets (Liver Disorders and Computer Hardware) demonstrate the superior performance of the proposed method, particularly in handling multicollinearity and capturing non-linear relationships. Results show significant improvements in predictive accuracy, as measured by R^2, MSE, AIC, and BIC, with tree-augmented variants (e.g., NR-Trees) achieving the highest performance. This work bridges the gap between interpretability and flexibility, offering a robust tool for complex data structures in fields like econometrics, epidemiology, and machine learning.
|