UCI Communities and Crime Dataset — Feature Selection & Model Comparison
Mean squared error across validation and test sets. RBF Kernel Ridge achieves the best generalization performance.
Feature selection matrix showing which features are selected by OLS p-value, Stepwise, Lasso, and ElasticNet methods. Sorted by consensus count.
How coefficient values change as L1 ratio increases (sparsity effect). Ridge (L1=0) vs ElasticNet (L1=0.33, 0.66) vs Lasso (L1=1.0).
Cross-validation MSE for RBF and Polynomial Kernel Ridge Regression across different hyperparameter combinations.
Summary of the most important insights from this multi-method feature selection and model comparison analysis.
RBF Kernel Ridge achieved the lowest test MSE (0.01767), demonstrating that non-linear relationships in crime data significantly improve predictions over linear methods.
15 features were consistently selected across all 4 methods (OLS, Stepwise, Lasso, ElasticNet), indicating strong agreement on the most predictive variables for crime rates.
Increasing L1 ratio systematically drives coefficients to zero, with Lasso creating 66% sparsity. Ridge maintains non-zero coefficients across all features.
Successfully reduced 101 initial features to 37 selected features (63% reduction) while maintaining model performance, indicating strong feature engineering.
RBF Kernel Ridge showed strong generalization with minimal validation-test gap (0.01753 vs 0.01767), suggesting stable performance on unseen data.
RBF kernel performance is sensitive to gamma (width parameter), while polynomial kernels show more robustness, with degree=1 (linear) performing surprisingly well.
Interactive data visualization dashboard | Built with vanilla JavaScript | UCI Communities and Crime Dataset