Using Cross-Validation, Probing, And Lasso In Gradient Boosting Variable Selection

Abstract

Combining the predictions of various models often results in a model with increased predictive performance in numerous problem domains. Recently, machine learning methods such as boosting method, have been widely used in many scientific fields.  Boosting is an example of a method that has shown a lot of potentials. on the practical side, Experimental studies have demonstrated that combining models using boosting methods creates more accurate regression models. it was presented  methods for variable selection dependent on model-based gradient boosting, Model-based boosting is a method that fits a statistical model and selection variables. Certain machine learning algorithms handle high-dimensional data processing to improve data visualization. by comparing the three methods (lasso, probing, and cross-validation) dependent on the value of MSE, the probing has the lowest MSE so it prefers. The simulation and the real-data example both indicate that the proposed method (probing) outperforms the other current methods.