A STABILITY-AWARE HYBRID FEATURE SELECTION ALGORITHM BASED ON FILTERED SPARSITY AND WRAPPER REFINEMENT

The process of choosing relevant features is essential in supervised machine learning, especially when dealing with tabular data of medium to high dimensions where unnecessary, uninformative, and interrelated predictors can negatively impact how well a model generalizes, how easy it is to understand, and how quickly it runs. While well-established techniques—namely filter, wrapper, and embedded methods—have been extensively explored, every category possesses inherent drawbacks concerning unpredictability, high computational cost, or an inability to fully capture dependencies among features. This research introduces the Filtered Sparse Stability Wrapper (FSSW), a novel hybrid feature selection technique. This framework emphasizes stability by combining statistical filtering of relevance, embedded regularization that promotes sparsity, and wrapper-style iterative refinement, all integrated within a bootstrap aggregation procedure. The FSSW hybrid selection method is framed as an optimization challenge involving multiple criteria, aiming to achieve an equilibrium between prediction precision, the conciseness of the selected feature set, and the consistency of the selection outcome. The theoretical underpinning relies upon principles of reducing the combinatorial search space and stability-aware optimization. Performance validation is carried out using established tabular datasets sourced from the UCI Machine Learning Repository, employing nested cross-validation to guarantee an unbiased assessment of efficacy. Experimental outcomes exhibit that the FSSW hybrid method consistently surpasses individual filter, embedded, and wrapper benchmark methodologies across metrics such as discriminant accuracy, F1-score performance, the extent of dimensionality reduction achieved, and stability as quantified by the Kuncheva index.

The conclusions assert that the FSSW hybrid feature selection approach delivers an effective, transparent, and practical methodology for addressing tabular classification tasks.