Smooth Sigmoid Surrogate (SSS): An Alternative to Greedy Search in Decision Trees

Xiaogang Su, George Ekow Quaye, Yishu Wei, Joseph Kang, Lei Liu, Qiong Yang, Juanjuan Fan, Richard A. Levine

Research output: Contribution to journalArticlepeer-review

Abstract

Greedy search (GS) or exhaustive search plays a crucial role in decision trees and their various extensions. We introduce an alternative splitting method called smooth sigmoid surrogate (SSS) in which the indicator threshold function used in GS is approximated by a smooth sigmoid function. This approach allows for parametric smoothing or regularization of the erratic and discrete GS process, making it more effective in identifying the true cutoff point, particularly in the presence of weak signals, as well as less prone to the inherent end-cut preference problem. Additionally, SSS provides a convenient means of evaluating the best split by referencing a parametric nonlinear model. Moreover, in many variants of recursive partitioning, SSS can be reformulated as a one-dimensional smooth optimization problem, rendering it computationally more efficient than GS. Extensive simulation studies and real data examples are provided to evaluate and demonstrate its effectiveness.

Original languageEnglish
Article number3190
JournalMathematics
Volume12
Issue number20
DOIs
StatePublished - Oct 2024

Keywords

  • CART
  • decision trees
  • end-cut preference
  • greedy search
  • recursive partitioning
  • sigmoid function

Fingerprint

Dive into the research topics of 'Smooth Sigmoid Surrogate (SSS): An Alternative to Greedy Search in Decision Trees'. Together they form a unique fingerprint.

Cite this