A novel logistic regression model based on density estimation

We propose a density-based logistic regression (DLR) model for classification to address the challenge of the nonlinear classification problem in this domain. Based on a Nadarays-Watson density estimator, the training data is mapped into a particular feature space. Then, an optimization model is set up to optimize the feature weights and the width in the Nadaraya-Watson density estimation algorithm. We show that it is superior to not only standard logistic regression but also kernel logistic regression (KLR) with radial basis function (RBF) kernels. The results show that DLR compares favorably against other nonlinear methods including KLR and support vector machine (SVM). The introduced approach achieves not only better classification accuracy but also better time efficiency. Another major advantage of our method is that it can be naturally extended to cope with hybrid data with both categorical features and numerical features. Moveover, our approach shares with logistic regression the same advantage of interpretability of the model, which is not obtained by kernel based methods such as KLR and SVM.