Comparisons of ADABOOST, KNN, SVM and logistic regression in classification of imbalanced dataset
Date of Original Version
Data mining classification techniques are affected by the presence of imbalances between classes of a response variable. The difficulty in handling the imbalanced data issue has led to an influx of methods, either resolving the imbalance issue at data or algorithmic level. The R programming language is one of the many tools available for data mining. This paper compares some classification algorithms in R for an imbalanced medical data set. The classifiers ADABOOST, KNN, SVM-RBF and logistic regression were applied to the original, random oversampling and undersampling data sets. Results show that ADABOOST, KNN and SVM-RBF exhibits over-fitting when applied to the original dataset. No over-fitting occurs for the random oversampling dataset where by SVM-RBF has the highest accuracy (Training: 91.5%, Testing: 90.6%), sensitivity (Training:91.0%, Testing: 91.0%), specificity (Training: 92.0%,Testing: 90.2%) and precision (Training:91.9%, Testing 90.5%) for training and testing data set. For random undersampling, no over-fitting occurs only for ADABOOST and logistic regression. Logistic regression is the most stable classifier exhibiting consistent training an testing results.
Communications in Computer and Information Science
Rahman, Hezlin Aryani Abd, Yap Bee Wah, Haibo He, and Awang Bulgiba. "Comparisons of ADABOOST, KNN, SVM and logistic regression in classification of imbalanced dataset." Communications in Computer and Information Science 545, (2015): 54-64. doi:10.1007/978-981-287-936-3_6.