Title: ANALYSIS OF RELATIONSHIP BETWEEN RÉNYI ENTROPY AND MARGINAL BAYES ERROR AND ITS APPLICATION TO WEIGHTED NAÏVE BAYES CLASSIFIERS
Abstract: A weighted naïve Bayes (WNB) classifier using Rényi entropy is discussed with its tree augmented extension. A WNB classifier is one solution for gaining efficiency against large-scale classification problems with an enormous amount of data and a lot of features. Such a WNB classifier has been studied so far, aiming at improving the prediction performance or at reducing the number of features. Among those studies, weighting with Shannon entropy has succeeded in reducing the number of features while keeping the classification performance. However, it has not been fully revealed how different weighting methods affect the performance of classification and feature selection. In this paper, it is analyzed by changing the weights using a parametric α-Rényi entropy. As the first clue, the relationship between Rényi entropy and the marginal Bayes error is analyzed in detail. It was revealed that the WNB classifiers becomes the regular (without weight) naïve Bayes classifier in one end (α = 0.0) and naïve Bayes classifier weighted by the marginal Bayes error in the other end (α = ∞). In addition, an extension of WNB classifiers to incorporate tree-structured correlation between features is discussed.
Publication Year: 2014
Publication Date: 2014-10-14
Language: en
Type: article
Indexed In: ['crossref']
Access and Citation
Cited By Count: 2
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot