Title: An experimental comparison of uncertain inference systems (artificial intelligence, probability, entropy)
Abstract: Uncertainty is a pervasive feature of the domains in which expert systems are supposed to function. There are several mechanisms for handling uncertainty, of which the oldest and most widely used is probability theory. It is the only one which is derived from a formal description of rational behavior. For use in patten-directed inference systems, or rule-based inference engines, artificial intelligence researchers have favored others, largely for reasons of simplicity and speed. We have developed techniques which measure how these alternatives approximate the results of probability theory, assess how well they perform by those measures, and find out what underlying features of a problem affect performance.
Because the amount of data required to fully specify a probability distribution is enormous, some technique must be used to estimate a distribution when only partial information is given. We give intuitive and axiomatic arguments, algebraic analysis, and numerical examples, that fitting maximum entropy priors and using minimum cross entropy updating are the most appropriate ways to do so.
For several uncertain inference systems, detailed analysis of operations have been performed to elucidate both which basic problem-features bias the answers and the directions of the biases. We present and discuss both the motivation and design of our analysis techniques, and the specific structures which were found to have strong effects on performance. The techniques have also been tried on several variations of a fragment from a real expert system, with qualitatively similar results.
We have found that the newer uncertain inference systems often re-incorporated features of general probability theory which have been eliminated in earlier systems. Moreover, we found that newer systems sometimes continued exactly the features which they were supposed to eliminate, albeit in different notation. For every simple uncertain inference system, we found not only structures for which the performance was very good, but also structures for which performance was worse than random guessing, or systematic bias was present, or data was interpreted as having the opposite of its true import.
Publication Year: 1986
Publication Date: 1986-01-01
Language: en
Type: article
Access and Citation
Cited By Count: 5
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot