Title: Framing effects in evaluation of accuracy of others' predictions
Abstract: Framing effects in evaluation of accuracy of others’ predictions Saiwing Yeung ([email protected]) Institute of Education, Beijing Institute of Technology, China Abstract Most predictions can be partitioned into two components: the predicted outcome, and the chance that one considers the out- come will happen. We studied how people evaluate predictions with binary outcomes. These predictions can be conveyed in two equivalent ways: one predicting an outcome with some probability, and the other predicting the other outcome with the probability of the complement of the first outcome. Although these two ways of stating the predictions are mathematically interchangeable, we hypothesized that people would judge the congruently stated prediction, one that has the same qualita- tive component as the actual outcome, as more accurate. We tested this hypothesis in four experiments. Results suggested that this effect is consistent across a number of domains; de- pends on the frame in which the prediction is stated; is robust regardless of whether the ratings were elicited in positive or negative terms; holds for both rating and choice tasks. Keywords: framing effects; probabilistic judgment; decision making. Probabilistic predictions are frequently encountered in ev- eryday life. For example, weather forecasts are often made in probabilistic terms (e.g. “chance of rain is 80%”). By comparing these statements against the actual outcomes, we can assess the predictors’ skills at predicting these events. It is important to be able to accurately evaluate other people’s predictions because it would then allow us to learn how good the predictors are in making predictions, to judge whether or to what degree should we trust the predictions, and to make decisions accordingly. For example, if a certain investment analyst predicts that there is a 99% chance that Acme Com- pany will declare bankruptcy, and that we consider this ana- lyst to be a good predictor, then it would be advantageous to sell stocks of Acme Company that we are holding. In this paper, we focus on one particular aspect of evaluat- ing predictions — how framing of predictions affect people’s evaluations. Framing effect is an extremely well-researched topic and has led to numerous scholarly work. It refers to a phenomenon in which people’s judgment, decisions, and ac- tions are influenced by frames, or presentation of information and its context. Framing effects have been found to influence people in var- ious ways in different contexts. Levin, Schneider, and Gaeth (1998) proposed a typology that categorized them into three main types. The first type, risky choice framing effect, in- duces a choice reversal effect between two logically equiva- lent gambles (Tversky & Kahneman, 1981). In a prototypical setup, participants see one of the two gambles: either choos- ing between a sure gain and a risky gain, or choosing between a sure loss and a risky loss. Previous research has found that a majority of the people would prefer the sure gain choice in the gain condition, and risky loss choice in the loss condition. The second type of framing effects was called attribute framing effects, as a single attribute within a given context presented in two logically equivalent frames has been shown to change people’s evaluations about the subject. For exam- ple, in Levin and Gaeth (1988), beef that was labeled as “75% lean” was rated as better tasting and less greasy than beef that was labeled as “25% fat.” Goal framing effects is the third type in Levin et al.’s ty- pology. Here negatively framed messages are found to be more persuasive than positively framed messages. Works by Meyerowitz and Chaiken (1987) demonstrated a typical setup of this problem. They found that women are more likely to perform breast self-examination (BSE) if they are told of the negative consequences of not performing BSE, compared to being told of the positive consequences of performing one. In the present study we report a new type of framing effect, in which people’s evaluation of a prediction with respect to the outcome is influenced by the frame in which the predic- tion is presented. We will focus on predictions in which there are clearly two possible outcomes (e.g. coin flips) and are stated with the subjective probability of said event happening (e.g. “80%”). Because there are exactly two outcomes, any predictions can be stated in two ways that are logically equiv- alent. For example, to say that there is a 99% chance that the world will be destroyed at end of 2012 is equivalent to a 1% chance that the world will not be destroyed at end of 2012. We argue, however, that people evaluate these predictions differently. As demonstrated by the framing effects liter- ature described earlier, people’s judgments are often influ- enced by how information is presented. In the context of pre- diction evaluation, we suggest that people would overweight the qualitative component of the prediction (the stated out- come), relative to its quantitative component (the chance that one considers the outcome will happen). To differentiate this from previously discovered types of framing effects, we will call this probabilistic statement framing effect (PSFE). We will next describe four experiments that were carried out to investigate this hypothesized effect. Pilot Experiment The main objective of the Pilot Experiment was to establish initial evidence about PSFE. To ensure the realism of the stimulus, we used a cover story about the 2012 U.S. presi- dential election which had just ended a few weeks prior. Methods The participants were recruited using Amazon Mechanical Turk (MTurk). Only workers who were residing in the U.S., were at least 18 years old, and had a lifetime acceptance rate with MTurk of 95% or over were allowed to participate 1 . 1 The same requirements applied to all experiments in this paper. Moreover, we disallowed participants from participating in more than one experiment in this paper (except for two participants who
Publication Year: 2013
Publication Date: 2013-01-01
Language: en
Type: article
Access and Citation
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot