The Base-Rate Fallacy in School Psychology: Implications for Decision-Making

This dissertation generalized the base-rate fallacy in diagnostic decision-making to the field of school psychology. This fallacy involves allowing information other than prior probabilities inappropriately to dominate probability judgments . Participants included 80 school psychologists who were asked to rate the probability of a learning-disability diagnosis and to state their degree of confidence in that diagnosis. In Study I, participants received only base-rate information. In Study 11, participants received either (a) relevant base-rate information plus irrelevant individuating information, (b) relevant base-rate information plus relevant diagnostic information, or (c) relevant base-rate information plus relevant diagnostic information plus irrelevant individuating information. Additionally, in Study II these three levels of information were completely crossed with a second independent variable, that is, salient link. Here, base-rate information was either linked or not linked with the diagnosis to be predicted. Results showed that (a) school psychologists appropriately used relevant base-rate information only when no other information was available, (b) when a salient link between relevant base-rate information and the diagnosis to be predicted was provided, diagnostic accuracy did not change, (c) when relevant diagnostic and irrelevant individuating information were provided in addition to base-rate information, diagnostic accuracy decreased relative to base-rate-only information, and (d) school psychologists were least confident of diagnoses for which they demonstrated most accuracy. Results were discussed in terms of implications for diagnostic decision making in the field of school psychology. Acknowledgment There are several people who provided ongoing help and assistance in completing this dissertation, but first and foremost, I would like to thank my major professor, Dr. W. Grant Willis. His insight, direction and clarity of thought was invaluable. There are no words to adequately express my appreciation for all he has contributed to this dissertation, as well as to my own professional development. I was extremely fortunate to have had Dr. David Faust as a member of my dissertation committee. I owe him a profound debt of gratitude for his ongoing support, advice, and insight in developing and completing this dissertation. I also want to thank Dr. Susan Brady and Dr. Dayle Joseph who helped to develop and refine this dissertation with their unique perspectives and ideas. I do not know how to begin to thank Dr. Michael Weiler who continually and unselfishly put aside his own work to answer "just one more question". His calm demeanor and sense of humor helped me make it through the most insane moments. He has been a constant source of inspiration and I feel very lucky to have him as a part of my life. Finally, I want to acknowledge my mother who deserves a part of this PhD. Without her, none of this would have been possible. I am very fortunate to have such a loving and unselfish role model.


Possible explanations
The base-rate fallacy  Table 1: Sample Characteristics for Study I 1 9      Introduction This dissertation examines the use of base rates by school psychologists in a diagnostic decision-making task. Current knowledge of diagnostic decision-making in school psychology is important because if suboptimal reasoning is used to make decisions about children and youth with disabilities, it is likely that many students will be misclassified. For example, children in need of special education and related services may not be identified, whereas those who do not need such services may be labeled as disabled. Children mislabeled as in need of services may then suffer social and emotional consequences. Fagley (1988) has called for research that examines whether school psychologists are prone to the same decision-making errors (i.e., underutilization or neglect of base rates) as has been documented in other fields (see Eddy, 1982;Schwartz, Gory, Kassirer, & Essig, 1973).
The present study was designed to investigate psychoeducational decision-making with respect to the processing of base-rate information and to examine conditions that may increase diagnostic decision-making accuracy. Base rates refer to the frequency of an event or diagnosis and are integral to diagnostic decision-making. Bayes theorem provides an explicit model of how to incorporate base-rate information into the decision-making process; however, research in the area of decision-making has documented that both clinicians and laypeople are subject to a systematic and predictable decision-making error, that is, the "base-rate fallacy" (Bar-Hillel & Fisch ho ff, 1981; Kahneman & Tversky, 1973;Lyon & Slovic;. This fallacy involves allowing information other than base rates to dominate when making probability judgments.
Thus, base-rate information often is overlooked even though it is essential to probabilistic decision-making.
In the following section, several areas of literature are reviewed that provide the theoretical underpinnings for this study. These include: (a) the research within cognitive psychology that examines the cognitive processes of human judgment and Kahneman, 1974) and because of the complexity and the variety of decisions we make (Nisbett & Ross, 1980). It has been documented, however, that the same processes and tools that can simplify a problem, when overapplied, also can lead to systematic and predictable errors (Tversky & Kahneman, 1974).
Thus, Tversky and Kahneman (1974) focused on gaining a better understanding of human information processing by examining: (a) the cognitive heuristics used to reduce complex decision-making tasks, and (b) the kinds of errors that are made through the use of these heuristics. They explained the justification for this research strategy: There are three related reasons for the focus on systematic errors and inferential biases in the study of reasoning. First, they expose some of our intellectual limitations and suggest ways to improve the quality of our thinking. Second, errors and biases often reveal the psychological processes .that govern judgment and inference. Third, mistakes and fallacies help the mapping of human intuitions by indicating which principles of statistics or logic are non-intuitive or counter-intuitive. (Kahneman & Tversky, 1982;p.124).
Heuristics. There are three main cognitive heuristics that can lead to systematic decision-making errors in prediction, in part, because they are not sensitive to baserate information. First, the representative heuristic refers to a decision-making strategy that relies on making a decision or prediction based on similarity. For example, research examining the representativeness heuristic has determined that people assess probabilities by . the degree to which the item to be predicted is similar or representative of their conception of a category (Kahneman & Tversky, 1972). One of the main problems with making probability judgments by representativeness is the neglect of prior probabilities, or base rates. Base rates have no effect on the representative heuristic, even though they should play a major role in assessing probabilities.
In other situations, people make probability judgments based on the availability or ease with which the category to be predicted can be brought to mind. The availability heuristic can be a useful tool for assessing probability because in many cases larger 3 classes of events are retrieved more quickly; however, availability also is influenced by other factors that have no bearing on the probability of occurrence. Specifically, availability can be influenced by familiarity, salience, the effectiveness of a search set, illusory correlation, and imaginability (Tversky & Kahn-eman, 1974). The influence of these factors leads to neglect of the base rates, and thus to errors in probability judgments .
Anchoring and adjustment, the third heuristic , rely on making probability judgments by starting from an initial value (anchor) and adjusting that value to reach a final estimate. This strategy of anchoring and adjustment has been demonstrated to result in predictable errors in which people insufficiently adjust their estimates by anchoring on the initial value ( Tversky & Kahneman, 1974). Anchoring can result when an initial starting value is provided or when an incomplete computation is made of the available data (e.g., failure to incorporate base-rate information) . Summary. In summary, the research within cognitive psychology has attempted to clarify the underlying cognitive processes of the human judge. The results of this research have documented the limitations of our information-processing systems and the resultant suboptimal reasoning strategies. Cognitive heuristics are a means by which we simplify complex problems, such as assessing probabilities . . These heuristics, however, can lead to serious judgment errors, in part, because they neglect base-rate information. Thus, one of the most important approaches used in studying judgment and decision making has been to analyze the systematic discrepancies between normative judgment models and actual judgments.

Research io Social Psychology
A cognitive approach to the study of human judgment within social psychology, which closely paralleled the study of human judgment within cognitive psychology, began to gain attention in the 1960s. This cognitive orientation stemmed from research such as Tajfel's (1969) "Cognitive Aspects of Prejudice" that applied a cognitive perspective 4 to the understanding of human behavior. In the early 1970s, the longstanding attitude-behavior link primarily used to explain human judgment was criticized, and a new emphasis was placed on the information-processing limitations, not the motivations, of the human judge (Israel & Tajfel, 1972). Further, the use of the computer to model human judgment was de-emphasized within cognitive psychology; instead, understanding the limitations of human judgment was stressed.
Schemas. Researchers in social psychology, as well as in cognitive psychology, emphasized the role of schemas in aiding human judgment and information processing.
Schemas are cognitive structures that are formed by past experiences that help us to organize and to understand information (Neisser, 1976 Schemas, like heuristics, however, can lead to biases and errors in judgment. These errors can occur when we miscategorize information or we oversimplify our processing of information. For example, our schemata can sometimes misdirect our attention and, thus, we ignore relevant information. We also can make errors by remembering schema-related details that did not actually occur (Schneider, 1988).
Heuristics also may guide the selection of schemas and, thus, inappropriate schemas may be activated (Nisbett & Ross, 1980). Further, it has been suggested that humans lack good schemata for dealing with probabilistic data. It appears that although the logical importance of base rates cannot be overstated, this kind of information fails to be effective in activating appropriate schemas.
Stereotypes can be thought of as schema-like concepts in that they guide us in encoding, retaining, and retrieving information in the social world. Hamilton and Rose (1980) suggested that we can understand stereotypes better by framing them as 5 correlational concepts. For example, the stereotype that women are bad drivers, is based on a perceived relationship between an identified group, that is, women, and some trait, that is, bad driving. If we examine how people form concepts and beliefs based on the covariation between two variables, we may understand stereotypes better .
Covarjatjon analysjs. The cognitive demands involved in accurately assessing covariation make us prone both to the development and maintenance of stereotypes (Hamilton & Rose, 1980). Determining when two (or more) variables truly are related, that is, covariation analysis, is a difficult cognitive task. In order to establish that a relationship exists between two variables, in this case a particular group and some trait, one must show that the trait occurs more frequently among that group than among other groups. Individuals have great difficulty analyzing covariation and thus commonly draw false conclusions about relationships between two variables (Arkes & Harkness, 1980;Gnys, Willis, & Faust, in press;Smedslund, 1963;Ward & Jenkins, 1965).
With respect to diagnostic decision making, this error occurs when decision makers consider only the frequency with which the sign is present and the disorder present, and fail to consider other situations in which (a) the sign is present but the condition absent, (b) the sign is absent but the condition is present, and (c) the sign is absent and the condition is absent. Failure to consider all of this information makes it impossible to determine whether a valid relationship exists between the sign and the condition.
These false associations between variables have been referred to as illusory correlations (Chapman & Chapman, 1967). The disregard of base rates is one of the main contributing factors to the formation of illusory correlations.
Here, the psychologist comes to believe that particular features of test performance provide trustworthy indicators of certain diagnostic categories, without attending to the base rates (Chapman & Chapman, 1967). For example, one commonly reported relationship 6 interpreted from the Draw-a-Person Test is that highly detailed drawings of eyes are associated with paranoia. Although research has failed to validate this (and many other sign-diagnosis relationships) on the Draw-a-Person Test, clinicians still often agree about such clinical correlates of Draw-a-Person performance (Ziskin & Faust, 1991 ).

Conclusions from Cognitive and Social Psychology
This section has highlighted the research in both cognitive and social psychology on human judgment and decision-making. Research has revealed some of the limitations of the human judge, and the cognitive strategies and rules used to compensate for these limitations. For example, cognitive heuristics and schemas facilitate processing of complex data. These decision aids, however, can lead to systematic errors, especially when dealing with probabilistic data, in part because they neglect base-rate information. The neglect of base rates has been implicated in diagnostic decision-making errors, as well as in errors of judgment in the social world. For example, stereotypes can be thought of as illusory correlations that develop and are maintained, in part, due to a neglect of base-rate information. In the following section, base rates and their role in diagnostic prediction are reviewed and the potential implications of the base-rate fallacy for the field of school psychology are discussed.

Base Rates
Base rates, as stated previously, refer to the frequency with which an event (i.e., a symptom or condition) occurs. For example, if school phobia exists in" 1 out of 100 junior high-school students, the base rate for school phobia in that population is 1 %.
Thus, base rates refer to the prevalence of a characteristic in a population.
One of the problems associated with the neglect of base-rate information, as discussed in the section on formation of stereotypes, is that a clinician may not know whether a sign is validly related to a condition. Knowledge of whether a sign is validly related to a condition, however, is not enough to determine whether use of that sign will be helpful in diagnostic prediction (Faust & Nurcombe, 1989). The clinician still needs 7 to know the base rate or frequency of the condition in the population. If one lacks knowledge of the base rate of the condition, then it is impossible to determine the diagnostic utility of the sign, that is, whether more erroneous classifications will be made when using the sign. The validity of diagnostic or predictive signs is never absolute or constant, but must be interpreted in relation to the prior odds or the base rates (Meehl & Rosen, 1955). Disorders that occur less frequently require a more accurate sign if more correct than false identifications are to be made. If use of a sign does not improve the accuracy one would achieve by relying on the base rates alone, then overall accuracy will be decreased by relying on the sign (Faust & Nurcombe, 1989).
The following example is adapted from Faust and Nurcombe (1989). Suppose that the prevalence of school phobia is 1 out of 1000 cases. School psychologist A relies on the base rates and consistently says that the condition is not present. School psychologist B only diagnoses the condition when there is overwhelming evidence.
Further, suppose that school psychologist B never misses a true case of school phobia, and only 1 out of 100 times, says the condition is present when it is not. On a sample of 1000 cases, school psychologist A will make one error, that is, will miss one case of school phobia when it is present. School psychologist B, on the other hand, will make approximately ten errors, that is, will detect the one case of school phobia, but also will misidentify nearly ten children (i.e., almost 1 of 100).
Thus, base rates are integral to diagnostic decision making both to determine: (a) whether a sign is validly related to a condition and, further, (b) whether use of that sign will increase diagnostic accuracy. As Faust and Nurcombe (1989) suggested, however, base rates should not be the sole criteria by which we guide our practice. Diagnostic decision making should be framed as a analysis of the kinds of errors we are willing to make (e.g., the consequences of overdiagnosing a condition versus missing a condition).
In some situations, missing a condition may have relatively minor consequences; in other circumstances, missing a condition can have life threatening consequences.
8 !gnorjng base rates. Research examining base rates in probabilistic decision making tasks has typically pitted base-rate information against some kind of individuating or diagnostic information (e.g., a personality description or the correct positive rate and false-alarm rate of a test) (Arkes, 1989). A consistent finding of this research is that individuals presented with base-rate information and case-specific, or individuating information, will defer to the individuating information (Kahneman & Tversky, 1973). For example, in their seminal work, Kahneman and Tversky (1973) presented participants with base-rate information (e.g., percentage of people employed as lawyers versus engineers), and individuating information (e.g., a brief personality description). The personality description was designed to be non-diagnostic in differentiating lawyers from engineers. Participants were asked to make a probability judgment (on a scale of 0 to 100) as to whether the person was an engineer or a lawyer.
Results of their study demonstrated that participants relied on the stereotypic description and ignored the base-rate data. Participants did use base-rate data when no other information was presented.
A similar finding was documented by Borgida and Nisbett (1977). In their study, participants were presented with base-rate information in the form of mean course ratings, and individuating information in the form of former students' descriptions of the course. Again, participants relied on the individuating information for course selection and neglected the base rates or mean course ratings.
Probability judgments also can be dominated by an accuracy effect (i.e., anchoring_ on the diagnosticity of specific information) (Lyon & Slavic, 1975). Tversky and Kahneman (1980) conducted a study in which participants were presented with a scenario and asked to make a probability judgment. They were told that 85% of the cabs in the city are blue and 15% are green. A cab has been involved in an accident and a witness has identified the cab as green. The witness was accurate in identifying each color 80% of the time and confused colors 20% of the time. Participants were asked to estimate the probability that the cab involved the accident was green as the witness had stated. In this Bayesian-inference problem, all normatively relevant information (i.e., base rates, correct positive rate, and false positive rate) was supplied.
Bayes rule provides a normative formula for this decision: Here, Pj is the probability that the cab is green; P is the base rate of green cabs in the city, p1 is the valid positive identification rate, Q is the base rate of blue cabs in the city, and P2 is the false positive identification rate. Applying Equation 1 to the previous example, the correct probability judgment is 41%, that is, .41 = (.15 x .80) I [ (.15 x .80) + (.85 x .20)]. Instead, participants tended to anchor on the hit-rate information without integrating pertinent base-rate information, and made estimates of 80%.
Studies have been conducted examining the stability of the base-rate fallacy (Bar-Hillel, 1975;Lyon & Slovic, 1976). For example, variations of one · of Tversky and Kahneman's original base-rate problems were presented to 350 participants. Some of the variables manipulated included order of information presentation, varying the extremity of the base rates (i.e., 90% versus 60%), and using a verbal description instead of actual statistics for the base-rate data (i.e., using words such as most instead of actual numbers). Results of this research consistently demonstrated that when other information was present, the base-rate information was neglected. Thus, the base-rate fallacy is considered to be a robust phenomenon (Ajzen, 1977;Bar-Hillel, 1980;Lyon & Slovic, 1976;Nisbett & Borgida, 1975).

Causality.
In an attempt to encourage people to use base-rate information, cognitive · psychologists have attempted a variety of interventions (Arkes, 1989).
One approach has been to enhance the causal link between the base-rate information and the category to be predicted. Nisbett and Ross (1980) concluded that individuals may lack an intuitive understanding of Bayesian statistics, but their judgments would be much less impaired if base rates could be made more salient. Situations in which base 1 0 rates appear more salient include conditions where no other information is presented or where the base rate has a causal interpretation.
Studies have manipulated causal relevance and demonstrated that base rates may dominate probability judgments in particular conditions (Ajzen, 1977). For example, in a study conducted by Ajzen, some participants were given a brief description of a student, Gary. Participants then were asked to make a probability judgment as to whether Gary passed or failed an exam. The first group of participants was told that Gary had been randomly sampled from a group of students, among which 75% had failed the exam and 25% had passed the exam. These participants ignored the normative relevance of the base rates in favor of the individuating information. Another group of participants, however, was told that the test was failed by 75% of the people in Gary's class and passed by 25% of the people in Gary's class. These individuals did attend to the high base rate of failures (i.e., 75%) and did not defer to the individuating information.
Thus, Ajzen concluded that base rates that are causally linked to the event or target to be predicted may influence probability judgments. Bar-Hillel (1980) discussed the relationship of causality to relevance , proffering that causality is only one aspect of relevance. She suggested that individuals may ignore base-rate information because they think it is irrelevant. Moreover, she stated that if the relevance of base-rate information is enhanced, individuals may be more likely to use this information. Results of her study indicated that two items of information may be used only if they appear equally relevant. Thus, one way relevance can be manipulated is by providing a causal link between the base-rate data and the outcome to be predicted. In this way, the base rate is interpreted as having some causal relation to the outcome to be predicted.

Research auestjons
This dissertation was an investigation of whether the base-rate fallacy generalized to the field of school psychology. Further, conditions were varied to 1 1 determine whether there are conditions favorable to the use of base-rate information.
Specifically, this study examined the effects of illusory-correlation information, accuracy-rate information, and salience of link, on the use of base-rate information. To clarify terminology, accuracy-rate information has been referred to in the research literature as diagnostic or true-positive rate information, illusory correlation has been referred to as individuating information or a belief in a relationship that does not exist, and salience of link has been referred to as relevance or causality.
In this study, the illusory-correlation was related to the diagnosis of a learning disability. The illusory-correlation information was included to elucidate how this kind of information, in combination with base-rate information and accuracy-rate information affects diagnostic accuracy. This is considered to be an important area of investigation as research has demonstrated that school psychologists are prone to the phenomenon of illusory correlation.
The choice of the illusory correlation was based on research (Gnys et al., in press) that has documented that some school psychologists believe in a systematic relationship between levels of intersubtest scatter on the Wechsler Intelligence Scale for Children--Revised (WISC-R) and the diagnosis of learning disabilities, although research has failed to validate this relationship (Kaufman, 1979). Moreover, research has documented that normal children show a mean range of seven (fil2. = 2) scaled points on the WISC-R (Kaufman, 1976). Thus, a scaled-score difference as large as nine points is unremarkable (Reynolds & Kaufman, 1990). These findings document that intersubtest scatter has negligible predictive value in the diagnosis of learning disabilities. Further, warnings have been advanced (Kaufman, 1976) that levels of scatter that frequently occur in the normal population should not influence diagnostic decisions.
The diagnostic information provided the accuracy rate of the test (i.e., the percentage of children diagnosed as learning disabled who are learning disabled) and the 1 2 false-positive rate (the number of children identified as learning disabled who are not learning disabled). In conditions where accuracy-rate information was presented, all the normatively relevant information necessary to make a Bayesian calculation was presented. Finally, salience of link was explored to examine whether providing a link between the base-rate information and the category to be predicted would increase diagnostic accuracy. These variables were manipulated systematically to provide directions for future intervention considerations.
In conclusion, this dissertation addressed the following questions: 1. Do school psychologists use base-rate information? It was predicted that school psychologists would use base-rate data when no other information was presented.
2. Does providing a salient link between the base rate and the category to be predicted increase accuracy? It was predicted that school psychologists' probability judgments would be more accurate when there was a salient link between the base rateinformation and the category to be predicted. 3. Does the kind of information presented with the base-rate information affect accuracy? It was predicted that school psychologists would be less accurate when presented with base-rate information plus illusory-correlation information plus accuracy-rate information versus when presented with either (a) base-rate information plus accuracy-rate information or (b) base-rate information plus illusory-correlation information ..
The first prediction was based on research showing that when base-rate information is presented alone, both lay people and professionals use that information appropriately when making probability judgments (e.g., Kahneman & Tversky, 1973).
The second prediction was based on limited research showing that when a salient link between the base-rate information and the category to be predicted is provided, judgmental accuracy improves (Ajzen, 1977). Finally, the third prediction was based on two areas of research showing that judgmental accuracy decreased when base-rate 1 3 information was combined either (a) with relevant diagnostic information (e. g., Lyon & Slovic, 1975)  Materials and procedure. All psychologists received a cover letter that explained the purpose of the survey and guaranteed their anonymity. A copy of this letter appears in Appendix A. The information packet comprised two parts. In the first section, respondents indicated: (a) highest academic degree, (b) year of degree, (c) gender, (d) geographic location, and (e) primary professional role (e.g., practitioner , trainer). In the second section, the packet included a case scenario providing base-rate information, specifically the percentage of boys diagnosed as learning disabled in a school (i.e., 10%).
The case scenario used in Study I appears in Appendix B.
Respondents first filled out the demographic information on the questionnaire.
They then were asked to rate the probability that the child described by the case scenario was learning disabled on a scale from · 0% (lowest--not probable at all) to 1 00% (highest--certain). They also were asked to rate their confidence in their decision on a Likert-type scale ranging from 1 (not confident) to 7 (very confident) . Respondents' probability ratings were used to calculate an accuracy score. This score represented the absolute difference between the participant's stated probability rating and the correct rating of 10%. Thus, lower difference scores represented higher accuracy. year of degree, (c) gender, (d) geographic location, and (e) primary professional role (e.g., practitioner, trainer).
The second section contained case-scenario information. Three kinds of information were provided in varying combinations. This included (a) base-rate information; (b) irrelevant, individuating information designed to assess susceptibility to illusory correlation; and (c) relevant, diagnostic accuracy -rate information. Every packet included base-rate information, specifically the percentage of boys diagnosed as learning disabled in a school (i.e., 10%). Two-thirds of the packets contained the irrelevant information. This was provided by including the summary page of the response booklet from the WISC-R. This summary page showed a range of intersubtest scatter of 9 scaled-score points. This range was chosen because of its fairly common occurrence and its lack of diagnostic utility (Kaufman, 1979;McDermott , Gluttin, Jones, Watkins, & Kush, 1989;Reynolds & Kaufman, 1989 Finally, in addition to the case-scenario information, half of the packets contained a statement clearly linking the base-rate information with the diagnosis to be predicted (i.e., "Because of an effective early-intervention program in John Smith's school, the incidence of learning disabilities among boys in this school is only 10%."), whereas the other half did not contain this statement. The case scenarios used in Study II appear in Appendix C.
Respondents first filled out the demographic information on the questionnaire.
They then were asked to rate the probability that the child represented by the casescenario information was learning disabled on a scale from 0% (lowest--not probable at all) to 100% (highest--certain). They also were asked to rate their confidence in their decision on a Likert-type scale ranging from 1 (not confident) to 7 (very confident).

Design.
Each participating school psychologist was assigned randomly to one of six conditions created by varying the levels of two independent variables. One independent variable, salience of link (i.e., provision of a link between the base 0 rate data and the category to be predicted), comprised two levels: (a) salient link between the base-rate data and category to be predicted and (b) no link provided between the baserate data and category to be predicted. The other independent variable, information, comprised three levels: (a) base-rate information plus relevant, that is, accuracyrate, information (BR + AR), (b) base-rate information plus irrelevant, that . is, illusory-correlation, information (BR + IC), and (c) base-rate information plus accuracy-rate information plus illusory-correlation information (BR + AR + IC).
The levels of these variables were completely crossed, resulting in a two-bythree between-groups design. Specifically, one group received a case scenario defined by BR + AR information with no link, one by BR + AR information with a salient link,  1988-1993 1983-1987 1978-1982 1973 -1977 1972 or before Results of Study I revealed that all respondents used base-rate information accurately when no other information was presented (M = O, .sJ2 = 0). That is, there was no difference between the reported and the optimal probability judgment for any of these respondents. The mean confidence rating was 2.7 (SQ= 1.7).

Analysis 11
Of the 300 surveys mailed for Study 11, 106 were returned, yielding an initial Most of the respondents (80%) were employed as school psychologists, but 6% indicated that their primary role was an administrator or supervisor, 7% a university trainer, and 7% another role. Of the 120 surveys used, 23% were from northeastern, 27% from north-central, 26% from western, and 24% from southern parts of the United States. The sample characteristics for Study II are presented in Table 2.  Table 3. Table 3 Two-way ANOVA     Table 5. Table 5 One-way ANOVA  Table 6 presents the mean probability ratings and the mean accuracy ratings (i.e., the absolute difference between correct and stated probability judgment) for each information condition.  26

Discussion
Based on research conducted with laypeople and professionals alike (e.g., Eddy, 1982;Kahneman & Tversky, 1973;Schwartz, Gory, Kassirer, & Essig, 1973), three major predictions were made. First, it was predicted that school psychologists would use base-rate information appropriately to make probabilistic diagnostic decisions in the absence of other clinical information. Second, it was predicted that providing a salient link between base-rate information and the diagnosis to be predicted would improve diagnostic accuracy. Finally, it was predicted that when school psychologists were presented with other clinical information (i.e., illusory-correlation and accuracy-rate information) in addition to base rates, a greater degree of error would be present when both individuating, that is. , illusory-correlation (IC), and diagnostic, that is, accuracy-rate (AR) information were presented together with base-rates (BR), than when either of these sources of information was presented alone with base rates.

Use of Base Rates
Results of the present investigation supported only the first hypothesis. Here, despite relatively low levels of diagnostic confidence, participants did use base-rate information appropriately in the absence of other clinical information. This clearly was shown in the responses of the school psychologists, every one of whom was completely accurate on the probabilistic diagnostic decision task that was administered (i.e., M difference between correct vs. stated probability = 0, .fil2 = 0). This result is consistent with other research (Kahneman & Tversky, 1972;Lyon & Slovic, 1976) showing that when base rates are presented alone, individuals are able to use that information accurately. Thus, this result generalizes the tendency to use base-rate information appropriately in the absence of other clinical information to the sample of school psychologists used in the present study. It shows that the school psychologists appropriately used these relevant base-rate data when making probabilistic diagnostic decisions about a case. Although not surprising, this finding is important because it demonstrates that when no other clinical information is present, school psychologists (like most other professionals and lay persons) use base rates accurately.

Salience of Link between Base Rate and Diagnosis
The second hypothesis, stating that school psychologists would be more accurate in their probabilistic diagnostic decisions when a salient link was provided between the base-rate information and the category to be predicted, was not supported. Responses of school psychologists who were provided with a salient link in their case-scenario information did not differ significantly from those who did not receive a salient link.
Thus, school psychologists were not more accurate in their probabilistic diagnostic decisions when they had a salient link provided between the base-rate information and the category to be predicted.
Other research examining salience as a potential intervention to remediate or correct the base-rate fallacy, has yielded equivocal results. Some research has documented that providing a salient link between the base-rate information and the category to be predicted enhances the relevance of the base rates and thereby improves accuracy (Bar-Hillel, 1980), whereas other atte_ mpts to improve accuracy by manipulating salience have proven to be unsuccessful (Lyon & Slovic, 1976). Because the power associated with this analysis was insufficient to guard adequately against type-II errors (i.e., ~ = .85), and because of the limited research conducted on salience especially within school psychology, salience currently should not be ruled out as a potential avenue for reducing the base-rate fallacy. Perhaps, a potentially more powerful link between the base-rate information and the category to be predicted may help to improve the perceived relevance of base-rate information and thereby increase diagnostic accuracy. This issue, of course, requires further empirical assessment.
Base-rate, lodividuatjna, and Diagnostic Information The third hypothesis, concerning differential levels of accuracy associated with combinations of base-rate, illusory-correlation (or individuating), and accuracy-rate (or diagnostic) information, was not supported. In contrast to what was predicted, psychologists who received both irrelevant illusory-correlation information as well as relevant accuracy-rate information with base rates were not the least accurate. Instead, their responses did not differ significantly from those who received either the relevant accuracy-rate information with base rates or those who received the irrelevant illusory-correlation information with base rates. Moreover, contrary to what might be expected, participants who received the irrelevant illusory-correlation information with base rates were more accurate than .those who received the relevant accuracy-rate information with base rates.
This third hypothesis was based on previous research conducted in school psychology and cognitive psychology that has documented the effects of illusorycorrelation and accuracy-rate information on probabilistic decision making.
Specifically, research in the field of school psychology (Gnys et al., in press) has revealed that school psychologists' diagnostic decisions are influenced adversely by an illusory belief in an association between particular psychometric results and the diagnosis of a learning disability. Further, research in cognitive psychology (Lyon & Slovic, 1976) has documented that individuals often rely on the accuracy rate of a test and fail to incorporate other relevant information (e.g., base rates) when making probability judgments. Research has not examined how these two sources of information, when combined, affect diagnostic decision-making. It seemed reasonable, however, to speculate that diagnostic decision-making errors associated with both of these sources of information combined (i.e., both irrelevant illusory-correlation information and relevant accuracy-rate information) would be potentiated (or compounded) over those associated with either source of information alone. Thus, probabilistic diagnostic decisions for situations involving base-rate , illusorycorrelation, and accuracy-rate information were predicted to be less accurate than for situations involving either (a) base-rate and illusory-correlation information or (b) base-rate and accuracy-rate information .
Possible explanations. As noted, however, this hypothesis was not supported in the present study. Moreover, the amount of error associated with adding relevant accuracy-rate information to base rates exceeded the amount of error associated with adding irrelevant illusory-correlation information to base rates. Results of a post-hoc analysis between information presented and year of terminal degree revealed that there were no significant differences according to this variable. Thus , year of terminal degree did not provide an explanation of the differential accuracy among the kinds of information presented. There are, however, several possible explanations for these differential levels of accuracy, including, but not limited to: (a) the extremity of the accuracy-rate information provided, (b) the substantative nature of the illusorycorrelation information provided, and (c) the assumption of additive effects of error associated with these two sources of information.
The extremity of the accuracy-rate information (i.e., 80%) may provide one explanation for the differential levels of accuracy. For example, research has documented that when individuals are presented with accuracy-rate information, they often base their decisions on that information and neglect other pertinent information. Lyon and Slovic (1976) suggested that probability estimates often are determined by the most salient features of case-specific evidence. In this investigation, the stated accuracy-rate information may have been particularly salient given its extreme nature, that is, the procedure was described as one that is accurate in identifying the disorder as present 80% of the time, and inaccurate in identifying the disorder as present 20% of the time. Thus, participants may have made diagnostic decisions primarily on the basis of accuracy rate, thereby increasing error . Unfortunately, the design of this investigation did not permit an examination of the sources of information upon which participants relied in order to make their diagnostic decisions. Thus, an evaluation of this possibility awaits empirical verification.
The substantive nature of the illusory correlation may also provide an explanation of why school psychologists were more accurate when presented with baserate and illusory-correlation information than with base-rate and accuracy -rate information. In the illusory-correlation conditions, participants were given the results of the WISC-R and were asked to make a probability judgment about the diagnosis of a learning disability. Contemporary definitions of learning disabilit ies rely on academicachievement information (Hammill, 1990;Hooper & Willis, 1989), and the psychoeducational assessment of learning disabilities typically includes the administration of both aptitude and achievement tests. Because academic-achievement information was not included with aptitude information in these conditions , participants may have perceived that an essential source of diagnostic information was missing, thereby becoming more conservative (and, therefore, more accurate) in their probability estimates than participants in the base-rate and accuracy-rate conditions.
Finally, the assumption of additive effects of error associated with these two sources of information (i.e. , accuracy-rate and illusory -correlation information) may have been incorrect. Instead, results of this study suggest that these two sources of information, when combined with base-rates, did not influence probabiiity judgments in a solely additive fashion. Compounding or potentiating error would occur if these sources of information were independent, but error associated with one source of information also could offset error that is inversely correlated another source of information. Again, however, the design of this investigation did not permit an examination of this issue, because diagnostic-information conditions (e.g., BR + IC; BR + AR) were treated as a between-groups rather than as a within-subjects variable .
The base-rate fallacy. Because salience of the link between base-rate and diagnosis did not affect the accuracy of probabilistic diagnostic decision-making of the school psychologists, the two levels of this variable (i.e., salient link present vs. salient link absent) were collapsed, thereby permitting a four-way comparison among levels of diagnostic information. Here, in Analysis 111, responses from Study I (involving only base-rate information) were compared with responses from Study II (involving BR + AR, BR + IC, and BR + AR + IC information). Results of this comparison revealed that participants who received base-rate information only were more accurate than all other participants. This finding is particularly important because it documented that when school psychologists were presented with base-rate information in combination with either irrelevant information, diagnostic information, or irrelevant and diagnostic information, as compared to base rates alone, their diagnostic accuracy decreased. This .
investigation, therefore, generalizes the base-rate fallacy to the present sample of school psychologists (Ajzen, 1977;Bar-Hillel, 1980;Lyon & Slovic, 1976 ;Nisbett & Borgida, 1975), and is perhaps the first study to demonstrate empirically that school psychologists are likely to neglect, or to underutilize , base rates in probabilistic decision tasks when either individuating information or diagnostic information is present.

Diagnostic Confidence
Results also revealed that participants presented with base-rate information only and those presented with base-rate and illusory-correlation information were less confident in their diagnostic judgments than those presented with base-rate and accuracy-rate information or with base-rate information combined with accuracy-rate and illusory-correlation information. There was no difference in confidence between participants who received base-rate information and those who received base-rate and illusory-correlation information. Similarly, there was no difference in confidence between participants who received base-rate and accuracy-rate information and those who received base-rate information combined with accuracy-rate and illusory- These results are consistent with other research that demonstrates that individuals who have considerable confidence in their decisions often make errors (Einhorn & Hogarth, 1978;Fischoff, 1977Fischoff, , 1982. Indeed, in this investigation, the most accurate school psychologists (i.e., base-rate only information) were less confident in their probabilistic diagnostic decisions than all groups but one (i.e., the base-rate plus illusory-correlation information group). Similarly, the participants who received illusory-correlation information reported lower confidence levels than the less accurate participants who received accuracy-rate information. These results suggest limited insight, in the sense that subjective levels of confidence did not match objective measures of accuracy for the present sample of school psychologists (cf., Fisch, Hammond, Joyce, & O'Reilly, 1981;Gauron & Dickinson, 1966).

Conclusions
Results of this investigation generalized the base-rate fallacy to the field of school psychology. School psychologists were able to use base-rate information accurately in the absence of other clinical information. Their accuracy, however, decreased when presented with additional information, either irrelevant or diagnostic.
This finding is consistent with other research examining base rates, and highlights the need to develop interventions to address the neglect of base-rate information for diagnostic decision making.
The intervention designed to reduce the error in school psychologists probability judgments by increasing the salience of the base-rate information was not effective. The power of this analysis, however, was insufficient and therefore this variable requires addditional empirical evaluation.
Contrary to prediction, results also documented that combining irrelevant individuating and relevant diagnostic information did not generate the greatest amount of error in school psychologists' probability judgments. This result suggests that error, in fact, was not compounded, and raises questions about how different kinds of information are aggregated in probabilistic diagnostic decision-making tasks.
Finally, results documented that the more confident school psychologists were not the most accurate. Thus, the higher confidence of some of the participants in this study was unwarranted based on their accuracy. This result suggests limited insight about diagnostic decision processes.
Implications. It has been demonstrated that the neglect or underutilization of base rates can lead to serious diagnostic decision-making errors. Meehl and Rosen (1955) warned psychologists against evaluating tests by their accuracy rate alone; and, instead, emphasized the importance of a decision-making framework that incorporates base rates, costs of error, goals, and other pertinent information.
The findings of this investigation, combined with other decision-making research in school psychology (Aspel, 1992;Gnys et al., in press;Kennedy, Faust, Willis, & Piotrowski, in press) raise concerns about the diagnostic strategies of school psychologists. It appears that school psychologists are prone to some of the same decision-making errors (e.g., illusory correlation, lack of awareness, the base-rate fallacy) that have been documented in other fields (see Eddy, 1982;Schwartz et al., 1973).
Given that diagnosis is a probabilistic task, these results highlight the need for efforts to improve diagnostic accuracy by eliminating the base-rate fallacy. Clearly, school psychologists demonstrated that they use base rates when that information was presented in isolation. Unfortunately, in a more typical diagnostic decision-making task much more information usually is presented. Base rates typically are not presented alone and even may be obfuscated by poorly operationalized or poorly validated diagnostic criteria. Moreover, in many judgment tasks there is more non-predictive than predictive information. Sometimes information about the base rate of a diagnosis is unavailable to a clinician. In some cases, national base-rate information may be available, but information about base rates specific to the sample in question may be lacking. Despite its importance to diagnostic decision making, however, Faust and Nurcombe (1989) noted that little effort seems to have been directed towards collecting base-rate information.
Of course, individual clinicians can make base-rate estimates based on representative samples of cases, but this requires that they first be aware of the importance of base-rate information for diagnostic accuracy. Unfortunately, results of this study suggest that school psychologists have limited awareness of the importance of base-rate information for diagnostic accuracy. Other research that has attempted to improve accuracy by identifying problematic judgment strategies and warning diagnosticians to avoid them have proven unsuccessful (Arkes, 1981;Kurtz & Garfield, 1978), perhaps because awareness of a problematic decision-making strategy inevitably does not address the cognitive processes that underlie that strategy (Faust & Nu rcombe, 1989).
Additional research is needed to explore potential avenues for improving the use of base-rate information in the presence of other clinical information. Results of this study present potential implications for trainers as well as practitioners. It has been suggested that one reason base rates may be neglected is because humans rely on heuristics to reduce the complexity of many cognitive tasks. Perhaps, providing alternative heuristics for probabilistic decision-making will enhance the use of baserates. Faust and Nurcombe (1989) suggested several strategies to guide clinical decision making based in part on principles of probabilistic reasoning. Of course, much research is needed to see whether teaching these strategies will improve probabilistic decision making.
In conclusion, research in the area of probabilistic diagnostic making has revealed numerous problematic judgment strategies. There is now a growing body of Please read the case described below and answer the proceeding questions .
(1) A procedure is used to diagnose learning disabilities.
(2) Because of an effective early intervention program in John Smith's school, the incidence of learning disabilities among boys in this schoo! is only 10% .
(3) John Sm ith has been tested and results are positive for a learning disability .
(4) Part of the procedure for diagnosing learning disabilit ies involves administering an intelligence test. Below you will find John Smith's WISC-R (Wechsler Intelligence Scale tor Children--Revised) results . Please read the case described below and answer the proceeding questions .
(1) A procedure is used to diagnose learning disabilities .
(2) When the results of the procedure are positive for a learning disability , the procedure is correct 80% of the time in ident ifying learning disabled students as learning disabled . The procedure is wrong 20% of the time, that is , it misidentifies children who are not learning disabled as learning disabled .
(3) A total of 10% of the boys in John Smith's school have a learning disab i lity .
(4) John Smith has been tested . and results are positive for a learning disability .
1.) Please indicate the probability that John Smith has a learning disability on a scale from 0% (lowest --not probable at all) to 100% (highest --certain) : There is a __ % that this child is learning disabled. Please read the case described below and answer the proceeding questions.
(1) A pro.cedure is used to diagnose learning disabilities .
(2) A total of 10% of the boys in John Smith's school have a learn ing disability.
(3) John Smith has been tested and results are positive for a learning disability .
(4) Part of the procedure tor diagnosing learning disabilities involves administering an intelligence test. Below you will find  There is a __ % that this child is learning· disabled .
2 ) Please rate your confidence in your dec ision on a scale of 1 (not at all confident ) to 7 (very confident) : Please read the case described below and answer the proceeding questions .
(1) A procedure is used to diagnose learning disabilities.
(2) When the results· of the procedure are positive for a learning disability, the procedure is correct 80% of the time in identifying learning disabled students as learning disabled . The procedure is wrong 20% of the time, that is , it misident ifies children who are not learning disabled as learning disabled.
(3) A total of 10% of the boys in John Smith's school have a learning disability .
(4) John Smith has been tested and results are positive for a learning disability .
(5) Part of the procedure for diagnosing learning disabilit ies involves administering an intelligence test. Below you will find John Smith's WISC-A (Wechsler Intelligence Scales for Child ren--Revised) results.   1.) Please indicate the probability that John Smith has a learning disability on a scale from 0% (lowest --not probable at all) to 100% (hig hest certa in): There is a __ % that this ch ild is learn ing disabled . Please read the case described below and answer the proceeding questions .
(1) A procedure is used to diagnose learning disabilities .
(2) When the results of the procedure are positive for a learning disability, the procedure i:; correct 80% cf the time in identifying learning disabled students as learning disabled . The procedure is wrong 20% of the time, that is , it misidentif ies children who are not learning disabled as learning disabled .
(3) Because of an effective early intervention program in John Smith's school, the incidence of learning disabilities among boys in this school is only 10%.
(5) . John has been tested and results are positive for a learning disability .
1.) Please indicate the probability that John Smith has a learning disa!Jility on a sca!e from 0~1= (lowest --not probable at a!I) tc 100% (highest --certain) : There is a __ % that th is child is learning disabled . Part II. Diagnostic Decision Making: Please read the case described below and answer the proceeding questions .
(1) A procedure is used to diagnose learning disabilities .
(2) When the results of the procedure are positive for a learning disability, the procedure is correct 80% of the time in identifying learning disabled students as learning disabled. The procedure is wrong 20% of the time, that is, it misidentifies children who are not learning disabled as learning disabled .

VHIAl TUTS
2.) Please rate your confide nce in your decision on a scale of 1 (not at all confident) to 7 (very con fident) :