Title: Meta-Analysis and Validity Generalization as Research Tools: Issues of Sample Bias and Degrees of Mis-Specification: Philip Bobko and Philip L. Roth
Abstract:In this chapter, the focus is on meta-analysis as a more general case of validity generalization. The latter is usually quite focused on the transportability or generalization of the validity of selec...In this chapter, the focus is on meta-analysis as a more general case of validity generalization. The latter is usually quite focused on the transportability or generalization of the validity of selection systems, and the effect size of interest is most often the Pearson product moment correlation coefficient between a selection test and a measure of job performance (the validity coefficient). In contrast, meta-analysis focuseson cumulating knowledge about any organizational interventions, which might include not only selection systems, but training systems, motivational interventions, types of reward structures, and so forth. Meta-analysis can also focus on naturally occurring effects (e.g., the analysis of individual differences across ethnic or gender subgroups). So, meta-analysis differs a bit from validity generalization in that it focuses on a variety of effect sizes (e.g., the standardized mean difference between two groups or two types of interventions) and not just on Pearson correlations (the typical index of empirical validity), (Note: the effects of such a distinction may be more apparent than real because many effect sizes are often transformations of Pearson r's.)To repeat, the current chapter is written with the more general notion of meta-analysis in mind, although many of the comments apply to the more specific notion of validity generalization. In that regard, Bobko and StoneRomero (1998) suggested that proponents of meta-analysis (or validity generalization) might be a bit overzealous in claims about what metaanalysis could or could not accomplish. The current authors believe that meta-analysis and validity generalization can be very useful (practically and theoretically) techniques. Indeed, the potential utility of validity generalization can be found in other chapters in this book. However, it is also the case that Bobko and Stone-Romero noted that caveats for metaanalysis and validity generalization are in order. Attention to these caveats will enhance the usefulness, both practical and theoretical, of metaanalyses and validity generalization studies. As such, we build on their earlier work and explicate several of the concerns about metaanalysis already mentioned in the literature, We also go beyond some of these earlier caveats and update the issues, given new developments in the field. Finally, we note that the more specific notion of validity generalization tends to appear in human resources (HR) and selection applications, whereas meta-analysis appears in more general topic domains. We consider some of the more general applications in organizational behavior (OB) and find that OB applications often have more desirable characteristics (e.g., a more complete set of variables in their nomological nets) than HR applications. We show that some valuable lessons for HR can be learned from such a comparison.Read More
Publication Year: 2013
Publication Date: 2013-03-07
Language: en
Type: book-chapter
Indexed In: ['crossref']
Access and Citation
AI Researcher Chatbot
Get quick answers to your questions about the article from our AI researcher chatbot