Conscientiousness and GMA as predictors of Job Performance - smaller and more complex than imagined.
By Paul Barrett
Remember, too, that being equal does not mean being the same. Now, let’s proceed. – Joanna L. Krotz
I was responding to a question from a consultant recently who asked: “what Conscientiousness & GMA measure would you recommend?”
My answer was: “After this ‘just published’ paper, I’m not sure I’d bother with either for any high-stakes job.”.
Harris-Watson, A.M., Kung, M., Tocci, M.C., Boyce, A.S., Weekley, J.A., Guenole, N., & Carter, N.T. (2022). The interaction between conscientiousness and general mental ability: Support for a compensatory interaction in task performance. Journal of Business and Psychology, 37, 4, 855-871.
Abstract: We propose a compensatory interactive influence of conscientiousness and GMA in task performance such that conscientiousness is most beneficial to performance for low-GMA individuals. Drawing on trait by trait interaction theory and empirical evidence for a compensatory mechanism of conscientiousness for low GMA, we contrast our hypothesis with prior research on a conscientiousness-GMA interaction and argue that prior research considered a different interaction type. We argue that observing a compensatory interaction likely requires (a) considering the appropriate interaction form, including a possible curvilinear conscientiousness-performance relationship; (b) measuring the full conscientiousness domain (as opposed to motivation proxies); (c) narrowing the criterion domain to reflect task performance; and (d) appropriate psychometric scoring of variables to increase power and avoid type 1 error. In four employee samples (N1 = 300; N2 = 261; N3 = 1,413; N4 = 948), we test a conscientiousness-GMA interaction in two employee samples. In three of four samples, results support a nuanced compensatory mechanism such that conscientiousness compensates for low to moderate GMA, and high conscientiousness may be detrimental to or unimportant for task performance in high-GMA individuals.
And from another “in press” article:
Sackett, P.R., Zhang, C., Berry, C.M., & Lievens, F. (2022). Revisiting meta-analytic estimates of validity in personnel selection: Addressing systematic overcorrection for restriction of range. Journal of Applied Psychology, Advance online publication, 1-104.
Abstract: This paper systematically revisits prior meta-analytic conclusions about the criterion-related validity of personnel selection procedures, and particularly the effect of range restriction corrections on those validity estimates. Corrections for range restriction in meta-analyses of predictor-criterion relationships in personnel selection contexts typically involve the use of an artifact distribution. After outlining and critiquing five approaches that have commonly been used to create and apply range restriction artifact distributions, we conclude that each has significant issues that often result in substantial overcorrection and that therefore the validity of many selection procedures for predicting job performance has been substantially overestimated. Revisiting prior meta-analytic conclusions produces revised validity estimates. Key findings are that most of the same selection procedures that ranked high in prior summaries remain high in rank, but with mean validity estimates reduced by .10-.20 points. Structured interviews emerged as the top-ranked selection procedure. We also pair validity estimates with information about mean Black-White subgroup differences per selection procedure, providing information about validity-diversity trade-offs. We conclude that our selection procedures remain useful, but selection predictor-criterion relationships are considerably lower than previously thought.
From p. 50: “Cognitive ability. The cognitive ability validity estimate decreased by .20, from .51 to .31. While this estimate is based on a number of additional meta-analyses, the prime driver of the decrease is a revised estimate of the degree to which the set of studies in question were affected by range restriction.”
p.51: “Conscientiousness. The conscientiousness validity estimate decreased by .12, from .31 to .19. This results from multiple new meta-analyses in this domain. Range restriction is not a significant factor in this domain”.
The reality is, unless you really are recruiting/selecting for potential high-autonomy/high-stakes entrant candidates, you might just as well hire from a well-designed biodata screen, with a random number generator for unbiased selection on the people who make it through the pre-screen.
Anyway, I’m writing further on this soon with some computational modelling to show how and why it works, especially for low-stakes high-volume assessment.