Case studies: Predicting the academic and work performance of accountants using the CPP

By Maretha Prinsloo on February 8, 2017

The following seven studies were conducted in two of the big four accounting firms in South Africa. The goal was to evaluate the degree by which the CPP predicts the academic and work performance of trainee and qualified accountants. More specifically:

• Do cognitive results as measured by the CPP predict the results of the financial qualifying exam (FQE I and II) as well as the work performance of postgraduate accounting students? (Note that at the time of these studies, the FQE I & II were the final qualifying chartered accountant exam in South Africa but has since been replaced by the PPE / QE II Financial Management exam.)
• Does the CPP predict the work performance of trainee accountants as indicated by 360 degree performance appraisal outcomes?
• Is there any adverse impact involved with CPP assessment, and can the CPP be used in an unbiased and fair manner to evaluate the cognitive preferences and capabilities across different gender and race groups in South Africa?
• Does the use of the CPP in selecting accounting bursars and placing young accountants, add value to the business?

The results of the various studies are briefly summarised here:

Study 1: CPP and the FQE I examination results (2001)
(n = 300)

The CPP results of a sample of 300 accounting students that managed to write their FQE I exams in 2001 were compared to their academic results.

It was found that a large number of the CPP dimensions significantly correlated with the examination results of the candidates, in particular those of: Analytical Skills, Logical Reasoning, Quick Insight Learning, Memory and Metacognitive awareness. Significant correlations of around 0.4 – 0.6 (p < 0.0001) were largely achieved between the exam pass rate and the above mentioned CPP dimensions. This study provides evidence that the CPP predicts the academic results of accounting students.

Study 2: CPP and pass rate for candidates from diverse socio-economic backgrounds (2005)
(n = 752)

The sample used for the purposes of this study was pre-selected using the CPP. Only those that showed the cognitive potential to deal with the complexity of the Tactical Strategy (SST level 3) or higher levels, were included: in other words, those who showed the cognitive capability to go beyond linear thinking in order to deal with interactive effects within tangible systems. A large number of individuals included in the study also showed the capacity to move beyond this tangible systems level of complexity to deal with intangible, fuzzy and dynamic systems.

In addition to the complexity preferences and capabilities of applicants, cognitive stylistic preferences were also used as a selection criterion. This was based on previous CPP research findings, which indicated that 73% of accountants show Logical-Analytical cognitive styles on the CPP.

Of the test candidates that were selected as bursars based on the above mentioned cognitive criteria, 651 of the total sample of 752 individuals passed and 101 of the 752 failed the FQE I exam. In other words, 86.5% of those selected based on their CPP results passed this challenging exam.

In terms of the exam results of students from disadvantaged socio-economic backgrounds in South Africa, this particular firm historically achieved pass rates of 4% on the FQE II exam. After implementing CPP assessment for purposes of selecting bursars from previously disadvantaged backgrounds, their pass rates improved to 64% on the FQE II – which was significantly higher than the national average pass rate of around 50% on the FQE II.

Study 3: CPP and Work Performance appraisals (2006)
(n = 752)

In a study using this same sample of trainee accountants in Assurances, multiple regression analysis was conducted to investigate CPP predictors of general work performance as indicated by the 360 degree evaluations of managers. The results indicate that the work performance of trainee accountants is best predicted by the following CPP dimensions:

• Potential for Level 3 functioning (positive correlation), plus
• Absence of Level 1 functioning (negative correlation), plus
• Absence of the tendency toward Quick Closure (negative correlation).

Study 4: CPP and Work Performance appraisals (2005)
(n = 40)

The purpose of the study was to identify guidelines for future selection and training initiatives in the accounting environment. CPP results were compared to the performance appraisal scores of trainee accountants. These performance scores were allocated by their mentors in terms of the following competencies and/or criteria: Business and financial acumen; Motivation; Interpersonal skills; Work performance; Academic achievement; Logical-analytical skills; Judgement capability; Self-improvement; and Learning.

Particular CPP scores differentially correlated with these dimensions and results of r = .4 to r = .61 were obtained. These correlations were mostly significant at a p < 0.01 level.

Study 5: CPP and differences between performance groups (2006)
(n = 69)

Using ANOVAs, the degree to which CPP results differentiated between competency-based performance groups was analysed. The competency dimensions were evaluated by the employer on a 5-point rating scale. Although many CPP dimensions correlated with the competency ratings, the only ones that differentiated between performance groups at a p < 0.01 significance level were Structuring and Metacognitive processes.

In other words, differences in the way in which performance groups understood, ordered and represented information, thus appear to significantly affect their performance within the accounting environment.

Study 6: CPP and job-related performance and competence in Tax (2005)
(n = 39)

A small qualitative and quantitative research study, in which the CPP results were correlated with job performance criteria, was undertaken in the Tax (n = 39) division in a big four accounting firm. The goal was to identify core cognitive dimensions that significantly predict job performance within the Tax environment in this organisation.

Correlations between CPP processing dimensions and General Job Performance indicated that job-related functioning seems to covary with the depth and effectiveness of exploration processes; the structuring and categorisation of information; and the integration and meaningful interpretation of fragmented information. An interpretation of the results indicated:

• The somewhat operational nature of the currently applied general performance rating, with an emphasis on tangible information and practical implications. Significant correlations between performance ratings and CPP results, namely exploration (r = .454; p < 0.015) and categorisation of tangible information (r = 0.41; p < 0.027) in particular, were found.
• A high correlation of 0.881 (p < 0.000) was found between the overall competency rating and the Potential Level of Work indication of the CPP
• Specific competency dimensions significantly correlated with particular CPP dimensions. The results showed correlations between r = 0.393 and r = 0.431, which met significance levels of p < 0.038 to p < 0.022 and indicated:
  • a strong element of cognitive functioning involved with Business Development competence
  • a verbal skills aspect involved with the People Development competency
  • the apparent non-cognitive nature of the Client Relations competency.
  

In an independent cross-cultural study of the CPP profiles, the academic performance and the work performance of 752 candidates from diverse backgrounds, ANOVA results of group differences show no significant differences between race and gender groups in terms of their potential for cognitive functioning as measured by the CPP.

Conclusion

The CPP thus seems to effectively predict the academic, work performance and competence of trainee and practicing accountants. No adverse impact was found. The CPP can therefore be used to prevent failure and manage risks involved with the selection of accounting bursary students and staff.