Here’s a question we’re often asked: “My candidate got the 42nd percentile in your test, is that enough skills for the role I have?”
We then have a discussion about what it means to compare a candidate to a norm group and assess whether the role is likely to be more or less demanding than average, to see if this score is a source of concern or not.
Basing assessment of the candidate on norm group comparison instead of using a competency level our client still knows a lot more about the capability of their candidate than they did before; including areas of strengths and weaknesses as well as an overall assessment of where they sit against their peers. That enables a strong hiring decision to be made.
At Accountests we’ve approached our testing using comparative data by way of a norm group to assess candidates. Although this process is hard to start with, (try getting 100 accountants to sit a test which highlights their ability – not easy!) it means those wanting to test only one or two candidates have something meaningful to compare to.
Wouldn’t it be great to set minimum competency levels, and design a test in which candidates need to achieve a ‘pass’ mark (known as a ‘cut score’) to have their application go forward?
There is recognised science around this using what is known as an ‘Angoff Method’. Under this method between five and twelve Subject Matter Experts are chosen to rate the questions based on a set of criteria that establish minimum competency levels. Once the experts have converged on the appropriate minimum ability requirements this sets the cut score for the test.
This approach is certainly possible for employers of large numbers of accountants at a certain level, or for groups of employers (e.g. local authorities) all of whom have staff at one level. We did use this approach for our Bookkeeper Test, but market reaction was that everyone wanted the ‘norm group’!
If you have interest in helping to develop competency based tests for your employer or industry group let us know!