Many clients who have used the test suite noted the Bookkeeper Test did not have a norm group, whilst the other tests all have one.
The Bookkeeper Test (“BKT”) assesses the candidate against a set of competencies. Assessing a candidate against a ‘bell curve’ provides a very quick snapshot of their ability compared to their peers, whereas comparisons to competencies requires more careful analysis. We’ve responded to your feedback and added a norm group to the BKT. However there were good reasons for not including the bell curve initially, and why the BKT norm group should be used with caution.
The original reason for not including the norm group in the BKT was that whilst accountants (especially those with memberships of a professional body) have a well understood competency expectation, bookkeepers come in all shapes and sizes, from those employed full time on accounts work or working across a number of smaller businesses, to those where producing monthly accounts from cloud software is a small part of a wider role. The level of expectation of the employer can also vary widely. Because of this we used a Expert Group of accountants to grade the BKT question set to reach a consensus view on the likely ability of a candidate to achieve in a role, based on their results in the Basic, Intermediate and Advanced questions. For example the Expert Group considered that a strong (>75%) showing in the Basic questions meant a bookkeeper could accurately process information through a cloud accounting package. An ability to manipulate or interrogate that information required high scores in the Intermediate and Advanced questions – to show greater a level of competency.
The procedure we used to develop the competencies is called a ‘Delphi’ qualitative analysis, and is a statistically recognised process. On the other hand the bell curve is relatively blunt instrument in these circumstances. If the role in question does not require a bookkeeper to reach Competency Level 2 (based on our Delphi process) then only strong performance at Competency Level 1 is required – whereas the norm group comparison is likely to see a suitable candidate score quite poorly in the overall test. Users of the report need to then take the time to critically consider both the level of competency required and that achieved to make sure the candidate is considered appropriately. Although the same argument can be made in relation to the Accounting Tests, in our view, the disparity is much wider for bookkeepers.
We’re always happy to discuss with clients which test is likely to be more appropriate for a particular position, and to assist with test interpretation.