Fairness in Testing: How to Ensure that Testing is Fair: responsibilities of the test usersContact Us
The constructs measured by the test must relate to the requirements of the role.
A thorough job analysis is a vital precursor to deciding which assessment techniques are appropriate. Before deciding whether to use tests at all, let alone which test to use, it is crucial to understand what attributes are required for effective job performance and the required level of the specified attributes. Once you have a clear specification of required attributes and the level at which they need to be, you can start to consider what assessment methods might be suitable. A psychometric test will not always be the most appropriate method. However if psychometric testing seems the best way of measuring one or more of the identified requirements, then your next responsibility is to choose tests that are appropriate. For each test considered you need to ask whether the constructs it measures match the requirements of the role; not just in terms of type of construct but in terms of difficulty level as well.
Note that the Law requires that the employer must be able to show that the tests used correspond to necessary role requirements.
Ensure that the chosen test is psychometrically robust.
Identifying a test that has the right content and difficulty level is just the first step in ensuring that the test will be fair. You next need to satisfy yourself that the test is psychometrically robust. You should consider the following:
How was the test developed? For which target group was it designed? How was it standardised? When was it last updated?
If the test is norm-referenced - are the available norm groups suitable for your purpose? What is the composition of the norm group you wish to use - does it represent the population it claims to represent (with respect to age, race, gender, occupation, educational level, social background etc.)? In selection, the norm group should be as representative of the applicant pool as possible.
What information is provided about the reliability and validity of the test? What is the standard error of measurement and how will you take account of that in your interpretation of the results?
What information is given about bias with respect to gender, ethnic origin etc.?
Pay attention to the test-takers welfare.
This means making sure that candidates are prepared for the use of tests and that they understand how and why the tests are being used.
An initial letter should be sent out to candidates explaining the role of the tests in the assessment process. The letter should also ask candidates to let the organisation know of any special requirements they may have and it should invite them to raise any concerns about testing with a named contact beforehand.
The administration of the test should not be rushed. It is important to establish rapport with candidates. They should be encouraged to ask questions and they should not feel embarrassed about taking too long over the example questions. Evidence suggests that rushing the administration process is proportionally more disadvantageous to people from ethnic minorities, people with disabilities and people who are less used to the job market.
Special care should be taken with people whose first language is not English, both in terms of ensuring they have understood the administration instructions and also in terms of the level of English required by the test. Unfairness is likely to arise where tests require reading skills that are not an integral part of the job.
Monitor test use for adverse impact.
It is important that test users continually monitor the impact of test use whether or not group differences have been identified previously. The following should be considered:
Job roles change over time so that a test that has always been used, while once relevant may have ceased to be so.
Test users should keep abreast of updates to the test and change to updated versions and more recent norm groups as they become available.
The relationship between performance on the test and job performance should be monitored.
Written by Wendy Lord: Chief Psychologist, Hogrefe-Cranbrook Learning Centre.
Fairness in Testing - Quick Links
