Assessment & Selection: Reflections on the use of different selection tools

This article is a reflective piece by Anthony Spencer on the use of different selection methods as used in day to day business psychology consultancy practice. Anthony touches upon various methods he has studied whilst doing a part-time MSc in Organisational Psychology at Alliance Manchester Business School as well as the implementation of such practices where appropriate in actual consultancy at Impact Consulting Psychologists.

My Reflections

Working in a voluntary capacity at Impact Consulting Psychologists whilst studying my MSc in Organisational Psychology has given me a practical insight into the ways in which selection and assessment approaches are implemented in both private and public business sectors. In particular, I have seen first-hand the preparation that goes into realising client-tailored assessment centres for senior leadership positions.

There is clear synergy between my MSc course and the practices that are carried out in the consultancy world in relation to senior level assessment centres. For example, within my academic studies I have learnt about the effectiveness of different selection methods and their respective ability to identify talent. These range from less effective methods such as unstructured interviews to more effective methods such as highly structured interviews, situational judgement tests, work samples and cognitive ability tests. Definitions of these selection methods are detailed in the following sections, it should be noted that the above selection tools are by no means exhaustive.

Highly structured interviews are defined by the use of pre-set and defined questions (which can sometimes be supplemented with the option for pre-determined probing). This type of interview contrasts with that of unstructured interviews where there are no pre-set questions (in a sense more of a casual conversation) and thus interviews can drift off onto tangents. The disadvantage of an unstructured approach is, if for example the interviewer and interviewee share similar hobbies or interests, the interview may diverge onto topics that may help to establish rapport in the client-candidate relationship but nevertheless and importantly, may not actually assess the skills, attributes and experience necessary for optimal job performance. This is an important point considering the body of research suggesting that people are generally attracted to those similar to themselves (Similarity-Attraction Paradigm), this illustrates how the use of unstructured interviews can become an issue for fairness and be impacted by employment law.

Structured interviews can help to overcome other types of bias such as the Confirmation Bias. Confirmation Bias is where for example, an interviewer may already have an idea of the suitability of the candidate prior to interview through access to pre-interview sources such as the application form, CV, LinkedIn profile etc. An example of this would be when an applicant has information on their CV that the interviewer resultantly forms an impression upon.  By knowing this information prior to interview, it may be possible that the interviewer overlooks key cues during the interview which are contrary to the initial perceptions and instead focuses on any information that confirms the existing beliefs already held about the interviewee. For a full review of the range of bias that can occur in interviews the reader is referred to Posthuma, Morgeson and Campion (2002). Whilst biasing factors can still present in structured interviews the structure, format and rating scales used provide a framework for minimising the influence of any such bias.

For example, by introducing mandatory time limits and deliberate sets of questions, the applicant is given more opportunity to combat against potential Confirmation Bias held by the interviewer. Being that structured interviews are based on ordered questions that are relatively systematic in their sequence, all interviewees are assessed on the same set of questions which results in a more accurate, reliable and importantly fairer approach to selection. Whilst there is ongoing debate regarding the effectiveness of different types of interviews in relation to their ability to predict job performance, Huffcutt, Culbertson and Weyhrauch (2014) show that generally the higher the structure of the interview, the more effective it is in predicting future job performance.

In the work that I observed at Impact Consulting Psychologists I noted the use of Situational Judgement Tests (SJTs) which are questionnaire based versions of situational structured interviews and may be more time-efficient and cost-effective in some circumstances. An SJT gives candidates a series of task-based scenarios to which they respond. An SJT may either elicit responses from candidates/delegates based on their most likely behavioural response(s) or alternatively in the form of a knowledge-response based format where candidates indicate the most appropriate response. In relation to behavioural-response SJTs, this type of SJT can be effective in assessing how someone may act in a certain situation, however responses given by candidates are often based on what they would ideally do and thus this behaviour may not always actually manifest in reality. Nevertheless, SJTs have been shown within the academic literature to be an effective method of selecting for managerial performance, particularly when they measure constructs such as interpersonal skills (Christian, Edwards & Bradley, 2010) and therefore can be a suitable choice for senior leader selection in particular.

SJTs themselves are generally shown to correlate somewhat with Cognitive Ability Tests, depending on the construct measured within the SJT. A Cognitive Ability Test can measure either general or specific intelligence e.g. numerical, verbal or speed and accuracy and thus can provide an overall picture of the cognitive suitability of the candidate. Whilst Cognitive Ability tests can be particularly effective in relation to senior positions because higher scores on the tests relate to higher performance on the job, such tests are generally less favourable in terms of the candidate experience – in other words candidates are less accepting of such tests when compared to other methods such as interviews and SJTs, possibly because the tests have lower face validity (they appear less relevant to the role). This may start to change as new forms of cognitive ability tests in the form of software based games become available on user-friendly platforms such as mobile devices; this assertion is merely speculative at this point however.

The work sample assessments used by Impact Consulting Psychologists comprise a range of exercises including in-trays, role plays and case studies. Work samples are essentially short versions of tasks carried out in the actual role. Work samples are generally favourable to both parties in that they give candidates a realistic job preview and thus allow them to self-select into the process, thus are one of the more preferred methods from the candidate perspective. Moreover, they give the client the opportunity to assess the competencies required by the role in tasks similar to that demonstrated in the actual job.

Favouring the use of research-based methods, Impact Consulting Psychologists are proficient in conducting, analysing and evaluating personality measures used in assessment and development centres. Personality measures are tools which indicate the personality type/traits of a candidate through a self-report questionnaire. One such example is the Occupational Personality Questionnaire (OPQ) which assesses the work-based personality of an individual, in other words the way in which their personality may manifest itself in-role. The OPQ distinguishes itself from other personality measures as it recognises that individuals may behave differently at work as opposed to privately. Employing the Occupational Personality Questionnaire as part of assessment processes is further evidence of the utilization of measures generally supported in the academic literature. Using an individual differences measure such as the OPQ enables Impact Consulting Psychologists to provide accurate candidate-specific insights into potential development areas for candidates, with this being a particularly useful practice for any organisation keen on developing their talent in the long term. The use of the OPQ gives a more objective measure of an individual’s occupational personality compared to more traditional approaches such as relying on the mere intuition of the interviewer. Also, the OPQ enables assessment of personality factors relevant to job-specific performance and the data derived is translated by Impact Consulting Psychologists into feedback that is easily interpreted by the client.

Working at Impact Consulting Psychologists I have gained a thorough understanding of the client-side work that they conduct in order to tailor their consulting to each individual client. By working with clients on their competency frameworks for example, Impact Consulting Psychologists are able to deeply understand the needs of the organisation and select the most effective assessment tools to spot talent. In this way, Impact Consulting Psychologists are able to deliver client-specific and more effective assessment centres, consistently.

This article does not have the scope to cover the entire breadth of what I have learnt since being at Impact Consulting Psychologists but what I hope to have done is highlight some of the key congruencies between the academic realm and real-world consultancy practice.

References

Christian, M. S., Edwards, B. D., & Bradley, J. C. (2010). Situational judgment tests: Constructs assessed and a meta‐analysis of their criterion‐related validities. Personnel Psychology, 63(1), 83-117.

Huffcutt, A. I., Culbertson, S. S., & Weyhrauch, W. S. (2014). Moving Forward Indirectly: Reanalyzing the validity of employment interviews with indirect range restriction methodology. International Journal of Selection and Assessment, 22(3), 297-309.

Posthuma, R. A., Morgeson, F. P., & Campion, M. A. (2002). Beyond employment interview validity: A comprehensive narrative review of recent research and trends over time. Personnel Psychology, 55(1), 1-81.