What are the psychometric properties of pre-employment tests?

Hiring the right candidate can take a lot of time: between 33 and 49 days on average, depending on the industry.[1] 

This can quickly turn into a problem. According to the recruitment agency Robert Half, 62% of job seekers lose interest in a job if they don’t hear back from the recruiter within two weeks.[2] High-quality candidates are usually less likely to wait for a job because they have many good opportunities. 

So, how do you hire faster without overlooking important details?

You could work on optimizing the resume-screening process. There is no 100% guarantee that you’ll always make a great hire, but there are always things you can do better. 

An even better option is to replace resumes and cover letters with skills-based hiring, which can be a game changer for your company. Using pre-employment tests to screen your candidates will help you make unbiased and objective hiring decisions.

And to make sure the skills assessments you implement are effective, you need to understand their psychometric properties.

In this article, we’ll help you learn more about pre-employment tests and their psychometric properties.

Let’s dive in.

What are the psychometric properties of pre-employment tests?

Do pre-employment tests actually work? How can you make sure they don’t produce false data that will only confuse you? Do they effectively measure the properties they claim to assess? How can we trust them?

To answer these questions, you need to look into the test’s psychometric properties.

Psychometric properties refer to the technical qualities of a test. When a certain aspect of an assessment meets scientific standards, we can consider it a psychometric property. 

Simply, psychometric properties are the characteristics of tests and other types of research instruments. They reveal information about the critical aspects of a test. There is no definitive list of them, but some are necessary for any type of assessment, such as validity and reliability.

A psychometric property of an assessment focuses on a particular characteristic. Some psychometric properties can be used to evaluate the test’s overall quality, whereas others focus on particular sections or individual items.

Psychometric properties are usually expressed quantitatively by a coefficient or an index. However, this is not the case for every psychometric property.

For example, the two essential psychometric properties, reliability and validity, are represented differently. The reliability coefficient is represented by a numerical value, whereas validity is too complex to be reduced to a single quantifiable value. 

Measuring the psychometric properties of tests is a complicated process, but it provides valuable insights. There are two reasons why you should learn about an assessment’s psychometric properties:

• It’s necessary for sound test construction: An assessment’s designers must be able to evaluate and describe its features. Similarly, when selecting a skills-testing platform, you need to be able to decide whether its assessments meet your criteria. 

• It helps you assess whether the data provided by the test can serve as a sound basis for decision-making: All kinds of professionals, from psychologists to marketing agents to educators, use the information provided by assessments in their decision-making processes. Research shows that 56% of employers use pre-employment tests to evaluate candidates’ skills objectively. The evidence to support these decisions can be found in a test’s psychometric properties.

Learning how well a pre-employment test functions can help you decide whether you can rely on the data it provides in your hiring process. To start exploring the psychometric properties of an assessment, you can divide them into two broad categories:

1. Reliability

2. Validity

Here, we’ll cover both types of psychometric properties in pre-employment tests.

Pre-employment tests’ reliability

Reliability is an essential component of every pre-employment test. A reliable test score stays consistent and accurate across multiple evaluations and test editions and after different raters grade the participants’ results.

Can the result be recreated multiple times? Does it produce similar results under set conditions? This means the test likely has high reliability.

If a pre-employment test produces stable results over time, after repeated administration, or under similar circumstances, it is trustworthy and reliable. 

There are many ways to evaluate a test’s psychometric properties to determine how reliable it is. Some include administering the same assessment to the same participants at different times or presenting them with variations of the same test to see if their results are consistent. Only pre-employment tests with proven reliability qualify for validity testing. 

There are four types of psychometric reliability for pre-employment tests:

1. Internal consistency reliability: Various items in the test are evaluated to see if they measure what the assessment is supposed to measure. Internal reliability between separate items is known as internal consistency.

2. Parallel forms reliability: Different tests that use the same content but with different procedures or equipment are administered to find out whether they will yield the same results for each participant.

3. Inter-rater reliability: Different examiners assess and score the pre-employment test in the same way to prove inter-rater consistency. The greater the extent to which they agree, the higher the inter-rater reliability. 

4. Test-retest reliability: The participant takes the same test repeatedly over time. The assessment is reliable if the participant’s scores are consistent.

Using pre-employment tests that have been proven to have high psychometric reliability is essential for an effective skills-based hiring process. Psychometric tests can be as reliable as medical tests if they are built correctly and the science behind them is sound. 

There can be minor discrepancies in a test’s reliability due to changes in the participants’ minds, feelings, current state, or circumstances, but this is to be expected for any assessment. Test-takers might suffer from social desirability issues and respond with what they believe is the appropriate answer, even if it’s not authentic.

Some other issues that can negatively impact the reliability of a pre-employment test are:

• The questions: The level of difficulty or ambiguity of the questions can negatively affect the assessment’s reliability. Therefore, it’s crucial to evaluate the questions for errors and biases.

• The administration: To avoid systematic errors that lower the pre-employment test’s reliability, the instructions accompanying the administration should be clear and well-defined. Any issues in the guidelines provided for the participants or the administrators can cause discrepancies in psychometric reliability.

• The environment: An uncomfortable environment, for example, one with audible or visual distractions, can influence the psychometric reliability score.

Pre-employment tests can be highly reliable and serve as a sound basis for decision-making as long as they are constructed and maintained by professionals who know the importance of psychometric properties.

Pre-employment tests’ validity

Psychometric validity is the property measuring an assessment’s capacity to evaluate the traits it claims to evaluate. Validity is measured by multiple data points and insights that focus on the relation between the personality traits that the test measures and the test’s construction.

Psychometric validity is a crucial element of any pre-employment test. If a test has high validity, its items remain firmly connected to the assessment’s core intended purpose.

If a pre-employment test scores high in reliability, its results will be consistent. But even if they are consistent, if the test doesn’t score high in validity, the results do not serve their purpose as a measure of the intended traits.

Evaluating the psychometric validity of pre-employment tests is a complex process. Psychometric properties cover a wide array of items and functions. Just like with reliability, there are four types of psychometric validity:

1. Construct validity: How well does the pre-employment test measure the construct it was created to measure?

2. Content validity: Is the content appropriate, and does it evaluate all aspects of the particular construct it is measuring?

3. Face validity: Does the pre-employment assessment appear as if it measures what it is intended to measure?

4. Criterion validity: Do the assessment scores correspond to a benchmark test?

How are pre-employment tests created? 

So, how are pre-employment tests created? Assessments must be developed by professional teams and supported by solid science to score high in tests of their psychometric properties.

At TestGorilla, we work with subject-matter experts to construct large question banks for each test and then assign them to skill areas. Question banks contain hundreds of questions to guarantee that candidates won’t see the same questions again if they retake the test. 

The algorithm that selects the questions for the pre-employment tests takes multiple factors into account:

• The skill area of each test: This is done to ensure each area receives equal attention.

• Whether a candidate has taken the same test before: The algorithm ensures that applicants don’t receive the same questions as when they previously took the test.

• The maximum exposure count: Each question has a limited lifetime – once it has been used a certain amount of times, it expires. This is done to protect the pre-employment test’s integrity and validity.

What’s the science behind pre-employment skills tests and their psychometric properties?

Pre-employment tests are an objective and standardized tool for gathering and evaluating data that can help you with the hiring process. Their purpose is to eliminate human bias from the screening process and provide you with the information to make data-driven decisions.

To achieve this effectively, TestGorilla uses a scientific, multilayered approach to develop its pre-employment tests. Here’s what we do: 

1. Build a framework: The psychometrics team follows well-established principles of test theory to construct the test’s framework.

2. Develop the questions: Subject-matter experts collaborate with our team to develop the questions for the test.

3. Peer review: Experts peer review the questions to ensure their accuracy.

4. Confirm consistency and fine-tune the test: We use advanced statistical algorithms to track down and analyze key indicators in our tests so that we can improve them. Measures such as Cronbach’s alpha help us confirm the internal consistency of each assessment.

5. Verify the test’s validity: We use the Uniform Guidelines on Employee Selection Procedures’ framework of employee selection and the U.S. Department of Labor’s skills analyses to verify the test’s validity.

6. Collect feedback: We also collect feedback from our customers about the performance of each candidate they hire. In this way, we can give data-driven recommendations about which tests to use for particular job roles.

Another aim of pre-employment tests is to eliminate unconscious bias from the hiring process.

For this reason, we’ve built our platform based on the standards of the Equal Employment Opportunity Commission to guarantee a fair hiring process. We also use differential item functioning studies on data collected from candidates to remove any adverse impact from the hiring process in advance.

Hire better with pre-employment skills tests

Better, faster, more reliable, unbiased, and data-driven – this is what your hiring process will look like when you start incorporating pre-employment skills tests. 

Our tests’ psychometric properties ensure they consistently measure what they are intended to and can provide a solid foundation for important hiring decisions.

Take a look at TestGorilla’s test library to find tests that will make your hiring process fairer and more objective. You can get started with a free plan today.

Sources

• Anders, George. (August 4, 2021). “Can you wait 49 days? Why getting hired takes so long in engineering”. LinkedIn. Retrieved January 9, 2023. https://www.linkedin.com/pulse/can-you-wait-49-days-why-getting-hired-takes-so-long-george-anders/

• “The Biggest Mistake You’re Making When Hiring”. (March 5, 2021). Robert Half. Retrieved January 9, 2023. https://www.roberthalf.com/blog/evaluating-job-candidates/the-biggest-mistake-youre-making-when-hiring