Legal risk and skills-based hiring: The legal defensibility of TestGorilla’s assessments

Science series materials are brought to you by TestGorilla’s Assessment Team: A group of IO psychology, data science, psychometric experts and assessment development specialists with a deep understanding of the science behind skills-based hiring.

In order to maintain legal compliance, avoid potential legal challenges, and participate ethically and responsibly in assessment practices, organizations must ensure that their assessment practices are legally defensible. 

If assessment practices are compliant with legal standards and influential guidelines for test use in personnel selection, then they are legally defensible. It’s hard to give a global definition of legal defensibility, since these rules depend on where you’re located, so you should consult a lawyer in your location if you’re unclear about what these rules are.

In this article, we’ll walk you through what legal defensibility means in the context of pre-employment testing and elaborate on some of the things TestGorilla is doing to ensure the legal defensibility of its tests.

Why should I care about legal defensibility?

Hiring always carries some kind of legal risk with it. This might sound negative, but it’s true. 

When hiring, you navigate all sorts of overlapping regulations and laws, and you’re often dealing with candidates who are strangers to you. From job posting wordage, to compensation and classification, to the depth of your due diligence vetting a candidate’s background information, all of your actions carry legal risk that can affect your organization’s bottom line. 

Legal defensibility applies to any and every hiring strategy

Employment laws and risks associated with violating them apply no matter how you hire. Whether you rely on science-based skills assessments or the gut feeling of your hiring manager, you still need to ensure the legal defensibility of your hiring strategy. At TestGorilla, we think that data-driven, skills-based hiring is the best way to hire, so we’ll be discussing legal defensibility in this context. But even if you don’t use pre-employment tests like ours, your decisions still carry inherent legal risk.

This being said, hiring is a necessary risk that’s always worth it. Companies can’t grow and do great things without great people, and you don’t find great people if you don’t hire. The key is to manage this risk effectively by using a hiring process that’s legally defensible and helps you identify top talent. 

What we mean by legal defensibility

When we talk about legally defensible tests at TestGorilla, we’re talking about tests that hiring managers should feel confident using to hire the best candidates. But to understand what it means for a test to be legally defensible, we must first acknowledge that it’s not just about the test itself – it’s about how that test is used. 

As an example: the same test can be low risk (and quite legally defensible) when used to hire for one job role; when used to hire for a different role, it might be high risk (and less legally defensible).

To put this into context even more, think about physical agility tests (PATs) that might require test takers to run a certain distance under a certain time, or be able to do a number of push-ups. The legal defensibility of a test like this depends on how it’s used. If you’re trying to hire a football player, great; this could be an effective and defensible test to use. But if you’re trying to hire a graphic designer? Maybe not so much. The use of a PAT in this instance with these facts doesn’t seem very defendable at all.

This example explains another important point, and another reason you should care about the legal defensibility of the tests you use to hire: If your test isn’t legally defensible, it’s probably not going to help you identify the best candidates for the job, either. Beyond the significant legal risk of non-defensible tests, they also tend to not be effective at identifying the best candidates for your job.

The key components of legal defensibility explained

Now that we’ve covered what legal defensibility is, let’s explore some of its core parts. The factors listed below are key components of legal defensibility of pre-employment tests in many jurisdictions across the world, including the US. However, the specifics and application of each of these factors may vary based on the laws and regulations in each specific country, region, state and/or local municipality.

Non-discriminatory

Legal discrimination refers to differential treatment of individuals based on certain protected classes or characteristics (race, gender, age, sexuality and disability status are a few examples, but many places have different categories). When people are treated unfairly or unfavorably because of their protected class or characteristics, illegal discrimination is often occuring.

Disparate treatment and disparate impact

For tests, we often think about discrimination in terms of disparate treatment and disparate impact. 

• Disparate treatment refers to people being treated differently because of their protected class or characteristics. This treatment is often intentional, where individuals are singled out and treated differently because of who they are. 

• Disparate impact (sometimes called adverse impact) refers to when a seemingly neutral employment policy or practice disproportionately affects individuals from a protected group. This sometimes is viewed as the opposite to intentional discrimination, as it typically results from policies or procedures that may appear neutral on the surface but have a discriminatory effect in practice. 

Intentional and unintentional discrimination can take place during pre-employment testing. We must consider both of these factors to monitor and minimize any detrimental impact associated with pre-employment tests.

US federal laws

In terms of discrimination in employment, there are many laws about this in the US, including from the federal government, state governments, and local or regional governments. Three of the most important US federal laws about discrimination and employment are:

• Title VII of the Civil Rights Act of 1964: This federal law prohibits employment discrimination based on race, color, religion, sex, and national origin for certain employers. 

• Title I of the Americans with Disabilities Act: This federal law, also known as “ADA”, prohibits employment discrimination against qualified persons with disabilities 

• The Age Discrimination in Employment Act: Also referred to by its acronym “ADEA”, this law prohibits employment discrimination against persons over age 40.

For federal laws, the Equal Employment Opportunity Commission (EEOC) leads the enforcement of these non-discriminatory laws in the US. They also help both employers and employees understand non-discrimination laws in employment. 

Job-relatedness

For a test to be legally defensible, it should also be sufficiently related to the job that you’re using it to hire for. The example we referenced previously demonstrates this well: If you’re using a PAT to hire your next graphic designer, then the test wouldn’t seem to relate to the job that your successful candidate will be doing. Because it lacks job-relatedness (i.e., it’s not based on the inherent requirements of the job), it’s not legally defensible to use a PAT test in this way. 

Unfortunately, things can get a bit complicated and interconnected here. Sometimes, job-relatedness can be used as a justification for using a legally discriminatory test: for example, if a job requires the ability to speak English, employers could use an English language test to discriminate against non-native English speakers. 

In this case, even though the test legally discriminates against a common protected class, its use seems legally defensible because of its job-relatedness. Because of this interconnection, it’s important to emphasize that people using a test for hiring, as well as those developing them, have a role in ensuring legal defensibility. 

Job analysis

Job analysis is a systematic process used to collect and analyze information about a job. You can learn more about this process in our guide to job tasks analysis, but at its core, this process is about taking a close look at a job and determining the skills necessary for that job. This might seem straightforward for some positions, but it can get very complex very quickly.

Consider a job role like “Administrative Assistant” or “Project Manager.” You might have a very clear idea of what skills these jobs require but that understanding may be quite different from someone on the other side of the world or even your company’s competitor across the street. There are helpful tools to help categorize and make these skills uniform, like the U.S. Department of Labor’s O*NET or the European Union’s ESCO, both of which TestGorilla use to help develop our role-specific tests. 

The goal of job analysis is to create an accurate and comprehensive understanding of the job's essential functions and requirements. Engaging in job analysis is also very important to minimize legal risk and increase the legal defensibility of your hiring. 

In short, conducting a job analysis is a vital step towards ensuring job-relatedness. If you understand the job you’re hiring for, you’re well equipped to choose the right tests to use in your hiring process.

How TestGorilla ensures the legal defensibility of tests

As the developers of tests, there are things TestGorilla can do to ensure they are legally defensible when used properly. 

We’ll walk you through what these things are, but first we want to emphasize that the legal defensibility of our tests can still be compromised if an employer doesn’t use them properly. The same goes for medicine – a certain drug might be safe to use for cancer treatment, but if someone starts using it for something else, and then gets sick, then it’s not very reasonable to hold the doctor accountable for its misuse.

Industry best practices

Overall, TestGorilla, follows industry best practices and relevant laws and regulations to maximize the legal defensibility of its tests. These include:

• The Uniform Guidelines on Employee Selection Procedures, developed by several federal agencies in the United States, including the US Equal Employment Opportunity Commission (EEOC).

• The Standards of Educational and Psychological Testing, developed by the American Educational Research Association (AERA), the American Psychological Association (APA) and the National Council on Measurement in Education (NCME).

• The Principles for the Validation and Use of Personnel Selection Procedures developed by The Society for Industrial and Organizational Psychology (SIOP).

These guidelines aren’t just about pre-employment tests, as they cover all methods of selection, but they serve as the backbone for our test development practices at TestGorilla. Although these are not laws themselves, they do provide helpful frameworks by agencies and industry groups to ensure that selection practices are fair and legal.

TestGorilla also understands that the scope of regulatory frameworks and best practices change as the world of work changes. Our assessment experts monitor developments in academic research, governmental regulation and industry standards to ensure that our processes maintain alignment. 

Test validity

Next up, test validity. This is connected to job-relatedness: if a test is to be valid, it must be related to the job skill it promises to assess – i.e., we have to be sure that the test is measuring what we expect it to measure. In other words, we have to carry out a test validation process.

“Validity is a key focus of test validation. It ensures that our tests measure what they are intended to measure. By validating our assessments, we ensure that they provide accurate and meaningful information about a candidate’s skills, abilities, and preferences.” 

– Lindie van der Westhuizen, Assessment scientist and IO psychologist

There are a few different kinds of validity, which you can learn more about in our brief introduction to test validation. In the context of this article, it’s important to know that we continually engage in data collection and test validation efforts to verify the quality of our tests (more on how we use this to continuously improve test quality later). 

Although test validity is something we are focussed on, we once again want to emphasize that our customers are the experts when it comes to the jobs they’re hiring for. We’re here to provide them with and help them select the best test, but, ultimately, only the employer can truly ensure job-relatedness.

Psychometric analysis and monitoring

In addition to our test validation efforts, TestGorilla uses demographic data to consistently analyze and monitor test results and trends. We analyze and monitor these data and adjust tests or items at all stages of the test development and validation process as needed. 

This type of analysis is vital to ensure that our tests perform how we want them to perform, ensuring a fair and compliant service.

Control of testing experience

Although it’s impossible to completely control the experience of a candidate on our platform, we strive to provide as uniform an experience as reasonable. Candidates taking an assessment for the same job on our platform will experience an identical candidate flow, including the registration page, the provision of instructions, the order of tests, and the test content itself. 

Our platform scores candidates’ tests in the same manner for every candidate, and provides results that are the same for each candidate, except for the differences based on their testing ability.

We can’t control the kind of device or keyboard a candidate uses (or if they had a good night’s sleep before their assessment), but we do provide a testing platform accessible to different kinds of devices with different levels of internet connectivity as well as test content that considers the varying kinds of culture and language used by diverse candidates from around the world.

Continuous test improvement

Finally, TestGorilla carries out continuous test improvement in order to maintain the legal defensibility of our tests. The parameters for the legal defensibility can shift and change – because, as we know, the skills needed for a particular job change as the world changes. Just because a test is valid when it’s created, doesn’t mean it will still be valid a few years down the line. 

Take, for example, programming languages and software applications. Most of us use some kind of software to do our jobs, and these are being changed and updated all the time. In fact, according to research from Lightcast, 37% of skills changed in the average job between 2017 and 2022. 

Continuous test improvement is part of our periodic and standard improvement process for tests. By ensuring that tests remain valid as job and skill requirements change, we’re making sure that, for the employers who use them properly, TestGorilla tests maintain their integrity and legal defensibility. For more information about our ongoing validation study initiative, and details about how to participate, you can read this blog.

Want to learn more about the science behind our tests?

Legal defensibility can be an intimidating concept to grapple with. But if you’re using skills tests intelligently and with integrity, and you’re sourcing those tests from a platform that, like TestGorilla, is working hard to ensure its tests are valid, up-to-date, and non-discriminatory, then you are setting yourself up for success. 

All of our tests are shepherded through their production by assessment development specialists with a deep understanding of the science behind skills-based hiring. Our Assessment team also includes a group of IO psychology, data science, and psychometric experts. To learn more about the science behind TestGorilla, check out these blogs from our science series: 

• A brief introduction to test validation

• How to prepare for a criterion-related validity study

• A brief introduction to candidate satisfaction

• A brief introduction to test order

• A brief introduction to behavioral competencies

Disclaimer: The information contained in this site is provided for informational purposes only and should not be construed as legal advice on any subject matter. If you have any questions on the legal defensibility of tests in your area, please consult with a lawyer. Please also familiarize yourself with the laws and regulations in your specific country and region regarding the use of pre-employment tests in selection.