A brief introduction to: Test validation

Science series materials are brought to you by TestGorilla’s Assessment Team: A group of IO psychology, data, and psychometric experts with a deep understanding of the science behind skills-based hiring.

At TestGorilla, we’re always working to improve the accuracy and reliability of our pre-hiring assessments to ensure a fair and bias-free screening experience for both candidates and employers. In this science series article, we’ll take a look at why test validation plays such a crucial role in our assessment creation process and how the team is working to improve this aspect of our tests to support user objectives. 

Let’s explore why test validation is important and how it benefits both organizations and individuals.

What is test validation?

Test validation is the process of gathering evidence to verify that the intended use, interpretations, and results of a test are accurate. It involves careful test design and development, data collection, analysis, and interpretation to ensure the validity and reliability of assessments.

Validity is a key focus of test validation. It ensures that our tests measure what they are intended to measure. By validating our assessments, we ensure that they provide accurate and meaningful information about a candidate’s skills, abilities, and preferences. Reliability is equally important. It refers to the consistency and stability of test scores over time. 

What are the different types of validity?

At TestGorilla, we consider four types of validity when developing our tests: content validity, construct validity, face validity, and criterion validity.

Content validity 

This is the extent to which a test covers a representative sample of the skills and knowledge content relevant to the topic in question. 

TestGorilla’s subject matter experts (SMEs) and assessment development specialists use a standardized test development process and formal test structures to ensure the skills and knowledge necessary for a particular topic are well-represented by the test and the test items, while irrelevant topics are discarded. 

Construct validity 

This is the extent to which the test accurately measures the specific construct that the test is intended to measure. The systematic and structured manner in which we develop our tests forms a key part of its construct validity. 

We also gather statistical evidence to support the construct validity of our tests, including evidence of convergent validity (the extent to which a test correlates with tests that measure similar/related constructs), discriminant validity (the extent to which a test correlates with tests that measure dissimilar/unrelated constructs), and factorial validity (the underlying factor structure of a test).

Face validity 

This type of validity focuses on whether a test appears to measure what it is intended to measure and whether, on the surface, the test feels relevant and appropriate for what it is supposed to be assessing. 

We evaluate this at the conclusion of each test and, again at the end of the assessment. At both stages, candidates are invited to provide real-time quantitative and qualitative feedback on their test experience and are surveyed about the perceived validity and relevance of the test items they have just completed, as well as their overall experience and satisfaction in using our tests. 

Criterion-related validity 

This is the degree to which test results align with some external outcome measure (e.g., job performance, turnover, organizational commitment). TestGorilla has launched several activities that aim to establish the degree to which candidates who perform well on our tests are successful in the roles for which they were assessed and hired. 

How does TestGorilla validate its tests?

TestGorilla adopts a rigorous multistage test development and review process, utilizing multiple independent subject matter experts and test development specialists to ensure reliable, valid, and fair measures. 

The process is managed by our Assessment team, who are responsible for designing tests that measure specific skills, knowledge, or competencies using proven test development theories and methods. The team works collaboratively with experienced subject matter experts to finalize questions for each test, which are then peer-reviewed. 

Next, we collect data from a diverse sample of individuals who take the test. Our team of organizational psychologists, psychometricians, and data scientists conduct ongoing statistical analyses to examine the quality of our tests and evaluate group differences, as well as the relationship between test scores and job-relevant work outcomes (e.g., self and managerial ratings of performance; turnover; sales volume figures; productivity metrics).

We continually engage in data collection and test validation efforts to verify the quality of our tests. Moreover, we use the gathered validation evidence to improve the quality of our tests, the reporting of results, and the user experience. 

How does test validation benefit hiring managers and candidates?

By using valid and reliable tests, our customers and candidates gain numerous advantages:

1. Improving hiring decisions

By using validated tests, clients can make more accurate and reliable hiring decisions. Validated assessments provide objective and standardized information about candidates’ skills, abilities, and potential, helping hiring managers identify the best fit for a role.

2. Fair and unbiased evaluation

Validated tests improve fairness and minimize bias in the selection process. They provide an equal opportunity for all candidates to demonstrate their capabilities, regardless of personal characteristics or backgrounds. This promotes diversity, inclusivity, and equal opportunities for candidates.

3. Enhanced predictive power

Validated tests have proven predictive power, meaning they can effectively predict candidates’ job performance and future success in a role. Backed by this data, hiring managers can have greater confidence in their selection decisions, knowing that the assessments are backed by evidence of their validity.

4. Time and cost savings

Test validation enables hiring managers to streamline their selection processes. By using validated assessments, they can efficiently evaluate a large pool of candidates and identify the most qualified individuals. This saves time and resources compared to traditional screening methods, such as unstructured interviews or resumes.

5. Increased candidate satisfaction

Validated tests make the evaluation process transparent and objective. Candidates appreciate the fairness and professionalism of assessments that are grounded in scientific rigor. This enhances their overall experience and perception of the selection process, leading to increased candidate satisfaction.

According to our internal data, a majority of candidates who took our assessments as part of a pre-employment screening were satisfied with the process, especially when a multi-measurement approach to testing was implemented. 

6. Personal development insights

Validated assessments offer valuable insights for candidates, helping them understand their strengths, areas for improvement, and potential career paths. This self-awareness can guide their personal development efforts and career planning.

7. Legal defensibility of test

Validation evidence helps establish a strong foundation for the defensibility of tests and helps to ensure they are compliant with legal standards and influential guidelines (e.g., Uniform Guidelines on Employee Selection Procedures, Equal Employment Opportunity Commission standards) on test use in personnel selection. Having validity evidence on tests enables employers to demonstrate that the assessments measure what they intend to measure and are predictive of job performance. 

Adhering to such standards helps organizations maintain legal compliance, avoid potential legal challenges, and promotes ethical and responsible assessment practices.

We’re on a mission to make hiring better. Join us by signing up for our new validation study initiative.

As part of our ongoing efforts to improve the accuracy, objectivity, and fairness of our tests, we’re conducting a series of criterion-related validation studies to understand the impact of TestGorilla’s assessments better on work-related outcomes. 

We have different types of initiatives that organizations can participate in – whether you have a large (more than 100) or a small number (less than 100) of hires. You only need to have tested and hired candidates using TestGorilla assessments and have (or be willing to gather) performance data on these hires. 

Organizations that sign up for the initiative and meet the qualifying criteria for large validation studies will receive a scientific report documenting the results of the research along with other observations and recommendations.  To sign up, simply fill out the form below.

With your help, we believe we can take our test development and optimization processes to the next level.

If you’re interested in being a part of our validation study initiative, fill out this form.

Learn more about the research that drives TestGorilla

Test validation is just one of the areas we’re investigating and improving upon. To read more about the research that’s helping us take our assessments forward, check out these blog posts about the science behind TestGorilla:

• A brief introduction to candidate satisfaction

• A brief introduction to test order

• A brief introduction to behavioral competencies