Understanding test timing: Cognitive load theory and talent assessment

Science series materials are brought to you by TestGorilla’s Assessment Team: A group of IO psychology, data science, psychometric experts and assessment development specialists with a deep understanding of the science behind skills-based hiring.

Have you ever wondered why our Culture Add test is 10 minutes long? Or why our Coding: Data Structures – Arrays test is 35? 

There’s more behind test timing than you might think: This seemingly simple aspect of talent assessment plays a crucial role in maintaining the balance between assessing candidate abilities and ensuring an efficient recruitment process. 

But why is this the case, and how do we maintain a balance? In this science series blog, we’ll explore cognitive load theory (CLT) and its implications for talent assessment in order to help you understand how we think about test timing at TestGorilla.

Cognitive load theory and its implications

The length of a test influences how candidates approach and also directly affects their performance. Too short, and you risk missing out on key insights into a candidate's capabilities; too long, and cognitive fatigue might set in, which could skew results.

To better understand why we choose certain durations for our tests, it’s important to understand the basics of cognitive load theory (CLT). 

Developed by John Sweller in 1988, CLT suggests that our working memory, or the mental space where we process new information, has limited capacity when processing new information. Overloading this capacity – which longer tests may inadvertently do – can lead to decreased performance or even inaccurate assessment outcomes.

Cognitive load

The “cognitive load” of CLT refers to the amount of mental effort used in working memory during problem-solving tasks. It is divided into three types:

1. Intrinsic load, which is inherent to the difficulty of the task itself.

2. Extraneous load, which is related to how information or tasks are presented.

3. Germane load, which is associated with construction and automation of schemas – patterns recognized by our brain helping us simplify complex information.

Understanding cognitive load helps us to design effective tests: It equips us to ensure that we don’t overload candidates' mental capacities, whilst still providing enough challenge for them to demonstrate their abilities effectively. 

Avoiding cognitive overload when designing tests

In order to avoid cognitive overload, we aim to do the following when we design tests: 

1. Optimize the intrinsic load: By making sure candidates are given an optimal time frame within which to complete the test, given the difficulty of the skills being evaluated. We want each test to be challenging, but if candidates   feel overly rushed – for example, if they’re required to complete complex calculations in under a minute – it means that the intrinsic load is unnecessarily overwhelming.

2. Minimize the extraneous load: By providing clear instructions for candidates. If instructions are confusing or unclear, the extraneous load increases and could be overwhelming. 

3. Maximize the germane load: With well-designed questions that help candidates’ brains to simplify the information that appears in the test. Poorly designed questions minimize the germane load and can lead to unnecessary overwhelm.

Getting both test content and timing right is crucial to ensuring our tests have the correct intrinsic load, and can accurately evaluate a candidate’s skills.. 

How long are TestGorilla tests?

Hopefully the above explains the thinking behind the timing of TestGorilla’s tests. Next, we’ll look at some more specific examples of test timing from our test library and walk you through which kinds of tests warrant a shorter or longer test time.

Note: You can find the exact duration of each of our tests individually in our test library. Simply click into the ‘details’ of the test and look for the ‘time’ panel on the right hand side.

10 minutes for scenario-based tests

Our scenario-based evaluations prioritize those job skills candidates will likely use the most. This approach allows us to reduce the number of questions asked without compromising on test effectiveness. Additionally, shorter tests encourage candidates to complete the assessment and allow recruiters to screen large volumes of applicants more efficiently without compromising accuracy - helping to reduce inefficiencies while enhancing productivity.

For this reason, many of TestGorilla’s scenario-based tests hover around the 10-minute mark because they strike an optimal balance between brevity (to avoid unnecessary stress, cognitive load, or disengagement) and comprehensiveness (enough time to assess core competencies).

We generally start scenario-based tests at 10 minutes while partnering with subject matter experts to determine the optimum number of questions for this timeframe, which typically ranges from 12-20 questions. 

Variability in test lengths

While many assessments aim for the sweet spot of approximately ten minutes, some may be slightly shorter or longer depending on what they’re designed to evaluate.

1. Shorter tests: When evaluating more straightforward skills that candidates can demonstrate within a short time window – such as typing – we learn towards briefer durations.

2. Longer tests: Conversely, when immersing candidates in coding challenges and similar simulations designed to replicate complex work tasks, we provide more time. 

In skills-based tests, time limits are deliberately set to increase challenge levels. Our goal is to make it challenging yet feasible for candidates possessing relevant competencies to complete within given boundaries.

Monitoring test timing

Using test completion data allows us to periodically review both question quantity and timing accuracy. If most candidates complete the test well under allocated time, we might reduce its duration or add more questions – or both! Conversely, if most struggle to finish within given limits, we may extend time allocation or include fewer items.

Consider timing when building assessments with TestGorilla

At TestGorilla, we understand that every organization has unique needs when it comes to pre-hire assessments. Therefore, we provide our customers with the flexibility to build their own assessments – a series of up to 5 tests along with custom questions. Additionally, they can also choose from pre-created assessments designed by our team.

Just as timing is a consideration for us when we build individual tests, it's equally crucial for our users to consider timing when combining tests into an assessment package. Here's why:

Avoiding cognitive overload across multiple tests

When multiple tests are bundled together into one assessment, cognitive load theory still applies but at a cumulative level. It’s important not to overload an assessment with long-duration tests which could lead candidates towards cognitive fatigue before they've completed all sections.

Striking a balance between shorter and longer duration tests within an assessment helps maintain candidate engagement throughout the process while ensuring comprehensive evaluation of various skills and competencies.

For instance, you might pair a single longer coding challenge like Coding: Intermediate-Level Algorithms (35 mins) – which requires deep thinking and problem-solving – with several shorter tests on related topics like clean code (10 mins) or a programming framework like Django (10 mins) – providing breadth without overwhelming depth in any single area.

In essence, careful consideration should be given not just to test durations individually, but also how these timings work collectively within your customized or chosen pre-made assessments.

Want to learn more about the science behind TestGorilla?

The goal of our science series blogs is twofold: To raise awareness about the science and thinking behind TestGorilla’s tests, and to help our users to create the best assessments possible with an optimal candidate experience. 

For more content that demystifies assessment science and gives you actionable ways to improve the way you use TestGorilla, check out: 

• A brief introduction to test order: 2 assessment science-backed tips on how to order your tests in an assessment

• A brief introduction to candidate satisfaction: 3 data-driven recommendations for improving candidate satisfaction with TestGorilla

• How to select tests for an assessment (and avoid a skill overlap): Clear guidelines on how to avoid testing for the same thing twice