How to Run a Bias Audit on Your AI Assessment Process

Author

Anjana Unni

Created on

June 22, 2026

Two pieces of legislation, California's ADS regulations and Colorado's AI Act, now make HR teams, not just their vendors, legally accountable for bias in AI-powered hiring tools. A bias audit is a structured process for measuring whether your assessment pipeline produces different outcomes across demographic groups. It involves mapping every AI touchpoint, requesting adverse impact data from vendors, running the 4/5ths rule calculation yourself, checking for proxy discrimination, and documenting findings in a format regulators can read.

What Is a Bias Audit in Hiring?

A bias audit is a structured, documented review of whether an assessment tool or process produces systematically different outcomes for people in legally protected groups, such as gender, race and ethnicity, age, national origin, disability status. It is not a values statement. It is not a diversity initiative. It is a measurement exercise with a specific output: evidence that your selection procedures treat candidates equitably, or a clear record of where they do not and what you did about it.

The concept has deep roots in I/O psychology. The Uniform Guidelines on Employee Selection Procedures, in place since 1978, require employers to evaluate any selection procedure that causes adverse impact, meaning disparate harm to a protected group. What has changed recently is who is explicitly named as responsible. Historically, vendors often absorbed the liability question. Now, HR teams are directly in scope under new state legislation.

Bias audits are distinct from what vendors typically provide. A vendor technical manual is not a bias audit. A diversity statement is not a bias audit. A one-time third-party review completed before your organization started using a tool is not a bias audit. A bias audit is a recurring, documented process applied to your specific implementation, with your applicant population, in your hiring context.

The 2023 EEOC Technical Assistance guidance on AI in employment selection confirmed that the Commission applies the same legal framework to algorithmic tools as to any other selection procedure. If a tool produces adverse impact, the employer using it bears responsibility, regardless of who built it.

Why This Is Urgent Right Now

Two laws have closed the gap between "vendor accountability" and "your accountability."

California ADS Regulations (effective October 1, 2025)

The California Civil Rights Council's regulations under the Fair Employment and Housing Act now define an Automated Decision System as any computational process that makes or assists in making employment decisions, such as screening resumes, scoring interviews, ranking candidates, assigning to training programs. The rules apply to employers with five or more California employees. Crucially, they apply to employer conduct, not just vendor products. If your system produces discriminatory outcomes, the obligation to demonstrate otherwise sits with you.

A parallel set of California regulations under the CCPA, finalised by the California Privacy Protection Agency in late 2025, adds an ADMT risk assessment requirement for any "significant decisions" about consumers, including employment decisions. Compliance deadline: January 1, 2027. That timeline sounds comfortable until you map how long proper documentation actually takes.

Colorado AI Act (SB 24-205, effective June 30, 2026)

Colorado's AI Act is the first comprehensive state law to create explicit statutory obligations for both developers and users of high-risk AI systems in consequential decisions. Employment is a covered category. The law establishes a duty to avoid algorithmic discrimination, requires impact assessments, and mandates disclosure when AI is used in a consequential employment decision. As of late April 2026, a federal court issued a stay pausing enforcement while litigation proceeds, but as employment attorneys note, the underlying statutory framework remains intact and enforcement could resume at any time. Organizations planning to operate in Colorado have no good reason not to continue building their compliance infrastructure.

Together, these two regimes signal where US employment law is heading. The organizations building audit capability now are ahead of the curve. Those waiting for a final stable version of every regulation are betting their documentation burden compounds faster than their readiness.

Step	What you'll be doing	What you need
1. Map AI touchpoints	Inventory every tool that scores, ranks, or filters candidates	Tool name, vendor, decision stage, input/output, last bias review
2. Request vendor data	Ask each vendor for adverse impact stats by demographic group	Pass rates by gender/race/age, sample sizes, methodology, audit date
3. Run the 4/5ths rule	Calculate selection rates across groups using your own data	Applicant data by demographic group; flag any ratio below 0.80
4. Check for proxies	Review model inputs for features that correlate with protected characteristics	List of model inputs; ask vendor about zip code, graduation year, school name
5. Document everything	Create a dated file regulators can read	Tool log, vendor requests + responses, your calculations, action taken, next review date

‍

What To Do If Your Vendor Can't Provide the Data

Some vendors will decline the request. Some will produce data that is technically responsive but analytically useless. Aggregate statistics that mask stage-level variation, or sample sizes too small for meaningful interpretation, are common examples. A few will send you a PDF with a lot of reassuring language and no actual numbers.

Here is what to do.

Escalate the request in writing. A formal written request creates a paper trail. It also signals that you are treating this as a compliance matter, not a preference.
Review the contract. California's regulations place explicit vendor management obligations on employers. If your contract does not include audit cooperation clauses, the next renewal should. Employment law firms that specialize in AI compliance have model clauses available.
Run what you can with your own data. You may not be able to audit the model internals, but you can audit the outputs. If you have applicant demographic data, you can calculate selection rates by group regardless of vendor cooperation. The output analysis is legitimate audit evidence even if it tells you nothing about why the tool behaves the way it does.
Treat the refusal as a risk signal. A vendor who cannot or will not produce adverse impact data by demographic group is a vendor you cannot fully account for under the new regulatory regime. That is a procurement and risk management consideration, not just a compliance one. Factor it into renewal decisions.
Consider replacing the tool. This sounds drastic. It is less drastic than defending a disparate impact claim with no documentation and a vendor who went silent on your data requests.

What Deeper Signals Does Differently

Deeper Signals assessments, the Core Drivers and Core Values diagnostics, are tested for adverse impact as part of their standard validation process. The normative database includes 90,000+ individuals. All models pass adverse impact testing across gender, age, and ethnicity, with full methodology documented in technical manuals.

This matters for HR teams running bias audits. Rather than relying on a general fairness commitment, you can point to a documented, quantified finding that these tools do not produce systematic group differences in outcomes.

The documents includes explicit adverse impact evidence alongside reliability and validity data. That is what a vendor response to a bias data request should look like. If you are evaluating assessment vendors under the new regulatory framework, it is a useful benchmark for what adequate documentation actually requires.

View the Deeper Signals science overview documents to understand the full validation approach.

Frequently Asked Questions

1. What is the four-fifths rule?

The four-fifths (or 80%) rule is an EEOC standard from 1978, still used as the primary first-pass test for adverse impact. If the selection rate for any demographic group falls below 80% of the selection rate for the highest-passing group, that is flagged as potential evidence of adverse impact. It is a starting threshold for investigation, not a legal safe harbour.

2. How is a bias audit different from what my vendor provides?

A vendor's technical manual documents how their tool performs on a general or proprietary sample. Your bias audit applies to your applicant population, your hiring context, and your implementation of the tool. The EEOC and state regulators are interested in what your tool does with your applicants, not what it did in the vendor's validation study.

3. What inputs create proxy discrimination risk?

Zip code, graduation year, school name, employment gaps, commute distance, and certain language patterns have all been identified in research and litigation as potential proxies for protected characteristics including race, age, and gender. Any input that correlates with demographic group membership, even indirectly, requires review.

4. What records do I need to keep?

Under California ADS regulations, ADS-related records must be retained for four years. Under Colorado's AI Act, deployers must retain completed impact assessments for three years after final deployment. Documents to retain include: your tool inventory, vendor data requests and responses, your own adverse impact calculations, proxy reviews, and any actions taken based on findings.

5. What if my vendor refuses to provide demographic pass-rate data?

Escalate the request in writing, review your contract for audit cooperation clauses, and run your own output-level analysis using applicant demographic data you hold. A vendor's refusal to provide adverse impact data is itself a risk indicator. It may be a factor in renewal or replacement decisions.

6. Where does Deeper Signals fit in this process?

Deeper Signals personality and values assessments are designed for use at the evaluation stage of the assessment process. Because the underlying models pass adverse impact testing across demographic groups, HR teams can use them with documented evidence of fairness, and point to that evidence in audit files, vendor reviews, and regulatory inquiries. The Deeper Signals platform also supports structured scoring and output documentation, which feeds directly into audit records.

How to Run a Bias Audit on Your AI Assessment Process

Author

Anjana Unni

Created on

June 22, 2026

Two pieces of legislation, California's ADS regulations and Colorado's AI Act, now make HR teams, not just their vendors, legally accountable for bias in AI-powered hiring tools. A bias audit is a structured process for measuring whether your assessment pipeline produces different outcomes across demographic groups. It involves mapping every AI touchpoint, requesting adverse impact data from vendors, running the 4/5ths rule calculation yourself, checking for proxy discrimination, and documenting findings in a format regulators can read.

What Is a Bias Audit in Hiring?

A bias audit is a structured, documented review of whether an assessment tool or process produces systematically different outcomes for people in legally protected groups, such as gender, race and ethnicity, age, national origin, disability status. It is not a values statement. It is not a diversity initiative. It is a measurement exercise with a specific output: evidence that your selection procedures treat candidates equitably, or a clear record of where they do not and what you did about it.

The concept has deep roots in I/O psychology. The Uniform Guidelines on Employee Selection Procedures, in place since 1978, require employers to evaluate any selection procedure that causes adverse impact, meaning disparate harm to a protected group. What has changed recently is who is explicitly named as responsible. Historically, vendors often absorbed the liability question. Now, HR teams are directly in scope under new state legislation.

Bias audits are distinct from what vendors typically provide. A vendor technical manual is not a bias audit. A diversity statement is not a bias audit. A one-time third-party review completed before your organization started using a tool is not a bias audit. A bias audit is a recurring, documented process applied to your specific implementation, with your applicant population, in your hiring context.

The 2023 EEOC Technical Assistance guidance on AI in employment selection confirmed that the Commission applies the same legal framework to algorithmic tools as to any other selection procedure. If a tool produces adverse impact, the employer using it bears responsibility, regardless of who built it.

Why This Is Urgent Right Now

Two laws have closed the gap between "vendor accountability" and "your accountability."

California ADS Regulations (effective October 1, 2025)

The California Civil Rights Council's regulations under the Fair Employment and Housing Act now define an Automated Decision System as any computational process that makes or assists in making employment decisions, such as screening resumes, scoring interviews, ranking candidates, assigning to training programs. The rules apply to employers with five or more California employees. Crucially, they apply to employer conduct, not just vendor products. If your system produces discriminatory outcomes, the obligation to demonstrate otherwise sits with you.

A parallel set of California regulations under the CCPA, finalised by the California Privacy Protection Agency in late 2025, adds an ADMT risk assessment requirement for any "significant decisions" about consumers, including employment decisions. Compliance deadline: January 1, 2027. That timeline sounds comfortable until you map how long proper documentation actually takes.

Colorado AI Act (SB 24-205, effective June 30, 2026)

Colorado's AI Act is the first comprehensive state law to create explicit statutory obligations for both developers and users of high-risk AI systems in consequential decisions. Employment is a covered category. The law establishes a duty to avoid algorithmic discrimination, requires impact assessments, and mandates disclosure when AI is used in a consequential employment decision. As of late April 2026, a federal court issued a stay pausing enforcement while litigation proceeds, but as employment attorneys note, the underlying statutory framework remains intact and enforcement could resume at any time. Organizations planning to operate in Colorado have no good reason not to continue building their compliance infrastructure.

Together, these two regimes signal where US employment law is heading. The organizations building audit capability now are ahead of the curve. Those waiting for a final stable version of every regulation are betting their documentation burden compounds faster than their readiness.

Step	What you'll be doing	What you need
1. Map AI touchpoints	Inventory every tool that scores, ranks, or filters candidates	Tool name, vendor, decision stage, input/output, last bias review
2. Request vendor data	Ask each vendor for adverse impact stats by demographic group	Pass rates by gender/race/age, sample sizes, methodology, audit date
3. Run the 4/5ths rule	Calculate selection rates across groups using your own data	Applicant data by demographic group; flag any ratio below 0.80
4. Check for proxies	Review model inputs for features that correlate with protected characteristics	List of model inputs; ask vendor about zip code, graduation year, school name
5. Document everything	Create a dated file regulators can read	Tool log, vendor requests + responses, your calculations, action taken, next review date

‍

What To Do If Your Vendor Can't Provide the Data

Some vendors will decline the request. Some will produce data that is technically responsive but analytically useless. Aggregate statistics that mask stage-level variation, or sample sizes too small for meaningful interpretation, are common examples. A few will send you a PDF with a lot of reassuring language and no actual numbers.

Here is what to do.

Escalate the request in writing. A formal written request creates a paper trail. It also signals that you are treating this as a compliance matter, not a preference.
Review the contract. California's regulations place explicit vendor management obligations on employers. If your contract does not include audit cooperation clauses, the next renewal should. Employment law firms that specialize in AI compliance have model clauses available.
Run what you can with your own data. You may not be able to audit the model internals, but you can audit the outputs. If you have applicant demographic data, you can calculate selection rates by group regardless of vendor cooperation. The output analysis is legitimate audit evidence even if it tells you nothing about why the tool behaves the way it does.
Treat the refusal as a risk signal. A vendor who cannot or will not produce adverse impact data by demographic group is a vendor you cannot fully account for under the new regulatory regime. That is a procurement and risk management consideration, not just a compliance one. Factor it into renewal decisions.
Consider replacing the tool. This sounds drastic. It is less drastic than defending a disparate impact claim with no documentation and a vendor who went silent on your data requests.

What Deeper Signals Does Differently

Deeper Signals assessments, the Core Drivers and Core Values diagnostics, are tested for adverse impact as part of their standard validation process. The normative database includes 90,000+ individuals. All models pass adverse impact testing across gender, age, and ethnicity, with full methodology documented in technical manuals.

This matters for HR teams running bias audits. Rather than relying on a general fairness commitment, you can point to a documented, quantified finding that these tools do not produce systematic group differences in outcomes.

The documents includes explicit adverse impact evidence alongside reliability and validity data. That is what a vendor response to a bias data request should look like. If you are evaluating assessment vendors under the new regulatory framework, it is a useful benchmark for what adequate documentation actually requires.

View the Deeper Signals science overview documents to understand the full validation approach.

Frequently Asked Questions

1. What is the four-fifths rule?

The four-fifths (or 80%) rule is an EEOC standard from 1978, still used as the primary first-pass test for adverse impact. If the selection rate for any demographic group falls below 80% of the selection rate for the highest-passing group, that is flagged as potential evidence of adverse impact. It is a starting threshold for investigation, not a legal safe harbour.

2. How is a bias audit different from what my vendor provides?

A vendor's technical manual documents how their tool performs on a general or proprietary sample. Your bias audit applies to your applicant population, your hiring context, and your implementation of the tool. The EEOC and state regulators are interested in what your tool does with your applicants, not what it did in the vendor's validation study.

3. What inputs create proxy discrimination risk?

Zip code, graduation year, school name, employment gaps, commute distance, and certain language patterns have all been identified in research and litigation as potential proxies for protected characteristics including race, age, and gender. Any input that correlates with demographic group membership, even indirectly, requires review.

4. What records do I need to keep?

Under California ADS regulations, ADS-related records must be retained for four years. Under Colorado's AI Act, deployers must retain completed impact assessments for three years after final deployment. Documents to retain include: your tool inventory, vendor data requests and responses, your own adverse impact calculations, proxy reviews, and any actions taken based on findings.

5. What if my vendor refuses to provide demographic pass-rate data?

Escalate the request in writing, review your contract for audit cooperation clauses, and run your own output-level analysis using applicant demographic data you hold. A vendor's refusal to provide adverse impact data is itself a risk indicator. It may be a factor in renewal or replacement decisions.

6. Where does Deeper Signals fit in this process?

Deeper Signals personality and values assessments are designed for use at the evaluation stage of the assessment process. Because the underlying models pass adverse impact testing across demographic groups, HR teams can use them with documented evidence of fairness, and point to that evidence in audit files, vendor reviews, and regulatory inquiries. The Deeper Signals platform also supports structured scoring and output documentation, which feeds directly into audit records.

How to Run a Bias Audit on Your AI Assessment Process

Author

Anjana Unni

Created on

June 22, 2026

Two pieces of legislation, California's ADS regulations and Colorado's AI Act, now make HR teams, not just their vendors, legally accountable for bias in AI-powered hiring tools. A bias audit is a structured process for measuring whether your assessment pipeline produces different outcomes across demographic groups. It involves mapping every AI touchpoint, requesting adverse impact data from vendors, running the 4/5ths rule calculation yourself, checking for proxy discrimination, and documenting findings in a format regulators can read.

What Is a Bias Audit in Hiring?

A bias audit is a structured, documented review of whether an assessment tool or process produces systematically different outcomes for people in legally protected groups, such as gender, race and ethnicity, age, national origin, disability status. It is not a values statement. It is not a diversity initiative. It is a measurement exercise with a specific output: evidence that your selection procedures treat candidates equitably, or a clear record of where they do not and what you did about it.

The concept has deep roots in I/O psychology. The Uniform Guidelines on Employee Selection Procedures, in place since 1978, require employers to evaluate any selection procedure that causes adverse impact, meaning disparate harm to a protected group. What has changed recently is who is explicitly named as responsible. Historically, vendors often absorbed the liability question. Now, HR teams are directly in scope under new state legislation.

Bias audits are distinct from what vendors typically provide. A vendor technical manual is not a bias audit. A diversity statement is not a bias audit. A one-time third-party review completed before your organization started using a tool is not a bias audit. A bias audit is a recurring, documented process applied to your specific implementation, with your applicant population, in your hiring context.

The 2023 EEOC Technical Assistance guidance on AI in employment selection confirmed that the Commission applies the same legal framework to algorithmic tools as to any other selection procedure. If a tool produces adverse impact, the employer using it bears responsibility, regardless of who built it.

Why This Is Urgent Right Now

Two laws have closed the gap between "vendor accountability" and "your accountability."

California ADS Regulations (effective October 1, 2025)

The California Civil Rights Council's regulations under the Fair Employment and Housing Act now define an Automated Decision System as any computational process that makes or assists in making employment decisions, such as screening resumes, scoring interviews, ranking candidates, assigning to training programs. The rules apply to employers with five or more California employees. Crucially, they apply to employer conduct, not just vendor products. If your system produces discriminatory outcomes, the obligation to demonstrate otherwise sits with you.

A parallel set of California regulations under the CCPA, finalised by the California Privacy Protection Agency in late 2025, adds an ADMT risk assessment requirement for any "significant decisions" about consumers, including employment decisions. Compliance deadline: January 1, 2027. That timeline sounds comfortable until you map how long proper documentation actually takes.

Colorado AI Act (SB 24-205, effective June 30, 2026)

Colorado's AI Act is the first comprehensive state law to create explicit statutory obligations for both developers and users of high-risk AI systems in consequential decisions. Employment is a covered category. The law establishes a duty to avoid algorithmic discrimination, requires impact assessments, and mandates disclosure when AI is used in a consequential employment decision. As of late April 2026, a federal court issued a stay pausing enforcement while litigation proceeds, but as employment attorneys note, the underlying statutory framework remains intact and enforcement could resume at any time. Organizations planning to operate in Colorado have no good reason not to continue building their compliance infrastructure.

Together, these two regimes signal where US employment law is heading. The organizations building audit capability now are ahead of the curve. Those waiting for a final stable version of every regulation are betting their documentation burden compounds faster than their readiness.

Step	What you'll be doing	What you need
1. Map AI touchpoints	Inventory every tool that scores, ranks, or filters candidates	Tool name, vendor, decision stage, input/output, last bias review
2. Request vendor data	Ask each vendor for adverse impact stats by demographic group	Pass rates by gender/race/age, sample sizes, methodology, audit date
3. Run the 4/5ths rule	Calculate selection rates across groups using your own data	Applicant data by demographic group; flag any ratio below 0.80
4. Check for proxies	Review model inputs for features that correlate with protected characteristics	List of model inputs; ask vendor about zip code, graduation year, school name
5. Document everything	Create a dated file regulators can read	Tool log, vendor requests + responses, your calculations, action taken, next review date

‍

What To Do If Your Vendor Can't Provide the Data

Some vendors will decline the request. Some will produce data that is technically responsive but analytically useless. Aggregate statistics that mask stage-level variation, or sample sizes too small for meaningful interpretation, are common examples. A few will send you a PDF with a lot of reassuring language and no actual numbers.

Here is what to do.

Escalate the request in writing. A formal written request creates a paper trail. It also signals that you are treating this as a compliance matter, not a preference.
Review the contract. California's regulations place explicit vendor management obligations on employers. If your contract does not include audit cooperation clauses, the next renewal should. Employment law firms that specialize in AI compliance have model clauses available.
Run what you can with your own data. You may not be able to audit the model internals, but you can audit the outputs. If you have applicant demographic data, you can calculate selection rates by group regardless of vendor cooperation. The output analysis is legitimate audit evidence even if it tells you nothing about why the tool behaves the way it does.
Treat the refusal as a risk signal. A vendor who cannot or will not produce adverse impact data by demographic group is a vendor you cannot fully account for under the new regulatory regime. That is a procurement and risk management consideration, not just a compliance one. Factor it into renewal decisions.
Consider replacing the tool. This sounds drastic. It is less drastic than defending a disparate impact claim with no documentation and a vendor who went silent on your data requests.

What Deeper Signals Does Differently

Deeper Signals assessments, the Core Drivers and Core Values diagnostics, are tested for adverse impact as part of their standard validation process. The normative database includes 90,000+ individuals. All models pass adverse impact testing across gender, age, and ethnicity, with full methodology documented in technical manuals.

This matters for HR teams running bias audits. Rather than relying on a general fairness commitment, you can point to a documented, quantified finding that these tools do not produce systematic group differences in outcomes.

The documents includes explicit adverse impact evidence alongside reliability and validity data. That is what a vendor response to a bias data request should look like. If you are evaluating assessment vendors under the new regulatory framework, it is a useful benchmark for what adequate documentation actually requires.

View the Deeper Signals science overview documents to understand the full validation approach.

Frequently Asked Questions

1. What is the four-fifths rule?

The four-fifths (or 80%) rule is an EEOC standard from 1978, still used as the primary first-pass test for adverse impact. If the selection rate for any demographic group falls below 80% of the selection rate for the highest-passing group, that is flagged as potential evidence of adverse impact. It is a starting threshold for investigation, not a legal safe harbour.

2. How is a bias audit different from what my vendor provides?

A vendor's technical manual documents how their tool performs on a general or proprietary sample. Your bias audit applies to your applicant population, your hiring context, and your implementation of the tool. The EEOC and state regulators are interested in what your tool does with your applicants, not what it did in the vendor's validation study.

3. What inputs create proxy discrimination risk?

Zip code, graduation year, school name, employment gaps, commute distance, and certain language patterns have all been identified in research and litigation as potential proxies for protected characteristics including race, age, and gender. Any input that correlates with demographic group membership, even indirectly, requires review.

4. What records do I need to keep?

Under California ADS regulations, ADS-related records must be retained for four years. Under Colorado's AI Act, deployers must retain completed impact assessments for three years after final deployment. Documents to retain include: your tool inventory, vendor data requests and responses, your own adverse impact calculations, proxy reviews, and any actions taken based on findings.

5. What if my vendor refuses to provide demographic pass-rate data?

Escalate the request in writing, review your contract for audit cooperation clauses, and run your own output-level analysis using applicant demographic data you hold. A vendor's refusal to provide adverse impact data is itself a risk indicator. It may be a factor in renewal or replacement decisions.

6. Where does Deeper Signals fit in this process?

Deeper Signals personality and values assessments are designed for use at the evaluation stage of the assessment process. Because the underlying models pass adverse impact testing across demographic groups, HR teams can use them with documented evidence of fairness, and point to that evidence in audit files, vendor reviews, and regulatory inquiries. The Deeper Signals platform also supports structured scoring and output documentation, which feeds directly into audit records.

How to Run a Bias Audit on Your AI Assessment Process

Author

Anjana Unni

Created on

June 22, 2026

Two pieces of legislation, California's ADS regulations and Colorado's AI Act, now make HR teams, not just their vendors, legally accountable for bias in AI-powered hiring tools. A bias audit is a structured process for measuring whether your assessment pipeline produces different outcomes across demographic groups. It involves mapping every AI touchpoint, requesting adverse impact data from vendors, running the 4/5ths rule calculation yourself, checking for proxy discrimination, and documenting findings in a format regulators can read.

What Is a Bias Audit in Hiring?

A bias audit is a structured, documented review of whether an assessment tool or process produces systematically different outcomes for people in legally protected groups, such as gender, race and ethnicity, age, national origin, disability status. It is not a values statement. It is not a diversity initiative. It is a measurement exercise with a specific output: evidence that your selection procedures treat candidates equitably, or a clear record of where they do not and what you did about it.

The concept has deep roots in I/O psychology. The Uniform Guidelines on Employee Selection Procedures, in place since 1978, require employers to evaluate any selection procedure that causes adverse impact, meaning disparate harm to a protected group. What has changed recently is who is explicitly named as responsible. Historically, vendors often absorbed the liability question. Now, HR teams are directly in scope under new state legislation.

Bias audits are distinct from what vendors typically provide. A vendor technical manual is not a bias audit. A diversity statement is not a bias audit. A one-time third-party review completed before your organization started using a tool is not a bias audit. A bias audit is a recurring, documented process applied to your specific implementation, with your applicant population, in your hiring context.

The 2023 EEOC Technical Assistance guidance on AI in employment selection confirmed that the Commission applies the same legal framework to algorithmic tools as to any other selection procedure. If a tool produces adverse impact, the employer using it bears responsibility, regardless of who built it.

Why This Is Urgent Right Now

Two laws have closed the gap between "vendor accountability" and "your accountability."

California ADS Regulations (effective October 1, 2025)

The California Civil Rights Council's regulations under the Fair Employment and Housing Act now define an Automated Decision System as any computational process that makes or assists in making employment decisions, such as screening resumes, scoring interviews, ranking candidates, assigning to training programs. The rules apply to employers with five or more California employees. Crucially, they apply to employer conduct, not just vendor products. If your system produces discriminatory outcomes, the obligation to demonstrate otherwise sits with you.

A parallel set of California regulations under the CCPA, finalised by the California Privacy Protection Agency in late 2025, adds an ADMT risk assessment requirement for any "significant decisions" about consumers, including employment decisions. Compliance deadline: January 1, 2027. That timeline sounds comfortable until you map how long proper documentation actually takes.

Colorado AI Act (SB 24-205, effective June 30, 2026)

Colorado's AI Act is the first comprehensive state law to create explicit statutory obligations for both developers and users of high-risk AI systems in consequential decisions. Employment is a covered category. The law establishes a duty to avoid algorithmic discrimination, requires impact assessments, and mandates disclosure when AI is used in a consequential employment decision. As of late April 2026, a federal court issued a stay pausing enforcement while litigation proceeds, but as employment attorneys note, the underlying statutory framework remains intact and enforcement could resume at any time. Organizations planning to operate in Colorado have no good reason not to continue building their compliance infrastructure.

Together, these two regimes signal where US employment law is heading. The organizations building audit capability now are ahead of the curve. Those waiting for a final stable version of every regulation are betting their documentation burden compounds faster than their readiness.

Step	What you'll be doing	What you need
1. Map AI touchpoints	Inventory every tool that scores, ranks, or filters candidates	Tool name, vendor, decision stage, input/output, last bias review
2. Request vendor data	Ask each vendor for adverse impact stats by demographic group	Pass rates by gender/race/age, sample sizes, methodology, audit date
3. Run the 4/5ths rule	Calculate selection rates across groups using your own data	Applicant data by demographic group; flag any ratio below 0.80
4. Check for proxies	Review model inputs for features that correlate with protected characteristics	List of model inputs; ask vendor about zip code, graduation year, school name
5. Document everything	Create a dated file regulators can read	Tool log, vendor requests + responses, your calculations, action taken, next review date

‍

What To Do If Your Vendor Can't Provide the Data

Some vendors will decline the request. Some will produce data that is technically responsive but analytically useless. Aggregate statistics that mask stage-level variation, or sample sizes too small for meaningful interpretation, are common examples. A few will send you a PDF with a lot of reassuring language and no actual numbers.

Here is what to do.

Escalate the request in writing. A formal written request creates a paper trail. It also signals that you are treating this as a compliance matter, not a preference.
Review the contract. California's regulations place explicit vendor management obligations on employers. If your contract does not include audit cooperation clauses, the next renewal should. Employment law firms that specialize in AI compliance have model clauses available.
Run what you can with your own data. You may not be able to audit the model internals, but you can audit the outputs. If you have applicant demographic data, you can calculate selection rates by group regardless of vendor cooperation. The output analysis is legitimate audit evidence even if it tells you nothing about why the tool behaves the way it does.
Treat the refusal as a risk signal. A vendor who cannot or will not produce adverse impact data by demographic group is a vendor you cannot fully account for under the new regulatory regime. That is a procurement and risk management consideration, not just a compliance one. Factor it into renewal decisions.
Consider replacing the tool. This sounds drastic. It is less drastic than defending a disparate impact claim with no documentation and a vendor who went silent on your data requests.

What Deeper Signals Does Differently

Deeper Signals assessments, the Core Drivers and Core Values diagnostics, are tested for adverse impact as part of their standard validation process. The normative database includes 90,000+ individuals. All models pass adverse impact testing across gender, age, and ethnicity, with full methodology documented in technical manuals.

This matters for HR teams running bias audits. Rather than relying on a general fairness commitment, you can point to a documented, quantified finding that these tools do not produce systematic group differences in outcomes.

The documents includes explicit adverse impact evidence alongside reliability and validity data. That is what a vendor response to a bias data request should look like. If you are evaluating assessment vendors under the new regulatory framework, it is a useful benchmark for what adequate documentation actually requires.

View the Deeper Signals science overview documents to understand the full validation approach.

Frequently Asked Questions

1. What is the four-fifths rule?

The four-fifths (or 80%) rule is an EEOC standard from 1978, still used as the primary first-pass test for adverse impact. If the selection rate for any demographic group falls below 80% of the selection rate for the highest-passing group, that is flagged as potential evidence of adverse impact. It is a starting threshold for investigation, not a legal safe harbour.

2. How is a bias audit different from what my vendor provides?

A vendor's technical manual documents how their tool performs on a general or proprietary sample. Your bias audit applies to your applicant population, your hiring context, and your implementation of the tool. The EEOC and state regulators are interested in what your tool does with your applicants, not what it did in the vendor's validation study.

3. What inputs create proxy discrimination risk?

Zip code, graduation year, school name, employment gaps, commute distance, and certain language patterns have all been identified in research and litigation as potential proxies for protected characteristics including race, age, and gender. Any input that correlates with demographic group membership, even indirectly, requires review.

4. What records do I need to keep?

Under California ADS regulations, ADS-related records must be retained for four years. Under Colorado's AI Act, deployers must retain completed impact assessments for three years after final deployment. Documents to retain include: your tool inventory, vendor data requests and responses, your own adverse impact calculations, proxy reviews, and any actions taken based on findings.

5. What if my vendor refuses to provide demographic pass-rate data?

Escalate the request in writing, review your contract for audit cooperation clauses, and run your own output-level analysis using applicant demographic data you hold. A vendor's refusal to provide adverse impact data is itself a risk indicator. It may be a factor in renewal or replacement decisions.

6. Where does Deeper Signals fit in this process?

Deeper Signals personality and values assessments are designed for use at the evaluation stage of the assessment process. Because the underlying models pass adverse impact testing across demographic groups, HR teams can use them with documented evidence of fairness, and point to that evidence in audit files, vendor reviews, and regulatory inquiries. The Deeper Signals platform also supports structured scoring and output documentation, which feeds directly into audit records.

How to Run a Bias Audit on Your AI Assessment Process

Customer

Job Title

Two pieces of legislation, California's ADS regulations and Colorado's AI Act, now make HR teams, not just their vendors, legally accountable for bias in AI-powered hiring tools. A bias audit is a structured process for measuring whether your assessment pipeline produces different outcomes across demographic groups. It involves mapping every AI touchpoint, requesting adverse impact data from vendors, running the 4/5ths rule calculation yourself, checking for proxy discrimination, and documenting findings in a format regulators can read.

What Is a Bias Audit in Hiring?

A bias audit is a structured, documented review of whether an assessment tool or process produces systematically different outcomes for people in legally protected groups, such as gender, race and ethnicity, age, national origin, disability status. It is not a values statement. It is not a diversity initiative. It is a measurement exercise with a specific output: evidence that your selection procedures treat candidates equitably, or a clear record of where they do not and what you did about it.

The concept has deep roots in I/O psychology. The Uniform Guidelines on Employee Selection Procedures, in place since 1978, require employers to evaluate any selection procedure that causes adverse impact, meaning disparate harm to a protected group. What has changed recently is who is explicitly named as responsible. Historically, vendors often absorbed the liability question. Now, HR teams are directly in scope under new state legislation.

Bias audits are distinct from what vendors typically provide. A vendor technical manual is not a bias audit. A diversity statement is not a bias audit. A one-time third-party review completed before your organization started using a tool is not a bias audit. A bias audit is a recurring, documented process applied to your specific implementation, with your applicant population, in your hiring context.

The 2023 EEOC Technical Assistance guidance on AI in employment selection confirmed that the Commission applies the same legal framework to algorithmic tools as to any other selection procedure. If a tool produces adverse impact, the employer using it bears responsibility, regardless of who built it.

Why This Is Urgent Right Now

Two laws have closed the gap between "vendor accountability" and "your accountability."

California ADS Regulations (effective October 1, 2025)

The California Civil Rights Council's regulations under the Fair Employment and Housing Act now define an Automated Decision System as any computational process that makes or assists in making employment decisions, such as screening resumes, scoring interviews, ranking candidates, assigning to training programs. The rules apply to employers with five or more California employees. Crucially, they apply to employer conduct, not just vendor products. If your system produces discriminatory outcomes, the obligation to demonstrate otherwise sits with you.

A parallel set of California regulations under the CCPA, finalised by the California Privacy Protection Agency in late 2025, adds an ADMT risk assessment requirement for any "significant decisions" about consumers, including employment decisions. Compliance deadline: January 1, 2027. That timeline sounds comfortable until you map how long proper documentation actually takes.

Colorado AI Act (SB 24-205, effective June 30, 2026)

Colorado's AI Act is the first comprehensive state law to create explicit statutory obligations for both developers and users of high-risk AI systems in consequential decisions. Employment is a covered category. The law establishes a duty to avoid algorithmic discrimination, requires impact assessments, and mandates disclosure when AI is used in a consequential employment decision. As of late April 2026, a federal court issued a stay pausing enforcement while litigation proceeds, but as employment attorneys note, the underlying statutory framework remains intact and enforcement could resume at any time. Organizations planning to operate in Colorado have no good reason not to continue building their compliance infrastructure.

Together, these two regimes signal where US employment law is heading. The organizations building audit capability now are ahead of the curve. Those waiting for a final stable version of every regulation are betting their documentation burden compounds faster than their readiness.

Step	What you'll be doing	What you need
1. Map AI touchpoints	Inventory every tool that scores, ranks, or filters candidates	Tool name, vendor, decision stage, input/output, last bias review
2. Request vendor data	Ask each vendor for adverse impact stats by demographic group	Pass rates by gender/race/age, sample sizes, methodology, audit date
3. Run the 4/5ths rule	Calculate selection rates across groups using your own data	Applicant data by demographic group; flag any ratio below 0.80
4. Check for proxies	Review model inputs for features that correlate with protected characteristics	List of model inputs; ask vendor about zip code, graduation year, school name
5. Document everything	Create a dated file regulators can read	Tool log, vendor requests + responses, your calculations, action taken, next review date

‍

What To Do If Your Vendor Can't Provide the Data

Some vendors will decline the request. Some will produce data that is technically responsive but analytically useless. Aggregate statistics that mask stage-level variation, or sample sizes too small for meaningful interpretation, are common examples. A few will send you a PDF with a lot of reassuring language and no actual numbers.

Here is what to do.

Escalate the request in writing. A formal written request creates a paper trail. It also signals that you are treating this as a compliance matter, not a preference.
Review the contract. California's regulations place explicit vendor management obligations on employers. If your contract does not include audit cooperation clauses, the next renewal should. Employment law firms that specialize in AI compliance have model clauses available.
Run what you can with your own data. You may not be able to audit the model internals, but you can audit the outputs. If you have applicant demographic data, you can calculate selection rates by group regardless of vendor cooperation. The output analysis is legitimate audit evidence even if it tells you nothing about why the tool behaves the way it does.
Treat the refusal as a risk signal. A vendor who cannot or will not produce adverse impact data by demographic group is a vendor you cannot fully account for under the new regulatory regime. That is a procurement and risk management consideration, not just a compliance one. Factor it into renewal decisions.
Consider replacing the tool. This sounds drastic. It is less drastic than defending a disparate impact claim with no documentation and a vendor who went silent on your data requests.

What Deeper Signals Does Differently

Deeper Signals assessments, the Core Drivers and Core Values diagnostics, are tested for adverse impact as part of their standard validation process. The normative database includes 90,000+ individuals. All models pass adverse impact testing across gender, age, and ethnicity, with full methodology documented in technical manuals.

This matters for HR teams running bias audits. Rather than relying on a general fairness commitment, you can point to a documented, quantified finding that these tools do not produce systematic group differences in outcomes.

The documents includes explicit adverse impact evidence alongside reliability and validity data. That is what a vendor response to a bias data request should look like. If you are evaluating assessment vendors under the new regulatory framework, it is a useful benchmark for what adequate documentation actually requires.

View the Deeper Signals science overview documents to understand the full validation approach.

Frequently Asked Questions

1. What is the four-fifths rule?

The four-fifths (or 80%) rule is an EEOC standard from 1978, still used as the primary first-pass test for adverse impact. If the selection rate for any demographic group falls below 80% of the selection rate for the highest-passing group, that is flagged as potential evidence of adverse impact. It is a starting threshold for investigation, not a legal safe harbour.

2. How is a bias audit different from what my vendor provides?

A vendor's technical manual documents how their tool performs on a general or proprietary sample. Your bias audit applies to your applicant population, your hiring context, and your implementation of the tool. The EEOC and state regulators are interested in what your tool does with your applicants, not what it did in the vendor's validation study.

3. What inputs create proxy discrimination risk?

Zip code, graduation year, school name, employment gaps, commute distance, and certain language patterns have all been identified in research and litigation as potential proxies for protected characteristics including race, age, and gender. Any input that correlates with demographic group membership, even indirectly, requires review.

4. What records do I need to keep?

Under California ADS regulations, ADS-related records must be retained for four years. Under Colorado's AI Act, deployers must retain completed impact assessments for three years after final deployment. Documents to retain include: your tool inventory, vendor data requests and responses, your own adverse impact calculations, proxy reviews, and any actions taken based on findings.

5. What if my vendor refuses to provide demographic pass-rate data?

Escalate the request in writing, review your contract for audit cooperation clauses, and run your own output-level analysis using applicant demographic data you hold. A vendor's refusal to provide adverse impact data is itself a risk indicator. It may be a factor in renewal or replacement decisions.

6. Where does Deeper Signals fit in this process?

Deeper Signals personality and values assessments are designed for use at the evaluation stage of the assessment process. Because the underlying models pass adverse impact testing across demographic groups, HR teams can use them with documented evidence of fairness, and point to that evidence in audit files, vendor reviews, and regulatory inquiries. The Deeper Signals platform also supports structured scoring and output documentation, which feeds directly into audit records.

Ready for your Spotlight?

Contact us to book your Customer Spotlight and showcase your work to an extensive, global audience!

Start your free trial today

Free access to Deeper Signals’ quick, scientific assessments, feedback tools, and more.

Start Free Trial

Articles

Is Skills-Based Hiring Actually Happening?

A guide to performative equity, the proxy swap hiding in plain sight, and the measurement infrastructure that makes skills-first hiring real.

Articles

High Performance vs. High Potential: Why Confusing Them Is Breaking Your Succession Pipeline

A guide to separating performance from potential, understanding why your strongest individual contributors often make weak first-line managers, and changing what your succession process actually measures.

Articles

Why SJTs Are Expanding How We Assess Talent

Learn why personality and cognitive ability are only part of the picture, and how SJTs help organizations understand workplace behavior in context.

Articles

From Assessment to Action: How AI Coaching Closes the Development Gap

The HR teams seeing real behavior change aren't running more assessments - they're using talent data the way drivers use GPS: live, in context, recalculating with every turn. A guide to how Sola turns Core Drivers results into nudges, manager 1:1 prompts, and learning paths.