South Africa: Employment Equity Act considerations when using AI in the workplace

Artificial intelligence (AI) systems are not neutral. They are designed, trained and deployed by human beings, and they learn from historical data that may itself reflect entrenched patterns of discrimination. Where AI is used to make or inform employment decisions, such as hiring, promotion, performance assessment, or pay determinations, there is a material risk that the system will unfairly discriminate, directly or indirectly, against persons on the basis of protected grounds.

Considering the Employment Equity Act, 1998 (EEA), there is a risk that unfair discrimination could arise when AI systems are deployed in employment decisions.

The prohibition of unfair discrimination under the EEA

Section 6 of the EEA prohibits unfair discrimination against an employee or applicant for employment in any employment policy or practice, on one or more of the listed grounds, such as race, gender, sex, pregnancy, marital status, age, disability, religion, HIV status and language, or on any other arbitrary ground. The protection extends to applicants for employment, meaning that the use of AI in recruitment processes falls squarely within the ambit of the EEA.

Critically, the EEA prohibits both direct and indirect discrimination. Direct discrimination occurs where a criterion expressly distinguishes on a listed ground. Indirect discrimination occurs where a facially neutral criterion or practice has the effect of disproportionately disadvantaging persons from a protected group. It is the latter form of discrimination that AI systems are most likely to produce.

Algorithm bias and the reproduction of historical discrimination

AI algorithms are trained on historical data. Where that data reflects existing demographic imbalances in a workforce or industry, the algorithm may learn to favour candidates who resemble the existing (potentially non-representative) workforce. The algorithm does not ‘intend’ to discriminate, but the outcome may nonetheless amount to unfair discrimination.

A prominent international example illustrates this risk: in 2015, a large technology retailer abandoned an AI recruitment tool after discovering that it systematically discriminated against female candidates. The algorithm had been trained on 10 years of historical hiring data, which reflected a predominantly male workforce. The system accordingly learned to penalise CVs containing indicators of female gender – including, reportedly, attendance at women’s colleges.

In the South African context, the risk is amplified by the country’s historical legacy of racial and gender exclusion. An algorithm trained on historical employment data may reproduce and entrench the very patterns of exclusion that the EEA and the Constitution seek to remedy.

AI and absolute barriers to advancement

Section 15(4) of the EEA expressly prohibits designated employers from taking any decision concerning an employment policy or practice that would establish an absolute barrier to the prospective or continued employment or advancement of people who are not from designated groups. This prohibition has been affirmed by our courts, which have held that any policy that lacks the numerical goals, targets, timeframes, monitoring mechanisms, and sufficient flexibility required by the EEA amounts in substance to quota-like exclusion, which is impermissible under section 15(3) and fails to meet both the constitutional standards in section 9(2) of the Constitution and the criteria set out in section 42 of the EEA.

Where AI systems are deployed to screen, shortlist, or rank candidates for appointment or promotion, there is a risk that the algorithm may function as an absolute barrier. For example, if an AI system is configured to exclude candidates who do not meet certain criteria (such as belonging to a ‘designated group’, age, years of experience, or qualifications) without the flexibility to consider alternative indicia of suitability, the system may effectively exclude persons from non-designated groups from fair competition, in addition to discriminating on the basis of age and/or arbitrary grounds. Affirmative action measures must remain flexible, avoid creating absolute barriers, and include mechanisms for deviation that prevent disproportionate harm. An AI system that operates as a rigid, categorical gate, without any deviation mechanism, and without flexibly taking into account a designated employer’s numerical targets, EE plan and workforce profile may therefore fall foul of the EEA.

AI-powered performance monitoring and disability discrimination

AI-powered performance monitoring tools which measure metrics such as keystrokes per minute, response times, screen activity and movement patterns present particular risks for employees with disabilities. Such tools may penalise employees who require reasonable accommodation (for example, employees with visual impairments who use screen readers, or employees with mobility impairments who type more slowly). Where the metrics generated by such tools are used to inform performance ratings or disciplinary action, the system may give rise to indirect discrimination on the ground of disability. In addition, rules pertaining to the interception and monitoring of communications may also apply, and employers ought to have appropriate policies and notifications in place for such monitoring and interception.

AI-driven pay algorithms and equal pay

Section 6(4) of the EEA provides that a difference in terms and conditions of employment between employees of the same employer performing the same or substantially the same work, or work of equal value, that is directly or indirectly based on a listed ground, or any other arbitrary ground constitutes unfair discrimination. Where AI-driven pay algorithms determine or recommend remuneration levels based on historical pay data, there is a risk that they will perpetuate existing gender or race-based pay gaps, giving rise to claims under section 6(4).

The ‘black box’ problem and transparency

A fundamental challenge with AI systems is the so-called ‘black box’ problem: the inability of users (including employers) to explain how an algorithm arrived at a particular outcome. Where an applicant or employee challenges an AI-driven decision as discriminatory on one or more of the listed grounds, the employer bears the onus (in terms of section 11 of the EEA) of proving that the discrimination did not take place, or that it is rational and not unfair, or that it is otherwise justifiable. If the employer cannot explain how the AI system reached its decision, it may be unable to discharge this onus.

Guardrails for recruitment agencies and third-party service providers

Where an employer engages a recruitment agency or other third-party service provider to conduct pre-screening, shortlisting, or other candidate assessment activities, the employer remains legally responsible for ensuring that such activities do not amount to unfair discrimination. An employer cannot escape liability under the EEA by outsourcing recruitment functions to an agency – if the agency’s screening process unfairly discriminates against applicants on a listed or arbitrary ground, the employer may be held accountable.

Employers should accordingly implement contractual and operational guardrails when engaging recruitment agencies that use AI systems. At a minimum, the employer should require the agency to disclose, prior to the commencement of the engagement, whether it uses any AI, algorithmic, or automated tools in candidate sourcing, screening, shortlisting, ranking, or assessment and whether it informs candidates of their practices. The employer should request full transparency on what AI systems are used by the agency, including the name and vendor of the system, and how those systems have been developed – in particular, the data on which they were trained and any bias testing or auditing that has been conducted.

The contract with the recruitment agency should also include appropriate warranties and indemnities against any claims arising from discriminatory screening practices.

Conclusion

The EEA imposes a strict prohibition on unfair discrimination, both direct and indirect, and extends its protection to applicants for employment. AI systems, by their nature, are capable of producing indirectly discriminatory outcomes, particularly where they are trained on historical data that reflects existing demographic imbalances. Employers deploying AI in HR must be alive to these risks, must audit their AI tools for bias, and must ensure that they can explain and justify the outcomes produced by such systems. The ‘black box’ defence is unlikely to succeed before the Commission for Conciliation, Mediation and Arbitration or the Labour Court. It is imperative that humans remain in the decision-making process and that decisions are not left solely to AI systems.

//By Melissa Cogger and Talita Laubscher, Partners, Bowmans