Tools that incorporate algorithms usingartificial intelligence (AI) have gradually made their way into every stage of the recruitment process. As early as 2018, 64% of the 9,000 recruiters surveyed in an online study reported using them sometimes or often in their work. 76% believed this technology would have a significant impact. A more recent survey even suggests a link between AI and performance: 22% of the highest-performing companies use “predictive” or “augmented” recruitment, compared to 6% of the lowest-performing organizations.
Is there cause for excitement? The promises made by market players are significant: time savings, more accurate profiling, the elimination of stereotypes… Beyond that, however, numerous technical, ethical, and legal questions arise.
To address this, a draft European Union regulation (“AI-ACT”) is currently being developed. It classifies systems intended for use in the recruitment or selection of individuals as “high-risk AI systems”—those that have potential impacts on fundamental rights. Specific rules regarding detailed disclosure, prior compliance, and regular audits are planned for these systems.
We want to use technology more! So we want to build trust in technology. And if technology poses risks, we want to identify those risks and address them. That way, we can take full advantage of all the other opportunities that technology offers. Our #AI legal proposal in a few seconds here ⬇️ pic.twitter.com/XvChlTwloL
— Margrethe Vestager (@vestager) April 22, 2021
Although French labor law and European legislation set forth general rules aimed at protecting job applicants, a European regulation on AI appears to be essential. The current reality is that there is a significant gap between the numerous promises of efficiency and objectivity made by these tools and the scarce scientific studies addressing these nonetheless fundamental issues. In particular, their ability to reduce discrimination has yet to be fully demonstrated.
AI-powered solutions now cover every stage of the recruitment process, each offering potential benefits to the organizations that implement them: economic benefits by making the process of selecting future employees faster and more efficient (some developers of these recruitment solutions even claim to cut the time needed to finalize a hire by three-quarters ); technical benefits, such as the ability to process large volumes of information and rank candidates according to desired criteria; and ethical benefits, such as avoiding the stereotypes that human recruiters may fall back on when reviewing a resume or cover letter.
When searching for candidates—a phase known as “sourcing”—the automated collection of online information about potential candidates (known as “web scraping”) is presented as a way to better align the hiring company’s needs with the candidates’ profiles. Analytical algorithms search for data found on resumes as well as information gathered from social media, which is supposed to help identify certain personality traits or specific skills in potential candidates.
During the initial screening of applications, chatbots—such as Randy, the conversational AI developed by Randstad—offer personalized assessments and guide candidates toward the most suitable roles. This is expected to improve the candidate experience, particularly by reducing stress and making the process more engaging. For the company, this presents an opportunity to redirect recruiters’ efforts toward more qualitative and complex tasks by freeing them from highly time-consuming steps.
As for the interview phase, automated video analysis tools—which candidates can sometimes use on their own—are rapidly evolving. For example, the American company HireVue offers to evaluate responses based on facial expressions and body language. The Swiss company Cryfe, meanwhile, analyzes a person’s “authenticity” by studying their verbal cues and body language.
All of this is supposedly done without relying on stereotypes related to a candidate’s physical appearance or language—and therefore without discrimination. At every stage of the hiring process, the proponents of these solutions promise companies that use them a more efficient, faster, and more inclusive hiring process.
Certain warning signs, however, should not be ignored. For example, several studies have shown that, far from reducing discriminatory biases, some predictive recruitment tools may actually create new biases in judgment.
Developers can incorporate their own biases as early as the tool’s programming stage. Algorithms that link facial expressions, personality traits, and competence, among other things, are based on questionable assumptions. Several studies conclude that decoding emotions is, on the one hand, highly complex and, on the other hand, culturally dependent. The error rate for recognizing an expression can thus range from 1% for a white man to 35% for a Black woman.
For so-called “machine learning” algorithms, which rely on data to train and adapt, discrimination can easily be perpetuated. Training datasets may be incomplete and biased, causing the resulting tools to perform poorly and even discriminate against minority groups.
The most famous example is thatof Amazon, which had to stop using an automated resume-screening tool in 2018. The tool systematically discriminated against women applying for technical or web developer positions based on hiring patterns from 2004 to 2014, which had favored men.
Amazon's sexist hiring algorithm might still be better than a human https://t.co/4qsU7gte22
— The Conversation (@ConversationUK) November 2, 2018
When it comes to tests that claim to be neutral, the “stereotype threat” is never far away. This is a psychological effect whereby, when faced with certain test situations, an individual may feel as though they are being judged through a negative stereotype directed at their group, which can cause stress and a decline in performance. For example, when a woman takes a math test, her score may be affected by stress caused by the internalized belief that women have inferior abilities to men in this subject.
Taking tests with a chatbot—which is supposed to be more fun and therefore less stressful for a candidate—could be a negative experience for certain groups of candidates. Taking tests with a chatbot—which is supposed to be more fun and therefore less stressful for a candidate—could nevertheless be a negative experience for certain groups of candidates. This is particularly true for candidates who are less familiar with digital and virtual environments; they may perform less well when faced with a digital selection process, due to negative generational stereotypes (as well as a lack of experience using this type of tool and the fear of performing worse than younger generations…).
The algorithm can also generate errors on its own by relying on misleading correlations caused by confounding variables. For example, playing golf may be an overrepresented hobby in the profiles of employees in executive positions. However, the association between this sport and job performance is by no means relevant. The worst part is that it is sometimes difficult to understand and identify the reasoning behind certaindeep learning algorithms due to the complexity of the process. In this case, we refer to a “black box” model.
Caution is therefore essential. Experts who have been working on artificial intelligence for many years sometimes even refer to “artificial incompetence” instead of artificial intelligence. Currently, tasks seem, for the most part, to be allocated with a certain degree of modesty in practice: prioritizing human intervention during the final decision-making phase, and considering the use of AI as a tool for preselection and as a decision-making aid.
[Nearly 80,000 readers rely on The Conversation’s newsletter to better understand the world’s major issues. Subscribe today]
However, the temptation to give in to the allure of algorithms is strong. As we show in a recently published study, recruiters do say they trust the recommendations of their peers more. In reality, though, they tend to follow the recommendations provided by a pre-screening algorithm more than those of their colleagues. This is true even when the algorithm suggests selecting the least qualified candidate.
Our observations therefore call for extreme vigilance: if recruiters blindly follow recommendations—even erroneous ones—provided by tools that lack transparency and explainability, the legal and reputational risks are significant for a company using these tools, particularly in cases of proven discrimination. The new regulations initiated by the European Union, which are expected to be passed this year, therefore appear to be entirely relevant.
An inexplicable algorithm is, in principle, an unacceptable algorithm. An explainable AI should follow three principles : the transparency the data used to create the model; theinterpretability, the ability to produce results that a user can understand; and theexplainability, the ability to understand the mechanisms that led to this result, along with the potential biases they entail. No doubt anticipating future challenges and changes in the legal framework, some companies are already offering adjustments that incorporate “explainable”—or, as they are also called, “transparent”—AI, if necessary, by practicing a form of affirmative action.
