A first-of-its-kind law in New York City aims to make the use of AI in hiring and promotion both clearer and fairer.
New York鈥檚 , which goes into effect on 5 July, requires employers to get an independent audit of their automated employment decision tools to ensure that they do not demonstrate significant bias based on sex, race or ethnicity 鈥� though it does not cover discrimination based on factors such as disability or age.
Bias in tools that rely on artificial intelligence is a well-known problem, given that these tools can replicate the human biases built into the data sets they are trained on. This is of particular concern in fair hiring practices now that millions of job applications are automatically screened by AI systems and other automated software before a human ever sets eyes on them.
Advertisement
New York is the first major US city to start requiring employers to show that such automated tools are not discriminating illegally against applicants or employees. Most employers now use automated tools of some kind for screening or ranking job applicants, and some estimates suggest that up to do so.
The law applies to any automated tool that 鈥渟ubstantially assists with an employment decision鈥� throughout the hiring or promotion process and 鈥渘ot just the final hiring or promotion decision鈥�, says a spokesperson for the NYC Department of Consumer and Worker Protection. Employers must publish the audit results on their websites and notify job applicants or employees ahead of time if such tools will be used. If the city agency finds an employer to have broken the law starting from 5 July 2023, it can charge $500 for first violations and up to $1500 for any additional violations.
But critics say that these safeguards have already been weakened by corporate pressure on city council members and the city鈥檚 enforcement agency since the time the law was first proposed and passed in 2021.
鈥淲hen the bill was first proposed in the New York City Council, it was vague, but at least it held out the possibility that companies would have to carry out pretty rigorous checks to make sure that their tools weren鈥檛 biased,鈥� says at the Center for Democracy & Technology, a non-profit based in Washington DC. 鈥淏ut the ordinance got watered down.鈥�
The law鈥檚 are so narrow that they appear to only cover companies that rely upon AI or algorithms without any human oversight 鈥� making it potentially easy for companies to claim they do not need to comply. 鈥淣o company is going to go out there and admit that they鈥檝e delegated decision-making authority on hiring to a machine,鈥� says Scherer.
This legal loophole is a problem because many human resources recruiters generally accept machine recommendations, say Scherer. That means the AI-powered employment decisions become the default even when there is technically human oversight.
The law also does not cover 鈥渙ld-school automation tools鈥� used in employment decision-making, such as simple keyword search algorithms, that can also include biases that filter out certain groups of people, says , chief data science officer at Harver, a company that offers automated tools based on AI for the hiring process.
Still, Polli supports the law鈥檚 push to make tech companies prove their AI-powered tools are unbiased. 鈥淚t was a bit disconcerting to see lots of other folks making such claims without producing any evidence,鈥� she says.
Harver plans to continue getting its products independently audited for the sake of employers using its tools, even if those employers are not technically required to abide by the law.
In principle, the NYC law shouldn鈥檛 even be necessary. It is merely asking employers to show that their automated tools comply with existing federal and state laws that prohibit discrimination in making employment decisions. 鈥淚t shouldn鈥檛 be a big surprise if employers have been doing what they鈥檙e supposed to do,鈥� says at Fortney Scott, a law firm in Washington DC.
But Beecher also described how the choice to be more transparent may leave employers vulnerable to lawsuits based on federal or state laws if the bias audits reveal problems. 鈥淐learly [an employer] does not want to post something and say, 鈥榖y the way, we have this biased AI we鈥檝e been using for the past year,鈥欌�� says Beecher.
Shortcomings aside, the NYC law reflects a broader push by state lawmakers to regulate how AI is used in employment screening and decisions. The US government鈥檚 has also begun issuing guidance warning companies against using automated tools in ways that violate existing federal anti-discrimination laws.
鈥淭his is just the beginning of AI regulation,鈥� says Beecher. 鈥淣ew York City was kind of out of the gate first, but I think you鈥檙e going to see a lot more regulatory activity.鈥�
