1
2 Comments

Are the following use cases safe uses of LLM in hiring?

I would like to have the opinions of the community here on this. As per the current draft of EU AI Act, this is mostly a high-risk use case, but these are permitted subject to regulations. If anybody knows actionable steps to do testing also let me know.

Currently, a lot of candidates are rejected due to automation like keyword matching. So we thought of using ChatGPT API to mine resumes for contextual results.

I am asking this question because I see "Automated determinations of eligibility for credit, employment, educational institutions, or public assistance services" as a disallowed usage of OpenAI technology and I agree to it fully. Still, are the following use cases legit and allowed?

  1. TAs will give criteria like "Candidate should have worked with Azure DataBricks and Spark", "Candidates must be using Java for more than 4 years", and "Candidate should be currently in a mid-senior manager role" and we feed this and an anonymized resume to ChatGPT API and ask it to say "yes, the candidate has bla bla...", "no, the candidate has bla bla..." in a json format.

  2. There will be open-ended questions like "Describe a time you had a challenge related to backend development. How did you approach it?" and criteria like "Looking for a candidate's ability to break down problems into smaller problems" and ask it to assign a score out of 10 and reason.

In both the above use-cases

  1. we are sending no PII to reduce any bias due to that, we prefer not sending even the gender. In the second use case, chatgpt doesn't even know about the person involved in answering the question.

  2. No hiring decision is made automatically, the scores are objective and the answers are disclosed with the possibility of human override. It is up to the TAs to decide based on the information available who to go for an interview with.

Let me know, how safe you think these approaches are and let me know if you have any ideas to make it more safe.

on May 12, 2023
  1. 1

    Unless I misunderstood something, the issue is not anonymity or lack thereof, but the very idea of decision-making by AI, or steps towards making that decision. In your #2, for instance, you would rely on AI to assign a score based on answers to open-ended questions. If I am a government bureaucrat interpreting this regulation literally, then to me it would definitely fall under "Automated determinations of eligibility for credit, employment, educational institutions, or public assistance services".

    1. 1

      Right, thank you for sharing your views. The data anonymity that we are doing is to reduce the bias of the AI towards a particular gender, ethnicity etc. For example, if an AI's job is to check if an answer is correct with respect to a question and matches specific criteria, it doesn't need to know who answered that question, secondly, for mining information like if a candidate has used XYZ technology, again it should not know who the person is, their gender or anything that is PII.