Facial Expression Detection Using AI: How It Can Improve the Hiring Process

By Danni White - Published on November 13, 2018
Facial Expression Detection Using AI: How It Can Improve the Hiring Process

One of the most interesting fields in the study of human-computer interaction is that of emotion recognition through the use of facial expression detection. Facial recognition technology is already an established system used to identify a person by analyzing their face from a digital image or video frame.

Facial recognition systems are inextricably linked with artificial intelligence technology. One of the ways that artificial intelligence is used in facial recognition is in facial expression recognition. And one of the ways in which artificial intelligence is providing facial emotion recognition is in job interviews.

AI is used to analyze the facial expressions of candidates during the hiring process in order to evaluate whether their personality is the right fit for the job, as well as determining the honesty of their answers. Some of the employers making use of this technology in their hiring process include the Boston Red Sox, Carnival Cruise Lines, Dunkin Donuts, IBM, and Unilever.

Examples of AI Usage in Hiring Process

In the case of Unilever, applicants often discover job opportunities through LinkedIn or Facebook and either upload their resumes or present their profiles on LinkedIn. Then they play several games based on neuroscience that are intended to assess their personalities to see if they will be a good fit for the job. After playing those games, they take part in a video interview with preset questions. In these interviews, artificial intelligence is used to detect and judge their facial expressions and personality traits. This facial expression software is provided by HireVue. The program selects the people it determines to be the best matches and notes what the artificial intelligence saw in the candidate and then sends this data to a human recruiter.

This approach to the hiring process has some benefits, both for the employer and the potential employee. First, it makes the process more convenient for everyone involved since candidates can take the interview whenever is best for them, and recruiters can review the data whenever they have time on their schedule. Second, properly programmed artificial intelligence technology does not have the unconscious biases that human recruiters may have. Third, it saves a lot of time for both the employer and the job applicants. Fourth, this easy-to-access approach greatly expands the recruitment pool, meaning that companies are able to grab hold of the best candidates, regardless of where they come from or what their socioeconomic status is.

The algorithms used in facial expression detection technology can analyze images of a job candidate and use data to indicate the current attitude of the person, or even figure out whether they might be given to a life of crime. The use of face recognition technology can greatly aid hiring managers in dealing with the tedious process of resume screening. This software can handle initial candidate screening on personality and job fitness, thus saving time and sparing the recruiter mental stress as he or she doesn’t have to analyze personal information about the candidate.

Personality screening is probably the most important advantage that facial expression software can provide to recruiters and hiring managers. Initial screening can be performed based on the data provided such as whether the person is anxious, excited, disinterested, interested, stressed, or relaxed. Software for facial emotion recognition can consider small movements around the eye or mouth and match these with a database full of character traits. This data will give recruiters an indication of whether the applicants are really who they seem to be, and also whether they and their personality will be the right fit for the culture of the company. Facial recognition software can be used to capture the facial expressions of current employees and compare candidates with them to see if their collective personalities will largely match.

However, there are some negative aspects and potential downsides to the use of facial expression detection. First, it can’t be denied that it’s at least a little bit disturbing that computer algorithms are analyzing candidates’ facial expressions, especially since those candidates may not be told that such a thing is happening. If people do know that artificial intelligence is analyzing their facial expressions to determine their fitness for a job, they may display self-consciousness and act differently than they otherwise would.

Second, the attitude of a human can change very easily for many different reasons. Simply because a person’s current mood may be unhappy, slightly angry, or otherwise perturbed does not mean that they are unfit for a job or that that is their constant demeanor. Human interviewers can take negative circumstances into account, but artificial intelligence may screen out potential candidates before that can take place.

Third, physical appearance issues could affect the way artificial intelligence assesses facial expressions. Facial tattoos, scars, Botox injections, and heavy or misapplied makeup can make technology view a person differently. A human interviewer can see these things and take them into account, however, a computer may simply screen that person out of the talent pool.

Facial expression detection using artificial intelligence could be very beneficial to people on both sides of the hiring process. However, companies using such software (and the makers of such software) will have to grapple with the concerns mentioned above.

Danni White | Danni White is the CEO of DW Creative Consulting Agency, a digital marketing firm specializing in elevating the visibility of small-to-midsize businesses and nonprofits. She is the author of 17 books and hosts the #Hashtags and Habits Podcast, which merges digital marketing, entrepreneurship, and personal growth.

Danni White | Danni White is the CEO of DW Creative Consulting Agency, a digital marketing firm specializing in elevating the visibility of small-to-midsize busi...

Related Posts.