Facial recognition firms got more than they bargained for when COVID-19 struck. While the technology to identify faces when wearing coverings like the medical face masks we see today has certainly existed, it was likely an outlier as opposed to an urgently essential feature of the software. Fast-forward to 2020 when mask-wearing has literally become the norm, and these tech companies realize that they must play catch-up or lose their business.
Fortunately for them, advancements in this field are ramping up.
According to new research by the US National Institute for Standards and Technology (NIST) released this week, facial recognition software has improved significantly this year in identifying faces, even when the individuals in question are wearing masks.
As The Verge explains, vendors, or the companies developing facial recognition software, “submit their facial recognition algorithms to NIST for testing as part of the Facial Recognition Vendor Test (FRVT).”
NIST then tests all of the submitted algorithms and tries to compile the findings to draw conclusions from them. The Institute has evaluated 152 algorithms so far (submitted both prior to and after mid-March 2020) based on the analysis of 6.2 million photographs. 65 new algorithms were provided to NIST since mid-March 2020.
The study focuses on false non-match rates (FNMR), which in biometrics concerns the failure to match two identical datasets – in this case, the same individual in 2 pictures, where they wear a mask in one but not the other. It is up to facial algorithms to be able to identify the person in the mask based on what they know about them in their databases.
“The initial results [of NIST’s research], published in July and August, showed that masks were thwarting facial recognition algorithms and increasing error rates by up to 99% in some cases,” CNET explained. “The error rates increased for every algorithm once researchers added masks to the test photos, even for facial recognition that was designed specifically for the coverings.”
“The error rates are still higher once a mask is factored in – jumping from 0.3% without masks to about 5% with masks,’ CNET continued. “Still, the NIST study said there was a ‘notable reduction in error rates’ compared to algorithms submitted before the pandemic.”
NIST said: “For some developers, false rejection rates in their algorithms submitted since mid-March 2020 decreased by as much as a factor of 10 over their pre-pandemic algorithms, which is evidence that some providers are adapting their algorithms to handle face masks.
“However, in the best cases, when comparing results for unmasked images to masked images, false rejection rates have increased from 0.3%-0.5% (unmasked) to 2.4%-5% (masked).”
Notably, the research paper states that the current performance of face recognition with face masks is comparable to the state-of-the-art on unmasked images in mid-2017, which shows a significant improvement.
However, the study does point to some limitations governing their findings. The images analyzed by NIST were originally photos used in government applications or taken at (land or air) borders where a crossing individual would be photographed. Masks were digitally added to these photos “given time and resource constraints,” which as a result prevents “an exhaustive simulation of the endless variations in color, design, shape, texture, bands, and ways masks can be worn.”
You can find the full report here.