Multiverse 2.357

Facial recognition systems measure and match facial feature patterns to identify people. The only thing required in terms of sensors is a camera, so it is well suited for use with mobile phones and CCTV cameras. This technology works well with crowds of people as well as with individuals, so it is not only used by apps and devices to identify the user, but also being used in concerts, sporting events and in airport terminals.

In the previous post, an introduction to biometric identification systems we mentioned the complexity of evaluating different use cases for these technoogiues, and discussed the problems around error rates. We also mentioned inherent bias showing up in facial recognition systems, and now we get to dive in a bit deeper.

Facial recognition systems have been shown to be amenable to spoofing as well. The Consumer Association of the Netherlands conducted tests and found they could unlock more than one third of the smartphones they tested using photographs. It’s worth noting that all nine Apple iPhones tested by the Dutch association passed.

More recently phone manufacturers have developed three dimensional algorithms that look for depth and cannot easily be fooled by photographs. Recent research suggests however, that by stitching together a group of photos, a 3D model can be constructed that will fool these. In fact, one Forbes reporter used a model of his head and was able to unlock four out of five phones he tested 6 – every phone except Apple iPhone X.

Fooling facial recognition systems can sometimes be accomplished by obscuring parts of the face to convince the algorithm that there is no face. Ordinary makeup has been used to this effect, and wearing strong or strategically placed light sources on the face can saturate the camera. Faces are easily photographed in public, and for that reason should never be used as biometric identifiers.

Facebook has long been seen as the leader in facial recognition technology, and has been aggressively pursuing it despite a history of friction with the EU over user consent. They have developed Augmented Reality technology to allow potential customers to try on cosmetics and sunglasses in their Instagram stores. Facebook has also been busy pairing this technology with their facial recognition system for physical store operators to use, as authentication and payment systems.

Amazon has been selling their technology to law enforcement agencies, drawing criticism from many and putting pressure on those like Microsoft who are advocating for regulation to prevent abuse. According to Microsoft CEO Satya Nadella, “This use of facial recognition technology could unleash mass surveillance on an unprecedental scale.”

Meanwhile, Amazon has also been building this capability into their smart home product lines seemingly intending to compete with Google in the category. NIST evaluated facial recognition systems in 2018 from 39 companies – notably not including products from Amazon, Google and IBM. Microsoft and China’s Yitu performed the best in what many experts call the gold standard in benchmarking. This testing is performed every four years, and the big takeaway this time was that dramatic improvement was noted, much of which was credited to the use of a special type of neural network.

Despite plentiful reasons for concern, facial recognition systems are being rolled out rapidly. This year they began to be deployed in US airports without warning or discussion, and caused concern over the lack of care taken to safeguard our personal data. This is part of a push by the current administration to put these systems in every major US airport by 2021. Of course concerned people and experts alike worry about the secure use and storage of this personal data, in light of incidents like a leak earlier this year of millions of facial recognition records in China.

We’ll take a look at specific systems, how they’re used and how they can be defeated in more detail later, but in the next post we’re going to examine some other popular biometric identification systems such as iris scans, fingerprint readers, and vein mapping technology – stay tuned!