What you need to know about facial recognition technology

For the state, face recognition is an important part of the security system and an impressive budget item. For journalists, it is either a panacea or an instrument of a world conspiracy. For business, a tool or a product. Whichever side you take, the basic questions still remain. Users habitually search for answers to them on the Internet (on average 28,704 face recognition queries per month), but they do not always find them. Correcting the situation.

Face recognition is a popular request of Internet users

What is face recognition

Let's separate the flies from the cutlets. Users are more likely to face face recognition in their own smartphones, where biometric identification is used to unlock the device and only its owner could access the data. A 3D camera is necessarily involved in the recognition process so that it is impossible to deceive the gadget with a photograph.

There is also identification of faces in real time and in real conditions: in this case, it is inextricably linked with video surveillance systems, where faces are literally "snatched" from the video stream filmed by cameras.

Imagine a high-quality modern CCTV camera placed just above the average human height in a well-lit place. Approximately the same number of approximately the same people pass in front of her every day. They do not move very quickly.

The captured video can be stored in the cloud archive. An analytical module is connected to the camera: a complex combination of algorithms (artificial intelligence, neural networks, that's all) plus a user interface. The module "snatches" faces from the video stream, determines gender and age, and enters the data into the database.

Gradually there are more images. The system remembers all recognized faces automatically and records them in the archive, and a user with admission indicates additional data: name, position, status, other marks ("VIP-guest" or "thief"). You can upload a photo of the required person, and the module will find all the detections of this person in the archive.

As soon as a person with a mark passes in front of the camera again, the system records this as an important event and sends a push notification to interested users.

Detection in the context of face recognition is a situation when the algorithm, in principle, understood that it was a face, and not an apple or a mermaid from a Starbucks mug. He first needs computing power for this, and only then can he match the face to the base or remember.

Face recognition does not always work correctly

If you've read the previous few paragraphs to the end, congratulations, you now know how facial recognition works in an ideal situation. The description is suitable for any system: from those used in the Moscow metro to solutions for small businesses.

The main thing to understand is that it is difficult to create an ideal situation in real life, especially when it comes to the whole city, and not an office or a store. For example, there are a lot of people on the subway, everyone is different, they walk fast. You need a lot of cameras, they cost money, and competent specialists should place them.

Is it possible to trick the face recognition algorithm

Despite the occasional blunders, the accuracy of machine recognition is often superior to that with which people determine faces. China to build giant facial recognition database to identify any citizen within seconds will soon appear in China, a system capable of finding a specific person among 1.3 billion other residents in 3 seconds with 90% accuracy.

And yet it is difficult to answer this question unequivocally, because there is no single ideal algorithm for face recognition. Big glasses, a pasted beard, a cap, a high speed of movement, special makeup (for example, a "Black Swan" lattice painted on the face, cats, circles and sticks. How to escape from face recognition systems using makeup) - all this can confuse the algorithm. Especially in the aggregate, because for recognition it is enough How to cheat recognition systems whether 70% of an open face is. Now imagine that it is necessary to use the above tricks in a real city. Doesn't sound so easy, right?

"Anti-recognition" glasses from Japan, which back in 2015

And here is such a 3D mask in 2014

Is it possible to recognize faces online

The Internet is a paradoxical place: people here can simultaneously worry about whether every second camera on the street detects their personality, and sincerely want to "recognize other people's faces from their photos online." Let's consider this face recognition trend separately.

The face recognition program is either the analytical module described above (CCTV camera + software + cloud storage), or software similar to the well-known (slightly scandalous) FindFace service. Today, it is, of course, impossible to download a face recognition program “for free and without registration” in the vast majority of cases.

The FindFace.ru web service, which helps to find people on the VKontakte social network by their photographs, was founded on February 18, 2016. Among other things, thanks to him, everyone could find profiles of girls who starred in porn films. Very soon, the service began to be used for many flash mobs to detect faces, which had every right to never be detected by anyone. A scandal erupted, which worked like a viral advertisement: the technology that formed the basis of the service received a number of prestigious awards and aroused the interest of customers from the state and business. Since September 1, 2018, the service no longer provides the FindFace Service, which was used to recognize protesters, announced the closure of the search for people by photo service, as it was transformed by NtechLab into a line of solutions for various business sectors.

The dream of the user who enters the request, obviously, looks like this: you go to the site, upload a photo of a person who was taken stealthily in the subway, the program recognizes the face and gives out a link to the profile on the social network. Yeah, got caught! Or like this: you download the program to your computer, connect your webcam to it and recognize the face of your cat. Success - now you will receive a notification every time the cat steals sausages.

Reality is cruel. The first site that offers you something like that refuses to work, and the second one requires programming skills in Python. More or less a dream-like application called SearchFace, which was recently restarted Searchface was restarted with authorization through VKontakte. But the social network has closed this feature called FindClone. You uploaded a photo, and the algorithm tried to recognize the same face in the VKontakte social network database. The application did not give out links to the profile, only the pictures themselves - and it doesn't matter who they were uploaded by. If a user has been active on a social network for a long time, the issuance of a photo created an eerie "biographical" effect, but if not, the recognized images could make them laugh.

Is it possible to recognize faces online

Actually, the SearchFace example clearly answers the question "How do social networks use face recognition?" It would be more accurate to formulate it this way: "How are social networks used for face recognition?" The answer is simple: like a database. An innumerable number of unique combinations of numbers (this is how the faces in the photo look for the algorithms of Facebook, VKontakte and others) form the basis for training neural networks that form the basis of one or another face recognition solution.

The solutions are all different, and neural networks are also different, and customers and service providers, as a rule, do not disclose details and technical features. In particular, the recognition module is able to determine gender and age due to the fact that it can learn from the information contained in Odnoklassniki, VKontakte, Instagram and Facebook.

How face recognition is programmed

You should never answer developer and developer questions if you are not a developer. Therefore, we turned to a specialist for help.

Dmitry Soshnikov Member of the Russian Association for Artificial Intelligence and senior expert in the development of AI and machine learning systems at Microsoft.

Face recognition (as well as other related operations) is a fairly common task. Therefore, many companies provide ready-made services in the form of cloud APIs (software intermediaries between applications) for a high-quality solution of these tasks. In addition to IT giants like Microsoft and Google, specialized companies, including Russian ones, are also engaged in facial recognition. Their products are evolving rapidly and provide even more exciting features such as identifying faces and silhouettes in crowds.

It is much more difficult to train a neural network from scratch. We need a large and high-quality set of initial data, that is, tens and hundreds of thousands (or even more!) Photographs of people. In addition, significant computational resources and knowledge of AI and machine learning will be required. Large companies have all these tools at their disposal, so they solve the problem much better.

There is also an intermediate solution - to use an already trained neural network, for example. This option, most likely, will work a little worse than a ready-made cloud service, but it will allow you to have full control over the system. This will require a certain level of understanding of the operation of neural networks and neural network frameworks and, most likely, some knowledge of the Python language, which has gained popularity as the main programming language among Data Science specialists.

Indeed, it is convenient to carry out various experiments, visualize data and perform efficient matrix calculations thanks to the excellent NumPy package. This is not the best language for industrial development, since it does not contain effective tools for creating large secure software systems, but there are no alternatives to it in the field of deep neural network training yet.

How facial recognition works in business

The demand for face recognition in fintech, retail and other types of business is directly related to the increased availability of technology. The mechanics are simple: all enterprises and all organizations have CCTV cameras, which are used as tools for data collection and subsequent analytics. In the world, surveillance systems shoot terabytes of video in Full HD per month, that is, there is really a lot of information for processing.

The required software for data analysis can be “flashed” onto the device by the manufacturer. On-board video analytics cameras are usually quite expensive.

An alternative option is analytics in the cloud, that is, a remote data center that connects to any inexpensive camera. This is an order of magnitude cheaper, plus it gives flexibility - you can adapt solutions for a specific business.

The popularity of face recognition technology in various fields of activity is increasing. For example, Sberbank is one of the leaders in announcing various high-profile face recognition projects, and it can argue that He recognizes you out of a thousand: the ATM will identify the client by the eyes with him in this regard, perhaps only Tinkoff. In 2017, Sberbank acquired Sberbank and invested 25.07% of VisionLabs in face recognition technology, which creates software for face recognition. In 2018, a financial institution managed to test face recognition in the Moscow metro and even catch 42 criminals 42 criminals were caught thanks to the Sberbank face recognition system, to test It will recognize you from a thousand: an ATM will identify a client by the eyes of ATMs with face identification so that attackers cannot withdraw money from other people's cards, as well as announce the collection of biometric data (audio recording of a voice,video of the face) of clients. In April this year, Sberbank control over the developer of voice and face recognition systems - the "Center for Speech Technologies" (MDT).

Another thing is that announcing, testing, piloting and buying solutions does not mean actually implementing. What exactly is now actually being used in Sberbank (and whether it is used), in fact, can only be said with certainty by German Gref.

With retail, everything is more transparent. Basically, there are three problems here that face recognition solves.

First, theft. The shops are run by scammers, and often the same people in the same network. Face recognition allows you to identify "drifting thieves" and other people who previously violated the order. As soon as the intruder entered the database once enters the store, the security will receive a notification in the messenger or in another convenient way.

Secondly, the difficulty of working with regular customers. There is simply not enough data on purchases and birthdays to personalize offers for VIPs and brand fans. Face recognition can be integrated with CRM - that is, software in which managers enter all the information on all transactions of the organization. In the case of thieves and VIPs, face recognition works in about the same way: the face is entered into a black or white list, and when it reappears, the system will beep to the person with access. Gender and age are determined automatically, and additional information will be added by the responsible employee.

Third, retail identification is used for targeted advertising. For example, some X5 Retail Group stores have installed the X5, which will include computer vision cameras to recognize facial expressions and age of customers. By analyzing this data, the system displays goods that a person may like on the monitor screen in the trading floor. Another vivid illustration is the case of Lolli & Pops, a large confectionery store in the United States. The face recognition system determines Your future in-store loyalty program will be fed by facial recognition of regular customers and sends notifications to their smartphones with products that they may like (taking into account individual preferences and even food allergies).

Another striking example of the use of technology in retail is stores without sellers and cash registers. For example, Alibaba Tao Cafe Amazon Go vs Alibaba Tao Cafe: Staffless Shop Showdown is a cafe and self-service store located in Hangzhou. It sells drinks, snacks, groceries, toys, backpacks and the like. Tao Cafe is only open to users of the Taobao website.

When buying drinks, a camera system with facial recognition support automatically identifies the customer, connects to his account in the online store and processes the payment. Shoppers exit through a space equipped with multiple sensors that identify both the customer and the goods. Scanning works even if the person puts the purchase in a pocket or bag.

How is facial recognition technology evolving

Face ID CCTV systems are truly taking over the world. In Moscow, the number of cameras in 2019 will reach High technologies and security: how many CCTV cameras will appear this year 174 thousand. This does not mean that all these devices by default can recognize a person: most often it is reported that the system for recognizing wanted criminals through video cameras will start working in Moscow in 2019 about 160 thousand cameras with this function. Nevertheless, at the end of 2018, the Moscow mayor's office announced the intention of the Moscow authorities in 2019, they are going to replace video cameras and launch a face recognition system to replace all video surveillance devices and form a completely innovative system next year.

The paradox is that 160 thousand is not that much. Especially when compared with another leader in search engine queries on the subject of face recognition - China. At the end of 2017 there was In Your Face: China's all-seeing state over 170 million CCTV cameras and over the next three years China's 'Big Brother' surveillance technology isn't nearly as all-seeing as the government wants you to think to connect to the network is still about 400 million.

Competent and correct use of face recognition works primarily to improve safety and comfort. People usually quickly gain confidence in technology that saves them from queuing for a football match (smiles at the camera - passed), prevents theft and hooliganism, or helps them spend less on purchases (loyalty programs). All this, of course, requires certain regulation - this is why laws on the protection of personal data are being adopted.

In the future, it is likely that the field of face recognition in video surveillance systems will be regulated in a similar way to the current practice of working with facial identification on the Internet. Privacy-minded people simply don’t upload too much on the Web - the partial fiasco of the SearchFace service proves that such a strategy is effective.

Of course, you cannot endlessly limit yourself to walking along the streets, where cameras are installed at every intersection, but the ability to preserve anonymity will be formed if there is a corresponding request from society.