CN114511909A - Face brushing payment intention identification method, device and equipment - Google Patents

Abstract

The embodiment of the specification discloses a method, a device and equipment for recognizing willingness to pay by brushing face. The scheme comprises the following steps: acquiring a face brushing image, and determining a candidate to be identified in the face brushing image; according to the located area of each candidate in the face brushing image, respectively generating corresponding mask images to distinguish the located area from other areas in the face brushing image; extracting the features of the face brushing image, and obtaining fusion features according to the features of the face brushing image and the mask image; and identifying whether each candidate has a face brushing willingness to pay according to the fusion characteristics. The safety of the face brushing payment can be improved.

Description

A method, device and device for recognizing willingness to pay for face-scanning

technical field

The present specification relates to the field of computer technology, and in particular, to a method, device, and device for recognizing the willingness to pay for face-scanning.

Background technique

With the development of computer and Internet technology, many businesses can be conducted online, which has promoted the development of various online business platforms. Among them, face-swiping payment refers to a new payment method based on artificial intelligence, machine vision, 3D sensing, big data and other technologies. By using face recognition as a payment method for identity verification, it brings great convenience to users. , which is generally favored by users.

At present, in the face-swiping payment scenario, after the paying user opens the face-swiping payment, he needs to stand in front of the device with the face-swiping payment function to perform face recognition. However, during the face-swiping process, there may be multiple users standing in front of the device, which will cause multiple users to appear in the face-swiping image collected by the device. At this time, when the device performs face recognition on the face-swiping image, it is difficult to determine which user is the current user to be paid, that is, which user has the willingness to pay for face-swiping. In other words, only the current user to be paid has the willingness to pay, while other users do not have the willingness to pay.

Based on this, face-swiping willingness to pay recognition is an important part of face-swiping security in the payment system, which helps to improve the face-swiping security experience. However, if the device recognizes other users and recognizes other users, there will be false face-swiping. payment, thereby reducing the security of face payment.

Based on this, a more secure identification scheme is required for face-scanning payment.

SUMMARY OF THE INVENTION

One or more embodiments of this specification provide a method, device, device, and storage medium for recognizing a willingness to pay by face-swiping, so as to solve the following technical problem: a more secure identification solution is required for face-swiping payment.

To solve the above technical problems, one or more embodiments of the present specification are implemented as follows:

One or more embodiments of this specification provide a method for recognizing the willingness to pay for face-scanning, including:

Obtaining a face brushing image, and determining a candidate to be identified in the brushing face image;

According to the area where each candidate is located in the face brushing image, a corresponding mask map is respectively generated to distinguish the located area from other areas in the face brushing image;

Extracting the features of the face brushing image, and obtaining fusion features according to the characteristics of the face brushing image and the mask map;

According to the fusion feature, identify whether each candidate has a willingness to pay for face-scanning.

One or more embodiments of the present specification provide a device for recognizing a willingness to pay for face-scanning, including:

an acquisition module, acquires a face brushing image, and determines a candidate to be identified in the brushing face image;

The generation module, according to the area where each candidate is located in the face brushing image, respectively generates a corresponding mask map to distinguish the area and other areas in the brushing face image;

an extraction module, extracting the features of the face brushing image, and obtaining fusion features according to the characteristics of the face brushing image and the mask map;

The identification module, according to the fusion feature, identifies whether each candidate has a willingness to pay for face-scanning.

One or more embodiments of this specification provide a device for recognizing the willingness to pay for face-scanning, including:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

Obtaining a face brushing image, and determining a candidate to be identified in the brushing face image;

According to the area where each candidate is located in the face brushing image, a corresponding mask map is respectively generated to distinguish the located area from other areas in the face brushing image;

Extracting the features of the face brushing image, and obtaining fusion features according to the characteristics of the face brushing image and the mask map;

According to the fusion feature, identify whether each candidate has a willingness to pay for face-scanning.

One or more embodiments of this specification provide a non-volatile computer storage medium for recognizing a face-to-pay willingness to pay, storing computer-executable instructions, and the computer-executable instructions are set to:

Obtaining a face brushing image, and determining a candidate to be identified in the brushing face image;

According to the area where each candidate is located in the face brushing image, a corresponding mask map is respectively generated to distinguish the located area from other areas in the face brushing image;

Extracting the features of the face brushing image, and obtaining fusion features according to the characteristics of the face brushing image and the mask map;

According to the fusion feature, identify whether each candidate has a willingness to pay for face-scanning.

The above-mentioned at least one technical solution adopted by one or more embodiments of this specification can achieve the following beneficial effects:

By generating corresponding mask maps for each candidate's location in the face-swiping image, the candidate's feature information can be more distinct, thereby increasing the number of candidates with and without the willingness to pay for face-swiping. Differences, through the fusion of features, it can identify whether each candidate has the willingness to pay for face brushing, and realize the enhanced image contrast effect, so as to realize the focus on the candidates with the willingness to pay for face brushing, which can be more Candidates with willingness to pay for face-scanning can be accurately distinguished from candidates who do not have the willingness to pay for face-scanning, and more targeted identification of candidates' willingness to pay for face-scanning in the face-scanning image can enhance the security experience of face-scanning and effectively guarantee the face-scanning payment. Usable security of face systems.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present specification or the prior art, the following briefly introduces the accompanying drawings required in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments described in this specification. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative labor.

FIG. 1 is a schematic flowchart of a method for recognizing a willingness to pay for face-scanning provided by one or more embodiments of this specification;

FIG. 2 is a schematic framework diagram of a face-scanning willingness-to-pay recognition system provided by one or more embodiments of the present specification;

3 is a schematic flowchart of a method for recognizing the willingness to pay for face brushing based on end-to-end learning of a deep convolutional neural network provided by one or more embodiments of this specification;

4 is a schematic structural diagram of a device for recognizing a willingness to pay for face-scanning provided by one or more embodiments of the present specification;

FIG. 5 is a schematic structural diagram of a device for recognizing a willingness to pay for face-scanning provided by one or more embodiments of the present specification.

Detailed ways

The embodiments of this specification provide a method, device, device, and storage medium for recognizing a willingness to pay for face-scanning.

In order to make those skilled in the art better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings in the embodiments of this specification. Obviously, the described The embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments of the present specification, all other embodiments obtained by persons of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present application.

FIG. 1 is a schematic flowchart of a method for recognizing the willingness to pay for face-scanning provided by one or more embodiments of the present specification. This process can be performed by an electronic device with a face-scanning payment function. The electronic device can be a terminal with an image data processing function, for example, a mobile terminal such as a mobile phone, tablet, and notebook, or a fixed terminal such as a desktop computer or a server. , some input parameters in the process or intermediate results allow manual intervention adjustments to help improve accuracy.

The flow in Figure 1 can include the following steps:

S102: Acquire a face brushing image, and determine a candidate to be identified in the face brushing image.

In one or more embodiments of this specification, after receiving the face-swiping payment instruction, the electronic device may obtain the face-swiping image through a pre-installed camera device, or the electronic device may generate the face-swiping payment instruction according to the payment order, The brush face image is acquired through the camera device. Wherein, the face brushing image may be a single frame image obtained from a video or an image.

The candidate to be identified refers to the user who needs to pay the relevant fees. It should be noted that if the candidate wants to pay by face-swiping, he needs to register his identity information on the corresponding client and enter the face information, which will be used in the candidate. When a person opens face-swiping payment, after identifying that the candidate has the willingness to recognize face-swiping payment, the candidate is authenticated through the pre-registered face information.

That is, the face information of the candidate is included in the face-swiping image, and whether the candidate has the willingness to pay for face-swiping can be obtained by identifying the face information, and then the identity of the candidate can be authenticated through the face information.

Wherein, the number of candidates in front of the camera device may be one or more, when the number of candidates is multiple, the face-scanning image includes multiple candidates, and when the number of candidates is one, the face-scanning image is A candidate is included in the image. At the same time, the face-swiping image includes not only the candidate's face information, but also other feature information of the candidate, such as torso information and limb information, and can also include other objects that do not need to be identified, such as the candidate's environment. Included tables, chairs, hanging objects, etc.

In addition, under normal circumstances, when the electronic device executes a single face-swiping payment instruction, it is to authenticate the identity of a specific candidate who has currently opened face-swiping payment, and the specific candidate usually also has the willingness to pay for face-swiping, that is to say , when executing a single face-swiping payment instruction, even if the face-swiping image includes multiple candidates, not all of the multiple candidates have the willingness to pay for face-swiping, and only a specific candidate has the willingness to pay for face-swiping, it can be considered that the Certain candidates are willing to pay safe, other candidates are willing to pay non-safe.

For example, in public places, offline Internet of Things (IoT) face-swiping devices are usually used for face-swiping payment. Among them, face-scanning IoT devices in public places refer to devices with face-scanning function set up in public consumption scenarios such as supermarkets/convenience stores/catering/hotels/campus education and medical care/campus education.

If candidate A clicks the face-swiping payment, the face-swiping payment will be enabled. The IoT face-swiping device receives the face-swiping payment instruction, and obtains the face-swiping image through the camera device. Since it is in an open public place, there are multiple candidates. In the scenario of queuing for payment, the face-swiping image obtained by the IoT device is very likely to include multiple candidates. However, among multiple candidates, only candidate A has the willingness to pay for face-swiping. At this time, it is necessary to identify A. On the premise that the candidate has the willingness to pay for face-scanning, the identity of candidate A is authenticated through the face information of candidate A, that is, other candidates do not actually have the willingness to pay for face-scanning.

Further, if candidate B ranks behind candidate A, even if candidate B does not have the willingness to pay for face-swiping, candidate A may be photographed by the camera equipment while candidate A is being photographed during the process of face-swiping payment authentication. Candidate B, as a result, both candidate A and candidate B are included in the photographed face brushing image. If the electronic device does not use candidate A as a face-swiping user in the process of recognizing the face-swiping image, but misidentifies candidate B, then the electronic device does not recognize whether candidate B has the willingness to pay for face-swiping, but directly The identity of candidate B is authenticated. After the authentication is passed, the payment will be made through the account of candidate B, thereby mistakenly brushing the assets of candidate B, resulting in the loss of candidate B's assets.

S104: According to the area where each candidate is located in the face brushing image, generate a corresponding mask map respectively to distinguish the located area from other areas in the face brushing image.

In one or more embodiments of this specification, the area may include the candidate's appearance feature information, such as face information, torso information, limb information, but in order to increase the accuracy of the recognition result, the area mainly includes Candidate's face information. At the same time, the location of the candidate can be determined according to the location information of the candidate in the face-swiping image.

Because the mask operation of the image refers to recalculating the value of each pixel in the image through the mask operator, the mask operator describes the influence of the pixel in the field on the value of the new pixel, and at the same time, according to the weight factor in the mask operator, the pixel is calculated. Weighted average, image mask operation is often used in areas such as image smoothing, edge detection, feature analysis, etc. Therefore, the mask operation can be used to distinguish the area where the candidate is located in the face brushing image and other areas in the face brushing image.

It should be noted that the area where a single candidate is located in the face brushing image corresponds to a single mask image, that is, if there are multiple candidates in the face brushing image, a corresponding image is generated for each candidate. The mask map of , and finally get multiple mask maps.

That is to say, in a single mask image, the area where you are located can be distinguished from other areas. For example, the filling value of the area where you are located is 1, and the filling value of other areas is 0 (other areas with higher discrimination can also be used). different padding values). That is, by generating a mask map corresponding to each candidate, the feature information of the candidate can be more distinct, thereby increasing the difference between those who have the willingness to pay for face-scanning and those who do not.

S106: Extract the features of the face brushing image, and obtain fusion features according to the characteristics of the face brushing image and the mask map.

In one or more embodiments of this specification, how to extract the features of the face-brushing image is not limited here, for example, the features of the face-brushing image are extracted through a feature extraction model. The features may include facial features, torso features, and limb features of each candidate. Wherein, the face feature may be the global feature of the candidate's face, and identifying the face through the global feature can improve the accuracy of the recognition result.

Of course, after the mask image is obtained, the fusion feature can be obtained by inputting the features of the face brush image and the mask image into the fusion feature extraction model.

By extracting the features of the face brushing image and combining the features of the face brushing image with the mask map, it is equivalent to adding a new channel to the characteristics of the face brushing image, that is, increasing the number of channels of the features of the face brushing image, thereby obtaining fusion. In the fusion feature, more attention is paid to the face features of the corresponding candidates in the face-swiping image, which can more accurately compare the candidates who have the willingness to pay for face-swiping and the candidates who do not have the willingness to pay for face-swiping in the face-swiping image. distinguish.

S108: According to the fusion feature, identify whether each candidate has a willingness to pay for face-scanning.

In one or more embodiments of the present specification, the face brushing image information is determined according to the fusion feature, and the face brushing image information includes feature information of the candidate. Among them, more attention is paid to the feature information of the candidate corresponding to the mask map of the fusion feature, and the feature information mainly includes the face information of the corresponding candidate. That is, when a mask map is generated, a recognition process is performed to identify whether the candidate corresponding to the mask map has the willingness to pay for face-scanning.

It should be noted that whether the corresponding candidate has the willingness to pay for face brushing can be identified in combination with the preset rules. For example, if the face area of the candidate is identified in the middle area, the candidate is considered to have the willingness to pay for face brushing. Or, if it is recognized that the face area of the candidate occupies most of the face-swiping image, it is considered that the candidate has the willingness to pay for face-swiping, or the candidate's face area is identified to occupy most of the face-swiping image, And if the face angle meets the preset angle threshold, it is considered that the candidate has the willingness to pay for face brushing.

Further, the fusion feature can be input into the willingness to pay recognition model, the image information of the face brushing can be identified through the willingness to pay recognition model, and the processing result is output according to the preset rules, and then the probability value of the willingness to pay for the face brushing can be generated according to the processing result, which can be obtained by brushing the face. The face-to-pay probability value determines whether the candidate has the willingness to pay for face-scanning.

Wherein, the processing result may be a vector generated by the willingness to pay recognition model and used to represent the probability value of the willingness to pay for brushing the face.

For example, if the probability value is greater than the preset probability threshold, it can be considered that the candidate has the willingness to pay for face-swiping, that is, the face-swiping payment instruction of the electronic device is generated after the candidate has enabled face-swiping payment, if the probability value is less than or equal to the preset probability threshold, it can be considered that the candidate does not have the willingness to pay for face recognition. That is, the face-swiping payment instruction of the electronic device is not generated after the candidate opens face-swiping payment, but is generated after other candidates open face-swiping payment.

Further, if the number of candidates whose probability value is greater than the preset probability is multiple, it means that the result of the face-scanning willingness-to-pay recognition is not credible, and the authentication failure will be prompted. Anyway, if there is no candidate with a probability value greater than the preset probability, it also means that the result of the face-scanning willingness-to-pay recognition is not credible, and the authentication failure will be prompted.

Through the method of Fig. 1, by generating corresponding mask maps for each candidate's location in the face-swiping image, the feature information of the candidate can be made more distinct, increasing the willingness to pay for face-swiping and those who do not. Differences in the willingness to pay for face-scanning. Through the fusion of features, it is possible to identify whether each candidate has the willingness to pay for face-scanning, which can enhance the image comparison effect, so as to realize the focus on candidates with the willingness to pay for face-scanning, which can be more In the face-swiping image, candidates with the willingness to pay for face-swiping are accurately distinguished from those who do not, and the willingness to pay for face-swiping of the candidates in the face-swiping image is more targeted, which can enhance the safety of face-swiping. experience.

Based on the method in FIG. 1 , the present specification also provides some specific implementations and extensions of the method, which will be described below.

In one or more embodiments of this specification, when generating the corresponding mask map,

Specifically, after the candidate is determined, extract the face area of the candidate in the face brushing image, for example, first extract the candidate's face in the face brushing image through the face extraction model, and then use the position information of the face, Determine the face area of the candidate. Then, the face area is processed to determine the face area selection frame. Among them, the face area selection frame can be displayed in various ways, such as circular frame, rectangular frame, irregular polygon frame, etc. However, there is a precondition, in order to ensure the accuracy of the recognition result, the face area selection frame must be All encompass the face area of the candidate.

After the face region selection box is obtained, the first filling region of the mask map corresponding to the candidate is determined according to the candidate's face region selection box. The shape of the first filling area may be displayed in various manners, which are not limited herein, for example, a circular area, a rectangular area, an irregular polygonal area, and the like. However, there is a precondition. In order to ensure the accuracy of the recognition result, when determining the first filling area, the selection frame of the face area should be combined to be as close to the actual face area as possible.

After the first filling area is determined, a second filling area other than the first filling area is continuously determined in the face brushing image, and different filling values are assigned to the first filling area and the second filling area.

It should be noted that, in order to match the first filling area in the mask image with the face area in the face brushing image as much as possible, after assigning different filling values to the first filling area and the second filling area, the resolution is generated. A mask map consistent with the resolution of the brush face image.

Further, since most of the face area is a circular area or an oval area, in order to make the first filled area fit the candidate's face area better, the first filled area is taken as a circular area.

Specifically, when the face area is processed and the face area selection frame is determined, the face area selection frame is determined as a rectangular frame. After obtaining the face area of the rectangular frame, calculate the face frame width and face frame height through the position of the rectangular frame in the face brushing image, and calculate the radius of the circular area through the face frame width and face frame height.

Among them, when calculating the radius of the circular area, since the face area is a circular area or an oval area, when the rectangular frame is initially generated, the face area is similar to the inscribed circle of the rectangular frame. At the same time, ensure that the first filling area includes all the face area as much as possible, and take the maximum value between half the width of the rectangular frame and half the length of the frame as the radius of the circular area.

Therefore, the center of the rectangular box is taken as the center of the circle, and the half length of the longest side of the rectangular box is determined as the radius, and then the circular area formed by the center and the radius is determined as the first filling area of the mask map corresponding to the candidate .

For example, suppose the position of the candidate's face rectangle in the face brushing image is (x ₁ , y ₁ , x ₂ , y ₂ ), where x ₁ and x ₂ are the position coordinates of the width of the rectangle on the x-axis, respectively , y ₁ and y ₂ are the position coordinates of the height of the rectangle on the y-axis, respectively.

Then the expression for calculating the width of the face frame is w=x ₂ -x ₁ , where w is the width of the face frame. It should be noted that the position coordinates of x1 are smaller _than the position coordinates of _x2 .

The expression for calculating the height of the face frame is h=y ₂ -y ₁ , where h is the height of the face frame. It should be noted that the position coordinates of y ₁ are smaller than the position coordinates of y ₂ .

其中，R为圆形区域半径。The expression for determining the radius of a circular area is

where R is the radius of the circular area.

R为

Based on this, the position coordinates of the center of the rectangular box are

R is

In one or more embodiments of the present specification, more intuitively, this solution is described in more detail with reference to FIGS. 2-3 .

FIG. 2 is a schematic framework diagram of a face-scanning payment willingness recognition system provided by one or more embodiments of the present specification.

In one or more embodiments of this specification, in order to more accurately identify the candidate's willingness to pay for face-swiping, it is proposed to use a deep convolutional neural network end-to-end learning method for the face-swiping images acquired during the face-swiping payment process. , which can realize the security detection of the willingness to pay for face brushing, and introduce the candidate area attention mechanism to integrate the mask map into the network learning, so that the willingness to pay for brushing the face of the candidate in the face brushing image can be more targeted, so as to It can enhance the security experience of face brushing.

As shown in Figure 2, the face-swiping willingness-to-pay recognition system is implemented by a deep convolutional neural network end-to-end learning method. The first convolutional network module, the implementation module of the attention mechanism in the region where the candidate is located, the third convolutional module, and the network output.

It should be noted that the first convolutional network and the third convolutional network have a specific correspondence, that is, the types of networks involved in the first convolutional network and the third convolutional network are not limited here. However, the two are not separated, for example, the first convolutional network and the third convolutional network belong to different parts of the same recognition network type.

Among them, the face-swiping image and the generated mask image are used as the input data of the deep convolutional neural network, and the network output is the probability value of the willingness to pay for face-swiping, that is, the willingness safety probability and the willingness non-safety probability.

Based on this, in the process of face-swiping willingness to pay recognition, the first convolutional network is used to extract the features of the face-swiping image, and the features and mask images of the face-swiping image are input into the implementation module of the attention mechanism in the region where the candidate is located. The implementation module of the attention mechanism in the region where the candidate is located processes the features and mask images of the face-swiping image, and outputs the fusion features. Finally, the fusion feature is input into the third convolutional network, and the fusion feature is processed by the third convolutional network to obtain the processing result.

The following will continue to explain how the implementation module of the attention mechanism in the region where the candidate is located specifically processes the features and mask images of the face-swiping image, and outputs the fusion features.

As shown in Figure 2, the implementation module of the attention mechanism in the region where the candidate is located includes the features of the face-swiping image, the mask map after down-resolution processing, the second convolutional network module, and the fusion features.

Specifically, in the process of face-swiping willingness to pay recognition, the face-swiping image is first input into the first convolutional network in the first convolutional network module. The code image is subjected to down-resolution processing to obtain a mask image after down-resolution processing, which is suitable for the features of the face-swiping image.

Then, the features of the face brushing image and the reduced-resolution mask image are input into the second convolutional neural network module, and the features of the brushed face image and the reduced-resolution mask image are fused through the second convolutional network. , to get the fusion feature.

Through the end-to-end learning method of the deep convolutional neural network, the recognition of the candidate's willingness to pay for face-swiping is realized, and the implementation module of the attention mechanism in the area where the candidate is located can make the network more targeted to learn and judge the candidates in the face-swiping image. the willingness to pay for face brushing. Especially in public places, it can effectively prevent the technical effect of candidate A's face-swiping and mistakenly swiping candidate B's assets.

More intuitively, FIG. 3 is a schematic flowchart of a method for recognizing the willingness to pay for face brushing based on end-to-end learning of a deep convolutional neural network according to one or more embodiments of the present specification.

The flow in Figure 3 may include the following steps:

S302: Acquire a face brushing image, and determine a candidate to be identified in the face brushing image.

S304: According to the area where each candidate is located in the face brushing image, generate a corresponding mask map respectively to distinguish the located area from other areas in the face brushing image.

S306: Extract the features of the face brushing image through the first convolutional network.

It should be noted that the first convolutional network is obtained through supervised training in advance.

S308: Perform a resolution reduction process on the mask image to adapt to the features of the face brushing image.

For example, by reducing the resolution of the mask map by means of nearest neighbor sampling, a mask map of the same resolution as the face brushing image is generated, which can be adapted to the characteristics of the brushing face image for processing by the second convolutional network.

S310: Through the second convolutional network, fuse the features of the face brushing image and the mask map after the resolution reduction process to obtain fused features. It should be noted that the second convolutional network is obtained through supervised training in advance.

In one or more embodiments of this specification, in the process of obtaining the fusion feature, according to the channel dimension, the feature of the face brushing image and the mask map after the resolution reduction process are connected, and the obtained feature is input into the first The second convolutional network is processed to obtain fused features.

The number of convolutional layers of the second convolutional network is not specifically limited here. That is, the number of convolutional layers of the second convolutional network may consist of one or more convolutional layers. At the same time, the resolution of the fusion feature is the same as that of the face brush image, and the number of feature channels of the fusion feature is the same as that of the face brush image.

S312: Input the fusion feature into a third convolutional network corresponding to the first convolutional network for processing, and obtain a processing result, wherein the first convolutional network and the third convolutional network are obtained in advance from The same convolutional network split is obtained. It should be noted that the third convolutional network is obtained through supervised training in advance.

In one or more embodiments of this specification, the first convolutional network and the third convolutional network are obtained by splitting from the same convolutional network in advance, for example, the convolutional network is resent, ShUffleNet V2, etc.

In the process of splitting, the first convolutional network and the third convolutional network can be used as the front and rear parts of the same convolutional network, respectively.

)，以便相应拆分卷积网络，完成模型构造，使得前一部分的卷积网络(第一卷积网络)恰好能够输出该分辨率的特征。For the split position, it can be determined by the resolution of the features of the face brushing image. Even if the feature extraction of the face image has not started (assuming that the model has not been constructed at this time), there can be an expected resolution for the features that can also brush the face image (referred to as the target resolution, for example, for the brush the resolution of the face image

), so as to split the convolutional network accordingly and complete the model construction, so that the convolutional network in the previous part (the first convolutional network) can just output the features of this resolution.

Among the convolutional layers in the same convolutional network, determine the convolutional layer that matches the target resolution. Finally, with the matching convolutional layer as the split point, the same convolutional network is split into the former part and the latter part, the former part is used as the first convolutional network, and the latter part is used as the third convolutional network.

S314: Generate a probability value according to the processing result to indicate whether the corresponding candidate has a willingness to pay for face-scanning.

For example, the probability value is compared with the set threshold probability, and if the probability value is greater than the set threshold probability, it is determined that the candidate's willingness is safe, that is, the candidate has the willingness to pay for face-scanning. If the probability value is less than or equal to the set threshold probability, it is determined that the candidate's willingness is not safe, that is, the candidate does not have the willingness to pay for face-scanning.

According to the previous description, the following continues to describe how to perform supervised training on the first convolutional network, the second convolutional network, and the third convolutional network.

In one or more embodiments of this specification, a training data set needs to be established first, and then network training is performed through the training data set.

Specifically, when establishing a training data set, first obtain the face-swiping sample images that include the confirmed face-swiping users, that is, the candidate enables face-swiping payment, and collects the face-swiping images through a camera device, regardless of whether there are many face-swiping images in the face-swiping images. A candidate to be identified, in the face-swiping image, the candidate is used as a face-swiping user, and the face-swiping image is used as a face-swiping sample image. For example, the face-swiping image is collected by the camera on the offline IoT face-swiping device. If candidate A enables face-swiping payment, candidate A will be used as the face-swiping user in the face-swiping image, and the face-swiping image will be used as the face-swiping image. Sample image.

It should be noted that each time the face-swiping payment is enabled, a corresponding face-swiping image will be collected. For example, if candidate A enables face-swiping payment, the IoT face-swiping device will collect images of nearby users to obtain a single face-swiping image; if candidate B enables face-swiping payment, the IoT face-swiping device will again Collect images for nearby users, so as to obtain a single face brushing image again.

After acquiring the face-swiping sample image, the positive samples are obtained by marking the face-swiping user as having the willingness to pay for face-swiping and generating the corresponding mask map. Wherein, when generating the corresponding mask image, the position of the face selection frame of the user who brushes the face in the face brushing image is selected from the face brushing image, and then the corresponding mask image is generated through the position. And when a face-swiping user is marked as having a willingness to pay for face-swiping, it can be marked with a willingness tag. For example, the willingness tag is {0, 1}, where 1 represents the willingness to pay for face-swiping, and 0 means no face-swiping willingness to pay. Willingness to pay.

After the face-swiping sample image is obtained, since there may be multiple candidates to be identified in the face-swiping sample image, if the face-swiping sample image also includes other users who are photographed incidentally, the other users can be marked as not having Face willingness to pay, and generate the corresponding mask map to get negative samples. For example, if candidate A enables face-swiping payment, in the face-swiping image, mark candidate B as not having the willingness to pay for face-swiping.

Finally, supervised training is performed on the first convolutional network, the second convolutional network, and the third convolutional network according to the obtained positive samples and negative samples.

Among them, during supervised training, the positive samples and negative samples are randomly sampled in the training data set, and the training batch and its corresponding label are generated. Then, the training batch and its corresponding labeled labels are fed into the initial deep convolutional neural network. The initial deep convolutional neural network includes untrained first convolutional network, second convolutional network, and third convolutional network.

Then, the initial deep convolutional neural network outputs the probability value, calculates the loss function through the probability value and the corresponding label, and uses the gradient descent method to perform network training without interrupting the optimized loss function, thus completing the supervised training and obtaining the deep convolution Neural Networks. Among them, the rules for identifying whether a candidate has the willingness to pay for face-scanning can be obtained through network training. For example, if the deep convolutional neural network recognizes that the candidate's face area is located in the middle area, it is considered that the candidate has the willingness to pay for face brushing.

Based on the same idea, one or more embodiments of the present specification also provide apparatuses and devices corresponding to the foregoing methods, as shown in FIG. 4 and FIG. 5 .

FIG. 4 is a schematic structural diagram of a device for recognizing a willingness to pay for face-scanning provided by one or more embodiments of the present specification, and the device includes:

Obtaining module 402, obtains a face brushing image, and determines a candidate to be identified in the brushing face image;

The generation module 404, according to the area where each candidate is located in the face brushing image, respectively generates a corresponding mask map to distinguish the area and other areas in the brushing face image;

Extraction module 406, extracts the features of the face brushing image, and obtains fusion features according to the characteristics of the face brushing image and the mask map;

The identification module 408, according to the fusion feature, identifies whether each of the candidates has a willingness to pay for face-scanning.

Optionally, the generating module 404, for each of the determined candidates, respectively, executes:

According to the face area selection frame of the candidate, determine the first filling area of the mask map corresponding to the candidate, and the second filling area other than the first filling area;

By assigning different filling values to the first filling area and the second filling area, the mask map whose resolution is consistent with the resolution of the face brushing image is generated.

Optionally, the face area selection frame is a rectangular frame;

The generating module 404 determines the center of the rectangular frame as the center of the circle, and determines the half length of the longest side of the rectangular frame as the radius;

The circular area formed based on the center of the circle and the radius is determined as the first filling area of the mask map corresponding to the candidate.

Optionally, the extraction module 406 extracts the features of the face brushing image through the first convolutional network;

performing down-resolution processing on the mask image to adapt to the features of the face brushing image;

Through the second convolutional network, the features of the face brushing image and the mask map after the resolution reduction process are fused to obtain fused features.

Optionally, the extraction module 406, according to the channel dimension, connects the feature of the face brushing image and the mask image after the resolution reduction process;

The features obtained by the connection are input into the second convolutional network for processing to obtain fusion features.

Optionally, the identifying module 408 inputs the fusion feature into a third convolutional network corresponding to the first convolutional network for processing, and obtains a processing result, wherein the first convolutional network and the The third convolutional network is split from the same convolutional network in advance;

A probability value is generated according to the processing result to indicate whether the corresponding candidate has a willingness to pay for face-scanning.

Optionally, the identification module 408 determines the target resolution, so as to be the resolution of the feature of the face brushing image;

In the convolutional layers in the same convolutional network, determine the convolutional layer matching the target resolution;

Using the matched convolutional layer as a splitting point, the same convolutional network is split into a former part and a latter part, the former part as the first convolutional network, and the latter part as the third Convolutional Networks.

Optionally, the device further includes a supervised training module, the supervised training module acquires a face-swiping sample image containing a user who has been confirmed as a face-swiping user;

By marking the face-swiping user as having a willingness to pay for face-swiping, and generating a corresponding mask map, a positive sample is obtained;

If the face-swiping sample image also includes other users who were photographed incidentally, then by marking the other users as not having the willingness to pay for face-swiping, and generating a corresponding mask image, a negative sample is obtained;

According to the obtained samples, supervised training is performed on the first convolutional network, the second convolutional network, and the third convolutional network.

Optionally, the face brushing image includes at least two human faces.

Optionally, the device is applied to an offline IoT face brushing machine, and the face brushing image is collected by the IoT face brushing machine for nearby users.

FIG. 5 is a schematic structural diagram of a device for recognizing the willingness to pay for face-scanning provided by one or more embodiments of the present specification, and the device includes:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

Obtaining a face brushing image, and determining a candidate to be identified in the brushing face image;

According to the area where each candidate is located in the face brushing image, a corresponding mask map is respectively generated to distinguish the located area from other areas in the face brushing image;

Extracting the features of the face brushing image, and obtaining fusion features according to the characteristics of the face brushing image and the mask map;

According to the fusion feature, identify whether each candidate has a willingness to pay for face-scanning.

Based on the same idea, one or more embodiments of this specification also provide a non-volatile computer storage medium corresponding to the above method for recognizing the willingness to pay for face-scanning, storing computer-executable instructions, the computer-executable instructions setting for:

Obtaining a face brushing image, and determining a candidate to be identified in the brushing face image;

According to the area where each candidate is located in the face brushing image, a corresponding mask map is respectively generated to distinguish the located area from other areas in the face brushing image;

Extracting the features of the face brushing image, and obtaining fusion features according to the characteristics of the face brushing image and the mask map;

According to the fusion feature, identify whether each candidate has a willingness to pay for face-scanning.

In the 1990s, improvements in a technology could be clearly differentiated between improvements in hardware (eg, improvements to circuit structures such as diodes, transistors, switches, etc.) or improvements in software (improvements in method flow). However, with the development of technology, the improvement of many methods and processes today can be regarded as a direct improvement of the hardware circuit structure. Designers almost get the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that the improvement of a method flow cannot be realized by hardware entity modules. For example, a Programmable Logic Device (PLD) (eg, Field Programmable Gate Array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. It is programmed by the designer to "integrate" a digital system on a PLD without having to ask a chip manufacturer to design and manufacture a dedicated integrated circuit chip. And, instead of making integrated circuit chips by hand, these days, most of this programming is done using "logic compiler" software, which is similar to the software compilers used in program development and writing, but before compiling The original code also has to be written in a specific programming language, which is called Hardware Description Language (HDL), and there is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language) , AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (RubyHardware Description Language), etc. The most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. It should also be clear to those skilled in the art that a hardware circuit for implementing the logic method process can be easily obtained by simply programming the method process in the above-mentioned several hardware description languages and programming it into the integrated circuit.

The controller may be implemented in any suitable manner, for example, the controller may take the form of, for example, a microprocessor or processor and a computer readable medium storing computer readable program code (eg software or firmware) executable by the (micro)processor , logic gates, switches, application specific integrated circuits (ASICs), programmable logic controllers and embedded microcontrollers, examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicon Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in the form of pure computer-readable program code, the controller can be implemented as logic gates, switches, application-specific integrated circuits, programmable logic controllers and embedded devices by logically programming the method steps. The same function can be realized in the form of a microcontroller, etc. Therefore, such a controller can be regarded as a hardware component, and the devices included therein for realizing various functions can also be regarded as a structure within the hardware component. Or even, the means for implementing various functions can be regarded as both a software module implementing a method and a structure within a hardware component.

The systems, devices, modules or units described in the above embodiments may be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or A combination of any of these devices.

For the convenience of description, when describing the above device, the functions are divided into various units and described respectively. Of course, when implementing this specification, the functions of each unit may be implemented in one or more software and/or hardware.

As will be appreciated by one skilled in the art, the embodiments of the present specification may be provided as a method, a system, or a computer program product. Accordingly, embodiments of this specification may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, embodiments of the present specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The specification is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the specification. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.

Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory in the form of, for example, read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed, or which are inherent to such a process, method, article of manufacture, or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture, or device that includes the element.

This specification may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including storage devices.

The various embodiments in this specification are described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the apparatus, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference may be made to some descriptions of the method embodiments for related parts.

The foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

The above descriptions are merely one or more embodiments of the present specification, and are not intended to limit the present specification. Various modifications and variations of the one or more embodiments of this specification are possible for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of one or more embodiments of this specification should be included within the scope of the claims of this specification.

Claims (21)

1. A face brushing willingness-to-pay recognition method comprises the following steps:

acquiring a face brushing image, and determining a candidate to be identified in the face brushing image;

according to the located area of each candidate in the face brushing image, respectively generating corresponding mask images to distinguish the located area from other areas in the face brushing image;

extracting the features of the face brushing image, and obtaining fusion features according to the features of the face brushing image and the mask image;

and identifying whether each candidate has a face brushing willingness to pay according to the fusion characteristics.

2. The method according to claim 1, wherein the generating a corresponding mask map according to the region of each candidate in the face brushing image respectively includes:

respectively aiming at each determined candidate, executing:

determining a first filling area of a mask image corresponding to the candidate and a second filling area outside the first filling area according to the face area selection box of the candidate;

generating the mask map having a resolution identical to that of the brush face image by assigning different fill values to the first fill region and the second fill region.

3. The method of claim 2, wherein the face region selection box is a rectangular box;

the determining, according to the face area selection box of the candidate, a first filling area of a mask map corresponding to the candidate specifically includes:

determining the center of the rectangular frame as a circle center, and determining the half length of the longest edge of the rectangular frame as a radius; and determining a circular area formed based on the circle center and the radius as a first filling area of the mask map corresponding to the candidate.

4. The method according to claim 1, wherein the extracting the features of the brushed face image and obtaining the fusion features according to the features of the brushed face image and the mask map specifically include:

extracting the characteristics of the brushing image through a first convolution network;

performing resolution reduction processing on the mask image to adapt to the characteristics of the face brushing image;

and fusing the characteristics of the brushing image and the mask image after resolution reduction processing through a second convolution network to obtain fused characteristics.

5. The method according to claim 4, wherein fusing, by using the second convolution network, the feature of the brushed image and the mask map after the resolution reduction processing to obtain a fused feature specifically includes:

according to the channel dimension, connecting the characteristics of the face brushing image with the mask image after resolution reduction processing; and inputting the connected features into the second convolution network for processing to obtain fusion features.

6. The method of claim 4, wherein identifying whether each candidate has a willingness to swipe a face according to the fusion features comprises:

inputting the fusion characteristics into a third convolution network corresponding to the first convolution network for processing to obtain a processing result, wherein the first convolution network and the third convolution network are obtained by splitting the same convolution network in advance; and generating a probability value according to the processing result to indicate whether the corresponding candidate has the willingness to pay by brushing the face.

7. The method according to claim 6, wherein the splitting specifically comprises:

determining a target resolution as a resolution of a feature of the brushed face image;

determining a convolutional layer matched with the target resolution in convolutional layers in the same convolutional network;

and taking the matched convolutional layer as a splitting point, splitting the same convolutional network into a former part and a latter part, wherein the former part is taken as the first convolutional network, and the latter part is taken as the third convolutional network.

8. The method of claim 6, prior to identifying whether each of the candidates has a willingness to swipe according to the fused features, the method further comprising:

acquiring a face brushing sample image containing a confirmed face brushing user;

marking the face brushing user as having a face brushing willingness to pay, and generating a corresponding mask map to obtain a positive sample;

if the face brushing sample image also contains other users shot by the way, marking the other users as not having the intention of face brushing payment, and generating a corresponding mask map to obtain a negative sample;

and carrying out supervised training on the first convolutional network, the second convolutional network and the third convolutional network according to the obtained samples.

9. A method as claimed in any one of claims 1 to 8, wherein the brushed face image comprises at least two human faces.

10. The method of any of claims 1-8 applied to an offline IoT facer, the facer image captured by the IoT facer for nearby users.

11. A brushing will-of-payment recognition device, comprising:

the acquisition module acquires a face brushing image and determines a candidate to be identified in the face brushing image;

the generating module is used for respectively generating corresponding mask images according to the areas of the candidate persons in the face brushing image so as to distinguish the areas from other areas in the face brushing image;

the extraction module is used for extracting the features of the face brushing image and obtaining fusion features according to the features of the face brushing image and the mask image;

and the identification module identifies whether each candidate has a face brushing willingness to pay according to the fusion characteristics.

12. The apparatus of claim 11, wherein the generating module performs, for each of the determined candidates:

determining a first filling area of a mask image corresponding to the candidate and a second filling area outside the first filling area according to the face area selection box of the candidate;

generating the mask map having a resolution identical to that of the brush face image by assigning different fill values to the first fill region and the second fill region.

13. The apparatus of claim 12, wherein the face region selection box is a rectangular box;

the generating module is used for determining the center of the rectangular frame as the circle center and determining the half length of the longest edge of the rectangular frame as the radius;

and determining a circular area formed based on the circle center and the radius as a first filling area of the mask map corresponding to the candidate.

14. The apparatus of claim 11, the extraction module to extract features of the brushed face image through a first convolutional network;

performing resolution reduction processing on the mask image to adapt to the characteristics of the face brushing image;

and fusing the characteristics of the face brushing image and the mask image after resolution reduction processing through a second convolution network to obtain fused characteristics.

15. The apparatus of claim 14, the extraction module to connect features of the brushed face image and the deresolved mask map according to a channel dimension;

and inputting the connected features into the second convolution network for processing to obtain fusion features.

16. The apparatus of claim 14, wherein the identifying module inputs the fused feature into a third convolutional network corresponding to the first convolutional network for processing, so as to obtain a processing result, and the first convolutional network and the third convolutional network are obtained by splitting the same convolutional network in advance;

and generating a probability value according to the processing result to indicate whether the corresponding candidate has the willingness to pay by brushing the face.

17. The apparatus of claim 16, the recognition module to determine a target resolution as a resolution of a feature of the brush face image;

determining a convolutional layer matched with the target resolution in convolutional layers in the same convolutional network;

and taking the matched convolution layer as a splitting point, splitting the same convolution network into a front part and a rear part, wherein the front part is taken as the first convolution network, and the rear part is taken as the third convolution network.

18. The apparatus of claim 16, further comprising a supervised training module that obtains a brush sample image containing confirmed brushing users;

marking the face brushing user as having a face brushing willingness to pay, and generating a corresponding mask map to obtain a positive sample;

if the face brushing sample image also contains other users shot by the way, marking the other users as not having the intention of face brushing payment, and generating a corresponding mask map to obtain a negative sample;

and carrying out supervised training on the first convolutional network, the second convolutional network and the third convolutional network according to the obtained samples.

19. An apparatus as claimed in any one of claims 11 to 16, wherein the brushed face image comprises at least two human faces.

20. The apparatus of any of claims 11-16, applied to an offline IoT facer, the facer image captured by the IoT facer for nearby users.

21. A brushing willingness-to-pay recognition device, comprising:

at least one processor; and the number of the first and second groups,

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to:

acquiring a face brushing image, and determining a candidate to be identified in the face brushing image;

according to the located area of each candidate in the face brushing image, respectively generating corresponding mask images to distinguish the located area from other areas in the face brushing image;

extracting the features of the face brushing image, and obtaining fusion features according to the features of the face brushing image and the mask image;

and identifying whether each candidate has a face brushing willingness to pay according to the fusion characteristics.

Images