TW202115622A - Face attribute recognition method, electronic device and computer-readable storage medium - Google Patents

Abstract

The present application provides a face attribute recognition method, an electronic device and a computer-readable storage medium. The method comprises: acquiring n feature images, wherein n feature images are obtained after one or more feature extraction of face images, and N is a positive integer; the attribute features corresponding to each feature image in n feature images are obtained. Among them, the attribute feature is the feature used to represent the target face attribute in the feature image; according to the attribute feature, the attribute recognition result of the target face attribute in the face image is determined.

Description

Face attribute recognition method, electronic equipment and computer readable storage medium

The present invention relates to the field of computer vision, and particularly relates to a face attribute recognition method, electronic equipment and computer-readable storage medium.

In recent years, the prevention, control and handling of emergencies have become more difficult. Important checkpoints such as airports, customs or border defenses, or public places such as streets, shopping malls, etc. have been installed with surveillance cameras. Face recognition technology is used to capture the faces of the cameras. Image attribute analysis can effectively assist security staff in responding to emergencies. For example, through the recognition of facial expression attributes to determine whether a person is aggressive, or by comparing and analyzing the collected face image of the target object with the face image database, the identity information of the target object can be obtained for the convenience of security staff Through these analysis results, security deployment and control and crisis management can be carried out in a timely manner.

The embodiments of the present invention provide a face attribute recognition method, an electronic device, and a computer-readable storage medium, which can efficiently and accurately recognize various attributes of a face image.

An embodiment of the present invention provides a face attribute recognition method, including: acquiring n feature images, where the n feature images are obtained by performing feature extraction on the face image one or more times, and n is positive An integer; acquiring the attribute feature corresponding to each of the n feature images, where the attribute feature is a feature in the feature image that is used to characterize the attributes of the target face; according to the attribute feature To determine the attribute recognition result of the target face attribute of the face image.

The aforementioned face attribute recognition method determines the attribute recognition result of the target face attribute in the face image by acquiring the characteristic image and the attribute characteristics corresponding to the characteristic image. Since the attribute features obtained in different feature images can reflect the face attributes to varying degrees, the use of attribute features corresponding to different feature images to predict face attributes can improve the accuracy and recognition of face attributes. effectiveness.

In some embodiments of the present invention, each of the feature attribute acquiring image feature corresponding to the n feature images, comprising: an image feature of each of the n feature images are input The corresponding attention model obtains the attention matrix corresponding to each of the feature images; and extracts the attribute feature corresponding to each feature image of the n feature images according to the attention matrix.

In the above method, by inputting a feature image into multiple attention models, multiple attention matrices corresponding to the feature image can be obtained, and one attention matrix can be used to extract a face attribute in the feature image Features, and a face attribute feature can be used to characterize a face attribute. Therefore, the recognition effect of a single face attribute can be improved through the attention model and the obtained attention matrix, thereby improving the recognition accuracy of all face attributes in the face image.

In some embodiments of the present invention, extracting attribute features corresponding to each of the n feature images according to the attention matrix includes: adding each feature image of the n feature images The image and the corresponding attention matrix are multiplied to obtain the attribute feature corresponding to each of the n feature images.

In the above method, the attention mechanism is used to multiply the feature image with the corresponding attention matrix, and the attribute features corresponding to the feature image can be obtained. These attribute features can locate the recognition area from the entire image to the area of interest , So as to filter out other noise and other factors that affect the attribute recognition result, and improve the accuracy and efficiency of attribute recognition.

In some embodiments of the present invention, the determining the attribute recognition result of the target face attribute of the face image according to the attribute feature includes: respectively predicting and obtaining the corresponding recognition result according to each of the attribute characteristics ; Perform weighted average calculation on the recognition result to determine the attribute recognition result of the target face attribute in the face image.

In the above method, the same attribute recognition is performed by performing a weighted average calculation on the attribute features corresponding to the attribute in different feature images, thereby improving the accuracy of the attribute recognition result.

In some embodiments of the present invention, the predicting and obtaining the corresponding recognition result respectively according to each of the attribute characteristics includes: performing attribute prediction processing on each of the attribute characteristics, and predicting to obtain the The recognition result corresponding to the attribute feature.

In the above method, the attribute prediction processing is performed on each attribute feature separately, and the recognition result of the corresponding attribute in each attribute characteristic can be obtained. According to these results, the influence factors of different attribute characteristics on the recognition result of the same attribute can be further obtained. In other words, it is possible to further determine the attributes in the face image that require deep-level attribute features for attribute recognition and the attributes that can be recognized by shallow-level features.

In some embodiments of the present invention, the preset attribute prediction processing includes global average pooling processing.

In the above method, by performing global average pooling on each extracted face attribute feature, the corresponding recognition result can be directly obtained, and the over-fitting phenomenon in the recognition process can be improved. At the same time, the recognition efficiency can be improved and the calculation of the model can be saved. The amount of time and budget.

In some embodiments of the present invention, before the obtaining the n feature images, the method further includes: obtaining the face image; performing feature extraction on the face image through a feature image extraction model , To obtain the n feature images.

In the above method, by extracting the feature image of the obtained face image, the noise in the face image can be preliminarily filtered, which facilitates the subsequent acquisition of attribute features, and improves the accuracy of the attribute recognition of the face image.

In some embodiments of the present invention, the feature image extraction model is obtained by training on face image samples and feature image samples, and the method further includes: using the feature image samples and the corresponding attention The matrix samples train the neural network to obtain the attention model.

In the above method, by using a large number of known feature image samples and known attention matrix samples, the attention model can be trained to improve the accuracy of the attention matrix. At the same time, it can also collect a large number of face image samples and features. Image samples are trained to obtain a feature image extraction model, which improves the multi-scale feature extraction, so that the recognition accuracy of face attributes is greatly improved.

In some embodiments of the present invention, the n feature images further include the face image.

In the above method, the attention model is actually a process of feature image extraction, so the face image can be directly input into the attention model for calculation, eliminating the need for feature image extraction model to extract features of the face image. Improve the efficiency of face attribute recognition.

An embodiment of the present invention provides a face attribute recognition device, including: a feature image extraction unit configured to obtain n feature images, where the n feature images are performed on the face image one or more times N is a positive integer obtained after feature extraction; the attribute feature extraction unit is configured to obtain the attribute feature corresponding to each feature image in the n feature images, wherein the attribute feature is in the feature image A feature used to characterize the attributes of the target face; the face attribute recognition unit is configured to determine the attribute recognition result of the target face attribute of the face image according to the attribute characteristics.

An embodiment of the present invention provides a storage medium in which a computer program is stored, and the computer program is executed by hardware to implement the above method.

The embodiment of the present invention provides a computer program product, and the above method will be executed when the computer program product is read and executed by a computer.

An embodiment of the present invention provides a computer cluster including: at least one computing node, each computing node includes a processor and a memory, and the processor executes the foregoing method by executing code in the memory.

An embodiment of the present invention provides a computer non-transitory storage medium, including instructions, which when the instructions run on at least one computing node in a computer node cluster, cause the computing node cluster to execute the above method.

An embodiment of the present invention provides an electronic device including a processor and a memory, and the processor executes the code in the memory to implement the above method.

The terms used in the embodiments of the present invention are only used to explain specific embodiments of the present invention, and are not intended to limit the present invention.

The application scenarios involved in the embodiments of the present invention include, but are not limited to, security monitoring scenarios, mobile phone entertainment scenarios, and are further applied to scenarios such as facial expression recognition and age recognition. In order to facilitate the understanding of the embodiments of the present invention, the application scenarios mainly involved in the embodiments of the present invention are first described here: security monitoring scenarios.

Security monitoring usually refers to the use of optical fiber, coaxial cable or microwave to transmit video signals in its closed loop, and an independent and complete system from photography to image display and recording. It can reflect the monitored object in real time, vividly, and truthfully. It can not only extend the observation distance, but also replace manual long-term monitoring in harsh environments, allowing people to see subtle features that cannot be seen by the human eye through monitoring, and cooperate with people. Face recognition equipment can also identify, alert and record relevant personnel in the monitoring area, and is one of the important means to maintain social security.

In the security monitoring scene, the face image of the target collected by the camera is usually recognized by multiple attributes through face recognition technology, and the results of multiple attribute recognition are combined to determine whether the target face is aggressive and the target face corresponds Identity information and so on. For example, first identify whether the target face image wears a hat, whether a backpack, whether to wear a mask, etc., and based on the results of multiple attribute recognition, combined with certain analysis strategies, determine whether the target face is aggressive; Or, first identify the eyes, nose, mouth, moles, scars and other attributes of the target face image, and identify the identity information corresponding to the target face based on the results of multiple attribute recognition, combined with a certain analysis strategy. It should be understood that the above examples are only for illustration, and the present invention is not specifically limited.

In related technologies, Multi-Task Convolution Neural Net (MTCNN) is often used to solve the problem of face attribute recognition. This method is based on the shared features of multi-branch networks, which can easily and effectively Each independent attribute is identified. However, in the actual application process, not all attributes need to be identified with the same features. For example, some attributes like wearing earrings and glasses need shallow features, while some attributes like facial expressions and gender need deep features. Hierarchical features, so when using MTCNN for face attribute recognition, there may be problems with inaccurate recognition results due to unobvious features.

Figure 1 is a schematic diagram of a process of identifying face attributes based on MTCNN in related technologies. It can be seen from FIG. 1 that the MTCNN includes an input layer 110, a common feature learning layer 120, and a fully connected layer 130. Wherein, the input layer 110 is configured to perform image preprocessing on the input face image, the common feature learning layer 120 is configured to extract the attribute features of the face image, and the fully connected layer 130 is configured to share the common feature learning layer 120. The output face attribute features are classified. It should be noted that the shared semantic feature learning layer 120 includes one or more convolutional layers and a fully connected layer, which are configured to filter and optimize face attribute features.

In specific implementation, for an input image (such as a face image), MTCNN first performs simple image processing on the input image through the input layer 110, for example, image binarization; and then through the shared feature learning layer 120 Feature extraction is performed on the input image. The shared feature learning layer 120 includes different convolutional layers and one or more fully connected layers. After the multi-layer convolutional layer, a fully connected layer is added to filter out irrelevant attribute features. The filtered attribute features are then extracted through the feature extraction of multiple convolutional layers to obtain shared features. For example, the filtered attribute features are again subjected to the convolution operation of the 10th convolutional layer to extract the features from the 10th convolutional layer. As a shared feature; finally, the shared feature is input to the fully connected layer 130, so that the shared feature is used to perform different attribute recognition tasks.

It can be understood that in the above-mentioned face attribute recognition process, MTCNN needs to solve the recognition problem of different attributes based on the common features extracted by the last convolutional layer. The neural network trained in this way, because each attribute recognition is based on the same common feature, it actually treats the features of each area of the picture equivalently, and does not treat a certain "area" "Too much attention will make some subtle features unable to be accurately recognized by the neural network. In other words, not all the features required for recognition of face attributes are the last layer obtained after multi-layer convolution extraction. feature. For example, performing three convolution operations on the face image of the target object can obtain three layers of features. In the case of judging whether the target object wears glasses, the convolutional neural network may only need shallow features (such as 2-layer features), you can get the characteristics of this attribute, so as to identify whether the target object wears glasses; in the case of judging the race of the target object, the convolutional neural network may require deep-level features (such as 4-layer features), In order to identify the race of the target object. At this time, when using the three-layer feature to determine whether the target object wears glasses, the feature information that characterizes the attribute may be lost due to the excessive extraction of features, which affects the accuracy of the attribute recognition result and reduces the efficiency of attribute recognition; When determining the race of the target object by layer features, it may be that the three-layer feature cannot fully characterize the attribute of race, thereby reducing the accuracy of the attribute recognition result.

Therefore, in view of the above-mentioned process of face attribute recognition in the security monitoring field, when attribute recognition is performed based on shared features, it is easy to cause problems such as feature loss or inconspicuous features. The embodiment of the present invention provides a method for face attribute recognition. By inputting features extracted from different convolutional layers into different face attribute recognition models, the accuracy and efficiency of face attribute recognition are improved.

Fig. 2 is a schematic structural diagram of a facial attribute recognition neural network provided by an embodiment of the present invention. As shown in Fig. 2, the facial attribute recognition neural network provided by an embodiment of the present invention includes: an input layer 210, a convolutional layer 220, An attention learning layer 230, a global average pooling layer (Global Average Pooling, GAP) 240, and a recognition layer 250. Hereinafter, each layer in the facial attribute recognition neural network provided by the embodiment of the present invention will be described in detail as follows.

Input layer 210:

輸入至輸入層，輸出和輸入相等，即，不對輸入的人臉圖像進行任何處理。為了陳述簡便，此處假設經輸入層輸出的圖像與輸入圖像相等，但是，在實際應用中，可以在輸入層中設置歸一化處理模組或者其他圖像預處理模組，此處不作具體限定。The face image of the target object

Input to the input layer, the output is equal to the input, that is, no processing is performed on the input face image. For the sake of simplicity, it is assumed that the image output through the input layer is equal to the input image. However, in practical applications, a normalization processing module or other image preprocessing module can be set in the input layer. Here There is no specific limitation.

Convolutional layer 220.

The input layer is an image convolution output as an input layer, or a plurality of convolutional convolutional layer is calculated to generate n feature images, wherein said n images are different levels of features of the image, it can be characterized al The attributes of the face at different levels. For ease of presentation, the embodiment of the present invention proposes the concept of "block", which divides the convolutional layer of the convolutional neural network into multiple blocks. Each block can include one or more convolution kernels. A feature image, in other words, blocks B ₁ , B ₂ ,..., B _n can output feature images I ₁ , I ₂ ,..., I _{n respectively} , where n is a positive integer.

輸出的特徵圖像I_{i- 1} 作為區塊B_i 的輸入，經過u 個卷積核

（l =1,2,…,u ）的卷積操作生成u 個特徵圖像

（l =1,2,…,u ），其中，

，i 、l 、u 均是正整數。將特徵圖像b_u 作為區塊B_i 輸出的特徵圖像I_i ，其中，特徵圖像b_u 是卷積得到的最後一個特徵圖像，也就是說，特徵圖像I_i 是將特徵圖像I_{i- 1} 經過u 次卷積操作提取到的特徵圖像。Wherein the image _{I 1, I 2, ...,} I n thus obtained may be: to characterize the block B _i I _i output from the image, for example, the block

The output feature image I _{i- 1 is} used as the input of block B _i , after u convolution kernels

( L =1,2,..., u ) convolution operation generates u feature images

( L =1,2,…, u ), where

, I , l , u are all positive integers. The feature image b _u as the block B _i I _i output from the feature image, wherein the image feature b _u is the last convolution of the feature image, i.e., image I _i is the characteristic feature of FIG. Like the feature image extracted from I _{i- 1} after u times of convolution operation.

（包括元素、尺寸、步長等等）、偏置值

可以是人為根據需要提取的特徵或者輸入圖像的大小等等進行設置。以卷積核

為例，當需要提取的特徵是人臉屬性的特徵時，卷積核

的元素可以採用局部二值模式（Local Binary Patterns，LBP）運算元的元素，又例如，當輸入的特徵圖像I_i 比較大時，卷積核

的大小也可以比較大，反之，當輸入的特徵圖像I_i 比較小時，卷積核

的大小也可以比較小，又例如，當輸入的特徵圖像I_i 比較大時，卷積核

的步長也可以比較大，反之，當輸入的特徵圖像I_i 比較小時，卷積核

的步長也可以比較小。It should be noted that the characteristic image I ₁ is obtained by inputting the original face image into the block B _1. In addition, among the various parameters of the above convolutional layer, the convolution kernel

(Including element, size, step size, etc.), offset value

It can be artificially set according to the features that need to be extracted or the size of the input image, etc. Convolution kernel

For example, when the feature to be extracted is the feature of the face attribute, the convolution kernel

The elements of can use the elements of Local Binary Patterns (LBP) operands. For example, when the input feature image I _{i is} relatively large, the convolution kernel

The size of can also be relatively large. Conversely, when the input feature image I _{i is} relatively small, the convolution kernel

The size of can also be relatively small, for example, when the input feature image I _{i is} relatively large, the convolution kernel

The step size of can also be relatively large, on the contrary, when the input feature image I _{i is} relatively small, the convolution kernel

The step size can also be relatively small.

Attention learning layer 230.

The block _{B 1, B 2, ...,} n B _n characteristic output from the image I _1, I _2, ..., wherein each image I _n in the input layer as attention learning, attention learning through layer After learning, n × m face attribute features are obtained, where each feature image corresponds to m face attribute features, m is a positive integer, and m represents the number of face attributes that need to be extracted.

（1-1）； 其中，Attention是一個注意力模型，

可以為特徵圖像I_i 與注意力矩陣V_ij 之間的映射關係，其中

可以通過大量的已知特徵圖像I 以及已知屬性的注意力矩陣進行訓練得到。為了陳述簡便，上面只陳述了特徵圖像I_i 的第j 個屬性的人臉屬性特徵A_ij 的提取，實際應用中，特徵圖像I_i 中的人臉屬性特徵A_{i 1} ,A_{i 2} ,…,A_im 的提取方式與特徵圖像I_i 的第j 個屬性的人臉屬性特徵A_ij 的提取方式類似，此處不再展開贅述。另外，特徵圖像I ₁ ,I ₂ ,…,I_n 中的第j 個屬性的人臉屬性特徵A _1j ,A _2j ,…,A_nj 的提取方式均與特徵圖像I_i 中第j 個屬性的人臉屬性特徵A_ij 的提取方式類似，此處不再展開贅述。 M facial attribute features to an image feature extraction as an example I _i: I _i input image feature learning attention layer through the m-th focus attention model learning attention generating the m matrix V _ij (j = 1,2,..., m ), where an attention matrix corresponds to a face attribute, that is, an attention matrix is used to extract the attribute characteristics of the corresponding face attribute, and j is a positive integer. The feature image I _i and the attention matrix V _{ij are} subjected to a dot product operation, and the face attribute feature A _ij of the j- th attribute corresponding to the feature image I _{i is} extracted. The acquisition process of each attention feature is as follows (1-1):

(1-1); Among them, Attention is an attention model,

It can be the mapping relationship between the feature image I _i and the attention matrix V _{ij, where}

It can be obtained by training a large number of known feature images I and attention matrices with known attributes. For the sake of simplicity, the above only stated the extraction of the face attribute feature A _ij of the j- th attribute of the feature image I _i . In practical applications, the face attribute features A _{i 1} , A _{i 2} in the feature image I _i ,..., The extraction method of A _im is similar to the extraction method of the face attribute feature A _ij of the j- th attribute of the feature image I _{i, and will not be repeated here.} In addition, the face attribute features A _{1 j} , A _{2 j} ,..., A _nj of the j- th attribute in the feature images I ₁ , I ₂ ,..., I _n are all extracted in the same way as the face attribute features in the feature image I _i . The method for extracting the face attribute features A _ij of j attributes is similar, and will not be repeated here.

Global average pooling layer 240:

（1-2）； 這裡，P_ij 表示特徵圖像I_i 的第j 個屬性的預測結果，GAP表示全域平均池化。The n × m face attribute features A ₁₁ , A ₁₂ ,..., A _{1 m} , A ₂₁ , A ₂₂ ,..., A _{2 m} ,..., A _{n 1} , A _{n 2} ,... , a _nm as average input gamut pooled layer, followed by pooling window through pooled to give n × m recognition results P _ij, wherein P _ij of each of the recognition process of obtaining the following equation (1-2):

(1-2); Here, P _ij represents the prediction result of the j- th attribute of the feature image I _i , and GAP represents global average pooling.

Recognition layer 250.

The n × m recognition results P _ij output by the global average pooling layer are classified into the recognition results of the first attribute P ₁₁ , P ₂₁ ,..., P _{n 1 according to} attributes, and the recognition results of the second attribute P ₁₂ , P ₂₂ ,..., P _{n 2} ,..., the recognition result of the m-th attribute P _{1 m} , P _{2 m} ,..., P _nm , the weighted average calculation is performed on the recognition result of each attribute to obtain the face image The recognition result of the attribute in.

（1-3）； 這裡，w_ij 是權重數值，w_ij 可以通過大量的已知屬性的預測結果以及對應的屬性識別結果進行訓練得到。為了陳述簡便，上面只陳述了人臉圖像的第j 個屬性的識別結果P _j 的獲取過程，實際應用中，人臉圖像的其他屬性的識別結果與人臉圖像的第j 個屬性的識別結果P _j 的獲取過程類似，此處不再進行贅述。Take the recognition result of the j-th face attribute as an example: calculate the weighted average of the recognition results P _{1 j} , P _{2 j} ,..., P _nj of the j- th attribute of the feature images I ₁ , I ₂ ,..., I _n , Obtain the recognition result P _j of the j-th face attribute of the face image, where the process of obtaining the recognition result P _j of the j-th attribute is as follows:

(1-3); Here, w _ij is the weight value, and w _ij can be obtained through training of a large number of known attribute prediction results and corresponding attribute recognition results. For the sake of simplicity, the above only stated the process of obtaining the recognition result P _j of the j- th attribute of the face image. In practical applications, the recognition results of other attributes of the face image are the same as the j- th attribute of the face image. The process of obtaining the recognition result P _j is similar, and will not be repeated here.

For example, the face image of the target object is input into the aforementioned facial attribute recognition neural network, and the input layer 210, the convolutional layer 220, the attention learning layer 230, the global average pooling layer 240, and the recognition layer 250 are calculated , The gender attribute result of the target object can be obtained. For example, the probability that the gender of the target object is male is 0.8, and the probability that the gender is female is 0.2, then the recognition layer 250 outputs the result that the target object is male.

It is understandable that the aforementioned facial attribute recognition neural network is merely an example. In practical applications, it can also be other deformed forms of the aforementioned neural network. For example, it can include more convolutional layers and more pooling layers. A fully connected layer can also be used instead of the global average pooling layer. In addition, more preprocessing such as binarization and standardization can also be performed on the input layer of the face image, which is not specifically limited in the embodiment of the present invention.

It should be noted that the above-mentioned face attribute recognition network is a convolutional neural network, where the convolutional neural network can include residual neural network (Residual Network, ResNet), VGG network (VGG Network, VGGNet), etc. Etc., there is no specific limitation here. The recognition of each attribute of the face image can be performed in the same convolutional neural network, or can be performed in different convolutional neural networks, and there is no specific limitation here. In addition, each feature image of the face image can be extracted with the same convolution kernel, or different convolution kernels can be used for extraction, which is not specifically limited here.

Based on the facial attribute recognition neural network shown in FIG. 2, an embodiment of the present invention provides a face attribute recognition method, as shown in FIG. 3, which is a flow chart of a face attribute recognition method provided by an embodiment of the present invention Schematic. The face attribute recognition method of the embodiment of the present invention includes the following steps.

S101. Acquire n feature images, where the n feature images are obtained by performing feature extraction on a face image one or more times, and n is a positive integer.

In the embodiment of the present invention, before obtaining the n feature images, first obtain the face image, and then perform the feature image extraction on the face image through the feature image extraction model to obtain the above n feature images . It should be understood that before performing feature image extraction on the collected face image, image preprocessing may also be performed on the face image, for example, image binarization processing or image standardization processing, etc., Then, feature extraction is performed on the preprocessed face image, so that the obtained feature image can contain more attribute features that can be used to identify the attributes of the face.

In the embodiment of the present invention, the feature image extraction model is obtained through training of face image samples and feature image samples. In the implementation, the feature image extraction model can be obtained by convolutional neural network training, or it can be obtained by the Scale-Invariant Feature Transform (SIFT) algorithm training, which is not specifically limited here. In the case of using a convolutional neural network to train to obtain a feature image extraction model, the feature image extraction model may include one convolutional layer or multiple convolutional layers, which is not specifically limited here.

In some embodiments of the present invention, the characteristic image may be a face image, or may be an image obtained by inputting a face image into one or more convolutional layers, which is not specifically limited here. In addition, the extraction of the aforementioned n feature images may use the same convolution kernel respectively, or may use different convolution kernels respectively, which is not specifically limited here.

S102. Input each feature image of the n feature images into a corresponding attention model to obtain an attention matrix corresponding to each feature image.

（1-4）； 其中，x 是特徵圖像，y 是注意力矩陣，

是注意力矩陣與輸入的特徵圖像之間的映射關係。

可以是通過大量的樣本集訓練得到，其中，每一個樣本集包括特徵圖像樣本以及對應的注意力矩陣樣本，一個樣本集用於訓練一個注意力模型。In the embodiment of the present invention, the neural network is trained using the above-mentioned characteristic image samples and the corresponding attention matrix samples to obtain an attention model. For example, take the feature image obtained by the target face image through 3 convolutional layers as the feature image sample, and extract the weight matrix of the target object's eye size in the face image as the attention matrix sample, using the above feature map The image sample and the attention matrix sample can be trained to obtain the attention model used to extract the eye size of the target object in the face image. In other words, the attention model reflects the mapping relationship between the attention matrix and the feature image. The attention model can be expressed as the following formula (1-4):

(1-4); where x is the feature image, y is the attention matrix,

It is the mapping relationship between the attention matrix and the input feature image.

It can be obtained by training a large number of sample sets, where each sample set includes a characteristic image sample and a corresponding attention matrix sample, and one sample set is used to train an attention model.

In the embodiment of the present invention, the attention model can be obtained through convolutional neural network training, where the convolutional neural network can be a network with a simple structure, such as a small network with only two convolutional layers, so that Efficiently and accurately extract the attribute characteristics of the target object's eye size in the face image. The convolutional neural network can also be a complex network with 10 convolutional layers, configured to extract the age of the target object in the face image. The attribute characteristics of the attributes are not specifically limited here, and the specific process of the attention model training is not repeated here. In addition, the convolutional neural network may be a residual neural network (Residual Network, ResNet), or a VGG network (VGG Network, VGGNet), etc., which is not specifically limited here.

In the embodiment of the present invention, in the case of recognizing m attributes in a face image, m attention models need to be trained, where one attention model is used to extract the attribute feature of one attribute. Wherein each of the n image feature image of m may be inputted attention model is obtained for each of m corresponding to the image feature matrix attention, and n corresponding to the image feature number n × m matrix attention . Among them, an attention matrix is used to extract a face attribute feature of the corresponding feature image, and m is a positive integer.

In the embodiment of the present invention, the above attributes may include many kinds. Taking a human face as an example, the attributes of a human face include local attributes and global attributes. The local attributes include hairstyle, eyebrow density, eye size, eyeball color, and nose bridge height. , Whether to wear glasses, whether to wear accessories, etc.; global attributes include gender, age, ethnicity, facial expressions, etc.

S103. According to the attention matrix, extract the attribute feature corresponding to each feature image in the n feature images, where the attribute feature is a feature used to characterize the attributes of the target face in the feature image.

In the embodiment of the present invention, each feature image of the n feature images is multiplied with the corresponding attention matrix to obtain the attribute feature corresponding to each feature image of the n feature images. Among them, the attribute feature is a feature used to characterize the attributes of the target face in the feature image. For example, when the recognized face attribute is whether to wear glasses, the attribute feature is a feature related to the eye area.

It will be appreciated, the case where m attributes to be identified, according to the characteristics of the image corresponding to the n n × m matrix number attention, the attention feature image corresponding to a matrix multiplication in the face image, n may be acquired attributes of the features of the image obtained with a number n × m, comprising: m attributes of a feature characteristic of the image _{a 11, a 12, ...,} a 1 m, the second feature image m attribute features A ₂₁ , A ₂₂ ,..., A _{2 m} , .., m attribute features A _{n 1} , A _{n 2} ,..., A _{nm of} the nth feature image, where A _nm is the nth The features used to characterize the face attributes of the m-th person in each feature image.

S104: Determine the attribute recognition result of the target face attribute in the face image according to the attribute feature.

In some embodiments of the present invention, determining the attribute recognition result of the target face attribute in the face image according to the attribute feature includes: predicting the corresponding recognition result according to each attribute feature; weighting the recognition result Average calculation to determine the attribute recognition result of the target face attribute in the face image.

In some embodiments of the present invention, predicting the corresponding recognition result according to each attribute feature includes: performing attribute prediction processing on each attribute feature in the attribute feature, and predicting the recognition result corresponding to the attribute feature.

In some embodiments of the present invention, the attribute prediction processing includes global average pooling processing. It is possible to perform global average pooling on the attribute feature A ₁₁ to obtain the attribute recognition result P ₁₁ of the first attribute of the first feature image, and perform global average pooling on the attribute feature A ₂₁ to obtain the second feature image The attribute recognition result of the first attribute P ₂₁ ,..., the attribute characteristic A _nm is pooled globally to obtain the attribute recognition result P _nm of the m- th attribute of the n- th characteristic image; for the attribute recognition result P ₁₁ , P ₂₁ , …, P _{n 1} perform weighted average calculation to obtain the attribute recognition result of the first attribute of the face image, and perform weighted average calculation on the attribute recognition results P ₁₂ , P ₂₂ , …, P _{n 2} to obtain the person The attribute recognition result of the second attribute of the face image,..., the weighted average calculation is performed on the attribute recognition results P _{1 m} , P _{2 m} ,..., P _nm to obtain the attribute recognition result of the m-th attribute of the face image .

The above face attribute recognition method can obtain multiple different levels of feature images through the feature image extraction model based on the acquired face images, and then input these different levels of feature images into the attention model to obtain the corresponding The attention matrix of, by multiplying the feature image and the attention matrix to obtain multiple attribute features, so as to recognize each attribute in the face image more efficiently and accurately.

The following takes the recognition attributes as whether to wear glasses, gender, and expression as an example to illustrate the face attribute recognition method shown in FIG. 3, as shown in FIG. 4, which is a method for identifying a target object according to an embodiment of the present invention. Schematic diagram of the process of identifying whether to wear glasses, gender, and facial expressions. The steps of the embodiment of the present invention may be as follows.

1. Obtain the face image of the target object.

2. After a convolution calculation, the first feature image is obtained.

3. Extract the characteristics of whether the target object wears glasses, the gender characteristics of the target object, and the expression characteristics of the target object in the first characteristic image. First, the first feature image is input into the first attention model, the second attention model, and the third attention model, respectively, to obtain the first attention matrix, the second attention matrix, and the third attention matrix. Among them, the first attention matrix is used to extract the characteristics of whether the target object wears glasses, the second attention model is used to extract the gender characteristics of the target object, and the third attention model is used to extract the expression characteristics of the target object; A feature image is respectively multiplied by the first attention matrix, the second attention matrix, and the third attention matrix, so as to extract the feature of whether the target object wears glasses, the gender feature of the target object in the first feature image The facial expression characteristics of the target object.

4. Perform three more convolution calculations on the first feature image to obtain the second feature image.

5. Extract the characteristics of whether the target object wears glasses, the gender characteristics of the target object, and the expression characteristics of the target object in the second characteristic image. Refer to step 3 for the implementation steps.

6. Perform five convolution calculations on the second feature image to obtain the third feature image.

7. Extract the characteristics of whether the target object wears glasses, the gender characteristics of the target object, and the expression characteristics of the target object in the third characteristic image. Refer to step 3 for the implementation steps.

8. Perform the global average pooling operation on the 9 features extracted in step 3, step 5 and step 7 to obtain corresponding recognition results respectively.

9. Perform a weighted average calculation on the recognition results of whether the target object wears glasses in the first characteristic image, the second characteristic image, and the third characteristic image to obtain whether the target object wears glasses or does not wear glasses.

。因此，人臉屬性識別裝置輸出目標對象未配戴眼鏡。For example, suppose that the probability of the target object wearing glasses in the first feature image is 0.4, and the probability of not wearing glasses is 0.6; the recognition result of the target object wearing glasses in the second feature image is 0.25, and the probability of not wearing glasses is 0.25 The probability of is 0.75; the recognition result of the target object wearing glasses in the third feature image is 0.2, and the probability of not wearing glasses is 0.8; and the weight of the first feature image in identifying whether the target object wears glasses is 0.2 , The weight of the second feature image in identifying whether the target object wears glasses is 0.4, and the weight of the second feature image in identifying whether the target object wears glasses is 0.4. After calculation, it can be known that the target object wears glasses The probability of is 0.4x0.2+0.25x0.4+0.2x0.4=0.26, and the probability of the target not wearing glasses is

. Therefore, the face attribute recognition device outputs that the target object does not wear glasses.

10. Perform a weighted average calculation on the gender recognition results of the target object in the first feature image, the second feature image, and the third feature image to obtain that the target object is male or female. See step 9 for implementation steps.

11. Perform a weighted average calculation on the recognition results of the expressions of the target object in the first feature image, the second feature image, and the third feature image to obtain whether the target object is happy, sad, or angry, and so on. See step 9 for implementation steps.

With reference to the related embodiments shown in Figures 1 to 4 above, the related devices involved in the embodiments of the present invention are described below.

Referring to FIG. 5, FIG. 5 is a schematic structural diagram of a face attribute recognition apparatus provided by an embodiment of the present invention. The face attribute recognition device in the embodiment of the present invention includes a feature image extraction unit 510, an attribute feature extraction unit 520, and a face attribute recognition unit 530.

The feature image extraction unit 510 is configured to obtain n feature images, where the n feature images are obtained by performing feature extraction on a face image one or more times, and n is a positive integer.

The attribute feature extraction unit 520 is configured to obtain the attribute feature corresponding to each feature image in the n feature images, where the attribute feature is a feature used to characterize the attributes of the target face in the feature image.

The face attribute recognition unit 530 is configured to determine the attribute recognition result of the target face attribute in the face image according to the attribute feature.

In the embodiment of the present invention, the attribute feature extraction unit 520 is further configured to: input each feature image of the n feature images into the corresponding attention model to obtain the attention matrix corresponding to each feature image; The attention matrix extracts the attribute feature corresponding to each of the n feature images.

In an embodiment of the present invention, the attribute feature extraction unit 520 is further configured to: n feature images corresponding to each of the attention image feature matrix multiplication to obtain the n wherein each feature image The attribute characteristics corresponding to the image.

In the embodiment of the present invention, the face attribute recognition unit 530 is further configured to: respectively obtain a corresponding recognition result according to each attribute feature; perform a weighted average calculation on the recognition result to determine the target face attribute in the face image Attribute recognition result.

In the embodiment of the present invention, the face attribute recognition unit 530 is further configured to perform attribute prediction processing on each attribute feature in the attribute feature, and predict the recognition result corresponding to the attribute feature.

In some embodiments of the present invention, the attribute prediction processing includes global average pooling processing.

In the embodiment of the present invention, before acquiring the n feature images, the face attribute recognition device 500 is further configured to: acquire a face image; perform feature extraction on the face image through a feature image extraction model , To obtain n feature images.

In some embodiments of the present invention, the characteristic image may be a face image, or may be obtained by inputting the face image into one or more convolutional layers, which is not specifically limited here. In addition, the extraction of n feature images may use the same convolution kernel or different convolution kernels, which is not specifically limited here.

In some embodiments of the present invention, the face attribute recognition device 500 further includes: a feature image extraction model training unit 540 and an attention model training unit 550.

The feature image extraction model training unit 540 is configured to train a feature image extraction model, and the feature image extraction model is obtained through training of face image samples and feature image samples. In the implementation, the feature image extraction model can be obtained by convolutional neural network training, or it can be obtained by the Scale-Invariant Feature Transform (SIFT) algorithm training, which is not specifically limited here. In the case of using a convolutional neural network to train to obtain a feature image extraction model, the feature image extraction model may include one convolutional layer or multiple convolutional layers, which is not specifically limited here.

（1-5）； 其中，x 是特徵圖像，y 是注意力矩陣，

是注意力矩陣與輸入的特徵圖像之間的映射關係。

可以是通過大量的樣本集訓練得到，其中，每一個樣本集包括特徵圖像樣本以及對應的注意力矩陣樣本，一個樣本集用於訓練一個注意力模型。The attention model training unit 550 is configured to train the neural network using the above-mentioned characteristic image samples and the corresponding attention matrix samples to obtain an attention model. For example, take the feature image obtained by the target face image through 3 convolutional layers as the feature image sample, and extract the weight matrix of the target object's eye size in the face image as the attention matrix sample, using the above feature map The image sample and the attention matrix sample can be trained to obtain the attention model used to extract the eye size of the target object in the face image. In other words, the attention model reflects the mapping relationship between the attention matrix and the feature image. The attention model can be expressed as the following formula (1-5):

(1-5); where x is the feature image, y is the attention matrix,

It is the mapping relationship between the attention matrix and the input feature image.

It can be obtained by training a large number of sample sets, where each sample set includes a characteristic image sample and a corresponding attention matrix sample, and one sample set is used to train an attention model.

In the embodiment of the present invention, the attention model can be obtained through convolutional neural network training, where the convolutional neural network can be a network with a simple structure, such as a small network with only two convolutional layers, so that Efficiently and accurately extract the attribute characteristics of the target object's eye size in the face image. The convolutional neural network can also be a complex network with 10 convolutional layers, configured to extract the age of the target object in the face image. The attribute characteristics of the attributes are not specifically limited here, and the specific process of the attention model training is not repeated here. In addition, the convolutional neural network may be a residual neural network (Residual Network, ResNet), or a VGG network (VGG Network, VGGNet), etc., which is not specifically limited here.

It can be understood that the recognition of attributes such as gender or expression of the target object belongs to global attribute recognition. Therefore, when judging the global attributes of the target object (such as gender or expression), multiple local attributes (such as hairstyle, whether to wear or not) can be judged first. Earrings or eye size, etc.). In other words, the above-mentioned face attribute recognition device may first recognize the local attribute characteristics of the target object, and comprehensively evaluate the global attributes of the face according to the recognition result of the local attributes.

It should be noted that the aforementioned facial recognition device can also be used in the security field to help security staff quickly and accurately screen offensive persons. For example, at an airport boarding gate, using the aforementioned facial recognition device can identify passengers’ Face attribute characteristics to determine whether the flight crew is aggressive. In the case that the flight crew is aggressive, the flight crew's facial image is compared with the existing facial image database to determine the flight crew's identity information.

The above-mentioned face attribute recognition device performs feature extraction on the acquired face image through the feature image extraction unit to obtain multiple feature images of different levels, and then combine these feature images of different levels in the attention feature learning unit Enter the attention model to obtain the corresponding attention matrix, and obtain multiple attribute features by multiplying the feature image and the attention matrix. Finally, in the face attribute recognition unit, the multiple attribute features are weighted and averaged, thereby Recognize each attribute in the face image more efficiently and accurately.

The face recognition apparatus of the embodiment of the present invention may be implemented in a single computing node, or may be implemented on a cloud computing infrastructure, which is not specifically limited here. The following will separately introduce how to implement a face recognition device on a single computing node and cloud computing infrastructure.

As shown in FIG. 6, the computing node 600 may include a processor 610, a memory 620, and a communication network 630. The processor is configured to run a feature image extraction unit 611, an attribute feature extraction unit 612, a face attribute recognition unit 613, a feature image extraction model training unit 614, an attention model training unit 615, and so on. The memory 620 is configured to store face images, feature images, attribute features, attention matrix, and so on. The computing node 600 also provides two external interfaces, namely, a management interface 640 for maintenance personnel of the device and a user interface 650 for users. Among them, the form of the interface can be various, such as web interface, command line tool, REST interface and so on.

In some embodiments of the present invention, the management interface 640 is configured to allow maintenance personnel to input a large number of face image samples and corresponding feature image samples to train the feature image extraction model; it can also input a large number of face image samples and corresponding feature image samples. The feature image samples and the corresponding attention matrix samples are used to train the attention model; the user interface 650 is configured for the user to input the face image of the target object that needs to be attributed, and through the user interface 650 outputs the recognition result of face attributes to the user.

It should be understood that the computing node 600 is only an example provided by the embodiment of the present invention, and the computing node 600 may have more or fewer components than shown, may combine two or more components, or may have Different configurations of components are realized.

As shown in FIG. 7, the cloud computing infrastructure may be a cloud service cluster 700. The cloud service cluster 700 is composed of nodes and a communication network between the nodes. The aforementioned nodes may be computing nodes or virtual machines running on computing nodes. Nodes can be divided into two categories according to their functions: computing nodes 710, storage nodes 720, and communication interfaces 730. The computing node 710 is configured to run a feature image extraction unit 711, an attribute feature extraction unit 712, a face attribute recognition unit 713, a feature image extraction model training unit 714, and an attention model training unit 715. The storage node 720 is configured to store face images, feature images, attribute features, and so on. The cloud service cluster 700 also provides two external interfaces, namely a management interface 740 for maintenance personnel of the question and answer engine and a user interface 750 for users. Among them, the form of the interface can be diverse, such as a web interface, a command line tool, a Representational State Transfer (REST) interface, and so on.

In some embodiments of the present invention, the management interface 740 is configured to allow maintenance personnel to input a large number of face image samples and corresponding feature image samples to train the feature image extraction model; it can also input a large number of face image samples and corresponding feature image samples. The feature image samples and the corresponding attention matrix samples are used to train the attention model; the user interface 750 is configured for the user to input the face image of the target object that needs to be attributed, and through the user interface 750 outputs the recognition result of face attributes to the user.

It should be understood that the cloud service cluster 700 is only an example provided by the embodiment of the present invention, and the cloud service cluster 700 may have more or fewer components than shown, two or more components may be combined, or It can be implemented with different configurations of components.

Referring to FIG. 8, an embodiment of the present invention provides a schematic structural diagram of a face attribute recognition apparatus 800 of another embodiment. The face attribute recognition apparatus of this embodiment can be implemented in a computer node as shown in FIG. 6, and includes at least: The processor 810, the communication interface 820, and the memory 830, wherein the processor 810, the communication interface 820, and the memory 830 are coupled through the bus 840.

The processor 810 includes one or more general-purpose processors, where the general-purpose processor can be any type of device capable of processing electronic instructions, including a central processing unit (Central Processing Unit, CPU), a microprocessor, a microcontroller, and a host Processor, controller and application specific integrated circuit (Application Specific Integrated Circuit, ASIC) and so on. The processor 810 reads the program code stored in the memory 830, and cooperates with the communication interface 820 to execute part or all of the steps of the method executed by the face attribute recognition apparatus 500 in the foregoing embodiment of the present invention.

The communication interface 820 may be a wired interface (for example, an Ethernet interface), and is configured to communicate with other computing nodes or devices.

The memory 830 may include volatile memory, such as random access memory (Random Access Memory, RAM); the memory may also include non-volatile memory (Non-Volatile Memory), such as read-only memory (Read Only Memory). -Only Memory, ROM, Flash Memory, Hard Disk Drive (HDD), or Solid-State Drive (SSD) memory may also include a combination of the above types of memory. The memory 813 can store program codes and program data. Among them, the program code includes the code of the feature image extraction unit, the code of the attribute feature extraction unit, the code of the face attribute recognition unit, the code of the feature image extraction model training unit, and the code of the attention model training unit. The program data includes a large number of face images, feature images and the corresponding attention matrix, which are used to train the feature image extraction model and attention model; the program data also includes a large number of weight values used to calculate the attribute recognition results, etc. Wait.

Wherein, the processor 810 is used to execute the following steps by calling the program code in the memory 830: The processor 810 is configured to obtain n feature images, where n feature images are performed on the face image once. Or obtained after multiple feature extractions, n is a positive integer; the processor 810 is configured to obtain the attribute feature corresponding to each feature image in the n feature images, where the attribute feature is used to characterize the target in the feature image The feature of the face attribute; the processor 810 is configured to determine the attribute recognition result of the target face attribute of the face image according to the attribute feature.

The processor 810 is further configured to train a feature image extraction model using face image samples and feature image samples; and use the feature image samples and corresponding attention matrix samples to train the neural network to obtain the attention Force model.

Refer to FIG. 9, which is a schematic structural diagram of a face attribute recognition apparatus according to another implementation manner according to an embodiment of the present invention. The face attribute recognition apparatus of this embodiment can be implemented in a cloud service cluster as shown in FIG. 7, and includes at least one computing node 910 and at least one storage node 920.

The computing node 910 includes one or more processors 911, a communication interface 912, and a memory 913, and the processor 911, the communication interface 912, and the memory 913 may be connected through a bus 914.

The processor 911 includes one or more general-purpose processors, where the general-purpose processor may be any type of device capable of processing electronic instructions, including a CPU, a microprocessor, a microcontroller, a main processor, a controller, and an ASIC, etc. . It can be a dedicated processor used only for the computing node 910 or can be shared with other computing nodes 910. The processor 911 reads the program code stored in the memory 913, and cooperates with the communication interface 912 to execute part or all of the steps of the method executed by the face attribute recognition apparatus 500 in the foregoing embodiment of the present invention.

The communication interface 912 may be a wired interface (for example, an Ethernet interface), configured to communicate with other computing nodes or users. When the communication interface 912 is a wired interface, the communication interface 912 may adopt a protocol family over Transmission Control Protocol/Internet Protocol (TCP/IP), for example, Robot as a Service (RAAS). ) Protocol, remote function call (Remote Function Call, RFC) protocol, simple object access protocol (Simple Object Access Protocol, SOAP) protocol, simple network management protocol (Simple Network Management Protocol, SNMP) protocol, public object request agent Architecture (Common Object Request Broker Architecture, CORBA) agreement and decentralized agreement, etc.

The memory 913 may include volatile memory, such as RAM; the memory may also include non-volatile memory, such as ROM, flash memory, HDD, or solid-state drive. SSD memory may also include the above-mentioned types of memory. The combination.

The storage node 920 includes one or more processors 911, a communication interface 912, and a memory 913. Among them, the storage controller 921, the communication interface 912, and the memory 913 can be connected through a bus 924.

The processor 911 includes one or more general-purpose processors, where the general-purpose processor may be any type of device capable of processing electronic instructions, including a CPU, a microprocessor, a microcontroller, a main processor, a controller, and an ASIC, etc. . It can be a dedicated processor used only for the storage node 920 or can be shared with other storage nodes 920. The processor 911 executes various types of digital storage instructions, such as software or firmware programs stored in the memory 913, which enables the storage node 920 to provide a wide variety of services. For example, the processor 911 can execute programs or process data to perform at least a part of the methods discussed herein.

The communication interface 912 may be a wired interface (for example, an Ethernet interface), configured to communicate with other computing devices or users.

The storage node 920 includes one or more storage controllers 921 and a storage array 922. Among them, the storage controller 921 and the storage array 922 can be connected through a bus 923.

The storage controller 921 includes one or more general-purpose processors, where the general-purpose processor may be any type of device capable of processing electronic instructions, including a CPU, a microprocessor, a microcontroller, a main processor, a controller, and an ASIC, etc. Wait. It can be a dedicated processor used only for a single storage node 920 or can be shared with the computing node 900 or other storage nodes 920. It can be understood that in this embodiment, each storage node includes a storage controller. In other embodiments, multiple storage nodes may share a storage controller, which is not specifically limited here.

The storage array 922 may include multiple memories. The memory may be a non-volatile memory, for example, ROM, flash memory, HDD or SSD memory may also include a combination of the above types of memory. For example, the storage array may be composed of multiple HDDs or multiple SDDs, or the storage array may be composed of HDDs and SDDs. Among them, multiple memories are combined in different ways with the assistance of the storage controller 921 to form a memory group, thereby providing higher storage performance than a single memory and providing data backup technology. Optionally, the storage array 922 may include one or more data centers. Multiple data centers can be set up at the same location, or in different locations, and there is no specific limitation here. The storage array 922 can store program codes and program data. Among them, the program code includes the code of the feature image extraction unit, the code of the attention feature learning unit, the code of the face attribute recognition unit, and the code of the attention model training unit. The program data includes a large number of face images, feature images and the corresponding attention matrix, which are used to train the feature image extraction model and attention model; the program data also includes a large number of weight values used to calculate the attribute recognition results, etc. Wait.

The computing node 910 is configured to perform the following steps by calling the program code in the storage node 920: The computing node 910 is configured to obtain n feature images, where n feature images are performed on the face image once or more. N is a positive integer obtained after sub-feature extraction; the computing node 910 is configured to obtain the attribute feature corresponding to each feature image in the n feature images, where the attribute feature is used to represent the target face in the feature image The feature of the attribute; the computing node 910 is configured to determine the attribute recognition result of the target face attribute of the face image according to the attribute feature.

The computing node 910 is also configured to use face image samples and feature image samples to train to obtain a feature image extraction model; and use the aforementioned feature image samples and corresponding attention matrix samples to train the neural network to obtain the attention Force model.

Multi-worker face attribute learning is currently a very hot and thorny problem in the field of video surveillance security, and it plays a vital role in the field of security. For example, it can be judged whether a person is aggressive through facial expression attribute recognition, and whether pedestrians are safe through identification of wearing dangerous objects. Related technologies can use a multi-branch neural network to share features and classify individual attributes (such as wearing earrings, eyes, and expressions). The network is very simple but has obvious shortcomings. Take a multiplexed convolutional neural network as an example. In this network, multiplexes often share all the features of the previous network, and then perform GAP in the last layer of the convolutional network to extract the shared semantic feature F, and use this F to perform Different classification tasks. The problem caused by this is that not all the features required by the attributes are the features of the last layer. For example, some subtle features such as wearing earrings and glasses may require relatively shallow two-dimensional features with an attention mechanism. Some attributes like gender and facial expressions require deeper semantic features.

The embodiments of the present invention are improved on the basis of the network, and provide a method that can recognize face attributes more efficiently and accurately. This method uses deep convolutional neural network (CNN) to extract facial image features, obtain attention features in different layers, and then use these attention features to classify, and finally all classification results are weighted and averaged to get the final Classification results. In implementation, the standard neural network can be divided into multiple blocks of different scales. In fact, many neural networks have been divided into blocks, such as VGG networks and residual networks; in areas of different scales N attention models are learned on block B _i , where N represents N tasks; N attention mechanisms are used for attribute classification, and on the same attribute, the prediction results P _ij on each block are weighted and averaged to get the final The result of P _{j ,} where i is the serial number of the block and j is the serial number of the attribute.

FIG. 10 is a schematic diagram of the implementation process of a face attribute recognition method provided by an embodiment of the present invention. As shown in FIG. 10, the neural network is divided into a plurality of blocks 101i of different scales, and the block 101i is one (N, W _i , H _i , C _i ) type tensor, where i=1,2..., n, n are the number of blocks, W _i is the image width corresponding to block 101i, and H _i is block 101i corresponding The image length of, C is the number of image channels corresponding to block 101i. For the face image to be recognized, feature extraction is performed on each block 101i, and a corresponding attention map 102i can be generated. The attention map 102i is a (N, W _i , H _i , 1) type tensor. Multiply each block 101i with the corresponding attention map 102i to obtain an attention feature, and then perform attribute classification on the attention feature through the GAP layer 1030 to obtain a prediction result 104i, which is a (N, 2) type Tensor. All prediction results 104i are weighted and averaged to obtain a total prediction result 1040. Finally, the total prediction result is used to classify the face attributes.

The face attribute recognition method provided by the embodiment of the present invention uses a multi-scale attention mechanism to dig deep into the features of different layers, and can use the features of each attribute at different depths in the network for attribute classification, thereby solving the problem of insufficient features in the classification in related technologies. A strong problem, compared with the related technology that only uses the last layer of semantic information for classification, it can recognize face attributes more efficiently and accurately.

In the above embodiments, it may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, it can be implemented in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on the computer, the processes or functions according to the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be from a website, computer, server, or data. The center transmits to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, a data center, or the like integrated with one or more available media. The usable medium may be a magnetic medium, (for example, a floppy disk, a storage disk, and a magnetic tape), an optical medium (for example, a Digital Video Disc (DVD)), or a semiconductor medium (for example, SSD). In the embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in an embodiment, reference may be made to related descriptions of other embodiments.

In the several embodiments provided by the present invention, it should be understood that the disclosed device may also be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or elements can be combined or integrated into Another system, or some features can be ignored or not implemented. In addition, the displayed or discussed indirect coupling or direct coupling or communication connection between each other may be indirect coupling or communication connection through some interfaces, devices or units, and may be in electrical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. on. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present invention.

In addition, the functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present invention essentially or the part that contributes to the existing technology or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium may include, for example, various media capable of storing program codes, such as a U disk, a removable hard disk, a read-only memory, a random access memory, a floppy disk, or an optical disk.

The above are only specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily think of various equivalent modifications within the technical scope disclosed by the present invention. Or replacement, these modifications or replacements should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the patent application.

Industrial applicability The present invention relates to a face attribute recognition method, an electronic device and a computer-readable storage medium. The method includes: obtaining n feature images, wherein the n feature images are performed on the face image once or more. Obtained after sub-feature extraction, n is a positive integer; acquiring the attribute feature corresponding to each feature image in the n feature images, where the attribute feature is used to characterize the target person in the feature image The feature of the face attribute; according to the attribute feature, the attribute recognition result of the target face attribute of the face image is determined. According to the face attribute recognition method of the embodiment of the present invention, the attribute features corresponding to different feature images can be used to predict the face attributes, which can improve the accuracy and efficiency of face attribute recognition.

110: Input layer 120: Shared feature learning layer 130: Fully connected layer 210: Input layer 220: Convolutional layer 230: Attention Learning Layer 240: Global average pooling layer 250: recognition layer 500: Face attribute recognition device 510: Feature image extraction unit 520: Attribute feature extraction unit 530: face attribute recognition unit 540: Feature image extraction model training unit 550: Attention Model Training Unit 600: compute node 610: processor 611: Feature image extraction unit 612: Attribute feature extraction unit 613: Face Attribute Recognition Unit 614: Feature image extraction model training unit 615: Attention Model Training Unit 620: memory 630: Communication Network 640: Management Interface 650: User Interface 700: Cloud service cluster 710: compute node 711: Feature image extraction unit 712: Attribute Feature Extraction Unit 713: Face Attribute Recognition Unit 714: Feature image extraction model training unit 715: Attention Model Training Unit 720: storage node 730: Communication interface 740: Management Interface 750: User Interface 800: Face attribute recognition device 810: processor 820: Communication interface 830: memory 840: Bus 900: compute node 910: compute node 911: processor 912: Communication Interface 913: Memory 914: Bus 920: storage node 921: Storage Controller 922: Storage Array 923: Bus 924: Bus S101~S104: steps

In order to explain the technical solutions of the embodiments of the present invention more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present invention. Ordinary technicians can obtain other drawings based on these drawings without creative work. Figure 1 is a schematic diagram of a process of identifying face attributes based on MTCNN in related technologies. Fig. 2 is a schematic structural diagram of a facial attribute recognition neural network provided by an embodiment of the present invention. FIG. 3 is a schematic flowchart of a method for identifying face attributes according to an embodiment of the present invention. FIG. 4 is a schematic diagram of a process for recognizing whether a target object wears glasses, gender, and expression provided by an embodiment of the present invention. Fig. 5 is a schematic structural diagram of a face attribute recognition device provided by an embodiment of the present invention. Fig. 6 is a schematic structural diagram of a computing node provided by an embodiment of the present invention. Fig. 7 is a schematic structural diagram of a cloud service cluster provided by an embodiment of the present invention. Fig. 8 is a schematic structural diagram of another face attribute recognition apparatus provided by an embodiment of the present invention. Fig. 9 is a schematic structural diagram of another face attribute recognition device provided by an embodiment of the present invention. FIG. 10 is a schematic diagram of the implementation process of a method for identifying face attributes according to an embodiment of the present invention.

S101~S104: steps

Claims (11)

A method for face attribute recognition includes: obtaining n feature images, where the n feature images are obtained after one or more feature extractions are performed on the face image, and n is a positive integer; The attribute feature corresponding to each feature image in the n feature images, where the attribute feature is a feature used to characterize the attributes of the target face in the feature image; and the attribute feature is determined according to the attribute feature The attribute recognition result of the target face attribute in the face image. The method of claim 1 request entries, wherein each of said acquiring a characteristic image corresponding to the characteristic attributes of the n feature images, comprising: a feature of each of the n image feature image The corresponding attention models are input respectively to obtain the attention matrix corresponding to each of the feature images; and the attribute feature corresponding to each feature image of the n feature images is extracted according to the attention matrix. The method according to claim 2, wherein extracting the attribute characteristics corresponding to each of the n characteristic images according to the attention matrix includes: combining each of the n characteristic images The feature image and the corresponding attention matrix are multiplied to obtain the attribute feature corresponding to each feature image of the n feature images. The method according to any one of claim items 1 to 3, wherein the determining the attribute recognition result of the target face attribute in the face image according to the attribute feature includes: According to each of the attribute characteristics, the corresponding recognition result is predicted; A weighted average calculation is performed on the recognition result to determine the attribute recognition result of the target face attribute in the face image. The method according to claim 4, wherein the predicting and obtaining the corresponding recognition result respectively according to each of the attribute characteristics includes: Attribute prediction processing is performed on each of the attribute characteristics, and the recognition result corresponding to the attribute characteristic is predicted. The method according to claim 5, wherein the attribute prediction processing includes global average pooling processing. The method according to any one of claim items 1 to 3, wherein, before the obtaining the n feature images, the method further includes: obtaining the face image; Perform feature extraction on the face image to obtain the n feature images. The method according to claim 7, wherein the feature image extraction model is obtained by training on face image samples and feature image samples, and the method further includes: Use the characteristic image samples and the corresponding attention matrix samples to train the neural network to obtain the attention model. The method according to any one of claim items 1 to 3, wherein the n characteristic images further include the face image. A computer-readable storage medium, the computer-readable storage medium stores a computer program, and the computer program is executed by hardware to implement the method described in any one of claim items 1 to 9. An electronic device includes a processor and a memory, and the processor executes the code in the memory to implement the method described in any one of claim items 1 to 9.

Images