KR20210113607A - Facial attribute recognition method and apparatus, electronic device and storage medium - Google Patents

Abstract

The present invention relates to a method and apparatus for recognizing a face attribute, an electronic device and a storage medium, the method comprising extracting facial features from an image (S10), and a plurality of types of faces preset based on the facial features performing attribute recognition to obtain a probability prediction result of each type of face attribute (S20), and determining a facial attribute recognition result in the image based on the probability prediction result of each type of face attribute (S30) By including, this method can simultaneously perform multiple types of face attribute recognition, and can improve recognition efficiency of face attribute recognition.

Description

Facial attribute recognition method and apparatus, electronic device and storage medium

The present invention claims the priority of a Chinese patent application filed with the Chinese Intellectual Property Office on December 29, 2018, the application number of which is 201811645459.1, and the title of the invention is "Method and Apparatus for Recognizing Facial Attributes, Electronic Devices and Storage Media"; The entirety of the invention is incorporated herein by reference.

The present invention relates to the field of image recognition technology, and more particularly, to a method and apparatus for recognizing facial attributes, an electronic device, and a storage medium.

In the conventional face attribute recognition technology, when it is necessary to perform a plurality of attribute recognition on a face, it is necessary to perform face recognition of a plurality of single attributes, and further match the face recognition results of a plurality of single attributes. Since the relation between the respective facial attributes is not taken into account, the accuracy of the facial recognition result of the plurality of attributes is poor, and the calculation speed is low.

The present invention provides technical means for facial attribute recognition.

According to one aspect of the present invention, extracting a facial feature from an image, performing a plurality of types of facial attribute recognition preset based on the facial characteristic to obtain a probability prediction result of each type of facial attribute, Provided is a method for recognizing a face attribute, comprising determining a result of recognizing a face attribute in the image based on a result of predicting probability of each type of face attribute.

According to an aspect of the present invention, a feature extraction module for extracting facial features from an image, and performing recognition of a plurality of types of facial attributes preset based on the facial features, a probability prediction result of each type of facial attributes It provides a face attribute recognition apparatus comprising: a probability prediction result acquisition module for obtaining .

According to one aspect of the present invention, there is provided an electronic device comprising a processor and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the method according to any one of the preceding claims.

According to one aspect of the present invention, there is provided a computer-readable storage medium storing computer program instructions, wherein when the computer program instructions are executed by a processor, the computer-readable storage medium realizes the method according to any one of the preceding claims. to provide.

According to an aspect of the present invention, there is provided a computer program comprising computer readable code, wherein the computer readable code executes instructions for realizing the facial attribute recognition method in a processor of the electronic device when the computer readable code is executed in an electronic device. to provide.

In an embodiment of the present invention, a facial feature is extracted from an image, and a plurality of types of preset facial attributes are recognized based on the facial features to obtain a probability prediction result of each type of facial attribute, and each type of face A face attribute recognition result in the image may be determined based on the attribute probability prediction result. Embodiments of the present invention can perform recognition of a plurality of types of facial attributes at the same time, and can improve recognition efficiency of facial attribute recognition.

It should be understood that the above general description and the following detailed description are merely exemplary and interpretative, and do not limit the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS Other features and aspects of the present invention will be clarified below by describing exemplary embodiments in detail with reference to the drawings.

Here, the drawings included as a part of this specification are suitable for the embodiment of the present invention, and together with the specification are used for the description of the technical means of the present invention.
1 is a flowchart of a face attribute recognition method according to an embodiment of the present invention.
2 is a flowchart of a face attribute recognition method according to an embodiment of the present invention.
3 is a schematic diagram of a training flow of a neural network in a facial attribute recognition method according to an embodiment of the present invention.
4 is a block diagram of an apparatus for recognizing facial attributes according to an embodiment of the present invention.
Fig. 5 is a block diagram of an electronic device according to an exemplary embodiment.
6 is a block diagram of an electronic device according to an exemplary embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Various exemplary embodiments, features and aspects of the present invention will be described in detail below with reference to the drawings. In the drawings, the same reference numerals indicate elements having the same or similar functions. Although the drawings have shown various aspects of the embodiments, the drawings are not necessarily drawn to scale unless otherwise noted.

As used herein, the term “exemplary” means “an example, used as an embodiment, or explanatory.” Any embodiment described herein as “exemplary” should not be construed as preferred or superior to other embodiments.

The term "and/or" in this specification simply describes a related relationship with a related object, and indicates that three relationships are possible, for example, A and/or B exists only in A, and A and B You may represent three cases in which all exist and only B exists. In addition, the term "one or more" in the present specification indicates any one of a plurality or any combination of at least two of the plurality, for example, A, B, and C including one or more of A, B, and C It may indicate including any one or a plurality of elements selected from the set.

In addition, various specific details are set forth in the following specific embodiments in order to more effectively explain the present invention. It should be understood by those skilled in the art that the present invention may be practiced without any specific details. In some embodiments, detailed descriptions of methods, means, elements and circuits known to those skilled in the art are not set forth in order to emphasize the spirit of the present invention.

1 is a flowchart of a face attribute recognition method according to an embodiment of the present invention. As shown in FIG. 1 , the face attribute recognition method includes steps S10 , S20 , and S30 .

In step S10, facial features are extracted from the image.

In one possible realization, the image may comprise a single image or a frame image of a video stream. The image may include each type of image such as an RGB (Red, Green, Blue) image.

In one possible implementation form, the original high-dimensional feature data of a face is described as low-dimensional data, and the extracted low-dimensional data may be used to effectively distinguish different faces. The facial features may include global facial features. The feature information is extracted from the entire face image, and may reflect the entire face information. The facial features may further include local facial features. The feature information is extracted from some pixels or a partial region of the face image, and may reflect local details of the face. In the present invention, the types and specific contents of facial features are not limited, and features according to each type of facial attributes can be extracted from an image according to the demand for facial attribute recognition.

In step S20, a plurality of preset types of facial attributes are recognized based on the facial features, and a probability prediction result of each type of facial attributes is obtained.

In one possible implementation form, the facial attribute may include one or more of age, gender, ornament, skin color, expression, and movement. The ornament includes one or more of ordinary glasses, sunglasses, a hat, and a mask, the expression includes one or more of joy, sadness, crying, anger, and excitement, and the action includes smoking a cigarette, drinking water Includes one or more of drinking, eating, putting on makeup, or making a phone call. According to demand, one or a plurality of attributes constituting one type of face attribute may be determined.

For example, a facial attribute may include age and gender (eg, a middle-aged man or an elderly man, etc.), or age, gender and ornament (eg, a middle-aged man wearing a hat, an elderly woman wearing glasses, etc.); or age, gender, skin color, facial expression, and behavior (eg, a middle-aged white male smiling on the phone, etc.).

In one possible implementation form, a plurality of types of facial attributes may be preset according to demand. In the present invention, the number and types of attributes included in each type of face attribute are not limited. For example, age(1), gender(1), and decoration(1) are facial attributes(1), age(2), gender(2), decoration(1), facial expression(1) and motion(1) Three types of facial attributes can be preset, such as face attribute (2) and gender (3), ornaments (2), facial expression (2), and facial attribute (3) that is motion (2).

In one possible implementation form, the facial features extracted from the image are distinguished according to each type of face attribute set in advance to obtain the characteristics of each type of facial attribute, and further classification processing is performed for the characteristics of each type of facial attribute. This may be performed to obtain a probability prediction result of each type of face attribute.

In step S30, a result of recognizing a face attribute in the image is determined based on a result of predicting the probability of each type of face attribute.

In one possible realization, the face attribute recognition result includes at least two attribute recognition results of the face.

In one possible implementation form, the facial attribute recognition result in the image may include one type or a plurality of types of facial attribute recognition results. A probability threshold may be set in advance, and one type or a plurality of types of face attributes having a probability prediction result greater than the probability threshold value may be determined as the face attribute recognition result in the image.

In this embodiment, facial features are extracted from the image, and a plurality of types of facial attributes are recognized in advance based on the facial features to obtain a probability prediction result of each type of facial attributes, and A result of recognizing a face attribute in an image may be determined based on the probability prediction result. In the embodiment of the present invention, a plurality of types of face attribute recognition can be performed simultaneously, and the recognition efficiency of face attribute recognition can be improved.

In one possible realization, the image includes an image of a road on which the vehicle is located and/or an image within the vehicle, which is captured by an in-vehicle camera installed in the vehicle, and the face in the image is a pedestrian on the road on which the vehicle is located, a driver of the vehicle. Includes the face of a passenger and/or a driver/passenger of another vehicle, and the attribute recognition result includes an attribute recognition result that satisfies one or more of a predetermined age, a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, and a predetermined motion do.

In one possible implementation mode, an on-vehicle camera may be installed in the vehicle. One in-vehicle camera may be installed to simultaneously capture an image in the vehicle and an image of a road on which the vehicle is located, and two cameras, a camera for capturing an image in the vehicle and a camera for capturing an image of a road on which the vehicle is located, are separately Although it may be provided, this is not limited in this invention. The in-vehicle camera may be a visible light camera, an infrared camera, a binocular camera, or the like, but an in-vehicle camera may be selected according to demand and application scenes.

In one possible implementation form, the on-vehicle camera may capture an image of a road on which the vehicle is located. In the image of the road on which the vehicle is located, the face of a pedestrian on the road may be included, or the face of a driver or passenger of another vehicle on the road may be included. According to the method according to an embodiment of the present invention, a result of recognition of facial attributes of a pedestrian on a road or a driver/passenger of another vehicle may be determined. A corresponding relationship between the face attribute recognition result and the preset first target object may be set in advance, and a face making a dangerous action (calling, putting on makeup, etc.) or a face with unstable emotions may be determined as the first target object. The first target object may be a target having a dangerous element that interferes with driving or causes a dangerous situation.

For example, a pedestrian A has an attribute recognition result 1 (children, running), a pedestrian B has an attribute recognition result 2 (elderly person, wearing sunglasses, standing), and a driver of the vehicle A is the attribute recognition result (3) (they are on the phone), and the passenger of the vehicle B is the attribute recognition result (4) (joy, smoking). The pedestrian A and the driver in the vehicle A may be determined as the first target object based on the attribute recognition result and the preset corresponding relationship between the first target object.

In one possible realization, the on-vehicle camera may capture an image within the vehicle that may include the vehicle's driver and passengers. The method according to an embodiment of the present invention may determine a result of recognition of facial attributes of a driver and/or a passenger of a vehicle. According to the demand, a face whose attribute recognition result satisfies at least one of a predetermined age, a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined expression, and a predetermined motion may be determined as the first target object. Here, the predetermined age may include children and the elderly, the predetermined gender may include women or men, the predetermined ornament may include sunglasses, the predetermined expression may include anger, sadness, etc. The action may include smoking a cigarette, putting on makeup, and the like. The respective setting conditions may be combined to obtain an attribute recognition result that satisfies the setting conditions, and a face that satisfies the setting conditions may be determined as the first target object. For example, the correspondence relationship between the face attribute recognition result and the second target object is set in advance, and a face performing a dangerous action (calling, putting on makeup, sleeping, etc.) or a face with unstable emotions is set as the second You may decide as a target object.

For example, the driver of a vehicle has an attribute recognition result (1) (eating something), a passenger (A) has an attribute recognition result (2) (wearing sunglasses, joy), and a passenger (B) has an attribute recognition result (3) ) (calling), the passenger C is the attribute recognition result (4) (joy, smoking). The driver of the vehicle may be determined as the second target object based on the attribute recognition result and the preset corresponding relationship between the second target object. A correspondence relationship between different attribute recognition results and a second target object may be set for the vehicle driver and the passenger.

In one possible implementation, the method

determining a first target object based on a result of recognizing face attributes in the image;

The method further includes transmitting presentation information so that the vehicle executes any one of maintaining speed, maintaining direction, decelerating, steering, and braking according to the determined first target object, and/or transmitting a control command to a vehicle control system. .

In one possible realization, the facial attribute recognition of an image in a vehicle and an image of a road on which the vehicle is located is applicable to an intelligent driving system, a driving assistance system, a safe driving system, and the like. The presentation information may be transmitted to the driver of the vehicle or the observer of the vehicle so that the vehicle executes the corresponding driving operation, and the set driving control information may be transmitted to the vehicle driving control system.

In one possible implementation form, the presentation information may be transmitted according to the determined first target object. In the present invention, the specific realization form and presentation content of the presentation information are not limited. For example, "There are children running ahead, so please be careful", "Drivers do not call. Please drive safely.” The driving control information set based on the determined first target object may be transmitted to the vehicle driving control system to control the vehicle to execute one or more of deceleration, steering, and braking. In addition to transmitting the presentation information, the set operation control information may be transmitted. In the present invention, this is not limited.

In this embodiment, an image in the vehicle and/or an image of a road on which the vehicle is located is captured by an in-vehicle camera installed in the vehicle, so that pedestrians on the road where the vehicle is located, drivers and passengers of the vehicle, and/or drivers and passengers of other vehicles Get the face attribute recognition result. The attribute recognition result according to the present embodiment can be used to enhance security such as intelligent driving and driving support.

In one possible implementation form, the image includes an observation image captured by an observation camera installed in an observation place including any one of a shopping center, a school, a large venue, a road surface, and an office building, and the attribute recognition result is a predetermined age; The attribute recognition result that satisfies at least one of a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, and a predetermined motion is included.

In one possible realization mode, the observation camera may be installed at a set position of an observation place such as a shopping center, a school, a large venue, a road surface, or an office building. For example, an observation camera may be installed at an entrance, an important road, or an important location. The observation camera may capture observation images inside or outside a shopping center, in a school yard or classroom, or inside a large venue. Since many people enter and exit the observation site, a plurality of faces may be included in the observation image. According to the method according to an embodiment of the present invention, a result of recognizing a face attribute in an observation image may be determined. Here, for determining the attribute recognition result that satisfies the setting condition as the second target object, reference may be made to the related description in the case of determining as the first target object, and redundant description will be omitted here.

For example, an observation camera for capturing an observation image may be provided in the shopping center to obtain the facial attribute recognition result of the staff or customer in the shopping center. An observation camera for capturing an observation image may be installed in a school classroom or a playground to obtain a result of recognition of facial attributes of teachers, students, or people outside the school. The facial attribute recognition result in the observation image can be used for security operation or directional transmission of setting information.

In one possible implementation, the method

determining a second target object based on a result of facial attribute recognition in the observation image;

The method further includes transmitting setting information at the observation location according to the determined second target object, or transmitting presentation information to an observer according to the determined second target object.

In one possible implementation mode, a correspondence relationship between the face attribute recognition result and a preset second target object may be preset. A face with unstable emotions may be determined as the second target object, or a face with which the attribute recognition result such as a woman who satisfies a setting condition is satisfied may be determined as the second target object. For example, during school hours, based on the facial attribute recognition result in the school observation image, the student may be the second target object, and a broadcast message to the effect of "Students, please leave the playground" may be sent out on the radio. . Among the facial attribute recognition results in the observation image of the shopping center, setting information such as advertisements for cosmetics may be reproduced on a large screen of the shopping center with the female customer as the second target object, and the male customer as the second target object Accordingly, setting information such as advertisements for automobiles may be reproduced on a large screen of a shopping center. According to the second target object, presentation information indicating that the second target object is a dangerous person or an observation target or the like may be transmitted to the observer at the observation location. In the present invention, the realization form and information content of setting information or presentation information are not limited.

In the present embodiment, an observation image is captured by an observation camera installed at the observation site to obtain a facial attribute recognition result in the observation image. The attribute recognition result according to the present embodiment can be used to enhance the security of the observation site, or to transmit setting information to the observation site.

2 is a flowchart of a face attribute recognition method according to an embodiment of the present invention. As shown in FIG. 2 , the step (S10) of the face attribute recognition method includes a step (S11) of performing convolutional processing on the image to extract convolutional features of the face.

In one possible implementation, the convolutional core is usually a two-dimensional matrix of m*n. Convolution processing is performed on image blocks at different positions in the image using preset convolutional cores and step widths to extract specific features from the image. For example, features such as the outline of the target object in the image and the shade of color may be extracted as convolutional features of the face.

Step S20 includes steps S21 and S22. In step S21, features of each type of face attribute are extracted from the convolutional features of the face to obtain attribute features of each type of face attribute.

In one possible implementation form, the extracted convolutional features of the face include features of a plurality of types of facial attributes. Based on the preset facial attributes, the attribute features of each type of facial attributes may be obtained from the extracted convolutional features. For example, the convolutional features of the face extracted from the image (1) include (feature information (1), feature information (2), ... feature information (100)), and based on three types of preset face attributes Thus, by distinguishing the convolutional features of the extracted face, the attribute features for the face attribute (1) (feature information (1), feature information (2), … feature information (50)), and for the face attribute (2) Attribute characteristics (feature information (20), characteristic information (21), ... characteristic information (80)), attribute characteristics (characteristic information (1), characteristic information (2), ... characteristic information (100) for face attribute (3) )) can be obtained.

In step S22, classification processing is performed on the attribute characteristics of each of the face attributes, and the probability prediction result of each face attribute is confirmed.

In one possible implementation mode, classification processing may be performed based on the attribute characteristics of each type of facial attribute to obtain a probability prediction result of each type of facial attribute. For example, for the image 1, the probability prediction result of the face attribute 1 is 10%, the probability prediction result of the face attribute 2 is 80%, and the probability prediction result of the face attribute 3 is 70%. . When the probability threshold is 50%, the face attribute recognition results in the image (1) are the face attribute (2) and the face attribute (3).

In this embodiment, after extracting the convolutional features of the image, and obtaining the attribute features of each type of facial attribute based on the convolutional characteristic, classification processing is performed on the attribute characteristics of each type of facial attribute to perform a classification process for each type of facial attribute You may make it obtain the probability prediction result of . By the convolutional feature and classification processing, the precision of the probabilistic prediction result of the face attribute can be further improved.

In one possible realization, step S11 of the facial attribute recognition method includes performing convolutional processing on the image by a convolutional layer of a neural network to extract convolutional features of the image. Step S21 includes extracting features of each type of facial attribute from the convolutional features of the face by the entire coupling layer of the neural network, and obtaining attribute features of each type of facial attribute. Step S22 includes performing classification processing on the attribute feature of each type of face attribute by the classification layer of the neural network, and confirming the probability prediction result of each type of face attribute.

In one possible implementation, the neural network may comprise an input layer, an intermediate layer and an output layer. The input layer, the intermediate layer, and the output layer all include a plurality of neurons, and a directed network having a variable weight between each neuron may be used. A neural network achieves the purpose of producing a model that simulates the relationship between input and output by repeatedly learning and training based on known information, and gradually adjusting and changing the neuron connection weights. The trained neural network may detect input information and provide output information corresponding to the input information by using the simulated input-output relationship model.

In one possible implementation, the neural network may include a plurality of convolutional layers, a full coupling layer and a classification layer, and the like. Convolutional processing is performed on the image by the convolutional layer of the neural network, and the convolutional features of the image are extracted. , the attribute characteristics of each type of facial attribute may be classified by the classification layer to obtain a probability prediction result of each type of facial attribute.

In this embodiment, an image is input to a neural network, and a convolutional feature of the image is extracted using the convolutional layer, the total coupling layer, and the classification layer of the neural network, and based on the convolutional feature, the properties of each type of face attribute It is also possible to obtain a feature and further perform classification processing on the attribute feature to obtain a probability prediction result of each type of face attribute of the image. With the powerful processing power of the neural network, the recognition efficiency of face attribute recognition can be further improved, and the precision of the recognition result can be further improved.

In one possible implementation, the neural network has been previously supervised by training based on a sample image set, and the sample image in the sample image set includes label information of a plurality of types of facial attributes.

In one possible realization, supervised training of the neural network may be performed using a sample image set comprising sample images including a plurality of types of facial attributes. For example, the sample image set may include N sample images, and each sample image may include label information of a plurality of types of facial attributes. For example, the face of the sample image 1 includes label information of three types of face attributes, each of which is a first type of face attribute, a fourth type of face attribute, and an eighth type of face attribute, and the sample image ( The face of 2) includes label information of two types of facial attributes, each of which is a second type of facial attribute and a fourth type of facial attribute, ... A neural network may be trained using a sample image set comprising sample images including a plurality of types of facial attributes.

In this embodiment, supervised training of the neural network may be performed using a sample image set comprising sample images including a plurality of types of facial attributes. The training efficiency of the neural network can be improved, and the trained neural network can recognize a plurality of types of facial attributes at the same time.

In one possible implementation form, the training process of the neural network is to obtain prediction results of each type of facial attribute of the sample image by the neural network, and a first method based on the prediction result of each type of facial attribute and the label information. respectively determining a loss, determining a second loss based on a first loss of each kind of facial attribute, and adjusting a network parameter of the neural network based on the second loss.

In one possible implementation mode, in the training process of the neural network, a sample image may be input to the neural network to obtain prediction results of face attributes of each type of the sample image. For example, the sample image 1 includes label information of face attribute 1, label information of face attribute 2, label information of face attribute 3, label information of face attribute 4, and face attribute 5 ), label information of 5 types of face attributes is included. Based on the actual attributes of the face in the sample image, label information of each face attribute may be obtained. For example, when the face attribute 3 does not exist for the face of the sample image, the label information of the face attribute 3 may be set to 0 or null. In the neural network, preset face attributes include five face attributes (1) to (5). For example, with respect to the prediction result of each type of facial attribute of the sample image 1, the prediction result of the face attribute (1) is 80%, the prediction result of the face attribute (2) is 40%, and the prediction result of the facial attribute (3) is 40%. The prediction result is 60%, the prediction result of the face attribute (4) is 10%, and the prediction result of the face attribute (5) is 20%.

In one possible implementation mode, the first loss of each type of facial attribute may be calculated based on the prediction result of each type of facial attribute and label information of each type of facial attribute. The first loss of the target facial attribute may be calculated based on a preset loss function, the prediction result of the target facial attribute, and label information of the target facial attribute. The target face attribute is a certain face attribute.

In one possible implementation, the first loss of each type of facial attribute in the sample image may be added to obtain a second loss.

In one possible implementation form, determining a weight loss of the target facial attribute based on a first loss of the target facial attribute including any one type of the facial attribute and the weight of the target facial attribute; and determining a second loss of the neural network based on a weight loss of the facial attribute.

In one possible implementation form, different weights may be set for each type of face attribute. For example, five types of facial attributes are preset, the weight of the face attribute (1) is set to K1, the weight of the face attribute (2) is set to be K2, ... The weight loss of the target facial attribute is obtained by multiplying the first loss of each type of facial attribute by the corresponding weight. In addition, the weight loss of each facial attribute may be added to obtain a second loss.

In one possible implementation, the second loss gradient may be propagated in the reverse direction to the neural network to adjust the network parameters of the neural network. Each sample image of the sample image set is sequentially input to the neural network until a preset procedure condition is satisfied or a preset number of training is reached, and the neural network is repeatedly trained to complete the training of the neural network. .

In this embodiment, a sample image is input to a neural network to obtain prediction results of each type of face attribute, and a first loss based on a prediction result of each type of face attribute and its label information is determined, respectively, and each type of face A second loss may be determined based on a first loss of an attribute, and a network parameter of the neural network may be adjusted based on the second loss. In the present embodiment, by adjusting the parameters of the neural network based on the loss determined based on the loss of each type of facial attribute, the relevance of the specific facial attribute is sufficiently reflected in the parameter adjustment of the neural network. It is possible to improve the precision of recognition of facial attributes of types.

3 is a schematic diagram of a training flow of a neural network of a facial attribute recognition method according to an embodiment of the present invention. As shown in FIG. 3 , the training flow of the neural network includes the following.

1. Extract facial features by the convolutional layer of the neural network.

2. By the entire bonding layer of the neural network, the attribute features of each kind of facial attribute are obtained according to each preset facial attribute. As shown in Fig. 3, the attribute characteristic of the facial attribute (1), the attribute characteristic of the facial attribute (2), ... Get the attribute feature of the face attribute (N).

3. Classification processing is performed on the attribute characteristics of each type of facial attribute by the classification layer of the neural network, and a probability prediction result of each type of facial attribute is obtained. A first loss of each type of face attribute is obtained based on the result of predicting the probability of each type of face attribute and the label information of the sample image.

4. A second loss is obtained based on the first loss of each type of facial attribute and a weight (K) corresponding to the various facial attributes.

5. Adjust the parameters of the neural network using the second loss to complete the training of the neural network.

In one possible implementation, the training method of the neural network comprises: performing one or more of inversion, cropping, translation, and addition of noise on a first image to obtain a noisy image of the first image; determining a noise image as a sample image to obtain the sample image set.

In one possible implementation form, noises such as horizontal inversion, random cropping, translation, and random disturbance of color space are added to the original image used for training the neural network to obtain a noise image of the original image, By taking both the image and the original image as sample images, a sample image set is obtained.

In this embodiment, by performing one or more of inversion, cropping, translation, and noise addition on the image, to obtain a noisy image of the image, the number of sample images can be increased and the generalization ability of the neural network can be increased. .

In one possible implementation, step S10 includes obtaining a first processed image by adjusting the resolution of the image, and extracting facial features from the first processed image.

In one possible realization, by adjusting the resolution of the image, the image whose resolution has been adjusted is input to the neural network to perform facial attribute recognition. For example, the resolution of the image may be adjusted to 224x224, 128x128, 96x96, or 64x64 according to the demand. The lower the resolution of the image, the faster the facial attribute recognition by the neural network. It is possible to satisfy the precision requirement of facial attribute recognition and increase the speed of facial attribute recognition by lowering the resolution of the image.

In the present embodiment, the speed of facial attribute recognition by the neural network can be increased by adjusting the resolution of the image.

In one possible realization, the method further comprises obtaining a coordinated neural network by adjusting the number of channels of the features of the neural network, performing convolution processing on the image by a convolutional layer of the neural network, Extracting the convolutional features includes performing convolutional processing on the image by a convolutional layer of the coordinated neural network to extract the convolutional features of the image.

In one possible implementation mode, the number of channels of the feature of the neural network may be adjusted, for example, the number of channels of the convolutional core of the convolutional layer of the neural network may be adjusted. By reducing the number of channels, the speed of facial attribute recognition by the neural network can be increased.

4 is a block diagram of an apparatus for recognizing facial attributes according to an embodiment of the present invention. As shown in FIG. 4 , the facial attribute recognition apparatus performs a feature extraction module 100 for extracting facial features from an image, and performs recognition of a plurality of preset types of facial attributes based on the facial features, each A probability prediction result acquisition module 200 for obtaining a probability prediction result of a type of face attribute, and an attribute recognition result acquisition for determining a result of recognizing a face attribute in the image based on a probability prediction result of each type of face attribute module 300 .

In one possible realization, the face attribute recognition result includes at least two attribute recognition results of the face.

In one possible realization, the facial attribute includes one or more of age, gender, ornament, skin color, expression, and motion.

In one possible realization, the ornament comprises at least one of ordinary glasses, sunglasses, a hat, and a mask, the expression comprises at least one of joy, sadness, crying, anger, and excitement, and the action comprises smoking a cigarette. , drinking water/beverage, eating, putting on makeup, or making a phone call.

In one possible realization, the image includes an image of a road on which the vehicle is located and/or an image within the vehicle, which is captured by an in-vehicle camera installed in the vehicle, and the face in the image is a pedestrian on the road on which the vehicle is located, a driver of the vehicle. Includes the face of a passenger and/or a driver/passenger of another vehicle, and the attribute recognition result includes an attribute recognition result that satisfies one or more of a predetermined age, a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, and a predetermined motion do.

In one possible implementation form, the apparatus includes a first target object determination module for determining a first target object based on a result of recognizing face attributes in the image, and a vehicle maintaining speed and direction according to the determined first target object It further includes a driving control module for transmitting presentation information and/or sending a control command to the vehicle control system to execute any one of holding, deceleration, steering, and braking.

In one possible implementation form, the image includes an observation image captured by an observation camera installed in an observation place including any one of a shopping center, a school, a large venue, a road surface, and an office building, and the attribute recognition result is a predetermined age; The attribute recognition result that satisfies at least one of a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, and a predetermined motion is included.

In one possible implementation form, the apparatus includes a second target object determining module configured to determine a second target object based on a result of facial attribute recognition in the observation image, and setting at the observation location according to the determined second target object. It further includes an observation control module for sending information, or for sending presentation information to an observer according to the determined second target object.

In one possible implementation, the feature extraction module performs convolutional processing on the image to extract convolutional features of the face.

In one possible implementation form, the probability prediction result acquisition module includes: a facial attribute feature extraction submodule for extracting features of each type of facial attribute from the convolutional features of the face to obtain attribute features of each type of facial attribute; and a probabilistic prediction result obtaining submodule for performing classification processing on the attribute characteristics of the type of face attribute, and confirming the probability prediction result of each type of face attribute.

In one possible realization, the feature extraction module performs convolution processing on the image by the convolutional layer of the neural network to extract convolutional features of the image, and the facial attribute feature extraction submodule is configured to: By extracting the features of each type of facial attribute from the convolutional features of the face by the concatenation layer, the attribute features of each type of facial attribute are obtained, and the probability prediction result acquisition submodule is configured to extract each type of facial attribute by the classification layer of the neural network. Classification processing is performed on the attribute characteristics of the type of face attribute, and the probability prediction result of each type of face attribute is confirmed.

In one possible implementation, the neural network has been previously supervised by training based on a sample image set, and the sample image in the sample image set includes label information of a plurality of types of facial attributes.

In one possible realization, the apparatus further comprises a training module for performing training of the neural network, wherein the training module obtains prediction results of each type of facial attribute of the sample image by the neural network, respectively; A first loss is respectively determined based on a prediction result of each type of facial attribute and its label information, a second loss is determined based on a first loss of each type of facial attribute, and a neural network is based on the second loss. Adjust the network parameters of the network.

In one possible realization, determining the second loss of the neural network based on the first loss of each type of facial attribute includes a first loss of a target facial attribute including any one type of facial attribute and a first loss of the target facial attribute. determining a weight loss of the target face attribute based on a weight, and determining a second loss of the neural network based on the weight loss of the target face attribute.

In one possible implementation, the training module is further configured to perform one or more of inversion, cropping, translation, and addition of noise on the first image to obtain a noise image of the first image, the first image and the noise An image is determined as a sample image to obtain the sample image set.

In one possible realization, the feature extraction module includes a first processed image acquisition module for obtaining a first processed image by adjusting the resolution of the image, and a first feature extraction for extracting facial features from the first processed image Contains submodules.

In one possible realization, the apparatus further comprises a channel adjustment module for adjusting the number of channels of the features of the neural network to obtain a coordinated neural network, wherein the feature extraction module is further configured to: Convolutional processing is performed on the image by the method to extract a convolutional feature of the image.

It should be understood that the embodiments of the respective methods mentioned in the present invention may be combined with each other to form embodiments as long as the principles and logic are not violated. Since there is a limit to the amount, a detailed description is omitted in the present invention.

In addition, for those skilled in the art, in the above method according to a specific embodiment, the description order of each step does not limit the execution process by strictly limiting the execution order, and the execution order of each step is determined specifically by its function and internal logic. have to understand

In some embodiments, functions or modules provided by the apparatus according to the embodiments of the present invention may be used to implement the methods described in the embodiments of the above-described methods, and for specific realizations thereof, refer to the description of the embodiments of the above-described methods. reference, and for the sake of brevity, a detailed description thereof is omitted here.

An embodiment of the present invention further provides a computer readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the computer readable storage medium realizes the method. The computer readable storage medium may be a nonvolatile computer readable storage medium.

In an embodiment of the present invention, there is further provided an electronic device comprising a processor and a memory for storing instructions executable by the processor, wherein the processor is configured to execute the method.

The electronic device may be provided as a terminal, server, or other type of device.

Fig. 5 shows a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a terminal such as a mobile phone, a computer, a digital broadcast terminal, a message transmitting/receiving device, a game console, a tablet type device, a medical device, a fitness device, or a personal digital assistant.

Referring to FIG. 5 , the electronic device 800 includes a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , and an input/output (I/O) interface. 812 , a sensor component 814 , and a communication component 816 .

The processing component 802 typically controls the overall operation of the electronic device 800 , such as operations related to display, phone call, data communication, camera operation, and recording operation. Processing component 802 may include one or more processors 820 executing instructions to carry out all or some steps of the method. Further, processing component 802 may include one or more modules for interaction with other components. For example, processing component 802 may include a multimedia module for interaction with multimedia component 808 .

The memory 804 is configured to store various types of data to support operation in the electronic device 800 . These data include, for example, commands of all application programs or methods operated by the electronic device 800 , contact data, phone book data, messages, photos, videos, and the like. Memory 804 may include, for example, static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory ( ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk, etc., can be realized by various types of volatile or nonvolatile storage devices, or combinations thereof.

The power component 806 supplies power to each component of the electronic device 800 . Power component 806 may include a power management system, one or more power sources, and other components related to power generation, management, and distribution for electronic device 800 .

The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and a user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). When the screen includes a touch panel, it may be realized as a touch screen that receives an input signal from a user. The touch panel includes one or more touch sensors for detecting touches, slides and gestures on the touch panel. The touch sensor may not only detect a boundary of a touch or slide operation, but also detect a duration and pressure associated with the touch or slide operation. In some embodiments, multimedia component 808 includes a front camera and/or a rear camera. When the electronic device 800 is in an operation mode, for example, a photographing mode or an imaging mode, the front camera and/or the rear camera may receive external multimedia data. Each front camera and rear camera may have a fixed optical lens system or focal length and optical zoom capability.

The audio component 810 is configured to output and/or input an audio signal. For example, the audio component 810 includes one microphone MIC, and the microphone MIC is when the electronic device 800 is in an operation mode, for example, a call mode, a recording mode, or a voice recognition mode. , configured to receive an external audio signal. The received audio signal may be further stored in memory 804 or transmitted via communication component 816 . In some embodiments, the audio component 810 further includes a speaker for outputting an audio signal.

I/O interface 812 provides an interface between processing component 802 and a peripheral interface module, which may be a keyboard, click wheel, button, or the like. These buttons may include, but are not limited to, a home button, a volume button, a start button, and a lock button.

The sensor component 814 includes one or more sensors for evaluating the condition of each side of the electronic device 800 . For example, the sensor component 814 may detect an on/off state of the electronic device 800 , eg, a relative positioning of components such as a display device and a keypad of the electronic device 800 , the sensor component 814 further indicates a change in the position of the electronic device 800 or a component in which the electronic device 800 is located, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration/deceleration of the electronic device 800, and the electronic device 800 ) can be detected. The sensor component 814 may include a proximity sensor configured to detect the presence of a nearby object in the absence of any physical contact. The sensor component 814 may further include a photosensor for use in imaging applications, such as CMOS or CCD image sensors. In some embodiments, the sensor component 814 may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

The communication component 816 is configured to realize wired or wireless communication between the electronic device 800 and another device. The electronic device 800 may access a wireless network based on a communication standard, for example, WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast related information from an external broadcast management system through a broadcast channel. In one exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate near field communication. For example, the NFC module can be realized by radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, the electronic device 800 includes one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs). , implemented by a controller, microcontroller, microprocessor or other electronic element, and may be used to execute the method.

In the exemplary embodiment, there is further provided a nonvolatile computer readable storage medium, for example, a memory 804 containing computer program instructions, the computer program instructions being executed by the processor 820 of the electronic device 800 . Once executed, the method can be executed.

In an embodiment of the present invention, a computer program product comprising computer readable code, wherein the computer readable code executes instructions for realizing the method according to any of the above embodiments to a processor of the device when the computer readable code is operated in a device. Additional products are provided.

The computer program product may be specifically realized by hardware, software, or a combination thereof. In one selectable embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, for example, a Software Development Kit (SDK for short) or the like.

6 shows a block diagram of an electronic device 1900 according to an exemplary embodiment. For example, the electronic device 1900 may be provided as a server. Referring to FIG. 6 , the electronic device 1900 includes a processing component 1922 including one or more processors and a memory 1932 for storing instructions executable by the processing component 1922 , for example, an application program. Includes memory resources represented by . The application program stored in the memory 1932 may include one or more modules each corresponding to one instruction group. Further, processing component 1922 is configured to execute the method by executing instructions.

The electronic device 1900 further includes a power component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and input/output ( I/O) interface 1958 . The electronic device 1900 may operate based on an operating system stored in the memory 1932 , for example, Windows Server ^TM , Mac OS X ^TM , Unix ^TM , Linux ^TM , FreeBSD ^{TM ,} or the like.

In the exemplary embodiment, there is further provided a nonvolatile computer readable storage medium, eg, a memory 1932 containing computer program instructions, the computer program instructions being stored in the processing component 1922 of the electronic device 1900 . If executed by the above method, it is possible to execute the method.

The present invention may be a system, method and/or computer program product. The computer program product may include a computer readable storage medium having computer readable program instructions for realizing each aspect of the present invention in the processor.

The computer-readable storage medium may be a tangible device capable of storing and storing instructions used in the instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electronic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), Static random access memory (SRAM), portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, e.g., punched cards or slots in which instructions are stored; such mechanical encoding devices, and any suitable combination of the above. A computer-readable storage medium as used herein is a transitory signal itself, for example, radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating via waveguides or other transmission media (eg, optical pulses passing through optical fiber cables). ), or an electrical signal transmitted via a wire.

The computer readable program instructions described herein may be downloaded from a computer readable storage medium to each computing/processing device, or externally via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. It may be downloaded to your computer or external storage device. The network may include copper transport cables, fiber optic transport, wireless transport, routers, firewalls, switchboards, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives a computer readable program command from the network, transmits the computer readable program command, and stores the computer readable program command in a computer readable storage medium in each computing/processing device.

Computer program instructions for carrying out the operations of the present invention may include assembly instructions, instruction set architecture (ISA) instructions, machine language instructions, machine dependent instructions, microcode, firmware instructions, state setting data or an object-oriented programming language such as Smalltalk, C++, etc.; and source code or target code written in any combination of one or more programming languages including a general procedural programming language such as "C" language or a similar programming language. The computer readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partly on a remote computer, or It may run entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any kind of network, including a local area network (LAN) or wide area network (WAN), or (eg, an Internet service provider via the Internet) may be connected to an external computer. In some embodiments, state information from computer readable program instructions is used to customize electronic circuitry, such as, for example, a programmable logic circuit, a field programmable gate array (FPGA) or a programmable logic array (PLA), the electronic circuit Each aspect of the present invention may be realized by executing computer readable program instructions by

Although each aspect of the present invention has been described herein with reference to a flowchart and/or block diagram of a method, apparatus (system) and computer program product according to an embodiment of the present invention, each block in the flowchart and/or block diagram, and the flowchart and It should be understood that all combinations of blocks in the block diagram may be realized by computer readable program instructions.

These computer readable program instructions are provided to a processor of a general purpose computer, dedicated computer, or other programmable data processing device, and when these instructions are executed by the processor of the computer or other programmable data processing device, one or more of the flowcharts and/or block diagrams The device may be manufactured to realize the functions/actions specified in the block. These computer readable program instructions may be stored in a computer readable storage medium, and may cause a computer, a programmable data processing apparatus, and/or other apparatus to operate in a specific manner. Thereby, a computer-readable storage medium having instructions stored thereon includes a product having instructions for realizing each aspect of a function/action specified in one or more blocks of the flowchart and/or block diagram.

The computer readable program instructions may be loaded into a computer, other programmable data processing device, or other device, and cause the computer, other programmable data processing device, or other device to execute a series of operational steps, thereby generating a process realized by the computer. . In this way, functions/operations specified in one or more blocks of the flowchart and/or block diagram are realized by instructions executed in a computer, other programmable data processing device, or other device.

Flowcharts and block diagrams in the drawings represent realizable system architectures, functions, and operations of systems, methods, and computer program products according to a plurality of embodiments of the present invention. In this regard, each block in the flowchart or block diagram may represent one module, program segment, or portion of an instruction, said module, program segment, or portion of instructions being one or more executable instructions for realizing a specified logical function. includes In some alternative implementations, the functions indicated in the blocks may be implemented in an order different from the order attached to the drawing. For example, two consecutive blocks may be executed substantially in parallel, or may be executed in the reverse order depending on the function involved. In addition, each block in the block diagram and/or the flowchart and the combination of the blocks in the block diagram and/or the flowchart may be realized by a dedicated system based on hardware for executing designated functions or operations, or dedicated hardware and a computer It should also be noted that it may be realized by a combination of commands.

As mentioned above, although each embodiment of this invention was described, the said description is only exemplary, and is not exhaustive, nor is it limited to each disclosed embodiment. Various modifications and changes will be apparent to those skilled in the art without departing from the scope and spirit of each described embodiment. The terminology selected herein is intended to advantageously interpret the principle, practical application, or improvement of existing techniques of each embodiment, or to enable others skilled in the art to understand each embodiment disclosed herein.

Claims (37)

extracting facial features from the image,
Recognizing a plurality of preset types of facial attributes based on the facial features to obtain a probability prediction result of each type of facial attributes;
and determining a face attribute recognition result in the image based on a probability prediction result of each type of face attribute. The method of claim 1,
The method of claim 1, wherein the facial attribute recognition result includes at least two attribute recognition results of the face. 3. The method according to claim 1 or 2,
The facial attribute comprises one or more of age, gender, ornament, skin color, expression, and motion. 4. The method of claim 3,
The ornament includes one or more of ordinary glasses, sunglasses, a hat, and a mask, the expression includes one or more of joy, sadness, crying, anger, and excitement, and the action includes smoking a cigarette, drinking water A method comprising one or more of drinking, eating, putting on makeup, and making a phone call. 5. The method according to claim 3 or 4,
The image includes an image of a road on which the vehicle is located and/or an image within the vehicle, which is captured by an in-vehicle camera installed in the vehicle,
The face in the image includes the face of a pedestrian on the road on which the vehicle is located, the driver/passenger of the vehicle and/or the face of the driver/passenger of another vehicle,
The attribute recognition result includes an attribute recognition result that satisfies at least one of a predetermined age, a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, and a predetermined motion. 6. The method of claim 5,
determining a first target object based on a result of recognizing face attributes in the image;
Transmitting presentation information so that the vehicle executes any one of maintaining speed, maintaining direction, decelerating, steering, and braking according to the determined first target object, and/or transmitting a control command to a vehicle control system , Way. 5. The method according to claim 3 or 4,
The image includes an observation image captured by an observation camera installed in an observation place including any one of a shopping center, a school, a large venue, a road surface, and an office building,
The attribute recognition result includes an attribute recognition result that satisfies at least one of a predetermined age, a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, and a predetermined motion. 8. The method of claim 7,
determining a second target object based on a result of facial attribute recognition in the observation image;
The method further comprising: transmitting setting information at the viewing location according to the determined second target object, or transmitting presentation information to an observer according to the determined second target object. 9. The method according to any one of claims 1 to 8,
Extracting facial features from images
performing convolution processing on the image to extract convolutional features of the face. 10. The method of claim 9,
Recognizing a plurality of preset types of facial attributes based on the facial features to obtain a probability prediction result of each type of facial attributes
extracting the characteristics of each type of facial attribute from the convolutional features of the face to obtain attribute characteristics of each type of facial attribute;
and performing classification processing on the attribute features of the face attributes of each type, and further comprising confirming a probability prediction result of each type of face attribute. 11. The method of claim 10,
Extracting the convolutional features of the face by performing convolution processing on the image
performing convolution processing on the image by a convolutional layer of a neural network to extract convolutional features of the image;
Extracting the characteristics of each type of facial attribute from the convolutional features of the face to obtain the attribute characteristics of each type of facial attribute
extracting features of each type of facial attribute from the convolutional features of the face by the entire coupling layer of the neural network, and obtaining attribute features of each type of facial attribute;
Performing classification processing on the attribute characteristics of each type of face attribute and confirming the probability prediction result of each type of face attribute
performing classification processing on the attribute feature of each type of facial attribute by a classification layer of the neural network, and confirming a probability prediction result of each type of facial attribute. 12. The method of claim 11,
The method of claim 1, wherein the neural network has been previously supervised by training based on a sample image set, and the sample image in the sample image set includes label information of a plurality of types of facial attributes. 13. The method of claim 12,
The training process of the neural network is
Obtaining prediction results of each type of face attribute of the sample image by a neural network, respectively;
Determining a first loss based on a prediction result of each type of face attribute and the label information, respectively;
determining a second loss based on the first loss of each type of facial attribute;
and adjusting a network parameter of the neural network based on the second loss. 14. The method of claim 13,
Determining the second loss of the neural network based on the first loss of each type of facial attribute comprises:
determining a weight loss of the target facial attribute based on a first loss of the facial attribute including any one type of target facial attribute and a weight of the target facial attribute;
and determining a second loss of the neural network based on the weight loss of each type of face attribute. 15. The method according to any one of claims 12 to 14,
The training method of the neural network is
performing one or more of inversion, cropping, translation, and addition of noise on the first image to obtain a noise image of the first image;
determining the first image and the noise image as sample images to obtain the set of sample images. 16. The method according to any one of claims 1 to 15,
Extracting facial features from images
adjusting the resolution of the image to obtain a first processed image;
and extracting facial features in the first processed image. 17. The method according to any one of claims 11 to 16,
Further comprising adjusting the number of channels of the feature of the neural network to obtain a coordinated neural network,
Extracting convolutional features of the image by performing convolution processing on the image by the convolutional layer of the neural network
performing convolution processing on the image by a convolutional layer of the coordinating neural network to extract convolutional features of the image. A feature extraction module for extracting facial features from an image;
a probability prediction result acquisition module for recognizing a plurality of types of facial attributes preset based on the facial features to obtain a probability prediction result of each type of facial attributes;
and an attribute recognition result acquisition module for determining a facial attribute recognition result in the image, based on a probability prediction result of each type of face attribute. 19. The method of claim 18,
The apparatus of claim 1, wherein the face attribute recognition result includes at least two attribute recognition results of the face. 20. The method according to claim 18 or 19,
wherein the facial attribute includes one or more of age, gender, ornament, skin color, expression, motion, and race. 21. The method of claim 20,
The ornament includes one or more of ordinary glasses, sunglasses, a hat, and a mask, the expression includes one or more of joy, sadness, crying, anger, and excitement, and the action includes smoking a cigarette, drinking water A device comprising one or more of drinking, eating, putting on makeup, and making a phone call. 22. The method of claim 20 or 21,
The image includes an image of a road on which the vehicle is located and/or an image within the vehicle, which is captured by an in-vehicle camera installed in the vehicle,
The face in the image includes the face of a pedestrian on the road on which the vehicle is located, the driver/passenger of the vehicle and/or the face of the driver/passenger of another vehicle,
and the attribute recognition result includes an attribute recognition result that satisfies at least one of a predetermined age, a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, a predetermined motion, and a predetermined race. 23. The method of claim 22,
a first target object determination module configured to determine a first target object based on a result of recognizing face attributes in the image;
adding a driving control module for sending presentation information and/or sending a control command to a vehicle control system so that the vehicle executes any one of speed maintenance, direction maintenance, deceleration, steering, and braking according to the determined first target object A device comprising: 22. The method of claim 20 or 21,
The image includes an observation image captured by an observation camera installed in an observation place including any one of a shopping center, a school, a large venue, a road surface, and an office building,
and the attribute recognition result includes an attribute recognition result that satisfies at least one of a predetermined age, a predetermined gender, a predetermined ornament, a predetermined skin color, a predetermined facial expression, a predetermined motion, and a predetermined race. 25. The method of claim 24,
a first target object determination module configured to determine a second target object based on a result of facial attribute recognition in the observation image;
The apparatus further comprising: an observation control module for transmitting setting information at the observation place according to the determined second target object, or for transmitting presentation information to an observer according to the determined second target object. 26. The method according to any one of claims 18 to 25,
and the feature extraction module performs convolution processing on the image to extract convolutional features of the face. 27. The method of claim 26,
The probability prediction result acquisition module is
a facial attribute feature extraction submodule for extracting features of each type of facial attribute from the convolutional features of the face to obtain attribute features of each type of facial attribute;
and a probabilistic prediction result acquisition submodule for performing classification processing on the attribute characteristics of the respective types of facial attributes to confirm the probability prediction results of each type of facial attributes. 28. The method of claim 27,
The feature extraction module performs convolution processing on the image by a convolutional layer of a neural network to extract convolutional features of the image,
the facial attribute feature extraction submodule extracts features of each type of facial attribute from the convolutional features of the face by the entire coupling layer of the neural network to obtain attribute features of each type of facial attribute;
and the probabilistic prediction result acquisition submodule performs classification processing on the attribute feature of each type of face attribute by the classification layer of the neural network, and confirms the probability prediction result of each type of face attribute. 29. The method of claim 28,
The apparatus of claim 1, wherein the neural network has undergone supervised training in advance based on a sample image set, and the sample image in the sample image set includes label information of a plurality of types of facial attributes. 30. The method of claim 29,
Further comprising a training module for performing training of the neural network,
The training module
Acquire prediction results of each type of face attribute of the sample image by a neural network,
determining a first loss based on a prediction result of each type of face attribute and its label information, respectively;
determine a second loss based on the first loss of each kind of facial attribute;
and adjusting a network parameter of the neural network based on the second loss. 31. The method of claim 30,
Determining the second loss of the neural network based on the first loss of each kind of facial attribute comprises:
determining a weight loss of the target facial attribute based on a first loss of the facial attribute including any one type of target facial attribute and a weight of the target facial attribute;
and determining a second loss of the neural network based on the weight loss of each type of face attribute. 32. The method according to any one of claims 29 to 31,
The training module further comprises:
performing one or more of inversion, cropping, translation, and addition of noise on the first image to obtain a noise image of the first image;
determining the first image and the noise image as sample images to obtain the set of sample images. 33. The method according to any one of claims 18 to 32,
The feature extraction module is
a first processed image acquisition module for obtaining a first processed image by adjusting the resolution of the image;
and a first feature extraction sub-module for extracting facial features from the first processed image. 34. The method according to any one of claims 28 to 33,
Further comprising a channel adjustment module for obtaining a coordinated neural network by adjusting the number of channels of the feature of the neural network,
The feature extraction module further comprises:
performing convolutional processing on the image by a convolutional layer of the coordinated neural network to extract convolutional features of the image. processor and
a memory for storing instructions executable by the processor;
The electronic device, wherein the processor is configured to execute the method of any one of claims 1 to 17. A computer readable storage medium having computer program instructions stored thereon, wherein when the computer program instructions are executed by a processor, the method according to any one of claims 1 to 17 is realized. A computer program comprising computer readable code, wherein the computer readable code, when executed in an electronic device, causes a processor of the electronic device to execute instructions for realizing the method of any one of claims 1 to 17.

Images