WO2021217912A1

WO2021217912A1 - Facial recognition-based information generation method and apparatus, computer device and storage medium

Info

Publication number: WO2021217912A1
Application number: PCT/CN2020/103796
Authority: WO
Inventors: 郑秦苏
Original assignee: 深圳壹账通智能科技有限公司
Priority date: 2020-04-28
Filing date: 2020-07-23
Publication date: 2021-11-04
Also published as: CN111639534A

Abstract

A facial recognition-based information generation method and apparatus, a computer device and a storage medium, which relate to the technical field of image recognition in artificial intelligence. Said method comprises: if a current user image acquired by a camera is received, performing facial recognition on the current user image, so as to obtain corresponding current user identity recognition information; if welcome information corresponding to the current user identity recognition information is stored, sending the welcome information to a corresponding display device for display; if a user facial image acquired by the camera is received, performing emotion recognition on the user facial image, so as to obtain corresponding current user emotion information; and if the current user emotion information indicates a happy emotion, automatically adding, to the welcome information, a label indicating that the information can be reserved. The present invention generates and deploys welcome information efficiently, and can acquire feedback information of the users on the welcome information on the basis of emotion recognition results of users.

Description

Information generation method, device, computer equipment and storage medium based on face recognition

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on April 28, 2020, the application number is 202010351274.0, and the invention title is "Information Generation Method, Apparatus, and Computer Equipment Based on Face Recognition". The entire content of the application is approved The reference is incorporated in this application.

Technical field

This application relates to the field of image recognition technology, and in particular to an information generation method, device, computer equipment, and storage medium based on face recognition.

Background technique

At present, the more common welcoming device is the door-front welcoming service equipment. The door-front welcoming service equipment mainly includes inkjet banners, voice broadcast welcoming equipment, and screen broadcast welcoming equipment.

Among them, inkjet banners take a long time to make and cannot be recycled and reused. The voice broadcast welcome device and the welcome word in the screen broadcast welcome device need to be edited manually by maintenance personnel. The inventor realized that the above-mentioned common door welcome service equipment requires labor The production or editing of the welcome speech results in low efficiency and single presentation, and it is impossible to detect the degree of satisfaction of the welcome user with the welcome speech.

Summary of the invention

The embodiments of the present application provide a method, device, computer equipment and storage medium for information generation based on face recognition, aiming to solve the problem of manual editing or production of door welcoming service equipment in the prior art, resulting in low efficiency and display mode Single problem.

In the first aspect, an embodiment of the present application provides a method for generating information based on face recognition, which includes:

Determine whether the current user image collected by the camera is received;

If the current user image collected by the camera is received, face recognition is performed on the current user image to obtain corresponding current user identification information;

Judging whether there is welcoming information corresponding to the current user identification information; wherein the welcoming information includes welcoming text information, welcoming audio information, or welcoming video information;

If the welcome information corresponding to the current user identification information is stored, sending the welcome information to the corresponding display device for display;

Determine whether the user's facial image collected by the camera is received;

If the user's facial image collected by the camera is received, emotion recognition is performed on the user's facial image to obtain corresponding current user emotional information;

Determine whether the current user emotional information is happy mood or angry mood; and

If the former user's emotion information is happy emotion, a retainable tag is automatically added to the welcome information.

In the second aspect, an embodiment of the present application provides an information generation device based on face recognition, which includes:

The first image judging unit is used to judge whether the current user image collected by the camera is received;

The user identification unit is configured to, if the current user image collected by the camera is received, perform face recognition on the current user image to obtain corresponding current user identification information;

The welcoming information judging unit is used to determine whether the welcoming information corresponding to the current user identification information is stored; wherein the welcoming information includes welcoming text information, welcoming audio information, or welcoming video information ；

The welcome information sending unit is configured to send the welcome information to the corresponding display device for display if the welcome information corresponding to the current user identification information is stored;

The second image judging unit is used to judge whether the user's facial image collected by the camera is received;

The emotion recognition unit is configured to perform emotion recognition on the user's facial image if the user's facial image collected by the camera is received, to obtain corresponding current user's emotion information;

An emotion judgment unit for judging whether the current user emotion information is happy mood or angry mood; and

The first label setting unit is configured to automatically add a retainable label to the welcome message if the previous user's emotional information is happy mood.

In a third aspect, an embodiment of the present application provides a computer device, which includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor executes the computer The program implements the information generation method based on face recognition described in the first aspect above.

In a fourth aspect, an embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the above-mentioned first On the one hand, the information generation method based on face recognition.

The embodiments of the present application provide a method, device, computer equipment and storage medium for information generation based on face recognition, including determining whether the current user image collected by the camera is received; if the current user image collected by the camera is received, correct Perform face recognition on the current user image to obtain corresponding current user identification information; determine whether there is welcome information corresponding to the current user identification information; wherein the welcome information includes welcome text information , Welcome audio information, or welcome video information; if the welcome information corresponding to the current user identification information is stored, send the welcome information to the corresponding display device for display; determine whether the camera is received The collected facial image of the user; if the facial image of the user collected by the camera is received, emotion recognition is performed on the facial image of the user to obtain the corresponding current user emotion information; it is judged whether the current user emotion information is happy emotion or angry emotion And if the former user’s emotional information is a happy mood, a retainable tag is automatically added to the welcoming information. This method realizes that the generation and deployment of welcome information is more efficient, and can obtain the user's feedback information on the welcome information based on the user's emotion recognition result.

Description of the drawings

In order to explain the technical solutions of the embodiments of the present application more clearly, the following will briefly introduce the drawings used in the description of the embodiments. Obviously, the drawings in the following description are some embodiments of the present application. Ordinary technicians can obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic diagram of an application scenario of a method for generating information based on face recognition provided by an embodiment of the application;

2 is a schematic flowchart of a method for generating information based on face recognition provided by an embodiment of the application;

FIG. 3 is a schematic diagram of a sub-process of a method for generating information based on face recognition provided by an embodiment of the application;

4 is a schematic diagram of another sub-flow of the method for generating information based on face recognition provided by an embodiment of the application;

FIG. 5 is a schematic block diagram of an information generation device based on face recognition provided by an embodiment of the application;

FIG. 6 is a schematic block diagram of subunits of the apparatus for generating information based on face recognition according to an embodiment of the application;

7 is a schematic block diagram of another subunit of the apparatus for generating information based on face recognition according to an embodiment of the application;

FIG. 8 is a schematic block diagram of a computer device provided by an embodiment of the application.

Detailed ways

The technical solutions in the embodiments of the present application will be described clearly and completely in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of this application.

It should be understood that when used in this specification and appended claims, the terms "including" and "including" indicate the existence of the described features, wholes, steps, operations, elements and/or components, but do not exclude one or The existence or addition of multiple other features, wholes, steps, operations, elements, components, and/or collections thereof.

It should also be understood that the terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in the specification of this application and the appended claims, unless the context clearly indicates other circumstances, the singular forms "a", "an" and "the" are intended to include plural forms.

It should be further understood that the term "and/or" used in the specification and appended claims of this application refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations .

Please refer to Figure 1 and Figure 2. Figure 1 is a schematic diagram of an application scenario of a method for generating information based on face recognition provided by an embodiment of this application; Figure 2 is a schematic flowchart of a method for generating information based on face recognition provided by an embodiment of this application The method for generating information based on face recognition is applied to a local terminal, and the method is executed by application software installed in the local terminal.

As shown in Figure 2, the method includes steps S110 to S180.

S110: Determine whether the current user image collected by the camera is received.

In this embodiment, by setting up a welcoming system (for example, exhibition venues, stadiums, athletes’ residences, high-end hotels, restaurants, tourist attractions, etc.) in places where welcoming guests are needed, the welcoming system includes at least a camera, a display device, and Server, this application describes the technical solution from the perspective of the server. Among them, the welcome information stored in the server and the massive face pictures that have been collected can be uploaded to the server by the user terminal (that is, a terminal such as a smart phone or tablet computer used by the user), and the user terminal performs the process through the user interaction interface corresponding to the server. The above upload operation.

In the welcoming system, the camera is equipped with an infrared sensor for detecting that the distance between the user and the camera is less than a preset distance threshold (such as 2-10m), and then collecting the user's current user image and sending it to the server. After the server receives the image of the current user and performs face recognition, if there is a target user, the welcome information corresponding to the target user is sent to the display device for display, thereby achieving the welcome effect.

If the camera detects that the distance between the user and the camera is less than the distance threshold, the camera collects the current user image and sends the current user image to the server.

S120: If the current user image collected by the camera is received, perform face recognition on the current user image to obtain corresponding current user identity recognition information.

In this embodiment, after the camera uploads the current user image, the server adopts the 1:N mode of face recognition to determine whether the current user image is a face image corresponding to the user to be welcomed.

Face recognition does a 1:N comparison, that is, after the server receives a photo of "me", it finds from the massive database of portraits that it matches the face data of the current user (ie "me") Image and match it to find out "who am I".

In one embodiment, as shown in FIG. 3, step S120 includes:

S121: Perform grayscale correction and noise filtering on the current user image to obtain a preprocessed picture;

S122: Obtain a feature vector corresponding to the preprocessed picture through a convolutional neural network model;

S123. Compare the feature vector corresponding to the current user image with the feature template stored in the face database to determine whether there is a feature vector corresponding to the current user image in the feature template stored in the face database The same feature template;

S124: If there is a feature template that is the same as the feature vector corresponding to the current user image among the feature templates stored in the face database, obtain corresponding current user identification information.

In this embodiment, the specific steps for face recognition in the 1:N mode are as the above steps S121-S124. The preprocessing of the current user image is based on the face detection result, the image is processed and finally serves the process of feature extraction . Due to various conditions and random interference, the original image obtained by the server cannot be used directly. It must be preprocessed in the early stages of image processing such as gray-scale correction and noise filtering. For face images, the preprocessing process mainly includes light compensation, gray scale transformation, histogram equalization, normalization, geometric correction, filtering and sharpening of the face image.

When obtaining the feature vector of the current user image, first obtain the pixel matrix corresponding to the preprocessed picture, and then use the pixel matrix corresponding to the preprocessed picture as the input of the input layer in the convolutional neural network model to obtain multiple features After that, the feature map is input to the pooling layer to obtain the one-dimensional vector corresponding to the maximum value corresponding to each feature map. Finally, the one-dimensional vector corresponding to the maximum value corresponding to each feature map is input to the fully connected layer to obtain the The feature vector corresponding to the processed image.

Among them, the feature templates stored in the face database can be actively uploaded to the server by the user, or they can be continuously added to the feature templates that do not exist in the historical face database during the process of continuous use by the user, so as to accumulate the face. Feature templates stored in the database. For example, before the welcoming user arrives at the welcoming site, the administrator of the server can directly upload the head image of the welcoming user to the server for feature vector extraction, thereby obtaining the characteristic template of the welcoming user, and then The feature template of the to-be-welcoming user can be stored in the face database.

The 1:N mode of face recognition is that after the server receives a photo of "me", it finds an image that matches the face data of the current user (ie "me") from the massive database of portraits, and Perform matching to find out "who am I", that is, the execution process of the above steps S121-S124.

Since the feature templates stored in the face database store the feature vectors corresponding to the massive amount of face images that have been collected, that is, each person’s face corresponds to a unique feature vector. With these massive feature templates as data After the foundation, it can be used to determine one or more people corresponding to the preprocessed picture, so as to realize face recognition.

Finally, the obtained user identification information can be the user's ID number. Since each citizen's ID number is unique, it can be used as its unique identification code.

In an embodiment, as shown in FIG. 3, after step S123, the method further includes:

S125: If there is no feature template that is the same as the feature vector corresponding to the current user image among the feature templates stored in the face database, a prompt is made that the target user is not recognized, and the step of ending the process is executed.

In this embodiment, when the feature vector corresponding to the current user image captured by the camera is not the same as the feature template stored in the face database, it means that the user corresponding to the current user image is not the user to be welcomed. Perform the following steps to welcome the guests, and then jump directly to the step that ends the process.

Wherein, if the current user image collected by the camera is not received, wait for the preset delay waiting time, and return to step S110;

That is, when the camera in the welcome system has not collected the current user image, the camera cannot send the current user image to the server. At this time, the server waits for the preset delay waiting time, and then returns to step S110. .

S130: Determine whether the welcome information corresponding to the current user identification information is stored; wherein the welcome information includes welcome text information, welcome audio information, or welcome video information.

In this embodiment, when it is recognized that the current user image corresponds to the user to be greeted, it is necessary to retrieve in the server whether the welcome information corresponding to the current user identification information is stored; wherein, in the welcome information It includes welcome text information, welcome audio information, or welcome video information. That is, it is judged whether the pre-edited welcome information corresponding to the user to be greeted is stored in the server. This judgment process is also to quickly retrieve the personalized welcome information corresponding to the user to be greeted.

S140: If the welcome information corresponding to the current user identification information is stored, the welcome information is sent to a corresponding display device for display.

In this embodiment, the welcome information includes welcome text information, welcome audio information, or welcome video information, that is, the specific content of the welcome information is diverse, and may be in plain text. Welcome text information (such as "Welcome XXX", the welcome text information is displayed on the display device scrolling or static display), or it can be welcome audio information (such as "Welcome XXX", the welcome audio information is displayed by The built-in speaker of the device is used for external playback. The total number of playbacks, playback period, and total playback duration can be user-defined settings), or welcome video information (such as playing through the display device through animated text and various background videos) For the composed welcome video, the total number of times of playing, the playing period and the total playing time can be user-defined settings). In addition to the information composed of one or more of the above three examples, it can also be other more complicated ways of welcoming guests. This application will not give detailed examples due to space limitations. Moreover, the welcome information in this application is pushed to the display device for display due to the specific identity of the user to be greeted, that is, the server can automatically control the welcome information displayed by the display device, and the display device is not for one-time use. It is reused many times to avoid the inefficient production process caused by the use of customized red silk banners with welcome words.

After the welcome information is sent to the display device for display, a variety of display methods for the welcoming user are realized at this time, but in order to further optimize the update or replacement of the welcome information, it is necessary to perform step S140 at this time Steps to identify the user’s emotions.

Wherein, the welcome information corresponding to the current user identification information is not stored in the server, the pre-stored general welcome information is acquired, and the general welcome information is sent to the corresponding display device for display.

That is, if the corresponding welcome information is not set for the user to be greeted, in order to quickly display the welcome information at this time, the pre-stored general welcome information can be obtained at this time, and the general welcome information can be sent to the corresponding display device. To display.

S150: Determine whether the user's facial image collected by the camera is received.

In this embodiment, when the user's facial image collected by the camera is not the same image as the current user image in step S110, and the collection time of the user's facial image is also different from the collection time of the current user's image, generally the user's facial image The acquisition time is later than the acquisition time of the current user image (for example, the acquisition time of the user's face image is 5-10s after the acquisition time of the current user image), that is, after the completion of the recognition of the current user image, the corresponding display After the welcome message, the camera collects an image of the user's face.

S160: If the user's facial image collected by the camera is received, perform emotion recognition on the user's facial image to obtain corresponding current user emotion information.

In this embodiment, the purpose of performing emotion recognition based on the user's facial image is to further determine the current emotion type of the user after seeing the welcome message. For example, if the user is happy after seeing the welcome message, it means that the user is satisfied with the content of the welcome message; for example, if the user is angry after seeing the welcome message, it means that the user is angry with the welcome message. The content is not satisfactory; for example, if the user is in a normal mood after seeing the welcome message, it means that the user has a neutral attitude towards the content of the welcome message.

In an embodiment, the current user identification information and the corresponding welcome information are stored in a blockchain network, and the information is shared between different platforms through the blockchain. When the current user identification information is obtained, the information stored in the blockchain network is used to find the welcome information corresponding to the current user identification information.

Blockchain is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information for verification. The validity of the information (anti-counterfeiting) and the generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

In an embodiment, as shown in FIG. 4, step S160 includes:

S161. Obtain the facial feature points in the user's facial image through a pre-stored face detector;

S162. Obtain a face bounding box that surrounds all the face feature points in the user's face image.

S163. Acquire a mouth feature point set included in the facial feature points in the user's facial image;

S164. Obtain the longitudinal coordinate spacing value between the uppermost mouth feature point and the lowermost mouth feature point in the mouth feature point set, and obtain the corresponding value according to the ratio of the longitudinal coordinate spacing value to the length of the face bounding box The value of the user's mouth opening degree;

S165. Determine whether the user's mouth opening degree value is less than a preset mouth opening degree threshold;

S166: If the user's mouth opening degree value is less than the mouth opening degree threshold value, the value of the current user's emotional information is set as an angry mood;

S167: If the user's mouth opening degree value is greater than or equal to the mouth opening degree threshold value, the current value of the user's emotional information is set as a happy mood.

In this embodiment, specifically, the face detector in OpenCV (a cross-platform computer vision library that implements many general algorithms in image processing and computer vision) is used to obtain 68 faces in the user's facial image. Feature points.

Generally, there are 16-20 mouth feature points among 68 facial feature points, and the above-mentioned 16-20 mouth feature points constitute a mouth feature point set. Among them, each of the 16-20 mouth feature points can be placed in the same Cartesian coordinate system as the face bounding box. At this time, the uppermost mouth feature point in the mouth feature point set has its y The value is the maximum among the 16-20 mouth feature points, and the y value of the lowermost mouth feature point in the mouth feature point set is the minimum among the 16-20 mouth feature points.

In steps S166 and S167, the current user emotional information is determined based on the relationship between the user's mouth opening degree value and the preset mouth opening degree threshold. In the above steps, only the distinction between happy emotions and unhappy emotions is used as an example for specific explanation. It is not limited to only identifying happy emotions and unhappy emotions.

In an embodiment, step S162 includes:

Invoke the face bounding box creation instruction to correspondingly create the current face bounding box; wherein the initial bounding box of the current face bounding box is a preset initial face bounding box;

Judging whether the facial feature points in the user's facial image are all located in the area within the current face bounding box;

If the facial feature points in the user's facial image are not all located in the area of the current face bounding box, the length and width of the initial face bounding box are expanded according to the preset expansion ratio value to update The current face bounding box, returning to the step of judging whether the face feature points in the user's facial image are all located within the area of the current face bounding box;

If the facial feature points in the user's facial image are all located in the area of the current face bounding box, the current face bounding box is taken as the face bounding box that encloses all the facial feature points in the user's facial image .

In this embodiment, through the heuristic method of gradually enlarging the length and width of the current face bounding box, a face bounding box that encloses all the face feature points in the user's face image can be obtained.

S170: Determine whether the current user emotion information is a happy mood or an angry mood.

In this embodiment, when the current user emotion information of the user to be welcomed is obtained through emotion recognition, it is necessary to determine whether the current emotion of the user is happy or angry, so as to determine whether the user is greeted by the user. Satisfaction of the information. Generally, the user is happy after seeing the welcome message (that is, happy emotion), indicating that the user is satisfied with the content of the welcome message; for example, if the user is angry after seeing the welcome message (that is, angry ), indicating that the user is not satisfied with the content of the welcome message.

S180: If the former user's emotional information is a happy mood, automatically add a retainable tag to the welcome message.

In this embodiment, if the current user’s emotional information is happy, it means that there is no need to modify the welcome information corresponding to the user, and the welcome information can be directly retained. Next time the user comes again, it can be called directly The welcome information corresponding to the user is displayed.

In an embodiment, after step S170, the method further includes:

If the former user's emotional information is angry, the tag to be adjusted is automatically added to the welcome information.

In this embodiment, if the current user emotional information is angry, it means that the welcome information corresponding to the user needs to be modified. Next time the user comes again, the modified welcome information corresponding to the user can be called To show.

In an embodiment, after the step of automatically adding a label to be adjusted to the welcome message if the previous user's emotional information is an angry mood, the method further includes:

Call the local current welcome information to be recommended, and update the welcome information labeled as to be adjusted to the current welcome information to be recommended.

In this embodiment, in order to adjust each label to the welcome information to be adjusted in time, the current welcome information to be recommended in the server can be obtained at this time, and the welcome information whose label is to be adjusted is updated to the current welcome information to be recommended. Guest information. In this way, the welcoming information that the user is not satisfied with can be adjusted in time.

This method realizes that the generation and deployment of welcome information is more efficient, and can obtain the user's feedback information on the welcome information based on the user's emotion recognition result.

The embodiment of the present application also provides an information generation device based on face recognition, and the information generation device based on face recognition is used to execute any embodiment of the aforementioned information generation method based on face recognition. Specifically, please refer to FIG. 5, which is a schematic block diagram of an information generating apparatus based on face recognition provided by an embodiment of the present application.

As shown in FIG. 5, the information generation device 100 based on face recognition includes: a first image determination unit 110, a user identity recognition unit 120, a welcome information determination unit 130, a welcome information transmission unit 140, and a second image determination unit 150 , The emotion recognition unit 160, the emotion judgment unit 170, and the first label setting unit 180.

The first image judging unit 110 is used to judge whether the current user image collected by the camera is received.

The user identification unit 120 is configured to, if the current user image collected by the camera is received, perform face recognition on the current user image to obtain corresponding current user identification information.

In an embodiment, as shown in FIG. 6, the user identification unit 120 includes:

The picture preprocessing unit 121 is configured to perform grayscale correction and noise filtering on the current user image to obtain a preprocessed picture;

The feature vector obtaining unit 122 is configured to obtain a feature vector corresponding to the preprocessed picture through a convolutional neural network model;

The vector comparison unit 123 is configured to compare the feature vector corresponding to the current user image with the feature template stored in the face database to determine whether the feature template stored in the face database is similar to the current one. Feature templates with the same feature vector corresponding to the user image;

The user information obtaining unit 124 is configured to obtain corresponding current user identification information if there is a feature template that is the same as the feature vector corresponding to the current user image in the feature templates stored in the face database.

In an embodiment, as shown in FIG. 6, the user identification unit 120 further includes:

The matching failure prompt unit 125 is configured to, if there is no feature template that is the same as the feature vector corresponding to the current user image among the feature templates stored in the face database, perform a prompt that the target user is not recognized, and execute the step of ending the process .

The welcoming information judging unit 130 is used to determine whether there is welcoming information corresponding to the current user identification information stored; wherein the welcoming information includes welcoming text information, welcoming audio information, or welcoming video information.

The welcome information sending unit 140 is configured to, if the welcome information corresponding to the current user identification information is stored, send the welcome information to the corresponding display device for display.

The second image judging unit 150 is used to judge whether the user's facial image collected by the camera is received.

The emotion recognition unit 160 is configured to, if the user's facial image collected by the camera is received, perform emotion recognition on the user's facial image to obtain corresponding current user emotion information.

In an embodiment, as shown in FIG. 7, the emotion recognition unit 160 includes:

The facial feature point acquiring unit 161 is configured to acquire the facial feature points in the user's facial image through a pre-stored face detector;

The face bounding box obtaining unit 162 is configured to obtain a face bounding box that encloses all the facial feature points in the user's facial image;

The mouth feature point set acquiring unit 163 is configured to acquire the mouth feature point set included in the facial feature points in the user's facial image;

The user mouth opening degree value obtaining unit 164 is used to obtain the vertical coordinate spacing value between the uppermost mouth feature point and the lowermost mouth feature point in the mouth feature point set, according to the vertical coordinate spacing value and the face boundary The ratio of the length of the box to obtain the corresponding value of the user's mouth opening degree;

The mouth opening degree value comparison unit 165 is configured to determine whether the user's mouth opening degree value is less than a preset mouth opening degree threshold;

An angry emotion recognition unit 166, configured to set the value of the current user's emotion information as an angry emotion if the user's mouth opening degree value is less than the mouth opening degree threshold;

The happy emotion recognition unit 167 is configured to set the value of the current user's emotion information as happy emotion if the user's mouth opening degree value is greater than or equal to the mouth opening degree threshold value.

In an embodiment, the face bounding box acquiring unit 162 includes:

The initial bounding box creation unit is used to call the face bounding box creation instruction to correspondingly create the current face bounding box; wherein the initial bounding box of the current face bounding box is a preset initial face bounding box;

An in-area judging unit, the user judges whether the facial feature points in the user's facial image are all located in the area within the current face bounding box;

The bounding box adjustment unit is configured to expand the length and width of the initial face bounding box according to a preset if the face feature points in the user's face image are not all located in the area within the current face bounding box The scale value is expanded to update the current face bounding box, and return to the step of determining whether the face feature points in the user's face image are all located in the area of the current face bounding box;

The bounding box selection unit is configured to use the current face bounding box as the face feature in the user facial image if the facial feature points in the user's facial image are all located in the area within the current face bounding box Click the bounding box of the face surrounded by all.

The emotion judging unit 170 is used to judge whether the current user emotion information is a happy mood or an angry mood.

The first tag setting unit 180 is configured to automatically add a retainable tag to the welcome message if the previous user's emotion information is happy emotion.

In an embodiment, the information generating apparatus 100 based on face recognition further includes:

The second label setting unit is configured to automatically add a label to be adjusted to the welcome message if the previous user's emotional information is angry.

The recommended welcome information update unit is used to call the local current welcome information to be recommended, and update the welcome information labeled as to be adjusted to the current welcome information to be recommended

The above-mentioned information generating apparatus based on face recognition can be implemented in the form of a computer program, and the computer program can be run on a computer device as shown in FIG. 8.

Please refer to FIG. 8, which is a schematic block diagram of a computer device according to an embodiment of the present application. The computer device 500 is a server, and the server may be an independent server or a server cluster composed of multiple servers.

Referring to FIG. 8, the computer device 500 includes a processor 502, a memory, and a network interface 505 connected through a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute the information generation method based on face recognition.

The processor 502 is used to provide calculation and control capabilities, and support the operation of the entire computer device 500.

The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute the information generation method based on face recognition.

The network interface 505 is used for network communication, such as providing data information transmission. Those skilled in the art can understand that the structure shown in FIG. 8 is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 500 to which the solution of the present application is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

Wherein, the processor 502 is configured to run a computer program 5032 stored in a memory to implement the information generation method based on face recognition disclosed in the embodiment of the present application.

Those skilled in the art can understand that the embodiment of the computer device shown in FIG. 8 does not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or less components than those shown in the figure. Or combine certain components, or different component arrangements. For example, in some embodiments, the computer device may only include a memory and a processor. In such an embodiment, the structures and functions of the memory and the processor are consistent with the embodiment shown in FIG. 8 and will not be repeated here.

It should be understood that in this embodiment of the application, the processor 502 may be a central processing unit (Central Processing Unit, CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), Application Specific Integrated Circuit (ASIC), Field-Programmable Gate Array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor.

In another embodiment of the present application, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium, or may be a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, where the computer program is executed by a processor to implement the information generation method based on face recognition disclosed in the embodiments of the present application.

Those skilled in the art can clearly understand that, for the convenience and conciseness of description, the specific working process of the above-described equipment, device, and unit can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. A person of ordinary skill in the art may be aware that the units and algorithm steps of the examples described in the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both, in order to clearly illustrate the hardware and software Interchangeability, in the above description, the composition and steps of each example have been generally described in accordance with the function. Whether these functions are executed by hardware or software depends on the specific application and design constraint conditions of the technical solution. Professionals and technicians can use different methods for each specific application to implement the described functions, but such implementation should not be considered beyond the scope of this application.

In the several embodiments provided in this application, it should be understood that the disclosed equipment, device, and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, or the units with the same function may be combined into one. Units, for example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some interfaces, devices or units, and may also be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. It includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), magnetic disk or optical disk and other media that can store program codes.

The above are only specific implementations of this application, but the protection scope of this application is not limited to this. Any person skilled in the art can easily think of various equivalents within the technical scope disclosed in this application. Modifications or replacements, these modifications or replacements shall be covered within the protection scope of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

An information generation method based on face recognition, which includes:

Determine whether the current user image collected by the camera is received;

If the current user image collected by the camera is received, face recognition is performed on the current user image to obtain corresponding current user identification information;

Judging whether there is welcoming information corresponding to the current user identification information; wherein the welcoming information includes welcoming text information, welcoming audio information, or welcoming video information;

If the welcome information corresponding to the current user identification information is stored, sending the welcome information to the corresponding display device for display;

Determine whether the user's facial image collected by the camera is received;

If the user's facial image collected by the camera is received, emotion recognition is performed on the user's facial image to obtain corresponding current user emotional information;

Determine whether the current user emotional information is happy mood or angry mood; and

If the former user's emotion information is happy emotion, a retainable tag is automatically added to the welcome information.
The method for generating information based on face recognition according to claim 1, wherein said performing face recognition on said current user image to obtain corresponding current user identification information comprises:

Performing grayscale correction and noise filtering on the current user image to obtain a preprocessed picture;

Obtaining a feature vector corresponding to the pre-processed picture through a convolutional neural network model;

The feature vector corresponding to the current user image is compared with the feature template stored in the face database to determine whether the feature vector stored in the face database is the same as the feature vector corresponding to the current user image. Feature template

If there is a feature template that is the same as the feature vector corresponding to the current user image among the feature templates stored in the face database, the corresponding current user identification information is obtained.
The method for generating information based on face recognition according to claim 1, wherein said performing emotion recognition on said user's facial image to obtain corresponding current user emotion information comprises:

Acquiring the facial feature points in the facial image of the user through a pre-stored facial detector;

Acquiring a face bounding box that surrounds all the face feature points in the user's face image;

Acquiring a mouth feature point set included in the facial feature points in the user's facial image;

Obtain the longitudinal coordinate spacing value between the uppermost mouth feature point and the lowermost mouth feature point in the mouth feature point set, and obtain the corresponding user according to the ratio of the longitudinal coordinate spacing value to the length of the face bounding box Mouth opening degree value;

Judging whether the user's mouth opening degree value is less than a preset mouth opening degree threshold;

If the user's mouth opening degree value is less than the mouth opening degree threshold value, the value of the current user's emotional information is set as an angry mood;

If the user's mouth opening degree value is greater than or equal to the mouth opening degree threshold value, the value of the current user's emotional information is set as a happy mood.
The method for generating information based on face recognition according to claim 3, wherein said acquiring a face bounding box enclosing all the face feature points in said user's facial image comprises:

Invoke the face bounding box creation instruction to correspondingly create the current face bounding box; wherein the initial bounding box of the current face bounding box is a preset initial face bounding box;

Judging whether the facial feature points in the user's facial image are all located in the area within the current face bounding box;

If the facial feature points in the user's facial image are not all located in the area of the current face bounding box, the length and width of the initial face bounding box are expanded according to the preset expansion ratio value to update The current face bounding box, returning to the step of judging whether the face feature points in the user's facial image are all located within the area of the current face bounding box;

If the facial feature points in the user's facial image are all located in the area of the current face bounding box, the current face bounding box is taken as the face bounding box that encloses all the facial feature points in the user's facial image .
The method for generating information based on face recognition according to claim 2, wherein the feature vector corresponding to the current user image is compared with a feature template stored in the face database to determine After whether there is a feature template that is the same as the feature vector corresponding to the current user image among the stored feature templates, it also includes:

If there is no feature template that is the same as the feature vector corresponding to the current user image among the feature templates stored in the face database, a prompt is made that the target user is not recognized, and the step of ending the process is executed.
The method for generating information based on face recognition according to claim 1, wherein after determining whether the current user emotion information is happy emotion or angry emotion, the method further comprises:

If the former user's emotional information is angry, automatically add a label to be adjusted to the welcome message;

Call the local current welcome information to be recommended, and update the welcome information labeled as to be adjusted to the current welcome information to be recommended.
The method for generating information based on face recognition according to claim 1, wherein the current user identification information and the corresponding welcome information are stored in a blockchain network.
The method for generating information based on face recognition according to claim 1, wherein said determining whether there is welcome information corresponding to said current user identification information is stored; wherein said welcome information includes welcome text After the message, the welcome audio message, or the welcome video message, it also includes:

The server does not store welcome information corresponding to the current user identification information, obtains the pre-stored general welcome information, and sends the general welcome information to the corresponding display device for display.
The method for generating information based on face recognition according to claim 1, wherein after the determining whether the current user image collected by the camera is received, the method further comprises:

If the current user image collected by the camera is not received, wait for the preset delay waiting time, and return to the step of determining whether the current user image collected by the camera is received.
An information generating device based on face recognition, which includes:

The first image judging unit is used to judge whether the current user image collected by the camera is received;

The user identification unit is configured to, if the current user image collected by the camera is received, perform face recognition on the current user image to obtain corresponding current user identification information;

The welcoming information judging unit is used to determine whether the welcoming information corresponding to the current user identification information is stored; wherein the welcoming information includes welcoming text information, welcoming audio information, or welcoming video information ；

The welcome information sending unit is configured to send the welcome information to the corresponding display device for display if the welcome information corresponding to the current user identification information is stored;

The second image judging unit is used to judge whether the user's facial image collected by the camera is received;

The emotion recognition unit is configured to perform emotion recognition on the user's facial image if the user's facial image collected by the camera is received, to obtain corresponding current user's emotion information;

An emotion judgment unit for judging whether the current user emotion information is happy mood or angry mood; and

The first label setting unit is configured to automatically add a retainable label to the welcome message if the previous user's emotional information is happy mood.
A computer device includes a memory, a processor, and a computer program that is stored on the memory and can run on the processor, wherein the processor implements the following steps when the processor executes the computer program:

Determine whether the current user image collected by the camera is received;

If the current user image collected by the camera is received, face recognition is performed on the current user image to obtain corresponding current user identification information;

Judging whether there is welcoming information corresponding to the current user identification information; wherein the welcoming information includes welcoming text information, welcoming audio information, or welcoming video information;

If the welcome information corresponding to the current user identification information is stored, sending the welcome information to the corresponding display device for display;

Determine whether the user's facial image collected by the camera is received;

If the user's facial image collected by the camera is received, emotion recognition is performed on the user's facial image to obtain corresponding current user emotional information;

Determine whether the current user emotional information is happy mood or angry mood; and

If the former user's emotion information is happy emotion, a retainable tag is automatically added to the welcome information.
The computer device according to claim 11, wherein said performing face recognition on said current user image to obtain corresponding current user identification information comprises:

Performing grayscale correction and noise filtering on the current user image to obtain a preprocessed picture;

Obtaining a feature vector corresponding to the pre-processed picture through a convolutional neural network model;

The feature vector corresponding to the current user image is compared with the feature template stored in the face database to determine whether the feature vector stored in the face database is the same as the feature vector corresponding to the current user image. Feature template

If there is a feature template that is the same as the feature vector corresponding to the current user image among the feature templates stored in the face database, the corresponding current user identification information is obtained.
The computer device according to claim 11, wherein said performing emotion recognition on said user's facial image to obtain corresponding current user emotion information comprises:

Acquiring the facial feature points in the facial image of the user through a pre-stored facial detector;

Acquiring a face bounding box that surrounds all the face feature points in the user's face image;

Acquiring a mouth feature point set included in the facial feature points in the user's facial image;

Obtain the longitudinal coordinate spacing value between the uppermost mouth feature point and the lowermost mouth feature point in the mouth feature point set, and obtain the corresponding user according to the ratio of the longitudinal coordinate spacing value to the length of the face bounding box Mouth opening degree value;

Judging whether the user's mouth opening degree value is less than a preset mouth opening degree threshold;

If the user's mouth opening degree value is less than the mouth opening degree threshold value, the value of the current user's emotional information is set as an angry mood;

If the user's mouth opening degree value is greater than or equal to the mouth opening degree threshold value, the value of the current user's emotional information is set as a happy mood.
The computer device according to claim 13, wherein said acquiring a face bounding box that encloses all the face feature points in the user's facial image comprises:

Invoke the face bounding box creation instruction to correspondingly create the current face bounding box; wherein the initial bounding box of the current face bounding box is a preset initial face bounding box;

Judging whether the facial feature points in the user's facial image are all located in the area within the current face bounding box;

If the facial feature points in the user's facial image are not all located in the area of the current face bounding box, the length and width of the initial face bounding box are expanded according to the preset expansion ratio value to update The current face bounding box, returning to the step of judging whether the face feature points in the user's facial image are all located within the area of the current face bounding box;

If the facial feature points in the user's facial image are all located in the area of the current face bounding box, the current face bounding box is taken as the face bounding box that encloses all the facial feature points in the user's facial image .
The computer device according to claim 12, wherein the feature vector corresponding to the current user image is compared with the feature template stored in the face database to determine whether the feature template is stored in the face database. After whether there is a feature template that is the same as the feature vector corresponding to the current user image, it also includes:

If there is no feature template that is the same as the feature vector corresponding to the current user image among the feature templates stored in the face database, a prompt is made that the target user is not recognized, and the step of ending the process is executed.
The computer device according to claim 11, wherein, after determining whether the current user emotion information is a happy mood or an angry mood, the method further comprises:

If the former user's emotional information is angry, automatically add a label to be adjusted to the welcome message;

Call the local current welcome information to be recommended, and update the welcome information labeled as to be adjusted to the current welcome information to be recommended.
The computer device according to claim 11, wherein the current user identification information and the corresponding welcome information are stored in a blockchain network.
11. The computer device according to claim 11, wherein said determining whether to store welcome information corresponding to said current user identification information; wherein said welcome information includes welcome text information and welcome audio information , Or after the welcome video message, it also includes:

The server does not store welcome information corresponding to the current user identification information, obtains the pre-stored general welcome information, and sends the general welcome information to the corresponding display device for display.
The computer device according to claim 11, wherein after said determining whether the current user image collected by the camera is received, it further comprises:

If the current user image collected by the camera is not received, wait for the preset delay waiting time, and then return to the step of determining whether the current user image collected by the camera is received.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program is executed by a processor to perform the following operations:

Determine whether the current user image collected by the camera is received;

If the current user image collected by the camera is received, face recognition is performed on the current user image to obtain corresponding current user identification information;

Judging whether there is welcoming information corresponding to the current user identification information; wherein the welcoming information includes welcoming text information, welcoming audio information, or welcoming video information;

If the welcome information corresponding to the current user identification information is stored, sending the welcome information to the corresponding display device for display;

Determine whether the user's facial image collected by the camera is received;

If the user's facial image collected by the camera is received, emotion recognition is performed on the user's facial image to obtain corresponding current user emotional information;

Determine whether the current user emotional information is happy mood or angry mood; and

If the former user's emotion information is happy emotion, a retainable tag is automatically added to the welcome information.