CN112633239A

CN112633239A - Micro-expression identification method and device

Info

Publication number: CN112633239A
Application number: CN202011637994.XA
Authority: CN
Inventors: 唐绮雯; 黄维登; 程亮; 宁博
Original assignee: Industrial and Commercial Bank of China Ltd ICBC
Current assignee: Industrial and Commercial Bank of China Ltd ICBC
Priority date: 2020-12-31
Filing date: 2020-12-31
Publication date: 2021-04-09

Abstract

The invention provides a micro-expression recognition method and device, and relates to the technical field of information security. The method comprises the following steps: obtaining a video to be identified; extracting key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2; intercepting a human face micro expression key area from each key frame image to obtain a micro expression key area image of each key frame image; encrypting the micro-expression key area images of the N key frame images to obtain micro-expression encryption information; and sending a micro-expression identification request to carry out micro-expression identification, wherein the micro-expression identification request comprises micro-expression encryption information. The device is used for executing the method. The micro expression identification method and the device provided by the embodiment of the invention can improve the safety of information transmission in the micro expression identification process.

Description

Micro-expression identification method and device

Technical Field

The invention relates to the technical field of information security, in particular to a micro-expression identification method and device.

Background

Micro-expressions are very fast expressions of short duration that can convey various emotions of a human. The micro-expression has good application prospect in the fields of medical treatment, safety protection and the like.

The micro expression video transmission encryption technology is characterized in that when a micro expression acquisition end carries out face micro expression video acquisition, video data is immediately encrypted, end-to-end transmission encryption during micro expression identification is realized, hackers are prevented from counterfeiting, tampering and embezzling micro expression video information, the safety of video identification data is ensured, and the safety of a micro expression identification process is improved. However, due to the addition of the video encryption function, the time consumption of the whole micro expression recognition process is increased while the security is improved, and the recognition efficiency of the micro expressions is reduced.

Disclosure of Invention

Aiming at the problems in the prior art, the embodiment of the invention provides a method and a device for identifying a micro expression, which can at least partially solve the problems in the prior art.

In a first aspect, the present invention provides a micro expression recognition method, including:

obtaining a video to be identified;

extracting key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2;

intercepting a human face micro expression key area from each key frame image to obtain a micro expression key area image of each key frame image;

encrypting the micro-expression key area images of the N key frame images to obtain micro-expression encryption information;

and sending a micro expression identification request to carry out micro expression identification, wherein the micro expression identification request comprises micro expression encryption information.

In a second aspect, the present invention further provides a micro expression recognition method, including:

receiving a micro-expression identification request, wherein the micro-expression identification request comprises micro-expression encryption information;

decrypting the micro-expression encryption information to obtain micro-expression key region graphs of the N key frame images; wherein N is a positive integer greater than or equal to 2;

and carrying out micro-expression recognition on the micro-expression key area graph of each key frame image to obtain a micro-expression recognition result of each micro-expression key area graph.

In a third aspect, the present invention provides a micro expression recognition apparatus, comprising:

the obtaining unit is used for obtaining a video to be identified;

the extraction unit is used for extracting key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2;

the intercepting unit is used for intercepting a human face micro expression key area from each key frame image to obtain a micro expression key area image of each key frame image;

the encryption unit is used for encrypting the microexpression key area images of the N key frame images to obtain microexpression encryption information;

the sending unit is used for sending a micro expression identification request to carry out micro expression identification, and the micro expression identification request comprises micro expression encryption information.

In a fourth aspect, the present invention further provides a micro expression recognition apparatus, including:

the system comprises a receiving unit, a processing unit and a processing unit, wherein the receiving unit is used for receiving a micro-expression identification request which comprises micro-expression encryption information;

the decryption unit is used for decrypting the micro-expression encryption information to obtain micro-expression key region graphs of the N key frame images; wherein N is a positive integer greater than or equal to 2;

and the identification unit is used for carrying out micro-expression identification on the micro-expression key area graph of each key frame image to obtain a micro-expression identification result of each micro-expression key area graph.

In a fifth aspect, the present invention provides an electronic device, including a first memory, a first processor, and a first computer program stored in the first memory and executable on the first processor, wherein the first processor implements the following micro-expression recognition method when executing the first computer program:

obtaining a video to be identified;

In a sixth aspect, the present invention provides a computer readable storage medium having stored thereon a first computer program which, when executed by a first processor, implements a micro-expression recognition method comprising:

obtaining a video to be identified;

In a seventh aspect, the present invention further provides an electronic device, including a second memory, a second processor, and a second computer program stored in the second memory and executable on the second processor, where the second processor implements the following micro-expression recognition method when executing the second computer program:

In an eighth aspect, the present invention also provides a computer-readable storage medium having stored thereon a second computer program which, when executed by a second processor, implements the following micro-expression recognition method:

The micro expression recognition method and the device provided by the embodiment of the invention can obtain a video to be recognized, extract key frame images of the video to be recognized, obtain N key frame images, intercept a human face micro expression key area from each key frame image, obtain a micro expression key area image of each key frame image, encrypt the micro expression key area images of the N key frame images to obtain micro expression encryption information, send a micro expression recognition request to perform micro expression recognition, and improve the safety of information transmission in the micro expression recognition process by obtaining and encrypting the N key frame images.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to the drawings without creative efforts. In the drawings:

fig. 1 is a schematic structural diagram of a micro expression recognition system according to a first embodiment of the present invention.

Fig. 2 is a flowchart illustrating a micro expression recognition method according to a second embodiment of the present invention.

Fig. 3 is a flowchart illustrating a micro expression recognition method according to a third embodiment of the present invention.

Fig. 4 is a flowchart illustrating a micro expression recognition method according to a fourth embodiment of the present invention.

Fig. 5 is a flowchart illustrating a micro expression recognition method according to a fifth embodiment of the present invention.

Fig. 6 is a flowchart illustrating a micro expression recognition method according to a sixth embodiment of the present invention.

Fig. 7 is a flowchart illustrating a micro expression recognition method according to a seventh embodiment of the present invention.

Fig. 8 is a flowchart illustrating a micro expression recognition method according to an eighth embodiment of the present invention.

Fig. 9 is a flowchart illustrating a micro expression recognition method according to a ninth embodiment of the present invention.

Fig. 10 is a flowchart illustrating a micro expression recognition method according to a tenth embodiment of the present invention.

Fig. 11 is a schematic flowchart of capturing a key region of a human face microexpression from each key frame image according to an eleventh embodiment of the present invention.

Fig. 12 is a flowchart illustrating an encryption process for micro-expression key region maps of N key frame images according to a twelfth embodiment of the present invention.

Fig. 13 is a flowchart illustrating a micro expression recognition method according to a thirteenth embodiment of the present invention.

Fig. 14 is a schematic structural diagram of a micro expression recognition device according to a fourteenth embodiment of the present invention.

Fig. 15 is a schematic structural diagram of a micro expression recognition device according to a fifteenth embodiment of the present invention.

Fig. 16 is a schematic physical structure diagram of an electronic device according to a sixteenth embodiment of the present invention.

Fig. 17 is a schematic physical structure diagram of an electronic device according to a seventeenth embodiment of the present invention.

Detailed Description

In order to make the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the embodiments of the present invention are further described in detail below with reference to the accompanying drawings. The exemplary embodiments and descriptions of the present invention are provided to explain the present invention and not to limit the present invention. It should be noted that the embodiments and features of the embodiments in the present application may be arbitrarily combined with each other without conflict.

Fig. 1 is a schematic structural diagram of a micro expression recognition system according to a first embodiment of the present invention, and as shown in fig. 1, the micro expression recognition system according to the embodiment of the present invention includes a video capture terminal 1 and a recognition server 2, where:

the video acquisition terminal 1 is in communication connection with the identification server 2; the video acquisition terminal 1 may acquire the video to be identified in real time through the camera or intercept a section of the video as the video to be identified, and set the video according to actual conditions, which is not limited in the embodiments of the present invention. The video acquisition terminal 1 includes, but is not limited to, a bank self-service terminal, a desktop computer, and the like.

The video acquisition terminal 1 is used for processing a video to be identified, acquiring micro-expression encryption information, carrying the micro-expression encryption information in a micro-expression identification request, and sending the micro-expression encryption information to the identification server 2.

And the identification server 2 receives the micro expression identification request, decrypts the micro expression encryption information, obtains a micro expression key area image of each key frame image, and then carries out micro expression identification on the micro expression key area image of each key frame image.

Fig. 2 is a schematic flow chart of a micro expression recognition method according to a second embodiment of the present invention, and as shown in fig. 2, the micro expression recognition method according to the embodiment of the present invention includes:

s201, acquiring a video to be identified;

specifically, the video acquisition terminal can obtain a video to be identified. The video to be identified may be acquired by the video acquisition terminal in real time through a camera, or may be captured from the video by the video acquisition terminal, and the video to be identified is set according to actual needs. The execution subject of the micro-expression recognition method provided by the embodiment of the invention includes but is not limited to a video acquisition terminal.

For example, when a customer transacts business to a self-service device of a bank business outlet, the self-service device may collect a video of a face of the customer for a period of time in real time through a camera to serve as a video to be recognized.

For example, when a person appearing in a surveillance video needs to be subjected to micro-expression recognition, a video segment including a face of the person can be captured from the surveillance video as a video to be recognized.

S202, extracting key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2;

specifically, after the video acquisition terminal obtains the video to be identified, the video to be identified may be subjected to key frame image extraction, and N key frame images are extracted, where the N key frame images are used for performing micro-expression identification at an identification server. Wherein N is a positive integer greater than or equal to 2.

S203, intercepting a human face micro expression key area from each key frame image to obtain a micro expression key area image of each key frame image;

specifically, after the video acquisition terminal obtains N key frame images, for each key frame image, a human face micro expression key area is captured from the N key frame images, and a micro expression key area image of each key frame image is obtained.

For example, if only one face region exists in a key frame image, the whole face region can be intercepted as a micro-expression key region map of the key frame image.

For example, if a plurality of face regions exist in a key frame image, one of the face regions may be intercepted as a micro-expression key region map of the key frame image.

S204, encrypting the micro-expression key region images of the N key frame images to obtain micro-expression encryption information;

specifically, the video acquisition terminal encrypts micro-expression key region maps of the N key frame images to obtain micro-expression encryption information. The encryption mode used for encrypting the micro-expression key area images of the N key frame images is set according to actual needs, and the embodiment of the invention is not limited.

S205, sending a micro-expression identification request, wherein the micro-expression identification request comprises micro-expression encryption information.

Specifically, after the video acquisition terminal obtains the micro-expression encryption information, the video acquisition terminal can send a micro-expression identification request to perform micro-expression identification. Wherein the micro expression identification request comprises micro expression encryption information.

For example, the video acquisition terminal may carry the micro expression encryption information in a micro expression identification request and directly send the micro expression encryption information to the identification server, and after receiving the micro expression identification request, the identification server may decrypt the micro expression encryption information to obtain micro expression key area maps of N key frame images, and then perform micro expression identification on the micro expression key area maps of each key frame image to obtain a micro expression corresponding to the micro expression key area map of each key frame image. The identification algorithm adopted for micro-expression identification of the micro-expression key area graph is selected according to actual needs, and the embodiment of the invention is not limited.

For example, the video acquisition terminal may carry the micro-expression encryption information in a micro-expression identification request and send the micro-expression encryption information to the service server, and the service server forwards the micro-expression identification request to the identification server, so that the identification server performs subsequent micro-expression identification.

The micro expression recognition method provided by the embodiment of the invention can obtain a video to be recognized, extract key frame images of the video to be recognized, obtain N key frame images, intercept a face micro expression key area from each key frame image, obtain a micro expression key area image of each key frame image, encrypt the micro expression key area images of the N key frame images to obtain micro expression encryption information, send a micro expression recognition request to perform micro expression recognition, and improve the safety of information transmission in the micro expression recognition process by obtaining and encrypting the N key frame images. In addition, the whole video to be identified does not need to be encrypted and transmitted, so that the time consumption of encryption and decryption is reduced while the information transmission safety is improved, and the micro expression identification efficiency is improved.

Fig. 3 is a schematic flow chart of a micro expression recognition method according to a third embodiment of the present invention, and as shown in fig. 3, on the basis of the foregoing embodiments, further performing key frame image extraction on the video to be recognized to obtain N key frame images includes:

s2021, if the current frame image comprises the face micro expression, comparing the current frame image with the previous frame image;

specifically, the video acquisition terminal acquires images from the video to be recognized frame by frame, and for each frame of image, whether the human face micro expression exists or not can be judged through the micro expression recognition model. And if the video acquisition terminal judges that the current frame image comprises the human face micro-expression, acquiring a previous frame image of the current frame image, and then comparing the current frame image with the previous frame image. The micro expression recognition model may be, for example, an Affectiva micro expression recognition model, and is selected according to actual needs, which is not limited in the embodiment of the present invention.

S2022, if there is a difference between the current frame image and the previous frame image, extracting the current frame image as a key frame image.

Specifically, when comparing the current frame image with the previous frame image of the current frame image, the video capture terminal may determine whether the current frame image is different from the previous frame image, and if the current frame image is different from the previous frame image, extract the current frame image as a key frame image.

Fig. 4 is a schematic flow chart of a micro expression recognition method according to a fourth embodiment of the present invention, and as shown in fig. 4, on the basis of the foregoing embodiments, further, the capturing a human face micro expression key region from each key frame image to obtain a micro expression key region map of each key frame image includes:

s401, positioning all face areas and respective position information in the key frame image through a face positioning algorithm;

specifically, the video acquisition terminal may locate all face regions and respective position information in the key frame image through a face location algorithm. The face positioning algorithm is selected according to actual needs, and the embodiment of the invention is not limited. The position information indicates the position of the corresponding face region in the key frame image.

For example, the face area is a rectangular area, and the coordinates of the upper left corner of the rectangular area and the length and width of the rectangular area may be used as the position information corresponding to the face area.

S402, if only one face area is obtained, the face area is intercepted based on the position information of the face area and is used as a micro-expression key area image of the key frame image.

Specifically, if the video acquisition terminal knows that there is only one face region, that is, only one face region is detected in the key frame image through a face positioning algorithm, the face region may be intercepted as a micro-expression key region map of the key frame image based on the position information of the face region, that is, a picture of the face region in the key frame image is taken as the micro-expression key region map.

Fig. 5 is a schematic flow chart of a micro expression recognition method according to a fifth embodiment of the present invention, and as shown in fig. 5, on the basis of the foregoing embodiments, further, the capturing a face micro expression key area from each key frame image to obtain a micro expression key area map of each key frame image includes:

s501, positioning all face areas and respective position information in the key frame image through a face positioning algorithm;

specifically, the video acquisition terminal may locate all face regions and respective position information in the key frame image through a face location algorithm. The specific implementation process of this step is similar to step S401, and is not described herein again.

S502, if M human face regions are obtained and preset parameters are started, intercepting the largest human face region based on the position information of the largest human face region in the M human face regions as a micro-expression key region graph of the key frame image; wherein M is a positive integer greater than or equal to 2.

Specifically, the video acquisition terminal may count the number of face regions in the key frame image, and if M face regions are known and preset parameters are turned on, may calculate the size of each face region in the M face regions, obtain the largest face region through comparison, and then intercept the largest face region as the micro-expression key region map of the key frame image based on the position information of the largest face region in the M face regions. Wherein M is a positive integer and M is greater than or equal to 2. The preset parameters are preset and used for indicating the judgment of the maximum face area.

Fig. 6 is a schematic flow chart of a micro expression recognition method according to a sixth embodiment of the present invention, and as shown in fig. 6, the step of capturing a micro expression key region of a human face from each key frame image to obtain a micro expression key region map of each key frame image includes:

s601, positioning all face areas and respective position information in the key frame image through a face positioning algorithm;

S602, if M personal face areas are obtained and preset parameters are not started, obtaining face areas meeting the centered face judgment rule according to each face area in the M personal face areas and the centered face judgment rule; wherein M is a positive integer greater than or equal to 2;

specifically, the video acquisition terminal may count the number of face regions in the key frame image, and if M face regions are known and a preset parameter is not turned on, it may be determined whether a face region satisfying the centered face determination rule exists according to position information of each face region in the M face regions and the centered face determination rule, and if a face region satisfying the centered face determination rule exists, it may be determined the face region of the centered face determination rule. The centered face judgment rule is used for determining a face region closest to the geometric center of gravity of the key frame image from the M face regions, and the face region is set according to actual needs. M is a positive integer greater than or equal to 2.

For example, the centered face determination rule is that the minimum distance between the geometric gravity center of each face region and the geometric gravity center of the picture to be recognized is obtained, and the minimum distance is smaller than a preset value. The video acquisition terminal can obtain the geometric barycentric coordinates of each face region according to the position information of each face region in the M personal face regions, then calculate the distance between the geometric barycentric coordinates of each face region and the geometric barycentric of the picture to be recognized, obtain the minimum distance from the M distances, compare the minimum distance with the preset value, and if the minimum distance is smaller than the preset value, take the face region corresponding to the minimum distance as the face region meeting the centered face judgment rule. The preset value is set according to actual experience, and the embodiment of the invention is not limited.

S603, based on the position information of the face region meeting the centered face judgment rule, intercepting the face region meeting the centered face judgment rule as a micro-expression key region map of the key frame image.

Specifically, after the video acquisition terminal obtains the face region meeting the centered face judgment rule, the face region meeting the centered face judgment rule may be intercepted as the micro-expression key region map of the key frame image according to the position information of the face region meeting the centered face judgment rule.

On the basis of the foregoing embodiments, further, the capturing the human face micro-expression key region from each key frame image, and obtaining the micro-expression key region map of each key frame image includes:

and if each face area in the M face areas does not meet the centered face judgment rule, intercepting the largest face area based on the position information of the largest face area in the M face areas as a micro-expression key area image of the key frame image.

Specifically, if the video acquisition terminal knows that an M-face region exists and a preset parameter is not started, it may be determined whether a face region satisfying the centered face determination rule exists according to the position information of each face region in the M-face region and the centered face determination rule, and if the face region satisfying the centered face determination rule does not exist, the size of each face region in the M-face region may be calculated, a largest face region is obtained through comparison, and then the largest face region is intercepted as the micro-expression key region map of the key frame image based on the position information of the largest face region in the M-face region.

For example, the centered face determination rule is that the minimum distance between the geometric gravity center of each face region and the geometric gravity center of the picture to be recognized is obtained, and the minimum distance is smaller than a preset value. The video acquisition terminal can obtain the geometric barycentric coordinate of each face area according to the position information of each face area in the M face areas, then calculate the distance between the geometric barycentric coordinate of each face area and the geometric barycentric of the picture to be recognized, obtain the minimum distance from the M distances, then compare the minimum distance with the preset value, and if the minimum distance is larger than or equal to the preset value, each face area in the M face areas does not meet the centered face judgment rule. The video acquisition terminal can calculate the size of each face area in the M personal face areas, obtains the largest face area through comparison, and then intercepts the largest face area as a micro-expression key area image of the key frame image based on the position information of the largest face area in the M personal face areas.

Fig. 7 is a schematic flowchart of a micro expression recognition method according to a seventh embodiment of the present invention, and as shown in fig. 7, on the basis of the foregoing embodiments, further encrypting the micro expression key area maps of the N key frame images, and obtaining micro expression encryption information includes:

s2041, encrypting the micro expression key region images of the N key frame images through a symmetric encryption key to obtain micro expression ciphertext; wherein the symmetric encryption key is randomly generated;

specifically, in order to encrypt the micro-expression key region maps of the N key frame images, the video capture terminal may randomly generate a symmetric encryption key, and encrypt the micro-expression key region maps of the N key frame images using the symmetric encryption key to obtain a micro-expression ciphertext. The algorithm used for generating the symmetric encryption key is set according to actual needs, and the embodiment of the invention is not limited.

S2042, encrypting the symmetric encryption key by using the asymmetric encryption public key to obtain a key ciphertext;

specifically, the video capture terminal encrypts the symmetric encryption key by using the asymmetric encryption public key to obtain a key ciphertext. Wherein the asymmetric encryption public key is preset. It is understood that step S2041 and step S2042 have no precedence relationship, and step S2041 or step S2042 may be executed first.

S2043, the micro expression ciphertext, the key ciphertext and the asymmetric encryption public key are used as the micro expression encryption information.

Specifically, after obtaining the micro expression ciphertext and the key ciphertext, the video acquisition terminal uses the micro expression ciphertext, the key ciphertext and the asymmetric encryption public key as the micro expression encryption information.

On the basis of the above embodiments, further, the micro expression recognition method provided by the embodiment of the present invention further includes:

performing completion operation on each intercepted key frame image to obtain each completed key frame image;

accordingly, sending the micro-expression recognition request comprises:

and sending a micro-expression identification request carrying the completed key frame images.

Specifically, for each key frame image, after the video acquisition terminal intercepts the key region of the face micro expression, the video acquisition terminal may perform a complementing operation on the intercepted key frame image, that is, color filling is performed on the position of the original key region of the face micro expression on the intercepted key frame image, so that the intercepted key frame image is complete, and the complemented key frame image is obtained. When color filling is performed, pure color filling may be adopted, and the pure color is, for example, black, and is set according to actual needs, which is not limited in the embodiment of the present invention.

When the video acquisition terminal sends the micro expression identification request, each complemented key frame image can be carried in the micro expression identification request to be sent.

Fig. 8 is a schematic flow chart of a micro expression recognition method according to an eighth embodiment of the present invention, and as shown in fig. 8, the micro expression recognition method according to the embodiment of the present invention includes:

s801, receiving a micro-expression identification request, wherein the micro-expression identification request comprises micro-expression encryption information;

specifically, the video acquisition terminal may send a micro-expression recognition request to the recognition server, where the micro-expression recognition request includes micro-expression encryption information. The identification server receives the micro-expression identification request. The micro-expression encryption information comprises micro-expression key area graphs of the N encrypted key frame images. The execution subject of the micro expression recognition method provided by the embodiment of the invention comprises but is not limited to a recognition server.

S802, decrypting the micro-expression encryption information to obtain micro-expression key area graphs of the N key frame images; wherein N is a positive integer greater than or equal to 2;

specifically, after receiving the micro expression recognition request, the recognition server decrypts the micro expression recognition request to obtain micro expression key region maps of the N key frame images. Wherein N is a positive integer greater than or equal to 2.

And S803, performing micro expression recognition on the micro expression key area image of each key frame image to obtain a micro expression recognition result of each micro expression key area image.

Specifically, after obtaining the micro-expression key region maps of the N key frame images, the identification server may perform micro-expression identification on the micro-expression key region map of each key frame image by using a micro-expression identification model, so as to obtain a micro-expression identification result of each micro-expression key region map. The micro expression recognition model may be, for example, an Affectiva micro expression recognition model, and is selected according to actual needs, which is not limited in the embodiment of the present invention.

The micro expression identification method provided by the embodiment of the invention can receive the micro expression identification request, decrypt the micro expression encryption information, obtain the micro expression key area maps of N key frame images, perform micro expression identification on the micro expression key area map of each key frame image, and obtain the micro expression identification result of each micro expression key area map, thereby reducing the time consumption for decryption caused by encrypting the micro expression image, and improving the micro expression identification efficiency while ensuring the information transmission safety.

Fig. 9 is a schematic flowchart of a micro expression recognition method according to a ninth embodiment of the present invention, and as shown in fig. 9, based on the foregoing embodiments, further, the micro expression encryption information includes a micro expression ciphertext, a key ciphertext, and an asymmetric encryption public key; correspondingly, the decrypting the micro-expression encryption information to obtain the micro-expression key region maps of the N key frame images comprises the following steps:

s8021, acquiring a corresponding asymmetric encryption private key according to the asymmetric encryption public key;

specifically, the micro expression encryption information includes a micro expression ciphertext, a key ciphertext, and an asymmetric encryption public key. The identification server can obtain an asymmetric encryption private key corresponding to the asymmetric encryption public key according to the query of the asymmetric encryption public key. The asymmetric encryption private key is stored in advance and corresponds to the asymmetric encryption public key.

S8022, decrypting the key ciphertext according to the asymmetric encryption private key to obtain a symmetric encryption key;

specifically, after obtaining the asymmetric encryption private key, the identification server decrypts the key ciphertext according to the asymmetric encryption private key, so as to obtain a symmetric encryption key.

S8023, decrypting the micro-expression ciphertext according to the symmetric encryption key to obtain micro-expression key region maps of the N key frame images.

Specifically, after obtaining the symmetric encryption key, the identification server may decrypt the micro-expression ciphertext according to the symmetric encryption key to obtain micro-expression key region maps of the N key frame images.

Fig. 10 is a schematic flow chart of a micro expression recognition method according to a tenth embodiment of the present invention, and as shown in fig. 10, the flow chart of the micro expression recognition method according to the embodiment of the present invention is as follows:

firstly, obtaining a video to be identified. The video acquisition terminal can acquire and obtain a video to be identified in real time or intercept a section of video as the video to be identified.

And secondly, judging the quality of the video image. The video acquisition terminal judges the image quality of a video to be identified, judges whether the video image has quality problems such as over-brightness, over-darkness and blurring, and returns to the first step to obtain the video to be identified again if the video image has the quality problems; if there is no image quality problem, the third step is entered. The judgment of the video image quality is performed by the prior art, and is not described herein.

And thirdly, judging whether the human face image exists or not. The video acquisition terminal judges whether a face image exists in the video to be identified, and if the face image is detected, the fourth step is carried out; if the face image is not detected, the video to be recognized is obtained again in the first step.

And fourthly, extracting operation. The video acquisition terminal extracts key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2.

And fifthly, intercepting operation. And the video acquisition terminal intercepts the human face micro-expression key area from each key frame image to obtain a micro-expression key area image of each key frame image.

And sixthly, encrypting operation. The video acquisition terminal encrypts the micro-expression key region images of the N key frame images to obtain micro-expression encryption information.

And step seven, sending a micro expression recognition request. The video acquisition terminal carries the micro expression encryption information in a micro expression identification request and sends the micro expression encryption information to the identification server so as to carry out micro expression identification.

Fig. 11 is a schematic flow chart of capturing a micro-expression key region from each key frame image according to an eleventh embodiment of the present invention, and as shown in fig. 11, the flow chart of capturing the micro-expression key region from each key frame image according to the embodiment of the present invention is as follows:

firstly, positioning a face area. The video acquisition terminal carries out face detection on the key frame image through a face region positioning algorithm so as to detect a face region and corresponding position information in the key frame image.

And secondly, judging whether a plurality of face areas exist. The video acquisition terminal counts the detected face regions, judges whether a plurality of face regions exist, and enters a third step if the plurality of face regions exist; if there is only one face region, then go to the sixth step.

And thirdly, judging whether to start preset parameters. The video acquisition terminal detects whether the preset parameters are started, and if the preset parameters are started, the sixth step is carried out; and if the preset parameters are not opened, entering the fourth step.

And fourthly, judging whether a centered human face exists or not. And the video acquisition terminal judges whether a face area meeting the centered face judgment rule exists according to the position information of each face area in the M personal face areas and the centered face judgment rule. If the face area meeting the centered face judgment rule exists, entering a seventh step; and if the face area meeting the centered face judgment rule does not exist, the step five is carried out. Wherein N is a positive integer greater than or equal to 2.

And fifthly, positioning the maximum face area. And the video acquisition terminal calculates the size of each face area in the M face areas, obtains the largest face area through comparison, and then enters the eighth step.

And sixthly, positioning the absolute maximum face area. And the video acquisition terminal calculates the size of each face area in the M face areas, and the largest face area is obtained through comparison.

And seventhly, positioning the centered face area. The video acquisition terminal obtains the geometric barycentric coordinates of each face area according to the position information of each face area in the M face areas, then calculates the distance between the geometric barycentric coordinates of each face area and the geometric barycentric of the picture to be recognized, obtains the minimum distance from the M distances, compares the minimum distance with a preset value, and if the minimum distance is smaller than the preset value, takes the face area corresponding to the minimum distance as the face area meeting the centered face judgment rule, and completes the positioning of the centered face area.

And eighthly, obtaining a micro-expression key area graph. And the video acquisition terminal intercepts the face area from the key frame image based on the position information of the face area as a micro-expression key area image of the key frame image.

Fig. 12 is a schematic flowchart of a process of encrypting the microexpression key area maps of N key frame images according to a twelfth embodiment of the present invention, and as shown in fig. 12, the process of encrypting the microexpression key area maps of N key frame images according to the embodiment of the present invention includes:

firstly, obtaining an asymmetric encryption public key. And the video acquisition terminal acquires the asymmetric encryption public key. Wherein the asymmetric encryption public key is preset.

And secondly, judging whether the asymmetric encryption public key is acquired. The video acquisition terminal acquires the asymmetric encryption public key, and then the third step is carried out; and if the asymmetric encryption public key is not acquired, entering a seventh step.

And thirdly, generating a symmetric encryption key. And the video acquisition terminal randomly generates a symmetric encryption key. The algorithm for generating the symmetric encryption key is set according to actual needs, and the embodiment of the invention is not limited.

And fourthly, encrypting by using a symmetric encryption key. And the video acquisition terminal encrypts the micro expression key region images of the N key frame images by using the symmetric encryption key to obtain the micro expression ciphertext.

And fifthly, encrypting by using the asymmetric encryption public key. And the video acquisition terminal encrypts the symmetric encryption key to obtain a key ciphertext.

And sixthly, acquiring micro-expression encryption information. The video acquisition terminal takes the micro expression ciphertext, the key ciphertext and the asymmetric encryption public key as micro expression encryption information, and the micro expression encryption information is carried in the micro expression identification request and is sent to the identification server.

And step seven, ending the encryption process.

Fig. 13 is a schematic flow chart of a micro expression recognition method according to a thirteenth embodiment of the present invention, and as shown in fig. 13, the flow chart of the micro expression recognition method according to the embodiment of the present invention is as follows:

the method comprises the following steps of firstly, receiving a micro expression recognition request. The identification server receives a face identification request sent by the video acquisition terminal, wherein the face identification request comprises micro expression encryption information.

And secondly, decrypting the micro-expression encryption information. The identification server obtains micro-expression encrypted information from the micro-expression identification request, and then decrypts the micro-expression encrypted information to obtain micro-expression key area maps of the N key frame images.

And thirdly, judging whether the decryption is successful. If the recognition server obtains the micro-expression key area graphs of the N key frame images, entering a fourth step; and if the micro-expression key area maps of the N key frame images are not obtained, the step five is proceeded.

And fourthly, performing micro-expression recognition. The recognition server adopts a micro expression recognition model to perform face recognition on the micro expression key region graph of each key frame image, and obtains the recognition result of each micro expression key region graph.

And step five, ending the process.

The micro expression recognition method provided by the embodiment of the invention solves the problem of safe information transmission in the current micro expression recognition process, meets the requirement of transmission safety, improves the safety protection capability of the micro expression recognition process, effectively reduces the time consumption of transmission encryption, solves the problem of large ciphertext length after encryption, reduces the time consumption of decryption of micro expression recognition, improves the efficiency of micro expression recognition under the condition of ensuring data safety, and gives consideration to safety and usability in the micro expression recognition process.

Fig. 14 is a schematic structural diagram of a micro expression recognition apparatus according to a fourteenth embodiment of the present invention, and as shown in fig. 14, the micro expression recognition apparatus according to the present invention includes an obtaining unit 1401, an extracting unit 1402, an intercepting unit 1403, an encrypting unit 1404, and a transmitting unit 1405, where:

the obtaining unit 1401 is used for obtaining a video to be identified; the extraction unit 1402 is configured to perform key frame image extraction on the video to be identified, so as to obtain N key frame images; wherein N is a positive integer greater than or equal to 2; the intercepting unit 1403 is used for intercepting the human face micro expression key area from each key frame image to obtain a micro expression key area image of each key frame image; the encryption unit 1404 is configured to encrypt the micro-expression key region maps of the N key frame images to obtain micro-expression encryption information; the sending unit 1405 is configured to send a micro expression recognition request to perform micro expression recognition, where the micro expression recognition request includes micro expression encryption information.

Specifically, the obtaining unit 1401 can obtain a video to be recognized. The video to be identified may be acquired by the video acquisition terminal in real time through a camera, or may be captured from the video by the video acquisition terminal, and the video to be identified is set according to actual needs, which is not limited in the embodiment of the present invention.

After obtaining the video to be identified, the extracting unit 1402 may extract key frame images from the video to be identified, and extract N key frame images, where the N key frame images are used for performing micro-expression identification at an identification server. Wherein N is a positive integer greater than or equal to 2.

After obtaining the N key frame images, for each key frame image, the capturing unit 1403 captures a micro expression key region of the face from the key frame image, and obtains a micro expression key region map of each key frame image.

The encryption unit 1404 encrypts the microexpression key region maps of the N key frame images to obtain microexpression encryption information. The encryption mode used for encrypting the micro-expression key area maps of the N key frame images is set according to actual needs, and the embodiment of the invention is not limited.

After obtaining the micro expression encryption information, the transmission unit 1405 may transmit a micro expression recognition request for micro expression recognition. Wherein the micro expression identification request comprises micro expression encryption information.

The micro expression recognition device provided by the embodiment of the invention can obtain a video to be recognized, extract key frame images of the video to be recognized, obtain N key frame images, intercept a face micro expression key area from each key frame image, obtain a micro expression key area image of each key frame image, encrypt the micro expression key area images of the N key frame images to obtain micro expression encryption information, send a micro expression recognition request to perform micro expression recognition, and improve the safety of information transmission in the micro expression recognition process by obtaining and encrypting the N key frame images. In addition, the whole video to be identified does not need to be encrypted and transmitted, so that the time consumption of encryption and decryption is reduced while the information transmission safety is improved, and the micro expression identification efficiency is improved.

Fig. 15 is a schematic structural diagram of a micro expression recognition device according to a fifteenth embodiment of the present invention, and as shown in fig. 15, the micro expression recognition device according to the embodiment of the present invention includes a receiving unit 1501, a decryption unit 1502, and a recognition unit 1503, where:

the receiving unit 1501 is configured to receive a micro expression recognition request, where the micro expression recognition request includes micro expression encryption information; the decryption unit 1502 is configured to decrypt the micro-expression encrypted information to obtain micro-expression key region maps of N key frame images; wherein N is a positive integer greater than or equal to 2; the identification unit 1503 is configured to perform micro-expression identification on the micro-expression key region map of each key frame image to obtain a micro-expression identification result of each micro-expression key region map.

Specifically, the receiving unit 1501 may transmit a micro expression recognition request including micro expression encryption information to the recognition server. The identification server receives the micro-expression identification request. The micro-expression encryption information comprises micro-expression key area graphs of the N encrypted key frame images.

After receiving the micro expression recognition request, the decryption unit 1502 decrypts the micro expression recognition request to obtain micro expression key region maps of the N key frame images. Wherein N is a positive integer greater than or equal to 2.

After obtaining the micro expression key region maps of the N key frame images, the identifying unit 1503 may perform micro expression identification on the micro expression key region map of each key frame image by using a micro expression identification model to obtain a micro expression identification result of each micro expression key region map. The micro expression recognition model may be, for example, an Affectiva micro expression recognition model, and is selected according to actual needs, which is not limited in the embodiment of the present invention.

The micro expression recognition device provided by the embodiment of the invention can receive the micro expression recognition request, decrypt the micro expression encryption information, obtain the micro expression key area maps of N key frame images, perform micro expression recognition on the micro expression key area map of each key frame image, and obtain the micro expression recognition result of each micro expression key area map, thereby reducing the time consumption for decryption caused by encrypting the micro expression image, and improving the micro expression recognition efficiency while ensuring the information transmission safety.

The embodiment of the apparatus provided in the embodiment of the present invention may be specifically configured to execute the processing flow of the corresponding method embodiment, and the functions of the apparatus are not described herein again, and refer to the detailed description of the corresponding method embodiment.

Fig. 16 is a schematic physical structure diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 16, the electronic device may include: a first Processor (Processor)1601, a first communication Interface (Communications Interface)1602, a first Memory (Memory)1603 and a first communication bus 1604, wherein the first Processor 1601, the first communication Interface 1602 and the first Memory 1603 are in communication with each other via the first communication bus 1604. The first processor 1601 may call logic instructions in the first memory 1603 to perform methods including, for example: obtaining a video to be identified; extracting key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2; intercepting a human face micro-expression key area from each key frame image to obtain a micro-expression key area image of each key frame image; encrypting the micro-expression key area images of the N key frame images to obtain micro-expression encryption information; and sending a micro-expression identification request to carry out micro-expression identification, wherein the micro-expression identification request comprises micro-expression encryption information.

The present embodiments disclose a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, the computer is capable of performing a method comprising, for example: obtaining a video to be identified; extracting key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2; intercepting a human face micro expression key area from each key frame image to obtain a micro expression key area image of each key frame image; encrypting the micro-expression key area images of the N key frame images to obtain micro-expression encryption information; and sending a micro-expression identification request to carry out micro-expression identification, wherein the micro-expression identification request comprises micro-expression encryption information.

The present embodiment provides a computer-readable storage medium storing a first computer program that causes a computer to execute a method, for example, including: obtaining a video to be identified; extracting key frame images of the video to be identified to obtain N key frame images; wherein N is a positive integer greater than or equal to 2; intercepting a human face micro expression key area from each key frame image to obtain a micro expression key area image of each key frame image; encrypting the micro-expression key area images of the N key frame images to obtain micro-expression encryption information; and sending a micro-expression identification request to carry out micro-expression identification, wherein the micro-expression identification request comprises micro-expression encryption information.

Fig. 17 is a schematic physical structure diagram of an electronic device according to another embodiment of the present invention, and as shown in fig. 17, the electronic device may include: a second Processor (Processor)1701, a second communication Interface (Communications Interface)1702, a second Memory (Memory)1703 and a second communication bus 1704, wherein the second Processor 1701, the second communication Interface 1702 and the second Memory 1703 complete communication with each other through the second communication bus 1704. The second processor 1701 may call logical instructions in the second memory 1703 to perform methods including, for example: receiving a micro-expression identification request, wherein the micro-expression identification request comprises micro-expression encryption information; decrypting the micro-expression encryption information to obtain micro-expression key region graphs of the N key frame images; wherein N is a positive integer greater than or equal to 2; and carrying out micro-expression recognition on the micro-expression key area graph of each key frame image to obtain a micro-expression recognition result of each micro-expression key area graph.

The present embodiments disclose a computer program product comprising a computer program stored on a computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, the computer is capable of performing a method comprising, for example: receiving a micro expression identification request, wherein the micro expression identification request comprises micro expression encryption information; decrypting the micro expression encryption information to obtain micro expression key region graphs of N key frame images; wherein N is a positive integer greater than or equal to 2; and carrying out micro-expression recognition on the micro-expression key area graph of each key frame image to obtain a micro-expression recognition result of each micro-expression key area graph.

The present embodiment provides a computer-readable storage medium storing a second computer program that causes a computer to execute a method including, for example: receiving a micro expression identification request, wherein the micro expression identification request comprises micro expression encryption information; decrypting the micro expression encryption information to obtain micro expression key area graphs of N key frame images; wherein N is a positive integer greater than or equal to 2; and carrying out micro expression recognition on the micro expression key area graph of each key frame image to obtain a micro expression recognition result of each micro expression key area graph.

Furthermore, the logic instructions in the first memory 1603 and the second memory 1703 can be implemented in the form of software functional units and stored in a computer readable storage medium when sold or used as a stand-alone product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, an apparatus, or a network device) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

In the description herein, references to the description of the term "one embodiment," a particular embodiment, "" some embodiments, "" e.g., "an example," "a particular example," or "some examples," etc., mean that a particular feature, structure, material, or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the invention. In this specification, the schematic representations of the terms used above do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

The above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention, and it should be understood that the above-mentioned embodiments are only exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims

1. A micro-expression recognition method is characterized by comprising the following steps:

obtaining a video to be identified;

and sending a micro-expression identification request to carry out micro-expression identification, wherein the micro-expression identification request comprises micro-expression encryption information.

2. The method according to claim 1, wherein the extracting key frame images from the video to be identified to obtain N key frame images comprises:

if the current frame image comprises the face micro expression, comparing the current frame image with the previous frame image;

and if the current frame image is different from the previous frame image, extracting the current frame image as a key frame image.

3. The method of claim 1, wherein the capturing the micro-expression key area of the face from each key frame image, and obtaining the micro-expression key area map of each key frame image comprises:

positioning all face areas and respective position information in the key frame image through a face positioning algorithm;

and if only one face area is known, intercepting the face area based on the position information of the face area to be used as a micro-expression key area image of the key frame image.

4. The method of claim 1, wherein the capturing the micro-expression key area of the face from each key frame image, and obtaining the micro-expression key area map of each key frame image comprises:

if M face regions are known and preset parameters are started, intercepting the largest face region based on the position information of the largest face region in the M face regions as a micro-expression key region graph of the key frame image; wherein M is a positive integer greater than or equal to 2.

5. The method of claim 1, wherein the capturing the micro-expression key area of the face from each key frame image, and obtaining the micro-expression key area map of each key frame image comprises:

if M personal face areas are obtained and preset parameters are not started, obtaining a face area meeting the centered face judgment rule according to each face area in the M personal face areas and the centered face judgment rule; wherein M is a positive integer greater than or equal to 2;

and intercepting the face area meeting the centered face judgment rule as a micro-expression key area image of the key frame image based on the position information of the face area meeting the centered face judgment rule.

6. The method of claim 5, wherein the step of capturing the micro-expression key area of the face from each key frame image to obtain the micro-expression key area map of each key frame image comprises:

7. The method according to claim 1, wherein the encrypting the microexpressing key area maps of the N key frame images to obtain microexpressing encryption information comprises:

encrypting the micro-expression key region images of the N key frame images by using a symmetric encryption key to obtain micro-expression ciphertext; wherein the symmetric encryption key is randomly generated;

encrypting the symmetric encryption key by using the asymmetric encryption public key to obtain a key ciphertext;

and taking the micro expression ciphertext, the key ciphertext and the asymmetric encryption public key as the micro expression encryption information.

8. The method of any one of claims 1 to 7, further comprising:

accordingly, sending the micro-expression recognition request comprises:

9. A micro-expression recognition method is characterized by comprising the following steps:

10. The method of claim 9, wherein the micro-expression encryption information comprises a micro-expression ciphertext, a key ciphertext, and an asymmetric encryption public key; correspondingly, the decrypting the micro-expression encryption information to obtain the micro-expression key region maps of the N key frame images comprises the following steps:

acquiring a corresponding asymmetric encryption private key according to the asymmetric encryption public key;

decrypting the key ciphertext according to the asymmetric encryption private key to obtain a symmetric encryption key;

and decrypting the micro expression ciphertext according to the symmetric encryption key to obtain micro expression key region graphs of the N key frame images.

11. A micro expression recognition device, comprising:

the obtaining unit is used for obtaining a video to be identified;

the encryption unit is used for encrypting the micro-expression key area images of the N key frame images to obtain micro-expression encryption information;

the sending unit is used for sending a micro-expression identification request to carry out micro-expression identification, and the micro-expression identification request comprises micro-expression encryption information.

12. A micro expression recognition device, comprising:

13. An electronic device comprising a first memory, a first processor and a first computer program stored on the first memory and executable on the first processor, wherein the first processor implements the method of any of claims 1 to 8 when executing the first computer program.

14. A computer-readable storage medium, on which a first computer program is stored, which, when being executed by a first processor, carries out the method of any one of claims 1 to 8.

15. An electronic device comprising a second memory, a second processor and a second computer program stored on the second memory and executable on the second processor, characterized in that the second processor implements the method of claim 9 or 10 when executing the second computer program.

16. A computer-readable storage medium, on which a second computer program is stored, which, when being executed by a second processor, carries out the method of claim 9 or 10.