WO2016165614A1 - Procédé de reconnaissance d'expression en vidéo instantanée et équipement électronique - Google Patents

Procédé de reconnaissance d'expression en vidéo instantanée et équipement électronique Download PDF

Info

Publication number
WO2016165614A1
WO2016165614A1 PCT/CN2016/079115 CN2016079115W WO2016165614A1 WO 2016165614 A1 WO2016165614 A1 WO 2016165614A1 CN 2016079115 W CN2016079115 W CN 2016079115W WO 2016165614 A1 WO2016165614 A1 WO 2016165614A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature point
point coordinate
face
feature
instant video
Prior art date
Application number
PCT/CN2016/079115
Other languages
English (en)
Chinese (zh)
Inventor
武俊敏
Original Assignee
美国掌赢信息科技有限公司
武俊敏
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 美国掌赢信息科技有限公司, 武俊敏 filed Critical 美国掌赢信息科技有限公司
Publication of WO2016165614A1 publication Critical patent/WO2016165614A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames

Definitions

  • the present invention relates to the field of video, and in particular, to an expression recognition method and an electronic device in an instant video.
  • instant video applications With the popularity of instant video applications on mobile terminals, more and more users realize interaction with others through instant video applications. Therefore, an expression recognition method in instant video is needed to satisfy users.
  • the instant video application realizes the personalized needs when interacting with others, and improves the user experience in the interactive scenario.
  • the prior art provides an expression recognition method, which specifically includes: acquiring a current frame picture to be recognized from a pre-recorded video, identifying a facial expression in the current frame picture, and continuing the above steps on other frame images. To identify facial expressions in the video frame picture in the video.
  • the method cannot recognize the facial expression in the real-time video in real time, and in the implementation process, since the method occupies a large amount of processing resources and storage resources of the device, the method has high requirements on the device, and the method cannot be applied.
  • mobile terminals such as smart phones and tablets, it is unable to meet the diverse needs of users and reduce the user experience.
  • the embodiment of the present invention provides an expression recognition method and an electronic device in an instant video.
  • the technical solution is as follows:
  • an expression recognition method in an instant video comprising:
  • the feature point is used to describe the current expression of the face
  • the feature vector includes feature point coordinates and texture feature point coordinates under a standard pose matrix
  • the texture feature points are used to uniquely determine the feature points
  • Obtaining a feature vector corresponding to at least one feature point of the face in the instant video frame includes:
  • the acquiring the at least one feature point coordinate and the at least one texture feature point coordinate in the standard pose matrix includes:
  • the at least one feature point is normalized to obtain the at least one of the standard pose matrix
  • the at least one texture feature point coordinate of the feature point includes:
  • Rotating the current pose matrix into a standard pose matrix and acquiring the at least one feature point coordinate and the at least one texture feature point coordinate under the standard pose matrix.
  • the identifying the feature vector corresponding to the at least one feature point includes:
  • the determining, according to the recognition result, that the current expression is one of a plurality of pre-stored expressions includes:
  • the recognition result is within the preset range, it is determined that the expression corresponding to the feature vector is one of a plurality of pre-stored expressions.
  • an electronic device comprising:
  • An acquiring module configured to acquire a feature vector corresponding to at least one feature point of a face in an instant video frame, where the feature point is used to describe a current expression of the face;
  • An identification module configured to identify a feature vector corresponding to the at least one feature point, and generate a recognition result
  • a determining module configured to determine, according to the recognition result, the current expression as one of a plurality of pre-stored expressions.
  • the acquiring module is further configured to acquire the at least one feature point coordinate and the at least one texture feature point coordinate in the standard pose matrix;
  • the identification module is further configured to generate a feature vector corresponding to the at least one feature point according to the at least one feature point coordinate and the at least one texture feature point coordinate in the standard pose matrix.
  • the acquiring module is further configured to acquire the at least one feature point coordinate of the face in the instant video frame and the at least one texture feature point coordinate;
  • the device further includes a processing module, configured to perform normalization processing on the at least one feature point to obtain the at least one feature point coordinate and the at least one texture feature point coordinate in the standard pose matrix.
  • the obtaining module is further configured to: according to the at least one feature point of a face in the instant video frame Obtaining, by the coordinates and the at least one texture feature point coordinate, the at least one feature point of the face in the instant video frame and the current pose matrix corresponding to the at least one texture feature point;
  • the processing module is further configured to rotate the current pose matrix into a standard pose matrix, and acquire the at least one feature point coordinate and the at least one texture feature point coordinate under the standard pose matrix.
  • the device further includes:
  • a calculation module configured to input a feature vector corresponding to the at least one feature point into a preset expression model library for calculation, and obtain the recognition result.
  • the determining module is specifically configured to:
  • the recognition result is within the preset range, it is determined that the expression corresponding to the feature vector is one of a plurality of pre-stored expressions.
  • an electronic device including a video input module, a video output module, a sending module, a receiving module, a memory, and the video input module, the video output module, the sending module, and the receiving And a processor coupled to the memory, wherein the memory stores a set of program code, the processor is configured to invoke program code stored in the memory, and perform the following operations:
  • the feature point is used to describe the current expression of the face
  • the processor is further configured to invoke program code stored in the memory, and perform the following operations:
  • the at least one feature point coordinate and the at least one texture according to the standard pose matrix Feature point coordinates, generating feature vectors corresponding to the at least one feature point.
  • the processor is further configured to invoke program code stored in the memory, and perform the following operations:
  • the processor is further configured to invoke the program code stored in the memory, and perform the following operations:
  • Rotating the current pose matrix into a standard pose matrix and acquiring the at least one feature point coordinate and the at least one texture feature point coordinate under the standard pose matrix.
  • the processor is further configured to invoke the program code stored in the memory, and perform the following operations:
  • the processor is further configured to invoke the program code stored in the memory, and perform the following operations:
  • the recognition result is within the preset range, it is determined that the expression corresponding to the feature vector is one of a plurality of pre-stored expressions.
  • An embodiment of the present invention provides an expression recognition method and an electronic device in an instant video, including: acquiring a feature vector corresponding to at least one feature point of a face in an instant video frame, where the feature point is used to describe the current face of the face An expression; identifying a feature vector corresponding to the at least one feature point to generate a recognition result; and determining, according to the recognition result, the current expression as one of a plurality of pre-stored expressions.
  • Obtaining feature points through feature points by acquiring feature points for describing a current expression of a face in an instant video
  • the corresponding feature vector can more accurately represent the current expression of the face, and then obtain the recognition result according to the feature vector by identifying the feature vector, which simplifies the complexity of the algorithm for recognizing the face in the instant video, so that the embodiment of the present invention provides
  • the method can be run on the mobile terminal to meet the diverse needs of the user and improve the user experience.
  • FIG. 1 is a schematic diagram of an interaction system according to an embodiment of the present invention
  • FIG. 2 is a schematic diagram of an interaction system according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of an interaction system according to an embodiment of the present invention.
  • FIG. 5 is a flowchart of an expression recognition method in an instant video according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
  • An embodiment of the present invention provides an expression recognition method in an instant video, where the method is applied to an interactive system including at least two mobile terminals and a server, wherein the mobile terminal can run an instant video program, and the user can run Instant video program on the mobile terminal to achieve interaction with others
  • the mobile terminal may be a smart phone, a tablet computer, or another mobile terminal.
  • the specific mobile terminal is not limited in the embodiment of the present invention.
  • the mobile terminal at least includes a video input module and a video display module, the video input module may include a camera, and the video display module may include a display screen, and the instant video program may implement real-time video input by controlling a video input module of the mobile terminal, and may also control the video.
  • the display module enables the display of instant video.
  • the interactive system can be referred to FIG. 1 , in which the mobile terminal 1 is an instant video sender, the mobile terminal 2 is an instant video receiver, and the instant video sent by the mobile terminal 1 is forwarded to the mobile terminal 2 via the server;
  • the user of the mobile terminal 1 and the user of the mobile terminal 2 can interact through the interactive system.
  • the execution body of the method provided by the embodiment of the present invention may be any one of the mobile terminal 1, the mobile terminal 2, and the server. If the execution subject of the method is the mobile terminal 1, the mobile terminal 1 After receiving the instant video input through the video input module of the user, performing facial expression recognition on the face in the instant video, forwarding the recognition result to the mobile terminal 2 via the server, and/or outputting the recognition result through the display screen of the same;
  • the execution body of the method is a server, and after the mobile terminal 1 and/or the mobile terminal 2 input the live video through the video input module of the method, the instant video is sent to the server, and the server recognizes the facial expression in the instant video.
  • the recognition result is sent to the mobile terminal 1 and/or the mobile terminal 2; if the execution subject of the method is the mobile terminal 2, the mobile terminal 1 sends the live video to the server after inputting the live video through its own video input module.
  • the server sends the instant video to the mobile terminal 2, and the mobile terminal 2 performs the facial expression in the instant video. Identification, forwarding the recognition result to the mobile terminal 1 via the server, and/or outputting the recognition result through its own display screen.
  • the specific implementation body of the method in the interaction system is not limited in the embodiment of the present invention.
  • the method provided by the embodiment of the present invention can also be applied to an interactive system including only the mobile terminal 1 and the mobile terminal 2.
  • the interactive system can be referred to FIG. 2, wherein the The mobile terminal in the interactive system is the same as the mobile terminal in the interactive system shown in FIG. 1, and details are not described herein again.
  • the execution body of the method provided by the embodiment of the present invention may be Any one of the mobile terminal 1 and the mobile terminal 2, if the execution subject of the method is the mobile terminal 1, the mobile terminal 1 performs an expression on the face in the instant video after inputting the live video through its own video input module Identifying, then transmitting the recognition result to the communication device 2, and/or outputting the recognition result through its own display screen; if the execution subject of the method is the mobile terminal 2, the mobile terminal 1 inputs the live video through its own video input module Sending the live video to the mobile terminal 2, the mobile terminal 2 performs expression recognition on the face in the instant video, and then transmits the recognition result to the mobile terminal 1, and/or outputs the recognition result through its own display screen.
  • the specific implementation body of the method in the interaction system is not limited in the embodiment of the present invention.
  • the method provided by the embodiment of the present invention can also be applied to an interactive system including only the mobile terminal 1 and the user.
  • the interactive system can be referred to FIG. 3, wherein the mobile terminal 1 includes at least a video input.
  • the module and the video display module, the video input module may include a camera, the video display module may include a display screen, and at least one instant video program may be run in the mobile terminal, and the instant video program controls the video input module and the video display module of the mobile terminal to perform instant video.
  • the mobile terminal receives the instant video input by the user, performs facial expression recognition on the instant video, and outputs the recognition result through the display screen of the user.
  • the mobile terminal in the embodiment of the present invention may be one or multiple, and the specific mobile terminal is not limited in the embodiment of the present invention.
  • embodiment of the present invention may further include other application scenarios, and the specific application scenario is not limited in the embodiment of the present invention.
  • An embodiment of the present invention provides an expression recognition method in an instant video. As shown in FIG. 4, the method includes:
  • the feature vector includes feature point coordinates and texture feature point coordinates under a standard pose matrix, and the texture feature points are used to uniquely determine feature points.
  • acquiring the feature vector corresponding to the at least one feature point of the face in the instant video frame includes:
  • the process of obtaining at least one feature point coordinate and at least one texture feature point coordinate under the standard pose matrix may be:
  • the at least one feature point is normalized, and at least one feature point coordinate and at least one texture feature point coordinate under the standard pose matrix are obtained.
  • the process of normalizing at least one feature point and acquiring at least one texture feature point coordinate of at least one feature point under the standard pose matrix may be:
  • Rotating the current pose matrix into a standard pose matrix and acquiring at least one feature point coordinate and at least one texture feature point coordinate under the standard pose matrix.
  • the feature vector corresponding to the at least one feature point is input into the preset expression model library for calculation, and the recognition result is obtained.
  • the recognition result is within the preset range, it is determined that the expression corresponding to the feature vector is one of a plurality of pre-stored expressions.
  • Embodiments of the present invention provide an expression recognition method and an electronic device in an instant video.
  • the feature vector corresponding to the feature point is obtained by the feature point, and the current expression of the face is more accurately represented, and then the feature is recognized.
  • the vector obtains the recognition result according to the feature vector, which simplifies the complexity of the algorithm for recognizing the face in the instant video, so that the method provided by the embodiment of the present invention can be run on the mobile terminal, satisfies the diversified needs of the user, and improves the vector. user experience.
  • An embodiment of the present invention provides an expression recognition method in an instant video. Referring to FIG. 5, the method flow includes:
  • the at least one feature point is used to describe a current expression of a face in an instant video.
  • the at least one feature point is used to describe the outline of the face detail, and the face detail includes at least the eyes, the mouth, the eyebrows, and the nose.
  • the manner in which the face feature points are obtained is not limited in the embodiment of the present invention.
  • the feature parameter may include coordinates of the feature point in a vector including at least a face face, and may further include the feature point including at least a face of the face The scale and direction of the vector indicated in the section.
  • a texture feature point is acquired near each feature point, and the texture feature point is used to uniquely determine the feature point, and the texture feature point does not change with changes in light, angle, and the like.
  • feature points and texture feature points can be extracted from the face by a preset extraction model or an extraction algorithm.
  • feature points and textures can be extracted from the face by other means.
  • Feature points, the specific extraction model, the extraction algorithm, and the extraction method are not limited in the embodiment of the present invention.
  • the texture feature point describes the region where the feature point is located
  • the texture feature point can be used to uniquely determine the feature point, so that the face detail is determined according to the feature point and the texture feature point, and the feature point in the instant video is guaranteed to be the same as the actual feature point.
  • a position ensures the recognition quality of the image details, thereby improving the reliability of the expression recognition.
  • the attitude matrix is used to indicate the scale and direction of the vector indicated by the three-dimensional coordinates of the feature point and the feature texture point corresponding to the feature point.
  • the process can be:
  • the normalization process may be:
  • the corresponding scale and direction are the scale and direction in the two-dimensional coordinates, so a preset conversion algorithm converts coordinates, scales and directions corresponding to the texture feature points corresponding to the at least one feature point and each feature point into two-dimensional coordinates, the at least one feature point and each feature in two-dimensional coordinates
  • the coordinates, scales, and directions corresponding to the texture feature points corresponding to the points; the specific algorithm and the conversion mode are not limited in the embodiment of the present invention.
  • step c can also be implemented in the following manner:
  • the current pose matrix is used to indicate the scale and direction of the vector indicated by the feature point;
  • the embodiment of the present invention does not limit the specific manner of rotating the current posture matrix into a standard posture matrix.
  • step 502 to step 503 at least one feature point is normalized to obtain at least one texture feature point coordinate of at least one feature point in the standard pose matrix, and in addition, other methods may be adopted.
  • the specific manner is not limited in the embodiment of the present invention.
  • the embodiment of the present invention normalizes the at least one feature point coordinate of the face in the instant video and the at least one texture feature point coordinate, so that the acquired pose matrix is not affected by, for example, illumination changes and perspective changes, and Compared with the traditional expression recognition, the expression recognition in the instant video is not changed by the change of the attitude zoom, so that the expression recognition is more accurate.
  • steps 501 to 503 are processes for acquiring at least one feature point coordinate and at least one texture feature point coordinate in the standard pose matrix, and the process may be implemented in other manners, in the embodiment of the present invention.
  • the specific method is not limited.
  • the obtained at least one feature point and the at least one texture feature point are acquired in the standard pose matrix, and the influence of external factors such as illumination and angle on the instant video face is excluded, so that the acquired feature point and the texture feature point are more comparable.
  • Sex makes the expression in the total recognition of instant video more accurate.
  • the orientation matrix indicates the direction and the scale of the feature point. Therefore, at least one feature point coordinate corresponding to the standard pose matrix and at least one texture feature point coordinate corresponding to the at least one feature point may be acquired according to the standard pose matrix.
  • the embodiment of the present invention does not limit the manner in which the feature vector corresponding to at least one feature point is generated according to at least one feature point coordinate and at least one texture feature point coordinate in the standard pose matrix. set.
  • steps 501 to 504 are the process of acquiring the feature vector corresponding to the at least one feature point of the face in the instant video frame, and the process may be implemented in other manners. The way is not limited.
  • the feature vector is input into a preset expression model corresponding to each expression for calculation.
  • the preset expression model can be a regression equation, which can be:
  • A is the regression coefficient
  • x is the feature vector
  • y is the recognition result
  • the result y is calculated according to the feature vector in the preset expression model corresponding to each expression, and the recognition result in the at least one preset expression model is obtained.
  • the step is to identify the feature vector corresponding to the at least one feature point, and to generate the process of the recognition result.
  • the process may be implemented in other manners, and the specific method is not limited in the embodiment of the present invention. .
  • the current expression is determined to be one of a plurality of pre-stored expressions according to the y value included in the recognition result of the feature vector in the preset expression model corresponding to each expression.
  • step 506 is a process for determining that the current expression is one of a plurality of pre-stored expressions according to the recognition result.
  • the process may be implemented in other manners. The process is not limited.
  • the method process further includes:
  • the number n of frames for recognizing the expression is determined in the process of the instant video, and the sum of the scores of each of the acquired expressions in the n frames is calculated, and the highest sum of the scores is the recognized expression in the n frames.
  • n is an integer greater than or equal to 2.
  • the facial expression in the instant video is constantly changing, at least one recognition result is generated by identifying the facial expression in the two or more instant video frames, and then determining the instant video frame according to the at least one recognition result.
  • the recognition result is generated by recognizing the facial expression in one frame, and the facial expression in the instant video is determined according to the recognition result, and the recognition result is more accurate, and the expression recognition can be further improved. Reliability to improve the user experience.
  • the method process further includes:
  • the models of each expression are respectively trained, and the preset expressions to be established are taken as positive samples, and the other preset expressions are used as negative samples, and the logistic regression equation indicated in step 505 is used for training.
  • the process may be:
  • the expression to be trained is taken as a positive sample, and the other expressions are used as negative samples.
  • the output result y 1
  • the output result y 0;
  • parameter A acquisition process in the logistic regression equation can be:
  • the instant expressions of all the obtained users in the instant video are input into a preset optimization formula to generate a parameter A, and the preset optimization formula may be:
  • J(A) represents the parameter A
  • y i is the predicted A value of the prediction function
  • y i ' is the true value of A.
  • step 508 is not required to be performed each time step 501 to step 506 is performed.
  • Embodiments of the present invention provide an expression recognition method and an electronic device in an instant video.
  • the feature vector corresponding to the feature point is obtained by the feature point, and the current expression of the face is more accurately represented, and then the feature vector is obtained according to the feature vector.
  • Obtaining the recognition result simplifies the complexity of the algorithm for recognizing the face in the instant video, so that the method provided by the embodiment of the present invention can be run on the mobile terminal, satisfies the diversified needs of the user, and improves the user experience.
  • the texture feature point describes the region where the feature point is located
  • the texture feature point can be used to uniquely determine the feature point, so that the face detail is determined according to the feature point and the texture feature point, and the feature point and the actual feature point in the instant video are guaranteed. In the same position, the recognition quality of the image details is ensured, thereby improving the reliability of the expression recognition.
  • the distortion rate in image processing is improved, and the reliability of image processing is increased.
  • the acquired pose matrix is not affected by, for example, illumination changes and viewing angle changes, and the like.
  • the expression recognition in the instant video is not changed by the change of the attitude zoom, so that the expression recognition is more accurate.
  • the acquired at least one feature point and the at least one texture feature point are acquired in the standard pose matrix, and the influence of external factors such as illumination and angle on the instant video face is excluded, so that the acquired feature point and the texture feature point are more Comparable, making the expression of total recognition in real-time video more accurate.
  • the computational complexity is reduced, and the recognition of the face is faster in the process of instant video, reducing the occupation of the system process, the occupation of processing resources and storage resources. , improve the operating efficiency of the processor.
  • An embodiment of the present invention provides an electronic device 6.
  • the electronic device 6 includes:
  • the obtaining module 61 is configured to acquire a feature corresponding to at least one feature point of the face in the instant video frame Vector, feature points are used to describe the current expression of the face;
  • the identification module 62 is configured to identify a feature vector corresponding to the at least one feature point, and generate a recognition result
  • the determining module 63 is configured to determine, according to the recognition result, that the current expression is one of a plurality of expressions stored in advance.
  • the obtaining module 61 is further configured to acquire at least one feature point coordinate and at least one texture feature point coordinate in the standard pose matrix;
  • the identification module 62 is further configured to generate a feature vector corresponding to the at least one feature point according to the at least one feature point coordinate and the at least one texture feature point coordinate in the standard pose matrix.
  • the obtaining module 61 is further configured to: acquire at least one feature point coordinate of the face in the instant video frame and at least one texture feature point coordinate;
  • the device further includes a processing module, configured to normalize the at least one feature point to obtain at least one feature point coordinate and at least one texture feature point coordinate in the standard pose matrix.
  • the obtaining module 61 is further configured to acquire, according to at least one feature point coordinate of the face and at least one texture feature point coordinate of the face in the instant video frame, at least one feature point of the face in the instant video frame and a current pose corresponding to the at least one texture feature point. matrix;
  • the processing module is further configured to rotate the current pose matrix into a standard pose matrix, and acquire at least one feature point coordinate and at least one texture feature point coordinate under the standard pose matrix.
  • the electronic device 6 further includes:
  • the calculation module is configured to input the feature vector corresponding to the at least one feature point into the preset expression model library for calculation, and obtain the recognition result.
  • the determining module 63 is specifically configured to:
  • the recognition result is within the preset range, it is determined that the expression corresponding to the feature vector is one of a plurality of pre-stored expressions.
  • An embodiment of the present invention provides an electronic device, which is obtained by using an instant video. Describe the feature points of the current expression of the face, so that the feature vector corresponding to the feature point is more accurately represented by the feature point, and then the recognition result is obtained by identifying the feature vector, and the recognition result is simplified according to the feature vector.
  • the complexity of the algorithm for recognizing a face in the instant video enables the method provided by the embodiment of the present invention to be run on the mobile terminal, satisfies the diverse needs of the user, and improves the user experience.
  • the electronic device 7 includes a video input module 71, a video output module 72, a sending module 73, a receiving module 74, a memory 75, and a video input module 71, and a video output.
  • the module 72, the transmitting module 73, the receiving module 74 and the processor 75 are connected to the processor 76, wherein the memory 75 stores a set of program codes, and the processor 76 is configured to call the program code stored in the memory 75 to perform the following operations:
  • the current expression is one of a plurality of expressions stored in advance.
  • the processor 76 is configured to call the program code stored in the memory 75, and perform the following operations:
  • the processor 76 is configured to call the program code stored in the memory 75, and perform the following operations:
  • the at least one feature point is normalized to obtain at least one feature point coordinate and at least one texture feature point coordinate under the standard pose matrix.
  • the processor 76 is configured to call the program code stored in the memory 75, and perform the following operations:
  • Rotating the current pose matrix into a standard pose matrix and acquiring at least one feature point coordinate and at least one texture feature point coordinate under the standard pose matrix.
  • the processor 76 is configured to call the program code stored in the memory 75, and perform the following operations:
  • the feature vector corresponding to the at least one feature point is input into the preset expression model library for calculation, and the recognition result is obtained.
  • the processor 76 is configured to call the program code stored in the memory 75, and perform the following operations:
  • the recognition result is within the preset range, it is determined that the expression corresponding to the feature vector is one of a plurality of pre-stored expressions.
  • An embodiment of the present invention provides an electronic device that obtains a feature point for describing a current expression of a face in an instant video, so that the feature vector corresponding to the feature point is more accurately represented by the feature point.
  • the current expression of the face by identifying the feature vector, and obtaining the recognition result according to the feature vector, simplifies the complexity of the algorithm for recognizing the face in the instant video, so that the method provided by the embodiment of the present invention can be run on the mobile terminal, satisfying The diverse needs of users have improved the user experience.
  • the electronic device provided by the foregoing embodiment is only illustrated by the division of each functional module. In actual applications, the functions may be assigned differently according to needs.
  • the function module is completed, that is, the internal structure of the electronic device is divided into different functional modules to complete all or part of the functions described above.
  • the electronic device and the method embodiment of the foregoing embodiments are in the same concept, and the specific implementation process is described in detail in the method embodiment, and details are not described herein again.
  • the storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé de reconnaissance d'expression dans une vidéo instantanée qui appartient au champ technique des vidéos. Le procédé consiste : à acquérir un vecteur caractéristique correspondant à au moins un point de caractéristique d'un visage humain dans une trame vidéo instantanée, le point caractéristique servant à la description d'une expression actuelle du visage humain (401) ; à reconnaître le vecteur caractéristique correspondant au(x) point(s) caractéristique(s) pour générer un résultat de reconnaissance (402) ; et à déterminer, en fonction du résultat de reconnaissance, que l'expression actuelle fait partie d'une pluralité d'expressions préenregistrées (403). Selon le procédé, des expressions de visage humain dans une vidéo instantanée sont reconnues en fonction de vecteurs caractéristiques, de sorte que les demandes diversifiées des utilisateurs sont remplies et que l'expérience de l'utilisateur est améliorée.
PCT/CN2016/079115 2015-04-16 2016-04-13 Procédé de reconnaissance d'expression en vidéo instantanée et équipement électronique WO2016165614A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201510182122.1 2015-04-16
CN201510182122.1A CN104794444A (zh) 2015-04-16 2015-04-16 一种即时视频中的表情识别方法和电子设备

Publications (1)

Publication Number Publication Date
WO2016165614A1 true WO2016165614A1 (fr) 2016-10-20

Family

ID=53559232

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/079115 WO2016165614A1 (fr) 2015-04-16 2016-04-13 Procédé de reconnaissance d'expression en vidéo instantanée et équipement électronique

Country Status (2)

Country Link
CN (1) CN104794444A (fr)
WO (1) WO2016165614A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541422A (zh) * 2020-12-08 2021-03-23 北京科技大学 光照和头部姿态鲁棒的表情识别方法、设备及存储介质

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104794444A (zh) * 2015-04-16 2015-07-22 美国掌赢信息科技有限公司 一种即时视频中的表情识别方法和电子设备
CN109309866B (zh) * 2017-07-27 2022-03-08 腾讯科技（深圳）有限公司 图像处理方法及装置、存储介质
CN109934156A (zh) * 2019-03-11 2019-06-25 重庆科技学院 一种基于elman神经网络的用户体验评价方法及系统
CN109978996B (zh) * 2019-03-28 2021-06-11 北京达佳互联信息技术有限公司 生成表情三维模型的方法、装置、终端及存储介质
CN110213667B (zh) * 2019-04-16 2022-04-05 佛山市丰智胜教育咨询服务有限公司 在线视频交互的网络保障方法、系统、设备及存储介质
CN111460945A (zh) * 2020-03-25 2020-07-28 亿匀智行(深圳)科技有限公司 一种基于人工智能获取rgb视频中3d表情的算法

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080107311A1 (en) * 2006-11-08 2008-05-08 Samsung Electronics Co., Ltd. Method and apparatus for face recognition using extended gabor wavelet features
CN103488293A (zh) * 2013-09-12 2014-01-01 北京航空航天大学 一种基于表情识别的人机情感交互系统及方法
CN104077579A (zh) * 2014-07-14 2014-10-01 上海工程技术大学 基于专家系统的人脸表情图像识别方法
CN104123545A (zh) * 2014-07-24 2014-10-29 江苏大学 一种实时表情特征提取及表情识别方法
CN104794444A (zh) * 2015-04-16 2015-07-22 美国掌赢信息科技有限公司 一种即时视频中的表情识别方法和电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080107311A1 (en) * 2006-11-08 2008-05-08 Samsung Electronics Co., Ltd. Method and apparatus for face recognition using extended gabor wavelet features
CN103488293A (zh) * 2013-09-12 2014-01-01 北京航空航天大学 一种基于表情识别的人机情感交互系统及方法
CN104077579A (zh) * 2014-07-14 2014-10-01 上海工程技术大学 基于专家系统的人脸表情图像识别方法
CN104123545A (zh) * 2014-07-24 2014-10-29 江苏大学 一种实时表情特征提取及表情识别方法
CN104794444A (zh) * 2015-04-16 2015-07-22 美国掌赢信息科技有限公司 一种即时视频中的表情识别方法和电子设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112541422A (zh) * 2020-12-08 2021-03-23 北京科技大学 光照和头部姿态鲁棒的表情识别方法、设备及存储介质
CN112541422B (zh) * 2020-12-08 2024-03-12 北京科技大学 光照和头部姿态鲁棒的表情识别方法、设备及存储介质

Also Published As

Publication number Publication date
CN104794444A (zh) 2015-07-22

Similar Documents

Publication Publication Date Title
WO2016165614A1 (fr) Procédé de reconnaissance d'expression en vidéo instantanée et équipement électronique
WO2016110199A1 (fr) Système, dispositif électronique et procédé de migration d'expression
WO2019128508A1 (fr) Procédé et appareil de traitement d'image, support de mémoire et dispositif électronique
US10559062B2 (en) Method for automatic facial impression transformation, recording medium and device for performing the method
US11163978B2 (en) Method and device for face image processing, storage medium, and electronic device
CN111476709B (zh) 人脸图像处理方法、装置及电子设备
WO2019201042A1 (fr) Procédé et dispositif de reconnaissance d'objet d'image, support de stockage et dispositif électronique
JP6330036B2 (ja) 画像処理装置及び画像表示装置
CN111783605B (zh) 一种人脸图像识别方法、装置、设备及存储介质
WO2012089902A1 (fr) Procédé, appareil et produit-programme informatique de regroupement d'images
KR20140010541A (ko) 이미지 내 사용자의 시선 보정 방법, 기계로 읽을 수 있는 저장 매체 및 통신 단말
CN111353336B (zh) 图像处理方法、装置及设备
US9747695B2 (en) System and method of tracking an object
US20180020180A1 (en) System and method for video call using augmented reality
CN114445562A (zh) 三维重建方法及装置、电子设备和存储介质
CN113380269B (zh) 视频图像生成方法、装置、设备、介质和计算机程序产品
US20160350622A1 (en) Augmented reality and object recognition device
CN115049819A (zh) 注视区域识别方法及装置
CN112714337A (zh) 视频处理方法、装置、电子设备和存储介质
CN110545386B (zh) 用于拍摄图像的方法和设备
CN117372657A (zh) 关键点旋转模型的训练方法及装置、电子设备和存储介质
CN116129526A (zh) 合拍控制方法、装置、电子设备及存储介质
CN110956576B (zh) 图像处理方法、装置、设备及存储介质
TW202046169A (zh) 電子裝置及人臉識別方法
CN110597384A (zh) 信息沟通方法及其系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16779589

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05/02/2018)

122 Ep: pct application non-entry in european phase

Ref document number: 16779589

Country of ref document: EP

Kind code of ref document: A1