WO2019184299A1

WO2019184299A1 - Microexpression recognition-based film and television scoring method, storage medium, and intelligent terminal

Info

Publication number: WO2019184299A1
Application number: PCT/CN2018/109820
Authority: WO
Inventors: 王鲜
Original assignee: 深圳创维－Rgb电子有限公司
Priority date: 2018-03-28
Filing date: 2018-10-11
Publication date: 2019-10-03
Also published as: CN108509893A

Abstract

The present disclosure relates to a microexpression recognition-based film and television scoring method, a storage medium, and an intelligent terminal. The method comprises: obtaining a microexpression of a user during watching of a film or television, recognizing and analyzing the microexpression, and outputting an analysis result; obtaining a category of the currently played film or television, and obtaining microexpression data corresponding to the category of the current film or television from a preset microexpression database; and automatically scoring the film or television according to the analysis result and the obtained microexpression data, and outputting a final score. A microexpression of a user during watching of a film or television is analyzed, and the film or television is automatically scored according to change of the microexpression, so that there is no need for the user to manually input a score, a scoring result is more real, and malicious and false scoring is effectively avoided.

Description

Film and television scoring method based on micro-expression recognition, storage medium and intelligent terminal

Technical field

The present disclosure relates to the field of consumer electronic products, for example, to a film recognition method based on micro-expression recognition, a storage medium, and a smart terminal.

Background technique

With the development of television and film and television industry, people's quality requirements for film and television dramas are getting higher and higher. Different people have different preferences for film and television dramas. Therefore, major film and television websites have also launched ratings for film and television. Features, user-friendly interaction and reference.

At present, most film and television scoring websites use a 10-point system. For example, many websites such as Douban and Time have their own algorithms to count users' scores for a certain movie. However, most websites currently use manual scoring system with manual input scores, which will make each person's scores too subjective, and even have false and malicious scores, resulting in the final film score is not true.

Therefore, the prior art has yet to be improved and developed.

Summary of the invention

The technical problem to be solved by the present disclosure is to provide a film and television scoring method, a storage medium and a smart terminal based on micro-expression recognition, aiming at solving the prior art video scores by manually inputting scores. The way, the score is too subjective, leading to problems such as untrue scores.

The technical solution adopted by the present disclosure to solve the technical problem is as follows:

A film and television scoring method based on micro-expression recognition, wherein the method comprises:

Obtaining a micro-expression when the user views the image, and identifying and analyzing the micro-expression, and outputting the analysis result;

Obtaining the category of the currently playing movie and TV drama, and obtaining the micro-expression data corresponding to the category of the current movie and television drama from the preset micro-expression database;

The movie drama is automatically scored according to the analysis result and the acquired micro-expression data, and the final score is output.

Optionally, the micro-expressions when the user is watching are obtained, and the micro-expressions are identified and analyzed, and the output analysis results specifically include:

The camera in the smart terminal acquires the face image of the user when watching the movie in real time;

Detecting the face image and marking the micro-expression image sequence;

Extracting a feature of the micro-expression image sequence, and identifying a category to which the micro-expression belongs according to the extracted feature;

Statistical analysis is performed on the types and quantities of the micro-expressions, and the analysis results are output.

Optionally, the obtaining a micro-expression when the user is watching the image, and identifying and analyzing the micro-expression, the output analysis result further includes:

The face image is preprocessed so that the pixels of the preprocessed face image are identical.

Optionally, the preprocessing the facial image specifically includes:

The Adaboost algorithm is used to detect the face in the image and perform cropping;

The bilinear difference algorithm is used to normalize the size of the cropped image.

Optionally, the micro-expression image sequence includes: a start frame, a duration, and an end frame of the appearance of the micro-expression.

Optionally, the extracting the feature of the micro-expression image sequence includes:

The dynamic spatiotemporal texture features of the micro-expression image sequence are extracted using the CBP-TOP algorithm.

Optionally, the analysis result includes: a type of micro-expression in a user watching process, and an actual proportion of each micro-expression; the types of the micro-expression include: happiness, surprise, disgust, sadness, fear, and anger.

Optionally, before the obtaining the category of the currently playing movie and TV and obtaining the micro-expression data corresponding to the category of the current movie and TV drama from the preset micro-expression database, the method further includes:

Pre-establishing a micro-expression database for acquiring corresponding micro-expression data according to the currently played movie drama category;

The micro-expression database includes: a theoretical micro-expression type corresponding to each film and television drama and a theoretical proportion of each micro-expression;

The categories of film and television dramas include: comedy, adventure, horror, crime, science fiction, martial arts, and love.

Optionally, the obtaining the category of the currently played movie and TV drama, and obtaining the micro-expression data corresponding to the category of the current movie and television drama from the preset micro-expression database specifically includes:

The smart terminal acquires the type of the currently played movie drama;

Finding corresponding micro-expression data from the micro-expression data according to the type of the film and television drama obtained;

Obtaining a theoretical micro-expression type corresponding to the current movie drama category included in the micro-expression data and a theoretical proportion of each micro-expression.

Optionally, the automatically scoring the movie drama according to the analysis result and the acquired micro-expression data, and outputting the final score specifically includes:

Compare the types of micro-expressions in the user's viewing process, and the actual proportion of each micro-expression to the theoretical micro-expressions in the acquired micro-expression data and the theoretical proportion of each micro-expression. ;

If the type of micro-expression in the process of watching the video coincides with the type of theoretical micro-expression, and the actual proportion of each micro-expression is equal to the theoretical proportion of each micro-expression, the movie drama is automatically rated as Pass score; the pass score is 6 points.

Optionally, the automatically scoring the movie drama according to the analysis result and the acquired micro-expression data, and outputting the final score further includes:

If the type of micro-expressions in the process of viewing is higher than the theoretical micro-expressions, and the actual proportion of each micro-expression is higher than the theoretical proportion of each micro-expression, the rating of the film and television drama is Increase the score on the passing score;

If the type of micro-expressions in the process of viewing is lower than the theoretical micro-expressions, and the actual proportion of each micro-expression is lower than the theoretical proportion of each micro-expression, the rating of the film and television drama is Reduce the score on the passing score.

Optionally, the method further includes:

Obtain the current user's viewing time. If the current user's viewing time is less than the preset standard duration, the user's micro-expression cannot be used to score the movie and TV drama.

Optionally, the method further includes:

The smart terminal encrypts the user's micro-expression and the user's final score.

A storage medium having stored thereon a plurality of instructions, wherein the instructions are adapted to be loaded and executed by a processor to implement the steps of the micro-expression recognition-based film and television scoring method of any of the above.

An intelligent terminal, comprising: a processor, a storage medium communicatively coupled to the processor, the storage medium being adapted to store a plurality of instructions; the processor being adapted to invoke an instruction in the storage medium to perform an implementation The step of the micro-expression recognition-based film and television scoring method according to any of the above.

The beneficial effects of the present disclosure: the present disclosure analyzes the micro-expressions when the user is watching the movie, and automatically scores the movie according to the change of the micro-expression, without the user manually inputting the score, so that the score result is more realistic, effectively avoiding malicious and false scoring Case.

DRAWINGS

1 is a flow chart of a first preferred embodiment of a method for scoring a film based on micro-expression recognition in the present disclosure.

2 is a flow chart of a micro-expression recognition method of the present disclosure.

3 is a flow chart of a second preferred embodiment of a method for scoring a film based on micro-expression recognition according to the present disclosure

4 is a functional block diagram of a smart terminal of the present disclosure.

detailed description

The present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the disclosure and are not intended to limit the disclosure.

Because the methods of scoring film and television dramas in the prior art basically adopt the method of manual manual input of scores, the scores are too subjective, and the scores of the films are not true. Although the scoring algorithms of some websites are more complicated, there are still cases of false and malicious scoring.

Example 1

The embodiment of the present disclosure provides a film and television scoring method based on micro-expression recognition. As shown in FIG. 1 , FIG. 1 is a flowchart of a first preferred embodiment of a method for scoring a film based on micro-expression recognition. The film recognition method based on micro-expression recognition comprises the following steps:

Step S100: Obtain a micro-expression when the user views the image, identify and analyze the micro-expression, and output the analysis result.

Preferably, the step S100 specifically includes:

Detecting the face image and marking the micro-expression image sequence;

In a specific implementation, the embodiment first installs a camera for capturing a user's face image on the smart terminal. Preferably, the embodiment discloses a specific flowchart of the micro-expression recognition, as shown in FIG. 2 .

Step 210: Acquire a face image. When the user is watching, the camera on the smart terminal will capture the face image of the user when watching the movie in real time.

Step 220: Pre-processing the face image. In this embodiment, an Adaboost (iterative algorithm) algorithm is used to detect a face in a micro-expression image, and the cropping is performed, and the size of the image is normalized by using a bilinear difference algorithm. The size of the pre-processed face image is 180×. 180 pixels, the image pixels after pre-processing are consistent, which is beneficial to the extraction of micro-expressions in the subsequent steps.

Step 230: Extract micro-feature features. In this embodiment, the face image is detected, and the micro-expression image sequence is marked. In this embodiment, a Birnbaum-Saunders distribution curve is used to establish a regression model to mark a sequence of facial micro-expression images, including a start frame Apex1, a duration and an end frame Apex2, which are used for facial micro-expressions, thereby completing facial micro-expression detection. . Further, in this embodiment, the dynamic spatiotemporal texture feature of the face micro-expression sequence is extracted using the CBP-TOP algorithm. The 8 frames of the face micro-expression sequence are divided into 16×16 non-overlapping blocks, the CBP-TOP feature is extracted on each sub-block, and the histogram is subjected to histogram statistics of CBP-TOP features, and finally The feature histograms of all sub-blocks are connected in series to form a histogram of the entire facial micro-expression sequence, thereby extracting dynamic spatiotemporal texture features of the facial micro-expression sequence, that is, obtaining deformation information of the facial micro-expression sequence in the XY plane, and Motion information of the XT plane and the YT plane. The CBP-TOP feature is a texture feature existing in the image itself, and the extraction process is calculated by the CBP code.

Step 240, feature judgment. This embodiment uses the ELM classifier for training and prediction, one for verifying the validity of the CBP-TOP algorithm, and the other for classifying and identifying the features of the face micro-expression sequence to determine which type of person the extracted feature belongs to. The face micro-expression recognizes the category to which the micro-expression belongs. In the above method for identifying a micro-expression, the CBP-TOP algorithm is as follows:

CBP-TOP=Σx,y,tI{fj(x,y,t)=i},i=0,...,nj-1;j=0,1,2---(1),]] >

Where: nj is the number of modes produced by the CBP operator in the jth plane, j=0,1,2 represent the XY, XT and YT planes respectively, and fj(x,y,t) represents the central pixel point (x,y, t) The decimal value of the CBP code in the jth plane, the definition of the function I(f) is as follows:

I(f)=1, if f is True0, if f is False---(2).]]>

Step 250: Output the analysis result. After identifying the type of micro-expression, the intelligent terminal will count the types and quantities of micro-expressions that appear in the process of viewing the user, and output the analysis result. The analysis results include: the types of micro-expressions in the process of viewing the user, and the actual proportion of each micro-expression. Preferably, in this embodiment, the types of micro-expressions are divided into happiness, surprise, disgust, sadness, fear, and anger. The above method for micro-expression recognition is more accurate, so that the smart terminal can accurately extract the micro-expressions from the face images, identify and classify them, and determine which category the micro-expressions belong to.

Further, in step S200, the category of the currently played movie and television drama is obtained, and the micro-expression data corresponding to the category of the current movie and television drama is obtained from the preset micro-expression database.

Preferably, the step S200 specifically includes:

The smart terminal acquires the type of the currently played movie drama;

In a specific implementation, the embodiment further establishes in the smart terminal a micro-expression database for acquiring corresponding micro-expression data according to the currently played movie drama category. The micro-expression database includes: a theoretical micro-expression type corresponding to each movie drama and a theoretical proportion of each micro-expression, that is, pre-defining the types of micro-expressions that the user will appear for each type of film and television drama, and each type The theoretical proportion of micro-expressions is used as a reference for scoring film and television dramas in subsequent steps. The categories of film and television dramas in this embodiment include: comedy, adventure, horror, crime, science fiction, martial arts, and love.

When the user is watching the movie, the smart terminal acquires the currently-played movie drama category, and searches for the theoretical micro-expression type corresponding to the category and the theoretical proportion of each micro-expression from the preset micro-expression database according to the movie drama category. . For example, if the currently-played film and television drama is a sci-fi film and television drama, the sci-fi film and television drama theory micro-expressions are both surprised and pleasant, and the surprised micro-expressions account for 40% of the theoretical proportion of the micro-expressions. The theoretical ratio is 20%.

Further, in step S300, the movie drama is automatically scored according to the analysis result and the acquired micro-expression data, and the final score is output.

Preferably, the step S300 specifically includes:

If the type of micro-expression in the process of watching the video coincides with the type of theoretical micro-expression, and the actual proportion of each micro-expression is equal to the theoretical proportion of each micro-expression, the movie drama is automatically rated as Pass score.

In a specific implementation, the present embodiment presets a standard condition, and when the type of the micro-expression and the proportion of each micro-expression in the user's viewing are in accordance with the standard condition, the film and television drama is rated as a passing score. In the present embodiment, the pass score is 6 points (the total score is ten basis). Specifically, in this embodiment, the types of micro-expressions in the process of viewing the user and the actual proportion of each micro-expression are respectively compared with the types of theoretical micro-expressions obtained from the micro-expression data and the expression of each micro-expression. The theoretical proportions are compared to determine whether the user's micro-expressions meet the conditions, and the match is scored 6 points.

For example, the theoretical micro-expression of comedy-like movies and TV shows is pleasant. If the user's pleasant micro-expression reaches 60%, the score can be 6 points. The theoretical micro-expression of the adventure film and television drama is surprised. If the user's surprised micro-expression reaches 60%. Comment 6 points; horror film and television drama theory micro-expression is fear and surprise, if the user's horrible micro-expression reaches 40% and the surprised micro-expression reaches 20%, you can score 6 points; criminal film and television drama theory micro-expression is disgust, fear And astonish, if the user's disgusting micro-expression reaches 10%, the fearful micro-expression reaches 20%, and the disgusting micro-expression reaches 30%, you can rate 6 points; the theoretical micro-expression of science fiction movies and TV dramas is surprised and pleasant, if the user is surprised The micro-expression reaches 40% and the pleasant micro-expression reaches 20% to score 6 points; the theoretical micro-expression of the martial arts drama is surprised, if the user's surprised micro-expression reaches 60%, then 6 points can be scored; The theoretical micro-expression is pleasure or sadness, and if the user's pleasant or sad micro-expression reaches 60%, he can score 6 points. The proportion of micro-expressions in each type of movie can be set according to the actual situation.

Further, if the type of the micro-expression in the process of viewing the user is higher than the theoretical micro-expression type, and the actual proportion of each micro-expression is higher than the theoretical proportion of each micro-expression, then the movie drama The score increases the score on the passing score (6 points); if the type of micro-expressions in the user's viewing process is lower than the theoretical micro-expression type, and the actual proportion of each micro-expression is lower than that of each micro-expression The theoretical proportion of the film and television drama score on the passing score (6 points) reduced scores, the final score is 0-10 points.

In order to further reduce the false and malicious scores, the embodiment also obtains the viewing time of the user when the user performs the viewing. If the viewing time of the current user is less than the preset standard duration, the user does not watch the complete movie. The user's micro-expressions cannot be used to score movies and TV shows to ensure that all users who score are valid and authentic. In addition, the embodiment also performs data encryption processing on the user's micro-expression and the user's score to ensure the security of the user data.

Example 2

Further, the present disclosure also provides a second preferred embodiment of the film recognition method based on micro-expression recognition. As shown in FIG. 3, the method includes the following steps:

Step 310: Collect a face image. The smart terminal collects the face image of the viewing user by using a preset camera.

Step 320, micro-expression recognition. The micro-feature feature is identified from the acquired face image.

Step 330: Record various micro-expression categories and proportions.

Step 340: Determine whether the movie ends. When yes, step 350 is performed; when no, step 310 is performed.

Step 350, analysis processing. Comparing the types of micro-expressions in the process of viewing the user and the actual proportion of each micro-expression with the theoretical micro-expressions obtained from the micro-expression data and the theoretical proportion of each micro-expression, thereby judging Whether the user's micro-expression is eligible and automatically scores.

Step 360: Output a movie score.

Example 3

Based on the foregoing embodiment, an embodiment of the present disclosure further provides an intelligent terminal, as shown in FIG. 4, including: a processor 10, a storage medium 20 connected to the processor 10; wherein the processing The device 10 is configured to invoke program instructions in the storage medium 20 to perform the method provided by the above embodiments, for example, to perform:

The present disclosure also provides a storage medium on which computer instructions are stored, the computer instructions causing a computer to perform the methods provided by the various embodiments described above.

All or part of the steps of the above embodiments may be completed by hardware, or may be executed by a program to execute related hardware. The program may be stored in a computer readable storage medium, and the storage medium may be a non-transitory storage medium, including a USB flash drive. A removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and the like, which can store a program code, or a transient storage medium.

Industrial applicability

The micro-expression recognition-based film and television scoring method, the storage medium and the intelligent terminal provide an analysis of the micro-expressions when the user is watching the movie, and automatically score the video according to the change of the micro-expression, without the user manually inputting the score, so that the user does not need to manually input the score. The scores are more realistic and effectively avoid malicious and false scoring.

It is to be understood that the application of the present disclosure is not limited to the above-described examples, and those skilled in the art can make modifications and changes in accordance with the above description, all of which are within the scope of the appended claims.

Claims

A film and television scoring method based on micro-expression recognition, characterized in that the method comprises:

Obtaining a micro-expression when the user views the image, and identifying and analyzing the micro-expression, and outputting the analysis result;

Obtaining the category of the currently playing movie and TV drama, and obtaining the micro-expression data corresponding to the category of the current movie and television drama from the preset micro-expression database;

The movie drama is automatically scored according to the analysis result and the acquired micro-expression data, and the final score is output.
The micro-expression recognition-based film and television scoring method according to claim 1, wherein the micro-expressions when the user views the images are acquired, and the micro-expressions are identified and analyzed, and the output analysis results specifically include:

The camera in the smart terminal acquires the face image of the user when watching the movie in real time;

Detecting the face image and marking the micro-expression image sequence;

Extracting a feature of the micro-expression image sequence, and identifying a category to which the micro-expression belongs according to the extracted feature;

Statistical analysis is performed on the types and quantities of the micro-expressions, and the analysis results are output.
The micro-expression recognition-based film and television scoring method according to claim 2, wherein the micro-expressions when the user views the images are acquired, and the micro-expressions are identified and analyzed, and the output analysis results further include:

The face image is preprocessed so that the pixels of the preprocessed face image are identical.
The method for pre-processing a face image according to the micro-expression recognition method according to claim 3, wherein the pre-processing of the face image comprises:

The Adaboost algorithm is used to detect the face in the image and perform cropping;

The bilinear difference algorithm is used to normalize the size of the cropped image.
The micro-expression recognition-based film and television scoring method according to claim 2, wherein the micro-expression image sequence comprises: a start frame, a duration, and an end frame of the appearance of the micro-expression.
The micro-expression recognition-based film and television scoring method according to claim 2, wherein the extracting the features of the micro-expression image sequence comprises:

The dynamic spatiotemporal texture features of the micro-expression image sequence are extracted using the CBP-TOP algorithm.
The micro-expression recognition-based film and television scoring method according to claim 2, wherein the analysis result comprises: a type of micro-expression in a user's viewing process, and an actual proportion of each micro-expression; The types of micro-expressions include: happiness, surprise, disgust, sadness, fear, and anger.
The method for scoring a film based on micro-expression according to claim 1, wherein the acquiring the category of the currently played movie and TV drama and obtaining the category of the current film and television drama from the preset micro-expression database Micro-expression data also includes:

Pre-establishing a micro-expression database for acquiring corresponding micro-expression data according to the currently played movie drama category;

The micro-expression database includes: a theoretical micro-expression type corresponding to each film and television drama and a theoretical proportion of each micro-expression;

The categories of film and television dramas include: comedy, adventure, horror, crime, science fiction, martial arts, and love.
The method for scoring a film based on micro-expression according to claim 1, wherein the acquiring the category of the currently played movie and TV drama and obtaining the category of the current film and television drama from the preset micro-expression database The micro-expression data specifically includes:

The smart terminal acquires the type of the currently played movie drama;

Finding corresponding micro-expression data from the micro-expression data according to the type of the film and television drama obtained;

Obtaining a theoretical micro-expression type corresponding to the current movie drama category included in the micro-expression data and a theoretical proportion of each micro-expression.
The micro-expression recognition-based film and television scoring method according to claim 1, wherein the automatically scoring the movie drama according to the analysis result and the acquired micro-expression data, and outputting the final score specifically includes:

Compare the types of micro-expressions in the user's viewing process, and the actual proportion of each micro-expression to the theoretical micro-expressions in the acquired micro-expression data and the theoretical proportion of each micro-expression. ;

If the type of micro-expression in the process of watching the video coincides with the type of theoretical micro-expression, and the actual proportion of each micro-expression is equal to the theoretical proportion of each micro-expression, the movie drama is automatically rated as Pass score; the pass score is 6 points.
The micro-expression recognition-based film and television scoring method according to claim 10, wherein the automatically scoring the movie drama according to the analysis result and the acquired micro-expression data, and outputting the final score further includes:

If the type of micro-expressions in the process of viewing is higher than the theoretical micro-expressions, and the actual proportion of each micro-expression is higher than the theoretical proportion of each micro-expression, the rating of the film and television drama is Increase the score on the passing score;

If the type of micro-expressions in the process of viewing is lower than the theoretical micro-expressions, and the actual proportion of each micro-expression is lower than the theoretical proportion of each micro-expression, the rating of the film and television drama is Reduce the score on the passing score.
The method according to claim 1, wherein the method further comprises:

Obtain the current user's viewing time. If the current user's viewing time is less than the preset standard duration, the user's micro-expression cannot be used to score the movie and TV drama.
The method according to claim 1, wherein the method further comprises:

The smart terminal encrypts the user's micro-expression and the user's final score.
A storage medium having stored thereon a plurality of instructions, wherein the instructions are adapted to be loaded and executed by a processor to implement the micro-expression recognition based on the above-described claims 1-13. The steps of the method.
An intelligent terminal, comprising: a processor, a storage medium communicatively coupled to the processor, the storage medium being adapted to store a plurality of instructions; the processor being adapted to invoke an instruction in the storage medium to The step of performing the micro-expression recognition-based film and television scoring method according to any one of claims 1 to 13 above.