CN112836679B - A Fast Expression Recognition Algorithm and System Based on Dual Model Probabilistic Optimization - Google Patents

Abstract

The present invention proposes a fast expression recognition algorithm and system based on dual model probability optimization, which mainly includes a face recognition cropping module, an image preprocessing module, a dual model prediction module and a combined probability optimization module; It is sent to the dual-model classifiers trained by the standard image and the binarized image for parallel judgment. At the same time, two optimization algorithms are proposed to optimize the combined probability of the output results of the dual-model, using two light weights with low accuracy. The probabilistic optimization of the recognition results of the neural network model achieves high-accuracy judgment and recognition. The invention can realize that the recognition rate of pain that lasts for a threshold time and above is not less than 99%, and greatly reduces the cost of calculation and storage, and can also effectively recognize other kinds of expressions.

Description

A Fast Expression Recognition Algorithm and System Based on Dual Model Probabilistic Optimization

technical field

The invention belongs to the field of image processing and emotion recognition, in particular to a fast expression recognition algorithm and system based on double-model probability optimization.

Background technique

With the development of society, people are paying more and more attention to acute diseases. Due to the rapid onset of acute diseases, severe symptoms and rapid changes, it is difficult to effectively prevent acute diseases and the onset of acute diseases is often accompanied by severe pain, which causes the disease to suffer. Unable to call for help, unable to receive timely treatment, the patient's life is greatly threatened. At present, for the identification of acute disease, personnel escort, video surveillance or installation of specific sensing devices on the patient's body are commonly used, which wastes human resources and brings inconvenience to the patient's movement. Although the existing neural network has a very high The accuracy rate is high, but the storage and computing costs are high, which has brought limitations to the promotion of special monitoring intelligent terminals.

Expressions are usually divided into 6 general expressions: happy, sad, fearful, angry, disgusted and surprised, the face and facial expressions represent the thinking behavior of the individual, the facial expressions convey 55% of the information, while only 38% and 7% of the information It is related to language and sound, so even though pain is not conceptualized as an emotion, it can be expressed through expressions. Traditional expression recognition algorithms are usually divided into two categories: one is static image classification, which has faster computing speed, but cannot capture the spatiotemporal features of facial expressions; the other is dynamic video classification, which can effectively capture the spatiotemporal characteristics of facial expressions. features, but the calculation is more complex. Most of them focus on changing the network structure and design mode. The accuracy of the model is very high, but the computational cost and storage cost are high, and it is not suitable for the deployment of intelligent terminals with low computing power.

CN111466878A discloses a real-time monitoring method and device for pain symptoms of bedridden patients based on facial expression recognition, the method includes: 1 establishing a pain expression training data set; 2 establishing and training a neural network model for analyzing pain expressions, so as to obtain a pain classification model; 3. Collect three real-time images of the face at the same time and input them into the neural network model after preprocessing, so as to obtain the probability corresponding to A pain level; and select the pain level corresponding to the maximum probability as the current moment detection image The pain level is higher than the threshold, and alarm processing is performed for the pain level exceeding the threshold to realize real-time monitoring. The invention can perform real-time and automatic accurate assessment of pain, so as to realize effective monitoring of bedridden patients.

CN106682616B discloses a neonatal pain expression recognition method based on dual-channel feature deep learning. The method firstly grayscales the newborn's facial image to extract the Local Binary Pattern (LBP) feature map; The features of the two channels of the LBP feature map are used for deep learning; finally, the softmax-based classifier is used to classify the expressions of the fusion features of the two channels, which are divided into four categories: calm, crying, mild pain, and severe pain. The method combines the feature information of two channels of gray image and its LBP feature map, which can effectively identify expressions such as calm, crying, mild pain, severe pain, etc. It has good robustness and provides a new method and approach for the development of a neonatal pain expression recognition system, but the computational cost is high, which is not conducive to the deployment of smart terminals.

CN202011036047.5 discloses an intelligent recognition method for the painful expression of the elderly on a nursing bed, which extracts painful expression features from face images in combination with a deep belief network. Problems are identified through semi-supervised learning methods by combining labeled and unlabeled samples, using generative models. The invention focuses on adjusting the structure and parameters of the neural network, which can effectively identify painful expressions, but the accuracy cannot be adjusted once the neural network is trained, which is due to the uncertainty in the training process of the neural network.

In order to solve the identification problem of acute illness, the present invention proposes an acute illness identification algorithm that can identify pain lasting a threshold time and above according to the patient's facial expression, and can effectively ignore occasional pain.

SUMMARY OF THE INVENTION

In order to solve the above problem, the present invention proposes an algorithm based on two simple models trained by static pictures, through a combined probability optimization algorithm to continuously judge the expression action peak value. It can quickly and accurately identify persistent pain for a set threshold time and above, and can ignore brief episodic pain.

In order to solve the above-mentioned technical problems, the present invention proposes a fast expression recognition algorithm based on the probability optimization of the dual models, which is mainly based on the combined probability optimization of the output results of the dual models to realize the recognition of facial expressions, and specifically includes the following technical solutions:

A fast expression recognition algorithm based on dual-model probability optimization, which specifically includes the following steps:

Step 1: Take the frame of the image sent by the camera, and cut out the face image as a standard image;

Through the library function provided by opencv, the video frame is sent to the Haar face cascader, the face in the image is retrieved through the classifier, and the face is cropped into a standard image to prepare for image preprocessing;

Step 2: Binarization preprocessing is performed on the standard image, and median filtering is performed to reduce the interference of invalid features to obtain a binary image;

The standard image is grayed and binarized, and at the same time, in order to reduce the influence of irrelevant features such as whiskers and spots, the image is denoised by the median filter algorithm;

Step 3: The standard image and the binary image are respectively sent to the Mini_Xception model and the CNN7 model for parallel judgment, and the facial expression category in the image is recognized;

The CNN7 model and the Mini_Xception model are used to judge the images separately. The facial expression categories are divided into two categories: normal and painful. During the judgment process, the pain is recorded as 1, and the normal as 0;

Step 4 If the output results of the two models are all painful, that is, when the judgment results of the CNN7 model and the Mini_Xception model are equal and both are 1, activate the optimization algorithm, go to Step 5, perform combined probability optimization and output the final result, otherwise go to Step 5 1;

Step 5: The counter sum is set to 1, and the following L images are counted and accumulated according to the optimization algorithm;

When the counter sum is set to 1, the judgment results of the subsequent L images are recorded. When using the optimization algorithm 1 to work, each time the double model is judged to be painful, the counter sum is increased by 1; when the optimization algorithm 2 is used to work , add W to sum when only model 1 is judged to be painful, and add 1-W to sum when only model 2 judges to be painful;

Step 6: When sum is accumulated to K or greater than K in L images, go to Step 7, otherwise go to Step 1;

When using optimization algorithm 1, when the counter sum is equal to the threshold K, enter the next step 7; when using the optimization algorithm 2, when the counter sum is greater than or equal to K, enter the next step 7;

Step 7 outputs an alarm to complete the judgment on the facial expression; go to step 1;

When the output conditions are met, the program will print Warning cyclically until manual mediation.

Further, the CNN7 model is mainly composed of two parts. The first part is composed of 7 groups of convolution modules, using a two-dimensional convolution layer (Conv2D layer), a Dropout layer and a maximum pooling layer (MaxPooling layer). The convolution modules all use Relu as the activation function. Relu can reduce the computational cost of the neural network and improve the efficiency of gradient descent and backpropagation; the second part consists of a fully connected layer, and the first fully connected layer has 128 neurons. The activation function is Relu, the second fully connected layer consists of 2 neurons, and the activation function is softmax to play a classification role.

The Mini_Xception mainly consists of three parts. The first part is two Conv2D layers. To reduce over-fitting, batch normalization layers (BN) are used and Relu is used as the activation function; the second part is composed of five dual-channel modules, of which The left channel consists of a Conv2D layer and a BN layer, the right channel consists of two modules, the first module consists of a depthwise separable convolutional layer, a BN layer and a Relu layer, and the second module consists of a depthwise separable convolutional layer It is composed of the BN layer. In order to reduce the amount of calculation, the MaxPooling layer is connected. Finally, the add layer is used to merge the two channels together and send it to the next module. The third part is composed of the Conv2D layer, the Dropout layer and the global average pooling layer. Finally, use The softmax activation function to classify the output features.

Further, the probabilistic optimization uses the mathematical model of the optimization algorithm to combine the output results of the dual models, and makes an overall judgment on the expression action. Therefore, two probability optimization algorithms are proposed in the present invention, both of which can solve the problem. More preferably, the present invention selects the second optimization algorithm, and the second optimization algorithm is an improvement of the first optimization algorithm, which can better coordinate the relationship between L and K, so as to reduce the false alarm interval as much as possible while ensuring the accuracy. False alarm rate.

Specifically, the first optimization algorithm is:

When the prediction results of the dual models are all pain, the counter sum = 1, and the following L images are taken. Every time the dual model predicts pain, the counter sum increases by 1. When the sum is equal to K, an interrupt is generated and an alarm is triggered. The formula for:

Among them, P is the total success rate of the algorithm, P ₀₀ is the probability that the dual models are simultaneously judged to be pain, K is the set threshold, L means that after the dual models are simultaneously judged to be pain, the subsequent L images are taken for statistics, P _start It represents the probability that when the pain sequence appears, one of the first n photos appears to be pain in the judgment of the dual model, and its formula is:

Specifically, the second optimization algorithm is:

Let the weight of model one be W, and the weight of model two to be 1-W. When the prediction results of both models are pain, the counter sum=1, and the following L images are taken. Each time the dual model predicts pain, The counter sum is incremented by 1. When model 1 judges pain and model 2 judges normal, sum adds W. When model 2 judges pain and model 1 judges normal, sum adds 1-W. When sum is greater than or equal to K, An interrupt is generated and an alarm is triggered; among them, K is the set threshold, and L means that after the dual models are simultaneously judged as pain, the subsequent L images are taken for statistics;

Assuming that 0≤a≤L and 0≤b≤L exist such that the formula aW+b(1-W)<K, the algorithm success rate formula is:

Among them, P ₁ represents the probability of correct judgment of model 1, P ₂ represents the probability of correct judgment of model 2, a and b respectively represent the number of correctly judged images, * represents all possible sets of (a, b); obviously in the interval [0, b The existence of real numbers a and b in L] makes the inequality aW+b(1-W)<K hold.

The purpose of the present invention is also to provide a fast expression recognition system based on dual-model probability optimization, which mainly includes four modules,

First, the face recognition cropping module, which crops the face from the image output by the camera as a standard image;

Second, the image preprocessing module, which binarizes the standard image and performs median filtering to reduce the interference of invalid features to obtain a binary image;

Third, the dual-model prediction module sends the standard image and the binary image to the Mini_Xception model and the CNN7 model for parallel judgment, and recognizes the facial expression category in the image; the facial expression category is divided into two categories: normal and painful, and in the judgment process The pain is recorded as 1 in the middle, and the normal value is 0, and the execution of the next step is determined by comparing the results;

Fourth, combine the probability optimization module to perform probability optimization on the output results of the Mini_Xception model and the CNN7 model and output the final results; the specific process is: determine whether the output results of the dual models are painful, if so, set the counter sum to 1, and then The L images obtained are counted and accumulated according to the optimization algorithm; when the sum of the L images is accumulated to the threshold K or greater than K, an alarm is output to complete the judgment of the expression action; otherwise, go to module one.

In order to realize the automatic identification of acute disease and reduce the cost of calculation and storage, the present invention proposes an acute disease recognition algorithm and system based on the peak value of the patient's painful expression and action. The dual-model classifiers trained on standard images and binary images make parallel judgments, and at the same time, two optimization algorithms are proposed to optimize the combined probability of the output results of the dual models, using two lightweight neural network models with low accuracy to identify the probability of the results The optimization realizes high-accuracy judgment and recognition. The invention can realize that the recognition rate of pain that lasts for a threshold time and above is not less than 99%, and greatly reduces the cost of calculation and storage, and can also effectively recognize other kinds of expressions.

Compared with the prior art, the present invention has the following beneficial effects and progress:

On the basis of the judgment of the neural network, the invention optimizes the combination of the output results of the neural network based on the optimization algorithm, and realizes the high-accuracy recognition of various persistent facial expressions. On the one hand, the present invention uses a neural network model with a simple structure and a simple algorithm, which effectively reduces the calculation cost and storage cost, and the calculation speed is relatively fast; Uncertainty and better controllability. At the same time, the invention has high application value in the fields of acute disease alarm, social emotion recognition and the like.

Description of drawings

Fig. 1 is the structural flow chart of the identification method of the present invention;

Figure 2 is a schematic diagram of the structure of CNN7;

Figure 3 is a schematic diagram of the Mini_Xception structure;

Figure 4 is a schematic diagram of the probabilistic optimization algorithm idea.

Detailed ways

The specific embodiments of the present invention will be described in further detail below in conjunction with the accompanying drawings:

This embodiment provides a fast expression recognition system based on dual-model probability optimization, which mainly includes four modules, as shown in FIG. 1 . First, the face recognition cropping module cuts the face from the image output by the camera as a standard image; second, the image preprocessing module binarizes the standard image, and is in the process of reducing the interference of invalid features. Value filtering to obtain a binary image; third, a dual-model prediction module to identify the facial expression categories in the image; fourth, a combined probability optimization module to perform probability optimization on the output results of the dual models and output the final result.

Before using the probability optimization algorithm to make an overall judgment on the model output results, a classification model needs to be used to classify the facial expressions. In this embodiment, the standard image and the binary image are respectively sent to the Mini_Xception model and the CNN7 model for parallel judgment to identify the person in the image. Facial expression category; the facial expression category is divided into two categories: normal and painful. In the judgment process, the pain is recorded as 1, and the normal is 0, and the execution of the next step is determined by comparing the results;

(1)CNN7

CNN7 is mainly composed of two parts. The first part is composed of 7 groups of convolution modules, using two-dimensional convolution layer (Conv2D layer), Dropout layer and maximum pooling layer (MaxPooling layer). These 7 groups of convolution modules all use Relu As an activation function, Relu can reduce the computational cost of neural networks and improve the efficiency of gradient descent and backpropagation. The second part consists of a fully connected layer, the first fully connected layer has 128 neurons, the activation function is Relu, the second fully connected layer consists of 2 neurons, and the activation function is softmax for classification. Its network structure is shown in Figure 2.

(2)Mini_Xception

Mini_Xception mainly consists of 3 parts. The first part consists of two Conv2D layers. In order to reduce over-fitting, batch normalization layer (BN) is used and Relu is used as the activation function. The second part consists of five dual-channel modules, of which the left channel is used. It consists of a Conv2D layer and a BN layer, the right channel consists of two modules, the first module consists of a depthwise separable convolutional layer, a BN layer, and a Relu layer, and the second module consists of a depthwise separable convolutional layer, BN In order to reduce the amount of calculation, the MaxPooling layer is connected, and finally the add layer is used to merge the two channels together and send it to the next module. The third part consists of the Conv2D layer, the Dropout layer, the global average pooling layer, and finally uses the softmax activation. function to classify the output features, and its structure is shown in Figure 3.

Based on the judgment accuracy of the two models, a mathematical model of the probabilistic optimization algorithm is established, the output results of the models are combined, and the overall expression of the expression is judged. The schematic diagram of the algorithm idea is shown in Figure 4;

(1) Optimization algorithm one

When both models predict pain, the counter sum = 1, and the following L images are taken. Every time the double model predicts pain, the counter sum increases by 1. When the sum is equal to K, an interrupt is generated and an alarm is triggered. Its formula is:

Among them, P is the total success rate of the algorithm, P ₀₀ is the probability that the dual models are simultaneously judged to be pain, K is the set threshold, L means that after the dual models are simultaneously judged to be pain, the subsequent L images are taken for statistics, P _start It represents the probability that when the pain sequence appears, one of the first n photos appears to be pain in the judgment of the dual model, and its formula is:

(2) Optimization algorithm two

Let the weight of model one be W, and the weight of model two to be 1-W. When the prediction results of both models are pain, the counter sum=1, and the following L images are taken. Each time the dual model predicts pain, The counter sum is incremented by 1. When model 1 judges pain and model 2 judges normal, sum adds W. When model 2 judges pain and model 1 judges normal, sum adds 1-W. When sum is greater than or equal to K, Generate an interrupt and trigger an alarm. Among them, K is the set threshold, and L indicates that after the dual models are simultaneously judged to be pain, the following L photos are taken for statistics.

Assuming that 0≤a≤L and 0≤b≤L exist such that the formula aW+b(1-W)<K, the algorithm success rate formula is:

Among them, P ₁ represents the probability of correct judgment of model 1, P ₂ represents the probability of correct judgment of model 2, a and b respectively represent the number of correctly judged images, and * represents all possible sets of (a, b). Obviously, there are real numbers a, b in the interval [0, L] so that the inequality aW+b(1-W)<K holds.

A fast expression recognition algorithm based on dual-model probability optimization provided by the present invention will be described below through two specific embodiments. During the experiment, 100 pain samples were input into the system to simulate pain lasting 4 seconds to test whether the system could realize an alarm. Each algorithm was tested for 2000 rounds to obtain data.

Example 1

A fast expression recognition algorithm based on dual-model probability optimization,

Step 1: Take the frame of the image sent by the camera, and cut out the face image as a standard image;

Through the library function provided by opencv, the video frame is sent to the Haar face cascader, the face in the image is retrieved through the classifier, and the face is cropped into a standard image of (224, 224) to prepare for image preprocessing;

Step 2: Binarization preprocessing is performed on the standard image, and median filtering is performed to reduce the interference of invalid features to obtain a binary image;

The standard image is grayed and binarized, and at the same time, in order to reduce the influence of irrelevant features such as whiskers and spots, the image is denoised by the median filter algorithm;

Step 3: The standard image and the binary image are respectively sent to the Mini_Xception model and the CNN7 model for parallel judgment, and the facial expression category in the image is recognized;

Use the CNN7 and Mini_Xception models to judge the images separately. The image categories are divided into two categories: normal and painful. During the judgment process, the pain is recorded as 1, and the normal is 0. The execution of the next step is determined by comparing the results;

Step 4 If the output results of the two models are all painful, that is, when the judgment results of the CNN7 model and the Mini_Xception model are equal and both are 1, activate the optimization algorithm 1, go to Step 5, perform combined probability optimization and output the final result, otherwise go to step 1;

Step 5: The counter sum is set to 1, and the following L images are counted, and accumulated according to the optimization algorithm 1, and the counter sum is increased by 1 every time the double model is judged to be painful;

Step 6: When sum is accumulated to K in L images, go to Step 7, otherwise go to Step 1;

Step 7 generates an interrupt and outputs an alarm, and the program will print Warning cyclically until manual mediation completes the judgment on the facial expression; go to step 1.

The experimental results of Example 1 are shown in Table 1.

Table 1 Algorithm-experimental results

Acc represents the overall recognition accuracy of facial expressions.

Example 2

A fast expression recognition algorithm based on dual-model probability optimization,

Step 1: Take the frame of the image sent by the camera, and cut out the face image as a standard image;

Through the library function provided by opencv, the video frame is sent to the Haar face cascader, the face in the image is retrieved through the classifier, and the face is cropped into a standard image of (224, 224) to prepare for image preprocessing;

Step 2: Binarization preprocessing is performed on the standard image, and median filtering is performed to reduce the interference of invalid features to obtain a binary image;

The standard image is grayed and binarized, and at the same time, in order to reduce the influence of irrelevant features such as whiskers and spots, the image is denoised by the median filter algorithm;

Step 3: The standard image and the binary image are respectively sent to the Mini_Xception model and the CNN7 model for parallel judgment, and the facial expression category in the image is recognized;

Use the CNN7 and Mini_Xception models to judge the images separately. The image categories are divided into two categories: normal and painful. During the judgment process, the pain is recorded as 1, and the normal is 0. The execution of the next step is determined by comparing the results;

Step 4 If the output results of the two models are all painful, that is, when the judgment results of the CNN7 model and the Mini_Xception model are equal and both are 1, activate the optimization algorithm 1, go to Step 5, perform combined probability optimization and output the final result, otherwise go to step 1;

Step 5: The counter sum is set to 1, and the following L images are counted, and accumulated according to the optimization algorithm 2;

Step 6: When sum is accumulated to be greater than or equal to K in L images, go to Step 7, otherwise go to Step 1;

Step 7 generates an interrupt and outputs an alarm, and the program will print Warning cyclically until manual mediation completes the judgment on the facial expression; go to step 1.

The experimental results of Example 2 are shown in Table 2.

Table 2 The experimental results of the second algorithm

Claims (6)

1. A quick expression recognition method based on double-model probability optimization is characterized by comprising the following steps:

step 1, sending an image into a camera to take a frame, and cutting out a face image as a standard image;

step 2, carrying out binarization pretreatment on the standard image, and simultaneously carrying out median filtering for reducing interference of invalid features to obtain a binary image;

step 3, respectively sending the standard image and the binary image into a Mini _ Xcenter model and a CNN7 model for parallel judgment, and identifying the facial expression category in the image; the facial expression categories are divided into normal and pain categories, and the pain is recorded as 1 and the normal is recorded as 0 in the judging process;

step 4, if the output results of the dual models are all painful, namely the judgment results of the CNN7 model and the Mini _ Xcenter model are equal and are both 1, activating an optimization algorithm, turning to step 5, performing combined probability optimization and outputting a final result, and turning to step 1 otherwise;

step 5, setting a counter sum to be 1, counting the L images, and performing sum accumulation according to an optimization algorithm;

step 6, when sum in the L images is accumulated to K or is larger than K, turning to step 7, otherwise, turning to step 1;

step 7, outputting an alarm to finish judging the expression action; turning to the step 1;

in the step 4, the probability optimization uses a mathematical model of an optimization algorithm to combine output results of the dual models, and judges the overall expression and action;

the optimization algorithm is selected from the following optimization algorithm I or optimization algorithm II:

(1) optimization algorithm one

When the dual-model prediction results are all painful, the counter sum is 1, the following L images are taken, the dual-model prediction is painful every time, the counter sum is added with 1, when sum is equal to K, interruption is generated and an alarm is triggered, and the formula is as follows:

wherein P is the algorithm assembly power, P₀₀Expressing the probability of pain caused by simultaneous judgment of the double models, K expressing a set threshold value, L expressing that the pain caused by simultaneous judgment of the double models occurs, taking the subsequent L pictures for statistics, and P_startWhen the pain sequence is listed, the probability that one of the previous n photos is suffering from dual-mode judgment is shown, and the formula is as follows:

(2) optimization algorithm two

Setting a CNN7 model as a first model, wherein the weight occupied by the CNN7 model is W, a Mini _ Xconcept model is a second model, the weight occupied by the Mini _ Xconcept model is 1-W, when the dual-model prediction results are all painful, a counter sum is 1, taking subsequent L images, and when the dual-model prediction is painful once, the counter sum is added with 1, when the first model judges that the pain is painful, the second model judges that the pain is normal, the counter sum is added with W, when the second model judges that the pain is painful, the first model judges that the pain is normal, the counter sum is added with 1-W, and when the sum is more than or equal to K, an interrupt is generated and an alarm is triggered; wherein, K is a set threshold value, L represents that a double model appears and is judged as pain at the same time, and then L pictures are taken for statistics;

if a is more than or equal to 0 and less than or equal to L and b is more than or equal to 0 and less than or equal to L, so that the formula aW + b (1-W) < K, the algorithm assembly power formula is as follows:

wherein P is₁Representing the probability that the model one judges correctly, P₂Representing the probability that the model II judges correctly, a and b respectively represent the number of images which judge correctly, and representing all possible sets of (a and b); apparently in the interval [0, L]The presence of real numbers a, b makes the inequality aW + b (1-W) < K true.

2. The method for recognizing the rapid expressions based on the dual-model probability optimization of the facial expression recognition system as claimed in claim 1, wherein the CNN7 model is mainly composed of two parts, the first part is composed of 7 sets of convolution modules, the Conv2D layer, the Dropout layer and the MaxPholing layer are used, and the 7 sets of convolution modules all use Relu as the activation function; the second part consists of a fully-connected layer, the first layer of the fully-connected layer has 128 neurons, the activation function is Relu, the second layer of the fully-connected layer consists of 2 neurons, and the activation function is softmax and plays a classification role.

3. The method for identifying rapid expressions based on bimodal probability optimization according to claim 1, wherein the Mini _ Xconcept consists of 3 parts, the first part is two Conv2D layers, BN is used for reducing overfitting, and Relu is used as an activation function; the second part consists of five dual-channel modules, wherein the left channel consists of a Conv2D layer and a BN layer, the right channel consists of two modules, the first module consists of a depth separable convolutional layer, a BN layer and a Relu layer, the second module consists of a depth separable convolutional layer and a BN layer, and a Max bonding layer is connected for reducing the calculation amount, and finally, the add layer is used for combining the two channels together and sending the two channels into the next module; the third part consists of a Conv2D layer, a Dropout layer and a global average pooling layer, and finally output features are classified by using a softmax activation function.

4. A fast expression recognition system based on double model probability optimization is characterized by mainly comprising four modules,

firstly, a face recognition cutting module cuts a face from an image output by a camera to be used as a standard image;

secondly, the image preprocessing module is used for carrying out binarization on the standard image and carrying out median filtering for reducing interference of invalid features to obtain a binary image;

thirdly, the dual-model prediction module is used for respectively sending the standard image and the binary image into a Mini _ Xcenter model and a CNN7 model for parallel judgment and identifying the facial expression category in the image; the facial expression categories are divided into normal and painful categories, the pain is recorded as 1 in the judging process, the pain is recorded as 0 in the normal condition, and the execution of the next step is determined by comparing results;

fourthly, a combined probability optimization module carries out probability optimization on output results of the Mini _ Xcenter model and the CNN7 model and outputs a final result; the specific process is as follows: judging whether the output results of the double models are painful, if so, setting a counter sum to be 1, counting the L images, and performing sum accumulation according to an optimization algorithm; when sum in the L images is accumulated to a threshold value K or is larger than K, outputting an alarm to finish the judgment of the expression action; otherwise, turning to a face recognition cutting module;

in the combined probability optimization module, probability optimization uses a mathematical model of an optimization algorithm to combine output results of the double models and judge the overall expression and action;

the optimization algorithm is selected from the following optimization algorithm I or optimization algorithm II:

(1) optimization algorithm one

When the dual-model prediction results are all painful, the counter sum is 1, the following L images are taken, the dual-model prediction is painful every time, the counter sum is added with 1, when sum is equal to K, interruption is generated and an alarm is triggered, and the formula is as follows:

wherein P is the algorithm assembly power, P₀₀Expressing the probability of pain caused by simultaneous judgment of the double models, K expressing a set threshold value, L expressing that the pain caused by simultaneous judgment of the double models occurs, taking the subsequent L pictures for statistics, and P_startWhen the pain sequence is listed, the probability that one of the previous n photos is suffering from dual-mode judgment is shown, and the formula is as follows:

(2) optimization algorithm two

Setting a CNN7 model as a first model, wherein the weight of the CNN7 model is W, a Mini _ Xception model is a second model, the weight of the Mini _ Xception model is 1-W, when the results of the double-model prediction are all painful, a counter sum is 1, taking subsequent L images, when the double-model prediction is painful every time, adding 1 to the counter sum, when the first model judges that the model is painful, and the second model judges that the model is normal, adding W to the counter sum, when the second model judges that the model is painful, and the first model judges that the model is normal, adding 1-W to the counter sum, and when the sum is more than or equal to K, generating interruption and triggering an alarm; wherein, K is a set threshold value, L represents that a double model appears and is judged as pain at the same time, and then L pictures are taken for statistics;

if a is more than or equal to 0 and less than or equal to L and b is more than or equal to 0 and less than or equal to L, so that the formula aW + b (1-W) < K, the algorithm assembly power formula is as follows:

wherein P is₁Representing the probability that the model one judges correctly, P₂Representing the probability that the model II judges correctly, a and b respectively represent the number of images which judge correctly, and representing all possible sets of (a and b); apparently in the interval [0, L]The presence of real numbers a, b makes the inequality aW + b (1-W) < K true.

5. The system for fast expression recognition based on dual-model probability optimization as claimed in claim 4, wherein the CNN7 model is mainly composed of two parts, the first part is composed of 7 sets of convolution modules, wherein Conv2D layer, Dropout layer and MaxPholing layer are used, and the 7 sets of convolution modules all use Relu as activation function; the second part consists of a fully-connected layer, the first layer of the fully-connected layer has 128 neurons, the activation function is Relu, the second layer of the fully-connected layer consists of 2 neurons, and the activation function is softmax and plays a classification role.

6. The system of claim 4, wherein the Mini _ Xconcept consists essentially of 3 parts, the first part is two Conv2D layers, BN is used for reducing overfitting, and Relu is used as an activation function; the second part consists of five dual-channel modules, wherein the left channel consists of a Conv2D layer and a BN layer, the right channel consists of two modules, the first module consists of a depth separable convolutional layer, a BN layer and a Relu layer, the second module consists of a depth separable convolutional layer and a BN layer, and a Max bonding layer is connected for reducing the calculation amount, and finally, the add layer is used for combining the two channels together and sending the two channels into the next module; the third part consists of a Conv2D layer, a Dropout layer and a global average pooling layer, and finally output features are classified by using a softmax activation function.

Images