CN111553299A - Method for extracting student face information to realize educational psychology analysis based on image big data - Google Patents

Method for extracting student face information to realize educational psychology analysis based on image big data Download PDF

Info

Publication number
CN111553299A
CN111553299A CN202010375024.0A CN202010375024A CN111553299A CN 111553299 A CN111553299 A CN 111553299A CN 202010375024 A CN202010375024 A CN 202010375024A CN 111553299 A CN111553299 A CN 111553299A
Authority
CN
China
Prior art keywords
image
data
layer
information
students
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010375024.0A
Other languages
Chinese (zh)
Inventor
刀锋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN202010375024.0A priority Critical patent/CN111553299A/en
Publication of CN111553299A publication Critical patent/CN111553299A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/24323—Tree-organised classifiers
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/253—Fusion techniques of extracted features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/24—Aligning, centring, orientation detection or correction of the image
    • G06V10/243—Aligning, centring, orientation detection or correction of the image by compensating for image skew or non-uniform image deformations
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/172—Classification, e.g. identification
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • General Health & Medical Sciences (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Molecular Biology (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a method for extracting facial information of students to realize educational psychology analysis based on image big data, which obtains image characteristics through a deep learning algorithm to further realize analysis and identification of data, improves the accuracy of reading facial expressions of the students, is beneficial for teachers to read the facial expressions of the students and obtain psychological changes of the students so as to give psychological guidance. The method comprises the steps of preprocessing an image to obtain pure data information; the image characteristics are extracted, the analysis of the image characteristic information is realized by extracting the local information and the global texture image characteristics from the local information extraction and the global information extraction, the image information is fused, the student expression and emotion are deeply learned, and the emotion change condition reflected by the image is obtained. The invention can enable the psychological teachers to actively learn and understand the expression and emotional state changes of students, help the teachers accurately and comprehensively master the overall situation of class students and promote the improvement of classroom teaching quality.

Description

Method for extracting student face information to realize educational psychology analysis based on image big data
Technical Field
The invention relates to the field of psychological education, in particular to a method for extracting face information of students to realize educational psychological analysis based on large image data.
Background
The emotion is one of the important research objects of psychological research, and is also the research field which is most closely combined with big data and most abundant in results so far. The facial expression recognition is to perform feature extraction work on expression images or videos generated by pulling muscles on a face through a computer, implement expression classification and expression recognition according to the current understanding experience and thought recognition of human beings, and extract and analyze human emotions from facial information. The correctness and the usefulness of the expression feature extraction are the key points of whether the expression can be correctly recognized. In the psychology teaching, the facial expressions of students directly reflect the psychological states of the students, and the internal psychological world of the students can be effectively read by reading the facial features of the students under different conditions so as to give correct psychological counseling.
In the conventional psychological research, in terms of the emotional fluctuation of students, the research is carried out through fluctuation rhythms on a daily cycle level, particularly the research mainly carried out around positive emotions and negative emotions, and through a large amount of data, the research and research are carried out in a laboratory, so that the emotions of the students are tested in a fluctuation mode, and although the emotions of the students are greatly deviated in the case that all links are done in place. Therefore, the current method for recognizing facial expressions of students by emotion still seems to be unconscious in assisting teachers in performing educational psychological analysis.
Disclosure of Invention
Aiming at the defects of the prior art, the invention discloses a method for extracting student face information based on image big data to realize educational psychology analysis, which obtains image characteristics through a deep learning algorithm to further realize data analysis and identification, improves the accuracy of reading student face expressions, is beneficial for teachers to read student expressions and obtain student psychological changes to give psychological guidance.
The invention adopts the following technical scheme:
a method for extracting student face information to realize educational psychology analysis based on image big data comprises the following steps:
(S1) constructing a student image database set to acquire student face image information;
(S2) image preprocessing; performing tilt correction, size normalization, histogram equalization and gray level normalization processing on a face picture in a database, cleaning a blurred picture, and acquiring pure image data information;
(S3) image feature extraction; inputting the preprocessed image data into a Convolutional Neural Network (CNN) data model for depth feature extraction; the extraction method comprises the steps of local image feature extraction, global image feature extraction and global texture image feature extraction; the CNN network data model comprises an input layer, a convolutional layer, a sub-sampling layer, a full connection layer and an output layer; the input layer is used for inputting picture information data, the convolutional layer is formed by arranging neurons into a characteristic plane, the number of characteristic surfaces in the sub-sampling layer is the same as that of the characteristic surfaces in the convolutional layer, the characteristic surfaces are arranged in a one-to-one corresponding relation so as to reduce characteristic dimensionality, the full connection layer is arranged to collect local characteristics of the convolutional layer and the sampling layer, and the collected data information result is output;
(S4) image data information fusion; respectively normalizing the three extracted features, and then performing cascade fusion; the image cascade fusion method is a multi-data characteristic association algorithm based on a fuzzy neural network; the multi-data feature association algorithm model structure comprises an input layer, a fuzzy layer, a rule reasoning layer and an inverse fuzzy layer; wherein:
the input layer is used for inputting variables, the fuzzification layer is used for realizing fuzzification of the input variables, the fuzzification degree is determined through a membership degree, and the membership degree is expressed by the following formula:
Figure BDA0002477512590000031
wherein, aijAnd bijRespectively representing the center and the width of the membership function; mu (x)i) Representing the degree of membership of the jth fuzzy subset of the ith input variable;
the rule reasoning layer represents fuzzy operation through different set nodes, one node represents a fuzzy rule, and the number of the nodes is between 2 and 300;
the output layer is subjected to defuzzification processing through a gravity center method, and output information is expressed by the following formula:
Figure BDA0002477512590000032
where m denotes the number of rules, wiIntensity, r, of the ith rule representing the rule heap layer outputiA conclusion of the ith rule is shown;
(S5) classifying the data after the image data information fusion; inputting the fused features into a trained strong classifier for training and classification; the strong classifier is based on a random forest algorithm model, and the training process of the strong classifier is as follows:
randomly sampling a fixed number of student image samples from a sample training set after image preprocessing, putting back one sample when acquiring one sample, and then re-sampling, wherein if the random sampling is performed for T times on N sample training sets, the results of the T times of sampling are different due to the randomness of the sampling, the data with the most frequency is output as the result of each time, and the data is used as a final data model, and the current node of the point is set as a leaf node; in the original sample training data set, assume the input as sample set D { (x)1,y1),(x2,y2),...,(xm,ym) }; the first output is output through a weak learner algorithm, namely weak classifiers are iterated for T times, and the weak classifiers are superposed and output as a final strong classifier;
(S6) outputting the expression data information; for the teacher to judge to recognize the corresponding psychological state of the student's facial information.
As a further aspect of the present invention, the method for normalizing the gray scale conversion in the step (S2) includes: expanding the gray distribution in the original image to an image with the whole gray level by using a gray stretching method, wherein the formula of the gray transformation normalization is as follows:
Figure BDA0002477512590000041
wherein I (I, j) and N (I, j) respectively represent the gray scale value of the original image and the gray scale value of the transformed image, and min and max respectively represent the minimum gray scale value and the maximum gray scale value of the original image.
As a further technical solution of the present invention, the maximum value method in the step (S2) is to use the maximum value of the three-component luminance in the color image as the gray value of the gray scale map, and the three components include an R component, a G component and a B component.
As a further technical solution of the present invention, the average value method in the step (S2) is to convert the images collected at different times and under different illumination into standard images with the same mean and variance of gray scale, and the calculation formula is
Figure BDA0002477512590000051
Figure BDA0002477512590000052
Wherein I (I, j), M, V respectively represent the gray value, mean, variance, N (I, j), M of the image before normalization0、V0Respectively representing the gray value, the mean value and the variance of the normalized image.
As a further technical solution of the present invention, in the weighted average method in step (S2), three components including an R component, a G component, and a B component are weighted and averaged by different weights, and a calculation formula is: f (i, j) ═ 0.30R (i, j) +0.59G (i, j) +0.11B (i, j) (6).
As a further technical solution of the present invention, the local image feature extraction method in the step (S3) is a direction gradient histogram method, and the direction gradient histogram method includes:
(1) performing convolution operation on the original image by using [ -1,0,1] gradient operator to obtain gradient components in the x direction;
(2) performing convolution operation on the original image by using a [1, 0-1 ] T gradient operator to obtain a gradient component in the y direction;
(3) the gradient size and direction of the pixel point are calculated by the following formula:
Gx(x,y)＝H(x+1,y)-H(x-1,y)； (7)
Gy(x,y)＝H(x,y+1)-H(x,y-1)； (8)
wherein G isx(x,y)、Gy(x, y) and H (x, y) are the horizontal gradient, vertical gradient and pixel value at pixel point (x, y) in the input image, respectively, and the gradient magnitude and gradient direction at pixel point (x, y) are:
Figure BDA0002477512590000053
Figure BDA0002477512590000061
as a further technical solution of the present invention, in the step (S3), the all image feature extraction method is a CNN depth feature extraction method.
As a further technical solution of the present invention, the global texture image feature extraction method in the step (S3) is a complete local binary pattern CLBP feature extraction method.
As a further technical solution of the present invention, the calculation process of the multi-data feature association algorithm of the fuzzy neural network in the step (S4) is as follows:
(1) assuming that each image target in the image data set is j, utilizing a membership formula
Figure BDA0002477512590000062
A new image set is constructed satisfying the inequality:
Dz(i,j)＜LDz(j)MIN(11)
wherein:
Dz(i,j)＝vi(k)'[St(k)]-1vi(k) (12)
Dz(i,j)MIN＝min{Dz(1,j),...,Dz(i,j),Dz(n,j)} (13)
wherein Dz(i,j)MINEfficient testing in evaluating tests for imagesAll measurements in a volume set have a minimum deviation from the estimated image target in position information, the minimum deviation being 0, so the relative deviation in position of the ith measurement image and the jth image target is D'z(i,j)；
Then the formula is: d'z(i,j)＝Dz(i,j)/Dz(j)MIN；
When D is presentz(j)MINWhen 0, there is a relative deviation of D'z(i,j)＝Dz(i,j)；
(2) Then calculating each measurement data i (i ═ 1, 2.. multidot.k) in the effective measurement data set of the image target j and the target; k is the correlation degree measurement A (i, j) of the number measured in the target effective measurement set, wherein A (i, j) is more than or equal to 0 and less than or equal to 1; the target association fuzzy inference model is calculated;
(3) the decision picture target j is associated with the measurement in its active measurement set having the greatest degree of association metric.
As a further technical solution of the present invention, the method for performing data dimension reduction by the random forest algorithm in the step (S5) is an operation for integration based on the construction of a decision tree.
Has the positive and beneficial effects that:
according to the invention, through a deep learning algorithm, the image characteristics are obtained, so that the analysis and the recognition of data are realized, the accuracy of reading the facial expressions of students is improved, and the method is beneficial for teachers to read the expression of the students and obtain the psychological changes of the students so as to give psychological guidance. According to the invention, pure data information is obtained by preprocessing the image; the image characteristics are extracted, the analysis of the image characteristic information is realized by extracting the local information and the global texture image characteristics from the local information extraction and the global information extraction, the image information is fused, the student expression and emotion are deeply learned, and the emotion change condition reflected by the image is obtained. The invention can enable a psychological teacher to actively learn and understand the expression and emotional state change of students, help the teacher accurately and comprehensively master the overall situation of class students and promote the improvement of classroom teaching quality.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings needed in the embodiments will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a schematic diagram of the overall architecture of the present invention;
FIG. 2 is a schematic diagram of a data model architecture of a convolutional neural network CNN according to the present invention;
FIG. 3 is a schematic diagram of a multi-data feature association algorithm architecture according to the present invention;
FIG. 4 is a diagram illustrating a training process of the strong classifier according to the present invention;
FIG. 5 is a diagram of an expression database in accordance with an embodiment of the present invention;
FIG. 6 is an exemplary diagram of a network structure of a facial expression feature extraction CNN according to the present invention;
FIG. 7 is a schematic diagram of the extraction of features from a convolutional layer of the present invention;
FIG. 8 is a schematic diagram of one embodiment of convolutional layer extraction features of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the accompanying drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1 to 4, a method for extracting facial information of students based on image big data to realize educational psychology analysis, wherein the method comprises the following steps:
(S1) constructing a student image database set to acquire student face image information;
(S2) image preprocessing; performing tilt correction, size normalization, histogram equalization and gray level normalization processing on a face picture in a database, cleaning a blurred picture, and acquiring pure image data information;
in this step, the method of the gray scale transformation normalization is as follows: the gray scale distribution in the original image is expanded to the image with the whole gray scale by using a gray scale stretching method, wherein the formula of the gray scale transformation normalization is as follows:
Figure BDA0002477512590000091
wherein I (I, j) and N (I, j) respectively represent the gray scale value of the original image and the gray scale value of the transformed image, and min and max respectively represent the minimum gray scale value and the maximum gray scale value of the original image.
In this step, the maximum value method is to take the maximum value of the luminance of three components in a color image, including an R component, a G component, and a B component, as the gradation value of a gradation map.
In the step, the average value method is to convert the images collected at different time and under different illumination into standard images with the same gray mean and variance, and the calculation formula is
Figure BDA0002477512590000092
Figure BDA0002477512590000093
Wherein I (I, j), M, V respectively represent the gray value, mean, variance, N (I, j), M of the image before normalization0、V0Respectively representing the gray value, the mean value and the variance of the normalized image.
In this step, the weighted average method is to perform weighted average on three components including R component, G component and B component with different weights, and the calculation formula is:
f(i,j)＝0.30R(i,j)+0.59G(i,j)+0.11B(i,j) (6)。
(S3) image feature extraction; as shown in fig. 2, inputting the preprocessed image data into a convolutional neural network CNN network data model for depth feature extraction; the extraction method comprises the steps of local image feature extraction, global image feature extraction and global texture image feature extraction; the convolutional neural network CNN network data model comprises an input layer, a convolutional layer, a sub-sampling layer, a full connection layer and an output layer; the input layer is used for inputting picture information data, the convolutional layer is formed by arranging neurons into a feature plane, the number of feature surfaces in the sub-sampling layer is the same as that of the feature surfaces in the convolutional layer, the feature surfaces are arranged in a one-to-one corresponding relation so as to reduce feature dimension, the full connection layer is arranged to collect local features of the convolutional layer and the sampling layer, and the collected data information result is output;
in this step, the local image feature extraction method is a direction gradient histogram method, and the direction gradient histogram method includes:
(1) performing convolution operation on the original image by using [ -1,0,1] gradient operator to obtain gradient components in the x direction;
(2) performing convolution operation on the original image by using a [1, 0-1 ] T gradient operator to obtain a gradient component in the y direction;
(3) the gradient size and direction of the pixel point are calculated by the following formula:
Gx(x,y)＝H(x+1,y)-H(x-1,y)； (7)
Gy(x,y)＝H(x,y+1)-H(x,y-1)； (8)
wherein G isx(x,y)、Gy(x, y) and H (x, y) are the horizontal gradient, vertical gradient and pixel value at pixel point (x, y) in the input image, respectively, and the gradient magnitude and gradient direction at pixel point (x, y) are:
Figure BDA0002477512590000111
Figure BDA0002477512590000112
in this step, all image feature extraction methods are CNN depth feature extraction methods.
In applying the CNN depth feature extraction method, referring to fig. 6-8, in an exemplary embodiment, the implementation steps are:
(1) during convolution, performing convolution calculation on an input image through a trainable filter, wherein in order to facilitate calculation, a letter is used for representation, the filter is set to be f (x), a bias parameter is set to be b (x), the obtained convolution is set to be C (x), feature extraction is mainly performed in a convolution layer, original characteristic information is enhanced through convolution calculation, noise is reduced, and an enhanced pixel signal is obtained through convolution filtering;
(2) and performing sub-sampling, respectively summing four pixels in the field during sampling, outputting a new pixel, performing weighting calculation through a scalar W (x +1), then increasing an offset b (x +1), and then inputting a sigmoid activation function to obtain a reduced feature mapping image S (x +1), wherein preferably, the reduced feature mapping image is one eighth of the original image.
(3) Multiplying the result output by the sub-sampling layer by a weight vector through the full connection layer, and transmitting the result to a sigmoid function through setting a bias quantity;
(4) the sample data information is then calculated by classification by a classifier, such as using logistic regression.
(5) In this step, the global texture image feature extraction method is a complete local binary pattern CLBP feature extraction method. Common classifiers are svm and knn classifiers, which are used for facial expression recognition and can be realized through matlab programming. The method for extracting the CLBP features by utilizing the complete local binary pattern has stronger identification capability, when the method is applied, the CLBP features are firstly extracted, and the assumed features are as follows:
Figure RE-GDA0002529648360000121
initializing the mode type corresponding to each sample feature, thenComprises the following steps:
Figure RE-GDA0002529648360000122
wherein p is the feature set
Figure RE-GDA0002529648360000123
Characteristic weft number, vector
Figure RE-GDA0002529648360000124
Is composed of
Figure RE-GDA0002529648360000125
Then extracting each expression feature of different types according to Fisher criterion, and sequencing the CLBP features from big to small during the step to obtain the final expression feature
Figure RE-GDA0002529648360000126
Then select out
Figure RE-GDA0002529648360000127
The sum of the median feature values and the feature type whose percentage of the sum of all feature values is greater than the threshold value ξ:
Figure RE-GDA0002529648360000128
wherein q is q selected feature values, and the q feature values are more representative features in the CLBP features of the adopted image sample block, and then the corresponding q feature types are as follows:
Figure RE-GDA0002529648360000129
then, for the screened CLBP features of each class, taking intersection of feature types of the same class of samples, and screening out the common feature types of the class:
Figure BDA00024775125900001210
when the user identifies the d-th category expression, other category samples can be used as negative samples, then the common characteristics of the negative samples are extracted, the intersection is taken from the characteristic types of the negative samples, and the characteristic types of the negative sample category are obtained, and then:
Figure BDA0002477512590000131
finally obtaining the characteristic type JC for identifying the d-type expressionsglobal＝JCiUJN。
(S4) image data information fusion; as shown in fig. 3, the three extracted features are normalized respectively, and then cascade fusion is performed; the image cascade fusion method is a multi-data characteristic association algorithm based on a fuzzy neural network; the multi-data feature association algorithm model structure comprises an input layer, a fuzzy layer, a rule reasoning layer and an inverse fuzzy layer; wherein:
the input layer is used for inputting variables, the fuzzification layer is used for realizing fuzzification of the input variables, the fuzzification degree is determined through a membership degree, and the membership degree is expressed by the following formula:
Figure BDA0002477512590000132
wherein, aijAnd bijRespectively representing the center and the width of the membership function; mu (x)i) Representing the degree of membership of the jth fuzzy subset of the ith input variable;
the rule reasoning layer represents fuzzy operation through different set nodes, one node represents a fuzzy rule, and the number of the nodes is between 2 and 300;
the output layer is subjected to defuzzification processing through a gravity center method, and output information is expressed by the following formula:
Figure BDA0002477512590000133
where m denotes the number of rules, wiRepresenting rule heap layer outputIntensity of the ith rule of (1), riIndicating the conclusion of the ith rule.
In this step, the calculation process of the multi-data feature association algorithm of the fuzzy neural network is as follows:
(1) assuming that each image target in the image data set is j, utilizing a membership formula
Figure BDA0002477512590000141
A new image set is constructed satisfying the inequality:
Dz(i,j)＜LDz(j)MIN(11)
wherein:
Dz(i,j)＝vi(k)'[St(k)]-1vi(k) (12)
Dz(i,j)MIN＝min{Dz(1,j),...,Dz(i,j),Dz(n,j)} (13)
wherein Dz(i,j)MINMinimum deviation in positional information for all measurements in the active measurement set at the time of image evaluation test from the evaluation image target, the minimum deviation being 0, so that the relative deviation in position of the ith measurement image and the jth image target is D'z(i,j)；
Then the formula is: d'z(i,j)＝Dz(i,j)/Dz(j)MIN；
When D is presentz(j)MINWhen 0, there is a relative deviation of D'z(i,j)＝Dz(i,j)；
(2) Then calculating each measurement data i (i ═ 1, 2.. multidot.k) in the effective measurement data set of the image target j and the target; k is the correlation degree measurement A (i, j) of the number measured in the target effective measurement set, wherein A (i, j) is more than or equal to 0 and less than or equal to 1; the target association fuzzy inference model is calculated;
(3) the decision picture target j is associated with the measurement in its active measurement set having the greatest degree of association metric.
(S5) classifying the data after the image data information fusion; as shown in fig. 4, the fused features are input into a trained strong classifier for training and classification; the strong classifier is based on a random forest algorithm model, and the training process of the strong classifier is as follows:
randomly sampling a fixed number of student image samples from a sample training set after image preprocessing, putting back one sample when acquiring one sample, and then re-sampling, wherein if the random sampling is performed for T times on N sample training sets, the results of the T times of sampling are different due to the randomness of the sampling, the data with the most frequency is output as the result of each time, and the data is used as a final data model, and the current node of the point is set as a leaf node; in the original sample training data set, assume the input as sample set D { (x)1,y1),(x2,y2),...,(xm,ym) }; the first output is output through a weak learner algorithm, namely weak classifiers are iterated for T times, and the weak classifiers are superposed and output as a final strong classifier;
in the step, the method for performing data dimension reduction by the random forest algorithm is to perform integrated operation on the basis of constructing a decision tree.
(S6) outputting the expression data information; for the teacher to judge to recognize the corresponding psychological state of the student's facial information.
According to the invention, the image characteristics are obtained through a deep learning algorithm, so that the analysis and the identification of data are realized, the accuracy of reading the facial expressions of students is improved, and the method is beneficial for teachers to read the expressions of the students and obtain the psychological changes of the students so as to give psychological guidance. The method comprises the steps of preprocessing an image to obtain pure data information; the image characteristics are extracted, the analysis of the image characteristic information is realized from the extraction of local information and the extraction of global texture image characteristics, the image information is fused, the student expression and emotion are studied deeply, and the emotion change condition reflected by the image is obtained. The invention can enable the psychological teachers to actively learn and understand the expression and emotional state changes of students, help the teachers accurately and comprehensively master the overall situation of class students and promote the improvement of classroom teaching quality.
Although specific embodiments of the present invention have been described above, it will be understood by those skilled in the art that these specific embodiments are merely illustrative and that various omissions, substitutions and changes in the form of the detail of the methods and systems described above may be made by those skilled in the art without departing from the spirit and scope of the invention. For example, it is within the scope of the present invention to combine the steps of the above-described methods to perform substantially the same function in substantially the same way to achieve substantially the same result. Accordingly, the scope of the invention is to be limited only by the following claims.

Claims (10)

1. A method for extracting student face information to realize educational psychology analysis based on image big data is characterized by comprising the following steps: the method comprises the following steps:
(S1) constructing a student image database set to acquire student face image information;
(S2) image preprocessing; performing tilt correction, size normalization, histogram equalization and gray level normalization processing on a face picture in a database, cleaning a blurred picture, and acquiring pure image data information, wherein the adopted method at least comprises gray level conversion normalization, a maximum value method, an average value method or a weighted average method to graye a color image during gray level normalization processing;
(S3) image feature extraction; inputting the preprocessed image data into a Convolutional Neural Network (CNN) data model for depth feature extraction; the extraction method comprises the steps of local image feature extraction, global image feature extraction and global texture image feature extraction; the convolutional neural network CNN network data model comprises an input layer, a convolutional layer, a sub-sampling layer, a full connection layer and an output layer; the input layer is used for inputting picture information data, the convolutional layer is formed by arranging neurons into a feature plane, the number of feature surfaces in the sub-sampling layer is the same as that of the feature surfaces in the convolutional layer, the feature surfaces are arranged in a one-to-one corresponding relation so as to reduce feature dimension, the full connection layer is arranged to collect local features of the convolutional layer and the sampling layer, and the collected data information result is output;
(S4) image data information fusion; respectively normalizing the three extracted features, and then performing cascade fusion; the image cascade fusion method is a multi-data characteristic association algorithm based on a fuzzy neural network; the multi-data feature association algorithm model structure comprises an input layer, a fuzzy layer, a rule reasoning layer and an inverse fuzzy layer; wherein:
the input layer is used for inputting variables, the fuzzification layer is used for realizing fuzzification of the input variables, the fuzzification degree is determined through a membership degree, and the membership degree is expressed by the following formula:
μ(xi)＝exp{-[(xi-aij)/bij]2} (1)
wherein, aijAnd bijRespectively representing the center and the width of the membership function; mu (x)i) Representing the degree of membership of the jth fuzzy subset of the ith input variable;
the rule reasoning layer represents fuzzy operation through different set nodes, one node represents a fuzzy rule, and the number of the nodes is between 2 and 300;
the output layer is subjected to defuzzification processing through a gravity center method, and output information is expressed by the following formula:
Figure FDA0002477512580000021
where m denotes the number of rules, wiIntensity, r, of the ith rule representing the rule heap layer outputiA conclusion of the ith rule is shown;
(S5) classifying the data after the image data information fusion; inputting the fused features into a trained strong classifier for training and classification; the strong classifier is based on a random forest algorithm model, and the training process of the strong classifier is as follows:
randomly sampling a fixed number of student image samples from a sample training set after image preprocessing, wherein each samplingCollecting a sample, putting back the sample, then re-sampling, if random sampling is carried out for T times on N sample training sets, the results of the T times of sampling are different due to the randomness of the sampling, the data with the most frequency is output as the result of each time, the data is used as a final data model, and the current node of the point is set as a leaf node; in the original sample training data set, assume the input as sample set D { (x)1,y1),(x2,y2),...,(xm,ym) }; the first output is output through a weak learner algorithm, namely weak classifiers are subjected to iteration T times, and the weak classifiers are superposed and output as a final strong classifier;
(S6) outputting the expression data information; for the teacher to judge to identify the psychological state of the student's facial information response.
2. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the method of normalization of gray scale conversion in the step (S2) is: the gray scale distribution in the original image is expanded to the image with the whole gray scale by using a gray scale stretching method, wherein the formula of the gray scale transformation normalization is as follows:
Figure FDA0002477512580000031
wherein I (I, j) and N (I, j) respectively represent the gray scale value of the original image and the gray scale value of the transformed image, and min and max respectively represent the minimum gray scale value and the maximum gray scale value of the original image.
3. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the maximum value method in the step (S2) is to take the maximum value of the luminances of three components in the color image, including the R component, the G component, and the B component, as the gradation value of the gradation map.
4. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the average value method in the step (S2) is to convert the images collected at different time and under different illumination into standard images with the same mean and variance of gray scale, and the calculation formula is
Figure FDA0002477512580000032
Figure FDA0002477512580000041
Wherein I (I, j), M, V respectively represent the gray value, mean, variance, N (I, j), M of the image before normalization0、V0Respectively representing the gray value, the mean value and the variance of the normalized image.
5. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the weighted average method in the step (S2) is to perform weighted average on three components including the R component, the G component, and the B component with different weights, and the calculation formula is as follows: f (i, j) ═ 0.30R (i, j) +0.59G (i, j) +0.11B (i, j) (6).
6. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the local image feature extraction method in the step (S3) is a direction gradient histogram method, and the direction gradient histogram method includes:
(1) performing convolution operation on the original image by using [ -1,0,1] gradient operator to obtain gradient components in the x direction;
(2) performing convolution operation on the original image by using a [1, 0-1 ] T gradient operator to obtain a gradient component in the y direction;
(3) the gradient size and direction of the pixel point are calculated by the following formula:
Gx(x,y)＝H(x+1,y)-H(x-1,y)； (7)
Gy(x,y)＝H(x,y+1)-H(x,y-1)； (8)
wherein G isx(x,y)、Gy(x, y) and H (x, y) are the horizontal gradient, vertical gradient and pixel value at pixel point (x, y) in the input image, respectively, and the gradient magnitude and gradient direction at pixel point (x, y) are:
Figure FDA0002477512580000051
Figure FDA0002477512580000052
7. the method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: all image feature extraction methods in the step (S3) are CNN depth feature extraction methods.
8. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the global texture image feature extraction method in the step (S3) is a complete local binary pattern CLBP feature extraction method.
9. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the calculation process of the multi-data feature association algorithm of the fuzzy neural network in the step (S4) is as follows:
(1) assuming that each image target in the image data set is j, using the membership formula mu (x)i)＝exp{-[(xi-aij)/bij]2Construct a new image set, satisfying the inequality:
Dz(i,j)＜LDz(j)MIN
(11)
wherein:
Dz(i,j)＝vi(k)'[St(k)]-1vi(k)
(12)
Dz(i,j)MIN＝min{Dz(1,j),...,Dz(i,j),Dz(n,j)}
(13)
wherein Dz(i,j)MINMinimum deviation in positional information for all measurements in the active measurement set at the time of image evaluation test from the evaluation image target, the minimum deviation being 0, so that the relative deviation in position of the ith measurement image and the jth image target is D'z(i,j)；
Then the formula is: d'z(i,j)＝Dz(i,j)/Dz(j)MIN；
When D is presentz(j)MINWhen 0, there is a relative deviation of D'z(i,j)＝Dz(i,j)；
(2) Then calculating each measurement data i (i ═ 1, 2.. multidot.k) in the effective measurement data set of the image target j and the target; k is the correlation degree measurement A (i, j) of the number measured in the target effective measurement set, wherein A (i, j) is more than or equal to 0 and less than or equal to 1; the target association fuzzy inference model is obtained through calculation;
(3) the decision picture target j is associated with the measurement in its active measurement set having the greatest degree of association metric.
10. The method for realizing educational psychology analysis by extracting facial information of students based on image big data as claimed in claim 1, wherein: the method for performing data dimension reduction by the random forest algorithm in the step (S5) is an integrated operation based on the construction of a decision tree.
CN202010375024.0A 2020-05-03 2020-05-03 Method for extracting student face information to realize educational psychology analysis based on image big data Pending CN111553299A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010375024.0A CN111553299A (en) 2020-05-03 2020-05-03 Method for extracting student face information to realize educational psychology analysis based on image big data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010375024.0A CN111553299A (en) 2020-05-03 2020-05-03 Method for extracting student face information to realize educational psychology analysis based on image big data

Publications (1)

Publication Number Publication Date
CN111553299A true CN111553299A (en) 2020-08-18

Family

ID=72000634

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010375024.0A Pending CN111553299A (en) 2020-05-03 2020-05-03 Method for extracting student face information to realize educational psychology analysis based on image big data

Country Status (1)

Country Link
CN (1) CN111553299A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114373214A (en) * 2022-01-14 2022-04-19 平安普惠企业管理有限公司 User psychological analysis method, device, equipment and storage medium based on micro expression

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108304826A (en) * 2018-03-01 2018-07-20 河海大学 Facial expression recognizing method based on convolutional neural networks
CN109165670A (en) * 2018-07-12 2019-01-08 江南大学 A kind of TS-RBF fuzzy neural network robust fusion algorithm applied to infra red flame identification
CN109214298A (en) * 2018-08-09 2019-01-15 盈盈（杭州）网络技术有限公司 A kind of Asia women face value Rating Model method based on depth convolutional network
CN109446980A (en) * 2018-10-25 2019-03-08 华中师范大学 Expression recognition method and device
US20200327308A1 (en) * 2017-11-22 2020-10-15 Zhejiang Dahua Technology Co., Ltd. Methods and systems for face recognition

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200327308A1 (en) * 2017-11-22 2020-10-15 Zhejiang Dahua Technology Co., Ltd. Methods and systems for face recognition
CN108304826A (en) * 2018-03-01 2018-07-20 河海大学 Facial expression recognizing method based on convolutional neural networks
CN109165670A (en) * 2018-07-12 2019-01-08 江南大学 A kind of TS-RBF fuzzy neural network robust fusion algorithm applied to infra red flame identification
CN109214298A (en) * 2018-08-09 2019-01-15 盈盈（杭州）网络技术有限公司 A kind of Asia women face value Rating Model method based on depth convolutional network
CN109446980A (en) * 2018-10-25 2019-03-08 华中师范大学 Expression recognition method and device

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
赵高峰;毕笃彦;孙卫;: "基于模糊神经网络的数据挖掘算法", 空军工程大学学报(自然科学版), no. 03 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114373214A (en) * 2022-01-14 2022-04-19 平安普惠企业管理有限公司 User psychological analysis method, device, equipment and storage medium based on micro expression

Similar Documents

Publication Publication Date Title
Valentin et al. Categorization and identification of human face images by neural networks: A review of the linear autoassociative and principal component approaches
CN112101241A (en) Lightweight expression recognition method based on deep learning
CN113011357B (en) Depth fake face video positioning method based on space-time fusion
CN112287891B (en) Method for evaluating learning concentration through video based on expression behavior feature extraction
CN111090764B (en) Image classification method and device based on multitask learning and graph convolution neural network
CN112528777A (en) Student facial expression recognition method and system used in classroom environment
CN113239916B (en) Expression recognition and classroom state evaluation method, device and medium
Aitkenhead et al. A neural network face recognition system
CN106909938A (en) Viewing angle independence Activity recognition method based on deep learning network
Trokielewicz et al. Presentation attack detection for cadaver iris
Bala et al. Visual routine for eye detection using hybrid genetic architectures
CN112149616A (en) Figure interaction behavior recognition method based on dynamic information
CN115346149A (en) Rope skipping counting method and system based on space-time diagram convolution network
Goderya et al. Morphological classification of galaxies using computer vision and artificial neural networks: A computational scheme
Zaeem et al. Cause and effect: Concept-based explanation of neural networks
CN111967361A (en) Emotion detection method based on baby expression recognition and crying
CN110210380B (en) Analysis method for generating character based on expression recognition and psychological test
CN116994295B (en) Wild animal category identification method based on gray sample self-adaptive selection gate
CN114492634A (en) Fine-grained equipment image classification and identification method and system
Nie Research on facial expression recognition of robot based on CNN convolution neural network
CN111553299A (en) Method for extracting student face information to realize educational psychology analysis based on image big data
CN106447691A (en) Weighted extreme learning machine video target tracking method based on weighted multi-example learning
CN114723998A (en) Small sample image classification method and device based on large-boundary Bayes prototype learning
CN114022698A (en) Multi-tag behavior identification method and device based on binary tree structure
CN113688739A (en) Classroom learning efficiency prediction method and system based on emotion recognition and visual analysis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination