CN113158825A - Facial expression recognition method based on feature extraction - Google Patents

Facial expression recognition method based on feature extraction Download PDF

Info

Publication number
CN113158825A
CN113158825A CN202110341583.4A CN202110341583A CN113158825A CN 113158825 A CN113158825 A CN 113158825A CN 202110341583 A CN202110341583 A CN 202110341583A CN 113158825 A CN113158825 A CN 113158825A
Authority
CN
China
Prior art keywords
sub
facial expression
pixel
neighborhoods
neighborhood
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110341583.4A
Other languages
Chinese (zh)
Inventor
郭晓金
张哲�
张震
刘煌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Chongqing University of Post and Telecommunications
Original Assignee
Chongqing University of Post and Telecommunications
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Chongqing University of Post and Telecommunications filed Critical Chongqing University of Post and Telecommunications
Priority to CN202110341583.4A priority Critical patent/CN113158825A/en
Publication of CN113158825A publication Critical patent/CN113158825A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/168—Feature extraction; Face representation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Health & Medical Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a facial expression recognition method based on feature extraction, which comprises the steps of firstly dividing a facial expression image into sub-regions, obtaining two eight-bit binary sequences for the same pixel point by adopting two AR-LGBP operators with different sizes, carrying out logical XOR operation on the two binary sequences in a one-to-one correspondence manner to obtain a new binary sequence, converting the sequence into decimal numerical values which are pixel values of the pixel point, calculating the pixel value of each pixel point in each sub-region according to the method to obtain a histogram of the sub-region, and connecting the histograms of the sub-regions to generate facial expression feature vectors; and finally, reducing the dimension of the generated feature vector through a principal component analysis algorithm, and then carrying out facial expression classification and recognition by combining an SVM classifier. The invention not only considers the pixel relation between neighborhoods, can increase the feature description capability, but also has expansibility, and can extract features under different scales.

Description

Facial expression recognition method based on feature extraction
Technical Field
The invention relates to the technical field of digital image processing, in particular to a feature extraction technology which is high in efficiency, low in complexity, good in robustness and strong in identification capability and is adopted in an expression identification process.
Background
At present, research into artificial intelligence has reached a high level, while relatively little research has been done into human emotion and cognition. In real life, people expect that computers can serve the society like human beings, are more intelligent during man-machine interaction, are far from having the perception capabilities of vision and hearing, and need to add emotion understanding and emotion recognition functions. The visual information reflected by the human face is the most direct and important carrier for human emotional expression and interaction, and researchers can deduce the true mind of the expressive person through the change of facial expressions. Therefore, expression recognition is very important for emotion research.
Due to the complexity of human emotion and human facial expression and the higher and higher requirements for emotion recognition accuracy in some fields at present, the characteristic extraction algorithm of the classical Local Binary Pattern (LBP) is greatly improved in the aspects of extracting texture characteristics and realizing time.
In the classic LBP algorithm, within a window range of 3 × 3, a central pixel value is used as a threshold value to perform binarization processing on 8 pixel points in a neighborhood. By comparing the central pixel gcAnd adjacent pixel giThe grey value is encoded in relation to its magnitude, the result being a binary number of eight bits, if gi≥gcIf so, it is 1, otherwise, it is 0. And then, the binary codes of the 8 sampling points in the neighborhood are connected clockwise from the upper left corner to form an LBP binary code sequence of the central pixel. Finally, each binary number is assigned a binomial coefficient 2nThe weighted sum results in the LBP decimal code for the center pixel. The LBP has the advantages of gray scale invariance, rotation invariance, strong anti-interference and texture discrimination capability, simple calculation and certain inhibition effect on illumination. The LBP calculation process only compares the central pixel with the adjacent pixels, does not consider the gray scale relation between the adjacent pixels, and cannot extract the non-local information.
To address this problem, the industry proposes a Local Gradient Binary Pattern (LGBP) solution. LGBP calculates the relation among local pixels from the horizontal direction, the vertical direction and the diagonal direction, and compares the edges in the horizontal direction, the vertical direction and the diagonal direction respectivelyAnd coding two pixel values at the edge, and if the comparison result is greater than or equal to 0, assigning the value to be 1, otherwise, assigning the value to be 0. And filling the result into 8 sampling points in the neighborhood clockwise from the upper left corner, and finally connecting the binary codes of the 8 sampling points to form a binary sequence of the LGBP. And respectively and accurately expressing the change condition of each expression area in the image according to the comparison result. Finally, each binary number is assigned a binomial coefficient 2nThe weighted sum yields the LGBP decimal code. LGBP has stronger discrimination ability than LBP, but the operator is easily influenced by noise, and the neighborhood size is fixed, and texture features cannot be well extracted under large scale.
Disclosure of Invention
In view of this, the present invention aims to provide a facial expression recognition scheme based on an improved LGBP, that is, an Asymmetric Local Gradient Binary Pattern (AR-LGBP), which not only considers the pixel relationship between neighborhoods, can increase feature description capability, but also has extensibility, and can extract features at different scales.
In order to achieve the above object, the technical scheme adopted by the invention is a facial expression recognition method based on feature extraction, which comprises the following steps:
step one, dividing the facial expression image into a plurality of sub-regions.
And step two, adopting an asymmetric local gradient binary pattern to calculate the pixel value of each pixel point in the sub-region, and obtaining a histogram of the pixel values of the sub-region.
And step three, connecting the histograms of the sub-regions to generate the facial expression feature vectors.
And step four, reducing the dimension of the feature vector in the step three through a Principal Component Analysis (PCA) algorithm, and then combining a Support Vector Machine (SVM) classifier to carry out facial expression classification and recognition.
On the basis of the scheme, the second step in the scheme is optimized by using logic operation in a digital circuit.
In a digital circuit, when the binary numbers a and b are the same, the output is 0, and when they are not the same, the output is 1. The idea after the scheme is optimized is as follows: and for the same pixel point, according to an AR-LGBP operator, obtaining two sequences PA and PB by adopting two neighborhoods with different sizes, carrying out logical XOR operation on the two sequences according to bits to obtain a new sequence P, converting the new sequence into 8-bit binary number, taking the value as a texture characteristic value of the pixel point, calculating each pixel point in each sub-region according to the method, finally obtaining a histogram of the sub-region, and finally continuing the step three and the step four to finish final identification. The new scheme is an exclusive or Asymmetric Local Gradient Binary Pattern (XOR Asymmetric Local Gradient Binary Pattern, XOR-AR-LGBP).
The invention adopts the technical scheme to bring the following technical effects: (1) compared with an LGBP scheme, the size of the sub-neighborhood of the AR-LGBP scheme can be changed according to m and n, the expansibility is realized, the characteristics of the image can be extracted at any scale, and the advantages of the characteristics at any scale can be extracted so as to have better recognition effect finally; and the digit of the binary sequence is constant and does not change along with the size change of the neighborhood of the operator, so the characteristic dimension of the operator is constant and the phenomenon of dimension disaster caused by the increase of the complexity of the algorithm is avoided.
(2) The advantage of the AR-LGBP scheme is inherited, the XOR-AR-LGBP scheme combines the XOR logical operation in the digital circuit, the strength change relation among sub-neighborhoods with different scales in the horizontal direction, the vertical direction and the diagonal direction can be reflected, the texture features of the image can be better extracted, and the method has better identification capability.
Drawings
FIG. 1 is a schematic diagram of the 3 × 3 neighborhood gray scale of AR-LGBP in the present invention;
FIG. 2 is a schematic diagram of the 3 × 3 neighborhood gray scale of the AR-LGBP processed in the present invention;
FIG. 3 is a schematic diagram of XOR-AR-LGBP encoding in the present invention.
Detailed Description
The invention is described in further detail below with reference to the figures and specific embodiments.
The design idea of the invention is to provide a method which can extract expression characteristics under multi-scale and comprehensively consider the pixel relation between neighborhoodsThe method of (a). FIG. 1 is a schematic diagram of the 3 × 3 neighborhood gray scale of AR-LGBP. The AR-LGBP operator neighborhood size is (2m +1) × (2n +1), where,
Figure RE-GDA0003104606420000021
w represents the image width, h represents the image height, m is used to determine the width of any sub-neighborhood, n is used to determine the height of any sub-neighborhood, the symbol
Figure RE-GDA0003104606420000022
Indicating a rounding down. The operator divides the neighborhood into 9 sub-neighborhoods, which are marked as Ri. These neighborhoods may be divided into four sub-neighborhoods R of size mxn1，R3，R5，R 72 sub-neighborhoods R of size mx 14，R 82 sub-neighborhoods R of size 1 Xn2，R 61 sub-neighborhoods R of size 1 × 19，R9Is the central pixel point.
For ease of computation, the AR-LGBP is processed, as shown in FIG. 2, by setting the pixel value of each sub-neighborhood to be
Figure RE-GDA0003104606420000031
The size of the pixel is equal to the mean value of all pixel points in the sub-neighborhood, and a formula can be used
Figure RE-GDA0003104606420000032
Is represented by, wherein, NiAnd the number of all pixel points in the ith sub-neighborhood is represented. p is a radical ofijAnd expressing the pixel value of the jth pixel point of the ith sub-neighborhood. Its binary sequence can be represented as: p0,P1…P7Wherein P is0：
Figure RE-GDA0003104606420000033
P1：
Figure RE-GDA0003104606420000034
P2:
Figure RE-GDA0003104606420000035
P3：
Figure RE-GDA0003104606420000036
P4：
Figure RE-GDA0003104606420000037
P5：
Figure RE-GDA0003104606420000038
P6：
Figure RE-GDA0003104606420000039
P7：
Figure RE-GDA00031046064200000310
The s function is a binary function, and the AR-LGBP coding formula is as follows:
Figure RE-GDA00031046064200000311
Pithe value of the ith bit in the binary sequence is represented, and x and y respectively represent the abscissa and the ordinate of the pixel point.
Further, the exclusive or operation in the digital circuit is combined with the AR-LGBP algorithm. In a digital circuit, when the binary numbers a and b are the same, the output is 0, and when they are not the same, the output is 1. The idea after the scheme is optimized is as follows: for the same pixel point, according to an AR-LGBP operator, two neighborhoods with different sizes are adopted to obtain two sequences PA and PB, the two sequences are subjected to logical XOR operation according to bits to obtain a new sequence P, the new sequence is converted into an 8-bit binary number, and the value is used as a texture characteristic value of the pixel point and can be expressed as follows:
Figure RE-GDA00031046064200000312
wherein PAiAnd PBiThe ith bit of the sequences PA and PB respectively,
Figure RE-GDA00031046064200000313
indicating a logical exclusive-or operation and L indicates the length of the sequence, where L is 8. As shown in FIG. 3, is XOR-AR-LGBThe P-coding process takes two neighborhood sizes of 5 × 5 and 3 × 3 as an example. For the same pixel, there is intensity variation between neighborhoods, such as 3 × 3 sub-neighborhoods R of the central pixel8Pixel value greater than R4I.e. by
Figure RE-GDA00031046064200000314
Figure RE-GDA00031046064200000315
The code of the AR-LGBP descriptor is PA (01010001)281. Center pixel point R95 x 5 sub-neighborhoods R8Pixel value less than R4I.e. by
Figure RE-GDA00031046064200000316
The code of the AR-LGBP descriptor is PB (00010001)217, the PA and PB carry out bitwise logical XOR operation to obtain the code of XOR-AR-LGBP
Figure RE-GDA00031046064200000317
And finally, reducing the dimension of the generated feature vector through a Principal Component Analysis (PCA) algorithm, and then combining a Support Vector Machine (SVM) classifier to carry out facial expression classification and recognition. The principal component analysis algorithm and the support vector machine classifier can be realized by adopting the method in the prior art.

Claims (5)

1. A facial expression recognition method based on feature extraction is characterized by comprising the following steps:
dividing a facial expression image into a plurality of sub-areas;
step two, adopting an asymmetric local gradient binary pattern to calculate the pixel value of each pixel point in the sub-region to obtain a histogram of the sub-region;
connecting the histograms of the sub-regions to generate facial expression characteristic vectors;
and step four, reducing the dimension of the feature vector in the step three through a principal component analysis algorithm, and then combining a support vector machine classifier to carry out facial expression classification and recognition.
2. The facial expression recognition method based on feature extraction as claimed in claim 1, wherein: the process of calculating the pixel value is as follows: two AR-LGBP operators with different scales are adopted for a certain pixel point to obtain two eight-bit binary sequences, the two binary sequences are subjected to logic exclusive-OR operation in a one-to-one correspondence mode to obtain a new binary sequence P, the binary sequence P is converted into an 8-bit binary number and is used as a texture characteristic value of the pixel point, and a decimal value converted from the new binary sequence is the pixel value of the pixel point.
3. The facial expression recognition method based on feature extraction as claimed in claim 2, wherein: the AR-LGBP operator neighborhood size is (2m +1) × (2n +1), wherein,
Figure FDA0002999763360000011
1≤
Figure FDA0002999763360000012
w represents the image width, h represents the image height, m is used to determine the width of any sub-neighborhood, n is used to determine the height of any sub-neighborhood, the symbol
Figure FDA0002999763360000013
Expressing rounding down, dividing the neighborhood into 9 sub-neighborhoods, and recording as RiEach sub-neighborhood has a pixel value of
Figure FDA0002999763360000014
The binary sequence P is represented as: p0,P1…P7Wherein P is0：
Figure FDA0002999763360000015
P1：
Figure FDA0002999763360000016
P2:
Figure FDA0002999763360000017
P3：
Figure FDA0002999763360000018
P4：
Figure FDA0002999763360000019
P5：
Figure FDA00029997633600000110
P6：
Figure FDA00029997633600000111
P7：
Figure FDA00029997633600000112
The s-function is a binary function.
4. The facial expression recognition method based on feature extraction as claimed in claim 3, wherein: the 9 sub-neighborhoods comprise four sub-neighborhoods R with the size of mxn1，R3，R5，R72 sub-neighborhoods R of size mx 14，R82 sub-neighborhoods R of size 1 Xn2，R61 sub-neighborhoods R of size 1 × 19，R9Is the central pixel point.
5. The facial expression recognition method based on feature extraction as claimed in claim 2, wherein: the texture characteristic value is
Figure FDA00029997633600000113
Wherein PAiAnd PBiThe ith bit of the sequences PA and PB respectively,
Figure FDA00029997633600000114
and expressing logical XOR operation, L expressing the length of the sequence, and x and y expressing the horizontal and vertical coordinates of the central pixel points of the neighborhood.
CN202110341583.4A 2021-03-30 2021-03-30 Facial expression recognition method based on feature extraction Pending CN113158825A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110341583.4A CN113158825A (en) 2021-03-30 2021-03-30 Facial expression recognition method based on feature extraction

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110341583.4A CN113158825A (en) 2021-03-30 2021-03-30 Facial expression recognition method based on feature extraction

Publications (1)

Publication Number Publication Date
CN113158825A true CN113158825A (en) 2021-07-23

Family

ID=76885440

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110341583.4A Pending CN113158825A (en) 2021-03-30 2021-03-30 Facial expression recognition method based on feature extraction

Country Status (1)

Country Link
CN (1) CN113158825A (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106127196A (en) * 2016-09-14 2016-11-16 河北工业大学 The classification of human face expression based on dynamic texture feature and recognition methods
CN108960041A (en) * 2018-05-17 2018-12-07 首都师范大学 Image characteristic extracting method and device

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106127196A (en) * 2016-09-14 2016-11-16 河北工业大学 The classification of human face expression based on dynamic texture feature and recognition methods
CN108960041A (en) * 2018-05-17 2018-12-07 首都师范大学 Image characteristic extracting method and device

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
于明: "基于LGBP特征和稀疏表示的人脸表情识别", 《计算机工程与设计》 *
毋小省等: "《局部二值模式及扩展算子》", 31 October 2018, 北京：北京航空航天大学出版社 *
黄丽雯等: "非对称方向性局部二值模式人脸表情识别", 《计算机工程与应用》 *

Similar Documents

Publication Publication Date Title
JP4429370B2 (en) Human detection by pause
CN106127196B (en) Facial expression classification and identification method based on dynamic texture features
Ke-Chen et al. Research and perspective on local binary pattern
Fu et al. Centralized binary patterns embedded with image euclidean distance for facial expression recognition
Zahran et al. A MODIFIED LBP METHOD TO EXTRACT FEATURES FROM COLOR IMAGES.
CN111340047B (en) Image semantic segmentation method and system based on multi-scale feature and foreground and background contrast
CN109325507B (en) Image classification method and system combining super-pixel saliency features and HOG features
CN107729890B (en) Face recognition method based on LBP and deep learning
Kang et al. Faces detection method based on skin color modeling
CN110826408B (en) Face recognition method by regional feature extraction
CN109388727A (en) BGP face rapid retrieval method based on clustering
CN113963041A (en) Image texture recognition method and system
Moh’d Shamaileh et al. A new feature-based wavelet completed local ternary pattern (Feat-WCLTP) for texture image classification
He et al. Local compact binary count based nonparametric background modeling for foreground detection in dynamic scenes
CN109086801A (en) A kind of image classification method based on improvement LBP feature extraction
CN113158825A (en) Facial expression recognition method based on feature extraction
Yang et al. Weber binary pattern and Weber ternary pattern for illumination-robust face recognition
CN114119658A (en) Following algorithm for multi-feature self-adaptive fusion
CN109241886B (en) Face recognition method and system based on OLBP and PCA
Bedi et al. Local binary pattem variants: a review
Nguyen et al. Locating robust patterns based on invariant of LTP-based features
CN113888747A (en) Image texture feature extraction method based on annular local ternary mode
Ardakany et al. An extended local binary pattern for gender classification
Li et al. Maximum Entropy based Local Multiple Patterns for Texture Classification
Zehani et al. Features extraction using different histograms for texture classification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20210723