CN112800865B - Method for identifying facial makeup features based on attribute U-net model and normalized correlation coefficient matching - Google Patents
Method for identifying facial makeup features based on attribute U-net model and normalized correlation coefficient matching Download PDFInfo
- Publication number
- CN112800865B CN112800865B CN202110037721.XA CN202110037721A CN112800865B CN 112800865 B CN112800865 B CN 112800865B CN 202110037721 A CN202110037721 A CN 202110037721A CN 112800865 B CN112800865 B CN 112800865B
- Authority
- CN
- China
- Prior art keywords
- facial makeup
- matching
- net model
- image
- correlation coefficient
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T11/00—2D [Two Dimensional] image generation
- G06T11/001—Texturing; Colouring; Generation of texture or colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
- G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Abstract
The invention discloses a method for identifying facial features based on attribute U-net model and normalized correlation coefficient matching, which comprises the steps of firstly, resizing the facial makeup of the Qin cavity, then adding attribute Gate into the tail end of each jump connection of the U-net model, realizing an Attention mechanism for the extracted features, enabling the extracted features to select focusing positions, and then cutting facial makeup images generated by the attribute U-net model to form template images; finally, carrying out template matching on the input target facial makeup image through an improved normalization correlation coefficient matching method, finding out a template image with highest similarity with the target image, outputting facial makeup characteristics represented by the template image, and deducing character figures represented by the facial makeup through the facial makeup characteristics; the problem of the characteristic recognition of the facial makeup of the Qin cavity is solved, the accuracy of the matching of the normalized correlation coefficient is improved, and the operation speed is greatly improved.
Description
Technical Field
The invention belongs to the technical field of computer graphics processing, and particularly relates to a method for matching and identifying facial makeup features based on an Attention U-net model and a normalized correlation coefficient.
Background
The Qin cavity originates from Qin land and is popular in northwest areas, and has extremely important guiding function on the whole Chinese dramatic art development. The main expression forms of the Qin cavity are singing, reciting, doing and playing, and the forms are very rich in the performance process, wherein the most important is to highlight the facial makeup art of the personality of the dramatic characters. According to the traditional Chinese 'different hearts', the facial makeup mainly shows the activities of the hearts of the figures, is a concentrated representation of the mental lives of the people, and is applied to dramatic performances in a form, and the concentrated and exaggerated representation is obvious, like the facial makeup art in the Qin cavity.
The facial makeup in the Qin cavity is of different types and has unique emphasis on color and conformation. Besides a certain cultural connotation, each facial makeup also permeates the loyal etiquette, happiness, fun and funeral in the life of Shanxi and even northwest people, and the cultural connotation of loyal and funeral.
Currently, the template matching algorithm applied to image recognition is one of the basic operations in image processing, pattern recognition, and computer vision. The template matching algorithm mainly comprises the following steps: square difference matching, standard square difference matching, which uses square difference to match, preferably 0. The worse the match, the larger the match value; correlation matching and standard correlation matching are adopted in the method, multiplication operation between a template and an image is adopted, so that a larger number indicates a higher matching degree, and 0 identifies the worst matching effect. Normalized correlation coefficient matching, such methods match the relative value of the template to its mean with the relative value of the image to its mean, 1 represents a perfect match, -1 represents a bad match, and 0 represents no correlation (random sequence). Matching of normalized correlation coefficients in actual image processing also needs to take into account factors such as scaling, rotation, etc. of the target image, which requires a large amount of computation of the cross-correlation coefficients between a certain region to be matched in the target image and the template image. And the larger the target image is, the larger the calculated multiplication times are, so that the normalized correlation coefficient matching algorithm has the defects of large calculated amount, long calculation time consumption and the like.
Disclosure of Invention
Aiming at the defects or shortcomings of the prior art, the invention aims to provide a method for recognizing facial makeup features based on the Attention U-net model and normalized correlation coefficient matching, which learns finer features in pictures, reduces the calculated amount of operation, improves the image matching accuracy, improves the algorithm operation speed, and ensures that the facial makeup features of the Qin cavity are recognized faster, more accurately and better in effect.
In order to achieve the above purpose, the technical scheme adopted by the invention is as follows:
the method for identifying facial makeup features based on the attribute U-net model and the normalized correlation coefficient matching comprises the following steps:
step 1: collecting facial makeup images of the Qin cavity, wherein the size is 256x256mm;
step 2: constructing an original U-net model network structure by using Keras, wherein the U-net model network structure consists of two parts, namely an encoder (encoder) and a decoder (decoder), which are symmetrical structures;
step 3: adding an Attention Gate model into the tail end of each jump connection of the original U-net model;
step 4: generating a gray Attention pattern of the ash chamber facial makeup by using an Attention U-net model;
step 5: clipping a gray attention graph characteristic part to be used as a template image;
step 6: carrying out normalization correlation coefficient matching on the Qin cavity facial makeup image and the template image in the step 1 to obtain a template image with highest similarity, and outputting facial makeup characteristics represented by the template image and character characteristics represented by the facial makeup characteristics;
the normalization process of the correlation coefficient is expressed as: :
wherein S (u, v) is the accumulated sum value of each pixel point, S 2 (u, v) is the cumulative square value of all pixels, S 2 (u+U-1，v+V-1)-S 2 (u-1，v+V-1)-S 2 (u+U-1，v-1)+S 2 (u-1, v-1) is a matching region in the template imageU is the pixel point of the template image, V is the pixel point of the template image matching the target image, U, V is the length and width of the target image size, FFT is the fast Fourier transform function, e t (u, v) zero-mean template image energy function,is the gray pixel average value of the template image.
Further, when template matching is performed on the template image and the target image in the step 6, the matching sequence is diffusion matched from the center of the target image to two sides.
The invention has the following effects:
according to the method for recognizing facial makeup features based on the Attention U-net model and the normalized correlation coefficient matching, firstly, the size of the facial makeup of the Qin cavity is reset, then the facial makeup of the Qin cavity is integrated based on the U-net model, and Attention Gates are used in a decoder part of the U-net model, so that an Attention U-net model is obtained. Experiments show that after the attribute Gates are integrated, the U-net model has higher precision; secondly, putting the processed ash-chamber facial makeup into an Attention U-net model to generate a gray Attention as a target image, and finding out through experiments, so that the facial makeup texture is clearer and the image template matching is more facilitated; then cutting the generated gray attention pattern facial makeup characteristic part to be used as a template image; and finally, carrying out template matching on the target image and the template image through an improved normalized correlation coefficient matching algorithm, obtaining the template image with the highest similarity by matching, and outputting the facial features represented by the template image and the character represented by the facial features. The method solves the problem of characteristic recognition of the facial makeup of the Qin cavity, improves the accuracy of matching the normalized correlation coefficient, and greatly improves the operation speed.
The invention can make more people know the artistic charm and cultural connotation of ancient games and the related colloquial, folk and folk stories and education functions by identifying the facial features of the Qin Chamber.
Drawings
FIG. 1 is a collected partial Qin cavity facial makeup image;
FIG. 2 is a diagram of a U-net model;
FIG. 3 is a gray scale map generated by U-net;
FIG. 4 is a view of the Attention Gate structure;
FIG. 5 is an overall block diagram of the Attention U-net model;
FIG. 6 is a gray Attention drawing of the Qin cavity facial makeup generated by the Attention U-net model;
FIG. 7 is a partial template diagram of a clip gray attention drawing;
FIG. 8 is a result diagram of facial feature recognition through an improved normalized correlation coefficient matching algorithm and an Attention U-net model;
FIG. 9 is a result diagram of facial feature recognition by an original normalized correlation coefficient matching algorithm;
FIG. 10 is a result diagram of facial feature recognition through an original normalized correlation coefficient matching algorithm and a U-net model;
Detailed Description
The present invention is described in further detail below in connection with specific examples, but is not limited thereto.
The invention provides a method for identifying facial makeup features based on an Attention U-net model and normalized correlation coefficient matching, which specifically comprises the following steps:
step 1: the data set is first collected on the Qin cavity website or website related to the Qin cavity facial makeup, and then the image size is changed to (256 ), as in FIG. 1, for two images collected.
Step 2: the original U-net model network structure is constructed by Keras, and the U-net model network structure consists of two parts, namely an encoder (decoder) and a decoder (decoder), which are symmetrical structures. The U-net model is shown in FIG. 2.
The encoder corresponds to an image downsampling process, the decoder corresponds to a feature map upsampling process, and skip connections exist between the respective encoder and decoder, which can help the upsampling layer recover the detail information of the image. The encoder extracts the characteristic information of the image through a typical convolutional neural network structure, and consists of a 3×3 convolutional layer, a ReLU function and a 2×2 maximum pooling layer, 4 downsampling is performed in total, the characteristic image size is reduced after pooling operation is performed each time, and the channel number is doubled. The decoder performs up-sampling by a 2 x2 deconvolution layer (or transpose convolution) to gradually recover the image information. Corresponding to the encoder section, the decoder section performs up-sampling 4 times in total, each up-sampling expanding the feature map size while halving the number of channels. The image generated by the U-net is shown in fig. 3.
Step 3: on the original U-net, add the attention mechanism:
the Attention U-net model has one more Attention Gate structure than the original version U-net model, as shown in FIG. 4. The Attention Gate is connected to the end of each jump connection, and an Attention mechanism is implemented for the extracted feature. The overall structure of the Attention U-net is shown in FIG. 5.
Wherein, the feature map g of the same layer is downsampled i (F g ×H g ×W g ×D g ) 1X 1 convolution operation to obtainFeature map of layer above upsampling layer +.>By 1X 1 convolution operation to obtainThe feature map obtained from the upper two parts +.>And->Adding and then performing ReLU to obtainσ 1 For the function to be activated by the ReLU, then a convolution operation of 1X 1 is used to obtain +.>Finally pair->Performing a sigmoid activation function to obtain the final attention coefficients (alpha' i )。
Step 4: and generating the gray Attention pattern of the ash chamber facial makeup by using an Attention U-net model.
In order to verify the effectiveness of the Attention mechanism for the U-net model, the invention generates the gray Attention pattern of the ash chamber facial makeup by using the Attention U-net model by using one facial makeup image. As shown in fig. 6, the texture in the facial makeup is found to be clearer and thus better than that of fig. 3.
Step 5: the gray attention map feature portion is clipped as a template image, and a part of the template image is not as shown in fig. 7.
Step 6: and (3) carrying out normalized correlation coefficient matching on the Qin cavity facial makeup image and the template image in the step (1), obtaining the template image with the highest similarity by matching, and outputting facial makeup characteristics represented by the template image and character characters represented by the facial makeup characteristics. As shown in fig. 8.
The similarity measurement is carried out on the two images, the data can be used for objectively evaluating and normalizing more accurately, and the normalization processing of the correlation coefficient is expressed as follows:
wherein X and Y are the sizes of the searched target images, the sizes of the target images used in the invention are unified to 256X256mm, V is the pixel point of the template image matched with the target image, U and V are the length and width of the target image, f (X, Y) is the pixel gray value with X as the abscissa and Y as the ordinate in the matching range, t (X-U, Y-V) is the pixel gray value in the template image, f u，v For the average value in the matching range,for template imagesGray pixel average, where f u，v Is calculated as follows:
(1) The denominator in the formula is a zero-mean function f (x, y) -f u，v Template function with zero mean valueIs a variance of (c).
Normalized correlation coefficient algorithm improvement:
from the formulas (1) and (2), it can be seen that when the target image is matched with the template image template, the number of multiplications required to be calculated is very large, which directly affects the operation speed of the algorithm, so that the number of multiplications calculated when the template is matched can be greatly reduced by adopting Fast Fourier Transform (FFT), the more obvious the calculated amount is saved when the template is matched, and the normalized correlation coefficient algorithm is finally improved by inverse transform processing:
wherein S is 2 (u+U-1，v+V-1)-S 2 (u-1，v+V-1)-S 2 (u+U-1，v-1)+S 2 (u-1, v-1) is the sum of the pixel gray values of the matching regions in the template image.
In order to verify the effectiveness of combining the Attention U-net model and the improved normalization correlation coefficient matching algorithm, the invention carries out facial feature recognition on a facial makeup image through the original normalization correlation coefficient matching algorithm, the result diagram is shown in fig. 9, and the facial makeup feature recognition is carried out through the original normalization correlation coefficient matching algorithm and the U-net model, and the result diagram is shown in fig. 10. Through comparison, the accuracy of facial feature recognition is higher, the recognition speed is faster, and the effect is better by combining the Attention U-net and the improved normalization correlation coefficient matching algorithm.
The present invention has been described in detail with reference to the above embodiments, and it should be understood by those skilled in the art that: modifications and equivalents may be made to the specific embodiments of the invention without departing from the spirit and scope of the invention, which are intended to be covered by the scope of the claims.
Claims (2)
1. The method for identifying facial makeup features based on the attribute U-net model and the normalized correlation coefficient matching is characterized by comprising the following steps:
step 1: collecting facial makeup images of the Qin cavity, wherein the size is 256x256mm;
step 2: constructing an original U-net model network structure by using Keras, wherein the U-net model network structure consists of two parts, namely an encoder (encoder) and a decoder (decoder), which are symmetrical structures;
step 3: adding an Attention Gate model into the tail end of each jump connection of the original U-net model;
step 4: generating a gray Attention pattern of the ash chamber facial makeup by using an Attention U-net model;
step 5: clipping a gray attention graph characteristic part to be used as a template image;
step 6: and (3) carrying out normalization correlation coefficient matching on the Qin cavity facial makeup image and the template image in the step (1), obtaining the template image with the highest similarity by matching, and outputting facial makeup characteristics represented by the template image and character characteristics represented by the facial makeup characteristics.
The normalization process of the correlation coefficient is expressed as:
where u is the pixel of the template image, v is the pixel of the template image matching the target image, FFT is the fast Fourier transform function, e t (u, v) is a zero-mean template image energy function,is the gray pixel average value of the template image.
2. The method for identifying facial features based on attribute U-net model and normalized correlation coefficient matching according to claim 1, wherein the method comprises the following steps: and (3) when template matching is carried out on the template image and the target image in the step (6), the matching sequence is diffusion matched from the center of the target image to two sides.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110037721.XA CN112800865B (en) | 2021-01-12 | 2021-01-12 | Method for identifying facial makeup features based on attribute U-net model and normalized correlation coefficient matching |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110037721.XA CN112800865B (en) | 2021-01-12 | 2021-01-12 | Method for identifying facial makeup features based on attribute U-net model and normalized correlation coefficient matching |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112800865A CN112800865A (en) | 2021-05-14 |
| CN112800865B true CN112800865B (en) | 2023-08-25 |
Family
ID=75810131
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110037721.XA Active CN112800865B (en) | 2021-01-12 | 2021-01-12 | Method for identifying facial makeup features based on attribute U-net model and normalized correlation coefficient matching |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112800865B (en) |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902710A (en) * | 2019-01-07 | 2019-06-18 | 南京热信软件科技有限公司 | A kind of fast matching method and device of text image |
| CN111028306A (en) * | 2019-11-06 | 2020-04-17 | 杭州电子科技大学 | AR2U-Net neural network-based rapid magnetic resonance imaging method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20200381096A1 (en) * | 2019-06-03 | 2020-12-03 | The Board Of Trustees Of The Leland Stanford Junior University | Methods of Predicting Disorder Progression for Control Arms Within an Experimental Trial |
-
2021
- 2021-01-12 CN CN202110037721.XA patent/CN112800865B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109902710A (en) * | 2019-01-07 | 2019-06-18 | 南京热信软件科技有限公司 | A kind of fast matching method and device of text image |
| CN111028306A (en) * | 2019-11-06 | 2020-04-17 | 杭州电子科技大学 | AR2U-Net neural network-based rapid magnetic resonance imaging method |
Non-Patent Citations (1)
| Title |
|---|
| 基于快速傅里叶变换的心电模板匹配算法设计;周酥;《医疗卫生装备》;20140515;全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112800865A (en) | 2021-05-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107730536B (en) | High-speed correlation filtering object tracking method based on depth features | |
| Zhang et al. | Automatic speechreading with applications to human-computer interfaces | |
| CN111160533A (en) | Neural network acceleration method based on cross-resolution knowledge distillation | |
| CN106709964B (en) | Sketch generation method and device based on gradient correction and multidirectional texture extraction | |
| CN110992374B (en) | Hair refinement segmentation method and system based on deep learning | |
| CN113139904B (en) | Image blind super-resolution method and system | |
| CN116452410A (en) | Text-guided maskless image editing method based on deep learning | |
| CN115019143A (en) | Text detection method based on CNN and Transformer mixed model | |
| CN116310693A (en) | Camouflage target detection method based on edge feature fusion and high-order space interaction | |
| CN114898438A (en) | Cross-modal depth forgery detection method based on time-frequency domain visual artifact characteristic adaptive fusion | |
| CN116129129A (en) | Character interaction detection model and detection method | |
| CN108830878B (en) | Target tracking method based on FPN neural network | |
| CN112801092B (en) | Method for detecting character elements in natural scene image | |
| CN108710836B (en) | Lip detection and reading method based on cascade feature extraction | |
| CN112800865B (en) | Method for identifying facial makeup features based on attribute U-net model and normalized correlation coefficient matching | |
| CN116434759B (en) | Speaker identification method based on SRS-CL network | |
| CN112800882A (en) | Mask face posture classification method based on weighted double-flow residual error network | |
| CN109165551B (en) | Expression recognition method for adaptively weighting and fusing significance structure tensor and LBP characteristics | |
| CN108460383B (en) | Image significance refinement method based on neural network and image segmentation | |
| CN112215868B (en) | Method for removing gesture image background based on generation of countermeasure network | |
| CN114821580A (en) | Noise-containing image segmentation method by stage-by-stage merging with denoising module | |
| CN113111848A (en) | Human body image analysis method based on multi-scale features | |
| Lee | Visual-speech-pass filtering for robust automatic lip-reading | |
| CN110427892B (en) | CNN face expression feature point positioning method based on depth-layer autocorrelation fusion | |
| CN113724273A (en) | Edge light and shadow fusion method based on neural network regional target segmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |