CN111611962A - A face image super-resolution recognition method based on fractional multi-set partial least squares - Google Patents
A face image super-resolution recognition method based on fractional multi-set partial least squares Download PDFInfo
- Publication number
- CN111611962A CN111611962A CN202010473054.5A CN202010473054A CN111611962A CN 111611962 A CN111611962 A CN 111611962A CN 202010473054 A CN202010473054 A CN 202010473054A CN 111611962 A CN111611962 A CN 111611962A
- Authority
- CN
- China
- Prior art keywords
- resolution
- fractional
- fmpls
- matrix
- face
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 29
- 239000011159 matrix material Substances 0.000 claims abstract description 41
- 238000012549 training Methods 0.000 claims abstract description 23
- 239000000284 extract Substances 0.000 claims abstract description 10
- 238000012360 testing method Methods 0.000 claims abstract description 8
- 230000001427 coherent effect Effects 0.000 claims abstract description 5
- 238000005457 optimization Methods 0.000 claims description 9
- 239000013598 vector Substances 0.000 claims description 9
- 238000000354 decomposition reaction Methods 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 3
- 238000000844 transformation Methods 0.000 claims description 3
- 238000013507 mapping Methods 0.000 abstract description 4
- 238000002474 experimental method Methods 0.000 description 8
- 238000000513 principal component analysis Methods 0.000 description 8
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 238000013135 deep learning Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000000052 comparative effect Effects 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000000491 multivariate analysis Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/172—Classification, e.g. identification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24147—Distances to closest patterns, e.g. nearest neighbour classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/16—Human faces, e.g. facial parts, sketches or expressions
- G06V40/168—Feature extraction; Face representation
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Computational Biology (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- Oral & Maxillofacial Surgery (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Image Analysis (AREA)
Abstract
本发明公开了基于分数阶多集偏最小二乘的人脸图像超分辨率识别方法,步骤1在训练阶段利用训练集学习不同分辨率视图之间的相关关系,使用PCA对图像降维,利用分数阶思想重新估计组内及组间协方差矩阵,并计算FMPLS投影矩阵,将主成分特征投影到FMPLS的一致相干子空间;步骤2在测试阶段,提取输入的多种低分辨率图像的主成分特征,并投影到相应的FMPLS子空间,通过邻域重建策略重建出输入的低分辨率图像的高分辨率特征;步骤3最后利用最近邻分类器进行人脸识别。本发明利用分数阶多集偏最小二乘,可以同时学习不同分辨率人脸视图之间的多种特定分辨率的映射,同时借助分数阶思想,重新估计协方差矩阵,以减少由样本数量不足、噪声等因素带来的影响。
The invention discloses a face image super-resolution recognition method based on fractional order multi-set partial least squares. Step 1 uses the training set to learn the correlation between views of different resolutions in the training stage, uses PCA to reduce the image dimension, and uses The fractional-order idea re-estimates the intra- and inter-group covariance matrices, calculates the FMPLS projection matrix, and projects the principal component features to the consistent coherent subspace of FMPLS; step 2 In the testing phase, extract the principal components of various input low-resolution images. The component features are projected to the corresponding FMPLS subspace, and the high-resolution features of the input low-resolution images are reconstructed through the neighborhood reconstruction strategy; Step 3 Finally, the nearest neighbor classifier is used for face recognition. The present invention utilizes fractional order multi-set partial least squares, can simultaneously learn a variety of specific resolution mappings between face views of different resolutions, and at the same time re-estimates the covariance matrix with the help of fractional order ideas, so as to reduce the problem caused by the insufficient number of samples , noise and other factors.
Description
技术领域technical field
本发明涉及人脸超分辨率识别领域,特别涉及一种基于分数阶多集偏最小二乘的人脸图像超分辨率识别方法。The invention relates to the field of face super-resolution recognition, in particular to a face image super-resolution recognition method based on fractional order multi-set partial least squares.
背景技术Background technique
在现实世界应用中,尤其是视频监控领域,由于光照条件差、成像距离较远姿态变化等因素,设备捕获到的人脸图像通常都是低分辨率的,这对传统的人脸识别算法来说是一大难题。为了解决这一问题,近些年来许多人脸超分辨率重建方法被提出,旨在将输入的低分辨率人脸图像恢复成高分辨率图像。In real-world applications, especially in the field of video surveillance, due to factors such as poor lighting conditions, long imaging distance, and posture changes, the face images captured by the device are usually low-resolution, which is difficult for traditional face recognition algorithms. Said it was a big problem. In order to solve this problem, many face super-resolution reconstruction methods have been proposed in recent years, aiming to restore the input low-resolution face images into high-resolution images.
多元分析方法常被运用于超分辨率重建以进行特征提取,其中较为流行的是主成分分析(Principal Component Analysis,PCA),特征提取步骤通常用来对数据降维并减少噪声。PCA通过保留合适的维度以提取人脸的有用信息并过滤噪声,Wang等人提出一个通过PCA得到图像线性组合系数来生成高分辨率人脸的框架。Multivariate analysis methods are often used in super-resolution reconstruction for feature extraction, and the more popular one is Principal Component Analysis (PCA). The feature extraction step is usually used to reduce data dimensionality and reduce noise. PCA extracts useful information of faces and filters noise by preserving suitable dimensions. Wang et al. propose a framework for generating high-resolution faces by obtaining image linear combination coefficients through PCA.
超分辨率方法可大致分为三类:基于插值的、基于重建的以及基于学习的方法,其中基于学习的方法旨在从训练集中学习高低分辨率图像之间的映射关系。近年来,将深度学习应用于人脸超分辨率技术取得了较大的成功。但是,基于深度学习的方法通常需要大量的训练数据以学习最优的参数,并且尽管硬件性能大幅提升,深度网络的学习过程仍然需要耗费大量时间,因此传统的超分辨率方法仍然有较大的研究价值。Super-resolution methods can be roughly divided into three categories: interpolation-based, reconstruction-based, and learning-based methods, where learning-based methods aim to learn the mapping relationship between high- and low-resolution images from a training set. In recent years, the application of deep learning to face super-resolution technology has achieved great success. However, deep learning-based methods usually require a large amount of training data to learn optimal parameters, and although the hardware performance is greatly improved, the learning process of deep networks still takes a lot of time, so traditional super-resolution methods still have a large research value.
在现实生活中,人们通常需要面对同一人脸具有多种不同分辨率视图的情况。对于多种不同的低分辨率视图,先前的方法通常需要分别训练和处理每一种分辨率视图,这种处理方式十分耗时且效率低下。换句话说,现有的经典方法很难将多个低分辨率视图同时映射到高分辨率人脸空间,迄今为止,共同学习多种视图之间关系还没有得到广泛关注;此外,由于训练样本不足或噪声等因素的影响,协方差矩阵可能会存在偏差。In real life, people often face situations where the same face has multiple views of different resolutions. For multiple different low-resolution views, previous methods typically require training and processing each resolution view separately, which is time-consuming and inefficient. In other words, it is difficult for existing classical methods to map multiple low-resolution views to the high-resolution face space simultaneously, and so far, co-learning the relationship between multiple views has not received extensive attention; in addition, due to the training samples Due to factors such as insufficient or noise, the covariance matrix may be biased.
发明内容SUMMARY OF THE INVENTION
本发明的目的是克服现有技术缺陷,提供一种基于分数阶多集偏最小二乘的人脸图像超分辨率识别方法,解决了现有技术中无法同时学习多种分辨率视图之间相关关系且协方差矩阵可能存在偏差的问题。The purpose of the present invention is to overcome the defects of the prior art, provide a face image super-resolution recognition method based on fractional order multi-set partial least squares, and solve the problem that the prior art cannot simultaneously learn the correlation between views of multiple resolutions relationship and the covariance matrix may be biased.
本发明的目的是这样实现的:一种基于分数阶多集偏最小二乘的人脸图像超分辨率识别方法,包括以下步骤:The purpose of the present invention is to achieve in this way: a kind of face image super-resolution recognition method based on fractional order multi-set partial least squares, comprising the following steps:
步骤1在训练阶段利用训练集学习不同分辨率视图之间的相关关系,使用PCA对图像降维,利用分数阶思想重新估计组内及组间协方差矩阵,计算FMPLS投影矩阵,将主成分特征投影到FMPLS的一致相干子空间;Step 1 In the training phase, use the training set to learn the correlation between views of different resolutions, use PCA to reduce the dimension of the image, use the fractional order idea to re-estimate the covariance matrix within and between groups, calculate the FMPLS projection matrix, and convert the principal component features. Projection to a consistent coherent subspace of FMPLS;
步骤2在测试阶段,提取输入的多种低分辨率图像的主成分特征,并投影到相应的FMPLS子空间,通过邻域重建策略重建出输入的低分辨率图像的高分辨率特征;Step 2: In the test phase, extract the principal component features of various input low-resolution images, and project them into the corresponding FMPLS subspace, and reconstruct the high-resolution features of the input low-resolution images through the neighborhood reconstruction strategy;
步骤3最后利用最近邻分类器预测人脸标签,输出超分辨率人脸识别结果。Step 3 Finally, the nearest neighbor classifier is used to predict the face label, and the super-resolution face recognition result is output.
作为本发明的进一步限定,所述步骤1具体包括以下步骤:As a further limitation of the present invention, the step 1 specifically includes the following steps:
(1)对于m个多维中心化随机变量其中di表示xi的维度,FMPLS目的为求解如下的最优化问题,以寻找一组线性变换 (1) For m multidimensional centralized random variables where d i represents the dimension of xi , and the purpose of FMPLS is to solve the following optimization problem to find a set of linear transformations
其中Tr(·)表示矩阵的迹,为经过分数阶重新估计过的协方差矩阵:where Tr( ) represents the trace of the matrix, is the fractionally re-estimated covariance matrix:
(2)对于利用分数阶重新估计协方差矩阵,使用特征值分解分解组内协方差矩阵并用分数阶参数αi为指数进行重新估计:(2) For the fractional re-estimation of the covariance matrix, use the eigenvalue decomposition to decompose the intra-group covariance matrix and use the fractional parameter α i as the index to re-estimate:
其中Qi为组内协方差Sii的特征向量组成的矩阵,是以分数阶参数αi为指数的特征值对角矩阵,ri=rank(Sii)且分数阶参数αi满足0≤αi≤1;使用奇异值分解分解组间协方差矩阵并用分数阶参数βij重新估计:where Q i is the matrix composed of the eigenvectors of the within-group covariance S ii , is a diagonal matrix of eigenvalues with fractional parameter α i as an index, r i =rank(S ii ) and fractional parameter α i satisfies 0≤α i ≤1; use singular value decomposition to decompose the covariance matrix between groups and use fractional The order parameter β ij is re-estimated:
其中Uij和Vij分别是左右奇异值向量组成的矩阵,是以分数阶参数βij为指数的奇异值对角矩阵,rij=rank(Sij)且分数阶参数βij满足0≤βij≤1;where U ij and V ij are matrices composed of left and right singular value vectors, respectively, is a diagonal matrix of singular values with the fractional parameter β ij as an index, r ij =rank(S ij ) and the fractional parameter β ij satisfies 0≤β ij ≤1;
(3)对于该最优化问题,利用递归方法求解,假设前k-1个方向已经得到,其中k≤d,对于第k个方向通过求解如下最优化问题得到:(3) For this optimization problem, use the recursive method to solve it, assuming the first k-1 directions has been obtained, where k≤d, for the kth direction It is obtained by solving the following optimization problem:
其可以由该特征值问题解得,其中为分块矩阵,其第(i,j)块为 同时:which can be determined by the eigenvalue problem solved, in which is a block matrix, and its (i,j)th block is at the same time:
为多元特征值,为单位矩阵; is the multivariate eigenvalue, is the unit matrix;
(4)对于训练集中的原始高分辨率图像集和它对应的m个低分辨率视图的图像集其中d0为高分辨率人脸图像列向量的维度,每个视图的样本数量为n个,其中所有的低分辨率视图都利用双三次插值算法上采样到了和高分辨率视图同样的尺寸;(4) For the original high-resolution image set in the training set and its corresponding image set of m low-resolution views where d 0 is the dimension of the high-resolution face image column vector, the number of samples in each view is n, and all the low-resolution views are upsampled to the same size as the high-resolution views using the bicubic interpolation algorithm;
(5)中心化各个图像块向量其中μi表示第i个分辨率视图人脸图像的均值,利用PCA提取各个视图的主成分特征 其中Pi为第i个视图的主成分系数;(5) Centralize each image block vector where μ i represents the mean of the face image of the i-th resolution view, and PCA is used to extract the principal component features of each view where P i is the principal component coefficient of the ith view;
(6)利用FMPLS计算投影方向将各个视图的主成分特征投影到潜在子空间中: (6) Calculate the projection direction using FMPLS Project the principal component features of each view into the latent subspace:
作为本发明的进一步限定,所述步骤2具体包括以下步骤:As a further limitation of the present invention, the step 2 specifically includes the following steps:
(1)对于输入的第i种低分辨率视图的低分辨率人脸通过提取其主成分特征由得出潜在FMPLS特征;(1) For the low-resolution face of the input i-th low-resolution view pass Extract its principal component features Depend on Derive potential FMPLS features;
(2)对于从Ci中搜索其k近邻重建得到最小化如下重建误差以得到权重系数 (2) For Search for its k nearest neighbors from C i to reconstruct Minimize the reconstruction error as follows to get the weight coefficients
将权重应用到C0中相应的高分辨率特征计算输入的低分辨率人脸图像的高分辨率重建特征 will weight applied to the corresponding high-resolution features in C0 Compute high-resolution reconstructed features of an input low-resolution face image
与现有技术相比,本发明的有益效果在于:本发明先抽取多视图人脸图像的主成分特征,再利用分数阶多集偏最小二乘重新估计组内及组间协方差矩阵并计算投影方向,提取潜在相干特征,通过邻域重建恢复输入图像的高分辨率特征,最后利用最近邻分类器预测人脸标签,输出超分辨率人脸识别结果,利用分数阶多集偏最小二乘,同时学习不同分辨率人脸视图之间的多种特定分辨率的映射;解决了低分辨率人脸图像因信息不足、姿态变化、光照影响等因素而给人脸识别带来的困难,以及传统人脸超分辨率算法无法同时处理多种分辨率人脸图像输入的问题,本发明利用分数阶思想重新估计组内及组间协方差矩阵以减少训练样本不足或噪声等因素带来的影响,在多视图场景下能够有效提升处理效率以及人脸识别率,并在训练样本较少的场景下更为稳定。Compared with the prior art, the beneficial effects of the present invention are: the present invention first extracts the principal component features of the multi-view face image, and then uses the fractional order multi-set partial least squares to re-estimate the intra-group and inter-group covariance matrices and calculate Projection direction, extract potential coherent features, restore high-resolution features of the input image through neighborhood reconstruction, and finally use the nearest neighbor classifier to predict face labels, output super-resolution face recognition results, and use fractional multi-set partial least squares , while learning multiple resolution-specific mappings between face views of different resolutions; solving the difficulties in face recognition caused by low-resolution face images due to insufficient information, pose changes, lighting effects, and other factors, and The traditional face super-resolution algorithm cannot deal with the problem of inputting face images with multiple resolutions at the same time. The present invention uses the fractional order idea to re-estimate the intra-group and inter-group covariance matrices to reduce the influence of factors such as insufficient training samples or noise. , which can effectively improve processing efficiency and face recognition rate in multi-view scenarios, and is more stable in scenarios with fewer training samples.
附图说明Description of drawings
图1是本发明的实现流程图。FIG. 1 is a flow chart of the implementation of the present invention.
具体实施方式Detailed ways
如图1所示的一种基于分数阶多集偏最小二乘的人脸图像超分辨率识别方法,包括以下步骤:As shown in Figure 1, a face image super-resolution recognition method based on fractional multi-set partial least squares includes the following steps:
步骤1在训练阶段利用训练集学习不同分辨率视图之间的相关关系,使用PCA对图像降维,利用分数阶思想重新估计组内及组间协方差矩阵,计算FMPLS投影矩阵,将主成分特征投影到FMPLS的一致相干子空间;Step 1 In the training phase, use the training set to learn the correlation between views of different resolutions, use PCA to reduce the dimension of the image, use the fractional order idea to re-estimate the covariance matrix within and between groups, calculate the FMPLS projection matrix, and convert the principal component features. Projection to a consistent coherent subspace of FMPLS;
(1)对于m个多维中心化随机变量其中di表示xi的维度,FMPLS目的为求解如下的最优化问题,以寻找一组线性变换 (1) For m multidimensional centralized random variables where d i represents the dimension of xi , and the purpose of FMPLS is to solve the following optimization problem to find a set of linear transformations
其中Tr(·)表示矩阵的迹,为经过分数阶重新估计过的协方差矩阵:where Tr( ) represents the trace of the matrix, is the fractionally re-estimated covariance matrix:
(2)对于利用分数阶重新估计协方差矩阵,使用特征值分解分解组内协方差矩阵并用分数阶参数αi为指数进行重新估计:(2) For the fractional re-estimation of the covariance matrix, use the eigenvalue decomposition to decompose the intra-group covariance matrix and use the fractional parameter α i as the index to re-estimate:
其中Qi为组内协方差Sii的特征向量组成的矩阵,是以分数阶参数αi为指数的特征值对角矩阵,ri=rank(Sii)且分数阶参数αi满足0≤αi≤1;使用奇异值分解分解组间协方差矩阵并用分数阶参数βij重新估计:where Q i is the matrix composed of the eigenvectors of the within-group covariance S ii , is a diagonal matrix of eigenvalues with fractional parameter α i as an index, r i =rank(S ii ) and fractional parameter α i satisfies 0≤α i ≤1; use singular value decomposition to decompose the covariance matrix between groups and use fractional The order parameter β ij is re-estimated:
其中Uij和Vij分别是左右奇异值向量组成的矩阵,是以分数阶参数βij为指数的奇异值对角矩阵,rij=rank(Sij)且分数阶参数βij满足0≤βij≤1。where U ij and V ij are matrices composed of left and right singular value vectors, respectively, is a diagonal matrix of singular values with the fractional-order parameter β ij as an index, r ij =rank(S ij ) and the fractional-order parameter β ij satisfies 0≤β ij ≤1.
(3)对于该最优化问题,利用递归方法求解,假设前k-1个方向已经得到,其中k≤d,对于第k个方向通过求解如下最优化问题得到:(3) For this optimization problem, use the recursive method to solve it, assuming the first k-1 directions has been obtained, where k≤d, for the kth direction It is obtained by solving the following optimization problem:
其可以由该特征值问题解得,其中为分块矩阵,其第(i,j)块为 同时:which can be determined by the eigenvalue problem solved, in which is a block matrix, and its (i,j)th block is at the same time:
为多元特征值,为单位矩阵。 is the multivariate eigenvalue, is the identity matrix.
(4)对于训练集中的原始高分辨率图像集和它对应的m个低分辨率视图的图像集其中d0为高分辨率人脸图像列向量的维度,每个视图的样本数量为n个,其中所有的低分辨率视图都利用双三次插值算法上采样到了和高分辨率视图同样的尺寸;(4) For the original high-resolution image set in the training set and its corresponding image set of m low-resolution views where d 0 is the dimension of the high-resolution face image column vector, the number of samples in each view is n, and all the low-resolution views are upsampled to the same size as the high-resolution views using the bicubic interpolation algorithm;
(5)中心化各个图像块向量其中μi表示第i个分辨率视图人脸图像的均值,利用PCA提取各个视图的主成分特征 其中Pi为第i个视图的主成分系数;(5) Centralize each image block vector where μ i represents the mean of the face image of the i-th resolution view, and PCA is used to extract the principal component features of each view where P i is the principal component coefficient of the ith view;
(6)利用FMPLS计算投影方向将各个视图的主成分特征投影到潜在子空间中: (6) Calculate the projection direction using FMPLS Project the principal component features of each view into the latent subspace:
步骤2在测试阶段,提取输入的多种低分辨率图像的主成分特征,并投影到相应的FMPLS子空间,通过邻域重建策略重建出输入的低分辨率图像的高分辨率特征;Step 2: In the test phase, extract the principal component features of various input low-resolution images, and project them into the corresponding FMPLS subspace, and reconstruct the high-resolution features of the input low-resolution images through the neighborhood reconstruction strategy;
(1)对于输入的第i种低分辨率视图的低分辨率人脸通过提取其主成分特征由得出潜在FMPLS特征;(1) For the low-resolution face of the input i-th low-resolution view pass Extract its principal component features Depend on Derive potential FMPLS features;
(2)对于从Ci中搜索其k近邻重建得到最小化如下重建误差以得到权重系数 (2) For Search for its k nearest neighbors from C i to reconstruct Minimize the reconstruction error as follows to get the weight coefficients
将权重应用到C0中相应的高分辨率特征计算输入的低分辨率人脸图像的高分辨率重建特征该重建出的高分辨率特征便可通过最近邻分类器用于人脸识别工作。will weight applied to the corresponding high-resolution features in C0 Compute high-resolution reconstructed features of an input low-resolution face image The reconstructed high-resolution features can then be used for face recognition through the nearest neighbor classifier.
步骤3利用最近邻分类器预测人脸标签,输出超分辨率人脸识别结果。Step 3 uses the nearest neighbor classifier to predict the face label, and outputs the super-resolution face recognition result.
为了测试本发明的有效性,分别在CMU PIE人脸数据库以及AT&T人脸数据库与其他六种方法进行对比测试。CMU PIE数据库包含68个不同人的人脸图像,每人24张,包含不同视角、光线条件、角度等,选择每人的偶数张人脸图像作为训练,剩下的作为测试。高分辨率人脸图像大小为64×64,下采样倍数分别为2倍、4倍以及8倍,其分辨率分别为32×32、16×16以及8×8。AT&T人脸数据库包含40个人的人脸图像,每人10张,分别选择每人的前Ntrain张人脸作为训练集进行实验(Ntrain分别为7、5、3),每次实验剩下的人脸作为测试。原始高分辨率为112×92,下采样倍数为2倍、4倍以及8倍,对应低分辨率为56×46、28×23以及14×12。In order to test the effectiveness of the present invention, comparative tests are performed on the CMU PIE face database and the AT&T face database with other six methods. The CMU PIE database contains 68 face images of different people, 24 for each person, including different viewing angles, light conditions, angles, etc., and the even-numbered face images of each person are selected as training, and the rest are used as testing. The size of the high-resolution face image is 64×64, the downsampling multiples are 2 times, 4 times and 8 times, and the resolutions are 32×32, 16×16 and 8×8, respectively. The AT&T face database contains 40 face images, 10 for each person, and the first N train faces of each person are selected as the training set for experiments (N trains are 7, 5, and 3 respectively), and the rest of each experiment is left. face as a test. The original high resolution is 112×92, the downsampling multiples are 2×, 4×, and 8×, and the corresponding low resolutions are 56×46, 28×23, and 14×12.
实验1基于CMU PIE人脸数据库的超分辨率识别对比实验Experiment 1 Comparison experiment of super-resolution recognition based on CMU PIE face database
在本实验中,本发明的邻域大小设置为40,保留95维FMPLS特征,同时分数阶参数α和β分别为0.4和0.8。所得人脸识别对比结果如表1所示,可以看出本发明在每种分辨率视图下都取得了最高的人脸识别率。In this experiment, the neighborhood size of the present invention is set to 40, 95-dimensional FMPLS features are retained, and the fractional-order parameters α and β are 0.4 and 0.8, respectively. The obtained face recognition comparison results are shown in Table 1. It can be seen that the present invention has achieved the highest face recognition rate under each resolution view.
表1七种方法在CMU PIE人脸数据库上不同缩放倍数的人脸识别率(%)Table 1. Face recognition rates (%) of seven methods on the CMU PIE face database with different scaling factors
实验2基于AT&T人脸数据库的超分辨率识别对比实验Experiment 2 Comparison experiment of super-resolution recognition based on AT&T face database
在本实验中,本发明的邻域大小设置为50,保留75维FMPLS特征,当使用每人前7张图片训练时,分数阶参数α=0.2,β=1;当使用每人前5张图片训练时,α=0.4,β=1;当使用每人前3张图片训练时,α=0.2,β=0.8。表2记录了七种方法在AT&T人脸数据库上不同缩放倍数以及不同训练样本数量情况下的人脸识别率。从表格中可以看出本发明的人脸识别率在三种不同训练样本情况下以及不同缩放倍数下都优于对比方法。In this experiment, the neighborhood size of the present invention is set to 50, and the 75-dimensional FMPLS feature is retained. When using the first 7 pictures of each person to train, the fractional parameters α=0.2, β=1; when using the first 5 pictures of each person to train , α=0.4, β=1; when using the first 3 images of each person to train, α=0.2, β=0.8. Table 2 records the face recognition rates of the seven methods on the AT&T face database with different scaling factors and different numbers of training samples. It can be seen from the table that the face recognition rate of the present invention is better than the comparison method in the case of three different training samples and different zoom factors.
表2七种方法在AT&T人脸数据库上不同缩放倍数以及不同训练样本数下的人脸识别率(%)Table 2. Face recognition rate (%) of seven methods on AT&T face database with different scaling factors and different training samples
综上所述,本发明提出了一种基于分数阶多集偏最小二乘的人脸图像超分辨率识别方法,本发明利用分数阶多集偏最小二乘,可以同时学习不同分辨率人脸视图之间的多种特定分辨率的映射,同时借助分数阶思想,重新估计协方差矩阵,以减少由样本数量不足、噪声等因素带来的影响,实验结果表明,本发明的识别效果在多视图情况下表现较为出色,同时在训练样本减少的场景下更为稳定。In summary, the present invention proposes a face image super-resolution recognition method based on fractional multi-set partial least squares. The present invention uses fractional multi-set partial least squares to simultaneously learn faces of different resolutions. Various specific resolution mappings between views, and at the same time, with the help of fractional order thinking, the covariance matrix is re-estimated to reduce the influence of factors such as insufficient number of samples and noise. The experimental results show that the recognition effect of the present invention is more It performs better in the view case, and is more stable in scenarios with fewer training samples.
本发明并不局限于上述实施例,在本发明公开的技术方案的基础上,本领域的技术人员根据所公开的技术内容,不需要创造性的劳动就可以对其中的一些技术特征作出一些替换和变形,这些替换和变形均在本发明的保护范围内。The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solutions disclosed in the present invention, those skilled in the art can make some substitutions and modifications to some of the technical features according to the disclosed technical contents without creative work. Modifications, replacements and modifications are all within the protection scope of the present invention.
Claims (3)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010473054.5A CN111611962A (en) | 2020-05-29 | 2020-05-29 | A face image super-resolution recognition method based on fractional multi-set partial least squares |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202010473054.5A CN111611962A (en) | 2020-05-29 | 2020-05-29 | A face image super-resolution recognition method based on fractional multi-set partial least squares |
Publications (1)
Publication Number | Publication Date |
---|---|
CN111611962A true CN111611962A (en) | 2020-09-01 |
Family
ID=72196511
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202010473054.5A Pending CN111611962A (en) | 2020-05-29 | 2020-05-29 | A face image super-resolution recognition method based on fractional multi-set partial least squares |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN111611962A (en) |
Cited By (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113537086A (en) * | 2021-07-20 | 2021-10-22 | 科大讯飞股份有限公司 | Image recognition method and device, electronic equipment and storage medium |
CN113887509A (en) * | 2021-10-25 | 2022-01-04 | 济南大学 | A Fast Multimodal Video Face Recognition Method Based on Image Collection |
CN114898423A (en) * | 2022-04-01 | 2022-08-12 | 东南大学 | Face recognition method based on improved recursive least square method |
CN115205941A (en) * | 2022-07-13 | 2022-10-18 | 山西大学 | Kinship Verification Method Based on Generalized Multi-View Graph Embedding |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101697197A (en) * | 2009-10-20 | 2010-04-21 | 西安交通大学 | Method for recognizing human face based on typical correlation analysis spatial super-resolution |
CN106096547A (en) * | 2016-06-11 | 2016-11-09 | 北京工业大学 | A kind of towards the low-resolution face image feature super resolution ratio reconstruction method identified |
-
2020
- 2020-05-29 CN CN202010473054.5A patent/CN111611962A/en active Pending
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN101697197A (en) * | 2009-10-20 | 2010-04-21 | 西安交通大学 | Method for recognizing human face based on typical correlation analysis spatial super-resolution |
CN106096547A (en) * | 2016-06-11 | 2016-11-09 | 北京工业大学 | A kind of towards the low-resolution face image feature super resolution ratio reconstruction method identified |
Non-Patent Citations (2)
Title |
---|
YUN-HAO YUAN 等: ""Fractional-order embedding multiset canonical correlations with applications to multi-feature fusion and recognition"" * |
YUN-HAO YUAN 等: ""Learning Simultaneous Face Super-Resolution Using Multiset Partial Least Squares"" * |
Cited By (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN113537086A (en) * | 2021-07-20 | 2021-10-22 | 科大讯飞股份有限公司 | Image recognition method and device, electronic equipment and storage medium |
CN113537086B (en) * | 2021-07-20 | 2024-08-27 | 科大讯飞股份有限公司 | Image recognition method, device, electronic equipment and storage medium |
CN113887509A (en) * | 2021-10-25 | 2022-01-04 | 济南大学 | A Fast Multimodal Video Face Recognition Method Based on Image Collection |
CN113887509B (en) * | 2021-10-25 | 2022-06-03 | 济南大学 | A Fast Multimodal Video Face Recognition Method Based on Image Collection |
CN114898423A (en) * | 2022-04-01 | 2022-08-12 | 东南大学 | Face recognition method based on improved recursive least square method |
CN114898423B (en) * | 2022-04-01 | 2024-12-24 | 东南大学 | A face recognition method based on improved recursive least squares method |
CN115205941A (en) * | 2022-07-13 | 2022-10-18 | 山西大学 | Kinship Verification Method Based on Generalized Multi-View Graph Embedding |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN111611962A (en) | A face image super-resolution recognition method based on fractional multi-set partial least squares | |
CN106952228B (en) | A single image super-resolution reconstruction method based on image non-local self-similarity | |
US8842891B2 (en) | Ultra-low dimensional representation for face recognition under varying expressions | |
US8483492B2 (en) | Method and apparatus for signal detection, classification and estimation from compressive measurements | |
Shi et al. | Exploiting multi-scale parallel self-attention and local variation via dual-branch transformer-CNN structure for face super-resolution | |
CN110400276B (en) | Hyperspectral image denoising method and device | |
CN113379597A (en) | Face super-resolution reconstruction method | |
Ullah et al. | Low resolution face recognition using enhanced SRGAN generated images | |
Dong et al. | Low-rank laplacian-uniform mixed model for robust face recognition | |
CN115588135A (en) | Unsupervised untrained hyperspectral image change detection method | |
CN113052016B (en) | A face super-resolution method based on multi-scale attention residual and equivariant mapping | |
Chen et al. | Eigen-patch: Position-patch based face hallucination using eigen transformation | |
Zhang et al. | Dyna-depthformer: Multi-frame transformer for self-supervised depth estimation in dynamic scenes | |
Zhang et al. | Learning-based face hallucination in DCT domain | |
CN111292237B (en) | Face image super-resolution reconstruction method based on two-dimensional multi-set partial least square | |
Li et al. | H-VFI: hierarchical frame interpolation for videos with large motions | |
Chen et al. | Video foreground detection algorithm based on fast principal component pursuit and motion saliency | |
CN111275624B (en) | Face image super-resolution reconstruction and recognition method based on multi-set canonical correlation analysis | |
CN116935488A (en) | An end-to-end action recognition method based on key points and convolutional neural network | |
CN111242082B (en) | Face super-resolution reconstruction and recognition method based on fractional-order orthogonal partial least squares | |
CN111292238B (en) | A Face Image Super-resolution Reconstruction Method Based on Orthogonal Partial Least Squares | |
Sharan et al. | RepAr-Net: Re-parameterized encoders and attentive feature arsenals for fast video denoising | |
CN104156698B (en) | Face identification method and device | |
CN115393491A (en) | Ink video generation method and device based on instance segmentation and reference frame | |
Hao et al. | Dsmisr: Differential siamese multi-scale attention network for iris image super resolution |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
RJ01 | Rejection of invention patent application after publication | ||
RJ01 | Rejection of invention patent application after publication |
Application publication date: 20200901 |