CN112184571A - Robust principal component analysis method based on non-convex rank approximation - Google Patents
Robust principal component analysis method based on non-convex rank approximation Download PDFInfo
- Publication number
- CN112184571A CN112184571A CN202010959975.2A CN202010959975A CN112184571A CN 112184571 A CN112184571 A CN 112184571A CN 202010959975 A CN202010959975 A CN 202010959975A CN 112184571 A CN112184571 A CN 112184571A
- Authority
- CN
- China
- Prior art keywords
- rank
- matrix
- convex
- principal component
- component analysis
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012847 principal component analysis method Methods 0.000 title claims abstract description 18
- 239000011159 matrix material Substances 0.000 claims abstract description 58
- 238000000034 method Methods 0.000 claims abstract description 19
- 230000006870 function Effects 0.000 claims description 35
- 238000012545 processing Methods 0.000 claims description 10
- 239000013598 vector Substances 0.000 claims description 7
- 238000010801 machine learning Methods 0.000 claims description 6
- 238000011084 recovery Methods 0.000 abstract description 13
- 230000000694 effects Effects 0.000 abstract description 6
- 238000000354 decomposition reaction Methods 0.000 abstract description 2
- 230000007547 defect Effects 0.000 description 4
- 238000013459 approach Methods 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 238000000513 principal component analysis Methods 0.000 description 2
- 238000005481 NMR spectroscopy Methods 0.000 description 1
- OAICVXFJPJFONN-UHFFFAOYSA-N Phosphorus Chemical compound [P] OAICVXFJPJFONN-UHFFFAOYSA-N 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 230000011218 segmentation Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
- G06F17/10—Complex mathematical operations
- G06F17/17—Function evaluation by approximation methods, e.g. inter- or extrapolation, smoothing, least mean square method
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Mathematical Physics (AREA)
- Pure & Applied Mathematics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Analysis (AREA)
- Computational Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Algebra (AREA)
- Databases & Information Systems (AREA)
- Software Systems (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computing Systems (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a robust principal component analysis method based on non-convex rank approximation, which solves the problem that the recovery effect of a traditional robust principal component analysis method on a low-rank matrix is poor, realizes the rank approximation of a low-rank matrix in an image by constructing a non-convex rank approximation function, reconstructs an image low-rank matrix recovery model based on the non-convex rank approximation function, and solves by using an alternating direction method to realize the low-rank matrix and noise matrix decomposition of original image data.
Description
Technical Field
The invention relates to the technical field of robust principal component analysis, in particular to a robust principal component analysis method based on non-convex rank approximation.
Background
During data acquisition, factors such as environment and equipment often cause acquired data to contain noise or outliers, for example, a transformed cloud layer causes a relief image of satellite remote sensing to contain occlusion and shadow; due to the influence of light, the human face image has difference in brightness; the method is influenced by object motion, noise inherent in recording equipment, defects of an imaging system, external interference and the like, the quality of a nuclear magnetic resonance image is degraded and the like, so that image denoising, image restoration, image reconstruction and image completion become current popular research subjects, the image denoising problem and the image reconstruction problem are to remove noise and outliers contained in an image, find a low-rank part in image data, namely a background in the image, and the motion segmentation and face recognition problem restores a low-rank matrix representation in the image data through a subspace clustering method.
The low-rank matrix recovery problem can be solved by adopting a classical principal component analysis method and a robust principal component analysis method, but a plurality of problems are encountered in the solving process:
1. when a classical principal component analysis algorithm is adopted to solve a low-rank matrix recovery model, singular values of a data matrix X are calculated by means of singular value matrix decomposition solving, k main singular values and corresponding left singular vectors are determined, and main characteristics of the model are reflected by the vectors.
2. In consideration of the fact that a low-rank part of data is recovered from data containing sparse and large noise, a traditional robust principal component analysis method is provided for solving a low-rank matrix recovery model, the model is converted into a solution of a convex relaxation problem by means of a compressive sensing theory, when singular vectors of the low-rank part of an image are reasonably distributed and non-zero elements of a noise and outlier matrix are uniformly distributed, the model can well decompose a low-rank matrix and a noise matrix from a matrix of real data, but a nuclear norm in the traditional robust principal component analysis method deviates from the rank of the low-rank matrix due to a large singular value, and the error of an approximate rank is large.
3. Under certain conditions, a low-rank matrix in the traditional robust principal component analysis method has low rank and sparsity at the same time, a noise matrix has sparsity and low rank at the same time, and the traditional low-rank matrix recovery model cannot solve the problems.
Disclosure of Invention
Aiming at the defects in the prior art, the invention aims to provide a robust principal component analysis method based on non-convex rank approximation, which can more accurately and quickly recover the low-rank part in the data and effectively remove the noise and the outlier in the image.
In order to achieve the aim, the invention provides the following technical scheme that the robust principal component analysis method based on non-convex rank approximation comprises a non-convex rank approximation function and a new norm, and is characterized in that: the non-convex rank approximation function is
X in the non-convex rank approximate function is a singular value, and the new norm is
Y in the new norm*Is the kernel norm of matrix Y.
The invention is further configured to:
s1: inputting video data x;
s2: vectorizing video data frames, and forming a video data frame matrix by using vectors;
s3: carrying out iterative processing by using an alternating direction method of large-scale machine learning processing, and setting an iterative stopping condition;
s4: respectively iterating an original data matrix X, a low-rank matrix Y and a noise matrix E;
s5: and outputting the optimal solution YE.
The invention is further configured to: the non-convex rank approximate function iterative solution method adopts an alternating direction method which utilizes large-scale machine learning processing.
In summary, the invention has the following advantages: according to the invention, a non-convex rank approximate function is designed to make up the deviation of the approximate rank of the nuclear norm, a low-rank matrix recovery model is reconstructed, and a new low-rank matrix recovery model is solved by using an alternating direction method, so that the non-convex rank approximate function can recover the low-rank part in data more accurately and more quickly than the nuclear norm, noise and outliers in an image are effectively removed, and the effect of solving the image processing problems of image reconstruction, image denoising and the like is remarkable.
Drawings
FIG. 1 is a diagram of the relationship between the rank function and the singular values of a low rank matrix according to the present invention;
FIG. 2 is a detailed flow chart of the method of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings.
from the aspect of mathematics, the essence of the research on image denoising, image restoration, image reconstruction and image completion problems is low-rank matrix recovery, so that the low-rank matrix recovery model provides a unified framework for the research, and each piece of image data is arranged into a column vector to form an image data matrix X e Rm×nSuppose X can be decomposed into the following form:
X＝Y+E (1)
X,Y,E∈Rm×ny is a low rank matrix representing the low rank part of the real data, and E is a sparse matrix representing noise or outliers in the real data.
The low rank matrix is obtained by assuming that X is (X)ij)m×n∈Rm×nWith rank r and r < min { m, n }, then X is low. The row and column of the low-rank matrix contain a large amount of redundant information, which is helpful for image reconstruction and image feature extraction.
Generally, data containing noise needs to establish a rank minimization model of formula (2), where η is a positive compromise factor, | · | | survivallRepresenting a certain norm of the matrix E and P is a dictionary matrix. When P ═ I (I is the identity matrix), equation (2) is a form of a common low rank matrix recovery model. P is constructed by selecting proper bases, and different subspaces are represented by using low-rank matrixThe data extraction is performed, so that the defect that the traditional robust principal component analysis method can only process the data from the same subspace is overcome.
For equation (2), the convex approximation using the kernel norm as a rank function, and considering more robustness to noise and outliers, translates to the following model:
||Y||*representing the kernel norm of the matrix Y, | | E | | non-woven phosphor2,1Representing the (2, 1) norm of matrix E. Equation (3) can be solved by using the augmented lagrange multiplier method.
The definition of the Y nuclear norm is as follows:
if a certain sigmaiToo large value of (Y) may result in too large rank estimation of the low rank matrix, and affect rank approximation effect. A general formal model is given on the basis of a non-convex rank approximation model:
where f (·) is some form of non-convex rank approximation function, a continuous non-convex function that monotonically increases over [0, ∞); gradient function of g
Is Lipschitz continuous. For the matrix Y ∈ Rm×nLet us order
The rank approximation function can be approximately replaced by F (Y), and accordingly a model based on a non-convex rank approximation can be obtained, the non-convex rank approximation model of the general model being:
wherein Y, E ∈ Rm×n，||E||0Is l of the matrix E0Norm, η being a positive compromise factor.
Due to the fact that the rank approximation function F (Y) and the formula (5) which are more accurate than the nuclear norm are adopted, the low-rank part in the original data can be better recovered, and the application effect of the model in image processing such as image reconstruction, image denoising and image recognition is improved. Therefore, a new non-convex rank approximate function is provided, a low-rank matrix recovery model is constructed based on the function, and the model is solved by using an alternating direction method.
New non-convex rank approximation function:
mu is a balance factor, mu > 0, gamma is a convergence rate parameter of f (x), and gamma > 0.
f (x) satisfies the property of the non-convex rank approximation function, and the following New-norm can be obtained based on the function in equation (6).
Definitions 1 (New-norm) assume Y ∈ Rm×nγ > 0 is a parameter, defined:
wherein sigmai(Y) is the ith singular value of the matrix Y.
Utilizing New-norm Y of Y to count YNApproximating the rank function rank (Y) of the substitute Y, the following low rank matrix recovery model is obtained:
for equation (16), convex relaxation is used to change it to an unconstrained optimized form:
in the formula eta1And η2Is made byIn balancing the non-negative parameters of data consistency.
Equation (9) is solved by means of an alternating direction method of large-scale optimization problems that are often encountered and handled in the field of machine learning. In the k +1 th iteration, equation (9) requires solving two subproblems:
(10) the objective function of the formula is a combination of a non-convex function and a convex function, the former being a non-convex function, and the latter being a convex function, which is solved by the differential of the convex function. Thus, the (k + 1) th iteration for Y in equation (11) can be expressed as:
the objective function of equation (12) calculates the partial derivative of Y and makes it zero, resulting in:
the solution for E may be iteratively thresholded using the singular values of E, whereby the k +1 th iteration for E in equation (11) may be represented as:
Ek+1＝μ2(Xk-Yk) (14)
wherein, X0＝PHX, matrix Xk＝Yk+Ek. The iterative formula of matrix X is:
Xk+1＝Yk+1+Ek+1-PH(X-PYk+1-Ek+1)) (15)
to maintain data consistency, from Yk+1+Ek+1Minus the residual PH(X-PYk+1-Ek+1) Obtaining new Xk+1。
As shown in fig. 1, the abscissa x in fig. 1 is a singular value, the true rank is 1 when the singular value is non-zero, the rank is 0 when the singular value is zero, and the horizontal straight line represents the rank of the true low-rank matrix; the curves represent the variation of the non-convex rank approximation function f (x). With the increase of x, the function value of f (x) approaches to 1, which is close to the real rank, and f (x) is intuitively taken as a rank approximation function, so that the approximation effect is good, and when the value of γ is small, for example, γ is 0.1 or γ is 0.5, the deviation between f (x) and the real rank is large; as the value of γ increases, the f (x) approaches to the true rank more and more well, and if γ is 2, the f (x) approaches to the true rank well.
In addition, fig. 1 also shows the effect of approximating the nuclear norm rank, the definition of the nuclear norm knows that the nuclear norm is the sum of all singular values, the straight line with the slope of 1 in fig. 1 represents the simplified case when only one singular value exists, the defect of approximating the nuclear norm rank can be seen from the figure, the degree of deviation between the nuclear norm and the real rank is larger along with the increase of the singular value, and therefore, when the nuclear norm is used for approximating the rank, the approximate deviation is not negligible.
After a new non-convex rank approximation function is defined, a novel robust principal component analysis method based on non-convex rank approximation is re-established, as shown in formula (16), and a specific operation process of the method is shown in fig. 2. Firstly, inputting video data to be decomposed; secondly, vectorizing the video data frames, and forming a video data frame matrix by the vectors; thirdly, performing iteration processing by using an alternating direction method of large-scale machine learning processing, setting an iteration stopping condition, and respectively performing iteration on the original data matrix X, the low-rank matrix Y and the noise matrix E; and finally, outputting the optimal solution Y, E obtained after iteration.
The above-mentioned embodiments are merely illustrative and not restrictive, and those skilled in the art can make modifications to the embodiments without inventive contribution as required after reading the present specification, but only protected by the patent laws within the scope of the claims.
Claims (3)
1. A robust principal component analysis method based on non-convex rank approximation comprises a non-convex rank approximation function and a new norm, and is characterized in that: the non-convex rank approximation function is
X in the non-convex rank approximate function is a singular value, and the new norm is
Y in the new norm*Is the kernel norm of matrix Y.
2. The robust principal component analysis method based on non-convex rank approximation as claimed in claim 1, wherein the main flow steps are as follows:
s1: inputting video data x;
s2: vectorizing video data frames, and forming a video data frame matrix by using vectors;
s3: carrying out iterative processing by using an alternating direction method of large-scale machine learning processing, and setting an iterative stopping condition;
s4: respectively iterating an original data matrix X, a low-rank matrix Y and a noise matrix E;
s5: the optimal solution Y E is output.
3. The robust principal component analysis method based on non-convex rank approximation as claimed in claim 1, wherein: the non-convex rank approximate function iterative solution method adopts an alternating direction method which utilizes large-scale machine learning processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010959975.2A CN112184571A (en) | 2020-09-14 | 2020-09-14 | Robust principal component analysis method based on non-convex rank approximation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010959975.2A CN112184571A (en) | 2020-09-14 | 2020-09-14 | Robust principal component analysis method based on non-convex rank approximation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112184571A true CN112184571A (en) | 2021-01-05 |
Family
ID=73920890
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010959975.2A Pending CN112184571A (en) | 2020-09-14 | 2020-09-14 | Robust principal component analysis method based on non-convex rank approximation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112184571A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112767261A (en) * | 2021-01-06 | 2021-05-07 | 温州大学 | Non-local denoising framework for color images and videos based on generalized non-convex tensor robust principal component analysis model |
| CN113705336A (en) * | 2021-07-15 | 2021-11-26 | 南京林业大学 | Flexible cutting smoke robust feature extraction method |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107133930A (en) * | 2017-04-30 | 2017-09-05 | 天津大学 | Ranks missing image fill method with rarefaction representation is rebuild based on low-rank matrix |
| CN109522971A (en) * | 2018-12-18 | 2019-03-26 | 重庆大学 | A kind of CS-MRI image reconstructing method based on classification image block rarefaction representation |
-
2020
- 2020-09-14 CN CN202010959975.2A patent/CN112184571A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107133930A (en) * | 2017-04-30 | 2017-09-05 | 天津大学 | Ranks missing image fill method with rarefaction representation is rebuild based on low-rank matrix |
| CN109522971A (en) * | 2018-12-18 | 2019-03-26 | 重庆大学 | A kind of CS-MRI image reconstructing method based on classification image block rarefaction representation |
Non-Patent Citations (2)
| Title |
|---|
| 潘鹏等: "一种新的基于非凸秩近似的鲁棒主成分分析模型", 《科学技术与工程》 * |
| 袁安富等: "基于低秩矩阵的自适应边缘检测算法", 《组合机床与自动化加工技术》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112767261A (en) * | 2021-01-06 | 2021-05-07 | 温州大学 | Non-local denoising framework for color images and videos based on generalized non-convex tensor robust principal component analysis model |
| CN112767261B (en) * | 2021-01-06 | 2024-02-13 | 温州大学 | Non-local denoising frame for color image and video based on generalized non-convex tensor robust principal component analysis model |
| CN113705336A (en) * | 2021-07-15 | 2021-11-26 | 南京林业大学 | Flexible cutting smoke robust feature extraction method |
| CN113705336B (en) * | 2021-07-15 | 2024-03-19 | 南京林业大学 | Flexible cutting smoke robust feature extraction method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Moore et al. | Panoramic robust pca for foreground–background separation on noisy, free-motion camera video | |
| Yuan et al. | l0tv: A new method for image restoration in the presence of impulse noise | |
| US20130191425A1 (en) | Method for Recovering Low-Rank Matrices and Subspaces from Data in High-Dimensional Matrices | |
| CN107038730B (en) | Sparse representation image reconstruction method based on Gaussian scale structure block grouping | |
| CN111028177A (en) | Edge-based deep learning image motion blur removing method | |
| CN106097278B (en) | Sparse model, reconstruction method and dictionary training method of multi-dimensional signal | |
| CN107133923B (en) | Fuzzy image non-blind deblurring method based on adaptive gradient sparse model | |
| Pajot et al. | Unsupervised adversarial image reconstruction | |
| CN112184571A (en) | Robust principal component analysis method based on non-convex rank approximation | |
| Huang et al. | Quaternion-based dictionary learning and saturation-value total variation regularization for color image restoration | |
| Javed et al. | SBMI-LTD: Stationary background model initialization based on low-rank tensor decomposition | |
| CN114119426A (en) | Image reconstruction method and device by non-local low-rank conversion domain and full-connection tensor decomposition | |
| CN110751599B (en) | Visual tensor data completion method based on truncated nuclear norm | |
| Wang et al. | Robust matrix completion based on factorization and truncated-quadratic loss function | |
| CN108876750B (en) | Method for removing Poisson noise from image based on non-local similarity low-rank matrix | |
| CN111986121B (en) | Based on Framelet l 0 Non-blind restoration method for fuzzy image with norm constraint | |
| Silva et al. | Efficient separable filter estimation using rank-1 convolutional dictionary learning | |
| Malézieux et al. | Dictionary and prior learning with unrolled algorithms for unsupervised inverse problems | |
| CN112270650A (en) | Image processing method, system, medium, and apparatus based on sparse autoencoder | |
| Asif et al. | Low-rank tensor ring model for completing missing visual data | |
| Xu et al. | Adaptive rank estimate in robust principal component analysis | |
| Gao et al. | Self-attention underwater image enhancement by data augmentation | |
| CN109002802A (en) | Video foreground separation method and system based on adaptive robust principal component analysis | |
| CN112001938A (en) | Background separation method based on robust principal component analysis | |
| Geng et al. | Robust core tensor dictionary learning with modified gaussian mixture model for multispectral image restoration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210105 |