CN113378415A - Multimedia data self-adaptive recovery method and device based on local and global constraints - Google Patents

Multimedia data self-adaptive recovery method and device based on local and global constraints Download PDF

Info

Publication number
CN113378415A
CN113378415A CN202110925533.0A CN202110925533A CN113378415A CN 113378415 A CN113378415 A CN 113378415A CN 202110925533 A CN202110925533 A CN 202110925533A CN 113378415 A CN113378415 A CN 113378415A
Authority
CN
China
Prior art keywords
data
local
matrix
global
noise
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110925533.0A
Other languages
Chinese (zh)
Other versions
CN113378415B (en
Inventor
郭丽
刘知贵
张小乾
李理
付聪
吴均
喻琼
张活力
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Southwest University of Science and Technology
Original Assignee
Southwest University of Science and Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Southwest University of Science and Technology filed Critical Southwest University of Science and Technology
Priority to CN202110925533.0A priority Critical patent/CN113378415B/en
Publication of CN113378415A publication Critical patent/CN113378415A/en
Application granted granted Critical
Publication of CN113378415B publication Critical patent/CN113378415B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2111/00—Details relating to CAD techniques
    • G06F2111/04—Constraint-based CAD
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2119/00—Details relating to the type or aim of the analysis or the optimisation
    • G06F2119/10—Noise analysis or noise optimisation

Abstract

The invention discloses a multimedia data self-adaptive recovery method and a device based on local and global constraints, which comprises the following steps: extracting feature data of the multimedia data, wherein the feature data comprise global features and local features; carrying out low-rank constraint on the characteristic data by using the constructed data global constraint model to obtain clean data; and extracting noise in the characteristic data by using the constructed noise data local similarity measurement model to obtain noise data. The invention aims to provide a multimedia data self-adaptive recovery method and device based on local and global constraints, which can improve the data recovery quality, especially have more obvious recovery effect on seriously damaged images, videos and other multimedia data, can provide more accurate image and video target and background information, and achieve more ideal denoising effect.

Description

Multimedia data self-adaptive recovery method and device based on local and global constraints
Technical Field
The invention relates to the technical field of data recovery, in particular to a multimedia data self-adaptive recovery method and device based on local and global constraints.
Background
Multimedia data such as images, videos and audios provide a large amount of colorful information for daily life and work, but the multimedia data structures often lead to a large amount of noise caused by data collectors, natural conditions, human factors and the like, so that the essential structure of the data is damaged, and certain difficulty is caused for data processing. For example, image data is often interfered by different light changes, occlusion, image corruption, gaussian noise and non-gaussian noise in practical applications; video data is often introduced with non-gaussian noise such as loss, destruction, outliers, and the like of motion trajectory features due to low acquisition frequency, mutual occlusion between moving bodies, self-occlusion of moving bodies, and the like.
At present, in order to deal with such noise present in multimedia, a Robust Principal Component Analysis (RPCA) is designed and used. The RPCA utilizes the low-rank property of clean data and the sparseness of noise data, and respectively carries out low-rank constraint and noise modeling on the clean data and the noise data, thereby realizing the separation of the clean data and the noise from the noise-containing data. Generally, the RPCA method can obtain a good effect when recovering data damaged by light shielding and gaussian noise, but has a poor data recovery effect when the data is seriously damaged and the shielding surface is large.
Disclosure of Invention
The invention aims to provide a multimedia data self-adaptive recovery method and device based on local and global constraints, which can improve the data recovery quality, especially have more obvious recovery effect on seriously damaged images, videos and other multimedia data, can provide more accurate image and video target and background information, and achieve more ideal denoising effect.
The invention is realized by the following technical scheme:
the self-adaptive multimedia data restoring method based on local and global constraints includes the following steps:
extracting feature data of the multimedia data, wherein the feature data comprise global features and local features;
carrying out low-rank constraint on the characteristic data by using the constructed data global constraint model to obtain clean data;
and extracting noise in the characteristic data by using the constructed noise data local similarity measurement model to obtain noise data.
Preferably, the data global constraint model is:
Figure 455450DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 67697DEST_PATH_IMAGE002
representing a clean data matrix, E representing a noisy data matrix,
Figure 260650DEST_PATH_IMAGE003
representing the enhanced truncation rank weighted nuclear norm of A, and r representing the number of removed larger singular values;
Figure 350966DEST_PATH_IMAGE004
representing an identity matrix of size r x r,
Figure 345598DEST_PATH_IMAGE005
representation matrix
Figure 597587DEST_PATH_IMAGE006
The trace of (a) is determined,
Figure 620381DEST_PATH_IMAGE007
representing the characteristic data, and C and D are unitary matrixes obtained by performing singular value decomposition on A respectively.
Preferably, the data global constraint model is solved by an alternating minimization method or an alternating direction multiplier method to obtain the clean data.
Preferably, the noise data local similarity metric model is:
Figure 593016DEST_PATH_IMAGE008
；
wherein the content of the first and second substances,
Figure 206269DEST_PATH_IMAGE009
is a Gaussian kernel function, and
Figure 301264DEST_PATH_IMAGE010
，
Figure 829328DEST_PATH_IMAGE011
which is indicative of the size of the kernel,
Figure 933551DEST_PATH_IMAGE012
representing the ith row of the matrix of noise data E.
Preferably, when the clean data is acquired, the method further comprises adaptively learning an affinity matrix of the feature data;
Figure 732136DEST_PATH_IMAGE013
wherein the content of the first and second substances,m is a feature data affinity matrix,
Figure 591507DEST_PATH_IMAGE014
it is shown that the affinity learning is performed,
Figure 13393DEST_PATH_IMAGE015
representing a smooth constraint on the affinity matrix, K being represented by
Figure 452464DEST_PATH_IMAGE016
A block diagonal matrix of block diagonal elements.
The multimedia data self-adaptive recovery device based on local and global constraints comprises a control module, a data acquisition module, a data processing module and a data display module;
the control module is used for controlling the working processes of the data acquisition module, the data processing module and the data display module;
the data acquisition module is used for acquiring multimedia data to be processed;
the data processing module is used for processing the multimedia data according to the multimedia data adaptive recovery method based on the local and global constraints as claimed in any one of claims 1-5;
and the data display module is used for displaying the multimedia data processed by the data processing module.
Preferably, the data processing module comprises an extraction unit, a first acquisition unit and a second acquisition unit;
the extraction unit is used for extracting feature data of the multimedia data, and the feature data comprises global features and local features;
the first acquisition unit is used for acquiring clean data; the clean data is obtained by carrying out low-rank constraint on the feature data through a constructed data global constraint model;
the second acquisition unit is used for acquiring noise data; and extracting the noise in the characteristic data by the noise data through a built noise data local similarity measurement model.
Preferably, the data global constraint model is:
Figure 774730DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 805003DEST_PATH_IMAGE002
representing a clean data matrix, E representing a noisy data matrix,
Figure 714184DEST_PATH_IMAGE003
representing the enhanced truncation rank weighted nuclear norm of A, and r representing the number of removed larger singular values;
Figure 488105DEST_PATH_IMAGE004
representing an identity matrix of size r x r,
Figure 667807DEST_PATH_IMAGE017
representation matrix
Figure 868981DEST_PATH_IMAGE018
The trace of (a) is determined,
Figure 62196DEST_PATH_IMAGE007
representing the characteristic data, and C and D are unitary matrixes obtained by performing singular value decomposition on A respectively.
Preferably, the noise data local similarity metric model is:
Figure 577491DEST_PATH_IMAGE008
；
wherein the content of the first and second substances,
Figure 467825DEST_PATH_IMAGE019
is a Gaussian kernel function, and
Figure 59474DEST_PATH_IMAGE020
，
Figure 989253DEST_PATH_IMAGE011
which is indicative of the size of the kernel,
Figure 153911DEST_PATH_IMAGE021
representing the ith row of the matrix of noise data E.
Preferably, the first obtaining unit further performs adaptive learning on an affinity matrix of the feature data when obtaining the clean data;
Figure 806741DEST_PATH_IMAGE013
wherein M is a feature data affinity matrix,
Figure 349717DEST_PATH_IMAGE022
it is shown that the affinity learning is performed,
Figure 953743DEST_PATH_IMAGE023
representing a smooth constraint on the affinity matrix, K being represented by
Figure 545261DEST_PATH_IMAGE024
A block diagonal matrix of block diagonal elements.
Compared with the prior art, the invention has the following advantages and beneficial effects:
1. the data recovery quality can be improved, particularly, the recovery effect on seriously damaged multimedia data such as images, videos and the like is more obvious, more accurate image and video target and background information can be provided, and a more ideal denoising effect is achieved;
2. the sum of all singular values is not minimized for the truncated kernel norm, only a small min is minimized (m,n)- r And the singular values are used for avoiding the interference generated by the larger singular values so as to better mine the global structure of the feature data.
Drawings
The accompanying drawings, which are included to provide a further understanding of the embodiments of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the principles of the invention. In the drawings:
FIG. 1 is a schematic flow chart of an adaptive recovery method according to the present invention;
FIG. 2 is a schematic diagram of video data recovery using the adaptive recovery method provided by the present invention;
fig. 3 is a block diagram of an adaptive recovery apparatus according to the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail below with reference to examples and accompanying drawings, and the exemplary embodiments and descriptions thereof are only used for explaining the present invention and are not meant to limit the present invention.
Example 1
The embodiment provides a multimedia data adaptive recovery method based on local and global constraints, as shown in fig. 1, including the following steps:
extracting characteristic data of the multimedia data;
the feature data in this embodiment includes global features (e.g., color features, texture features, shape features, spatial relationship features, or optical flow features of an image, where the color features include color histograms and color sets) and local features (including edges, corners, lines, curves, and regions of special attributes, etc.);
since the global features are insensitive to the changes of the direction, the size and the like of the image or the image area and cannot well capture the local feature description of the object in the image, the global features of the multimedia data are not suitable for processing the situations of image mixture and occlusion, and the local features are the features extracted from the local area of the image and comprise edges, corners, lines, curves, special attribute areas and the like.
It should be noted that how to extract feature data from multimedia data is the prior art, and therefore this application does not describe it, and the improvement point of this step is: in the prior art, most of the images are subjected to denoising processing by only extracting global features, and the scheme combines the global features and the local features and adopts a multi-mode of combining the global features and the local features to perform denoising processing on the images.
Carrying out low-rank constraint on the characteristic data by using the constructed data global constraint model to obtain clean data;
generally, the clean data matrix has a low-rank characteristic, and people often perform low-rank constraint on data in the field of data processing to mine a potential low-rank structure of the data. For example, the RPCA data recovery method and its extension method both use a standard kernel norm to perform low-rank constraint on data, and when this constraint mode performs low-rank constraint on noisy data, the rank components of the data matrix are treated the same, i.e. the influence of larger rank components and their sensitivity to noise are not considered, and it is difficult to implement low-rank representation in practical application. Therefore, in the scheme, in order to better utilize the prior characteristics of the observed data, the data is subjected to low-rank constraint by using the enhanced truncated rank weighted norm, that is: the first r larger eigenvalues are removed first, and then different weights are given to the remaining singular values according to different importance of the singular values. The method comprises the following steps:
the first step is as follows: and (3) truncating the first r larger singular values, wherein when the data matrix has a low-rank structure, noise is contained in relatively small eigenvalues, and the error of the noise is controlled by using the sum of the small eigenvalues, so that the low-rank characteristic of the data is better represented.
The second step is that: and (4) distributing the weight of the eigenvalue, endowing different weights according to the importance of the eigenvalue components, endowing the eigenvalue with higher importance with a larger weight, and endowing a smaller weight with a smaller singular value of the eigenvalue.
Specifically, the data global constraint model in the scheme is as follows:
Figure 380493DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 766475DEST_PATH_IMAGE002
representing a clean data matrix, E representing a noisy data matrix,
Figure 874109DEST_PATH_IMAGE003
representing the enhanced truncation rank weighted nuclear norm of A, and r representing the number of removed larger singular values;
Figure 306534DEST_PATH_IMAGE004
representing an identity matrix of size r x r,
Figure 917644DEST_PATH_IMAGE025
representation matrix
Figure 84314DEST_PATH_IMAGE026
The trace of (a) is determined,
Figure 148085DEST_PATH_IMAGE007
representing the characteristic data, and C and D are unitary matrixes obtained by performing singular value decomposition on A respectively.
The data global constraint model in the scheme is a non-convex model, in order to solve the model accurately at high speed, the scheme adopts an alternate minimization method or an alternate direction multiplier method to solve low-rank components contained in the multimedia data, and the low-rank components correspond to the de-noised clean data.
The method for solving the low-rank component contained in the multimedia data by adopting an alternating minimization method or an alternating direction multiplier method is the prior art, so the specific solving process is not explained; the improvement point of the step is that: in the prior art, the standard nuclear norm is mostly adopted to carry out low-rank constraint on data, and the enhanced truncation rank weighting nuclear norm is adopted to carry out low-rank constraint on the data in the schemem,n)- r And the singular values are used for avoiding the interference generated by the larger singular values so as to better mine the global structure of the feature data.
Further, in order to better enhance the low rank recovery effect and recover clean data, the scheme further utilizes the principle that the same type of data (clean data is the same type and noise data is the other type) has very high similarity, affinity learning is added on the basis of a data global constraint model, and the affinity matrix is subjected to smooth constraint to adaptively learn the affinity matrix of the noise-containing data.
The process of adaptively learning the affinity matrix of the noisy data comprises the following steps:
Figure 658570DEST_PATH_IMAGE027
wherein M is a data affinity matrix,
Figure 389765DEST_PATH_IMAGE028
it is shown that the affinity learning is performed,
Figure 461758DEST_PATH_IMAGE029
representing a smooth constraint on the affinity matrix, K is
Figure 278404DEST_PATH_IMAGE030
A block diagonal matrix of block diagonal elements.
In summary, the clean data of the present embodiment is obtained by the following formula:
Figure 329930DEST_PATH_IMAGE013
extracting noise in the characteristic data by using the constructed noise data local similarity measurement model to obtain noise data;
in the existing RPCA (robust principal component analysis) and extension methods, the L1 norm is often adopted
Figure 915632DEST_PATH_IMAGE031
Modeling data noise is generally better for linear data recovery, but it is less effective for noise modeling and data recovery when dealing with non-linear data (e.g., complex noise).In the application, most of the extracted feature data is nonlinear data, and the noise data in the multimedia data cannot be effectively removed by adopting the conventional RPCA (robust principal component analysis) and expansion method. And traditional sparse-based representation
Figure 158526DEST_PATH_IMAGE032
Compared with the denoising method, the noise data local similarity measurement model has a better data recovery effect, and can obtain higher-quality denoising data, so that the reliability of subsequent multimedia signal processing and analysis is effectively guaranteed. Specifically, the noise data local similarity metric model of the present application is shown as follows:
Figure 196889DEST_PATH_IMAGE008
wherein the content of the first and second substances,
Figure 783597DEST_PATH_IMAGE033
is a Gaussian kernel function, and
Figure 489385DEST_PATH_IMAGE034
，
Figure 903180DEST_PATH_IMAGE011
which is indicative of the size of the kernel,
Figure 694418DEST_PATH_IMAGE035
representing the ith row of the matrix of noise data E.
In summary, the present application realizes classification of noise data and clean data by the following formula, and obtains recovered clean data and noise data, as shown in fig. 2.
Figure 87747DEST_PATH_IMAGE036
。
Example 2
The embodiment provides a multimedia data self-adaptive recovery device based on local and global constraints, which comprises a control module, a data acquisition module, a data processing module and a data display module;
in this embodiment, as shown in fig. 3, the control module is configured as a microcomputer and is mainly responsible for sending signal commands and controlling terminals, that is: the data acquisition module is used for acquiring data and transmitting the data to the data processing module;
the data acquisition module is used for acquiring multimedia data to be processed;
the data processing module is used for processing the multimedia data according to the multimedia data self-adaptive recovery method based on the local constraint and the global constraint provided by the embodiment 1, removing noise data and recovering clean data;
and the data display module is used for converting the recovered clean data into corresponding video, image and audio data according to the requirements of the user and displaying the video, image and audio data on the user terminal.
Specifically, the data processing module of the present embodiment includes an extraction unit, a first acquisition unit, and a second acquisition unit;
the extraction unit is used for extracting feature data of the multimedia data, and the feature data comprises global features and local features;
a first acquisition unit configured to acquire clean data; the clean data is obtained by low-rank constraint on the feature data through a constructed data global constraint model;
wherein, the data global constraint model is as follows:
Figure 648041DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 29475DEST_PATH_IMAGE002
representing a clean data matrix, E representing a noisy data matrix,
Figure 354015DEST_PATH_IMAGE003
representing the enhanced truncation rank weighted nuclear norm of A, and r representing the number of removed larger singular values;
Figure 846307DEST_PATH_IMAGE004
representing an identity matrix of size r x r,
Figure 261108DEST_PATH_IMAGE037
representation matrix
Figure 518170DEST_PATH_IMAGE038
The trace of (a) is determined,
Figure 18422DEST_PATH_IMAGE007
representing the characteristic data, and C and D are unitary matrixes obtained by performing singular value decomposition on A respectively.
A second acquisition unit configured to acquire noise data; extracting noise in the characteristic data by the noise data through a built noise data local similarity measurement model;
the noise data local similarity measurement model is as follows:
Figure 720930DEST_PATH_IMAGE039
wherein the content of the first and second substances,
Figure 255816DEST_PATH_IMAGE033
is a Gaussian kernel function, and
Figure 415271DEST_PATH_IMAGE034
，
Figure 402819DEST_PATH_IMAGE011
which is indicative of the size of the kernel,
Figure 440176DEST_PATH_IMAGE021
representing the ith row of the matrix of noise data E.
Further, in order to better enhance the low rank recovery effect and recover clean data, the scheme further utilizes the principle that the same type of data (clean data is the same type and noise data is the other type) has very high similarity, affinity learning is added on the basis of a data global constraint model, and the affinity matrix is subjected to smooth constraint to adaptively learn the affinity matrix of the noise-containing data. The process of adaptively learning the affinity matrix of the noisy data comprises the following steps:
Figure 829569DEST_PATH_IMAGE040
wherein M is a data affinity matrix,
Figure 440153DEST_PATH_IMAGE041
it is shown that the affinity learning is performed,
Figure 914996DEST_PATH_IMAGE042
representing a smooth constraint on the affinity matrix, K is
Figure 818362DEST_PATH_IMAGE043
A block diagonal matrix of block diagonal elements.
To sum up, the clean data in this embodiment is obtained by the following formula:
Figure 577108DEST_PATH_IMAGE013
。
the above-mentioned embodiments are intended to illustrate the objects, technical solutions and advantages of the present invention in further detail, and it should be understood that the above-mentioned embodiments are merely exemplary embodiments of the present invention, and are not intended to limit the scope of the present invention, and any modifications, equivalent substitutions, improvements and the like made within the spirit and principle of the present invention should be included in the scope of the present invention.

Claims (10)

1. The method for adaptively recovering the multimedia data based on the local constraint and the global constraint is characterized by comprising the following steps of:
extracting feature data of the multimedia data, wherein the feature data comprise global features and local features;
carrying out low-rank constraint on the characteristic data by using the constructed data global constraint model to obtain clean data;
and extracting noise in the characteristic data by using the constructed noise data local similarity measurement model to obtain noise data.
2. The adaptive multimedia data recovery method based on local and global constraints according to claim 1, wherein the data global constraint model is:
Figure 249417DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 635399DEST_PATH_IMAGE002
representing a clean data matrix, E representing a noisy data matrix,
Figure 272527DEST_PATH_IMAGE003
represents the enhanced truncation rank weighted kernel norm of a, r represents the number of larger singular values removed,
Figure 870999DEST_PATH_IMAGE004
representing an identity matrix of size r x r,
Figure 278846DEST_PATH_IMAGE005
representation matrix
Figure 570150DEST_PATH_IMAGE006
The trace of (a) is determined,
Figure 915812DEST_PATH_IMAGE007
number of representation featuresAccordingly, C and D are unitary matrices obtained by singular value decomposition of a, respectively.
3. The adaptive multimedia data recovery method based on local and global constraints according to claim 2, wherein the global constraint model of the data is solved by an alternating minimization method or an alternating direction multiplier method to obtain the clean data.
4. The adaptive multimedia data restoration method based on local and global constraints according to claim 1, wherein the noise data local similarity metric model is:
Figure 442608DEST_PATH_IMAGE008
wherein the content of the first and second substances,
Figure 580329DEST_PATH_IMAGE009
is a Gaussian kernel function, and
Figure 416435DEST_PATH_IMAGE010
，
Figure 498661DEST_PATH_IMAGE011
which is indicative of the size of the kernel,
Figure 438935DEST_PATH_IMAGE012
representing the ith row of the matrix of noise data E.
5. The adaptive multimedia data recovery method based on local and global constraints according to any one of claims 1-4, further comprising performing adaptive learning on the affinity matrix of the feature data when acquiring the clean data;
Figure 306528DEST_PATH_IMAGE013
wherein M is an affinity matrix for the feature data,
Figure 64268DEST_PATH_IMAGE014
it is shown that the affinity learning is performed,
Figure 774736DEST_PATH_IMAGE015
representing a smooth constraint on the affinity matrix, K being represented by
Figure 629952DEST_PATH_IMAGE016
A block diagonal matrix of block diagonal elements.
6. The multimedia data self-adaptive recovery device based on local and global constraints is characterized by comprising a control module, a data acquisition module, a data processing module and a data display module;
the control module is used for controlling the working processes of the data acquisition module, the data processing module and the data display module;
the data acquisition module is used for acquiring multimedia data to be processed;
the data processing module is used for processing the multimedia data according to the multimedia data adaptive recovery method based on the local and global constraints as claimed in any one of claims 1-5;
and the data display module is used for displaying the multimedia data processed by the data processing module.
7. The adaptive multimedia data restoration apparatus based on local and global constraints according to claim 6, wherein the data processing module comprises an extraction unit, a first acquisition unit and a second acquisition unit;
the extraction unit is used for extracting feature data of the multimedia data, and the feature data comprises global features and local features;
the first acquisition unit is used for acquiring clean data; the clean data is obtained by carrying out low-rank constraint on the feature data through a constructed data global constraint model;
the second acquisition unit is used for acquiring noise data; and extracting the noise in the characteristic data by the noise data through a built noise data local similarity measurement model.
8. The apparatus for adaptive multimedia data restoration based on local and global constraints according to claim 7, wherein the data global constraint model is:
Figure 742265DEST_PATH_IMAGE001
wherein the content of the first and second substances,
Figure 405327DEST_PATH_IMAGE002
representing a clean data matrix, E representing a noisy data matrix,
Figure 212878DEST_PATH_IMAGE003
represents the enhanced truncation rank weighted kernel norm of a, r represents the number of larger singular values removed,
Figure 354009DEST_PATH_IMAGE004
representing an identity matrix of size r x r,
Figure 429150DEST_PATH_IMAGE005
representation matrix
Figure 404059DEST_PATH_IMAGE006
The trace of (a) is determined,
Figure 682594DEST_PATH_IMAGE007
representing the characteristic data, and C and D are unitary matrixes obtained by performing singular value decomposition on A respectively.
9. The apparatus for adaptive multimedia data restoration based on local and global constraints according to claim 7, wherein the noise data local similarity metric model is:
Figure 909307DEST_PATH_IMAGE008
wherein the content of the first and second substances,
Figure 730632DEST_PATH_IMAGE009
is a Gaussian kernel function, and
Figure 1077DEST_PATH_IMAGE010
，
Figure 907853DEST_PATH_IMAGE011
which is indicative of the size of the kernel,
Figure 908563DEST_PATH_IMAGE012
representing the ith row of the matrix of noise data E.
10. The apparatus for adaptive multimedia data restoration based on local and global constraints according to any one of claims 7-9, wherein the first obtaining unit further performs adaptive learning on the affinity matrix of the feature data when obtaining the clean data;
Figure 709029DEST_PATH_IMAGE013
wherein M is a feature data affinity matrix,
Figure 291320DEST_PATH_IMAGE014
it is shown that the affinity learning is performed,
Figure 295179DEST_PATH_IMAGE015
representing a smooth constraint on the affinity matrix, K being represented by
Figure 112963DEST_PATH_IMAGE016
A block diagonal matrix of block diagonal elements.
CN202110925533.0A 2021-08-12 2021-08-12 Multimedia data self-adaptive recovery method and device based on local and global constraints Active CN113378415B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110925533.0A CN113378415B (en) 2021-08-12 2021-08-12 Multimedia data self-adaptive recovery method and device based on local and global constraints

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110925533.0A CN113378415B (en) 2021-08-12 2021-08-12 Multimedia data self-adaptive recovery method and device based on local and global constraints

Publications (2)

Publication Number Publication Date
CN113378415A true CN113378415A (en) 2021-09-10
CN113378415B CN113378415B (en) 2021-11-02

Family

ID=77577011

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110925533.0A Active CN113378415B (en) 2021-08-12 2021-08-12 Multimedia data self-adaptive recovery method and device based on local and global constraints

Country Status (1)

Country Link
CN (1) CN113378415B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102509268A (en) * 2011-11-09 2012-06-20 北方民族大学 Immune-clonal-selection-based nonsubsampled contourlet domain image denoising method
CN103049892A (en) * 2013-01-27 2013-04-17 西安电子科技大学 Non-local image denoising method based on similar block matrix rank minimization
CN105872315A (en) * 2016-04-01 2016-08-17 广西大学 Video denoising method for mixed noise
CN105931264A (en) * 2016-04-14 2016-09-07 西安电子科技大学 Sea-surface infrared small object detection method
CN106504207A (en) * 2016-10-24 2017-03-15 西南科技大学 A kind of image processing method
CN108765330A (en) * 2018-05-21 2018-11-06 西北大学 Image de-noising method and device based on the joint constraint of global and local priori
CN109754018A (en) * 2019-01-09 2019-05-14 北京工业大学 A kind of image-recognizing method of the low-rank locality preserving projections based on F norm
CN110992292A (en) * 2019-12-09 2020-04-10 河北工业大学 Enhanced low-rank sparse decomposition model medical CT image denoising method
CN111951181A (en) * 2020-07-14 2020-11-17 浙江工业大学 Hyperspectral image denoising method based on non-local similarity and weighted truncation kernel norm
CN111951186A (en) * 2020-07-16 2020-11-17 南京邮电大学 Hyperspectral image denoising method based on low-rank and total variation constraint

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102509268A (en) * 2011-11-09 2012-06-20 北方民族大学 Immune-clonal-selection-based nonsubsampled contourlet domain image denoising method
CN103049892A (en) * 2013-01-27 2013-04-17 西安电子科技大学 Non-local image denoising method based on similar block matrix rank minimization
CN105872315A (en) * 2016-04-01 2016-08-17 广西大学 Video denoising method for mixed noise
CN105931264A (en) * 2016-04-14 2016-09-07 西安电子科技大学 Sea-surface infrared small object detection method
CN106504207A (en) * 2016-10-24 2017-03-15 西南科技大学 A kind of image processing method
CN108765330A (en) * 2018-05-21 2018-11-06 西北大学 Image de-noising method and device based on the joint constraint of global and local priori
CN109754018A (en) * 2019-01-09 2019-05-14 北京工业大学 A kind of image-recognizing method of the low-rank locality preserving projections based on F norm
CN110992292A (en) * 2019-12-09 2020-04-10 河北工业大学 Enhanced low-rank sparse decomposition model medical CT image denoising method
CN111951181A (en) * 2020-07-14 2020-11-17 浙江工业大学 Hyperspectral image denoising method based on non-local similarity and weighted truncation kernel norm
CN111951186A (en) * 2020-07-16 2020-11-17 南京邮电大学 Hyperspectral image denoising method based on low-rank and total variation constraint

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
LI, LI等: "Method to improve convergence performance of iterative learning control systems with bounded noise", 《JOURNAL OF THE FRANKLIN INSTITUTE-ENGINEERING AND APPLIED MATHEMATICS》 *
黄槿: "基于块协同法向滤波的网格保特征去噪算法", 《中国优秀硕士学位论文全文数据库 (信息科技辑)》 *

Also Published As

Publication number Publication date
CN113378415B (en) 2021-11-02

Similar Documents

Publication Publication Date Title
Liu et al. Knowledge-driven deep unrolling for robust image layer separation
CN111028177A (en) Edge-based deep learning image motion blur removing method
WO2021048607A1 (en) Motion deblurring using neural network architectures
CN108171133B (en) Dynamic gesture recognition method based on characteristic covariance matrix
CN107274365A (en) A kind of mine image intensification method based on unsharp masking and NSCT algorithms
CN113128368A (en) Method, device and system for detecting character interaction relationship
CN111898571A (en) Action recognition system and method
CN113378415B (en) Multimedia data self-adaptive recovery method and device based on local and global constraints
Murugan et al. Performance evaluation of face recognition using Gabor filter, log Gabor filter and discrete wavelet transform
CN113658065A (en) Image noise reduction method and device, computer readable medium and electronic equipment
CN109410134A (en) A kind of self-adaptive solution method based on image block classification
Singh Recognizing hand gestures for human computer interaction
CN110598647B (en) Head posture recognition method based on image recognition
CN117274059A (en) Low-resolution image reconstruction method and system based on image coding-decoding
Hou et al. Relative gradients for image lighting correction
Li et al. Real-time action recognition by feature-level fusion of depth and inertial sensor
CN114140828B (en) Real-time lightweight 2D human body posture estimation method
Jian et al. Towards reliable object representation via sparse directional patches and spatial center cues
Zhang et al. High-frequency attention residual gan network for blind motion deblurring
CN114511487A (en) Image fusion method and device, computer readable storage medium and terminal
CN112883916B (en) Video denoising method based on image block similarity prior
CN108256572B (en) Indoor visual feature classification method based on improved naive Bayes
CN112767261A (en) Non-local denoising framework for color images and videos based on generalized non-convex tensor robust principal component analysis model
Qian et al. Real-world underwater image enhancement via degradation-aware dynamic network
Li et al. An enhanced image denoising method using method noise

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant