CN110381313A - Video compress sensing reconstructing method based on LSTM network Yu the blind assessment of image group quality - Google Patents

Video compress sensing reconstructing method based on LSTM network Yu the blind assessment of image group quality Download PDF

Info

Publication number
CN110381313A
CN110381313A CN201910610758.XA CN201910610758A CN110381313A CN 110381313 A CN110381313 A CN 110381313A CN 201910610758 A CN201910610758 A CN 201910610758A CN 110381313 A CN110381313 A CN 110381313A
Authority
CN
China
Prior art keywords
image group
lstm network
frame
reconstruction
measurement vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201910610758.XA
Other languages
Chinese (zh)
Other versions
CN110381313B (en
Inventor
刘浩
魏冬
周健
田伟
陈根龙
黄荣
孙韶媛
李德敏
周武能
魏国林
廖荣生
黄震
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Donghua University
National Dong Hwa University
Original Assignee
Donghua University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Donghua University filed Critical Donghua University
Priority to CN201910610758.XA priority Critical patent/CN110381313B/en
Publication of CN110381313A publication Critical patent/CN110381313A/en
Application granted granted Critical
Publication of CN110381313B publication Critical patent/CN110381313B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The present invention relates to a kind of video compress sensing reconstructing method based on LSTM network Yu the blind assessment of image group quality, it wherein rebuilds end and receives frame measurement vector code stream, combination forms continuous image group measurement vector, multi-frame joint iterative reconstruction based on LSTM network is executed to each image group measurement vector, obtain corresponding reconstruction image group, final reconstructed frame is exported one by one, while deciding whether the parameter sets of update LSTM network according to the duration aspect that the number of iterations reaches maximum value.The present invention can model sparse prior to be combined with Data-drive mode, helps to promote the quality for rebuilding video.

Description

Video compress sensing reconstructing method based on LSTM network Yu the blind assessment of image group quality
Technical field
The present invention relates to video compress sensing reconstructing technical fields, more particularly to one kind based on LSTM network and image group The video compress sensing reconstructing method of the blind assessment of quality.
Background technique
The rise of compressed sensing technology provides a kind of novel signal acquisition and Restoration Mechanism, is managed according to compressed sensing By, it is only necessary to original signal is projected and obtains a small amount of measured value on random base, in a certain transform domain have it is sparse or The signal of nearly rarefaction representation can be restored by these measured values.In compressed sensing video communication, measurement end with again It is extremely asymmetric to build end, measurement end is information-physics emerging system, limited, signal acquisition and transmission with physics and computing resource Collaboration essential characteristics, the reconstruction end for having sufficient resources such as carries out and then needs to restore original signal under no feedback channel.
Video compress perception generallys use the communication construction of " every frame independent measurement, multi-frame joint reconstruct ", complicated by calculating Degree is transferred to from measurement end rebuilds end, and extremely simple measurement end design is highly suitable for visual sensing resource-constrained in sensing network Device.Measurement end carries out independent observation coding using identical observing matrix to every frame image of video, generates continuous frame observation Vector is sent as code stream.After reconstruction end receives code stream, it is combined into continuous image group measurement vector, multiframe connection It is different using the degree of Space-Time redundancy to close reconstruct, the speed and quality of video reconstruction is also different.
The restructuring procedure of video compress perception can not obtain original picture signal, and reconstruction property assessment is difficult to reference to original Image.The blind assessment of video quality is trained using the image pattern in typical image database, passes through supervision type pattern-recognition The blind assessment models of video features variation are established with statistical regression, the quality that can execute multiple image without original image is commented Estimate.The Video-BLIINDS and VIIDEO that Bovik et al. is proposed are two kinds of blind assessment levels of typical video quality, wherein Video-BLIINDS criterion is the frequency domain statistical model based on Space-Time natural scene, and VIIDEO criterion is based on front and back frame difference The statistical model of cloth.The blind assessment of the quality of reconstruction image group can extract the own feature of multiple image, help to restore video The structural information of signal.
Deep learning has shown the performance for the prospect of having much in machine vision and image recovery tasks, and compressed sensing is deep Degree study, which can make full use of, rebuilds end resource, preferably reconstructs the vision signal of dynamic change.Shot and long term remembers (LSTM) net Network executes the long-term sequence modeling based on attention model, more complicated Space-Time information can be expressed, based on LSTM network Deep learning mechanism helps to restore the detailed information of vision signal.
Summary of the invention
The view based on LSTM network Yu the blind assessment of image group quality that technical problem to be solved by the invention is to provide a kind of Sparse prior can be modeled and be combined with Data-drive mode by frequency compressed sensing reconstructing method, helped to be promoted and rebuild view The quality of frequency.
The technical solution adopted by the present invention to solve the technical problems is: providing one kind based on LSTM network and image group matter Measure the video compress sensing reconstructing method of blind assessment, comprising the following steps:
(1) end receiving frame measurement vector code stream is rebuild, combination forms continuous image group measurement vector；
(2) the 1st image group measurement vector GMV is utilized1Reconstruction image group training LSTM network parameter sets；
(3) for n-th image group measurement vector GMVn, the multi-frame joint iterative reconstruction based on LSTM network is executed, In, n >=2, stop condition is when the number of iterations reaches maximum value K or residual error l2Norm | | Rn,j||2Less than threshold value resMin or figure As organizing blind quality Qb nHigher than threshold value qMax, so that the recovery of n-th image group is completed, by reconstructed frame F thereinnAs final N-th of reconstructed frame；After completing the recovery of continuous α image group, if the final recovery of each image group is that the number of iterations reaches The case where just stopping to maximum value K then enters step (4)；Otherwise, subsequent multi-frame joint iterative reconstruction still uses LSTM network Parameter current set §*, and jump to step (5)；
(4) it rebuilds end and utilizes n-th image group measurement vector GMVnReconstruction image group GnTraining LSTM network；
(5) if still having image group measurement vector to be reconstructed, return step (3), continuation restores image one by one Group；Otherwise, remaining reconstructed frame F is exportedn+1、…、Fn+L-1, as final (n+1)th ..., n+L-1 reconstructed frame, complete video Reconstruct.
Each image group measurement vector includes L frame measurement vector in the step (1), wherein L >=2, each frame observation Vector contains M measured value.
For the 1st image group measurement vector GMV in the step (2)1, rebuild end and use image reconstruction algorithm extensive frame by frame Multiple 1st reconstruction image group G1={ F1,F2,...,FL, then by (G1,GMV1) it is used as reference data pair, for training LSTM The parameter sets § of network1, to obtain the parameter current set § of LSTM network*=§1。
The multi-frame joint iterative reconstruction based on LSTM network is by frame measurement vector GMV in the step (3)n(:,i) The i-th frame residual error vector R is initialized one by onen,j(:, i), and by the residual error vector R of initializationn,j(:, i) it is used as the defeated of LSTM network Enter；The LSTM network of the i-th frame image in iteration j is exported using transition matrix UBe converted to base vectorNcell is the number of LSTM network neural member；By base vector zn,j(:, i) further input into softmax Layer, it follows that the nonzero probability of each element in the i-th frame sparse spike selects the element with maximum probability and added To the supported collection of frame sparse spike；Finally, finding each frame sparse spike in iteration j one by one by least squares estimate {Sn,j(:,i)}I=1,2 ..., L。
The probability that residual error vector after rebuilding calculating successive ignition in end in the step (3) is zero according to residual error coefficient is to residual Poor coefficient is weighted, and is obtained the residual error minimization problem of a weighting, is solved this using Split Bregman iterative algorithm and ask Topic.
The blind quality of image group is assessed by Video-BLIINDS or VIIDEO criterion in the step (3).
End is rebuild in the step (4) uses image reconstruction algorithm to restore n-th of reconstruction image group G frame by framen={ Fn, Fn+1,...,Fn+L-1, by (Gn,GMVn) it is used as reference data pair, the parameter sets § of training LSTM networkn, update LSTM network Parameter current set §*=§n。
When the trained LSTM network, using LSTM network to the reconstruction image group G of trainingnSparse coding is carried out, and Rarefaction representation is carried out to data-oriented with the LSTM network, obtains coefficient matrix；Fixed coefficient matrix later successively updates LSTM Each atom of network makes it closer characterize the reconstruction image group G of trainingn。
Beneficial effect
Due to the adoption of the above technical solution, compared with prior art, the present invention having the following advantages that and actively imitating Fruit: the present invention considers the Space-Time sparse features of continuous multiple frames image, proposes and incorporates sparse prior modeling and data accumulating drive Dynamic video compress sensing reconstructing method, this method constructs LSTM network by the sparsity of image group measurement vector, in multiframe The blind assessment of image group quality is added under the basic framework of joint sparse, seriatim executes the multi-frame joint iteration based on LSTM network Reconstruct.The present invention can consider a large amount of frames simultaneously, without making linear hypothesis to object of which movement, can comprehensively reflect that object is transported Dynamic information, facilitates the structure and detailed information of restoring multiple image on the whole, to promote the quality for rebuilding video.
Detailed description of the invention
Fig. 1 is the timing diagram of image group measurement vector Yu frame measurement vector；
Fig. 2 is the overview flow chart of video compress sensing reconstructing method；
Fig. 3 is the multi-frame joint iterative reconstruction flow chart based on LSTM network.
Specific embodiment
Present invention will be further explained below with reference to specific examples.It should be understood that these embodiments are merely to illustrate the present invention Rather than it limits the scope of the invention.In addition, it should also be understood that, after reading the content taught by the present invention, those skilled in the art Member can make various changes or modifications the present invention, and such equivalent forms equally fall within the application the appended claims and limited Range.
In compressed sensing video communication, measurement end executes every frame to original video or by the sub-video frame by frame that block divides Independent measurement, and send frame measurement vector code stream.It rebuilds end and receives code stream, combination forms continuous image group measurement vector, often A image group measurement vector contains L frame measurement vector, and i indicates the frame number (1≤i≤L) in same image group, each frame Measurement vector contains M measured value,It indicates n-th image group measurement vector (n >=1), L frame measurement vector row Arrange into GMVnL column, wherein GMVn(:, i) indicate the i-th frame measurement vector.Based on continuous frame measurement vector code stream, Fig. 1 is provided The timing diagram of image group measurement vector, wherein the frame number sum L=3 that each image group measurement vector includes.It rebuilds It holds and the multi-frame joint iterative reconstruction based on shot and long term memory (LSTM) is executed to each image group measurement vector, before video One image group has stronger correlation with latter image group, rebuilds end and is adaptively determined according to the recent situation of iterative reconstruction Whether the parameter current set of LSTM network is updated.
In rebuilding video, each reconstruction image group contains L reconstructed frame, Gn={ Fn,Fn+1,...,Fn+L-1Indicate n-th A reconstruction image group, Qb nIndicate GnThe blind quality of image group, Rn,jIndicate the n-th image group residual error vector in iteration j. FnIt indicates n-th of reconstructed frame, contains N number of pixel.Sn,jIndicate reconstruction image group GnCorresponding image group is sparse in iteration j Vector.Based on LSTM network and the blind assessment of image group quality, Fig. 2 gives the overall procedure of video compress sensing reconstructing method Figure, mainly comprises the steps that
Step 1: in initialization operation, when serial number n=1.Since primitive frame can not obtain, rebuilds end and use by the 1st Image group measurement vector GMV1The reconstruction image group training LSTM network parameter of recovery.For GMV1, rebuild end and become frame by frame using complete The image reconstruction algorithm that difference minimizes restores the 1st reconstruction image group G1={ F1,F2,...,FL, then by (G1,GMV1) conduct Reference data pair, training LSTM network, obtains its parameter sets §1, parameter current set § as LSTM network*=§1。
Step 2: for n-th image group measurement vector GMVn, wherein n >=2 rebuild end and execute based on LSTM network Multi-frame joint iterative reconstruction calculates residual error in each iteration, then according to residual error coefficient be zero probability to residual error coefficient into Row weighting forms the residual error minimization problem of a weighting, and solves the problems, such as this using Split Bregman iterative algorithm.It is more The stop condition of frame Joint iteration reconstruct is that the number of iterations reaches maximum value K or residual error l2Norm | | Rn,j||2Less than threshold value ResMin or the blind quality Q of image groupb nHigher than threshold value qMax.The blind assessment of image group quality using Video-BLIINDS or VIIDEO criterion, the bigger quality of value are better.Cooperate the qMax newly introduced, K is largerly optional, and resMin is smalllyer optional.Weight It builds end and completes image group measurement vector GMVnRecovery, one by one obtain reconstructed frame { Fn+i-1=Ψ Sn,j(:,i)}I=1,2 ..., L, into And obtain reconstruction image group Gn={ Fn,Fn+1,...,Fn+L-1, and export reconstructed frame FnAs n-th final of reconstructed frame, remaining Reconstructed frame Fn+1、…、Fn+L-1Original state as the same temporal frame of subsequent image group.Adjacent image group has approximate multiframe Joint sparse characteristic, the parameter sets of LSTM network are usually in the recovery of multiple images group without updating.It is α continuous when completing When the recovery of image group, if the number of iterations has continued α image group the case where reaching maximum value and stop, continuing step 3；It is no Then, subsequent image group measurement vector still uses the parameter current set § of LSTM network*, and jump to step 4.
In the multi-frame joint iterative reconstruction of the step, L frame sparse spike would be combined into an image group measurement vector into Row integrated restoration, L reconstructed frame are exported by frame sequence and frame per second, form reconstruction image group.In video reconstruction,It is Gauss Random matrix,It is double tree wavelet transformation bases, observing matrixFor any image group measurement vector, Fig. 3 gives the multi-frame joint iterative reconstruction flow chart based on LSTM network.Image group residual error vector Rn,jThe i-th frame residual error arrow Measure Rn,j(:, i)=GMVn(:,i)-Aj·Sn,j(:, i), frame number i=1,2 ..., L, AjBeing only includes corresponding Sn,jSupport element of set The matrix of the A column of element, Sn,j(:, i) it is image group sparse spike Sn,jThe i-th frame sparse spike.L is an image group observation arrow Frame number sum in amount, i.e. Sn,jThe quantity of column.The joint sparse dependence of multiple frame measurement vectors is often gradual change, The conditional probability by calculating residual error is needed dynamically to obtain this dependence.The item that this method passes through each vector of calculating Part probability obtains this dependence, these probability are speculated using the LSTM network of data-driven, completes GMVn(:, i) and AjLeast-squares estimation.Assuming that Sn,jRespectively column are common sparse, i.e., the nonzero element of each vector appears in and other vectors Identical position, it means that frame sparse spike supported collection having the same.This method passes through the i-th frame measurement vector GMVn(:, I) the i-th frame residual error vector R is initialized one by onen,j(:, i), these residual error vectors are used as the input of LSTM network；Then using conversion Matrix U exports the LSTM network of the i-th frame in iteration jIt is transformed to base vectorIt will zn,j(:, i) softmax layers are input to, output is expressed as conditional probability, thus finds out each element in the i-th frame sparse spike Nonzero probability, select the element with maximum probability and be added to the supported collection of frame sparse spike；Later, pass through minimum Two, which multiply the estimation technique, finds the i-th frame sparse spike, while calculating the i-th new frame residual error vector, as LSTM net in next iteration The input of network.
Step 3: rebuilding end utilizes n-th image group measurement vector GMVnReconstruction image group training LSTM network parameter. Rebuild end uses the image reconstruction algorithm of total variation minimization to restore n-th of reconstruction image group G frame by framen={ Fn,Fn+1,..., Fn+L-1, by (Gn,GMVn) it is used as reference data pair, the parameter sets § of re -training LSTM networkn, and update LSTM network Parameter current set §*=§n。
In the step 3, if the number of iterations has continued α image group the case where reaching maximum value and stop, starting LSTM net The training and update of network parameter.Reconstruction image group GnAnd its corresponding image group measurement vector GMVnReference data is constituted to (Gn, GMVn).In order to find out LSTM network parameter, need to minimize the known of conditional probability that LSTM network provides and reference data pair Cross entropy cost function between probability.Training process alternately uses LSTM network to the reconstruction image of training first Group GnSparse coding is carried out, i.e., fixed LSTM network carries out rarefaction representation to data-oriented with the LSTM network, i.e. use is few as far as possible Coefficient as far as possible approximatively indicate image group measurement vector GMVn, obtain coefficient matrix；Fixed coefficient matrix later, successively more Each atom (each column of LSTM network) of new LSTM network, makes it closer characterize the reconstruction image group G of trainingn。 Serial number can just carry out again when the training process of LSTM network parameter is generally spaced longer, and α value is smaller, rebuild the quality of video It is more stable, but more computing resources need to be consumed, updated LSTM set of network parameters is used for the recovery of subsequent image group.
Step 4: n=n+1 jumps back to step 2 if rebuilding end still has image group measurement vector to be reconstructed, The above process is repeated, continues to restore subsequent image group；Otherwise, it rebuilds end and exports GnRemaining reconstructed frame Fn+1、…、 Fn+L-1, as final (n+1)th ..., n+L-1 reconstructed frame, to complete video reconstruction.
It is not difficult to find that the present invention considers the Space-Time sparse features of continuous multiple frames image, proposes involvement sparse prior and build The video compress sensing reconstructing method of mould and data accumulating driving, this method are constructed by the sparsity of image group measurement vector The blind assessment of image group quality is added in LSTM network under the sparse basic framework of multi-frame joint, seriatim executes and is based on LSTM net The multi-frame joint iterative reconstruction of network.The present invention can consider that a large amount of frames can without making linear hypothesis to object of which movement simultaneously Comprehensively reflect object motion information, facilitate the structure and detailed information of restoring multiple image on the whole, to promote weight Build the quality of video.

Claims (8)

1. a kind of video compress sensing reconstructing method based on LSTM network Yu the blind assessment of image group quality, which is characterized in that packet Include following steps:
(1) end receiving frame measurement vector code stream is rebuild, combination forms continuous image group measurement vector；
(2) the 1st image group measurement vector GMV is utilized1Reconstruction image group training LSTM network parameter sets；
(3) for n-th image group measurement vector GMVn, execute the multi-frame joint iterative reconstruction based on LSTM network, wherein and n >= 2, stop condition is when the number of iterations reaches maximum value K or residual error l2Norm | | Rn,j||2It is blind less than threshold value resMin or image group Quality Qb nHigher than threshold value qMax, so that the recovery of n-th of reconstruction image group is completed, by reconstructed frame F thereinnAs final N reconstructed frame；After completing the recovery of continuous α image group, if the final recovery of each image group is that the number of iterations reaches The case where maximum value K just stops then entering step (4)；Otherwise, subsequent multi-frame joint iterative reconstruction still uses LSTM network Parameter current set §*, and jump to step (5)；
(4) it rebuilds end and utilizes n-th image group measurement vector GMVnReconstruction image group GnTraining LSTM network；
(5) if still having image group measurement vector to be reconstructed, return step (3) continues to restore image group one by one；It is no Then, remaining reconstructed frame F is exportedn+1、…、Fn+L-1, as final (n+1)th ..., n+L-1 reconstructed frame, complete video reconstruction.
2. the video compress sensing reconstructing side according to claim 1 based on LSTM network Yu the blind assessment of image group quality Method, which is characterized in that each image group measurement vector includes L frame measurement vector in the step (1), wherein L >=2, each Frame measurement vector contains M measured value.
3. the video compress sensing reconstructing side according to claim 1 based on LSTM network Yu the blind assessment of image group quality Method, which is characterized in that for the 1st image group measurement vector GMV in the step (2)1, rebuild end and use image reconstruction frame by frame Algorithm restores the 1st reconstruction image group G1={ F1,F2,...,FL, then by (G1,GMV1) it is used as reference data pair, for instructing Practice the parameter sets § of LSTM network1, to obtain the parameter current set § of LSTM network*=§1。
4. the video compress sensing reconstructing side according to claim 1 based on LSTM network Yu the blind assessment of image group quality Method, the multi-frame joint iterative reconstruction based on LSTM network is by frame measurement vector GMV in the step (3)n(:, i) it is first one by one Beginningization the i-th frame residual error vector Rn,j(:, i), and by the residual error vector R of initializationn,j(:, i) it is used as the input of LSTM network；It uses Transition matrix U exports the LSTM network of the i-th frame image in iteration jBe converted to base vectorNcell is the number of LSTM network neural member；By base vector zn,j(:, i) further input into softmax Layer, it follows that the nonzero probability of each element in the i-th frame sparse spike selects the element with maximum probability and added To the supported collection of frame sparse spike；Finally, finding each frame sparse spike in iteration j one by one by least squares estimate {Sn,j(:,i)}I=1,2 ..., L。
5. the video compress sensing reconstructing side according to claim 4 based on LSTM network Yu the blind assessment of image group quality Method, which is characterized in that rebuild in the step (3) end calculate the residual error vector after successive ignition according to residual error coefficient be zero it is general Rate is weighted residual error coefficient, the residual error minimization problem of a weighting is obtained, using Split Bregman iterative algorithm solution The certainly problem.
6. the video compress sensing reconstructing side according to claim 1 based on LSTM network Yu the blind assessment of image group quality Method, the blind quality of image group is assessed by Video-BLIINDS or VIIDEO criterion in the step (3).
7. the video compress sensing reconstructing side according to claim 1 based on LSTM network Yu the blind assessment of image group quality Method, which is characterized in that rebuild end in the step (4) and image reconstruction algorithm is used to restore n-th of reconstruction image group G frame by framen= {Fn,Fn+1,...,Fn+L-1, by (Gn,GMVn) it is used as reference data pair, the parameter sets § of training LSTM networkn, update LSTM The parameter current set § of network*=§n。
8. the video compress sensing reconstructing side according to claim 1 based on LSTM network Yu the blind assessment of image group quality Method, which is characterized in that when the trained LSTM network, using LSTM network to the reconstruction image group G of trainingnCarry out sparse volume Code, and rarefaction representation is carried out to data-oriented with the LSTM network, obtain coefficient matrix；Fixed coefficient matrix later, successively more Each atom of new LSTM network, makes it closer characterize the reconstruction image group G of trainingn。
CN201910610758.XA 2019-07-08 2019-07-08 Video compression sensing reconstruction method based on LSTM network and image group quality blind evaluation Active CN110381313B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201910610758.XA CN110381313B (en) 2019-07-08 2019-07-08 Video compression sensing reconstruction method based on LSTM network and image group quality blind evaluation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201910610758.XA CN110381313B (en) 2019-07-08 2019-07-08 Video compression sensing reconstruction method based on LSTM network and image group quality blind evaluation

Publications (2)

Publication Number Publication Date
CN110381313A true CN110381313A (en) 2019-10-25
CN110381313B CN110381313B (en) 2021-08-31

Family

ID=68252277

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201910610758.XA Active CN110381313B (en) 2019-07-08 2019-07-08 Video compression sensing reconstruction method based on LSTM network and image group quality blind evaluation

Country Status (1)

Country Link
CN (1) CN110381313B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112085102A (en) * 2020-09-10 2020-12-15 西安电子科技大学 No-reference video quality evaluation method based on three-dimensional space-time characteristic decomposition

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102630011A (en) * 2012-03-31 2012-08-08 浙江师范大学 Compressive perceptual coding and decoding method and system in video sensor network
CN107154064A (en) * 2017-05-04 2017-09-12 西安电子科技大学 Natural image compressed sensing method for reconstructing based on depth sparse coding
CN107317583A (en) * 2017-05-18 2017-11-03 湖北工业大学 Variable step size distributed compression based on Recognition with Recurrent Neural Network perceives method for reconstructing
WO2019081937A1 (en) * 2017-10-26 2019-05-02 Gb Gas Holdings Limited Determining operating state from complex sensor data

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102630011A (en) * 2012-03-31 2012-08-08 浙江师范大学 Compressive perceptual coding and decoding method and system in video sensor network
CN107154064A (en) * 2017-05-04 2017-09-12 西安电子科技大学 Natural image compressed sensing method for reconstructing based on depth sparse coding
CN107317583A (en) * 2017-05-18 2017-11-03 湖北工业大学 Variable step size distributed compression based on Recognition with Recurrent Neural Network perceives method for reconstructing
WO2019081937A1 (en) * 2017-10-26 2019-05-02 Gb Gas Holdings Limited Determining operating state from complex sensor data

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112085102A (en) * 2020-09-10 2020-12-15 西安电子科技大学 No-reference video quality evaluation method based on three-dimensional space-time characteristic decomposition
CN112085102B (en) * 2020-09-10 2023-03-10 西安电子科技大学 No-reference video quality evaluation method based on three-dimensional space-time characteristic decomposition

Also Published As

Publication number Publication date
CN110381313B (en) 2021-08-31

Similar Documents

Publication Publication Date Title
CS Kumar et al. Depthnet: A recurrent neural network architecture for monocular depth prediction
CN110969577B (en) Video super-resolution reconstruction method based on deep double attention network
CN110119780B (en) Hyper-spectral image super-resolution reconstruction method based on generation countermeasure network
Zhang et al. Group-based sparse representation for image restoration
US20190124346A1 (en) Real time end-to-end learning system for a high frame rate video compressive sensing network
CN109241491A (en) The structural missing fill method of tensor based on joint low-rank and rarefaction representation
CN113177882B (en) Single-frame image super-resolution processing method based on diffusion model
CN107968962A (en) A kind of video generation method of the non-conterminous image of two frames based on deep learning
CN107274347A (en) A kind of video super-resolution method for reconstructing based on depth residual error network
Lecouat et al. Fully trainable and interpretable non-local sparse models for image restoration
CN107133930A (en) Ranks missing image fill method with rarefaction representation is rebuild based on low-rank matrix
CN108923984B (en) Space-time video compressed sensing method based on convolutional network
CN109949217A (en) Video super-resolution method for reconstructing based on residual error study and implicit motion compensation
CN110363068A (en) A kind of high-resolution pedestrian image generation method based on multiple dimensioned circulation production confrontation network
CN109102461A (en) Image reconstructing method, device, equipment and the medium of low sampling splits' positions perception
Zha et al. The power of triply complementary priors for image compressive sensing
CN110599585A (en) Single-image human body three-dimensional reconstruction method and device based on deep learning
CN114926336A (en) Video super-resolution reconstruction method and device, computer equipment and storage medium
CN110381313A (en) Video compress sensing reconstructing method based on LSTM network Yu the blind assessment of image group quality
CN113487481B (en) Circular video super-resolution method based on information construction and multi-density residual block
Wu et al. Blind super-resolution for remote sensing images via conditional stochastic normalizing flows
CN111882512B (en) Image fusion method, device and equipment based on deep learning and storage medium
CN114998104B (en) Super-resolution image reconstruction method and system based on hierarchical learning and feature separation
Liu et al. An end-to-end multi-scale residual reconstruction network for image compressive sensing
CN114494472A (en) Image compression method based on depth self-attention transformation network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant