CN105741252A - Sparse representation and dictionary learning-based video image layered reconstruction method - Google Patents

Abstract

The invention discloses a sparse representation and dictionary learning-based video image layered reconstruction method. The main objective of the invention is to solve the problem of long consumed time in video image reconstruction in the prior art. The method includes the following steps of: (1) obtaining a sample set; (2) layering images in the sample set; (3) training the images of the sample set before and after layering so as to obtain high-resolution dictionaries and low-resolution dictionaries of the sample set before and after layering; (4) dividing an image to be reconstructed into a main region, a sub region or a region-of-non-interest; (5) reconstructing the main region according to the high-resolution dictionaries and the low-resolution dictionaries of the sample set after layering; reconstructing the sub region according to the high-resolution dictionaries and the low-resolution dictionaries of the sample set before layering; (7) reconstructing the region-of-non-interest; (8) fusing a reconstructed main region and a reconstructed sub region into a reconstructed region-of-non-interest so as to obtain a complete reconstructed image. With the method of the invention adopted, the reconstruction time of the image is reduced. The method can be used for the processing of medical images, natural images and remote sensing images.

Description

Video image grade reconstruction method based on rarefaction representation Yu dictionary learning

Technical field

The invention belongs to video and technical field of image processing, relate to the super resolution ratio reconstruction method of a kind of video image, can be used for medical image, natural image and remote sensing images etc. and generally require the occasion of high-definition picture.

Background technology

Due to the impact of the factors such as restriction and atmospheric interference of imaging system inherence build-in attribute, can cause that the problems such as image quality is poor, resolution is low occur in the single image obtained or video.How based on the video image of existing hardware condition and acquisition, recover its style as much as possible or improve its quality index such as resolution, definition, being always up the hot issue in video image scientific research and engineer applied.Super-resolution rebuilding is a kind of technology that can be effectively improved and improve video image level of resolution, and its single frames to obtaining or multiframe low-resolution image utilize the priori such as mathematical model of image to rebuild, and then obtain high-definition picture.

Current super-resolution rebuilding mainly has three kinds of methods: interpolation method, reconstruction method and the method based on study.Traditional interpolation method has arest neighbors interpolation method, bilinear interpolation and bicubic interpolation method, although interpolation method algorithm is simple, be easily achieved, but the edge rebuilding image has discontinuous, ringing effect or the shortcoming such as overall polarisation is sliding.Reconstruction method is devoted to the acquisition procedure to low-resolution image and carries out effective and reasonable modeling, the priori being formed corresponding high resolution information by regularization mode is used restraint, image super-resolution rebuilding problem is converted to the low-resolution image estimation problem to high-definition picture, is namely converted to the optimal solution problem with restriction criterion cost method.Based on the main stream approach that the super resolution ratio reconstruction method learnt is image-recovery technique field in recent years, its thought comes from machine learning.Freeman et al. proposes a kind of super resolution ratio reconstruction method based on sample, the method first passes through machine learning and high-resolution and low-resolution sample image is carried out block division, utilize Markov network that the spatial relationship of image is modeled, each piece of low-resolution image to be reconstructed seeks most suitable position in Markov grid in the model set up, and realizes super-resolution rebuilding with this.Although the method can restore more detailed information, but full images region is processed by it, it usually needs longer reconstruction time, and is not suitable for the reconstruction of the video image comprising multiple mobile object.

Summary of the invention

Present invention aims to the deficiency of above-mentioned prior art, a kind of video image grade reconstruction method based on rarefaction representation and dictionary learning is proposed, for comprising the reconstruction of the video image of multiple mobile object, can while ensureing video image main contents reconstruction quality, reduction reconstruction time, the real-time reconstruction for video lays the foundation.

The technical thought realizing the present invention is: utilize form Component Analysis method that the image in sample set is layered, utilize KSVD algorithm that the image before and after layering is trained respectively, obtain each training dictionary, utilize Snake algorithm by image division to be reconstructed for area-of-interest and region of loseing interest in, according to moving target size, area-of-interest is further divided into main region and sub-region, adopt doubledictionary learning method that main region is carried out super-resolution rebuilding, adopt individual character allusion quotation learning method that sub-region is carried out super-resolution rebuilding, adopt interpolation method that region of loseing interest in is interpolated reconstruction, the main region of converged reconstruction, sub-region and region of loseing interest in, obtain the original image rebuild.Its concrete steps include as follows:

(1) from sample database, sample set I={I is obtained_h,I_l,Represent high-resolution sample set,Represent low resolution sample set, with the high-definition picture of content same in sample set IAnd low-resolution imageConstitute sample to image

(2) utilize form Component Analysis method that the image in sample set I carries out texture layer and structural stratification, obtain high-resolution texture layer I_ht, high resolution structures layer I_hsWith low resolution texture layer I_lt, low resolution structure sheaf I_ls；

(3) utilize KSVD algorithm to the high-resolution sample image I in sample set I_hWith low resolution sample image I_lIt is trained, obtains high-resolution dictionary D_hWith low-resolution dictionary D_l；

(4) utilize KSVD algorithm that each layered image in sample set I is trained, obtain texture high-resolution dictionary D_ht, structure high-resolution dictionary D_hsWith texture low-resolution dictionary D_lt, structure low-resolution dictionary D_ls；

(5) low-resolution video single-frame images to be reconstructed is divided into area-of-interest and region of loseing interest in；

(6) area-of-interest of low-resolution video single-frame images to be reconstructed is divided into main region and sub-region；

(7) adopt doubledictionary learning method that main region carries out super-resolution rebuilding, adopt individual character allusion quotation learning method that sub-region carries out super-resolution rebuilding, adopt interpolation method that region of loseing interest in is interpolated reconstruction；

(8) what the main region of reconstruction, sub-region be fused to reconstruction loses interest in region, obtains complete reconstruction image.

The present invention has the advantage that compared with prior art

1. video image is carried out grade reconstruction by the present invention, zones of different in video image is adopted the method for reconstructing of different accuracy grade, main region is carried out the reconstruction based on doubledictionary study, sub-region is carried out the reconstruction of single dictionary learning, to loseing interest in, region is interpolated reconstruction, can improving the existing super resolution ratio reconstruction method based on dictionary learning and act on the problem that the reconstruction time brought in full images region is longer, the real-time reconstruction for video lays the foundation；

2. when the present invention extracts the area-of-interest of video image, utilize the Snake algorithm detection accurate closed contour of moving target, using comprise accurate closed contour minimum rectangular area as area-of-interest, the area-of-interest obtained can be made minimum at the same time domain comprising moving target；

3. the moving target in video image is divided into major heading and time target by the present invention according to elemental area size, utilize this parameter of elemental area to characterize the primary and secondary of target, directly effectively target can be carried out major minor sorting, can not increase again computation complexity, the reconstruction time of further downscaled video image on the whole；

4. the present invention is using the minimum rectangular area that comprises major heading in low-resolution video single-frame images to be reconstructed as main region, and when can make based on the super-resolution rebuilding algorithm of doubledictionary study, main region to be rebuild, action scope is minimum, the reconstruction time of reduction main region；

5. the present invention is when calculating the rarefaction representation of each piecemeal of main region, utilize searching algorithm search piecemeal best matching blocks in each three two field pictures in front and back, using the weighted sum of the best matching blocks rarefaction representation rarefaction representation as piecemeal, this utilize the rarefaction representation that before and after video, frame temporal correlation obtains, coefficient is more accurate, can further improve the reconstruction effect of main region in single-frame images to be reconstructed；

To sum up, low-resolution video image can be carried out grade reconstruction by the present invention effectively, while ensureing the reconstruction quality of these main contents of major heading, reduces the reconstruction time of video image, and the real-time reconstruction for video establishes circuit base.

Accompanying drawing explanation

Fig. 1 be the present invention realize general flow chart；

Fig. 2 is the texture layer rarefaction representation and the structure sheaf rarefaction representation sub-process figure that calculate main region in the present invention.

Specific embodiments

Below in conjunction with accompanying drawing 1, the step of the present invention is described in further detail:

Step 1. obtains sample set.

Pictures PASCALVOC committee provided are as sample database, and this data base includes the mankind, animal, the vehicles and indoor four big class totally 20 catalogues: wherein, animal includes bird, cat, cattle, Canis familiaris L., horse, sheep；The vehicles include aircraft, bicycle, ship, bus, car, motorcycle, train；Indoor include bottle, chair, dining table, potted plant, sofa, TV.

Under each catalogue, randomly choose 10 width images, obtain 200 width sample images.High-resolution sample set is constituted with the 200 width sample images obtainedThe 200 width sample images obtained are carried out 3 times of down-samplings respectively, obtains 200 width low-resolution images, constitute low resolution sample set with this 200 width low-resolution imageHigh-resolution sample set I_hWith low resolution sample set I_lCollectively form sample set I={I_h,I_l}；The high-definition picture of same contentAnd low-resolution imageConstitute sample to image

Step 2. utilizes form Component Analysis method that the image in sample set carries out texture layer and structural stratification.

The core of form component analysis is: be indicated sparse for image aspects optimum.Assuming that image X to be processed comprises Γ different form, namely image X comprises the layering of Γ mutually different background transparent, { X_λ, λ=1,2 ...., Γ }, X=X₁+X₂+...+X_λ+...+X_Γ, MCA method uses dictionary { T one group excessively complete₁,T₂,...,T_λ,...,T_ΓΓ the layering of image X is described, λ layer X_λDictionary T can only be used_λAtom carry out rarefaction representation, with other dictionaries T_γThe atom of (γ ≠ λ) cannot represent, so, it is possible to cross complete dictionary { T by building one group₁,T₂,...,T_λ,...,T_ΓRealize the Γ layer to image X and be layered.

Picture breakdown is two different forms by this example, is texture layer X by picture breakdown_tWith structure sheaf X_s, accordingly, it would be desirable to built complete dictionary { T_t,T_s, T_tFor describing the dictionary of image texture information, T_sFor describing the dictionary of image structure information.

Instrument for building texture dictionary has Gabor transformation, dct transform etc., and the instrument for building structure dictionary has wavelet transformation, curve wave conversion, ridgelet transform, profile wave convert etc..The selection of dictionary is generally chosen according to fidelity measurement functions or other similar approach, but this method too complex selecting optimum dictionary according to theory function, so, in a lot of image processing work, generally according to user experience, image is analyzed, it is selected to better represent the conventional conversion of texture or structure, the texture part of image and structure division are separated.This example selects but is not limited to the texture dictionary with dct transform structure image, builds the structure dictionary of image with profile wave convert.Implement step as follows:

2.1) texture dictionary is built

To 200 samples in sample set I to image I_i, i=1,2 ..., 200 make dct transform respectively, obtain high-resolution sample set I_h200 dct transform matrixes and low resolution sample set I_l200 dct transform matrixes, using the dct transform matrix dictionary as image, obtain high-resolution sample set I_h200 texture dictionariesWith low resolution sample set I_l200 texture dictionaries

2.2) structure dictionary is built

To 200 samples in sample set I to image I_i, i=1,2 ..., 200 make profile wave convert respectively, obtain high-resolution sample set I_h200 profile wave convert matrixes and low resolution sample set I_l200 profile wave convert matrixes, using the profile wave convert matrix dictionary as image, obtain high-resolution sample set I_h200 structure dictionariesWith low resolution sample set I_l200 structure dictionaries

2.3) matching pursuit algorithm is utilized to calculate optimum texture sparse coefficient and optimum structure sparse coefficient

In order to obtain the texture layer of high-resolution sample imageAnd structure sheafNeed to calculate high-resolution sample imageAt high-resolution texture dictionaryWith high resolution structures dictionaryUnder optimum rarefaction representation, namely solve following optimization problem

$\begin{array}{cc}\left\{{\mathrm{\α}}_{{\mathrm{ht}}_i}^{*},{\mathrm{\α}}_{{\mathrm{hs}}_i}^{*}\right\}=\underset{\left\{{\mathrm{\α}}_{{\mathrm{ht}}_i},{\mathrm{\α}}_{{\mathrm{hs}}_i}\right\}}{Arg}\min \left\{\left|\right|{\mathrm{\α}}_{{\mathrm{ht}}_i}|{|}_1+|\left|{\mathrm{\α}}_{{\mathrm{hs}}_i}\right|{|}_1\right\}& s.t\left|\right|I_{h_i}-T_{{\mathrm{ht}}_i}\×{\mathrm{\α}}_{{\mathrm{ht}}_i}-T_{{\mathrm{hs}}_i}\×{\mathrm{\α}}_{{\mathrm{hs}}_i}|{|}_2\≤\mathrm{\ϵ},i=1,2,\mathrm{...},200\end{array}$

Wherein ε=1.0 × 10^-6For degree of rarefication empirical value,WithThe high-resolution texture sparse coefficient respectively calculated and high resolution structures sparse coefficient,WithThe high-resolution optimum texture sparse coefficient respectively tried to achieve and high-resolution optimum structure sparse coefficient.

The algorithm solving above-mentioned optimization problem has matching pursuit algorithm, base tracing algorithm, orthogonal matching pursuit algorithm etc..Wherein, matching pursuit algorithm is a kind of greedy algorithm, and it tries to achieve sparse signal representation by being progressively similar to, and its principle is simple, it is simple to realizes, is current signal Its Sparse Decomposition most common method.Therefore, this example adopts but is not limited to matching pursuit algorithm and image carries out Its Sparse Decomposition, follow-up based on being also use matching pursuit algorithm that image is carried out Its Sparse Decomposition in the image reconstruction of dictionary learning.

To low resolution sample imageDo same process, obtain low resolution optimum texture sparse coefficient ${\mathrm{\α}}_{{\mathrm{lt}}_i}^{*},i=1,2,...,200$ With low resolution optimum structure sparse coefficient ${\mathrm{\α}}_{{\mathrm{ls}}_i}^{*},i=1,2,...,200.$

2.4) texture layer and the structural stratification of image are calculated:

2.4a) according to high-resolution texture dictionaryWith high-resolution optimum texture sparse coefficient ${\mathrm{\α}}_{{\mathrm{ht}}_i}^{*},i=1,2,...,200,$ Obtain high-resolution texture layer $I_{{\mathrm{ht}}_i}=T_{{\mathrm{ht}}_i}\×{\mathrm{\α}}_{{\mathrm{ht}}_i}^{*},i=1,2,...,200,$ And remember $I_{ht}=\{I_{{\mathrm{ht}}_i},i=1,2,...,200\};$

2.4b) according to high resolution structures dictionaryWith high-resolution optimum structure sparse coefficient ${\mathrm{\α}}_{{\mathrm{hs}}_i}^{*},i=1,2,...,200,$ Obtain high resolution structures layer $I_{{\mathrm{hs}}_i}=T_{{\mathrm{hs}}_i}\×{\mathrm{\α}}_{{\mathrm{hs}}_i}^{*},i=1,2,...,200,$ And remember $I_{hs}=\{I_{{\mathrm{hs}}_i},i=1,2,...,200\};$

2.4c) according to low resolution texture dictionaryWith low resolution optimum texture sparse coefficient ${\mathrm{\α}}_{{\mathrm{lt}}_i}^{*},i=1,2,...,200,$ Obtain low resolution texture layer $I_{{\mathrm{lt}}_i}=T_{{\mathrm{lt}}_i}\×{\mathrm{\α}}_{{\mathrm{lt}}_i}^{*},i=1,2,...,200,$ And remember $I_{lt}=\{I_{{\mathrm{lt}}_i},i=1,2,...,200\};$

2.4d) according to low resolution structure dictionaryWith low resolution optimum structure sparse coefficient ${\mathrm{\α}}_{{\mathrm{ls}}_i}^{*},i=1,2,...,200,$ Obtain low resolution structure sheaf $I_{{\mathrm{ls}}_i}=T_{{\mathrm{ls}}_i}\×{\mathrm{\α}}_{{\mathrm{ls}}_i}^{*},i=1,2,...,200,$ And remember $I_{ls}=\{I_{{\mathrm{ls}}_i},i=1,2,...,200\}.$

Step 3. utilizes KSVD algorithm that the image in sample set is trained.

Image super-resolution rebuilding based on dictionary learning, generally require the substantial amounts of sample image of training to obtain high-resolution dictionary and low-resolution dictionary, the efficiency of training dictionary is very big by the impact of dictionary atom number, therefore, selects a method that can effectively reduce dictionary atom number particularly significant.

The method of dictionary learning is broadly divided into two big classes: non-supervisory dictionary learning and supervision dictionary learning.Non-supervisory dictionary learning is intended to study one and has the good dictionary representing ability, and supervision dictionary learning, owing to considering the identification of dictionary, is usually used in computer identification mission.Based in the image super-resolution rebuilding of dictionary learning, it is necessary to obtain the optimum rarefaction representation of image, and good dictionary that corresponding rarefaction representation can be made to have is higher openness, therefore the method that this example selects non-supervisory dictionary learning trains each tomographic image.

The representative method of non-supervisory dictionary learning has MOD method and KSVD method, the object function of two kinds of method optimizations is identical, but when utilizing matching pursuit algorithm to carry out dictionary iteration, dictionary Global Algorithm is once obtained by MOD method, KSVD method has then carried out the optimization of sequential update row on the basis of MOD method, each iteration only updates the string of dictionary, and namely iteration only updates an atom of dictionary every time.The optimization of this sequential update of KSVD method row can effectively reduce the atom number in dictionary, and the atom after training still can all information of the initial dictionary of linear expression, so, this example adopts but is not limited to KSVD Algorithm for Training sample image.

This example utilizes KSVD Algorithm for Training high-resolution sample image the same with the step of low resolution sample image, existing to train high-resolution sample image I_hFor example, implement step as follows:

3.1) by high-resolution sample image I_hCarry out overlap partition

By high-resolution sample image I_hIn every piece imageCarrying out overlap partition according to array scan mode, piecemeal is sized to 9 × 9 block of pixels, and horizontally and vertically each overlapping pixel in direction, obtains high-resolution sample block collectionWherein,Represent high-resolution sample image I_hA sample block, m=1,2 ..., M, M represents high-resolution sample image I_hPiecemeal number.

3.2) high-resolution dictionary D is built_hInitial value

Take high-resolution sample block collection Y_hIn front 1024 sample block, it is made dct transform, obtain the dct transform matrix of 1024 9 × 9 sizes, the dct transform matrix of each 9 × 9 sizes is opened into column vector, obtaining 1024 length is the column vector of 81, by 1024 column vectors by row combination, obtain the matrix of 81 × 1024 sizes, using this matrix as high-resolution dictionary D_hInitial value.

3.3) optimum high-resolution dictionary is calculated

Utilize KSVD algorithm, by following optimization process to high-resolution dictionary D_hIt is updated, until high-resolution sample block collection Y_hAt high-resolution dictionary D_hUnder rarefaction representation be optimum rarefaction representation

$\begin{array}{ccc}{D}_h^{*}=\underset{D_h}{Arg}\min \left\{\underset{m=1}{\overset{M}{\mathrm{\Σ}}}\left|\right|y_{h_m}-D_h\×{\mathrm{\α}}_{h_m}|{|}_2^2\right\}& s.t& \left|\right|{\mathrm{\α}}_{h_m}|{|}_0\≤\mathrm{\ϵ},m=1,2,\mathrm{...},M\end{array}$

Wherein,For sample blockAt high-resolution dictionary D_hUnder rarefaction representation, ε=1.0 × 10^-6For degree of rarefication empirical value,For the optimum high-resolution dictionary tried to achieve.

Consider that low resolution sample image is obtained according to 3 times of down-samplings by high-resolution sample image, so at training low resolution sample image I_lTime, this example is by step 3.1) in piecemeal be sized such that 3 × 3 block of pixels so that step 3.2) in the dictionary initial value that obtains be sized to 9 × 1024, other operations are with step 3.1) 3.3) described in, obtain optimum low-resolution dictionary

Step 4. utilizes KSVD algorithm that each layered image in sample set is trained.

According to step 3.1) 3.3) to high-resolution texture layer I_htWith high resolution structures layer I_hsProcess, obtain texture high-resolution dictionaryWith structure high-resolution dictionary

At training low resolution texture layer I_ltWith low resolution structure sheaf I_lsTime, this example is by step 3.1) in piecemeal be sized such that 3 × 3 block of pixels, so that step 3.2) in the dictionary initial value that obtains be sized to 9 × 1024, other operations are with step 3.1) 3.3) described in, obtain optimum texture low-resolution dictionaryWith optimum structure low-resolution dictionary

Low-resolution video single-frame images to be reconstructed is divided into area-of-interest and region of loseing interest in by step 5..

Low-resolution video single-frame images to be reconstructed is carried out area-of-interest and the problem of region division of loseing interest in by this example, can regard the partition problem of foreground image and background image in machine vision as.In field of machine vision, the method separating video image foreground and background mainly has two big classes, one class is that video or image sequence are carried out background modeling, obtain background image, foreground image is obtained by video image subtracting background to be detected, this kind of method requires that input is for multiple image, conventional method has mixed Gaussian background modeling method, optical flow method, but the background that these algorithms extract still contains fuzzy moving target, by them with the moving target in area-of-interest being made clear not in this example.

Another kind of is utilize the movable information extracting directly moving target in video image, using motion target area as foreground image, except foreground image partly as background image.This kind of method generally extracts the motion vector information in video image code stream, combining form processes the binary image obtaining characterizing moving region, but the description of moving target is generally of bigger deviation by this binary image, if now can inevitably make moving target lack to some extent by the minimum rectangular area comprising moving target as foreground area.

Considering that Snake algorithm can detect the relatively precise boundary of target in broad image, this example adopts Snake algorithm to extract the area-of-interest of image to be reconstructed, it is intended to making area-of-interest is the minimum rectangular area comprising relatively precise motion target.Implement step as follows:

5.1) binary image characterizing moving target is obtained:

5.1a) from the H.264 code stream of low-resolution video single-frame images to be reconstructed, extract movable information, obtain the motion vector field MV of present frame；

5.1b) characterize grey scale pixel value by vector length, and gray value is standardized to [0,255] scope, the motion vector field MV of present frame is converted into the gray-scale map G characterizing current frame motion region；

5.1c) the gray-scale map G characterizing current frame motion region is carried out Morphological scale-space, obtain the binary image BW of moving target.

5.2) Snake algorithm is utilized to extract the relatively precise boundary of moving target:

5.2a) extract the Guan Bi outline of moving target binary image BW, obtain curve v (s)=[x (s), y (s)], x (s), the abscissa of the point on y (s) respectively contour curve and vertical coordinate, parameter s ∈ [0,1], using this curve initial profile as Snake algorithm；

5.2b) utilize Snake algorithm that curve v (s) is deformed so that it is to approach relatively precise boundary v (s) of moving target^*, this process can be converted into and seek following optimal solution

$\begin{array}{c}v{\left(s\right)}^{*}=\underset{v\left(s\right)}{Arg}\min {\∫}_0^1{E}_{snake}\left(v\left(s\right)\right)ds\\ =\underset{v\left(s\right)}{Arg}\min {\∫}_0^1\[E_{\mathrm{int}}\left(v\left(s\right)\right)+E_{image}\left(v\left(s\right)\right)+E_{con}\left(v\left(s\right)\right)\]ds\end{array}$

Wherein,Represent internal energy, v_s、v_ssThe respectively single order of v (s) and second dervative, the weighting parameter of α (s), β (s) respectively controlling curve v (s) tension force and slickness, determines curve v (s) at the extension of certain point and degree of crook；E_image(v (s)) represents the energy that image active force produces, and in order to highlight the marked feature of image, is generally designed by gradation of image, gradient information, and guiding curve v (s) approaches to edge contour；E_conRepresenting the energy that outside limits power produces, this portion of energy is set to 0 by this example；v(s)^*Relatively precise boundary for moving target.

5.3) area-of-interest and region of loseing interest in are obtained

Low-resolution video single-frame images to be reconstructed will comprise more accurate closed contour v (s) of moving target^*Minimum rectangular area extract, as area-of-interest P, by except area-of-interest partly as the region B that loses interest in.

The area-of-interest of low-resolution video single-frame images to be reconstructed is divided into main region and sub-region by step 6..

For comprising the scene of two or more multiple moving target, if all of moving target carrying out the super-resolution rebuilding of same precision regardless of primary and secondary, the reconstruction time that can make video image is long, takies too much calculating resource.Meanwhile, be no matter computer when carrying out Digital Image Processing, or when video image is observed by subjective observation person, often more concerned with the information of main target in video image.So, main target in video image is carried out high-precision super-resolution rebuilding by this example, and by-end carries out the reconstruction of relatively low precision, so can while ensureing the reconstruction quality of video image main contents, reduction reconstruction time, improves and rebuilds efficiency.

Considering shooting video or during image, focal object tends to take up more elemental area, this example is using the bigger target of elemental area in low-resolution video single-frame images to be reconstructed as major heading, and the less target of elemental area is as secondary target.Implement step as follows:

6.1) step 5.2 is utilized) more accurate closed contour v (s) of moving target that obtains^*, calculate the elemental area A={A of each target₁, A₂..., A_n..., A_N, wherein A_nRepresent the elemental area of the n-th target, n=1,2 ..., N, N represents the number of moving object in video sequences；

6.2) utilize K-means algorithm by object pixel area A={A₁, A₂..., A_n..., A_NIt being divided into two classes by size, the class that area is big is designated as major heading A_m, the class that area is little is designated as time target A_sub；

6.3) major heading A will be comprised_mMinimum rectangular area as main region P_m, except main region partly as sub-region P_sub；

6.4) record minimum rectangular area position Pos=[row, col, del in low-resolution video single-frame images to be reconstructed_row, del_col], the ranks coordinate that wherein (row, col) is minimum rectangular area top left corner pixel, del_row、del_colRespectively minimum rectangular area obtains line number and columns.

Step 7. adopts the method that doubledictionary learns that main region is rebuild.

The problem that the reconstruction time brought in full images region is longer is acted in order to improve the existing image super-resolution rebuilding method based on dictionary learning, video image is carried out grade reconstruction by this example, wherein the main region comprising major heading in video image is carried out the super-resolution rebuilding based on doubledictionary study, implements step as follows:

7.1) according to step 2) by main region P_mIt is divided into texture layer P_mtWith structure sheaf P_ms；

7.2) texture layer rarefaction representation and the structure sheaf rarefaction representation of main region are calculated:

Existing image super-resolution rebuilding method utilizes the dictionary of training directly image to be reconstructed to be processed mostly, and these methods have rebuilds effect preferably.The input of this example is video information, when single-frame images to be reconstructed is rebuild, for improving the reconstruction effect of image further, directly single-frame images to be reconstructed is not processed, and it is combined with the temporal correlation between frame of video and spatial coherence, namely select each three two field pictures in front and back adjacent with video single-frame images to be reconstructed as reference picture, by image to be reconstructed is rebuild by the reconstruction of reference picture indirectly.

With reference to Fig. 2, being accomplished by of this step

7.2a) select front and back each three two field pictures adjacent with main region place frame as reference picture, obtain reference picture collection P_r={ P_rj, j=1,2 ..., 6}, P_rjRepresent a frame reference picture；

7.2b) according to step 2) by reference picture collection P_rIt is divided into texture layer P_rt={ P_rtj, j=1,2 ..., 6} and structure sheaf P_rs={ P_rsj, j=1,2 ..., 6}, wherein P_rtjFor reference picture P_rjTexture layer, P_rsjFor reference picture P_rjStructure sheaf；

7.2c) by the texture layer P of main region_mtCarrying out overlap partition according to array scan mode, piecemeal is sized to 3 × 3 block of pixels, and horizontally and vertically each overlapping pixel in direction, obtains the piecemeal collection of main region texture layerWhereinRepresent the texture layer P of main region_mtA piecemeal, n=1,2 ..., N, N represents the texture layer P of main region_mtPiecemeal number；

7.2d) use the ParallelComputingToolbox workbox in Matlab, set up six parallel task Pro_j, j=1,2 ..., 6, each task Pro_jOnly process for reference picture texture layer P_rtjOperation；

7.2e) at task Pro_j, j=1,2 ..., 6 times, to main region texture layer P_mtEach piecemealUtilize Three Step Search Algorithm at reference picture texture layer P_rtjMiddle search best matching blocksBlock matching criterion adopts MAD criterion, namely minimizes mean absolute error function MAD (d_h,d_v):

$MAD(d_h,d_v)=\frac{1}{RC}\underset{r=1}{\overset{R}{\Σ}}\underset{c=1}{\overset{C}{\Σ}}|f(r,c)-f_{rj}(r+d_h,c+d_v)|$

Wherein, R, C respectively piecemealLine number and columns, (r c) represents that in piecemeal, coordinate is (r, pixel brightness value c), f to f_rj(r+d_h,c+d_v) represent reference picture texture layer P_rtjMiddle coordinate is (r+d_h,c+d_v) pixel brightness value, (d_h,d_v) for moving displacement vector, d_hFor horizontal direction displacement, d_vFor vertical direction displacement；

7.2f) according to texture low-resolution dictionaryCalculate match blockRarefaction representationWhereinForInverse matrix；

7.2g) calculate reference picture texture layer P_rtjIn most match blockWeight coefficientComputing formula is as follows

$w_{j_n}=\frac{1}{\sqrt{(y_{t_n}-y_{{\mathrm{rtj}}_n}^{*}){(y_{t_n}-y_{{\mathrm{rtj}}_n}^{*})}^T}};$

7.2h) to match blockRarefaction representationIt is weighted summation, obtains main region texture layer piecemealTexture layer rarefaction representationNoteTexture layer rarefaction representation for main region；

7.2i) by the structure sheaf P of main region_msAccording to step 7.2c) 7.2h) process, obtain the structure sheaf rarefaction representation of main region

7.3) according to main region P_mTexture layer rarefaction representationWith texture high-resolution dictionaryObtain the reconstruction image of main region texture layer $P_{mt}^{*}=D_{ht}^{*}\×{\mathrm{\β}}_{mt}^{*};$

7.4) the structure sheaf rarefaction representation according to main regionWith structure high-resolution dictionaryObtain the reconstruction image of main region structure sheaf $P_{ms}^{*}=D_{hs}^{*}\×{\mathrm{\β}}_{ms}^{*};$

7.5) by the reconstruction image of main region texture layerReconstruction image with main region structure sheafMerge, obtain the reconstruction image of complete main region.

Step 8. adopts the method for single dictionary learning that sub-region is rebuild.

Reconstruction time for downscaled video image, ensure the reconstruction quality of video image main contents simultaneously, video image is carried out grade reconstruction by this example, the sub-region comprising time target wherein carries out the super-resolution rebuilding based on single dictionary learning, implement step as follows in video image:

8.1) according to step 3) the optimum low-resolution dictionary that obtainsCalculate sub-region P_subRarefaction representation ${\mathrm{\β}}_{sub}=\left(D_l^{*}\right)\×P_{sub},$ WhereinForInverse matrix；

8.2) the rarefaction representation β according to sub-region_subWith step 3) the optimum high-resolution dictionary that obtainsObtain the reconstruction image of sub-region $P_{sub}^{*}=D_h^{*}\×{\mathrm{\β}}_{sub}.$

Step 9. adopts interpolation method that region of loseing interest in is rebuild.

Current super-resolution rebuilding mainly has three kinds of methods: interpolation method, reconstruction method and based on study method.Interpolation method algorithm is simple, be easily achieved, and rebuilds quality relatively other two classes method deviations of image.This example is intended to video image is carried out grade reconstruction, the area-of-interest comprising moving target carries out the reconstruction based on study, makes moving target have and rebuild effect preferably；Interpolation method is adopted to rebuild in region of loseing interest in, although so can sacrifice the reconstruction quality in region of loseing interest in, but can while ensureing the reconstruction quality of these main contents of moving target, the reconstruction time of downscaled video image.

Interpolation method mainly has arest neighbors interpolation method, bilinear interpolation and bicubic interpolation method.The arest neighbors interpolation method pixel value to each interpolation point, be taken in original image around respective point that in 4 consecutive points, Euclidean distance is the shortest one, this method simply easily realizes, amount of calculation is only small, but the picture quality after interpolation is not high, blocking artifact and sawtooth effect usually occurs.

The bilinear interpolation pixel value to each interpolation point, corresponding weights are determined with the distance of adjacent 4 points according to it, the pixel value of interpolation point is determined by the weighted sum of the pixel value of adjacent 4 points, this method amplifies the image smoothing that the image produced produces than arest neighbors interpolation method, do not have the situation that gray value is discontinuous, but owing to bilinear interpolation has the character of low pass filter, make high fdrequency component impaired, when amplification increases, image after amplification also there will be significantly block phenomenon, makes image outline thicken to a certain extent.

Bicubic interpolation method utilizes the gray value of 16 points around interpolation point to make cubic interpolation, considers not only the gray scale impact of 4 direct neighbor points, and considers the impact of gray-value variation rate between each consecutive points, rebuilds effect and be better than above two method.This example adopts but is not limited to bicubic interpolation method and the region B that loses interest in is rebuild, and its formula for interpolation is as follows:

F (i+u, j+v)=A^*B^*C^*

A^*=[S (1+u) S (u) S (1-u) S (2-u)]

$B^{*}=\left[\begin{array}{cccc}f\left(i-1,j-2\right)& f\left(i,j-2\right)& f\left(i+1,j-2\right)& f\left(i+2,j-2\right)\\ f\left(i-1,j-1\right)& f\left(i,j-1\right)& f\left(i+1,j-1\right)& f\left(i+2,j-1\right)\\ f\left(i-1,j\right)& f\left(i,j\right)& f\left(i+1,j\right)& f\left(i+2,j\right)\\ f\left(i-1,j+1\right)& f\left(i,j+1\right)& f\left(i+1,j+1\right)& f\left(i+2,j+1\right)\end{array}\right]$

C^*=[S (1+v) S (v) S (1-v) S (2-v)]^T

$S\left(w\right)=\left\{\begin{array}{c}1-2|w{|}^2+|w{|}^3,\left|w\right|<1;\\ 4-8\left|w\right|+5|w{|}^2-|w{|}^3,1\&le;\left|w\right|<2;\\ 0,\left|w\right|\&GreaterEqual;2\end{array}\right.$

Wherein, i, j are nonnegative integer, represent interpolation point row-coordinate in original image and row coordinate respectively；U, v are the floating number that (0,1) is interval, represent interpolation point and the closest pixel distance in horizontally and vertically direction respectively；(i j) represents that original image is at coordinate (i, j) pixel value at place to f；S (w) is bicubic interpolation basic function, and independent variable w is taken absolute value by independent variable w ∈ R, | w | expression.

Step 10. is by step 7) the reconstruction main region that obtains and step 8) the reconstruction sub-region that obtains, according to step 6.4) locus Pos=[row, col, the del that record_row,del_col], it being fused to step 9) reconstruction that obtains loses interest in region, obtains complete reconstruction image.

Above description is only example of the present invention, does not constitute any limitation of the invention.Obviously for those skilled in the art; after having understood present disclosure and principle; all it is likely to when without departing substantially from the principle of the invention, structure; carry out the various corrections in form and in details and change, but these based on the correction of inventive concept and change still within the claims of the present invention.

Claims (7)

1., based on a video image grade reconstruction method for rarefaction representation and dictionary learning, comprise the steps:

(1) from sample database, sample set I={I is obtained_h,I_l,Represent high-resolution sample set,Represent low resolution sample set, with the high-definition picture of content same in sample set IAnd low-resolution imageConstitute sample to image $I_i=\left\{I_{h_i},I_{l_i}\right\};$

(2) utilize form Component Analysis method that the image in sample set I carries out texture layer and structural stratification, obtain high-resolution texture layer I_ht, high resolution structures layer I_hsWith low resolution texture layer I_lt, low resolution structure sheaf I_ls；

(3) utilize KSVD algorithm to the high-resolution sample image I in sample set I_hWith low resolution sample image I_lIt is trained, obtains high-resolution dictionary D_hWith low-resolution dictionary D_l；

(4) utilize KSVD algorithm that each layered image in sample set I is trained, obtain texture high-resolution dictionary D_ht, structure high-resolution dictionary D_hsWith texture low-resolution dictionary D_lt, structure low-resolution dictionary D_ls；

(5) low-resolution video single-frame images to be reconstructed is divided into area-of-interest and region of loseing interest in；

(6) area-of-interest of low-resolution video single-frame images to be reconstructed is divided into main region and sub-region；

(7) adopt doubledictionary learning method that main region carries out super-resolution rebuilding, adopt individual character allusion quotation learning method that sub-region carries out super-resolution rebuilding, adopt interpolation method that region of loseing interest in is interpolated reconstruction；

(8) what the main region of reconstruction, sub-region be fused to reconstruction loses interest in region, obtains complete reconstruction image.

2. the video image grade reconstruction method based on rarefaction representation and dictionary learning according to claim 1, it is characterized in that: step (2) utilizes form Component Analysis method the image in sample set I carries out texture layer and structural stratification, carry out as follows:

(2a) to sample to image I_iMake dct transform, constitute high-resolution texture dictionary by the data after conversionWith low resolution texture dictionary

(2b) to sample to image I_iMake profile wave convert, constitute high resolution structures dictionary by the data after conversionWith low resolution structure dictionary

(2c) matching pursuit algorithm is utilized to calculate high-definition pictureAt high-resolution texture dictionaryWith high resolution structures dictionaryUnder optimum rarefaction representation, be converted into following optimization process by this calculating process

$\begin{array}{cc}\{{\mathrm{\&alpha;}}_{{\mathrm{ht}}_i}^{*},{\mathrm{\&alpha;}}_{{\mathrm{hs}}_i}^{*}\}=\underset{\{{\mathrm{\&alpha;}}_{{\mathrm{ht}}_i},{\mathrm{\&alpha;}}_{{\mathrm{hs}}_i}\}}{Arg}min\left\{\right|\left|{\mathrm{\&alpha;}}_{{\mathrm{ht}}_i}\right|{|}_1+\left|\right|{\mathrm{\&alpha;}}_{{\mathrm{hs}}_i}\left|{|}_1\right\}& s.t\left|\right|I_{h_i}-T_{{\mathrm{ht}}_i}\&times;{\mathrm{\&alpha;}}_{{\mathrm{ht}}_i}-T_{{\mathrm{hs}}_i}\&times;{\mathrm{\&alpha;}}_{{\mathrm{hs}}_i}|{|}_2\&le;\mathrm{\&epsiv;}\end{array},$

Wherein ε is degree of rarefication empirical value,WithRespectively utilize high-resolution texture sparse coefficient and high resolution structures sparse coefficient that matching pursuit algorithm calculates,WithThe high-resolution optimum texture sparse coefficient respectively tried to achieve and high-resolution optimum structure sparse coefficient；

(2d) low-resolution image is calculated according to step (2c)At low resolution texture dictionaryWith low resolution structure dictionaryUnder optimum rarefaction representation, obtain low resolution optimum texture sparse coefficientWith low resolution optimum structure sparse coefficient

(2e) according to high-resolution texture dictionaryWith high-resolution optimum texture sparse coefficientObtain high-resolution texture layerNoteHigh-resolution texture layer for sample set I；According to high resolution structures dictionaryWith high-resolution optimum structure sparse coefficientObtain high resolution structures layerNoteHigh resolution structures layer for sample set I；

(2f) according to low resolution texture dictionaryWith low resolution optimum texture sparse coefficientObtain low resolution texture layerNoteLow resolution texture layer for sample set I；According to low resolution structure dictionaryWith low resolution optimum structure sparse coefficientObtain low resolution structure sheafNoteLow resolution structure sheaf for sample set I.

3. the video image grade reconstruction method based on rarefaction representation and dictionary learning according to claim 1, it is characterised in that: step (3) utilizes KSVD algorithm that the image in sample set I is trained, carries out as follows:

(3a) by the high-resolution sample image I in sample set I_hCarry out overlap partition, obtain high-resolution sample block collection Represent high-resolution sample image I_hA sample block, m=1,2 ..., M, M represents high-resolution sample image I_hPiecemeal number；

(3b) at high-resolution sample block collection Y_hIn randomly select a sample block, it is made dct transform, with conversion after data constitute high-resolution dictionary D_hInitial value；

(3c) KSVD algorithm is utilized, by following optimization process to high-resolution dictionary D_hIt is updated, until high-resolution sample block collection Y_hAt high-resolution dictionary D_hUnder rarefaction representation be optimum rarefaction representation:

$\begin{array}{cc}{D}_h^{*}=\underset{D_h}{Arg}min\left\{\underset{m=1}{\overset{M}{\&Sigma;}}\right||y_{h_m}-D_h\&times;{\mathrm{\&alpha;}}_{h_m}|{|}_2^2\}& s.t\left|\right|{\mathrm{\&alpha;}}_{h_m}|{|}_0\&le;\mathrm{\&epsiv;},m=1,2,...,M\end{array}$

Wherein,For sample blockAt high-resolution dictionary D_hUnder rarefaction representation, ε is degree of rarefication empirical value,For the optimum high-resolution dictionary tried to achieve；

(3d) by the low resolution sample image I of sample set I_lProcess according to step (3a) (3c), obtain optimum low-resolution dictionary

4. the video image grade reconstruction method based on rarefaction representation and dictionary learning according to claim 1, it is characterized in that: low-resolution video single-frame images to be reconstructed is divided into area-of-interest and region of loseing interest in by step (5), carries out as follows:

(5a) low-resolution video single-frame images to be reconstructed is carried out moving object detection, obtain the binary image of moving target；

(5b) using the Guan Bi outline of moving target binary image as the initial profile of Snake algorithm, by the successive iteration process of Snake algorithm, the accurate closed contour of moving target is obtained；

(5c) using the minimum rectangular area that comprises the accurate closed contour of moving target in low-resolution video single-frame images to be reconstructed as area-of-interest P, except area-of-interest partly as the region B that loses interest in.

5. the video image grade reconstruction method based on rarefaction representation and dictionary learning according to claim 1, it is characterized in that: the area-of-interest of low-resolution video single-frame images to be reconstructed is divided into main region and sub-region by step (6), carries out as follows:

(6a) utilize the accurate closed contour of the moving target that step (5b) obtains, calculate the elemental area of each target；

(6b) according to elemental area size target is divided into major heading and time target；

(6c) using the minimum rectangular area that comprises major heading as main region P_m, except main region partly as sub-region P_sub。

6. the video image grade reconstruction method based on rarefaction representation and dictionary learning according to claim 1, it is characterised in that: step (7) adopts the method for doubledictionary study main region is carried out super-resolution rebuilding, carry out as follows:

(7a) according to step (2) by main region P_mIt is divided into texture layer P_mtWith structure sheaf P_ms；

(7b) select the reference picture of main region, utilize texture layer rarefaction representation and the structure sheaf rarefaction representation of reference picture, calculate the texture layer rarefaction representation of main regionWith structure sheaf rarefaction representation

(7c) according to main region P_mTexture layer rarefaction representationWith texture high-resolution dictionaryObtain the reconstruction image of main region texture layerStructure sheaf rarefaction representation according to main regionWith structure high-resolution dictionaryObtain the reconstruction image of main region structure sheaf

(7d) by the reconstruction image of main region texture layerReconstruction image with main region structure sheafMerge, obtain the reconstruction image of complete main region.

7. the video image grade reconstruction method based on rarefaction representation and dictionary learning according to claim 1, it is characterized in that: step (7b) selects the reference picture of main region, utilize texture layer rarefaction representation and the structure sheaf rarefaction representation of reference picture, calculate the texture layer rarefaction representation of main regionWith structure sheaf rarefaction representationCarry out as follows:

(7b1) using each for the front and back of main region place frame three frames as reference picture, reference picture collection P is obtained_r={ P_rj, P_rjRepresent a frame reference picture, j=1,2 ..., 6；

(7b2) according to step (2) by reference picture collection P_rIt is divided into texture layer P_rt={ P_rtjAnd structure sheaf P_rs={ P_rsj, P_rtjFor reference picture P_rjTexture layer, P_rsjFor reference picture P_rjStructure sheaf；

(7b3) by the texture layer P of main region_mtCarry out overlap partition, obtain the piecemeal collection of main region texture layer Represent the texture layer P of main region_mtA piecemeal, n=1,2 ..., N, N represents the texture layer P of main region_mtPiecemeal number；

(7b4) to main region texture layer P_mtEach piecemealUtilize Three Step Search Algorithm at reference picture texture layer P_rtjThe most match block of middle search

(7b5) according to texture low-resolution dictionaryCalculate match blockRarefaction representationWhereinForInverse matrix；

(7b6) reference picture texture layer P is calculated_rtjIn most match blockWeight coefficient w_jn, computing formula is as follows

$w_{j_n}=\frac{1}{\sqrt{\left(y_{{\mathrm{mt}}_n}-y_{{\mathrm{rtj}}_n}^{*}\right){\left(y_{{\mathrm{mt}}_n}-y_{{\mathrm{rtj}}_n}^{*}\right)}^T}};$

(7b7) to match blockRarefaction representationIt is weighted summation, obtains main region texture layer piecemealTexture layer rarefaction representationNoteTexture layer rarefaction representation for main region；

(7b8) by the structure sheaf P of main region_msProcess according to step (7b3) (7b7), obtain the structure sheaf rarefaction representation of main region