CN106791836A - It is a kind of to be based on a pair of methods of the reduction compression of images effect of Multi net voting - Google Patents

Abstract

The a kind of of proposition is based on a pair of methods of the reduction compression of images effect of Multi net voting in the present invention, and its main contents includes：Input jpeg compressed image, it is proposed that component generates artifact-free candidate image, measurement assembly assessment exports quality, and its process is, using a pair of Multi net votings, including suggestion element and measuring cell；Then suggestion component exports a series of artifact-free candidate images using jpeg compressed image as input, and its output quality is further estimated by measurement assembly.The present invention is realized from a jpeg compressed image and effectively recovers artifact-free image, loss is perceived by merging, natural loss and JPEG lose these three loss functions to set up measurement assembly, provide the user a series of candidate images, and allow the image that user selects them to be liked, the pinch effect of image is reduced, the Quality of recovery of image is substantially increased.

Description

It is a kind of to be based on a pair of methods of the reduction compression of images effect of Multi net voting

Technical field

The present invention relates to image processing field, a pair of reduction compression of images effects of Multi net voting are based on more particularly, to a kind of The method answered.

Background technology

In the epoch of this information explosion, the amount of images propagated on network increases sharply.People would generally have with image Compression is damaged to save bandwidth and memory space.Wherein, JPEG is most widely used lossy compression.However, image is damaged Compression can cause information to be lost and compression artefacts, and this seriously reduces Consumer's Experience.Therefore, how to recover attractive in appearance without artifact Image is more and more paid close attention to by people.It has been proposed that many methods suppress JPEG compression effect, such as sparse volume is used Code rebuilds artifact free image, but this method would generally be along with noisy edge and non-natural region；And use ARCNN To recover image, image can be made to become excessively smooth, compared to the picture of compression, the texture that it contains is less.

The present invention propose it is a kind of be based on a pair of methods of the reduction compression of images effect of Multi net voting, using one-to-many net Network, including suggestion element and measuring cell；Then suggestion component exports a series of without artifact using jpeg compressed image as input Candidate image, further estimated by measurement assembly its output quality.The present invention realizes effective from a jpeg compressed image Ground recovers artifact-free image, and loss is perceived by merging, and natural loss and JPEG lose these three loss functions to set up survey Amount component, provides the user a series of candidate images, and allows the image that user selects them to be liked, and reduces the compression of image Effect, substantially increases the Quality of recovery of image.

The content of the invention

The problem of picture quality can be reduced for lossy compression method, one-to-many net is based on it is an object of the invention to provide one kind The method of the reduction compression of images effect of network, using a pair of Multi net votings, including suggestion element and measuring cell；Suggestion component with Then jpeg compressed image exports a series of artifact-free candidate images as input, further estimates that its is defeated by measurement assembly Mass.

To solve the above problems, present invention offer is a kind of to be based on a pair of methods of the reduction compression of images effect of Multi net voting, Its main contents includes：

(1) it is input into jpeg compressed image；

(2) suggestion component generates artifact-free candidate image；

(3) measurement assembly assessment output quality.

Wherein, a pair of described Multi net votings, it is broken down into suggestion element and measuring cell；Suggestion component is with JPEG compression Then image exports a series of artifact-free candidate images as input, and its output quality is further estimated by measurement assembly；One Multi net voting is realized from a jpeg compressed image and effectively recover artifact-free image, provide the user a series of high-quality Candidate image, and allow the image that user selects them to be liked.

Wherein, the image of described input JPEG compression, jpeg compressed image is generated using the jpeg coder of MATLAB； Image is divided into 8 × 8 encoding blocks by jpeg coder first, then to each block application discrete cosine transform (DCT)；In DCT Afterwards, each in 64 DCT coefficients is equably quantified together with quantization table；During decoding, jpeg decoder is to quantifying to be Number performs inverse DCT to obtain pixel value.

Wherein, described suggestion component generates artifact-free candidate image, it is proposed that component provides a F model, will reflect It is deep layer CNN to penetrate F exploitations；One-to-many attribute is enabled, auxiliary variable Z is introduced in a network as hiding additional input；Net Network is using compression image Y as input；It is adopted from the normal distribution centered on zero with standard deviation 1 to Z simultaneously Sample, both Y and Z then be fed in network to carry out Nonlinear Mapping；

With the Z for sampling as two inputs of different branches, the output of the two branches is cascaded compression image Y；In level On connection Feature Mapping, further perform polymerization sub-network and predicted without artifact with generating；

In component is advised, each branch includes 5 remaining units, and the sub-network that is polymerized includes 10 remaining units； Each remaining unit includes two batch standardization layers, two ReLU layers and two convolutional layers；

Before compression image is forwarded into network, down-sampling is carried out to it by 4 × 4 convolutional layers of stride -2；Finally, Network is exported and up-sampled by 4 × 4 uncoiling laminations of stride -2, to keep image size.

Further, described up-sampling, is up-sampled using the uncoiling lamination of stride -2 that wave filter size is 4, will Filter represents [w₁,w₂,w₃,w₄]；Assuming that applying deconvolution to the constant [..., c ...] of an input, wherein c is scalar；In advance Phase output should be constant；However, reality output is c^*；If it is required that reality output met anticipated output, the filtering trained Device should meet w₁+w₃=w₂+w₄；

Make final output for constant, (can be expressed as deconvolution output is obtained using " mobile and average " strategy Deconv after), following two steps are performed：

1) repeat deconv and moved to right 1 pixel；

2) average deconv and shifted version.

Wherein, described measurement assembly assessment output quality, an output is obtained from suggestion componentAfterwards, adopt Estimated with measurement assemblyIt is whether satisfactory, therefore define three measurement loss functions：Perceive loss, natural loss and JPEG loses.

Further, described perception loss, the feature for the depth network of the pre-training of image classification can be fine Ground description perception information；The feature extracted from lower level tends to retain accurate information on photo, and higher level feature is to face Color, texture and shape difference are constant；Therefore, loss is perceived to be defined to promoteSimilar high-level characteristic is shared with X：

Wherein, φ is the function from network calculations, H_φIt is characteristic size.

Further, described natural loss, it is intended that it is " nature " image to recover artifact free image as far as possible, because This builds a complementary network D to distinguish image be from suggestion component F generations or a natural image；Network D performs two and enters System classification, and it is the probability of " nature " to export input；Second loss of this probability as measurement component is added toIt is negative right On number, excitationWith high probability：

Network D is also required to training, and it uses binary system entropy loss as its optimization aim：

From formula (2) and formula (3) as can be seen that network F and network D contends with one other：It is artifact-free that network F attempts generation ImageIt is difficult to be distinguished with natural image for network D, while training network D, it is to avoid network F produces pseudomorphism.

Further, described JPEG losses, Joint Photographic Experts Group is made up of various predefined parameters, by using these ginsengs Number, we can obtain the lower and upper limit of pixel value；For compression, jpeg coder divides input picture by quantifying table DCT coefficient, is then rounded to immediate integer by result；The quantization table that jpeg decoder is multiplied by below is depressurized；Therefore, Relation between compression image Y and corresponding uncompressed image X can be expressed as：

Wherein, X^dctAnd Y^dctIt is respectively the DCT coefficient of X and Y, Q is quantization table, and i and j is the index in DCT domain, formula (4) imply that we write following DCT coefficient range constraint：

So the artifact free image of each recoveryAlso formula (5) should be met, it is proposed that following JPEG loses：

Wherein,It isSize；As can be seen that JPEG losses are the L for truncating₂Loss；DCT coefficient fall lower limit/on Outside limit (such as：) reconstructed imageWill be penalized.

Further, three described measurement loss functions, measurement assembly is set up by merging these three loss functions：

One-to-many Web vector graphic batch gradient decline is optimized；Image is prepared as the benefit of network inputs size identical Fourth；By λ₁It is set to 0.1, λ₂Then need some specially treateds；Jpeg coder is performed to each 8 × 8 non-overlapped encoding block respectively Quantify；And for the piece with coding block boundaries misalignment, we can not obtain its DCT coefficient；Therefore, we are according to given benefit Fourth sets different λ₂Value；In general, network training includes two key steps in each iterative process：

1) discriminant amendment component F, is optimized with equation (3) and differentiates network D；

2) corrective networks D, is optimized with measurement component (i.e. formula (7)) and proposes component F；If input block and JPEG encoding blocks Boundary alignment, then by λ₂It is set to 0.1；Otherwise by λ₂It is set to 0.

Brief description of the drawings

Fig. 1 is that the present invention is a kind of is based on a pair of system flow charts of the method for the reduction compression of images effect of Multi net voting.

Fig. 2 is that the present invention is a kind of is based on a pair of schematic flow sheets of the method for the reduction compression of images effect of Multi net voting.

Fig. 3 is that the present invention is a kind of is based on a pair of suggestion component generation nothings of the method for the reduction compression of images effect of Multi net voting The candidate image of artifact.

Fig. 4 is that the present invention is a kind of is based on a pair of " mobile and average " plans of the method for the reduction compression of images effect of Multi net voting Slightly.

Specific embodiment

It should be noted that in the case where not conflicting, the feature in embodiment and embodiment in the application can phase Mutually combine, the present invention is described in further detail with specific embodiment below in conjunction with the accompanying drawings.

Fig. 1 is that the present invention is a kind of is based on a pair of system flow charts of the method for the reduction compression of images effect of Multi net voting.It is main To include input jpeg compressed image, it is proposed that component generates artifact-free candidate image, measurement assembly assessment output quality.

Wherein, jpeg compressed image is generated using the jpeg coder of MATLAB；Image is divided into 8 by jpeg coder first × 8 encoding blocks, then to each block application discrete cosine transform (DCT)；Each after DCT, in 64 DCT coefficients Equably quantified together with quantization table；During decoding, jpeg decoder performs inverse DCT to obtain pixel value to quantization parameter.

Wherein, measurement assembly assessment output quality, an output is obtained from suggestion componentAfterwards, using measurement Component is estimatedIt is whether satisfactory, therefore define three measurement loss functions：Loss is perceived, natural loss and JPEG are damaged Lose.

Loss is perceived, the feature for the depth network of the pre-training of image classification can well describe perception information； The feature extracted from lower level tends to retain accurate information on photo, and higher level feature is to color, texture and shape difference It is different constant；Therefore, loss is perceived to be defined to promoteSimilar high-level characteristic is shared with X：

Wherein, φ is the function from network calculations, H_φIt is characteristic size.

Natural loss, it is intended that it is " nature " image to recover artifact free image as far as possible, therefore structure one is additional Network D is come to distinguish image be from suggestion component F generations or a natural image；Network D performs binary class, and exports defeated It is the probability of " nature " to enter；Second loss of this probability as measurement component is added toNegative logarithm on, excitationWith height Probability：

Network D is also required to training, and it uses binary system entropy loss as its optimization aim：

From formula (2) and formula (3) as can be seen that network F and network D contends with one other：It is artifact-free that network F attempts generation ImageIt is difficult to be distinguished with natural image for network D, while training network D, it is to avoid network F produces pseudomorphism.

JPEG loses, and Joint Photographic Experts Group is made up of various predefined parameters, and by using these parameters, we can obtain The lower and upper limit of pixel value；For compression, jpeg coder divides the DCT coefficient of input picture by quantifying table, then will Result is rounded to immediate integer；The quantization table that jpeg decoder is multiplied by below is depressurized；Therefore, compression image Y and right Relation between the uncompressed image X for answering can be expressed as：

Wherein, X^dctAnd Y^dctIt is respectively the DCT coefficient of X and Y, Q is quantization table, and i and j is the index in DCT domain, formula (4) imply that we write following DCT coefficient range constraint：

So the artifact free image of each recoveryAlso formula (5) should be met, it is proposed that following JPEG loses：

Wherein,It isSize；As can be seen that JPEG losses are the L for truncating₂Loss；DCT coefficient fall lower limit/on Outside limit (such as：) reconstructed imageWill be penalized.

Measurement assembly is set up by merging these three loss functions：

One-to-many Web vector graphic batch gradient decline is optimized；Image is prepared as the benefit of network inputs size identical Fourth；By λ₁It is set to 0.1, λ₂Then need some specially treateds；Jpeg coder is performed to each 8 × 8 non-overlapped encoding block respectively Quantify；And for the piece with coding block boundaries misalignment, we can not obtain its DCT coefficient；Therefore, we are according to given benefit Fourth sets different λ₂Value；In general, network training includes two key steps in each iterative process：

1) discriminant amendment component F, is optimized with equation (3) and differentiates network D；

2) corrective networks D, is optimized with measurement component (i.e. formula (7)) and proposes component F；If input block and JPEG encoding blocks Boundary alignment, then by λ₂It is set to 0.1；Otherwise by λ₂It is set to 0.

Fig. 2 is that the present invention is a kind of is based on a pair of schematic flow sheets of the method for the reduction compression of images effect of Multi net voting.Adopt A pair of Multi net votings are used, it is broken down into suggestion element and measuring cell；Suggestion component using jpeg compressed image as input, then A series of artifact-free candidate images are exported, its output quality is further estimated by measurement assembly.

Fig. 3 is that the present invention is a kind of is based on a pair of suggestion component generation nothings of the method for the reduction compression of images effect of Multi net voting The candidate image of artifact.Suggestion component provides a F model, is deep layer CNN by mapping F exploitations；One-to-many attribute is enabled, Auxiliary variable Z is introduced in a network as hiding additional input；Network is using compression image Y as input；It is from tool simultaneously The normal distribution centered on zero for having standard deviation 1 is sampled to Z, then is fed in network to carry out by both Y and Z Nonlinear Mapping；

With the Z for sampling as two inputs of different branches, the output of the two branches is cascaded compression image Y；In level On connection Feature Mapping, further perform polymerization sub-network and predicted without artifact with generating；

In component is advised, each branch includes 5 remaining units, and the sub-network that is polymerized includes 10 remaining units； Each remaining unit includes two batch standardization layers, two ReLU layers and two convolutional layers；

Before compression image is forwarded into network, down-sampling is carried out to it by 4 × 4 convolutional layers of stride -2；Finally, Network is exported and up-sampled by 4 × 4 uncoiling laminations of stride -2, to keep image size.

Fig. 4 is that the present invention is a kind of is based on a pair of " mobile and average " plans of the method for the reduction compression of images effect of Multi net voting Slightly.During suggestion component generates artifact-free candidate image, entered using the uncoiling lamination of stride -2 that wave filter size is 4 Row up-sampling, [w is represented by filter₁,w₂,w₃,w₄]；Assuming that deconvolution is applied to the constant [..., c ...] of an input, its Middle c is scalar；Anticipated output should be constant；However, reality output is c^*；If it is required that reality output meets anticipated output, The filter then trained should meet w₁+w₃=w₂+w₄；

Make final output for constant, (can be expressed as deconvolution output is obtained using " mobile and average " strategy Deconv after), following two steps are performed：

1) repeat deconv and moved to right 1 pixel；

2) average deconv and shifted version.

For those skilled in the art, the present invention is not restricted to the details of above-described embodiment, without departing substantially from essence of the invention In the case of god and scope, the present invention can be realized with other concrete forms.Additionally, those skilled in the art can be to this hair Bright to carry out various changes and modification without departing from the spirit and scope of the present invention, these improvement also should be regarded as of the invention with modification Protection domain.Therefore, appended claims are intended to be construed to include preferred embodiment and fall into all changes of the scope of the invention More and modification.

Claims (10)

1. it is a kind of to be based on a pair of methods of the reduction compression of images effect of Multi net voting, it is characterised in that main to include input JPEG Compression image (one)；Suggestion component generates artifact-free candidate image (two)；Measurement assembly assessment output quality (three).

2. based on a pair of Multi net votings described in claims 1, it is characterised in that it is broken down into suggestion element and measurement unit Part；Then suggestion component exports a series of artifact-free candidate images using jpeg compressed image as input, is entered by measurement assembly One step estimates its output quality；One-to-many real-time performance from a jpeg compressed image effectively recovers artifact-free image, Provide the user a series of high-quality candidate images, and allow the image that user selects them to be liked.

3. based on the image () for being input into JPEG compression described in claims 1, it is characterised in that using the JPEG of MATLAB Encoder generates jpeg compressed image；Image is divided into 8 × 8 encoding blocks by jpeg coder first, then to each block application Discrete cosine transform (DCT)；After DCT, each in 64 DCT coefficients is equably quantified together with quantization table；Solution During code, jpeg decoder performs inverse DCT to obtain pixel value to quantization parameter.

4. artifact-free candidate image (two) is generated based on the suggestion component described in claims 1, it is characterised in that suggestion group Part provides a F model, is deep layer CNN by mapping F exploitations；One-to-many attribute is enabled, auxiliary variable Z is introduced in a network As hiding additional input；Network is using compression image Y as input；It is with zero from standard deviation 1 simultaneously The normal distribution of the heart is sampled to Z, then is fed in network to carry out Nonlinear Mapping by both Y and Z；

With the Z for sampling as two inputs of different branches, the output of the two branches is cascaded compression image Y；It is special in cascade Levy on mapping, further perform polymerization sub-network to generate without artifact prediction；

In component is advised, each branch includes 5 remaining units, and the sub-network that is polymerized includes 10 remaining units；It is each Individual remaining unit includes two batch standardization layers, two ReLU layers and two convolutional layers；

Before compression image is forwarded into network, down-sampling is carried out to it by 4 × 4 convolutional layers of stride -2；Finally, network Export and up-sampled by 4 × 4 uncoiling laminations of stride -2, to keep image size.

5. based on the up-sampling described in claims 4, it is characterised in that use the deconvolution of stride -2 that wave filter size is 4 Layer is up-sampled, and filter is represented into [w₁,w₂,w₃,w₄]；Assuming that applying uncoiling to the constant [..., c ...] of an input Product, wherein c is scalar；Anticipated output should be constant；However, reality output is c^*；If it is required that reality output meets be expected Output, the then filter trained should meet w₁+w₃=w₂+w₄；

Make final output for constant, deconvolution output (being expressed as deconv) can obtained using " mobile and average " strategy Afterwards, following two steps are performed：

1) repeat deconv and moved to right 1 pixel；

2) average deconv and shifted version.

6. based on measurement assembly assessment output quality (three) described in claims 1, it is characterised in that obtained from suggestion component One outputAfterwards, estimated using measurement assemblyIt is whether satisfactory, therefore define three measurement loss letters Number：Loss is perceived, natural loss and JPEG lose.

7. based on the perception loss described in claims 6, it is characterised in that for the depth network of the pre-training of image classification Feature perception information can be described well；The feature extracted from lower level tends to retain accurate information on photo, and Higher level feature is to color, and texture and shape difference are constant；Therefore, loss is perceived to be defined to promoteSimilar height is shared with X Layer feature：

$L_{percept}(\hat{X},X)=\frac{1}{H_{\mathrm{\φ}}}\left|\right|\mathrm{\φ}\left(\hat{X}\right)-\mathrm{\φ}\left(X\right)|{|}_2^2---\left(1\right)$

Wherein, φ is the function from network calculations, H_φIt is characteristic size.

8. based on the natural loss described in claims 6, it is characterised in that it is desirable that recovering artifact free image as far as possible It is " nature " image, therefore builds a complementary network D to distinguish image be from the figure naturally of suggestion component F generations or Picture；Network D performs binary class, and it is the probability of " nature " to export input；Using this probability as the second of measurement component Loss is added toNegative logarithm on, excitationWith high probability：

$L_{natural}\left(\hat{X}\right)=-log\left(D\right(\hat{X}\left)\right)---\left(2\right)$

Network D is also required to training, and it uses binary system entropy loss as its optimization aim：

$L_D(\hat{X},X)=-\left(log\right(D\left(X\right))+\log (1-D\left(\hat{X}\right)\left)\right)---\left(3\right)$

From formula (2) and formula (3) as can be seen that network F and network D contends with one other：Network F attempts to generate artifact-free imageIt is difficult to be distinguished with natural image for network D, while training network D, it is to avoid network F produces pseudomorphism.

9. based on the JPEG losses described in claims 6, it is characterised in that Joint Photographic Experts Group is by various predefined parameter groups Into by using these parameters, we can obtain the lower and upper limit of pixel value；For compression, jpeg coder throughput Change the DCT coefficient that table divides input picture, result is then rounded to immediate integer；Jpeg decoder is multiplied by amount below Change table is depressurized；Therefore, the relation between compression image Y and corresponding uncompressed image X can be expressed as：

$Y^{dct}(i,j)=ROUND\left(\frac{X^{dct}(i,j)}{Q(i,j)}\right)*Q(i,j)---\left(4\right)$

Wherein, X^dctAnd Y^dctIt is respectively the DCT coefficient of X and Y, Q is quantization table, and i and j is the index in DCT domain, and formula (4) is dark Show that we write following DCT coefficient range constraint：

$Y^{dct}-\frac{Q}{2}\≤X^{dct}\≤Y^{dct}+\frac{Q}{2}---\left(5\right)$

So the artifact free image of each recoveryAlso formula (5) should be met, it is proposed that following JPEG loses：

$L_{jpeg}(\hat{X},Y)=\frac{1}{H_{\hat{X}}}MAX({\left({\hat{X}}^{dct}-{\hat{Y}}^{dct}\right)}^2-{\left(\frac{Q}{2}\right)}^2,0)---\left(6\right)$

Wherein,It isSize；As can be seen that JPEG losses are the L for truncating₂Loss；DCT coefficient falls outside lower limit/upper limit (such as：) reconstructed imageWill be penalized.

10. based on three measurement loss functions described in claims 6, it is characterised in that lose letter by merging these three Count to set up measurement assembly：

$L(\hat{X},X,Y)=L_{percept}(\hat{X},X)+{\mathrm{\λ}}_1{L}_{natural}\left(\hat{X}\right)+{\mathrm{\λ}}_2{L}_{jpeg}(\hat{X},Y)---\left(7\right)$

One-to-many Web vector graphic batch gradient decline is optimized；Image is prepared as the patch of network inputs size identical；By λ₁ It is set to 0.1, λ₂Then need some specially treateds；Jpeg coder performs quantization to each 8 × 8 non-overlapped encoding block respectively； And for the piece with coding block boundaries misalignment, we can not obtain its DCT coefficient；Therefore, we set according to given patch Put different λ₂Value；In general, network training includes two key steps in each iterative process：

1) discriminant amendment component F, is optimized with equation (3) and differentiates network D；

2) corrective networks D, is optimized with measurement component (i.e. formula (7)) and proposes component F；If input block and JPEG coding block boundaries Alignment, then by λ₂It is set to 0.1；Otherwise by λ₂It is set to 0.