CN107492085A - Stereo image quality evaluation method based on dual-tree complex wavelet transform - Google Patents

Stereo image quality evaluation method based on dual-tree complex wavelet transform Download PDF

Info

Publication number
CN107492085A
CN107492085A CN201710597141.XA CN201710597141A CN107492085A CN 107492085 A CN107492085 A CN 107492085A CN 201710597141 A CN201710597141 A CN 201710597141A CN 107492085 A CN107492085 A CN 107492085A
Authority
CN
China
Prior art keywords
image
original
wavelet
stereo
picture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201710597141.XA
Other languages
Chinese (zh)
Other versions
CN107492085B (en
Inventor
侯春萍
林洪湖
岳广辉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tianjin University
Original Assignee
Tianjin University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tianjin University filed Critical Tianjin University
Priority to CN201710597141.XA priority Critical patent/CN107492085B/en
Publication of CN107492085A publication Critical patent/CN107492085A/en
Application granted granted Critical
Publication of CN107492085B publication Critical patent/CN107492085B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20212Image combination
    • G06T2207/20221Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Image Analysis (AREA)

Abstract

Stereo image quality evaluation method is referred to based on dual-tree complex wavelet transform entirely the present invention relates to a kind of, step is as follows:1) carries out left image respectively for original three-dimensional image and distortion stereo-picture and right image merges;2) does dual-tree complex wavelet transform for the original three-dimensional image and left image of distortion stereo-picture, right image and composite diagram;3) extracts contrast, structure and brightness from original graph and each wavelet sub-band of distortion map;4) makes energy calculation to each wavelet sub-band of original three-dimensional image, gain control method is used to make weight with each sub-belt energy after left image, right image and composite diagram the progress wavelet decomposition of original three-dimensional image, with reference to features such as brightness, contrast and the structures of original image and each wavelet sub-band of distorted image, stereo image quality evaluation is carried out, obtains final quality evaluation fraction.

Description

Stereo image quality evaluation method based on dual-tree complex wavelet transform
Technical field
The invention belongs to the objective evaluation system of image processing field, specifically stereo image quality, is related to using double trees The objective image evaluation method of complex wavelet transform and stereo-picture composite diagram.
Background technology
With the arrival in multimedia messages epoch, people as the data signal active influence of representative and is changed using image and video Live and work mode.Due to the fast development of imaging technique and display device, the popularization of stereo-picture, it is greatly enhanced The visual quality experience of user.But stereo-picture can produce during acquisition, pretreatment, coding, transmission, decoding etc. Degradation problems, therefore stereo image quality evaluation turns into an important research topic.Stereo image quality evaluation now In method, most straightforward approach is exactly that left images are commented respectively using the statistical nature of plane picture quality evaluating method Valency, finally prediction obtains the quality of stereo-picture, but such method does not have the visually-perceptible mechanism for considering that the mankind are complicated, because This evaluation poor-performing.Composite diagram [1] incorporates the parallax information of visually-perceptible so that stereo image quality is evaluated and subjective assessment Uniformity is higher.Therefore, the present invention proposes the traditional plane picture quality evaluating method of application in left image, right image and synthesis Feature is extracted on each subband of small echo of figure, carries out stereo image quality evaluation.
[1]Chen M J,Su C C,Kwon D K,et al.Full-reference quality assessment of stereopairs accounting for rivalry[J].Signal Processing:Image Communication,2013,28(9):1143-1155.
[2]Wang Z,Bovik A C,Sheikh H R,et al.Image quality assessment:from error visibility to structural similarity[J].IEEE transactions on image processing,2004,13(4):600-612.
The content of the invention
It is an object of the invention to for three-dimensional distorted image quality evaluation problem, it is preferably vertical to propose that one kind can obtain The complete of body image quality evaluation effect refers to stereo image quality evaluation method.The present invention is on original three-dimensional image left image, the right side Wavelet decomposition is carried out on image and composite diagram and distortion stereo-picture left image, right image and composite diagram, in each wavelet sub-band Upper extraction is than degree, structure and brightness, with each wavelet sub-band energy of original three-dimensional image left image, right image and composite diagram Make weight fusion character pair, the method for carrying out stereo image quality evaluation.Technical scheme is as follows:
It is a kind of that stereo image quality evaluation method is referred to based on dual-tree complex wavelet transform entirely, utilize the left figure of stereo-picture Picture, right image and each wavelet sub-band energy of composite diagram three weight each wavelet sub-band feature and carry out quality to distortion stereo-picture Evaluation, step are as follows:
1) carries out left image respectively for original three-dimensional image and distortion stereo-picture and right image merges
For original three-dimensional image and distortion stereo-picture, it is poor that left view is calculated by left image and right image, uses left image Left disparity map is added to obtain with left view difference, then weight is calculated to synthesize composite diagram by left image and left disparity map, passes through normalization Gabor filtered energies respond to obtain image weights;
2) it is multiple small to do double trees for the original three-dimensional image and left image of distortion stereo-picture, right image and composite diagram by Wave conversion
Each image is resolved into 3 layers, every layer of 6 direction, i.e.,:They distinguish table Show n-th layer ± 15 °, ± 45 °, the wavelet coefficient of ± 75 ° of subbands, n=1,2,3;Obtain 18 wavelet sub-bands;
3) extracts contrast, structure and brightness from original graph and each wavelet sub-band of distortion map
By the contrast of original three-dimensional image and the left image of distortion stereo-picture, right image and each wavelet sub-band of composite diagram Degree, structure and brightness are as feature extraction;
4) makes energy calculation to each wavelet sub-band of original three-dimensional image, and Fusion Features
Each wavelet sub-band of original three-dimensional image is made energy calculation, uses gain control method with original stereo figure The left image of picture, right image and composite diagram carry out each sub-belt energy after wavelet decomposition and make weight, with reference to original image and distortion The features such as brightness, contrast and the structure of each wavelet sub-band of image, stereo image quality evaluation is carried out, final quality is obtained and comments Valency fraction.
The present invention carries out wavelet decomposition in the left image, right image and composite diagram of original three-dimensional image, after wavelet decomposition Each sub-belt energy as gain control weight, brightness, contrast and the architectural feature of each image are merged, to distortion stereo-picture Quality evaluation, obtain quality evaluation fraction.The present invention and subjective picture quality evaluation uniformity are strong, and performance is better than what be presently, there are Most of stereo image quality evaluation algorithms.
Brief description of the drawings
The flow chart of Fig. 1 present invention
Tri- layers of dual-tree complex wavelet transforms of Fig. 2
Fig. 3 uniformity
Embodiment
The present invention proposes a kind of based on extracting wavelet decomposition in the left image, right image and composite diagram of original three-dimensional image The full reference image quality appraisement method of weight that is controlled as gain of each sub-belt energy.To make technical scheme more Add clear, with reference to figure 1, of the invention comprises the following steps that.
1. left images merge
After different degrees of alteration occurs in stereo-picture, due to the presence of parallax, the subjective quality of image can not only lead to The average of left images quality is crossed to obtain, the stereo-picture for merging parallax information is different from plane picture, left images fusion Obtain composite diagram and mainly consider parallax information, therefore the present invention, to being handled as follows to left and right figure, left images carry out Gabor filters Ripple, the multiple Gabor filtering of two dimension are defined as follows:
Wherein R1=xcos θ+ysin θ, R2=-sin θ+ycon θ.σxyIt is standard deviation, ζxyIt is spatial frequency, θ is filter Ripple direction.Energy response value of the left images in all yardsticks and direction is respectively GEL, GER
The calculating of left and right weight is obtained by normalizing Gabor filtered energies response assignment, is defined as:
Composite diagram is defined as by the present invention:
C (x, y)=wL(x,y)·IL(x,y)+wR(x+d,y)·IR(x+d,y)
(4)
Wherein, C represents composite diagram, ILAnd IRLeft images are represented, d is parallax, wLAnd wRFor left and right weight, for solid The quality evaluation of image, present invention utilizes the parallax information of left images to synthesize composite diagram, is prepared for extraction feature.
2. dual-tree complex wavelet transform
The present invention carries out wavelet decomposition on left image, right image and composite diagram, because the peak value of wavelet coefficient can be anti- Reflect the definition of image.Dual-tree complex wavelet transform is developed by traditional wavelet, and traditional wavelet transformation is broken down into water Flat, vertical and 3 directions of tilted direction detailed information, dual-tree complex wavelet transform inherit the good characteristic of traditional wavelet, The directivity information of image can also be described more, Complex Wavelet Transform is realized by real DWT, by real part and Imaginary part is separated, and the wavelet conversion coefficient of real and imaginary parts is obtained by two groups of parallel real filter groups.The present invention is made Three layers of dual-tree complex wavelet transform such as Fig. 2.
Each image is resolved into 3 layers by the present invention, every layer of 6 direction, i.e.,:They ± 15 ° of expression n-th layer respectively, ± 45 °, the wavelet coefficient of ± 75 ° of subbands, n=1,2,3.Each subband of small echo is carried out Energy balane, as the final weight for calculating mass fraction
3. character representation
These three aspects of contrast, structure and brightness all directly affect picture quality, between them again independently of each other.This hair It is bright using contrast, structure and the brightness of the left images of stereo-picture and each wavelet sub-band of composite diagram as feature extraction. Picture contrast, structure and brightness are extracted separately below:
1) contrast is defined as the standard deviation sigma of signalx, the similarity of contrast is defined as:
C1It is the constant of a very little.
To two signals x and y, brightness is defined as base average, i.e.,:
In formula, N is the length of signal.
2) structural similarity is defined as:
3) similarity of brightness is defined as:
In formula, C2It is the constant of a very little, ensuresWith 0 very close to when numerical value stablize it is qualitative.
Image structure similarity (SSIM) [2] is defined as:
SSIM (x, y)=[L (x, y)]α[C(x,y)]β[F(x,y)]γ (9)
In formula, α, beta, gamma is the number more than 0, and the size by adjusting this 3 parameters can change 3 components in quality Importance in fraction.
4. Fusion Features
The present invention uses gain control method to carry out each sub-belt energy work after wavelet decomposition with left and right image and composite diagram Weight, with reference to features such as the brightness of each wavelet sub-band of image, contrast and architectural features, carry out stereo image quality evaluation.Its Effect is that simple linear combination mode is not reached.Formula is as follows:
WhereinEL、ERIt is similar,Represent that left image carries out i-th layer of j-th of direction after wavelet decomposition Energy:
η,Respectively Li,jThe real and imaginary parts in (i-th layer of j-th of direction of left image),Formula it is similar.The structural similarity of original left image and distortion right image is represented,
Q is final mass fraction.
5. database selects
Two open test storehouses of present invention selection, they are the asymmetric stereo-picture test libraries that LIVE laboratories provide LIVE-3D II and symmetrical stereo-picture test library LIVE-3D I.In LIVE 3D rendering quality evaluations storehouse II, totally 360 width lose True stereo-picture and 8 width original images, the storehouse includes symmetrical and asymmetric 2 kinds of distortions, also comprising 5 kinds of type of distortion:JPEG is pressed Contracting, JPEG2000 (JP2K), Gaussian Blur (Gaussian blur, GBLUR), white noise (white noise, WN) and fast Weak (fast fading, FF), and provide the subjective scoring difference and parallax value of every group of distortion stereo-picture.Scheme in LIVE3D As in quality evaluation storehouse I, totally 365 width distortion stereo-pictures and 20 width original images, are that left and right distortion level identical is symmetrical Distortion, include 5 kinds of type of distortion:JPEG,JP2K,GBLUR,WN,FF.
For prove image prediction objective quality scores that the inventive method obtains and subjective quality scores have it is very high consistent Property, prediction objective quality scores can accurately reflect the quality of image, the inventive method is surveyed in symmetrical and asymmetric stereo-picture Tested on examination storehouse LIVE-3D II and LIVE-3D I, take 3 measurement Objective image quality evaluation algorithms commonly used in the world Index evaluation the inventive method performance, 3 indexs are respectively Spearman sequence coefficient correlation (Spearman rank- Order correlation coefficient, SRCC), Pearson's linearly dependent coefficient (Pearson linear Correlation coefficient, PLCC) and root-mean-square error (Root Mean Squared Error, RMSE), wherein, PLCC and RMSE indexs weigh the forecasting accuracy of objective algorithm, and SRCC indexs weigh the prediction monotonicity of objective algorithm.PLCC It is smaller closer to 1, RMSE value with SRCC value, illustrate that algorithm performance is better, prediction objective quality scores and subjective quality point Several correlations is higher.
6. compare and parser performance
Present invention verification algorithm performance on stereo-picture test library LIVE-3D I and LIVE-3D II, from following contrast In understand that the inventive method achieves good effect, the inventive method vertical stops image quality evaluating method uniformity with subjective It is high.Table 1 represents performance of the present invention on LIVE 3D Phase I test libraries, it can be seen that the present invention is for right from table 1 Claim Gaussian Blur distortion PLCC to reach 0.944,0.937 is reached to white Gaussian noise distortion SPRCC.It can see by table 2, this hair It is bright to reach 0.971, SPRCC for asymmetric Gaussian Blur mixing distortion PLCC on LIVE 3D Phase II test libraries and reach 0.947.Tables 1 and 2 has carried out contrast of the present invention on LIVE storehouses with other methods, as can be seen from the table, the present invention Method and the degree of correlation of subjective assessment, order of accuarcy are significantly improved, and Fig. 3 represents the correlation of subjective and objective fraction, says The prediction objective quality scores of bright the inventive method are high with subjective quality scores correlation.
Table 1 is in LIVE 3D Phase I to the uniformity result of calculation of 5 kinds of distortions
Table 2 is in LIVE 3D Phase II to the uniformity result of calculation of 5 kinds of mixing distortions
The inventive method has advantages below:
(1) the inventive method and subjective stereo image quality evaluation uniformity are high.
(2) different from plane picture quality evaluation, the present invention considers the distinctive parallax information of stereo-picture, proposes from conjunction Brightness of image, structure and contrast metric are extracted on into each subband of small echo of figure, test result indicates that the inventive method performance is excellent In the most of stereo image quality evaluation algorithms that presently, there are.

Claims (1)

1. a kind of refer to stereo image quality evaluation method entirely based on dual-tree complex wavelet transform, the left figure of stereo-picture is utilized Picture, right image and each wavelet sub-band energy of composite diagram three weight each wavelet sub-band feature and carry out quality to distortion stereo-picture Evaluation.Step is as follows:
1) carries out left image respectively for original three-dimensional image and distortion stereo-picture and right image merges
For original three-dimensional image and distortion stereo-picture, it is poor that left view is calculated by left image and right image, with left image and a left side Parallax is added to obtain left disparity map, then calculates weight by left image and left disparity map to synthesize composite diagram, by normalizing Gabor filters Wave energy responds to obtain image weights;
2) does dual-tree complex wavelet change for the original three-dimensional image and left image of distortion stereo-picture, right image and composite diagram Change
Each image is resolved into 3 layers, every layer of 6 direction, i.e.,:They represent n-th respectively ± 15 ° of layer, ± 45 °, the wavelet coefficient of ± 75 ° of subbands, n=1,2,3;Obtain 18 wavelet sub-bands;
3) extracts contrast, structure and brightness from original graph and each wavelet sub-band of distortion map
By original three-dimensional image and the left image of distortion stereo-picture, the contrast of right image and each wavelet sub-band of composite diagram, knot Structure and brightness are as feature extraction;
4) makes energy calculation to each wavelet sub-band of original three-dimensional image, and Fusion Features
Each wavelet sub-band of original three-dimensional image is made energy calculation, uses gain control method with original three-dimensional image Each sub-belt energy after left image, right image and composite diagram progress wavelet decomposition makees weight, with reference to original image and distorted image The features such as brightness, contrast and the structure of each wavelet sub-band, stereo image quality evaluation is carried out, obtain final quality evaluation point Number.
CN201710597141.XA 2017-07-20 2017-07-20 Stereo image quality evaluation method based on dual-tree complex wavelet transform Active CN107492085B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710597141.XA CN107492085B (en) 2017-07-20 2017-07-20 Stereo image quality evaluation method based on dual-tree complex wavelet transform

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710597141.XA CN107492085B (en) 2017-07-20 2017-07-20 Stereo image quality evaluation method based on dual-tree complex wavelet transform

Publications (2)

Publication Number Publication Date
CN107492085A true CN107492085A (en) 2017-12-19
CN107492085B CN107492085B (en) 2021-01-15

Family

ID=60643480

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710597141.XA Active CN107492085B (en) 2017-07-20 2017-07-20 Stereo image quality evaluation method based on dual-tree complex wavelet transform

Country Status (1)

Country Link
CN (1) CN107492085B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108269253A (en) * 2018-01-11 2018-07-10 天津大学 Stereo image quality evaluation method based on wavelet transformation and local structure feature
CN110363763A (en) * 2019-07-23 2019-10-22 上饶师范学院 Image quality evaluating method, device, electronic equipment and readable storage medium storing program for executing
CN112330757A (en) * 2019-08-05 2021-02-05 复旦大学 Complementary color wavelet measurement for evaluating color image automatic focusing definition

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103152600A (en) * 2013-03-08 2013-06-12 天津大学 Three-dimensional video quality evaluation method
CN104918039A (en) * 2015-05-05 2015-09-16 四川九洲电器集团有限责任公司 Image quality evaluation method and image quality evaluation system
CN105959684A (en) * 2016-05-26 2016-09-21 天津大学 Stereo image quality evaluation method based on binocular fusion

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103152600A (en) * 2013-03-08 2013-06-12 天津大学 Three-dimensional video quality evaluation method
CN104918039A (en) * 2015-05-05 2015-09-16 四川九洲电器集团有限责任公司 Image quality evaluation method and image quality evaluation system
CN105959684A (en) * 2016-05-26 2016-09-21 天津大学 Stereo image quality evaluation method based on binocular fusion

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
张达奇: "全参考图像主观质量客观评价算法研究", 《中国优秀硕士学位论文全文数据库(电子期刊)》 *
熊润生: "基于小波变换的无参考立体图像质量评价", 《计算机科学》 *

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108269253A (en) * 2018-01-11 2018-07-10 天津大学 Stereo image quality evaluation method based on wavelet transformation and local structure feature
CN110363763A (en) * 2019-07-23 2019-10-22 上饶师范学院 Image quality evaluating method, device, electronic equipment and readable storage medium storing program for executing
CN112330757A (en) * 2019-08-05 2021-02-05 复旦大学 Complementary color wavelet measurement for evaluating color image automatic focusing definition
CN112330757B (en) * 2019-08-05 2022-11-29 复旦大学 Complementary color wavelet measurement for evaluating color image automatic focusing definition

Also Published As

Publication number Publication date
CN107492085B (en) 2021-01-15

Similar Documents

Publication Publication Date Title
CN108876735B (en) Real image blind denoising method based on depth residual error network
CN105959684B (en) Stereo image quality evaluation method based on binocular fusion
CN107393542B (en) Bird species identification method based on two-channel neural network
CN110969124A (en) Two-dimensional human body posture estimation method and system based on lightweight multi-branch network
CN108765414B (en) No-reference stereo image quality evaluation method based on wavelet decomposition and natural scene statistics
CN107341786A (en) The infrared and visible light image fusion method that wavelet transformation represents with joint sparse
CN107123091A (en) A kind of near-infrared face image super-resolution reconstruction method based on deep learning
CN109167996B (en) It is a kind of based on convolutional neural networks without reference stereo image quality evaluation method
Li et al. Deep learning methods in real-time image super-resolution: a survey
CN110070574B (en) Binocular vision stereo matching method based on improved PSMAT net
CN105844635B (en) A kind of rarefaction representation depth image method for reconstructing based on structure dictionary
CN101562675B (en) No-reference image quality evaluation method based on Contourlet transform
CN112288668B (en) Infrared and visible light image fusion method based on depth unsupervised dense convolution network
CN104376565B (en) Based on discrete cosine transform and the non-reference picture quality appraisement method of rarefaction representation
CN104199627B (en) Gradable video encoding system based on multiple dimensioned online dictionary learning
CN109242834A (en) It is a kind of based on convolutional neural networks without reference stereo image quality evaluation method
CN107492085A (en) Stereo image quality evaluation method based on dual-tree complex wavelet transform
CN109523513A (en) Based on the sparse stereo image quality evaluation method for rebuilding color fusion image
CN111539247A (en) Hyper-spectrum face recognition method and device, electronic equipment and storage medium thereof
CN108259893B (en) Virtual reality video quality evaluation method based on double-current convolutional neural network
CN107071423A (en) Application process of the vision multi-channel model in stereoscopic video quality objective evaluation
CN108257098A (en) Video denoising method based on maximum posteriori decoding and three-dimensional bits matched filtering
Li et al. Recent advances and challenges in video quality assessment
CN114266957A (en) Hyperspectral image super-resolution restoration method based on multi-degradation mode data augmentation
CN114926337A (en) Single image super-resolution reconstruction method and system based on CNN and Transformer hybrid network

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant