CN110246111B

CN110246111B - No-reference stereoscopic image quality evaluation method based on fusion image and enhanced image

Info

Publication number: CN110246111B
Application number: CN201811498041.2A
Authority: CN
Inventors: 李素梅; 丁义修; 常永莉
Original assignee: Tianjin University Marine Technology Research Institute
Current assignee: Tianjin University Marine Technology Research Institute
Priority date: 2018-12-07
Filing date: 2018-12-07
Publication date: 2023-05-26
Anticipated expiration: 2038-12-07
Also published as: CN110246111A

Abstract

The non-reference stereoscopic image quality evaluation method based on the fusion image and the enhanced image comprises the steps of firstly, fusing a left viewpoint and a right viewpoint of a stereoscopic image in a red channel, a green channel and a blue channel based on the characteristics of binocular fusion, binocular competition, binocular inhibition and the like of a human visual system to obtain a color fusion image; secondly, a stereo matching algorithm is used for obtaining a parallax image of the distorted stereo image pair, and the gradient weight of the parallax image is used for weighting the gray level image of the color fusion image; thirdly, generating an enhanced image according to the fusion image and the parallax map; then, natural statistical features are extracted from the fused image and the enhanced image in a spatial domain, and kurtosis and skewness features are extracted from the disparity map; and finally, fusing the extracted features and sending the fused features into support vector regression (support vector regression, SVR) to obtain the quality of the stereoscopic image to be evaluated.

Description

No-reference stereoscopic image quality evaluation method based on fusion image and enhanced image

Technical Field

The invention belongs to the field of image processing, relates to objective evaluation research of stereoscopic image quality, and particularly relates to a non-reference stereoscopic image quality objective evaluation method based on a fusion image and an enhanced image.

Background

With the rapid development of 3D technology, stereoscopic image quality evaluation has become one of the indispensable research directions in the 3D field. At present, the evaluation method of the stereoscopic image quality can be divided into subjective evaluation and objective evaluation, wherein the subjective evaluation accords with human visual characteristics, but the realization process is tedious, time-consuming and labor-consuming; the objective evaluation method is simpler and quicker to realize and has good operability, so a large number of students throw into the field of objective evaluation ^[1-3] 。

Objective quality assessment is classified into three categories according to the degree of using the original image information: full reference stereoscopic image quality assessment ^[4-6] They make full use of the information of the original image as a reference to evaluate the distorted image; semi-reference stereoscopic image quality assessment ^[7-8] Performing quality evaluation by using partial information of the original image; ginseng-freeExamination stereo image quality evaluation ^[9-10] The quality evaluation can be completed by only utilizing the characteristics of the distorted image, and the method has good applicability.

At present, many students start from left and right views of a stereoscopic image, respectively perform feature extraction on the left and right views, and then obtain an evaluation result according to the features of the left and right views, and the method often cannot well evaluate an asymmetric stereoscopic image. Literature ^[3] A gradient dictionary learning method for performing color visual characteristics on left and right views respectively is provided, so that sparse representation is used for performing feature extraction; literature ^[10] Luminance statistics features are extracted for the left and right views respectively, and then depth and structure statistics features are further extracted by combining the disparity map with the left and right views respectively. However, in practice, after receiving the information of the left and right viewpoints, the human eye first forms a binocular fusion image from the brain, and then perceives the obtained fusion image. To better simulate this characteristic, some students began using binocular fusion images for stereoscopic image quality evaluation. Shen (Chinese character) ^[11] Considering the importance of spatial frequency to human eyes, the left and right views are processed by Gabor filters, and the processed left and right views are added to form a fusion image, and the model can only accord with human visual characteristics to a certain extent. Levelt ^[12] Based on the binocular competition characteristic of human eyes, a linear model of a fusion image is provided, and the left view and the right view are weighted respectively and then added to obtain the fusion image; literature ^[13][14] Such a linear model is improved in view of the importance of parallax compensation and contrast sensitivity in human visual characteristics, respectively. Ding for more accurate simulation of human visual properties ^[15] Several binocular fusion models are proposed based on gain control and gain enhancement.

Reference to the literature

[1] Chen M J, Cormack L K, Bovik A C. No-reference quality assessment of natural stereopairs.[J]. IEEE Transactions on Image Processing, 2013, 22(9):3379-3391.

[2] Zhou W, Jiang G, Yu M, et al. Reduced-reference stereoscopic image quality assessment based on view and disparity zero-watermarks[J]. Signal Processing Image Communication, 2014, 29(1):167-176.

[3] Yang J, An P, Ma J, et al. No-reference stereo image quality assessment by learning gradient dictionary-based color visual characteristics[C]// IEEE International Symposium on Circuits and Systems. IEEE, 2018.

[4] Shao F, Lin W, Gu S, et al. Perceptual Full-Reference Quality Assessment of Stereoscopic Images by Considering Binocular Visual Characteristics[J]. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 2013, 22(5):1940-1953.

[5] Zhang Y, Chandler D M. 3D-MAD: A Full Reference Stereoscopic Image Quality Estimator Based on Binocular Lightness and Contrast Perception[J]. IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society, 2015, 24(11):3810-25.

[6] Lin Y, Yang J, Wen L, et al. Quality Index for Stereoscopic Images by Jointly Evaluating Cyclopean Amplitude and Cyclopean Phase[J]. IEEE Journal of Selected Topics in Signal Processing, 2017, PP(99):1-1.

[7] Qi F, Zhao D, Gao W. Reduced Reference Stereoscopic Image Quality Assessment Based on Binocular Perceptual Information[J]. IEEE Transactions on Multimedia, 2015, 17(12):2338-2344.

[8] Ma J, An P, Shen L, et al. Reduced-Reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics and Structural Degradation[J]. IEEE Access, 2017, PP(99):1-1.

[9] Sazzad Z M P, Horita Y. No-reference stereoscopic image quality assessment[J]. Proceedings of SPIE - The International Society for Optical Engineering, 2010, 7524(2):75240T-75240T-12.

[10] Fang Y, Yan J, Wang J. No reference quality assessment for stereoscopic images by statistical features[C]// Ninth International Conference on Quality of Multimedia Experience. IEEE, 2017.

[11] Shen L, Lei J, Hou C. No-reference stereoscopic 3D image quality assessment via combined model[J]. Multimedia Tools & Applications, 2017(9):1-18.

[12] W.J.M. Levelt, On Binocular Rivalry, Mouton, The Hague, Paris, 1968.

[13] Chen M J , Su C C , Kwon D K , et al. Full-reference quality assessment of stereoscopic images by modeling binocular rivalry[C]// Signals, Systems & Computers. IEEE, 2013.

[14] Lu K , Zhu W . Stereoscopic Image Quality Assessment Based on Cyclopean Image[C]// Dependable, Autonomic & Secure Computing, Intl Conf on Pervasive Intelligence & Computing, Intl Conf on Big Data Intelligence & Computing & Cyber Science & Technology Congress. IEEE, 2016.

[15] Ding J, Klein S A, Levi D M. Binocular combination of phase and contrast explained by a gain-control and gain-enhancement model[J]. Journal of Vision, 2013, 13(2):13.

[16] Liu L, Liu B, Su C C, et al. Binocular spatial activity and reverse saliency driven no-reference stereopair quality assessment[J]. Signal Processing Image Communication, 2017.

[17] Yang J, Wang Y, Li B, et al. Quality assessment metric of stereo images considering cyclopean integration and visual saliency[J]. Information Sciences An International Journal, 2016, 373(C):251-268.

[18] Xu X, Zhao Y, Ding Y. No-reference stereoscopic image quality assessment based on saliency-guided binocular feature consolidation[J]. Electronics Letters, 2017, 53(22):1468-1470.

[19] Ma J, An P, Shen L, et al. SSIM-based binocular perceptual model for quality assessment of stereoscopic images[C]// Visual Communications and Image Processing. IEEE, 2018.

[20] Shao F, Li K, Lin W, et al. Learning Blind Quality Evaluator for Stereoscopic Images Using Joint Sparse Representation[J]. IEEE Transactions on Multimedia, 2016, 18(10):2104-2114.

[21] Ding Y, Zhao Y. No-reference quality assessment for stereoscopic images considering visual discomfort and binocular rivalry[J]. Electronics Letters, 2018, 53(25):1646-1647.

[22] Yue G, Hou C, Jiang Q, et al. Blind Stereoscopic 3D Image Quality Assessment via Analysis of Naturalness, Structure, and Binocular Asymmetry[J]. Signal Processing, 2018。

Disclosure of Invention

In order to solve the problems in the prior art, the non-reference stereoscopic image quality evaluation method based on the fusion image and the enhanced image not only has good consistency with subjective evaluation of human eyes, but also can effectively evaluate symmetric distortion and asymmetric distortion stereoscopic images, and promotes the development of stereoscopic imaging technology on a certain basis.

The non-reference stereoscopic image quality evaluation method based on the fusion image and the enhanced image comprises the following specific contents:

acquisition of color fusion image

Firstly, simulating human eye characteristics, and performing Gabor filtering on three channels of red, green and blue; secondly, the gain control theory states that the left eye applies gain control to the right eye in proportion to the contrast energy of the input signal; and applies gain control, called gain enhancement, to the gain control from the right eye; the right eye applies gain control and gain enhancement to the left eye as well; and then respectively generating weights for the left view and the right view according to the total contrast energy, giving the weights to the left view and the right view, and finally summing to obtain a color fusion image, wherein the detailed process is as follows:

1. gabor filter mimics the receptive field:

wherein

，/>

and />

Respectively representing left and right viewpoints; />

An intensity value representing the left or right view at each spatial location; />

Is per image and with spatial frequency +.>

And angle->

Gabor filter of->

Convolutionally derived response, wherein->

With 6 scales>

Has 8 directions, upper corner mark->

Representing the number of the feature images, 48 feature images can be obtained in total; />

Representing the magnitude of each response, +.>

Representing the phase of each responseA bit;

2. gain control

And gain enhancement->

：

The left view and the right view of the stereoscopic image are processed by Gabor filter to obtain 48 feature images with different scales and different directions, and the 48 feature images are arranged according to the ascending order of average brightness value to obtain a set

The method comprises the steps of carrying out a first treatment on the surface of the Gain enhancement +.>

And gain control

：

3. Total contrast energy:

applying contrast sensitivity functions to feature maps

To get +.>

Formula (4) wherein

Calculating the weight by the formula (5)>

Then the total comparative energy of the gain control is obtained by equation (6)>

And gain-enhanced total specific energy +.>

；

4. And (3) a left and right image fusion process:

the image fusion process is performed in three channels of the color image, red, green, and blue, wherein,

weight for left view, +.>

For right view weights, the final fused image is derived from equation (9):

/>

represents R, G, B channel,)>

The fusion image representing the channel can be obtained after three-channel fusion into a color fusion image +.>

；

Secondly, obtaining a disparity map and a disparity gradient weight:

processing the distorted stereo image pair by using a stereo matching algorithm based on structural similarity to obtain a parallax image; then calculating kurtosis and skewness of the parallax map by using a statistical method;

weighting the normalized fused image with weights generated by parallax gradients to predict visual saliency, the parallax gradient weights being generated as in equation (10), wherein

Gradient magnitude representing disparity map:

third, obtaining an enhanced image:

the parallax compensation is applied to the fusion image in a multiplication mode, and the fusion image subjected to the parallax compensation are multiplied to form a reinforced image, wherein the reinforced image can highlight the texture of the picture; the enhanced image calculation method is represented by formula (11), wherein,

representing a reinforced image; />

A gray scale map representing the fused image; />

Representing horizontal parallax; />

Representing the spatial coordinates of the image; />

Normalization and feature extraction of the image:

1. normalization of images:

respectively carrying out average contrast ratio normalization (mean subtracted contrast normalized, MSCN) on the fused image and the reinforced image thereof, wherein the normalization can remove local correlation of the image and make brightness values of the image tend to Gaussian distribution; calculating the subtracted mean contrast normalization coefficient as in equation (12), the MSCN coefficients of the resulting fused image may further weight the fused image as in equation (15):

/>

wherein ,

gray-scale map for fusion or enhancement image, < >>

Representing the height and width of the image, respectively; />

Is a constant; />

Represents a local mean; />

Representing local variance;

is a circularly symmetric Gaussian weighting function sampled to 3 standard deviations,/o>

The window of the Gaussian filter is set to +.>

；

2. Fitting a Gaussian model to extract characteristics:

the Gaussian model is used for capturing the change rule of statistical features in a natural scene in a spatial domain and evaluating the quality of a plane image, and the statistical features of the natural scene have very important roles in simulating a human visual system; the two Gaussian models are applied to stereoscopic image quality evaluation, so that good results are obtained;

to capture the differences in the case of different distortion types, the weighted and enhanced images are extracted by fitting a generalized gaussian distribution (generalized Gaussian distribution, GGD) and an asymmetric generalized gaussian distribution (asymmetric generalized Gaussian distribution, AGGD), respectively, at two scales, the process of which can be divided into two phases:

in the first stage, a GGD model is used to fit the MSCN coefficient distribution of the weighted image and the enhanced image, the GGD model can be used to effectively capture the statistical characteristics of the distorted image, and the zero-mean GGD model calculation method is as follows:

wherein ,

is a gamma function, the zero-mean GGD model makes the MSCN coefficient distribution approximately symmetrical,/A->

Control the general shape of the gaussian distribution, +.>

Representing variance, the degree of shape change can be controlled, so using these two parameters

To capture information of the image as a feature of the first stage;

in the second stage, the AGGD model is used for fitting MSCN coefficients multiplied by adjacent elements in the image in pairs, and the weighted image and the reinforced image are respectively fitted along four directions, namely a horizontal direction H, a vertical direction V, a main diagonal direction D1 and a secondary diagonal direction D2; the image calculation method for the four directions is as follows:

/>

the commonly used AGGD model is as follows:

wherein ,

shape parameters

Control the shape of the distribution->

For the mean value of the AGGD model, the scale parameter +.>

、/>

Control the distribution of left and right sides, respectively, will +.>

The four parameters are taken as the extracted characteristics of AGGD, and 16 characteristics are taken in four directions;

3. extracting kurtosis and skewness of the disparity map:

the statistical data is modified in a specific way by different distortions of the image, kurtosis can describe the flatness or the abrupt degree of the image, skewness can describe the distortability of the image, and the statistical characteristics of the parallax image under different distortions are captured by using the kurtosis and the skewness, and the formula (27) is as follows:

and />

Represents kurtosis and skewness of disparity map, respectively, ">

Representing a disparity map->

Is the mean value of the disparity map;

fifthly, feature fusion and SVR:

because the images show different characteristics under different scales, 72 characteristics can be obtained by utilizing the GDD and AGDD models to extract the characteristics of the weighted fusion image and the reinforced image based on different scales; the kurtosis and skewness characteristics of the parallax images are combined to form 74 characteristics; then the 74 obtained features are fused and sent into SVR and subjective evaluation value for fitting; wherein, the nonlinear regression function uses a logistic function, and the kernel function of the SVR uses a radial basis function.

Drawings

FIG. 1 is a block diagram of an algorithm of the present invention;

FIG. 2 is a schematic diagram of a color fusion image fusion process;

FIG. 3 is a fused image and enhanced image MSCN coefficient distribution map (ori: original image. Wn: white noise. Jp2k: JPEG2000. JPEG: JPEG compression. Blur: gaussian blur. Ff: fast fading);

FIG. 4 is a MSCN coefficient distribution map (ori: original image, wn: white noise, jp2k: JPEG2000. JPEG: JPEG compression, blur: gaussian blur, ff: fast) of a fused image multiplied by adjacent pixels in the horizontal direction.

Detailed Description

First, a color fusion image and a disparity map are formed from left and right views. And obtaining an enhanced image according to the parallax image and the fusion image. Considering the importance of parallax information, the parallax images are subjected to multi-angle mining, not only are the statistical features of the parallax images extracted, but also parallax gradient weights are calculated, and fusion images are weighted to better accord with the human eye characteristics. And then, capturing statistical features of the weighted fusion image and the enhanced image by adopting a Gaussian model, and finally fusing all the features and fitting with subjective scores. Experimental results show that the algorithm disclosed by the invention is excellent in performance, can well accord with subjective evaluation of human beings, and is accurate in model prediction result. The experimental structure is shown in fig. 1, and the color fusion image fusion process is shown in fig. 2.

In the technical scheme of the patent, when the parallax gradient weight, the kurtosis and the skewness of the parallax map and all the factors are the same, three other methods for evaluating the quality of the stereoscopic image can be formed, and the quality of the stereoscopic image can be evaluated more excellently through the performance comparison by the first three methods in the table 1. However, the PLCC, SROCC and RMSE indices are inferior to the methods described herein, which illustrates that the present invention can be used to derive several other non-optimal but viable methods according to the technical scheme, and further integrate the comparisons of the remaining methods in Table 1, the scores were greatly reduced without using enhanced images, while SINQ was used in the framework presented herein ^[16] In the case of the multiplied images, the scoring is inferior to the method, and the superiority of the enhanced image is reflected. The information of the enhanced image, the parallax gradient weight, the kurtosis and the skewness of the parallax image come from different angles in the parallax image, the information plays a very remarkable role, the quality score is improved to a great extent, the method is perfected, and the best implementation scheme of the invention is obtained.

Table 1 comparison of the performance of the methods herein

Fig. 3 (a) and (b) are distributions of MSCN coefficients of a fused image and an enhanced image, respectively, and fig. 4 (a) and (b) are distributions of MSCN coefficients multiplied by horizontally adjacent elements of the fused image and the enhanced image, respectively, wherein the original images and distortion maps of the fused image and the enhanced image use the same scene. Different distortion types have different shapes according to different statistical characteristics, and in fig. 3 and fig. 4, the distorted images cause the original image distribution to have different degrees of extrusion or diffusion. According to different deformation modes and degrees of image distribution, different distortion types of the image can be approximately embodied.

The distribution of the original image in fig. 3 presents a gaussian distribution, and the introduction of different distortions causes a different degree of pinching or spreading of this distribution. In fig. 3 (a), the distribution of the jp2k distorted image is obviously squeezed, and the distribution is similar to the laplace distribution; the distribution of the wn distortion image is diffused, and still shows Gaussian distribution. In fig. 3 (b), the peak of the wn-distorted image distribution is significantly shifted.

The distribution of the original image in fig. 4 shows left-right asymmetry, and the introduction of distortion type causes the extrusion or diffusion phenomenon to occur in the distribution, and the degree of asymmetry varies differently. FIGS. 4 (a), (b) show an asymmetry phenomenon in the original image; the distribution of the wn distortion image not only generates a diffusion phenomenon, but also has more obvious asymmetry degree compared with the original image distribution; the distribution of the jp2k distorted image is less likely to be squeezed, and the degree of asymmetry is more pronounced than the original image distribution.

From the above analysis, it is clear that the statistical features of the image MSCN coefficients can reflect to some extent the differences of different distorted images, which can be quantified. Literature is used herein ^[9] The difference is quantified by a method for extracting features, the weighted image and the enhanced image are respectively subjected to fitting generalized Gaussian distribution and asymmetric generalized Gaussian distribution under two scales to obtain statistical features, and the process can be divided into two stages.

The present invention performs performance testing of the proposed algorithm on two disclosed stereo image databases (LIVE Phase i and LIVE Phase ii). The LIVE Phase I database comprises 365 symmetrically distorted stereo image pairs and 20 original stereo image pairs; LIVE Phase ii contains a total of 360 and 8 original stereo image pairs of symmetrical and asymmetrical distorted stereo image pairs. The eigenvectors were fed into the SVR and fitted to DMOS values to give PLCC (Pearson's Correlation Coefficient), SROCC (Spearman's Rank Ordered Correlation Coefficient), and RMSE (Root Mean Squared Error) scores were used to measure the quality of the results. The lower the RMSE value is, the higher the PLCC and SROCC values are, which shows that the better the performance of the algorithm provided by the invention is, and the obtained objective quality score has better consistency with the subjective quality score.

The invention compares and analyzes the three-dimensional image quality evaluation result published by the prior art. Literature ^[18] Performing non-reference quality evaluation based on a multi-scale feature fusion method; literature ^[17][19] Performing full-reference stereoscopic image quality evaluation by using the fusion image; literature ^[20] Providing a method for joint sparse representation without reference to perform quality evaluation; literature ^[6][16][21] Performing non-reference quality evaluation by using the fusion image; literature ^[22] Evaluation is performed by analyzing natural statistical properties, structural properties, and asymmetry of the distorted image. Table 2 shows the results of all algorithms on LIVE Phase I and LIVE Phase II databases. The best performing algorithm is represented in the table by bold fonts.

As can be seen from Table 2, the performance of other methods on the LIVE Phase I database is significantly higher than that on the LIVE Phase II database due to the presence of a large number of asymmetrically distorted images in the LIVE Phase II database, and the performance of the methods herein on both databases is close and excellent. This demonstrates that the method herein conforms to the visual characteristics of the human eye and enables accurate assessment of symmetrically distorted and asymmetrically distorted images. The advantages of the methods herein are evident compared to existing full-reference and no-reference methods. PLCC was 0.9583, SROCC was 0.9507 and RMSE was 4.3811 on LIVE Phase I database; PLCC was 0.9575, SROCC was 0.9542 and RMSE was 3.0689 on LIVE Phase II database. The method can obtain good performance without using original image information, and the performance is higher than that of other reference-free methods, and the method has good practicability and robustness.

Table 2 overall performance comparison of different methods

Because of the adoption and improvement of the literature [16]Tables 3 and 4 are methods and literature herein ^[16] Detailed comparison of the medium index.As can be seen from Table 3, the individual distortion types and overall scores of the methods herein are higher on the LIVE Phase I database than in the literature ^[16] As can be seen from Table 4, the method herein has a single distortion type score and a total score on the LIVE Phase II database that are higher than those of the literature ^[16] This phenomenon is not only reflected in the methods herein compared with the literature ^[16] The method has the advantages that the indexes are improved, the quality of the symmetrical distortion and the asymmetrical distortion stereoscopic image can be effectively evaluated, the method is more in line with the visual characteristics of human eyes, and the method is suitable for asymmetrical distortion stereoscopic images.

TABLE 3 Performance ratio of two different methods on LIVE Phase I database

Table 4 shows a comparison of the performance of two different methods on LIVE Phase II.

/>

Claims

1. The non-reference stereoscopic image quality evaluation method based on the fusion image and the enhanced image is characterized by comprising the following steps of: detailed description of the preferred embodiments

Acquisition of color fusion image

Firstly, simulating human eye characteristics, and performing Gabor filtering on three channels of red, green and blue;

secondly, the left eye applies gain control to the right eye in proportion to the contrast energy of the input signal according to the gain control theory; and applies gain control, called gain enhancement, to the gain control from the right eye; the right eye applies gain control and gain enhancement to the left eye as well; and then respectively generating weights for the left view and the right view according to the total contrast energy, giving the weights to the left view and the right view, and finally summing to obtain a color fusion image, wherein the detailed process is as follows:

1. gabor filter mimics the receptive field:

wherein v is { l, r }, l and r represent left and right views respectively; i _v (ζ, η) represents the intensity value of the left or right view at each spatial position;

is per image and with spatial frequency f _s Convolving the resulting response with Gabor filter g at angle θ, where f _s The upper corner mark n represents the number of the feature images, and 48 feature images can be obtained in total, wherein the 6 scales theta have 8 directions; />

Representing the magnitude of each response, +.>

Representing the phase of each response;

2. gain control gc and gain enhancement ge:

Gain enhancement ge and gain control gc are obtained by equations (2) and (3):

3. total contrast energy:

applying contrast sensitivity functions to feature maps

To get +.>

As in equation (4), where v ε { l, r }, n=1, 2,3 … 48, weight +.>

The total comparative energy TCE of the gain control is then obtained by equation (6) _v And gain-enhanced total specific energy +.>

A(f)＝2.6(0.192+0.114f)exp[-(0.114f) ^1.1 ] (4)

4. And (3) a left and right image fusion process:

the image fusion process is performed in three channels of red, green and blue of the color image, wherein G _l Weight G for left view _r For right view weights, the final fused image is derived from equation (9):

i represents R, G, B channel, C ⁱ (x, y) representing the fusion image of the channel, obtaining a color fusion image C after three-channel fusion _r (x，y)；

Secondly, obtaining a disparity map and a disparity gradient weight:

Gradient magnitude representing disparity map:

third, obtaining an enhanced image:

the parallax compensation is applied to the fusion image in a multiplication mode, and the fusion image subjected to the parallax compensation are multiplied to form a reinforced image, wherein the reinforced image can highlight the texture of the picture; the enhanced image calculating method is represented by formula (11), wherein P represents an enhanced image; c represents a gray scale image of the fusion image; d represents horizontal parallax; x, y represent the spatial coordinates of the image;

P(x，y)＝C(x，y)·C(x+d(x，y)，y) (11)

normalization and feature extraction of the image:

1. normalization of images:

respectively carrying out average-reduction contrast normalization MSCN operation on the fused image and the reinforced image thereof, wherein the normalization can remove the local correlation of the image and lead the brightness value of the image to be prone to Gaussian distribution; calculating the subtracted mean contrast normalization coefficient as in equation (12), the MSCN coefficients of the resulting fused image may further weight the fused image as in equation (15):

MSCN _w ＝W(x，y)·C ^MSCN (x，y) (15)

wherein C is a gray scale of the fused or enhanced image, x.epsilon.1, 2..M, y.epsilon.1, 2..N; m and N respectively represent the height and width of the image; a is a constant; mu (mu) _C Represents a local mean; sigma (sigma) _C Representing local variance; omega= { omega _k，l I k= -k..k, l= -l..l..l } is a circularly symmetric gaussian weighting function sampled to 3 standard deviations, k=l=3, the window of the gaussian filter is set to 7×7;

2. fitting a Gaussian model to extract characteristics:

in order to capture the difference under different distortion types, the weighted image and the enhanced image are respectively subjected to feature extraction under two scales by fitting the generalized Gaussian distribution GGD and the asymmetric generalized Gaussian distribution AGGD, and the process can be divided into two stages:

wherein Γ (·) is a gamma function, the zero-mean GGD model causes the MSCN coefficient distribution to be approximately symmetrical, α controls the approximate shape of the Gaussian distribution, σ ² Representing variance, the degree of shape change can be controlled, so that the two parameters (alpha, sigma ² ) To capture information of the image as a feature of the first stage;

the commonly used AGGD model is as follows:

/>

wherein ,

shape parameter v controls the shape of the distribution, eta is the mean value of the AGGD model, and the scale parameter sigma _l ² 、σ _r ² Control the distribution of left and right sides respectively, will (v, eta, sigma) _l ² ，σ _r ² ) The four parameters are taken as the extracted characteristics of AGGD, and 16 characteristics are taken in four directions;

3. extracting kurtosis and skewness of the disparity map:

k and S represent kurtosis and skewness of the disparity map, respectively, d represents the disparity map, and E (d) is the average value of the disparity map;

fifthly, feature fusion and SVR: