CN110853027A - Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation - Google Patents

Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation Download PDF

Info

Publication number
CN110853027A
CN110853027A CN201911124950.4A CN201911124950A CN110853027A CN 110853027 A CN110853027 A CN 110853027A CN 201911124950 A CN201911124950 A CN 201911124950A CN 110853027 A CN110853027 A CN 110853027A
Authority
CN
China
Prior art keywords
local
color
image
gaussian
features
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201911124950.4A
Other languages
Chinese (zh)
Inventor
方玉明
姚怡茹
鄢杰斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Individual
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to CN201911124950.4A priority Critical patent/CN110853027A/en
Publication of CN110853027A publication Critical patent/CN110853027A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/90—Determination of colour characteristics
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30168—Image quality inspection

Abstract

The invention relates to a three-dimensional synthetic image non-reference quality evaluation method based on local variation and global variation, which is characterized by comprising the following steps: firstly, for local change detection, extracting the structure and color characteristics of a synthesized image by using a Gaussian derivative; secondly, coding is carried out by using a local binary pattern based on the structure characteristic and the color characteristic to obtain a structure characteristic graph and a color characteristic graph, and the structure characteristic graph and the color characteristic graph are calculated to obtain the structure characteristic graph and the color characteristic graph, so that distortion information of the local structure and the color is obtained; then, for global change detection, extracting brightness characteristics to evaluate the naturalness of the three-dimensional synthetic image; and finally, training a random forest regression model to map the extracted features to subjective quality scores based on the extracted visual features. The experimental results on the three disclosed databases show that the method shows good effectiveness and superiority compared with the existing non-reference image quality evaluation method and full-reference three-dimensional synthetic image quality evaluation method.

Description

Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation
Technical Field
The invention belongs to the technical field of multimedia, particularly belongs to the technical field of digital image and digital image processing, and particularly relates to a three-dimensional synthetic image non-reference quality evaluation method based on local variation and global variation.
Background
Free-view video (FVV) and three-dimensional film and television can bring people with a feeling of being personally on the scene, and the technology of the FVV and the three-dimensional film and television has attracted a great deal of attention from the academic world and the industrial world in the past decades. Generally, one method for obtaining free-view video is to take different images of the same scene from different viewing angles by using a plurality of cameras, and then to stitch the images taken from different viewing angles together. As the demand for better experience increases, the number of views in free-view video continues to grow, and the load on storage and transmission also increases. To solve this problem, multi-view video deepening (MVD) technology has been developed.
The multi-view video deepening technology only requires an original view map and depth maps of other cameras, and the remaining virtual views can be generated by a rendering-based deepening picture technology (DIBR). However, the DIBR process causes new visual distortions such as blurring, discontinuities, blocking and image stretching, which are clearly different from conventional distortions such as Gaussian blur, Gaussian noise and white noise, which can significantly affect the end user experience. Therefore, it is necessary to design a reliable and effective quality evaluation method for three-dimensional synthetic images to predict the visual quality of the three-dimensional synthetic images. In the past years, some quality evaluation methods aiming at synthesis distortion have been proposed, but the prediction effects of the methods are not accurate enough, so the invention provides a three-dimensional synthesis image quality evaluation method based on local variation and global variation, and the method can effectively predict the visual quality of a synthesis image.
Disclosure of Invention
The invention relates to a three-dimensional synthetic image non-reference quality evaluation method based on local variation and global variation, which is characterized by comprising the following steps: firstly, for local change detection, extracting the structure and color characteristics of a synthesized image by using a Gaussian derivative; secondly, coding is carried out by using a local binary pattern based on the structure characteristic and the color characteristic to obtain a structure characteristic graph and a color characteristic graph, and the structure characteristic graph and the color characteristic graph are calculated to obtain the structure characteristic graph and the color characteristic graph, so that distortion information of the local structure and the color is obtained; then, for global change detection, extracting brightness characteristics to evaluate the naturalness of the three-dimensional synthetic image; and finally, training a random forest regression model to map the extracted features to subjective quality scores based on the extracted visual features. The experimental results on the three disclosed databases show that the method shows good effectiveness and superiority compared with the existing non-reference image quality evaluation method and full-reference three-dimensional synthetic image quality evaluation method.
In order to achieve the purpose, the invention adopts the technical scheme that:
a three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation is characterized in that: the method comprises the following steps:
A. extracting the structure and color characteristics of the synthesized image by using a Gaussian derivative;
B. coding the obtained structure and color characteristics by using a local binary pattern respectively to obtain a structure characteristic diagram and a color characteristic diagram, and calculating the structure characteristic diagram and the color characteristic diagram respectively based on the structure characteristic diagram and the color characteristic diagram to obtain distortion information of the local structure and the color;
C. extracting brightness characteristics to evaluate the naturalness of the three-dimensional synthetic image for global change;
D. and (4) combining the extracted characteristic information, and using a random forest regression model to learn the mapping relation between the visual characteristics and the subjective quality scores to predict the quality scores of the three-dimensional synthetic images.
Further, structural features and color features of the image are extracted using gaussian derivatives.
Further, the structural features of the image are extracted by using Gaussian derivatives, and the method specifically comprises the following steps:
A. the local Taylor series expansion can represent the local characteristics of the image, and the coefficients of the local Taylor series can be obtained through local Gaussian derivatives; the gaussian derivative of an image can be defined as follows:
Figure BDA0002276501970000021
where m ≧ 0 and n ≧ 0 are derivatives along the horizontal x and vertical y directions, the symbol denotes the convolution operation; gσ(x, y, σ) is a Gaussian function whose standard deviation σ is defined as follows:
Figure BDA0002276501970000022
B. using the second order Gaussian derivative to extract structural features, first calculating the sum of m + n < 1 > and 2
Figure BDA0002276501970000023
By calculating the Gaussian derivatives, the resulting matrix
Figure BDA0002276501970000024
Can be expressed as:
Figure BDA0002276501970000031
further, extracting color features of the image by using Gaussian derivatives, which comprises the following specific steps:
A. two color features that are not affected by luminance are employed on the first order gaussian derivative of the color channel, one of which is defined as follows:
Figure BDA0002276501970000032
wherein the content of the first and second substances,
Figure BDA0002276501970000033
r, G and B respectively represent red, green and blue channels in a color space; another color characteristic
Figure BDA0002276501970000034
The definition is as follows:
Figure BDA0002276501970000035
wherein R ', G', and B 'respectively represent gaussian first derivative values of R, G, and B channels in the horizontal direction, and ρ ═ 2R' -G '-B', δ ═ 2G '-R' -B ', τ ═ 2B' -R '-G'.
Further, local binary patterns are respectively used for the obtained structure and color characteristics to obtain structure and color characteristic maps, and quality characteristics are calculated according to the structure and color characteristics, so that distortion information of the local structure and color is obtained.
Further, a local binary pattern method is used for obtaining a structure diagram to obtain distortion information of a local structure, and the specific steps are as follows:
A. binary pattern method pair using local rotation invariance
Figure BDA0002276501970000037
Each pixel of the image is operated on, the calculation is based on
Figure BDA0002276501970000038
Characteristic map of absolute value
Figure BDA0002276501970000039
The calculation formula is as follows:
Figure BDA00022765019700000310
wherein s ∈ { s ∈ [)1,s2,s3,s4,s5}; LBP stands for LBP operation; riu2 represents a rotation invariant unified mode; d and E represent the number and the calculation radius of surrounding pixels, the number D of the surrounding pixels is set to be 8, and the calculation radius E is 1; thereby obtaining 5 characteristic maps which are respectively
Figure BDA00022765019700000311
And
Figure BDA00022765019700000312
wherein
Figure BDA00022765019700000313
Describing the relationship between the central pixel point and the adjacent pixel points in the local area;
B. representing local structure distortion information by using weighted histogram, and using same local binary pattern operator pair
Figure BDA00022765019700000314
The pixels of (a) are accumulated to obtain a weighted histogram, which is defined by the following formula:
Figure BDA0002276501970000041
Figure BDA0002276501970000042
wherein N represents the number of picture pixels; k denotes the index of LBP, K ∈ [0, D +2 ]],Is a weight value which is a characteristic map
Figure BDA0002276501970000044
And summarizing the Gaussian derivatives to fuse and map pixel values in the Gaussian derivatives according to the LBP image intensity value, and obtaining a characteristic vector through normalization operation to enhance the change of high contrast in the image area so as to reflect local structure distortion information.
Further, obtaining a chromaticity diagram by using coding to obtain distortion information of local chromaticity; wherein the content of the first and second substances,
A. obtaining a feature vector by using a local binary model: at the extracted color feature x1Carry out LBPriu2Operation acquisition feature map
Figure BDA0002276501970000045
Then theConverting the feature map into a feature vector, which defines the formula:
Figure BDA0002276501970000046
Figure BDA0002276501970000047
wherein the content of the first and second substances,
Figure BDA0002276501970000048
is a weight value obtained by a local binary pattern operator, and the value is a characteristic diagram
Figure BDA0002276501970000049
B. The local chrominance information is represented by a weighted histogram: at the extracted color feature x2Carry out LBPriu2Operation acquisition feature map
Figure BDA00022765019700000410
Then x2The weighted histogram calculation of (a) is defined as follows:
Figure BDA00022765019700000411
Figure BDA00022765019700000412
wherein, the weight value
Figure BDA00022765019700000413
Is a characteristic map
Figure BDA00022765019700000414
Finally, a single feature vector representing the color information of the image is calculated by the following formula:
Figure BDA00022765019700000415
further, for global variation, the naturalness of the three-dimensional synthetic image is evaluated by extracting the luminance features: wherein the content of the first and second substances,
A. fitting luminance coefficient, luminance coefficient (L), using Gaussian distribution′) The definition is as follows:
where (i, j) represents the spatial position of the pixel, and i e {1,2, …, a }, j e {1,2, …, b }, where a and b represent the height and width of the image, respectively, and μ (i, j) and σ (i, j) are defined as follows:
Figure BDA0002276501970000052
Figure BDA0002276501970000053
where ω is a two-dimensional, centrosymmetric gaussian weighting function, ω ═ ωa,b|a∈[-3,3],b∈[-3,3]}；
The luminance parameter L' (i, j) is then fitted using a zero-mean generalized gaussian distribution, which defines the formula:
Figure BDA0002276501970000054
wherein the content of the first and second substances,
Figure BDA0002276501970000055
and is
Figure BDA0002276501970000056
Where parameter α controls the shape of the distribution, σ controls the variance;
B. subsequently, 4 parameters including shape parameters and variance of the generalized gaussian distribution and kurtosis and skewness of the luminance coefficient are calculated on 5 scales of the composite image, resulting in a total of 20-dimensional features; in addition, a laplacian pyramid image is computed from the difference between the composite image and its low-pass filtered image, the shape coefficients and variances are obtained using a generalized gaussian distribution model to fit the pixel values in the laplacian pyramid, the kurtosis and skewness of the laplacian pyramid are computed, and the four parameters are extracted from the five scales, yielding a total of 20-dimensional features.
Further, a quality prediction model is trained using a random forest regression method, wherein,
A. selecting feature information, namely obtaining a total 310-dimensional feature vector through local features and global features, wherein the total 310-dimensional feature vector comprises 270-dimensional local variation features and 40-dimensional global naturalness features;
B. and training a visual quality prediction model by using a random forest regression method, and mapping the quality characteristics to subjective evaluation. The three-dimensional synthetic view quality database is randomly divided into a training set and a testing set to be trained for 1000 times, and finally, a pierce linear correlation coefficient mean (PLCC), a pierce Mandarin level correlation coefficient (SRCC), a Kendall level correlation coefficient (KRCC) and a Root Mean Square Error (RMSE) are used as final results.
Drawings
FIG. 1 is a block diagram of the algorithm of the present invention.
Detailed Description
The technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. It is to be understood that the described embodiments are merely exemplary of the invention, and not restrictive of the full scope of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Wherein technical features, abbreviations/abbreviations, symbols and the like referred to herein are explained, defined/explained on the basis of the known knowledge/common understanding of a person skilled in the art.
The invention designs a novel and effective three-dimensional synthetic view non-reference quality evaluation method (LVGC) based on local variation and global variation, which can effectively improve the quality evaluation effect of the three-dimensional synthetic image and is inspired by the fact that the three-dimensional synthetic image degradation of local and global distribution and the human visual system are very sensitive to the structure, color information and global natural change.
The specific operation of each part of the invention is as follows:
(1) structural feature extraction:
research shows that local taylor series expansion can represent local features of a picture and coefficients of the local taylor series can be obtained through local gaussian distribution, and the gaussian distribution of one image can be defined as:
Figure BDA0002276501970000061
where m ≧ 0 and n ≧ 0 are derivatives along the horizontal (defined as x) and vertical (defined as y) directions, in particular, the notation x denotes the convolution operation, G ≧ 0σ(x, y, σ) is a gaussian function, and its positive deviation σ is defined as follows:
Figure BDA0002276501970000062
inspired by other researches, the invention applies a second order Gaussian derivative to extract structural features; firstly, m + n is more than or equal to 1 and less than or equal to 2 to obtain the compound
Figure BDA0002276501970000063
By calculating the Gaussian derivatives, matrices
Figure BDA0002276501970000064
Can be expressed as:
Figure BDA00022765019700000713
subsequently, the locally uniform rotation invariant binary pattern operator (ULBP) pair is utilized
Figure BDA0002276501970000071
Each pixel in the image is operated to realize rotation invariance, and the calculation result is based on
Figure BDA0002276501970000072
Characteristic map of absolute value
Figure BDA0002276501970000073
The calculation formula is as follows:
Figure BDA0002276501970000074
wherein s ∈ { s ∈ [)1,s2,s3,s4,s5}; LBP stands for LBP operation; riu2 represents a rotationally invariant uniform pattern; d and E represent the number of surrounding pixels and the adjacent radius. Specifically, the number of peripheral pixel points D is set to 8, and the adjacent radius E is set to 1, so as to obtain 5 feature maps (each of which is represented by
Figure BDA0002276501970000075
And
Figure BDA0002276501970000076
) Wherein
Figure BDA0002276501970000077
The relationship between the center pixel and the neighboring pixels in the local area is described, and the local detail can effectively capture the complex degradation caused by different distortion types.
Although the local binary pattern can detect differences between the center pixel and its neighboring pixels, it cannot accurately capture gradient information, it encodes the differences of neighboring pixels, which impairs the ability of the local binary pattern to distinguish local variations. This is critical, and the change of local contrast has a great influence on the evaluation of the visual quality of the picture. It is well known that contrast transformations are highly correlated with picture visual quality. Thus, the present invention accumulates by the same local binary pattern operatorTo obtain a weighted histogram, defined as:
Figure BDA0002276501970000079
Figure BDA00022765019700000710
wherein N represents the number of picture pixels; k denotes the index of LBP, K ∈ [0, D +2 ]]And is and
Figure BDA00022765019700000711
as a weight, the value is a feature map
Figure BDA00022765019700000712
The method adopts Gaussian derivatives to collect, fuse and map pixel values of image intensity in the Gaussian derivatives, and obtains characteristic vectors through normalization operation; through these operations, the variation of high contrast in the picture region can be enhanced.
(2) Extracting color features:
in order to extract color characteristics, the invention adopts two color brightness which is not influenced by brightness on a chromaticity channel of a first-order Gaussian derivative; experiments prove that the first-order Gaussian derivative information of the color can be used for perceiving the degradation of local structures, wherein one color characteristic can be defined as:
wherein the content of the first and second substances,r, G, B represent the red, green and blue channels, respectively, in color space. Then, at x1Carry out LBPriu2Operating to extract feature maps
Figure BDA0002276501970000083
Then, the feature map is converted into a feature vector, and a calculation formula is defined as follows:
Figure BDA0002276501970000084
wherein the content of the first and second substances,
Figure BDA0002276501970000086
is a weight value, which is a characteristic map
Figure BDA0002276501970000087
Another characteristic is that
Figure BDA0002276501970000088
It is defined as follows:
Figure BDA0002276501970000089
Figure BDA00022765019700000810
where R ', G', and B 'respectively represent gaussian first derivative values of R, G, B channels in the horizontal direction, and ρ ═ 2R' -G '-B', δ ═ 2G '-R' -B ', τ ═ 2B' -R '-G'. Then, will
Figure BDA00022765019700000811
Operation applied to x2Calculating to obtain a characteristic map
Figure BDA00022765019700000812
The weight histogram is calculated as follows:
Figure BDA00022765019700000813
wherein the content of the first and second substances,
Figure BDA00022765019700000815
is defined as
Figure BDA00022765019700000816
The calculation formula is as follows:
Figure BDA00022765019700000817
color features are invariant to luminance and luminance-related scene information effects such as shadows, and therefore, they can represent powerful structural information as they are unaffected by illumination; furthermore, image distortions caused by a single factor (such as a blurring factor) can damage the structure of the image, but they are not necessarily related to the effect associated with brightness.
(3) Image naturalness characterization:
the brightness distortion of the three-dimensional composite image may affect the naturalness, and a high-definition three-dimensional composite image should have the natural characteristics of a natural picture. Therefore, the invention uses the quality characteristics based on brightness to evaluate the naturalness of the three-dimensional synthetic image, and takes the brightness parameters of the natural image into consideration to follow a Gaussian distribution, and uses the brightness coefficient to calculate the naturalness of the synthetic image; the luminance parameter (L') is defined as follows:
Figure BDA0002276501970000091
where (i, j) represents the spatial index, and i ∈ {1,2, …, a }, j ∈ {1,2, …, b }, where a and b represent the height and width of the image, respectively.
In particular, μ (i, j) and σ (i, j) are defined as follows:
Figure BDA0002276501970000092
Figure BDA0002276501970000093
where ω is a 2D centrosymmetric gaussian weight function sampled and rescaled to unity magnitude over three standard deviations, { ω ═ ωa,b|a∈[-3,3],b∈[-3,3]}。
The luminance parameter L' (i, j) is fitted with a zero mean generalized gaussian distribution, and the formula is defined as follows:
Figure BDA0002276501970000094
wherein the content of the first and second substances,
Figure BDA0002276501970000095
and is
Figure BDA0002276501970000096
Parameter α controls the general shape of the distribution, σ control variance, two parameters (α, σ)2) The evaluation is obtained through the model; and the kurtosis and skewness of the luminance coefficients are calculated from classical distributions from more than 5, yielding a total of 20 features.
Then, a Laplace pyramid is calculated by the difference between the synthesized image and its low-pass filtered version, a generalized Gaussian distribution model is used to fit the pixel value distribution in the Laplace pyramid, the kurtosis and skewness of the Laplace pyramid are taken as features, the invention extracts quality-conscious features from five levels, and 20 features are generated in total.
(4) Regression model and quality prediction:
studies have shown that multi-scale properties exist in the human visual system when perceiving visual information, and therefore extracting visual features that extract pictures on multiple scales can be better characterized. By representing the model through local features and global features, the invention can obtain a 310-dimensional feature vector in total, and the 310-dimensional feature vector comprises 270-dimensional local variation features and 40-dimensional global naturalness features; then, training a visual quality prediction model by using a random forest method, so that the quality characteristics can be mapped into subjective evaluation; by arbitrarily partitioning the database: 80% of the image samples and corresponding subjective evaluation scores were used for training in the database, and the remaining 20% were used for testing; finally, the pierce linear correlation coefficient mean (PLCC), the impersonate scale correlation coefficient (SRCC), the kender scale correlation coefficient (KRCC), and the Root Mean Square Error (RMSE) are summarized as final results.
The process of the invention is shown in figure 1, and the specific process is as follows:
step 1: extracting structural and color features using gaussian derivatives;
step 2: using local binary patterns to encode a structural and color feature map for computing quality-aware features to derive local structural and color distortions;
and step 3: extracting brightness characteristics through global change to evaluate the naturalness of the three-dimensional synthetic image;
and 4, step 4: based on the extracted feature information, a quality prediction model is trained using random forest regression mapping from visual features to subjective scores.
The pierce linear correlation coefficient mean (PLCC), the impersonate rank correlation coefficient (SRCC), the kendell correlation coefficient (KRCC), and the Root Mean Square Error (RMSE) were used as final comparison results. Generally speaking, higher PLCC and SRCC and lower RMSE values represent better performance of the algorithm, i.e., better algorithm prediction accuracy. In order to verify the performance of the algorithm provided by the invention, the algorithm is compared with the existing reference and non-reference quality evaluation methods in three public databases MCL-3D, IRCCyN/IVC and IETR-DIBR, and the comparative evaluation methods comprise PSNR, SSIM, BRISQE, NIQE, BIQI, NRSL, CM-LOG, MP-PSNRr, MW-PSNR, MW-PSNRr, LOGS, Ref, APT and NIQSV +; the first seven methods are quality evaluation methods for natural pictures, and the last seven methods are quality evaluation methods designed particularly for synthetic view angles.
The MCL-3D database contains 693 stereoscopic groups of pictures selected from nine depth image information sources. The IRCCyN/IVC DIBR database consists of 12 reference pictures chosen from three MVD sequences and 84 synthetic pictures generated by 7 different DIBR techniques. The IETR DIBR database consists of 150 synthetic pictures generated by 10 MVD sequences and 7 latest DIBR technologies, and similar to the RCCyN/IVC DIBR database, the IETR DIBR database is also mainly concerned with rendering distortion.
Table 1: comparison of the present invention with existing full reference methods
Figure BDA0002276501970000111
Figure BDA0002276501970000121
Table 1 shows the comparison of the proposed method with the existing full reference method, from which the proposed no reference method of the present invention performs better.
Table 2: comparison of the present invention with existing reference-free methods
Figure BDA0002276501970000122
Figure BDA0002276501970000131
Table 2 is the comparison of the proposed method to the existing no reference method, from which the proposed no reference method of the present invention performed better on the tested database.
The above-described embodiments are illustrative of the present invention and not restrictive, it being understood that various changes, modifications, substitutions and alterations can be made herein without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims (9)

1. A three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation is characterized in that: the method comprises the following steps:
A. extracting the structure and color characteristics of the synthesized image by using a Gaussian derivative;
B. coding the obtained structure and color characteristics by using a local binary pattern respectively to obtain a structure characteristic diagram and a color characteristic diagram, and calculating the structure characteristic diagram and the color characteristic diagram respectively based on the structure characteristic diagram and the color characteristic diagram to obtain distortion information of the local structure and the color;
C. extracting brightness characteristics to evaluate the naturalness of the three-dimensional synthetic image for global change;
D. and (4) combining the extracted characteristic information, and using a random forest regression model to learn the mapping relation between the visual characteristics and the subjective quality scores to predict the quality scores of the three-dimensional synthetic images.
2. The method of claim 1, wherein: structural features and color features of the image are extracted using gaussian derivatives.
3. The method of claim 2, wherein: the structural features of the image are extracted by using Gaussian derivatives, and the method comprises the following specific steps:
A. the local Taylor series expansion can represent the local characteristics of the image, and the coefficients of the local Taylor series can be obtained through local Gaussian derivatives; the gaussian derivative of an image can be defined as follows:
Figure FDA0002276501960000011
where m ≧ 0 and n ≧ 0 are derivatives along the horizontal x and vertical y directions, the symbol denotes the convolution operation; gσ(x, y, σ) is a Gaussian function whose standard deviation σ is defined as follows:
B. using the second order Gaussian derivative to extract structural features, first calculating the sum of m + n < 1 > and 2
Figure FDA0002276501960000014
By calculating the Gaussian derivatives, the resulting matrix
Figure FDA0002276501960000015
Can be expressed as:
Figure FDA0002276501960000013
4. the method of claim 2, wherein: extracting color features of the image by using Gaussian derivatives, which comprises the following specific steps:
A. two color features that are not affected by luminance are employed on the first order gaussian derivative of the color channel, one of which is defined as follows:
Figure FDA0002276501960000021
wherein the content of the first and second substances,
Figure FDA0002276501960000022
r, G and B respectively represent red, green and blue channels in a color space; another color characteristic
Figure FDA0002276501960000025
The definition is as follows:
Figure FDA0002276501960000024
wherein R ', G', and B 'respectively represent gaussian first derivative values of R, G, and B channels in the horizontal direction, and ρ ═ 2R' -G '-B', δ ═ 2G '-R' -B ', τ ═ 2B' -R '-G'.
5. The method of claim 2, wherein: and obtaining a structure characteristic graph and a color characteristic graph by respectively using the obtained structure and color characteristics through a local binary pattern, and calculating quality characteristics so as to obtain distortion information of the local structure and color.
6. The method of claim 5, wherein: obtaining a structure chart by using a local binary pattern method to obtain distortion information of a local structure, wherein the method comprises the following specific steps of:
A. binary pattern method pair using local rotation invariance
Figure FDA0002276501960000026
Each pixel of the image is operated on, the calculation is based onCharacteristic map of absolute value
Figure FDA0002276501960000028
The calculation formula is as follows:
Figure FDA0002276501960000029
wherein s ∈ { s ∈ [)1,s2,s3,s4,s5}; LBP stands for LBP operation; riu2 represents a rotation invariant unified mode; d and E represent the number and the calculation radius of surrounding pixels, the number D of the surrounding pixels is set to be 8, and the calculation radius E is 1; thereby obtaining 5 characteristic maps which are respectively
Figure FDA00022765019600000210
And
Figure FDA00022765019600000211
wherein
Figure FDA00022765019600000212
Describing the relationship between the central pixel point and the adjacent pixel points in the local area;
B. representing local structure distortion information by using weighted histogram, and using same local binary pattern operator pair
Figure FDA00022765019600000213
The pixels of (a) are accumulated to obtain a weighted histogram, which is defined by the following formula:
Figure FDA0002276501960000031
Figure FDA0002276501960000032
wherein N represents the number of picture pixels; k denotes the index of LBP, K ∈ [0, D +2 ]],
Figure FDA0002276501960000038
Is a weight value which is a characteristic map
Figure FDA0002276501960000039
And summarizing the Gaussian derivatives to fuse and map pixel values in the Gaussian derivatives according to the LBP image intensity value, and obtaining a characteristic vector through normalization operation to enhance the change of high contrast in the image area so as to reflect local structure distortion information.
7. The method of claim 5, wherein: obtaining a chromaticity diagram by using coding to obtain distortion information of local chromaticity; wherein the content of the first and second substances,
A. obtaining a feature vector by using a local binary model: at the extracted color feature x1Carry out LBPriu2Operation acquisition feature mapThe feature map is then converted into feature vectors, which define the formula:
Figure FDA0002276501960000033
Figure FDA0002276501960000034
wherein the content of the first and second substances,
Figure FDA00022765019600000311
is a weight value obtained by a local binary pattern operator, and the value is a characteristic diagram
Figure FDA00022765019600000312
B. The local chrominance information is represented by a weighted histogram: at the extracted color feature x2Carry out LBPriu2Operation acquisition feature map
Figure FDA00022765019600000313
Then x2The weighted histogram calculation of (a) is defined as follows:
Figure FDA0002276501960000035
Figure FDA0002276501960000036
wherein, the weight value
Figure FDA00022765019600000314
Is a characteristic map
Figure FDA00022765019600000315
Finally, a single feature vector representing the color information of the image is calculated by the following formula:
Figure FDA0002276501960000037
8. the method of claim 1, wherein: for global variation, the naturalness of the three-dimensional synthetic image is evaluated by extracting the brightness features: wherein the content of the first and second substances,
A. the luminance coefficient is fitted using a gaussian distribution, and the luminance coefficient (L') is defined as follows:
Figure FDA0002276501960000041
where (i, j) represents the spatial position of the pixel, and i e {1,2, …, a }, j e {1,2, …, b }, where a and b represent the height and width of the image, respectively, and μ (i, j) and σ (i, j) are defined as follows:
Figure FDA0002276501960000042
Figure FDA0002276501960000043
where ω is a two-dimensional, centrosymmetric gaussian weighting function, ω ═ ωa,b|a∈[-3,3],b∈[-3,3]}；
The luminance parameter L' (i, j) is then fitted using a zero-mean generalized gaussian distribution, which defines the formula:
Figure FDA0002276501960000044
wherein the content of the first and second substances,
Figure FDA0002276501960000045
and is
Figure FDA0002276501960000046
Where parameter α controls the shape of the distribution, σ controls the variance;
B. subsequently, 4 parameters including shape parameters and variance of the generalized gaussian distribution and kurtosis and skewness of the luminance coefficient are calculated on 5 scales of the composite image, resulting in a total of 20-dimensional features; in addition, a laplacian pyramid image is computed from the difference between the composite image and its low-pass filtered image, the shape coefficients and variances are obtained using a generalized gaussian distribution model to fit the pixel values in the laplacian pyramid, the kurtosis and skewness of the laplacian pyramid are computed, and the four parameters are extracted from the five scales, yielding a total of 20-dimensional features.
9. The method of claim 1, wherein: a quality prediction model is trained using a random forest regression method, wherein,
A. selecting feature information, namely obtaining a total 310-dimensional feature vector through local features and global features, wherein the total 310-dimensional feature vector comprises 270-dimensional local variation features and 40-dimensional global naturalness features;
B. and training a visual quality prediction model by using a random forest regression method, and mapping the quality characteristics to subjective evaluation. The three-dimensional synthetic view quality database is randomly divided into a training set and a testing set to be trained for 1000 times, and finally, a pierce linear correlation coefficient mean (PLCC), a pierce Mandarin level correlation coefficient (SRCC), a Kendall level correlation coefficient (KRCC) and a Root Mean Square Error (RMSE) are used as final results.
CN201911124950.4A 2019-11-18 2019-11-18 Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation Pending CN110853027A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911124950.4A CN110853027A (en) 2019-11-18 2019-11-18 Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911124950.4A CN110853027A (en) 2019-11-18 2019-11-18 Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation

Publications (1)

Publication Number Publication Date
CN110853027A true CN110853027A (en) 2020-02-28

Family

ID=69600595

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911124950.4A Pending CN110853027A (en) 2019-11-18 2019-11-18 Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation

Country Status (1)

Country Link
CN (1) CN110853027A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112288699A (en) * 2020-10-23 2021-01-29 北京百度网讯科技有限公司 Method, device, equipment and medium for evaluating relative definition of image
CN112785494A (en) * 2021-01-26 2021-05-11 网易（杭州）网络有限公司 Three-dimensional model construction method and device, electronic equipment and storage medium
CN113643262A (en) * 2021-08-18 2021-11-12 上海大学 No-reference panoramic image quality evaluation method, system, equipment and medium
CN115511833A (en) * 2022-09-28 2022-12-23 广东百能家居有限公司 Glass surface scratch detection system
CN116309216A (en) * 2023-02-27 2023-06-23 南京博视医疗科技有限公司 Pseudo-color image fusion method and image fusion system based on multiple wave bands
CN116758060A (en) * 2023-08-10 2023-09-15 江苏森标科技有限公司 Vertical basket of flowers visual detection system of battery piece

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106408561A (en) * 2016-09-10 2017-02-15 天津大学 Texture feature-based image quality evaluating method without reference
CN108010024A (en) * 2017-12-11 2018-05-08 宁波大学 It is a kind of blind with reference to tone mapping graph image quality evaluation method
CN109919959A (en) * 2019-01-24 2019-06-21 天津大学 Tone mapping image quality evaluating method based on color, naturality and structure
CN110046673A (en) * 2019-04-25 2019-07-23 上海大学 No reference tone mapping graph image quality evaluation method based on multi-feature fusion
CN110363704A (en) * 2019-05-29 2019-10-22 西北大学 Merge the image super-resolution rebuilding model construction and method for reconstructing of form and color

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106408561A (en) * 2016-09-10 2017-02-15 天津大学 Texture feature-based image quality evaluating method without reference
CN108010024A (en) * 2017-12-11 2018-05-08 宁波大学 It is a kind of blind with reference to tone mapping graph image quality evaluation method
CN109919959A (en) * 2019-01-24 2019-06-21 天津大学 Tone mapping image quality evaluating method based on color, naturality and structure
CN110046673A (en) * 2019-04-25 2019-07-23 上海大学 No reference tone mapping graph image quality evaluation method based on multi-feature fusion
CN110363704A (en) * 2019-05-29 2019-10-22 西北大学 Merge the image super-resolution rebuilding model construction and method for reconstructing of form and color

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112288699A (en) * 2020-10-23 2021-01-29 北京百度网讯科技有限公司 Method, device, equipment and medium for evaluating relative definition of image
CN112288699B (en) * 2020-10-23 2024-02-09 北京百度网讯科技有限公司 Method, device, equipment and medium for evaluating relative definition of image
CN112785494A (en) * 2021-01-26 2021-05-11 网易（杭州）网络有限公司 Three-dimensional model construction method and device, electronic equipment and storage medium
CN112785494B (en) * 2021-01-26 2023-06-16 网易（杭州）网络有限公司 Three-dimensional model construction method and device, electronic equipment and storage medium
CN113643262A (en) * 2021-08-18 2021-11-12 上海大学 No-reference panoramic image quality evaluation method, system, equipment and medium
CN115511833A (en) * 2022-09-28 2022-12-23 广东百能家居有限公司 Glass surface scratch detection system
CN116309216A (en) * 2023-02-27 2023-06-23 南京博视医疗科技有限公司 Pseudo-color image fusion method and image fusion system based on multiple wave bands
CN116309216B (en) * 2023-02-27 2024-01-09 南京博视医疗科技有限公司 Pseudo-color image fusion method and image fusion system based on multiple wave bands
CN116758060A (en) * 2023-08-10 2023-09-15 江苏森标科技有限公司 Vertical basket of flowers visual detection system of battery piece
CN116758060B (en) * 2023-08-10 2023-10-27 江苏森标科技有限公司 Vertical basket of flowers visual detection system of battery piece

Similar Documents

Publication Publication Date Title
CN110853027A (en) Three-dimensional synthetic image no-reference quality evaluation method based on local variation and global variation
CN107767413B (en) Image depth estimation method based on convolutional neural network
Yue et al. Combining local and global measures for DIBR-synthesized image quality evaluation
Guttmann et al. Semi-automatic stereo extraction from video footage
CN104899845B (en) A kind of more exposure image fusion methods based on the migration of l α β spatial scenes
Jiang et al. Image dehazing using adaptive bi-channel priors on superpixels
RU2426172C1 (en) Method and system for isolating foreground object image proceeding from colour and depth data
CN105894484B (en) A kind of HDR algorithm for reconstructing normalized based on histogram with super-pixel segmentation
Dudhane et al. C^ 2msnet: A novel approach for single image haze removal
CN108898575B (en) Novel adaptive weight stereo matching method
CN109685045B (en) Moving target video tracking method and system
CN107635136B (en) View-based access control model perception and binocular competition are without reference stereo image quality evaluation method
CN109345502B (en) Stereo image quality evaluation method based on disparity map stereo structure information extraction
AU2016302049C1 (en) 2D-to-3D video frame conversion
CN108596975A (en) A kind of Stereo Matching Algorithm for weak texture region
CN109242834A (en) It is a kind of based on convolutional neural networks without reference stereo image quality evaluation method
CN108537782A (en) A method of building images match based on contours extract with merge
CN107633495A (en) A kind of infrared polarization based on complementary relationship and the more embedded fusion methods of algorithm 2D VMD of intensity image
CN112950596A (en) Tone mapping omnidirectional image quality evaluation method based on multi-region and multi-layer
Kuo et al. Depth estimation from a monocular view of the outdoors
CN110910365A (en) Quality evaluation method for multi-exposure fusion image of dynamic scene and static scene simultaneously
CN115953321A (en) Low-illumination image enhancement method based on zero-time learning
CN109218706A (en) A method of 3 D visual image is generated by single image
CN105528772B (en) A kind of image interfusion method based on directiveness filtering
Calagari et al. Data driven 2-D-to-3-D video conversion for soccer

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination