JP2015232770A - Image quality evaluation device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To evaluate image quality in consideration of an error evaluation value and an image feature value.SOLUTION: An image quality evaluation device 1 includes: an error evaluation value calculation part 10 that calculates an error evaluation value that is an index of an error between an evaluation object image and a reference image; an image feature value calculation part 20 that calculates an image feature value that is an index of the image feature of the evaluation object image, which is calculated by convolution on the evaluation object image; and an addition part 30 that calculates the sum or the weighted sum of the error evaluation value and the image feature value as an evaluation value of the evaluation object image.

Description

  The present invention relates to an image quality evaluation apparatus and program for objectively evaluating image quality.

  As an image quality evaluation method, there is a reference image that can be referred to as an ideal image, and there is a method for evaluating the degree of deterioration by comparing a reference image with an evaluation target image. This method is FR (Full Reference). This is called a type objective evaluation method. As a typical example of the FR type objective evaluation method, there is a method using an objective evaluation index such as PSNR (Peak Signal to Noise Ratio) or SSIM (Structural Similarity). PSNR is an index obtained by logarithmically evaluating the square error between the image to be evaluated and the reference image, but SNR (Signal to Noise Ratio) is widely used for evaluating various signals such as voice and radio waves. It is most popular as an objective evaluation index because it is familiar and easy to implement with hardware and software. Further, SSIM is an index for evaluating the similarity of the local structure between the evaluation target image and the reference image, and has recently been attracting attention as being able to obtain an evaluation close to human subjectivity.

  On the other hand, there is a method for evaluating image quality using only image feature quantities such as a TV (Total Variation) norm for the evaluation target image in a situation where no reference image exists. This method is an NR (Non Reference) type objective evaluation method. is called. Further, as an NR type objective evaluation method, a method of estimating PSNR from DCT coefficient information of an MPEG-2 bit stream has been put into practical use (for example, see Patent Document 1).

Japanese Patent No. 4309703

  However, the PSNR may not match the subjective evaluation value, that is, human intuition. For example, if the entire image is shifted by one pixel, the subjective evaluation is hardly lowered, but the PSNR may be greatly lowered, and this tendency is particularly strong in the case of a fine image. In addition, slight differences in image contrast and geometric distortions that do not significantly affect subjective evaluation also cause a significant decrease in PSNR.

  Since SSIM focuses on local structure, problems such as those in the case of PSNR are unlikely to occur, and the obtained index is also close to subjectivity. However, there is a problem that the calculation cost is high because it is necessary to obtain the average, variance, covariance, and the like of the target image and the reference image in the block while scanning a local block set in the screen.

  Moreover, although the technique of patent document 1 is also implemented in hardware and real-time processing is possible, the obtained index is an estimated value of PSNR. In order to realize an evaluation closer to the subjective evaluation value, it is necessary to consider an image feature amount such as a pixel value variation of the evaluation target image in addition to an error of the evaluation target image with respect to a reference image such as PSNR.

  An object of the present invention made in view of such circumstances is to consider an image quality value in consideration of an error evaluation value that is an index of error between an evaluation target image and a reference image and an image feature value that is an index of an image feature amount of the evaluation target image. It is an object of the present invention to provide an image quality evaluation apparatus and program capable of evaluating the image quality.

  In order to solve the above problems, an image quality evaluation apparatus according to the present invention is an image quality evaluation apparatus that outputs an evaluation value of an image quality of an evaluation target image, and an error evaluation value that is an index of an error between the evaluation target image and the reference image. An error evaluation value calculation unit to calculate, an image feature value calculation unit to calculate an image feature value, which is an index of an image feature amount of the evaluation target image, calculated by a convolution operation of the evaluation target image, and the error evaluation value And an adding unit that calculates a sum of the image feature values or a weighted sum as an evaluation value of the evaluation target image.

  Furthermore, in the image quality evaluation apparatus according to the present invention, the error evaluation value calculation unit includes a subtraction unit that calculates a difference value between the evaluation target image and the reference image for each pixel, and the difference value is stored in a first lookup table. A first LUT conversion unit that converts the difference values converted by the first LUT conversion unit within a predetermined area, and a first summation unit that calculates a summation value as the error evaluation value It is characterized by providing.

  Furthermore, in the image quality evaluation apparatus according to the present invention, the error evaluation value calculation unit further includes a second LUT conversion unit that converts the total value with reference to a second look-up table, and the second LUT conversion unit. The total value converted by the conversion unit is used as the error evaluation value.

  Furthermore, in the image quality evaluation apparatus according to the present invention, the image feature value calculation unit includes a convolution operation unit that performs a convolution operation on each of the evaluation target images using one or more types of convolution coefficients to generate a convolution image. A third LUT converter that converts the pixel values of the convolution image with reference to a third lookup table, and a sum value obtained by summing the pixel values converted by the third LUT converter in a predetermined area And a second summation unit that calculates the image feature value as the image feature value.

  In order to solve the above problem, a program according to the present invention causes a computer to function as the image quality evaluation apparatus.

  According to the present invention, it is possible to evaluate image quality in consideration of an error evaluation value that is an index of error between an evaluation target image and a reference image and an image feature value that is an index of an image feature amount of the evaluation target image.

It is a block diagram which shows the structural example of the image quality evaluation apparatus which concerns on one Embodiment of this invention. It is a flowchart which shows the process example of the image quality evaluation apparatus which concerns on one Embodiment of this invention.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration of an image quality evaluation apparatus according to an embodiment of the present invention. In the example illustrated in FIG. 1, the image quality evaluation apparatus 1 includes an error evaluation value calculation unit 10, an image feature value calculation unit 20, a delay unit 30, and a second addition unit 40.

  The image quality evaluation apparatus 1 evaluates the image quality in the region B with respect to the input of the evaluation target image Y and the reference image X, and outputs an evaluation value e (B). In the following description, the pixel value at the image coordinates (i, j) of the evaluation target image Y is denoted as Y (i, j), and the pixel value at the image coordinates (i, j) of the reference image X is X (i, j). j).

  The error evaluation value calculation unit 10 calculates an error evaluation value that is an index of error between the evaluation target image Y and the reference image X. The error evaluation value calculation unit 10 includes a subtraction unit 11, a first LUT conversion unit 12, a first summation unit 13, and a second LUT conversion unit 14. Note that the second LUT conversion unit 14 is not an essential constituent block and may be omitted.

  The image feature value calculation unit 20 calculates an image feature value that is an index of the image feature amount of the evaluation target image Y, which is calculated by a convolution operation. The image feature value calculation unit 20 includes a convolution coefficient table 21, one or more convolution operation units 22, a third LUT conversion unit 23, a first addition unit 24, and a second summation unit 25. In the embodiment shown in FIG. 1, the number of convolution operation units 22 is two (22-1 and 22-2).

  The image quality evaluation apparatus 1 performs the calculation shown in Expression (1). Note that LUT. [·] Represents a one-dimensional lookup table. [•] [•] represent a two-dimensional lookup table.

The image quality evaluation apparatus 1 sequentially inputs a pixel value sequence obtained by raster scanning the reference image X by the scanning unit 2 and a pixel value sequence obtained by raster scanning the evaluation target image Y by the scanning unit 3.

  The subtraction unit 11 subtracts the pixel value X (i, j) of the reference image X from the pixel value Y (i, j) of the evaluation target image Y that is sequentially input, d = Y (i, j) −X (I, j) is calculated and output to the first LUT converter 12.

The first LUT conversion unit 12 converts the difference value d with reference to the lookup table LUT _D. The value set in LUT _D is arbitrary, but it is preferable that the output value does not decrease as the absolute value of the difference value d increases. For example, the first LUT conversion unit 12 performs the conversion shown in Expression (2) or Expression (3). Further, the conversion value may be clipped as shown in Expression (4).

The first summation unit 13 can be configured by an adder circuit and a delay circuit (z ⁻¹ ). The first summation unit 13 calculates a summation value by accumulating error evaluation values that arrive from the first LUT conversion unit 12 every moment. The data is output to the conversion unit 14. Note that the initial value of the output value of the delay circuit is 0. The first summation unit 13 scans all the pixel values Y (i, j) of the evaluation target image Y and the pixel values X (i, j) of the reference image X within the region of (i, j) ∈B. Thus, the total value shown in the equation (5) is calculated. When the error evaluation value calculation unit 10 does not include the second LUT conversion unit 14, the total value is output to the delay unit 30 as an error evaluation value.

The second LUT conversion unit 14 converts the sum value calculated by the first summation unit 13 with reference to the lookup table LUT _E , and outputs the converted value to the delay unit 30 as an error evaluation value. For example, when the second LUT conversion unit 14 performs the conversion of Expression (6), when the first LUT conversion unit performs the conversion of Expression (2), the second LUT conversion unit 14 refers to the evaluation target image Y. RMSE (Root Mean Square Error) in the region B of the image X is output. Here, N represents the number of pixels in the region B.

The convolution coefficient table 21 holds sets of convolution coefficients (convolution kernels) as many as the number of convolution operation units 22. In the example of FIG. 1, the convolution coefficient table 21 holds two sets (C ₀ and C ₁ ) of secondary convolution coefficients of 3 × 3 pixels. For example, as the convolution coefficients C ₀ and C ₁ , gradients (primary differentiation) in the horizontal direction and the vertical direction are set as in Expression (7).

The convolution operation unit 22-1 performs a convolution operation on the evaluation target image Y using the convolution coefficient C ₀ stored in the convolution coefficient table 21 to generate a convolution image Z ₀ and outputs the convolution image Z ₀ to the third LUT conversion unit 23. To do. The convolution operation unit 22-2 performs a convolution operation on the evaluation target image Y using the convolution coefficient C ₁ stored in the convolution coefficient table 21 to generate a convolution image Z _1, and outputs it to the third LUT conversion unit 23. To do. The convolution operation units 22-1 and 22-2 can be configured by an adder circuit, a multiplier circuit, and a delay circuit. The pixel value Z (i, j) at the image coordinates (i, j) of the convolution image Z is expressed by Expression (8).

In Expression (8), m is an index for distinguishing a plurality of convolution operation units. In the example of FIG. 1, mε {0, 1}. Further, the region K _m is the expansion of the m-th convolution coefficient C _m kernel, and is represented by Expression (9) in the example of FIG.

The third LUT conversion unit 23 refers to the lookup table LUT _F [m] for each pixel value Z _m (i, j) of the convolution image Z _m calculated by the convolution operation unit 22-1 and the convolution operation unit 22-2. The converted image W _m is output to the first adder 24. For example, the third LUT conversion unit 23 performs the conversion shown in Expression (10). The pixel value W _m (i, j) at the image coordinates (i, j) of the converted image W _m is expressed by Expression (11).

The first addition unit 24 adds the converted image W _m converted by the third LUT conversion unit 23 for each pixel and outputs the result to the second summation unit 25.

The second summation unit 25 can be composed of an addition circuit and a delay circuit (z ⁻¹ ), and obtains the summation by accumulating values that arrive from the first addition unit 24 every moment. Note that the initial value of the output value of the delay circuit (z ⁻¹ ) is 0. In the example of FIG. 1, after 2L + 2 clocks have elapsed after the number of clocks required to scan all within the region of (i, j) εB, the second summation unit 25 performs the summation value represented by Expression (12). Are calculated as image feature values.

  The delay unit 30 adjusts the timing of the output signals of the error evaluation value calculation unit 10 and the image feature value calculation unit 20. In the example of FIG. 1, considering the delay in the convolution operation unit 22, the number of horizontal pixels in the region B is set to L, and the delay unit 30 compensates for the time required for scanning for 2L + 2 pixels.

  The second adder 40 calculates the sum or weighted sum of the error evaluation value input from the error evaluation value calculator 10 and the image feature value input from the image feature value calculator 20 as the evaluation value e of the evaluation target image Y. Output as (B). In the example of FIG. 1, the second adder 40 outputs the value of the expression (1) after 2L + 2 clocks have elapsed after the number of clocks required to scan all within the region of (i, j) εB. . For example, when the set values of the formulas (2), (6), (7), and (10) are employed, the formula (1) is expressed by the formula (13).

  Expression (13) represents the sum of the RMSE planting and the TV norm value in the region B of the evaluation target image Y. The index of Expression (13) is an index that takes into account both the similarity of the evaluation target image Y to the reference image X and the small variation in the pixel value of the evaluation target image Y. That is, Expression (13) is an objective index that takes into consideration both the small error and the small ringing.

  FIG. 2 is a flowchart illustrating a processing example of the image quality evaluation apparatus 1. Although step S1 for calculating the error evaluation value and step S2 for calculating the image feature value can be performed in parallel at the same time, FIG. 2 is a flowchart in which step S2 is performed after step S1 for convenience of explanation.

In step S1 for calculating the error evaluation value, a difference value d between the evaluation target image Y and the reference image X is calculated for each pixel (step S11), and the difference value d is converted by referring to the first lookup table LUT _D. In step S12, a first total value obtained by summing the converted difference value d in the predetermined area B is calculated (step S13). Then, the first total value is converted with reference to the second lookup table LUT _E to obtain an error evaluation value (step S14). This step S14 may not be necessary. In this case, the first total value calculated in step S13 is set as the error evaluation value.

  For example, non-linear calculation or linear calculation including square calculation and absolute value calculation is performed in step S12, and square error sum (SSD) or absolute value error sum (SAD) is calculated in step S13. An error evaluation value including is obtained. In step S14, an error evaluation value including RMSE and PSNR is obtained.

In step S2 for calculating the image feature value, a convolution image is generated by performing each convolution operation on the evaluation target image Y using one or more types of convolution coefficients (step S21), and the pixel value of the convolution image is set to the third value. Is converted with reference to the look-up table LUT _F (step S22), and a second total value obtained by summing the converted pixel values in the predetermined area B is calculated as an image feature value (step S23).

  For example, in step S21, one or more filtering processes including a horizontal or vertical gradient (first derivative) calculation are performed, and in step S22, a non-linear calculation or a linear calculation including a square calculation or an absolute value calculation is performed. In S23, for example, an image feature value including the absolute value sum (TV norm) of the gradient in the horizontal direction and the vertical direction is obtained.

  Finally, the error evaluation value calculated in step S1 and the sum or weighted sum of the image feature values calculated in step S2 are calculated as the evaluation value e (B) of the evaluation target image Y (step S3).

  Therefore, according to the image quality evaluation apparatus 1 according to the present invention, the error evaluation value that is an index of error between the evaluation target image Y and the reference image X and the image feature value that is an index of the image feature amount of the evaluation target image Y are obtained. The image quality can be evaluated in consideration. Further, the evaluation value can be obtained only by a simple circuit such as a look-up table, an adder circuit, a multiplier circuit, and a delay circuit, and mounting by a logic circuit such as an FPGA (Field Programmable Gate Array) becomes easy. Further, by providing the second LUT conversion unit 14, a nonlinear evaluation function such as a logarithm can be applied as the error evaluation value.

  As a modification of the image quality evaluation apparatus 1, as with the evaluation target image Y, the reference image X is also subjected to image feature quantities by the convolution coefficient table 21, the convolution operation unit 22, the third LUT conversion unit 23, and the first addition unit 24. The error or ratio between the image feature quantity of the reference image X and the image feature quantity of the evaluation target image Y may be used as the final image feature quantity of the evaluation target image Y.

  In addition, a computer can be suitably used to cause the image quality evaluation apparatus 1 to function as described above, and such a computer stores a program describing processing contents for realizing each function of the image quality evaluation apparatus 1. This program can be realized by reading out and executing this program by the CPU of the computer. This program can be recorded on a computer-readable recording medium.

  Although the above embodiment has been described as a representative example, it will be apparent to those skilled in the art that many changes and substitutions can be made within the spirit and scope of the invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and changes can be made without departing from the scope of the claims. For example, a plurality of constituent blocks described in the embodiments can be combined into one, or one constituent block can be divided.

DESCRIPTION OF SYMBOLS 1 Image quality evaluation apparatus 10 Error evaluation value calculation part 11 Subtraction part 12 1st LUT conversion part 13 1st summation part 14 2nd LUT conversion part 20 Image feature value calculation part 21 Convolution coefficient table 22 Convolution calculation part 23 3rd LUT conversion part 24 1st Adder 25 Second total sum 30 Delay unit 40 Second adder

Claims (5)

An image quality evaluation apparatus that outputs an evaluation value of an image quality of an evaluation target image,
An error evaluation value calculation unit that calculates an error evaluation value that is an index of error between the evaluation target image and the reference image;
An image feature value calculation unit that calculates an image feature value that is an index of an image feature amount of the evaluation target image, calculated by a convolution operation of the evaluation target image;
An adder that calculates a sum or weighted sum of the error evaluation value and the image feature value as an evaluation value of the evaluation target image;
An image quality evaluation apparatus comprising: The error evaluation value calculation unit
A subtraction unit that calculates a difference value between the evaluation target image and the reference image for each pixel;
A first LUT converter that converts the difference value with reference to a first lookup table;
A first summation unit that calculates, as the error evaluation value, a summation value obtained by summing the difference values transformed by the first LUT transformation unit within a predetermined area;
The image quality evaluation apparatus according to claim 1, further comprising: The error evaluation value calculation unit
A second LUT converter that converts the sum value with reference to a second lookup table;
The image quality evaluation apparatus according to claim 2, wherein a sum value converted by the second LUT conversion unit is used as the error evaluation value. The image feature value calculation unit
A convolution operation unit that performs a convolution operation on each of the evaluation target images using one or more types of convolution coefficients, and generates a convolution image;
A third LUT converter that converts the pixel value of the convolution image with reference to a third lookup table;
A second summation unit that calculates a sum value obtained by summing the pixel values converted by the third LUT conversion unit in a predetermined area as the image feature value;
The image quality evaluation apparatus according to claim 1, further comprising:   The program for functioning a computer as an image quality evaluation apparatus as described in any one of Claim 1 to 4.

Images