JPH11215523A - Image quality evaluation device for evaluating deterioration in image quality accompanying image processing - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image quality evaluation device that evaluates image quality deterioration accompanying image processing, where the evaluation result is well in agreement with a subjective evaluation by humans. SOLUTION: This device is provided with a differential arithmetic section 1 that applies differential arithmetic operation between both image signals before and after image processing, a time space arithmetic section 2 that includes a time frequency filter 13 and a space frequency filter 14, each receiving a half of outputs of the differential arithmetic section 1 to calculate a response in temporal frequency and spatial frequency, where the calculation of the response of the temporal frequency filter 13 and the spatial frequency filter 14 is controlled by an image signal level before or after the image processing, and a time space integration section 3 that applies integration in space and time directions to signals outputted from the time space arithmetic section 2 to output an evaluation value.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image quality evaluation apparatus, and more particularly, to an image quality evaluation apparatus which evaluates image quality deterioration due to image processing such as digital compression / expansion on behalf of a human and outputs an evaluation value close to the subjective evaluation of a human. The present invention relates to a possible image quality evaluation device.

[0002]

2. Description of the Related Art Digital compression / expansion of image signals, component-to-composite conversion, or analog processing involves image quality deterioration although varying in degree. Conventionally, in order to evaluate the degree of image quality degradation with an evaluation value close to the subjective evaluation, a compressed image evaluation system (Television Society of Japan) which devised spatial processing such as weighting in consideration of visual characteristics in blocks Technical Report ITE Technical Report
Vol. 20, No. 35, "Development of Digital Compressed Image Evaluation System", announced on June 20, 1996). In the field of noise meters, there is also a device (a noise meter manufactured by Rohde & Schwarz) that measures through a weighting filter adapted to the characteristics of visual perception.

[0003]

However, since a digitally compressed / decompressed coded image is subjected to various signal processings on the image before the compression / decompression, the signal distortion due to the processing is reduced in the screen. It is not constant in the time direction and changes in various factors such as the type of image, complexity and vicinity of the edge, and it is difficult to bring the measurement result close to the subjective evaluation value in the previous measurement using the above-mentioned measurement methods and devices. there were.

In addition, the evaluation of signal distortion due to digital compression / expansion can be approximated to a subjective evaluation value to some extent by a method of weighting the evaluation value based on the complexity of the image or whether or not the image is near an edge. , Paper (N
HK Giken R & D No. 36, "Study of Objective Evaluation Scale for Digitally Coded Images for Broadcasting," published on May 15, 1995). Could not approach the subjective evaluation value.

SUMMARY OF THE INVENTION An object of the present invention is to solve the above-mentioned problems (problems to be solved) of the conventional image quality evaluation apparatus, and to provide an image processing apparatus capable of obtaining a result whose evaluation result accurately matches a subjective evaluation value performed by a human. It is an object of the present invention to provide an image quality evaluation device for evaluating image quality degradation accompanying the image quality.

[0006]

According to the image quality evaluation apparatus of the present invention, when a human performs a subjective evaluation of a television image or the like, the characteristics of human vision have a large brightness dependency, and the image is bright in the image. The present inventors have focused on the fact that the same amount of distortion or noise looks completely different between a place (high signal level) and a dark place (low signal level). By configuring the image quality evaluation device using a spatio-temporal filter depending on the signal level (brightness), an image quality evaluation device for evaluating image quality degradation due to image processing that results from approaching a subjective evaluation value is obtained. It has been realized.

That is, an image quality evaluation apparatus for evaluating image quality deterioration due to image processing according to the present invention comprises: a difference operation section for performing a difference operation between image signals before and after image processing; In order to calculate the response at the time frequency and the spatial frequency with respect to the output signal from the calculating unit, the variable output unit includes a variable time frequency filter and a spatial frequency filter to which the output of the difference calculating unit is supplied. A spatio-temporal calculation unit configured to control the response of the variable time-frequency filter and the spatial-frequency filter by an image signal level before or after performing the image processing; At least a spatiotemporal integration unit that performs integration in the spatial and temporal directions on an output signal from the spatial operation unit and outputs an evaluation value It is an feature.

Further, in the image quality evaluation apparatus for evaluating image quality deterioration accompanying image processing according to the present invention, the spatio-temporal integration section includes the spatio-temporal operation via the variable time-frequency filter and the spatial-frequency filter, respectively. A multiplication circuit that multiplies each of the output signals by a predetermined coefficient, a power operation circuit that performs a predetermined power operation on the multiplication result, and an addition circuit that synthesizes a result of the power operation. It is characterized by the following.

Further, according to the present invention, there is provided an image quality evaluation apparatus for evaluating image quality deterioration due to image processing, wherein the difference calculation section and the spatiotemporal calculation section separately perform a luminance signal component and a plurality of color signal components separately. It is characterized in that calculation is performed.

Further, according to the present invention, there is provided an image quality evaluation apparatus for evaluating image quality deterioration due to image processing, wherein the spatiotemporal arithmetic unit supplies a finite band white noise as an input signal to the arithmetic unit. Is calibrated so as to obtain an output change that matches the visual sensitivity characteristic of.

[0011]

DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below based on embodiments of the present invention with reference to the accompanying drawings. FIG.
Indicates the signal level dependency of the visual sensitivity measured by the present inventors as a characteristic of human vision as a relation of a detection limit with respect to a signal level (a noise level when an input is random noise). The figure shows random noise (white noise), fixed pattern noise (signal), and color signals (Gch, Rch, Bch) of the luminance channel (Ych). Above each measurement curve is a detectable range, and below is a non-detectable range. As will be described later, these curves are used for gain calibration of a spatiotemporal filter (described later) constituting the apparatus of the present invention.

FIG. 2 shows the dependence of visual brightness on the frequency characteristics in the time direction (time frequency characteristics). In FIG. 2, the brightness is a parameter for the relationship between time frequency and visual sensitivity. It is expressed as FIG.
Shows the dependence of visual brightness on the frequency characteristics (spatial frequency characteristics) in the spatial direction. In the drawing, the brightness is expressed as a parameter regarding the relationship between the spatial frequency and the visual sensitivity. In these figures, the relationship between brightness and signal level is determined by the setting of viewing conditions. In FIGS. 2 and 3, a unit td (troland) of the fundus illuminance of the human eye is used as the unit of brightness.

An image quality evaluation apparatus for evaluating image quality deterioration accompanying image processing according to the present invention realizes the time-frequency characteristics and the spatial frequency characteristics shown in FIGS. Is performed so that an evaluation result close to the subjective evaluation value is obtained.

FIG. 4 is a block diagram showing the basic configuration of the image quality evaluation apparatus according to the present invention. In the figure, 12,
The four blocks denoted by 13, 14, and 15 and having characteristic curves drawn in the frames are spatiotemporal filters each having the drawn characteristics. In FIG. 4, a digital image before (before processing) image processing (for example, image compression / expansion processing) and a digital image after (after processing) image processing (for example, image compression / expansion processing) are processed by the differentiator 11 of the difference calculation unit 1. It is assumed that the signals are supplied to the input terminal to be subtracted and the input terminal to be subtracted. As a result, the output terminal of the differentiator outputs, for example, an amount of deterioration due to compression / expansion (data lost during compression does not return to its original state even by expansion), and this is output to the next stage of the spatiotemporal operation unit 2.
Supplied to

The spatio-temporal operation unit 2 includes four spatio-temporal frequency filters 12, 13, 14 and 15 in principle, of which the time-frequency filter 13 and the spatial-frequency filter 14 are controllable in terms of characteristics. . The output from the difference calculation unit 1 is first split into two in a spatiotemporal calculation unit 2, and a fixed spatial frequency filter 12 whose characteristics are indicated by a curve a and a variable spatial frequency filter 14 (that is, whose characteristics can be controlled) Supplied to And the filter 12
Is applied to a variable (ie, characteristic controllable) time-frequency filter 13 cascaded to it.
4 are respectively supplied to a fixed time-frequency filter 15 whose characteristics are cascaded and whose characteristic is shown by a curve d.
A filtered output is obtained from each of the filters 13 and 15.

Here, the characteristics of the variable time frequency filter 13 and the variable spatial frequency filter 14 are as shown in FIGS. 2 and 3, respectively. As can be seen from FIGS. 2 and 3, the parameter is the brightness of the image (corresponding to the level of the image signal).
-1, b-2, b-3, and b-4, and in the case of the filter 14, from the curves c-1, c-2, c-3, and c-4, the brightness, that is, the image signal Switch to a specific curve according to the level of. Therefore, in the example shown in FIG.
The curve selection of the variable filters 13 and 14 is controlled by an image signal before or after processing supplied thereto (in this example, the processed image signal is used). It should be noted that about three types of curves for the variable filters 13 and 14 are enough to constitute the image quality evaluation apparatus according to the present invention.

In the above example, the variable filters 13 and 14 are described as selecting one of the four types of curves. However, this is done by weighting a plurality of curves and adding them all. When used, not only the same effect (selection effect) can be obtained, but also there is an advantage that the brightness parameter can be set continuously. A configuration example in this case will be described later.

In the next-stage spatiotemporal integration unit 3, first,
Outputs of the variable filter 13 and the fixed filter 15 from the spatio-temporal operation unit 2 are supplied to multipliers 17 and 18, respectively, where the control signal from the level-dependent control circuit 16 (in the spatio-temporal operation unit 2) is also used. Each filter (1
The coefficient to be multiplied with the output signal of (3, 15) is controlled. For example, by changing the multiplication coefficients of the multipliers 17 and 18 depending on the signal level of the processed image, the filter 1
The outputs of 3 and 15 can be matched to the visual sensitivity characteristics. By changing or changing the variable precision with an accuracy of about 4 bits (16 levels), sufficient precision close to the visual characteristics can be obtained.

The exponentiators 19 and 20 of the spatiotemporal integration unit 3 can obtain a signal output proportional to a visual stimulus value by using, for example, a squaring circuit. Even when the variable filter 13 and the fixed filter 15 are different, the addition can be performed by the adder 21 without canceling each other. Furthermore, if the multipliers (19, 20) are placed after the adder 21, integration other than the square can be performed. However, when the multiplier is combined with the multiplier to make the square or more, the integration result is relatively local. Evaluation value gets worse due to typical distortion. In addition, when the value is set to the square or less, the characteristic is such that the evaluation value is deteriorated with respect to the distortion having a relatively large area.

In addition to the above, as a setting of the multiplier, there is a method in which the multipliers 19 and 20 are fixed to the square, and the multiplier after the adder 21 is made variable to cope with integration other than the square. The threshold processing circuit 22 at the subsequent stage of the adder 21 sets a signal of a level which is not visually perceived to be zero, and the next integration circuit 23 prevents the signal from being integrated. By performing the spatio-temporal integration and the threshold processing as described above, the degree of coincidence between the evaluation output value of the evaluation device according to the present invention and the subjective evaluation value can be further improved.

Here, a specific configuration example of the variable spatial frequency filter 14 included in the spatiotemporal operation unit 2 in FIG. 4 is shown in FIG. 5 and will be described. This example is not a type of selecting one from a plurality of curves, but a variable filter of a type that can change to an intermediate characteristic between the curves.

In FIG. 5, the variable spatial frequency filter 14 includes three fixed spatial frequency filters 24 and 25.
And 26. Each of the fixed spatial frequency filters 24, 25 and 26 has four tap coefficients having different tap coefficients between the filters 24, 25 and 26 so that each has a bandpass characteristic corresponding to a specific brightness. Spatial axis digital filter

[Outside 1] And differentiators 27, 28 and 29. Output of each fixed filter 24, 25 and 26 (taken out of differentiators 27, 28 and 29 respectively)
Are multiplied by predetermined coefficients (weights) by gain adjusters 30, 31, and 32, respectively, and then added by adder 33, that is, weighted addition. The coefficient is supplied as a control signal from the level-dependent control circuit as described above. As a result, a variable spatial frequency filter that can be changed to an intermediate characteristic between the characteristics of the three fixed filters is obtained.

The variable time-frequency filter 13 included in the spatio-temporal operation unit 2 in FIG. 4 can also be configured in the same manner as the variable spatial-frequency filter 14 described above, and a description thereof will be omitted. The difference is that the four spatial axis digital filters replace the four time axis digital filters.

The image quality evaluation apparatus according to the present invention described above has been described with reference to FIG. 4 showing the basic configuration without particularly discriminating the luminance signal and the chrominance signal. However, as shown in FIG. Since the characteristics of the signal level dependency with respect to are different, the image signal to be evaluated is divided into a luminance signal and a chrominance signal, and the signals are input. Can be further improved.

Finally, a method of calibrating the gain of the time filter will be described with reference to an actual example. In FIG. 4, a finite band,
For example, white noise in the band of a television signal is converted into variable spatiotemporal filters 13 and 14 of the image quality evaluation device according to the present invention.
When input to the spatio-temporal operation unit 2 (in the spatio-temporal operation unit 2), the output signals from the spatio-temporal operation unit are summed by the adder 21 (in the spatio-temporal integration unit 3). Therefore, in FIG.
When white noise of the same amount as the random noise curve is input, the spatio-temporal filters 13 and 1 are controlled so that the output of the adder 21 is constant regardless of the signal level.
A gain of 4 can be calibrated.

Also, when the signal components of only Gch, Rch, and Bch are input, when the same amount of white noise as the curve of FIG. 1 is input, the output of the adder similarly becomes a constant signal level. Thus, the variable gain characteristic can be calibrated. When the image quality evaluation device according to the present invention is configured to include a plurality of variable spatiotemporal filters, a plurality of parameters are contradictory by using different calibration methods for each spatiotemporal filter. Can be calibrated without

[0027]

According to the present invention, an evaluation value that accurately matches the subjective evaluation value can be obtained without performing a human subjective evaluation. Further, since real-time processing can be performed at a relatively easy hardware processing speed, it is also possible to constantly monitor image quality deterioration due to compression / decompression processing. Furthermore, if a standard original image is determined and, for example, the image is built in the evaluation device according to the present invention, the same original image can be converted into a VTR at the transmission point, for example, with respect to image quality deterioration due to distortion in transmission from a distance. It can be easily evaluated by reproduction or the like.

[Brief description of the drawings]

FIG. 1 shows the signal level dependence of visual sensitivity.

FIG. 2 shows the brightness dependence of visual time-frequency characteristics.

FIG. 3 shows brightness dependency of a spatial frequency characteristic of vision.

FIG. 4 is a block diagram showing a basic configuration of an image quality evaluation device according to the present invention.

FIG. 5 shows a specific configuration example of a variable spatial frequency filter in FIG.

[Explanation of Signs] 1 Difference calculation unit 2 Spatiotemporal calculation unit 3 Spatiotemporal integration unit 11 Differentiator 12 Fixed spatial frequency filter 13 Variable time frequency filter 14 Variable spatial frequency filter 15 Fixed time frequency filter 16 Level dependent control Circuits 17, 18 Multipliers 19, 20 Power units 21 Adders 22 Threshold processing circuits 23 Integrator circuits 24, 25, 26 Fixed spatial frequency filters 27, 28, 29 Differential units 30, 31, 32 Gain adjusters 33 Adders

Claims (4)

[Claims] 1. A difference calculation unit for calculating a difference between two image signals before and after image processing, and a response at a time frequency and a spatial frequency is calculated for an output signal from the difference calculation unit. And a variable time-frequency filter and a spatial-frequency filter to which the output of the difference calculation unit is respectively supplied, and the response of the variable time-frequency filter and the spatial-frequency filter is calculated by the image processing. A spatio-temporal operation unit configured to be controlled by an image signal level before or after performing the operation, and integrates an output signal from the spatio-temporal operation unit in a spatial direction and a time direction. An image quality evaluation device for evaluating image quality deterioration accompanying image processing, characterized in that the image quality evaluation device comprises at least a spatiotemporal integration unit for outputting an evaluation value. 2. The image quality evaluation device according to claim 1, wherein
A spatio-temporal integration unit that multiplies each of the spatio-temporal operation unit output signals through the variable time-frequency filter and the spatial frequency filter by a predetermined coefficient; An image quality evaluation apparatus for evaluating image quality deterioration accompanying image processing, characterized by including at least a power operation circuit for performing a power operation of (i) and an addition circuit for synthesizing the result of the power operation. 3. The image quality evaluation device according to claim 1, wherein the difference calculation unit and the spatiotemporal calculation unit perform calculations on a luminance signal component and a plurality of color signal components, respectively. An image quality evaluation apparatus for evaluating image quality deterioration accompanying image processing characterized by the following. 4. The image quality evaluation device according to claim 1, wherein the spatio-temporal operation unit supplies a finite band white noise as an input signal to the operation unit.
An image quality evaluation apparatus for evaluating image quality deterioration due to image processing, characterized in that calibration is performed so as to obtain an output change matching the human visual sensitivity characteristics.