JP7154522B2 - Image quality evaluation equipment suitable for ultra-high-definition images - Google Patents

Description

The present invention relates to an image quality evaluation device suitable for ultra-high definition video.

In conventional high-definition broadcasting or full high-definition broadcasting, the video signal encoded by the video transmission source such as a key station is decoded after being transmitted via terrestrial waves, satellite broadcasting, cable TV networks, the Internet, etc. , is viewed as high-definition video or full high-definition video at video transmission destinations such as homes. However, depending on the transmission environment, block noise or the like may occur in the video after decoding. Therefore, there is a need for image quality evaluation that evaluates video after decoding based on video before encoding.

As one method for evaluating the image quality of video, there is a method called subjective evaluation. In this method, dozens of subjects are gathered, a video is presented to the subjects, the subjects are subjectively rated, and the numerical value obtained by statistically processing the scores is defined as the quality of the video. A representative method for subjective image evaluation is ITU-R Recommendation BT. 500-11, ITU-T Recommendation P. 910, etc.

However, subjective image evaluation must meet the strict viewing conditions stipulated by the Recommendation, and there are difficulties such as the need to recruit a large number of subjects. Hard to say.

On the other hand, by analyzing the video signal, one or more numerical indices representing the characteristics of the video called video feature quantity are extracted, and the image quality evaluation of the video is mechanically calculated from the video feature quantity. A technique called image quality evaluation is also known (see Patent Document 1). Such an objective image quality evaluation can usually be performed by a machine or the like. For example, by operating a plurality of machines 24 hours a day, it is possible to evaluate the image quality of a huge amount of content in a short period of time. The "subjective evaluation value" described in Patent Document 1 is an objective image quality evaluation value obtained mechanically.

JP-A-8-205156

The present invention provides an image quality evaluation apparatus suitable for ultra-high-definition video, which divides a screen displayed based on a digital video signal for one frame into a plurality of areas and obtains an image quality evaluation value for each area. , an extracting unit for dividing each region into a plurality of small blocks, obtaining a signal variance value for each of the small blocks, and averaging each region to extract an activity value; and a determination unit that determines that noise has occurred in the area when the objective evaluation value exceeds a threshold. A noise sensitivity contour line is formed on the screen by grouping the regions based on the activity value, and the image quality evaluation value is weighted for each group according to the noise sensitivity contour line .

In such cases, the objective image quality evaluation methods used in conventional high-definition broadcasting, full high-definition broadcasting, etc. often average the image of the entire screen in line with the conventional viewing mode. It turned out that it is not suitable for evaluating the image quality of high-definition images.

Therefore, the present invention has been made in view of the above problems, and it is an object of the present invention to provide an image quality evaluation apparatus capable of appropriately evaluating the image quality of ultra-high definition video such as 4K or 8K.

The present invention provides an image quality evaluation device suitable for ultra-high-definition video,
an evaluation unit that divides a screen displayed based on a digital video signal for one frame into a plurality of regions and obtains an image quality evaluation value for each region;
an extraction unit that further divides each region into a plurality of small blocks, obtains a signal variance value for each of the small blocks, averages each region, and extracts an activity value;
a calculation unit for calculating an objective evaluation value by weighting the image quality evaluation value according to the activity value for each region;
and a determination unit that determines that noise has occurred in the area when the objective evaluation value exceeds a threshold.

INDUSTRIAL APPLICABILITY The present invention can provide an image quality evaluation apparatus capable of appropriately evaluating image quality of ultra-high definition video such as 4K or 8K.

FIG. 10 is a diagram for explaining an example of segmentation and setting of small blocks in an 8K frame for activity detection. Here, the small block size is set to 8 pixels × 8 lines, and the region size is 960 pixels × 536 lines. have set. 1 is a block diagram showing the configuration of an image quality evaluation apparatus according to this embodiment; FIG. (a) is a diagram showing a screen displayed by a 4K or 8K digital video signal. When the inside of the screen is uniform like this, the gaze point is often concentrated on the center of the screen. (b) is a diagram showing noise sensitivity contour lines corresponding to the screen of (a). (a) is a diagram showing a screen displayed by a 4K or 8K digital video signal, showing a screen in which an object attracting attention points is framed in from the upper left. (b) is a diagram showing noise sensitivity contour lines corresponding to the screen of (a), and the shape of the noise sensitivity contour line is also changed because the area where the points of interest gather is different from that in FIG. (a) is a diagram showing a screen displayed by a 4K or 8K digital video signal, showing a state in which an object to attract attention points is positioned at the center of the screen. (b) is a diagram showing the noise sensitivity contour lines corresponding to the screen of (a). Since the center of the screen is the area where points of interest gather, the shape of the noise sensitivity contour lines also returns to the shape of FIG. (a) is a diagram showing a screen displayed by a 4K or 8K digital video signal, and shows a screen in which an object that attracts the gaze point is framed out from the lower right. (b) is a diagram showing the noise sensitivity contour lines corresponding to the screen of (a). Since the area where gaze points gather is symmetrical with that of FIG. 4, the shape of the noise sensitivity contour line is also symmetrical with that of FIG. .

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described with reference to the drawings. In this specification, "noise" means FR noise (double stimulation method: Full Reference method) obtained by comparing the reference video signal and the evaluation target video signal, and the evaluation target video signal alone. NR noise that can be evaluated (single stimulation method: No Reference method), and RR noise (feature value comparison method: Reduced Reference method) obtained by comparing the feature amounts of the reference video signal and the evaluation target video signal corresponding to the middle thereof ).

Also, in this specification, the term "super-high-definition video" refers to video of 4K or 8K or higher. For SDTV (Standard definition television) and HDTV (High-definition television), ITU-T J. An objective evaluation device has been developed in accordance with the 144 method, etc., but when viewing ultra-high-definition video such as 4K and 8K, the movements of the viewer's gaze point are greatly different. With only the 144 method, the correlation coefficient with subjective evaluation is often low, and sufficient evaluation accuracy cannot be obtained. Therefore, there is a demand for an image quality evaluation apparatus suitable for super-high-definition video.

Here, when a viewer views ultra-high-definition video of 4K or 8K or higher, it is assumed that the viewer approaches the screen and observes the displayed image while staring at it. Therefore, in the subjective image quality evaluation of ultra-high-definition video, the occurrence of noise is strictly detected (high noise sensitivity) in the display area where the viewer gazes, that is, where the gaze point is concentrated. For images other than the above, the occurrence of noise is often overlooked to some extent (noise sensitivity is low). Therefore, if the objective image quality evaluation performed mechanically follows the subjective image quality evaluation, the evaluation result of the subjective image quality evaluation can be approximated. However, it is difficult to objectively grasp what image area the viewer is gazing at. Therefore, in the present embodiment, the concept of "activity" is used to guess what the viewer is gazing at. Activities are described below.

FIG. 1 is a diagram for explaining activities. Although the screen is divided into 64 areas in the following example, the number of divisions is not limited to 64. Here, when a digital video signal is input, the activity extraction unit (not shown) extracts lines V1 to V2 and images H1 to H1 in one frame with reference to FIG. The range of H2 is divided into 64 regions, and calculation is performed for each region. Specifically, a video level and an activity, that is, a video activity are calculated for each region. Here, the Video Level is the average value of the pixel values included in the image frame, and is also referred to as the luminance signal level. Alternatively, the level of color difference signals may be used. Furthermore, as the video activity, when the variance is obtained for each small block included in the image, the average value of the pixels in the frame of this variance may be used, or simply the pixel value in the frame of the image frame may be used.

More specifically, if the frame ends to H1 and H2 are 0 pixels, and the frame ends to V1 and V2 are each 16 lines, the frame to be inspected can be made up of 7680 pixels in the horizontal direction and 4288 lines in the vertical direction. One region obtained by dividing this into 64 has 960 pixels and 536 lines. Here, as shown in FIG. 1B, a small block of m lines and n pixels is formed in one region. That is, the luminance value of each pixel in the small block can be represented by Y(m,n). Here, it is preferable to divide the luminance signal Y into small blocks of 8 pixels×8 lines. When using the luminance signal Y, the number of small blocks in one region is N=120×67=8040. When the color difference signals Pb and Pr are used, it is preferable to divide them into small blocks of 4 pixels×8 lines.

Furthermore, for each small block, the mean of the signal is obtained as the DC component and the variance as the AC component. That is, finding the variance as video activity means extracting high frequency components. Formula (1) is a formula for obtaining the average A(k) for the luminance signal Y within the small block #k, and formula (2) is for the variance V(k) for the luminance signal Y within the small block #k. is a formula for obtaining As a result, in the 64 regions, the average A(k) and the variance V(k) are obtained according to the number of blocks (k=1 to 8040).

Furthermore, the average A(k) and the variance V(k) obtained according to formulas (1) and (2) are averaged for each region. Formula (3) is a formula for obtaining the video average FkA of each area, and formula (4) is a formula for obtaining the activity average FkV (activity value) of each area.

In general, the activity average FkV indicates a high value for a shooting object (for example, an in-focus object) in the area (areas A and B), and as this is displaced, the high activity area moves, and the area A is the maximum value of noise sensitivity, the shape of the noise sensitivity contour changes. On the other hand, an area (areas C and D) such as a background where there is no object to be photographed has a low activity average FkV value and a low noise sensitivity. In this embodiment, the regions are grouped based on the activity average FkV obtained for each region, and the image quality evaluation value calculated for each region is weighted.

FIG. 2 is a block diagram showing the configuration of the image quality evaluation apparatus according to this embodiment. In FIG. 2, a 4K or 8K digital video signal input from the input unit 1 of the video distribution source is encoded by the encoder 11, transmitted to the video distribution destination via the video network VNW, and then decoded by the decoder 17. After being decoded, it is output from the output unit 2, and the super-high-definition video VD can be viewed via a dedicated television.

In parallel with this, a part of the digital video signal encoded in the encoder 11 is decoded by another decoder 12 without going through the video network VNW and input to the FR noise detector (evaluation section) 13 . A part of the digital video signal input from the input unit 1 is also input to the FR noise detector 13 as a reference signal and compared with the digital video signal decoded by the decoder 12 by the double stimulation method. Based on the difference between the two signals, an FR image quality evaluation value representing the degree of FR noise is generated. ) 14 for weighting. At this time, when one frame is divided into a plurality of regions, the FR image quality evaluation value is obtained for each region (the same applies to the NR image quality evaluation value and the RR image quality evaluation value). A weighted FR image quality evaluation value is output from the FR noise weighting device 14 . Weighting will be described later. An alarm is issued from the alarm device 15 as necessary.

Further, a part of the signal decoded by the decoder 17 of the video distribution destination is input to the NR noise detector (evaluator) 18 and evaluated by the single stimulation method. As a result, an NR image quality evaluation value representing the degree of NR noise is output from the NR noise detector 18 and input to the NR noise weighting device (extraction unit, calculation unit, determination unit) 19 for weighting. A weighted NR image quality evaluation value is then output from the NR noise weighting device 19 . Weighting will be described later. An alarm is issued from the alarm device 20 as necessary.

Further, a part of the digital video signal input from the input unit 1 of the video distribution source is input as a reference signal to the first RR feature amount detector 16, and the feature amount A is extracted. The extracted feature amount A is output from the first RR feature amount detector 16 and sent to the RR noise detector 22 via an IP network INW such as the Internet. Separately, part of the signal decoded by the decoder 17 of the video distribution destination is input to the second RR feature amount detector 21 to extract the feature amount B. FIG. The extracted feature quantity B is output from the second RR feature quantity detector 21 and is similarly sent to the RR noise detector (evaluation unit) 22 via the IP network INW. The RR noise detector 22 compares the received feature amounts A and B, outputs an RR image quality evaluation value representing the degree of RR noise, and outputs an RR noise weighting device (extractor, calculator section, determination section) 23 and weighting is performed. A weighted RR image quality evaluation value is then output from the RR noise weighting device 23 . Weighting will be described later. An alarm is issued from the alarm device 24 as necessary. The FR image quality evaluation value, NR image quality evaluation value, and RR image quality evaluation value are also disclosed in Japanese Patent Application Laid-Open No. 8-205156, Japanese Patent Application No. 2015-543639, and Japanese Patent Application No. 2007-553821. are used as image quality evaluation values. Also, other image quality evaluation values may be used.

The FR noise weighting device 14, the NR noise weighting device 19, and the RR noise weighting device 23 are assumed to include the activity extractor described with reference to FIG. A value obtained by weighting the image quality evaluation value is called an objective evaluation value, and the objective evaluation value calculated in the present embodiment is highly accurate and can substitute for human subjective evaluation. Weighting will be described below using the FR noise weighting device 14 as an example, but other noise weighting devices function similarly.

FIGS. 3A to 6A are diagrams showing screens (one frame) displayed by 4K or 8K digital video signals, and FIGS. 3B to 6B are diagrams, respectively. 3(a) to 6(a) showing noise sensitivity contours corresponding to the screens; FIG. In FIGS. 3 to 6, the screen is divided into 8×8=64 areas and shown by grids.

In this embodiment, 64 areas of one frame are divided into four groups A to D, and a weighting coefficient to be multiplied by the image quality evaluation value is assigned to each group. Specifically, the weighting factor is set to 1.375 in Group A, in which the area is hatched in the figure, and the weighting factor is 1.125 in Group B, in which the area is outlined in white. The weighting factor is set to 0.875 for group C indicated by , and the weighting factor is set to 0.625 for group D indicated by a dark tone in the figure. When the weighting factor is 1, the image quality evaluation value remains unchanged.

However, the weighting of group A is arbitrarily changed between 1.25 and 1.5 (preferably 1.3 to 1.45), and the weighting of group B is 1.05 to 1.25 (preferably 1.1 to 1.2), the weighting of group C is arbitrarily changed between 0.75 and 0.95 (preferably 0.8 to 0.9), and group D may be arbitrarily changed between 0.5 and 0.75 (preferably 0.5 and 0.75). Also, the number of divided groups is not limited to four, and may be two, three, or five or more.

In FIG. 3(a), the screen shows a uniformly calm sea surface SE. Therefore, since the object to be photographed is substantially uniform over the entire screen, the activity average FkV in each area is substantially equal. In such a case, the viewer viewing the screen often vaguely directs his or her line of sight to the center of the screen.

Therefore, as shown in FIG. 3B, the FR noise weighting device 14 sets the central 2×2=4 region to group A, and sets the surrounding regions to groups B, C, and D in order. Let this be the default setting. In this default setting, the image quality evaluation value is multiplied by a weighting factor of 1.375 for Group A of the four central regions, followed by 1.125 for Group B, 0.875 for Group C, and 0.875 for Group D. Each is multiplied by 0.625, the amount of noise in the entire screen is calculated, an FR objective evaluation value is obtained, and an alarm is issued from the alarm device 15 after comparison with a threshold value. As a result, the noise sensitivities of the four central regions are increased, and the noise sensitivities of the other regions decrease toward the periphery. In consideration of the human visual field characteristics, only the central four areas that are the effective visual field when viewing 4K and 8K video on a large screen are grouped into group A, and the other areas are grouped into one of groups B to D. good too.

Next, as shown in FIG. 4A, it is assumed that the boat BT has entered the frame from the upper left of the screen. A viewer who sees such a screen often first turns his or her line of sight to the boat BT. In such a video, the activity average FkV sharply increases around the area including the frame-in boat BT. In addition, since the wave that has been pushed around the boat BT spreads around the boat BT, the activity average FkV also increases in the area around the boat BT accordingly.

Therefore, the FR noise weighting device 14, as shown in FIG. Group B consists of 5 areas (including the wave that has been pushed by the image), Group C consists of 5 areas around the area of Group B, and Group D consists of the rest.

Looking at this grouping from another point of view, referring to FIG. It can be said that one noise sensitivity contour line is formed between the regions of , and one noise sensitivity contour line is formed between the group C region and the group D region. According to such a noise sensitivity contour line, the distribution of activity averages FkV can be understood at a glance, that is, the distribution of weighting coefficients can be understood. The image shown in FIG. 4A and the noise sensitivity contour lines shown in FIG. 4B may be displayed side by side on a monitor (not shown), or both may be displayed in an overlapping manner.

In the FR noise weighting device 14, the objective evaluation value calculated by multiplying the image quality evaluation value of the area of group A by a weighting factor of 1.375 is compared with the threshold, and the image quality evaluation value of the area of group B is weighted by a weighting factor of 1.375. The objective evaluation value calculated by multiplying by 125 is compared with the threshold, the objective evaluation value calculated by multiplying the image quality evaluation value of the region of group C by a weighting factor of 0.875 is compared with the threshold, and the image quality evaluation value of the group C is compared with the threshold. The objective evaluation value calculated by multiplying the image quality evaluation value of the area by a weighting factor of 0.625 is compared with a threshold value. A warning will be issued from That is, the group A area has the highest noise sensitivity, and the group D area has the lowest noise sensitivity.

When grouped in this way, the area away from the boat BT is grouped into group D and has a smaller weighting factor than the area including the boat BT. Even if there is noise in the region where the Although this is an objective evaluation performed mechanically, it can be brought closer to a subjective evaluation, and the evaluation efficiency can be greatly improved. According to the results of studies conducted by the present inventors, by weighting the image quality evaluation value in this way, the correlation coefficient with the subjective evaluation is 0.9214 (a perfect match if the correlation value is 1), which is high. It turns out that a correlation is required.

However, extreme weighting may greatly increase the noise sensitivity in the group A region and greatly decrease the noise sensitivity in the group D region, resulting in a large difference from the subjective evaluation value. Therefore, the FR noise weighting device 14 can also perform the following detection in parallel with noise detection.
(1) Average or individual weighted image quality evaluation value AV (objective evaluation value) in the group A region and average or individual weighted image quality evaluation value BV (objective evaluation value) in the group B region A difference from the value is taken, and if the difference exceeds a predetermined value δ, an alarm is issued.
(2) Average or individual weighted image quality evaluation value BV (objective evaluation value) in the group B area and average or individual weighted image quality evaluation value CV (objective evaluation value) in the group C area A difference from the value is taken, and if the difference exceeds a predetermined value δ, an alarm is issued.
(3) Average or individual weighted image quality evaluation value CV (objective evaluation value) in the group C area and average or individual weighted image quality evaluation value DV (objective evaluation value) in the group D area A difference from the value is taken, and if the difference exceeds a predetermined value δ, an alarm is issued.

Furthermore, as shown in FIG. 5A, when the boat BT moves toward the center of the screen, the viewer's line of sight often moves toward the center together with the boat BT. Therefore, the FR noise weighting device 14, as shown in FIG. (including waves pushed by the image) are group B, 20 regions around the region of group B are group C, and the rest are group D.

The FR noise weighting device 14 multiplies the image quality evaluation value obtained for each region by a weighting coefficient corresponding to the set group to calculate the objective evaluation value in the screen, or calculates the image quality evaluation value after weighting as described above. (Objective evaluation values) AV, BV, CV, and DV are calculated, and when these exceed threshold values, it is determined that noise has occurred, and the alarm device 15 issues an alarm in association with that area.

In addition, it is desirable to perform normalization so that the average value of the objective evaluation values does not change even if the weighting coefficient is changed. As a specific example of normalization, the sum Σ(AV+BV+CV+DV) of the weighted image quality evaluation values (objective evaluation values) is divided by (1.375×4+1.125×12+0.875×20+0.625×28). , weighted image quality evaluation values (objective evaluation values) AV, BV, CV, and DV. Other statistical methods may be used for normalization.

Furthermore, as shown in FIG. 6A, when the boat BT moves toward the lower left of the screen, the viewer's line of sight often moves toward the lower left along with the boat BT. Therefore, the FR noise weighting device 14, as shown in FIG. are group B, five areas around the area of group B are group C, and the rest are group D.

The FR noise weighting device 14 multiplies the image quality evaluation value obtained for each region by a weighting coefficient corresponding to the set group, and obtains the above-described weighted image quality evaluation values (objective evaluation values) AV, BV, and CV. , DV are calculated, and when these exceed the threshold values, the alarm device 15 issues an alarm in association with the area.

After that, when the boat BT frames out of the screen, as shown in FIG. 3(a), the calm sea surface SE is displayed as an image again. Assume that only the central 2×2=4 area is group A, and the surrounding areas are sequentially grouped B, C, and D, and the default setting is restored. Thereafter, when a change occurs in the screen, the activity average FkV increases, so the same process can be performed.

According to the present embodiment, PSNR (Peak Signal-to-Noise Ratio) and DSCQS (Double Stimulus Continuous Quality Scale) in ultra-high-definition video are objectively evaluated. It can be done numerically. In addition, by applying the real-time type rather than the storage type, it is possible to evaluate the ultra-high definition video signal with a huge amount of data inexpensively and easily without accumulating it in a huge SSD server. As described above, objective evaluation highly correlated with subjective evaluation results can be realized based on evaluation values weighted specifically for ultra-high-definition video.

Although the present invention has been described above with reference to the embodiments, the present invention is not necessarily limited to the above embodiments, and can be variously modified and implemented within the scope of its technical ideas. For example, in the above embodiments, all of the FR noise weighting device, the NR noise weighting device, and the RR noise weighting device weight the image quality evaluation value. Also, the grouping in the default setting is divided into four groups A to D. For example, groups A and B are group C, groups C and D are group D, and two groups are set as the default setting. Also good.

1 Input Part 2 Output Part 11 Encoder 12 Decoder 13 FR Noise Detector 14 FR Noise Weighting Device 15 Alarm Device 16 First RR Feature Amount Detector 17 Decoder 18 NR Noise Detector 19 NR Noise Weighting Device 20 Alarm Device 21 Second RR Feature Amount detector 22 RR noise detector 23 RR noise weighting device 24 alarm device INW network VD video VNW video network

Claims (4)

In the image quality evaluation equipment suitable for ultra-high-definition images,
an evaluation unit that divides a screen displayed based on a digital video signal for one frame into a plurality of regions and obtains an image quality evaluation value for each region;
an extraction unit that further divides each region into a plurality of small blocks, obtains a signal variance value for each of the small blocks, averages each region, and extracts an activity value;
a calculation unit for calculating an objective evaluation value by weighting the image quality evaluation value according to the activity value for each region;
a determination unit that determines that noise has occurred in the region when the objective evaluation value exceeds a threshold ;
A noise sensitivity contour line is formed on the screen by grouping the regions based on the activity value, and the image quality evaluation value is weighted for each group according to the noise sensitivity contour line. Evaluation device. 2. The image quality evaluation apparatus according to claim 1 , wherein the objective evaluation value is obtained for each group, and an alarm is output when the difference exceeds a predetermined value. 3. The image quality evaluation apparatus according to claim 1 , wherein the objective evaluation value is normalized based on the weighting coefficient for each group and the sum of the objective evaluation values. 4. The image quality evaluation value of the area corresponding to the center of the screen is weighted more heavily than the image quality evaluation values of other areas when the variance values of the signals obtained for each of the areas are substantially equal. The image quality evaluation apparatus according to any one of claims 1 to 3 .

Images