JP6076110B2 - Discomfort degree estimation apparatus and discomfort degree estimation program - Google Patents

Description

  The present invention relates to a technique for estimating the degree of discomfort for an image that frequently changes in luminance.

  Discomfort induced by images in which frequent changes in brightness, such as blinking, occur over a large area as the display of a broadcast, recorded video, personal computer video, etc. increases in size at home. In some cases, viewers may experience photosensitivity attacks and other health problems.

  In order to prevent such health damage, video broadcasted by public broadcasters and broadcasters affiliated with the National Broadcasting Corporation has been recommended by ITU-R BT.1702 `` Guidance for the reduction of recombinant epileptic seizures caused by In accordance with the “Guidelines for Video Techniques such as Animation” compliant with “television”, luminance reversal with a luminance difference exceeding 10% of the maximum luminance occurs 7 times or more per second, except for live broadcasts and news programs with high speed. The area is limited so that it does not exceed 25% of the entire screen.

  Many devices that determine whether images meet this criterion have been published and sold both domestically and abroad, but any of these devices are dangerous and likely to cause photosensitive seizures if the above conditions are exceeded. For example, “video with luminance inversion of 9.99% on the entire screen occurring 30 times per second” is determined as “safe” but “dangerous”. It is far greater than the viewer's discomfort ("video with a luminance inversion of 10.01% in the portion of 25.001% of the entire screen occurring seven times per second"). Discomfort level is high.

  In order to alleviate such an event, “Harding FPA” manufactured by CRS in the UK is most used as a flashing video analysis device in broadcasting stations all over the world, and is substantially an international standard device. “Harding FPA” manufactured by CRS in the UK displays the “Danger Level” of the video in units of frames and implements a function that issues “Caution” even if it is not judged as “Dangerous”. As for the “risk level”, the latter is determined to be “danger”, and the former is determined to be “caution” that does not reach “danger”.

  Accordingly, the applicant of the present invention has invented a discomfort level estimation device that estimates the discomfort level by using the “discomfort level” obtained by quantifying the viewer's discomfort as a criterion for determining the quality of the video (Patent Document 1). reference).

JP 2011-238121 A

  However, in the embodiment of Patent Document 1, the determination is made on the basis of multiple levels (16 levels in the example in the document), and the number of determination processing processes for estimation of discomfort is 10 of “Harding FPA” described above. Therefore, when constructing the discomfort level estimation device described in Patent Document 1 with dedicated hardware, an expensive LSI with a number of logic gates of several million gates is required, and the price of the device itself increases. . In addition, even when the dissatisfaction level estimation apparatus described in Patent Document 1 is configured by mounting the process in software on a computer, a plurality of high-spec CPUs are required to realize real-time processing due to the large number of processing processes. A high-performance machine will be required, and the price will increase.

Furthermore, in the example of Patent Document 1, since the insensitivity is estimated based only on the difference between the maximum brightness and the minimum brightness, the same applies when the brightness changes to a sine wave or a rectangular wave. Although the degree of discomfort is determined, in the actual evaluation result, the degree of discomfort when changing to a rectangular wave shape is higher, and there is an error in the discomfort degree estimation result. Further, the ratio of the luminance at the time point to the maximum luminance that can be displayed on the screen is 0.3 → 0.1 → 0.2 → 0.1 → 0.3 → 0.2 → 0.3 per second.
In the example of Patent Document 1, when the determination luminance difference is 0.09, it is assumed that there are six luminance inversions, and when the determination luminance difference is 0.20, there are two luminance inversions. Although the discomfort level of “high brightness difference 0.09 and luminance inversion 6 times” having a higher discomfort level is assumed to be an estimation result, the actual evaluation result has a higher discomfort level “luminance difference 0.164 and luminance inversion 6 times” This is equivalent to the discomfort degree in the case where the discomfort is to be performed, and there is an obvious error in the discomfort degree estimation result.

  As described above, there has not yet been a technique that can estimate the discomfort degree for a video with complicated and frequent luminance changes in real time and with high accuracy using inexpensive equipment.

  The present invention was devised in view of the above circumstances, and when estimating the degree of discomfort for complex and frequently changing luminance images, the estimation accuracy is increased and the number of processing processes is reduced, and the processing speed is increased. It is an object of the present invention to provide a discomfort degree estimation device and a discomfort degree estimation program capable of realizing low cost and high accuracy.

  In order to solve the above problems, a discomfort degree estimation apparatus according to the present invention is a discomfort degree estimation apparatus that estimates a discomfort degree with respect to a luminance change in a video, and includes a block luminance value calculation unit, a movement vector detection unit, and a block-specific magnitude. A measurement unit and a discomfort degree calculation unit are provided.

  With this configuration, the discomfort degree estimation device calculates a block luminance value that is a luminance value for each block of an image included in the video by the block luminance value calculation unit, and the calculation by the movement vector detection unit. Based on the block luminance value, a movement vector between one frame of the subject in the video is detected, and the block-by-block magnitude measurement unit detects the block luminance value in the block and the movement vector detected in the image one frame before. Based on the inter-frame luminance difference value, which is the difference between the block luminance value in the block corrected in step 1, and the block-by-block magnitude, which is the magnitude of the luminance change in each of the blocks, is calculated by the discomfort degree calculation unit. All the block-specific magnitudes By adding over the number of frames predetermined measurement interval in, it calculates the discomfort degree. In this discomfort level estimation device, when estimating the discomfort level for a complex and frequently changing brightness image, the estimation accuracy is increased, the number of processing processes is reduced, and high speed, low cost and high accuracy are realized. Can do.

  Further, the discomfort degree estimation device may be configured to measure the magnitude of each block by squaring after correcting the inter-frame luminance difference value by a frequency sensitivity correction filter by the block magnitude measurement unit.

  The discomfort degree estimation device may further include an average luminance calculation unit, and the block-by-block magnitude measurement unit of the discomfort degree estimation device may include an average luminance correction unit. With this configuration, the discomfort level estimation apparatus averages the block luminance value of each block over the number of frames of the predetermined measurement interval by the average luminance calculation unit, so that the entire screen during the predetermined measurement interval is averaged. An average luminance is calculated, and the discomfort level is corrected based on the average luminance by the average luminance correction unit. With this discomfort level estimation device, it is possible to obtain a discomfort level that takes into account the influence of contrast of luminance changes.

  Further, the discomfort degree estimation device may further include a discomfort degree correction unit. With this configuration, the discomfort degree estimation device multiplies the corrected discomfort degree at the previous measurement time point by a constant greater than 0 and less than 1, and adds it to the discomfort degree at the current measurement time point, thereby obtaining the current measurement time point. The post-correction discomfort level is obtained as the discomfort level. With this discomfort level estimation device, it is possible to obtain a discomfort level that takes into account the time accumulation effect.

  Further, the discomfort degree estimation device may further include a logarithmic conversion unit. With this configuration, the discomfort degree estimation device logarithmically converts the discomfort degree by the logarithmic conversion unit and outputs the result as an estimated discomfort degree. In this discomfort degree estimation device, it is possible to obtain an estimated discomfort degree in which the correspondence with the psychological evaluation value is almost linear.

  The present invention can also be embodied as a discomfort level estimation program that causes a computer to function as the above-described discomfort level estimation device.

  According to the present invention, it is possible to increase the estimation accuracy and reduce the number of processing processes, thereby realizing high speed, low cost, and high accuracy.

It is a block diagram which shows the discomfort degree estimation apparatus which concerns on embodiment of this invention. It is a figure which shows the frequency sensitivity characteristic of the discomfort degree by the brightness | luminance difference between frames. It is a figure which shows the discomfort degree by the combination of a brightness | luminance difference and a brightness | luminance inversion number.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings as appropriate. As shown in FIG. 1, a discomfort level estimation apparatus 1 according to an embodiment of the present invention estimates a discomfort level with respect to a luminance change based on a motion vector in a video. As a functional unit, a block luminance value calculation unit 10 A movement vector detection unit 20, a block-by-block magnitude measurement unit 30, an average luminance calculation unit 40, a discomfort degree calculation unit 50, a discomfort degree correction unit 60, and a logarithmic conversion unit 70.

<Block luminance value calculation unit>
The block luminance value calculation unit 10 receives a video signal a from an external device or the like, calculates a block luminance value for each of a plurality of blocks obtained by dividing an image included in a video, and has a pixel luminance. A value determining unit 11 and a block luminance value determining unit 12 are provided.

The pixel luminance value determining unit 11 determines a pixel luminance value that is a luminance value of each pixel of the image and outputs the pixel luminance value to the block luminance value determining unit 12. In the present embodiment, the pixel luminance value determining unit 11 determines a pixel luminance value Y that is a luminance value for each pixel based on the RGB values included in the acquired video signal, and the determination result is used as a block luminance value determining unit. 12 is output. When the RGB method is adopted in the video signal, the pixel luminance value Y is set to the R value (0 to 255), G value (0 to 255), and B value (0 to 255) of one pixel included in the video signal. ) And is expressed as follows (in the case of high-definition video).
Y = 0.212 · R + 0.701 · G + 0.087 · B

  The block luminance value determination unit 12 determines a block luminance value based on the determined pixel luminance value, and outputs the block luminance value to the movement vector detection unit 20, the block-by-block magnitude measurement unit 30, and the average luminance calculation unit 40. In this embodiment, the block luminance value determination unit 12 is a square pixel (approximately 6 to 9 pixels in both horizontal and vertical directions) close to a half cycle of the spatial frequency that is most sensitive to luminance changes in the human visual system under standard viewing conditions. The block luminance value is determined for each block. Here, the standard viewing condition refers to a viewing condition for viewing from a distance three times the height of the screen in the case of a high-definition video, for example. In the present embodiment, the block luminance value determination unit 12 calculates an average value (for example, an arithmetic average value) of the pixel luminance values Y of the pixels included in the block, thereby determining the average value as the block luminance value. . Note that the method of determining the block luminance value by the block luminance value determining unit 12 is not limited to the above-described one, and for example, a configuration in which the pixel luminance value Y of the central pixel of the block is adopted as the block luminance value may be employed.

<Movement vector detection unit>
Based on the calculated block luminance value, the movement vector detection unit 20 detects a movement vector between one frame of the subject in the video, and outputs it to the block-by-block magnitude measurement unit 30.
In the present embodiment, the motion vector detection unit 20 performs block matching on a macroblock of a specific size (for example, the size of the macroblock is about 8 × 8 to 16 × 16) between two consecutive images. Is used to detect, as a movement vector, the block movement amount that minimizes the sum of the inter-frame differences (absolute value sum or square sum) of the luminance values of each pixel from which the direct current component is removed. It is desirable to limit the search range of the movement vector to a range (about 1/10 to 1/5 of the screen height) that can be sufficiently followed by the visual system in both the upper, lower, left, and right directions.

<Magnitude measurement unit by block>
The block-by-block magnitude measuring unit 30 calculates the block based on the inter-frame luminance difference value that is the difference between the block luminance value in the block and the block luminance value in the block corrected with the motion vector detected in the image one frame before. Block-by-block magnitude that is the magnitude of the luminance change in each of the blocks, and for each block, a block-by-block luminance difference determining unit 31 (31-1, 31-2, ..., 31-n); The block-specific frequency sensitivity correction filter 32 (32-1, 32-2,..., 32-n) and the block-specific corrected luminance difference square unit 33 (33-1, 33-2,..., 33-n) Provide a set.

  The block-by-block luminance difference determining unit 31 detects the block luminance value of the corresponding block in the current frame (m-th image) and the movement vector detection unit 20 in the previous frame (m−1-th image). The difference value with the block luminance value of the block indicated by the movement vector, that is, the inter-frame luminance difference is calculated and output to the block-specific frequency sensitivity correction filter 32.

  The block-specific frequency sensitivity correction filter 32 corrects the transition of the inter-frame luminance difference in each block with the frequency sensitivity characteristic of discomfort shown in FIG. 2, and the obtained block-specific corrected luminance difference is a block-specific corrected luminance difference square unit. To 33. Here, the inter-frame luminance difference corresponds to a value obtained by multiplying the luminance amplitude by the frequency, and the frequency sensitivity characteristic in FIG. 2 is expressed by the sensitivity divided by the square root of the frequency. The corrected luminance difference for each block, which is the difference, corresponds to a value obtained by frequency sensitivity correction of the luminance amplitude multiplied by the square root of the frequency. 2 is an infinite impulse response (IIR) type digital filter having about 8 taps, for example, a four-stage low-pass filter and a four-stage high-pass filter. Can be realized by appropriately arranging and connecting them in series.

  The block-specific corrected luminance difference square unit 33 squares the block-specific corrected luminance difference, thereby multiplying the square of the luminance amplitude in each block by the frequency, that is, a value corresponding to the magnitude in the earthquake (hereinafter referred to as a block-specific magnitude). Is output to the discomfort degree calculation unit 50. The magnitude of each block is corrected by a frequency sensitivity characteristic of discomfort (a characteristic obtained by squaring the characteristic shown in FIG. 2 and then multiplying the frequency).

Here, “magnitude” corresponds to “earthquake magnitude” when the retina is replaced with the ground surface and the luminance change is replaced with ground motion. In the present invention, the brightness change corresponding to the earthquake motion is detected. By performing the frequency sensitivity correction, the discomfort level corresponding to the seismic intensity of the earthquake is derived.
Magnitude is expressed as a logarithmically converted value in order to correspond linearly to the sensory value, but since the value before logarithmic conversion is used in the calculation process, in the present invention, the magnitude before logarithmic conversion is expressed as “magnitude”. (Magnitude by block, total magnitude) ”, and the discomfort level is also referred to as“ discomfort level ”before logarithmic conversion.

<Average luminance calculation unit>
Based on the block luminance value of all blocks determined by the block luminance value determining unit 12, the average luminance calculating unit 40 averages the block luminance values of each block over the number of frames in a predetermined measurement interval (for example, 1 second). As a result, the average luminance of the entire screen during a predetermined measurement interval is calculated and output to the discomfort degree calculation unit 50.

<Discomfort calculation unit>
The discomfort level calculation unit 50 calculates the discomfort level based on the block-specific magnitude measured by the block-specific magnitude measurement unit 30, and includes a total magnitude calculation unit 51 and an average luminance correction unit 52.

  The sum magnitude calculation unit 51 calculates the sum magnitude by adding the block magnitudes of each block output from the block magnitude measurement unit 30 over the number of frames of the predetermined measurement interval, thereby correcting the average luminance. To the unit 52.

The average luminance correction unit 52 is used to obtain an uncomfortable degree in consideration of the influence of contrast of luminance change, and is obtained by the total magnitude calculation unit 51 using the average luminance value output from the average luminance calculation unit 40. The degree of discomfort at the time of the current measurement is calculated by correcting the total sum magnitude.
In the present embodiment, the average luminance correction unit 52 multiplies the sum magnitude by a value obtained by multiplying a reciprocal of the ratio of the average luminance with respect to the luminance of 100% white by the γ power (0 ≦ γ ≦ 1). calculate. That is, as the average luminance is lower and the luminance change is higher in contrast, the degree of discomfort, which is the corrected value, increases. However, it is desirable that the minimum value of the average luminance is set to be about 2% of the luminance value of 100% white in order to emphasize dark noise and determine that the degree of discomfort is high. .

<Discomfort correction unit>
The discomfort level correction unit 70 is for obtaining the discomfort level in consideration of the time accumulation effect. The greater the discomfort level after correction at the previous measurement time, the greater the corrected discomfort level at the current measurement time. Then, the discomfort degree at the time of the current measurement output from the discomfort degree calculation unit 50 is corrected to calculate the corrected discomfort degree and output to the logarithmic conversion unit 70. In this embodiment, the discomfort degree correction unit 60 includes an addition unit 61, a storage unit 62, and a multiplication unit 63, and stores the corrected discomfort degree at the time of the previous measurement for one measurement section in the storage unit 62. In addition, at the current measurement time point, a value obtained by multiplying the post-correction discomfort level at the previous measurement time point by a constant α (0 <α <1) by the multiplication unit 63 is output from the discomfort degree calculation unit 50 by the addition unit 61. This is realized by adding to the discomfort level at. Here, the value of the constant α represents the accumulation effect of the discomfort level when the blinking continues for a long time. When the measurement interval is 1 second, the estimation error of the discomfort level is minimum when about 0.84. It becomes. Note that the discomfort degree correction unit 60 outputs an uncorrected discomfort degree because the previous output is not stored in the storage unit 62 during the first correction.

<Logarithm conversion unit>
The logarithmic conversion unit 60 calculates the logarithmic value of the corrected discomfort level output from the discomfort level correction unit 60 so that the correspondence between the estimated discomfort level and the psychological evaluation value is almost linear, and outputs the logarithmic value as the estimated discomfort level b. To do.

  In the invention described in Patent Document 1, the discomfort degree is calculated based on the combination of the brightness / darkness luminance difference (luminance amplitude) and the number of luminance inversions (twice the frequency) based on the table in FIG. As a result of investigating the relationship between the magnitude of each block in the present embodiment and the discomfort level shown in FIG. 3, if the magnitude of each block has the same value in any combination of luminance amplitude and frequency, the rating value It has been confirmed that the same discomfort occurs within the 90% confidence interval. Therefore, when the maximum luminance, the minimum luminance, and the luminance inversion period of the luminance change are all constant, the estimated discomfort level according to the present embodiment matches the estimated discomfort level according to the invention described in Patent Document 1. In addition, the discomfort level estimation apparatus 1 according to the present embodiment is a brightness change that performs a complex brightness transition that cannot be accurately estimated by the invention described in Patent Document 1 described in the above-described problem. However, the degree of discomfort can be estimated with the same accuracy as in the case of steady luminance changes.

  Further, in the block-specific discomfort degree calculation unit in the invention described in Patent Document 1, in order to calculate the discomfort degree by the combination of the multi-stage brightness / luminance difference and the number of brightness inversions for each block, the number of stages of the brightness / darkness brightness difference, Each block discomfort level calculation unit requires one multiple of the number of processing steps (circuit scale in the case of an apparatus) necessary for the brightness difference between dark and dark, but the discomfort level estimation apparatus 1 according to the present embodiment has a magnitude. Already corresponds to the discomfort level, the block-based magnitude measurement unit 30 may perform one stage of processing, and the total number of processing steps (circuit scale in the case of an apparatus) is estimated according to Patent Document 1. It can be suppressed to one-tenth or less of the apparatus. That is, the discomfort degree estimation apparatus 1 according to the present embodiment can increase the estimation accuracy of the discomfort degree and reduce the number of processing processes, thereby realizing high speed, low cost, and high precision.

  Further, the discomfort level estimation device 1 is suitable for determining whether or not a video is good and setting a target level to which a frequent luminance change included in the video should be reduced when used by a video content creator at the production stage. Therefore, not only can the time, labor, and cost required for production be reduced, but also the safety and comfort of the supplied video content can be improved.

  In addition, when used on the viewer side of a video, the discomfort level estimation device 1 applies to a video produced and distributed without guaranteeing that it is safe and comfortable with respect to frequent luminance changes before or after viewing. By estimating the degree of discomfort immediately before display during viewing and outputting it to the display or speaker, a warning can be issued during viewing, so that it is possible to prevent health damage and inducing discomfort due to video sickness.

  As mentioned above, although embodiment of this invention was described with reference to embodiment, this invention is not limited to the said embodiment, A design change is possible suitably in the range which does not deviate from the summary of this invention. For example, the present invention can also be embodied as a discomfort level estimation program that causes a computer to function as the discomfort level estimation device 1. Further, when the degree of discomfort calculated by the logarithmic conversion unit 70 exceeds a threshold, it is determined that the video includes an unpleasant luminance change, and the determination result is output to an external device such as a display. Good. Further, when it is not necessary to approximate the discomfort level to the linear perceptual amount, the logarithmic conversion unit 70 can be omitted, and the corrected discomfort level calculated by the discomfort level correction unit 60 can be output as the discomfort level. It is. Further, when correcting the video, it is necessary to grasp the magnitude in a short time without accumulation of time, so the discomfort degree correction unit 60 and the logarithmic conversion unit 70 are omitted, and the discomfort degree calculation unit 50 calculates the image. It is also possible to output the degree of discomfort. In addition, the discomfort degree correction unit 60 may be omitted, and the discomfort degree calculated by the logarithmic conversion unit 70 logarithmically transforming the discomfort degree calculated by the discomfort degree calculation unit 50 is also possible. That is, the discomfort degree estimation apparatus 1 of the present invention can change the discomfort level that is output according to the use of the discomfort level.

DESCRIPTION OF SYMBOLS 1 Discomfort degree estimation apparatus 10 Block luminance value calculation part 20 Movement vector detection part 30 Magnitude measurement part according to block 40 Average luminance calculation part 50 Discomfort degree calculation part 60 Discomfort degree correction part 70 Logarithmic conversion part

Claims (6)

A discomfort level estimation device for estimating a discomfort level for luminance changes in a video,
A block luminance value calculation unit that calculates a block luminance value that is a luminance value for each block of the image included in the video;
Based on the calculated block luminance value, a movement vector detection unit that detects a movement vector of a subject in a video between one frame;
Based on the inter-frame luminance difference value, which is the difference between the block luminance value in the block and the block luminance value in the block corrected with the movement vector detected in the image one frame before, each of the blocks A block-by-block magnitude measurement unit that measures the block-by-block magnitude, which is the magnitude of the luminance change in
An unpleasant degree calculation unit for calculating an unpleasant degree by adding the calculated magnitude by block over the number of frames of a predetermined measurement interval in all blocks;
A discomfort degree estimation device comprising: 2. The discomfort degree estimation according to claim 1, wherein the block-by-block magnitude measurement unit measures the block-by-block magnitude by squaring after correcting the inter-frame luminance difference value by a frequency sensitivity correction filter. apparatus. Further, an average luminance calculation unit that calculates the average luminance of the entire screen during the predetermined measurement interval by averaging the block luminance value of each block over the number of frames of the predetermined measurement interval,
The discomfort degree estimation device according to claim 1, wherein the block-by-block magnitude measurement unit includes an average brightness correction unit that corrects the discomfort degree based on the average brightness. Further, the uncomfortable degree after correction at the current measurement time point is calculated by multiplying the uncomfortable degree after correction at the previous measurement time point by a constant larger than 0 and smaller than 1 and adding it to the uncomfortable degree at the current measurement time point. The discomfort degree estimation apparatus according to any one of claims 1 to 3, further comprising a discomfort degree correction unit obtained as a degree. 5. The discomfort degree estimation apparatus according to claim 1, further comprising a logarithmic conversion unit that logarithmically converts the discomfort degree and outputs the estimated discomfort degree. To estimate the discomfort for brightness changes in the video,
A block luminance value calculation unit for calculating a block luminance value which is a luminance value for each block of the image included in the video;
A movement vector detection unit that detects a movement vector between one frame of a subject in the video based on the calculated block luminance value;
Based on the inter-frame luminance difference value, which is the difference between the block luminance value in the block and the block luminance value in the block corrected with the movement vector detected in the image one frame before, each of the blocks A block-by-block magnitude measuring unit that measures a block-by-block magnitude that is a magnitude of a luminance change in
An unpleasant degree calculation unit for calculating an unpleasant degree by adding the calculated magnitudes of the respective blocks over the number of frames of a predetermined measurement interval in all blocks;
An unpleasantness degree estimation program characterized by functioning as

Images