JP2011238120A - Discomfort degree estimation device and discomfort degree estimation program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a discomfort degree estimation device for estimating a discomfort degree to be sensed by a viewer by global motions, based on the global motions.SOLUTION: The discomfort degree estimation device 1 includes: a global motion detection part 10 for detecting a plurality of kinds of global motions of an image; a Fourier transformation part 20 for calculating frequency components of the plurality of kinds of global motions by performing discrete Fourier transformation for a longer time as a frequency is lower; a weighting part 30 for performing weighting in the frequency components of the plurality of kinds of global motions; a discomfort degree calculation part 40 for calculating a discomfort degree by calculating a nonlinear sum of the weighted frequency components of the plurality of kinds of global motions and transforming the calculated nonlinear sum into a logarithm; and a discomfort degree correction part 50 for correcting the discomfort degree by multiplying a previous time discomfort degree by a constant which is larger than zero and smaller than one, and adding the previous time discomfort degree after the multiplication of the constant to a this time discomfort degree.

Description

  The present invention relates to a discomfort level estimation apparatus and a discomfort level estimation program for estimating a discomfort level of an image.

  As the display when watching broadcasts, recorded videos, personal computer videos, etc. at home increases in screen size, there is a tendency for discomfort induced by screen motion (video motion), so-called global motion, to increase. In some cases, viewers may experience video sickness resulting in health damage. In order to prevent such a situation, it is almost essential to install a “camera shake correction” function in a hand-held video camera or the like. In addition, a lot of “Video Stabilization” software has been developed to reduce the global motion contained in the captured video. Furthermore, an attempt has been made to estimate an unpleasant state by searching for a relationship between global motion and physiological indices such as pulse and blood pressure (see Non-Patent Document 1).

“Feasibility Study Report on Practical Use of Visual Sickness Guidelines Verification System”, Mechanical Systems Promotion Association, March 2008

  However, the above-mentioned “camera shake correction” cannot correct or inadequate long-period shaking, and “image stabilization” eliminates or reduces image shaking within a short period or converts it to smooth screen movement. It is not based on an objective standard that it can be left as long as it is shaken by what kind of property and how much. In addition, even if physiological indices such as pulse and blood pressure can be used as this objective standard, it is possible to specify only the conditions for reaching a health hazard and the current physiological state for those conditions. .

  On the other hand, suppliers of video content such as broadcasts and movies need to be careful not to cause discomfort associated with global motion to the majority of viewers in addition to video safety during the video production stage. . However, the above-described conventional technology does not specifically grasp the viewer's psychological “discomfort” with respect to global motion, but to determine the quality of a video based on the “discomfort” due to global motion. It was unsuitable.

  The present invention has been developed in view of the above-described circumstances, and provides a discomfort level estimation device and a discomfort level estimation program capable of estimating a discomfort level felt by a viewer based on global motion based on the global motion. This is the issue.

  In order to solve the above-mentioned problem, a discomfort degree estimation apparatus according to the present invention is a discomfort degree estimation apparatus that estimates a discomfort degree based on a global motion indicating screen motion in a video, and includes a global motion detection unit and a discrete Fourier transform unit. And a weighting unit, a discomfort calculation unit, and a discomfort correction unit.

  With this configuration, the discomfort level estimation device detects a plurality of types of global motion between temporally adjacent images of the video by a global motion detection unit, and performs a discrete Fourier transform unit on the plurality of types of global motion. The frequency components of the plurality of types of global motion are calculated by performing discrete Fourier transform in a longer period as the frequency is lower, and the frequency components of the plurality of types of global motion are respectively calculated by the weighting unit. Weighting is performed, and the discomfort degree calculation unit calculates a non-linear sum of the frequency components of the plurality of types of global motions, thereby calculating the discomfort degree. In order to increase the discomfort level, Degrees to correct for.

  With this discomfort level estimation device, the discomfort level can be estimated based on global motion regardless of the viewer's physiological index. In addition, since the weighting unit weights multiple types of global motion, the relationship between the frequency components of multiple types of global motion and the level of discomfort resulting from them is adjusted, and unpleasantness based on multiple types of global motion. Degrees can be handled in the same row. Further, since the discomfort level correcting unit corrects the current discomfort level based on the previous discomfort level, it is possible to output the discomfort level reflecting the accumulation effect when the global motion continues for a long time.

  The global motion detection unit may be configured to detect global motion related to vertical movement, horizontal movement, rotation, and zoom as the plurality of types of global motion.

  The weighting unit, for the frequency components of the plurality of types of global motion calculated by the Fourier transform unit, respectively, calculates the reciprocal number of the discomfort threshold obtained in advance for each frequency of the plurality of types of global motion. A configuration may be used in which weighting is performed by multiplying frequency components of various types of global motion.

  The discomfort degree calculation unit may output the non-linear sum converted to a logarithm as the discomfort degree to the discomfort degree correction unit.

  In this discomfort degree estimation apparatus, the uncomfortable degree before correction can be calculated as a value that increases at equal intervals.

  The discomfort level correcting unit multiplies the previous discomfort level by a constant larger than 0 and smaller than 1, and adds the previous discomfort level multiplied by the constant to the current discomfort level, thereby obtaining the discomfort level. May be configured to calculate.

  The present invention can also be embodied as a discomfort level estimation program that causes a computer to function as the above-described discomfort level estimation device.

  According to the present invention, it is possible to estimate the degree of discomfort felt by the viewer by global motion based on the global motion.

It is a block diagram which shows the discomfort degree estimation apparatus which concerns on embodiment of this invention. It is a table | surface which shows the relationship between the frequency of the frequency component calculated by a discrete Fourier-transform part, and the time window length (frame number) of the time window for calculating the said frequency component. It is a graph which shows an example of the frequency characteristic of the reciprocal number of a discomfort threshold. It is a detailed block diagram which shows the discomfort degree calculation part of FIG.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings as appropriate. Similar parts are denoted by the same reference numerals, and redundant description is omitted. In the present invention, “global motion” refers to movement of the entire screen (motion of the screen) in the motion of the video, and is distinguished from “local motion” in which the subject moves locally in the video.

  As shown in FIG. 1, the discomfort level estimation apparatus 1 according to the embodiment of the present invention includes a global motion detection unit 10, a discrete Fourier transform unit 20, a weighting unit 30, and a discomfort level calculation unit 40 as functional units. The discomfort degree correction unit 50 is provided.

<Global motion detector>
The global motion detection unit 10 acquires a video signal, and based on the acquired video signal, detects a plurality of types of global motion (difference between frames) between temporally adjacent images of the video in time series, The detection result is output to the discrete Fourier transform unit 20. In this embodiment, the global motion detection unit 10 includes, as a plurality of types of global motion, a global motion related to vertical movement (vertical direction (tilt) movement amount of the imaging device) and a global motion related to horizontal movement amount (horizontal direction of the imaging device ( Pan) movement amount), global motion related to rotation (rotation amount around the optical axis (roll) of the image pickup apparatus), and global motion related to zoom (change amount of zoom magnification of the image pickup apparatus).

  In addition, regarding the global motion related to the vertical movement, the case where the subject in the image moves upward is positive, and the case where the subject moves downward is negative. As for the global motion related to the left / right movement, the positive is when the subject in the image moves to the right, and the negative when the subject moves to the left. In addition, regarding global motion related to rotation, a positive value is obtained when the subject in the image rotates counterclockwise, and a negative value when the object rotates clockwise. As for the global motion related to zoom, a positive value is given when the size of the subject in the image is enlarged, and a negative value is given when the size is reduced.

  Further, the global motion detection unit 10 may be configured to normalize the plurality of types of global motion as the average moving viewing angle amount in the entire screen when the viewer views the video under the standard viewing condition. Here, the standard viewing condition refers to a viewing condition for viewing from a distance three times the height of the screen in the case of a high-definition video, for example. With such a configuration, it is possible to treat each global motion having different units with the same number of pixels (global motion related to vertical movement and horizontal movement), angle (global motion related to rotation), and size ratio (global motion related to zoom).

  As a general global motion detection method, the global motion detection unit 10 divides the screen into a plurality of blocks, calculates the amount of movement of each block image between frames by block matching, and corrects the block position. The global motion value is calculated when the corresponding square error of is minimum. In such a calculation method, the global motion detection unit 10 can appropriately adopt a known method such as a RANSAC (RANdom SAmple Consensus) method, an MPDE (MultiPartial Differential Equation) method, or the like.

<Discrete Fourier transform unit>
The discrete Fourier transform unit 20 acquires a plurality of types of global motion detected by the global motion detection unit 10, and performs discrete Fourier transform on the acquired plurality of types of global motion in a longer period as the frequency is lower. As a result, frequency components (values proportional to the absolute value of the amplitude) of a plurality of types of global motion are calculated, and the calculation result is output to the weighting unit 30.

  In the present embodiment, the discrete Fourier transform unit 20 performs a discrete Fourier transform every 1 [second]. This is because most viewers (average raters in the evaluation experiment) take 2 [seconds] at the earliest to reach the threshold, and therefore, the discomfort can be estimated at intervals of 1 [second]. This is because sufficient accuracy can be obtained.

  In the present embodiment, the discrete Fourier transform unit 20 includes a discrete Fourier transform unit 21 that performs a discrete Fourier transform on a global motion related to vertical movement, and a left-right move that performs a discrete Fourier transform on a global motion related to a horizontal movement. A discrete Fourier transform unit 22 for rotation, a discrete discrete Fourier transform unit 23 for performing discrete Fourier transform on a global motion related to rotation, a discrete Fourier transform unit 24 for zooming performing discrete Fourier transform on a global motion related to zoom, Is provided.

  FIG. 2 is a table showing the relationship between the frequency component frequency calculated by the discrete Fourier transform unit and the time window length (number of frames) of the time window for calculating the frequency component. As shown in FIG. 2, the discrete Fourier transform unit 20 has a frequency component of 0.06 [Hz] or less (time window length 34.13 [seconds]) that is recognized as a substantially constant speed motion, not as a global motion. 0 to 2 periods) is excluded from the calculation of frequency components.

  Moreover, since it is clear from the evaluation experiment that it takes 3 to 5 cycles for the global motion to be recognized as unpleasant, the discrete Fourier transform unit 20 has the lowest frequency (1 cycle) in each time window. ) To calculate the frequency component of global motion at a frequency about 3 to 5 times that of).

  That is, since the time window in which the frequency (two cycles) twice the lowest frequency is slightly smaller than 0.06 [Hz] may be used as the longest time window, in this embodiment, the discrete Fourier transform unit 20 is In an image of 30 [Hz] that is a standard frame frequency, the longest time window is set to 1024 frames (34.13 [seconds]), and discrete Fourier transform is performed.

  Then, the discrete Fourier transform unit 20 calculates the frequency component of global motion at a frequency of about 3 to 5 times the lowest frequency in each time window length while halving the time window length. In addition, since the global motion of 10 [Hz] or more is recognized as a double image rather than a screen shake, the discrete Fourier transform unit 20 sets the time window to 16 frames (0.53 [seconds]). To perform a discrete Fourier transform.

  The discrete Fourier transform unit 21 for vertical movement sets the number of frames to 1024 and calculates the frequency component of global motion related to vertical movement of 0.088 [Hz], 0.117 [Hz], and 0.146 [Hz]. Then, the number of frames is set to 512, the frequency component of global motion related to vertical movement of 0.176 [Hz], 0.234 [Hz], and 0.293 [Hz] is calculated, and the number of frames is set to 256. Then, the frequency component of the global motion related to the vertical movement of 0.352 [Hz], 0.469 [Hz], 0.586 [Hz] is calculated, the number of frames is set to 128, and 0.703 [Hz ], 0.938 [Hz], 1.172 [Hz], the global motion frequency component for vertical movement is calculated, the number of frames is set to 64, and 1.40 The frequency component of the global motion related to the vertical movement of [Hz], 1.875 [Hz], 2.344 [Hz] is calculated, the number of frames is set to 32, and 2.813 [Hz], 3.750 [ The frequency component of the global motion related to the vertical movement of [Hz] and 4.688 [Hz] is calculated, the number of frames is set to 16, 5.625 [Hz], 7.500 [Hz], and 9.375 [Hz]. ] The frequency component of the global motion regarding the vertical movement of 11.250 [Hz], 13.125 [Hz], and 15.000 [Hz] is calculated. Note that the left-right moving discrete Fourier transform unit 22, the rotating discrete Fourier transform unit 23, and the zooming discrete Fourier transform unit 24 also calculate the frequency components of each global motion in the same manner as the up-and-down moving discrete Fourier transform unit 21.

  In the present embodiment, since there is no overlap or omission in the frequency at which the discrete Fourier transform is performed in each time window, the discrete Fourier transform unit 20 can calculate the frequency component in a state where the total energy of the global motion is stored. . Therefore, the discomfort degree estimation apparatus 1 according to the present embodiment does not need to include a filter bank that performs wavelet transform or the like.

  The discrete Fourier transform unit 20 sets the number of frames (time window length) to a power of 2 that is convenient for repeatedly halving, but the frequency component calculated by performing the discrete Fourier transform is Since there are three in each time window (six only in the shortest time window), there is no need to perform Fast Fourier Transform (FFT), and the number of frames need not be set to a power of two. .

<Weighting section>
The weighting unit 30 acquires a plurality of types of global motion frequency components output from the discrete Fourier transform unit 20, and applies weights to the acquired plurality of types of global motion frequency components, In the present embodiment, as the weighting, the reciprocal number of the discomfort threshold obtained in advance for each frequency is multiplied by the frequency components of a plurality of types of global motion, and the multiplication result is output to the discomfort degree calculation unit 40. In the present embodiment, the weighting unit 30 includes a vertical movement weighting unit 31, a horizontal movement weighting unit 32, a rotation weighting unit 33, and a zoom weighting unit 34.

  FIG. 3 is a graph showing an example of the frequency characteristic of the reciprocal of the unpleasant threshold. Here, the unpleasant threshold value is a value obtained by calculating the frequency component threshold value (unpleasant threshold value) of the global motion that the viewer feels unpleasant for each frequency by the geometric average of many evaluators. . The vertical axis in FIG. 3 (the reciprocal of the unpleasant threshold) is a linear scale, and for example, [sec / deg] that is the reciprocal of the average visual speed can be adopted. Further, the reciprocal of the unpleasant threshold is meaningful by the ratio (relative value) of values between frequencies, and the value itself can be appropriately set by those skilled in the art.

The weighting unit for vertical movement 31 acquires the frequency component of the global motion related to the vertical movement output from the discrete Fourier transform unit 21 for vertical movement, weights the obtained plurality of frequency components, and sets the weighted result as the discomfort level. Output to the calculation unit 40. In the present embodiment, the frequency component related to the vertical movement of 0.088 [Hz] is a ₁ , the frequency component related to the vertical movement of 0.117 [Hz] is the vertical movement of a ₂ ,..., 15.000 [Hz]. 15. The frequency component for A is ₂₄ , the reciprocal of the unpleasant threshold for up / down movement of 0.088 [Hz] is k ₁ , the reciprocal of the unpleasant threshold for up / down movement of 0.117 [Hz] is k ₂ ,. 000 when the reciprocal of the discomfort threshold for vertical movement of the [Hz] set to _{k 24,} elevating weighting unit 31, as a result of the _{weighting, k 1 · a 1, k} 2 · a 2, ···, k 24 Output a ₂₄ to the discomfort level calculation unit 40.

The left / right moving weighting unit 32 acquires the frequency component of the global motion related to the left / right movement output from the left / right moving discrete Fourier transform unit 22, weights the acquired plurality of frequency components, and determines the weighted result as the discomfort level. Output to the calculation unit 40. In the present embodiment, the frequency component related to the horizontal movement of 0.088 [Hz] is a ₂₅ , the frequency component related to the horizontal movement of 0.117 [Hz] is the horizontal movement of a ₂₆ ,..., 15.000 [Hz]. 15. The frequency component related to A is ₄₈ , the reciprocal of the discomfort threshold relating to 0.088 [Hz] left-right movement is k ₂₅ , the reciprocal of the discomfort threshold relating to 0.117 [Hz] left-right movement is k ₂₆ ,. 000 when the reciprocal of the discomfort threshold for lateral movement of the [Hz] set to k _48, lateral movement weighting unit 32, k ₂₅ by multiplying the reciprocal of the pre-stored discomfort threshold value (weight value) and a frequency component A ₂₅ , k ₂₆ · a ₂₆ ,..., K ₄₈ · a ₄₈ are calculated, and the calculation result is output to the discomfort degree calculation unit 40 as a weighting result.

The rotation weighting unit 33 acquires the frequency component of the global motion related to the rotation output from the rotation discrete Fourier transform unit 23, weights the plurality of acquired frequency components, and sets the weighted result as the discomfort degree calculation unit 40. Output to. In the present embodiment, a frequency component related to rotation of 0.088 [Hz] is a ₄₉ , a frequency component related to rotation of 0.117 [Hz] is a frequency component related to rotation of a ₅₀ ,..., 15.000 [Hz]. was a _{a 72,} 0.088 _k 49 the inverse of the discomfort threshold for rotation of [Hz], 0.117 [Hz] k 50 the inverse of the discomfort threshold for rotation, ..., of 15.000 [Hz] when the reciprocal of the discomfort threshold for rotating and k _72, rotation weighting unit 33 in advance reciprocal of the stored discomfort threshold (weight value) and k _{49 ·} a ₄₉ by multiplying the frequency _components, k ₅₀ · a ₅₀ ,..., k ₇₂ · a ₇₂ are calculated, and the calculation result is output to the discomfort degree calculation unit 40 as a result of weighting.

The zoom weighting unit 34 acquires the frequency component of the global motion related to the zoom output from the zoom discrete Fourier transform unit 24, weights the acquired plurality of frequency components, and sets the weighted result as the discomfort degree calculation unit 40. Output to. In the present embodiment, a frequency component related to a zoom of 0.088 [Hz] is a ₇₃ , a frequency component related to a zoom of 0.117 [Hz] is a frequency component related to a zoom of a ₇₄ ,. Is a ₉₆ , the reciprocal of the discomfort threshold for zoom at 0.088 [Hz] is k ₇₃ , and the reciprocal of the discomfort threshold for zoom at 0.117 [Hz] is k ₇₄ ,..., 15.000 [Hz]. when the reciprocal of the discomfort threshold for zooming and k _96, zooming weighting unit _{34, k 73 · a 73, k} 74 · by multiplying the reciprocal of the pre-stored discomfort threshold value (weight value) and a frequency component a ₇₄ ,..., k ₉₆ · a ₉₆ are calculated, and the calculation result is output to the discomfort degree calculation unit 40 as a weighted result.

<Discomfort calculation unit>
Discomfort calculating unit 40 obtains the frequency component _{k 1 · a 1 ~k 96 ·} a 96 weighted by the weighting unit 30, based on such a weighted frequency components _{k 1 · a 1 ~k 96 ·} a 96 Then, the discomfort degree c is calculated. In the present embodiment, the discomfort degree calculation unit 40 repeatedly calculates the discomfort degree c every second and outputs it to the discomfort degree correction unit 50. FIG. 4 is a detailed block diagram illustrating the discomfort level calculation unit of FIG. As shown in FIG. 4, the discomfort degree calculation unit 40 includes a component addition unit 41, a logarithmic conversion unit 42, and a maximum value detection unit 43 within a predetermined time.

The component addition unit 41 obtains the weighted frequency components k1 · a1 to k96 · a96 output from the weighting unit 30, calculates the nonlinear sum b by performing nonlinear addition, and adds the calculated result to the component To the unit 42.
_{b = {Σ (k n ·} a n) γ} 1 / γ

That is, the component adding unit 41 adds all the weighted frequency components k _n · a _n (n = 1, 2,... 96) to the γ power and calculates the 1 / γ power. Therefore, the nonlinear sum b is obtained. Here, the non-linear sum b is a sum of weighted frequency components (amplitude sum) when γ = 1, and is a sum of global motion energy when γ = 2. When examining each evaluator of the subjective evaluation experiment, when γ is about 2, an unpleasant value d described later is suitable for the subjective evaluation experiment. However, when two types of global motion with the same unpleasant value by rating are synthesized, there is a standard deviation of about 30% around the average value when viewed by each evaluator. The value increases by about 4%. In this case, γ is about 1.8. Therefore, the value of γ is
1.8 ≦ γ ≦ 2.0
It is desirable to set so as to satisfy.

The logarithmic conversion unit 42 calculates the discomfort degree c by converting the non-linear sum b output from the component addition unit 41 into a logarithm, and outputs the calculation result to the maximum value detection unit 43 within a predetermined time.
c = log _X b

In this way, the degree of discomfort c is calculated by logarithmically transforming the non-linear sum b. This is a measure based on the fact that the values obtained by logarithmically transforming the nonlinear sum b at the boundary of the category of “unbearable and uncomfortable” are almost equally spaced. Here, when the nonlinear sum b is about 6.5 times, the discomfort level increases by one category.
X≈6.5
It is desirable to be set to.
The discomfort threshold described above is a threshold indicating a boundary between “somewhat discomfort” and “discomfort” in the category of the subjective evaluation experiment.

  Further, the nonlinear sum b has a critical value that does not reach the discomfort threshold even if it accumulates for a long time below a certain value. When the nonlinear sum b is equal to the critical value, the base of the logarithm is such that the discomfort degree c becomes zero. X is set, and the logarithmic conversion unit 42 is configured not to output the discomfort level c (or output 0 as the discomfort level c) when the discomfort level c is negative. Such a critical value is desirably set to a value about 1/50 of the discomfort threshold as a value before logarithmic conversion according to a subjective evaluation experiment.

  The maximum value detection unit 43 within a predetermined time detects the maximum value of the discomfort degree c during a predetermined period (for example, 5 to 7 [seconds]) immediately before the discrete Fourier transform processing time point by the discrete Fourier transform unit 20, The maximum value is output to the adding unit 51 of the discomfort level correcting unit 50 as the discomfort level c. This means that the rating discomfort for a video in which the period of no global motion is intermittent is about 3 seconds is almost the same as the rating discomfort for a video in which global motion of the same amplitude is continuously generated (that is, Even if there is no global motion for about 3 seconds, if global motion has occurred before and after that, humans will feel discomfort influenced by the previous global motion).

  In the present embodiment, the maximum value detection unit 43 within the predetermined time includes a storage unit 43a and a maximum value detection unit 43b.

  The storage unit 43a stores the most recent predetermined number (for example, 5 to 7) of the discomfort degree c output from the logarithmic conversion unit 42, and stores the predetermined number of discomfort levels c to the maximum value detection unit 43b. This is a shift register to output.

  The maximum value detection unit 43b compares a predetermined number of discomfort levels c output from the storage unit 43a, detects the maximum discomfort level c, and outputs it to the addition unit 51.

<Discomfort correction unit>
Returning to Figure 1, discomfort degree correcting unit 50, the previous (one second ago) unpleasantness c (hereinafter, the previous and unpleasantness c _m) is too large this unpleasantness c (hereinafter, this unpleasantness The uncomfortable degree d after correction is calculated by correcting the current uncomfortable degree _{cm + 1} so that (c _{m + 1} ) becomes large. In the present embodiment, the discomfort degree calculation unit 50 includes an addition unit 51, a storage unit 52, and a multiplication unit 53.

The adding unit 51 acquires the previous discomfort level _cm output from the discomfort level calculation unit 40, and outputs the acquired previous discomfort level _cm to the storage unit 52.

The storage unit 52 is a shift register that stores only one previous discomfort degree _cm output from the addition unit 51 and outputs the stored previous discomfort degree _cm to the multiplication unit 53.

The multiplication unit 53 acquires the previous discomfort degree _cm output from the storage unit 52, multiplies the acquired previous discomfort degree _cm by a constant p (0 <p <1), and calculates the calculation result p · c _m is output to the adder 51. The constant p represents an effect of accumulating discomfort when the global motion continues for a long time. In order to minimize an estimation error of discomfort, it is desirable that p is about 0.005.

Then, the adding unit 51 obtains the current discomfort level _{cm + 1} that is output from the discomfort level calculation unit 40, and adds the previous discomfort level p · _cm multiplied by a constant to the current discomfort level _{cm + 1.} As a result, the corrected discomfort level d is calculated, and the calculated discomfort level d is output to an external device (notification unit for notifying the user of the discomfort level d) such as a display or a speaker, and a new previous discomfort level is output. It outputs to the storage unit 52 as the degree.
d = c _{m + 1} + p · c _m
The adding unit 51 calculates and outputs the corrected discomfort degree d every 1 [second] which is the timing of the discrete Fourier transform by the discrete Fourier transform unit 20.

  The discomfort degree estimation apparatus 1 according to the embodiment of the present invention has a discomfort degree that is close to the average value in the subjective evaluation experiment, even for images with screen fluctuations in which frequency components of multiple types of global motion change over time. d can be estimated. Further, the estimated discomfort degree d can be used as an index of the degree of video correction when the video content producer performs video correction to reduce screen shaking.

That is, the discomfort level estimation apparatus 1 is suitable for determining whether a video is good or not, and for setting a target for how much the screen shaking included in the video should be reduced, when used by the video content creator at the production stage. Since the degree d can be presented to the video content producer, not only can the time, labor, and cost required for production be reduced, but also the safety and comfort of the supplied video content can be improved.
Further, when used on the viewer side of a video, the discomfort level estimation device 1 is pre-viewing or watching a video that has been produced and distributed without guaranteeing that the screen is safe and comfortable with respect to shaking. Since the degree of discomfort d is estimated and output to the display or the speaker immediately before the display of the image, a warning can be issued at the time of viewing, so that it is possible to prevent health damage and inducing discomfort due to video sickness.

  As mentioned above, although embodiment of this invention was described with reference to embodiment, this invention is not limited to the said embodiment, A design change is possible suitably in the range which does not deviate from the summary of this invention. For example, the present invention can also be embodied as a discomfort level estimation program that causes a computer to function as the discomfort level estimation device 1. In addition, when the discomfort degree d determined by the discomfort degree determination unit 40 exceeds a threshold, it is determined that the video includes an unpleasant global motion, and the determination result is output to an external device such as a display. May be. Further, the logarithmic conversion unit 42 and the maximum value detection unit 43 within a predetermined time can be omitted.

DESCRIPTION OF SYMBOLS 1 Discomfort degree estimation apparatus 10 Global motion detection part 20 Discrete Fourier transform part 30 Weighting part 40 Discomfort degree calculation part 50 Discomfort degree correction part

Claims (6)

A discomfort level estimation device that estimates a discomfort level based on global motion indicating a screen shake in a video,
A global motion detector that detects a plurality of types of global motion between temporally adjacent images of the video;
A discrete Fourier transform unit that calculates a frequency component of the plurality of types of global motion by performing discrete Fourier transform in a longer period as the frequency is lower for each of the detected types of global motion,
A weighting unit that weights the calculated frequency components of the plurality of types of global motion,
A discomfort degree calculating unit that calculates the discomfort degree by calculating a nonlinear sum of weighted frequency components of the plurality of types of global motion;
A discomfort level correction unit that corrects the current discomfort level so that the current discomfort level increases as the previous discomfort level increases;
A discomfort degree estimation device comprising: The said global motion detection part detects the global motion regarding a vertical movement, a left-right movement, a rotation, and a zoom as said multiple types of global motion. The discomfort degree estimation apparatus of Claim 1 characterized by the above-mentioned. The weighting unit, for the frequency components of the plurality of types of global motion calculated by the Fourier transform unit, respectively, calculates the reciprocal number of the discomfort threshold obtained in advance for each frequency of the plurality of types of global motion. 3. The discomfort degree estimation apparatus according to claim 1, wherein weighting is performed by multiplying frequency components of different types of global motion. The unpleasantness degree calculation unit outputs the conversion of the non-linear sum into a logarithm as the unpleasantness level to the unpleasantness degree correction unit. The unpleasantness according to any one of claims 1 to 3 Degree estimation device. The discomfort level correcting unit multiplies the previous discomfort level by a constant larger than 0 and smaller than 1, and adds the previous discomfort level multiplied by the constant to the current discomfort level after correction. The discomfort degree estimation apparatus according to any one of claims 1 to 4, wherein the discomfort degree of the current time is obtained. A discomfort degree estimation program that estimates discomfort based on global motion indicating screen sway in a video,
Computer
A global motion detector for detecting a plurality of types of global motion between temporally adjacent images of the video,
A discrete Fourier transform unit for calculating the frequency components of the plurality of types of global motion by performing discrete Fourier transform on the detected types of global motions in a longer period as the frequency is lower,
A weighting unit for weighting the calculated frequency components of the plurality of types of global motion,
A discomfort degree calculating unit for calculating the discomfort degree by calculating a nonlinear sum of weighted frequency components of the plurality of types of global motion; and
A discomfort level correction unit that corrects the current discomfort level so that the current discomfort level increases as the previous discomfort level increases;
An unpleasantness degree estimation program characterized by functioning as

Images