JP2008072665A - Video evaluation device, video frame rate determination apparatus, video evaluation method, video frame rate determination method, video evaluation program, and video frame rate determination program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To evaluate smoothness in the motion of a video image while taking into account the motion featured value of the relevant video image, a frame rate of the video image, and the analysis result of a spatial frequency in a frame image constituting the video image. <P>SOLUTION: A video evaluation device 10 comprises: a motion featured value extraction unit 102 which extracts the motion featured value of an input video image on the basis of a frame image constituting the input video image; a frame rate acquisition unit 103 which acquires the frame rate of the input video image; a spatial frequency analysis unit 104 which analyzes a spatial frequency in the frame image constituting the input video image; and an evaluation unit 105 which calculates an evaluation value indicating the smoothness in the motion of the video image, using the motion feature amount, the frame rate and the analysis result of the spatial frequency. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video evaluation apparatus, a video evaluation method, and a video evaluation program for evaluating the smoothness of motion of a video composed of a plurality of frame images.

  The present invention also relates to a video frame rate determination device, a video frame rate determination method, and a video frame rate determination program for determining a frame rate of a video composed of a plurality of frame images.

  In general, processing including acquisition, storage, transmission, display, encoding, and decoding for “video” is performed in a state where the number of frames per second is constant, that is, at a fixed frame rate.

  For example, according to the National Television Standards Committee (NTSC) standard adopted in the United States and Japan, the frame rate is defined as 29.97 fps, and the national television standard adopted in Europe. In the phase alternating line (PAL) standard established by the committee, the frame rate is defined as 25 fps (see Non-Patent Document 1).

  In some cases, the above-described image processing is performed at a fixed frame rate of 24 fps or 15 fps.

  Here, “video” is composed of a plurality of “frame images” that are a single still image, and “frame rate” is the number of frame images processed per second. “Fps (frame per second)” is a unit representing a frame rate.

  In a video processed at a fixed frame rate, if the number of frame images processed per second increases, the time interval between successive frame images is shortened, so that it can be processed as a smoother motion video. .

  For example, a video processed at a frame rate of 30 fps can be expressed with a smooth motion because the motion of the video can be expressed more finely in time than a video processed at a frame rate of 15 fps.

  On the other hand, when the number of frame images processed per second in a video processed at a fixed frame rate increases, the processing amount per unit time, the amount of data accompanying the processing, and the power consumption increase.

  For example, in the acquisition of video, as the frame rate increases, the number of frame images to be acquired per unit time increases, and the processing amount per unit time and the power consumption associated with the processing increase.

  In video storage, as the frame rate increases, the number of frames to be stored per unit time increases, and the amount of data associated with processing per unit time increases.

  There is also a video processing method in which the frame rate is variable according to the video processing amount and data amount.

For example, in video encoding, if the amount of data to be encoded is large, the buffer memory occupancy at the time of encoding exceeds a specified value, so the frame rate is reduced and encoding is performed per unit time. The number of frames to be reduced is reduced (see Patent Document 1).
Recommendation ITU-R BT.470-6, Conventional Television Systems JP-A-11-205791

  However, when processing a video at a fixed frame rate, increasing the frame rate for processing in order to achieve smooth motion increases the processing amount, the amount of data accompanying the processing, and the power consumption. There was a problem.

  Conversely, when processing a video at a fixed frame rate, reducing the frame rate to reduce the amount of processing, the amount of data accompanying the processing, and the power consumption reduces the smoothness of the motion of the video, There was a problem that it became an image of jerky movements.

  Also, when processing a video at a variable frame rate, the frame rate (the time of the frame to be processed) is determined according to only the amount of processing and the amount of data accompanying the processing without considering the smoothness of the motion of the video. Similarly, when the (interval) is changed, there is a problem in that the smoothness of the motion of the image is reduced, resulting in a jerky motion image.

  Here, the smoothness of the motion of the video is related to the characteristics of the video. For example, when there is no motion of the video or when the motion of the video is very small, even if the frame rate is lowered, the smoothness of motion does not decrease.

  In general, when the spatial frequency in an image is high, it is necessary to increase the frame rate in order to accurately grasp the movement. When the frame rate is decreased, the smoothness of the movement decreases.

  Therefore, the present invention has been made in view of the above-described problems, and takes into account the motion feature amount of the video, the frame rate of the video, and the analysis result of the spatial frequency in the frame image constituting the video. Providing a video evaluation apparatus, a video evaluation method, and a video evaluation program capable of evaluating the smoothness of movement with higher accuracy according to the characteristics of the video by evaluating the smoothness of the movement of the video Objective.

  Another object of the present invention is to provide a video frame rate determination device, a video frame rate determination method, and a video frame rate determination program that determine a frame rate that maintains an evaluation value indicating the smoothness of motion at a certain level or higher.

  A first feature of the present invention is a video evaluation device that evaluates the smoothness of motion of a video composed of a plurality of frame images, and based on the frame images constituting the input video, A motion feature amount extraction unit that extracts a motion feature amount, a frame rate acquisition unit that acquires a frame rate of the input video, and a spatial frequency analysis unit that analyzes a spatial frequency in a frame image constituting the input video And an evaluation unit that calculates an evaluation value indicating the smoothness of the motion of the video by using the extracted motion feature amount, the acquired frame rate, and the analysis result of the spatial frequency. This is the gist.

  A second feature of the present invention is a video evaluation device that evaluates the smoothness of motion of a video composed of a plurality of frame images, and based on the inputted frame images that constitute the video, Using a motion feature amount extraction unit that extracts a motion feature amount, a frame rate acquisition unit that acquires a frame rate of the input video, the extracted motion feature amount, and the acquired frame rate, Based on a comparison result between a spatial frequency range determination unit that determines a spatial frequency range to be evaluated for the video, and a signal intensity value in the spatial frequency range in a frame image constituting the input video and a predetermined threshold value And an evaluation unit that calculates an evaluation value indicating the smoothness of the motion of the video.

  In the second aspect of the present invention, the spatial frequency determination unit includes a Nyquist boundary line indicating a limit value of a spatial frequency capable of correctly displaying a motion of a video having a predetermined motion feature amount at a predetermined frame rate, and a predetermined frame. The spatial frequency range may be determined using a video quality boundary line indicating a limit value of a spatial frequency at which a quality related to motion of a video having a predetermined motion feature amount at a rate is maintained at a certain level or higher.

  According to a third aspect of the present invention, there is provided a video evaluation apparatus according to the first or second aspect of the present invention, wherein the video frame rate determination apparatus determines a frame rate of a video composed of a plurality of frame images. A storage unit that stores the video, a frame rate generation unit that generates a first frame rate of the video, the video that is stored in the storage unit, and the frame rate generation unit. A frame rate determination for determining a second frame rate of the video by using a video signal generation unit that generates a video signal including the first frame rate and the evaluation value calculated by the video evaluation device; It is made a summary.

  According to a fourth aspect of the present invention, there is provided a video evaluation method for evaluating the smoothness of motion of a video composed of a plurality of frame images, wherein the video based on the input frame images constituting the video A step of extracting a motion feature, a step of acquiring a frame rate of the input video, a step of analyzing a spatial frequency in a frame image constituting the input video, and the extracted motion feature And a step of calculating an evaluation value indicating the smoothness of the motion of the video using the acquired frame rate and the analysis result of the spatial frequency.

  A fifth feature of the present invention is a video evaluation method for evaluating the smoothness of motion of a video composed of a plurality of frame images, wherein the video based on the input frame images constituting the video A step of extracting a motion feature, a step of acquiring a frame rate of the input video, a target of evaluation of the video using the extracted motion feature and the acquired frame rate The smoothness of the motion of the video is shown based on a result of comparing the signal intensity value in the spatial frequency range in the frame image constituting the input video and a predetermined threshold in the step of determining the spatial frequency range of And a step of calculating an evaluation value.

  In the fifth aspect of the present invention, in the step of determining the spatial frequency range, a Nyquist boundary line indicating a limit value of a spatial frequency capable of correctly displaying a motion of a video having a predetermined motion feature amount at a predetermined frame rate; The spatial frequency range may be determined using a video quality boundary line indicating a limit value of a spatial frequency at which a quality related to a motion of a video having a predetermined motion feature amount at a predetermined frame rate is maintained above a certain level. .

  According to a sixth aspect of the present invention, there is provided a video evaluation method according to the fourth or fifth aspect of the present invention, wherein the video frame rate determination method determines a frame rate of a video composed of a plurality of frame images. An image including a step of generating a first frame rate of the image, the image stored in the storage unit, and the first frame rate generated in the step of generating the frame rate. The gist of the present invention is to include a step of generating a signal and a step of determining a second frame rate of the video using the evaluation value calculated by the video evaluation method.

  A seventh feature of the present invention is a video evaluation device that causes a computer to function as the video evaluation device according to the first or second feature of the present invention.

  The eighth feature of the present invention is summarized as a video frame rate determination program that causes a computer to function as the video frame rate determination device according to the third feature of the present invention.

  As described above, according to the present invention, the smoothness of the motion of the video is considered in consideration of the motion feature quantity of the video, the frame rate of the video, and the analysis result of the spatial frequency in the frame image constituting the video. By performing the evaluation, it is possible to provide a video evaluation device, a video evaluation method, and a video evaluation program that can evaluate the smoothness of motion with higher accuracy according to the characteristics of the video.

  In addition, according to the present invention, it is possible to provide a video frame rate determination device, a video frame rate determination method, and a video frame rate determination program that determine a frame rate that maintains an evaluation value indicating the smoothness of motion at a certain level or more.

(Image evaluation apparatus according to the first embodiment of the present invention)
With reference to FIG. 1 thru | or FIG. 8, the structure of the image | video evaluation apparatus 10 which concerns on the 1st Embodiment of this invention is demonstrated.

  The video evaluation apparatus 10 according to the present embodiment is configured to evaluate the smoothness of motion of a video composed of a plurality of frame images.

  The video evaluation apparatus 10 may be physically realized by a computer including a CPU (Central Processing Unit), a storage device such as a memory, a communication device, and the like. The video evaluation apparatus 10 may be realized by a mobile communication terminal such as a mobile phone. That is, an apparatus capable of performing information processing can be widely applied to the video evaluation apparatus 10.

  As shown in FIG. 1, the video evaluation apparatus 10 includes, as its functional components, an input unit 101, a motion feature amount extraction unit 102, a frame rate acquisition unit 103, a spatial frequency analysis unit 104, and an evaluation unit. 105.

  The input unit 101 is configured to acquire a video signal 107 input from the outside.

  Note that the video signal 107 is a signal including a video composed of a plurality of consecutive frame images that are a single still image and information related to the frame rate of the video.

  The input unit 101 is configured to decompose a video extracted from the video signal 107 input from the outside into a plurality of frame images 108 and output the decomposed frame images 108 to the motion feature amount extraction unit 102. .

  The input unit 101 is configured to output information 109 about the frame rate extracted from the video signal 107 input from the outside to the frame rate acquisition unit 103.

  The input unit 101 is configured to decompose a video extracted from the video signal 107 input from the outside into a plurality of frame images 110 and output the decomposed frame images 110 to the spatial frequency analysis unit 104. Yes.

  The motion feature amount extraction unit 102 extracts the motion feature amount 111 of the video based on the frame image 108 input by the input unit 101, and outputs the extracted motion feature amount 111 of the video to the evaluation unit 105. It is configured.

  Specifically, first, the motion feature amount extraction unit 102 reads two continuous frame images from the input unit 101. For the sake of explanation, the two consecutive frame images are sequentially named as a frame image P0 and a frame image P1.

  Second, the motion feature amount extraction unit 102 divides the frame image P1 read from the input unit 101 into blocks of a certain size (8 pixels × 8 pixels).

  Third, the motion feature quantity extraction unit 102 searches for similar image signal patterns within a predetermined search range of the frame image P0 by block matching for each block of the frame image of the frame image P1.

  Fourth, the motion feature amount extraction unit 102 is a motion that is a spatial displacement amount between the image signal pattern of the block in the frame image P1 and the image signal pattern in the frame image P0 that is most similar to the image signal pattern. Detect vectors. FIG. 2 is a diagram showing a state of motion vector search by block matching.

  Fifth, the motion feature amount extraction unit 102 calculates the magnitude of the motion in the x direction and the y direction of each block of the frame image P1 using the motion vector of each block of the frame image P1.

  In FIG. 2, the motion vector of the block of the frame image P1 is (MVx, MVy), and the magnitudes of motion in the x direction and y direction of the block are | MVx | and | MVy | (|| Is a symbol for calculating an absolute value).

  Sixth, the motion feature quantity extraction unit 102 among the magnitudes of motion in the x direction and y direction of all the blocks in the frame image P1, the maximum value | MVx of the magnitudes of motion in the x direction and y direction, respectively. | Max and | MVy | max are output to the evaluation unit 105 as the motion feature amount 111.

  The frame rate acquisition unit 103 is configured to acquire the above-described video frame rate 112 from the information 109 about the frame rate input by the input unit 101 and output the frame rate 112 to the evaluation unit 105.

  Specifically, when the information 109 regarding the frame rate input by the input unit 101 is the frame rate value itself, the frame rate acquisition unit 103 sets the frame rate value f as the frame rate 112 and evaluates the unit 105. It is configured to output to.

  In addition, when the information 109 regarding the frame rate input by the input unit 101 is the time interval between successive frame images, the frame rate acquisition unit 103 uses the frame rate calculated from the time interval between successive frame images as the frame rate. 112 is configured to output to the evaluation unit 105.

  When the time interval between successive frames is s seconds, the frame rate value f is calculated by 1 / s (fps).

  The method by which the frame rate acquisition unit 103 calculates the frame rate from the information 109 regarding the frame rate input by the input unit 101 is not limited to the above method.

  The spatial frequency analysis unit 104 is configured to analyze the spatial frequency in the frame image 110 input by the input unit 101.

  Specifically, the spatial frequency analysis unit 104 calculates a cumulative value of power at each spatial frequency in the frame image 110 input by the input unit 101, and uses the calculated cumulative power value as a spatial frequency feature amount 113 ( It is configured to output to the evaluation unit 105 as a spatial frequency analysis result).

  The function of the spatial frequency analysis unit 104 will be specifically described with reference to FIG. As shown in FIG. 3, the spatial frequency analysis unit 104 includes a two-dimensional Fourier transform unit 1041 and a cumulative value calculation unit 1042.

  The two-dimensional Fourier transform unit 1041 performs a two-dimensional Fourier transform process on the frame image P1 (1041) input by the input unit 101 to calculate the power 1043 at each frequency, and to the accumulated value calculation unit 1042. Configured for output.

  FIG. 4A shows a frame image P1 input by the input unit 101, and FIG. 4B shows calculation of power 1043 at each frequency by performing a two-dimensional Fourier transform process on the frame image P1. A two-dimensional Fourier transform image F1 including the result is shown.

  The accumulated value calculation unit 1042 uses the power 1043 at each frequency input by the two-dimensional Fourier transform unit 1041, as shown in FIG. 4C, in the ν direction at the spatial frequency μ in the horizontal direction (x direction). All powers are added together, and a μ component cumulative value that is a cumulative value of power at the spatial frequency μ is calculated.

  Further, the cumulative value calculation unit 1042 uses the power 1043 at each frequency input by the two-dimensional Fourier transform unit 1041, and, as shown in FIG. 4D, μ at the spatial frequency ν in the vertical direction (y direction). All the powers in the direction are added together, and a ν component cumulative value that is a cumulative value of power at the spatial frequency ν is calculated.

  Further, the cumulative value calculation unit 1042 is configured to output the calculated μ component cumulative value and ν component cumulative value to the evaluation unit 105 as the spatial frequency feature value 113 (spatial frequency analysis result).

  The evaluation unit 105 includes the motion feature amount extracted by the motion feature amount extraction unit 102, the frame rate acquired by the frame rate acquisition unit 103, and the spatial frequency feature amount 113 (spatial frequency analysis from the spatial frequency analysis unit 104). And the evaluation value 114 indicating the smoothness of the motion of the input video is calculated and output.

  Specifically, the evaluation unit 105 includes the x-direction motion magnitude | MVx | max and the y-direction motion magnitude | MVy | max input from the motion feature quantity extraction unit 102, and the frame rate acquisition unit 103. The evaluation value 114 is calculated by using the frame rate value f input from, and the μ component cumulative value and ν component cumulative value input from the spatial frequency analysis unit 104.

  The function of the evaluation unit 105 will be described in detail with reference to FIG. As illustrated in FIG. 5, the evaluation unit 105 includes a memory 10501, a μ-direction spatial frequency range determination unit 10502, an x-direction motion smoothness value calculation unit 10503, a ν-direction spatial frequency range determination unit 10504, and a y-direction. A smoothness value calculation unit 10505 and an evaluation value calculation unit 10506 are provided.

  The μ-direction spatial frequency range determination unit 10502 obtains the frame rate acquisition by the magnitude (speed) | MVx | max of the motion in the x direction extracted from the motion feature value 111 input from the motion feature value extraction unit 102. The μ direction spatial frequency range information 10507 is acquired using the frame rate value f (112) input from the unit 103, and is output to the x direction motion smoothness value calculation unit 10503. .

  Specifically, the μ-direction spatial frequency range determination unit 10502, based on “Nyquist boundary information in the x direction” stored in the memory 10501 in advance, as shown in FIG. The spatial frequency μ1 of the point at the magnitude | MVx | max of the movement in the x direction on the Nyquist boundary line is obtained.

  Also, the μ-direction spatial frequency range determination unit 10502, based on the “motion smoothness boundary information” stored in the memory 10501 in advance, as shown in FIG. On the smoothness boundary line, the spatial frequency μ0 of the point at the magnitude | MVx | max of the movement in the x direction is acquired.

  Then, the μ-direction spatial frequency range determination unit 10502 outputs the acquired μ0 and μ1 to the x-direction motion smoothness value calculation unit 10503 as the μ-direction spatial frequency range information 10507.

  Here, the “Nyquist boundary line in the x direction at the frame rate f” indicates a boundary at which the motion in the x direction of the video can be correctly displayed at the predetermined frame rate f.

  For example, “the Nyquist boundary line in the x direction at the frame rate f” indicates the limit value μ1 of the spatial frequency that can correctly display the motion of the video having the predetermined motion feature quantity (| MVx | max) at the predetermined frame rate f. .

  That is, the “Nyquist boundary line in the x direction at the frame rate f” is an image having the spatial frequency μ in the horizontal direction (x direction) and the motion magnitude in the x direction is | MVx | This is a boundary where the magnitude of the apparent movement of the video in the x direction is maintained at | MVx |.

  Note that in the region above the boundary line, the video appears to be stationary, appears to move in the opposite direction, or appears to move with a magnitude of movement different from | MVx |.

  The “smooth boundary line in the x direction at the frame rate f” is a video quality boundary line regarding the smoothness of the motion in the x direction, and the motion in the x direction of the video is smooth at a predetermined frame rate f. It shows the relationship between the magnitude of the perceived limit movement and the spatial frequency.

  For example, in the “smooth boundary line in the x direction at the frame rate f”, the quality related to the motion of the video having the predetermined motion feature quantity (| MVx | max) at the predetermined frame rate f is maintained above a certain level. Indicates the limit value of the spatial frequency.

  Note that the motion of the video in the x direction appears jerky above the boundary.

  The “smooth boundary line in the x direction at the frame rate f” is obtained when the magnitude of the motion in the x direction of the video having the spatial frequency μ in the horizontal direction (x direction) is changed at the frame rate f. In addition, it is determined in advance by subjective evaluation for judging whether or not the motion of the video looks jerky.

  In the memory 10501, the Nyquist boundary line in the x direction and the smoothness boundary line of motion at each frame rate are stored in advance as Nyquist boundary line information in the x direction and smoothness boundary line of motion, respectively.

  Similarly, the ν-direction spatial frequency range determination unit 10504 has a y-direction motion magnitude (speed) | MVy | max extracted from the motion feature value 111 input from the motion feature value extraction unit 102, and The ν-direction spatial frequency range information 10508 is acquired using the frame rate value f (112) input from the frame rate acquisition unit 103, and is output to the y-direction motion smoothness value calculation unit 10505. Has been.

  Specifically, the ν-direction spatial frequency range determination unit 10504 generates the y direction at the frame rate f as shown in FIG. 7 based on the “Nyquist boundary information in the y direction” stored in the memory 10501 in advance. The spatial frequency ν1 of the point at the magnitude | MVy | max of the movement in the y direction on the Nyquist boundary line is obtained.

  Also, the ν-direction spatial frequency range determination unit 10504, based on the “motion smoothness boundary information” stored in the memory 10501 in advance, as shown in FIG. On the smoothness boundary line, the spatial frequency ν0 of the point at the magnitude | MVy | max of the movement in the y direction is acquired.

  The ν-direction spatial frequency range determination unit 10504 outputs the acquired ν0 and ν1 to the y-direction motion smoothness value calculation unit 10505 as the ν-direction spatial frequency range information 10508.

  Here, the “Nyquist boundary line in the y direction at the frame rate f” indicates a boundary at which the motion in the y direction of the video can be correctly displayed at the predetermined frame rate f.

  For example, “the Nyquist boundary line in the y direction at the frame rate f” indicates the limit value ν1 of the spatial frequency at which a motion of a video having a predetermined motion feature quantity (| MVy | max) can be correctly displayed at a predetermined frame rate f. .

  That is, the “Nyquist boundary line in the y direction at the frame rate f” is an image having a spatial frequency ν in the vertical direction (y direction) and a motion magnitude in the y direction of | MVy | Is the boundary where the magnitude of the apparent movement in the y direction of the video is maintained at | MVy |.

  Note that in the region above the boundary line, the image may appear to be stationary, appear to move in the opposite direction, or appear to move with a magnitude of movement different from | MVy |.

  The “smooth boundary line in the y direction at the frame rate f” is a video quality boundary line relating to the smoothness of the y direction motion, and the y direction motion of the video is smooth at a predetermined frame rate f. It shows the relationship between the magnitude of the perceived limit movement and the spatial frequency.

  For example, the “smooth boundary line in the y direction at the frame rate f” keeps the quality related to the motion of a video having a predetermined motion feature quantity (| MVy | max) at a predetermined frame rate f above a certain level. Indicates the limit value of the spatial frequency.

  Note that the motion of the video in the y direction appears jerky above the boundary.

  The “smooth boundary line in the y direction at the frame rate f” is obtained when the magnitude of the movement in the y direction of the video having the spatial frequency ν in the vertical direction (y direction) is changed at the frame rate f. In addition, it is determined in advance by subjective evaluation for judging whether or not the motion of the video looks jerky.

  In the memory 10501, the Nyquist boundary line in the y direction and the smoothness boundary line of motion at each frame rate are stored in advance as Nyquist boundary line information in the y direction and smoothness boundary line of motion, respectively.

  The x-direction motion smoothness value calculation unit 10503 outputs the μ component accumulated value extracted from the spatial frequency feature value 113 input from the spatial frequency analysis unit 104 and the μ direction spatial frequency range determination unit 1052. Using the directional spatial frequency range information 10507, the smoothness value Sx (10509) of the motion in the x direction is calculated by adding all the accumulated values of the μ component in the μ directional spatial frequency range between μ0 and μ1. The evaluation value calculation unit 10506 is configured to output the evaluation value.

  Also, the y-direction motion smoothness value calculation unit 10505 is input from the ν component accumulated value extracted from the spatial frequency feature quantity 113 input from the spatial frequency analysis unit 104 and the ν direction spatial frequency range determination unit 10504. By using the ν direction spatial frequency range information 10508 and adding all the ν component accumulated values in the ν direction spatial frequency range between ν0 and ν1, the smoothness value (Sy) 10510 of the motion in the y direction is obtained. It is configured to calculate and output to the evaluation value calculation unit 10506.

  FIG. 8A is a diagram illustrating a region of accumulated values to be added to calculate the smoothness value Sx of the motion in the x direction, and FIG. 8B illustrates the smoothness value Sy of the motion in the y direction. It is a figure which shows the area | region of the cumulative value added in order to calculate.

  The evaluation value calculation unit 10506 includes an x-direction motion smoothness value 10509 input from the x-direction motion smoothness value calculation unit 10503, and a y-direction motion smoothness value calculation unit 10505. The smoothness value 10510 is compared, and the larger one is set as the smoothness value S of the motion in the frame image.

  The evaluation value calculation unit 10506 is configured to compare the motion smoothness value S with a predetermined threshold value and calculate the motion smoothness evaluation value E in the frame image.

  Specifically, the evaluation value calculation unit 1053 evaluates the smoothness of motion when the smoothness value (signal intensity value) of motion in a predetermined frame image is “S” and the predetermined threshold is “T”. The value is configured to calculate “E” by “TS”.

  Further, the evaluation value calculation unit 10506 calculates an average value of the evaluation values E indicating the smoothness of movement in all the frame images, and outputs it as an evaluation value 114 indicating the smoothness of movement.

(Video Evaluation Method According to First Embodiment of the Present Invention)
Hereinafter, the video evaluation method according to the present embodiment will be described with reference to FIGS. 9 to 11.

  First, the overall operation of the video evaluation method according to the present embodiment will be described with reference to FIG.

  As shown in FIG. 9, in step S <b> 901, the input unit 101 reads an externally input video and information regarding the frame rate of the video.

  In step S902, the motion feature amount extraction unit 101 extracts a motion feature amount of the video.

  In step S903, the frame rate acquisition unit 103 acquires the frame rate of the video from the information regarding the input frame rate.

  In step S904, the spatial frequency analysis unit 104 analyzes the spatial frequency feature amount in the frame image constituting the video.

  In step S905, the evaluation unit 105 evaluates the smoothness of the motion of the input video using the motion feature quantity of the video, the spatial frequency feature quantity, and the frame rate of the video.

  Second, the processing method of the spatial frequency analysis unit 104 will be described with reference to FIG.

  As shown in FIG. 10, in step S1001, the two-dimensional Fourier transform unit 1041 calculates the power at each frequency by performing a two-dimensional Fourier transform process on the input video.

  In step S10021, the cumulative value calculation unit 1042 uses the power at each frequency to add all the power in the ν direction at the spatial frequency μ in the horizontal direction, thereby accumulating the μ component that is the cumulative value of the power at the spatial frequency μ. Calculate the value.

  In step S10022, the cumulative value calculation unit 1042 calculates a ν component cumulative value that is a cumulative value of power at the spatial frequency ν by adding all the power in the μ direction at the spatial frequency ν in the vertical direction.

  Here, step S10021 and step S10022 are combined to form step S1002.

  Third, the processing method of the evaluation unit 105 will be described with reference to FIG.

  As illustrated in FIG. 11, in step S11011, the μ-direction spatial frequency range determination unit 10502 stores in advance in the memory 10501 using the motion magnitude | MVx | max of the video in the x direction and the frame rate f. The Nyquist boundary information in the x direction and the smoothness boundary information of the motion are acquired.

  In step S11012, the μ-direction spatial frequency range determination unit 10502, based on the acquired “Nyquist boundary line information”, moves the motion in the x direction on the Nyquist boundary line in the x direction at the frame rate f as shown in FIG. The spatial frequency μ1 of the point at the size | MVx | max is acquired.

  Also, in step S11013, the μ-direction spatial frequency range determination unit 10502, based on the acquired “motion smoothness boundary information”, as shown in FIG. 6, the motion smoothness boundary in the x direction at the frame rate f. On the line, the spatial frequency μ0 of the point at the magnitude | MVx | max of motion in the x direction is obtained.

  In step S11014, the μ direction spatial frequency range determination unit 10502 determines the μ direction spatial frequency range from μ0 to μ1.

  Here, step S11011 to step S11014 are collectively referred to as step S1101.

  In step S11021, the ν-direction spatial frequency range determination unit 10504 uses the y-direction motion magnitude | MVy | max of the video and the frame rate f to store the y-direction Nyquist previously stored in the memory 10501. Boundary line information and motion smoothness boundary line information are acquired.

  In step S11022, the ν-direction spatial frequency range determination unit 10504, based on the acquired “Nyquist boundary line information”, moves the y-direction motion on the y-direction Nyquist boundary line at the frame rate f as shown in FIG. Obtain the spatial frequency ν1 of the point at the magnitude | MVy | max.

  Further, in step S11023, the ν-direction spatial frequency range determination unit 10504, based on the acquired “motion smoothness boundary information”, as shown in FIG. 7, the motion smoothness boundary in the y direction at the frame rate f. On the line, the spatial frequency ν0 of the point at the magnitude | MVy | max of movement in the y direction is obtained.

  In step S11024, the ν direction spatial frequency range determination unit 10504 determines the ν direction spatial frequency range from ν0 to ν1.

  Here, step S1101021 to step S11024 are collectively referred to as step S1102.

  In step S1103, the x-direction motion smoothness value calculation unit 10503 calculates the x-direction motion smoothness value Sx by adding all the μ component accumulated values in the μ-direction spatial frequency range between μ0 and μ1. To do.

  In step S1104, the y-direction motion smoothness value calculation unit 10505 adds all the ν component accumulated values in the ν-direction spatial frequency range between ν0 and ν1, thereby adding the y-direction motion smoothness value Sy. Is calculated.

  In step S11051, the evaluation value calculation unit 10506 compares the smoothness value Sx of the motion in the x direction with the smoothness value Sy of the motion in the y direction.

  When the smoothness value Sx of the motion in the x direction is larger than the smoothness value Sy of the motion in the y direction, in step S11052, the evaluation value calculation unit 10506 determines the smoothness value S of the motion in the frame image as the x direction. The smoothness value Sx of the movement of

  In other cases, in step S11053, the evaluation value calculation unit 10506 sets the motion smoothness value S in the frame image as the motion smoothness value Sy in the y direction.

  In step S11054, the evaluation value calculation unit 10506 calculates an evaluation value E indicating the smoothness of motion in the frame image by “ST” using the predetermined threshold T.

  In step S11055, the evaluation value calculation unit 10506 calculates the average value of the evaluation values E for smoothing the motion in all the frame images).

  Here, steps S11051 to S11055 are collectively referred to as step S1105.

(Video evaluation program according to the first embodiment of the present invention)
With reference to FIG. 12, a video evaluation program 50 for causing a computer to function as the video evaluation apparatus 10 according to the present embodiment described above will be described.

  As shown in FIG. 12, the video evaluation program 50 includes a main module program 1201 that supervises processing, an input module 1202, a motion feature amount extraction module 1203, a frame rate acquisition module 1204, a spatial frequency analysis module 1205, And an evaluation module 1206.

  The functions that the input module 1202, motion feature amount extraction module 1203, frame rate acquisition module 1204, spatial frequency analysis module 1205, and evaluation module 1206 cause the computer to perform are the corresponding input unit 101, motion feature amount extraction unit 102, frame rate, respectively. The functions of the acquisition unit 103, the spatial frequency analysis unit 104, and the evaluation unit 105 are the same.

  Note that the video evaluation program 50 is provided by a storage medium such as a CD-ROM, a DVD, or a ROM, or a semiconductor memory, for example.

  The video evaluation program 40 may be provided via a network as a computer data signal superimposed on a carrier wave.

(Operation / Effect of Image Evaluation Apparatus According to First Embodiment of the Present Invention)
According to the video evaluation apparatus 10 according to the present embodiment, the input unit 101 separates and outputs a video extracted from a video signal input from the outside into information about a plurality of frame images and frame rates, and motion characteristics. The amount extraction unit 102 extracts the motion feature amount of the video based on the input frame image, the frame rate acquisition unit 103 acquires the frame rate from the input frame rate information, and the spatial frequency analysis unit 104 Calculates the spatial frequency feature amount in the input frame image, and the evaluation unit 105 evaluates the smoothness of the motion of the input video from the motion feature amount, the spatial frequency feature amount, and the frame rate. The smoothness of the motion can be evaluated with higher accuracy in consideration of the motion feature amount, the frame rate, and the spatial frequency.

(Modification 1)
Note that the video signal 107 is a signal including information related to the motion of the video and information related to the spatial frequency of the video in addition to the video composed of a plurality of frame images and information related to the frame rate of the video. Also good.

  In such a case, the motion feature amount extraction unit 102 may be configured to calculate a motion feature amount directly from the information regarding the motion of the video input by the input unit 101 and output the motion feature amount to the evaluation unit 105.

  Further, the spatial frequency analysis unit 104 may be configured to calculate a spatial frequency feature amount directly from the information about the spatial frequency of the video input by the input unit 101 and output the spatial frequency feature amount to the evaluation unit 105.

  In addition, the motion feature quantity extraction unit 102 does not necessarily have to be configured to detect motion vectors in units of blocks of 8 pixels × 8 pixels, but detects motion vectors in units of blocks of an arbitrary size. It may be configured to perform.

  Further, the motion feature quantity extraction unit 102 does not necessarily need to be configured to detect motion vectors in units of blocks.

  For example, the motion feature amount extraction unit 102 may be configured to perform a motion search on the entire frame image and detect one motion vector for each frame image.

  Further, the motion feature amount extraction unit 102 may be configured to detect a motion vector for each pixel in the frame image.

  Further, the motion feature quantity extraction unit 102 may be configured to divide the frame image for each object region in the frame image and detect a motion vector for each object region.

  The motion feature quantity extraction unit 102 does not necessarily use block matching as a motion search method, and can use a density gradient method or any other method for detecting the amount of motion displacement of video.

  The motion feature amount output by the motion feature amount extraction unit 102 is not limited to the maximum value of the motion vector in the x direction and the y direction of each block of each frame image, but an average value, an intermediate value, a minimum value Value or mode value.

  Also, the motion feature amount output by the motion feature amount extraction unit 102 may be the size of one motion vector detected in each frame image unit in the x direction and the y direction.

  In addition, the motion feature amount output by the motion feature amount extraction unit 102 is obtained by dividing each frame image for each object region in the frame image, and detecting the magnitude of the motion vector detected for each object region in the x and y directions. It may be a maximum value, an average value, an intermediate value, a minimum value, or a mode value.

  Further, the motion feature amount output by the motion feature amount extraction unit 102 does not necessarily have to be one each in the x direction and the y direction in frame image units, but in the x direction and y in block units or object units. There may be one in each direction.

  In such a case, the spatial frequency feature amount output by the spatial frequency analysis unit 104 may be a spatial frequency feature amount analyzed for each block of each frame image, or each frame image may be an object in the frame image. It may be a spatial frequency feature amount divided into regions and analyzed for each object region.

  Further, in such a case, the smoothness value of the motion in the x direction and the y direction calculated by the evaluation unit 105 may be one for each block or object.

  In this case, the evaluation unit 105 calculates the smoothness value of the maximum motion among the smoothness values of the motion in the x direction and the y direction calculated for each block unit or object unit in a frame image. The smoothness value S of the motion in the image may be used.

  In addition, the motion feature amount output by the feature amount extraction unit 102 does not necessarily need to be one in each of the x direction and the y direction, and may be one in any two orthogonal directions.

  In such a case, the spatial frequency feature amount output by the spatial frequency analysis unit 104 does not necessarily have to be in the μ direction and the ν direction, and any two orthogonal directions in which the motion feature amount is extracted by the feature amount extraction unit 102 It may be a spatial frequency feature value analyzed for.

  In this case, the smoothness value of the motion calculated by the evaluation unit 105 does not necessarily have to be in the x direction and the y direction, and any two orthogonal values obtained by extracting the motion feature amount by the feature amount extraction unit 102. You may calculate about a direction.

  Further, the power at each frequency calculated by the spatial frequency analysis unit 104 is not necessarily calculated using a two-dimensional Fourier image, and any method for calculating the power amount at each frequency of the frame image can be used. .

  Further, the method of acquiring the Nyquist boundary line information and the motion smoothness boundary line information in the x direction and the y direction that are stored in the memory in advance by the evaluation unit 105 may be a method of referring to a lookup table. However, mathematical expressions representing Nyquist boundary line information and motion smoothness boundary line information may be held and derived from the mathematical expressions.

  In addition, the motion smoothness value calculated by the evaluation unit 105 does not necessarily need to be the larger value of the motion smoothness value in the x direction and the motion smoothness value in the y direction. The sum or average value of the smoothness value of the motion in the y direction and the smoothness value of the motion in the y direction, or the smaller one of the smoothness value of the motion in the x direction and the smoothness value of the motion in the y direction. May be.

  The evaluation value E indicating the smoothness of the motion of each frame image calculated by the evaluation unit 105 is not necessarily calculated by “ST” using the smoothness value S of the motion and the predetermined threshold T. May be.

  As the evaluation value E indicating the smoothness of motion in each frame image calculated by the evaluation unit 105, any function that decreases the evaluation value E as the motion smoothness value S increases may be used.

  Also, the evaluation value E indicating the smoothness of the motion of each frame image calculated by the evaluation unit 105 may be determined to be a constant value by comparing the smoothness value S of the motion with a predetermined threshold T and depending on the magnitude relationship. Good.

  The evaluation value output by the evaluation unit 105 is not limited to the average value of the evaluation values E indicating the smoothness of motion in all the frame images included in the input video signal, but is included in the input video signal. It may be a maximum value, an intermediate value, a minimum value, or a mode value of the evaluation value E indicating the smoothness of motion in all frame images.

  Further, the evaluation value output by the evaluation unit 105 does not necessarily have to be one evaluation value through all the frame images included in the input video signal. One evaluation value may be used for each image, each frame image block, or each frame image object.

(Video evaluation apparatus according to the second embodiment of the present invention)
A video evaluation apparatus 20 according to the second embodiment of the present invention will be described with reference to FIGS. Hereinafter, the video evaluation apparatus 20 according to the present embodiment will be described by focusing on differences from the video evaluation apparatus 10 according to the first embodiment described above.

  As shown in FIG. 13, the video evaluation apparatus 20 includes, as its functional components, an input unit 1301, a motion feature amount extraction unit 1302, a frame rate acquisition unit 1303, a spatial frequency range determination unit 1304, and an evaluation. Part 1305.

  The input unit 1301 is configured to decompose a video extracted from the video signal 1306 input from the outside into a plurality of frame images 1307 and output the frame images 1307 to the motion feature amount extraction unit 1302.

  The input unit 1301 is configured to decompose a video extracted from the video signal 1306 input from the outside into a plurality of frame images 1308 and output the frame image 1308 to the evaluation unit 1305.

  Further, the input unit 1301 is configured to output information 1309 related to the frame rate extracted from the video signal 1306 input from the outside to the frame rate acquisition unit 1303.

  The motion feature amount extraction unit 1302 extracts a motion feature amount of the input video based on the frame image 1307 input by the input unit 1301 and outputs the motion feature amount 1310 to the spatial frequency range determination unit 1304 as a motion feature amount 1310. It is configured.

  Specifically, the motion feature amount extraction unit 1302 is configured to extract the motion feature amount of the input video in the same manner as the motion feature amount extraction unit 102 described above.

  The frame rate acquisition unit 1303 is configured to acquire the frame rate 1311 of the input video from the information 1309 related to the frame rate input by the input unit 1301 and output it to the spatial frequency range determination unit 1304.

  Specifically, the frame rate acquisition unit 1303 is configured to acquire the frame rate 1311 of the input video in the same manner as the frame rate acquisition unit 103 described above.

  The spatial frequency range determination unit 1304 uses the motion feature amount 1310 input from the motion feature amount extraction unit 1302 and the frame rate 1311 input from the frame rate acquisition unit 1303 to evaluate the smoothness of video motion. Is determined, and spatial frequency range information 1312 indicating the determined spatial frequency range is output to the evaluation unit 1305.

  With reference to FIG. 14, the method by which the spatial frequency range determination unit 1304 determines the spatial frequency range information 1312 will be specifically described.

  As shown in FIG. 14, the spatial frequency range determination unit 1304 includes a memory 13041, a Nyquist boundary frequency detection unit 13042, a smoothness boundary frequency detection unit 13043, and a spatial frequency range information determination unit 13044. Yes.

  The memory 13041 stores Nyquist boundary information in the x and y directions at each frame rate, and motion smoothness boundary information (video quality boundary regarding motion smoothness) information.

  The Nyquist boundary line information and the motion smoothness boundary line information in the x direction and the y direction at each frame rate are stored in advance in the memory 10501 of the evaluation unit 105 in the first embodiment described above. Same as direction Nyquist boundary information and motion smoothness boundary information.

  The Nyquist boundary frequency detection unit 13042 uses the motion feature amount 1310 input from the motion feature amount extraction unit 1302 and the frame rate 1311 input from the frame rate acquisition unit 1303 to generate a “frame corresponding to the motion feature amount 1310. The spatial frequency on the “Nyquist boundary line in the x direction and the y direction at the rate f” is acquired from the memory 13041 and is output to the spatial frequency range information determination unit 13044 as the Nyquist boundary frequency 13045.

  That is, when the magnitude of motion in the x direction at the frame rate f is | MVx | max, the Nyquist boundary frequency detection unit 13042 is on the “Nyquist boundary line in the x direction at the frame rate f” as shown in FIG. Is obtained from the memory 13041.

  Further, when the magnitude of the movement in the y direction at the frame rate f is | MVy | max, the Nyquist boundary frequency detection unit 13042 is on the “Nyquist boundary line in the y direction at the frame rate f” as shown in FIG. Is obtained from the memory 13041.

  The Nyquist boundary frequency detection unit 13042 is configured to output the spatial frequencies μ1 and ν1 to the spatial frequency range information determination unit 13044 as the Nyquist boundary frequency 13045.

  The motion smoothness boundary frequency detection unit 13043 corresponds to the motion feature amount 1310 using the motion feature amount 1310 input from the motion feature amount extraction unit 1302 and the frame rate 1311 input from the frame rate acquisition unit 1303. The spatial frequency on the “smooth boundary line in the x and y directions at the frame rate f” is acquired from the memory 13041 and is output to the spatial frequency range information determining unit 13044 as the smoothness boundary frequency 13046 of the motion. It is configured.

  That is, when the magnitude of the motion in the x direction at the frame rate f is | MVx | max, the motion smoothness frequency detection unit 13043 indicates “smooth motion in the x direction at the frame rate f as shown in FIG. The spatial frequency μ 0 on the “border line” is obtained from the memory 13041.

  In addition, when the magnitude of the motion in the y direction at the frame rate f is | MVy | max, the smoothness boundary frequency detection unit 13043 of motion, as illustrated in FIG. The spatial frequency ν 0 on the “smooth boundary line” is obtained from the memory 13041.

  The motion smoothness boundary frequency detection unit 13043 is configured to output the spatial frequencies μ0 and ν0 to the spatial frequency range information determination unit 13044 as the motion smoothness boundary frequency 13046.

  The spatial frequency range information determination unit 13044 uses the Nyquist boundary frequency 13045 input from the Nyquist boundary frequency detection unit 13042 and the motion smoothness boundary frequency 13046 input from the motion smoothness boundary frequency detection unit 13043. The spatial frequency range information 1312 is determined and output as the output of the spatial frequency range determination unit 1304.

  That is, as shown in FIG. 6, when the Nyquist boundary line frequency in the x direction is μ1 and the smoothness boundary line frequency in the x direction is μ0, the spatial frequency range information determining unit 13044 The spatial frequency range information is determined as μ0 ≦ μ ≦ μ1.

  Also, as shown in FIG. 7, when the Nyquist boundary line frequency in the y direction is ν1 and the smoothness boundary line frequency in the y direction is ν0, the spatial frequency range information determining unit 13044 The spatial frequency range information is determined as ν0 ≦ ν ≦ ν1.

  The evaluation unit 1305, based on the frame image 1308 input by the input unit 1301 and the spatial frequency range information 1312 input by the spatial frequency range determination unit 1304, the spatial frequency in the frame image constituting the input video. A signal intensity value within the range is calculated, an evaluation value indicating the smoothness of the motion of the video is calculated based on a comparison result between the signal intensity value and a predetermined threshold, and an evaluation value 1314 is output as the output of the evaluation device 20. Is configured to output.

  As illustrated in FIG. 15, the evaluation unit 1305 includes a two-dimensional Fourier transform unit 13051, a signal intensity value calculation unit 13052, and an evaluation value calculation unit 13053.

  A two-dimensional Fourier transform unit 13051 performs a two-dimensional Fourier transform process on the frame image 1308 input by the input unit 1301 to create a two-dimensional Fourier image 13054 including power at each frequency, and a signal intensity value calculation unit It is configured to output to 13052.

  The signal strength value calculation unit 13052 uses the two-dimensional Fourier image 13054 input by the two-dimensional Fourier transform unit 13051 and the spatial frequency range information 1312 input by the spatial frequency range determination unit 1304 to obtain a signal strength value 13055. It is configured to calculate and output to the evaluation value calculation unit 13053.

  The signal strength value calculation method by the signal strength value calculation unit 13052 will be specifically described with reference to FIG.

  FIG. 16A is a diagram for explaining a method for calculating a signal intensity value in the x direction, and FIG. 16B is a diagram for explaining a method for calculating a signal intensity value in the y direction.

  First, the signal intensity value calculation unit 13052 adds all the powers in the ν direction at the spatial frequency μ in the x direction for the power of each frequency component of the two-dimensional Fourier image 13054 to obtain the power at the spatial frequency μ. The μ component cumulative value, which is a cumulative value, is calculated.

  Second, the signal intensity value calculation unit 13052 uses the spatial frequency range information 1312 to accumulate the μ component in the x direction spatial frequency range μ0 ≦ μ ≦ μ1 (broken line hatched portion in FIG. 16A). Is added to calculate the signal intensity value in the x direction.

  Third, the signal intensity value calculation unit 13052 adds the power in the μ direction at the spatial frequency ν in the y direction for the power of each frequency component of the two-dimensional Fourier image 13054 to thereby obtain the power at the spatial frequency ν. Ν component cumulative value that is the cumulative value of is calculated.

  Fourthly, the signal intensity value calculation unit 13052 uses the spatial frequency range information 1312 to accumulate the ν component values in the y direction spatial frequency range ν0 ≦ ν ≦ ν1 (solid hatched portion in FIG. 16B). Is added to calculate the signal intensity value in the y direction.

  Fifth, the signal strength value calculation unit 13052 compares the signal strength in the x direction with the signal strength in the y direction, and outputs the larger value as the signal strength value 13055 to the evaluation value calculation unit 13053.

  The evaluation value calculation unit 13053 is configured to output an evaluation value 1314 as a comparison result between the signal strength value 13055 input from the signal strength value calculation unit 13052 and a predetermined threshold value.

  Specifically, the evaluation value calculation unit 13053 sets the signal strength value in a certain frame image to “S” and the predetermined threshold value to “T”, and the evaluation value E of the smoothness of motion is “TS”. It is configured to calculate.

  Further, the evaluation value calculation unit 13053 is configured to calculate an average value of the evaluation values E indicating the smoothness of motion in all the frame images and output the average value as the evaluation value 1314 of the smoothness of motion.

(Video Evaluation Method According to Second Embodiment of the Present Invention)
Hereinafter, the video evaluation method according to the present embodiment will be described with reference to FIGS. 17 to 19.

  First, the overall operation of the video evaluation method according to the present embodiment will be described with reference to FIG.

  As shown in FIG. 17, in step S1801, the input unit 1301 reads an externally input video and information on the frame rate of the video.

  In step S1802, the motion feature amount extraction unit 1302 extracts the motion feature amount of the video.

  In step S1803, the frame rate acquisition unit 1303 acquires the frame rate of the video based on the input information about the frame rate.

  In step S1804, the spatial frequency range determination unit 1304 determines a spatial frequency range to be evaluated for the smoothness of motion of the video.

  In step S1805, the evaluation unit 1305 evaluates the smoothness of the motion of the video.

  Second, the processing method of the spatial frequency range determination unit 1304 will be described with reference to FIG.

  As shown in FIG. 18, in step S1901, the spatial frequency range determination unit 1304 reads the motion feature quantity and frame rate of the input video.

  In step S1902, the Nyquist boundary frequency detection unit 13042 detects from the memory 13041 the Nyquist boundary frequency corresponding to the motion feature amount at the frame rate.

  In step S1903, the motion smoothness boundary frequency detection unit 13043 detects from the memory 13041 the motion smoothness boundary frequency corresponding to the motion feature amount at the frame rate.

  In step S1904, the spatial frequency range information determination unit 13044 determines the spatial frequency range information indicating the spatial frequency range between the Nyquist boundary frequency and the smoothness boundary frequency of motion.

  Third, the processing method of the evaluation unit 1305 will be described with reference to FIG.

  As shown in FIG. 19, in step S2001, the two-dimensional Fourier transform unit 13051 performs a two-dimensional Fourier transform process on the input frame image.

  In step S20021, the signal intensity value calculation unit 13052 accumulates the power of each frequency component of the two-dimensional Fourier image within the spatial frequency range described above to calculate the accumulated value of the power.

  In step S20022, the signal strength value calculation unit 13052 calculates the signal strength value by adding all the accumulated power values in the spatial frequency range.

  Here, step S20021 and step S20022 are collectively referred to as step S2002.

  In step S20031, the evaluation value calculation unit 13053 calculates an evaluation value in each frame image based on the comparison result between the calculated signal strength value and a predetermined threshold value.

  In step S20032, the evaluation value calculation unit 13053 calculates an average value of evaluation values indicating the smoothness of movement in all frame images, and outputs the average value as an evaluation value indicating the smoothness of movement.

  Here, step S20321 and step S20322 are combined into step S2032.

(Video evaluation program according to the second embodiment of the present invention)
With reference to FIG. 20, a video evaluation program 60 for causing a computer to function as the video evaluation device 20 according to the present embodiment described above will be described.

  As shown in FIG. 20, the video evaluation program 60 includes a main module program 2101 that supervises processing, an input module 2102, a motion feature amount extraction module 2103, a frame rate acquisition module 2104, and a spatial frequency range determination module 2105. And an evaluation module 2106.

  The functions that the input module 2102, the motion feature amount extraction module 2103, the frame rate acquisition module 2104, the spatial frequency range determination module 2105, and the evaluation module 2106 perform to the computer are the corresponding input unit 1301, motion feature amount extraction unit 1302, The functions of the frame rate acquisition unit 1303, the spatial frequency range determination unit 1304, and the evaluation unit 1305 are the same.

  Note that the video evaluation program 60 is provided by a storage medium such as a CD-ROM, a DVD, or a ROM, or a semiconductor memory, for example.

  The video evaluation program 60 may be provided via a network as a computer data signal superimposed on a carrier wave.

(Operation / Effect of Video Evaluation Apparatus According to Second Embodiment of the Present Invention)
According to the video evaluation apparatus 20 according to the present embodiment, the input unit 1301 outputs information about a plurality of frame images and frame rates from a video extracted from a video signal input from the outside, and extracts a motion feature amount. The unit 1302 extracts the motion feature amount of the video based on the input frame image, and the frame rate acquisition unit 1303 acquires the frame rate of the video based on the input frame rate information, and the spatial frequency The range determination unit 1304 determines a spatial frequency range to be evaluated for the smoothness of the motion of the video, the evaluation unit 1305 calculates a signal strength value within the spatial frequency range in the frame image, and the signal strength Since the quality related to the motion of the video is evaluated by comparing the value with a predetermined threshold, the frame level of the video is Depending on the preparative and the video motion features and the spatial frequency information, it is possible to evaluate the smoothness of the higher accuracy of the motion.

  In addition, according to the video evaluation device 20 according to the present embodiment, the spatial frequency range determination unit 1304 maintains the Nyquist boundary line that is a boundary that can correctly display the motion of the video and the quality related to the motion of the video at a certain level or more. The spatial frequency range to be evaluated for the video is determined using the motion smoothness boundary line that is the boundary to be evaluated, so that the evaluation is performed by paying attention to the spatial frequency range related to the smoothness of the motion of the video be able to.

(Modification 2)
Note that the video signal 1306 may be a signal including information related to the motion of the video in addition to the video composed of a plurality of frame images and information related to the frame rate of the video.

  In such a case, the motion feature amount extraction unit 1302 is configured to directly calculate the motion feature amount from the information regarding the motion of the video input by the input unit 1301 and output the motion feature amount to the spatial frequency range determination unit 1304. Also good.

  The motion feature amount extraction unit 1302 does not necessarily have to be configured to detect motion vectors in units of blocks of 8 pixels × 8 pixels, but detects motion vectors in units of blocks of an arbitrary size. It may be configured to perform.

  Further, the motion feature quantity extraction unit 1302 does not necessarily need to be configured to detect a motion vector in units of blocks.

  For example, the motion feature amount extraction unit 1302 may be configured to perform motion search on the entire frame image and detect one motion vector for each frame image.

  The motion feature amount extraction unit 1302 may be configured to detect a motion vector for each pixel in the frame image.

  Further, the motion feature quantity extraction unit 1302 may be configured to divide the frame image for each object region in the frame image and detect a motion vector for each object region.

  In addition, the motion feature amount extraction unit 1302 does not necessarily need to use block matching as a motion search method, and can use a density gradient method or any other method for detecting the amount of motion displacement of video.

  In addition, the motion feature amount output by the motion feature amount extraction unit 1302 is not limited to the maximum value of the magnitude of the motion vector in the x direction and the y direction of each block of each frame image, but an average value, intermediate value, Value or mode value.

  Further, the motion feature amount output by the motion feature amount extraction unit 1302 may be the size of one motion vector detected in each frame image unit in the x direction and the y direction.

  In addition, the motion feature amount output by the motion feature amount extraction unit 1302 is obtained by dividing each frame image for each object region in the frame image, and the magnitude of the motion vector detected for each object region in the x and y directions. It may be a maximum value, an average value, an intermediate value, a minimum value, or a mode value.

  Further, the motion feature amount output by the motion feature amount extraction unit 1302 does not necessarily have to be one in the x direction and y direction in units of frame images, but in the x direction and y in units of blocks or objects. There may be one in each direction.

  In such a case, the spatial frequency range information output by the spatial frequency range determination unit 1304 may be spatial frequency range information determined for each block of each frame image, or each frame image may be included in the frame image. The spatial frequency range information may be divided into object areas and determined for each object area.

  Further, in this case, the signal intensity value in the x direction and the y direction calculated by the evaluation unit 1305 may be one for each block unit or object unit.

  In such a case, the evaluation unit 1305 uses the maximum signal strength value among the signal strength values in the x direction and the y direction calculated for each block unit or object unit in a frame image as the signal strength in the frame image. It may be a value.

  Further, the motion feature amount output by the motion feature amount extraction unit 1302 does not necessarily have to be one in each of the x direction and the y direction, and may be one in any two orthogonal directions. .

In such a case, the spatial frequency range information output by the spatial frequency range determination unit 1304 does not necessarily have to be in the x direction and the y direction, and any orthogonal 2 in which the motion feature amount is extracted by the motion feature amount extraction unit 1302. It may be spatial frequency range information determined for one direction.

  In such a case, the signal intensity values calculated by the evaluation unit 1305 do not necessarily have to be in the x direction and the y direction, and any two orthogonal directions in which the motion feature amount is extracted by the motion feature amount extraction unit 1302 May be calculated.

  The method of acquiring the Nyquist boundary line information and the motion smoothness boundary line information in the x direction and the y direction stored in advance in the memory by the spatial frequency range determination unit 1304 is a method of referring to a lookup table. Alternatively, mathematical expressions representing Nyquist boundary line information and motion smoothness boundary line information may be held and derived from the mathematical expressions.

  In addition, the signal intensity value calculated by the evaluation unit 1305 is not necessarily calculated using a two-dimensional Fourier image, and any method for calculating the electric energy at each frequency of the frame image can be used.

  Further, the signal strength value calculated by the evaluation unit 1305 does not necessarily need to be the larger one of the signal strength value in the x direction and the signal strength value in the y direction, and the signal strength value in the x direction and y It may be the sum or average value with the signal strength value in the direction, or the smaller one of the signal strength value in the x direction and the signal strength value in the y direction.

  Further, the evaluation value E indicating the smoothness of the motion of each frame image calculated by the evaluation unit 1305 does not necessarily have to be calculated by “ST” using the signal intensity value S and the predetermined threshold value T. Good.

  As the evaluation value E indicating the smoothness of motion in each frame image calculated by the evaluation unit 1305, any function that decreases the evaluation value E as the signal strength value S increases may be used.

  Further, the evaluation value E of the smoothness of motion of each frame image calculated by the evaluation unit 1305 may be determined to be a constant value by comparing the signal intensity value S with a predetermined threshold value T and the magnitude relationship thereof.

  The evaluation value output by the evaluation unit 1305 is not limited to the average value of the evaluation values E indicating the smoothness of motion in all frame images included in the input video signal, but is included in the input video signal. It may be a maximum value, an intermediate value, a minimum value, or a mode value of the evaluation value E indicating the smoothness of motion in all frame images.

  Further, the evaluation value output by the evaluation unit 1305 does not necessarily need to be one evaluation value through all the frame images included in the input video signal. One evaluation value may be used for each image, each frame image block, or each frame image object.

(Video frame rate determination device according to the third embodiment of the present invention)
With reference to FIG. 21, a video frame rate determining apparatus 220 according to a third embodiment of the present invention will be described.

  The video frame rate determination device 220 according to the present embodiment may be physically realized by a computer including a CPU (Central Processing Unit), a storage device such as a memory, a communication device, and the like.

  Further, the video frame rate determination device 220 may be realized by a mobile communication terminal such as a mobile phone. That is, a device capable of information processing can be widely applied to the video frame rate determination device 220.

  As shown in FIG. 21, the video frame rate determination device 220 functionally includes a storage unit 2201, a frame rate generation unit 2202, a video signal generation unit 2203, a video signal evaluation unit 2204, and a frame rate determination unit 2205. And.

  Note that the video frame rate determination device 220 is configured to determine the frame rate 2211 of the video 2206 composed of a plurality of frame images 2208.

  The accumulation unit 2201 is configured to acquire and accumulate a video 2206 that is a target for which the frame rate 2211 is to be determined.

  Further, the storage unit 2201 extracts a plurality of frame images 2208 constituting the stored video 2206 according to the frame rate 2207 input from the frame rate generation unit 2202, and outputs the extracted frame images 2208 to the video signal generation unit 2203. It is configured as follows.

  The frame rate generation unit 2202 generates a first frame rate 2207 that is an object for evaluating the smoothness of motion, and outputs the first frame rate 2207 to the storage unit 2201, the video signal generation unit 2203, and the frame rate determination unit 2205. It is configured.

  The video signal generation unit 2203 receives the video signal 2209 including the frame image 2208 input from the storage unit 2201 (video 2206 stored in the storage unit 2201) and the frame rate 2207 input from the frame rate generation unit 2202. It is configured to generate and output to the video signal evaluation unit 2204.

  The video signal evaluation unit 2204 uses the video signal 2209 input from the video signal generation unit 2203 to evaluate the smoothness of the motion of the video included in the video signal 2209, thereby obtaining the evaluation value 2210 described above. It is configured to calculate and output to the frame rate determination unit 2205.

  Note that the video signal evaluation unit 2204 has the same function as that of the video evaluation device 10 according to the first embodiment or the video evaluation device 20 according to the second embodiment.

  The frame rate determination unit 2205 uses the frame rate 2207 input from the frame rate generation unit 2202 and the evaluation value 2210 input from the video signal evaluation unit 2204 to perform a second frame rate for processing the above-described video. The frame rate 2211 is determined and output.

  Specifically, the frame rate determination unit 2205 uses the frame rate 2207 input from the frame rate generation unit 2202 and the relationship between the evaluation value 2210 input from the video signal evaluation unit 2204 and a certain fixed value, The second frame rate 2211 is determined.

  When the evaluation value 2210 input from the video signal evaluation unit 2204 becomes larger than a certain fixed value, the frame rate determining unit 2205 sets the second frame rate 2211 to the first frame rate 2207 and a certain fixed value. The output is reduced in proportion to the difference.

  In addition, when the evaluation value 2210 input from the video signal evaluation unit 2204 is smaller than a certain fixed value, the frame rate determining unit 2205 sets the second frame rate 2211 as the first frame rate 2207 and a certain fixed value. The output is increased in proportion to the difference.

  Further, in the video frame rate determination device 220 according to the present embodiment, the frame rate generation unit 2202 generates various values of the first frame rate 2207, and the evaluation value 2210 calculated by the video signal evaluation unit 2204 is the above-described value. The frame rate determination unit 2205 may output the same frame rate as the first frame rate 2207 as the second frame rate 2211.

  Note that the certain value described above used for comparison in the frame rate determining unit 2205 may be determined in advance or may be given from the outside.

  Further, the first frame rate 2207 generated by the frame rate generator 2202 may be determined in advance or may be given from the outside.

  The frame rate generation unit 2202 may be configured to select and generate one or more frame rates from a plurality of predetermined frame rates.

(Video frame rate determination method according to the third embodiment of the present invention)
With reference to FIG. 22, the video frame rate determination method according to the present embodiment will be described.

  As shown in FIG. 22, in step S2301, the frame rate generator 2102 generates a first frame rate.

  In step S2302, the accumulation unit 2201 extracts the accumulated frame images at the first frame rate.

  In step S2303, the video signal generation unit 2203 generates a video using a plurality of frame images extracted from the storage unit 2201, and includes the generated video and a first frame rate for evaluating the smoothness of the video. Generate a video signal.

  In step S2304, the video signal evaluation unit 2204 evaluates the smoothness of the motion of the video included in the video signal.

  In step S2305, the frame rate determination unit 2205 performs processing on the video using the first frame rate input from the frame rate generation unit 2202 and the evaluation value E input from the video signal evaluation unit 2204. A second frame rate is determined.

(Video frame rate determination program according to the third embodiment of the present invention)
With reference to FIG. 23, a video frame rate determination program 240 for causing a computer to function as the video frame rate determination device 220 according to the third embodiment will be described.

  As shown in FIG. 24, the video frame rate determination program 240 includes a main module program 2401 that supervises processing, a storage module 2402, a frame rate generation module 2403, a video signal generation module 2404, and a video signal evaluation module 2405. A frame rate determination module 2406.

  The functions that the storage module 2402, the frame rate generation module 2403, the video signal generation module 2404, the video signal evaluation module 2405, and the frame rate determination module 2406 perform in the computer are the corresponding storage unit 2201, frame rate generation unit 2202, and video, respectively. The functions are the same as those of the signal generation unit 2203, the video signal evaluation unit 2204, and the frame rate determination unit 2205.

  The video frame rate determination program 240 is provided by a storage medium such as a CD-ROM, a DVD, or a ROM, or a semiconductor memory, for example.

  The video frame rate determination program 240 may be provided via a network as a computer data signal superimposed on a carrier wave.

(Operation / Effect of Video Frame Rate Determination Device According to Third Embodiment of the Present Invention)
According to the video frame rate determination device 22 according to the present embodiment, the video signal evaluation unit 2204 evaluates the smoothness of the video at the first frame rate generated by the frame rate generation unit 2202, and the second frame rate. In order to determine 2211, the frame rate of the video can be determined using an evaluation value indicating the smoothness of the motion of the video so that the smoothness of the motion of the video has a certain value (evaluation value). it can.

It is a functional block diagram which shows the structure of the image | video evaluation apparatus which concerns on the 1st Embodiment of this invention. It is a figure showing the mode of search of the motion vector by block matching. It is a functional block diagram which shows the structure of the spatial frequency analysis part of the image | video evaluation apparatus which concerns on the 1st Embodiment of this invention. (A) is a figure which shows an example of the frame image input into the spatial frequency analysis part of the image | video evaluation apparatus which concerns on the 1st Embodiment of this invention, (b) is the 1st Embodiment of this invention. It is a figure which shows an example of the two-dimensional Fourier-transform image obtained by carrying out the two-dimensional Fourier transform of the frame image input into the spatial frequency analysis part of the image | video evaluation apparatus which concerns on (c), (c) is the 1st of this invention FIG. 7 is a diagram illustrating a method of calculating a μ component cumulative value that is a cumulative value of power of a two-dimensional Fourier transform image at a spatial frequency μ in the horizontal direction (x direction) in the spatial frequency analysis unit of the video evaluation device according to the embodiment. And (d) is a cumulative value of the power of the two-dimensional Fourier transform image at the spatial frequency ν in the vertical direction (y direction) in the spatial frequency analysis unit of the video evaluation device according to the first embodiment of the present invention. ν component cumulative value It is a diagram showing a method of calculating. It is a functional block diagram which shows the structure of the evaluation part of the image | video evaluation apparatus which concerns on the 1st Embodiment of this invention. FIG. 5 is a diagram illustrating a method for acquiring μ-direction spatial frequency range information from Nyquist boundary information in the x direction and smoothness boundary information of motion in the evaluation unit of the video evaluation device according to the first embodiment of the present invention. . It is a figure showing the method to acquire (nu) direction spatial frequency range information from the Nyquist boundary line information of y direction, and the smoothness boundary information of a motion in the evaluation part of the image | video evaluation apparatus which concerns on the 1st Embodiment of this invention. . (A) is a figure which shows the area | region of the cumulative value added in order to calculate the smoothness value Sx of the motion of a x direction in the evaluation part of the image | video evaluation apparatus which concerns on the 1st Embodiment of this invention, ( b) is a diagram showing a cumulative value region to be added in order to calculate the smoothness value Sy of the movement in the y direction in the evaluation unit of the video evaluation device according to the first embodiment of the present invention; It is a flowchart explaining the image | video evaluation method which concerns on the 1st Embodiment of this invention. It is a flowchart explaining the processing method of the spatial frequency analysis part in the image | video evaluation method which concerns on the 1st Embodiment of this invention. It is a flowchart explaining the processing method of the evaluation part in the image | video evaluation method which concerns on the 1st Embodiment of this invention. It is a figure which shows the image | video evaluation program which concerns on the 1st Embodiment of this invention. It is a functional block diagram which shows the structure of the image | video evaluation apparatus which concerns on the 2nd Embodiment of this invention. It is a functional block diagram which shows the structure of the spatial frequency range determination part of the image | video evaluation apparatus which concerns on the 2nd Embodiment of this invention. It is a functional block diagram which shows the structure of the evaluation part of the image | video evaluation apparatus which concerns on the 2nd Embodiment of this invention. (A) is a figure which shows the area | region of the accumulation value added in order to calculate the signal strength value of a x direction in the evaluation part of the image | video evaluation apparatus which concerns on the 2nd Embodiment of this invention, (b). FIG. 10 is a diagram illustrating a region of cumulative values to be added to calculate a signal intensity value in the y direction in the evaluation unit of the video evaluation device according to the second embodiment of the present invention. It is a flowchart explaining the image | video evaluation method which concerns on the 2nd Embodiment of this invention. It is a flowchart explaining the processing method of the spatial frequency range determination part in the image | video evaluation method which concerns on the 2nd Embodiment of this invention. It is a flowchart explaining the processing method of the evaluation part in the image | video evaluation method which concerns on the 2nd Embodiment of this invention. It is a figure which shows the image | video evaluation program which concerns on the 2nd Embodiment of this invention. It is a functional block diagram which shows the structure of the video frame rate determination apparatus which concerns on the 3rd Embodiment of this invention. It is a flowchart explaining the video frame rate determination method which concerns on the 3rd Embodiment of this invention. It is a figure which shows the video frame rate determination program which concerns on the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10, 20 ... Video evaluation apparatus 101, 1301 ... Input part 102, 1302 ... Motion feature-value extraction part 103, 1303 ... Frame rate acquisition part 104 ... Spatial frequency analysis part 1041, 13051 ... Two-dimensional Fourier transform part 1042 ... Cumulative value calculation Unit 1304 ... spatial frequency range determination unit 13042 ... Nyquist boundary frequency detection unit 13043 ... smoothness boundary frequency detection unit 13044 ... frequency range information determination unit 105, 1305 ... evaluation unit 10501, 13041 ... memory 10502 ... μ-direction spatial frequency range Determination unit 10503 ... x-direction motion smoothness value calculation unit 10504 ... v-direction spatial frequency range determination unit 10505 ... y-direction motion smoothness value calculation units 10506, 13053 ... evaluation value calculation unit 13052 ... signal intensity value calculation unit 220 ... Video frame rate determination device 2201... Storage unit 2202 ... Frame rate generation unit 2203 ... Video signal generation unit 2204 ... Video signal evaluation unit 2205 ... Frame rate determination unit 50, 60 ... Video evaluation program 1201, 2101, 2401 ... Main modules 1202, 2102, ... Input modules 1203, 2103 ... motion feature quantity extraction modules 1204, 2104 ... frame rate acquisition module 1205 ... spatial frequency analysis modules 1206, 2106 ... evaluation module 2105 ... spatial frequency range determination module 240 ... video frame rate determination program 2402 ... storage module 2403 ... frame rate generation module 2404 ... Video signal generation module 2405 ... Video signal evaluation module 2406 ... Frame rate determination module

Claims (10)

A video evaluation device for evaluating the smoothness of movement of a video composed of a plurality of frame images,
A motion feature amount extraction unit that extracts a motion feature amount of the video based on the input frame image constituting the video;
A frame rate acquisition unit for acquiring a frame rate of the input video;
A spatial frequency analysis unit for analyzing a spatial frequency in a frame image constituting the input video;
An evaluation unit that calculates an evaluation value indicating the smoothness of the motion of the video using the extracted motion feature amount, the acquired frame rate, and the analysis result of the spatial frequency. Characteristic video evaluation device. A video evaluation device for evaluating the smoothness of movement of a video composed of a plurality of frame images,
A motion feature amount extraction unit that extracts a motion feature amount of the video based on the input frame image constituting the video;
A frame rate acquisition unit for acquiring a frame rate of the input video;
A spatial frequency range determining unit that determines a spatial frequency range to be evaluated for the video, using the extracted motion feature amount and the acquired frame rate;
An evaluation unit that calculates an evaluation value indicating the smoothness of the movement of the video based on a comparison result between a signal intensity value within the spatial frequency range in the frame image constituting the input video and a predetermined threshold value; A video evaluation apparatus characterized by:   The spatial frequency determination unit has a Nyquist boundary line indicating a limit value of a spatial frequency capable of correctly displaying a motion of a video having a predetermined motion feature amount at a predetermined frame rate, and a predetermined motion feature amount at a predetermined frame rate. The video evaluation apparatus according to claim 2, wherein the spatial frequency range is determined using a video quality boundary line indicating a limit value of a spatial frequency at which the quality related to the motion of the video is maintained at a certain level or more. A video frame rate determination device comprising the video evaluation device according to any one of claims 1 to 3, wherein the video frame rate determination device determines a frame rate of a video composed of a plurality of frame images.
A storage unit for storing the video;
A frame rate generator for generating a first frame rate of the video;
A video signal generation unit that generates a video signal including the video stored in the storage unit and the first frame rate generated by the frame rate generation unit;
A video frame rate determination device comprising: a frame rate determination unit that determines a second frame rate of the video using the evaluation value calculated by the video evaluation device. A video evaluation method for evaluating the smoothness of movement of a video composed of a plurality of frame images,
Extracting a motion feature amount of the video based on the input frame image constituting the video;
Obtaining a frame rate of the input video;
Analyzing a spatial frequency in a frame image constituting the inputted video;
Using the extracted motion feature quantity, the acquired frame rate, and the spatial frequency analysis result to calculate an evaluation value indicating the smoothness of the motion of the video, Video evaluation method. A video evaluation method for evaluating the smoothness of movement of a video composed of a plurality of frame images,
Extracting a motion feature amount of the video based on the input frame image constituting the video;
Obtaining a frame rate of the input video;
Determining a spatial frequency range to be evaluated for the video, using the extracted motion feature amount and the acquired frame rate;
Calculating an evaluation value indicating the smoothness of the motion of the video based on a comparison result between a signal intensity value within the spatial frequency range in the frame image constituting the video and the predetermined threshold value. A video evaluation method characterized by   In the step of determining the spatial frequency range, a Nyquist boundary line indicating a limit value of a spatial frequency capable of correctly displaying motion of a video having a predetermined motion feature amount at a predetermined frame rate, and a predetermined motion feature at a predetermined frame rate 7. The video according to claim 6, wherein the spatial frequency range is determined using a video quality boundary line indicating a limit value of a spatial frequency at which quality related to motion of the video having a quantity is maintained at a certain level or more. Evaluation methods. A video frame rate determination method for determining a frame rate of a video composed of a plurality of frame images, comprising the video evaluation method according to any one of claims 5 to 7.
Generating a first frame rate of the video;
Generating a video signal including the video stored in the storage unit and the first frame rate generated in the step of generating the frame rate;
And a step of determining a second frame rate of the video using the evaluation value calculated by the video evaluation method. Computer
A video evaluation program that causes the video evaluation apparatus according to any one of claims 1 to 3 to function. Computer
5. A video frame rate determination program that functions as the video frame rate determination device according to claim 4.

Images