JP5101673B2 - Video evaluation device, frame rate determination device, video processing device, video evaluation method, and video evaluation program - Google Patents

Description

  The present invention relates to a video evaluation device, a frame rate determination device, a video processing device, a video evaluation method, and a video evaluation program.

  Video processing including video acquisition, storage, transmission, display, encoding, decoding, and the like is generally performed at a fixed frame rate. Here, the frame rate refers to the number of frames processed per second. The fixed frame rate means a frame rate in which the number of frames processed per second is constant. A specific example of the fixed frame rate is defined as 29.97 fps (frame per second) in the National Television Standards Committee (NTSC) standard adopted in the United States and Japan, for example. Further, it is defined as 25 fps in the Phase Alternating Line (PAL) standard by the National Television Standards Committee adopted in Europe. Furthermore, a fixed frame rate of 15 fps or 24 fps may be used. Note that the “video” is configured by consecutive “frame images” that are individual still images.

  When video processing is performed at a fixed frame rate, increasing the frame rate shortens the time interval between successive frames. This makes it possible to process a video with smoother motion. For example, an image processed at a frame rate of 30 fps has a greater number of frames per unit time than an image processed at a frame rate of 15 fps, so that the movement of the image is expressed more finely, and the movement is smoother as a whole. It is expressed by

  In addition to the above-described video processing at a fixed frame rate, video processing at a variable frame rate is also performed. The video processing with the variable frame rate is to change the frame rate according to the video processing amount and data amount. For example, when encoding video, if it is determined that the amount of data to be encoded is large, the frame rate is decreased to reduce the number of frames to be encoded per unit time. This is because the time required for video processing increases as the amount of data increases. Here, when the time interval between successive frame images is T at a variable frame rate, the frame rate between the two frame images is 1 / T.

  A technique for changing such a frame rate is disclosed in Patent Document 1, for example.

Japanese Patent Laid-Open No. 11-112940

  By the way, when processing a video at a fixed frame rate, if the frame rate is increased in order to realize smooth motion, the processing amount, data amount and power consumption accompanying the video processing increase. More specifically, for example, when acquiring a video, the number of frames to be acquired per unit time increases, so that the processing amount and the power consumption accompanying the processing increase. In addition, when storing video, the amount of data increases due to an increase in the number of frames to be stored per unit time.

  On the other hand, if the frame rate is reduced to reduce the amount of processing, data, and power consumption associated with video processing, the smoothness of the video motion will be reduced, resulting in a jerky motion video. .

  Also, when processing video at a variable frame rate, if the frame rate is changed only according to the amount of video processing or the amount of data, the smoothness of the video motion will be reduced, resulting in a jerky motion video. End up.

  In this way, if the frame rate is changed without considering the motion characteristics of the video, an unnecessarily large amount of processing, data, and power consumption will be required, and the video will be jerky. There is a problem that it ends up.

  Therefore, in order to solve the above-described problems, the present invention provides a video evaluation apparatus, a frame rate determination apparatus, a video processing apparatus, a video evaluation method, and a frame rate determination method that determine a frame rate according to the smoothness of video motion. An object is to provide a video processing method, a video evaluation program, a frame rate determination program, and a video processing program.

  The video evaluation apparatus according to the present invention includes a change amount detecting means for detecting a change amount indicating a degree of change between each frame image based on a plurality of frame images included in the input video signal, and the change amount and between each frame image. Evaluation value calculation means for calculating an evaluation value related to the smoothness of the motion of the input video signal based on the time interval and outputting the evaluation value to the outside.

  Further, the video evaluation method of the present invention includes a change amount detection step for detecting a change amount indicating a degree of change between each frame image based on a plurality of frame images included in the input video signal, the change amount and each frame. And an evaluation value calculation step of calculating an evaluation value related to the smoothness of the motion of the input video signal based on the time interval between the images and outputting the evaluation value to the outside.

  Furthermore, the video evaluation program of the present invention includes a change amount detecting means for detecting a change amount indicating a degree of change between each frame image based on a plurality of frame images included in the input video signal, and a change amount. And an evaluation value calculation means for calculating an evaluation value related to the smoothness of the motion of the input video signal based on the time interval between the frame images and outputting the evaluation value to the outside.

  According to these inventions, the amount of change is detected based on a plurality of frame images included in the input video signal, and based on the amount of change and the time interval between each frame image of the input video signal, the movement of the input video signal is detected. An evaluation value for evaluating the smoothness of the image is calculated. Therefore, it is possible to evaluate the smoothness of the motion of the video at the frame rate at the time of video processing according to the amount of change of the input video signal input according to the predetermined frame rate. In addition, since the evaluation value obtained by this evaluation is output to the outside, the frame rate can be determined based on this evaluation value. That is, the frame rate can be determined according to the smoothness of the motion of the video.

  In the video evaluation apparatus of the present invention, the evaluation value calculation means calculates a time change amount according to the change amount and the time interval between the frame images based on the change amount and the time interval between the frame images, It is preferable to calculate the evaluation value using the amount of change. In this way, it is possible to calculate an evaluation value in consideration of the amount of change and the amount of time change calculated according to the time interval between the frame images.

  In the video evaluation apparatus of the present invention, it is preferable that the change amount is a value based on a difference in luminance value between the frame images or a motion vector between the frame images. In this way, the evaluation value can be calculated based on the difference in luminance value between the frame images or the motion vector between the frame images.

  In the video evaluation apparatus of the present invention, the video evaluation apparatus further includes a feature value calculation unit that calculates a motion feature value indicating a motion feature of the input video signal based on the amount of change, and the evaluation value calculation unit includes the motion feature value and each of the motion feature values. It is preferable to calculate the evaluation value based on the time interval between the frame images. In this way, it is possible to calculate an evaluation value in consideration of the motion feature value indicating the motion feature of the input video signal.

  In the video evaluation device of the present invention, the evaluation value calculation means calculates a change amount and a time change amount according to the time interval between the frame images based on the motion feature value and the time interval between the frame images, It is preferable to calculate the evaluation value using the time change amount. In this way, it is possible to calculate an evaluation value that takes into account the amount of change and the amount of time change according to the time interval between the frame images.

  In the video evaluation device of the present invention, it is preferable that the amount of change is a motion vector between the frame images, and the motion feature value is a value calculated based on the magnitude of the motion vector. In this way, the evaluation value can be calculated based on the magnitude of the motion vector between the frames.

  The frame rate determining apparatus according to the present invention includes a frame rate generating means for generating a first frame rate, and a change amount indicating a degree of change between the respective frame images based on a plurality of frame images included in the input video signal. A change amount detecting means for detecting, an evaluation value calculating means for calculating an evaluation value relating to the smoothness of the motion of the input video signal based on the time interval between the frame images corresponding to the change amount and the first frame rate; Frame rate determining means for determining a second frame rate for processing the input video signal using the value and outputting the second frame rate to the outside.

  The frame rate determination method of the present invention also includes a frame rate generation step for generating a first frame rate and a change indicating the degree of change between the frame images based on a plurality of frame images included in the input video signal. A change amount detecting step for detecting the amount; an evaluation value calculating step for calculating an evaluation value related to the smoothness of the motion of the input video signal based on the time interval between the frame images corresponding to the change amount and the first frame rate; And a frame rate determination step of determining a second frame rate for processing the input video signal using the evaluation value and outputting the same to the outside.

  Furthermore, the frame rate determination program according to the present invention causes a computer to perform a degree of change between frame images based on frame rate generation means for generating a first frame rate and a plurality of frame images included in an input video signal. An evaluation value for calculating an evaluation value related to the smoothness of the motion of the input video signal based on a change amount detecting means for detecting a change amount indicating the amount of change and a time interval between the frame images corresponding to the change amount and the first frame rate The calculation means and the evaluation value are used to determine a second frame rate for processing the input video signal and to function as a frame rate determination means for outputting to the outside.

  According to these inventions, the change amount is detected based on a plurality of frame images included in the input video signal, and the input video is based on the change amount and the time interval between the frame images corresponding to the first frame rate. An evaluation value for evaluating the smoothness of the movement of the signal is calculated, and a second frame rate for processing the input video signal is determined using this evaluation value. That is, the smoothness of the motion of the video at the first frame rate is evaluated according to the change amount of the input video signal input according to the first frame rate, and the second frame rate is evaluated using this evaluation. Is determined. Therefore, the input video signal can be read at the second frame rate determined in accordance with the evaluation of the smoothness of the video motion. That is, it is possible to determine the frame rate of the input video signal according to the smoothness of the video motion and to read the input video signal while maintaining the smoothness of the video motion.

  In the frame rate determining apparatus of the present invention, the frame rate determining means compares the evaluation value with a predetermined specified value, and if the evaluation value is larger than the specified value, the second frame rate is set to the first frame rate. When the evaluation value is smaller than the specified value, the second frame rate is preferably larger than the first frame rate. In this way, it is possible to determine the second frame rate that is considered so that the evaluation value for evaluating the smoothness of the motion of the video falls within the predetermined evaluation standard. In other words, the input video signal can be read while maintaining the smoothness of the video motion within a predetermined reference range.

  The video processing apparatus according to the present invention includes a buffer unit that stores an input video signal, a frame rate generation unit that generates a first frame rate, and a plurality of frame images included in the input video signal. And an evaluation value relating to the smoothness of the motion of the input video signal based on a change amount detecting means for detecting a change amount indicating the degree of change in the frame and a time interval between the frame images corresponding to the change amount and the first frame rate. An evaluation value calculation means for calculating, a frame rate determination means for determining a second frame rate for processing the input video signal using the evaluation value, and a buffer means using the second frame rate. Video processing means for reading the input video signal and performing video processing.

  The video processing method of the present invention also includes a buffer step for storing an input video signal, a frame rate generation step for generating a first frame rate, and each frame based on a plurality of frame images included in the input video signal. A change amount detecting step for detecting a change amount indicating a degree of change between images, and an evaluation on the smoothness of the motion of the input video signal based on a time interval between frame images corresponding to the change amount and the first frame rate. An evaluation value calculation step for calculating a value, a frame rate determination step for determining a second frame rate for processing the input video signal using the evaluation value, and a buffer means using the second frame rate A video processing step of reading the input video signal and performing video processing.

  Further, the video processing program of the present invention is based on a plurality of frame images included in the input video signal, buffer means for storing the input video signal, frame rate generation means for generating the first frame rate, and the computer. A change amount detecting means for detecting a change amount indicating a degree of change between the frame images, and a smooth movement of the input video signal based on a time interval between the frame images corresponding to the change amount and the first frame rate. An evaluation value calculating means for calculating an evaluation value relating to the length, a frame rate determining means for determining a second frame rate for processing the input video signal using the evaluation value, and a buffer using the second frame rate A video processing unit that reads the input video signal stored in the processing unit and performs video processing; .

  According to these inventions, the change amount is detected based on a plurality of frame images included in the input video signal, and the input video is based on the change amount and the time interval between the frame images corresponding to the first frame rate. An evaluation value for evaluating the smoothness of the movement of the signal is calculated, and a second frame rate for processing the input video signal is determined using this evaluation value. Also, video processing of the input video signal is performed using the second frame rate. That is, the smoothness of the motion of the video at the first frame rate is evaluated according to the change amount of the input video signal input according to the first frame rate, and the second frame rate is evaluated using this evaluation. Is determined. Also, video processing of the input video signal is performed using the second frame rate determined based on the evaluation value for the input video signal. Therefore, the video processing of the input video signal can be performed based on the second frame rate determined according to the evaluation of the smoothness of movement. That is, it is possible to determine the frame rate according to the smoothness of the video motion and to perform video processing of the input video signal while maintaining the smoothness of the video motion.

  [Other embodiments]

  The video evaluation apparatus according to the present invention includes a displacement amount detection means for detecting a motion displacement amount indicating a degree of displacement of a motion of an input video signal based on a plurality of frame images included in the input video signal, the motion displacement amount and the input Evaluation value calculation means for calculating and outputting an evaluation value for evaluating the smoothness of the motion of the input video signal based on the frame rate of the video signal.

  The video evaluation method of the present invention includes a motion displacement amount detection step for detecting a motion displacement amount indicating a degree of motion displacement of an input video signal based on a plurality of frame images included in the input video signal, and a motion displacement An evaluation value calculating step of calculating and outputting an evaluation value for evaluating the smoothness of the motion of the input video signal based on the amount and the frame rate of the input video signal.

  Furthermore, the video evaluation program of the present invention includes a displacement amount detection means for detecting a motion displacement amount indicating a degree of displacement of the motion of the input video signal based on a plurality of frame images included in the input video signal. Based on the amount of motion displacement and the frame rate of the input video signal, an evaluation value for evaluating the smoothness of the motion of the input video signal is calculated and functioned as an evaluation value calculation means for outputting to the outside.

  According to these inventions, the motion displacement amount is detected based on a plurality of frame images included in the input video signal, and the smoothness of the motion of the input video signal is based on the motion displacement amount and the frame rate of the input video signal. An evaluation value for evaluating is calculated. Therefore, the smoothness of the motion of the video at the frame rate at the time of video processing can be evaluated according to the amount of displacement of the motion of the input video signal input according to the predetermined frame rate. In addition, since the evaluation value obtained by this evaluation is output to the outside, the frame rate can be determined based on this evaluation value. That is, the frame rate can be determined according to the smoothness of the motion of the video.

  In the video evaluation apparatus of the present invention, the video evaluation device further includes a feature value calculation unit that calculates a motion feature value indicating a motion feature of the input video signal based on the motion displacement amount, and the evaluation value calculation unit includes the motion feature value and It is preferable to calculate the evaluation value based on the frame rate of the input video signal. In this way, it is possible to calculate an evaluation value in consideration of the motion feature value indicating the motion feature of the input video signal.

  In the video evaluation device of the present invention, the evaluation value calculation means calculates a motion amount indicating a degree of motion between the frames based on the motion displacement amount and the frame rate of the input video signal, and evaluates using the motion amount. It is preferable to calculate the value. Further, the evaluation value calculation means may calculate a motion amount indicating a degree of motion between the frames based on the motion feature value and the frame rate of the input video signal, and calculate the evaluation value using the motion amount. preferable. In this way, it is possible to calculate an evaluation value in consideration of the amount of motion indicating the degree of motion between the frames.

  In the video evaluation device of the present invention, it is preferable that the motion displacement amount is a motion vector, and the motion feature value is a value calculated based on the magnitude of the motion vector. In this way, the evaluation value can be calculated based on the magnitude of the motion vector between the frames.

  The frame rate determining apparatus according to the present invention includes a frame rate generating means for generating a first frame rate and a motion displacement indicating a degree of motion displacement of the input video signal based on a plurality of frame images included in the input video signal. A displacement amount detecting means for detecting the amount; an evaluation value calculating means for calculating an evaluation value for evaluating the smoothness of the motion of the input video signal based on the motion displacement amount and the first frame rate; And a frame rate determining means for determining a second frame rate for processing the input video signal and outputting the same to the outside.

  According to the present invention, the motion displacement amount is detected based on a plurality of frame images included in the input video signal, and the smoothness of the motion of the input video signal is evaluated based on the motion displacement amount and the first frame rate. An evaluation value is calculated, and a second frame rate for processing the input video signal is determined using the evaluation value. That is, the smoothness of the motion of the video at the first frame rate is evaluated according to the amount of displacement of the motion of the input video signal input according to the first frame rate. A frame rate is determined. Therefore, the input video signal can be read at the second frame rate determined in accordance with the evaluation of the smoothness of the video motion. That is, it is possible to determine the frame rate of the input video signal according to the smoothness of the video motion and to read the input video signal while maintaining the smoothness of the video motion.

  In the frame rate determining apparatus of the present invention, the frame rate determining means compares the evaluation value with a predetermined specified value, and if the evaluation value is larger than the specified value, the second frame rate is set to the first frame rate. When the evaluation value is smaller than the specified value, the second frame rate is preferably larger than the first frame rate. In this way, it is possible to determine the second frame rate that is considered so that the evaluation value for evaluating the smoothness of the motion of the video falls within the predetermined evaluation standard. In other words, the input video signal can be read while maintaining the smoothness of the video motion within a predetermined reference range.

  The video processing apparatus of the present invention includes a buffer unit that stores an input video signal, a frame rate generation unit that generates a first frame rate, and a plurality of frame images included in the input video signal. An evaluation value for evaluating the smoothness of the motion of the input video signal is calculated based on the displacement amount detecting means for detecting the motion displacement amount indicating the degree of motion displacement, and the motion displacement amount and the first frame rate. Evaluation value calculation means, frame rate determination means for determining a second frame rate for processing the input video signal using the evaluation value, and input stored by the buffer means using the second frame rate Video processing means for reading video signals and performing video processing.

  According to the present invention, the motion displacement amount is detected based on a plurality of frame images included in the input video signal, and the smoothness of the motion of the input video signal is evaluated based on the motion displacement amount and the first frame rate. An evaluation value is calculated, and a second frame rate for processing the input video signal is determined using the evaluation value. Also, video processing of the input video signal is performed using the second frame rate. That is, the smoothness of the motion of the video at the first frame rate is evaluated according to the amount of displacement of the motion of the input video signal input according to the first frame rate. A frame rate is determined. Also, video processing of the input video signal is performed using the second frame rate determined based on the evaluation value for the input video signal. Therefore, the video processing of the input video signal can be performed based on the second frame rate determined according to the evaluation of the smoothness of movement. That is, it is possible to determine the frame rate according to the smoothness of the video motion and to perform video processing of the input video signal while maintaining the smoothness of the video motion.

  According to the video evaluation device, the frame rate determination device, the video processing device, the video evaluation method, the frame rate determination method, the video processing method, the video evaluation program, the frame rate determination program, and the video processing program according to the present invention Since the frame rate can be determined in accordance with the smoothness of the image, it is possible to provide a smooth motion image while reducing the processing amount, the data amount, and the power consumption.

It is a figure which shows the function structure of the image | video evaluation apparatus in the modification of 1st Embodiment. It is a figure for demonstrating the method to detect displacement amount. (A) shows the frame image P0, and (b) shows the frame image P1. It is a figure for demonstrating the method of evaluating the smoothness of a motion. (A) is a figure for demonstrating the method of calculating the amount of motion between frames, (b) is a figure for demonstrating the method of calculating an evaluation value based on the amount of motion between frames. It is a flowchart which shows the flow of the image | video evaluation process in the modification of 1st Embodiment. It is a figure for demonstrating the range calculated | required based on the magnitude | size and direction of each motion vector. It is a figure which illustrates the module structure of the video evaluation program in the modification of 1st Embodiment. It is a figure which shows the function structure of the frame rate determination apparatus in 2nd Embodiment as an example. (A) illustrates the first frame rate, and (b) illustrates the sampling rate of the input video signal. It is a flowchart which shows the flow of the frame rate determination process in 2nd Embodiment. It is a figure which illustrates the module structure of the frame rate determination program in 2nd Embodiment. It is a figure which shows the function structure of the video processing apparatus in 3rd Embodiment. It is a flowchart which shows the flow of the video processing in 3rd Embodiment. It is a figure which illustrates the module structure of the video processing program in 3rd Embodiment. It is a figure which shows the function structure of the image | video evaluation apparatus in 1st Embodiment. It is a figure for demonstrating the method to detect the variation | change_quantity. It is a figure for demonstrating the method of evaluating the smoothness of a motion. (A) is a figure for demonstrating the method of calculating a time variation | change_quantity, (b) is a figure for demonstrating the method of calculating an evaluation value based on a time variation | change_quantity. It is a flowchart which shows the flow of the image | video evaluation process in 1st Embodiment. It is a figure which illustrates the module structure of the image | video evaluation program in 1st Embodiment.

  Embodiments of a video evaluation device, a frame rate determination device, a video processing device, a video evaluation method, a frame rate determination method, a video processing method, a video evaluation program, a frame rate determination program, and a video processing program according to the present invention will be described below. This will be described with reference to the drawings. In addition, in each figure, the same code | symbol is attached | subjected to the same element and the overlapping description is abbreviate | omitted.

  [First embodiment]

  First, a first embodiment of the present invention will be described. FIG. 14 is a diagram illustrating a functional configuration of the video evaluation apparatus 140 according to the first embodiment.

  Here, the video evaluation device 140 is physically a computer including a CPU (Central Processing Unit), a storage device such as a memory, a communication device, and the like. Therefore, the video evaluation apparatus 140 may be a fixed communication terminal such as a PC terminal or a mobile communication terminal such as a mobile phone. That is, as the video evaluation device 140, devices capable of information processing can be widely applied.

  A functional configuration of the video evaluation apparatus 140 will be described with reference to FIG. As illustrated in FIG. 14, the video evaluation device 140 includes a change amount detection unit 1401 and an evaluation value calculation unit 1402.

  The change amount detection unit 1401 decomposes an input video signal 1403 input as a moving image signal from the outside into frame images. The change amount detection unit 1401 detects a change amount indicating the degree of change between the frame images of the input video signal based on the plurality of decomposed frame images. The change amount detection unit 1401 outputs the detected change amount 1405 to the evaluation value calculation unit 1402.

  Here, a method for detecting the change amount 1405 will be described in detail with reference to FIG. The change amount detection unit 1401 sequentially reads two consecutive frame images decomposed from the input video signal 1403. Here, for convenience of explanation, two consecutive frame images will be described as a frame image P0 and a frame image P1 in the order of reading. The change amount detection unit 1401 obtains a difference in luminance value between pixels located at the same coordinates in the read frame image P1 and frame image P0, and calculates the square value of this difference for each pixel included in the entire frame image. To calculate. The change amount detection unit 1401 detects the change amount 1405 by calculating the average value of the square values calculated for each pixel. Therefore, the calculated average value is output to the evaluation value calculation unit 1402 as the change amount 1405.

  The evaluation value calculation unit 1402 is based on the change amount 1405 received from the change amount detection unit 1401 and the time interval between the frame images based on the frame rate information 1404 received from the outside. Determine the amount of time change according to the time interval. The evaluation value calculation unit 1402 calculates an evaluation value for evaluating the smoothness of the motion of the input video based on the calculated time change amount. The evaluation value calculation unit 1402 outputs the calculated evaluation value 1406 to the outside. Here, the outside corresponds to, for example, a device that determines an optimal frame rate for video processing of the input video signal 1403 based on the evaluation value 1406. By outputting the evaluation value 1406 to such an external device, it becomes possible to determine a frame rate according to the smoothness of the video motion by the input video signal 1403.

  Here, with reference to FIG. 16, a method for evaluating the smoothness of movement will be specifically described. First, with reference to FIG. 16A, for example, a method for determining the amount of time change S1 at time T1 will be described. Evaluation value calculation section 1402 calculates time change amount S1 based on time interval Δt1 and change amount Δc1 between time T0 of frame image P0 and time T1 of frame image P1 based on frame rate information 1404. Note that the time interval Δt1 between the time T0 and the time T1 is, for example, 1 / F1 second when the frame rate of the frame rate information 1404 at the time T1 is F1fps.

  A method for determining the amount of time change will be described more specifically. As shown in FIG. 16A, the time change amount S1 at time T1 is, for example, Δt1 · Δc1 (when the time interval between time T0 and time T1 is Δt1 and the change amount at time T1 is Δc1. The area of the hatched portion S1 shown in FIG. Similarly, the time change amount S2 at time T2 is, for example, Δt2 · Δc2 (see FIG. 16A) when the time interval between time T1 and time T2 is Δt2 and the change amount at time T2 is Δc2. (The area of the shaded portion S2 shown).

  Next, with reference to FIG. 16B, a method for calculating the evaluation value based on the time change amount will be described. The evaluation value calculation unit 1402 calculates, for all frame images included in the input video signal, an evaluation value of the smoothness of motion at time Tn (n: positive integer; the same applies hereinafter) of each frame image. More specifically, the evaluation value calculation unit 1402 calculates the evaluation value of the smoothness of movement at the time Tn using the time change amount Sn at the time Tn of each frame image. More specifically, the evaluation value calculation unit 1402 calculates the evaluation value of the smoothness of the movement at the time Tn by, for example, the following expression using the time change amount Sn, α / Sn (α is a constant). In addition, the evaluation value calculation unit 1402 calculates an evaluation value of the smoothness of the motion at time Tn, for example, by the following expression using the time change amount Sn, a × exp−bSn + c (a, b, and c are constants). May be. The evaluation value calculation unit 1402 calculates the average value of the evaluation values of the smoothness of motion at all times of the input video based on the calculated evaluation values. This average value is output to the outside as the final evaluation value 1406 of the input video.

  Next, the flow of the video evaluation process in the video evaluation device 140 of the first embodiment will be described with reference to FIG.

  First, the change amount detection unit 1401 sequentially reads two consecutive frame images (frame image P0 and frame image P1) decomposed from the input video signal 1403 (step S1701).

  Next, the change amount detection unit 1401 obtains a difference in luminance value between pixels located at the same coordinates in the read frame image P1 and frame image P0, and includes the square value of this difference in the entire frame image. Calculation is performed for each pixel. (Step S1702).

  Next, the change amount detection unit 1401 calculates an average value of the square values calculated for each pixel (step S1703). This calculated average value is output to the evaluation value calculation unit 1402 as the change amount 1405.

  Next, the evaluation value calculation unit 1402 determines the change amount 1405 and each frame based on the change amount 1405 received from the change amount detection unit 1401 and the time interval between frame images based on the frame rate information 1404 received from the outside. A time change amount corresponding to the time interval between images is calculated (step 1704).

  Next, the evaluation value calculation unit 1402 calculates the evaluation value of the smoothness of movement of each frame image at time Tn for all frame images included in the input video signal using the time change amount (step S1705). .

  Next, the evaluation value calculation unit 1402 calculates the average value of the evaluation values of the smoothness of motion at all times of the input video based on the calculated evaluation values (step S1706).

  Next, the evaluation value calculation unit 1402 outputs the calculated average value to the outside as the evaluation value 1406 of the smoothness of the movement with respect to the entire input video signal 1403 (step 1707).

  As described above, according to the video evaluation device 140 in the first embodiment, the amount of change is calculated based on the square value of the difference in luminance value between frames of a plurality of frame images included in the input video signal. . Also, an evaluation value for evaluating the smoothness of the motion of the input video signal is calculated based on the amount of change and the time interval between the frame images based on the frame rate of the input video signal. Therefore, based on the amount of change in the luminance value of the input video signal input according to the predetermined frame rate, the smoothness of the motion of the video at the frame rate during video processing can be evaluated. Further, since the evaluation value obtained by this evaluation is output to the outside, the frame rate can be determined externally based on this evaluation value. That is, the frame rate can be determined according to the smoothness of the motion of the video.

  Note that the unit of change amount detection by the change amount detection unit 1401 described above is not limited to the entire frame image described above. For example, a block unit, a pixel unit, an object region unit, or the like may be used.

  Further, the change amount detection method by the change amount detection unit 1401 is not limited to using the average value of the square values calculated for each pixel included in the frame image. For example, the maximum value, intermediate value, or minimum value of the square value calculated for each pixel may be used, or the square value of the maximum value, intermediate value, or minimum value of the square value may be used. Alternatively, a variance value of the square value in the entire frame image may be used.

  Further, the method of detecting the change amount by the change amount detection unit 1401 is not limited to using the square value of the difference in luminance value between pixels located at the same coordinates of each frame described above. For example, a difference value of luminance values between pixels located at the same coordinate in each frame, or an absolute value of this difference may be used.

  In addition to the above-described values, the change amount detection method by the change amount detection unit 1401 can be any method that represents a change in the input video signal between frames, such as a motion vector between frame images of the input video signal. A value can be used.

  Further, the method of calculating the time change amount by the evaluation value calculation unit 1402 is not limited to the method using the above-described formula (for example, Δt1 · Δc1), and the time interval between the frame images based on the change amount and the frame rate information. What is necessary is just to be able to figure out based on.

  Further, the time change amount calculated by the evaluation value calculation unit 1402 does not necessarily have to be one for each frame. For example, the number may be one for each block, pixel, or object.

  Also, the final evaluation value 1406 of the input video calculated by the evaluation value calculation unit 1402 is not limited to the average value of the evaluation values of the smoothness of the motion at all times of the input video described above. For example, it may be the maximum value, intermediate value, or minimum value of the evaluation value of the smoothness of motion at all times of the input video.

  Further, the final evaluation value 1406 of the input video calculated by the evaluation value calculation unit 1402 does not necessarily have to be one for all frame images included in the input video signal. For example, it may be one for several frame images, one frame image, block, pixel, and object.

  Finally, a video evaluation program 180 for causing a computer to function as the above-described video evaluation device 140 will be described with reference to FIG.

  As illustrated in FIG. 18, the video evaluation program 140 includes a main module program 1801 that supervises processing, a change amount detection module 1802, and an evaluation value calculation module 1803. The functions that the change amount detection module 1802 and the evaluation value calculation module 1803 cause the computer to perform are the same as the functions of the change amount detection unit 1401 and the evaluation value calculation unit 1402 described above, respectively.

  Note that the video evaluation program 180 is provided by a storage medium such as a CD-ROM, a DVD, or a ROM, or a semiconductor memory, for example. The video evaluation program 180 may be provided via a network as a computer data signal superimposed on a carrier wave.

  [Modification of First Embodiment]

  Next, a modification of the first embodiment described above will be described. FIG. 1 is a diagram illustrating a functional configuration of a video evaluation device 10 according to a modification of the first embodiment.

  Here, the video evaluation device 10 is physically a computer including a CPU (Central Processing Unit), a storage device such as a memory, a communication device, and the like. Therefore, the video evaluation apparatus 10 may be a fixed communication terminal such as a PC terminal or a mobile communication terminal such as a mobile phone. That is, as the video evaluation device 10, devices capable of information processing can be widely applied.

  The functional configuration of the video evaluation apparatus 10 will be described with reference to FIG. As illustrated in FIG. 1, the video evaluation device 10 includes a displacement amount detection unit 101, a feature value calculation unit 102, and an evaluation value calculation unit 103.

  The displacement amount detection unit 101 decomposes an input video signal 104 input as a moving image signal from the outside into frame images. The displacement amount detection unit 101 detects a displacement amount (motion displacement amount) indicating the degree of displacement of the motion of the input video signal based on the plurality of decomposed frame images. The displacement amount detection unit 101 outputs the detected displacement amount 106 to the feature value calculation unit 102.

  Note that the amount of displacement is not limited to that indicating the degree of displacement of the motion of the input video signal, but may be any amount that indicates the degree of change between the frame images of the input video signal.

  Here, with reference to FIG. 2, a method for detecting the displacement 106 will be described in detail. The displacement detection unit 101 sequentially reads two consecutive frame images decomposed from the input video signal 104. Here, for convenience of description, two consecutive frame images will be described as a frame image P0 (see FIG. 2A) and a frame image P1 (see FIG. 2B) in the order of reading. The displacement amount detection unit 101 divides the read frame image P1 into blocks having a predetermined size. For each block of the frame image P1, the displacement amount detection unit 101 searches the frame image P0 for an image signal pattern that is most similar to the image signal pattern of each block. This search can be realized, for example, by using a search process based on block matching (correlation method) shown in FIG. The displacement amount detection unit 101 detects a motion vector V (MVx, MVy), which is a spatial displacement amount between the signal patterns of both images, based on the signal patterns of both images determined to be most similar by the search process. . This motion vector V is output to the feature value calculation unit 102 as the displacement amount 106.

  The feature value calculation unit 102 calculates a motion feature value 107 indicating the motion feature of the input video signal based on the displacement amount 106 received from the displacement amount detection unit 101. The feature value calculation unit 102 outputs the calculated motion feature value 107 to the evaluation value calculation unit 103.

  Here, a method for calculating the motion feature value will be specifically described. Using the motion vector of each block of the frame image P1 received as the displacement amount 106, the feature value calculation unit 102 obtains the magnitude of the motion vector of each block of the frame image P1. The magnitude of the motion vector is obtained by (MVx2 + MVy2) 1/2 when the x component and y component of the motion vector of an arbitrary block on the frame image P1 are MVx and MVy, respectively. The feature value calculation unit 102 calculates a value that is a feature of the frame image P1 based on the size of each motion vector. This calculated value is output to the evaluation value calculation unit 103 as the motion feature value 107. As the value (motion feature value 107) calculated by the feature value calculation unit 102, for example, the maximum value, average value, intermediate value, or minimum value of the magnitude of the motion vector obtained for each block included in the frame image is applicable. To do.

  Based on the motion feature value 107 received from the feature value calculation unit 102 and the time interval between frame images based on the frame rate information 105 received from the outside, the evaluation value calculation unit 103 determines the degree of motion between the frame images. The amount of motion between frames indicating The evaluation value calculation unit 103 calculates an evaluation value for evaluating the smoothness of the motion of the input video based on the calculated inter-frame motion amount. The evaluation value calculation unit 103 outputs the calculated evaluation value 108 to the outside. Here, the outside corresponds to, for example, a device that determines an optimal frame rate for video processing of the input video signal 104 based on the evaluation value 108. By outputting the evaluation value 108 to such an external device, it becomes possible to determine a frame rate according to the smoothness of the video motion by the input video signal 104.

  When calculating the evaluation value, it is not always necessary to calculate based on the inter-frame motion amount. For example, the evaluation value may be calculated based on the displacement 106 and the amount of time change corresponding to the time interval between the frame images. This time change amount can be determined based on the motion feature value 107 received from the feature value calculation unit 102 and the time interval between frame images based on the frame rate information 105 received from the outside.

  Here, with reference to FIG. 3, a method for evaluating the smoothness of movement will be described in detail. First, with reference to FIG. 3A, for example, a method for determining the inter-frame motion amount S1 at time T1 will be described. The evaluation value calculation unit 103 is based on the time interval Δt1 between the time T0 of the frame image P0 and the time T1 of the frame image P1 based on the frame rate information 105, and the motion feature value Δd1 of the frame image P1. Is determined. Note that the time interval Δt1 between the time T0 and the time T1 is, for example, 1 / F1 second when the frame rate of the frame rate information 105 at the time T1 is F1fps.

  A method for determining the amount of motion between frames will be described more specifically. As shown in FIG. 3A, the inter-frame motion amount S1 at time T1 is, for example, Δt1 · when the time interval between time T0 and time T1 is Δt1 and the motion feature value at time T1 is Δd1. Δd1 / 2 (the area of the shaded portion S1 shown in FIG. 3A). Similarly, the inter-frame motion amount S2 at time T2 is, for example, Δt2 · Δd2 / 2 (FIG. 3) when the time interval between time T1 and time T2 is Δt2 and the motion feature value at time T2 is Δd2. The area of the shaded portion S2 shown in FIG.

  Next, a method for calculating the evaluation value based on the inter-frame motion amount will be described with reference to FIG. The evaluation value calculation unit 103 calculates, for all frame images included in the input video signal, an evaluation value of the smoothness of motion at time Tn (n: positive integer; the same applies hereinafter) of each frame image. More specifically, the evaluation value calculation unit 103 calculates an evaluation value of the smoothness of motion at time Tn using the inter-frame motion amount Sn at time Tn of each frame image. More specifically, the evaluation value calculation unit 103 calculates the evaluation value of the smoothness of the motion at time Tn, for example, by the following equation using the interframe motion amount Sn, α / Sn (α is a constant). . Further, the evaluation value calculation unit 103 calculates the evaluation value of the smoothness of the motion at the time Tn, for example, by the following equation using the interframe motion amount Sn, a × exp−bSn + c (a, b, and c are constants). May be. The evaluation value calculation unit 103 calculates the average value of the evaluation values of the smoothness of motion at all times of the input video based on the calculated evaluation values. This average value is output to the outside as the final evaluation value 108 of the input video.

  Next, the flow of the video evaluation process in the video evaluation apparatus 10 according to the modification of the first embodiment will be described with reference to FIG.

  First, the displacement amount detection unit 101 sequentially reads two consecutive frame images (frame image P0 and frame image P1) decomposed from the input video signal 104 (step S401).

  Next, the displacement amount detection unit 101 divides the read frame image P1 into blocks of a predetermined size (step S402).

  Next, the displacement amount detection unit 101 searches the frame image P0 for an image signal pattern most similar to the image signal pattern of each block for each block of the frame image P1 (step S403).

  Next, the displacement amount detection unit 101 detects a motion vector (MVx, MVy), which is a spatial displacement amount between the image signal patterns, based on the signal patterns of both images determined to be the most similar by the search. (Step S404). The detected motion vector is output to the feature value calculation unit 102 as the displacement amount 106.

  Next, the feature value calculation unit 102 obtains the size of the motion vector of each block of the frame image P1 using the motion vector of each block of the frame image P1 included in the displacement amount 106 (step S405).

  Next, the feature value calculation unit 102 calculates a value that is a feature of the frame image P1 based on the size of each motion vector (step S406). This calculated value is output to the evaluation value calculation unit 103 as the motion feature value 107.

  Next, the evaluation value calculation unit 103 determines between the frame images based on the motion feature value 107 received from the feature value calculation unit 102 and the time interval between frame images based on the frame rate information 105 received from the outside. A motion amount between frames, which is a motion amount, is calculated (step 407).

  Next, the evaluation value calculation unit 103 calculates, for all frame images included in the input video signal, an evaluation value of the smoothness of the motion of each frame image at the time Tn using the inter-frame motion amount (step S408). ).

  Next, the evaluation value calculation unit 103 calculates an average value of the evaluation values of the smoothness of motion at all times of the input video based on the calculated evaluation values (step S409).

  Next, the evaluation value calculation unit 103 outputs the calculated average value to the outside as the evaluation value 108 of the smoothness of motion for the entire input video signal 104 (step 410).

  As described above, according to the video evaluation device 10 in the modification of the first embodiment, the motion vector is detected based on the plurality of frame images included in the input video signal, and the motion is detected based on the magnitude of the motion vector. A feature value is calculated. An evaluation value for evaluating the smoothness of the motion of the input video signal is calculated based on the motion feature value and the time interval between the frame images based on the frame rate of the input video signal. Therefore, the smoothness of the video motion at the frame rate during video processing can be evaluated according to the magnitude of the motion vector of the input video signal input according to the predetermined frame rate. Further, since the evaluation value obtained by this evaluation is output to the outside, the frame rate can be determined externally based on this evaluation value. That is, the frame rate can be determined according to the smoothness of the motion of the video.

  Note that the search unit of the image signal pattern by the displacement amount detection unit 101 described above is not limited to the block unit described above. For example, it may be a frame unit, a pixel unit, an object area unit, or the like. Further, the search processing method by the displacement amount detection unit 101 is not limited to the block matching described above. For example, a concentration gradient method may be used.

  Further, the displacement amount detection unit 101 may read the input video signal 104 including the motion vector of the video as the input video signal 104 described above. In this case, the displacement amount detection unit 101 detects a motion vector from the input video signal 104 received from the outside, and outputs the detected motion vector to the feature value calculation unit 102 as the displacement amount 106.

  Further, the motion feature value 107 calculated by the feature value calculation unit 102 is not limited to the maximum value, average value, intermediate value, or minimum value of the magnitude of the motion vector obtained for each block described above. For example, it may be the size of one motion vector obtained for each frame image, or the maximum value, average value, or intermediate value of the size of the motion vector obtained for each pixel or object area in the frame image. Or it may be a minimum value.

  Further, the motion feature value 107 calculated by the feature value calculation unit 102 does not necessarily need to be one for each frame. For example, the number may be one for each block, pixel, or object. The motion feature value 107 includes a plurality of circles centered at the origin shown in FIG. 5 and a plurality of lines extending radially from the origin with respect to the distribution of motion vectors obtained in units of blocks, pixels, or objects. There may be one for each unit of the range R (for example, R1, R2, R3) delimited by.

  Here, with reference to FIG. 5, a method for calculating the motion feature value 107 in the range R unit will be specifically described taking a case where one frame image is divided into nine blocks as an example. First, one motion vector is obtained for each of the nine divided blocks. The obtained motion vectors are referred to as motion vectors V1 to V9. Next, the motion vectors V1 to V9 are projected on the graph shown in FIG. For example, when the motion vectors V1 to V4 are included in the range R1 shown in FIG. 5, the motion vectors V5 and V6 are included in the range R2 shown in FIG. 5, and the motion vectors V7 to V9 are included in the range R3 shown in FIG. To do. In this case, for example, the motion vector VR1 calculated as the average value of the motion vectors V1 to V4 is obtained as the motion feature value 107 in the range R1, and the motion vector VR2 calculated as the average value of the motion vectors V5 and V6. Is obtained as the motion feature value 107 in the range R2, and the motion vector VR3 calculated as the average value of the motion vectors V7 to V9 is obtained as the motion feature value 107 in the range R3. The method for obtaining the motion vector is not limited to the illustrated block unit, and may be, for example, a pixel unit or an object unit.

  Further, the method of calculating the inter-frame motion amount by the evaluation value calculation unit 103 is not limited to the method using the above-described formula (for example, Δt1 · Δd1 / 2). For example, the following expression represented by using a motion vector (MVx, MVy) as the displacement amount 106 and a time interval between frame images based on the frame rate information 105, β · MVx · Δt1 / 2 + γ · MVy · Δt1 / 2 (Β and γ may be constants).

  Further, the inter-frame motion amount calculated by the evaluation value calculation unit 103 is not necessarily one per frame. For example, the number may be one for each block, pixel, object, or range R (see FIG. 5) described above.

  In addition, the evaluation value of the smoothness of the motion at the time Tn of each frame image calculated by the evaluation value calculation unit 103 is α / Sn (α is a constant) or a × exp−bSn + c (a, b, c are The calculation is not limited to a constant). For example, it may be calculated by a function using a motion vector (MVx, MVy) as the displacement amount 106 and a time interval between frame images based on the frame rate information 105.

  Further, the final evaluation value 108 of the input video calculated by the evaluation value calculation unit 103 is not limited to the average value of the evaluation values of the smoothness of motion at all times of the input video described above. For example, it may be the maximum value, intermediate value, or minimum value of the evaluation value of the smoothness of motion at all times of the input video.

  Further, the final evaluation value 108 of the input video calculated by the evaluation value calculation unit 103 does not necessarily have to be one for all frame images included in the input video signal. For example, there may be one frame image, one frame image, block, pixel, object, or one for the above-described range R (see FIG. 5).

  Finally, a video evaluation program 50 for causing a computer to function as the above-described video evaluation apparatus 10 will be described with reference to FIG.

  As shown in FIG. 6, the video evaluation program 50 includes a main module program 501 that supervises processing, a displacement amount detection module 502, a feature value calculation module 503, and an evaluation value calculation module 504. The functions that the displacement amount detection module 502, the feature value calculation module 503, and the evaluation value calculation module 504 cause the computer to perform are the same as the functions of the displacement amount detection unit 101, the feature value calculation unit 102, and the evaluation value calculation unit 103, respectively. It is.

  Note that the video evaluation program 50 is provided by a storage medium such as a CD-ROM, a DVD, or a ROM, or a semiconductor memory, for example. The video evaluation program 50 may be provided via a network as a computer data signal superimposed on a carrier wave.

  In addition, the video evaluation apparatus 10 according to the modification of the first embodiment integrates the displacement amount detection unit 101 and the feature value calculation unit 102 into a new change amount detection unit, whereby the video evaluation apparatus according to the first embodiment. The same functional configuration as that of 140 can be obtained.

  [Second Embodiment]

  Next, a second embodiment of the present invention will be described. FIG. 7 is a diagram illustrating a functional configuration of the frame rate determination device 70 in the second embodiment.

  Here, the frame rate determination device 70 is physically a computer including a CPU (Central Processing Unit), a storage device such as a memory, a communication device, and the like. Therefore, the frame rate determination device 70 may be a fixed communication terminal such as a PC terminal or a mobile communication terminal such as a mobile phone. That is, as the frame rate determining device 70, devices capable of information processing can be widely applied.

  A functional configuration of the frame rate determining apparatus 70 will be described with reference to FIG. As illustrated in FIG. 7, the frame rate determination device 70 includes a frame rate generation unit 701, a video evaluation unit 702, and a frame rate determination unit 703.

  The frame rate generator 701 generates a first frame rate 705. The frame rate generation unit 701 outputs the generated first frame rate 705 to the video evaluation unit 702 and the frame rate determination unit 703.

  The video evaluation unit 702 has the same function as the function of the video evaluation device 140 described in the first embodiment described above or the function of the video evaluation device 10 described in the modification of the first embodiment. That is, the video evaluation unit 702 has the same functions as the functions of the change amount detection unit 1401 and the evaluation value calculation unit 1402, or the functions of the displacement amount detection unit 101, the feature value calculation unit 102, and the evaluation value calculation unit 103. Have

  The video evaluation unit 702 reads frame images at time intervals corresponding to the first frame rate 705 from the input video signal 704 inputted as a moving image signal from the outside, and evaluates the smoothness of the motion for the entire input video signal 704. A value 706 is calculated. The video evaluation unit 702 outputs the calculated evaluation value 706 of the smoothness of the motion to the frame rate determination unit 703.

  Here, there is no particular problem even if the first frame rate 705 is different from the sampling rate of the input video signal 704. For example, the first frame rate 705 shown in FIG. 8A is 1/15 seconds, whereas the sampling rate of the input video signal 704 shown in FIG. 8B is 1/30 seconds. is there.

  The frame rate determination unit 703 determines the second frame rate 707 based on the evaluation value 706 received from the video evaluation unit 702 and the first frame rate 705 received from the frame rate generation unit 701. The frame rate determination unit 703 outputs the determined second frame rate 707 to the frame rate determination device 70 as a frame rate for processing the input video signal 704.

  More specifically, the frame rate determination unit 703 makes the second frame rate 707 smaller than the first frame rate 705, for example, when the evaluation value 706 is larger than a predetermined specified value. For example, when the evaluation value 706 is smaller than a predetermined specified value, the frame rate determination unit 703 makes the second frame rate 707 larger than the first frame rate 705. For example, when the evaluation value 706 matches a predetermined specified value, the frame rate determining apparatus 70 sets the second frame rate 707 to the same frame rate as the first frame rate 705.

  In this way, it is possible to determine the second frame rate so that the evaluation value for evaluating the smoothness of the motion of the video is within the range of the predetermined evaluation standard. In other words, the input video signal can be read while maintaining the smoothness of the video motion within a predetermined reference range. The predetermined specified value described above may be set in advance or may be given from the outside.

  Next, the flow of frame rate determination processing in the frame rate determination device 70 of the second embodiment will be described with reference to FIG.

  First, the frame rate generator 701 generates a first frame rate 705 (step S901).

  Next, the video evaluation unit 702 reads frame images from the input video signal 704 at time intervals corresponding to the first frame rate 705 (step 902).

  Next, the video evaluation unit 702 calculates a motion smoothness evaluation value 706 for the entire input video signal 704 based on each frame image (step 903). That is, based on each frame image, the video evaluation processing from step S1701 to S1707 described in the first embodiment (see FIG. 17) or steps S401 to S410 described in the modification of the first embodiment described above. By performing the video evaluation process up to (see FIG. 4), the evaluation value 706 of the smoothness of the motion for the entire input video signal 704 is calculated.

  Next, the frame rate determination unit 703 determines the second frame rate 707 based on the evaluation value 706 received from the video evaluation unit 702 and the first frame rate 705 received from the frame rate generation unit 701. (Step 904).

  Next, the frame rate determination unit 703 outputs the second frame rate 707 to the outside as a frame rate for processing the input video signal 704 (step 905).

  As described above, according to the frame rate determination device 70 in the second embodiment, the motion of the input video signal is smoothed based on the change amount of the input video signal and the time interval between the frame images based on the first frame rate. An evaluation value for evaluating the image quality is calculated, and a second frame rate for processing the input video signal is determined using the evaluation value. That is, the smoothness of the motion of the video at the first frame rate is evaluated according to the change amount of the input video signal input according to the first frame rate, and the second frame rate is evaluated using this evaluation. Is determined.

  Also, a motion vector is detected based on a plurality of frame images included in the input video signal, and a motion feature value is calculated based on the magnitude of the motion vector. Then, an evaluation value for evaluating the smoothness of the motion of the input video signal is calculated based on the motion feature value and the time interval between the frame images based on the first frame rate. A second frame rate for processing the input video signal is determined. That is, the smoothness of the motion of the video at the first frame rate is evaluated in accordance with the feature amount of the motion of the input video signal input according to the first frame rate. A frame rate is determined.

  Therefore, the frame rate of the input video signal can be determined according to the smoothness of the video motion, and the input video signal can be read while maintaining the smoothness of the video motion.

  Finally, a frame rate determination program 100 for causing a computer to function as the above-described frame rate determination device 70 will be described with reference to FIG.

  As shown in FIG. 10, the frame rate determination program 100 includes a main module program 1001 that controls processing, a frame rate generation module 1002, a video evaluation module 1003, and a frame rate determination module 1004. The functions that the frame rate generation module 1002, the video evaluation module 1003, and the frame rate determination module 1004 cause the computer to perform are the same as the functions of the above-described frame rate generation unit 701, video evaluation unit 702, and frame rate determination unit 703, respectively. .

  The frame rate determination program 100 is provided by a storage medium such as a CD-ROM, DVD, or ROM, or a semiconductor memory, for example. The frame rate determination program 100 may be provided via a network as a computer data signal superimposed on a carrier wave.

  [Third embodiment]

  Next, a third embodiment of the present invention will be described. FIG. 11 is a diagram illustrating a functional configuration of the video processing apparatus 110 according to the third embodiment.

  Here, the video processing device 110 is physically a computer including a CPU (Central Processing Unit), a storage device such as a memory, and a communication device. Therefore, the video processing apparatus 110 may be a fixed communication terminal such as a PC terminal or a mobile communication terminal such as a mobile phone. That is, as the video processing device 110, devices capable of information processing can be widely applied.

  The functional configuration of the video processing apparatus 110 will be described with reference to FIG. As illustrated in FIG. 11, the video processing apparatus 110 includes a buffer unit 1101, a frame rate determination unit 1102, and a video processing unit 1103.

  The buffer unit 1101 temporarily stores an input video signal 1104 input as a moving image signal from the outside in a buffer on the memory. The buffer unit 1101 outputs the input video signal 1104 to the frame rate determination unit 1102. Note that the input video signal 1104 temporarily stored in the buffer is read as a processed video signal 1106 by a video processing unit 1103 described later.

  The frame rate determination unit 1102 has the same function as that of the frame rate determination device 70 described in the second embodiment. That is, the frame rate determination unit 1102 has the same function as the functions of the frame rate generation unit 701, the video evaluation unit 702, and the frame rate determination unit 703 described above. The frame rate determination unit 1102 uses the input video signal 1104 received from the buffer unit 1101 to calculate a video processing frame rate 1105 corresponding to the second frame rate 707 described above. The frame rate determination unit 1102 outputs the calculated video processing frame rate 1105 to the video processing unit 1103.

  The video processing unit 1103 reads the processed video signal 1106 from the buffer unit 1101 at time intervals corresponding to the video processing frame rate 1105, and performs video processing based on the processed video signal 1106. Specifically, the video processing unit 1103 is a frame at a time corresponding to the video processing frame rate 1105 received from the frame rate determination unit 1102 among the frame images included in the processed video signal 1106 stored in the buffer unit 1101. Read only the image and perform video processing. The video processing unit 1103 outputs the video generated by the video processing to the outside. The video processing performed by the video processing unit 1103 corresponds to, for example, video acquisition, storage, transmission, encoding, and decoding.

  Next, the flow of video processing in the video processing apparatus 110 of the third embodiment will be described with reference to FIG.

  First, the buffer unit 1101 outputs an input video signal 1104 input as a moving image signal from the outside to the frame rate determination unit 1102, and temporarily stores the input video signal 1104 in a buffer on the memory ( Step S1201).

  Next, the frame rate determination unit 1102 calculates the video processing frame rate 1105 using the input video signal 1104 received from the buffer unit 1101 (step 1202). That is, by using the input video signal 1104, the frame rate determination process (see FIG. 9) from steps S901 to S905 described in the second embodiment described above is performed, thereby the video processing frame rate that is the second frame rate. 1105 is calculated.

  Next, the video processing unit 1103 reads the processed video signal 1106 from the buffer unit 1101 at a time interval corresponding to the video processing frame rate 1105, and performs video processing based on the processed video signal 1106 (step 1203).

  Next, the video processing unit 1103 outputs the video generated by the video processing to the outside (step 1204).

  As described above, according to the video processing device 110 in the third embodiment, the smoothness of the movement of the input video signal based on the change amount of the input video signal and the time interval between the frame images based on the first frame rate. Is evaluated, and a second frame rate for processing the input video signal is determined using the evaluation value. Then, video processing of the input video signal is performed using the second frame rate. That is, the smoothness of the motion of the video at the first frame rate is evaluated according to the change amount of the input video signal input according to the first frame rate, and the second frame rate is evaluated using this evaluation. Is determined. Then, the video processing of the input video signal is performed using the second frame rate determined based on the evaluation value for the input video signal.

  Also, a motion vector is detected based on a plurality of frame images included in the input video signal, and a motion feature value is calculated based on the magnitude of the motion vector. Then, an evaluation value for evaluating the smoothness of the motion of the input video signal is calculated based on the motion feature value and the time interval between the frame images based on the first frame rate. A second frame rate for processing the input video signal is determined. Furthermore, video processing of the input video signal is performed using the second frame rate. That is, the smoothness of the motion of the video at the first frame rate is evaluated according to the feature value of the motion of the input video signal input according to the first frame rate, and the second A frame rate is determined. Then, the video processing of the input video signal is performed using the second frame rate determined based on the evaluation value for the input video signal.

  Therefore, the video processing of the input video signal can be performed based on the second frame rate determined according to the evaluation of the smoothness of movement. That is, it is possible to determine the frame rate according to the smoothness of the video motion and to perform video processing of the input video signal while maintaining the smoothness of the video motion.

  Note that the above-described video processing device 110 is applicable to, for example, a video acquisition device (for example, a camera), a video transmission device, a video encoding device, or a video decoding device.

  First, when the video processing device 110 is applied to a video acquisition device, each function described above functions as described below. The buffer unit 1101 buffers the input video signal 1104 input at a sampling rate (for example, 30 fps). The frame rate determination unit 1102 uses the input video signal 1104 received from the buffer unit 1101 to calculate an optimal video processing frame rate 1105 (for example, 15 fps) when the video acquisition device acquires video, and the video processing unit 1103. The video processing unit 1103 samples the input video signal 1104 stored in the buffer unit 1101 using the video processing frame rate 1105 (for example, 15 fps) received from the frame rate determination unit 1102. The video processing unit 1103 acquires the processed video signal 1106 obtained by this sampling at a video processing frame rate 1105 (for example, 15 fps).

  Further, when the video processing device 110 is applied to a video storage device, each function described above functions as described below. The buffer unit 1101 buffers the input video signal 1104 input at a sampling rate (for example, 30 fps). The frame rate determination unit 1102 uses the input video signal 1104 received from the buffer unit 1101 to calculate an optimal video processing frame rate 1105 (for example, 15 fps) when the video storage device stores video, and the video processing unit 1103. The video processing unit 1103 samples the input video signal 1104 stored in the buffer unit 1101 using the video processing frame rate 1105 (for example, 15 fps) received from the frame rate determination unit 1102. The video processing unit 1103 accumulates the processed video signal 1106 obtained by this sampling at a video processing frame rate 1105 (for example, 15 fps).

  Further, when the video processing device 110 is applied to a video transmission device, each function described above functions as described below. The buffer unit 1101 buffers the input video signal 1104 input at a sampling rate (for example, 30 fps). The frame rate determination unit 1102 uses the input video signal 1104 received from the buffer unit 1101 to calculate an optimal video processing frame rate 1105 (for example, 15 fps) when the video transmission apparatus transmits video, and the video processing unit 1103. The video processing unit 1103 samples the input video signal 1104 stored in the buffer unit 1101 using the video processing frame rate 1105 (for example, 15 fps) received from the frame rate determination unit 1102. The video processing unit 1103 transmits the processed video signal 1106 obtained by this sampling at a video processing frame rate 1105 (for example, 15 fps).

  Further, when the video processing device 110 is applied to a video encoding device, each function described above functions as described below. The buffer unit 1101 buffers the input video signal 1104 input at a sampling rate (for example, 30 fps). The frame rate determination unit 1102 uses the input video signal 1104 received from the buffer unit 1101 to calculate an optimal video processing frame rate 1105 (for example, 15 fps) when the video encoding device encodes video, The data is output to the processing unit 1103. The video processing unit 1103 samples the input video signal 1104 stored in the buffer unit 1101 using the video processing frame rate 1105 (for example, 15 fps) received from the frame rate determination unit 1102. The video processing unit 1103 encodes the processed video signal 1106 obtained by this sampling at a video processing frame rate 1105 (for example, 15 fps).

  Further, when the video processing device 110 is applied to a video decoding device, each function described above functions as described below. The buffer unit 1101 buffers the input video signal 1104 input at a sampling rate (for example, 30 fps). The frame rate determination unit 1102 uses the input video signal 1104 received from the buffer unit 1101 to calculate an optimal video processing frame rate 1105 (for example, 15 fps) when the video decoding apparatus decodes video, and performs video processing Output to the unit 1103. The video processing unit 1103 samples the input video signal 1104 stored in the buffer unit 1101 using the video processing frame rate 1105 (for example, 15 fps) received from the frame rate determination unit 1102. The video processing unit 1103 decodes the processed video signal 1106 obtained by this sampling at a video processing frame rate 1105 (for example, 15 fps).

  Finally, a video processing program 130 for causing a computer to function as the above-described video processing device 110 will be described with reference to FIG.

  As shown in FIG. 13, the video processing program 130 includes a main module program 1301 that supervises processing, a buffer module 1302, a frame rate determination module 1303, and a video processing module 1304. The functions that the buffer module 1302, the frame rate determination module 1303, and the video processing module 1304 cause the computer to perform are the same as the functions that the buffer unit 1101, the frame rate determination unit 1102, and the video processing unit 1103 have.

  The video processing program 130 is provided by a storage medium such as a CD-ROM, a DVD, or a ROM, or a semiconductor memory, for example. The video processing program 130 may be provided via a network as a computer data signal superimposed on a carrier wave.

(About the multifaceted nature of the present invention)
The technical idea of the present invention is expressed in many ways as follows.
(1) Based on a plurality of frame images included in the input video signal, change amount detecting means for detecting a change amount indicating the degree of change between the frame images;
Evaluation value calculation means for calculating an evaluation value related to the smoothness of movement of the input video signal based on the amount of change and the time interval between the frame images, and outputting the evaluation value to the outside;
A video evaluation apparatus comprising:
(2) The evaluation value calculation means calculates a time change amount according to the change amount and a time interval between the frame images based on the change amount and a time interval between the frame images, and changes the time change. The video evaluation apparatus according to claim 1, wherein the evaluation value is calculated using a quantity.
(3) The video evaluation apparatus according to claim 1 or 2, wherein the amount of change is a value based on a difference in luminance value between the frame images.
(4) The video evaluation apparatus according to claim 1 or 2, wherein the amount of change is a motion vector between the frame images.
(5) It further comprises feature value calculation means for calculating a motion feature value indicating a motion feature of the input video signal based on the amount of change,
The video evaluation apparatus according to claim 1, wherein the evaluation value calculation unit calculates the evaluation value based on the motion feature value and a time interval between the frame images.
(6) The evaluation value calculation means calculates the change amount and the time change amount according to the time interval between the frame images based on the motion feature value and the time interval between the frame images, The video evaluation apparatus according to claim 5, wherein the evaluation value is calculated using a change amount.
(7) The change amount is a motion vector between the frame images.
The video evaluation apparatus according to claim 5, wherein the motion feature value is a value calculated based on a magnitude of the motion vector.
(8) Frame rate generating means for generating a first frame rate;
Based on a plurality of frame images included in the input video signal, change amount detection means for detecting a change amount indicating the degree of change between the frame images,
Evaluation value calculating means for calculating an evaluation value related to the smoothness of the motion of the input video signal based on the time interval between frame images corresponding to the change amount and the first frame rate;
Frame rate determining means for determining a second frame rate for processing the input video signal using the evaluation value and outputting the second frame rate to the outside;
A frame rate determining apparatus comprising:
(9) The frame rate determination means compares the evaluation value with a predetermined specified value, and when the evaluation value is larger than the specified value, the second frame rate is determined as the first frame rate. 9. The frame according to claim 8, wherein the second frame rate is set to be larger than the first frame rate when the frame rate is smaller than the frame rate and the evaluation value is smaller than the specified value. Rate determination device.
(10) buffer means for storing an input video signal;
Frame rate generating means for generating a first frame rate;
Based on a plurality of frame images included in the input video signal, change amount detection means for detecting a change amount indicating the degree of change between the frame images;
Evaluation value calculating means for calculating an evaluation value related to the smoothness of the motion of the input video signal based on the time interval between frame images corresponding to the change amount and the first frame rate;
Frame rate determining means for determining a second frame rate for processing the input video signal using the evaluation value;
Video processing means for performing video processing by reading the input video signal stored by the buffer means using the second frame rate;
A video processing apparatus comprising:
(11) Based on a plurality of frame images included in the input video signal, a change amount detecting step for detecting a change amount indicating a degree of change between the frame images;
An evaluation value calculation step of calculating an evaluation value related to the smoothness of movement of the input video signal based on the amount of change and the time interval between the frame images and outputting the evaluation value to the outside;
A video evaluation method comprising:
(12) Connect the computer
Based on a plurality of frame images included in the input video signal, change amount detection means for detecting a change amount indicating the degree of change between the frame images,
A video evaluation program for functioning as an evaluation value calculation means for calculating an evaluation value related to smoothness of movement of the input video signal based on the change amount and a time interval between the frame images and outputting the evaluation value to the outside.

  DESCRIPTION OF SYMBOLS 10 ... Video | video evaluation apparatus, 101 ... Displacement amount detection part, 102 ... Feature value calculation part, 103 ... Evaluation value calculation part, 104 ... Input video signal, 105 ... Frame rate information , 106 ... displacement amount, 107 ... motion feature value, 108 ... evaluation value, 50 ... video evaluation program, 502 ... displacement amount detection module, 503 ... feature value calculation module, 504 ... Evaluation value calculation module, 70 ... Frame rate determination device, 701 ... Frame rate generation unit, 702 ... Video evaluation unit, 703 ... Frame rate determination unit, 704 ... Input video signal 705 ... first frame rate, 706 ... evaluation value, 707 ... second frame rate, 100 ... frame rate determination program, 1002 ... frame rate Raw module, 1003 ... Video evaluation module, 1004 ... Frame rate determination module, 110 ... Video processing device, 1101 ... Buffer unit, 1102 ... Frame rate determination unit, 1103 ... Video processing 1104: Input video signal, 1105 ... Video processing frame rate, 1106 ... Processed video signal, 1107 ... Video, 130 ... Video processing program, 1302 ... Buffer module, 1303 ... .. Frame rate determination module, 1304... Video processing module, 140... Video evaluation device, 1401... Change amount detection unit, 1402. ... Frame rate information, 1405 ... Change amount, 1406 ... Evaluation value, 180 ... Image evaluation program, 1802 ... variation detection module, 1803 ... evaluation value calculation module.

Claims (10)

Based on a plurality of frame images included in the input video signal, change amount detection means for detecting a change amount indicating the degree of change between the frame images,
Based on the change amount and the time interval between the frame images, a time change amount corresponding to the change amount and the time interval between the frame images is determined, and the movement of the input video signal is performed using the time change amount. An evaluation value calculation means for calculating an evaluation value related to the smoothness of the image and outputting it to the outside,
The evaluation value calculating means calculates the time change amount by multiplying the change amount by the time interval, and when the calculated time change amount is Sn, the expression α / Sn (α is a constant) or the expression The evaluation value is calculated by a × exp ^−bSn + c (a, b, and c are constants).
A video evaluation apparatus characterized by that.   The video evaluation apparatus according to claim 1, wherein the change amount is a value based on a difference in luminance value between the frame images.   The video evaluation apparatus according to claim 1, wherein the change amount is a motion vector between the frame images. Further comprising a feature value calculating means for calculating a motion feature value indicating a motion feature of the input video signal based on the change amount;
The evaluation value calculating means calculates, as the time change amount, an inter-frame motion amount based on a value obtained by multiplying the motion feature value by the time interval instead of the change amount. Item 1. The video evaluation apparatus according to Item 1. The amount of change is a motion vector between the frame images,
The video evaluation apparatus according to claim 4 , wherein the motion feature value is a value calculated based on a magnitude of the motion vector. Frame rate generating means for generating a first frame rate;
Based on a plurality of frame images included in the input video signal, change amount detection means for detecting a change amount indicating the degree of change between the frame images,
Based on the time interval between frame images corresponding to the change amount and the first frame rate, a time change amount corresponding to the change amount and the time interval between the frame images is determined, and the time change amount is used. Evaluation value calculation means for calculating an evaluation value related to the smoothness of movement of the input video signal;
A frame rate determining unit that determines a second frame rate for processing the input video signal using the evaluation value and outputs the second frame rate to the outside;
The evaluation value calculating means calculates the time change amount by multiplying the change amount by the time interval, and when the calculated time change amount is Sn, the expression α / Sn (α is a constant) or the expression The evaluation value is calculated by a × exp ^−bSn + c (a, b, and c are constants).
A frame rate determining apparatus characterized by that. The frame rate determining means compares the evaluation value with a predetermined specified value, and if the evaluation value is larger than the specified value, the second frame rate is set to be higher than the first frame rate. 7. The frame rate determining apparatus according to claim 6 , wherein when the evaluation value is smaller than the specified value, the second frame rate is made larger than the first frame rate. . Buffer means for storing an input video signal;
Frame rate generating means for generating a first frame rate;
Based on a plurality of frame images included in the input video signal, change amount detection means for detecting a change amount indicating the degree of change between the frame images;
Based on the time interval between frame images corresponding to the change amount and the first frame rate, a time change amount corresponding to the change amount and the time interval between the frame images is determined, and the time change amount is used. Evaluation value calculation means for calculating an evaluation value related to the smoothness of movement of the input video signal;
Frame rate determining means for determining a second frame rate for processing the input video signal using the evaluation value;
Video processing means for performing video processing by reading the input video signal stored by the buffer means using the second frame rate;
The evaluation value calculating means calculates the time change amount by multiplying the change amount by the time interval, and when the calculated time change amount is Sn, the expression α / Sn (α is a constant) or the expression The evaluation value is calculated by a × exp ^−bSn + c (a, b, and c are constants).
A video processing apparatus characterized by that. Based on a plurality of frame images included in the input video signal, a change amount detecting step for detecting a change amount indicating a degree of change between the frame images;
Based on the change amount and the time interval between the frame images, a time change amount corresponding to the change amount and the time interval between the frame images is determined, and the movement of the input video signal is performed using the time change amount. An evaluation value calculation step of calculating an evaluation value related to the smoothness of the image and outputting it to the outside,
The evaluation value calculating step calculates the time change amount by multiplying the change amount by the time interval, and when the calculated time change amount is Sn, α / Sn (α is a constant) or The evaluation value is calculated by a × exp ^−bSn + c (a, b, and c are constants).
A video evaluation method characterized by the above. Computer
Based on a plurality of frame images included in the input video signal, change amount detection means for detecting a change amount indicating the degree of change between the frame images,
Based on the change amount and the time interval between the frame images, a time change amount corresponding to the change amount and the time interval between the frame images is determined, and the movement of the input video signal is performed using the time change amount. Function as an evaluation value calculation means for calculating an evaluation value related to the smoothness of the image and outputting it to the outside.
The evaluation value calculating means calculates the time change amount by multiplying the change amount by the time interval, and when the calculated time change amount is Sn, the expression α / Sn (α is a constant) or the expression The evaluation value is calculated by a × exp ^−bSn + c (a, b, and c are constants).
A video evaluation program characterized by this.

Images