JP5441555B2 - Video signal processing apparatus and video signal processing method - Google Patents

Description

  The present invention relates to a video signal processing apparatus and a video signal processing method, and more particularly, to a video signal processing apparatus and a video signal processing method that insert an interpolation frame between two consecutive frames to increase the number of video signal frames.

  A motion vector representing the motion of the image is detected from images of two consecutive frames of the input video signal, an interpolation frame to be inserted between the two frames is generated based on the motion vector, and the number of frames of the video signal is determined. There are increasing frame rate conversion technologies.

  One method for detecting motion vectors is a block matching method. In the block matching method, the correlation between the block in the previous frame and the block in the subsequent frame of the two frames is calculated while shifting the relative position of both blocks within a predetermined range, and from the relative position with the highest correlation. A motion vector is calculated. For example, when there is a repetitive pattern region such as a striped pattern or a lattice pattern in the image, the correlation of block matching becomes high even at a relative position that coincides with the repetitive pattern cycle. If the relative position having the highest correlation coincides with the cycle of the repetitive pattern, an erroneous motion vector unrelated to the original image motion is detected. When an interpolated frame image generated based on such an inappropriate motion vector is inserted between two frames and video processing is performed to increase the number of frames, a greatly disturbed video is generated. Also, if there are areas where different motions coexist in the image, such as images with different motions of the background and foreground, the correlation of block matching will be high at the relative position corresponding to each motion, and it will correspond to either motion The motion vector is detected from the relative position. When video processing for increasing the number of frames is performed by inserting an interpolated frame image generated based on the motion vector between two frames, a video in which other motion portions are disturbed is generated.

  In order to solve this problem, a technique for determining the reliability of a detected motion vector has been proposed. For example, in Patent Document 1, the second maximum correlation point is detected from a portion excluding the periphery of the first maximum correlation point of block matching, and the second maximum correlation point is not in contact with the excluded portion. A technique for determining the reliability of a detected motion vector is described. In Patent Document 2, the search range for block matching is divided into several regions, the maximum correlation point is detected for each region, and the reliability of the detected vector is determined based on the correlation value of each of these correlation points. Techniques for judging sex are described.

Japanese Patent Laid-Open No. 07-050815 JP 2008-035404 A

In the technique described in Patent Document 1, when there is a correlation point as high as the first maximum correlation point around the first maximum correlation point, the reliability of the detected motion vector is not determined to be low. was there. Further, in the technique described in Patent Document 2, when the maximum correlation point between the foreground motion vector and the background motion vector is in the same region of the divided search range, the reliability of the detected motion vector is low. Sometimes it was not judged. For this reason, an interpolation frame is generated based on an inappropriate motion vector, and a frame number conversion process is performed using the generated interpolation frame, so that the output video may be disturbed.

  The present invention provides a video signal processing apparatus and a video signal processing method capable of suppressing occurrence of disturbance in an output video in video processing in which an interpolation frame is inserted between two frames to increase the number of frames. For the purpose.

The video signal processing apparatus according to the present invention is
A video signal processing apparatus that generates an image of an interpolation frame to be inserted between two consecutive frames of an input video signal,
A block obtained by dividing an image of a previous frame of the two frames, and a block of a subsequent frame at a relative position represented by a shift amount within a predetermined search range with respect to the position of the block; the sum of absolute differences is the total sum of the absolute differences of each other corresponding pixel values, calculating means for calculating for each shift amount in the search range,
Based on the shift amount within the search range where the sum of absolute differences calculated by the calculating means is minimized, a motion vector representing the motion of the image between the block of the previous frame and the block of the subsequent frame is detected. Motion vector detecting means for performing,
There are a plurality of shift amounts within the search range where the sum of absolute differences calculated by the calculating means is a minimum, and the second minimum value within the search range is predetermined among the plurality of minimum values of the sum of absolute differences. A determination unit that determines that the motion vector detected by the motion vector detection unit is low in reliability when the threshold is less than or equal to
Generating means for generating an image of the interpolated frame based on a determination result by the determining means;
It is characterized by providing.

The video signal processing method according to the present invention includes:
A video signal processing method for generating an image of an interpolation frame to be inserted between two consecutive frames of an input video signal,
A step of inputting a video signal;
A block obtained by dividing an image of a previous frame of two consecutive frames of the input video signal, and a relative position represented by a shift amount within a predetermined search range with respect to the position of the block and a block of a following frame, the difference absolute value sum is the total sum of the corresponding difference absolute value of pixel values each other, a calculating step of calculating for each shift amount in the search range,
Based on the shift amount within the search range in which the sum of absolute differences calculated by the calculation step is minimized, a motion vector representing the motion of the image between the block of the previous frame and the block of the subsequent frame is detected. A motion vector detection step to perform,
There are a plurality of shift amounts within the search range where the sum of absolute differences calculated by the calculation step is a minimum, and the second minimum value within the search range is predetermined among the plurality of minimum values of the sum of absolute differences. A determination step of determining that the motion vector detected by the motion vector detection step is low in reliability when the threshold value is less than or equal to
A generation step of generating an image of the interpolation frame based on a determination result by the determination step;
It is characterized by having.

  According to the present invention, it is possible to prevent the output video from being disturbed in the process of generating the interpolation frame based on the motion vector detected from the images of two consecutive frames and increasing the number of frames. It becomes.

1 is a block diagram illustrating a schematic configuration of a frame rate conversion apparatus according to a first embodiment. The figure explaining the example of the block division of the image of the frame 0 The figure which shows the example of the search range of the motion vector of block matching The figure which shows the example of distribution of the difference absolute value sum The figure which shows the example of the production | generation method of the interpolation frame image based on a motion vector Examples of images where inappropriate motion vectors are easily detected The figure which shows the example of distribution of the difference absolute value sum The figure which shows the example of distribution of the difference absolute value sum The block diagram which shows the structure of the multiple peak detection part in Example 1. FIG. The figure which shows the example of the detection method of the peak of difference absolute value sum The block diagram which shows the structure of the multiple peak detection part in Example 2. FIG. The block diagram which shows the structure of the multiple peak detection part in Example 3. The block diagram which shows the structure of the multiple peak detection part in Example 4. FIG. The block diagram which shows the structure of the multiple peak detection part in Example 5. FIG.

Example 1
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of a video signal processing apparatus according to the present invention. This video signal processing device is a frame rate conversion device that inserts an interpolation frame between two consecutive frames of an input video signal to increase the number of frames of the video signal. Such a frame rate conversion device can display a smoother motion when, for example, a moving image such as a moving picture taken with a television signal or a video camera is displayed on a liquid crystal monitor or digital television. It can be used to process video signals. In addition to such real-time video processing at the time of video display, it can also be used for video processing for converting an existing video to a video with a high frame rate at the time of video editing. The frame rate conversion apparatus of the present embodiment can be used not only for increasing the number of frames per unit time but also for simply increasing the number of video frames.

  As shown in FIG. 1, the frame rate conversion apparatus 1 according to the present embodiment includes a block buffer 10, a block buffer 11, a difference absolute value sum calculation unit 12, a motion vector detection unit 13, a multiple peak detection unit 14, and an interpolation image generation. The unit 15 is configured. The frame rate conversion apparatus 1 receives two consecutive frame images, detects a motion vector from the two frame images, and inserts the motion vector between the previous frame and the subsequent frame based on the motion vector. Generate and output an interpolated frame image. For example, a frame memory is provided in the previous stage of the frame rate conversion apparatus 1, and an image of the previous frame held in the frame memory is input to the block buffer 10.

<Block buffer 10>
The block buffer 10 cuts out an image in a specific area of the image of the previous frame (referred to as frame 0) from two consecutive frames, and outputs it as block data 0. Hereinafter, as shown in FIG. 2, an example in which an image of frame 0 of 128 dots × 96 dots is divided into 12 rectangular blocks of 32 dots × 32 dots and each block is output as block data 0 will be described as an example. To do. As will be described later, each block data 0 is an image serving as a unit for detecting a motion vector. Hereinafter, block data 0 (hereinafter also referred to as BD0) output from the block buffer 10 is represented by BD0 using block numbers (n, m) (n = 0-3, m = 0-2) shown in FIG. _{n, m} . The pixel data at the horizontal coordinate i and the vertical coordinate j in the block data BD0 _{n, m} is represented as BD0 _{n, m} (i, j). If the image data of frame 0 is FD0 and the pixel data at coordinates (x, y) in the image data FD0 of frame 0 is represented as FD0 (x, y), the pixel at coordinates (i, j) in block data 0 Data BD0 _{n, m} (i, j) is expressed as shown in equation (1).

<Block buffer 11>
The block buffer 11 cuts out a block having the same shape as the block data 0 from the image of the subsequent frame (referred to as frame 1) of the two consecutive frames, and outputs it as block data 1. The position where the block data 1 is cut out from the image of the frame 1 is a position where the relative position with respect to the position where the block data 0 is cut out from the frame image 0 is represented by a shift amount within a predetermined range (referred to as a search range). The block data 1 is cut out from the image of the frame 1 for all shift amounts within the search range and sequentially output. The search range is determined according to the magnitude of the motion vector to be detected. For example, when the size of the motion vector to be detected is horizontal ± 24 dots and vertical ± 15 dots, the search range is defined as horizontal ± 24 dots and vertical ± 15 dots. Then, at the position where the relative position to the position where the block data 0 is cut out in the frame 0 image is represented by a shift amount within a range of ± 24 dots in the horizontal direction and ± 15 dots in the vertical direction, Block data 1 is cut out. If the block data 0 is at a position close to the edge of the image of the frame 0, the position where the corresponding block data 1 should be cut out may protrude from the frame 1, but the value of the protruding pixel is, for example, a predetermined fixed value. And In this case, the number of possible combinations of horizontal and vertical shift amounts is 1440, and 1440 pieces of block data 1 are output corresponding to one block data 0.

  FIG. 3 shows one block data 0 and the existence range of block data 1 output corresponding thereto. In relation to one block data 0 drawn in the center in FIG. 3, 1440 pieces of block data 1 whose positions are shifted by one dot in the horizontal or vertical direction within the range surrounded by the outer solid line are output. . Such block data 1 is output for each of the 12 pieces of block data 0.

Block data 1 having a shift amount (is, js) corresponding to block data 0 (BD0 _{n, m} ) is represented as BD1 _{n, m} (is, js). The pixel data at the horizontal coordinate i and the vertical coordinate j in the block data BD1 _{n, m} (is, js) is represented as BD1 _{n, m} (i, j, is, js). If the image data of frame 1 is FD1, and the pixel data at coordinates (x, y) in the image data FD1 of frame 1 is FD1 (x, y), the pixel BD1 _{n, m} (i, j) of block data 1 , Is, js) is expressed as in equation (2).

差分絶対値和計算部１２は、各ブロックデータ０について、サーチ範囲内のシフト量毎に差分絶対値和を計算する。本実施例の差分絶対値和計算部１２は、本発明における「算出手段」に相当する。これにより、各ブロックデータ０について、差分絶対値和の分布が得られる。図４は、本実施例における差分絶対値和の分布例を、シフト量に対応するｉｓ−ｊｓ平面上の各点において高さ方向に差分絶対値和の値をプロットすることによって可視化した図である。 <Absolute Difference Sum Calculation Unit 12>
The difference absolute value sum calculation unit 12 calculates the absolute value of the difference between the input block data 0, the block data 1 of the corresponding shift amount (is, js), and the corresponding pixel value, The sum of absolute differences is calculated over all pixels. This sum is called a difference absolute value sum (SAD). The sum of absolute differences between the block data 0 (BD0 _{n, m} ) and the block data 1 (BD1 _{n, m} (is, js)) of the corresponding shift amount (is, js) is SAD _{n, m} (is , Js), SAD _{n, m} (is, js) is expressed by the following equation (3).

The difference absolute value sum calculation unit 12 calculates the difference absolute value sum for each block data 0 for each shift amount within the search range. The difference absolute value sum calculation unit 12 of this embodiment corresponds to a “calculation unit” in the present invention. Thereby, the distribution of the sum of absolute differences is obtained for each block data 0. FIG. 4 is a diagram visualized by plotting the value of the sum of absolute differences in the height direction at each point on the is-js plane corresponding to the shift amount in the distribution example of the sum of absolute differences in this embodiment. is there.

<Motion vector detection unit 13>
The sum of absolute differences represents the degree of difference between block data 0 and block data 1 corresponding thereto. As the value of the sum of absolute differences SAD _{n, m} (is, js) is smaller, the block data BD0 _{n, m} and the block data BD1 _{n, m} (is, js) of the corresponding shift amount (is, js) The correlation is high. A location where the value of the sum of absolute differences is relatively small in the distribution of sum of absolute differences is referred to as a correlation point. When the amount of shift from the position of block data 0 to the position of block data 1 matches the motion vector between the image of block data 0 and the image of block data 1, the correlation between block data 0 and block data 1 is Get higher. Therefore, the motion vector detection unit 13 detects a motion vector based on the shift amount (is, js) that minimizes the sum of absolute differences. In this embodiment, the sum of absolute differences is obtained for all shift amounts (is, js) within the search range, and block matching is performed by searching for the shift amount with the highest degree of correlation between block data 0 and block data 1. Is done. The motion vector detection unit 13 of the present embodiment corresponds to “motion vector detection means” in the present invention.

  For example, when the motion between the image of frame 0 and the image of frame 1 is a simple motion such as translation, the distribution of the sum of absolute differences has a sharp peak as shown in FIG. The distribution is a distribution, and the sum of absolute differences is the minimum value at this peak. Accordingly, the motion vector detection unit 13 detects a motion vector based on the shift amount (is, js) corresponding to the peak difference absolute value sum.

<Interpolated image generation unit 15>
The interpolation image generation unit 15 generates and outputs an image of an interpolation frame to be inserted between the frame 0 image and the frame 1 image based on the motion vector input from the motion vector detection unit 13. An example of an interpolation frame image generation method will be described with reference to FIG. In the example of FIG. 5, the motion vector is (Vx, Vy), and the coordinates of the pixel P of the image of the interpolation frame to be generated are (x, y). At this time, the pixel P has a relative position with respect to the coordinate (x, y) and the relative position with respect to the coordinate (x, y) and the pixel A ₀ of the frame 0 whose relative position is (−Vx / 2, −Vy / 2). / 2, Vy / 2) and calculating the average value of the pixel A ₁ of the frame 1 image. The interpolated image generation unit 15 of the present embodiment corresponds to “generation means” in the present invention.

<Error detection of motion vectors (repeated pattern)>
By the way, in an image having a repetitive pattern as in the right region of FIG. 6, the correlation between the block data 0 and the block data 1 becomes high at a shift amount that matches the period of the repetitive pattern. Therefore, in the difference absolute value sum distribution, the difference absolute value sum is small not only in the shift amount that matches the motion vector but also in the shift amount that matches the cycle of the repetitive pattern. Therefore, a plurality of shift amounts with high correlation appear in block matching. As described above, the motion vector detection unit 13 detects a shift amount that minimizes the sum of the absolute differences as a motion vector. Therefore, if the sum of absolute differences in the shift amount corresponding to the repetition pattern period is smaller than the sum of absolute differences in the shift amount corresponding to the motion vector, the shift amount corresponding to the repetition pattern period is erroneously detected as a motion vector. End up.

  An example of the distribution of the sum of absolute differences in an image having a repeated pattern is shown in FIG. In FIG. 7, the correlation point A represents the sum of absolute differences in the shift amount corresponding to the motion vector, and the correlation point B represents the sum of absolute differences in the shift amount corresponding to the cycle of the repeated pattern. When the difference absolute value sum at the correlation point B is smaller than the difference absolute value sum at the correlation point A, the shift amount corresponding to the correlation point B is detected as a motion vector. That is, a shift amount that is irrelevant to the original image motion and corresponds to the cycle of the repetitive pattern is erroneously detected as a motion vector. Then, an image of the interpolation frame is generated based on the motion vector erroneously detected in this way, and the generated interpolation frame is inserted between frame 0 and frame 1.

  For example, assuming that the shift amount corresponding to the correlation point A, that is, the original motion vector is (0, 0), and the shift amount corresponding to the correlation point B, that is, the cycle of the repetitive pattern is (L, 0). The motion vector detected by the detection unit 13 is (L, 0). Therefore, each pixel of the image of the interpolation frame is in the relative position (L / 2, 0) with respect to the pixel of the frame 0 image in the relative position (-L / 2, 0) with respect to the coordinate. It is generated by the average value of pixels of a certain frame 1 image. However, the motion vector corresponding to the motion of the image is (0, 0). In this case, there is actually no motion in the image. Therefore, when the generated interpolated frame is inserted between the frame 0 and the frame 1, the video after the frame rate conversion is greatly disturbed.

<Error detection of motion vectors (coexistence of different motions)>
In addition, in an image in which different motions coexist in the image as in the left area of FIG. 6, the correlation between block data 0 and block data 1 is high in the shift amount corresponding to the motion vector indicating each motion. Become. Therefore, in the distribution of the sum of absolute differences, the sum of absolute differences becomes a small value in the shift amount corresponding to each motion vector. For this reason, a plurality of shift amounts with high correlation appear in block matching. Then, the shift amount having the minimum difference absolute value sum is detected as a motion vector.

  FIG. 8 shows an example of the distribution of the sum of absolute differences in an image in which the motion is different between the foreground and the background as in the left area of FIG. In FIG. 8, correlation point A represents the sum of absolute differences in the shift amount corresponding to the foreground motion vector, and correlation point B represents the sum of absolute differences in the shift amount corresponding to the background motion vector. When the difference absolute value sum at the correlation point A is smaller than the difference absolute value sum at the correlation point B, the shift amount corresponding to the correlation point A is detected as a motion vector. Then, an image of an interpolation frame is generated based on the detected motion vector, and the generated interpolation frame is inserted between frame 0 and frame 1. The video after frame rate conversion generated in this way is displayed smoothly with respect to the foreground movement, but is greatly disturbed with respect to the background.

<Multiple Peak Detection Unit 14>
A configuration unique to the video signal processing apparatus of the present embodiment for suppressing the generation of a distorted video due to frame rate conversion will be described below. The multiple peak detection unit 14 determines the reliability of the motion vector detected by the motion vector detection unit 13, and if it is determined that the reliability of the detected motion vector is low, it is based on the detected motion vector. A signal for suppressing the image generation of the interpolation frame is output. FIG. 9 shows a configuration diagram of the multiple peak detection unit 14. The multiple peak detection unit 14 includes a peak detection unit 16, a comparison unit 17, a count unit 18, and a determination unit 19.

<Peak detection unit 16>
The peak detection unit 16 receives the input of the difference absolute value sum from the difference absolute value sum calculation unit 12 and detects the minimum peak of the difference absolute value sum. An example of a method for detecting the peak of the sum of absolute differences by the peak detector 16 will be described with reference to FIG. FIG. 10 is a distribution diagram in which the sum of absolute differences at each shift amount calculated for a certain block data 0 is recorded at a corresponding position in the block matching search range and visualized. In this embodiment, the search range is −24 ≦ is ≦ + 24 and −15 ≦ js ≦ + 15, but in FIG. 10, only the ranges of −5 ≦ is ≦ + 5 and −2 ≦ js ≦ + 2 are shown enlarged for the sake of simplicity. ing.

  The peak detector 16 compares the difference absolute value sum at the shift amount of interest with the difference absolute value sum at the eight shift amounts adjacent thereto. When the difference absolute value sum in the shift amount of interest is minimum, the sum of difference absolute values is determined to be a peak. In the example shown in FIG. 10, the absolute difference value sums 10, 20, and 75 at three locations surrounded by a double line are determined to be peaks.

  Note that the peak determination method is not limited to the above-described method as long as the difference absolute value sum (referred to as a maximum correlation point) that is a minimum value in the distribution of the difference absolute value sum can be detected. For example, if the sum of absolute differences at the shift amount of interest is smaller than the sum of absolute differences of shift amounts at four locations adjacent to the shift amount in the horizontal direction or the vertical direction, the sum of absolute differences at the shift amount of interest May be determined as a peak. Further, when the sum of absolute differences in the target shift amount is smaller than the sum of absolute differences in the eight shift amounts adjacent to the shift amount and the sixteen shift amounts outside the shift amount, The sum of absolute differences may be determined as a peak.

<Comparator 17>
Even the sum of absolute differences at a shift amount with a low correlation degree of block matching may be detected as a peak by the peak detector 16. Such a low correlation degree peak can occur randomly in a region where the block matching correlation is low in the search range. Therefore, the comparison unit 17 compares the sum of absolute difference values determined as peaks by the peak detection unit 16 with a predetermined threshold value (referred to as a multiple peak detection threshold value), and determines a peak that is equal to or less than the multiple peak detection threshold value as an effective peak. judge. For example, when the multiple peak detection threshold is 50, in the example of FIG. 10, two peaks of the difference absolute value sums 10 and 20 are detected among the detected three peaks (difference absolute value sums 10, 20 and 75). It is determined as a valid peak. The comparison unit 17 extracts a peak having a certain correlation or more by block matching from the peaks detected by the peak detection unit 16. The multiple peak detection threshold in this embodiment corresponds to a “threshold” in the present invention.

<Counter 18>
The counting unit 18 counts how many effective peaks exist in the search range. When the count value is 2 or more, it is determined that the reliability of the motion vector detected by the motion vector detection unit 13 is low, and a signal indicating this is output. In the case of block data including images with repeated patterns and different motions coexisting, there is a high possibility that two or more effective peaks are detected in the search range, so the reliability of the motion vector detected by block matching is low. Will be judged.

<Determining unit 19>
The determination unit 19 determines whether or not the block data determined by the motion vector detection unit 13 as having low reliability among all the block data constituting the frame 0 image has a predetermined ratio or more. Determine. The determination unit 19 prohibits generation of an image of the interpolation frame based on the detected motion vector when the block data determined to have low reliability of the detected motion vector has a predetermined ratio or more. Output a signal.

  When the interpolated image generating unit 15 receives the prohibition signal from the determining unit 19, the interpolated image generating unit 15 generates the frame 0 image or the frame 1 image without generating an interpolated frame image based on the motion vector detected by the motion vector detecting unit 13. Output as an interpolated frame image. That is, in this case, a process for increasing the frame rate is performed by inserting a frame 0 image or a frame 1 image as an interpolation frame between frame 0 and frame 1. As a result, an interpolation frame based on a motion vector detected by block matching is not generated for an image that is likely to have an inappropriate motion vector detected by block matching. Therefore, frame rate conversion processing can be performed while suppressing disturbance of the video after frame rate conversion, even for video signals where there are regions that contain repetitive patterns or regions where multiple different motions coexist in the image. Can be performed. When there is only one effective peak within the block matching search range, the motion vector detected from the shift amount corresponding to the peak is an appropriate motion vector that correctly indicates the motion of the image between the block data. Conceivable. Accordingly, in this case, the interpolation image generation unit 15 generates an image of the interpolation frame based on the motion vector, block data 0, and block data 1 detected by the motion vector detection unit 13, as described above.

  In this embodiment, the example in which it is determined that the reliability of the detected motion vector is low when there are a plurality of effective peaks in the search range is described. However, when there are a plurality of peaks in the search range. Alternatively, it may be determined that the reliability of the detected motion vector is low.

(Example 2)
FIG. 11 is a block diagram of the multiple peak detector 14 in the second embodiment. The multiple peak detection unit 14 according to the second embodiment includes a peak detection unit 16, a comparison unit 17, a determination unit 19, a first minimum peak value memory unit 20, and a second minimum peak value memory unit 21. In the present embodiment, the reliability of the motion vector detected by the motion vector detection unit 13 based on the value of the sum of absolute differences of the peak having the second smallest absolute difference value among the peaks detected by the peak detection unit 16. Determine gender. Since the configuration other than the multiple peak detection unit 14 is the same as that of the first embodiment, the description thereof will be omitted, and only the multiple peak detection unit 14 will be described below.
The operations of the peak detection unit 16 and the determination unit 19 in the present embodiment are the same as those in the first embodiment.

<First Minimum Peak Value Memory Unit 20>
The first minimum peak value memory unit 20 includes a difference absolute value sum (first minimum difference absolute value sum) of peaks having a minimum difference absolute value sum (referred to as a first minimum peak) among all peaks existing in the search range. Remember). While the detection of the peak for each block by the peak detection unit 16 continues, the difference absolute value sum stored in the first minimum peak value memory unit 20 is the peak having the smallest difference absolute value sum among the already detected peaks. It is updated with the sum of absolute differences. When the peak detection by the peak detection unit 16 is completed, the first minimum peak value memory unit 20 stores the first minimum difference absolute value sum.

<Second Minimum Peak Value Memory Unit 21>
The second minimum peak value memory unit 21 calculates a sum of absolute differences (second minimum absolute difference) of a peak (referred to as a second minimum peak) having the second smallest absolute difference value among all peaks existing in the search range. Value sum). While the peak detection unit 16 continues to detect the peak for each block, the difference absolute value sum stored in the second minimum peak value memory unit 21 is the second smallest absolute difference sum among the already detected peaks. It is updated with the sum of absolute differences of peaks. When the peak detection by the peak detection unit 16 is completed, the second minimum peak value memory unit 21 stores the second minimum difference absolute value sum.

A specific processing flow will be described. When a new peak is detected by the peak detector 16, the sum of absolute differences SAD _new of the peak is input to the first minimum peak value memory unit 20 and the second minimum peak value memory unit 21. Then, the input difference absolute value sum SAD _new , the difference absolute value sum SAD _min1 stored in the first minimum peak value memory unit 20, and the difference absolute value stored in the second minimum peak value memory unit 21 A comparison is made with the sum SAD _min2 .
(A) When the input value is smaller than the value of the first minimum peak value memory unit 20 (SAD _new <SAD _min1 ), the value of the second minimum peak value memory unit 21 is the first minimum peak value memory unit 20 _Is updated with the value SAD _min1 . Further, the value of the first minimum peak value memory unit 20 is updated with the input value SAD _new .
(B) When the input value is equal to or larger than the value of the first minimum peak value memory unit 20 and smaller than the value of the second minimum peak value memory unit 21 (SAD _min1 ≦ SAD _new <SAD _min2 ), the second minimum The value in the peak value memory unit 21 is updated with the input value SAD _new .

  By repeating such a processing flow, the first minimum difference absolute value sum and the second minimum difference absolute value sum are obtained when the peak detection by the peak detection unit 16 is completed for a certain block. In the example of FIG. 10, the first minimum difference absolute value sum is 10 and the second minimum difference absolute value sum is 20.

<Comparator 17>
The comparison unit 17 compares the second minimum difference absolute value sum with a plurality of peak detection threshold values, and determines that the reliability of the motion vector detected by the motion vector detection unit 13 is low when the sum is equal to or less than the plurality of peak detection threshold values. And a signal indicating this is output. In the example of FIG. 10, if the second minimum difference absolute value sum is 20, and the multiple peak detection threshold is 50 as in the first embodiment, the motion vector detected by the motion vector detection unit 13 for the block of FIG. Is determined to be unreliable.
When there is no second minimum peak, it means that there is only one peak in the search range. Therefore, in this case, the comparison unit 17 determines that the motion vector detected by the motion vector detection unit 13 is reliable. When there is no first minimum peak, there is a minimum point of the sum of absolute differences at the end of the search range, which means that no peak exists in the search range. Therefore, in this case, the comparison unit 17 determines that the reliability of the motion vector detected by the motion vector detection unit 13 is low.

  According to the configuration of the second embodiment, an interpolated frame is not generated based on a motion vector detected by block matching for an image in which an inappropriate motion vector is likely to be erroneously detected. Therefore, even if a frame rate conversion process for inserting an interpolated frame between two frames is performed on a video signal including such an image, it is possible to accurately suppress the occurrence of disturbance in the converted video. It becomes possible.

(Example 3)
In the first embodiment and the second embodiment, an example in which a plurality of peak detection threshold values are input to the comparison unit 17 as fixed values has been described. For example, in a noisy image, the correlation of block matching becomes lower overall due to the influence of noise, so the distribution of sum of absolute differences has a dull peak, and the sum of absolute differences at the peak is the average value. Get closer. Therefore, in the third embodiment, the multiple peak detection threshold is adaptively set according to the image. Since the configuration other than the multiple peak detection unit 14 is the same as that of the first embodiment, the description thereof will be omitted, and only the multiple peak detection unit 14 will be described below.
FIG. 12 is a block diagram of the multiple peak detector 14 in the third embodiment. The multiple peak detection unit 14 according to the third embodiment is configured by adding a threshold value calculation unit 22 to the configuration of the multiple peak detection unit 14 according to the second embodiment. The threshold calculation unit 22 receives the input of the first minimum difference absolute value sum from the first minimum peak value memory unit 20 and calculates a plurality of peak detection thresholds by the following equation (4).
Multiple peak detection threshold = first minimum difference absolute value sum + α (α is a fixed constant) (4)
The threshold value calculation unit 22 in this embodiment corresponds to setting means in the present invention.

The comparison unit 17 compares the second minimum difference absolute value sum with the plurality of peak detection threshold values calculated by the threshold value calculation unit 22, and the motion detected by the motion vector detection unit 13 when the value is equal to or less than the plurality of peak detection threshold values. It is determined that the reliability of the vector is low, and a signal indicating this is output.
According to the configuration of the third embodiment, the multi-peak detection threshold value can be adaptively set according to the overall degree of correlation between the image of frame 0 and the image of frame 1 of the input signal. More accurate determination can be made. Therefore, it is possible to suppress the generation of an interpolated frame image based on an inappropriate motion vector, and to suppress the occurrence of disturbance in the video after the frame rate conversion.

Example 4
As a method of adaptively setting a plurality of peak detection thresholds for the overall degree of correlation between two frames as in the third embodiment, the maximum absolute difference sum (referred to as the maximum absolute difference sum) within the search range is used. A plurality of peak detection thresholds may be set accordingly.

FIG. 13 is a block diagram of the multiple peak detector 14 in the fourth embodiment. The multiple peak detection unit 14 of the fourth embodiment is configured by adding a maximum value memory unit 23 to the configuration of the multiple peak detection unit 14 of the third embodiment.
The maximum value memory unit 23 stores the maximum value among the already calculated absolute difference sums while the calculation of the absolute difference sum is continued in the block matching for each block. A specific processing flow is as follows. When the value of the newly inputted difference absolute value sum is larger than the value stored in the maximum value memory unit 23, the value of the maximum value memory unit 23 is set by the inputted difference absolute value sum. Update.
The threshold value calculation unit 22 receives an input of the maximum difference absolute value sum from the maximum value memory unit 23 and an input of the first minimum difference absolute value sum from the first minimum peak value memory unit 20, and the following equation (5) The multiple peak detection threshold is calculated using a calculation formula.
Multiple peak detection threshold = first minimum difference absolute value sum + (maximum difference absolute value sum−first minimum difference absolute value sum) × α (α is a value of 0 <α <1) (5)
The threshold value calculation unit 22 in this embodiment corresponds to setting means in the present invention.

The comparison unit 17 compares the second minimum difference absolute value sum with the plurality of peak detection threshold values calculated by the threshold value calculation unit 22, and the motion detected by the motion vector detection unit 13 when the value is equal to or less than the plurality of peak detection threshold values. It is determined that the reliability of the vector is low, and a signal indicating this is output.
According to the configuration of the fourth embodiment, the multiple peak detection threshold can be adaptively set according to the overall degree of correlation between the frame 0 image and the frame 1 image of the input signal. More accurate determination can be made. Therefore, it is possible to suppress the generation of an interpolated frame image based on an inappropriate motion vector, and to suppress the occurrence of disturbance in the video after the frame rate conversion.

(Example 5)
As a method of adaptively setting a plurality of peak detection threshold values according to the overall degree of correlation between two frames as in the third embodiment, the average value of the sum of absolute differences within the search range (referred to as the sum of absolute differences) A plurality of peak detection thresholds may be set according to the above.

FIG. 14 is a block diagram of the multiple peak detector 14 in the fifth embodiment. The multiple peak detection unit 14 of the fifth embodiment is configured by adding an average value calculation unit 24 to the configuration of the multiple peak detection unit 14 of the third embodiment.
The average value calculation unit 24 calculates and outputs the average value of the sum of absolute differences within the search range.
The threshold value calculation unit 22 receives the input of the absolute difference sum average from the average value calculation unit 24 and the input of the first absolute minimum sum of absolute values from the first minimum peak value memory unit 20, and the following equation (6) The multiple peak detection threshold is calculated using a calculation formula.
Multiple peak detection threshold = first minimum difference absolute value sum + (difference absolute value sum average−first minimum difference absolute value sum) × α (α is a value of 0 <α <1) (6)
The threshold value calculation unit 22 in this embodiment corresponds to setting means in the present invention.

The comparison unit 17 compares the second minimum difference absolute value sum with the plurality of peak detection threshold values calculated by the threshold value calculation unit 22, and the motion detected by the motion vector detection unit 13 when the value is equal to or less than the plurality of peak detection threshold values. It is determined that the reliability of the vector is low, and a signal indicating this is output.
According to the configuration of the fifth embodiment, the multi-peak detection threshold value can be adaptively set according to the overall degree of correlation between the frame 0 image and the frame 1 image of the input signal. More accurate determination can be made. Therefore, it is possible to suppress the generation of an interpolated frame image based on an inappropriate motion vector, and to suppress the occurrence of disturbance in the video after the frame rate conversion.

10: block buffer, 11: block buffer, 12: absolute difference sum calculator, 13: motion vector detector, 14: multiple peak detector, 15: interpolated image generator

Claims (10)

A video signal processing apparatus that generates an image of an interpolation frame to be inserted between two consecutive frames of an input video signal,
A block obtained by dividing an image of a previous frame of the two frames, and a block of a subsequent frame at a relative position represented by a shift amount within a predetermined search range with respect to the position of the block; A calculating means for calculating a sum of absolute differences that is a sum of absolute differences between corresponding pixel values for each shift amount in the search range;
Based on the shift amount within the search range where the sum of absolute differences calculated by the calculating means is minimized, a motion vector representing the motion of the image between the block of the previous frame and the block of the subsequent frame is detected. Motion vector detecting means for performing,
There are a plurality of shift amounts within the search range where the sum of absolute differences calculated by the calculating means is a minimum, and the second minimum value within the search range is predetermined among the plurality of minimum values of the sum of absolute differences. A determination unit that determines that the motion vector detected by the motion vector detection unit is low in reliability when the threshold is less than or equal to
Generating means for generating an image of the interpolated frame based on a determination result by the determining means;
A video signal processing apparatus comprising:   When the generation unit includes a predetermined percentage or more of blocks determined to have low motion vector reliability among a plurality of blocks constituting the previous frame, either the image of the previous frame or the image of the subsequent frame The image of the interpolation frame is generated. Otherwise, the image of the interpolation frame is generated based on the motion vector detected by the motion vector detection means, the image of the previous frame, and the image of the subsequent frame. The video signal processing apparatus according to claim 1. A value obtained from the minimum value within the search range of the minimum value of the sum of absolute differences calculated by the calculating means and a predetermined fixed value as a threshold value = minimum value + fixed value The video signal processing apparatus according to claim 1, further comprising a setting unit configured to set to Based on the maximum value within the search range of the difference absolute value sum calculated by the calculation means and the minimum value within the search range of the minimum value of the difference absolute value sum calculated by the calculation means The threshold is threshold = minimum value + (maximum value−minimum value) × α (α is a value of 0 <α <1).
3. The video signal processing apparatus according to claim 1, further comprising setting means for setting to a value obtained by the above. Based on an average value in the search range of the sum of absolute differences calculated by the calculation means and a minimum value in the search range of a minimum value of the sum of absolute differences calculated by the calculation means The threshold is threshold = minimum value + (average value−minimum value) × α (α is a value of 0 <α <1).
3. The video signal processing apparatus according to claim 1, further comprising setting means for setting to a value obtained by the above. A video signal processing method for generating an image of an interpolation frame to be inserted between two consecutive frames of an input video signal,
A step of inputting a video signal;
A block obtained by dividing an image of a previous frame of two consecutive frames of the input video signal, and a relative position represented by a shift amount within a predetermined search range with respect to the position of the block A calculation step of calculating a difference absolute value sum that is a sum of absolute difference values of corresponding pixel values of a block in a certain subsequent frame for each shift amount in the search range;
Based on the shift amount within the search range in which the sum of absolute differences calculated by the calculation step is minimized, a motion vector representing the motion of the image between the block of the previous frame and the block of the subsequent frame is detected. A motion vector detection step to perform,
There are a plurality of shift amounts within the search range where the sum of absolute differences calculated by the calculation step is a minimum, and the second minimum value within the search range is predetermined among the plurality of minimum values of the sum of absolute differences. A determination step of determining that the motion vector detected by the motion vector detection step is low in reliability when the threshold value is less than or equal to
A generation step of generating an image of the interpolation frame based on a determination result by the determination step;
A video signal processing method comprising: In the generating step, when a plurality of blocks constituting the previous frame include a predetermined percentage or more of blocks determined to have low motion vector reliability, either the previous frame image or the subsequent frame image The image of the interpolation frame is generated. Otherwise, the image of the interpolation frame is generated based on the motion vector detected by the motion vector detection step, the image of the previous frame, and the image of the subsequent frame. The video signal processing method according to claim 6 . A value obtained by the threshold value = minimum value + fixed value based on the minimum value within the search range of the minimum value of the sum of absolute differences calculated by the calculating step and a predetermined fixed value. The video signal processing method according to claim 6, further comprising a setting step of setting to. Based on the maximum value within the search range of the difference absolute value sum calculated by the calculation step and the minimum value within the search range of the minimum value of the difference absolute value sum calculated by the calculation step The threshold is threshold = minimum value + (maximum value−minimum value) × α (α is a value of 0 <α <1).
The video signal processing method according to claim 6 or 7, further comprising a setting step of setting to a value obtained by the above. Based on the average value within the search range of the difference absolute value sum calculated by the calculation step, and the minimum value within the search range of the minimum value of the difference absolute value sum calculated by the calculation step The threshold is threshold = minimum value + (average value−minimum value) × α (α is a value of 0 <α <1).
The video signal processing method according to claim 6 or 7, further comprising a setting step of setting to a value obtained by the above.

Images