JP4312219B2 - Frame interpolation device, frame frequency conversion device, and frame interpolation method - Google Patents

Description

The present invention relates to a frame interpolation apparatus and a frame interpolation method for generating an image of an interpolated frame existing between two frames using two frames of images included in a video signal. The invention also about the frame frequency conversion equipment provided with a frame interpolation apparatus.

  A hold-type display such as a liquid crystal display continues to display the same image for one frame period, and when the object in the image moves, the tracking of the human eye to the moving object moves continuously, There is a problem that the edge portion looks blurry because the object moves discontinuously in units of one frame. On the other hand, it is conceivable to increase the number of display frames by interpolating frames to make the object move smoothly.

  In addition, for materials in which film images such as movies are converted into television signals, two (2 fields) or 3 (fields) are created from the same frame because of the difference in frame frequency between the two (film images and television signals). If the image is displayed as it is, there is a problem that the motion is blurred or judder with a jerky motion occurs.

  Similarly, a material obtained by converting computer-processed video into a TV signal is an image in which two (2 fields) are made from the same frame.

  The conventional frame interpolation apparatus and frame interpolation method is a zero hold method in which interpolation is performed with the same image as the previous frame with respect to the interpolated frame, or between the previous frame image and the next frame image with respect to the interpolated frame. Although there is an average value interpolation method for interpolating with an average image, the zero hold method does not move smoothly with respect to an image moving in a fixed direction, so the problem of blurring of the hold type display is still not solved. Further, the average value interpolation method has a problem that a moving image becomes a double image.

  As an improvement measure, an image is divided into a plurality of pixel blocks, a frame one frame before the interpolation frame is used as a first reference frame, and a frame one frame after the interpolation frame is used as a second reference frame. Find a pixel block having the highest correlation with the pixel block in the first reference frame from the second reference frame, and interpolate an image of the pixel block of the interpolation frame from the found pixel block pair (for example, Patent Reference 1). In this method, since an interpolation frame is generated in units of pixel blocks, data read / write in a memory (mainly SDRAM is used) necessary for processing becomes complicated, and hardware implementation is difficult. There is.

  As another improvement measure, from the pixel having the largest correlation between the pixel on the first reference frame and the pixel on the second reference frame that is point-symmetric with respect to the interpolation pixel of the interpolation frame. Some generate an interpolation pixel of an interpolation frame (see, for example, Patent Document 2). In this method, because correlation is detected in units of pixels, it may be detected that the correlation between pixels is large even though the contents of the image are different, and an interpolated image may not be obtained correctly.

JP 2004-357215 A (page 9, FIG. 16) Japanese Patent Laying-Open No. 2006-129181 (page 8, FIG. 3)

  The conventional frame interpolating apparatus is configured as described above, and there is a problem that motion is blurred or judder in which motion is jerky occurs. In addition, in the method of generating an interpolation frame in units of blocks, there is a problem that data read / write with respect to the memory becomes complicated and hardware implementation is difficult. In addition, there is a problem that the correlation cannot be detected correctly in the method of detecting the correlation in pixel units.

The frame interpolation apparatus of the present invention
A frame interpolation device for generating a pixel value of an interpolation pixel in an interpolation frame between a first reference frame and a second reference frame,
One of the pixel block pair consisting of the pixel block of the first reference frame and the pixel block of the second reference frame that is point-symmetric with respect to the position of the interpolation pixel, and the pixel block pair Extraction means for extracting a plurality of pixel block pairs each replaced by a pixel block shifted by one pixel;
For each of a plurality of pixel block pairs extracted by the extraction means, similarity calculation means for calculating the similarity between two pixel blocks based on the pixel values of the pixels in the two pixel blocks;
Reference pixel selection means for selecting, as a reference pixel, a pixel located at the center of each pixel block constituting a pixel block pair that has the largest similarity calculated by the similarity calculation means or a correction of the similarity;
You comprising the interpolation means for generating the interpolated pixel based on reference pixels selected by the reference pixel selecting means.

  According to the present invention, the position of the pixel block is located at the center of each of the pair of pixel blocks having the highest degree of similarity and thus having a mutual distance closest to the amount of image motion between the first reference frame and the second reference frame. Since the pixel value of the interpolation pixel can be determined based on the pixel value of the pixel to be interpolated, the interpolation can be performed accurately.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Embodiment 1 FIG.
FIG. 1 is a block diagram showing a frame interpolation apparatus according to Embodiment 1 of the present invention. The illustrated frame interpolation apparatus includes an input terminal 1, a delay unit 2, an extraction unit 3, a similarity calculation unit 4, a reference pixel selection unit 5, a symmetric position interpolation unit 6, and an output terminal 7.

  The video signal input from the input terminal 1 is input to the delay unit 2, delayed by one frame period, and input to one of the extraction units 3 as a first reference frame. The video signal input to the input terminal 1 is input to the other input of the extraction unit 3 as a second reference frame. The extraction unit 3 extracts a plurality of pairs of pixel blocks that are substantially point-symmetrical with respect to the interpolation pixel from the first reference frame and the second reference frame, and are composed of the same number of pixels.

  As shown in FIG. 2, the pixels in each frame are arranged in a matrix in the frame F (on the screen constituting the frame). That is, they are aligned with each other in the horizontal direction (main scanning direction) and the vertical direction (sub-scanning direction). The position of each pixel in the frame F is represented by (h, v) by a horizontal coordinate value h and a vertical coordinate value v. The coordinate origin is at the leftmost and uppermost position of the frame, the horizontal coordinate value h increases rightward by one for each pixel interval, and the vertical coordinate value v decreases downward. Assume that the value increases by 1 for each interval.

  When generating an interpolation pixel at an arbitrary position (h, v) in the interpolation frame, as the pixel block BLa in the first reference frame and the pixel block BLb in the second reference frame, for example, the center position thereof is in the horizontal direction. Is extracted in the range from (h-2) to (h + 2), and in the vertical direction is extracted from (v-2) to (v + 2). There are 15 pixel blocks satisfying such conditions in each of the first reference frame and the second reference frame. Each pixel block is composed of, for example, pixels of 3 rows and 5 columns.

  Such pixel blocks are illustrated in FIGS. 3A to 3D and FIGS. 4A to 4D. Among them, what are shown in FIGS. 3A to 3D are pixel blocks BLa in the first reference frame, and those shown in FIGS. 4A to 4D are those in the second reference frame. This is a pixel block BLb.

  The pixel block BLa (1) shown in FIG. 3 (a) and the pixel block BLb (1) shown in FIG. 4 (a) constitute a pixel block pair composed of pixel blocks located at point symmetry with respect to the interpolation pixel. is doing. Similarly, a pixel block consisting of a pixel block in which the pixel block BLa (2) shown in FIG. 3B and the pixel block BLb (2) shown in FIG. Make up a pair. The pixel block BLa (3) shown in FIG. 3 (c) and the pixel block BLb (3) shown in FIG. 4 (c) constitute a pixel block pair composed of pixel blocks located at point symmetry with respect to the interpolation pixel. is doing. Similarly, a pixel block composed of pixel blocks BLa (15) shown in FIG. 3D and pixel blocks BLb (15) shown in FIG. Make up a pair.

  In this way, the extraction unit 3 sequentially extracts pixel block pairs composed of two pixel blocks located at point-symmetric positions with the interpolation pixel as the center.

  The extraction means 3 further includes not only a pair of pixel blocks located at point symmetry with respect to the interpolated pixel as shown in FIGS. 3A to 3D but also one of them to the left of one pixel. Also extracted are those replaced with pixel blocks shifted to. For example, the pixel block BLb (1) shown in FIG. 4A is shifted to the left by one pixel as shown in FIG. 5A, and the pixel block BLb (1) shown in FIG. A pixel block pair consisting of a block BLa (1) and a pixel block BLb in which the pixel block BLb (2) shown in FIG. 4B is shifted to the left by one pixel as shown in FIG. 5B. FIG. 5C shows a pixel block pair composed of (2) and the pixel block BLa (2) shown in FIG. 3B. Similarly, the pixel block BLb (3) shown in FIG. As shown in FIG. 4, a pixel block pair composed of the pixel block BLb (3) shifted to the left by one pixel and the pixel block BLa (3) shown in FIG. The pixel block BLb (15) shown is shown in FIG. A pixel block BLb (15) which is shifted by one pixel left so that, to extract the composed pixel block pair out with pixel block BLa (15) shown in Figure 3 (d).

  The pixel blocks illustrated in FIGS. 3A to 3D and the pixel blocks illustrated in FIGS. 4A to 4D are horizontally aligned with each other (between the pixels corresponding to each other inside them). There is a difference (distance) in the position of an even number of pixels, that is, 4 pixels, 2 pixels, and 0 pixels, but the pixel blocks shown in FIGS. 3A to 3D and FIGS. The pixel blocks shown in (1) have an odd number of pixels in the horizontal direction, that is, a difference (distance) of 3 pixels and 1 pixel.

  The extraction means 3 further includes pixels constituting a pair of pixel blocks located at point-symmetrical positions with respect to the interpolation pixel as illustrated in FIGS. 3A to 3D and FIGS. 3A to 3D. A block obtained by replacing one of the blocks with a pixel block shifted upward by one pixel is also extracted. For example, a pixel block pair composed of a pixel block obtained by shifting the pixel block shown in FIG. 4A by one pixel and a pixel block shown in FIG. 3A is extracted.

  The extraction means 3 further constitutes a pair of pixel blocks located at point symmetry with respect to the interpolation pixel as illustrated in FIGS. 3 (a) to 3 (d) and FIGS. 3 (a) to 3 (d). A pixel block in which one pixel block is replaced with a pixel block shifted to the left by one pixel and upward by one pixel (that is, obliquely to the upper left) is also extracted. For example, a pixel block pair composed of a pixel block obtained by shifting the pixel block shown in FIGS. 4A to 4D diagonally to the upper left by one pixel and a pixel block shown in FIGS. 3A to 3D. To extract.

  The pixel block pair extracted by the extraction unit 3 is input to the similarity calculation unit 4 and the similarity between the pixel blocks is calculated for each pixel block pair. For example, when calculating the similarity between the pixel block BLa in FIG. 3A and the pixel block BLb in FIG. 4A, the similarity calculation unit 4 calculates the pixels a1 to 1 in the pixel block BLa in FIG. The sum of absolute differences between a15 and the pixels b1 to b15 (pixels at the positions corresponding to the pixels a1 to a15 (the same position in the pixel block)) in the pixel block BLb of FIG. (Referred to as SAD) is calculated as the similarity according to the following equation.

  Similarly, the sum SAD of absolute difference values is calculated for other pixel block pairs, and the calculation result is output to the reference pixel selection means 5 as the similarity of each pixel block pair. Here, the smaller the sum SAD value, the greater the similarity, and the greater the sum SAD, the smaller the similarity.

  The reference pixel selection means 5 selects a pixel (center pixel) located at the center of each pixel block constituting the pixel block pair having the smallest sum SAD value of each pixel block pair as a reference pixel, and symmetrical position interpolation means 6 is supplied. Since the selection of the reference pixel is also the selection of the direction of the reference pixel with respect to the interpolation pixel, it also has the meaning of selecting the interpolation direction.

Here, the pixel located at the center of each pixel block (center pixel) refers to a pixel located in a central row and a central column constituting each pixel block. For example, when the pixel block is composed of pixels of 3 rows and 5 columns, it means the pixels of the second row and the third column. Generally, each pixel block is composed of pixels of n rows and m columns, and when n and m are odd numbers, the pixels of the (n + 1) / 2 rows and the (m + 1) / 2 columns are referred to.
When n and m are even numbers, the pixel in the n / 2th row or the (n / 2) + 1th row, the m / 2th column or the (m / 2) + 1th column is handled as the central pixel. That is, the center of the pixel block is not limited to the accurate center position, but may be a position near the accurate center position.
In order to uniquely determine the center position, it is desirable that n and m are odd numbers.

The symmetric position interpolation unit 6 calculates an average value of the pixel values of the reference pixels selected by the reference pixel selection unit 5 out of the pixel block pairs output from the extraction unit 3, and calculates the calculated result as an interpolation pixel of the interpolation pixel. Are output to the output terminal 7 as pixel values.
By performing the above operation on all the interpolation pixels of the interpolation frame, an interpolation image of the interpolation frame is obtained.

In the above example, the extracting unit 3 uses the pixel block pair (a) that is point-symmetric with respect to the interpolation pixel and one reference frame such as the second reference frame for such a pixel block pair. A pixel block pair (b) in which the inner pixel block is shifted to the left by one pixel (b), a pixel block pair (c) that is shifted by one pixel, and a pixel block pair (d) that is shifted by one pixel diagonally to the upper left are extracted. However, the present invention is not limited to this, and in order to reduce the scale of hardware, in addition to the pixel block pair (a), a pixel block pair (b) in which the pixel block in one reference frame is shifted to the left by one pixel. ) Only, or only the pixel block pair (c) shifted up by one pixel, or only the pixel block pair (d) shifted by one pixel diagonally to the upper left may be extracted.
Also, instead of “one pixel left”, one shifted to “one pixel right” may be used, and instead of “one pixel up”, one shifted “one pixel down” may be used, Instead of “one pixel left diagonally upper”, one shifted “one pixel right diagonally downward”, “one pixel left diagonally downward”, or “one pixel right diagonally upward” may be used.
Furthermore, a pixel block pair in which the pixel block in the first reference frame is shifted may be used instead of the pixel block pair in which the pixel block in the second reference frame is shifted.
Further, in addition to the pixel block pair composed of pixel blocks that are point-symmetric with respect to the interpolation pixel, one pixel block of the pixel block pair composed of pixel blocks that are point-symmetric with respect to the interpolation pixel is one pixel. Instead of using a pixel block pair replaced with a shifted one, a pixel block in a point-symmetrical position with respect to the interpolation pixel is used instead of a pixel block pair consisting of a pixel block in a point-symmetrical position with respect to the interpolation pixel. It is also possible to use only a pixel block pair in which one pixel block of the pixel block pair consisting of is replaced by one pixel block shifted.

  As described above, when extracting a block pair in which one pixel block of a pixel block pair composed of pixel blocks located symmetrically with respect to the interpolation pixel is shifted to one pixel, the first reference is made. Even when there is an odd number of pixels in the shifted direction between the frame and the second reference frame (the image of the subject has moved by an odd number of pixels), a pair of distances that are closest to the amount of movement A pixel located at the center of each pixel block can be selected as a reference pixel, and there is an effect that interpolation can be performed more accurately.

  For example, when extracting a block pair in which one pixel block of a pixel block pair composed of pixel blocks located symmetrically with respect to the interpolation pixel is shifted to the left by one pixel or to the right by one pixel, Even when there is an odd number of pixels in the horizontal direction between one reference frame and the second reference frame (the subject image has moved by an odd number of pixels), a pair of distances that are closest to the amount of movement A pixel located at the center of each pixel block can be selected as a reference pixel, and there is an effect that interpolation can be performed more accurately.

  Similarly, when a block pair in which one block of a pixel block pair composed of pixel blocks located symmetrically with respect to the interpolation pixel is shifted one pixel up or one pixel down is extracted, A pair having a mutual distance closest to the amount of movement even when there is an odd number of pixels in the vertical direction between the first reference frame and the second reference frame (the subject image has moved by an odd number of pixels). The pixel located in the center of each pixel block can be selected as a reference pixel, and there is an effect that interpolation can be performed more accurately.

  Similarly, one block of a pixel block pair consisting of pixel blocks that are point-symmetric with respect to the interpolation pixel is shifted one block diagonally to the upper left or one pixel diagonally to the lower right, or one pixel diagonally to the lower left. When a block pair shifted one pixel diagonally to the upper right is extracted, there is an odd pixel movement in the diagonal direction between the first reference frame and the second reference frame (the subject Even when the image moves by an odd number of pixels), the pixel located at the center of each of the pair of pixel blocks having the mutual distance closest to the amount of motion can be selected as the reference pixel, and interpolation is performed more accurately. There is an effect that can be.

  On the other hand, when a pixel block pair composed of pixel blocks that are point-symmetric with respect to the interpolation pixel is extracted, there is even-numbered pixel movement between the first reference frame and the second reference frame. When the subject image has moved by an even number of pixels, the pixel located at the center of each of the pair of pixel blocks having the mutual distance closest to the amount of motion can be selected as a reference pixel, and interpolation There is an effect that it can be performed accurately.

  In addition to the pixel block pair consisting of pixel blocks that are point-symmetric with respect to the interpolation pixel, one pixel block of the pixel block pair consisting of pixel blocks that are point-symmetric with respect to the interpolation pixel is shifted to one pixel. When the block pair is extracted, even if the movement in the shifted direction between the first reference frame and the second reference frame is for odd pixels or even pixels. Even if there is even-numbered pixel movement, a pixel located at the center of each of a pair of pixel blocks having the mutual distance closest to the amount of movement can be selected as a reference pixel, and interpolation can be performed more accurately. it can.

  When it is determined that a block pair obtained by shifting one pixel block of a pixel block pair consisting of pixel blocks located symmetrically with respect to the interpolation pixel to one pixel has the maximum similarity, In addition to these pixels, a pixel shifted by one pixel may be selected as a reference pixel. For example, when it is determined that a block pair obtained by shifting one pixel block of a pixel block pair composed of pixel blocks located symmetrically with respect to an interpolation pixel to the left by one pixel has the maximum similarity, It is possible to select not only the center pixel but also a pixel adjacent to the right side of the center pixel as a reference pixel, obtain an average of the four selected reference pixels, and use this average as the pixel value of the interpolation pixel. good.

  The symmetrical position interpolating means 6 uses the average value of the pixels located at the center of the pixel blocks constituting the pixel block pair as the pixel value of the interpolated pixel, but is located at the center of either one of the pixel block pairs. The pixel value of the pixel to be used may be used as the pixel value of the interpolation pixel.

  Further, as shown in FIG. 2, the similarity calculation unit 4 calculates the total SAD for all the pixels (a1 to a15 and b1 to b15) in the pixel block, but in order to reduce the hardware scale, The sum of absolute values of the differences between the pixel values of the pixel after performing equal thinning processing on the pixels in one pixel block, for example, the pixel block BLa, and the pixel at the corresponding position in the other pixel block BLb is obtained. It's also good. For example, in the block BLa, pixels (n is an integer of 2 or more) are extracted in the horizontal direction, and m pixels (m is an integer of 2 or more) in the vertical direction. It is also possible to extract each pixel (one for every m), find the absolute difference value of the pixel values of these pixels and the corresponding pixels in the block BLb, and find the sum of them. An example of the thinning-out process when m and n are 2 is shown in FIGS. In the example shown in the figure, every other pixel in the vertical direction and the horizontal direction, and only the pixels arranged in a checkered pattern (shown by hatching) are extracted, and the absolute difference value is obtained only for these pixels, Find the sum SAD of them.

  As described above, in the first embodiment, a pixel block pair including the pixel block BLa of the first reference frame and the pixel block BLb of the second reference frame that is substantially point-symmetrical with respect to the position of the interpolation pixel. A plurality of extraction means 3 for extracting, and a similarity calculation means 4 for calculating the similarity of two pixel blocks based on the pixel values of the pixels in the two pixel blocks, for each of a plurality of extracted pixel block pairs; A reference pixel selection unit 5 that selects, as a reference pixel, a pixel located at the center of a pair of pixel blocks having the largest similarity calculated by the similarity calculation unit 4 and a reference pixel selected by the reference pixel selection unit 5 are used. Since the symmetric position interpolation means 6 for generating the interpolation pixel is provided, the similarity is the highest, so that the image between the first reference frame and the second reference frame is the same. Since the pixel value of the interpolated pixel can be determined based on the pixel value of the pixel located at the center of each of the pair of pixel blocks having the mutual distance closest to the amount of movement, the interpolation can be performed accurately. it can.

  Further, a sum of absolute values of differences between pixel values of at least some of the pixels in one pixel block constituting the pixel block pair and corresponding pixels in the other pixel block is output as similarity. Therefore, there is an effect that the similarity can be obtained easily and accurately, and a more accurate interpolation image can be obtained.

  In this case, the sum of the absolute values of the differences between the pixel values of one pixel block constituting the pixel block pair and the pixel value at the corresponding position in the other pixel block is output as the similarity. Thus, the similarity can be obtained more accurately, and a more accurate interpolation image can be obtained.

  On the other hand, the sum of the absolute values of the difference between the pixel values of the pixels after the pixels of one pixel block constituting the pixel block pair and the pixels at the corresponding positions in the other pixel block are obtained. Is output as the similarity, the similarity can be calculated with a relatively small hardware scale.

Embodiment 2. FIG.
In the first embodiment, the similarity calculation result of the similarity calculation unit 4 is similarly evaluated for any pixel block pair and used to select the reference pixel. However, the pixel block pair farther from the position of the interpolation pixel is used. Since the reliability tends to be low, there is a possibility that an interpolation pixel is erroneously generated. In the second embodiment, the degree of similarity is corrected to a smaller value for a pixel block pair farther from the position of the interpolated pixel (correction is performed by multiplying the sum SAD of absolute difference values by a larger coefficient). By using the corrected similarity as a selection index, the reference pixel is selected more appropriately and the reliability of the generation of the pixel value of the interpolation pixel is increased.

  FIG. 8 shows a block diagram of a frame interpolation apparatus according to the second embodiment. The illustrated frame interpolating apparatus is generally the same as the frame interpolating apparatus of FIG. 1, but differs in that similarity correction means 13 is added. Components having the same reference numerals as those in FIG. 1 operate in the same manner as in the first embodiment.

  The similarity correction unit 13 sets the distance between the interpolation pixel and the pixel (center pixel) located at the center of the pixel block with respect to the sum SAD of the absolute difference values of each pixel block pair calculated by the similarity calculation unit 4. Accordingly, SAD × K is calculated for the sum SAD using a correction coefficient K as shown in FIG. 9 and the calculation result is output to the illumination pixel selection means 5 as a selection index SI. Since the larger the sum SAD means that the similarity is smaller, the larger the coefficient K applied to the sum SAD is, the larger the selection index SI is, and the similarity is corrected to a smaller value (the corrected similarity is more It means that it is a small value).

  For example, when the distance between the interpolated pixel and the central pixel of each of the pair of pixel blocks used for similarity calculation is 1 pixel or less, K is 1, and for 2 pixels or more, K is 1.5. In the range from 1 pixel to 2 pixels, K is gradually increased from 1 to 1.5.

  As the distance between the interpolated pixel and the central pixel of each of the pair of pixel blocks used for similarity calculation, for example, the longer distance in the horizontal direction and the longer distance in the vertical direction are used. In this case, in the example described with reference to FIGS. 3A to 3D and FIGS. 4A to 4D, the horizontal distance between the interpolated pixel and the center pixel of the farthest pixel block is 2. When the pixel block (the distance in the vertical direction is one pixel) and the two pixel blocks constituting the pixel block pair are “farthest pixel blocks”, the similarity of the pixel block pair used for calculating the similarity The coefficient K with respect to is 1.5.

  When the distance from the center pixel to the interpolated pixel is different between one pixel block and the other pixel block of the pixel block pair used for calculating the position of the interpolated pixel and the similarity, the average is used. For example, as shown in FIGS. 3A and 5A, the horizontal distance from the center pixel of one pixel block of the pixel block pair used for calculating the similarity to the interpolation pixel is 2 pixels ( When the distance in the vertical direction is 1 pixel) and the horizontal distance from the center pixel of the other pixel block to the interpolation pixel is 1 pixel (the vertical distance is also 1 pixel), the average of the distance is 1.5 pixels and the correction coefficient K is 1.25.

  Based on the similarity correction unit 13, the reference pixel selection unit 5 determines the center pixel of a pair of pixel blocks having the highest corrected similarity for each interpolation pixel (the value of the selection index SI = SAD × K is the smallest). It selects as a reference pixel and outputs it to the symmetrical position interpolation means 6. Thereafter, the pixel value of the interpolation pixel is calculated as in the first embodiment, and an image of the interpolation frame is obtained.

  As described above, in the second embodiment shown in FIG. 8, the selection for generating the selection index SI used for the selection of the reference pixel in the reference pixel selection unit 5 by the similarity calculation unit 4 and the similarity correction unit 13 is selected. An index generating means 32 is configured. On the other hand, in the first embodiment shown in FIG. 1, it can be seen that the output itself of the similarity calculation means 4 is the selection index SI, and that the selection index generation means is constituted only by the similarity selection means 4. it can.

In the above example, the longer of the horizontal distance and the vertical distance is used for determining the correction coefficient as the distance between the center pixel and the interpolation pixel of the pixel block. Instead, the horizontal distance and the vertical distance are used instead. The average of the distance in the direction may be used as the distance between the pixel blocks, and the square root of the sum of the square of the horizontal distance and the square of the vertical distance, that is,
{(Horizontal distance) ² + (vertical distance) ² } ^1/2
May be used as the distance between the center pixel of the pixel block and the interpolated pixel.
If the distance from the center pixel to the interpolated pixel is different between one pixel block and the other pixel block of the pixel block pair used for calculating the position of the interpolated pixel and the similarity, the average is used. However, the longer one may be used instead, and the shorter one may be used.

  As in the above embodiment, the pixel block pair whose center position is far from the interpolated pixel is corrected so that the similarity is smaller, and the corrected similarity is used as a selection index. Therefore, there is an effect that a more accurate interpolation image can be obtained.

Embodiment 3 FIG.
The similarity calculation unit 4 according to the first embodiment is a flat image (an image with little change in density between pixels over a wide range of the screen) or an image with a change in the calculation process of the similarity between pixel block pairs. Although the evaluation was performed in the same manner, the flatness of the image tends to increase the degree of similarity. Therefore, there is a possibility that the selection of the reference pixel is not appropriate. In the third embodiment, a pixel block pair including a flat image is evaluated with a lower similarity, and a pixel block pair including a changed image is evaluated with a higher similarity. Therefore, each pixel block is determined according to the degree of image variation. By correcting the similarity of a pair and using the corrected similarity as a selection index, the reliability of selection of a reference pixel is increased.

FIG. 10 shows a frame interpolation apparatus according to the third embodiment. The illustrated interpolation apparatus is generally the same as that shown in FIG. 1 except that a correction control means 27 and a similarity correction means 28 are added. Components having the same reference numerals as those in FIG. 1 operate in the same manner as in the first embodiment.
In the embodiment of FIG. 10, the selection index generation unit 32 is configured by the similarity calculation unit 4, the correction control unit 27, and the similarity correction unit 28.

  The correction control unit 27 includes a horizontal direction variation degree calculation unit 27a, a vertical direction variation degree calculation unit 27b, an addition unit 27c, and a correction coefficient generation unit 27d.

  The variation degree calculation means 27a calculates the variation degree for each pixel block pair. A method of calculating the degree of variation will be described by taking the pixel block pair in FIGS. 3A and 4A as an example. As the degree of variation in the horizontal direction, calculation is performed according to the following equation (2).

The above equation is a function indicating a larger value as the change in the horizontal direction is larger, and a smaller value as the change in the horizontal direction is smaller.
The vertical variation degree calculation means 27b further calculates the vertical variation degree by the following equation (3).

  The above equation is a function indicating a larger value as the change in the vertical direction is larger, and a smaller value as the change in the vertical direction is smaller.

The adding means 27c adds the horizontal variation degree calculation result CH and the vertical variation degree calculation result CV, and outputs the addition result as a total variation degree CT.
The correction coefficient generation unit 27d outputs a correction coefficient D according to the total variation degree CT. The relationship between the overall variation degree CT and the correction coefficient D is as shown in FIG. 11, for example. Above the first predetermined value Tha, the correction coefficient D is 1, and the second smaller than the first predetermined value. Below the predetermined value Thb, the correction coefficient D is 2, and in the range between the first predetermined value Tha and the second predetermined value Thb, the correction coefficient D increases from 1 to 2 as the overall variation degree CT decreases. Increasing gradually.
The correction coefficient generation means 27d may be constituted by a conversion table, for example, or may be constituted by a calculation means.
The correction coefficient D output from the correction coefficient generation unit 27 d is supplied to the similarity correction unit 28 as an output of the correction control unit 27.

The similarity correction unit 28 calculates SAD × D for the sum SAD output from the similarity calculation unit 4 using the correction coefficient D, and outputs the calculation result as a selection index SI.
As a result of such correction, when the total variation degree CT is equal to or greater than the first predetermined value Tha, the coefficient for the total SAD is 1, so that the amount of similarity correction is zero, but the total variation degree CT is Below the second predetermined value Thb, the total SAD is doubled, the similarity is corrected to a smaller value (the corrected similarity is set to a smaller value), and the overall variation degree CT is the first Between the predetermined value Tha and the second predetermined value Thb, as the total variation degree CT decreases, the coefficient D for the total SAD is increased and the similarity is corrected to a smaller value (corrected similarity) Is a smaller value).

  Based on the output of the similarity correction unit 28, the reference pixel selection unit 5 has the center of a pair of pixel blocks having the highest corrected similarity for each interpolation pixel (the value of the selection index SI = SAD × D is the smallest). A pixel is selected as a reference pixel and output to the symmetrical position interpolation means 6. Thereafter, the pixel value of the interpolation pixel is calculated as in the first embodiment, and an image of the interpolation frame is obtained.

  With the above configuration, the degree of variation of the pixel values of the pixels in the pixel block is calculated, and when the degree of variation is small, the degree of similarity is corrected so as to be smaller. This has the effect of solving the problem that a correct interpolated image cannot be obtained.

Embodiment 4 FIG.
In the first to third embodiments, the similarity calculation unit 4 obtains the sum of absolute values of differences between a pair of pixel blocks located in point symmetry with respect to the interpolation pixel, and uses it as an index indicating the similarity. The central pixel of the pixel block pair having the highest similarity is selected as the reference pixel, but the reference pixel selection result for each interpolation pixel (target interpolation pixel) is determined for the interpolation pixels around the target interpolation pixel. If the reference pixel is changed using the approximate direction (interpolation direction) with respect to the interpolation pixel, an interpolation image of a better interpolation frame can be obtained.

  FIG. 12 shows a frame interpolation apparatus according to the fourth embodiment. The illustrated interpolating apparatus is generally the same as the interpolating apparatus shown in FIG. 1, but differs in that reference pixel changing means 16 and delay compensating means 17 and 18 are added. Components having the same reference numerals as those in FIG. 1 operate in the same manner as in the first embodiment.

  The reference pixel changing means 16 is arranged in the horizontal direction and continuously subjected to interpolation processing, for example, for each of a plurality of, for example, seven interpolation pixels, reference pixel selection means 4 for each of the interpolation pixels. , The direction for the interpolated pixel is stored. For example, the reference pixel changing unit 16 divides the approximate direction (interpolation direction) of the reference pixels sequentially selected by the reference pixel selection unit 5 for each interpolation pixel with respect to the interpolation pixel by dividing a predetermined number of interpolation pixels in the horizontal direction. For example, 7 pixels are held. Specifically, information indicating the area to which the reference pixel selected by the reference pixel selection unit 5 for each interpolation pixel belongs is stored for a predetermined number of interpolation pixels, for example, seven pixels, as information indicating the general interpolation direction.

  For example, when the center pixel of the pixel block of the first reference frame in the pixel block pair determined by the reference pixel selection unit 5 is in the region UL of FIG. 13A, the first flag “FUL” 13B, the second flag “FDL” is set when it is in the region DL of FIG. 13B, and the third flag “FDR” is set when it is in the region DR of FIG. If it is in the area UR of 13 (d), the fourth flag “FUR” is set. In FIG. 13, when it is in the third row of the first to fourth columns, the first flag “FUL” and the second flag “FDL” are both set (in the first area and in the second area When the third row of the sixth to ninth columns is in the third row, the third flag “FDR” and the fourth flag “FUR” are both set (in the third region, 1st flag “FUL” and 4th flag “FUR” are both set (in the first region), when in the fifth column of the first to second rows. And the second flag “FDL” and the third flag “FDR” are both set (when the fourth flag is in the fifth column of the fourth to fifth rows). Means that it is in the third area, and is in the fifth column of the third row. Flags “FUL”, “FDL”, “FDR”, “FUR” are all set (in the first area, in the second area, in the third area, and in the fourth area Means that

  The reference pixel changing unit 16 has three interpolation pixels (n−) whose approximate interpolation direction for each target interpolation pixel (nth interpolation pixel) is located on the left side of the target interpolation pixel and is continuous to the target interpolation pixel. The flag set for each of the predetermined number of interpolation pixels existing in the left direction of the interpolation pixel of interest is all coincident with the rough interpolation direction for each of the (3, n-2, n-1th interpolation pixels). At least one of the flags set for the target interpolation pixel), or three interpolation pixels (n + 1, n + 2, n + 3th) located on the right side of the target interpolation pixel and continuing to the target interpolation pixel At least one of the flags set for each of the predetermined number of interpolation pixels present in the right direction of the target interpolation pixel matches all of the rough interpolation directions for each of the interpolation pixels). If that matches one of the raised flag for the target interpolation pixel) as it is the reference pixels a reference pixel selected in the reference pixel selecting means 5 for the interpolation pixel, and outputs it to the symmetrical position interpolator 6. On the other hand, if none of the above, instead of the reference pixel selected by the reference pixel selection means 5, reference is made to the pixel of the first reference frame and the pixel of the second reference frame existing at the same position as the interpolation pixel. It outputs to the symmetrical position interpolation means 6 as a pixel.

  In other words, the reference pixel changing unit 16 has a rough interpolation direction for each target interpolation pixel (n-th interpolation pixel) located on the left side of the target interpolation pixel and continuing to the target interpolation pixel. Unlike the rough interpolation direction for any of the three interpolation pixels (n-3, n-2, n-1th interpolation pixel) (any of the predetermined number of interpolation pixels present in the left direction of the target interpolation pixel) None of the flags set for the target interpolation pixel match with any of the flags set for the target interpolation pixel), and three interpolation pixels that are located on the right side of the target interpolation pixel and are continuous to the target interpolation pixel ( Any of the flags set for any one of the predetermined number of interpolation pixels existing in the right direction of the target interpolation pixel is different from the approximate interpolation direction for any of (n + 1, n + 2, n + 3th interpolation pixels). However, instead of the reference pixel selected by the reference pixel selection means 5, the first reference frame existing at the same position as the interpolation pixel is used. The pixel and the pixel of the second reference frame are output to the symmetrical position interpolation means 6 as a reference pixel. Otherwise, the reference pixel selected by the reference pixel selection means 5 is output to the symmetrical position interpolation means 6 as it is.

  The delay compensation means 17 and 18 compensate the delay time required for correction of the interpolation direction of the reference pixel changing means 16 and output the output of the extraction means 3 to the symmetrical position interpolation means 6. The symmetric position interpolation unit 6 uses the pixel value of the reference pixel output from the interpolation direction correction unit 16 to generate the pixel value of the interpolation pixel as in the first embodiment.

  According to the above configuration, the reference pixels that are aligned in the horizontal direction and are successively subjected to the interpolation process and selected by the reference pixel selection unit 5 for each of the plurality of interpolation pixels are associated with the interpolation pixel. The direction of the reference pixel for each target interpolation pixel is stored, and the direction of the reference pixel selected for any of a predetermined number of interpolation pixels in the left direction of the target interpolation pixel When the reference pixel selected for any of the predetermined number of interpolation pixels existing in the right direction of the target interpolation pixel is different from the direction with respect to the interpolation pixel, the reference pixel selection unit 5 selects the reference pixel. In place of the reference pixel, the pixel of the first reference frame at the same position as the position of the target interpolation pixel and the second position at the same position as the position of the target interpolation pixel Because with a reference pixel change unit 16, reference pixels pixels irradiation frame, it is possible to select the reference pixels more properly.

Embodiment 5 FIG.
In the first to fourth embodiments, the same interpolated frame is generated regardless of whether it is a moving image or a still image. However, by adaptively changing the processing between a moving image and a still image, A better interpolation image can be obtained. Hereinafter, Embodiment 5 will be described with reference to FIG.

  The interpolation apparatus shown in FIG. 14 has the same position interpolation means 8, in addition to the delay means 2, the extraction means 3, the similarity calculation means 4, the reference pixel selection means 5 and the symmetric position interpolation means 6 as in FIG. A first difference absolute value calculation unit 9, a second difference absolute value calculation unit 10, a mixing control unit 11, and a mixing unit 12 are provided. Note that components denoted by the same reference numerals as those in FIG. 1 operate in the same manner as in the first embodiment.

The same-position interpolation unit 8 calculates an average value of the pixels of the first reference frame and the pixels of the second reference frame that are present at the same position as the interpolation pixel, and outputs the average value to the mixing unit 12.
In the first embodiment, the interpolation means is constituted only by the symmetric position interpolation means 6, whereas in the fifth embodiment, the interpolation means 30 is constituted by the symmetric position interpolation means 6, the same position interpolation means 8, and the mixing means 12. Is configured.

  The first difference absolute value calculation means 9 is the same as the central pixel of the pixel block existing in the first reference frame of the pixel block pair extracted by the extraction means 3 (the same position in the frame, that is, in the frame). The absolute value of the difference from the pixel of the second reference frame existing at the position where the vertical coordinate value and the horizontal coordinate value are equal is output to the mixing control means 11.

The second difference absolute value calculation means 10 is the center pixel of the pixel block existing in the second reference frame of the pixel block pair extracted by the extraction means 3 and the pixel of the first reference frame existing at the same position. Is calculated and output to the mixing control means 11.
These absolute difference values represent the amount of image change between the first reference frame and the second reference frame.

  The mixing control unit 11 includes a pixel block pair having the reference pixel selected by the reference pixel selection unit 5 as a central pixel, that is, a pair of pixel blocks determined by the reference pixel selection unit 5 to have the highest similarity. The larger one of the output of the first difference absolute value calculation means 8 and the output of the second difference absolute value calculation means 9 is selected, and a mixing coefficient M proportional to the selected difference absolute value is generated, and mixing is performed. Output to means 12.

When the mixing coefficient M is equal to or greater than the predetermined value Thm, the mixing unit 12 selects the output Pa of the symmetric position interpolation unit 6 and outputs it to the output terminal 7 as in the first embodiment. When it is less than the predetermined value Thm, the output signal Pa of the symmetric position interpolation means 6 and the output signal Pb of the same position interpolation means 8 are mixed at a ratio according to the magnitude of the mixing coefficient M and output to the output terminal 7. That is, when the mixing coefficient M is relatively large, the ratio α (0 ≦ α ≦ 1) of the output signal Pa of the symmetric position interpolation means 6 is increased, and the ratio β (= 1) of the output signal Pb of the same position interpolation means 8. -Α) is reduced. When the mixing coefficient M is relatively small, the ratio α of the output signal Pa of the symmetric position interpolation unit 6 is decreased, and the ratio β of the output signal Pb of the same position interpolation unit 8 is increased. In other words, the mixing unit 12 increases the ratio α of the output signal Pa of the symmetric position interpolation unit 6 and decreases the ratio β of the output signal Pb of the same position interpolation unit 8 as the mixing coefficient M increases. To do.
An example of the relationship between the mixing coefficient M and the mixing ratios α and β of the output signals Pa and Pb is shown in FIG.

As described above, in the fifth embodiment, the interpolation unit 30 configured by the symmetric position interpolation unit 6, the same position interpolation unit 8, and the mixing unit 12 corresponds to the mixing coefficient M generated by the mixing control unit 11. In addition to the reference pixel selected by the reference pixel selection means 5, the pixel of the first reference frame at the same position as the position of the interpolation pixel and the pixel of the second reference frame at the same position as the position of the interpolation pixel Based on the above, a pixel value of the interpolation pixel is generated. Specifically, the interpolation result (first interpolation value) using the reference pixel selected by the reference pixel selection means 5, the pixel of the first reference frame at the same position as the interpolation pixel, and the interpolation The interpolation result (second interpolation value) using the pixel of the second reference frame at the same position as the pixel position as the reference pixel is determined according to the mixing coefficient M generated by the mixing control unit 11 (mixing coefficient). The pixel value of the interpolation pixel is generated by mixing (at a ratio according to M).
For this reason, there is an effect that an appropriate interpolated image can be obtained according to the magnitude of the absolute difference (change amount) of the pixel value between frames, and accordingly, depending on whether the image is a moving image or a still image.

Embodiment 6 FIG.
In the first embodiment, the center pixel of the pixel block pair of the pixel block pair having the highest similarity (the value of the sum of absolute differences SAD is the smallest) is selected as the reference pixel. When the highest similarity (similarity of the pixel block pair having the highest similarity) calculated by the similarity calculation means 4 for the interpolated pixel is smaller than the threshold (the total SAD value of the pixel block pair having the smallest total SAD value) Is larger than a predetermined value), it is assumed that there is no pixel suitable for interpolation in the vicinity of the interpolation pixel, and the average of the pixel of the first reference frame and the pixel of the second reference frame at the same position as the interpolation pixel If an interpolation pixel is generated using the value, an interpolation image of a better interpolation frame can be obtained.

FIG. 16 shows a frame interpolation apparatus according to the sixth embodiment. The illustrated interpolation apparatus is generally the same as that shown in FIG. 1, except that the same position interpolation means 8, the mixing control means 14, and the mixing means 15 are added. Components having the same reference numerals as those in FIG. 1 operate in the same manner as in the first embodiment.
In the sixth embodiment shown in FIG. 16, as in the fifth embodiment shown in FIG. 14, the symmetric position interpolation means 6, the same position interpolation means 8, and the mixing means 12 constitute the interpolation means 30.

  Similarly to the same position interpolation unit 8 shown in FIG. 14, the same position interpolation unit 8 calculates an average value of the pixels of the first reference frame and the pixels of the second reference frame at the same position as the interpolation pixel. Calculate and output to the mixing means 15 as the second interpolation value.

The mixing control unit 14 determines a pair of pixel blocks determined to have the highest similarity by the reference pixel selection unit 5 among the similarities calculated by the similarity calculation unit 4 for each interpolation pixel (that is, absolute difference). The value of the sum of absolute differences SAD of a pair of pixel blocks determined to have the smallest sum of values SAD) is compared with two predetermined values Th1 and Th2 (Th1 <Th2), and according to the comparison result The mixing ratio β shown in the following equations (4A) to (4C) is output to the mixing means 15.
When 0 ≦ SAD <Th1, β = 0 (4A)
When Th1 ≦ SAD <Th2, β = (SAD−Th1) / (Th2−Th1) (4B)
When Th2 ≦ SAD, β = 1 (4C)
Such a relationship between the total SAD and the mixing coefficient β is shown in FIG.

  The mixing means 15 outputs the same position interpolation means 8 output (second interpolation value) Pb as the output (first interpolation value) Pa of the symmetrical position interpolation means 6 according to the mixing ratio β output from the mixing control means 14. Are mixed as shown in the following equation (5), and the mixing result Pc is output to the output terminal 7.

  Pc = β × Pb + (1−β) × Pa (5)

  In addition, although the function of the mixing control means 14 was shown as generating the mixing ratio β represented by the equations (4A) to (4C), the generation of the mixing ratio β may be performed by calculation. You may make it perform by table conversion.

  As described above, in the sixth embodiment, the interpolation means 30 has the sum SAD of absolute differences between pixel blocks having the reference pixel as the central pixel equal to or greater than the first predetermined value Th1, and therefore the degree of similarity is the first. If it is less than or equal to the threshold value of 1, the first interpolation value obtained by using the pixel value of the reference pixel selected by the reference pixel selection means 5 according to the similarity (output of the symmetric position interpolation means 6) ) And the second interpolation value (2) obtained using the pixel of the first reference frame at the same position as the interpolation pixel and the pixel of the second reference frame at the same position as the interpolation pixel. By mixing the output of the same position interpolation means 8) by the mixing means 15, an interpolation value of the interpolation pixel is generated.

The interpolating means 30 also sets the mixing ratio β to 0 when the sum SAD is less than the first predetermined value Th1, and therefore the similarity is larger than the first threshold, and is selected by the reference pixel selecting means 5. The first interpolation value (output of the symmetric position interpolation unit 6) obtained using the pixel value of the reference pixel thus obtained is used as the output of the interpolation unit 30 as it is.
The interpolating means 30 further sets the mixing ratio β to 1 when the total SAD is equal to or greater than the second predetermined value Th2 (Th2> Th1), and therefore the similarity is equal to or smaller than the second threshold value. The second interpolation value (of the same position interpolation means 8) obtained by using the pixel of the first reference frame at the same position as the position and the pixel of the second reference frame at the same position as the position of the interpolation pixel. Output) is used as the output of the interpolation means 30 as it is.

  In order to perform such switching, it is possible to select or mix interpolation values generated by different interpolation methods according to the degree of similarity between pixel blocks having a reference pixel as a central pixel, and more appropriately. There is an effect that a correct interpolation image can be obtained.

The features of Embodiments 1 to 6 can be used in combination with each other. For example, the similarity correction unit 13 of the second embodiment may be used in combination with the correction control unit 27 and the similarity correction unit 28 of the third embodiment. In this case, the similarity corrected by the similarity correction unit 13 may be further corrected by the similarity correction unit 28, and the similarity corrected by the similarity correction unit 13 and the similarity corrected by the similarity correction unit 28 ( They may be combined (for example, by taking an average of them).
Further, the reference pixel changing means 16 described in the fourth embodiment may be incorporated in the frame interpolation apparatus of the second or third embodiment.
Furthermore, the first difference absolute value calculation means 9, the second difference absolute value calculation means 10, the mixing control means 11, the same position interpolation means 8, and the mixing means 12 described in the fifth embodiment are the same as those in the second to fourth embodiments. The mixing control unit 14, the same position interpolation unit 8, and the mixing unit 12 described in the sixth embodiment may be combined with the second to fourth embodiments.
Furthermore, the modification described most in detail with respect to the first embodiment can also be applied to the second to sixth embodiments.

Embodiment 7 FIG.
Embodiments 1 to 6 show examples of interpolation image generation apparatuses for interpolation frames. However, a frame output by doubling the frame rate of an input video signal using any of Embodiments 1 to 6 The frame frequency conversion device may be configured using rate conversion means.

FIG. 18 shows a block diagram of a frame frequency conversion apparatus according to the seventh embodiment. Reference numeral 19 denotes an input terminal, 20 denotes a frame interpolator, 21 denotes writing / reading means, 22, 23, 24 and 25 denote frame memories, and 26 denotes an output terminal.
As the frame interpolation device 20, any of those described in the first to sixth embodiments may be used.

  From the video signal input from the input terminal 19, the frame interpolation device 20 sequentially generates an image (interpolated image) of an interpolation frame by any one of the methods described in the first to sixth embodiments. The writing / reading means 21 alternately writes video signals (hereinafter referred to as original images) input from the input terminal 19 into the frame memory 22 and the frame memory 23 in units of frames. Further, the output signal (interpolated image) of the frame interpolation device 20 is alternately written in the frame memory 24 and the frame memory 25 in units of frames. Next, the original image is read out and output to the output terminal 26 from the frame memory 22 or the frame memory 23 in which the writing operation is not performed at twice the writing speed. Subsequently, an interpolated image is read out and output to the output terminal 24 from the frame memory 24 or the frame memory 25 where the writing operation is not performed at twice the writing speed. As a result, a good video signal having a frame frequency twice that of the input video signal is obtained at the output terminal 24.

  With the above configuration, the frame frequency conversion is performed using the frame interpolation apparatus according to the first to sixth embodiments, so that there is an effect that a good frame frequency conversion of an interpolated image can be obtained.

  In addition, the interpolation apparatus of Embodiments 1-6 removes the 2nd frame when the frame of the same content is input 2 consecutively, not only when converting a frame frequency, Processing (frame conversion processing) in which frames between frames are generated by interpolation may be performed.

  As an application example of the present invention, it can be applied to frame frequency conversion for television and frame frequency conversion for commercial monitors. Even if the frame frequency is not converted, it can be applied to frame conversion in which one redundant frame (a frame having the same content as the immediately preceding frame) is removed and a new frame is generated.

It is a block diagram which shows the interpolation apparatus of Embodiment 1 of this invention. FIG. 3 is a diagram showing a pair of pixel blocks that are sequentially selected for reference pixel selection in the first embodiment. (A)-(d) is a figure which shows the pixel block in the 1st reference frame selected sequentially for selection of the reference pixel in Embodiment 1. FIG. (A)-(d) is a figure which shows the pixel block in the 2nd reference frame selected sequentially for selection of the reference pixel in Embodiment 1. FIG. (A)-(d) is a figure which shows the pixel block in the 2nd reference frame selected sequentially for selection of the reference pixel in Embodiment 1. FIG. 4 is a diagram showing pixel blocks in a first reference frame that are sequentially selected for selection of reference pixels in Embodiment 1. FIG. FIG. 6 is a diagram illustrating pixel blocks in a second reference frame that are sequentially selected for selection of reference pixels in the first embodiment. It is a block diagram which shows the interpolation apparatus of Embodiment 5 of this invention. 10 is a diagram illustrating a relationship between a distance between a reference pixel and an interpolated pixel and a correction coefficient K in Embodiment 2. FIG. It is a block diagram which shows the interpolation apparatus of Embodiment 3 of this invention. FIG. 10 is a diagram illustrating a relationship between a total variation degree CT and a correction coefficient D in the third embodiment. It is a block diagram which shows the interpolation apparatus of Embodiment 4 of this invention. (A)-(d) is a figure which shows each area | region used for determination of the general direction with respect to the interpolation pixel of the reference pixel selected by the reference pixel selection means 5 in Embodiment 4. FIG. It is a block diagram which shows the interpolation apparatus of Embodiment 5 of this invention. It is a figure which shows the relationship between the mixing coefficient M in Embodiment 5, and mixing ratio (alpha) and (beta). It is a block diagram which shows the interpolation apparatus of Embodiment 6 of this invention. FIG. 20 is a diagram illustrating a relationship between a sum SAD of absolute differences and a mixing ratio β in the sixth embodiment. It is a block diagram which shows the frame frequency converter of Embodiment 7 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Input terminal, 2 Delay means, 3 Extraction means, 4 Similarity calculation means, 5 Reference pixel selection means, 6 Symmetric position interpolation means, 7 Output terminal, 8 Same position interpolation means, 9 1st difference absolute value calculation means, 10 second difference absolute value calculation means, 11 mixing control means, 12 mixing means, 13 similarity correction means, 14 mixing control means, 15 mixing means, 16 reference pixel changing means, 17 delay compensation means, 18 delay compensation means, 19 input terminals, 20 interpolators, 21 write / read means, 22 frame memories, 23 frame memories, 24 frame memories, 25 frame memories, 26 output terminals, 27 correction control means, 28 similarity correction means, 30 interpolation means, 32 selections Indicator generation means.

Claims (22)

A frame interpolation device for generating a pixel value of an interpolation pixel in an interpolation frame between a first reference frame and a second reference frame,
One of the pixel block pair consisting of the pixel block of the first reference frame and the pixel block of the second reference frame that is point-symmetric with respect to the position of the interpolation pixel, and the pixel block pair Extraction means for extracting a plurality of pixel block pairs each replaced by a pixel block shifted by one pixel;
For each of a plurality of pixel block pairs extracted by the extraction means, similarity calculation means for calculating the similarity between two pixel blocks based on the pixel values of the pixels in the two pixel blocks;
Reference pixel selection means for selecting, as a reference pixel, a pixel located at the center of each pixel block constituting a pixel block pair that has the largest similarity calculated by the similarity calculation means or a correction of the similarity;
An interpolating unit that generates the interpolated pixel based on the reference pixel selected by the reference pixel selecting unit.   The extracting means sets one pixel block to one pixel block pair consisting of a pixel block at a point-symmetrical position with respect to the interpolation pixel and a pixel block pair consisting of a pixel block at the point-symmetrical position. 2. The frame interpolation device according to claim 1, wherein a pixel block pair replaced with a pixel block shifted to the left of the pixel or to the right of one pixel is extracted.   The extracting means sets one pixel block to one pixel block pair consisting of a pixel block at a point-symmetrical position with respect to the interpolation pixel and a pixel block pair consisting of a pixel block at the point-symmetrical position. The frame interpolating apparatus according to claim 1, wherein a pixel block pair replaced with a pixel block shifted by one pixel or one pixel below is extracted.   The extracting means sets one pixel block to one pixel block pair consisting of a pixel block at a point-symmetrical position with respect to the interpolation pixel and a pixel block pair consisting of a pixel block at the point-symmetrical position. 2. The pixel block pair replaced by a pixel block shifted to the upper right pixel, the lower right pixel, the upper left pixel, or the lower left pixel pixel is extracted. Frame interpolator.   The similarity calculation means obtains the sum of absolute values of differences between pixel values of at least some of the pixels in one pixel block constituting the pixel block pair and pixels at corresponding positions in the other pixel block, 5. The frame interpolating apparatus according to claim 1, wherein the sum is output as the similarity.   The similarity calculation means obtains a sum of absolute values of differences between pixel values of all pixels in one pixel block constituting the pixel block pair and pixels at corresponding positions of the other pixel block. The frame interpolation apparatus according to claim 5.   The similarity calculation means calculates a difference between a pixel value of a pixel after performing equal thinning processing on pixels in one pixel block constituting a pixel block pair and a pixel at a corresponding position in the other pixel block. 6. The frame interpolating apparatus according to claim 5, wherein a sum of absolute values of the frame interpolation is obtained. Similarity correction that outputs a corrected similarity that is smaller than the similarity of the pixel block pair as the distance from the interpolation pixel of the pixel located at the center of each pixel block constituting the pixel block pair is longer Further comprising means,
The frame interpolation device according to claim 1, wherein the reference pixel selection unit selects the reference pixel based on the corrected similarity. Correction control means for calculating the degree of variation of the pixel values of the pixels in the pixel block and detecting the calculated degree of variation;
The degree of variation detected by the correction control means further includes a similarity correction means for outputting a corrected similarity that is smaller than the similarity output by the similarity calculation means,
The frame interpolation device according to claim 1, wherein the reference pixel selection unit selects the reference pixel based on the corrected similarity. Alignment in the horizontal direction, storing the direction of the reference pixel selected by the reference pixel selection means for each of the plurality of interpolation pixels that are continuously subjected to interpolation processing, with respect to the interpolation pixel,
The direction of the reference pixel with respect to the interpolation pixel of interest is different from the direction of the reference pixel selected with respect to the interpolation pixel of any of a predetermined number of interpolation pixels existing in the left direction of the interpolation pixel of interest. When the reference pixel selected for any of the predetermined number of interpolation pixels existing in the right direction of the target interpolation pixel is different from the direction of the interpolation pixel, the reference pixel selected by the reference pixel selection unit Instead, there is provided a reference pixel changing unit that uses the pixel of the first reference frame at the same position as the target interpolation pixel and the pixel of the second reference frame at the same position as the target interpolation pixel as a reference pixel. The frame interpolation device according to claim 1, wherein the frame interpolation device is a frame interpolation device. The interpolation means sets the average value of the pixel values of the reference pixels selected by the reference pixel selection means as the pixel value of the interpolation pixels, or sets one pixel value of the reference pixels as the pixel value of the interpolation pixels. The frame interpolation apparatus according to claim 1, wherein The interpolation means includes
Symmetric position interpolation means for generating a first interpolation value for the interpolation pixel using the reference pixel selected by the reference pixel selection means;
The same generation of a second interpolation value for the interpolation pixel using the pixel of the first reference frame at the same position as the interpolation pixel and the pixel of the second reference frame at the same position as the interpolation pixel Position interpolation means;
The frame interpolation apparatus according to claim 1, further comprising a mixing unit that mixes the first interpolation value and the second interpolation value. Of the pair of pixel blocks having the highest similarity calculated by the similarity calculation means, the pixel at the center position of the pixel block in the first reference frame, and the pixel block in the first reference frame First difference absolute value calculation means for calculating a difference absolute value between a pixel in a central position and a pixel in a second reference frame at the same position;
Of the pair of pixel blocks having the highest similarity calculated by the similarity calculation means, the pixel at the center position of the pixel block in the second reference frame, and the pixel block in the second reference frame Second difference absolute value calculation means for calculating a difference absolute value between the pixel in the first position and the pixel in the first reference frame at the same position;
A mixing control unit that generates a mixing coefficient from the absolute difference value calculated by the first absolute difference value calculating unit and the absolute difference value calculated by the second absolute difference value calculating unit;
The frame interpolation according to claim 12, wherein the mixing unit mixes the first interpolation value and the second interpolation value at a ratio corresponding to a mixing coefficient generated by the mixing control unit. apparatus.   The mixing control means calculates a value proportional to a larger one of the difference absolute value calculated by the first difference absolute value calculation means and the difference absolute value calculated by the second difference absolute value calculation means. 14. The frame interpolating apparatus according to claim 13, wherein the frame interpolating apparatus outputs the mixed coefficient. The mixing means selects the output of the symmetrical position interpolation means when the mixing coefficient is equal to or greater than a predetermined value;
The frame interpolation device according to claim 14, wherein when the mixing coefficient is less than a predetermined value, the first interpolation value and the second interpolation value are mixed at a ratio corresponding to the mixing coefficient.   16. The mixing unit according to claim 15, wherein the mixing means increases the ratio of the first interpolation value and decreases the ratio of the second interpolation value as the mixing coefficient increases. The frame interpolation apparatus described.   The interpolation means, when the largest similarity calculated by the similarity calculation means for each interpolation pixel is equal to or less than a first threshold value, the first interpolation value and the second interpolation value, The frame interpolation apparatus according to claim 12, wherein the interpolation value of the interpolation pixel is generated by mixing at a ratio corresponding to the magnitude of the similarity.   In the mixing, the mixing means increases the ratio of the first interpolation value and increases the ratio of the second interpolation value as the similarity calculated by the similarity calculation means for each pixel increases. The frame interpolation apparatus according to claim 17, wherein the frame interpolation apparatus is made smaller. The mixing unit selects and outputs the second interpolation value when the maximum similarity calculated by the similarity calculation unit for each interpolation pixel is equal to or smaller than a second threshold value. The frame interpolation device according to claim 17.   The mixing means selects and outputs the output of the symmetric position interpolation means when the largest similarity calculated by the similarity calculation means for each interpolation pixel is larger than the first threshold value. The frame interpolation apparatus according to claim 17, wherein   A frame frequency conversion device comprising the frame interpolation device according to claim 1. A frame interpolation method for generating a pixel value of an interpolation pixel in an interpolation frame between a first reference frame and a second reference frame,
One of the pixel block pair consisting of the pixel block of the first reference frame and the pixel block of the second reference frame that is point-symmetric with respect to the position of the interpolation pixel, and the pixel block pair An extraction step of extracting a plurality of pixel block pairs obtained by replacing each pixel block with a pixel block shifted by one pixel;
A similarity calculation step of calculating the similarity of two pixel blocks based on the pixel values of the pixels in the two pixel blocks for each of a plurality of pixel block pairs extracted by the extraction step;
A reference pixel selection step of selecting, as a reference pixel, a pixel located at the center of each pixel block that constitutes a pixel block pair that has the largest similarity calculated by the similarity calculation step or a correction of the similarity;
An interpolation step for generating the interpolation pixel based on the reference pixel selected in the reference pixel selection step.

Images