JP2010098513A - Apparatus and method for generating interpolated image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique of detecting a higher-precision vector on a search range boundary in block matching of an image. <P>SOLUTION: An interpolated image generation apparatus includes: a frame memory for holding a first image of an input moving video; a moving vector search part for detecting a vector candidate of a small block and outputting the vector information of these vector candidates after detecting and outputting two moving vectors whose block sizes differ as a large one and a small one; a search range boundary vector detecting part for detecting a search range boundary vector derived from vector information on the vector candidate of the small block output from the moving vector search part; a vector selection part for selecting either of the output of the moving vector of the large one or the small one of the two block sizes of the moving vector search part based on the detection result of the search range boundary vector; and an interpolation frame preparing part for creating an interpolation frame using a selected moving vector. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a frame interpolation circuit, a frame interpolation method, and a display device.

  When a moving image is displayed on a liquid crystal display device (LCD), the LCD displays a frame image (hereinafter simply referred to as a frame) at a rate of 60 frames / second, for example. This frame is a progressive scanning signal obtained by processing an interlace of 60 fields / second, for example. That is, the LCD continues to display one frame for 1/60 second.

  When viewing such a video displayed on the LCD, the image of the previous frame remains as an afterimage in human eyes. For this reason, a moving object in the image may appear blurred or the movement of the object may appear unnatural. Such a phenomenon becomes more prominent as the screen becomes larger.

  In order to prevent such blurring of moving images, a method of displaying a moving image by inserting an interpolation frame between two consecutive frames is known (Patent Document 1). In this method, the motion vector (the direction and distance the object moved) of each block is detected by matching the image blocks that make up the frame between two or more input frames of the previous frame and the subsequent frame. Is done. Using the motion vector of each block, a new interpolated frame located between the input frames is created. By inserting an interpolated frame between two input frames, the number of frames is increased to display a moving image.

  The block matching is a method for detecting which image block in a certain frame matches an image block of a predetermined size. The difference between the pixels corresponding to each other in the image block in the previous frame and any of the image blocks in the subsequent frame is calculated, and the subsequent frame in which the accumulated value (SAD: Sum of Absolute Difference) is minimized The middle image block is detected as the image block most similar to the previous frame middle image block. The difference in the position of the most similar image block between the previous frame and the subsequent frame is detected as a motion vector.

  There is also a device to change the size of this block. For example, in the detection of a motion vector for creating an interpolation frame, there is one that improves the detection accuracy of a motion vector as follows for an image including a periodic pattern.

  That is, as a solving means, a motion detection process using two blocks having different sizes is performed, and a motion vector detected by a small block is usually employed. If multiple reliable motion vector candidates are found when detecting a vector in a small block, refer to the vector detected using the large block, and detect the large block from the motion vectors detected in the small block. The closest motion vector is adopted as the motion vector of the block used for creating the interpolation frame (Patent Document 2).

The above is effective for the purpose, but on the other hand, another problem is that when the input signal has a motion near the boundary of the search range or beyond the search range, vector misdetection may occur during vector detection. However, a technique for dealing with this problem is not disclosed.
JP 2005-6275 A JP 2008-35404 A

  It is an object of the present invention to provide a technique for detecting a vector with higher accuracy at a search range boundary in image block matching.

  In order to solve the above problem, an interpolation image generation apparatus according to the present invention holds the first image in an interpolation image generation apparatus that generates an interpolation image between a first image and a second image of an input moving image. And a motion vector of a block having two different sizes from the first image held in the frame memory and output, and a vector candidate of a small block is detected to detect the vector candidates. A motion vector search unit for outputting vector information, a search range boundary vector detection unit for detecting a search range boundary vector from vector information of vector candidates of the small blocks output from the motion vector search unit, and the motion vector search unit One of the motion vector outputs of two large and small blocks is selected based on the detection result of the search range boundary vector. A torque selector, characterized by comprising an interpolation frame creation unit for creating an interpolation frame by using the motion vector selected nuclear.

  According to the present invention, a technique for detecting a vector with higher accuracy at a search range boundary in block matching of an image can be obtained.

An embodiment according to the present invention will be described with reference to FIGS.
FIG. 1 is a block diagram showing an embodiment of an interpolation frame creating apparatus 1 according to the present invention.
The motion vector search unit 3 detects and outputs two large and small motion vectors from frames before and after the interpolation frame from the input image signal of 60 frames / second. Further, the motion vector search unit 3 detects vector information (vector value, correlation value, eg, SAD value) of a small block vector candidate, and outputs it to the search range boundary vector detection unit 4 which is a feature of this embodiment. The search range boundary vector detection unit 4 is configured to detect a search range boundary vector, and the motion vector search unit 3 selects an optimal vector based on this signal and outputs the vector to the interpolation frame generation unit 5. ing. The interpolation frame generation unit 5 generates an interpolation frame using the frame memory 2 based on this vector, and outputs it to the panel 7 at an output image signal of 120 frames / second. The control unit 6 controls each component described above.

Next, a block matching processing method will be described. FIG. 2 is a diagram showing block matching processing.
As a motion vector detection method using block matching, in addition to a method of obtaining an SAD by simply shifting an image block in a previous frame in a subsequent frame, an image block located at a point target position as shown in FIG. There is a method for obtaining a motion vector through mutual block matching processing.

  That is, in the method of FIG. 2, the image blocks at the point target positions on the front frame 20 and the rear frame 22 sandwiching the interpolation image block 41 in the interpolation frame 21 are compared for each pixel. To calculate SAD. A vector connecting the image blocks that are most similar (having the smallest SAD) is determined as a motion vector. This comparison is performed in a predetermined search range 40 in the previous frame 20 and a corresponding search range 42 in the rear frame 22.

  When the combination of image blocks most similar to each other is, for example, the image block 43 and the image block 44, a vector from the image block 43 to the image block 44 is determined as the motion vector of the interpolated image block 41. Based on this motion vector and image data of image blocks 43 and 44 that are most similar to each other, an interpolation image block 41 in the interpolation frame 21 is created.

  As a correlation value serving as a reference for the magnitude of correlation, a sum of absolute differences (SAD), a sum of coincidence pixels corresponding to highly correlated pixels (Sum of Agreement Pixels: SAP), and the like can be considered. SAD can be obtained by the following equation.

Here, x is the position of the pixel in the frame, B is a set of pixel positions in the block, N is the number of frames, d is the motion vector, and f (x, N) is the luminance component of the target pixel. That is, d in which the SAD is minimum in the equation (1) is the motion vector MV from the N frame to the N−1 frame, and the previous block indicated by MV is the candidate interpolation block. Moreover, SAP can be calculated | required by the following formula | equation.

Expressions (2) and (3) obtain the number of pixels in which the absolute value difference between the luminance components in the block is smaller than the threshold Th. That is, d in which the SAP is maximum in the equations (2) and (3) is the motion vector MV from the N frame to the N−1 frame, and the previous block indicated by MV is the candidate interpolation block. In SAP, the correlation is higher when the value is larger as opposed to SAD.

FIG. 3 is a block diagram showing an embodiment of the motion vector search unit 3 and the search range boundary vector detection unit 4 according to the present invention.
The motion vector search unit 3 includes a small block vector detection unit 31 that detects and outputs a small block vector, a large block vector detection unit 32 that detects and outputs a vector of a large block, and features of the present embodiment. A small block vector candidate detecting unit 33 for detecting a vector candidate of a small block and obtaining vector information (vector value and SAD value) of the vector candidate and a vector selecting unit 34 are configured. In addition, there is a search range boundary vector detection unit 4 that detects a search range boundary vector from the vector information obtained by the small block vector candidate detection unit 33. The vector selection unit 34 may be provided outside the motion vector search unit 3.

Next, the image block size and motion vector search range will be described.
4 shows a specific example of the size of a small block and its search range, and FIG. 5 shows a specific example of the size of a large block and its search range. These blocks and search ranges can be applied to the block matching process shown in FIG. In order to simplify the description, the block matching process in the horizontal direction will be mainly described.

  In FIG. 4, SB is a small block, and SSR is a search range of the small block SB. The size of the small block SB is, for example, 64 pixels × 2 pixels. The small block SB is shifted on the previous frame 20 by −12 to +12 pixels in the horizontal direction and −2 to +2 pixels in the vertical direction. At the same time, the small block SB is shifted on the rear frame 22 by +12 to -12 pixels in the horizontal direction and +2 to -2 pixels in the vertical direction.

  That is, when the small block SB is shifted by, for example, +12 pixels on the previous frame 20 with the interpolation image block 41 (a block having a size matching the small block SB here) inserted on the interpolation frame 21 as the center of the point object. Then, the small block SB is shifted by -12 pixels on the rear frame 22. SAD is calculated by comparing pixel values of corresponding image blocks in both frames 20 and 22 in units of corresponding pixels. Accordingly, in the case of block matching using the small block SB, the search range SSR on the previous frame 20 and the subsequent frame 22 is 88 pixels × 6 pixels here. The vicinity of the search range boundary described later can be defined, for example, when the absolute value is shifted by 9 pixels or more in the horizontal direction in the SSR, or when the absolute value is shifted by 2 pixels in the horizontal direction.

  As shown in FIG. 5, the size of the large block LB is, for example, 640 pixels × 4 pixels. Similar to the small block SB, the large block LB is shifted by −12 to +12 pixels in the horizontal direction and −2 to +2 pixels in the vertical direction on the previous frame 20. At the same time, the large block LB is shifted on the rear frame 22 by +12 to −12 pixels in the horizontal direction and +2 to −2 pixels in the vertical direction.

  That is, when the large block LB is shifted by, for example, +12 pixels on the previous frame 20 with the interpolated image block 41 (a block having a size corresponding to the large block LB here) inserted on the interpolation frame 21 as the center of the point object. Then, the large block LB is shifted by -12 pixels on the rear frame 22, and the SAD is calculated by comparing the pixel values of the corresponding image blocks in the frames 20 and 22 in units of corresponding pixels. Therefore, in the case of block matching using a large block, the search range LSR on the previous frame 20 and the subsequent frame 22 are both 664 pixels × 8 pixels here.

  FIG. 6 is a relationship diagram between vector values and vector values of vector candidates for small block vector candidate detection of the motion vector detection unit of the present embodiment. Vector information consisting of the vector values and SAD values of these vector candidates is output to the search range boundary vector detection unit. In this example, the vector value V1 of the vector candidate 1, the SAD value SAD (V1), the vector value V2 of the vector candidate 2 having the smallest SAD value, and the SAD value SAD (V2) are output to the search range detection vector detection unit. The search range detection vector portion is ΔSAD (V1, V2) = | SAD (V1) −SAD (V2) | is smaller than a certain threshold value THR_SAD, and the distance D (V1, V2) = | V1-V2 | If a certain threshold value THR_D is larger than the threshold value THR_D and these vector candidates are located in the vicinity of the search range boundary, it is determined to be a motion vector at the search range boundary.

  FIG. 7A is a diagram in which vector candidates and original motion vectors are plotted on the search range of the horizontal vector on the horizontal axis and the vertical vector on the vertical axis with respect to the diagram of FIG. FIG. 7B is a diagram in which a large block vector and an original motion vector are plotted as in FIG. 7A.

  In the conventional method, an interpolation frame is generated by the vector V2 which is the minimum value of the SAD of the small block. However, as can be seen from FIG. 7A, since V2 is a vector in the reverse direction compared to the original motion vector, an interpolation frame is generated with a false detection vector.

  In the present embodiment, when the search range boundary is detected, the vector selection unit determines the large block vector V3 in FIG. 7B, and the vector can be corrected by V3 close to the original motion vector. The interpolation frame can be generated by a vector in the same direction as the original vector.

  FIG. 8 is a flowchart of this embodiment. First, when there is a single vector candidate obtained from the small block vector candidate detection unit 33 (No in step S21), the small block vector is output (step S24). If there is no vector candidate, if there is a large block vector, the large block vector is output.

  When there are a plurality of vector candidates (Yes in step S21), all these vector candidates are located in the vicinity of the search range boundary, and the vector candidate is smaller than the threshold THR_SAD having ΔSAD with the vector candidate having the smallest SAD value. If there is no (No in step S22), a small block vector having the smallest SAD value is output (step S24).

  If there is a vector candidate (Yes in step S22), the search range boundary vector detection unit 4 determines that a motion vector at the search range boundary has been detected, and outputs a large block vector (step S23). After step S23 or step S24, it is determined in step S25 whether or not the processing has been completed for all the small blocks. If not completed, the process returns to step S21, and if completed, the process ends.

  In this embodiment, it is easy to understand that the motion vector detection using the large block is performed before the motion vector candidate detection using the small block, but such a motion vector detection is performed in parallel. Or conversely, it may be performed in series.

  That is, motion vector candidates may be detected using small blocks, and after detecting the motion vector at the search range boundary, motion vectors may be detected using the large blocks and the large block vectors may be output. Serial processing can reduce the circuit scale compared to parallel processing.

  As described above, as an improvement method in the present embodiment, motion near the search range boundary or over the search range is detected for the input signal, and an optimal vector is selected to create an interpolation frame.

  As an effect, the vector candidate detection unit detects the search range boundary vector and selects the optimal vector even when there is motion near the search range boundary or over the search range, which is related to the detection of the backward motion vector. High vectors can be detected.

  As described above, when the search range boundary vector is detected according to this embodiment, it is possible to interpolate with a vector with higher accuracy by correcting with a vector of a large block.

  In addition, this invention is not limited to the said embodiment, In the range which does not deviate from the summary, it can implement in various modifications. For example, the vector selection unit 34 normally selects an output in accordance with an instruction from the search range boundary vector detection unit 4, but the process when there is no vector candidate is output in accordance with an instruction from the small block vector candidate detection unit 33. May be selected.

  Various inventions can be formed by appropriately combining a plurality of constituent elements disclosed in the above-described embodiments. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements according to different embodiments may be appropriately combined.

The block block diagram which shows one Embodiment of this invention. The figure which shows the block matching process of the embodiment. The block block diagram shown in order to demonstrate the detail of the principal part of the embodiment. The figure which shows the specific example of the size of a small block, and its search range. The figure which shows the specific example of the size of a large block, and its search range. FIG. 7 is a relationship diagram between vector values and SAD values of vector candidates for small block vector candidate detection according to the embodiment. The vector plot for demonstrating the embodiment. FIG.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Interpolation frame production apparatus, 2 ... Frame memory, 3 ... Motion vector search part, 4 ... Search range boundary vector detection part, 5 ... Interpolation frame generation part, 6 ... Control part, 7 ... Panel, 31 ... Small block vector detection 32, large block vector detection unit, 33 ... small block vector candidate detection unit, 34 ... vector selection unit.

Claims (6)

In an interpolated image generating apparatus that generates an interpolated image between a first image and a second image of an input moving image,
A frame memory for holding the first image;
From the first image held in the frame memory, motion vectors of blocks of two different sizes are detected and output, and vector candidates of small blocks are detected and vector information of the vector candidates is output. A motion vector search unit;
A search range boundary vector detection unit that detects a search range boundary vector from vector information of vector candidates of the small blocks output from the motion vector search unit;
A vector selection unit that selects one of the motion vector outputs of the two large and small blocks of the motion vector search unit based on a detection result of the search range boundary vector;
An interpolation image generation apparatus comprising: an interpolation frame generation unit that generates an interpolation frame using a selected motion vector.   The interpolated image generation apparatus according to claim 1, wherein when the number of vector candidates is singular, the vector selection unit selects an output of a motion vector of a small block.   The interpolated image generation apparatus according to claim 1, wherein the vector information includes a SAD value.   The interpolated image generation apparatus according to claim 1, wherein when there is no vector candidate, the vector selection unit selects an output of a motion vector of a large block.   The interpolated image generation apparatus according to claim 1, further comprising a panel that inputs an output of the interpolation frame creation unit and displays the output as a moving image. An interpolation image generation method for generating an interpolation image between a first image and a second image of an input moving image,
A motion vector search step of detecting and outputting motion vectors of blocks of two different sizes from the first image, and detecting vector candidates of small blocks and outputting vector information of the vector candidates;
A search range boundary vector detection step of detecting a search range boundary vector from vector information of vector candidates of the small block output in the motion vector search step;
A vector selection step of selecting any one of the motion vector outputs of the two large and small blocks in the motion vector search step based on the detection result of the search range boundary vector;
And an interpolation frame generating step of generating an interpolation frame using the selected motion vector.

Images