JP5762006B2 - Image processing apparatus and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus and an image processing method.

  By detecting motion information (motion vector) between two temporally continuous frames of the input video signal and generating an interpolated frame image that interpolates between those frames, There is a technique for increasing the frame rate (frame rate conversion processing). When generating an image of an interpolation frame, each frame of the input video signal is divided into a plurality of blocks, and a motion vector is detected for each block. Then, an interpolation frame is generated using the detected motion vector. If such a technique is used, the motion of the video can be expressed smoothly.

By the way, if the video includes a telop, the viewer can easily focus on the telop. Therefore, a large image quality improvement effect can be expected by generating an interpolated frame image and smoothly expressing the motion of the telop. Patent Document 1 discloses a technique for detecting a telop area from a video signal and generating an interpolated frame image using a motion vector based on the motion of the telop as a motion vector of a block corresponding to the telop area. Yes. According to the technique disclosed in Patent Document 1, the movement of a telop can be expressed smoothly.
There is also a technique for generating an image of an interpolated frame in which image quality deterioration at an end portion of a moving region is suppressed. Such a technique is disclosed in Patent Document 2 and Patent Document 3, for example.

JP 2008-107753 A JP 2009-141878 A JP 2009-055340 A

  However, a block at the end portion of a telop (end block) often includes a portion other than the telop (hereinafter referred to as background) more than the telop. In addition, there are many cases where the motion is different between the telop and the background. Therefore, in the end block, the motion vector of the telop is difficult to detect, and the background motion vector is easy to detect. When an interpolated frame image is generated using a background motion vector as a motion vector of a block including a telop, the block appears to lack a telop. As described above, since the telop is an object that attracts attention, the image quality is greatly deteriorated due to the collapse of the telop.

  An object of the present invention is to provide an image processing apparatus and an image processing method capable of generating an interpolated frame image in which image quality deterioration in a telop area is suppressed.

A first aspect of the present invention is an image processing apparatus for converting a frame rate by inserting an interpolation frame between frames of an input video signal, and dividing the frame of the input video signal into a plurality of blocks. A first motion vector detecting means for detecting a motion vector for each block; and a plurality of blocks that are smaller than a block obtained by dividing at least a part of a frame of an input video signal by the first motion vector detecting means. A second motion vector detecting means for detecting a motion vector for each small block, and a telop detecting means for detecting a block including a telop among the blocks divided by the first motion vector detecting means, Interpolation frame generation means for generating an image of the interpolation frame using a motion vector, and the interpolation frame generation unit Among the block including the telop, to a region of the block that the block adjacent free of telop, using the motion vector detected by the second motion vector detecting means generates an image of the interpolation frame, the remaining An image processing apparatus is characterized in that an interpolation frame image is generated for the block area using the motion vector detected by the first motion vector detecting means.

According to a second aspect of the present invention, there is provided an image processing device for converting a frame rate by inserting an interpolated frame between frames of an input video signal, and dividing the frame of the input video signal into a plurality of blocks. A motion vector detection unit for detecting a motion vector for each block; an acquisition unit for acquiring information on a motion vector of a telop included in the input video signal; and generating an image of the interpolation frame using the motion vector. Interpolated frame generating means, wherein the motion vector detecting means is a sum of absolute values of differences between pixel values of images in the blocks for each block of one of two temporally continuous frames. A region of the image where SAD is minimum is detected from the other frame, and for each block of the one frame, the block and the detected region The positional deviation amount is output as a motion vector, and the minimum SAD is output as a minimum SAD. The interpolation frame generation means acquires the motion vector detected by the motion vector detection means by the acquisition means. Of the blocks equal to the motion vector of the telop and the blocks adjacent to the block, a block having a minimum SAD that is smaller than the minimum SAD by at least a predetermined value is acquired by the acquisition unit for the adjacent block area. An image of an interpolation frame is generated using the motion vector of the telop, and an image of the interpolation frame is generated using the motion vector detected by the motion vector detection means for the remaining block area. An image processing apparatus.

According to a third aspect of the present invention, there is provided an image processing method executed by an image processing device for converting a frame rate by inserting an interpolated frame between frames of an input video signal. Are divided into a plurality of blocks, and a first motion vector detecting step for detecting a motion vector for each block, and a block obtained by dividing at least a partial area of a frame of an input video signal by the first motion vector detecting step. A block including a telop among the blocks divided in the first motion vector detection step, and a second motion vector detection step of detecting a motion vector for each small block, and dividing the block into a plurality of small blocks that are smaller blocks A telop detection step for detecting, and an interpolation frame for generating an image of the interpolation frame using a motion vector. Includes a chromatography beam generating step, a, in the interpolation frame generation step, the blocks including the telop, to a region of the block that the block adjacent free of telop is detected by the second motion vector detection step An image of an interpolation frame is generated using the obtained motion vector, and an image of the interpolation frame is generated using the motion vector detected in the first motion vector detection step for the remaining block area. This is an image processing method.

According to a fourth aspect of the present invention, there is provided an image processing method executed by an image processing apparatus for converting a frame rate by inserting an interpolated frame between frames of an input video signal. Is divided into a plurality of blocks, a motion vector detection step for detecting a motion vector for each block, an acquisition step for acquiring motion vector information of a telop included in the input video signal, and the motion vector anda interpolation frame generation step of generating an image of the interpolation frame, and in the motion vector detection step, for each block of one frame of the two temporally successive frames, the pixel values of the image in the block An area of an image in which the SAD, which is the sum of absolute values of the differences between the two frames, is detected from the other frame, and the one frame For each block, the amount of positional deviation between the block and the detected area is output as a motion vector, and the minimum SAD is output as a minimum SAD. In the interpolation frame generation step, the motion vector detection step Among the blocks that are equal to the motion vector of the telop acquired in the acquisition step and the blocks adjacent to the block, the block having the minimum SAD that is smaller than the minimum SAD by at least a predetermined value is adjacent. For a block area, an interpolated frame image is generated using the motion vector of the telop acquired in the acquisition step, and for the remaining block area, the motion vector detected in the motion vector detection step. Can be used to generate an interpolated frame image. Which is the image processing method characterized.

  According to the present invention, it is possible to generate an interpolated frame image in which image quality deterioration in the telop area is suppressed.

1 is a block diagram illustrating an example of a configuration of an image processing apparatus according to a first embodiment. The figure which shows an example of a telop area | region and a telop edge area | region. The figure which shows an example of the setting method of a small block. The figure which shows an example of the image of the interpolation frame produced | generated. The figure which shows an example of the image of the interpolation frame produced | generated. FIG. 9 is a block diagram illustrating an example of a configuration of an image processing apparatus according to a second embodiment. The figure which shows an example of a telop area | region. The figure which shows an example of distribution of minimum SAD, a telop area | region, and a telop edge area | region. The figure which shows an example of the image of the interpolation frame produced | generated.

<Example 1>
Hereinafter, an image processing apparatus according to Embodiment 1 of the present invention and an image processing method executed by the apparatus will be described.
The image processing apparatus according to this embodiment converts the frame rate by inserting an interpolation frame between frames of the input video signal. In this embodiment, it is assumed that one interpolated frame is inserted between frames of a video signal having a frame rate of 60 fps, and the frame rate of the video signal is converted to 120 fps which is doubled. The frame rate before and after conversion is not limited to this. For example, the frame rate before conversion may be 120 fps. Two or more interpolation frames may be inserted between the frames. Specifically, two interpolation frames may be inserted between the frames, and the frame rate of the input video signal may be converted to a triple frame rate.
In this embodiment, it is assumed that a video signal is input in units of frames.

FIG. 1 is a block diagram illustrating the configuration of the image processing apparatus according to the present embodiment.
In FIG. 1, the frame delay unit 101 delays the input video signal by one frame period, and outputs it to the motion vector detection unit 102, the small block motion vector detection unit 103, and the interpolation unit 104.

  The motion vector detection unit 102 divides the frame of the input video signal into a plurality of blocks, and detects a motion vector for each block. In the present embodiment, a motion vector between the input current frame (current frame) and the previous frame (previous frame) acquired from the frame delay unit 101 is detected. The detected motion vector is output to the telop detection unit 105 and the interpolation method control unit 107.

Hereinafter, a block matching method will be described as an example of a motion vector detection method for each block.
In the block matching method, first, an image of one frame of two temporally continuous frames (current frame and previous frame) is divided into a plurality of blocks in a matrix form.
One of the plurality of blocks is set as a target block.
Next, the region of the image having the highest correlation with the block of interest is detected from the other frame.
Then, the amount of positional shift between the target block and the detected area is detected as a motion vector.
Note that, similarly to the one frame, the other frame may be divided into a plurality of blocks. Then, a block having the highest correlation with the block of interest may be detected from the plurality of blocks, and a shift amount between the position of the block of interest and the detected block may be detected as a motion vector.

The correlation is represented, for example, by the sum of absolute values (SAD) of differences between the pixel values of the image in the block of interest and the pixel values of the image in the detected area. In that case, the correlation increases as the SAD decreases. That is, the image area where the SAD is minimum is the image area having the highest correlation. Hereinafter, such SAD (minimum SAD) is referred to as minimum SAD.
In the present embodiment, the motion vector detection unit 102 detects, from the other frame, an image region in which SAD is minimized for each block of one of two temporally continuous frames. Then, for each block of one frame, the amount of positional deviation between that block and the detected area is output as a motion vector.
Note that the motion vector detection method for each block is not limited to the above method. For example, the correlation may be expressed by a value other than SAD. Specifically, the correlation may be represented by the absolute value of the difference between the representative pixel value of the image in the target block and the representative pixel value of the image in the detected area. The representative pixel value is a pixel value representing a block (region) such as a maximum value, a minimum value, an average value, or a mode value of an image. The motion vector may be detected using a gradient method. Any method may be used as long as a motion vector is detected for each block.

  The small block motion vector detection unit 103 divides at least a partial region of the frame of the input video signal into small blocks that are smaller than the blocks divided by the motion vector detection unit 102, and performs motion for each small block. A vector (small block motion vector) is detected. Hereinafter, blocks divided by the motion vector detection unit 102 are referred to as “blocks”, and blocks divided by the small block motion vector detection unit 103 are referred to as “small blocks”. The motion vector detected by the motion vector detection unit 102 is described as “motion vector”, and the motion vector detected by the small block motion vector detection unit 103 is described as “small block motion vector”. The small block motion vector may be detected using the same method (block matching method) as the motion vector detection unit 102, or may be detected using a method different from the motion vector detection unit 102.

In this embodiment, the size of the small block is set to a quarter of the size of the block divided by the motion vector detection unit 102. However, the size of the small block is not limited to this. The size of the small block may be a fixed value or a variable value. Further, the size of the small blocks may not be uniform within one frame.
For example, the size of the small block is set to a predetermined size. Then, the size of the small block is left as it is for a small block region having a sufficiently small minimum SAD. On the other hand, for the small block area where the minimum SAD is not sufficiently small (greater than a predetermined value), the size of the small block is made smaller. Thereby, the size of the small block can be optimized. This is shown in FIG.
Further, the small block motion vector detection unit 103 acquires the motion of the telop (motion vector) from the telop detection unit 105, and detects the motion vector of the telop as the small block motion vector with priority or limited. May be.

  The interpolation unit 104 generates an interpolation frame image using the interpolation motion vector acquired from the interpolation method control unit 107 (interpolation frame generation processing), and outputs the generated image. In this embodiment, for each block, an image having the size of the block (an image of an interpolation frame) is generated.

The telop detection unit 105 detects the telop region and the movement of the telop from the input video signal, and outputs the detection result to the telop end detection unit 106. Specifically, the telop detection unit 105 detects a block including a telop among the blocks divided by the motion vector detection unit 102 as a telop area (telop area).
The telop area and its motion are detected by statistically analyzing the spatial distribution of motion vectors, as in the technique disclosed in Patent Document 1, for example. However, the detection method of the telop area and its movement is not limited to this. For example, the telop area and its motion may be detected using the pixel value of the image of the input frame, the feature amount regarding the difference in pixel value between frames, or other additional information. The telop area may be detected by any method as long as a set of blocks including the telop is detected.

  The telop edge detection unit 106 detects the telop edge area (telop edge area) using the detection result of the telop area acquired from the telop detection unit 105, and outputs the detection result to the interpolation method control unit 107. Specifically, among the blocks in the telop area (blocks including the telop), the area of the block adjacent to the block not including the telop is detected as the telop end area. For the telop edge area, for example, a block corresponding to the telop area outputs 1 and a block other than 0 outputs 0, thereby creating a two-dimensional binary data array representing the detection result of the telop area. It can be detected by performing a general edge detection process on the value data array. However, the detection method of the telop edge area is not particularly limited. For example, using the position coordinates of each block in the telop area, a block located at the end of the telop area (ie, the telop end area) among the blocks in the telop area may be detected.

  The interpolation method control unit 107 acquires a motion vector from the motion vector detection unit 102, a small block motion vector from the small block motion vector detection unit 103, and information on a telop end region from the telop end detection unit 106. Then, the small block motion vector acquired from the small block motion vector detection unit 103 is output as the motion vector for interpolation of the block in the telop edge region. And the motion vector acquired from the motion vector detection part 102 is output as a motion vector for interpolation of another block.

Therefore, the interpolation unit 104 uses the small block motion vector detected by the small block motion vector detection unit 103 to interpolate an interpolated frame for a block area adjacent to a block that does not include the telop among blocks including the telop. Generate an image of In other words, in a block area adjacent to a block that does not include a telop among blocks including a telop, an image of an interpolation frame is generated for each small block.
Then, the interpolation unit 104 generates an image of the interpolation frame using the motion vector detected by the motion vector detection unit 102 for the remaining block area among the blocks including the telop. That is, in an area other than the telop edge area, an interpolated frame image is generated for each block divided by the motion vector detection unit 102.

Hereinafter, the effect of the present embodiment will be described.
FIG. 2 is a diagram illustrating an example of a part of an input frame image. In the example of FIG. 2, the text “RRRR” that is a telop moves from right to left on a stationary background including a plurality of triangular objects. In FIG. 2, the area within the range indicated by the bold line is a telop area, and the blocks in the area include the telop. A block with a cross pattern (cross mark) in the telop area indicates a block in the telop edge area. 4 and 5 are diagrams showing examples of images of interpolation frames output next to the frames of FIG. 4 and 5 are regions having the same position and size as the region of FIG.

  FIG. 4 is an example of an interpolated frame image generated using the motion vector detected by the motion vector detection unit 102 as the motion vector for interpolation. Of the blocks in the telop area shown in FIG. 2, in the block in the telop edge area, the background area tends to be larger than the telop area, so the background motion vector (0 (still) in this embodiment) Is easy to detect. On the other hand, among the blocks in the telop area, the telop motion vector is detected in the block outside the telop edge area. For this reason, in the example of FIG. 4, since there is a block in which the interpolation frame generation process using the telop motion vector is not performed in the telop edge region, the image quality is greatly deteriorated.

  Therefore, in this embodiment, for the telop edge region, an interpolation frame image is generated using the small block motion vector detected by the small block motion vector detection unit 103. On the other hand, for other regions, an interpolated frame image is generated using the motion vector detected by the motion vector detecting unit 102.

  An example of an interpolated frame image generated by this embodiment is shown in FIG. As shown in FIG. 5, in the telop edge area, the motion vector (small) detected for each small block smaller than the block shown in FIGS. An image of an interpolation frame is generated using a block motion vector. Compared to the block shown in FIG. 2, the ratio of the area of the telop included in one block is likely to be larger in the small block shown in FIG. For this reason, in a small block including a telop, the motion vector of the telop is more easily detected than in the case of FIG. As a result, as shown in FIG. 5, it is possible to generate an interpolated frame in which image quality deterioration (telop disturbance) in the telop area is suppressed.

  As described above, according to the present embodiment, for the telop edge region, an image of an interpolation frame is generated using the small block motion vector detected by the small block motion vector detection unit 103. Then, for the areas other than the telop edge area in the telop area, an interpolation frame image is generated using the motion vector detected by the motion vector detection unit 102. For this reason, in the telop edge region, it is determined in more detail whether the block includes a telop. As a result, the interpolation frame generation processing using the motion vector of the telop is easily performed at the end of the telop, and the interpolation frame in which the image quality deterioration (telop disturbance) in the telop area is suppressed can be generated. .

Also, when an interpolated frame is generated using the entire motion vector in the telop area of FIG. 2 as the telop motion vector, the background around the telop is dragged by the telop motion, and the image quality deteriorates. According to the present embodiment, in the telop edge area, an image of an interpolation frame is generated for each small block smaller than the blocks in other areas. Therefore, it is possible to reduce the area of the background (region other than the telop) in which the telop motion vector is the motion vector for interpolation, and to reduce image quality degradation caused by the background around the telop being dragged by the motion of the telop. can do.
Further, according to the present embodiment, the blocks in which the small block motion vector is used as the interpolation motion vector are limited to the blocks in the telop edge region. If the size of the block used for block matching is reduced, the amount of information contained in one block is reduced, so that erroneous detection of motion vectors is likely to occur. An erroneous detection of a motion vector leads to degradation of image quality. For example, if interpolation frame generation processing using small block motion vectors is performed on the entire telop area in FIG. 2, even if the image quality at the telop edge is improved, motion vectors are erroneously detected in blocks other than the telop edge. As a result, the image quality may be deteriorated as compared with the case of FIG. In this embodiment, the interpolation frame generation process using the small block motion vector is limited to the telop edge region, so that it is possible to suppress erroneous detection of the motion vector and, in turn, deterioration of the image quality of the telop. .

In this embodiment, for each block, the interpolation motion vector is switched between the small block motion vector acquired from the small block motion vector detection unit 103 and the motion vector acquired from the motion vector detection unit 102. The present invention is not limited to this configuration.
For example, the interpolation unit 104 generates a first (interpolated frame) image using the small block motion vector detected by the small block motion vector detection unit 103, and uses the motion vector detected by the motion vector detection unit 102. May be used to generate a second (interpolated frame) image. Then, the interpolation unit 104 generates a final interpolated frame image by synthesizing the first image and the second image at the boundary between the block region adjacent to the block not including the telop and the other region. May be. With such a configuration, it is possible to reduce a sense of incongruity caused by a sudden change in image processing (motion vector) between the telop area and other areas with the telop edge area as a boundary.

As described above, in the small block divided by the small block motion vector detection unit 103, the motion vector of the telop is more easily detected than the block divided by the motion vector detection unit 102. Therefore, at the boundary portion, the first image and the second image may be weighted and synthesized so that the weight of the first image increases toward the inside of the telop area.
The boundary portion may be the entire telop edge area or a part of the telop edge area (for example, a small block area adjacent to a block not including the telop). The boundary includes an area around the telop edge area, for example, an area of a block that does not include a telop within a predetermined range from the telop edge area, an area of a block that includes the telop, or both areas. May be. The boundary may be defined in any way as long as the above effect is obtained.

  In addition, the interpolation unit 104 uses a motion vector obtained by synthesizing the small block motion vector detected by the small block motion vector detection unit 103 and the motion vector detected by the motion vector detection unit 102 at the boundary. An image may be generated. Even with such a configuration, an effect similar to the above effect can be obtained.

Note that the small block motion vector detection unit 103 may divide the entire frame image into a plurality of small blocks, or divide a partial area of the frame (for example, only the telop end area) into a plurality of small blocks. May be. If the area for detecting the small block motion vector is made small (for example, if the area for detecting the small block motion vector is limited to the telop edge area only), the processing load for detecting the small block motion vector can be reduced. . Moreover, the erroneous detection of the motion vector mentioned above can be suppressed.
Note that “adjacent” may include being adjacent in an oblique direction, and may be limited to being adjacent in the horizontal direction and the vertical direction.

<Example 2>
Hereinafter, an image processing apparatus according to Embodiment 2 of the present invention and an image processing method executed by the apparatus will be described.
FIG. 6 is a block diagram illustrating the configuration of the image processing apparatus according to the present embodiment.

In FIG. 6, the frame delay unit 201 delays the input video signal by one frame period and outputs it to the motion vector detection unit 202 and the interpolation unit 203.
The motion vector detection unit 202 divides each frame of the input video signal into a plurality of blocks, and detects a motion vector for each block. Specifically, the motion vector detection unit 202 detects, from the other frame, an image area in which SAD is minimized for each block of one of two temporally continuous frames. Then, for each block of one frame, the shift amount of the position between the block and the detected area is output as a motion vector, and the minimum SAD is output as the minimum SAD. The motion vector and the minimum SAD are output to the telop detection unit 204, the telop end detection unit 205, and the interpolation method control unit 206.
The interpolation unit 203 generates an interpolation frame image using the interpolation motion vector acquired from the interpolation method control unit 206 (interpolation frame generation processing), and outputs the generated image.

The telop detection unit 204 acquires information on the motion vector of the telop included in the input video signal. In this embodiment, a telop area and a motion (motion vector) of the telop are detected from the input video signal. The detection result is output to the telop edge detection unit 205.
Specifically, the telop detection unit 204 generates a motion vector histogram acquired from the motion vector detection unit 202.
Next, when the frequency of the category corresponding to the horizontal or vertical motion vector of the histogram is equal to or greater than a predetermined number, the telop detection unit 204 detects the motion vector of the category as the telop motion vector. When a plurality of motion vectors having different sizes belong to a horizontal or vertical vector category whose frequency is equal to or greater than a predetermined number, the representative value of the size of the motion vector belonging to the category is used as the motion vector of the telop. Should be the size. The representative value is, for example, a maximum value, a minimum value, an average value, a mode value, or the like.
Then, the telop detection unit 204 detects, as a telop region, a block region in which the motion vector detected by the motion vector detection unit 202 is equal to the detected telop motion vector. It should be noted that the motion vector detected by the motion vector detection unit 202 and the motion vector of the telop detected by the telop detection unit 204 may not completely match. That is, the telop area may be detected in consideration of the error.
Further, the telop detection unit 204 assumes that a background motion vector is detected at the end of the telop, and also sets a block adjacent to the block as the telop area as the telop area. In other words, in this embodiment, a block where the motion vector detected by the motion vector detection unit 202 is equal to the motion vector of the telop acquired by the telop detection unit 204 and a block adjacent to the block are defined as telop regions.
Note that the telop area and its motion detection method are not limited thereto. Information on the telop area and the motion vector of the telop may be acquired from the outside. Information on the telop area and the motion vector of the telop may be added to the video signal as metadata.

  The telop end detection unit 205 detects the telop end region using the minimum SAD acquired from the motion vector detection unit 202 and the output of the telop detection unit 204, and outputs the detection result to the interpolation method control unit 206. Specifically, among the blocks in the telop area, the area of the block adjacent to the minimum SAD block that is at least a predetermined value smaller than its own minimum SAD is detected as the telop edge area.

  The interpolation method control unit 206 acquires a motion vector from the motion vector detection unit 202, a telop motion vector from the telop detection unit 204, and information on a telop end region from the telop end detection unit 205. Then, the motion vector of the telop acquired from the telop detection unit 204 is output as the motion vector for interpolation of the block in the telop edge region. Then, the motion vector acquired from the motion vector detection unit 202 is output as the motion vector for interpolation of other blocks.

Therefore, the interpolation unit 203 generates an interpolated frame image using the motion vector of the telop acquired by the telop detection unit 204 for the block in the telop end region among the blocks in the telop region.
Then, the interpolation unit 203 generates an interpolated frame image using the motion vector detected by the motion vector detection unit 202 for the remaining block region among the blocks in the telop region.

Hereinafter, the effect of the present embodiment will be described.
FIG. 7 is a diagram illustrating an example of a part of an input frame image. As shown in FIG. 7, in this embodiment, a block in which the motion vector detected by the motion vector detection unit 202 is equal to the motion vector of the telop acquired by the telop detection unit 204 and the area of the block adjacent to the block are telops. It is considered as an area. In the case of such a configuration, as shown in FIG. 7, the telop area may be larger than the area composed of blocks including the telop. That is, the telop area may include a block area that does not include the telop (including only the background). When the telop edge area is detected for such a telop area by the same method as in the first embodiment, a block including only the background is set as the telop edge area, and the background around the telop is largely disturbed by the movement of the telop. There is a risk of becoming. On the contrary, if only the block in which the motion vector of the telop is detected is detected as the telop area, the telop area may be smaller than the area including the block including the telop. When a telop edge area is detected for the detected telop area in the same manner as in the first embodiment, an interpolation frame generation process using a background motion vector is performed at the edge part of the telop (a block including the telop). The expected image quality improvement effect cannot be obtained.

Therefore, in this embodiment, the minimum SAD is analyzed to detect the telop edge region with high accuracy.
An example of the minimum SAD of each block when the frame of FIG. 7 is input is shown in FIG.
A block that does not include a telop (background) has a minimum SAD of zero. This means that in the block, perfect matching is obtained between frames, and a very high correlation is obtained.
On the other hand, the minimum SAD of the block including the telop is larger than that of the background block (300 in the example of FIG. 8). This means that the correlation between frames is lower in a block including a telop than in a background block. In a block including a telop, a telop (foreground) and a background are mixed and their movements are different, so that the relative positional relationship between the telop and the background changes for each frame. Also, by moving the telop (foreground), the background that was hidden under the telop in one frame appears in the next frame, or the background that was viewed in reverse is hidden in the telop in the next frame. Therefore, the correlation between frames is reduced in a block including a telop, and the minimum SAD is increased as compared with the background block.

It is considered that the background block is adjacent to the block at the end of the telop. In other words, the block at the end of the telop is considered to be adjacent to a block having a minimum SAD that is at least a predetermined value smaller than its own minimum SAD.
Therefore, in the present embodiment, among the blocks in the telop area, the area of the block adjacent to the minimum SAD block that is at least a predetermined value smaller than its own minimum SAD is detected as the telop end area. Thereby, the telop edge region can be detected with high accuracy.
The telop edge region is subjected to interpolation frame generation processing using the telop motion vector acquired by the telop detection unit 204, and the other region is detected by the motion vector detection unit 202. An interpolation frame generation process is performed using the motion vector.
As a result, as shown in FIG. 9, the interpolation frame generation processing using the motion vector of the telop is performed on the block area including the telop, and the deterioration of the image quality in the telop area can be suppressed.

As described above, according to the present embodiment, a block in which the motion vector detected by the motion vector detection unit 202 is equal to the motion vector of the telop acquired by the telop detection unit 204 and a block area adjacent to the block are included. The telop area. Of the blocks in the telop area, the area of the block adjacent to the minimum SAD block that is at least a predetermined value smaller than its own minimum SAD is set as the telop end area. Thereby, the telop edge region can be detected with high accuracy.
An interpolation frame image is generated for the telop edge region using the telop motion vector acquired by the telop detection unit 204. An interpolation frame image is generated using a motion vector detected by the motion vector detection unit 202 for an area other than the telop edge area in the telop area. Thereby, the interpolation frame generation processing using the motion vector of the telop can be performed on the block region including the telop, and the deterioration of the image quality in the telop region can be suppressed.

In this embodiment, the interpolation motion vector is switched between the telop motion vector acquired from the telop detection unit 204 and the motion vector acquired from the motion vector detection unit 202 for each block. Is not limited to this configuration.
For example, the interpolation unit 203 generates a first (interpolated frame) image using the telop motion vector acquired by the telop detection unit 204, and uses the motion vector detected by the motion vector detection unit 202. Two (interpolated frame) images may be generated. Then, the interpolation unit 203 synthesizes the first image and the second image at the boundary between the area of the block adjacent to the block having the minimum SAD that is smaller than the minimum SAD by at least a predetermined value and the other area. Thus, an image of the final interpolation frame may be generated. With such a configuration, it is possible to reduce a sense of incongruity caused by a sudden change in image processing (motion vector) between the telop area and other areas with the telop edge area as a boundary.

Specifically, at the boundary portion, the first image and the second image may be weighted and synthesized so that the weight of the first image increases toward the inside of the telop area.
The boundary part may be a telop edge area or an area other than the telop edge area. The boundary portion may include an area around the telop edge area, for example, an area of a block not including a telop within a predetermined range from the telop edge area, an area of a block including the telop, or both areas. Good. The boundary may be defined in any way as long as the above effect is obtained.

  In addition, the interpolation unit 104 generates an image of an interpolation frame at the boundary using a motion vector obtained by combining the motion vector of the telop acquired by the telop detection unit 204 and the motion vector detected by the motion vector detection unit 202. May be. Even with such a configuration, an effect similar to the above effect can be obtained.

102 motion vector detection unit 103 small block motion vector detection unit 104 interpolation unit 105 telop detection unit 202 motion vector detection unit 203 interpolation unit 204 telop detection unit

Claims (6)

An image processing apparatus for converting a frame rate by inserting an interpolation frame between frames of an input video signal,
First motion vector detection means for dividing a frame of an input video signal into a plurality of blocks and detecting a motion vector for each block;
The at least part of the frame of the input video signal is divided into a plurality of small blocks that are smaller than the blocks divided by the first motion vector detecting means, and a motion vector for each small block is detected. Two motion vector detection means;
A telop detection means for detecting a block including a telop among the blocks divided by the first motion vector detection means;
Interpolation frame generation means for generating an image of the interpolation frame using a motion vector;
Have
The interpolation frame generation means includes a block including the telop.
For an area of a block adjacent to a block that does not include a telop, an interpolation frame image is generated using the motion vector detected by the second motion vector detection unit,
An image processing apparatus for generating an interpolated frame image for the remaining block area using the motion vector detected by the first motion vector detecting means. The interpolation frame generation means includes an image generated using the motion vector detected by the second motion vector detection means at a boundary portion between a block area adjacent to the block not including the telop and another area, and the image An image generated using the motion vector detected by the first motion vector detection means is synthesized, or the motion vector detected by the second motion vector detection means and the first motion vector detection means are detected. The image processing apparatus according to claim 1, wherein an image of the interpolation frame is generated by using a motion vector synthesized with a motion vector. 3. The second motion vector detecting means divides only a block area adjacent to a block not including a telop among the blocks including the telop into the plurality of small blocks. Image processing apparatus. An image processing method executed by an image processing apparatus for converting a frame rate by inserting an interpolation frame between frames of an input video signal,
A first motion vector detection step of dividing a frame of the input video signal into a plurality of blocks and detecting a motion vector for each block;
First, a region of at least a part of the frame of the input video signal is divided into a plurality of small blocks that are smaller than the block divided in the first motion vector detection step, and a motion vector for each small block is detected. Two motion vector detection steps;
A telop detection step of detecting a block including a telop among the blocks divided in the first motion vector detection step;
An interpolation frame generation step of generating an image of the interpolation frame using a motion vector;
Have
In the interpolation frame generation step, among the blocks including the telop,
An interpolated frame image is generated using the motion vector detected in the second motion vector detection step for an area of a block adjacent to a block not including a telop,
An image processing method, wherein an image of an interpolation frame is generated for the remaining block area using the motion vector detected in the first motion vector detection step. In the interpolation frame generation step, an image generated using the motion vector detected in the second motion vector detection step at a boundary portion between a block area adjacent to the block not including the telop and another area, and the block The image generated using the motion vector detected in the first motion vector detection step is synthesized, or the motion vector detected in the second motion vector detection step and the motion vector detected in the first motion vector detection step The image processing method according to claim 4 , wherein an image of the interpolation frame is generated by using a motion vector synthesized with a motion vector. The said 2nd motion vector detection step WHEREIN: Only the area | region of the block which the block which does not contain a telop among the blocks which contain the said telop is adjacent is divided | segmented into these several small blocks, The Claim 4 or 5 characterized by the above-mentioned. The image processing method as described.

Images