JP2013219453A - Video signal processing apparatus and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video signal processing apparatus in which a sense of visual incompatibility caused by discontinuity of interpolation images can be reduced.SOLUTION: A motion vector detector 2 detects a motion vector V1 over a wider range than a range of a moving object with a past frame or a present frame of a video signal Sin as a search reference. An interpolation frame generator 3 generates an interpolation frame f0.5. A motion vector filter 4 applies filtering to the motion vector V1 and outputs a result as a motion vector V2. A differential calculator 5 calculates a differential between the motion vectors V1 and V2 and generates a boundary determination signal Sjb. An interpolation frame filter 6 applies filtering to the interpolation frame f0.5 on the basis of the boundary determination signal Sjb and defines a result as an interpolation frame f0.5'.

Description

  The present invention relates to a video signal processing apparatus and method for generating an interpolation frame using a motion vector of an image, and more particularly, a non-interpolated image generated when a motion vector detected in a motion region affects a background of a still region. The present invention relates to a video signal processing apparatus and method capable of reducing visual discomfort caused by continuity.

  When a moving image is displayed on an image display device using a liquid crystal panel, an afterimage tends to occur. Therefore, in order to reduce the afterimage and improve the moving image characteristics, the number of frames is increased by interpolating interpolated frames between the actual frames of the video signal. For example, the frame rate (frame frequency) of the vertical frequency 60 Hz is doubled to 120 Hz. Alternatively, the image is displayed after being converted to a frame rate higher than that. In an image display device that displays an image by converting the frame rate of a video signal, a motion vector of the image is detected, each interpolation pixel is generated using the motion vector, and an interpolation frame that is interpolated between actual frames is generated. .

  As one of the methods for detecting the motion vector of an image, the motion vector is detected as a reference frame when searching for a direction having the highest correlation with the past frame or the current frame, which is an actual frame positioned before and after the interpolation frame. There is a way to do it. In the method of detecting a motion vector using the past frame or the current frame as a search reference, the detection position of the pixel (block) that detected the motion vector in the reference frame and the interpolation that generates the interpolation pixel (interpolation block) in the interpolation frame The position will shift.

  Therefore, when a predetermined object moves in the moving region, the motion vector of the object is detected in a range wider than the range of the moving object. In this way, it is possible to generate an interpolated image interpolated with the correct motion vector in the interpolated frame even using a motion vector detected using the past frame or the current frame as a search reference.

JP 2009-55340 A

  However, when a motion vector of an object is detected in a range wider than the range of the moving object and used to generate an interpolated image in the interpolation frame, a part of the motion vector detected in the motion area is adjacent to the motion area. This is also applied when generating an interpolated image of the background of the still region. Then, in the background, there is a problem that the boundary between the region interpolated by applying the motion vector of the moving object and the region correctly interpolated as a still image becomes discontinuous, resulting in visual discomfort.

  In view of such a problem, the present invention detects a motion vector of an object in a wider range than the range of a moving object and generates an interpolation frame to generate a visual result due to discontinuity of an interpolation image. It is an object of the present invention to provide a video signal processing apparatus and method that can reduce a sense of discomfort.

  In order to solve the above-described problems of the related art, the present invention uses a past frame or a current frame of an input video signal (Sin) as a search reference, and has a wider range than the range of a moving object in the video signal. Using the motion vector detection unit (2) for detecting the first motion vector (V1) indicating the motion of the object, the past frame, the current frame, and the first motion vector, the past frame and the An interpolation frame generation unit (3) for generating a first interpolation frame (f0.5) to be interpolated between the current frame and a first filter processing on the first motion vector; A motion vector filter unit (4) that outputs as a second motion vector (V2), and by calculating a difference between the first motion vector and the second motion vector, A predetermined region including a boundary line between a first region in which an interpolated image is generated based on the first motion vector in a background of a still region to be placed and a second region in which an interpolated image is generated based on the still vector Based on the boundary determination signal, a difference calculation unit (5) that generates a boundary determination signal (Sjb) indicating a boundary area that is a range area, and a second value for the boundary area in the first interpolation frame. There is provided an image signal processing device comprising an interpolation frame filter unit (6) that performs a filter process and outputs a second interpolation frame (f0.5 ′).

  In the video signal processing apparatus, the motion vector filter unit is preferably a low-pass filter.

  In the video signal processing apparatus, the interpolation frame filter unit is preferably a low-pass filter.

  In the video signal processing device, the difference calculation unit generates a signal including a plurality of steps in accordance with a distance from the boundary line in the boundary region as the boundary determination signal, and the interpolation frame filter unit It is preferable that the number of taps in the second filter processing is made different according to the plurality of steps based on the boundary determination signal.

  In the above video signal processing device, it is preferable that the interpolation frame filter unit increases the number of taps in the second filter processing as it is closer to the boundary line in the boundary region.

  In order to solve the above-described problems of the conventional technology, the present invention uses the past frame or the current frame of the input video signal (Sin) as a search reference, and is wider than the range of moving objects in the video signal. Detecting a first motion vector (V1) indicating a motion of the object in a range, and using the past frame, the current frame, and the first motion vector, between the past frame and the current frame; A first interpolation frame (f0.5) to be interpolated is generated, a first filtering process is performed on the first motion vector, and it is output as a second motion vector (V2). An interpolation image is generated based on the first motion vector in the background of the still region located outside the object by calculating the difference between the motion vector of the second motion vector and the second motion vector Generating a boundary determination signal (Sjb) indicating a boundary region that is a region in a predetermined range including a boundary line between the first region and the second region where the interpolation image is generated based on the still vector, and the boundary determination signal The video signal processing method is characterized in that a second filter process is performed on the boundary region in the first interpolation frame and the result is output as a second interpolation frame (f0.5 ′). To do.

  In the video signal processing method, the first filter processing is preferably filter processing using a low-pass filter.

  In the video signal processing method, the second filter processing is preferably filter processing using a low-pass filter.

  In the video signal processing method, as the boundary determination signal, a signal including a plurality of steps is generated in the boundary region according to a distance from the boundary line, and according to the plurality of steps based on the boundary determination signal It is preferable to vary the number of taps in the second filter processing.

  In the video signal processing method, it is preferable that the number of taps in the second filter processing is increased as the boundary line is closer to the boundary line.

  According to the video signal processing apparatus and method of the present invention, the motion vector of an object is detected in a wider range than the range of the moving object, and this is caused by the discontinuity of the interpolation image that occurs when the interpolation frame is generated. Visual discomfort can be reduced.

It is a block diagram which shows one Embodiment of the video signal processing apparatus of this invention. It is a figure for demonstrating the detection operation of the motion vector which made the search reference | standard the present flame | frame by the motion vector detection part 2 in FIG. It is a figure for demonstrating the detection operation | movement which detects a motion vector in the range wider than the range of the moving object by the motion vector detection part 2 in FIG. It is a figure for demonstrating the visual phenomenon which generate | occur | produces when the motion vector detected in the range wider than the range of the moving object is applied also to the background of a still region. It is a figure for demonstrating operation | movement of the motion vector filter part 4 in FIG. It is a figure for demonstrating operation | movement of the difference calculation part 5 in FIG. It is an operation of the interpolation frame filter unit 6 in FIG. 1, and is a diagram for explaining an effect according to one embodiment.

  Hereinafter, an embodiment of a video signal processing apparatus and method according to the present invention will be described with reference to the accompanying drawings. 1 shows a frame frequency conversion device as an example of a video signal processing device. In FIG. 1, each pixel data of the input video signal Sin is sequentially input to the frame memory 1, the motion vector detection unit 2, the interpolation frame generation unit 3, and the frame frequency conversion unit 7. The video signal Sin has a frame frequency of 60 Hz, for example.

  The frame memory 1 outputs the input pixel data with a delay of one frame period. Assume that the current frame of the video signal Sin is the current frame f0, and the previous frame output from the frame memory 1 is the past frame f1. The interpolation frame generated by the interpolation frame generation unit 3 is defined as an interpolation frame f0.5. Each pixel data of the past frame f1 output from the frame memory 1 is sequentially input to the motion vector detection unit 2 and the interpolation frame generation unit 3.

  The motion vector detection unit 2 uses, for example, a matching method and uses the pixels of the current frame f0 and the past frame f1, and each interpolation pixel of the interpolation frame f0.5 to be interpolated between the current frame f0 and the past frame f1. A motion vector V1 for generation is detected. The motion vector V1 is input to the interpolation frame generation unit 3, the motion vector filter unit 4, and the difference calculation unit 5.

  The motion vector V1 may be detected in units of pixels, or may be detected in units of blocks composed of a plurality of pixels. A block is assumed to be one or a plurality of pixels. Although not particularly illustrated, the motion vector detection unit 2 includes a plurality of line memories, a pixel delay unit, a subtractor for obtaining a difference between pixel data, and the like.

  The operation of detecting the motion vector V1 by the motion vector detection unit 2 will be further described with reference to FIG. FIG. 2 shows a state where the object OB located in the past frame f1 has moved to the illustrated position in the current frame f0. Ideally, an object OBi based on an interpolation image as shown by a broken line is generated in the interpolation frame f0.5. In the present embodiment, the motion vector detection unit 2 detects the motion vector V1 using the current frame f0 as a search reference. The motion vector V1 may be detected using the past frame f1 as a search reference, or the motion vector V1 may be detected using both the past frame f1 and the current frame f0 as a search reference.

  As shown in FIG. 2, using the current frame f0 as a search criterion, correlations (difference values) between the block of interest in the current frame f0 and a plurality of blocks in the past frame f1 are detected as indicated by thin arrow lines. By comparing the correlations with each other, the direction with the smallest difference value is determined as the direction with the highest correlation. In this way, the motion vector detection unit 2 can detect the motion vector V1 indicating the direction with the highest correlation and the amount of motion.

  When the motion vector V1 detected using the current frame f0 as a search reference is applied to the interpolation frame f0.5 as indicated by the broken arrow line, the detection position of the motion vector V1 and the interpolation position for generating the object OBi are shifted. . Therefore, the motion vector detection unit 2 is configured to detect the motion vector V1 of the object OB in a range wider than the range of the moving object OB, as shown in FIG.

  In FIG. 3, the motion vector V1 applied to the object OB in the current frame f0 indicates the motion vector detected as shown in FIG. 2 in each block for detecting the motion vector V1. The motion vector V1 located on both outer sides of the object OB is a motion vector detected by configuring so as to detect the motion vector V1 of the object OB in a range wider than the range of the moving object OB. The motion vector V1 located on both outer sides of the object OB correctly indicates the moving direction of the object OB.

  If the motion vector V1 is detected only in the range of the moving object OB, the motion vector V1 located on both outer sides of the object OB does not become a stationary vector or a motion vector V1 that correctly indicates the moving direction of the object OB. Or

  There are various methods for extending the detection range of the motion vector V1 in this way, and any method may be employed. For example, the detection range of the motion vector V1 can be expanded by adding the difference value obtained in the neighboring block to the difference value obtained in the target block from which the motion vector V1 is to be detected. Further, the detection range of the motion vector V1 can be expanded by referring to the detection result of the motion vector V1 detected in the past frame.

  The interpolation frame generation unit 3 generates the interpolation frame f0.5 based on the motion vector V1 detected by the motion vector detection unit 2 as described above. As shown in FIG. 3, by applying a motion vector V1 indicating the motion of the object OB detected in a wider range than the range of the object OB to the interpolation frame f0.5 as indicated by a broken arrow line, the motion vector V1 is obtained. Are not shifted from the interpolation position for generating the object OBi. Therefore, it is possible to generate the object OBi by the interpolation image interpolated by the correct motion vector V1 within the interpolation frame f0.5.

  In FIG. 3, the region located on the right side of the object OB is the background Rbg of the still image. As shown in FIG. 3, a part of the motion vector V1 with an expanded detection range is also applied to the background Rbg. A region Ra adjacent to the object OBi in the background Rbg is an interpolation image generated based on the motion vector V1. The region Rb outside the region Ra is a still image interpolation image generated by a still vector.

  A visual phenomenon that occurs when the motion vector V1 generated to generate the object OBi is also applied to the background Rbg of the still region will be described with reference to FIG. FIG. 4 shows, as an example, a still image in which the background Rbg has gray levels sequentially increasing from the left side to the right side of the figure. As shown in FIG. 4A, the motion vector V1 is applied to the region Ra surrounded by the broken line, and the motion vector V1 is not applied to the region Rb. A correctly detected still vector is applied to the region Rb.

  FIG. 4B shows an interpolation image generated in the interpolation frame f0.5. The portion corresponding to the region Ra is an interpolated image IMa generated based on the motion vector V1, and the portion corresponding to the region Rb is an interpolated image IMb generated based on the still vector. As shown in FIG. 4B, a boundary line Lbdr is generated at the boundary between the interpolated image IMa and the interim image IMb because the interpolated image IMa and the interim image IMb are discontinuous. This boundary line Lbdr is visually uncomfortable.

  Therefore, in the video signal processing apparatus and method of the present embodiment, the configuration shown in FIG. 1 is used to make the boundary line Lbdr inconspicuous, thereby reducing visual discomfort. In FIG. 1, a motion vector filter unit 4 performs a filtering process on an input motion vector V1 and outputs it as a motion vector V2. The filter in the motion vector filter unit 4 is preferably a low-pass filter. As the motion vector filter unit 4, a 3-tap low-pass filter can be used.

  The operation of the motion vector filter unit 4 will be described with reference to FIG. FIG. 5A is the same as FIG. The motion vector V1 in the region Ra has a predetermined motion vector value. In the region Rb, the motion vector value is 0 because it is a static vector. Therefore, the motion vector value of the motion vector V1 changes sharply as shown by the solid line in FIG. 5B in accordance with the pixel position in the horizontal direction. When the motion vector filter unit 4 applies a filtering process to the motion vector V1 using a low-pass filter, a motion vector V2 having a gentle characteristic indicated by a broken line is obtained. A hatched portion that is the difference between the motion vector V1 and the motion vector V2 becomes a boundary region Rbdr of a predetermined region including a boundary between the region Ra and the region Rb.

  The width of the boundary region Rbdr can be adjusted by appropriately setting a filter in the motion vector filter unit 4.

  Returning to FIG. 1, the difference calculation unit 5 calculates the absolute difference between the motion vector V1 and the motion vector V2. A plurality of threshold values are set in the difference calculation unit 5, and the difference calculation unit 5 compares the difference absolute value between the motion vector V1 and the motion vector V2 with the plurality of threshold values, thereby comparing the region Ra and the region Rb. The distance from the boundary of is roughly detected. The difference calculation unit 5 generates a boundary determination signal Sjb having a plurality of steps according to the distance from the boundary between the region Ra and the region Rb.

  The operation of the difference calculation unit 5 that generates the boundary determination signal Sjb will be described with reference to FIG. In FIG. 6, for the sake of simplification, two threshold values TH1 and TH2 are shown. In the portion outside the boundary region Rbdr, the difference absolute value between the motion vector V1 and the motion vector V2 is 0, and the difference calculation unit 5 sets the boundary determination signal Sjb to “0”. The difference calculation unit 5 sets the boundary determination signal Sjb to “1” if the absolute difference value exceeds 0 and reaches the threshold value TH1, and sets the boundary determination signal Sjb if the absolute difference value exceeds the threshold value TH1 and reaches the threshold value TH2. Set to “2”. The difference calculation unit 5 sets the boundary determination signal Sjb to “3” when the absolute difference value exceeds the threshold value TH2. The numerical values “0” to “3” of the boundary determination signal Sjb are merely examples.

  The boundary determination signal Sjb generated by the difference calculation unit 5 is input to the interpolation frame filter unit 6. The interpolation frame filter unit 6 performs a filtering process on the interpolation frame f0.5 generated by the interpolation frame generation unit 3, and outputs the result as an interpolation frame f0.5 '. The filter in the interpolation frame filter unit 6 is preferably a low-pass filter. The interpolation frame filter unit 6 preferably increases the number of taps of the low-pass filter as the difference absolute value between the motion vector V1 and the motion vector V2 increases, according to the step by the boundary determination signal Sjb.

  When the boundary determination signal Sjb is “0” to “3”, when the boundary determination signal Sjb is “3”, the filter processing range is widened as the largest number of taps, and when the boundary determination signal Sjb is “1” What is necessary is just to narrow the range of filter processing as the minimum tap number. More threshold values may be set in the difference calculation unit 5, and the boundary determination signal Sjb may be a signal having more steps. If the value of the boundary determination signal Sjb is “3”, the number of taps of the low-pass filter may be 3 taps, and if the value of the boundary determination signal Sjb is “15”, the number of taps of the low-pass filter may be 15 taps.

  FIG. 7A shows an interpolation image by the interpolation frame f0.5 output from the interpolation frame generation unit 3, and is the same as FIG. 4B. A boundary line Lbdr is generated between the interpolated image IMa and the inter-image IMb. FIG. 7B is an interpolation image by the interpolation frame f0.5 ′ output from the interpolation frame filter unit 6. Since the interpolation frame filter unit 6 performs filtering on the boundary region Rbdr, the interpolated image in the boundary region Rbdr is corrected and the boundary line Lbdr becomes inconspicuous. This can reduce the visual discomfort. The interpolated frame f0.5 'is input to the frame frequency conversion unit 7.

  The frame frequency conversion unit 7 alternately inputs the current frame f0 and the interpolation frame f0.5 ′ in the order of the interpolation frame f0.5 ′ and the current frame f0 at 120 Hz that is twice the frequency of the video signal Sin. Output to. As a result, the video signal Sout having a frame frequency of 120 Hz is output from the frame frequency converter 7.

  The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the scope of the present invention. In the present embodiment described above, the filtering process in the horizontal direction has been described. However, the filtering process in the vertical direction may be added.

  Moreover, although this embodiment demonstrated the frame frequency converter which converts a frame frequency into 2 times as an example of a video signal processing apparatus, the frame frequency converter which converts a frame frequency into 3 times or more may be sufficient. Furthermore, the present invention is not applied only to the frame frequency conversion apparatus, but can be applied to any video signal processing apparatus and method for generating an interpolation frame based on a motion vector.

DESCRIPTION OF SYMBOLS 1 Frame memory 2 Motion vector detection part 3 Interpolation frame production | generation part 4 Motion vector filter part 5 Difference calculation part 6 Interpolation frame filter part 7 Frame frequency conversion part

Claims (10)

A motion vector detection unit for detecting a first motion vector indicating the motion of the object in a wider range than the range of the moving object in the video signal using a past frame or a current frame of the input video signal as a search reference; ,
An interpolation frame generation unit that generates a first interpolation frame to be interpolated between the past frame and the current frame using the past frame, the current frame, and the first motion vector;
A motion vector filter unit that performs a first filter process on the first motion vector and outputs a second motion vector;
By calculating the difference between the first motion vector and the second motion vector, an interpolated image is generated based on the first motion vector in the background of the still region located outside the object. A difference calculating unit that generates a boundary determination signal indicating a boundary region that is a region of a predetermined range including a boundary line between the first region and a second region in which an interpolation image is generated based on a still vector;
An interpolation frame filter unit that performs a second filter process on the boundary region in the first interpolation frame based on the boundary determination signal, and outputs the result as a second interpolation frame;
A video signal processing apparatus comprising:   The video signal processing apparatus according to claim 1, wherein the motion vector filter unit is a low-pass filter.   The video signal processing apparatus according to claim 1, wherein the interpolation frame filter unit is a low-pass filter. The difference calculation unit generates a signal including a plurality of steps according to a distance from the boundary line in the boundary region as the boundary determination signal,
The video according to any one of claims 1 to 3, wherein the interpolation frame filter unit varies the number of taps in the second filter processing according to the plurality of steps based on the boundary determination signal. Signal processing device.   The video signal processing apparatus according to claim 4, wherein the interpolation frame filter unit increases the number of taps in the second filter processing as it is closer to the boundary line in the boundary region. Detecting a first motion vector indicating a motion of the object in a wider range than a range of a moving object in the video signal, using a past frame or a current frame of the input video signal as a search reference;
Using the past frame, the current frame, and the first motion vector to generate a first interpolation frame to be interpolated between the past frame and the current frame;
Applying a first filtering process to the first motion vector and outputting it as a second motion vector;
By calculating the difference between the first motion vector and the second motion vector, an interpolated image is generated based on the first motion vector in the background of the still region located outside the object. Generating a boundary determination signal indicating a boundary region that is a region in a predetermined range including a boundary line between the first region and a second region in which an interpolation image is generated based on the static vector;
A video signal processing method comprising: performing a second filter process on the boundary region in the first interpolation frame based on the boundary determination signal and outputting the result as a second interpolation frame.   7. The video signal processing method according to claim 6, wherein the first filter processing is filter processing using a low-pass filter.   8. The video signal processing method according to claim 6, wherein the second filter processing is filter processing using a low-pass filter. As the boundary determination signal, a signal including a plurality of steps is generated in accordance with the distance from the boundary line within the boundary region,
The video signal processing method according to any one of claims 6 to 8, wherein the number of taps in the second filter processing is made different according to the plurality of steps based on the boundary determination signal.   The video signal processing method according to claim 9, wherein the number of taps in the second filter processing is increased as the boundary line is closer to the boundary area.