JP2013201607A - Motion vector correction device and method, and, video signal processing device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a motion vector correction device which is appropriate for a case in which a plurality of interpolation frames are inserted among actual frames of video signal, capable of further reducing visual discomfort of interpolation images even if error detection occurs when a motion vector is relatively large.SOLUTION: A motion vector detection part 2 detects a motion vector, using pixel data in at least two actual frames of video signal, which is required when generating interpolation pixel data constituting a plurality of interpolation frames inserted between two actual frames. A motion vector correction part 4 performs correction such that a motion vector is attenuated according to the phase of a plurality of interpolation frames between two actual frames.

Description

  The present invention relates to a motion vector correction apparatus and method, and a video signal processing apparatus and method that can reduce visual discomfort in an interpolated image due to erroneous detection of a motion vector.

  When a moving image is displayed on an image display device using a liquid crystal panel, an afterimage tends to occur. Therefore, in order to reduce the afterimage, an interpolation frame is interpolated between the actual frames of the video signal to increase the number of frames. For example, the frame rate at a vertical frequency of 60 Hz is converted to a double vertical frequency of 120 Hz or higher. It is done to display. In a video signal processing apparatus that performs frame rate conversion, a motion vector of an image is detected, each interpolation pixel is generated using the motion vector, and an interpolation frame that is interpolated between actual frames is generated.

  The motion vector is generated based on a limited range of pixel data within a limited frame of, for example, two consecutive frames. Therefore, it is difficult to completely eliminate false detection of motion vectors. Therefore, technical improvements have been made to reduce false detection of motion vectors. The present applicant has disclosed a motion vector correction apparatus and method, and a video signal processing apparatus and method capable of reducing the visual discomfort of an interpolated image even when a motion vector is erroneously detected in Patent Document 1. Proposed.

JP2011-239325A

  In Patent Document 1, as a video signal processing device, a frame rate conversion device that interpolates one interpolated frame between real frames of a video signal and converts the frame rate to twice is taken as an example. In recent years, a frame rate conversion device has been introduced that interpolates three interpolation frames between actual frames of a video signal and converts the frame rate by four times. Therefore, in a video signal processing apparatus that interpolates two or more interpolated frames between real frames of the video signal, it is required to further reduce the visual discomfort of the interpolated image even when erroneous detection of a motion vector occurs. .

  In order to meet such a demand, the present invention is suitable for interpolating a plurality of interpolated frames between actual frames of a video signal, and even when an erroneous detection occurs when a motion vector is relatively large, an interpolated image It is an object of the present invention to provide a motion vector correction apparatus and method, and a video signal processing apparatus and method that can further reduce visual discomfort.

  In order to solve the above-described problems of the related art, the present invention uses a plurality of interpolated frames to be interpolated between two actual frames using pixel data in at least two actual frames in an input video signal. A motion vector detection unit (2) for detecting a motion vector necessary for generating the interpolated pixel data to constitute, and attenuating the motion vector according to the phase of the plurality of interpolation frames between the two real frames There is provided a motion vector correction device comprising a motion vector correction unit (4) for correcting the motion.

  In the motion vector correction device, the motion vector correction unit is configured to reduce motion vector attenuation used when generating interpolation pixel data constituting an interpolation frame having a phase away from the temporal center between the two actual frames. It is preferable that the degree is larger than the degree of attenuation of the motion vector used when generating the interpolated pixel data constituting the interpolated frame having the phase located at the center.

  In addition, in order to solve the above-described problems of the related art, the present invention uses a plurality of interpolations to interpolate between two actual frames using pixel data in at least two actual frames in an input video signal. A motion vector necessary for generating interpolation pixel data constituting a frame is detected, and the motion vector is corrected so as to be attenuated according to the phase of the plurality of interpolation frames between the two actual frames. A motion vector correction method is provided.

  In the motion vector correction method, the degree of motion vector attenuation used when generating interpolation pixel data constituting an interpolation frame having a phase away from the temporal center between the two actual frames is positioned at the center. It is preferable that the degree of attenuation of the motion vector used when generating the interpolated pixel data constituting the interpolated frame of the phase to be larger is set.

  Furthermore, in order to solve the above-described problems of the conventional technology, the present invention uses a plurality of interpolations to interpolate between two actual frames using pixel data in at least two actual frames in an input video signal. A motion vector detector (2) for detecting a first motion vector necessary for generating interpolated pixel data constituting a frame; and the plurality of interpolations between the two actual frames by using the first motion vector. A motion vector correction unit (4) that outputs a second motion vector that is corrected to attenuate according to the phase of the frame, and selects pixel data in the two actual frames according to the second motion vector Then, the selected pixel data is mixed at a ratio corresponding to the phase of each of the plurality of interpolation frames to generate interpolation pixel data constituting the plurality of interpolation frames. To provide a video signal processing apparatus characterized by comprising a pixel generating unit (5).

  In the video signal processing apparatus, the motion vector correction unit may reduce motion vector attenuation used when generating interpolation pixel data constituting an interpolation frame having a phase away from the temporal center between the two actual frames. It is preferable that the degree is larger than the degree of attenuation of the motion vector used when generating the interpolated pixel data constituting the interpolated frame having the phase located at the center.

  Furthermore, in order to solve the above-described problems of the related art, the present invention uses a plurality of pixel data in at least two real frames in an input video signal to interpolate between the two real frames. A first motion vector necessary for generating interpolation pixel data constituting an interpolation frame is detected, and the first motion vector is attenuated according to the phase of the plurality of interpolation frames between the two actual frames. To correct the second motion vector, select pixel data in the two real frames according to the second motion vector, and select the selected pixel data according to the phase of each of the plurality of interpolation frames. The video signal processing method is characterized in that the interpolated pixel data constituting the plurality of interpolated frames is generated by mixing at a certain ratio.

  In the video signal processing method, the degree of motion vector attenuation used when generating interpolation pixel data constituting an interpolation frame having a phase away from the temporal center between the two actual frames is positioned at the center. It is preferable that the degree of attenuation of the motion vector used when generating the interpolated pixel data constituting the interpolated frame of the phase to be larger is set.

  According to the motion vector correction apparatus and method and the video signal processing apparatus and method of the present invention, when a plurality of interpolation frames are interpolated between real frames of a video signal, false detection is performed when the motion vector is relatively large. Even when this occurs, it is possible to further reduce the visual discomfort of the interpolated image.

It is a block diagram which shows one Embodiment of this invention. It is a characteristic view for demonstrating operation | movement of the motion vector correction | amendment part 4 in FIG. It is a characteristic view for demonstrating operation | movement of the motion vector correction | amendment part 4 in FIG. FIG. 6 is a characteristic diagram illustrating correction characteristics when the motion vector correction unit 4 in FIG. 1 uses the first parameter set. FIG. 10 is a characteristic diagram illustrating correction characteristics when the motion vector correction unit 4 in FIG. 1 uses a second parameter set. It is a figure which shows the production | generation operation | movement of the interpolation pixel for demonstrating the effect | action by one Embodiment of this invention.

  DESCRIPTION OF EXEMPLARY EMBODIMENTS Hereinafter, an embodiment of a motion vector correction apparatus and method, and a video signal processing apparatus and method according to the invention will be described with reference to the accompanying drawings. An embodiment of the present invention shown in FIG. 1 shows a frame rate conversion device that interpolates three interpolation frames between real frames of a video signal and converts the frame rate to four times as an example of a video signal processing device. ing. A frame rate conversion device that converts the frame rate to three times may be used as long as it is a video signal processing device that interpolates two or more interpolation frames between actual frames of the video signal.

  The video signal processing apparatus of the present invention is not limited to the frame rate conversion apparatus, but may be a film judder removal apparatus provided with a motion vector detection apparatus. The present invention can be applied to any video signal processing device that includes a motion vector detection device and interpolates two or more interpolation frames between real frames of a video signal.

  In FIG. 1, each pixel data of a video signal Sin having a frame frequency of 60 Hz is sequentially input to a frame memory 1, a motion vector detection unit 2, an interpolation pixel generation unit 5, and a frame frequency conversion unit 6. The frame memory 1 outputs the input pixel data with a delay of one frame. The current frame of the input video signal Sin is F0, and the frame one frame before the current frame output from the frame memory 1 is the previous frame F1. The current frame F0 and the previous frame F1 are actual frames. Each pixel data of the frame F1 is sequentially input to the motion vector detection unit 2 and the interpolation pixel generation unit 5.

  The motion vector detection unit 2 uses, for example, a matching method to generate each interpolation pixel data of an interpolation frame to be interpolated between the current frame F0 and the previous frame F1, using the pixel data of the current frame F0 and the previous frame F1. A motion vector V1 is detected. The motion vector V1 may be detected in units of pixel data, or may be detected in units of blocks composed of a plurality of pixel data. Although not particularly illustrated, the motion vector detection unit 2 includes a plurality of line memories, a pixel delay unit, a subtractor for obtaining a difference between pixel data, and the like. The motion vector V1 is input to the motion vector correction unit 4. In addition to the current frame F0 and the previous frame F1, the motion vector V1 may be detected using a frame earlier than the previous frame F1.

  The correction parameter setting unit 7 is input with a phase signal Sph indicating the frame phase of the interpolated frame to be interpolated between the current frame F0 and the previous frame F1. The correction parameter setting unit 7 supplies parameter data Dpr to the motion vector correction unit 4 when the motion vector correction unit 4 generates a later-described motion vector V3 according to the input phase signal Sph.

The operation of the motion vector correction unit 4 will be described with reference to FIGS. The motion vector correction unit 4 corrects the input motion vector V1 to the motion vector V2 according to the specification shown in FIG. The motion vector correction unit 4 uses the motion vector V1 as it is as the motion vector V2 in the range from 0 to the threshold s that is the first parameter, and when the motion vector V1 exceeds the threshold s. The motion vector V2 is generated based on the following equation (1).
V2 = V1 * m / n + s * {(nm) / n} (1)

  M and n in Equation (1) are predetermined natural numbers where m <n, and the values may be set as appropriate. Let m be the second parameter and n be the third parameter. The second and third parameters m and n determine the slope of the portion of the motion vector V2 that exceeds the threshold s. The threshold value s is a starting point for attenuating the motion vector V1.

The motion vector correction unit 4 further corrects the motion vector V2 to the motion vector V3 according to the specification shown in FIG. The motion vector correction unit 4 uses the motion vector V2 as it is as the motion vector V3 in the range from 0 to the threshold r which is the fourth parameter. The motion vector correction unit 4 holds the value of the motion vector V3 at a constant value r in the range h where the magnitude of the motion vector V2 is from the threshold value r to the threshold value t that is the fifth parameter. Furthermore, when the magnitude of the motion vector V2 exceeds the threshold t, the motion vector correction unit 4 generates a motion vector V3 based on the following equation (2).
V3 = 2 × r + h−V2 (2)

  When the magnitude of the motion vector V2 exceeds the threshold t according to the equation (2), the magnitude of the motion vector V3 is attenuated at a predetermined rate and goes to zero.

  The frame phases of three interpolation frames interpolated between the current frame F0 and the previous frame F1 are set to ph0, ph1, and ph2 in the order of the interpolation frames from the current frame F0 side to the previous frame F1 side. When the phase signal Sph is the frame phase ph0, ph2, the correction parameter setting unit 7 uses the first parameter set (s1) as the parameter set (s, m, n, r, t) including the first to fifth parameters. , M1, n1, r1, t1) are supplied to the motion vector correction unit 4 as parameter data Dpr. The first parameter set (s1, m1, n1, r1, t1) has a relatively high rate at which the motion vector V3 attenuates.

  Further, when the phase signal Sph is the frame phase ph1, the correction parameter setting unit 7 uses the second parameter set (s2) as the parameter set (s, m, n, r, t) including the first to fifth parameters. , M2, n2, r2, t2) are supplied to the motion vector correction unit 4 as parameter data Dpr. The second parameter set (s2, m2, n2, r2, t2) is a value at which the rate at which the motion vector V3 attenuates is relatively small.

  FIG. 4 shows correction characteristics when the motion vector correction unit 4 corrects the motion vector V1 to the motion vector V3 using the first parameter set (s1, m1, n1, r1, t1). In FIG. 4, the motion vector V3 is held at a constant value r1 in the range h1. FIG. 5 shows correction characteristics when the motion vector correction unit 4 corrects the motion vector V1 to the motion vector V3 using the second parameter set (s2, m2, n2, r2, t2). In FIG. 5, the motion vector V3 is held at a constant value r2 in a range h2 larger than the range h1.

  As can be seen by comparing FIGS. 4 and 5, the motion vector correction unit 4 attenuates the motion vector V 1 when the frame phase ph 1 corresponds to the temporal center between the current frame F 0 and the previous frame F 1. When the frame phases are ph0 and ph2 that are relatively small and far from the temporal center, the degree of attenuation of the motion vector V1 is relatively large.

  In FIG. 4, if the motion vector V1 is a vector V10, the motion vector V3 corrected by the motion vector correction unit 4 is a vector V31. On the other hand, in FIG. 5, when the motion vector V1 is the same vector V10 as the vector V10 shown in FIG. 4, the motion vector V3 corrected by the motion vector correction unit 4 is a vector V32 larger than the vector V31. Thus, the motion vector correction unit 4 attenuates the motion vector V1 in accordance with the frame phase of the interpolated frame interpolated between the current frame F0 and the previous frame F1, and corrects it to the motion vector V3. The degree of is different.

  The value of the parameter set (s, m, n, r, t) is varied according to the frame phase of the interpolation frame interpolated between the current frame F0 and the previous frame F1. This is because when the motion vector V1 is erroneously detected, the degree of influence on the visual discomfort varies depending on the frame phase of the interpolation frame. As a result of verification by the present inventor, the degree to which the visual discomfort is affected when the frame phase of the interpolated frame is at a position distant from the temporal center position of the current frame F0 and the previous frame F1. It became clear that it was big.

  Therefore, in the present embodiment, the frame parameters ph0 and ph2 use the first parameter set (s1, m1, n1, r1, t1) to achieve the characteristics shown in FIG. 4, while the frame phase ph1 uses the second parameter. The set (s2, m2, n2, r2, t2) is used to obtain the characteristics shown in FIG. In the frame phases ph0 and ph2, it is assumed that the current frame F0 and the previous frame F1 are temporally the same position when viewed from the temporally central position, and the visual detection is made when the motion vector V1 is erroneously detected. The first parameter set (s1, m1, n1, r1, t1), which is the same parameter set, is used assuming that the degree of influence on the uncomfortable feeling is the same.

  Here, the case where three interpolation frames are interpolated between the current frame F0 and the previous frame F1 has been described. However, no matter how many interpolation frames are used, the temporal relationship between the current frame F0 and the previous frame F1 is not limited. The value of the parameter set (s, m, n, r, t) may be set according to the temporal position viewed from the central position. It is preferable to set the value of the parameter set (s, m, n, r, t) so that the degree of attenuation increases as the distance from the central position increases. Further, the value of the parameter set (s, m, n, r, t) may be set according to the degree of influence on the visual discomfort when the motion vector V1 is erroneously detected.

  When an even number of interpolated frames are interpolated between the current frame F0 and the previous frame F1, there is no interpolated frame having a temporally central frame phase between the current frame F0 and the previous frame F1. In such a case, the interpolated frame having the closest frame phase from the center position may be equivalent to the center frame phase.

  The values of the first to fifth parameters of the first parameter group (s1, m1, n1, r1, t1) and the second parameter group (s2, m2, n2, r2, t2) are all different. It may be the same or partly the same. As described above, when the values of the parameter sets (s, m, n, r, t) are made different according to the frame phase of the interpolation frame, it is not necessary to make all the values of the first to fifth parameters different. The values of the first to fifth parameters may be made different so that the degree of attenuation differs when the motion vector V1 is attenuated and corrected to the motion vector V3.

  As described above, the motion vector V3 output from the motion vector correction unit 4 is input to the interpolation pixel generation unit 5. Although not particularly illustrated, the interpolation pixel generation unit 5 delays the pixel data of the current frame F0 in the horizontal direction and the vertical direction, and generates a plurality of pixel data in a predetermined horizontal direction and vertical direction in the current frame F0. A line memory and a pixel delay unit, and a plurality of line memories and pixels for delaying the pixel data of the previous frame F1 in the horizontal direction and the vertical direction and generating pixel data in a predetermined horizontal direction and vertical direction in the previous frame F1 And a delay device. In addition, the interpolation pixel generation unit 5 includes a mixing unit that generates and generates interpolation pixel data by selecting and mixing the pixel data in the current frame F0 and the pixel data in the previous frame F1 according to the motion vector MV3.

  A phase signal Sph is input to the interpolation pixel generation unit 5. When mixing the pixel data in the current frame F0 and the pixel data in the previous frame F1, the mixing unit of the interpolation pixel generation unit 5 determines the mixing ratio according to the frame phase indicated by the phase signal Sph. Mix. Thereby, the interpolation pixel generation unit 5 can generate interpolation pixel data constituting an interpolation frame of each frame phase to be interpolated between the current frame F0 and the previous frame F1.

  The frame frequency conversion unit 6 is sequentially input with the pixel data of the current frame F0 and the interpolation pixel data of each of the three interpolation frames output from the interpolation pixel generation unit 5. A phase signal Sph is also input to the frame frequency converter 6. The frame frequency conversion unit 6 can be configured by a time series conversion memory. The frame frequency conversion unit 6 generates a current frame F0 that is a real frame based on the pixel data of the current frame F0 that is sequentially input, and 3 based on the pixel data of the frame phases ph0, ph1, and ph2 that are sequentially input. Generate one interpolated frame.

  The frame frequency converting unit 6 outputs a video signal having a frame frequency of 240 Hz at an output frequency of 240 Hz in the order of an interpolation frame of frame phase ph2, an interpolation frame of frame phase ph1, an interpolation frame of frame phase ph0, and a current frame F0. Output Sout.

  The effect | action by this embodiment is demonstrated using FIG. FIG. 6A is an operation according to the present embodiment, and shows an interpolation pixel generation operation according to the present embodiment. FIG. 6B shows an interpolation pixel generation operation with the configuration described in Patent Document 1 for comparison.

  First, FIG. 6B will be described. In FIG. 6B, it is assumed that the motion vector V1 is the vector V10, and the motion vector V1 is attenuated and corrected to the vector V30 by the configuration described in Patent Document 1. In the configuration described in Patent Document 1, the vector V30 is used when generating any interpolated pixel data of the frame phases ph0, ph1, and ph2. The pixel data d10 in the previous frame F1 and the pixel data d00 in the current frame F0 are mixed at a ratio corresponding to the phase of each of the frame phases ph0, ph1, and ph2 to generate interpolated pixel data i00, i10, and i20. .

  On the other hand, in this embodiment, if the motion vector V1 is the vector V10, the motion vector V1 is attenuated and corrected to the vector V32 at the frame phase ph1, and is corrected to the vector V32 at the frame phases ph0 and ph2. Further attenuated and corrected to vector V31. In the frame phase ph1, the pixel data d12 in the previous frame F1 and the pixel data d02 in the current frame F0 are mixed at a rate corresponding to the phase of the frame phase ph1 to generate the interpolated pixel data i11. In the frame phases ph0 and ph2, the pixel data d11 in the previous frame F1 and the pixel data d01 in the current frame F0 are mixed in proportions corresponding to the respective phases of the frame phases ph0 and ph2, and the interpolated pixel data i01 and i21 are obtained. Generated.

  In the present embodiment, the motion vector V1 detected by the motion vector detection unit 2 is corrected to be attenuated by the correction characteristics shown in FIG. 4 or 5 by the motion vector correction unit 4 regardless of the state of the motion vector V1. Yes. A motion vector state determination unit as described in Patent Document 1 may be provided to correct the motion vector V1 to the motion vector V3 according to the state of the motion vector V1.

  In this case, for example, a motion vector state determination unit is provided before the correction parameter setting unit 7, and the determination signal determined by the motion vector state determination unit is input to the correction parameter setting unit 7. Then, the correction parameter setting unit 7 may be configured to further adjust the value of the parameter set (s, m, n, r, t) determined according to the frame phase of the interpolation frame according to the determination signal. .

  The present invention is not limited to the embodiment described above, and various modifications can be made without departing from the gist of the present invention. The present invention includes a program for causing a computer to realize the functions of the apparatus described in this embodiment. These programs may be read from a recording medium and loaded into a computer, or may be transmitted via a communication network and loaded into a computer.

DESCRIPTION OF SYMBOLS 1 Frame memory 2 Motion vector detection part 4 Motion vector correction | amendment part 5 Interpolation pixel production | generation part 6 Frame frequency conversion part 7 Correction parameter setting part

Claims (8)

Using the pixel data in at least two actual frames in the input video signal, a motion vector necessary for generating interpolation pixel data constituting a plurality of interpolation frames to be interpolated between the two actual frames is detected. A motion vector detection unit to perform,
A motion vector correction unit that corrects the motion vector to attenuate according to the phase of the plurality of interpolation frames between the two real frames;
A motion vector correction apparatus comprising:   The motion vector correction unit is located at the center of the degree of attenuation of the motion vector used when generating interpolation pixel data constituting an interpolation frame having a phase away from the temporal center between the two actual frames. 2. A motion vector correction apparatus according to claim 1, wherein the motion vector correction apparatus uses a degree of attenuation of a motion vector used when generating interpolated pixel data constituting a phase interpolation frame. Using the pixel data in at least two actual frames in the input video signal, a motion vector necessary for generating interpolation pixel data constituting a plurality of interpolation frames to be interpolated between the two actual frames is detected. And
The motion vector correction method, wherein the motion vector is corrected so as to be attenuated according to the phase of the plurality of interpolation frames between the two actual frames.   The degree of motion vector attenuation used when generating interpolated pixel data constituting an interpolated frame having a phase away from the temporal center between the two actual frames constitutes the interpolated frame having the phase located at the center. 4. The motion vector correction method according to claim 3, wherein the motion vector correction level is larger than the degree of attenuation of the motion vector used when generating the interpolated pixel data. First motion required when generating interpolated pixel data constituting a plurality of interpolated frames to be interpolated between the two actual frames using pixel data in at least two actual frames in the input video signal A motion vector detection unit for detecting a vector;
A motion vector correction unit that corrects the first motion vector to be attenuated according to the phase of the plurality of interpolation frames between the two real frames, and outputs the second motion vector as a second motion vector;
The pixel data in the two real frames is selected according to the second motion vector, and the selected pixel data is mixed at a ratio corresponding to the phase of each of the plurality of interpolation frames to thereby combine the plurality of interpolation frames. An interpolation pixel generation unit that generates interpolation pixel data to be configured;
A video signal processing apparatus comprising:   The motion vector correction unit is located at the center of the degree of attenuation of the motion vector used when generating interpolation pixel data constituting an interpolation frame having a phase away from the temporal center between the two actual frames. 6. The video signal processing apparatus according to claim 5, wherein the degree of attenuation of a motion vector used when generating interpolated pixel data constituting a phase interpolated frame is made larger. First motion required when generating interpolated pixel data constituting a plurality of interpolated frames to be interpolated between the two actual frames using pixel data in at least two actual frames in the input video signal Detect the vector
Correcting the first motion vector to a second motion vector by attenuating according to the phase of the plurality of interpolated frames between the two real frames;
The pixel data in the two real frames is selected according to the second motion vector, and the selected pixel data is mixed at a ratio corresponding to the phase of each of the plurality of interpolation frames to thereby combine the plurality of interpolation frames. A method of processing a video signal, comprising generating interpolated pixel data.   The degree of motion vector attenuation used when generating interpolated pixel data constituting an interpolated frame having a phase away from the temporal center between the two actual frames constitutes the interpolated frame having the phase located at the center. 8. The video signal processing method according to claim 7, wherein the degree of attenuation of the motion vector used when generating the interpolated pixel data is larger.

Images