JP5526918B2 - Video signal processing apparatus and method - Google Patents

Description

  The present invention relates to a video signal processing apparatus and method for interpolating an interpolated frame between real frames of a video signal, such as a frame rate converting apparatus and a film judder removing apparatus.

  When a moving image is displayed on an image display device using a liquid crystal panel, an afterimage tends to occur. Therefore, in order to reduce the afterimage, an interpolation frame is interpolated between the actual frames of the video signal to increase the number of frames. For example, the frame rate at a vertical frequency of 60 Hz is converted to a double vertical frequency of 120 Hz or higher. It is done to display. In a video signal processing apparatus that performs frame rate conversion, a motion vector of an image is detected, each interpolation pixel is generated using the motion vector, and an interpolation frame that is interpolated between actual frames is generated.

  In addition, when a film image is converted into a video signal having a vertical frequency of 60 Hz by, for example, 2-3 pulldown, unnaturalness is likely to occur in the motion of the image. This unnatural motion of the image is called film judder. In order to remove (reduce) film judder, there is a film judder removal apparatus that interpolates an interpolation frame between actual frames. Also in the film judder removal apparatus, a motion vector of an image is detected, each interpolation pixel is generated using the motion vector, and an interpolation frame is generated. Note that the film judder removal apparatus may or may not convert the frame rate.

JP 2006-337448 A

  In a video signal processing device that detects a motion vector of an image, generates each interpolation pixel using the motion vector, and generates an interpolation frame, it is required to reduce erroneous interpolation as much as possible in order to obtain a high-quality video. . In order to improve the performance of generating an interpolation frame by reducing erroneous interpolation, the circuit scale of a motion vector detection unit and an interpolation unit that generates interpolation pixel data based on a motion vector generally increases. However, if the circuit scales of the motion vector detection unit and the interpolation unit are increased, the cost increases. Accordingly, there is a need for a video signal processing apparatus and method that can suppress an increase in circuit scale without significantly reducing the performance of generating an interpolation frame.

  The present invention has been made in view of such problems, and an object of the present invention is to provide a video signal processing apparatus and method capable of suppressing an increase in circuit scale without significantly reducing the performance of generating an interpolation frame. .

In order to solve the above-described problems of the prior art, the present invention forms an interpolated frame to be interpolated between the two actual frames using pixel data in at least two actual frames in the input video signal. A motion vector detection unit (2) for detecting a motion vector necessary for generating interpolation pixel data, and a plurality of pixel data in each of the two actual frames required for generating the interpolation pixel data. and generating pixel data generating unit that (31, 32), within one frame of said two real frames for generating the interpolated pixel data in response to the motion vector, the pixel position based on the motion vector A determination unit (33 to 35) that determines whether or not a situation occurs in which data is not generated by the pixel data generation unit; Determining if the status is determined not to occur, the within two actual frame, to generate the interpolated pixel data by using the position of the pixel data based on the motion vector, the determining unit, and the situation may occur In this case, the pixel data at the position based on the motion vector and the same horizontal and vertical positions as the interpolated pixel data in the other of the two real frames excluding the pixel data of the one frame And an interpolation pixel data generation unit (36-38) for generating the interpolation pixel data by mixing the pixel data of the two real frames .

In addition, in order to solve the above-described problems of the related art, the present invention uses the pixel data in at least two actual frames in the input video signal to interpolate an interpolation frame between the two actual frames. A motion vector detecting step for detecting a motion vector necessary for generating the interpolated pixel data to be configured; and generating a plurality of pixel data in each of the two actual frames required for generating the interpolated pixel data and pixel data generation step of, said within one frame of the two actual frame, the position of the pixel data is the pixel data generated based on the motion vector for generating the interpolated pixel data in response to the motion vector A determination step for determining whether or not a situation that is not generated in the step occurs; and If the status is determined not to occur, the within two actual frame, the generated interpolated pixel data by using the position of the pixel data based on the motion vector, when the situation in the determining step occurs If determined, the pixel data at the position based on the motion vector in the other frame of the two real frames excluding the pixel data of the one frame, and the same horizontal and There is provided an image signal processing method characterized by including an interpolation pixel data generation step of generating the interpolation pixel data by mixing each pixel data of the two real frames in a vertical position .

  According to the video signal processing apparatus and method of the present invention, it is possible to suppress an increase in circuit scale without significantly reducing the performance of generating an interpolation frame.

It is a block diagram which shows one Embodiment of the video signal processing apparatus of this invention. It is a figure for demonstrating operation | movement of the motion vector detection part 2 in FIG. FIG. 3 is a block diagram illustrating a specific configuration example of delay units 31 and 32 in FIG. 1. It is a figure for demonstrating the production | generation method of the interpolation pixel by one Embodiment.

  The video signal processing apparatus of the present invention will be described below with reference to the accompanying drawings. The embodiment of the present invention shown in FIG. 1 is a frame rate conversion device that interpolates three interpolated frames between real frames of a video signal and converts the frame rate to four times as an example of a video signal processing device. Is shown. The video signal processing apparatus of the present invention is not limited to the frame rate conversion apparatus, and may be a film judder removal apparatus, as long as it interpolates interpolation frames between actual frames of the video signal.

  In FIG. 1, each pixel data of the video signal Sin having a frame frequency of 60 Hz is sequentially input to the frame memory 1, the motion vector detection unit 2, the interpolation unit 3, and the frame frequency conversion unit 4. The frame frequency conversion unit 4 includes a time series conversion memory. The frame memory 1 outputs the input pixel data with a delay of one frame. The current frame of the input video signal Sin is F0, and the frame one frame before the current frame output from the frame memory 1 is F1. The current frame F0 and the frame F1 are actual frames.

  The motion vector detection unit 2 uses, for example, a matching method to generate each interpolation pixel data of an interpolation frame to be interpolated between the current frame F0 and the frame F1, using the pixel data of the current frame F0 and the frame F1. The motion vector MV is detected. Although not particularly illustrated, the motion vector detection unit 2 includes a plurality of line memories, a pixel delay unit, a subtractor for obtaining a difference between pixel data, and the like. The motion vector MV is input to the delay units 31 and 32 and the determination units 33 to 35. In addition to the current frame F0 and the frame F1, the motion vector MV may be detected using a frame earlier than the frame F1.

  As shown in FIG. 2, when three interpolation frames Fi1, Fi2, and Fi3 are inserted between the current frame F0 and the previous frame F1, the pixel data Pf00 of the current frame F0 and the pixel data Pf10 of the frame F1 It is necessary to insert interpolated pixel data Pfi1, Pfi2, and Pfi3 indicated by broken lines between. The motion vector detection unit 2 generates a motion vector MVi2 for generating interpolated pixel data Pfi2 located at the center of the pixel data Pf00 and the pixel data Pf10. Then, the motion vector detection unit 2 generates a motion vector MVi1 for generating the interpolated pixel data Pfi1 and a motion vector MVi3 for generating the inter-pixel data Pfi3 by shifting the motion vector MVi2. The motion vector MV is a general term for the motion vectors MVi1 to MVi3.

  The delay unit 31 is necessary to generate the interpolated pixel data Pfi1, Pfi2, and Pfi3 by sequentially delaying the input pixel data Pf0 of the current frame F0 in the horizontal direction in units of pixels and sequentially in units of lines. A plurality of pixel data on the current frame F0 are generated. Similarly, the delay unit 32 generates the interpolated pixel data Pfi1, Pfi2, and Pfi3 by sequentially delaying the pixel data Pf1 of the input frame F1 in the horizontal direction in units of pixels and sequentially in units of lines. A plurality of pixel data on the necessary frame F1 are generated. In other words, the delay units 31 and 32 are pixel data generation units that generate a plurality of pixel data in each of the two actual frames necessary for generating the interpolation pixel data.

  A specific configuration example of the delay units 31 and 32 will be described with reference to FIG. The delay unit 31 and the delay unit 32 have the same configuration. The pixel data Pf0 of the current frame F0 is sequentially input to the delay unit 31, and the pixel data Pf1 of the frame F1 output from the frame memory 1 is sequentially input to the delay unit 32. The pixel data Pf0 is arbitrary pixel data such as the pixel data Pf0-4, Pf0-3, Pf0-2, Pf0-1, Pf00, Pf01, Pf02, etc. in FIG. 2, and the pixel data Pf1 is the pixel data Pf1 in FIG. -4, Pf1-3, Pf1-2, Pf1-1, Pf10, Pf11, Pf12...

Hereinafter, the delay unit 31 to which the pixel data Pf0 is input will be described as a representative. In FIG. 3, _m line memories (m is an integer of 2 or more) of line memories 301 _{1 to} 301 m are connected in series, and input pixel data Pf 0 is one line by the line memories 301 _{1 to} 301 _m . Delayed by one. Pixel data Pf0 is n connections pixel delay units 302 ₀₁ to 302 _0n in series (n is an integer of 2 or more) are delayed by one pixel by pixel delayer. Pixel data output from the line memory 301 ₁ is delayed by one pixel by the pixel connected to the delay unit 302 ₁₁ to 302 _1n in series. Similarly, pixel data output from a line memory (not shown ₎ between the line memories 301 _{1 to} 301 _m is also delayed pixel by pixel by n pixel delay units. Pixel data output from the line memory 301 _m is delayed pixel by pixel by pixel delay units 302 _{m1 to} 302 _mn connected in series.

  The number m of line memories and the number n of pixel delays shown in FIG. 3 are the pixels in the horizontal and vertical directions necessary for generating the interpolated pixel data Pfi1, Pfi2, Pfi3 based on the shift amount indicated by the motion vector MV. What is necessary is just to set so that it may become the number of data. The numbers m and n are arbitrary numbers, and are optimal numbers in consideration of both the performance of generating an interpolation frame (accuracy of generating interpolation pixel data) and the circuit scale.

Pixel delay units _{302 01 ~302 0n, 302 11 ~302} 1n, ..., pixel data output from the 302 _m1 to 302 _mn are input to the selection unit 303. The selection unit 303 receives the motion vectors MVi1 to MVi3 output from the motion vector detection unit 2. The selection unit 303 in the delay unit 31 is based on the pixel data Pf0 of the current frame F0 necessary for generating the interpolation pixel data Pfi1 based on the shift amount indicated by the motion vector MVi1 and the shift amount indicated by the motion vector MVi2. Pixel data Pf0 of the current frame F0 necessary for generating the interpolation pixel data Pfi2 and pixel data Pf0 of the current frame F0 required for generating the interpolation pixel data Pfi3 based on the shift amount indicated by the motion vector MVi3 Select and output.

  The selection unit 303 in the delay unit 32 performs interpolation based on the pixel data Pf1 of the frame F1 necessary for generating the interpolation pixel data Pfi1 based on the shift amount indicated by the motion vector MVi1 and the shift amount indicated by the motion vector MVi2. The pixel data Pf1 of the frame F1 necessary for generating the pixel data Pfi2 and the pixel data Pf1 of the frame F1 required for generating the interpolation pixel data Pfi3 based on the shift amount indicated by the motion vector MVi3 are selected. Output.

  In the example of FIG. 2, the selection unit 303 in the delay unit 31 selects and outputs the pixel data Pf0-1, Pf0-2, Pf0-3, and the selection unit 303 in the delay unit 32 includes the pixel data Pf13, Pf12 and Pf11 are selected and output. Here, the case where a pair of pixel data is output by the delay units 31 and 32 has been described. However, as will be described later, one pixel data of a pair of pixel data originally required based on the motion vector MV is delayed. In some cases, the signals are not output from the units 31 and 32.

  As shown in FIG. 4, the pixel data Pf0 generated by the delay unit 31 is in the range from the pixel data Pf0-4 to the pixel data Pf04 in the horizontal direction, and the pixel data Pf1 generated by the delay unit 32 is the pixel data in the horizontal direction. It is assumed that the range is from Pf1-4 to pixel data Pf14. Actually, the pixel range in the horizontal direction is wider, but here pixel data Pf0 and Pf1 for nine pixels are used for simplification. In this case, the pair of pixel data that the delay units 31 and 32 should originally output based on the motion vector MVi1 are the pixel data Pf0-2 and Pf16, but the pixel data Pf16 is delayed as shown by the two-dot chain line. Pixel data that is not output from the unit 32. Similarly, a pair of pixel data that should be output by the delay units 31 and 32 based on the motion vector MVi3 is pixel data Pf0-6 and Pf12, but the pixel data Pf0-6 is indicated by a two-dot chain line. Pixel data not output from the delay unit 31.

  Although FIG. 4 shows the case of the horizontal direction, it is possible that one pixel data is not output from the delay units 31 and 32 even in the vertical direction.

  1 determine the pixel data in one frame out of the pixel data in two real frames for generating the interpolation pixel data Pfi1, Pfi2, and Pfi3 according to the motion vector MV. It is determined whether or not a situation that is not generated by 31 and 32 occurs. Since the determination units 33 to 35 know how much the delay units 31 and 32 generate pixel data for generating interpolation pixel data in the horizontal direction and the vertical direction, the input motion vector MV Based on the above, it can be determined whether or not the above situation occurs.

  When the determination units 33 to 35 determine that one of the pixel data is not output, the determination indicates which pixel data in the current frame F0 and the frame F1 is output and which pixel data is not output. A signal is generated and supplied to the interpolation pixel data generation units 36-38. The interpolation pixel data generation unit 36 generates interpolation pixel data Pfi1. The interpolation pixel data generation unit 37 generates interpolation pixel data Pfi2. The interpolation pixel data generation unit 38 generates interpolation pixel data Pfi3.

  In FIG. 1, the determination unit 34 for supplying a determination signal to the interpolation pixel data generation unit 37 is provided. However, when the interpolation pixel data Pfi2 of the interpolation frame Fi2 located at the center between the current frame F0 and the frame F1 is generated. Therefore, a pair of pixel data is always output from the delay units 31 and 32. Therefore, the determination unit 34 may be deleted.

  When a pair of pixel data is supplied from the delay units 31 and 32, the interpolation pixel data generation units 36 to 38 adaptively mix the pair of pixel data according to the distance to generate the interpolation pixel data Pfi1 to Pfi3. . In the case of FIG. 2, the interpolation pixel data generation unit 36 calculates (2/3 × Pf0-1 + 1/3 × Pf13) to generate interpolation pixel data Pfi1, and the interpolation pixel data generation unit 37 calculates (1/2 * Pf0-2 + 1/2 * Pf12) is generated to generate the interpolation pixel data Pfi2, and the interpolation pixel data generation unit 38 calculates (1/3 * Pf0-3 + 2/3 * Pf11) to calculate the interpolation pixel data Pfi3. Should be generated.

  In the case of FIG. 4, the interpolation pixel data generation unit 36 uses the pixel data Pf0-2 as it is as the interpolation pixel data Pfi1, and the interpolation pixel data generation unit 38 sets the pixel data Pf12 as it is as the interpolation pixel data Pfi3. As described above, in the present embodiment, when the interpolation pixel data Pfi1 and Pfi3 are generated, if one of the pair of pixel data to be originally used does not exist, the interpolation pixel data Pfi1 and Pfi3 Interpolated pixel data Pfi1 and Pfi3 are generated using only one pixel data having a short distance.

  If one of the pixel data is directly used as the interpolation pixel data Pfi1 and Pfi3, if there is a high possibility of erroneous interpolation, one pixel data and the other pixel data are mixed to generate the interpolation pixel data Pfi1 and Pfi3. Is preferred. Other preferable pixel data is pixel data Pf00 and Pf10 at the same horizontal and vertical positions as the interpolation pixel data Pfi1 and Pfi3. The pixel data Pf00 and Pf10 are pixel data used when generating the interpolated pixel data Pfi1 to Pfi3 at the time of still image interpolation. In this case, the pixel data Pf00 and Pf10 may be supplied from the delay units 31 and 32 to the interpolation pixel data generation units 36 and 38.

  The interpolated pixel data generation units 36 and 38 preferably mix the one pixel data and the pixel data Pf00 and Pf10 when the motion vector detection unit 2 determines that the motion is greater than a predetermined value. If the pixel data Pf00 and Pf10 are mixed with the one of the pixel data when the movement is larger than a predetermined value, the possibility of failure in interpolation can be reduced.

  In this embodiment, the interpolated pixel data Pfi1 to Pfi3 are generated using pixel data in two real frames, but the interpolated pixel data Pfi1 to Pfi3 are generated using pixel data in three or more real frames. May be.

  The frame frequency conversion unit 4 is sequentially input with the pixel data of the frame F0 and the interpolation pixel data Pfi1, Pfi2, and Pfi3 output from the interpolation pixel data generation units 36 to 38. The frame frequency conversion unit 4 is an image data of a frame F0 that is an actual frame based on pixel data of a frame F0 that is sequentially input, and an image of an interpolation frame Fi1 that is an interpolation frame based on the interpolation pixel data Pfi1 that is sequentially input. Data, image data of an interpolation frame Fi2 that is an interpolation frame based on sequentially input interpolation pixel data Pfi2, and image data of an interpolation frame Fi3 that is an interpolation frame based on sequentially input interpolation pixel data Pfi3 To do.

  Then, the frame frequency conversion unit 4 outputs the image data of the frame F0 and the image data of the interpolation frames Fi1 to Fi3 in the order of the interpolation frame Fi3, the interpolation frame Fi2, the interpolation frame Fi1, and the frame F0 at a frequency of 240 Hz. A video signal Sout having a frame frequency of 240 Hz is output.

  The present invention is not limited to the embodiments described above, and various modifications can be made without departing from the scope of the present invention. In the present embodiment, the frame rate conversion device that converts the frame rate to four times has been described as an example of the video signal processing device. However, the frame rate conversion device that converts the frame rate to three times may be used. When the video signal processing apparatus of the present invention is applied to a frame rate conversion apparatus, it is suitable for use in a frame rate conversion apparatus that converts the frame rate to three times or more.

DESCRIPTION OF SYMBOLS 1 Frame memory 2 Motion vector detection part 3 Interpolation part 4 Frame frequency conversion part 31, 32 Delay part (pixel data generation part)
33 to 35 determination unit 36 to 38 Interpolated pixel data generation unit

Claims (2)

A motion for detecting a motion vector necessary for generating interpolated pixel data constituting an interpolated frame to be interpolated between the two real frames using pixel data in at least two real frames in the input video signal A vector detector;
A pixel data generation unit for generating a plurality of pixel data in each of the two real frames necessary for generating the interpolation pixel data;
The interpolation pixel within one frame of said two real frames for generating data, the position of the pixel data based on the motion vectors are not generated by the pixel data generator situation occurs in response to the motion vector A determination unit for determining whether or not to do;
When the determination unit determines that the situation does not occur, the interpolation pixel data is generated using pixel data at a position based on the motion vector in the two real frames, and the determination unit includes: If it is determined that the situation occurs, there before Symbol in the other frame of the two actual frame, the pixel data of the position based on the motion vector, to the same horizontal and vertical positions and the interpolated pixel data An image signal processing apparatus comprising: an interpolation pixel data generation unit configured to mix the pixel data of the two actual frames and generate the interpolation pixel data. A motion for detecting a motion vector necessary for generating interpolated pixel data constituting an interpolated frame to be interpolated between the two real frames using pixel data in at least two real frames in the input video signal A vector detection step;
A pixel data generation step for generating a plurality of pixel data in each of the two real frames necessary for generating the interpolation pixel data;
Within one frame of said two real frames for generating the interpolated pixel data in response to the motion vector, the position of the pixel data based on the motion vector is not generated status by the pixel data generation step A determination step for determining whether or not to occur;
When it is determined in the determination step that the situation does not occur, the interpolation pixel data is generated using pixel data at a position based on the motion vector in the two real frames, and the determination step includes if the status Te is determined to occur, the other in the frame, and the pixel data of the position based on the motion vector, the same horizontal and vertical positions and the interpolated pixel data of the previous SL two actual frame And interpolating pixel data generating step of generating the interpolated pixel data by mixing the pixel data of the two real frames .

Images