JP2006310985A - Frame rate conversion apparatus, and display apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a frame rate conversion technology for properly capable of satisfying both the smoothed motion of an output image and an uncomplicated means for generating an interpolation frame. <P>SOLUTION: The frame rate conversion apparatus includes: a generating means that detects a motion vector on the basis of first and second consecutive frames among a plurality of frames received at an input frame rate, corrects the magnitude of the motion vector on the basis of a correction coefficient; and generates an interpolation frame on the basis of the motion vector after the correction and the first and second frames; and an output means that outputs at least one of the received frames and the generated interpolation frame at an output frame rate. The correction coefficient has a value not changed in the midst of the frame rate conversion. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to frame rate conversion.

  Examples of the display device include various devices such as a CRT (Cathode Ray Tube) display, a plasma display, and a liquid crystal display. Various display devices sequentially display frames (hereinafter referred to as input frames) constituting the input moving image. At this time, if the frame rate of the input moving image (hereinafter, input frame rate) and the frame rate of the moving image at the time of output (hereinafter, output frame rate) are different, the input moving image In order to display the image in accordance with the output frame rate, it is necessary to convert the frame rate.

Conventionally, for example, there are techniques disclosed in Patent Documents 1 and 2 as frame rate conversion methods. The technique disclosed in Patent Document 1 converts an input frame rate into an output frame rate by repeatedly outputting a certain input frame. The technique disclosed in Patent Document 2 obtains a motion vector from the position of an object on a certain input frame and the position of the same object on the next input frame, and based on the motion vector, A prediction screen (hereinafter referred to as “interpolation frame” for convenience) calculated as a frame between the input frame rate and the other frame is calculated, and the interpolation frame is output to convert the input frame rate into the output frame rate.
JP 2001-111968 Japanese Patent Laid-Open No. 2003-069961

  According to the techniques of Patent Documents 1 and 2 described above, there are the following problems. Hereinafter, when the output frame rate is 1.2 times the input frame rate, specifically, for example, when the input frame rate is 50 fps (frames / second) and the output frame rate is 60 fps. An example will be described with reference to FIGS.

  When the input frame rate is 50 fps, one input frame is input every 1/50 seconds as shown in FIG. 14A. Since the output frame rate is 60 fps, five input frames 201A to 201E are input and six frames are output within the same time (here, 1/10 second) (hereinafter, the output frames are referred to as “output frames”). ”).

  When the technique disclosed in Patent Document 1 is diverted to such a frame rate conversion, it is considered that the frame rate conversion is performed in the manner illustrated in FIG. 14B. That is, four input frames out of the five input frames are each output once, and one input frame (for example, the third input frame 201C) is output twice in succession. However, in this frame rate conversion method, as can be seen from the one-dot chain line in FIG. 14B, there arises a problem that the smoothness of the motion of the image is impaired (sometimes referred to as motion judder interference).

  On the other hand, when the technique disclosed in Patent Document 2 is diverted to the above-described frame rate conversion, it is considered that the frame rate conversion is performed in the manner illustrated in FIG. 14C. That is, one interpolation frame is generated for each two consecutive input frames based on a motion vector obtained from two consecutive input frames, and one output frame is output from the five input frames. Yes (for example, only the first frame 201A), and the other five frames 202A to 202E to be output are all considered to be interpolation frames (by the way, 202A is an interpolation frame based on 201A and 201B, 202B is 201B And 202C is an interpolation frame based on 201C and 201D, 202D is an interpolation frame based on 201D and 201E, and 202E is an interpolation frame based on 201E and 201F). However, according to this frame rate conversion method, the problem that the smoothness of the motion of the image is impaired can be solved, but another problem occurs. That is, as can be seen from FIG. 14C, the temporal position of the interpolation frame differs from the temporal position of each of the two input frames for every two consecutive input frames. In other words, the ratio of the temporal distance between the input frame and the correction frame and the temporal distance between the correction frame and the next input frame (hereinafter referred to as “time ratio” for convenience) is variously converted (for example, 5/6, 4/6, 3/6, 2/6, and 1/6). For this reason, when generating an interpolated frame, a correction coefficient for correcting the magnitude of the motion vector is adjusted in accordance with the time ratio (hereinafter, this correction coefficient is referred to as a “scaling ratio”). Therefore, it is necessary to correct or change the magnitude of the motion vector with the adjusted scaling ratio (hereinafter, this correction or changing process is referred to as “scaling”). Means (for example, hardware, computer program, or a combination thereof) become complicated. Further, it is considered that the load for generating the interpolation frame is large.

  The present invention has been made in view of the above problems, and its purpose is to moderate both smoothing the motion of the output image and not complicating the means for generating the interpolation frame. It is an object of the present invention to provide a frame rate conversion technique that satisfies the above requirements. Another object of the present invention is to provide a frame rate conversion technique that can satisfy both the smoothing of the motion of the output image and the suppression of the load of generating the interpolation frame.

  A frame rate conversion device according to the first aspect of the present invention is a device that performs frame rate conversion for outputting a frame input at an input frame rate at an output frame rate, and inputs a plurality of frames at the input frame rate. And a motion vector is detected based on the first and second frames that are continuous in the plurality of input frames, the magnitude of the detected motion vector is corrected based on a correction coefficient, and the corrected Generating means for generating an interpolated frame predicted as a frame between the first and second frames based on a motion vector and the first and second frames; and a plurality of input frames Output means for outputting at least one and the generated interpolation frame at an output frame rate. The correction coefficient is a value that is not changed during the frame rate conversion.

  A frame rate conversion device according to the second aspect of the present invention is a device that performs frame rate conversion for outputting a frame input at an input frame rate at an output frame rate, and inputs a plurality of frames at the input frame rate. And a motion vector is detected based on the first and second frames that are continuous in the plurality of input frames, the magnitude of the detected motion vector is corrected based on a correction coefficient, and the corrected Generating means for generating an interpolated frame predicted as a frame between the first and second frames based on a motion vector and the first and second frames; and a plurality of input frames Output means for outputting at least one and the generated interpolation frame at an output frame rate. The first time distance, which is the time from when the first frame is input to when the frame is output, is m, and is the time from when the output is performed until the second frame is input. When the second temporal distance is n, the correction coefficient is m / (m + n) or n / (m + n). The value of m + n is (for example, always) 2 to the power of x (x is an integer of 1 or more).

  In one embodiment, the correction factor may be a half.

  A frame rate conversion device according to a third aspect of the present invention is a device that performs frame rate conversion for outputting a frame input at an input frame rate at an output frame rate, and inputs a plurality of frames at the input frame rate. Means for generating an interpolated frame that is predicted as a frame between the first and second frames based on successive first and second frames in the plurality of input frames; and Output means for outputting all or a part of the plurality of input frames and the generated interpolated frame at an output frame rate. The generation unit generates the interpolation frame when the output unit outputs the interpolation frame, and does not generate the interpolation frame otherwise. The output means outputs the input frame more than the interpolation frame.

  In one embodiment, the output means includes a first temporal distance that is a time from when the first frame is input to when the frame is output, and the second frame after the output is performed. Based on the first frame and / or the second frame and the first and second frames based on at least one of a second temporal distance that is a time until the is input One frame is selected from the interpolated frames, and the selected frame is output. The output means includes a first position in which a temporal position of an output frame is in a range closer to the first frame based on at least one of the first temporal distance and the second temporal distance. Belonging to the second range, a second range that is closer to the second frame, and an intermediate range that is a range between the first range and the second range. If it is determined that it belongs to the first range, the first frame is selected. If it is determined that it belongs to the second range, the second frame is selected. When it is determined that it belongs to the intermediate range, the interpolation frame can be selected. The intermediate range may be subdivided into a plurality of sub-intermediate ranges. In that case, the generation unit selects a correction coefficient corresponding to a sub-intermediate range to which the temporal position of the output frame belongs, from a plurality of types of correction coefficients, and based on the selected correction coefficient Interpolated frames can be generated.

  Each means described above can be realized by hardware, a computer program, or a combination thereof. Each means may be constituted by one element or may be constituted by a plurality of elements (the plurality of elements may be distributed at a plurality of locations).

  A display device according to a fourth aspect of the present invention includes the above-described frame rate conversion device and a display unit that displays a frame output from the frame rate conversion device.

  According to the frame rate device according to the first and second aspects of the present invention, both smoothing the motion of the output image and not complicating the means for generating the interpolation frame are moderately achieved. Can be satisfied.

  According to the frame rate device according to the third aspect of the present invention, it is possible to appropriately satisfy both of smoothing the motion of the output image and suppressing the load of generating the interpolation frame.

  Hereinafter, some embodiments of the present invention will be described with reference to the drawings. In the following description, as described above, the correction coefficient for correcting the magnitude of the motion vector is referred to as “scaling ratio”, and the process for correcting or changing the magnitude of the motion vector is referred to as “scaling”. To do.

  FIG. 1 shows a configuration example of a display device according to a first embodiment of the present invention.

  The display device 401 can be various display devices such as a CRT (Cathode Ray Tube) display, a plasma display, and a liquid crystal display. Further, the display device 401 may be a television receiver or a display device mounted on or connected to a personal computer. The display device 401 includes a moving image input unit 402, a frame rate conversion unit 403, and a display unit (for example, a screen) 404.

  The moving image input unit 402 inputs a moving image to the frame rate conversion unit 403. The input moving image may be, for example, a video signal extracted from an analog or digital broadcast signal, or a moving image file (for example, input via a communication network or a storage medium (for example, a memory card)). MPEG format file).

  The frame rate conversion unit 403 can be configured by hardware, a computer program, or a combination thereof. The frame rate conversion unit 403 converts an input frame rate, which is a frame rate of the input moving image, into a predetermined output frame rate. In other words, the frame rate conversion unit 403 increases (or decreases) the number of output frames more than the number of input frames in the common unit time based on the input frame rate and the output frame rate. Like that. The “common unit time” refers to the reciprocal 1 / Z seconds of the greatest common divisor Z of the input frame rate F1 and the output frame rate F2. Specifically, for example, when the input frame rate is 50 fps and the output frame rate is 60 fps, since the greatest common divisor of 50 and 60 is 10, the common unit time is “1/10 second”. For example, when the input frame rate is 24 fps and the output frame rate is 60 fps, the greatest common divisor of 24 and 60 is 12, so the common unit time is “1/12 seconds”. The common unit time may be a time in different time zones instead of the same time zone for input and output. The frame rate conversion unit 403 outputs the frames after the rate rate conversion to the display unit 404, thereby sequentially displaying the frames at the output frame rate on the display unit 404, that is, displaying the moving image at the output frame rate. Can do.

  Hereinafter, the frame rate conversion unit 403 will be described in detail.

  FIG. 2 shows a configuration example of the frame rate conversion unit 403.

  The frame rate conversion unit 403 includes a frame delay unit 2, a motion vector calculation unit 3, a vector conversion unit 4, an interpolation frame generation unit 5, a buffer memory unit 6, a frame output position determination unit 7, and a frame output control. Part 8. Hereinafter, the components 2 to 8 will be described while describing the processing performed by the frame rate conversion unit 403.

  A moving image (video signal) input from the moving image input unit 402 via the input unit 1 is output to the frame delay unit 2, the motion vector calculation unit 3, the interpolation frame generation unit 5, and the buffer memory unit 6. Is done.

  The frame delay unit 2 is a storage device such as a memory and holds one frame (video signal for one frame). When the next one frame is newly input, the currently held one frame is stored. The result is output to the motion vector calculator 3 and the next newly input frame is held. Thereby, the frame delay unit 2 can output the video signal delayed by one frame to the motion vector calculation unit 3 and the interpolation frame generation unit 5. The storage capacity of the frame delay unit 2 can be set to a capacity that can store a video signal for one frame.

  The motion vector calculation unit 3 includes a frame input via the input unit 1 (that is, a video signal before one frame delay) and a frame input from the frame delay unit 2 (that is, a video signal delayed by one frame). Based on the above, a motion vector is calculated for each block or each pixel. The motion vector calculation unit 3 inputs information representing the calculated motion vector to the vector conversion unit 4.

Here, an example of a motion vector will be described with reference to FIG. FIG. 3 shows an example of the concept of motion vectors. The video signal delayed by one frame by the frame delay unit 2 corresponds to the first input frame A, and the video signal before one frame delay is the first. This corresponds to 2 input frames B. An output frame output between the first input frame 201 and the second input frame 202 is an interpolation frame 503. When attention is paid to a certain pixel on the interpolation frame, in order to obtain a motion vector corresponding to this pixel, first, the motion amount is estimated from the correlation of the image pattern from the second input frame B to the first input frame A. Thus, the motion vector V is obtained. When the temporal distance between the first input frame A and the interpolation frame 503 (output frame) is m and the temporal distance between the interpolation frame 503 (output frame) and the second input frame B is n, the interpolation frame 503 is obtained. The motion vector Vpr corresponding to the amount of movement from the first input frame A to the first input frame A and the motion vector Vct corresponding to the amount of movement from the interpolation frame 503 to the second input frame B are the following (1) and (2) The formula of
Vpr = V × m / (m + n) (1)
Vct = −V × n / (m + n) (2)
Can be obtained. Note that the temporal distance m is, for example, from the input of the first input frame to the output of the output frame, assuming that the start of the common unit time is the same at the input and output of the frame. The temporal distance n can be expressed as the time from when the output frame is output until the second input frame is input.

  The vector conversion unit 4 performs a scaling process on the motion vector V detected by the motion vector calculation unit 3 to motion vectors Vpr and Vct that match the output frame position (for example, the temporal distance m from the first frame 501).

  The time ratio, which is the ratio between m and n, varies in various ways. In this embodiment, the scaling ratio (m / (m + n) in the above equation (1), n / (in the equation (2)). m + n)) is fixed without changing according to the change of the time ratio. Specifically, for example, in this embodiment, the scaling ratio is a fixed value ½ regardless of the values of m and n. Even if it is a fixed value, the value of P in the scaling ratio Q / P (Q is an integer of 1 or more) is 2 to the power of x (x is an integer of 1 or more) because of the design of the vector conversion unit 4. It is preferable.

  When the scaling ratio is set to a fixed value ½, the motion vector scaling process can be realized by only the bit shift of the digital data displayed in binary, so that the vector conversion unit 4 can be easily designed. The vector conversion unit 4 outputs motion vector information representing the motion vector after the scaling process to the interpolation frame generation unit 5.

  The interpolation frame generation unit 5 includes the motion vector information input from the vector conversion unit 4, the first input frame A input from the frame delay unit 2, and the second input frame input via the input unit 1. Based on B, an interpolation frame 503 that is predicted as a frame between the first input frame A and the second input frame B is generated, and the generated interpolation frame 503 is used as a buffer memory unit (for example, a memory) 6. Can be output. Specifically, for example, the interpolation frame generation unit 5 calculates an average value for each pixel in the first input frame A and the second input frame B, and outputs data including the average value as an interpolation frame. can do.

  The interpolation frame generation method by the interpolation frame generation unit 5 is not limited to the above. For example, only the motion vector Vpr is used to position the pixel of the first input frame A in the corresponding position on the interpolation frame 503. Other motion compensation methods such as moving to may be followed.

  The interpolated frame 503 (in other words, the interpolated data signal) generated by the interpolated frame generating unit 5 is output to the buffer memory unit 6, and is stored in the buffer memory unit 6 by a write control signal from the frame output control unit 8. Written to the area. The second input frame (for example, a frame before delay input substantially simultaneously with the interpolation frame) is also written into a certain storage area of the buffer memory unit 6 by the write control signal. The first input frame written in the buffer memory unit 6 before the second input frame is buffered until the third frame, which is the next frame after the second input frame, is input. It may be stored in the memory unit 6. In other words, the buffer memory unit 6 has a storage capacity sufficient to store, for example, three frames of the interpolation frame and the first input frame and the second input frame that are the basis for generating the interpolation frame. it can. The storage area of the buffer memory unit 6 may be divided into, for example, two input sub areas for storing two input frames and one interpolation sub area for storing one interpolation frame. In this case, the first input frame and the second input frame may be alternately written in one input sub-region (for example, an odd-numbered input frame is written in one input sub-region. And even-numbered input frames may be written in the other input sub-region). The rate of writing to the buffer memory unit 6 can be the input frame rate described above, and the rate of reading from the buffer memory unit 6 can be the output frame rate described above. Therefore, for example, the frame (video signal) input via the input unit 1 and the generated interpolated frame are written into the buffer memory unit 6 substantially simultaneously at the input frame rate, and written into the buffer memory unit 6. Either the input frame or the interpolated frame is output to the display unit 404 as the output frame 9 at the output frame rate.

  An input synchronization signal corresponding to the input frame rate is input to the frame output position determination unit 7 via the input unit 10, and an output synchronization signal corresponding to the output frame rate is input via the input unit 11. The frame output position determination unit 7 determines an output frame position at the time of frame rate conversion, and outputs a determination signal representing information regarding the determined output frame position to the frame output control unit 8. The frame output control unit 8 outputs a write control signal to the buffer memory unit 6 at a timing according to the input frame rate, or outputs a read control signal to the buffer memory unit 6 at a timing according to the output frame rate. At that time, the frame output control unit 8 switches the frame output control to switch between the input frame written in the buffer memory unit 6 and the interpolation frame based on the determination signal from the frame output position determination unit 7. I do.

  The output frame position determination method and frame output control will be described below.

  An example of a method for determining the output frame position in the present embodiment will be described. Consider a case in which an interpolation frame 503 located between the first input frame A and the second input frame B as shown in FIG. 4 is generated from the first input frame A and the second input frame B. . First, the frame output position determination unit 7 outputs a temporal distance m from the first input frame A to the output frame and a temporal distance n from the output frame to the second input frame B as an input synchronization signal and an output. (For example, the time distance m is detected from the difference in input timing between the input synchronization signal and the output synchronization signal, and the time distance n is determined from the difference between the output synchronization signal and the next input synchronization signal. To detect). In the example of FIG. 4, m: n is for 1: 1.

As shown in FIG. 5, frame output control is performed based on two determination threshold values. For example, the temporal distance from the input of the first input frame A to the output of the output frame, and the temporal distance from the output of the output frame to the input of the second input frame A Two ratios M1: N1 and M2: N2 are prepared (where M1, N1, M2, and N2 are all integers of 1 or more, and M1 ≦ N1, M1 < M2, M2 ≦ N2, and N1 ≧ N2.) More specifically, for example, M1 is a time distance from the input of the first input frame A to the output of the output frame, and outputs the first input frame A itself as an output frame. It can be said that it is a good time distance. N1 can be a value obtained by subtracting M1 from the time distance from the input of the first input frame A to the input of the second input frame A. Further, for example, M2 is a time distance from the input of the first input frame A to the output of the output frame, and is based on the first input frame A and the second input frame B. It can be assumed that the obtained interpolated frame 503 is a temporal distance that may be output as an output frame. N2 can be a value obtained by subtracting M2 from the temporal distance from the input of the first input frame A to the input of the second input frame A. The first determination threshold value TH1 and the second determination threshold value TH2 are expressed by the following equations (3), (4),
TH1 = M1 / (M1 + N1) (3)
TH2 = M2 / (M2 + N2) (4)
Can be defined in

In the frame output position determination unit 7, a first determination threshold value TH1 and a second determination threshold value TH2 are set in advance. Then, the frame output position determination unit 7 makes the following determination based on the detected m and n.
(A) When m / (m + n) <TH1, it is determined to output the first input frame A itself as an output frame;
(B) When TH1 ≦ m / (m + n) ≦ TH2, it is determined to output the interpolation frame 503 as an output frame;
(C) When m / (m + n)> TH2, decide to output the second input frame B itself as an output frame;
It can be performed. At least one of the first determination threshold value TH1 and the second determination threshold value TH2 (or the above-described M1, N1, M1, and N2) may be a fixed value that should not be changed by the user or the like. The variable value may be changed by the user or the like. The frame output position determination unit 7 outputs a determination signal representing the determination results (a) to (c) to the frame output control unit 8, and the frame output control unit 8 generates an output frame based on the determination signal. A read control signal designating a frame to be output (for example, when it is determined to output the first input frame A, a read control signal designating an address of an area in which the first input frame A is stored) ) Is output to the buffer memory unit 6. Accordingly, the buffer memory unit 6 outputs a type of frame according to the determination results (a) to (c) from the buffer memory unit 6.

  FIG. 6 shows frame transition when the frame rate conversion 403 according to this embodiment is applied to frame rate conversion from 50 fps to 60 fps.

  In this figure, S1 to S4 correspond to S1 to S4 in FIG. That is, S1 indicates the transition of the input frame 1 input to the buffer memory unit 6 at 50 fps. S2 indicates a transition of the input frame obtained by delaying the input frame by one frame by the frame delay unit 2. S3 indicates a transition of an interpolation frame that is generated based on the input frame of S1 and the input frame of S2 and is input to the buffer memory unit 6. S4 indicates the transition of the output frame output from the buffer memory unit 6 at 60 fps.

  When converting the frame rate from 50 fps to 60 fps, it is necessary to convert from 5 frames to 6 frames every 1/10 second. As can be seen from the arrangement order of the output frames in S4 of FIG. 6, when the frame output control is performed according to the determination result described above, for example, when the input timing and the output timing are the same, The interpolation frame input to the buffer memory unit 6 can be output from the buffer memory unit 6 at the input timing. In other words, instead of outputting the second input frame as appropriate, an interpolation frame generated based on the second input frame and the corresponding first input frame is output, and thereafter By outputting the second input frame, frame rate conversion can be supported. When the interpolation frame is not output, the frame output control unit 8 adjusts the output timing with the read control signal, and outputs the input frame 1 input to the buffer memory unit 6 with a delay from the buffer memory unit 6. Therefore, the output timing of the input frame can be matched with the output frame rate.

  When the interpolation frame is generated by changing the scaling ratio every time the time ratio between m and n changes, as in the conventional case, for example, in frame rate conversion from 50 fps to 60 fps, output is performed every 1/10 second. As many as five frames among the six frames to be processed become interpolation frames. In this case, the motion of the output image can be smoothed, but there is a drawback that the configuration of the frame rate conversion unit 403 (particularly, for example, the interpolation frame generation unit 5) is complicated. On the other hand, when an input frame is simply output repeatedly without generating an interpolation frame, for example, in the frame rate conversion from 50 fps to 60 fps, the same input frame is repeatedly output once every 1/10 second. . For this reason, there exists a fault that the smoothness of the motion of an image will be impaired.

  However, in the frame rate conversion by the frame rate conversion unit 403 of this embodiment, the input frame is not repeatedly output, and is generated based on the output of the input frame itself and the input frames before and after the delay. The output of the interpolated frame can be switched appropriately. Accordingly, it is possible to appropriately satisfy the reduction of the unnaturalness of the motion of the image and the prevention of the complicated configuration of the frame rate conversion unit 403. For example, in the case of frame rate conversion from 50 fps to 60 fps, FIG. 14A to FIG. 14C described above and a diagram illustrating transition of output frames output from the frame rate conversion unit 403 according to the present embodiment. 7 (reference numbers 201A to 201F in FIG. 7 are input frames themselves, and reference number 203 is an interpolation frame generated based on the input frames 201E and 201F).

  By the way, the frame output position determination unit 7 described above can be realized by a circuit configuration illustrated in FIG. 8, for example. That is, the frame output position determination unit 7 includes a line counter 607, a first data holding unit 608, a second data holding unit 609, a comparison unit 610, a first multiplier 611, and a second multiplication. A container 612 is provided.

  The line counter 607 receives an input horizontal timing signal 601 that is a timing signal of an input frame rate and an input vertical timing signal 602. The line counter 607 is reset, for example, at the rising edge of the input vertical timing signal 602, counts up at, for example, the rising edge of the input horizontal timing signal 601, and outputs the number of lines as a count value. The count value output here is the number of lines in one frame.

  The first data holding unit 608 is, for example, a register, and holds the count value output from the line counter 607, for example, at the rising edge of the output vertical timing signal 603, so that the first input frame to the interpolated frame can be obtained. The number of lines S at the temporal distance m can be held.

  The second data holding unit 609 is, for example, a register, and holds the count value output from the line counter 607, for example, at the rising edge of the input vertical timing signal 602, so that the first input frame to the interpolated frame can be obtained. It is possible to hold the total number of lines S + T at m + n, which is the sum of the temporal distance m and the temporal distance n from the interpolation frame to the second input frame.

  The first multiplier 611 multiplies the first determination threshold 604 (TH1 shown in FIG. 5) by the total number of lines S + T output from the second data holding unit 609, thereby making the first determination. The threshold is normalized to a value converted into the number of lines, and the normalized first determination threshold value TH1 ′ is output to the comparison unit 610.

  Similarly, the second multiplier 612 multiplies the second determination threshold value 605 (TH2 shown in FIG. 5) by the total number of lines S + T output from the second data holding unit 609 to obtain the second Is normalized to a value converted into the number of lines, and the normalized second determination threshold value TH2 ′ is output to the comparison unit 610.

When the first determination threshold value replaced with the number of lines is TH1 ′ and the second determination threshold value replaced with the number of lines is TH2 ′, TH1 ′ = TH1 × (S + T), and TH2 ′ = TH2 × (S + T). The comparison unit 610 calculates the line number S output from the first data holding unit 608, the first determination threshold value TH1 ′ replaced with the line number, and the second determination threshold value TH2 ′. Compare each. As a result, the comparison unit 610, for example,
(1) If S <TH1 ′, the value 0 is output.
(2) When TH1 ′ ≦ M ≦ TH2 ′, value 1 is output,
(3) If M> TH2 ′, output value 2;
It can be performed.

  As described above, the frame output position determination unit 7 can be realized with a simple circuit configuration.

  The second embodiment of the present invention will be described below. In the following description, differences from the above-described first embodiment will be mainly described, and description of common points with the first embodiment will be omitted or simplified (this is the following third embodiment). The same applies to the examples).

  FIG. 9 shows a circuit configuration example of the frame output position determination unit 7 in the second embodiment of the present invention.

  In this second embodiment, instead of normalizing the first determination threshold value TH1 and the second determination threshold value TH2 with a multiplier, they are normalized in advance from the input units 701 and 702, respectively. The first determination threshold value TH1 ′ and the second determination threshold value TH2 ′ are input to the comparator 610.

  Thereby, the first multiplier 611 and the second multiplier 612 provided in the frame output position determination unit 7 of the first embodiment are not necessary, and the frame output position determination unit 7 can be constructed with a simpler configuration. it can.

  The third embodiment of the present invention will be described below.

  FIG. 10 shows a configuration example of the frame rate conversion unit in the third embodiment of the present invention. In particular, this configuration example is for conversion to an output frame rate higher than the input frame rate.

  In this frame rate conversion unit 403, a frame buffer (for example, a memory) of the buffer memory unit 6 is provided as a frame rate increasing unit 21 in the previous stage of the buffer memory unit 6. Further, an output for switching and outputting the video signal (input frame) output from the frame rate increasing unit 21 and the video signal (interpolation frame) output from the interpolation frame generating unit 5 at the subsequent stage of the buffer memory unit 6. A switching unit 22 is provided.

  In the third embodiment, a video signal (input frame) input from the input unit 1 is written to the frame rate increasing unit 21 and converted from a video signal at the input frame rate to a video signal at the output frame rate. . When the output frame rate is higher than the input frame rate, the number of output frames is larger than the number of input frames in the common unit time. In order to cope with this, in the present embodiment, the number of output frames is increased by repeatedly outputting a certain input frame of the frame rate increasing unit 21. The video signal (frame) output from the frame rate increasing unit 21 at the output frame rate is input to the frame delay unit 2, the motion vector calculation unit 3, the interpolation frame generation unit 5, and the output switching unit 22. Thereafter, the frame delay unit 2, the motion vector calculation unit 3, the vector conversion unit 4, and the interpolation frame generation unit 5 perform the same processing as in the first embodiment.

  The frame output position determination unit 7 detects the output frame position when converting from the input frame rate to the output frame rate from the input synchronization signal and the output synchronization signal, and sends a determination signal based on the detection result to the frame output control unit 7. Output. This operation is performed according to the output frame rate, unlike the first embodiment, which is performed according to the input frame rate. The frame output control unit 7 generates a frame switching signal for controlling the output switching unit 22 based on the determination signal from the frame output position determination unit 7. The output switching unit 22 can operate as a selector that switches between the frame output from the frame rate increasing unit 21 and the interpolated frame output from the interpolated frame generating unit 5 according to the frame switching signal.

  FIG. 11 shows video frame signals of respective parts by frame rate conversion from 50 fps to 60 fps in the third embodiment of the present invention.

  In this figure, S1 and S5 to S7 correspond to S1 to S4 in FIG. That is, S1 indicates the transition of the input frame 1 input to the buffer memory unit 6 at 50 fps. S5 indicates the transition of the frame output from the frame rate increasing unit 2. S6 indicates the transition of the input frame obtained by delaying the input frame by one frame by the frame delay unit 2. S7 shows the transition of the interpolation frame generated based on the input frame of S5 and the input frame of S6 and input to the output switching unit 22. S8 shows the transition of the frame output at 60 fps while the input frame of S5 and the interpolation frame of S7 are switched.

  As can be seen from the frame transition of S8, in this embodiment, the input frame is not repeatedly output, and is generated based on the output of the input frame itself and the input frames before and after the extension. The output of the interpolation frame is appropriately switched. In the third embodiment, when the output frame position is determined, for example, a determination signal indicating which of the input frame and the interpolated frame is output by replacing the B output range in FIG. 5 with the A output range. May be output.

  The preferred embodiments of the present invention have been described above, but these are examples for explaining the present invention, and the scope of the present invention is not limited to these embodiments. The present invention can be implemented in various other forms.

  For example, in the first to third embodiments, the frame rate conversion from 50 fps (frames / second) to 60 fps has been described as an example. However, the frame rate conversion technique of the present invention is applicable to various frame rate conversions. Can be applied.

  Specifically, for example, even when the input frame rate is 24 fps and the output frame rate is 60 fps, the frame rate conversion technique according to the embodiment of the present invention can be applied as shown in FIG. In this case, the same input frame (or the same interpolation frame) may be repeatedly output, although not as much as the prior art illustrated in FIG. 12A). Note that when this frame rate conversion is performed by the conventional technique of repeatedly outputting input frames, the same input frame may be repeatedly output every 1/12 seconds, for example, as illustrated in FIG. 12A. Many (the parallelogram in the figure indicates a frame, and the number in the frame indicates the frame number). Further, when this frame rate conversion is performed by another prior art that outputs an interpolation frame while adjusting the scaling ratio of the motion vector, as illustrated in FIG. 12B, the frame rate is output every 1/12 seconds. Four of the two frames are interpolated frames, and each interpolated frame is created with different scaling ratios even if they are made from the same two input frames (in the figure, V + w (for example, 1 + 2) frame means an interpolation frame created based on the vth input frame and the wth input frame).

  Similarly, for example, when the input frame rate is 60 fps and the output frame rate is 90 fps, the frame rate conversion technique according to the embodiment of the present invention can be applied as illustrated in FIG. 13C. When this frame rate conversion is performed by the conventional technique of repeatedly outputting an input frame, the conversion is as illustrated in FIG. 13A, and another conventional technique of outputting an interpolation frame while adjusting the scaling ratio of the motion vector. When using technology, the conversion is as illustrated in FIG. 13B.

  Depending on the ratio of the output frame rate to the input frame rate (in other words, the conversion magnification), the temporal distance m between the first input frame and the output frame and the output frame and the second input frame The sum with the temporal distance n may not be 2 to the power of x. For example, in the conversion from 60 fps to 90 fps illustrated in FIGS. 13A to 13C, m + n = 3 and 2 x power is not obtained. Even in such a case, the frame rate conversion unit generates an interpolated frame as m + n = 2 to the power of x (for example, as described above, m: n = 1: 1).

  For example, the input frame rate can be automatically detected from, for example, an input broadcast signal, and the output frame rate can also be automatically detected from a connected display unit. Further, the output frame rate may be manually set by the user via an operation unit (not shown), for example.

  For example, in the frame output position determination, two determination threshold values are used, but the number of determination threshold values may be larger or smaller. By setting the number of determination thresholds to three or more, it is possible to make a detailed determination as to what to output as an output frame. In that case, for example, the output range of the interpolation frame may be subdivided into a plurality of sub-ranges, and the motion vector scaling ratio for creating the interpolation frame may be different for each sub-range (that is, a plurality of types) Scaling ratios may be provided). Specifically, for example, the frame output position determination unit 7 notifies the motion vector calculation unit 3 which sub-range the frame output position belongs to, and the motion vector calculation unit 3 includes a plurality of types of scaling ratios. From the above, a scaling ratio corresponding to the notified sub-range may be selected. In this case, the value of P of the scaling ratio Q / P (that is, the value of m + n) is preferably 2 to the power of x (x is an integer of 1 or more) regardless of the values of m and n.

  Further, for example, the frame rate conversion unit may always generate an interpolation frame, or may not generate an interpolation frame for the amount that is not output when no interpolation frame is output. Specifically, for example, the frame rate conversion unit determines, based on the input frame rate and the output frame rate, at which timing the frame to be output should be an interpolation frame, and the interpolation frame that is output at the determined timing It is not necessary to generate an interpolation frame that generates only the other and is not output.

  For example, the determination threshold value is not limited to the above-described value, and may be M1 or M2 for comparison with the temporal distance m, for example.

  In the embodiment described above, all the input frames are all output as output frames, but not all the input frames are necessarily output. Specifically, for example, at least one input frame may not be output depending on the ratio between the input frame rate and the output frame rate, and an interpolation frame may be output instead.

FIG. 1 shows a configuration example of a display device according to a first embodiment of the present invention. FIG. 2 shows a configuration example of the frame rate conversion unit 403. FIG. 3 is an explanatory diagram of motion vectors. FIG. 4 is an explanatory diagram of determination of an output frame position. FIG. 5 is an explanatory diagram of determination of a frame to be output as an output frame. FIG. 6 shows frame transition when the frame rate conversion 403 according to the first embodiment is applied to frame rate conversion from 50 fps to 60 fps. FIG. 7 shows transitions of output frames output from the frame rate conversion unit 403 according to the first embodiment. FIG. 8 shows a circuit configuration example of the frame output position determination unit 7 in the first embodiment of the present invention. FIG. 9 shows a circuit configuration example of the frame output position determination unit 7 in the second embodiment of the present invention. FIG. 10 shows a configuration example of the frame rate conversion unit in the third embodiment of the present invention. FIG. 11 shows video frame signals of respective parts by frame rate conversion from 50 fps to 60 fps in the third embodiment of the present invention. FIG. 12A is an example of a transition of a frame output when one conventional technique converts 24 fps to 60 fps, and shows the result of consideration by the present applicant. FIG. 12B is an example of the transition of the frame output when the similar frame rate conversion is performed by another conventional technique, and shows the result of consideration by the present applicant. FIG. 12C shows an example of transition of a frame output when similar frame rate conversion is performed by the frame rate conversion technique according to the embodiment of the present invention. FIG. 13A is an example of a transition of a frame output when one conventional technique converts 60 fps to 90 fps, and shows the result of consideration by the present applicant. FIG. 13B is an example of a transition of a frame that is output when the same frame rate conversion is performed by another conventional technique, and shows the result of consideration by the present applicant. FIG. 13C shows an example of transition of frames output when similar frame rate conversion is performed by the frame rate conversion technique according to the embodiment of the present invention. FIG. 14A is a diagram showing an example of transition of frames input at an input frame rate of 50 fps. FIG. 14B is a diagram showing a result of examination by the applicant of the problem in the case of converting 50 fps to 60 fps by one conventional technique. FIG. 14C is a diagram showing a result of examination by the applicant of a problem when 50 fps is converted to 60 fps by another conventional technique.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Frame (video signal) input part 2 ... Frame delay part 3 ... Motion vector calculation part 4 ... Vector conversion part 5 ... Interpolation frame production | generation part 6 ... Buffer memory part 7 ... Frame output position determination part 8 ... Frame output control part DESCRIPTION OF SYMBOLS 9 ... Output frame 10 ... Input synchronization signal input unit 11 ... Output synchronization signal input unit 401 ... Display device 402 ... Moving image input unit 403 ... Frame rate conversion unit 404 ... Display unit

Claims (11)

In an apparatus for performing frame rate conversion for outputting a frame input at an input frame rate at an output frame rate,
An input means for inputting a plurality of frames at an input frame rate;
A motion vector is detected based on the first and second consecutive frames in the plurality of input frames, the magnitude of the detected motion vector is corrected based on a correction coefficient, and the corrected motion vector and Generating means for generating an interpolated frame predicted as a frame between the first and second frames based on the first and second frames;
Output means for outputting at least one of the plurality of input frames and the generated interpolation frame at an output frame rate;
The correction coefficient is a value that does not change during the frame rate conversion.
Frame rate conversion device. In an apparatus for performing frame rate conversion for outputting a frame input at an input frame rate at an output frame rate,
An input means for inputting a plurality of frames at an input frame rate;
A motion vector is detected based on the first and second consecutive frames in the plurality of input frames, the magnitude of the detected motion vector is corrected based on a correction coefficient, and the corrected motion vector and Generating means for generating an interpolated frame predicted as a frame between the first and second frames based on the first and second frames;
Output means for outputting at least one of the plurality of input frames and the generated interpolation frame at an output frame rate;
The first time distance, which is the time from when the first frame is input to when the frame is output, is m, and is the time from when the output is performed until the second frame is input. When the second temporal distance is n, the correction coefficient is m / (m + n) or n / (m + n),
The value of m + n is 2 to the power of x (x is an integer of 1 or more).
Frame rate conversion device. The correction factor is one half.
The frame rate conversion apparatus according to claim 1 or 2. In an apparatus for performing frame rate conversion for outputting a frame input at an input frame rate at an output frame rate,
An input means for inputting a plurality of frames at an input frame rate;
Generating means for generating an interpolated frame predicted as a frame between the first and second frames based on the first and second consecutive frames in the plurality of input frames;
Output means for outputting all or a part of the plurality of input frames and the generated interpolation frame at an output frame rate;
The generation unit generates the interpolation frame when the output unit outputs the interpolation frame, and if not, does not generate the interpolation frame,
The output means outputs the input frame more than the interpolation frame;
Frame rate conversion device. The output means includes a first temporal distance, which is a time from when the first frame is input to when the frame is output, and after the output is performed until the second frame is input. The first frame and / or the second frame based on at least one of a second temporal distance that is time, and the interpolated frame created based on the first and second frames, Select one frame from and output the selected frame.
The frame rate conversion apparatus according to claim 1, 2 or 4. The output means includes a first position in which a temporal position of an output frame is in a range closer to the first frame based on at least one of the first temporal distance and the second temporal distance. Belonging to the second range, a second range that is closer to the second frame, and an intermediate range that is a range between the first range and the second range. If it is determined that it belongs to the first range, the first frame is selected. If it is determined that it belongs to the second range, the second frame is selected. If it is determined that it belongs to the intermediate range, the interpolation frame is selected;
The frame rate conversion apparatus according to claim 5. The intermediate range is subdivided into a plurality of sub-intermediate ranges;
The generating means selects a correction coefficient corresponding to a sub-intermediate range to which the temporal position of the output frame belongs from among a plurality of types of correction coefficients, and selects the interpolation frame based on the selected correction coefficient. Generate,
The frame rate conversion apparatus according to claim 2 and 6. The frame rate conversion device according to any one of claims 1 to 7,
And a display unit configured to display a frame output from the frame rate conversion device. In a method of performing frame rate conversion in which a frame input at an input frame rate is output at an output frame rate,
Inputting a plurality of frames at an input frame rate;
A motion vector is detected based on the first and second consecutive frames in the plurality of input frames, the magnitude of the detected motion vector is corrected based on a correction coefficient, and the corrected motion vector and Generating an interpolated frame predicted as a frame between the first and second frames based on the first and second frames;
Outputting at least one of the input plurality of frames and the generated interpolated frame at an output frame rate;
The correction coefficient is a value that does not change during the frame rate conversion.
Frame rate conversion method. In a method of performing frame rate conversion in which a frame input at an input frame rate is output at an output frame rate,
Inputting a plurality of frames at an input frame rate;
A motion vector is detected based on the first and second consecutive frames in the plurality of input frames, the magnitude of the detected motion vector is corrected based on a correction coefficient, and the corrected motion vector and Generating an interpolated frame predicted as a frame between the first and second frames based on the first and second frames;
Outputting at least one of the input plurality of frames and the generated interpolated frame at an output frame rate;
The first time distance, which is the time from when the first frame is input to when the frame is output, is m, and is the time from when the output is performed until the second frame is input. When the second temporal distance is n, the correction coefficient is m / (m + n) or n / (m + n),
The value of m + n is 2 to the power of x (x is an integer of 1 or more).
Frame rate conversion method. In a method of performing frame rate conversion in which a frame input at an input frame rate is output at an output frame rate,
Inputting a plurality of frames at an input frame rate;
Generating an interpolated frame that is predicted as a frame between the first and second frames based on successive first and second frames in the plurality of input frames;
Outputting all or a part of the plurality of input frames and the generated interpolation frame at an output frame rate. In the generating step, the output means outputs the interpolation frame. The interpolated frame is generated, otherwise the interpolated frame is not generated,
In the outputting step, the input frame is output more than the interpolation frame.
Frame rate conversion method.

Images