JP2007082096A - Frame frequency conversion method, frame frequency conversion apparatus, and frame frequency conversion program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate an output dynamic image for progressive scanning with frame frequency different from an input dynamic image, without losing smoothness of animation, on the basis of the input dynamic image for interlace scanning. <P>SOLUTION: The frame frequency conversion method comprises: an animation interpolating pixel generation means 405 for generating animation interpolating pixels regarding each frame (target pixel) of an output frame (target output frame) to be included in the output dynamic image, on the basis of a pixel of an input field included in the input dynamic image by means of a first weighted addition; a still image interpolating pixel generation means 407 for generating still image interpolating pixels, regarding the target pixel on the basis of a pixel of the input field included in the input dynamic image by means of a second weighed addition; a moving amount generation means 421 for generating moving amount, regarding the target pixel based on the input dynamic image; and a target pixel generation means 423 for generating the target pixel, based on the moving image interpolating pixel and still image interpolating pixel by means of a third weighed addition that utilizes weighting corresponding to the moving amount. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a frame frequency conversion method, a frame frequency conversion device, and a frame for generating an output moving image of a progressive scan having a frame frequency different from that of the input moving image based on an input moving image of interlaced scanning or progressive scanning. It relates to a frequency conversion program.

  In recent years, a personal computer has a built-in or connected TV tuner, and the personal computer can display, record, and play back TV broadcasts received by the TV tuner. ing.

  In Japan and several other countries, the NTSC system is adopted for television broadcasting. In the NTSC system, the field frequency is about 59.94 Hz (= 60 × (1000/1001) Hz) instead of 60 Hz in order to adjust the frequency with the color subcarrier frequency. Therefore, the frame frequency is not about 30 Hz but about 29.97 Hz (= 30 × (1000/1001) Hz). The frame frequency of a moving image obtained by performing IP (Interlace / Progressive) conversion on the signal output from the NTSC tuner is 59.94 Hz.

  On the other hand, a typical refresh rate (corresponding to a frame frequency) of a television monitor of a personal computer is just 60 Hz. Many personal computers can also set other refresh rates.

  Then, since the frame frequency of the television image to be displayed is different from the refresh rate of the television monitor, the personal computer needs to perform processing for solving this difference.

Conventionally, as a process for this, some frames are thinned out or displayed twice (for example, see Patent Document 1).
JP 2004-80327 A

  However, when the frame frequency is adjusted by thinning out some frames or displaying twice, the smoothness of the moving image is lost before and after the thinning and at the twice-displayed portion, which is unnatural. .

  Therefore, the present invention is to generate a progressive scan output moving image having a frame frequency different from that of the input moving image based on an input moving image of interlace scanning or progressive scanning without losing the smoothness of the moving image. It is an object of the present invention to provide a frame frequency conversion method, a frame frequency conversion device, and a frame frequency conversion program.

According to a first aspect of the present invention, in the frame frequency conversion method for generating an progressive moving output moving image having a frame frequency different from that of the input moving image based on the input moving image of the interlace scanning, At least one moving image interpolation pixel for each pixel (hereinafter referred to as “target pixel”) of each output frame (hereinafter referred to as “target output frame”) to be included in the output moving image is included in the input moving image. A moving image interpolation pixel generation step for generating by a first weighted addition based on at least one pixel of one input field;
A still image interpolation pixel generating step for generating a still image interpolation pixel for the target pixel by a second weighted addition based on at least one pixel of at least one input field included in the input moving image; A motion amount generation step for generating a motion amount for the target pixel based on the input moving image, and the target pixel according to the motion amount based on the moving image interpolation pixel and the still image interpolation pixel. And a target pixel generation step generated by a third weighted addition using a weight. A frame frequency conversion method is provided.

  In the frame frequency conversion method, in the moving image interpolation pixel generation step, an input field (hereinafter referred to as “first input field”) immediately before the target output frame among a plurality of input fields included in the input moving image. .) Of at least one pixel that is spatially closest to the target pixel, and of the pixels in the next field of the first field (hereinafter referred to as “second input field”). The moving image interpolation pixel may be generated based on at least one pixel or both of which are spatially closest to the target pixel.

  In the frame frequency conversion method, in the still image interpolation pixel generation step, the target pixel among the pixels of a plurality of input fields (hereinafter referred to as “preceding input field group”) preceding the target output frame is selected. The target pixel among the pixels in the same spatial position and the closest position in time, and pixels in a plurality of input fields after the preceding input field group (hereinafter referred to as “subsequent input field group”) The still image interpolation pixel may be generated based on at least a pixel located at the same spatial position and the closest position in time or both.

  In the frame frequency conversion method, the motion amount generation step includes a first motion amount calculation step of calculating a first motion amount for the target pixel from a plurality of odd fields included in the input moving image, and the input A second motion amount calculating step for calculating a second motion amount for the target pixel from a plurality of even fields included in the moving image; and the motion based on the first motion amount and the second motion amount. And a final motion amount calculating step for calculating the amount.

  In the frame frequency conversion method, in the first motion amount calculation step, an odd input field (hereinafter referred to as “first odd input”) immediately before the target output frame among a plurality of odd fields included in the input moving image. Of at least one pixel spatially closest to the target pixel and an odd field next to the first odd field (hereinafter referred to as a “second odd field”). The first motion amount is calculated based on at least one pixel that is spatially closest to the target pixel among the pixels. In the second motion amount calculation step, the first motion amount is included in the input moving image. An even input field (hereinafter referred to as a “first even input field”) immediately before the target output frame among a plurality of even fields. Of at least one pixel that is spatially closest to the target pixel and pixels of the even field next to the first even field (hereinafter referred to as “second even field”). The second motion amount may be calculated based on at least one pixel that is spatially closest to the target pixel.

  The frame frequency conversion method may further include a step of estimating the timing of the vertical synchronization signal corresponding to the target output frame based on the output frame frequency and the scanning line number sequentially sampled from the graphic unit. May be.

  According to a second aspect of the present invention, in the frame frequency conversion method for generating a progressive scan output moving image having a frame frequency different from that of the input moving image based on the input moving image of progressive scanning, the input An input frame (hereinafter referred to as “first input frame”) immediately before each output frame (hereinafter referred to as “target output frame”) to be included in the output moving image among a plurality of input frames included in the moving image. .) To the target output frame and the period from the target output frame to the next input frame of the first input frame (hereinafter referred to as “second input frame”). And a weight calculating step for calculating a weight of the second input frame, and each pixel of the target output frame (hereinafter referred to as “target pixel”) A weighted addition of at least one pixel of the first input frame and at least one pixel of the second input frame based on a weight of the first input frame and a weight of the second input frame. A frame frequency conversion method comprising: a target pixel generation step for generating the target pixel.

  In the frame frequency conversion method described above, in the target pixel generation step, the first input frame of the pixel of the second input frame (hereinafter referred to as “subsequent pixel”) used for generating the target pixel. When the amount of motion with respect to a pixel (hereinafter referred to as “preceding pixel”) that is spatially the same as the subsequent pixel is less than a predetermined value, the weighted addition is not performed, and the subsequent pixel is used. The target pixel may be generated without using the pixel.

  The frame frequency conversion method may further include a step of estimating the timing of the vertical synchronization signal corresponding to the target output frame based on the output frame frequency and the scanning line number sequentially sampled from the graphic unit. May be.

  The frame frequency conversion method may further include an interlace / progressive conversion step of generating the progressive scan input moving image based on the interlaced scan moving image, and the interless / progressive conversion step includes the interlaced scan step. A first motion amount calculating step for calculating a first motion amount for each pixel of each frame of the progressive moving input moving image from a plurality of odd fields included in the moving image, and the interlaced scanning moving image A second motion amount calculating step for calculating a second motion amount for each pixel of each frame of the progressive scan input moving image from a plurality of even fields included; the first motion amount and the second motion; On the basis of the amount, for each pixel of each frame of the progressive scan input moving image A final motion amount calculating step for calculating the amount of motion, and each pixel of each frame of the progressive scan input moving image, a moving image interpolation pixel and a still image interpolation pixel obtained on the basis of the interlaced scan moving image. Based on this, an intermediate pixel generation step of generating by weighted addition using a weight corresponding to the motion amount calculated in the final motion amount calculation step may be provided.

  According to a third aspect of the present invention, in a method for controlling a frame frequency of a television monitor, a plurality of frames to be output to the television monitor are frame-buffered according to a frame frequency of an output unit of the plurality of frames. A step of sequentially writing to the television monitor, a step of sequentially reading out the plurality of frames from the frame buffer in accordance with a frame frequency of the television monitor, and a frame number of the television monitor based on the number of frames stored in the frame buffer. And a control step for controlling the frequency.

  In the above method, the output unit of the frame further includes an interlace / progressive conversion step of generating a progressive scan moving image based on the interlaced scan moving image, and the interlace / progressive conversion step includes the interlace / progressive conversion step. A first motion amount calculating step for calculating a first motion amount for each pixel of each frame of the progressive scan moving image from a plurality of odd fields included in the less scan moving image; and the interlaced scan moving image. A second motion amount calculating step for calculating a second motion amount for each pixel of each frame of the progressive scan moving image from a plurality of even fields included in the first motion amount, and the first motion amount and the second motion Based on the amount, the amount of motion for each pixel of each frame of the progressive scan moving image is calculated. The final motion amount calculating step to be output, and each pixel of each frame of the progressive scan moving image, based on the moving image interpolation pixel and the still image interpolation pixel obtained based on the moving image of the interlace scanning, A pixel generation step of generating by weighted addition using a weight corresponding to the motion amount calculated in the final motion amount calculation step.

  According to the present invention, in the frame frequency conversion method for generating an output moving image of progressive scanning having a frame frequency different from that of the input moving image based on the input moving image of interlace scanning, the output moving image includes the output moving image. A moving image interpolation pixel for each pixel (hereinafter referred to as “target pixel”) of each output frame to be performed (hereinafter referred to as “target output frame”) is set as at least one of at least one input field included in the input moving image. Based on one pixel, a moving image interpolation pixel generation step generated by a first weighted addition, and a still image interpolation pixel for the target pixel, at least one of at least one input field included in the input moving image A still image interpolation pixel generation step generated by the second weighted addition based on the pixel, and a motion for the target pixel; A motion amount generating step for generating an amount based on the input moving image, and a target pixel based on the moving image interpolation pixel and the still image interpolation pixel and using a weight corresponding to the motion amount. The target pixel generation step generated by the weighted addition of 3 is provided, so that it is possible to avoid missing some input fields or outputting some input fields as they are multiple times. Smoothness is not lost.

  The best mode for carrying out the present invention will be described below in detail with reference to the drawings.

[Embodiment 1]
FIG. 1 is a block diagram showing the overall configuration of the frame frequency conversion apparatus according to the first embodiment.

  Referring to FIG. 1, an IP (Interlace / Progressive) conversion processing unit 101 receives, for example, an interlaced moving image output from an NTSC tuner, and performs interlaced / progressive conversion on the moving image. Then, the IP conversion processing unit 101 writes the progressive scan moving image obtained by this conversion into the frame buffer 102 for each frame. Also, the IP conversion processing unit 101 refers to the current time input from the system clock 103 of the personal computer, adds the current time to each frame of the progressive scan moving image, and also associates this with each frame. Write to buffer 102. The IP conversion processing performed by the IP conversion processing unit 101 will be described later, but the IP conversion processing unit 101 may perform a commonly used IP conversion process.

  The vertical synchronization signal acquisition unit 104 acquires the vertical synchronization signal of the output video signal from the graphic card 105 inserted in the motherboard of the personal computer or the graphic chip built in the motherboard. The frequency of this vertical synchronizing signal is, for example, 60 Hz. Then, when the vertical synchronization signal acquisition unit 104 acquires the vertical synchronization signal from the graphic card 105 or the like, the interpolation processing unit 106 is activated immediately after that. The vertical synchronization signal acquisition unit 104 supplies the current time obtained from the system clock 103 to the interpolation processing unit 106 at the time of activation. Note that the output frame time may be offset so that the interpolation processing unit 106 can acquire the input frame with a time later than the output frame time from the frame buffer 102. That is, for example, the interpolation processing unit 106 can acquire an input frame with a time later than the time of the output frame by setting the time of the output frame to a time 45 milliseconds past the time obtained from the system clock. You may do it.

  When started, the interpolation processing unit 106 reads a necessary frame from the frame buffer 102, generates an output frame based on the read frame, and writes the generated frame in the display memory 105-1 in the graphic card 105.

  The graphic card 105 outputs the output frame written in the display memory 105-1 to the television monitor 107.

  Next, the configuration of the IP conversion processing unit 101 and the IP conversion processing performed by this will be described.

  FIG. 2 is a block diagram illustrating a configuration of the IP conversion unit 101.

  The first field memory 201-1 stores image data of a field one field ahead of the field to be processed. The second field memory 201-2 stores image data of a field to be processed. The third field memory 201-3 stores image data of the field immediately before the field to be processed.

  The image data writing unit 203 writes the image data in the first to third field memories 201-1, 201-2, and 201-3. Note that the first to third field memories 201-1, 201-2, and 201-3 have a ring buffer configuration, and the image data of the three fields are stored in these field memories in a circular manner. May be.

  The moving image interpolation pixel generation unit 205 generates a moving image interpolation pixel based on the image data stored in the second field memory 201-2.

  The moving image interpolation pixel generation unit 205 basically has, for example, the field number currently being processed as n and the vertical direction of the pixel currently being processed as a pixel to be interpolated in that field. When the position is i and the horizontal position of the pixel currently being processed as a pixel to be interpolated in that field is j, the moving image interpolation pixel A (n, i, j) is represented by the equation (1). Generate. Here, i is a position when the actual scanning line and the scanning line to be interpolated are counted together.

ここで、ｘ（ｎ，ｉ−１，ｊ）は、処理の対象となっているフィールドの処理の対象となっている画素の１つ上の画素の値であり、ｘ（ｎ，ｉ＋１，ｊ）は、処理の対象となっているフィールドの処理の対象となっている画素の１つ下の画素の値である。上記では、これらの２つの画素の値の平均値を処理の対象となっている画素の値としているが、これらの２つの画素のうちの１つの画素の値を処理の対象となっている画素の値としてもよい。また、処理の対象となっているフィールドの画素のうちの処理の対象となっている画素の所定の近傍の画素の値の加重平均を処理の対象となっている画素の値としてもよい。

Here, x (n, i−1, j) is the value of the pixel one pixel above the pixel to be processed in the field to be processed, and x (n, i + 1, j). ) Is the value of the pixel one pixel below the pixel to be processed in the field to be processed. In the above, the average value of the values of these two pixels is the value of the pixel to be processed, but the value of one pixel of these two pixels is the pixel to be processed. It is good also as the value of. Also, a weighted average of the values of pixels in a predetermined vicinity of the pixel to be processed among the pixels in the field to be processed may be used as the value of the pixel to be processed.

  The still image interpolation pixel generation unit 207 generates still image pixels based on the pixel data stored in the first field memory 201-1 and the pixel data stored in the third field memory 201-3. Generate. For example, the number of the field currently being processed is n, the vertical position of the pixel currently being processed as a pixel to be interpolated in that field, i, and the pixel to be interpolated in that field If the horizontal position of the pixel currently being processed is j, a moving image interpolation pixel B (n, i, j) is generated according to equation (2).

ここで、ｘ（ｎ−１，ｉ，ｊ）は、処理の対象となっているフィールドの１つ前のフィールドの画素のうち、処理の対象となっている画素と同一の位置（ｉ，ｊ）にある画素の値であり、ｘ（ｎ＋１，ｉ，ｊ）は、処理の対象となっているフィールドの１つ後のフィールドの画素のうち、処理の対象となっている画素と同一の位置（ｉ，ｊ）にある画素の値である。上記では、これらの２つの画素の値の平均値を処理の対象となっている画素の値としているが、これらの２つの画素のうちの１つの画素の値を処理の対象となっている画素の値としてもよい。また、処理の対象となっているフィールドの近傍のフィールドの画素のうちの処理の対象となっている画素と同一の位置にある画素の値の加重平均を処理の対象となっている画素の値としてもよい。

Here, x (n−1, i, j) is the same position (i, j) as the pixel to be processed among the pixels in the field immediately before the field to be processed. ), And x (n + 1, i, j) is the same position as the pixel to be processed among the pixels in the next field after the field to be processed. This is the value of the pixel at (i, j). In the above, the average value of the values of these two pixels is the value of the pixel to be processed, but the value of one pixel of these two pixels is the pixel to be processed. It is good also as the value of. Also, the value of the pixel subject to processing is the weighted average of the values of the pixels in the same position as the pixel subject to processing among the pixels in the field in the vicinity of the field subject to processing. It is good.

  The difference calculation unit 209 obtains a difference C (n, i, j) for the pixel to be processed in the field to be processed according to the equation (3).

ここで、ｘ（ｎ−１，ｉ，ｊ）は、処理の対象となっているフィールドの１つ前のフィールドの画素のうち、処理の対象となっている画素と同一の位置（ｉ，ｊ）にある画素の値であり、ｘ（ｎ＋１，ｉ，ｊ）は、処理の対象となっているフィールドの１つ後のフィールドの画素のうち、処理の対象となっている画素と同一の位置（ｉ，ｊ）にある画素の値である。差分Ｃ（ｎ，ｉ，ｊ）は、動き量の基本となる。

Here, x (n−1, i, j) is the same position (i, j) as the pixel to be processed among the pixels in the field immediately before the field to be processed. ), And x (n + 1, i, j) is the same position as the pixel to be processed among the pixels in the next field after the field to be processed. This is the value of the pixel at (i, j). The difference C (n, i, j) is the basis of the amount of motion.

  The difference correction unit 211 calculates the corrected difference D (n, i, j) by correcting C according to the equation (4).

ここで、Δｃの値は、ノイズのレベル等により決定され、補正値保持部２１３に格納されている。

Here, the value of Δc is determined by the noise level or the like and stored in the correction value holding unit 213.

  The difference accumulating unit 215 updates the accumulated motion amount m (i, j) stored in the accumulated motion amount memory 217 according to the equation (5).

ここで、ＴＨの値は、視覚特性等により決定され、しきい値保持部２１９に格納されている。ＴＨの値は、例えば、１６である。

Here, the value of TH is determined by visual characteristics or the like and stored in the threshold value holding unit 219. The value of TH is 16, for example.

  If a certain amount of motion is detected by calculating the cumulative motion amount m (i, j) according to the above equation, the cumulative motion amount m (i, j) A value obtained by adding 1 to the accumulated motion amount in the previous field and subtracting 1 or when the motion amount is large is a threshold value TH, and decreases by 1 every time one field has passed since then. By doing so, the motion detection is tailed. Thereby, the possibility that the moving image part is erroneously determined to be a still image part after movement can be reduced. When the moving image part is erroneously determined to be a still image part, an interpolation value completely different from the interpolation value calculated when correctly processed as the moving image part is calculated, which causes a visual problem. On the other hand, if the still image portion is erroneously determined to be a moving image portion, only the resolution is lowered, and this is not a visual problem. In particular, even if the still image portion around the portion where the motion is actually detected is erroneously determined to be a moving image portion, the reduction in the resolution of the portion is not visually recognized due to the masking effect due to the portion that actually moves. . Therefore, the advantage of introducing the formula (5) and reducing the possibility that the moving image portion is erroneously determined as the still image portion is that the possibility that the still image portion is erroneously determined as the moving image portion is increased. Therefore, the introduction of the equation (5) is useful for improving the image quality as a whole. In addition, since many still image parts such as a telop part in which characters are fixed, no motion is detected from the part, even if the expression (5) is introduced, it is erroneously a moving image part. It will not be judged.

  The motion amount calculation unit 221 calculates the motion amount M (i, j) according to Equation (6).

合成補間画素生成部２２３は、式（７）に従って合成補間画素ｘ’（ｎ，ｉ，ｊ）を計算する。

The synthetic interpolation pixel generation unit 223 calculates the synthetic interpolation pixel x ′ (n, i, j) according to the equation (7).

選択部２２５は、ライン毎に実在画素ｘ（ｎ，ｉ，ｊ）と補間画素ｘ’（ｎ，ｉ，ｊ）を切り換えて、プログレッシブ走査の画像データを出力する。

The selection unit 225 switches the actual pixel x (n, i, j) and the interpolation pixel x ′ (n, i, j) for each line, and outputs progressive scan image data.

  The IP conversion processing unit 101 whose configuration is shown in FIG. 2 can be realized by hardware, but the computer records a program for causing the computer to function as the IP conversion processing unit 101 whose configuration is shown in FIG. It can also be realized by reading from the medium and executing it.

  Next, the effect of calculating the motion amount M (i, j) according to the equation (6) will be described.

  As shown in FIG. 3, it is assumed that a black vertical line moves to the right on a white flat background. Further, in order to facilitate the consideration, it is assumed that the motion accumulation amount m (i, j) is calculated according to the equation (8).

そうすると、式（６）を用いずに、仮に、式（９）に従って、動き量Ｍ（ｉ，ｊ）を計算することとすると、図３に示すように、動きが検出されずに、静止画補間画素により補間が行われることとなり、コーミングが発生する。

Then, if the motion amount M (i, j) is calculated according to the equation (9) without using the equation (6), as shown in FIG. Interpolation is performed by the interpolation pixel, and combing occurs.

すなわち、図３において、四角の画素は元々ある画素であり、丸の画素は補完するべき画素である。例えば、画素１３４と画素３３４との差分の絶対値が画素２３４の動き量となるが、この値はゼロであるため、画素２３４は、静止画画素であると間違って判断される。そうすると、画素１３４と画素３３４の平均値が画素２３４の補間値となり、画素２３４は白となってしまう。画素２１４、２５４についても同様である。そうすると、画素２２４、２４４、２６４が正しく黒であるのに対し、画素２１４、２３４、２５４は間違って白となってしまい、コーミングが発生する。

That is, in FIG. 3, square pixels are originally pixels, and round pixels are pixels to be complemented. For example, the absolute value of the difference between the pixel 134 and the pixel 334 is the amount of motion of the pixel 234. Since this value is zero, the pixel 234 is erroneously determined to be a still image pixel. Then, the average value of the pixel 134 and the pixel 334 becomes the interpolation value of the pixel 234, and the pixel 234 becomes white. The same applies to the pixels 214 and 254. Then, while the pixels 224, 244, and 264 are correctly black, the pixels 214, 234, and 254 are erroneously white and combing occurs.

  On the other hand, if the motion amount M (i, j) is calculated according to the equation (6) according to the present invention, the motion is correctly detected and interpolation is performed by the moving image interpolation pixels as shown in FIG. No combing occurs.

  That is, in FIG. 4, as in the case of FIG. 3, square pixels are originally pixels, and round pixels are pixels to be complemented. For example, the larger of the sum of the absolute value of the difference between the pixel 044 and the pixel 244 and the threshold value TH in the absolute value of the difference between the pixel 134 and the pixel 334 is the amount of movement of the pixel 234. Since the value is the threshold value TH, the pixel 234 is correctly determined as a moving image pixel. Then, the average value of the pixel 224 and the pixel 244 becomes the interpolation value of the pixel 234, and the pixel 234 becomes black. The same applies to the pixels 214 and 254. Then, since the pixels 224, 244, and 264 are correctly black and the pixels 214, 234, and 254 are also correctly black, no combing occurs.

  Thus, by calculating the motion amount M (i, j) according to the equation (6), it is possible to avoid erroneously determining that a moving image pixel is a still image pixel.

  In Expression (6), when the input field corresponding to the number n is an even field, m (i, j) is a motion amount detected from the even field, and m (i + 1, j) is , The amount of motion detected from the odd field. Similarly, in Equation (6), when the input field corresponding to the number n is an odd field, m (i, j) is the amount of motion detected from the odd field, and m (i + 1, j) Is the amount of motion detected from the even field. That is, the motion amount M (i, j) is used by utilizing the fact that the motion that could not be detected from one of the even field and the odd field can be detected from the other field. I am going to do that.

  Next, an interlace / progressive conversion method performed by the IP conversion processing unit 101 whose configuration is shown in FIG. 2 will be described.

  Referring to FIG. 5, first, the moving image interpolation pixel generation unit 205 generates a moving image interpolation pixel according to Expression (1) (step S301). Next, the still image interpolation pixel generation unit 207 generates a still image interpolation pixel according to Expression (2) (step S303). Next, the difference calculation unit 309 calculates the difference according to the equation (3) (step S305). Next, the difference correction unit 211 corrects the difference according to the equation (4) (step S307). Next, the difference accumulating unit 215 updates the accumulated motion amount according to the equation (5) (step S309). Next, the motion amount calculation unit 221 calculates the motion amount according to Equation (6) (step S311). Next, the synthetic | combination interpolation pixel production | generation part 223 produces | generates a synthetic | combination interpolation pixel according to Formula (7) (step S315).

  Next, the configuration of the interpolation processing unit 106 and the interpolation processing performed by this will be described.

  Referring to FIG. 6, the output frame time input unit 106-1 receives the time of each output frame from the vertical synchronization signal acquisition unit 104.

  The input frame time reference unit 106-2 refers to the time of each input frame stored in the frame buffer 102, and inputs the input frame time (first input) immediately before the time of the output frame currently being processed. Frame time) and an input frame time (second input frame time) immediately after the first input frame time.

  The first input frame reading unit 106-3 receives the first input frame time from the input frame time reference unit 106-2, and further, the frame buffer receives an input frame (first input frame) corresponding to the first input frame time. Read from 102.

  The second input frame reading unit 106-4 receives the second input frame time from the input frame time reference unit 106-2, and further, the frame buffer stores an input frame (second input frame) corresponding to the second input frame time. Read from 102.

  The weight calculation unit 106-5 weights the first input frame and the second input frame based on the output frame time tout, the first input frame time tin1, and the second input frame time tin2 according to the following equation (10). Weights w1 and w2 for addition are calculated.

加重加算部１０６−６は、下に示す式（１１）に従って、第１入力フレームの各画素ｘ１（ｉ、ｊ）と第２入力フレームの各画素ｘ２（ｉ、ｊ）を加重加算することにより、出力フレームの各画素ｙ（ｉ、ｊ）を生成し、これを表示メモリ１０５−１に書き込む。

The weighted addition unit 106-6 performs weighted addition of each pixel x1 (i, j) of the first input frame and each pixel x2 (i, j) of the second input frame in accordance with Expression (11) shown below. , Each pixel y (i, j) of the output frame is generated and written into the display memory 105-1.

次に、図６にその構成を示す補間処理部１０６により行われる補間処理方法について説明する。

Next, an interpolation processing method performed by the interpolation processing unit 106 whose configuration is shown in FIG. 6 will be described.

  Referring to FIG. 7, first, the output frame time input unit 106-1 receives the output frame time from the vertical synchronization signal acquisition unit 104 (step S321). Next, the input frame time reference unit 106-2 detects the first input frame time and the second input frame time (steps S323 and S325). Next, the weight calculation unit 106-5 calculates a weight for weighted addition (step S327). Next, steps S331 to S337 are repeated (step S329). In step S331, the first input frame reading unit 106-3 reads the pixels of the first input frame from the frame buffer 102. Next, the second input frame reading unit 106-4 reads the pixels of the second input frame from the frame buffer 102 (step S333). Next, the weighted addition unit 106-6 generates a pixel of the output frame by weighted addition (step S335), and writes this in the display memory 105-1 (step S337). Step S329 may be repeated for each of a plurality of pixels to reduce the number of accesses to the memory.

  Note that the interpolation processing unit 106 whose configuration is shown in FIG. 6 can be realized by hardware, but the computer stores a program for causing the computer to function as the interpolation processing unit 106 whose configuration is shown in FIG. It can also be realized by reading and executing.

  According to the first embodiment, each pixel of the output frame is determined based on the pixel of the first input frame and the pixel of the second input frame, and the period from the first input frame to the output frame and from the output frame to the second dormitory frame. Since the weighted addition is performed with the weight based on the period of time, unnaturalness does not occur in the motion of the output moving image, and the motion becomes smooth.

[Embodiment 2]
The second embodiment is the same as the first embodiment regarding the basic configuration. However, compared to the first embodiment, the second embodiment looks at the amount of motion for each pixel of the second input frame and determines that the amount of motion is less than a predetermined value. The difference is that the pixel of the output frame is generated from the pixel of the first input frame without using the pixel of the frame. Thereby, since the weighted addition process by the weighted addition unit 106-6 can be omitted for such a pixel, the amount of calculation can be reduced. Even if it is omitted as such, the image quality of the output frame does not deteriorate so much.

  FIG. 8 is a block diagram showing the overall configuration of the frame frequency conversion apparatus according to the second embodiment. As is clear from comparing FIG. 8 with FIG. 1, the frame frequency conversion device according to the second embodiment is different from the frame frequency conversion device according to the first embodiment in that a motion amount buffer 108 is added, and the connection increases accordingly. The point and interpolation processing unit 106 is changed to the interpolation processing unit 106B.

  The motion amount buffer 108 stores the motion amount calculated by the motion amount calculation unit 221 (FIG. 2) for each pixel of each input frame stored in the frame buffer 102.

  FIG. 9 is a block diagram illustrating a configuration of the interpolation processing unit 106B according to the second embodiment. As is clear from comparison of FIG. 9 with FIG. 2, the interpolation processing unit 106B according to the second embodiment is different from the interpolation processing unit 106 according to the first embodiment in that the second input frame pixel motion amount reading unit 106-7 is omitted. The part 106-8 and the switch 106-9 are added, and the connection is increased accordingly.

  The second input frame pixel motion amount reading unit 106-7 reads the motion amount corresponding to each pixel of the second input frame from the motion amount buffer 108.

  The omission determination unit 106-8 determines whether or not the motion amount read by the second input frame pixel motion amount reading unit 106-7 is less than a predetermined threshold value. For the pixel, the readout by the second input frame readout unit 106-4 and the weighted addition by the weighted addition unit 106-6 are suspended, and for the switch 106-9, the pixel read out by the first input frame readout unit 106-3 To select. Otherwise, the second input frame reading unit 106-4 and the weighted addition unit 106-6 operate in the same manner as in the first embodiment, and the switch 106-9 selects the output of the weighted addition unit 106-6. .

  The predetermined threshold value to be compared with the motion amount read by the second input frame pixel motion amount reading unit 106-7 is, for example, a value that the motion amount can take from 0 to X, and the larger the X, the more the motion amount. X may be used when it represents large. However, the value may be smaller than X. An appropriate value is set in consideration of the degree of image quality deterioration and the amount of calculation amount reduction. The CPU's computing power and current load may be detected and taken into account. The larger the value of X, the greater the effect of reducing the amount of computation, but the degree of image quality deterioration is relatively large.

  Next, an interpolation processing method performed by the interpolation processing unit 106B whose configuration is shown in FIG. 9 will be described.

  Referring to FIG. 10, first, the output frame time input unit 106-1 receives the output frame time from the vertical synchronization signal acquisition unit 104 (step S341). Next, the input frame time reference unit 106-2 detects the first input frame time and the second input frame time (steps S343 and S345). Next, the weight calculation unit 106-5 calculates a weight for weighted addition (step S347). Next, steps S351 to S363 are repeated (step S349). In step S351, the first input frame reading unit 106-3 reads the pixels of the first input frame from the frame buffer 102. Next, the second input frame pixel motion amount reading unit 106-7 reads the current motion amount of the pixel from the motion amount buffer 108 (step S353). Next, the omission determination unit 106-8 determines whether or not the read motion amount is less than a predetermined threshold value (step S355).

  If so (YES in step S355), the switch 106-9 selects the first frame pixel (step S357) and writes it in the display memory 105-1 (step S365).

  Otherwise (NO in step S355), the second input frame reading unit 106-4 reads the pixels of the second input frame from the frame buffer 102 (step S359). Next, the weighted addition unit 106-6 generates pixels of the output frame by weighted addition (step S361). Next, the switch 106-9 selects a pixel generated by weighted addition (step S363) and writes it in the display memory 105-1 (step S365). The number of accesses to the memory may be reduced by repeating step S349 for each of a plurality of pixels.

  Further, instead of performing the calculation of the motion amount and the determination based on the calculation and the conditional operation based on the determination for each pixel, for example, the calculation may be performed for each block of M × N pixels. Accordingly, the motion amount generation method in the IP conversion processing unit 101 may be changed to one for each block.

  Although the interpolation processing unit 106B whose configuration is shown in FIG. 9 can be realized by hardware, the computer uses a recording medium to store a program for causing the computer to function as the interpolation processing unit 106B whose configuration is shown in FIG. It can also be realized by reading and executing.

[Embodiment 3]
In the first and second embodiments, the IP conversion processing unit 101 and the interpolation processing unit 106 or 106B are provided separately, but in the third embodiment, these are integrated.

  The principle of the third embodiment will be described with reference to FIGS. 11 and 12. 11 and 12, the time axis is horizontal and the vertical axis of the screen is vertical, the input fields f (n−1), f (n), f (n + 1), f (n + 2) and the output frame F (m ) At a certain horizontal position. In addition, what is indicated by a double circle in the output frame is a pixel (target pixel) y (m, i, j) that is currently generated. Here, i is a line number in the frame, and j is a pixel number in the line. The input field f (n) is a field immediately before the output frame F (m), and the input field f (n + 1) is an input field immediately after the input field f (n).

  FIG. 11A shows the target pixel y (m) when the input field f (n) has a pixel x (n, i, j) spatially located at the same position as the target pixel y (m, i, j). , I, j) shows an example of a method for generating a moving image interpolation pixel ym (m, i, j). In the input field f (n), the pixel spatially closest to the target pixel y (m, i, j) is the pixel x (n, i, j). In the input field f (n + 1), the target pixel y ( The pixels spatially closest to m, i, j) are x (n + 1, i-1, j) and x (n + 1, i + 1, j).

  When the time of the output frame F (m) is tout, the time of the input field f (n) is tin1, and the time of the input field f (n + 1) is tin2, the moving image interpolation pixel ym (m, i, j) is generated. For this purpose, the weights w1 and w2 in the weighted addition are calculated by the following equation (12).

そして、動画補間画素ｙｍ（ｍ、ｉ、ｊ）を下に示す式（１３）により算出する。

Then, the moving image interpolation pixel ym (m, i, j) is calculated by the following equation (13).

但し、動画補間画素ｙｍ（ｍ、ｉ、ｊ）を下に示す式（１４）又は式（１５）により算出してもよい。

However, the moving image interpolation pixel ym (m, i, j) may be calculated by the following formula (14) or formula (15).

また、重みｗ１＞ｗ２の場合に、式（１４）を利用し、そうでない場合に、式（１５）を利用するようにしてもよい。

Further, when the weight w1> w2, the expression (14) may be used, and if not, the expression (15) may be used.

  Also, input pixels other than the input pixels appearing in Expression (13), Expression (14), and Expression (15) (for example, pixels in the vicinity of the position (i, j) of the fields f (n) and f (n + 1)) are also included. The moving image interpolation pixel ym (m, i, j) may be calculated by using this.

  FIG. 11B shows the target pixel y when the input field f (n + 1) includes a pixel x (n, i ′, j) that is spatially located at the same position as the target pixel y (m, i ′, j). An example of a method for generating a moving image interpolation pixel ym (m, i ′, j) for (m, i ′, j) is shown. In the input field f (n + 1), the pixel spatially closest to the target pixel y (m, i ′, j) is the pixel x (n + 1, i ′, j). In the input field f (n), the target pixel The pixels spatially closest to y (m, i ′, j) are x (n, i′−1, j) and x (n, i ′ + 1, j).

  When the time of the output frame F (m) is tout, the time of the input field f (n) is tin1, and the time of the input field f (n + 1) is tin2, the moving image interpolation pixel ym (m, i ′, j) is generated. The weights w1 and w2 in the weighted addition for calculating are calculated by the equation (12) shown above.

  Then, the moving image interpolation pixel ym (m, i ', j) is calculated by the following equation (16).

但し、動画補間画素ｙｍ（ｍ、ｉ’、ｊ）を下に示す式（１７）又は式（１８）により算出してもよい。

However, the moving image interpolation pixel ym (m, i ′, j) may be calculated by the following expression (17) or expression (18).

また、重みｗ１＞ｗ２の場合に、式（１７）を利用し、そうでない場合に、式（１８）を利用するようにしてもよい。

Further, when weight w1> w2, the expression (17) may be used, and if not, the expression (18) may be used.

  In addition, input pixels other than the input pixels appearing in the equations (16), (17), and (18) (for example, pixels in the vicinity of the position (i, j) of the field f (n) or f (n + 1)) are also included. The moving image interpolation pixel ym (m, i ′, j) may be calculated by using this.

  FIG. 12A illustrates the target pixel y (m) when the input field f (n) includes a pixel x (n, i, j) spatially located at the same position as the target pixel y (m, i, j). , I, j) shows an example of a method for generating a still image interpolation pixel ys (m, i, j). Basically, the still image interpolation pixel ys (m, i, j) is a still image interpolation pixel ys (m, i, j) among pixels of one or more input frames near the output frame F (m). ) Based on pixels located in the same spatial position. In the input field f (n), the pixel spatially located at the same position as the target pixel y (m, i, j) is the pixel x (n, i, j), and in the input field f (n + 2), the space In particular, the pixel at the same position as the target pixel y (m, i, j) is x (n + 2, i, j).

  When the time of the output frame F (m) is tout, the time of the input field f (n) is tin1, and the time of the input field f (n + 2) is tin2, the still image interpolation pixel ys (m, i, j) is generated. The weights w1 and w2 in the weighted addition for calculating are calculated by the equation (12) shown above.

  Then, the still image interpolation pixel ys (m, i, j) is calculated by the following equation (19).

但し、静止画補間画素ｙｓ（ｍ、ｉ、ｊ）を下に示す式（２０）又は式（２１）により算出してもよい。

However, the still image interpolation pixel ys (m, i, j) may be calculated by the following equation (20) or equation (21).

また、重みｗ１＞ｗ２の場合に、式（２０）を利用し、そうでない場合に、式（２１）を利用するようにしてもよい。

Further, when weight w1> w2, the expression (20) may be used, and if not, the expression (21) may be used.

  In addition, input pixels other than the input pixels appearing in Expression (19), Expression (20), and Expression (21) (for example, x (n−2, i, j), x (n + 4, i, j)) are also used. The still image interpolation pixel ys (m, i, j) may be calculated.

  FIG. 12B shows the target pixel y when the input field f (n + 1) includes a pixel x (n + 1, i ′, j) spatially identical to the target pixel y (m, i ′, j). An example of a method for generating a still image interpolation pixel ys (m, i ′, j) for (m, i ′, j) is shown. Basically, the still image interpolation pixel ys (m, i ′, j) is a still image interpolation pixel ys (m, i ′) among pixels of one or more input frames near the output frame F (m). , J) is generated on the basis of pixels located at the same spatial position. In the input field f (n−1), the pixel spatially located at the same position as the target pixel y (m, i ′, j) is the pixel x (n−1, i ′, j), and the input field f In (n + 1), the pixel spatially located at the same position as the target pixel y (m, i ′, j) is x (n + 1, i ′, j).

  When the time of the output frame F (m) is tout, the time of the input field f (n−1) is tin1, and the time of the input field f (n + 1) is tin2, the still image interpolation pixel ys (m, i ′, j ) To calculate the weights w1 and w2 in the weighted addition for generating () by the equation (12) shown above.

  Then, the still image interpolation pixel ys (m, i ', j) is calculated by the following equation (22).

但し、静止画補間画素ｙｓ（ｍ、ｉ’、ｊ）を下に示す式（２３）又は式（２４）により算出してもよい。

However, the still image interpolation pixel ys (m, i ′, j) may be calculated by the following equation (23) or equation (24).

また、重みｗ１＞ｗ２の場合に、式（２３）を利用し、そうでない場合に、式（２４）を利用するようにしてもよい。

Further, when the weight w1> w2, the expression (23) may be used, and if not, the expression (24) may be used.

  In addition, input pixels other than the input pixels appearing in Expression (22), Expression (23), and Expression (24) (for example, x (n−3, i, j), x (n + 3, i, j)) are also used. The still image interpolation pixel ys (m, i ′, j) may be calculated.

  In this way, the moving image interpolation pixel ym (m, i, j) and the still image interpolation pixel ys (m, i, j) for the target pixel are calculated. Here, i represents i and i 'as a representative.

  Then, the motion amount M (i, j) for the target pixel is obtained, and the moving image interpolation pixel ym (m, i, j) and the still image interpolation pixel ys (m, i, j) are calculated with weights based on the motion amount. The target pixel y (m, i, j) is generated by weighted addition.

  As a method for calculating the motion amount M (i, j) and a weighted addition method for generating the target pixel y (m, i, j), the same method as in the first embodiment can be used. Hereinafter, these will be described.

  When the input field f (n) includes a pixel x (n, i, j) that is spatially identical to the target pixel y (m, i, j), the difference C (m, i, j) is determined according to equation (25) below.

入力フィールドｆ（ｎ＋１）に対象画素ｙ（ｍ、ｉ’、ｊ）と空間的に同一位置の画素ｘ（ｎ＋１、ｉ’、ｊ）がある場合には、対象画素についての差分Ｃ（ｍ、ｉ’、ｊ）を下に示す式（２６）に従って求める。

If the input field f (n + 1) includes a pixel x (n + 1, i ′, j) spatially identical to the target pixel y (m, i ′, j), the difference C (m, i ′, j) is obtained according to the equation (26) shown below.

差分Ｃ（ｍ，ｉ，ｊ）又はＣ（ｍ，ｉ’，ｊ）は、動き量の基本となる。

The difference C (m, i, j) or C (m, i ′, j) is the basis of the amount of motion.

  Next, the corrected difference D (m, i, j) is obtained by correcting C (m, i, j) according to the equation (27) below. Here, i represents i and i ', and so on.

ここで、Δｃの値は、ノイズのレベル等により決定された値である。

Here, the value of Δc is a value determined by the noise level or the like.

  Next, the accumulated motion amount m (i, j) stored in the memory is updated according to the equation (28) shown below.

ここで、ＴＨの値は、視覚特性等により決定された値である。ＴＨの値は、例えば、１６である。

Here, the value of TH is a value determined by visual characteristics or the like. The value of TH is 16, for example.

  The effect of calculating the cumulative motion amount m (i, j) according to the equation (28) shown above is as described in the first embodiment.

  Next, the motion amount M (i, j) is calculated according to the following equation (29).

式（２９）において、ｍ（ｉ、ｊ）とｍ（ｉ＋１、ｊ）のうちの何れか一方は入力偶数フィールドから検出された累積動き量であり、他方は入力奇数フィールドから検出された累積動き量である。

In Equation (29), one of m (i, j) and m (i + 1, j) is the cumulative motion amount detected from the input even field, and the other is the cumulative motion detected from the input odd field. Amount.

  Next, the target pixel y (m, i, j) is calculated according to the equation (30) shown below.

次に、実施形態３によるフレーム周波数変換装置の構成について図１３を参照して説明する。

Next, the configuration of the frame frequency conversion apparatus according to the third embodiment will be described with reference to FIG.

  Referring to FIG. 13, the image data writing unit 403 inputs the interlaced scanning moving image output from the NTSC tuner or the like, and stores each field in the first to fourth field memories 401-1 to 401-4. Write. Also, the time when each field is written is obtained from the system clock 441 and is also written into the field memory. Further, whether each field is an even field or an odd field is detected from, for example, a synchronization signal of an input moving image, or is input from an NTSC tuner, and information related thereto is also written in the field memory. Even when the number of field memories is increased and the output frame time fluctuates, the input fields f (n−1), f (n), f (n + 1), f (n + 2) are surely stored in any field memory. ) May be stored.

  The vertical synchronization signal acquisition unit 104 acquires the vertical synchronization signal of the progressive moving image output from the graphic card 105, and acquires the time at which the vertical synchronization signal was acquired from the system clock 103 as the time of the output frame. Note that the output frame time may be offset so that an input frame with a time later than the output frame time can be used. That is, for example, by setting the time of the output frame to 45 milliseconds past the time obtained from the system clock, the input frame with a time later than the time of the output frame may be used. .

  The output frame time input unit 432 acquires the time of each output frame F (m) from the vertical synchronization signal acquisition unit 104.

  The input field time reference unit 435 receives the times of the fields f (n−1), f (n), f (n + 1), and f (n + 2) from the first to fourth field memories 401-1 to 401-4. To get.

  The even / odd field reference unit 437 includes fields f (n−1), f (n), f (n + 1), and f (n + 2) from the first to fourth field memories 401-1 to 401-4 as even fields. Get information about whether the field is an odd field.

  The weight calculation unit 439 calculates the time of the output field F (m), the time of each of the fields f (n−1), f (n), f (n + 1), and f (n + 2) and the fields f (n−1), f (N), f (n + 1) and f (n + 2) for each pixel y (m, i, j) of the output frame F (m) based on information about whether it is an even field or an odd field, respectively. The weights for generating the moving image interpolation pixel ym (m, i, j) and the still image interpolation pixel ys (m, i, j) are calculated.

  That is, for example, whether the moving image interpolation pixel ym (m, i, j) is calculated based on the equation (13) or the equation (16) is the input field f (n) is an even field. Or an odd field, and information on whether the line number i of the target pixel y (m, i, j) is an even number or an odd number, and further, weighted addition in each case Is calculated by substituting the time of the output field and the times of the fields f (n) and f (n + 1) into the equation (12). As described above, the expression (14) or the expression (15) is used fixedly or adaptively instead of the expression (13), or the expression (17) or the expression (18) is used instead of the expression (16). May be calculated in a fixed or adaptive manner.

  Further, for example, whether the still image interpolation pixel ys (m, i, j) is calculated based on the equation (19) or the equation (22) is an even field. It is determined on the basis of information on whether there is an odd field or an odd field and information on whether the line number i of the target pixel y (m, i, j) is even or odd, and the weights in each case The weight in addition is calculated by substituting the time of the output field, the time of the fields f (n) and f (n + 2) or the time of the output field, and the fields f (n−1) and f (n + 1) into the equation (12). To do. As described above, the expression (20) or the expression (21) is used fixedly or adaptively instead of the expression (19), or the expression (23) or the expression (24) is used instead of the expression (22). May be calculated in a fixed or adaptive manner.

  The moving image interpolation pixel generation unit 405 inputs the weight necessary for generating the moving image interpolation pixel ym (m, i, j) for each target pixel y (m, i, j) from the weight calculation unit 329, and Based on the weight, necessary pixels are read from the second to third field memories 401-2 to 401-3, and the still image interpolation pixel ym (m, i, j) is determined based on the input weight and the read pixel. Generate.

  The still image interpolation pixel generation unit 407 inputs a weight necessary for generating the still image interpolation pixel ys (m, i, j) for each target pixel y (m, i, j) from the weight calculation unit 329. The necessary pixels are read from the first to fourth field memories 401-1 to 401-4 based on the weights, and the still image interpolation pixel ys (m, i, j is based on the input weights and the read pixels. ) Is generated.

  The difference calculation unit 409 relates to information on whether the field f (n) is an even field or an odd field and whether the line number i of the target pixel y (m, i, j) is an even number or an odd number. Based on the information, it is determined which pixel in which field should be calculated, and based on the result, necessary pixels are determined in the first field memory 401-1 and the third field memory 401-. 3 or the second field memory 401-2 and the fourth field memory 401-4, and based on the read pixel, the difference C (m, i, j) based on the equation (25) or the equation (26) Is calculated.

  The difference correction unit 411 calculates the corrected difference D (m, i, j) by correcting C according to the equation (27).

  Here, the value of Δc is determined by the noise level or the like and stored in the correction value holding unit 413.

  The difference accumulating unit 415 updates the accumulated motion amount m (i, j) stored in the accumulated motion amount memory 417 according to the equation (28).

  Here, the value of TH is determined by visual characteristics and the like, and is stored in the threshold value holding unit 419. The value of TH is 16, for example.

  The motion amount calculation unit 421 calculates the motion amount M (i, j) according to the equation (29).

  The synthetic interpolation pixel generation unit 423 calculates a synthetic interpolation pixel y (m, i, j) according to the equation (30). The combined interpolation pixel y (m, i, j) is written into the display memory 105-1.

  The frame frequency conversion device whose configuration is shown in FIG. 13 can be realized by hardware, but the computer uses a recording medium to store a program for causing the computer to function as the frame frequency conversion device whose configuration is shown in FIG. It can also be realized by reading and executing.

[Embodiment 4]
In the first to third embodiments, it is assumed that the graphic card outputs a vertical synchronization signal to the outside, and the vertical synchronization signal acquisition unit 104 acquires the vertical synchronization signal. However, there are cases where the vertical synchronizing signal of the output moving image cannot be obtained from a graphic card or the like. However, even in such a case, the current scan line value (scan line number) and refresh rate (frame frequency) of the output moving image can often be obtained from a graphic card or the like.

  The fourth embodiment corresponds to the case where the vertical synchronization signal of the output moving image cannot be obtained from the graphic card or the like, but the current scan line value and the refresh rate of the output moving image can be obtained from the graphic card or the like. That is, in Embodiments 1 to 3, one output frame is generated every time the vertical synchronization signal of each output frame is acquired. In Embodiment 4, a vertical synchronization signal of each output frame is generated. Is estimated based on the scan line value and the refresh rate, and one output frame is generated at the time when a new vertical synchronization signal is predicted to be generated.

  Next, this method will be described. Referring to FIG. 14, in steps S501 to S507 on the left side, an estimated value of vertical resolution is calculated. Here, the vertical resolution is the number of scan lines per frame (hereinafter simply referred to as “the number of scan lines”).

  First, the scan line number MaxScanLine is initialized to zero (step S501). Next, the current scan line value A is obtained by sampling from a graphic card or the like (step S503). Next, it is determined whether or not the scan line value A acquired in step S503 is larger than the currently held scan line number MaxScanLine. If so, the scan line value A is substituted into the scan line number MaxScanLine ( Step S505). Next, it is determined whether or not a predetermined time has elapsed from the time when step S501 is executed. If so, the process is terminated, and if not, the process returns to step S503 (step S507). By continuing the loop of steps S503 to S507 for a predetermined time, the scan line number MaxScanLine becomes an estimated value of the actual scan line number.

  If the loop from step S503 to S507 is exited, the current scan line value A is obtained by sampling from a graphic card or the like (step S511). Next, by multiplying the reciprocal of the refresh rate (= frame period) by (current scan line value A / scan line number MaxScanLine), an elapsed time from the last vertical synchronization signal generation time to the current time is estimated ( Step S513). Next, the time of occurrence of the last vertical synchronization signal is estimated by subtracting the elapsed time estimated in step S513 from the current system time acquired from the system clock (step S515). Next, it is determined whether or not the vertical synchronization signal whose generation time is estimated in step S515 of the current loop is the next vertical synchronization signal of the vertical synchronization signal whose generation time is estimated in step S515 of the previous loop. (Step S517). For this purpose, for example, whether the occurrence time estimated in step S515 of the current loop is later than the time obtained by adding half the time of the frame period to the occurrence time estimated in step S515 of the previous loop. to decide. By doing so, even if the estimated scan line number MaxScanLine includes an error with respect to the actual scan line number, the determination in step S517 can be made accurate. For reference, when it is determined whether or not the occurrence time estimated in step S515 of the current loop matches the occurrence time estimated in step S515 of the previous loop, the determination in step S517 is accurate. I can't make it.

  If so (YES in step S517), an output frame is generated (step S519), and then the process returns to step S511.

  Otherwise (NO in step S517), the process immediately returns to step S511.

  By doing so, it is possible to generate an output frame in synchronization with the vertical synchronization signal of the output moving image.

[Embodiment 5]
In the first to fourth embodiments, it is assumed that the frame frequency of the television monitor 107 cannot be adjusted from the outside. However, the fifth embodiment corresponds to a case where the frame frequency of the television monitor 107 can be adjusted from the outside. It is a thing.

  Referring to FIG. 15, the IP conversion processing unit 101 is the same as the IP conversion processing unit 101 of the first embodiment.

  The frame buffer 601 can store a plurality of frames output from the IP conversion processing unit 101, and has, for example, a ring buffer configuration. Each frame output from the IP conversion processing unit 101 is temporarily written in the frame buffer 601 in synchronization with the vertical synchronization signal of the moving image output from the IP conversion processing unit 101 and synchronized with the vertical synchronization signal from the graphic card 105. Are read out and written to the display memory 105-1.

  The frequency adjustment unit 603 calculates the number of frames stored in the frame buffer 601 based on the difference between the write address and the read address of the ring buffer, and the number of frames is greater than a predetermined threshold value TH1. In this case, a signal for increasing the frame frequency of the television monitor 107 is output to the television monitor 107, and when the number of frames is smaller than a predetermined threshold value TH2, the frame frequency of the television monitor 107 is decreased. The signal is output to the television monitor 107. By doing so, overflow and underflow of the frame buffer 601 can be avoided, and as a result, the frame frequency of the television monitor 107 can be matched with the frame frequency of the moving image output from the IP conversion unit 101.

It is a block diagram which shows the structure of the frame frequency converter by Embodiment 1 of this invention. It is a block diagram which shows the example of a structure of the IP conversion process part shown in FIG. It is a 1st figure for demonstrating the effect of the motion amount detection method which the IP conversion process part by embodiment of this invention performs. It is a 2nd figure for demonstrating the effect of the motion amount detection method which the IP conversion process part by embodiment of this invention performs. FIG. 2 is a flowchart showing an IP conversion processing method performed by an IP conversion processing unit whose configuration is shown in FIG. It is a block diagram which shows the example of a structure of the interpolation process part shown in FIG. FIG. 6 is a flowchart showing an interpolation method performed by an interpolation processing unit whose configuration is shown in FIG. It is a block diagram which shows the structure of the frame frequency converter by Embodiment 2 of this invention. It is a block diagram which shows the example of a structure of the interpolation process part shown in FIG. FIG. 9 is a flowchart showing an interpolation method performed by an interpolation processing unit whose configuration is shown in FIG. It is the 1st figure for demonstrating the principle of Embodiment 3 of this invention, and is for demonstrating the method to produce | generate a moving image interpolation pixel. It is the 1st figure for demonstrating the principle of Embodiment 3 of this invention, and is for demonstrating the method to produce | generate a still image interpolation pixel. It is a block diagram which shows the structure of the frame frequency converter by Embodiment 3 of this invention. 6 is a flowchart illustrating a frame frequency conversion method including a vertical synchronization signal estimation method according to Embodiment 4 of the present invention. It is a block diagram which shows the structure of the interlace / progressive conversion apparatus which has a television monitor frame frequency adjustment function by Embodiment 5 of this invention.

Explanation of symbols

101 IP conversion processing unit 102 Frame buffer 103 System clock 104 Vertical synchronization signal acquisition unit 105 Graphic card 106, 106B Interpolation processing unit 107 Television monitor 108 Motion amount buffer

Claims (25)

In a frame frequency conversion method for generating an output moving image of progressive scanning having a frame frequency different from the input moving image based on an input moving image of interlace scanning,
Video interpolation pixels for each pixel (hereinafter referred to as “target pixel”) of each output frame (hereinafter referred to as “target output frame”) to be included in the output moving image are included in at least the input moving image. A moving image interpolation pixel generating step for generating by a first weighted addition based on at least one pixel of one input field;
A still image interpolation pixel generating step for generating a still image interpolation pixel for the target pixel by a second weighted addition based on at least one pixel of at least one input field included in the input moving image;
A motion amount generating step for generating a motion amount for the target pixel based on the input moving image;
A target pixel generation step of generating the target pixel by a third weighted addition using a weight according to the amount of motion based on the moving image interpolation pixel and the still image interpolation pixel;
A frame frequency conversion method comprising: The frame frequency conversion method according to claim 1,
In the moving image interpolation pixel generation step, the pixel of the input field (hereinafter referred to as “first input field”) immediately before the target output frame among the plurality of input fields included in the input moving image. Of the pixels in the next field of the first field (hereinafter referred to as “second input field”) at least one pixel that is spatially closest to the target pixel, spatially closest to the target pixel A frame frequency conversion method, characterized in that the moving image interpolation pixel is generated based at least on at least one pixel in the position or both. The frame frequency conversion method according to claim 1,
In the still image interpolation pixel generation step, the target pixel is in the same spatial position as a pixel of a plurality of input fields (hereinafter referred to as “preceding input field group”) preceding the target output frame. A pixel that is closest in time and a spatially identical position to the target pixel among pixels of a plurality of input fields after the preceding input field group (hereinafter referred to as “successive input field group”). A frame frequency conversion method, characterized in that the still image interpolation pixel is generated based on at least a pixel located at a position closest in time or both. The frame frequency conversion method according to claim 1,
The movement amount generation step includes:
A first motion amount calculating step for calculating a first motion amount for the target pixel from a plurality of odd fields included in the input moving image;
A second motion amount calculating step of calculating a second motion amount for the target pixel from a plurality of even fields included in the input moving image;
A final motion amount calculating step for calculating the motion amount based on the first motion amount and the second motion amount;
A frame frequency conversion method comprising: The frame frequency conversion method according to claim 4, wherein
In the first motion amount calculation step, pixels of an odd input field (hereinafter referred to as “first odd input field”) immediately before the target output frame among a plurality of odd fields included in the input moving image. Among the target pixels of at least one pixel that is spatially closest to the target pixel and pixels in an odd field next to the first odd field (hereinafter referred to as “second odd field”). Calculating the first amount of motion based on at least one pixel located spatially closest;
In the second motion amount calculating step, pixels of an even input field (hereinafter referred to as “first even input field”) immediately before the target output frame among a plurality of even fields included in the input moving image. Among the target pixels of at least one pixel that is spatially closest to the target pixel and pixels in the even field next to the first even field (hereinafter referred to as “second even field”). A frame frequency conversion method, wherein the second motion amount is calculated on the basis of at least one pixel located at a spatially closest position. The frame frequency conversion method according to claim 1,
A frame frequency conversion method, further comprising: estimating a timing of a vertical synchronization signal corresponding to the target output frame based on an output frame frequency and a scanning line number output from the graphic unit and sequentially sampled. In a frame frequency conversion method for generating an output moving image of progressive scanning having a frame frequency different from the input moving image based on the input moving image of progressive scanning,
Among the plurality of input frames included in the input moving image, an input frame (hereinafter referred to as “first input frame”) immediately before each output frame (hereinafter referred to as “target output frame”) to be included in the output moving image. ”) To the target output frame and a period from the target output frame to the next input frame of the first input frame (hereinafter referred to as“ second input frame ”). A weight calculating step of calculating a weight of the input frame and a weight of the second input frame;
Each pixel of the target output frame (hereinafter referred to as “target pixel”) is defined as at least one pixel of the first input frame and at least one pixel of the second input frame. A target pixel generation step for generating by weighted addition based on a weight and a weight of the second input frame;
A frame frequency conversion method comprising: The frame frequency conversion method according to claim 7,
In the target pixel generation step, spatially the same as the subsequent pixel of the first input frame of the pixel of the second input frame (hereinafter referred to as “subsequent pixel”) used to generate the target pixel. When the amount of motion with respect to the pixel at the position (hereinafter referred to as “preceding pixel”) is less than a predetermined value, the weighted addition is not performed, the preceding pixel is used, and the subsequent pixel is not used. A frame frequency conversion method characterized by generating a pixel. The frame frequency conversion method according to claim 7,
A frame frequency conversion method, further comprising: estimating a timing of a vertical synchronization signal corresponding to the target output frame based on an output frame frequency and a scanning line number output from the graphic unit and sequentially sampled. The frame frequency conversion method according to claim 7,
An interlace / progressive conversion step of generating an input moving image for progressive scanning based on a moving image for interlace scanning;
The interlace / progressive conversion step includes:
A first motion amount calculating step of calculating a first motion amount for each pixel of each frame of the progressive scan input video from a plurality of odd fields included in the interlace scan video;
A second motion amount calculating step of calculating a second motion amount for each pixel of each frame of the progressive scan input video from a plurality of even fields included in the interlace scan video;
A final motion amount calculating step of calculating a motion amount for each pixel of each frame of the progressive moving image based on the first motion amount and the second motion amount;
Each pixel of each frame of the progressive scan input moving image is calculated in the final motion amount calculation step based on the moving image interpolation pixel and the still image interpolation pixel obtained based on the interlaced scanning moving image. An intermediate pixel generating step for generating by weighted addition using a weight corresponding to the amount of motion;
A frame frequency conversion method comprising: In a method for controlling the frame frequency of a television monitor,
A writing step of sequentially writing a plurality of frames to be output to the television monitor in a frame buffer in accordance with a frame frequency of an output unit of the plurality of frames;
A step of sequentially reading out the plurality of frames from the frame buffer according to a frame frequency of the television monitor;
A control step of controlling the frame frequency of the television monitor based on the number of frames stored in the frame buffer;
A method comprising the steps of: The method of claim 11, wherein
The frame output unit further includes an interlace / progressive conversion step of generating a progressive scan moving image based on the interlace scan moving image,
The interlace / progressive conversion step includes:
A first motion amount calculating step of calculating a first motion amount for each pixel of each frame of the progressive scan moving image from a plurality of odd fields included in the interlaced scan moving image;
A second motion amount calculating step of calculating a second motion amount for each pixel of each frame of the progressive scan moving image from a plurality of even fields included in the interlaced scan moving image;
A final motion amount calculating step for calculating a motion amount for each pixel of each frame of the progressive scan moving image based on the first motion amount and the second motion amount;
Each pixel of each frame of the progressive scan moving image was calculated in the final motion amount calculation step based on the moving image interpolation pixel and still image interpolation pixel obtained based on the interlaced moving image. A pixel generation step of generating by weighted addition using a weight according to the amount of motion;
A method comprising the steps of: In a frame frequency conversion device for generating an output moving image of progressive scanning having a frame frequency different from that of the input moving image based on an input moving image of interlace scanning,
Video interpolation pixels for each pixel (hereinafter referred to as “target pixel”) of each output frame (hereinafter referred to as “target output frame”) to be included in the output moving image are included in at least the input moving image. A moving image interpolation pixel generating means for generating the first weighted addition based on at least one pixel of one input field;
Still image interpolation pixel generation means for generating a still image interpolation pixel for the target pixel by second weighted addition based on at least one pixel of at least one input field included in the input moving image;
A motion amount generating means for generating a motion amount for the target pixel based on the input moving image;
Target pixel generation means for generating the target pixel by a third weighted addition using a weight according to the amount of motion based on the moving image interpolation pixel and the still image interpolation pixel;
A frame frequency conversion device comprising: The frame frequency conversion device according to claim 13, wherein
In the moving image interpolation pixel generation means, the pixel of the input field (hereinafter referred to as “first input field”) immediately before the target output frame among the plurality of input fields included in the input moving image. Of the pixels in the next field of the first field (hereinafter referred to as “second input field”) at least one pixel that is spatially closest to the target pixel, spatially closest to the target pixel A frame frequency conversion device, characterized in that the moving image interpolation pixel is generated based on at least one pixel in the position or both. The frame frequency conversion device according to claim 13, wherein
In the still image interpolation pixel generation means, the target pixel is in the same spatial position as the target pixel among the pixels of a plurality of input fields (hereinafter referred to as “preceding input field group”) preceding the target output frame. A pixel that is closest in time and a spatially identical position to the target pixel among pixels of a plurality of input fields after the preceding input field group (hereinafter referred to as “successive input field group”). A frame frequency conversion device, characterized in that the still image interpolation pixel is generated based on at least a pixel located at a position closest in time or both. The frame frequency conversion device according to claim 13, wherein
The movement amount generating means includes
First motion amount calculating means for calculating a first motion amount for the target pixel from a plurality of odd fields included in the input moving image;
Second motion amount calculating means for calculating a second motion amount for the target pixel from a plurality of even fields included in the input moving image;
Final motion amount calculating means for calculating the motion amount based on the first motion amount and the second motion amount;
A frame frequency conversion device comprising: The frame frequency conversion device according to claim 16,
In the first motion amount calculating means, pixels of an odd input field (hereinafter referred to as “first odd input field”) immediately before the target output frame among a plurality of odd fields included in the input moving image. Among the target pixels of at least one pixel that is spatially closest to the target pixel and pixels in an odd field next to the first odd field (hereinafter referred to as “second odd field”). Calculating the first amount of motion based on at least one pixel located spatially closest;
In the second motion amount calculation means, pixels of an even input field (hereinafter referred to as “first even input field”) immediately before the target output frame among a plurality of even fields included in the input moving image. Among the target pixels of at least one pixel that is spatially closest to the target pixel and pixels in the even field next to the first even field (hereinafter referred to as “second even field”). A frame frequency conversion device, wherein the second motion amount is calculated on the basis of at least one pixel at a spatially closest position. The frame frequency conversion device according to claim 13, wherein
A frame frequency conversion apparatus, further comprising means for estimating a timing of a vertical synchronization signal corresponding to the target output frame based on an output frame frequency and a scanning line number output from the graphic unit and sequentially sampled. In a frame frequency conversion device for generating a progressive scanning output moving image having a frame frequency different from that of the input moving image based on the input moving image of progressive scanning,
Among the plurality of input frames included in the input moving image, an input frame (hereinafter referred to as “first input frame”) immediately before each output frame (hereinafter referred to as “target output frame”) to be included in the output moving image. ”) To the target output frame and a period from the target output frame to the next input frame of the first input frame (hereinafter referred to as“ second input frame ”). Weight calculating means for calculating the weight of the input frame and the weight of the second input frame;
Each pixel of the target output frame (hereinafter referred to as “target pixel”) is defined as at least one pixel of the first input frame and at least one pixel of the second input frame. Target pixel generation means for generating by weighted addition based on a weight and a weight of the second input frame;
A frame frequency conversion device comprising: The frame frequency conversion device according to claim 19,
The target pixel generation means is spatially the same as the subsequent pixel of the first input frame of the pixel of the second input frame (hereinafter referred to as “subsequent pixel”) used for generating the target pixel. When the amount of motion with respect to the pixel at the position (hereinafter referred to as “preceding pixel”) is less than a predetermined value, the weighted addition is not performed, the preceding pixel is used, and the subsequent pixel is not used. A frame frequency conversion device for generating a pixel. The frame frequency conversion device according to claim 19,
A frame frequency conversion apparatus, further comprising means for estimating a timing of a vertical synchronization signal corresponding to the target output frame based on an output frame frequency and a scanning line number output from the graphic unit and sequentially sampled. The frame frequency conversion device according to claim 19,
Further comprising interlace / progressive conversion means for generating the input moving image of the progressive scanning based on the moving image of the interlace scanning,
The interlace / progressive conversion means includes:
First motion amount calculation means for calculating a first motion amount for each pixel of each frame of the progressive scan input moving image from a plurality of odd fields included in the interlaced scan moving image;
Second motion amount calculating means for calculating a second motion amount for each pixel of each frame of the progressive scan input video from a plurality of even fields included in the interlace scan video;
A final motion amount calculating means for calculating a motion amount for each pixel of each frame of the input moving image of the progressive scan based on the first motion amount and the second motion amount;
Each pixel of each frame of the progressive scan input moving image is calculated by the final motion amount calculation means based on the moving image interpolation pixel and the still image interpolation pixel obtained based on the interlaced scanning moving image. Intermediate pixel generating means for generating by weighted addition using a weight according to the amount of motion;
A frame frequency conversion device comprising: In a device for controlling the frame frequency of a television monitor,
Writing means for sequentially writing a plurality of frames to be output to the television monitor in a frame buffer in accordance with a frame frequency of an output unit of the plurality of frames;
Read means for sequentially reading the plurality of frames from the frame buffer in accordance with the frame frequency of the television monitor;
Control means for controlling the frame frequency of the television monitor based on the number of frames stored in the frame buffer;
A device comprising: 24. The apparatus of claim 23.
The frame output unit further includes an interlace / progressive conversion unit that generates a progressive scan moving image based on the interlaced scan moving image,
The interlace / progressive conversion means includes:
First motion amount calculating means for calculating a first motion amount for each pixel of each frame of the progressive scan moving image from a plurality of odd fields included in the interlaced scan moving image;
Second motion amount calculating means for calculating a second motion amount for each pixel of each frame of the progressive scan moving image from a plurality of even fields included in the interlaced scan moving image;
A final motion amount calculating means for calculating a motion amount for each pixel of each frame of the progressive scan moving image based on the first motion amount and the second motion amount;
Each pixel of each frame of the progressive scan moving image was calculated by the final motion amount calculation means based on the moving image interpolation pixel and the still image interpolation pixel obtained based on the interlaced scan moving image. Pixel generation means for generating by weighted addition using a weight according to the amount of motion;
A device comprising:   A program for causing a computer to perform the method according to any one of claims 1 to 12.

Images