JP3721941B2 - Scanning line interpolation device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention according to the scanning line interpolation apparatus, more particularly to a scanning line interpolation apparatus for converting into progressive scanning image signal interlaced image signal.
[0002]
[Prior art]
FIG. 4 is a diagram showing a scanning line structure. The scanning line structure shown in FIG. 4A is an interlace (interlaced scanning) signal used as a standard television image signal such as NTSC or HDTV, and one frame has a time (t) and a vertical direction (V ), And when there are many high frequency components in the vertical direction of the image, interlace interference such as line flickering occurs.
[0003]
On the other hand, the scanning line structure shown in FIG. 4B is a progressive scanning signal, which is also referred to as a progressive signal or a non-interlace signal, and there is no vertical shift, so there is no interlace interference. The scanning line structure shown in FIG. 4C is a signal obtained by performing scanning line interpolation, and is created by interpolating the pixels of the scanning line x of the portion thinned out by interlacing with the pixels of the peripheral scanning line ○, A process for removing interlace interference is performed by converting to a progressive scanning signal. Such a processing method is called progressive scanning conversion or double-density conversion.
[0004]
FIG. 5 is a block diagram showing a conventional scanning line interpolation apparatus. This is a conventional example of a progressive scan conversion using the interpolation method according to motion compensation which is described in JP-A-6-133280. In FIG. 5, the interlaced image signal input from the input terminal 1 is supplied to the motion vector detection circuit 2, the motion compensation interpolation circuit 3, the interpolation selection determination circuit 6, the intra-field interpolation circuit 7 and the delay compensation circuit 9. The motion vector detection circuit 2 detects the motion vector V of the input image for each block of the image and supplies it to the motion compensation interpolation circuit 3 and the interpolation selection determination circuit 6.
[0005]
The motion vector V includes two components, a horizontal motion component Vx and a vertical motion component Vy. The motion compensation interpolation circuit 3 performs motion compensation on the input signal using the average value of the pixels in the preceding and following fields corresponding to the motion vector V (Vx, Vy) as a new pixel, and obtains a motion compensated output signal. To the mixing circuit 8. The interpolation selection determination circuit 6 determines for each pixel whether or not the motion vector V is accurate, determines a mixing coefficient Km, and supplies the mixing coefficient Km to the mixing circuit 8.
[0006]
The intra-field interpolation circuit 7 performs intra-field interpolation using the average value of the upper and lower scanning lines as a new scanning line, obtains an interpolated output signal, and supplies it to the mixing circuit 8. The mixing circuit 8 mixes the output signal of the motion compensation interpolation circuit 3 and the output signal of the intra-field interpolation circuit 7 according to the mixing coefficient Km, obtains an interpolation signal, and supplies it to the double speed conversion circuit 10. The delay compensation circuit 9 delay-compensates the input signal for the time required for the interpolation signal generation process, obtains the current line signal, and supplies it to the double speed conversion circuit 10. The double speed conversion circuit 10 alternately reads the current line signal and the interpolation signal at a speed twice that of the input signal, sequentially obtains the scanning signal, and outputs it from the output terminal 11.
[0007]
FIG. 6 is a diagram for explaining the operation of the conventional example. In the case of the progressive scan conversion shown in FIG. 5, in the case of an image moving in the vertical direction with Vx = 0 (pixel / field) and Vy = 2 (scan line / field), as shown in FIG. In addition, the average value of the pixels C ′ and D ′ in the preceding and following fields corresponding to the detected motion vector V (0, 2) can be set as the new pixel Q. The progressive scan conversion operation described above will be described in the time-vertical spatio-temporal spectrum region.
[0008]
FIG. 6B shows an image in which a subject having a ν component of 0 to fsv / 2 moves in the vertical direction with Vx = 0 and Vy = 2 as shown in FIG. 6A. The original spectrum of the image extends linearly from the origin in the upper left direction, and the aliasing spectrum of the sampling in the time direction by the field also extends in the upper left direction from the point (f, v) = (fst, 0). X existing in (f, ν) = (± fst / 2, fsv / 2) is a carrier of space-time sampling by interlace scanning, and the aliasing component thereby extends from each x in the lower right direction. . The spectrum of aliasing due to interlace is indicated by a broken line in the figure, and this is a component that causes interlace interference.
[0009]
FIG. 6C shows a frequency response when motion compensation is performed using the detected motion vector V (0, 2). A region indicated by diagonal lines in the figure is a blocking region when interpolation is viewed as a filter. Therefore, the spectrum of the region not shown by the diagonal lines becomes the spectrum of the progressive scan image signal after the progressive scan conversion, the original image spectrum of the solid line is not damaged at all, and the folded spectrum of the broken line is completely removed. That is, if motion compensation interpolation is performed, the original image component is reproduced at full resolution, and interlace interference is completely removed. FIG. 6 shows an image of Vx = 0 and Vy = 2, but the same holds true if Vx = n and Vy = 2m (n and m are integers).
[0010]
FIG. 6D shows a frequency response when intra-field interpolation is performed. In the intra-field interpolation, the original spectrum and the folded spectrum cannot be separated, the high band of the original spectrum is cut, and the high band of the folded spectrum remains in the time high band. That is, with intra-field interpolation, the resolution of the image is degraded and line flicker is not removed.
[0011]
[Problems to be solved by the invention]
FIG. 7 is a diagram for explaining the operation of the conventional example. In motion compensated interpolation, the operation differs depending on whether Vy (number of scanning lines / field) is even or odd. For example, when Vx = 0 and Vy = 1, as shown in FIG. 7A, there are no pixels in the preceding and following fields corresponding to Vx = 0 and Vy = 1. This is not limited to Vx = 0 and Vy = 1, but is valid if Vx = n and Vy = 2m + 1 (n and m are integers). As described above, since motion compensation cannot be performed on an image in which Vy (number of scanning lines / field) shows an odd number of motions, a motion vector detection circuit is not limited so that only motion vectors having an even number of Vy are detected. Don't be.
[0012]
However, restricting Vy to an even number in motion vector detection does not verify all the possibilities, leading to a decrease in motion vector detection accuracy. When Vy = 1, even if motion compensation is performed using the average value of pixels C and C ′ in the previous field as a virtual pixel corresponding to Vy = 1, when Vy = 1, C = A, C ′ = Since B, interpolation equivalent to intra-field interpolation is performed.
[0013]
This can also be explained in the time-vertical spatiotemporal spectral region. FIG. 7B shows an image in which a subject having a ν component of 0 to fsv / 2 moves in the vertical direction with Vy = 1. In the case of Vy = 1, the original spectrum and the folded spectrum completely overlap on the f−ν region. FIG. 7C shows the frequency response of motion compensation interpolation, and FIG. 7D shows the frequency response of intra-field interpolation. In any of the drawings, the original spectrum and the folded spectrum cannot be separated, and the high frequency of the original spectrum is cut, so that the resolution of the image cannot be prevented even by motion compensation interpolation. Note that the high frequency region of the folded spectrum remains, but the line flicker is inconspicuous because it is a low frequency region.
[0014]
As described above, in an image in which Vy (number of scanning lines / field) shows an odd number of motions, motion compensation interpolation cannot be performed or even if motion compensation interpolation is performed, it is equivalent to intra-field interpolation, There is a problem that only a deteriorated progressive scanning signal can be obtained.
The present invention has been made in order to solve the above-described problem, and it is not necessary to provide a special restriction on motion vector detection, and in principle it is difficult to obtain an interpolation signal in the vertical direction (scanning component). An object of the present invention is to provide a scanning line interpolating apparatus capable of obtaining a high-resolution progressive scanning signal even for an image having an odd number of lines / field.
[0015]
[Means for Solving the Problems]
To achieve the above object, the scanning line interpolation apparatus for converting a scanning line is not in the input image signal to interlaced input image signal to the interpolation and sequentially scanned image signal said input image signal, the input image signal A motion vector detection circuit for detecting a motion vector of the input image signal, a motion compensation interpolation circuit for performing motion compensation on the input image signal according to the motion vector, and intra-field interpolation for performing intra-field interpolation on the input image signal circuit and the motion vector to determine accurately whether the motion vector detected for each pixel by the detection circuit, and the interpolation selection determination circuit for outputting a mixing coefficient, the input image signal is time - vertical - horizontal 3 A spatio-temporal coordinate determination circuit for determining where in the three-dimensional spatio-temporal frequency domain, and a determination result by the spatio-temporal coordinate determination circuit And a shift amount generation circuit for generating a shift amount in both the horizontal and vertical directions based on the spectrum, and the input image signal when the vertical motion component (number of scanning lines / field) of the motion vector is an odd number. A three-dimensional interpolation circuit that performs a three-dimensional interpolation process based on both the horizontal and vertical shift amounts, an output signal of the motion compensation interpolation circuit, and an output signal of the three-dimensional interpolation circuit, and the motion vector of the case vertical movement component is odd selects the output signal of the 3-dimensional interpolator, selecting an output signal of said motion-compensated interpolation circuit when the vertical movement component is even of the motion vector a selection circuit for and outputting a mixing circuit mixing combined and output in accordance with an output signal to the mixing coefficient of the output signal and the field interpolation circuit of the selection circuit, the input image signal and the previous Mixing circuit output signal and the read out alternately at twice the rate of each signal, provide a scanning line interpolation apparatus characterized by being configured and a double-speed conversion circuit for converting the sequential scanning signals To do.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a block diagram showing an embodiment of the present invention. In FIG. 1, an interlaced image signal input from an input terminal 1 includes a motion vector detection circuit 2, a motion compensation interpolation circuit 3, a three-dimensional interpolation circuit 4, an interpolation selection determination circuit 6, an intrafield interpolation circuit 7, and a delay compensation circuit 9. Has been supplied to. The motion vector detection circuit 2 detects a motion vector V (Vx, Vy) of the input image for each block of the image and supplies it to the motion compensation interpolation circuit 3 and the interpolation selection determination circuit 6.
[0017]
The motion vector V includes two components, a horizontal motion component Vx and a vertical motion component Vy. The motion compensation interpolation circuit 3 performs motion compensation on the input signal using the average value of the pixels in the preceding and following fields corresponding to the motion vector V (Vx, Vy) as a new pixel, and obtains a motion compensated output signal. To the selection circuit 5. The three-dimensional interpolation circuit 4 performs interpolation by a method described later when the vertical motion component Vy (number of scanning lines / field) of the motion vector V is an odd number, and supplies the output signal to the selection circuit 5. . The selection circuit 5 selects the output signal of the motion compensation interpolation circuit 3 when the vertical motion component Vy (number of scanning lines / field) of the motion vector V is an even number, and the vertical motion component Vy of the motion vector V When (the number of scanning lines / field) is an odd number, the output signal of the three-dimensional interpolation circuit 4 is selected and supplied to the mixing circuit 8. The interpolation selection determination circuit 6 determines for each pixel whether or not the motion vector V is accurate, determines a mixing coefficient Km, and supplies the mixing coefficient Km to the mixing circuit 8.
[0018]
The intra-field interpolation circuit 7 performs intra-field interpolation using the average value of the upper and lower scanning lines as a new scanning line, obtains an interpolated output signal, and supplies it to the mixing circuit 8. The mixing circuit 8 mixes the output signal of the selection circuit 5 and the output signal of the intra-field interpolation circuit 7 according to the mixing coefficient Km, obtains an interpolation signal, and supplies it to the double speed conversion circuit 10. The delay compensation circuit 9 delay-compensates the input signal for the time required for the interpolation signal generation process, obtains the current line signal, and supplies it to the double speed conversion circuit 10. The double speed conversion circuit 10 alternately reads the current line signal and the interpolation signal at a speed twice that of the input signal, sequentially obtains the scanning signal, and outputs it from the output terminal 11.
[0019]
FIG. 2 is a block diagram showing an embodiment of the three-dimensional interpolation circuit 4 in FIG. In FIG. 2, the interlaced image signal input from the input terminal 21 is supplied to the field delay circuit 22, the vertical HPF 23, the interframe difference detection circuit 24, the vertical gradient detection circuit 25, and the horizontal gradient detection circuit 26. The field delay circuit 22 obtains a signal obtained by delaying the input signal by a time corresponding to one field and supplies it to the field delay circuit 27 and the vertical LPF 28. The field delay circuit 27 further obtains a signal obtained by delaying the input signal by a time corresponding to one field and supplies it to the vertical HPF 29. Therefore, different signals are supplied to the vertical HPF 23, the vertical LPF 28, and the vertical HPF 29 by one field.
[0020]
The vertical LPF 28 is a low-pass filter in the vertical direction, extracts the low-frequency signal of the current field that is the field to be interpolated, and supplies it to the adder circuit 36. Similarly, the vertical HPF 23 is a high-pass filter in the vertical direction. The vertical HPF 23 extracts a high-frequency signal of a field subsequent to the current field, which is an interpolated field, and supplies it to the image shift circuit 30. Similarly, the vertical HPF 29 takes out the high-frequency signal of the previous field with respect to the current field which is the field to be interpolated, and supplies it to the image shift circuit 31.
[0021]
The inter-frame difference detection circuit 24 detects an inter-frame difference corresponding to the motion vector V supplied from the input terminal 32 and supplies it to the spatio-temporal coordinate determination circuit 33. The vertical gradient detection circuit 25 detects the degree of signal change in the vertical direction and supplies it to the spatio-temporal coordinate determination circuit 33. The horizontal gradient detection circuit 26 detects the degree of signal change in the horizontal direction and supplies it to the spatio-temporal coordinate determination circuit 33. The spatio-temporal coordinate determination circuit 33 compares the output signals of the inter-frame difference detection circuit 24, the vertical gradient detection circuit 25, and the horizontal gradient detection circuit 26 to determine which coordinate in the three-dimensional frequency domain the input image is at. Then, the determination result is obtained and supplied to the shift amount generation circuit 34. The shift amount generation circuit 34 determines the shift amount S of the image shift circuit 30 and the image shift circuit 31 according to the determination result of the spatio-temporal coordinate determination circuit 33 and the motion vector V. The shift amount S is composed of two components, a horizontal shift amount Sx and a vertical shift amount Sy.
[0022]
The image shift circuit 30 and the image shift circuit 31 spatially move the input image according to the shift amount S supplied from the shift amount generation circuit 34, and supply the output to the adder circuit 35, respectively. The field spatially moved by the image shift circuit 30 and the field spatially moved by the image shift circuit 31 are reversed in time relationship as viewed from the current field that is the field to be interpolated, so that the moving directions are also reversed. The adder circuit 35 adds the high-frequency signals of the front and rear fields that have been spatially moved, and supplies them to the adder circuit 36. The adder circuit 36 adds the high-frequency signals of the front and rear fields that have been spatially moved to the low-frequency signal of the current field obtained by the vertical LPF 28, and outputs the result from the output terminal 37.
[0023]
FIG. 3 is a diagram for explaining the operation of the present invention. The operation of progressive scan conversion according to the present invention will be described in a time-vertical-horizontal three-dimensional space-time spectrum region. FIG. 3A is a diagram showing an image moving in the vertical direction at Vx = 0 and Vy = 1 in the time-vertical-horizontal three-dimensional spatio-temporal frequency domain, as in FIG. When FIG. 3A is projected onto the f-ν plane, FIG. 7B is obtained. The plane in contact with the μ axis in FIG. 3A is the original spectrum of the image moving in the vertical direction with Vx = 0, Vy = 1, and at the same time, (f, ν, t) = (± fst / 2, It is also a folded spectrum extending from the carrier x of space-time sampling by interlace scanning existing in fsv / 2,0).
[0024]
As shown in FIG. 7B, the original spectrum and the folded spectrum that overlap completely in the f-ν region are also spectra that do not overlap if considered in the f-ν-μ region as shown in FIG. You can see that it exists. For example, the solid line extending in the upper right direction from the origin is the spectrum of the diagonally rising edge component of the image moving in the vertical direction with Vy = 1, and the folded spectrum indicated by the broken line extending from x to the lower left It can be seen that they do not overlap at all.
[0025]
As described above with reference to FIG. 7, when Vy (number of scanning lines / field) is an odd number, even if motion compensation is performed using the motion vector V (Vx, Vy), the original spectrum and the folded spectrum cannot be separated. FIG. 3B shows a three-dimensional frequency response for an image moving in the vertical direction with Vx = 0 and Vy = 1. The cubic region in the figure is a passband when interpolation is viewed as a filter, and the hatched partial plane not included in this cubic region is a spectrum that is removed by sequential scanning conversion. When FIG. 3B is projected onto the f-ν plane, FIG. 7C is obtained.
[0026]
FIG. 3C shows the three-dimensional frequency response of the interpolation method of the present invention. In the interpolation method of the present invention, the pass band of the interpolation filter can be varied by the difference in spatio-temporal coordinates even on the same plane, so that it is possible to improve that the high band of the original spectrum of the solid line is cut, Image resolution degradation can be improved.
As described above, the scanning line interpolation apparatus of the present invention can obtain a high-resolution sequential scanning signal for an interlaced video signal including an image having an odd number of vertical motion components (number of scanning lines / field). .
In the embodiment of the present invention shown in FIG. 1, the case of combining with motion compensation interpolation has been described. However, the three-dimensional interpolation circuit 4 which is a feature of the present invention has a motion with an odd number of Vy (number of scanning lines / field). Of course, it is only necessary to operate in this case, so that it may be combined with scanning line interpolation other than motion compensation interpolation.
[0027]
【The invention's effect】
The scanning line interpolation apparatus of the present invention does not require any special restriction on motion vector detection, and in principle, it is difficult to obtain an interpolation signal, and the number of vertical motion components (number of scanning lines / field) is an odd number. Also for an image, there is an extremely excellent effect that a high-resolution progressive scanning signal can be obtained.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of the present invention.
FIG. 2 is a block diagram showing an embodiment of the three-dimensional interpolation circuit 4 in FIG.
FIG. 3 is a diagram for explaining the operation of the present invention.
FIG. 4 is a diagram showing a scanning line structure.
FIG. 5 is a block diagram showing a conventional example.
FIG. 6 is a diagram for explaining the operation of a conventional example.
FIG. 7 is a diagram for explaining the operation of a conventional example.
[Explanation of symbols]
1, 2, 32 Input terminal 2 Motion vector detection circuit 3 Motion compensation interpolation circuit 4 3D interpolation circuit 5 Selection circuit 6 Interpolation selection determination circuit 7 In-field interpolation circuit 8 Mixing circuit 9 Delay compensation circuit 10 Double speed conversion circuit 11, 37 Output Terminal 22, 27 Field delay circuit 23, 29 Vertical HPF
24 Inter-frame difference detection circuit 25 Vertical gradient detection circuit 26 Horizontal gradient detection circuit 28 Vertical LPF
30, 31 Image shift circuit 33 Spatio-temporal coordinate determination circuit 34 Shift amount generation circuit 35, 36 Addition circuit

Claims (3)

In the scanning line interpolation apparatus which interpolates the scan lines in the input image signal to interlaced input image signal into a progressive scanning image signal said input image signal,
A motion vector detection circuit for detecting a motion vector of the input image signal;
A motion compensation interpolation circuit that performs motion compensation on the input image signal according to the motion vector;
An intra-field interpolation circuit for performing intra-field interpolation on the input image signal;
The motion vector the motion vector detected by the detection circuit to determine exactly whether each pixel, the interpolation selection determination circuit for outputting a mixing coefficient,
Based on the spatio-temporal coordinate determination circuit for determining where the input image signal is located in the time-vertical-horizontal three-dimensional spatio-temporal frequency region, based on the determination result by the spatio-temporal coordinate determination circuit and the motion vector And a shift amount generation circuit for generating a shift amount in both the horizontal and vertical directions, and when the motion component in the vertical direction (number of scanning lines / field) of the motion vector is an odd number, the input image signal is converted into the horizontal and vertical directions. A three-dimensional interpolation circuit that performs a three-dimensional interpolation process based on the amount of shift in both directions ;
The inputs the output signal of the output signal of the 3-dimensional interpolator for motion compensated interpolation circuit selects the output signal of the 3-dimensional interpolation circuit in the case where the vertical movement component of the motion vector is odd, the a selection circuit configured to select an output signal of said motion-compensated interpolation circuit when the vertical movement component of the motion vector is even,
A mixing circuit for outputting engaged mixed according to the mixing coefficient and the output signal of the output signal and the field interpolation circuit of the selection circuit,
Reading an output signal of the input image signal and the mixing circuit alternately at twice the speed of the respective signals, characterized by being configured with a double-speed conversion circuit for converting the sequential scanning signals Scanning line interpolation device. The three-dimensional interpolation circuit
An inter-frame difference detection circuit that detects an inter-frame difference of an image from a field that is temporally changed with respect to the input image signal;
A vertical gradient detection circuit that detects the degree of signal change in the vertical direction of an image from a field that is temporally changed with respect to the input image signal;
A horizontal gradient detection circuit that detects the degree of signal change in the horizontal direction of the image from a field that is temporally mixed with respect to the input image signal;
The spatio-temporal coordinate determination circuit compares the output signals of the inter-frame difference detection circuit, the vertical gradient detection circuit, and the horizontal gradient detection circuit, so that the input image signal is at any coordinate in the three-dimensional spatio-temporal frequency domain. 2. The scanning line interpolating apparatus according to claim 1, wherein the scanning line interpolating apparatus is configured to determine whether or not there is any. The three-dimensional interpolation circuit
A vertical LPF that extracts the low-frequency signal of the current field of the input image signal;
A vertical HPF that extracts a high-frequency signal of a field before and after the input image signal in time,
In accordance with the shift amount generated by the shift amount generation circuit, an image shift circuit that spatially shifts the high-frequency signal of the previous and subsequent fields in the vertical and horizontal directions;
3. The scanning according to claim 1, further comprising: an adding circuit that adds an output signal of the vertical LPF and an output signal of the image shift circuit to obtain an output signal interpolated by scanning lines. Line interpolation device.

Images