JP3546698B2 - Scan line interpolation circuit - Google Patents

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a scanning line interpolation circuit used for converting an interlaced image into a progressive image.
[0002]
[Prior art]
Standard television signals such as NTSC signals and HDTV signals are interlaced (interlaced scanning) signals. 5A and 5B are diagrams showing a scanning line structure. FIG. 5A shows an interlaced signal, FIG. 5B shows a progressive (sequential scanning) signal, and FIG. 5C shows a signal obtained by converting an interlaced signal into a progressive signal by scanning line interpolation. Is shown. Note that the progressive signal may be referred to as a non-interlaced signal. In FIG. 5, a circle indicates a scanning line, and a cross indicates an interpolated scanning line.
[0003]
In FIG. 5, the vertical direction V is the vertical direction of the screen, and the horizontal direction t is the time direction. As shown in FIG. 5A, the interlace signal is composed of two fields in which one frame is shifted in time and in the vertical direction. On the other hand, the progressive signal has no shift in the scanning line structure as shown in FIG. In an interlaced signal, when the number of high frequency components in the vertical direction of an image increases, interlace interference such as generation of line flicker occurs. On the other hand, in a progressive signal, there is no interlace interference.
[0004]
Therefore, as shown in FIG. 5C, a processing method for removing interlace interference by interpolating a scan line of a part decimated by interlace with a peripheral scan line and converting the interpolated scan line into a progressive signal. is there. Such a processing method is referred to as progressive scan conversion or double density conversion.
[0005]
Conventionally, scanning line interpolation for progressive scan conversion and double-density conversion is performed by motion adaptive processing. That is, as shown in FIG. 6, when the image is stationary, a new scanning line is generated by performing inter-field interpolation with the average value of the pixels A and B in the preceding and succeeding fields as a new pixel Q indicated by x. I do. If the image is moving, a new scanning line is generated by performing intra-field interpolation with a new pixel Q indicated by the average value of the upper and lower pixels C and D.
[0006]
[Problems to be solved by the invention]
By the way, the progressive scan conversion by the inter-field interpolation or the intra-field interpolation can be regarded as a spatio-temporal filter in the time-vertical region for the input signal, and the inter-field interpolation is a time low-pass filter (hereinafter, the low-pass filter is abbreviated as LPF). ), Intra-field interpolation has the property of a vertical LPF.
[0007]
FIG. 7A shows a spectrum of a still image with respect to a time frequency f and a vertical spatial frequency ν. In a still image, as shown in FIG. 7A, the spectrum of the image exists only on the axis of f = 0. Note that the spectrum that spreads around the circle at f = 60, −60 (Hz) is an even-order harmonic spectrum. The blocking area of the inter-field interpolation which is the interpolation processing in the case of the still image is indicated by the hatched portion in FIG. 7A, and the spectrum of the non-hatched area corresponds to the output signal after the sequential scan conversion and the output signal. Become.
[0008]
Therefore, in the still image, by performing inter-field interpolation, the original spectrum including the even-order harmonic spectrum is not impaired, and f = 30, −30 (Hz) indicated by a broken line that causes interlace interference is present. , Only the odd-order harmonic spectrum spreading around. As a result, a progressive signal without image quality degradation can be obtained.
[0009]
On the other hand, FIG. 7B shows a spectrum of a moving image related to the time frequency f and the vertical spatial frequency ν. Generally, in a moving image, as shown in FIG. 7B, the inter-field correlation is small, and the spectrum of the image often spreads to a high time region. The reason why inter-field interpolation cannot be used for a moving image can be explained from this. That is, in the inter-field interpolation, a spectrum in a high time range, which is a great feature of a moving image, is removed.
[0010]
Therefore, a vertical filter having a characteristic of a stop band indicated by a shaded portion in FIG. 7B, that is, an intra-field interpolation, must be used so as not to remove a spectrum in a high time band in a moving image. However, in the fixed vertical LPF by the intra-field interpolation, although the spectrum in the high time region is preserved, a part of the original spectrum is lost, and the odd-order harmonic spectrum is not completely removed. Therefore, in the motion adaptive processing by the conventional scanning line interpolation circuit, the resolution deteriorates in the moving part, and interlace interference such as line flicker and jaggedness is not completely removed.
[0011]
SUMMARY OF THE INVENTION The present invention has been made in view of such a problem, and even if a moving part exists in an image during scanning line interpolation, high-quality scanning line interpolation is performed without deterioration in resolution or interlace interference. It is an object to provide a scanning line interpolation circuit capable of obtaining an image.
[0012]
[Means for Solving the Problems]
SUMMARY OF THE INVENTION The present invention provides a scanning line interpolation circuit for interpolating a scanning line of an input image signal, wherein the scanning line is located vertically above and below a current field which is an interpolated field. An intra-field interpolation means for generating an intra-field interpolation signal; a current-field low-frequency signal generation means for generating a current-field low-frequency signal which is a frequency component lower in the vertical direction of the current field; A post-field high-band signal generating means (3) for generating a post-field high-band signal which is a high frequency component in the vertical direction after the current field, and a vertical-field signal in front of the current field in the vertical direction. A front field high band signal generating means (8) for generating a front field high band signal which is a high frequency component in the direction; Moving direction detecting means (4) for detecting a moving direction and a moving amount of an image from the field, and a first shifting means for shifting the post-field high-frequency signal in accordance with the moving direction detected by the moving direction detecting means ( 9), second shifting means (10) for shifting the previous field high-frequency signal in accordance with the motion direction detected by the motion direction detecting means, and the second shift means shifted by the first and second shift means. A spatio-temporal interpolation signal generating means (13, 14) for adding a rear field high frequency signal, the previous field high frequency signal, and the current field low frequency signal to generate a spatiotemporal interpolation signal; Adaptive mixing means (11, 12, 15) for adaptively mixing the intra-field interpolation signal and the spatiotemporal interpolation signal in accordance with the motion amount detected by It is to provide a scanning line interpolation circuit, characterized in that configuration was.
[0013]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a scanning line interpolation circuit according to the present invention will be described with reference to the accompanying drawings. FIG. 1 is a block diagram showing an embodiment of the scanning line interpolation circuit of the present invention, FIG. 2 is a block diagram showing an example of a specific configuration of the motion direction detector 4 in FIG. 1, and FIG. 3 is a scanning line of the present invention. FIG. 4 is a block diagram showing an example in which the interpolation circuit is applied to a motion adaptive interpolation device, FIG. 4 is a block diagram showing an example in which the scanning line interpolation circuit of the present invention is applied to a progressive scan conversion device, and FIG. 8 is a scanning line interpolation circuit of the present invention. FIG. 5 is a diagram showing an image spectrum for explaining the following.
[0014]
First, the principle of the scanning line interpolation circuit of the present invention will be described with reference to FIG. The spectrum of an image when the image is moving in a vertical or horizontal direction has a characteristic that the spectrum spreads with directionality. For example, when the image is moved at a constant speed at a horizontal movement speed Vx = 0 and a vertical movement speed Vy = 2 (scanning line / field), the spectrum of the image is in the f-ν plane as shown in FIG. It exists only on the fixed straight line shown by the thick solid line above.
[0015]
Therefore, in the present invention, the motion direction and the motion amount of the image are detected from a field temporally before and after the current field. This makes it possible to distinguish the spectra of a plurality of characteristic images. Also, based on the detected motion direction, the front and rear field high frequency signals are spatially shifted up, down, left and right and added to the current field low frequency signal to obtain a spatiotemporal interpolation signal. This makes it possible to perform an interpolation process having an optimal frequency characteristic on each characteristic image spectrum.
[0016]
For example, for an image having the spectrum shown in FIG. 8, by performing spatio-temporal interpolation in which a shaded portion (filled in black) in FIG. 8 is a stop band, a progressive signal without image quality degradation is performed. Can be obtained.
[0017]
Here, the configuration and operation of the scanning line interpolation circuit of the present invention will be specifically described. In FIG. 1, an interlaced image signal input from an input terminal 1 is input to a field delay unit 2, a vertical high-pass filter (hereinafter, high-pass filter is abbreviated to HPF) 3, and a motion direction detection unit 4.
[0018]
The field delay unit 2 delays the input image signal by a time corresponding to one field, and inputs the delayed image signal to the intra-field interpolator 5, field delay unit 6, and vertical LPF 7. The field delay unit 6 delays the input image signal by a time corresponding to one field, and inputs the delayed image signal to the motion direction detection unit 4 and the vertical HPF 8. Therefore, different image signals are input to the vertical HPF 3, the intra-field interpolator 5 and the vertical LPF 7, and the vertical HPF 8, each of which differs by one field. The image signal output from the field delay unit 2 is defined as a current field.
[0019]
The vertical HPF 3 extracts a high-frequency signal of a field temporally later than the current field, which is the interpolated field, and inputs the signal to the image shifter 9. Similarly, the vertical HPF 8 extracts a high-frequency signal of a field temporally before the current field, which is the interpolated field, and inputs the signal to the image shifter 10. That is, the vertical HPF 3 is a post-field high-band signal generation unit that generates a post-field high-band signal that is a high-frequency component in the vertical direction after the current field in time, and the vertical HPF 8 is Means for generating a front-field high-band signal that is a high-frequency component in the vertical direction immediately before the current field.
[0020]
The motion direction detection unit 4 uses a configuration and method to be described later to set a value (hereinafter, motion direction value) s indicating a motion direction of an image between two fields (one frame) and a value (movement direction) indicating an amount of motion of the image. And the motion direction value s of the image is input to the image shifters 9 and 10, and the motion amount value k of the image is input to the multipliers 11 and 12. That is, the movement direction detection unit 4 is a movement direction detection unit that detects a movement direction and a movement amount of an image from a field temporally before and after the current field.
[0021]
The image shifter 9 and the image shifter 10 spatially move the input high-frequency signal according to the motion direction value s, and input the high-frequency signal to the adder 13. Note that the field that moves spatially by the image shifter 9 and the field that spatially moves by the image shifter 10 have the opposite temporal relationship with respect to the current field that is the interpolated field, so that the moving directions are also reversed. . The adder 13 adds the high-frequency signals of the preceding and succeeding fields that have been spatially moved, and inputs the sum to the adder 14.
[0022]
The vertical LPF 7 takes out the low-frequency signal of the current field and inputs it to the adder 14. The vertical LPF 7 is a current-field low-band signal generating unit that generates a current-field low-band signal that is a low-frequency component of the current field in the vertical direction. The adder 14 adds the result of the addition of the high-frequency signals of the preceding and succeeding fields input from the adder 13 and the low-frequency signal of the current field input from the vertical LPF 7 to generate a spatiotemporal interpolation signal. That is, the adder 13 and the adder 14 are spatio-temporal interpolation signal generation means for generating a spatio-temporal interpolation signal by adding the rear field high band signal, the previous field high band signal, and the current field low band signal. This spatiotemporal interpolation signal is input to the multiplier 11.
[0023]
Further, the intra-field interpolator 5 adds the scanning lines spatially located above and below the interpolated scanning line to generate an intra-field interpolation signal. In the intra-field interpolator 5, a delay equivalent to the delay generated when the spatio-temporal interpolation signal is generated is made, and the intra-field interpolation signal synchronized with the spatio-temporal interpolation signal is input to the multiplier 12.
[0024]
As described above, the motion amount value k is input to the multipliers 11 and 12 from the motion direction detector 4. The multiplier 11 multiplies the input spatiotemporal interpolation signal by (1−k), the multiplier 12 multiplies the input intra-field interpolation signal by k, and inputs each multiplication result to the adder 15. The adder 15 adds the spatio-temporal interpolation signal weighted by the motion amount value k and the intra-field interpolation signal, and outputs a final interpolation signal from the output terminal 16. Note that the multipliers 11 and 12 and the adder 15 are adaptive mixing means for adaptively mixing the intra-field interpolation signal and the spatiotemporal interpolation signal according to the motion amount value k detected by the motion direction detection unit 4.
[0025]
Next, a detailed configuration and operation of the movement direction detection unit 4 in FIG. 1 will be described. 2, the input interlaced image signal is input to an image shifter 17, and the image signal output from the field delay 6 is input to an image shifter 18. The image shift setting unit 19 sequentially generates image shift values according to a preset search range, and inputs the image shift values to the image shift unit 17, the image shift unit 18, and the motion direction determination unit 23. Note that the search range means a range in which an input signal is shifted in order to detect a moving direction and a moving amount of an image signal.
[0026]
For example, assuming that the search range is ± 1 in a scanning line unit in the vertical direction and ± 2 in a pixel unit in the horizontal direction, the image shift setting unit 19 sequentially shifts 15 image shift values of 3 × 5 (vertical × horizontal). Will occur. The image shifter 17 and the image shifter 18 spatially move the input image signal according to the image shift value and input the image signal to the subtracter 20. Note that the field spatially moved by the image shifter 17 and the field spatially moved by the image shifter 18 have opposite time relations with respect to the current field, which is the interpolated field, so that the moving directions are also opposite. To
[0027]
The subtractor 20 subtracts the output of the image shifter 17 from the output of the image shifter 18 to obtain an inter-frame difference, and inputs the difference signal (inter-frame difference value) to the absolute value converter 21. The absolute value converter 21 converts the input difference value between frames into an absolute value and inputs the absolute value to the spatial LPF 22. The spatial LPF 22 smoothes a spatial change by applying a spatial low-pass filter to the absolute value of the inter-frame difference value, and inputs the result to the motion direction determiner 23. As described above, each image shift value is input to the motion direction determiner 23 from the image shift setting device 19, and the motion direction determiner 23 calculates an inter-frame difference value of each image shift value input from the spatial LPF 22. Compare.
[0028]
As a result of the comparison between the inter-frame difference values, the motion direction determiner 23 outputs, to the output terminal 24, the image shift value at which the inter-frame difference value becomes the minimum as the motion direction value s of the image. This movement direction value s is input to the image shifters 9 and 10 in FIG. 1 as described above. Further, the motion direction determiner 23 inputs the minimum inter-frame difference value to the nonlinear converter 25. The non-linear converter 25 performs non-linear conversion on the input inter-frame difference value, and outputs the result as an image motion amount value k. That is, the case where the input inter-frame difference value is equal to or less than the noise level is set to 0, and the case where the input inter-frame difference value is equal to or higher than the predetermined level is set to 1, and the interval is linearly converted.
[0029]
This predetermined level is a level at which the intra-field interpolation signal obtained by the intra-field interpolator 5 in FIG. 1 is clearly more appropriate than the spatio-temporal interpolation signal obtained by the processing up to the adder 14. Set. The motion amount value k thus obtained is output from the output terminal 26. This motion amount value k is input to the multipliers 11 and 12 in FIG. 1 as described above.
[0030]
The scanning line interpolation circuit of the present invention is configured as described above. The scanning line interpolation circuit of the present invention is used as a scanning line interpolation circuit of a motion adaptive interpolation device as shown in FIG. 3, and as a scanning line interpolation circuit of a sequential scanning line conversion device as shown in FIG. Can be
[0031]
First, in FIG. 3, an image signal input from the input terminal 1 is input to a scanning line interpolation circuit 30, an inter-field interpolator 31, and a motion detector 33. The scanning line interpolation circuit 30 is the scanning line interpolation circuit according to the present invention shown in FIG. The scanning line interpolation circuit 30 generates a motion interpolation signal and inputs the signal to the adaptive mixer 32. As described with reference to FIG. 6, the inter-field interpolator 31 generates an average value of the pixels A and B in the temporally preceding and succeeding fields of the interpolated field as a stationary interpolation signal, and inputs the interpolated signal to the adaptive mixer 32. The motion detector 33 detects the motion of the image. The adaptive mixer 32 adaptively mixes the motion interpolation signal and the stationary interpolation signal according to the magnitude of the motion detected by the motion detector 33, and outputs a final interpolation signal from an output terminal 34.
[0032]
Next, in FIG. 4, the input interlaced image signal is input to the scanning line interpolation circuit 40 and the field delay unit 41. The scanning line interpolation circuit 40 is the scanning line interpolation circuit according to the present invention shown in FIG. The interpolation signal generated by the scanning line interpolation circuit 40 is input to the line buffer 42. The field delay unit 41 delays the input image signal by the same amount as the delay generated by the scanning line interpolation circuit 40, and inputs the same to the line buffer 43. The line buffer 42 and the line buffer 43 hold a signal for one line and read out the signal at twice the speed of the input signal. The outputs of the line buffers 42 and 43 are input to a switch 44, and are alternately selected by the switch 44 to form a sequential scanning signal, which is output from an output terminal 45.
[0033]
As described above, the scanning line interpolation circuit of the present invention can detect the motion direction and size of an image and change the characteristics of the spatio-temporal interpolation filter optimal for the spectrum of the image at that time. At the time of scanning line interpolation such as scan conversion and double-density conversion, resolution is not degraded, and interlace interference such as line flicker and jaggedness does not occur.
[0034]
【The invention's effect】
As described in detail above, the scanning line interpolation circuit of the present invention includes an intra-field interpolation unit that generates an intra-field interpolation signal from scanning lines spatially located above and below a current field that is an interpolated field; A current-field low-frequency signal generating means for generating a current-field low-frequency signal that is a vertically low-frequency component, and a post-field high-frequency signal that is a vertically high-frequency component temporally behind the current field. A post-field high-frequency signal generating means for generating, a front-field high-frequency signal generating means for generating a front-field high-frequency signal that is a high frequency component in the vertical direction in front of the current field in time, Moving direction detecting means for detecting the moving direction and the moving amount of the image from the temporally preceding and succeeding fields, and the moving direction detecting means First shift means for shifting the rear-field high-frequency signal in accordance with the moving direction, second shifting means for shifting the preceding-field high-frequency signal in accordance with the motion direction detected by the motion direction detecting means, A spatiotemporal interpolation signal generating means for generating a spatiotemporal interpolation signal by adding the post-field high-frequency signal and the previous field high-frequency signal shifted by the second shift means and the current field low-frequency signal; In accordance with the amount of motion detected by (1), adaptive mixing means for adaptively mixing the intra-field interpolation signal and the spatio-temporal interpolation signal is provided. A high-quality scanning line interpolation image can be obtained without causing deterioration or interlace interference.
[Brief description of the drawings]
FIG. 1 is a block diagram showing one embodiment of the present invention.
FIG. 2 is a block diagram showing an example of a specific configuration of a motion direction detector 4 in FIG.
FIG. 3 is a block diagram showing an example in which the present invention is applied to a motion adaptive interpolation device.
FIG. 4 is a block diagram showing an example in which the present invention is applied to a progressive scan conversion device.
FIG. 5 is a diagram showing various scanning line structures.
FIG. 6 is a diagram showing inter-field interpolation and intra-field interpolation.
FIG. 7 is a diagram showing a spectrum of an image of inter-field interpolation and intra-field interpolation for explaining a conventional problem.
FIG. 8 is a diagram showing an image spectrum for explaining the present invention.
[Explanation of symbols]
2, 6 field delay unit 3, 8 vertical high-pass filter 4 motion direction detection unit 5 intra-field interpolator 7 vertical low-pass filter 9, 10 image shifter 11, 12 multiplier 13, 14, 15 adder

Claims (1)

In a scanning line interpolation circuit that interpolates a scanning line of an input image signal,
Intra-field interpolation means for generating an intra-field interpolation signal from scanning lines spatially located above and below the current field which is the field to be interpolated,
Current field low band signal generating means for generating a current field low band signal which is a low frequency component in the vertical direction of the current field,
A post-field high-band signal generation unit that generates a post-field high-band signal that is a high frequency component in the vertical direction after the current field,
A previous-field high-band signal generation unit that generates a previous-field high-band signal that is a high frequency component in the vertical direction before the current field in time,
A movement direction detection means for detecting a movement direction and a movement amount of an image from a field temporally before and after the current field,
First shifting means for shifting the post-field high-frequency signal according to the movement direction detected by the movement direction detection means;
Second shifting means for shifting the previous field high-frequency signal according to the movement direction detected by the movement direction detection means;
A spatio-temporal interpolation signal for generating a spatio-temporal interpolation signal by adding the post-field high band signal and the pre-field high band signal shifted by the first and second shift means and the current field low band signal. Generating means;
A scanning line interpolation circuit comprising: adaptive mixing means for adaptively mixing the intra-field interpolation signal and the spatiotemporal interpolation signal in accordance with the amount of motion detected by the motion direction detection means.

Images