JPS58205377A - Adaptive type time spatial filter - Google Patents

Abstract

PURPOSE:To reproduce an excellent picture suitable for the property of a picture signal, by changing the passing characteristics in response to the amount of movement of the picture. CONSTITUTION:An input picture signal is applied to a vertical filter 5 comprising a 1H delay circuit and a time filter 6 comprising a field memory, and output picture signals of the filters 5 and 6 are synthesized at an adder ADD4 and a sequential scanning picture signal output is obtained. Further, a filter control circuit 7 outputs a filter coefficient control signal so as to change the filter characteristics suitably for a still and a synamic picture based on the time spatial picture information from the filters 5, 6, allowing to control the filters 5 and 6.

Description

[Detailed description of the invention]

The present invention relates to an adaptive ##j spatial filter used for interpolation adapted to a 1 mm image size during scan conversion of an image signal such as a television signal. It adapts to the characteristics of the image signal and reproduces a high-quality image [7]. A conventional standard television signal uses 2:l innotarcing scanning, has a number of scanning lines of 525, a field frequency (l U H2s), a frame frequency of 80 Hz, and is divided vertically, that is, in the scanning f-line array direction, and in the time axis direction, that is, in the field and The image signal is made up of IC P in the frame direction.Therefore, the UFI+ 11
When the tg signal is expressed as a two-dimensional spectrum with vertical spatial frequency and temporal frequency as axes, as is well known, the converted signal spectrum is It exhibits a spectral structure that appears periodically and repeatedly on the coordinates. In Figure 1, which shows the frequency characteristics of the conventional spatio-temporal interpolation filter, point A is the repeating component closest to the baseband spectrum including the origin among the sampled tube wave numbers that are scattered by the 2:1 interlace scan f. It shows the center cylinder wave number, and since the baseband spectrum is distributed around this frequency point A (7), the frequency point A
When the baseband spectrum of the i-center toss is superimposed, it becomes a so-called aliasing problem, which causes image disturbances peculiar to ink-lace scanning images such as interline clitska, and deteriorates the image quality (2). In order to prevent the occurrence of such image disturbance, conventionally, a 2;
Interfering spectrum components appearing based on interlaced scanning are removed by a spatiotemporal interpolation filter having the configuration shown in FIG. 2, and a 2:1 interlaced scanning image is converted into a sequential scanning image at 60 frames per second by interpolation. It was 7. Although a considerable improvement in image quality was obtained by taking such measures, the pass characteristics of the spatio-temporal interpolation filter with the configuration shown in FIG.

[7. Optimal passing characteristics that are originally different between still images and moving images. Therefore, conversely speaking, depending on the conventional spatio-temporal interpolation filter of this kind, the configuration V which obtains the optimal pass characteristics for both still images and moving images is different. Range filter (V) [PF
) 2 and a vertical low-pass filter (VLPF) a are each composed of 1'C, f, and 14 with a period (]H) delay circuit as a unit delay element in one horizontal scan. By changing the ratio of mixing the delayed output signals of the delay output signals, the respective passing characteristics are changed, and the total passing + fNote is obtained.For example, for the six parts in the frequency characteristics shown in the first diagram, As indicated by the arrows in the figure, components in the time axis direction and the vertical spatial frequency axis force direction increase and decrease in mutually opposite directions. - Considering the distribution of the baseband spectrum of the image signal, for still images, there is no baseband spectrum component in the time axis direction, and the spectral component appears only on the vertical spatial frequency axis, so interpolation It is desirable that the filter exhibits a pass characteristic in which the pass band is not limited in the vertical spatial frequency axis direction and the pass band is limited only in the time axis direction. However, for example, as shown in FIG. 3(a), for a moving image in which a rectangular image moves in the right direction of the arrow, the interpolated signal by the interpolation filter in the time axis direction includes the 8th As shown in Figure (b), the outline formed by the vertical sides of the rectangular national electric line is drawn by a brush in the time axis direction. ! , discontinuity of the I image occurs, significantly degrading the image quality. On the other hand, in the interpolated signal by the interpolation filter in the vertical spatial frequency axis direction, vertical blurring occurs in the contour formed by the horizontal sides of the rectangular image, as shown in FIG. Deterioration is less noticeable with filters than with filters. Therefore, in the conventional spatio-temporal interpolation filter having the configuration shown in the second figure, the shaded area shown in the first figure exhibits a pass characteristic intermediate between a temporal filter and a vertical spatial filter, so that it is suitable for both still images and moving images. It was designed to be a transit area. Therefore, the normal II! of si obtained by a normal television camera! II
Although it exhibits a good interpolation filter effect for image numbers, it does not work well for all kinds of image signals, including pan-quality television 1III image signals and electronically configured images (-1). The disadvantage is that the spatio-temporal interpolation filter does not exhibit optimal pass characteristics. 4. The purpose of the present invention is to eliminate the above-mentioned conventional drawbacks, and to obtain optimal pass characteristics for both still images and moving images. An object of the present invention is to provide an adaptive spatio-temporal filter that exhibits characteristics and is capable of reproducing a beautiful, fleshy image without producing even a single sliver in the contour of a still image. The filter is composed of a temporal filter and a vertical spatial filter, and in the temporal and spatial interpolation filter used for interpolation of signal tarlace and progressive scan conversion of -ftI! According to the result of configuring the entire filter and detecting the inter-frame high-frequency components and the correlation components between both the horizontal and vertical sides of the image GL signal,
By forming a spoofing signal related to the 1IiII movement represented by the 1Iil image signal and changing the pass characteristics of the spatiotemporal filter, the spatiotemporal filter is adjusted according to the motion weight of the IIkl image. It is characterized in that t7 is changed so as to change the passing characteristics. Referring to Figure rkI below, the present invention will be described in detail. First, an example of the basic configuration of the adaptive spatio-temporal filter of the present invention is shown in FIG. In the illustrated configuration, an interlaced set meal image signal is input and a 1llrK scanning th image signal is output. Along with supplying those filters 5
The output images of 6 and 6 are combined in a soft signal adder ADD4 to obtain a +w-order fmi signal output. Furthermore, the filter control circuit 7 adaptively changes the filter characteristics for the still image and the #I image based on the spatio-temporal image information from the filters 5 and 6. Letter coefficient %I
II control signal is output to control each filter b and 6. In the figure, the solid line represents the flow of the image signal, and the one-dot chain line represents the flow of the coefficient f111 signal. The basic configuration of such a filter control circuit 70 is shown in FIG. In the filter control circuit with the illustrated configuration, the time difference component detection circuit 8 determines the change in the image signal in the time axis direction, but since there is a response even for flickering images, it does not necessarily function as an IclI image detection circuit. (, No. (7) Therefore, the horizontal/yellow phase open component detection circuit 9 detects the difference (1) in Fig. 8 (1). , at the normal spatial frequency given to an image with 525 set lines, /2 cycles/l[l1tf
]? #iJ component is detected, and the time difference component detection output signal and the horizontal/vertical correlation component detection output signal are respectively weighted and guided to the discrimination processing circuit 12 via circuits IO and 11 to calculate their mutual product. By finding the above-mentioned coefficient control signal, the coefficient control signal depends on both the amount of movement and the contour shoe (7), so that the filter coefficient is fully controlled. The operating characteristics of the adaptive spatio-temporal filter of the present invention having the basic configuration shown in FIG. 4 will be explained with reference to FIGS. , and the frequency points AVi are as described above with reference to FIG. As mentioned above, the coefficient control output value becomes larger as η=0.η
=η. and l=η. The custom-made filters are shown in the same figure (al, (bl) and (c), respectively.The filter characteristics in the same figure (a) when η=0 and 1 are shown in gold for a still image, The purpose of using the illustrated passing percentage is to avoid excessive vertical resolution deterioration due to interpolation. However, in the case of moving images, In addition, as the filter coefficient control output value η increases, that is, the amount of motion of the image increases, the outline of the vertical side of the rectangular image becomes blurred, which causes image quality deterioration. Same figure (C)
Change the filter characteristics as shown in . Due to such a change in filter characteristics, for example, the above-mentioned square ll1II
The outline blur of the image is shown in Figure 8(b).
This will move to the horizontal side shown in Figure (C). [7] The blurring of the outline of a rectangular flaw is visually much less noticeable when it occurs on its horizontal side than on its vertical side, so as mentioned above, the blurring of the outline of a rectangular flaw is much less noticeable when it occurs on its horizontal side. By changing the filter characteristics accordingly, the deterioration in image quality caused by the sleeve 111 filter can be sufficiently eliminated. When converting a 2:l interlaced IIkI image signal of 60 fields per second to a sequentially running 1-picture image signal of 60 frames per second, the interlacing IC Sakaki Sei is based on the basic configuration shown in Figure 4. An example of the detailed configuration of the adaptive time ill filter of the present invention is shown in FIG. But [2nd, 7th]
The configuration of the spatio-temporal filter of the present invention shown in FIG. The vertical high-pass filter 2 and the vertical low-pass filter 8 can be modified so as to obtain a filter output suitable for supplying to the horizontal/vertical correlation detection circuit 9 in the filter control circuit 7. Furthermore, the filter control circuit 7 has the basic configuration shown in FIG. The absolute value α' of , is determined by the absolute value unit 18, and then,
The coefficient multiplier 10 multiplies the weighting coefficient C'fr to form the time difference component α. That is, α'=1/81=l/, -/work1 (1) α=C
α′(2) On the other hand, in the horizontal/vertical correlation component detection circuit 9, the filter output obtained from the vertical high-low pass filters 2 and 3 is
, t! t, which will be described later with reference to FIGS. 9 and 10, are taken out as delayed outputs of each stage in the spatiotemporal transpersion filter, and a correlation component between these M extended output image signals is formed. Therefore, in the vertical direction, each stage of the low-pass filter 2.8 gold eggplant transversal filter has a delay output of 1! Mkz
As shown in □, t, H1 No. 81! 81 (a), the filter output images of high and low pass filters 2 and 8 are represented by solid lines and dotted lines, respectively. FIG. 8(b) is a pixel signal component corresponding to each point shown in FIG. It is expressed as 1 in a matrix. In addition, as mentioned above, the vertical high-pass filter (VHP
F) 2 Oh! bimi [direction low range 7 (ruta (VLPF)
3 are as shown in FIGS. 9 and 10, respectively.
%IH delay circuits 21-1 to 21-m and their respective stage delayed output signals are processed by n-stage th elementary period τ delay circuits 22-n to 2.
2-1. t, delay time %2B and coefficient unit (a) 24 or coefficient unit (1)) 25 in order, adder DD
The above-mentioned horizontal/vertical correlation component detection circuit 9 is composed of a vertical high/low pass filter 2.81 as shown in FIG. ′# Pixel period τ delay output signal k of each stage in the II successful transversal filter
The coefficients t□, t. (+1126 Omahi coefficient unit (-) > 27 alternately 7
After combining by the counter 28, the combined output signal ν
is led to the adder ADD 6 via the e-column auxiliary unit 29, and the horizontal and vertical correlation components β' are derived as the combined output signal. Therefore, the delay circuit 12' in the filter control circuit 7 of the spatio-temporal filter of the present invention shown in FIG. 7(a)K, and the vertical tube low-pass filters 2 and 3 shown in FIGS. This is for adding a delay of time t1 to the image signal in the t□ delay circuit 23 in order to synchronize the timing of the image signal and the coefficient control signal in each stage. Therefore, in the horizontal/vertical correlation component detection circuit 9 in the filter control circuit 7 in the spatio-temporal filter of the present invention having the configuration shown in FIG. For example, the correlation component β' is detected as a blur that occurs on the vertical contour such as the vertical side of the rectangular image mentioned above, and the detection output β' is the first
In FIG. 1, the combined output signal ν of the adder 28-1 at each stage in the horizontal/vertical correlation component detection circuit 9 described above is expressed as follows. Furthermore, the coefficient unit (+1) 26 and the coefficient unit (-1) in the horizontal/vertical correlation component detection circuit 9 having the configuration shown in FIG.
) 27 alternately inverts the polarity of the delayed output image signals of each stage, and the delayed output image signals of each stage constituting the composite value ν of the alternately inverted delayed output image signals □1.
t kJ, *...v km, i represent detection points arranged vertically across two fields on the screen shown in FIG. 8(a). Therefore, each coefficient unit (+1
), (-1), the composite value ν1 of each stage of the alternately inverted delayed output image signal is 525 /2 (
The contour blur of the vertical side of the rectangular image shown in Figure 8) is the vertical component 2 of sz5/(vicle/picture 1LA). Since the above-mentioned composite value ν is large, the composite value ν□ is also a large value.This composite value ν, is calculated over several points in the horizontal direction as shown in Figure No. value β
' is determined in order to detect the movement t of the image in the horizontal direction, and the faster the wJ movement of the image, the larger the blurred area on the vertical side, so the absolute value 1ν of the composite value ν, 1
The composite value β', which is the sum of the values, also shows a large value. 'Also, in the configuration of FIG. 7 (I3), the filter control circuit 7
The input signal γ of the control signal processing circuit 14 at
′ is the formula%formula%(
8) (5) (6) It becomes. Note that the coefficient d in equation (6) is a coefficient that weights the horizontal/vertical correlation component β, and is multiplied by the coefficient unit 11 by 1. Weighting is applied. However, in moving images with fast image movement, interpolation in the time axis direction will cause the same deterioration in image quality as in the past. That is, as shown in FIG. 7(b), the control signal r that appears as an inter-frame difference component is generated as the difference between the first field and the third field, and therefore, when the image movement is extremely fast, teeth,
Although an inter-frame difference signal is generated during the illustrated period c, which is the moving region, the signal level of the control signal γ becomes zero during the τ' period, which corresponds to the second field. The control signal processing circuit 14 in the filter control circuit 7 in the configuration example of the spatio-temporal filter of the present invention shown in FIG. 6 in Figure 7) when the signal γ is applied between the frames of the waveform shown in Figure 7 (0), the output waveform becomes dull as shown in Figure 7 (0). Regarding b), the above-mentioned period τ' of zero coefficient control signal disappears. However, the innocuous response length τ' shown here is the highest value that is required for the correction shown in FIG. In the case where a simple low-pass filter 16 is used to correct the time delay caused by the filter output of the coefficient control signal, the seventh
A drawback arises in that an extra coefficient control signal is generated for a period f before and after the inter-frame difference signal waveform shown in FIG. 3(b). In order to eliminate such drawbacks and obtain the f'C coefficient control signal, an example of the configuration of the g processing circuit 14 is shown in FIG. 7(d). a) ~ billion)
is shown in FIG. 7(e). In the 0 control signal processing circuit 14 according to the illustrated configuration, a filter corresponding to waveform (b) obtained by applying r<r inter-frame difference signal γ to waveform (a) using a low-pass filter (LPF) 1 (i VC4 Output 11 A delay circuit (D
)1? -1 output signal (a), output signal (b) obtained by filter output signal 4g (0) obtained from an equivalent delay circuit (D) 17-2, and filter output signal 1b (C) 1 warehouse and kenam
The waveform (e
), and the delayed output signal (b) and the filter output No. 16 (0) are added to the adder ADD.
Formed by 1. The synthesized waveform shown in waveform (f) is obtained, and the synthesized waveform shown in waveform (f) is supplied to the underflow circuit 20 (7). If it is extracted as an output value and non-additively mixed with the delayed output signal (1)) in the NAM circuit 19, the zero value period τ of the input frame difference signal r, as shown in Tsuzumi Cg)
A positive output signal level is also obtained at `, and a processed output control signal r having an appropriate waveform is extracted without producing an extra positive output signal in the front and rear dynamic and static regions. However, the delay time of delay circuit 17-1.174 is LPF
equal to the impulse response length of 16. Next, the input/output characteristics with the coefficient control signal η obtained from the judgment/control circuit 15 which receives the processed output control signal r as input are as shown in FIG.
For a level change of ze......, γ, the output signal η becomes η0. η2. ......, η. The coefficient multiplier 241 of the vertical high-pass filter (VHPF) and low-pass filter (VLPF) shown in FIG. 9 and FIG. 1θ, respectively, takes discrete values. And in step 251, the coefficient values al and bi to be multiplied by the delayed image signal of each stage are changed. Note that although the coefficient control signal η can generally be a continuous value, it is preferable to use a discrete value when implementing it in hardware, and even if the filter coefficient value is continuously changed, the effect will not change. Since it is small, it is usually sufficient to use discrete theory. The input/output characteristics shown in Figure 12 can be easily tested using the comparator sound. The relationship between the characteristics of the space-time and interpolation filters is set as shown in Figure 6 (a1 to (C)). That is, at η = 0. is the filter %Note for stationary surface 1, so it is a temporal filter consisting only of field memory as a spatiotemporal interpolation filter, and when η>0, the filter characteristics shown in Figure 7 (b) to (C1) The parameters of the filter control circuit 7 are set so as to match the blurring characteristics of the image contour shown in FIG. 3(b) or (C1) and reduce image quality deterioration.
For the filter characteristics shown in Fig. 6 (al K), the coefficient value of the vertical low pass filter (VLPF) 8 is set to i = o, and the vertical high pass filter (VHPF) 2 is set to trl through. ), the coefficient value a of the vertical high-pass filter (VHPFl 2)
As i = O, commission←÷2 ・
, act as a temporal filter and a vertical spatial filter, respectively. In the above explanation, the example of converting from 2:1 interlaced scanning to 11@ next scanning was unsuccessful, but if multiplexing is performed on the main scan shown in FIGS. 4, 5, and 7 (al Vc) It can be extended and applied to the case of converting from interlaced scanning to progressive scanning. Great effects can be expected by applying an adaptive spatio-temporal interpolation filter to a receiver for pan-quality television broadcasting using transmission.As is clear from the above description, according to the present invention, As an interpolation filter used when converting an interlaced scan image signal to a progressive scan image signal, an adaptive spatiotemporal filter that exhibits good interpolation characteristics for both still and moving images can be configured, and It is possible to create sequentially scanned images without blurring.In other words, by controlling the filter characteristics according to the results of detecting all the time difference components and horizontal and vertical correlation components of the image signal, it is possible to create a progressively scanned image that does not cause blurring. Regardless, the best interpolation effect can be achieved regardless of the degree of image movement i.
Optimal filtering percentage adapted to the nature of the image can be set at any time, and supplement 1ii: The configuration of the filter can be simplified. Note that the spatio-temporal filter of the present invention is applicable not only to conversion from a 2:1 interlaced scanning image signal, but also to conversion from multiple interlaced scanning i! Similar good effects can be obtained with the interpolation filter used when converting an IllI image signal to a progressively scanned image signal, and the transmitter/receiver system can be configured as a progressive scan type, and the transmission system can be configured as an interlaced scan type. ,
Low transmission rate that allows scanning format compatibility on the receiver side,
This method can be effectively applied to tube-quality telephony systems.

[Brief explanation of the drawing]

Figure 1 is a block diagram showing the configuration of a conventional spatio-temporal interpolation filter, Figure 2 is a characteristic curve diagram showing the frequency characteristics of the spatio-temporal interpolation filter, and Figure 8 (a) to (0) ii images. Fig. 4 is a block diagram showing the basic configuration of the adaptive spatio-temporal filter of the present invention, and Fig. 5 is a diagram showing examples of contour blur generation by the interpolation filter. A block diagram showing the basic configuration of the filter control circuit in the filter, FIG. Figures (al to e) are block diagrams and waveform diagrams showing the detailed configuration of the adaptive spatio-temporal filter, the form of filter control signal formation, the configuration of the processing circuit, and examples of operation waveforms of each part, respectively; (a) and 2) are diagrams showing examples of the arrangement of image signal detection points that are used as the basis for forming characteristic control signals for the adaptive spatio-temporal filter, and FIG. 7 is a block diagram showing an example of a detailed configuration of a vertical high-pass filter, FIG. 10 is a block diagram showing example 1 of a detailed configuration of a vertical low-pass filter in the adaptive spatio-temporal filter, and FIG. Similarly, a block diagram showing an example of the detailed configuration of the horizontal/vertical correlation component detection circuit in the adaptive spatio-temporal filter is shown in FIG. This is a characteristic curve diagram showing an example. 1-1.1-1...Field memory, 2...Vertical high-pass filter, 8...Vertical low-pass filter, 4
, 12'... Delay circuit, 5... Horizontal/vertical filter, 6... Time filter, 7... Filter control circuit, 8... Time difference component detection circuit, 9... Horizontal/vertical correlation Component detection circuit, 10.11... Weighting circuit, 12
...Motion discrimination processing circuit,]8. Absolute value unit, 14..Control signal processing circuit, 15..Judgment control circuit,]C6..
Low-pass filter, 17-1.17-2...delay circuit, ]
8゜19... NAM circuit, 20... Underflow circuit, 21-1 to 21-m, - IH delay circuit, 22-i
-τ delay circuit, 28-1 to 28-n...t0 delay circuit, 24-1 to 24-n, 25-1 to 25-n...coefficient unit, 26-i, 27-1... Coefficient unit, 28-1~2
8-n...Adder, 29-1 to 29-n...Absolute value unit, ADD to ADD6...Adder, SUB□, SUB,
...Subtractor, MUL...Multiplier. Patent Applicant Japan Broadcasting Association Figure 1 Figure 2 Figure 3 (b) -1111- (C) EEEE b) (C) (d) Fig. 7 (e) Fig. S (a) (b) Fig. 9 Upper blade F Fig. 1 O Dog de Q

Claims (1)

[Claims] 1. In a spatio-temporal interpolation filter that consists of a temporal filter and a vertical spatial filter and is used for interpolation in interlace/progressive scan conversion of image signals, the temporal filter and the pre-positioned linear filter constitute a spatio-temporal filter whose pass characteristics can be controlled. Then, according to the rice milling that detected the inter-frame high-frequency components of the -(II times the order) and the correlation components in both the horizontal and vertical directions, a control signal related to the movement violet of the 1Ilii image represented by the image 1g is formed, and the control signal is An adaptive spatio-temporal filter characterized in that the pass characteristics of the spatio-temporal filter can be changed according to the movement of the image by controlling the pass characteristics of the spatio-temporal filter.