JPH0324838B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to an adaptive spatiotemporal filter used for interpolation adapted to an image signal during scan conversion of an image signal such as a television signal. In this way, it is possible to reproduce a high-quality image that adapts to the characteristics of the image signal.

A conventional standard television signal uses 2:1 interlaced scanning with 525 scanning lines, a field frequency of 60 Hz, and a frame frequency of 30 Hz, in the vertical direction, that is, the scanning line arrangement direction, and in the time axis direction, that is, the field and frame direction. It is a sampled image signal. Therefore, when such an image signal is expressed as a two-dimensional spectrum with vertical spatial frequency and temporal frequency as axes, as is well known, the sampled signal spectrum has a baseband spectrum that is equal to the sampling frequency and its harmonic frequencies. It exhibits a spectral structure that appears periodically and repeatedly on the coordinates.

In Fig. 1, which shows the frequency characteristics of a conventional spatio-temporal interpolation filter of this kind, point A is the center of the repetitive component closest to the baseband spectrum including the origin among the sampling frequencies appearing by 2:1 interlaced scanning. Since the baseband spectrum is centered around this frequency point A and is distributed around it, the baseband spectrum centered at frequency point A is superimposed on the baseband spectrum including the origin. In some cases, the signal becomes a false signal due to so-called aliasing, causing image disturbances such as interline flicker peculiar to interlaced scanning images, and significantly deteriorating the image quality.

In order to prevent the occurrence of such image disturbance, conventionally, the interference spectrum component that appears near the frequency point A in the frequency characteristic shown in FIG. This was removed by an interpolation filter, and the 2:1 interlace scan image was converted into a sequential scan image at 60 frames per second by interpolation. Although a considerable improvement in image quality has been obtained by taking such measures, the pass characteristics of the spatio-temporal interpolation filter with the configuration shown in FIG. This is a compromise between the optimal passing characteristics that are originally different between images and videos. Therefore, to put it another way, it has been difficult to obtain optimal passing characteristics for both still images and moving images using conventional spatio-temporal interpolation filters of this type. The reason for this drawback of the conventional spatio-temporal interpolation filter is that in the conventional spatio-temporal interpolation filter having the configuration shown in FIG. Each VLPF) 3 is constructed by combining one horizontal scanning period (1H) delay circuit as a unit delay element, and by changing the ratio of mixing the delayed output signals of each unit delay element, each For example, with respect to the shaded area in the frequency characteristics shown in Figure 1, the components in the time axis direction and the vertical spatial frequency axis direction are opposite to each other, as shown by the arrows in the figure. One example of this is that it increases and decreases in both directions.

On the other hand, when considering the distribution of the baseband spectrum of an image signal, for still images, there is no time axis component of the baseband spectrum, and the spectral component appears only on the vertical spatial frequency axis, so it can be used as an interpolation filter. It is desirable to have a pass characteristic in which the pass band is not limited in the vertical spatial frequency axis direction and the pass band is limited only in the time axis direction. However, for example, for a moving image in which a rectangular image moves to the right of the arrow as shown in FIG. 3a, the interpolated signal by the interpolation filter in the time axis direction is As shown in , image discontinuity occurs in the contour formed by the vertical sides of the rectangular image due to blurring in the time axis direction, significantly degrading the image quality. On the other hand, in the interpolated signal produced by the interpolation filter in the vertical spatial frequency axis direction, vertical blurring occurs in the contour formed by the horizontal sides of the rectangular image, as shown in Fig. Deterioration is less noticeable than blur.
Therefore, in the conventional spatio-temporal interpolation filter having the configuration shown in the second figure, the diagonal line shown in the first figure exhibits a passage characteristic intermediate between a temporal filter and a vertical spatial filter, so that it is suitable for both still images and moving images. area as the passing area. Therefore, although the interpolation filter effect is almost good for ordinary image signals obtained by ordinary television cameras, it does not work well for ordinary image signals, including high-definition television image signals and electronically constructed image signals. The drawback is that it is not a spatio-temporal interpolation filter that exhibits optimal pass characteristics for all types of image signals.

The purpose of the present invention is to eliminate the above-mentioned conventional drawbacks, exhibit optimal passing characteristics for both still images and moving images, and reproduce images of good quality without blurring the contours of still images. An object of the present invention is to provide an adaptive spatio-temporal filter that can

That is, the adaptive spatio-temporal filter of the present invention has two
A time filter that adds input and output image signals at both ends of a cascade of field memories, and a filter output signal of the time filter and an intermediate output signal of the cascade of the field memories as inputs, each of which has a plurality of The filter output signals of a vertical high-pass filter and a vertical low-pass filter are obtained by weightedly adding together sequential interstage delay output signals in a cascade connection of line-period delay elements, respectively. a space-time filter, the space-time filter comprising a filter output signal of the vertical spatial filter delayed and added to the intermediate output signal in the cascade connection of the field memories; The sequential inter-stage delay outputs in the cascade connection of the plurality of line period delay elements each forming a range filter are delayed equally to each other by successive pixel-by-pixel delays for at least one stage, and then a time difference component detection circuit which performs weighted addition to each other to obtain the respective filter output signals, and which detects and outputs the absolute value of the difference between the input and output image signals in the cascade connection of the field memories; and the vertical high-pass filter; Detecting the absolute value of the value obtained by adding the inter-stage delayed output signals, which are sequentially delayed pixel by pixel in the cascade connection of the plurality of line period delay elements of the vertical low-pass filter, with opposite polarities. A filter control circuit is provided, which includes a horizontal/vertical correlation component detection circuit as an output, and a motion discrimination processing circuit that outputs a coefficient control signal obtained by weightedly multiplying the detection output signals of these detection circuits as an image motion discrimination output, The weighted counts of the weighted addition in the vertical high-pass filter and the vertical low-pass filter are respectively controlled by the count control signal of the filter control circuit, thereby adapting to the motion of the image represented by the input pixel signal. This feature is characterized in that the transmission characteristics are changed by changing the transmission characteristics.

The present invention will be described in detail below with reference to the drawings.

First, an example of the basic configuration of the adaptive spatio-temporal filter of the present invention is shown in FIG. In the illustrated configuration,
It receives an interlace scan image signal as input and outputs a sequential scan image signal.The input image signal is supplied to a vertical filter 5 consisting of a 1H delay circuit and a time filter 6 consisting of a field memory, and the outputs of these filters 5 and 6 are The image signals are combined by an adder ADD4 to obtain a sequentially scanned image signal output. In addition, the filter control circuit 7
controls each filter 5 and 6 by outputting a filter coefficient control signal that adaptively changes filter characteristics for still images and moving images based on spatio-temporal image information from filters 5 and 6. In the figure, the solid line represents the flow of the image signal, and the dashed line represents the flow of the coefficient control signal.

The basic configuration of the filter control circuit 7 is explained in the fifth section.
As shown in the figure. In the filter control circuit with the illustrated configuration, the time difference component detection circuit 8 determines the amount of change in the image signal in the time axis direction, but since there is a response even for flickering images, it does not necessarily function as a moving image detection circuit. . Therefore, horizontal
The vertical correlation component detection circuit 9 detects the blurring of the contour of the vertical side of the rectangular image by the time interpolation filter shown in FIG. The time difference component detection output signal and the horizontal/vertical correlation component detection output signal are guided to the discrimination processing circuit 12 via weighting circuits 10 and 11, respectively, and their mutual products are calculated to perform the above-described coefficient control. By using a signal, the coefficient control signal controls the filter coefficient depending on both the amount of motion and the amount of outline blurring of the image.

The operating characteristics of the adaptive spatio-temporal filter of the present invention having the basic configuration shown in FIG. 4 will be explained with reference to FIGS. ,
This is as described above with reference to FIG. Furthermore, η is the coefficient control signal output value of the filter control circuit 7, which increases as the amount of image movement and the amount of outline blurring increases. , η = η _e and η = η _n .
Shown in b and c, respectively. However, η
The filter characteristic shown in FIG. 5A, where 0 is set, indicates the case of a still image, and by using the illustrated passing characteristic, resolution deterioration in the vertical direction due to interpolation is prevented. However, on the other hand, in the case of a moving image, as described above with reference to FIG. , as the amount of image movement increases, the filter characteristics are changed as shown in FIG. Due to such changes in filter characteristics, for example,
The blurred outline of the rectangular image described above will shift from the vertical sides shown in FIG. 3b to the horizontal sides shown in FIG. 3c. Therefore, the outline blurring of a rectangular image is visually much less noticeable when it occurs on its horizontal sides than on its vertical sides. By changing , the deterioration in image quality caused by the interpolation filter can be sufficiently removed.

Next, when converting a 2:1 interlaced scanning image signal of 60 fields per second to a progressive scanning image signal of 60 frames per second, the adaptive space-time of the present invention configured for interpolation based on the basic configuration shown in FIG. An example of the detailed configuration of the filter is shown in FIG. 7a, and its operation mode will be explained with reference to FIGS. 7b to 7e. Therefore, the configuration of the spatio-temporal filter of the present invention shown in FIG. 7a is obtained by adding a filter control circuit 7 to the conventional configuration shown in FIG. The vertical high-pass filter 2 and the vertical low-pass filter 3 are designed to obtain a filter output suitable for supplying to the horizontal/vertical correlation detection circuit 9 in the filter control circuit 7. It has been transformed into. Further, the filter control circuit 7 has the basic configuration shown in FIG. 5, but the time difference component detection circuit 8 has the following:
Image signal with 2 field difference or 1 frame difference
The absolute value α' of the difference signal ₃ between ₁ and ₂ is determined by an absolute value unit 13, and then multiplied by a weighting coefficient c by a coefficient unit 10 to form a time difference component α. That is, α′=| ₃ |=| ₂ − ₁ | (1) α=cα′ (2) On the other hand, in the horizontal/vertical correlation component detection circuit 9, the filter obtained from the vertical high/low band filters 2 and 3 The outputs k _{l1 and l2} are taken out as delayed outputs of each stage in a time-space transversal filter, which will be described later with reference to FIGS. 9 and 10, and correlation components between these delayed output image signals are formed.

Therefore, the vertical high and low pass filters 2 and 3
As shown in FIG. 8a, the delayed output signals k _l1 and l2 of each stage of the transversal filter forming These are pixel signal components corresponding to each point indicated by a circle in the figure, and FIG. 8b shows each of these points collectively in a matrix.

As mentioned above, the vertical high-pass filter (VHPF) 2 and the vertical low-pass filter (VLPF) 3 are shown in FIGS. 9 and 10, respectively.
As shown in the figure, 1H delay circuits 21-1 to 21-
m and the delayed output signal of each stage are transferred to n stages of pixel period τ delay circuits 22-n to 22-1, _t1 delay circuit 2
It consists of a transversal filter that is synthesized by an adder ADD ₅ through a coefficient unit A24 and a coefficient unit B25 sequentially. , vertical high and low pass filters 2,
The pixel period τ delayed output signals k _{l1, l2} of each stage in the transversal filter constituting the transversal filter 3
1) The synthesized output signal ν is alternately passed through the coefficient unit 26 and the coefficient unit (-1) 27, and then synthesized by the adder ₂₈ .・The vertical correlation component β′ is derived.

Therefore, the delay circuit 1 in the filter control circuit 7 of the spatio-temporal filter of the present invention shown in FIG.
2' and the _t1 delay circuit 23 in the vertical high/low pass filters 2 and 3 shown in FIGS. This is for adding a delay of t ₁ to the image signal.

Therefore, in the horizontal/vertical correlation component detection circuit 9 in the filter control circuit 7 in the spatio-temporal filter of the present invention having the configuration shown in FIG.
By applying arithmetic processing to the image signal using the configuration shown in the figure, the correlation component β' is detected as a blur that occurs in the vertical contour, such as the vertical side of the rectangular image mentioned above, and the detected output β' is as shown in FIG. The combined output signal ν _i of the adder 28-i at each stage in the horizontal/vertical correlation component detection circuit 9 described above is expressed as follows.

β′＝ _o 〓 ⁱ⁼¹ |ν _i | (3) ν _i = _n 〓 ^l=1 (−1) ^l-1 k _l,i (4) Also, the horizontal and vertical correlation components with the configuration shown in Figure 11 The coefficient unit (+1) 26 and the coefficient unit (-1) 27 in the detection circuit 9 are used to alternately invert the polarity of the delayed output image signals of each stage, and produce a composite value ν _i of the alternately inverted delayed output image signals. On the screen shown in FIG. 8a, the delayed output image signals k _1,i , k _2,i, ..., k _n,i of each stage constituting the detection image are arranged vertically over two fields. represents a point.
Therefore, the composite value ν _i of each stage of the alternately inverted delayed output image signals from each coefficient unit (+1) and (-1) in the configuration shown in FIG. 11 is 52 at the vertical spatial frequency.
It becomes a correlation component of 5/2 (cycle/screen height). On the other hand, since the contour blur on the vertical side of the rectangular image shown in FIG. 3b has a large vertical component of 525/2 (cycle/screen height), the above-mentioned composite value ν _i also shows a large value. The reason why this composite value ν _i is determined over several points in the horizontal direction as shown in FIG. 11, and the composite value β' is determined is to detect the amount of movement of the image in the horizontal direction. The faster the image moves, the larger the contour blur area on the vertical side becomes, so the composite value β', which is the sum of the absolute values |ν _i | of the composite value ν _i , also exhibits a large value.

In addition, in the configuration of FIG. 7a, the filter control circuit 7
The input signal γ of the control signal processing circuit 14 in
From equations (2) and (3) above, γ=α・β (5) α=cα′ β=d・β′ (6) Note that the coefficient d in equation (6) is a weighting coefficient for the horizontal/vertical correlation component β, and is multiplied by the coefficient unit 11 and weighted.

On the other hand, in a moving image with fast image movement, as shown in FIG. Axial interpolation results in conventional image quality degradation. That is, as shown in FIG. 7b, the control signal γ, which appears as an inter-frame difference, is generated as the difference between the first field and the third field, so if the movement of the image is extremely fast, Although an inter-frame difference signal occurs in the τ'' period shown in the figure, the signal level of the control signal γ becomes zero in the τ' period corresponding to the second field. The control signal processing circuit 14 in the filter control circuit 7 in the configuration example of the spatial filter is as follows:
The purpose is to generate the control signal γ even during the τ' period, and it can be constructed with a low-pass filter 16 as shown in FIG. 7c. The seventh low-pass filter 16 with impulse response length τ'
When the inter-frame difference signal γ having the waveform shown in Fig. 7b is applied, the impulse response is distorted as shown in Fig. 7c, as shown in Fig. 7b.
Therefore, the above-mentioned period τ' of zero coefficient control signal disappears.

However, the impulse response length τ' shown here is the maximum value that requires the correction shown in FIG. 7d.

However, when the control signal processing circuit 14 is constructed of a simple low-pass filter 16 as shown in FIG. A drawback arises in that an extra coefficient control signal is generated for a period τ before and after the interframe difference signal waveform shown in FIG. FIG. 7 d shows a configuration example of the control signal processing circuit 14 that eliminates this drawback and makes it possible to obtain a coefficient control signal throughout the period of the inter-frame difference signal γ shown in FIG. Figures a to g are shown in Figure 7e. In the control signal processing circuit 14 having the configuration shown in the figure, the inter-frame difference signal γ shown in waveform a is passed through a low-pass filter (LPF).
The output signal a of the delay circuit D17-1, which corrects the timing of the filter output corresponding to the waveform b obtained by guiding the waveform B to the input frame difference signal γ, and the filter output signal c are guided to the equivalent delay circuit D17-2. output signal b and the filter output signal c itself.
A synthesized output of waveform e is obtained by supplying it to the NAM circuit 18 and performing non-additive mixing, and the difference between the delayed output signal b and the added output signal d of the filter output signal c by the adder ADD ₄ is calculated by the subtracter SUB ₂ . The synthesized waveform shown in waveform f is obtained, and the synthesized output signal f is supplied to the underflow circuit 20, and the negative value input is made into a zero output, and the positive value input is directly taken out as an output value, and is sent to the NAM circuit 19. If non-additive mixing is performed with the delayed output signal b, a positive output signal level can be obtained even during the zero value period τ' of the input inter-frame difference signal γ, as shown in waveform g. A processed output control signal γ^ having an appropriate waveform that does not generate an extra positive value output signal in the region is extracted. However, the delay time of delay circuits 17-1 and 17-2 is
Take it equal to the impulse response length of LPF16.

Next, the coefficient control signal η obtained from the determination/control circuit 15 which receives the processed output control signal γ^
_The input/output characteristics of the input signal γ are as shown in Fig. 12, and the output signal η changes as _{η 1 , η 2 ,} . ..., η _n , and with the coefficient control signal η of such value,
Coefficient value a to be multiplied by the delayed image signal of each stage in the coefficient units 24 _i and 25 _i of the vertical high-pass filter (VHPF) and low-pass filter (VLPF) shown in FIGS. 9 and 10, respectively. Vary _i and b _i . Note that although the coefficient control signal η can generally be a continuous value, it is preferable to use a discrete value when implementing it in hardware, and even if the filter coefficient value is continuously changed, the effect will not change. Since the number is small, it is usually sufficient to use discrete values. The input/output characteristics shown in Figure 12 can be easily realized using a comparator, and the relationship between the coefficient control signal η and the change in the characteristics of the spatio-temporal interpolation filter is shown in Figures 6 a to c.
Configure as shown in . In other words, when η=0, the still image filter characteristics are used, so the spatio-temporal interpolation filter is a time filter consisting only of field memory, and when η>0,
By changing the filter characteristics shown in b to c of the same figure,
Each parameter of the filter control circuit 7 is set so that the blurring of the image outline shown in FIG. 3b or c matches the visual characteristics and deterioration of the image quality is reduced. For the filter characteristics shown in Fig. 6a, the coefficient value b _i of the vertical low pass filter (VLPF) 3 is set to 0, and the vertical high pass filter (VHPF) 2 is passed through. For the filter characteristics shown in , the coefficient value a _i =0 of the vertical high-pass filter (VHPF) 2 is set to act as a temporal filter and a vertical spatial filter, respectively.

In the above explanation, an example was described in which 2:1 interlaced scanning is converted to sequential scanning, but the configurations shown in FIGS. 4, 5, and 7a convert from multiple interlaced scanning to sequential scanning. It can also be extended and applied when converting to Great effects can be expected by applying an adaptive spatio-temporal interpolation filter to television broadcast receivers.

As is clear from the above description, according to the present invention, as an interpolation filter used when converting an interlace scan image signal to a progressive scan image signal,
It is possible to construct an adaptive spatio-temporal filter that exhibits good interpolation characteristics for both still images and moving images, and it is possible to reproduce progressively scanned images without blurring the contours of still images. That is,
By controlling the filter characteristics according to the results of detecting all the time difference components and horizontal and vertical correlation components of the image signal, it can be used regardless of whether it is a still image or a moving image.
Regardless of the amount of image movement, the best interpolation effect can be achieved, the optimal filter characteristics that suit the nature of the image can be set at any time, and the configuration of the interpolation filter can be simplified. can.

Note that the spatio-temporal filter of the present invention has the same good effect as an interpolation filter not only for conversion from a 2:1 interlace scan image signal, but also for conversion from a multiple interlace scan image signal to a sequential scan image signal. It is also effective for low transmission rate, high-quality television systems where the transmitter/receiver system is in a sequential scanning format and the transmission system is in an interlaced scanning format, so that compatibility between the scanning formats can be achieved on the receiving side. Can be applied.

[Brief explanation of drawings]

Fig. 1 is a characteristic curve diagram showing the frequency characteristics of a conventional spatio-temporal interpolation filter, Fig. 2 is a block diagram showing the configuration of the spatio-temporal interpolation filter, and Figs. 3 a to c show contour blurring due to the image interpolation filter. 4 is a block diagram showing the basic configuration of the adaptive spatio-temporal filter of the present invention, and FIG. 5 is a diagram showing the basic structure of the filter control circuit in the adaptive spatio-temporal filter. FIGS. 6 a to 6 c are characteristic curve diagrams sequentially showing examples of changes in filter characteristics of the adaptive spatiotemporal filter, and FIGS. 7 a to e are block diagrams showing the adaptive spatiotemporal filter. A block diagram and a waveform diagram showing examples of the detailed configuration of the spatio-temporal filter, the form of filter control signal formation, the configuration of the processing circuit, and the operation waveforms of each part, respectively, and FIGS. 8a and 8b are for the adaptive spatio-temporal filter. Diagrams each showing an example of the configuration and arrangement of image signal detection points used as the basis for forming a characteristic control signal, and FIG. 9 is a block diagram showing an example of the detailed configuration of a vertical high-pass filter in the adaptive spatio-temporal filter. , FIG. 10 is a block diagram showing an example of the detailed configuration of the vertical low-pass filter in the adaptive space-time filter, and FIG. 11 is a block diagram showing the detailed structure of the vertical low-pass filter in the adaptive space-time filter.
FIG. 12 is a block diagram showing an example of the detailed configuration of the vertical correlation component detection circuit, and a characteristic curve diagram showing an example of the input/output characteristics of the determination/control circuit in the adaptive spatiotemporal filter. 1-1, 1-2...field memory, 2...
Vertical high-pass filter, 3... Vertical low-pass filter, 4, 12'... Delay circuit, 5... Horizontal/vertical filter, 6... Time filter, 7... Filter control circuit, 8... Time difference component Detection circuit, 9...
Horizontal/vertical correlation component detection circuit, 10, 11... Weighting circuit, 12... Movement discrimination processing circuit, 13...
...Absolute value unit, 14...Control signal processing circuit, 15...
...Judgment control circuit, 16...Low-pass filter, 17-
1, 17-2... delay circuit, 18, 19...
NAM circuit, 20...underflow circuit, 21
-1 to 21-m...1H delay circuit, 22-i...
τ delay circuit, 23-1 to 23-n... _t1 delay circuit, 24-1 to 24-n, 25-1 to 25-n...
...Coefficient unit, 26-i, 27-i...Coefficient unit, 28
-1 to 28-n...Adder, 29-1 to 29-n
...Absolute value unit, ADD ₁ to ADD ₆ ...Adder,
SUB ₁ , SUB ₂ ...Subtractor, MUL...Multiplier.

Claims (1)

[Claims] 1. A time filter that adds input and output image signals at both ends of a cascade connection of two field memories, and a filter output signal of the time filter and an intermediate output in the cascade connection of the field memories. The filter output signals of a vertical high-pass filter and a vertical low-pass filter, each of which is obtained by weighted addition of sequential interstage delay output signals in a cascade connection of a plurality of line period delay elements, are mutually input. A space-time filter comprising a vertical spatial filter that adds the sum to obtain a filter output, and adds the delayed intermediate output signal in the cascade connection of the field memories to the filter output signal of the vertical spatial filter, Sequential inter-stage delay outputs in the cascade connection of the plurality of line period delay elements constituting the high-pass filter and the vertical low-pass filter, respectively, are sequentially delayed in pixel units for at least one stage, respectively. A time difference component detection circuit that delays each other equally and adds weight to each other to obtain the respective filter output signals, and outputs the absolute value of the difference between the input and output image signals in the cascade connection of the field memories. and adding the inter-stage delayed output signals, which are sequentially delayed pixel by pixel in the cascade connection of the plurality of line period delay elements of the vertical high-pass filter and the vertical low-pass filter, with opposite polarities to each other. A horizontal/vertical correlation component detection circuit whose detection output is the absolute value of the obtained value, and a motion discrimination processing circuit whose image motion discrimination output is a coefficient control signal obtained by weighted multiplication of the detection output signals of these detection circuits. A filter control circuit is provided, and the weighted counts of the weighted addition in the vertical high-pass filter and the vertical low-pass filter are respectively controlled by the count control signal of the filter control circuit. 1. An adaptive spatio-temporal filter characterized in that its passage characteristics are changed in accordance with the movement of an image represented by a signal.