JPH0444488A - Encoder for television signal - Google Patents

Abstract

PURPOSE:To reduce fog, glare and flicker of a video image at a decoder side by providing a filter in a direction of time applying band limit to a still picture signal in the direction of time and varying a delay time of a delay means with its moving vector. CONSTITUTION:The encoder is provided with a filter 6 in a direction of time by using a delay means 24 and adding a prescribed time difference to a video signal so as to apply band limit in the direction of time to the still picture signal. Thus, invasion of a component with a high frequency in the direction of time in the still picture signal is prevented and production of reflected wave distortion or the like is not caused during the processing of the still picture signal. Moreover, when a pattern itself is moved regularly, the delay time of the delay means 24 of the filter 6 is changed in response to the moving vector, Thus, the contrast of the still picture signal is kept to be almost that of the original signal and a picture with high resolution is reproduced by the correction at the decoder side by means of the moving vector.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention is applicable to, for example, a MUSE type encoder and an EDT.
The present invention relates to a television signal encoding device suitable for application to a V-scheme encoder and the like.

[Summary of the Invention] The present invention divides the video signal of each pixel into a still image signal with a wide transmission band in the spatial frequency domain and a video signal with a narrow transmission band in the spatial frequency domain, and divides the still image signal into multiple fields or a video signal with a narrow transmission band in the spatial frequency domain. In a television signal encoding device that transmits divided frames into multiple frames and also detects and transmits a motion vector indicating the movement of the entire screen when the entire screen is moving regularly, using a delay means. By providing a filter in the time direction that limits the band in the time direction of the still image signal by adding the video signals with a predetermined time difference, and by changing the delay time in the delay means according to the motion vector, For example, when panning a stationary object, it is possible to reduce the blurring of the image on the decoder side of the stationary object, and it is also possible to reduce the glare and flickering of the image on the decoder side of the stationary telop on the screen at the time of panning. This is what I did.

[Prior Art] In order to increase the image resolution and improve the image quality in a television receiver (decoder), it is necessary to widen the transmission band from the encoder to the decoder in the spatial frequency domain of the video signal. There is a limit to the expansion of the transmission band in transmission channels such as satellite broadcasting.

For this reason, for example, the MUSE method is used to transmit high-definition signals over one channel of satellite broadcasting, and EDTV and ACT are used to improve image quality using transmission paths such as the VHF band.
In V, etc., each pixel is divided into a still pixel with a low temporal frequency (temporal frequency) of the corresponding video signal and a video pixel with a high temporal frequency, and the still pixel is divided into a wide spatial frequency over multiple fields. The transmission band of the transmission path is substantially expanded by transmitting a video signal of a relatively narrow band for moving pixels.

Also, if the camera is flashing, the screen will be processed as a video and the resolution will be poor, but since the human line of sight moves to follow the image of a specific object on the screen, it will be processed as a still image. The same level of resolution is required. Therefore, in the MUSE method, for example, the encoder side detects a motion vector indicating the amount of motion of the entire west plane and transmits it to the decoder side, and the decoder side uses the motion vector to obtain a resolution comparable to that of a still image.

Furthermore, since processing is performed on the assumption that the temporal frequency of still pixels is low, and it is necessary to reduce aliasing distortion in the temporal direction, a low-pass filter (temporal filter) in the temporal direction is provided at the initial stage of encoding. This is being considered.

Figure 4 shows an example in which a temporal filter is added to a conventional MUSU encoder. In Figure 4, the three primary color signals R, G, and B of a high-definition camera, etc. are each passed through a band-limiting low-pass filter (IR). )～(IB)
, A/D converters (2R) to (2B) with a sampling frequency of 48.6MHz, and a correction circuit for inverse γ correction (3R)
~ (3B) is supplied to the matrix circuit (4),
A luminance signal Y and color difference signals RY and BY are generated from this matrix circuit (4).

Of these, the color difference signals R-Y and B-Y are subjected to static motion adaptive processing and time compression processing in the color difference signal processing circuit (5), and the signals after this processing are supplied to the transmission circuit side (not shown). be done. On the other hand, the luminance signal Y is supplied to a temporal filter (6), an intra-field blurring filter (15), and a motion vector detection circuit (17), which will be described later.

The transmission band during this original sampling is 1125 TV lines in the vertical direction and 24.3 TV lines in the horizontal direction, as shown in Figure 5A.
MHz. In addition, the field frequency is 60Hz
Therefore, according to the Nyquist condition, the transmission band in the time direction is 30 Hz.

The temporal filter (6) includes a frame delay circuit (7) consisting of a frame memory that delays the input signal by one frame, an adder circuit (8), and a multiplier circuit (9) that reduces the input signal by half. Consists of cascading connections,
The input and output of the frame delay circuit (7) are added by an adder circuit (8) and supplied to the multiplier circuit (9). Therefore, if the luminance signal Y of the nth field is expressed as Y (n) and this luminance signal Y (n) is supplied to the temporal filter (6), the luminance output from the frame delay circuit (7) The signal becomes the luminance signal Y(n-2) of the (n-2)th field, and the luminance signal Y of the nth field output from the temporal filter (6).
T (n) is YT (n) = (Y (n) + Y (n-2))
/2...(1) It can be expressed as follows. This temporal filter (6) limits the temporal transmission band of the luminance signal YT (n) to approximately <7.5 Hz as shown in FIG. 5B.

This luminance signal YT (n) is further processed by an interfield filter (10) having the characteristics shown in FIG. A low-pass filter with an off frequency of 12 MHz (1
2) to one input section of a multiplier circuit (13), and the output of this multiplier circuit (13) is supplied to an adder circuit (14). The luminance signal supplied to this adder circuit (14) can be considered to be a still image signal whose temporal band is approximately 7.5 Hz or less.

In addition, a video signal with a temporal frequency up to 30 Hz in the luminance signal Y is processed using an intra-field bris filter (15
), the output of this multiplier circuit (16) is supplied to an adder circuit (14), and the weighted mixed output of this adder circuit (14) is sent to the transmitting circuit side. is supplied to As is clear from FIGS. 5C and 5D, the transmission band for still images is twice as large in the vertical direction and about 3/2 times as large in the horizontal direction as compared to the transmission band for moving images. and,
The transmission circuit (not shown) subjects the weighted mixed output to frame offset subsampling (FO3) or line offset subsampling (LO3).

Therefore, for still pixels, the MUSE encoder sends the data of 1/2 sample points of the image data of one frame in one frame period, and in the next frame period, the data of the next one A different one of the frame's image data
/2 sample point data will be transmitted. On the other hand, for video pixels, the encoder transmits data of 1/2 sample points of the image data of each field during each field period. Correspondingly, on the encoder side, for still pixels, 2 frames of image data are combined to reproduce 1 frame image, and for video pixels, 1 field of image data is synthesized from 1 field of image data depending on the inner field width. Reproduce.

Therefore, as a whole, the sampling frequency of the video signal of a still pixel is 15H2, and the sampling frequency of the video signal of a video pixel is 60Hz. Therefore, in order to suppress the occurrence of aliasing distortion according to the Nyquist condition, for the still image signal, It is necessary to limit the transmission band to 7.5Hz using a temporal filter (6), and it is necessary to limit the transmission band to 30H2 for video signals, but since the transmission band of normal video signals is about 308Z or less, No temporal filter is provided for the signal.

(17) is a motion vector detection circuit that calculates the brightness level of the pixel at coordinates (i, j) of the previous frame by y (i.

j), the luminance level of a pixel shifted by (m, n) from the same position in the current frame is x(i+m).

j 10n), this motion vector detection circuit (17
) is l x (i+m, j+n) −y (i, j
) with a predetermined weight added to the total area of the west face, which is the number 10.
Calculate for 00 points. When the sum is the minimum (
m, n) are motion vectors, and the level of their sum is motion information. Specifically, when the sum can be considered to be approximately 0, it is a completely still image (more precisely, it is a completely translated image), when the sum is large enough, it is a quasi-still image (quasi-parallelly translated image), and when the sum exceeds a predetermined level, it is a completely still image. This becomes a normal moving image (normal image), and in the case of a normal image, the decoder side does not use the motion vector.

At the stage of transmission to the decoder, the motion vector is composed of a 4-bit horizontal component and 3-bit vertical component, and 1 horizontal bit corresponds to 1 period of 32 MHz (rlcKJ). ), one bit in the vertical direction corresponds to the interval (IH) between the horizontal lines, and the positive direction is when the west face moves to the right and when it moves downward, respectively. The two components are as follows.

Horizontal component Knee 8 to +7 CK/frame Vertical component Knee 4 to +38/field Therefore, the vertical component is detected at an interval of one frame, converted into a change in interval of one field, and transmitted.

The motion vector and motion information detected in the motion vector detection circuit (17) are supplied to a motion area detection circuit (18) and a control signal encoder (19), and the motion area detection circuit (18) detects the luminance of the current frame that is supplied. The degree of movement is detected for each pixel using the difference between the signal and the luminance signal of the previous frame, and a motion coefficient signal whose value gradually changes from 0 to 1 corresponding to the range from a completely still pixel to a moving pixel is generated. generate. This motion coefficient signal is fed directly and via an inverter (20) to multiplication circuits (16) and (13), respectively. Therefore, in the case of a still pixel, the adder circuit (1
4) outputs a still image signal, in the case of a video pixel, an adder circuit (14) outputs a video signal, and for pixels in the intermediate area, a weighted mixed signal of the still image signal and a video signal is sent to the adder circuit. (14) is output.

However, when the motion information supplied by the motion vector detection circuit (17) corresponds to a still state such as a completely still image or a semi-still image, the motion area detection circuit (18) converts the motion coefficient signal in the direction corresponding to the still pixel. (i.e., in the direction of 0).As a result, when the camera is panning, even if the movement of each pixel is determined to be a moving image,
The adder circuit (14) mainly outputs still image signals with high resolution.

Further, the motion vector and motion information are supplied from the control signal encoder (19) to the transmission circuit side, and thereby the motion vector and motion information are also supplied to the decoder side.

When the camera is panned, motion information corresponding to a completely still image, two components of a motion vector, and a high resolution video signal corresponding to still pixels are transmitted from the decoder to the encoder side. For each transmitted field (1
Assume that the image corresponding to the video signal (at an interval of /60 seconds) is an image that moves approximately parallel to the right by α per field, as shown in FIG. At this time, assuming that the current field is at a position of 3/60 seconds on the time axis, the decoder side recognizes from the motion information that the image of the current field is a completely still image (completely parallel movement WM), The final high-resolution image data of the current field is restored from the image data of the previous four field images (21) in FIG.

In this case, these four images (21) are in a relationship where they are moving parallel to each other, so if they were simply combined, a 2M image etc. would be generated, so the motion vector supplied from the encoder side would be horizontal. If the direction is α/field, then on the decoder side it is 0.1/60.2/6 on the time axis.
Each field image at the 0 second position is horizontally 3α
, 2α, and α, and these translated images and a field image located at a position of 3/60 seconds are combined. Similarly, to obtain a field image at the 5760 second position (not shown), the image data of the previous four field images (22) are combined. For images panned in this way, the encoder can restore the original image at high resolution in the same way as still images.

In addition, in Figure 6, (23) indicates a subtitle for breaking news, weather forecast, etc.
) information is, for example, the camera and matrix (4) in the example in Figure 4.
), so it remains completely stationary on the screen even when panning.

[Problem to be solved by the invention] When performing correction using a motion vector as described above,
Panning the camera should not reduce the resolution, but in reality, depending on the image, the resolution may be worse than before panning, resulting in some so-called blurring. Possible causes of this blur include the characteristics of the camera itself and the fact that the values of the transmitted motion vector are discrete.
This is also due to the reasons explained below.

That is, as shown in FIG. 7, assuming that the horizontal spatial frequency of the image to be panned (the output of the television camera) at time t is 0 is, for example, 2.7 MH2 (period: 375 ns), the image is horizontally It is assumed that panning is performed at 125 ns/frame. Since the original sampling frequency is 48.6 MHz, its 375 ns is 1
It corresponds to a length of B (=48.6/2.7) pixels.

Also, since there are approximately 1440 pixels in the horizontal direction, the panning of 125 ns (6 pixels)/frame is fast enough to switch the entire horizontal screen in 8 (=1440/6/30) seconds. handle.

Furthermore, as is clear from Fig. 7, the video signal in this case is 37
Since the phase is reversed every 60 seconds, the period is essentially 6/6.
It vibrates in 0 (1/10) seconds, that is, at a frequency of 10 Hz. Therefore, the video signal is 7.5 in Figure 4.
This signal is removed by the temporal filter (6) for removing signals above about Hz, and only a small amount of the video signal whose band is greatly limited by the intra-field filter (15) is transmitted to the decoder side. Even if the decoder performs correction using a motion vector, the original high-resolution image cannot be restored.

To explain this in more detail with reference to FIG. 8, in this case, the luminance signals Y (n) and (n-2) of the nth frame in the temporal filter (6) of FIG.
The luminance signal Y(n-2) of the frame is shown in FIGS. 8B and A, respectively.
It becomes as shown in. Therefore, its temporal filter (6
) is the output YT (n) of the average value of the two luminance signals, as shown in Figure 8C, the output YT (n) is the human power signal Y
(n), and the contrast of the image becomes ambiguous. Even if the encoder side corrects and synthesizes the four-field luminance signals with reduced contrast using the α/field motion vector as shown in FIG. 9, the contrast of the resulting luminance signal remains poor.

As mentioned above, if you pan an image that has a fine structure in that direction, it will essentially vibrate at a high frequency, and this will limit the transmission band in the time direction for the still image signal. Because the temporal filter (6) blocks signals in the band that should originally be passed during panning, blurring occurs in the decoded image even if correction using motion vectors is performed.

Conversely, the signal of an image that is stationary on the screen during panning, such as the telop (23) in Figure 6, passes through the temporal filter (6) in Figure 4 and is transmitted to the decoder side. However, when the decoder side performs correction using motion vectors, the signals of images such as the telop (23) are synthesized with their phases shifted, which may result in double images or the like.

In other words, if the horizontal frequency of the image such as the telop (23) is the same as the waveform at the time of the example in FIG. However, such high frequency signals may cause aliasing distortion due to the thinning process during interpolation in the encoder.
As a result, there is a possibility that flickering, glare, etc. may occur in the reproduced image.

In view of the above, the present invention provides an image transmission system that performs correction using motion vectors. For example, when panning a stationary object, it is possible to reduce the blurring of the reproduced image on the decoder side of the stationary object, and to It is an object of the present invention to reduce glare and flickering of a reproduced image on the decoder side of a stationary telop or the like on a screen.

[Means for Solving the Problems] The television signal encoding device according to the present invention converts the video signal of each pixel into a still image signal having a wide transmission band in the spatial frequency domain and a still picture signal having a wide transmission band in the spatial frequency domain, as shown in FIG. The still image signal is divided into multiple fields or multiple frames and is transmitted separately from the video signal, which has a narrow transmission band, and when the entire screen is making regular movements (e.g. panning, zooming, etc.), the entire screen is transmitted. In a television signal encoding device that detects and transmits a motion vector indicating the motion of a still image signal, the video signal is added with a predetermined time difference using a delay means (24) to limit the bandwidth of the still image signal in the time direction. A filter (6) in the time direction is provided, and the delay time in the delay means (24) is corrected in accordance with the motion vector.

[Operation] According to the present invention, since the filter (6) in the time direction is provided, a component with a high frequency in the time direction is prevented from being mixed into the still image signal, which is originally a component with a low frequency in the time direction. This prevents aliasing distortion from occurring during processing of the still image signal.

Furthermore, when the entire screen moves regularly, each pixel is generally determined to be a video pixel, but on the decoder side, motion vectors are used to stack multiple fields or multiple frames of still image signals while shifting their phases. By doing so, it should be possible to reproduce images with high resolution.

However, if the filter (6) in the time direction is operated regularly, when the entire screen moves regularly, the contrast of the still image signal becomes small due to the addition in the filter (6), and the decoder Even if correction is performed on the side using motion vectors, it will not be possible to reproduce a high-resolution image.

As a countermeasure against this, in the present invention, when the entire screen moves regularly, the delay means (24) of the filter (6)
) is changed according to the motion vector. Therefore, for example, when the entire screen is moving to the right, the delay time in the delay means (24) is shortened and the previous video signal is substantially moved to the right, so that the still image The contrast of the signal can be maintained at the same level as the original signal, and if the decoder side performs correction using the motion vector, it is possible to reproduce a high-resolution image.

On the other hand, for an image that is partially still while the entire screen is moving regularly, such as a subtitle during panning, the delay time in the delay means (24) may be changed according to the motion vector. Therefore, the contrast of the still image signal becomes small. Therefore, even if the decoder side processes a signal with a small contrast by correcting it using a motion vector, there is no noticeable deterioration in image quality.

[Embodiment] Hereinafter, an embodiment of the present invention will be explained with reference to FIGS. Portions corresponding to those in FIG. 4 are designated by the same reference numerals, and detailed explanation thereof will be omitted.

FIG. 1 shows the encoder of this example, and in this FIG. 1, the sampling frequency is 48.6 from the matrix (4).
Luminance signal Y and color difference signals R-Y, B- obtained by processing three primary color signals that have been A/D converted at MHz and inversely corrected.
Y is output, and this color difference signal is supplied to a static-motion adaptive processing circuit and a transmission circuit (not shown). Further, the luminance signal Y is supplied to a temporal filter (6) for still image signals, an intra-field blurring filter (15) for moving image signals, a motion vector detection circuit (17), and a motion area detection circuit (18).

In the temporal filter (6) of this example, (24) is a frame delay circuit made of an image memory having a capacity larger than the luminance signal for E frames, and (25) specifies the read address of this frame delay circuit (24). address change circuit, (8) is an adder circuit, (9) is an input with 1
In this example, the motion vector MV and motion information output from the motion vector detection circuit (17) are supplied to the address change circuit (25), and this address change circuit (25) Frame delay circuit (24)
Normally, the time difference between the input and output of the frame delay circuit (24) is one frame, but as will be described later, during panning etc., the read address of the frame delay circuit (24) is changed according to the motion vector. The time difference changes depending on the size. Therefore, its frame delay circuit (
24) is the luminance signal Y (n) of the nth field, the luminance signal output from the frame delay circuit (24) is the luminance signal of the (n-2)th field. Y(
n-2+β).

The two luminance signals Y (n) and Y (n-2+β) are input to an adder circuit (8), and the output of this adder circuit (8) is supplied to a multiplier circuit (9). The output of this multiplication circuit (9) becomes the luminance signal YT (n) of the nth field output from the temporal filter (6), that is, YT (n
) = (Y (n) +Y (n-2+β))/2・
...(2) holds true. Therefore, this temporal filter (6
) can be considered a two-tap transversal filter. This luminance signal YT (n) is supplied to an interfield blurring filter (10) for still image signals. The other configurations are the same as the example shown in FIG.

To explain the panning operation of this example with reference to FIG. 2, in this example as well, similar to the example of FIG. It is assumed that panning is performed at 125 ns/frame. In this case, the luminance signal Y of the nth field
If (n) is as shown in FIG. 2B, the luminance signal Y(n-2) of the (n-2)th field, which is a signal obtained by delaying the luminance signal by one frame, is as shown in FIG. 2A. As shown by the broken line, it is delayed by β (=125ns) in the horizontal direction (time direction) with respect to the luminance signal Y (n).

Therefore, in the temporal filter (6) in FIG. 1, the address change circuit (25) delays the read address of the frame delay circuit (24) with respect to the write address by a time equal to two fields minus time Correction is performed using motion vectors.

At this time, the phase of the luminance signal Y (n-2+β) read out from the frame delay circuit (24) becomes equal to the phase of the luminance signal Y (n), as shown by the solid line in FIG. 2A. Therefore,
Temporal filter (6) of this example determined by equation (2)
As shown in Figure 2C, the contrast of the luminance signal YT(n) outputted from the transmitter is the same as that of the original luminance signal Y(n), and this high-contrast luminance signal is directly transmitted to the transmitting circuit (not shown). The signal is supplied to the decoder side via the signal.

Therefore, on the decoder side, by replacing α with β in FIG. 6 and performing correction using the motion vector, there is an advantage that the panned image can be reproduced with high resolution.

Next, we will explain the processing of an image that is stationary on the screen during panning (for example, the telop (23) in the example in Figure 6). In this case, the luminance signal Y (n) of the still image of the nth frame is Assuming that it is as shown in Figure 3B, the number (
The phase of the luminance signal Y(n-2) of frame n-2) is the third
As shown in FIG. A, this is the same phase as the luminance signal Y (n). In this case, in the temporal filter (6) of FIG. 1, the luminance signal Y(n-2+β) is read out from the frame delay circuit (24) by the action of the address change circuit (25); 2+β) is ahead of the phase of the luminance signal Y (n) by β, as shown by the solid line of the third [ilA.

Therefore, the temporal filter (6) determined by equation (2)
As shown in FIG. 3C, the contrast of the luminance signal YT (n) that is the output of is worse than that of the original luminance signal Y (n), and this low contrast signal is transmitted to the decoder side. Then, on the decoder side, the phase of the low contrast signal is changed to β/
Even if the signals are synthesized while shifting each frame, the contrast of the resulting signal remains low.

Therefore, even if the frequency of the signal in the time direction exceeds the upper limit of 7.5 Hz for still image signals, aliasing distortion etc. will not be noticeable in subsampling during subsequent interpolation, and so-called glare and flickering will be reduced. There is a benefit of improved image quality. In this case, the contrast is low only in the outline of the telop, etc., and the telop etc. itself is reproduced satisfactorily.

In this regard, for pixels that are stationary on the screen,
When correction is performed using a motion vector, the pixel is determined to be a moving pixel by motion area detection on the decoder side, so a moving image signal based on intra-field interpolation is mainly used. Therefore, a relatively good reproduced image can be obtained. Furthermore, for images that remain stationary on the resurface, such as subtitles during panning,
On the encoder side, the delay time in the frame delay circuit (24) may be forcibly set to one frame, and on the decoder side, correction using the motion vector may not be performed.

Further, although the above-described embodiment is realized by a circuit configuration in which individual circuits are connected, it can be easily realized by software such as a microprocessor.

It should be noted that the present invention is not limited to the above-mentioned embodiments, but may be applied to, for example, full-screen zooming and rotation, and may be applied to an EDTV encoder, etc., and various configurations may be adopted without departing from the gist of the present invention. Of course.

[Effects of the Invention] According to the present invention, for example, when panning a stationary object, it is possible to reduce the blurring of the reproduced image on the decoder side of the stationary object, and it is also possible to reduce the blurring of the reproduced image on the decoder side of the stationary object, and to reduce the blurring of the reproduced image on the decoder side of the stationary object. This has the advantage of reducing glare and flickering of the reproduced image on the side.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing the main parts of an encoder according to an embodiment of the present invention, and FIGS. 2 and 3 are timing charts for explaining the operation of a temporal filter according to the embodiment, respectively.
Fig. 4 is a block diagram showing a conventional MUSE type encoder, Fig. 5 is a diagram showing the transmission bands of each filter, etc. in the example shown in Fig. 4, and Fig. 6 is a diagram showing motion vector correction in a conventional decoder during panning. Diagrams for explanation: FIG. 7 is a timing chart showing the output of the TV camera on the encoder side during panning; FIG. 8 is a timing chart for explaining the operation of a conventional temporal filter; FIG. 9 is a timing chart for explaining the operation of a conventional temporal filter FIG. 6 is a timing chart diagram illustrating motion vector correction on the decoder side. (6) is a temporal filter, (8) is an adder circuit, (9
) is a multiplication circuit, (10) is an interfield brifilter,
(15) is an intra-field bris filter, (17) is a motion vector detection circuit, (24) is a frame delay circuit, (2
5) is an address change circuit.

Claims (1)

[Claims] The video signal of each pixel is divided into a still image signal with a wide transmission band in the spatial frequency domain and a video signal with a narrow transmission band in the spatial frequency domain, and the still image signal is divided into multiple fields or multiple frames. In a television signal encoding device that detects and transmits a motion vector indicating the movement of the entire screen when the entire screen is moving regularly, the television signal encoder detects and transmits a motion vector indicating the movement of the entire screen. The present invention is characterized in that a time-direction filter is provided that limits the time-direction band of the still image signal by adding the video signal, and the delay time in the delay means is changed in accordance with the motion vector. equipment for encoding television signals.