JPH0548030B2 - - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to an image signal interpolation method for restoring the frequency band of an image signal subjected to band compression through sampling, particularly subsampling, and in particular to an image signal interpolation method for restoring the frequency band of an image signal subjected to band compression by subsampling. It is designed to prevent this.

Generally, when transmitting wideband image signals such as high-definition television signals, it is extremely important to make the frequency band of the transmission path required for transmission as narrow as possible from the standpoint of effective use of the transmission band. .

Therefore, in order to reduce the required bandwidth of the transmission path, one of the band compression methods applied to the image signal to be transmitted is to utilize the inter-frame correlation of the image signal, and to compress the image signal in pixel units. There is a band compression method that sub-samples the signal and transmits it. Among such pixel-based subsampling, 2:1 subsampling, which samples every other pixel signal in the same way as so-called 2:1 dot interlacing, is the simplest example. As shown in the figure, if every other pixel signal is subsampled on every other scanning line that has been main sampled using interlaced scanning, the restoration of the original image signal can be completed using two frames worth of sample pixel signals. Therefore, in principle, the transmission bandwidth required for transmitting image signals subjected to such subsampling for successive frames can be halved.

However, in order to restore the frequency band of an image signal transmitted by halving the required frequency band by such 2:1 subsampling, a frame memory is usually provided and each frame is sequentially For each missing pixel signal, the corresponding pixel signal in the previous frame temporarily stored in the frame memory is read out and replaced. However, although such alternative interpolation of pixel signals can completely restore the original image for still images, it cannot restore changes due to the movement of the same pixel point between two adjacent frames for moving images. This results in the generation of false signals, making it impossible to faithfully restore the original motion image.

On the other hand, for moving images, even if there is considerable blurring in the moving parts of the image, the visual sense cannot sufficiently follow the movement of the moving parts, so visually the image quality does not deteriorate that much. It has been common practice to make use of visual characteristics to interpolate missing pixel signals in the moving part due to subsampling with adjacent pixel signals in the same field. Conventionally, it has been demonstrated that such pixel interpolation can be applied to motion compensated interframe coding devices and the like to obtain considerable effects.

However, such conventional pixel interpolation has the following specific problems. In other words, (1) new information is added to the restored image by switching between pixel interpolation by interpolation using interframe correlation and pixel interpolation by interpolation between adjacent pixels within the same field, depending on the presence or absence of movement in the image. This creates an unnatural feeling.

(2) If the detection of moving parts of an image were performed simply according to the magnitude of the difference between frames, for example, in edge parts where the high-frequency components of the image signal increase, it would be difficult to detect small movements in the image. Motion detection is incomplete, with large inter-frame differences often occurring, resulting in detection results equivalent to large motion in the image, and new image quality deterioration occurs as a result of such erroneous detection of motion.

Therefore, in all conventional image signal interpolation methods, it is difficult to sufficiently restore the frequency band of the original image signal by performing pixel interpolation to obtain a sufficiently high-quality image. is difficult,
There was a drawback.

The object of the present invention is to solve the above-mentioned conventional problems and eliminate their drawbacks, and to perform pixel interpolation using an appropriate interpolation signal without introducing new unnaturalness.
An object of the present invention is to provide an image signal interpolation method that can restore the frequency band of an original image signal.

That is, in restoring the frequency band of an image signal whose required frequency band has been reduced by sampling at a ratio of at least 2:1, the image signal interpolation method of the present invention extracts the motion of the image represented by the image signal by at least two frames. The absolute value of the difference between two frames between the pixel of interest that differs in period, the absolute value of the difference between pixels created by the pixel of interest and pixels adjacent in the line direction in the same field, and the adjacent pixels in a quincunx relationship with the pixel of interest. Detection is performed based on the absolute value of the difference between lines created by the comrades, and the mixing ratio of the inter-frame interpolated image signal and the intra-field interpolated image signal is continuously changed according to the detected amount of movement of the image. The feature is that they are mixed with each other.

The invention will be explained in detail below by way of example embodiments with reference to the drawings.

First, FIG. 2 shows an example of the basic configuration of an image signal interpolation device according to the present invention. Although the illustrated basic configuration uses two frame memories 1 and 2, the same effects as described later can be obtained even if the configuration is simplified and a single frame memory is used. However, when only a single frame memory is used, as described above, the motion part of the image is affected by the false signal caused by the interpolation using interframe correlation. This causes a positional shift in the direction, and the blurring is similar to the normal image blurring caused by a simple intra-field two-dimensional filter action, and image quality deterioration due to the positional shift occurs.As will be explained later, interpolation between frames and When interpolation and interpolation within the same field are simultaneously used and switched, the positional shift of the image becomes noticeable, so that the intended effect of the image signal interpolation according to the method of the present invention cannot be sufficiently obtained.

Therefore, in the illustrated configuration, image signals of three adjacent frames are sequentially extracted from each input/output terminal of the two cascade-connected frame memories 1 and 2 and supplied to the motion amount detection circuit 4, which will be described later. The amount of movement in the image is detected in this way. The detection output signal signal level of this motion amount detection circuit 4 changes depending on the magnitude of image movement, and the signal level is high in parts of the image with large movement, and low in parts of the image with small movement. .

On the other hand, for example, as an interpolated image signal obtained by performing pixel interpolation on an input image signal subjected to band compression by 2:1 subsampling, two frames obtained from both ends of the cascade connection of two frame memories 1 and 2 are used. Image signals b and c with periodic distance are sent to adder 5 and 1/
The signal averaged by the 2 multiplier 6 and the image signal a are led to the switch S, and the interpolated image signal for the still image obtained by alternately switching every sample and the 2 frame memories 1 and 2 are connected in series. An image signal a with an intermediate frame period obtained from an intermediate connection point of the connection is guided to a two-dimensional filter 3, and an interpolated image signal for a motion image subjected to interpolation is guided to multipliers 8 and 7, respectively. As shown in the figure, the mixing ratio α of both interpolated image signals is multiplied by (1-α) and α, respectively, and then guided to the adder 9.
Both interpolated image signals are mixed appropriately and extracted as an output interpolated image signal, and the mixing ratio α of both interpolated image signals is set to the detection signal level of the motion amount detection circuit 4 described above. control and
The mixing ratio α is adjusted at an appropriate speed according to changes in the amount of image movement.
change continuously and smoothly. Therefore, switching between the inter-frame interpolated image signal and the intra-frame inter-pixel interpolated image signal depending on the presence or absence of image movement is the same as in the conventional interpolation method; Since the switching is performed continuously and smoothly, there is no possibility that new image quality deterioration will occur due to the switching of interpolated image signals, unlike in the conventional case.

Next, FIG. 3 shows an example of a detailed configuration of the motion amount detection circuit 4 for accurately detecting the motion of an image required for switching the interpolated image signals in the manner described above. Detection of the amount of motion of an image using the illustrated configuration is performed according to the following principle.

In other words, when detecting the motion of an image based on the presence or absence of a correlation between image signals between two adjacent frames, for example, the magnitude of the inter-frame difference obtained between the two frames directly represents the amount of image motion. For example, even if the movement in the image is slight, if there is movement in the edge part of a plane image where there are many high-frequency components of the image signal, the movement edge part will have a large frame between two frames. There will be a difference between the two. Therefore, the motion amount detection circuit used in the interpolation method of the present invention does not depend only on the magnitude of the inter-frame difference as described above, but also on the so-called in-field detection circuit within the same field with the configuration shown in FIG. 4, for example. The difference between pixels, that is, the magnitude of the difference between two pixel signals,
That is, it is possible to accurately detect the movement of an image by taking into consideration the magnitude of the high frequency component of the image signal.

Therefore, as for the configuration arrangement of the pixel group shown in FIG. Inter-pixel differences D _e1 and D _e2 between two pixels separated one after the other by a pixel interval and the pixel of interest, and adjacent scans located at each vertex of an approximately square with the pixel of interest as the intersection of diagonals as shown in the figure. The inter-line differences D _l1 and D _l2 between two pixels on the line and the normal two-frame difference D _f regarding the pixel of interest itself are used to detect the amount of motion, and therefore the above-mentioned inter-frame interpolated image signal and field By setting the mixing ratio α with the internally interpolated image signal, motion amount detection is performed in consideration of the above-mentioned interframe difference and image signal high-frequency components. The above-mentioned differences D _e1 , D e2 , D _l1 , D e2 , D _l1 ,
The absolute value of both D _l2 and D _f is used for motion amount detection.

According to the inter-pixel differences D _e1 , D _e2 and the line-to-line differences D _l1 , D _l2 described above, it is possible to determine the state of change in the image signal level in a minute image area centered on the pixel of interest, that is, the image signal Since it is possible to determine the magnitude of the high frequency component, if the maximum absolute value D of the absolute values of each of these differences is selected, the magnitude relationship between the maximum absolute value D of and the absolute value of the inter-frame difference D _f is determined. By combining these, it is possible to accurately detect the amount of motion taking into account the presence of high-frequency components of the image signal. In other words, if D≡max{D _e1 , D _e2 , D _l1 , D _l2 }, then by the function f α=f (D _f , D), the interpolated image signal described above and the pixel in the field are The mixing ratio α with the interpolated image signal can be appropriately selected. For example, if the function f is set as f (D _f , D) = D _f /D, (1) When the amount of motion of the video is small, D _f → small,
Since D becomes large, α becomes 0, which increases the mixing ratio of interframe interpolated image signals.

(2) When the amount of image movement is large, D _f → large,
Since D becomes small, α becomes 1, which increases the mixing ratio of interpolated image signals between pixels between fields.

Note that when D _f → large and D → large, for example, it is assumed that the amount of movement at the edge part of the image is large, but in reality, when the edge part moves quickly, it is difficult to image such a moving edge. Since the high-frequency component in the output signal is much lower than in the case of a stationary edge, it approximates the case where D _f → large and D → small. Strictly speaking, however, it is necessary to select an appropriate configuration of the function f (D _f , D) that takes such cases into consideration so that appropriate motion amount detection can be performed even in such cases.

In addition, since the calculation of the function f (D _f , D) based on the magnitude relationship of D _f and _D needs to be performed at a high image signal rate, A function f calculated in advance for
Preferably, the read-on memory ROM storing the values of is accessed using the measured values of the absolute values D _f and D to determine the required function value.

In addition, according to the above-mentioned motion amount detection process, effective and accurate motion amount detection can be performed taking into account the presence of the image signal high frequency component, but as seen in the circuit configuration shown in the second figure, the signal processing system Since there is a multiplication circuit inside, in order to prevent the generation of undesired high-frequency components during the multiplication process, the detection output signal is guided to an appropriate low-pass filter to convert the function value to an appropriate value. It is preferable to extract the mixing ratio.

Therefore, in order to detect the amount of motion by the above-described process based on the detection of inter-pixel differences using the configuration arrangement shown in FIG. This is an illustrated configuration example. That is, in the illustrated configuration, the image signal a taken out from the intermediate connection point of the two frame memories 1 and 2 cascaded in the second illustrated configuration is supplied to the line memories 10 and 11 similarly cascaded; The image signal taken out from the intermediate connection point is supplied to the one-clock delay devices 16 and 17 connected in series in the same way, and the frame memory 1 and
The image signals b and c of the difference between the two frames extracted from both ends of the cascade connection of the two frames are supplied to the subtracter 24. Next, the image signals taken out from both ends of the cascade connection of the line memories 10 and 11 are supplied to the subtracter 12 to extract the sequential line differences, and the subtracted output is led to the absolute value unit 13 to calculate the absolute value. demand. On the other hand, the pixel signal of the difference between two pixels taken out from both ends of the cascade connection of the one-clock delay devices 16 and 17 is supplied to the subtracter 18 to extract the difference between the pixels sequentially. and find its absolute value. Next, the inter-line difference from the absolute value unit 13 is transmitted through the 1-clock delay device 14, which corrects the positional deviation of 1 pixel interval between the inter-line difference detection pixel point and the inter-pixel difference detection pixel point in the configuration shown in FIG. The absolute value and the inter-pixel difference absolute value from the absolute value unit 19 are supplied to the maximum value unit 15 to obtain the maximum absolute value between the two, and the maximum absolute value output is connected in cascade to two 1-clock delay units 20. , 21 and the maximum absolute value from both ends of the cascade is supplied to the maximum value unit 22,
The maximum absolute value between the maximum absolute value in the left half pixel group and the maximum absolute value in the right half pixel group in the configuration shown in FIG. 4 is determined, and the maximum absolute value D according to the above formula is obtained. On the other hand, the inter-frame difference obtained from the subtracter 24 is guided to the absolute value unit 25 to obtain its absolute value, and two 1-clock delay devices connected in cascade are used to match the timing with the maximum absolute value D mentioned above. Inter-frame difference absolute value via 26, 27
By accessing D _f and the maximum absolute value D from the maximum value unit 22, the required function value f corresponding to these measured values, that is, the required mixing ratio α, is retrieved from the read-only memory (ROM) 23.

As is clear from the above, according to the present invention, the frequency band of an image signal subjected to pixel-by-pixel subsampling for band compression can be processed without deterioration of image quality, regardless of the movement of the image. Since it is possible to restore the image signal, it is possible to reduce the required amount of transmission by subjecting the image signal to sub-sampling on a pixel basis, and then perform appropriate encoding and transmission.

Note that such pixel-by-pixel subsampling can also be applied to analog transmission of image signals to obtain the effect of compressing the required transmission band, so the interpolation method of the present invention can be widely applied to general transmission of wideband image signals. be able to.

[Brief explanation of drawings]

FIG. 1 is a diagram showing an example of a mode of image signal subsampling, and FIG. 2 is a block diagram showing an example of the basic configuration of an image signal interpolation circuit according to the method of the present invention.
FIG. 3 is a block diagram showing an example of a detailed configuration of a motion amount detection circuit in the interpolation circuit, and FIG. 4 is a diagram showing an example of a difference detection mode in the motion amount detection circuit. 1, 2... Frame memory, 3... Two-dimensional filter, 4... Motion amount detection circuit, 5, 9... Adder, 6, 7, 8... Multiplier, 10, 11... Line memory, 12 ,18,24...subtractor,13,
19, 25... Absolute value unit, 14, 20, 21, 2
6, 27...1 clock delay device, 15, 22...
Maximum value unit, 23...Read-only memory.

Claims (1)

[Scope of Claims] 1. In restoring the frequency band of an image signal whose required frequency band has been reduced by sampling at a ratio of at least 2:1, the motion of the image represented by the image signal is determined by a pixel of interest that differs by at least two frame periods. The absolute value of the difference between two frames between two frames, the absolute value of the difference between pixels created by the pixel of interest and pixels adjacent in the line direction in the same field, and the line created by adjacent pixels in a quincunx relationship with the pixel of interest. Detection is performed based on the absolute value of the inter-frame difference, and the inter-frame interpolated image signal and the intra-field interpolated image signal are mixed with each other at a mixing ratio that is continuously changed according to the detected amount of image motion. An image signal interpolation method characterized by: 2. An image signal in the interpolation method according to claim 1, wherein the inter-frame interpolated image signal and the intra-field interpolated image signal temporally correspond to each other. Interpolation method. 3. In the interpolation method according to claim 1 or 2, the maximum absolute value of the absolute value of the difference between two frames, the absolute value of the difference between pixels, and the absolute value of the difference between lines; An image signal interpolation method characterized in that a motion of the image is detected based on a ratio between the two images.