JPS58191586A - Picture signal inserting system - Google Patents

Abstract

PURPOSE:To restore the frequency band of an original picture signal, by mixing picture signals with the mixing ratio changed continuously in response to the amount of detection of the movement of the picture signals after interpolation between frames and between fields and performing picture element interpolation without the generation of unnatural feeling. CONSTITUTION:The picture signals of adjacent frames are extracted sequentially from input and output terminals of two frame memories 1, 2 of cascade connection, applies to a movement amount detection circuit 4 and the movement of the picture is detected. The picture signal after interpolation for picture element interpolation is obtained from both the terminals of the memories 1, 2 to an input picture signal for band compression at least 2:1 sampling and averaged at an adder 5 and a multiplier 6. The picture signal between intermediate frame periods between the memories 1, 2 is applied to a two-dimension filter 3 to output the picture signal after interpolation, and the signal is applied to a multiplier 7 and the output of the multiplier 6 is applied to a multiplier 8. The signals are mixed at an adder 9 in the mixing ratio of continuous change in response to the detection of the movement of the picture signal detected at the circuit 4.

Description

[Detailed description of the invention]

The present invention relates to an image signal interpolation method for restoring the frequency band of an image signal that has been subjected to band compression by subsampling in sampling percentage, and in particular to a method for preventing image quality deterioration of moving images due to interpolation. It is. In general, when transmitting wideband image signals such as high-definition television programs, it is extremely important to make the frequency band of the transmission line required for transmission as narrow as possible from the viewpoint of effective use of the transmission band. be. Therefore, in order to narrow the required bandwidth of the transmission path, one of the band compression methods applied to the image signal to be transmitted is to use a 1m interframe phase of the image signal, and to compress the image signal pixel by pixel. There is a band compression method that sub-samples the signal and transmits it. Among such pixel-based subsampling, the simplest example is 2:1 subsampling, which samples every other pixel signal in the same way as the so-called 2:l dot interlace, as shown in FIG. If every other pixel signal is sub-sampled on every other scanning line subjected to main sampling by interlaced scanning, the restoration of the original Im signal is completed with the sample Im elementary signal for two frames. Therefore, by performing such subsampling on nine successive frames, the transmission bandwidth required for transmitting nine image signals can be theoretically reduced. However, in order to transmit the required frequency band by half by such 2:1 subsampling and restore the frequency band of a good image signal on the receiving side of the transmission path, a frame memory is usually provided and each frame is sequentially transmitted. The missing pixel signals are replaced by all corresponding IIjI pixel signals in the previous frame temporarily stored in the frame memory. However, although it is possible to completely restore the #'i original image for a still image by using alternative compensation flJIti for such a pixel signal, for a moving image, changes due to the movement of the same pixel point between two adjacent frames Restore I-
Therefore, false signals are generated, and the original motion image cannot be faithfully restored. On the other hand, for a moving image, even if there is blur equivalent to the pressure of the moving part of the image, the visual sense cannot sufficiently follow the movement of the moving part - so the image quality does not visually deteriorate much. Taking advantage of this visual characteristic, it has been common practice to interpolate the loss of pixel signals in the moving part due to subsampling with adjacent pixel signals in the same field. Conventionally, it has been demonstrated that such pixel interpolation can be applied to motion compensated interframe coding devices and the like to obtain significant effects. However, such conventional pixel interpolation has the following specific problems. That is, (1) the image glK restored by switching between pixel interpolation by interpolation using frame fliJ1 correlation and iii pixel interpolation by interpolation between adjacent pixels in the same field, depending on the presence or absence of image movement; A new unnaturalness arises. (2) If the detection of moving parts of an image is performed simply according to the magnitude of the difference between frames, for example, in edge parts where the high-frequency components of the image signal increase, even a slight movement of the image A large difference between frames occurs, resulting in a detection result equivalent to that of a large movement in the image.
Motion detection is incomplete, and new image quality degradation occurs due to such erroneous detection of motion. (-7 Therefore, in the conventional image signal interpolation method,
Either method has the disadvantage that it is difficult to sufficiently restore the frequency band of the original image signal by performing pixel interpolation to obtain an image of sufficiently high quality. The object of the present invention is to solve the above-mentioned conventional problems and eliminate their drawbacks, and to use a suitable interpolation signal without introducing new artifacts. An object of the present invention is to provide an image signal interpolation method that performs il cumulative interpolation to restore the frequency band of an original image signal. That is, the image signal interpolation method of the present invention has at least 2:
In restoring the frequency band of an image signal whose required frequency band has been reduced by sampling at a ratio of Detection is performed based on the respective absolute values of intra-field pixel differences between the pixel and each of a plurality of pixels adjacent to the pixel within the same field, and inter-frame interpolated image signals and intra-field interpolation are performed. The present invention is characterized in that the detected image signals are mixed with each other at a mixing ratio that is continuously changed by five times the amount of detected image motion. In the following, the invention will be described in detail by way of example embodiments with reference to the drawings. FIG. 2 shows an example of the basic configuration of an image signal interpolation device according to the present invention. In addition, although two frame memories 1 and 2 are used in the principle configuration trap shown in the figure,
Even if the configuration is simplified and a single frame memory is used, the same effects as described later can be obtained. however,
In the case where only a single frame memory is used, as mentioned above, the moving part of the KSFh image may be misaligned in the direction of its movement due to false signals caused by interpolation using interframe correlation in the moving part of the image. This results in the same blurring as normal image blurring caused by a simple intra-field two-dimensional filter effect, as well as image quality deterioration due to the positional shift K. Since the positional deviation of K iil f& is noticeable when interpolation and interpolation are used simultaneously, the intended effects of image signal interpolation according to the method of the present invention cannot be sufficiently obtained. In the illustrated configuration, image signals of eight adjacent frames are sequentially extracted 12 from each input/output terminal of two cascade-connected frame memories 1 and 2 and supplied to the recommended amount detection circuit 4. , as will be described later (7), detects the amount of movement of the image.The detection output signal level of the movement amount detection circuit 8 changes depending on the size of the movement of the image, and in parts of the image where the movement is large, the signal level changes. The signal level is high and the signal level is low in parts with small image movement.On the other hand, for example, as an interpolated image signal obtained by performing pixel interpolation on an input image signal that has been subjected to band compression using 2:1 subsampling, 2 frame, memory, 2 frame cycle distance image signals obtained from both ends of the 2 cascade connection,
The interpolated image signal for still images obtained by averaging c by the adder 5 and the -multiplier 6, and the image signal a of the intermediate frame period obtained from the intermediate connection point of the cascade connection of the two frame memories 1.2 are divided into two. The interpolated image signal for the motion image that has been guided to the dimensional filter 8 and subjected to interpolation is guided to the multipliers 8 and 7, respectively, and as shown in the figure, the mixing ratio α of both interpolated image signals is calculated. , the multiplication of (l-α) and α is calculated and then sent to the adder 9, and both interpolated image signals are appropriately mixed and extracted as an output internalized image signal. The mixing ratio α of the interpolated image signals is controlled 17 by the detection signal level of the telekinetic motion detection circuit 4, and the mixing ratio α is adjusted at an appropriate speed according to the change in the image motion amount.
change continuously and smoothly. Therefore, ii! Il
Switching between the inter-frame interpolated image signal and the inter-pixel interpolated image within the same field (lI signal) depending on the presence or absence of movement is the same as in the conventional interpolation method, but the switching This is done continuously and smoothly, so there is no risk of new image quality deterioration due to the switching of the interpolated image signal, as in the conventional case.Next, the interpolated image signal as described above An example of the detailed configuration of the motion amount detection circuit for accurately detecting the image motion required for switching is shown in Figure @8.The principle of image motion amount detection using the illustrated configuration is as follows. In other words, when detecting image motion based on the presence or absence of correlation between image signals between eight adjacent frames, for example, the magnitude of the inter-frame difference obtained between the two frames is teeth,
It does not directly represent the amount of movement in the image; for example, even if the movement in the image is slight, if there is movement in the edge part of the image where there are many high-frequency components of the image signal, the amount of movement in the edge part will be A large difference will occur between two frames. Therefore, in the motion amount detection circuit using the interpolation method of the present invention, the trap does not depend only on the size of the difference between frames as described above, but the so-called so-called within the same frame due to the configuration arrangement 9 as shown in FIG. The in-frame pixel difference, that is, the magnitude of the difference between 8 pixel signals, that is, the magnitude of the high frequency component of the image signal is also taken into consideration to accurately detect the movement of the image. [2] The arrangement of the pixel group shown in FIG. 4 is as follows:
Focusing on a central pixel on a scanning line located in the center of eight sequentially adjacent scanning lines in the same frame, the pixel is located between the pixel of interest and three pixels spaced 22 pixels apart from each other, sandwiching the pixel of interest. Difference between each pixel. , and Tori. , and giii on the adjacent scanning lines located at each vertex of the #1 square with the attention iii element as the intersection of the diagonal lines as shown in the figure.
The line-to-line difference Dt and Dt between the elementary elements,
By detecting the amount of motion and using the normal inter-frame difference D/ regarding the pixel of interest itself, the mixing ratio α of the inter-frame interpolated image signal and the intra-field inter-pixel interpolated image signal described above is set. , the amount of motion is detected taking into account the above-mentioned inter-frame difference and image signal high-frequency components. Note that each of the above-mentioned differences Del I Delml Dtl 1 D, . In both cases, the absolute value of D7 is used for motion amount detection. I7 (7, the above-mentioned difference between each pixel De1' DeU
and the difference between each line, 11 Dtil, it is possible to determine the state of change in the image signal level in a minute image area centered on the pixel of interest, that is, the magnitude of the individual signal ﾾ area component. If you choose the maximum absolute value among the absolute values of each difference, its maximum absolute mD
By combining the magnitude relationship between the absolute value of the inter-frame difference DI and the absolute value of the inter-frame difference DI, accurate motion amount detection can be performed taking into account the presence of the high frequency component of the image signal. That is, D ; maX (D61 e D61 * DtX t
Dtl), then the mixing ratio α of the inter-frame interpolated image signal and the intra-field inter-pixel interpolated image signal is determined by the function f α-/(D7, D).
can be selected appropriately. For example, if the function f is set as follows: (1) When the amount of image movement is one small, Df → small, D
→ becomes large, so α → 0.9 This increases the mixing ratio of interframe interpolated image signals. (2) When the amount of image movement is large, Df→large, D→
Since α becomes smaller, α→l, which increases the mixing ratio of the intra-field self-interpolated image signal. Note that when Dj→Large or D→Large, for example, it is assumed that the amount of movement at the edge part of the image is large, but in reality, when the movement of the edge part is fast, the imaging output of such a moving edge Since the high-frequency components in the signal are much lower than in the case of a stationary edge, they are approximated by the cases where Df→large and D→small. However, (~, strictly speaking, in order to perform appropriate motion amount detection even in such a case, it is necessary to select an appropriate configuration of the function / (D7, D) that takes such a case into consideration. / , a function based on the magnitude relationship of D /
Since the calculation of (Dl, D) needs to be performed at a high image signal rate, a read-only system is used that stores the values of the function f that have been calculated in advance for various combinations of the absolute values D and D7. Absolute value DI, D of memory ROM
It is preferable to obtain the desired function value by accessing the measured value of . According to the above-described motion amount detection process, effective and accurate motion amount detection can be performed taking into account the presence of high-frequency components of the image signal, but as shown in the circuit configuration shown in the second figure, the signal processing system Since there is a multiplication circuit inside, the multiplication circuit guides the detection output signal to an appropriate low-pass filter and converts it to an appropriate value in order to prevent undesired high-frequency components from occurring during the calculation process. It is preferable to extract the function value or the mixing ratio. 17 or 1. Then, a motion amount detection circuit is constructed in accordance with the above-described process faithfully so that a motion hoe can be detected by the above-described process based on the detection of inter-pixel differences using the configuration shown in FIG.

[- is the configuration example shown in Figure 8. That is, in the configuration shown in the figure, the image signal af taken out from the intermediate connection point of the two frame memories 1.2 connected in cascade in the configuration shown in the second figure is transmitted to the line memories 10.2, which are connected in cascade in the same way. I
The image signals taken out from the intermediate connection point are supplied to l clock delay devices 16 and 17 which are connected in cascade in the same way, and the two frames taken out from both ends of the cascade connection of the frame memories 1. One frame difference image signal S, c
is supplied to the subtracter 24. Next, line memory 10.
The 1iIil signals taken out from both ends of the cascade connection of 11 are supplied to a subtracter 12 to extract sequential line-to-line differences, and the subtracted output is led to an absolute value unit 18 to obtain its absolute value. On the other hand, the pixel signal of the difference between two pixels taken out from both ends of the cascade connection of the clock delay devices 16 and 17 is supplied to the subtracter 18 to extract the difference between the pixels sequentially, and the subtracted output is sent to the absolute value unit 19. and find the absolute value of τ. Next, the inter-line difference from the absolute value unit 18 is transmitted through the 1-clock delay device 14, which corrects the positional deviation of 1 pixel interval between the inter-line difference detection pixel point and the inter-pixel difference detection pixel point in the configuration shown in FIG. The absolute value and the inter-pixel difference absolute value from the absolute value unit 19 are supplied to the maximum value unit 15 to obtain the maximum absolute value between the two, and the maximum absolute value output is connected in cascade to two 1-clock delay a.
r=g O, B 1, and the maximum absolute value from both ends of the cascade connection is supplied to the maximum value unit 22, and the maximum absolute value in the left half pixel group in the configuration shown in FIG. Find the maximum value with respect to the maximum resultant 1 value in the right half pixel group, and extract the maximum absolute f[D according to the above formula. On the other hand, the inter-frame difference obtained from the subtracter 24 is guided to the absolute value unit 25 to obtain its absolute value, and in order to match the timing with the maximum absolute value mentioned above, two 1-clock delay units connected in cascade are used. Inter-frame difference absolute value DJ via devices 26 and 27
and the maximum absolute value from the maximum value unit 22 (
7, the required function value f1 corresponding to those measured values, that is, the required mixture ratio α, is stored in a read-only memory (ROM).
Take it out from 28. As is clear from the above description, according to the present invention, the frequency band of an image signal can be subsampled in pixel units for band compression without deterioration of image quality, regardless of image movement. Therefore, after subsampling the image signal in units of K11ii elements to reduce the required transmission amount, appropriate encoded transmission can be performed. Note that such pixel-by-pixel subsampling can also be applied to analog transmission of image signals to obtain the effect of compressing the required transmission band, so the interpolation method of the present invention can be widely applied to general transmission of wideband image signals. be able to.

[Brief explanation of the drawing]

FIG. 1 is a diagram showing an example of S+-like image signal subsampling, FIG. 2 is a block diagram showing an example of the basic configuration of a 1iii+ signal interpolation circuit according to the method of the present invention, and FIG. FIG. 4 is a block diagram showing an example of a detailed configuration of a motion amount detection circuit in the interpolation circuit, and FIG. 4 is a diagram showing an example of a mode of difference detection in the motion amount detection circuit. ], 2... Frame memory, 8... Two-dimensional filter, 4... Motion amount detection circuit, 5, 9... Adder, 617
18 Multiplier, 10.11...Line memory, 12゜1
8.24... Subtractor, 18, 19, 2 digits... Absolute value unit, 14, 20, 21, 26, 2 digits... 1 clock delay sharpener, 15.22... Maximum value unit, 28...Read-only memory. Mutual Applicant: Japan Broadcasting Association Figure 1--<To→<-+-E-0-One Piece One Beto-11 Water-+-H--0-One Yu-Hee◇--Juice 111 Figure 3 Figure 4 A

Claims (1)

[Claims] L: In restoring the frequency band of an image signal whose required frequency band has been reduced by sampling at a ratio of at least 2:1, the motion of the image represented by the image signal is different by at least one frame period. Detection is performed based on the absolute value of the inter-frame difference between pixels and the absolute value of the intra-field inter-pixel difference between the wJ element and each of a plurality of pixels adjacent to the pixel within the same field, and the frame An image signal interpolation method characterized in that an interpolated image signal and an intra-field interpolated image signal are mixed with each other at a mixing ratio that is continuously changed according to the detected amount of motion of the image. In the interpolation method according to claim 1, the inter-frame interpolated mi defect signal and the intra-field interpolated image signal temporally correspond to each other. My signal interpolation method.龜 Claims 1 to 9 are based on the interpolation method described in Clause S, based on the ratio of the absolute value of the inter-frame difference to the maximum absolute value of the absolute value of the intra-field pixel difference. An image signal interpolation method characterized in that pressure is applied to detect movement of the image.