KR100203677B1 - Improved image coding system having functions for controlling generated amount of coded bit stream - Google Patents

Abstract

The present invention, in the coding system including MC-DCT, quantization, calculates the complexity of the image to be currently encoded based on the spatial complexity of the previous frame reconstructed for motion estimation and compensation, and according to the calculation result 1 The present invention relates to a video encoding system having a bit generation amount adjusting function for adaptively adjusting a bit generation amount after encoding by selectively removing high frequency components of an input video signal using a dimensional low pass filter. A spatial complexity value is calculated for each reconstructed previous frame for motion estimation and compensation, and the average spatial complexity value for the plurality of preset previous frames is calculated by averaging the spatial complexity values of each calculated previous frame. Calculation means; The calculated average spatial complexity value of the plurality of frames is referred to as the complexity of the frame to be currently encoded by encoding means having DCT, quantization, and entropy encoding, and is input for encoding based on the detected spatial complexity value of each frame. Control means for generating a filter control signal for limiting the frequency passband of the current frame; And based on the generated control signal, provide the input current frame to the encoding means as a current frame signal for motion estimation and compensation, or apply one-dimensional low pass filtering to the input current frame to limit the pass band. By including the one-dimensional low-pass filtering means for providing the encoding means as a current frame signal for motion estimation and compensation to the encoding means, to effectively reduce the amount of bits generated after encoding without undue step size increase during quantization in the encoding means. It is adjustable.

Description

Image Coding System with Bit Rate Control

1 is a block diagram of a video encoding system having a bit generation amount adjusting function according to a preferred embodiment of the present invention.

2 is an illustration of an 8x8 pixel block as an example in accordance with the present invention.

3 is an exemplary view showing a one-dimensional low pass filter coefficient of order 7 as an example according to the present invention.

4 is an exemplary view illustrating a horizontal filtering process at (0,4) position and a vertical filtering process at (3,0) position as an example according to the present invention.

* Explanation of symbols for main parts of the drawings

100, 170: frame memory 110: subtractor

120: image encoding block 130: entropy encoding block

140: transmission buffer 150: video decoding block

160: adder 180: block on the current frame side

210: space complexity calculation block 220: filter control block

230: low pass filtering block

The present invention relates to a video encoding system for compressing and encoding a video signal. More particularly, the present invention relates to a video encoding system for compressing and encoding a video signal, and more specifically, to compress and encode a video signal using motion compensation differential pulse code modulation (MC-DPCM). The present invention relates to a video encoding system having a bit generation amount adjustment function suitable for adaptively adjusting the amount of generated bits after encoding by referring to a variation of an input video signal predicted based on the complexity of a previous frame.

As is well known in the art, the transmission of discrete video signals can maintain better image quality than analog signals. When a video signal consisting of a series of image “frames” is represented in digital form, a significant amount of data must be transmitted, especially for high quality televisions (aka HDTVs). However, since the usable frequency range of the conventional transmission channel is limited, in order to transmit a large amount of digital data, it is necessary to compress the transmitted data and reduce the amount thereof. Among the various compression techniques for compressing data, hybrid coding techniques combining probabilistic coding techniques with temporal and spatial compression techniques are known to be the most efficient, and these techniques have already been enacted by the standards standard, for example, by the World Organization for Standardization. It is widely disclosed in recommendations such as MPEG-1 and MPEG-2.

Most hybrid coding techniques use motion compensated DPCM (differential pulse code modulation), two-dimensional DCT (discrete cosine change), quantization of DCT coefficients, VLC (variable length coding), and the like. The motion compensation DPCM determines a motion of an object between a current frame and a previous frame, and predicts a current frame according to the motion of the object to generate a difference signal representing a difference between the current frame and a predicted value. This can be done for example by Staffan Ericsson's Fixed and Adaptive Predictors for Hybrid Predictive / Transform Coding, IEEE Transactions on Communication, CCM-33, NO.12 (Dec. 1985), or A motion Compensated Interframe Coding Scheme by Ninomiy and Ohtsuka. for Television Pictures, IEEE Transactions on Communication, COM-30, NO.1 (1982, January).

In general, two-dimensional DCT converts digital image data blocks, such as 8x8 blocks, into DCT conversion coefficients by using or removing spatial redundancy between image data. This technique is described in Chen and Pratt's Scene Adaptive Coder, IEEE Transactions on Communication, COM-32, NO.3 (March 1984). The DCT conversion coefficient may be processed through a quantizer, zigzag scan, VLC, etc. to effectively reduce (or compress) the amount of data to be transmitted.

More specifically, the motion compensation DPCM predicts the current frame from the previous frame according to the motion of the object estimated between the current frame and the previous frame. The estimated motion may be represented by a two-dimensional motion vector representing the displacement between the previous frame and the current frame.

Typically, there are several approaches to estimating the displacement of an object. These are generally classified into two types, one of which is a block-based motion estimation method using a block matching algorithm, and the other is a pixel-based motion estimation method using a pixel circulation algorithm.

In the motion estimation method for estimating the displacement of an object as described above, the displacement is obtained for each pixel by using the pixel-based motion estimation method. This method has the advantage of being able to estimate pixel values more accurately and easily handle scale changes (e.g., zooming, which is a movement perpendicular to the image plane), while the motion vectors for each pixel Since it is determined, it is impossible to transmit substantially all the motion vectors to the receiver as large amounts of motion vectors occur.

In addition, in block-by-block motion estimation, a block having a predetermined size of the current frame is moved by one pixel in a search range of a previous frame and compared with the corresponding blocks to determine an optimal matching block having a minimum error value. From this, the interframe displacement vector (the extent to which the block has moved between frames) for the entire block is estimated for the current frame to be transmitted. Here, in determining the similarity between two corresponding blocks between the current frame and the previous frame, the mean absolute difference and the mean square difference, etc., as known in the art, are mainly used.

On the other hand, the image bit stream encoded by the encoding technique as described above, that is, the encoding scheme such as motion compensation DPCM, two-dimensional DCT, DCT coefficient quantization and VLC (or entropy encoding) is transmitted to the output side of the image encoding system. The next transmission point stored in the buffer is sent to the transmitter for transmission to the remote destination. At this time, the transfer time here is related to the size (i.e. capacity) and transfer rate of the transfer buffer and is controlled so that no malfunction (data overflow or data underflow) occurs in the transfer buffer.

More specifically, the amount of bits generated for each frame at the time of encoding varies due to various factors (for example, the complexity of the image). In view of this, in the image encoding system, the average bit rate may be kept constant. Control of the output buffer. That is, the video encoding system checks the bit generation amount up to the frame currently encoded based on the data fullness state information of the output transmission buffer and adjusts the bit amount to be allocated in the current frame. In other words, in the conventional typical video encoding system, the amount of bits generated in the encoding system is adjusted by controlling the quantization step size (QP) substantially based on the data full state information of the output transmission buffer, that is, if the amount of bits generated before has been large, The bit generation amount is controlled by reducing the bit generation amount by adjusting the step size largely, and in the opposite case, by adjusting the quantization step size small to increase the bit generation amount.

However, as described above, the conventional method of adjusting the bit generation amount by adjusting the quantization step size based on the data fullness state information of the output side transmission buffer is performed when encoding and transmitting video data corresponding to each frame at the same data rate. In the case where the image to be encoded is complex (a large amount of high frequency components are generated), a large amount of bits is generated, which causes a problem that the quantization step size becomes large, resulting in severe image quality degradation in the reproduced image. The high frequency component generated here is a component that is substantially insensitive to human visual characteristics (a component that hardly affects the image quality of a reproduced video).

Accordingly, the present invention is to solve the above-described problems of the prior art, in the coding system including MC-DCT, quantization, to be currently encoded based on the spatial complexity of the previous frame reconstructed for motion estimation and compensation An image having a bit generation amount adjustment function that can adaptively adjust the bit generation amount after encoding by calculating the complexity of the image and selectively removing high frequency components of the input video signal using a one-dimensional low pass filter according to the calculation result. The purpose is to provide an encoding system.

In order to achieve the above object, the present invention provides a discrete cosine transform, quantization, and an error signal between an input current frame and a prediction frame obtained through motion estimation and compensation in units of macroblocks using the current frame and the reconstructed previous frame. Compression encoding is performed through encoding means including entropy encoding to generate an encoded bit stream, and the quantization has a bit generation amount adjusting function of adjusting a step size based on the fullness information of the bit stream stored in an output buffer. In the image encoding system, a spatial complexity value is calculated for each of the previous frames reconstructed for the motion estimation and compensation, and the averaged spatial complexity values of the respective previous frames are respectively calculated for a plurality of preset previous frames. Calculate the average spatial complexity value Space complexity calculation means; The calculated average spatial complexity value of the plurality of frames is referred to as the complexity of the frame to be currently encoded by the encoding means, and the frequency of the current frame input for encoding based on the detected spatial complexity value of each frame. Control means for generating a filter control signal for limiting the pass bandwidth; And based on the generated filter control signal, the input current frame is provided to the encoding means as the current frame signal for motion estimation and compensation, or one-dimensional low pass filtering is applied to the input current frame to pass the band. And a one-dimensional low pass filtering means for providing a frame signal from which a high frequency component is removed to the encoding means as a current frame signal for motion estimation and compensation by limiting the frequency difference. To provide.

The above and other objects and various advantages of the present invention will become more apparent from the preferred embodiments of the present invention described below with reference to the accompanying drawings by those skilled in the art.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram of a video encoding system having a bit generation amount adjusting function according to a preferred embodiment of the present invention. As shown in the figure, the image encoding system of the present invention includes a first frame memory 100, a subtractor 110, an image encoding block 120, an entropy encoding block 130, a transmission buffer 140, and image decoding. Block 150, adder 160, second frame memory 170, current frame side block 180, spatial complexity calculation block 210 filter control block 220, and low pass filtering block 230. .

Referring to FIG. 1, the input current frame signal is stored in the first frame memory 100 and then input to the low pass filtering block 230, and the low pass filtering block 230 includes the filter control block 220 described later. Selectively filter the frequency of the input frame signal, i.e., one-dimensional low pass filtering, according to a control signal (a logic signal of level 0 or 1) calculated based on the spatial complexity of the previous frame reconstructed and reconstructed for encoding provided from The high frequency component of the input video signal (which is a relatively insensitive part of the human eye) is filtered. The detailed operation of the low pass filtering block 230 is described with reference to FIGS. 2 to 4. It will be described later in detail. Then, the current frame signal in which the high frequency component is removed or not removed from the input image is provided to the subtractor 110 and the current frame side block 180 through the line L11, respectively.

First, in the subtractor 110, the motion compensated prediction is performed on the moving object provided from the current frame edge block 180 described later through the line L19 from the current frame signal provided from the first frame memory 100 through the line L11. The current frame signal is subtracted, and as a result, the data, that is, the error signal representing the differential pixel value, is obtained by using discrete cosine transform (DCT) and one of quantization methods well known in the art through the image coding block 120. By encoding a series of quantized DCT transform coefficients. In this case, the quantization of the error signal in the image encoding block 120 is performed based on the quantization parameter QP determined according to the data fullness state information provided from the output side transmission buffer 140 described later through the line L21. The size is adjusted.

Meanwhile, although the detailed illustration in FIG. 1 is omitted, in the low pass filtering block 230, the filter control signal 0 based on the calculated spatial complexity information of the previous frame provided from the filter control block 220 which will be described later according to the present invention. Or one-dimensional low-pass filtering based on a logic signal of 1 to remove (ie, filter) a high frequency component that is relatively insensitive to human visual characteristics. Accordingly, the present invention can appropriately adjust the quantization stem size in the quantization step of the image encoding block 120 that causes deterioration of image quality in the reproduced image even in a complex image accompanied by an increase in the encoded bit generation amount. A detailed process of selectively limiting the passband of the input frame by using the filter control signal generated based on the spatial complexity value information of the previous frame calculated as described above will be described in detail later.

Next, the quantized DCT transform coefficients on line L13 are sent to entropy coding block 130 and image decoding block 150, respectively. Here, the quantized Dct transform coefficients provided to the entropy coding block 130 are encoded, for example, through a variable length coding scheme, and provided to the transmission buffer 140 on the output side. It is delivered to the transmitter not shown for the transmission of.

On the other hand, the quantized DCT transform coefficients on the line L13 provided from the image coding block 120 to the image decoding block 150 are converted into a frame signal reconstructed again through inverse quantization and inverse discrete cosine transform, and then adder 160. The adder 160 adds the reconstructed frame signal from the image decoding block 150 and the predicted current frame signal provided from the current frame side block 180 described later through line L19 to reconstruct the previous image. A frame signal is generated, and the previous frame signal reconstructed as described above is stored in the second frame memory 170. Accordingly, the immediately previous frame signal for every frame encoded through this path is continuously updated, and the reconstructed previous frame signal thus updated is provided to the current frame side block 180 described below for motion estimation and compensation. do.

In addition, the reconstructed reconstructed previous frame signal stored in the second frame memory 170 is provided to the spatial complexity calculation block 210 described below through line L16 for calculating the reconstruction degree of the frame according to the present invention.

On the other hand, in the current frame side block 180, the current frame signal in which the high frequency component on the line L11 provided from the low pass filtering block 230 according to the present invention is selectively removed or the high frequency component has not been removed and the above-described product is removed. Based on the reconstructed previous frame signal on the line L15 provided from the two-frame memory 170, a predetermined search range (for example, 16 × 16 or 32 × 32 search range) of the previous frame reconstructed using the block matching algorithm is predetermined. Predicts the current frame in units of blocks of (e.g., 8x8 or 16x16 DCT blocks) and then generates a predicted current frame signal on line L19 to the subtractor 110 and adder 160, respectively. Is provided. At this time, the switch SW on the line L19 is turned on / off according to the control signal CS from the system controller (not shown). When the switch SW is on, the current encoding mode is the inter mode. In contrast, when off, the current encoding mode is intra mode. Accordingly, the subtractor 110 provides an error signal between the current frame signal and the predicted frame signal to the image encoding block 120 during inter-mode encoding, and transmits the current frame signal itself to the image encoding block 120 during intra-mode encoding. to provide.

The current frame side block 180 also generates a set of motion vectors for each selected block (8x8 or 16x16 blocks) on line L17 and provides it to the entropy coding block 130 described above. Here, the sets of detected motion vectors are predicted in a block (8x8 or 16x16 block) of the current frame and a preset search area (e.g., 16x16 or 32x32 search range) in the previous frame. It is the displacement between the most similar blocks. Accordingly, in the above-described etropy coding block 130, the quantized DCT change coefficients on the line L13 together with the sets of the motion vectors on the line L17 are generated by encoding the encoded bit stream through, for example, a variable length encoding technique. do.

Meanwhile, according to the present invention, the spatial complexity calculation block 210 calculates the spatial complexity of the image for the restored previous frame provided from the second frame memory 170 through the line L16, that is, the variance value of the image signal (standard deviation). Calculate the spatial complexity related to the information amount of the video signal to be encoded, and refer to the calculated spatial complexity of the previous frame as the image complexity of the frame to be currently encoded. Based on the spatial complexity of the video encoding block 120, the high frequency component relatively insensitive to human visual characteristics is adaptively (or selectively) removed (ie, filtered) before the quantization step in the image encoding block 120.

In the present invention, the variance value (standard deviation) of the video signal is used when calculating the spatial complexity of the reconstructed previous frame. In the case of the variance value used here, when the value is large, the result of performing the DCT is a high frequency component The distribution of transform coefficients is inadequate for compressing the data because it will contain many (components with relatively insensitive human visual characteristics). A good image for ideal data compression is the case where there is no high frequency component but only DC component, and the distribution of the transformed coefficient has only one value at the position of (0,0). In addition, when the variance value is large, motion compensation may not be performed properly, and as a result, the structure of the motion compensated image is not suitable for encoding. Therefore, such spatial complexity can be interpreted as the amount of information of the video signal, that is, the amount of data to be encoded and transmitted.

For example, assuming that one frame has a size of M × N, Ip is a picture of a previous frame reconstructed and reconstructed after encoding, and Ic is a picture of a current frame input for encoding. The pixel value of the Ip image at the position of x, y) is Ip (x, y). At this time, the complexity of the currently input Ic image is calculated for the image signal of the previous frame reconstructed after coding (DCT, quantization) for motion estimation and compensation. ACT) can be used.

In Equation (1), MIp means an average value for the reconstructed previous frame Ip image, and the average value MIp for this Ip image is calculated as follows (2).

As described above, the reason for using the ACT value of the previous frame reconstructed as the complexity of the current frame to be encoded is that the characteristics of the video signal do not change rapidly every frame.

Therefore, the calculated complexity ACT value is related to the amount of information (bit generation amount) of the image. If the current encoded image is complex, the calculated complexity ACT value will be relatively large, and vice versa The complexity, ACT, will be small.

In view of this aspect, the complexity ACT value between the restored previous frames calculated at the time of encoding may be interpreted as a value related to the amount of information to be transmitted. In the calculation of the complexity ACT value of the reconstructed previous frame according to the present invention, the encoding system stores the reconstructed previous frame signal required in the process of performing motion estimation and compensation in the frame memory. Using part of, you can easily get the complexity ACT value of the previous recovered frame. Then, the complexity ACT value calculated in the spatial complexity calculation block 210 is provided to the next filter control block 220.

On the other hand, the filter control block 220 generates a filter control signal for the one-dimensional filtering of the input image on the line L23 based on the complexity ACT value provided from the filter control block 210 to generate a low pass filtering block 230. In other words, in the filter control block 220, if the average complexity ACT value input from the spatial complexity calculation block 210 is greater than a predetermined threshold value, a high level logic signal is generated to the low pass filtering block 230. If the calculated average ACT value is smaller than the predetermined threshold value, a low level logic signal is generated and provided to the low pass filtering block 230. In this case, the preset threshold may be set to an average value of the ACT values generated over a predetermined time in the course of encoding. As an example, when the frame rate of the video signal is 30, 30 QE values calculated for 1 second may be averaged and compared with this value and the ACT value generated every frame. That is, when the average value of the ACT value generated during the previous 30 frames is C, the filter control block 220 generates a high level logic signal as the filter control signal on the line L23 if it is larger than the ACT value C generated in the current frame. If small, a low level logic signal is generated as the filter control signal on the line L23. Of course, the ACT average value C for the previous 30 frames will be continuously updated by the ACT value for every frame that is generated, at which time the oldest ACT value of the ACT values for the previous 30 frames is discarded.

Meanwhile, the low pass filtering block 230 passes the input video signal as it is, based on the filter control signal C from the filter control block 220 (that is, the subtractor 110 and the current frame through the line L11). Image signal is removed on the line L11 by removing the high frequency component which is relatively insensitive to human visual characteristics by limiting the pass band by directing to the side block 180) or by applying one-dimensional low pass filtering to the input image signal. do. That is, the low pass filtering block 230 removes high frequency components from the input image through one-dimensional low pass filtering when the filter control signal on the line L23 is a high level logic signal, and the filter control signal on the line L23 is low level logic. In the case of a signal, a high frequency component is removed from the input image without being removed.

More specifically, the low pass filtering method of an input video signal according to the present invention is performed by multiplying a low pass filter by an input video signal. That is, when the value of each pixel for an image of one block of N × M (for example, 8 × 8 block) is f (x, y), one block is shown as shown in FIG. 2 as an example. In the case of an 8x8 block, the position values x and y of the pixels in the horizontal and vertical directions have integer values between 0 and 7, and each value has a level value between 0 and 255. That is, as can be seen from FIG. 2, the position values in the horizontal and vertical directions of each pixel of the 8x8 block have a value of f (0,0) to f (7,7).

On the other hand, as the low pass filter in the present invention, as shown in FIG. 3 as an example, it is assumed that the one-dimensional low pass filter coefficient has seven orders. The low pass filter is an example of a low pass filter that band-limits the signal to have a frequency bandwidth of fs / 4 because the frequency bandwidth is fs / 2 when the sampling frequency of the input video signal is fs.

Accordingly, in the low pass filtering block 230, the low pass filtering of the video signal according to the present invention is performed by performing one-dimensional low pass filtering on each direction since the input video signal is a one-dimensional signal in the horizontal and vertical directions. Can be implemented. This process is illustrated in detail in FIG. 4, which shows the horizontal filtering at position (0.4) and the vertical filtering at position (3,0). That is, if the low pass filtered signal in the vertical direction at the position of (c, y) is z (x, y), it is calculated by the following expression (3).

In the above expression (3), T means the order of the filter, so T = 7. Thus, u and v values have integer values between -3 and 3. In the above expression (3), the k (u) value is a filter coefficient value, and the f (x, y) value is a pixel value. In the above Equation (3), if the value of f (x, yu) is less than 0, the value is 0. If the value is larger than the value of M-1 corresponding to the image size of one frame, the value of M-1 is M-1. Do it. This is a method of setting the end value when the pixel value exists only in the area, that is, 0 to M-1, and this filtering method is a technique known in the art.

Therefore, if filtering is performed at all pixel positions as in Equation (3), as shown in FIG. 4, for example, one-dimensional filtering in the horizontal and vertical directions can be obtained. In this case, according to the present invention, in the process of performing such filtering at all pixel positions, the horizontal filtering may be performed first, and the vertical filtering may be performed again or the order of the horizontal filtering may be changed again. .

As a result, the low pass filtering block 230 restricts the pass band by applying one-dimensional low pass filtering to the input video signal when the filter control signal on the line L23 is high level, that is, when it is determined that the image complexity is large. Thus, an image signal from which a high frequency component relatively insensitive to human visual characteristics is removed is generated on the line L11 and provided to the subtractor 110 and the current frame prediction block 180 of FIG.

Accordingly, in the image encoding block 120 of FIG. 1, in the case of a complex image, encoding (quantization) is performed in a state in which a high frequency component of an image that is relatively insensitive to human vision is removed through one-dimensional low pass filtering as described above. Therefore, the low frequency signal, which is a visually important component, can be encoded with less quantization error. If the image is not a complex image but does not limit the passband of the frequency (removing the high frequency components) as in the present invention, as a result, the amount of bits generated after encoding increases and the quantization step size becomes large. In the low frequency band), a lot of quantization errors are generated, and as a result, image quality deterioration due to quantization will be caused in the playback image of the receiver.

As described above, according to the present invention, the complexity of the input image to be currently encoded is calculated by using spatial complexity information calculated for each frame by using a previous frame reconstructed and reconstructed after encoding for motion estimation and compensation during encoding. If the current input image is a complex image, the high frequency component of the image that is insensitive to human vision is first removed based on the calculation result, and then MC-DCT, quantization, and the like are encoded. By doing so, it is possible to effectively adjust the amount of bits generated after encoding without excessively increasing the step size in the quantization step. Therefore, according to the present invention, it is possible to effectively reduce image quality deterioration due to quantization error inevitably present in a reproduced image when reconstructing and displaying an encoded image.

Claims (5)

Compression means including discrete cosine transform, quantization, and entropy encoding for the error signal between the input current frame and the prediction frame obtained through motion estimation and compensation in units of macroblocks using the current frame and the reconstructed previous frame. A video encoding system having a bit generation amount adjusting function of encoding and generating a coded bit stream, wherein the quantization is adjusted based on the full state information of the bit stream stored in an output buffer. Computing a spatial complexity value for each of the previous frames reconstructed for compensation, and calculating an average spatial complexity value for a plurality of preset previous frames by averaging the spatial complexity values of the respective previous frames. Way; The calculated average spatial complexity value of the plurality of frames is referred to as the complexity of the frame to be currently encoded by the encoding means, and the frequency of the current frame input for encoding based on the detected spatial complexity value of each frame. Control means for generating a filter control signal for limiting the pass bandwidth; And based on the generated filter control signal, one-dimensional low pass filtering is applied to the encoding means as a current frame signal for motion estimation and compensation to remove the high frequency component. And a one-dimensional low-pass filtering means for providing the frame signal to the encoding means as a current frame signal for motion estimation and compensation, or further comprising a one-dimensional low pass filtering means for the input current frame. The filter control signal of claim 1, wherein the filter control signal comprises: an average error value for a plurality of frames transmitted per second obtained by averaging the calculated spatial complexity value of each frame, the spatial complexity value of each frame, and the standard of the average error value. An image encoding system having a bit generation amount adjustment function, which is determined according to a deviation. The image encoding system of claim 2, wherein the filter control signal is a logic signal of a high or low level. The image encoding system of claim 1, wherein the one-dimensional low pass filtering of the current frame is performed in units of 8 × 8 blocks. 5. The method of claim 1 or 4, wherein the one-dimensional low pass filtering for the 8x8 block is performed sequentially in the horizontal-vertical or vertical-horizontal direction of each pixel position in the 8x8 block. An image encoding system having a bit generation amount adjustment function.