KR20010034437A - Signal processing apparatus and method - Google Patents

Abstract

Signal processing device comprising a test circuit (EXAM), an adjustable filter (FIL) and a signal processor (PRC). The signal processor (PRC) is, for example, a video encoder for encoding a series of pictures according to the MPEG standard. The test circuit EXAM tests the signal S to be processed to obtain the distortion indication IND. The distortion indication IND is determined if the signal S is processed by the signal processor PRC (PRC [ S]) indicates to what extent distortion (DIST) will be introduced. The adjustable filter FIL filters the signal S in dependence on the distortion indication IND to obtain a filtered signal SF. The signal processor PRC processes the filtered signal SF. Thus, the signal processing apparatus filters the signal to be processed in a proactive manner to calculate the distortion introduced otherwise by processing the signal. Thus, satisfactory signal quality can be obtained. For example, in video encoding applications the filter may reduce the details contained in a series of pictures. This allows a series of pictures to be coded with sufficient accuracy without introducing block effects that would occur if the pictures were not filtered. Moreover, since the filter is adjusted proactively, the series of images is filtered more uniformly than when the filter is retroactively adjusted. Thus, there will be a relatively small change in resolution from one picture to another, which contributes to a satisfactory overall picture quality.

Description

Signal processing apparatus and method

Document JP-A-09,009,260 describes a dynamic picture encoding device. The following is an exact citation of the English abstract. For AD converted image data, the low frequency components are passed by a two-dimensional block device in a variable low-pass spatial filter, and the high frequency encoding process is performed in the compressor 13, which are temporarily stored in a FIFO memory, and the FIFO memory The amount of compressed data storage therein is controlled by the FIFO control device. Control data for each block device from the compressor and FIFO stored information, which is block activity information, are input to a band controller, and a cutoff frequency for each block is set and notified to the variable low-pass spatial filter from the band controller. .

The present invention relates to signal processing, for example, to coding a series of pictures according to a standard defined by the Moving Picture Experts Group (MPEG).

1 is a conceptual diagram illustrating the basic features of the invention as claimed in claim 1;

2 is a block diagram illustrating an example of a video encoder.

3 is a block diagram illustrating a video coding apparatus according to the present invention.

FIG. 4 is a graph showing a first filter control characteristic for the video coding apparatus shown in FIG.

FIG. 5 is a graph showing a second filter control characteristic for the video coding apparatus shown in FIG.

It is an object of the present invention to provide signal processing that allows for better quality.

The present invention contemplates the following aspects. Processing of the signal can cause distortion. For example, coding a series of pictures is a form of signal processing that causes distortion. In order not to exceed any limit by the amount of coded data obtained, it is necessary to code certain pictures with relatively low accuracy. If a picture is coded with relatively low accuracy, the picture is distorted in the sense that it occurs at the end where block effects are decoded.

It is possible to filter the signal before processing and calculate the distortion caused by processing the signal. However, if the signal is filtered, it is also distorted in the sense that some signal elements are weakened or even eliminated. Therefore, if the signal is filtered before being processed, a tradeoff is made between the distortion caused by the filter process and the distortion caused by the process. For example, if an image is filtered, the resulting filtered image will contain fewer details than the original image. As a result, the filtered picture can be coded with higher accuracy than the original picture, while taking into account some limitations by the amount of coded data obtained. However, the filtered image will have a lower resolution than the original image. Therefore, filtering the picture before it is coded creates a tradeoff between coding accuracy and picture resolution, because both affect the overall quality of the picture.

In the background art, the spatial low pass filter, which filters the image data before being coded, appears to be controlled by the action of the amount of coded data contained in the FIFO output buffer. As a result, it does not initially prevent the production of a relatively large amount of coded data. If a relatively large amount of coded data is actually produced, the picture data yet to be coded will be relatively largely distorted. This is because the image data must be filtered relatively large, coded with relatively low accuracy, or both in order to prevent the limit being exceeded by the amount of coded data obtained. Therefore, the picture data still to be coded will have a relatively inferior quality, since the resolution is low, there are some block effects or both.

According to the invention the test circuit tests the signal to be processed by the signal processor to obtain a distortion indication. The distortion indication indicates how far distortion will be introduced if such a signal is processed by the signal processor. The adjustable filter filters the signal in dependence on the distortion indication to obtain a filtered signal. The signal processor processes the filtered signal.

Therefore, in the present invention the adjustable filter is adjusted in a proactive manner, while in the background the spatial low pass filter is adjusted in a retrospective manner. Thus, there is no substantial delay in making a compromise between the distortion generated by the process and the distortion generated by the filter process in the present invention, whereas in the background art there are essentially some delays in achieving such a compromise. have. Therefore, the present invention allows for a better compromise between the two types of distortions. As a result, the present invention allows for better quality.

For example, the present invention can be applied to coding a series of pictures. Assume that the test circuit tests the picture to be coded or a part from it. In doing so, it can be indicated that a relatively large amount of coded data will be obtained if such pictures are coded. As mentioned above, this potentially causes distortion of one or more pictures to be subsequently coded. Since the present invention allows a picture to be filtered depending on the display, the coding of the picture can be prevented from actually producing a large amount of coded data. Thus, this contributes to the quality of the one or more subsequent pictures. Therefore, in fact, the present invention permits a quality exchange condition between the image tested and the one or more subsequent images. Thus, the present invention allows for a satisfactory and homogeneous picture quality.

DETAILED DESCRIPTION The present invention and additional features to be used at will in order to advantageously implement the present invention will be apparent and elucidated with reference to the drawings described below.

First of all, we will make some references to the use of reference symbols. Similar entities are represented by the same letter code throughout the drawings. Various similar entities will be shown in a single drawing. In this case a number is added to the character code to distinguish similar entities from each other. The number will be between parentheses if the number of similar entities is a continuous variable. Any number of reference signs in the above description and claims will be omitted where appropriate.

1 shows the basic features of the invention in solid lines. The signal processing apparatus includes a test circuit EXAM, a preprocessor FIL and a signal processor PRC. The test circuit EXAM tests the signal S to be processed to obtain the distortion indication IND. The distortion indication IND indicates to what extent distortion DIST is to be introduced if the signal S is PRC [S] processed by the processor PRC. The preprocessor FIL preprocesses the signal S depending on the distortion indication IND to obtain a preprocessed signal SF. The signal processor PRC processes the preprocessed signal SF.

The features shown in FIG. 1 can be applied, for example, to coding a series of pictures. In the application the signal S is a data stream representing the series of pictures to be coded. The signal processor PRC is a video encoder that distorts one or more pictures to a relatively large range. This will be explained in more detail below.

2 shows an example of a video encoder. The video encoder provides an MPEG data stream OUT having a certain bit rate R in response to an image data stream IN representing a series of pictures. The image data stream IN is obtained by, for example, converting an analog video signal into a digital video signal. The video encoder includes the following functional blocks: input buffer IBUF, subtractor SUB, memory MEM, motion estimator, compensator MEC, discrete cosine transformer DCT, inverse discrete cosine transformer IDCT, quantizer Q, inverse quantizer IQ, adder ADD, variable length encoder VLC, output buffer OBUF and controller CON.

The video encoder shown in FIG. 2 codes most pictures in the following manner. The input buffer IBUF stores a picture to be temporarily coded. The picture to be coded is, in fact, subdivided into large blocks of 16 by 16 pixels. It is assumed that the memory MEM includes a decoded version of a previously coded picture which will be referred to below as the previous picture. For each large block of the picture to be coded, the motion estimator and compensator MEC find a similar large block of 16 by 16 pixels in the previous picture. The motion estimator and compensator MEC feed the found large block to the subtractor SUB. The subtractor SUB subtracts from the large block to be coded a similar large block provided by the motion estimator and compensator MEC to provide a differential large block. The discrete cosine transformer DCT converts the differential large block into frequency coefficients of one block.

The quantizer Q divides each frequency coefficient by a value proportional to the quantization parameter QP and rounds the result of the division up to the nearest integer. Thus, the quantization device Q provides a block of quantized frequency coefficients. The inverse quantization device IQ and the inverse discrete cosine transformer ICDT provide a decoded version of the differential large block at the basis of the block of quantized frequency coefficients. The adder ADD adds similar large blocks in the previous picture, found by the motion estimator and compensator MEC, to the decrypted version of the differential large blocks. Thus, the adder ADD provides a decrypted version of the large block being coded, which is stored in the memory MEM. The memory MEM will therefore contain a decoded version of the picture currently being coded if all the large blocks contained in the picture are coded. As a result, in this way, it is achieved that the decrypted version of the preceding picture is included in the memory MEM when the picture is coded.

The variable length encoder VLC converts a block of quantized frequency coefficients into a series of variable length codes. The block of quantized frequency coefficients generally includes a number of frequency coefficients having a value of zero, where the higher number as the quantization parameter QP has a higher value. As a result, a series of variable length codes contain relatively few bits if the quantization parameter QP has a high value and vice versa. The variable length codes are temporarily stored in the output buffer OBUF. The output buffer OBUF outputs variable length codes belonging to already coded large blocks at bit rate R.

The controller CON controls the quantization parameter QP based on the following standard. The amount of coded data contained in the output buffer OBUF, which will next be referred to as output buffer full F, must be between the minimum and maximum levels. This means that if the output buffer OBUF is relatively full, the quantization parameter QP will generally have a relatively high value. If the output buffer OBUF is relatively empty, the quantization parameter QP will have a relatively low value.

More specifically, the controller CON calculates a target amount of coded data or a portion thereof for the picture to be coded. The target amount of coded data depends on the output buffer full F. In other words, if the output buffer OBUF is relatively full, the target will be relatively low and vice versa. The controller CON calculates a value for the quantization parameter QP based on the target amount of coded data and recent coding experiences. These recent coding experiences include the amount of data contained in the recently coded portions of the picture and the value of the quantization parameter QP applied to the coding of that portion. Assume that the recently coded portion of the picture contains a relatively large amount of coded data. From this the controller CON infers that the quantization parameter QP should have a slightly lower value than the most recent value to prevent the target amount of coded data from being exceeded.

The video encoder shown in FIG. 2 distorts the picture relatively largely if the picture or a portion thereof is coded with a quantization parameter QP having a relatively high value. If the quantization parameter QP has a relatively high value, the number of different values that the quantized frequency coefficient will have is relatively low. Moreover, in practice, many frequency coefficients will be rounded to zero values. As a result, the picture will be relatively largely distorted at the end of decoding. For example, an object that includes a relatively large number of details, or a portion thereof, will be represented by relatively large and uniform blocks. This phenomenon is referred to below as the block effect.

For example, block effects occur when there is a scene change in a series of pictures. In general, the first picture of the new scene will be relatively different from the last picture of the preceding scene. Thus, if the first picture of the new scene is coded, many blocks of frequency coefficients will contain many frequency coefficients with substantial values. If the blocks are quantized with a quantization parameter QP with a name only value, a relatively large amount of coded data will be obtained. To prevent this, the controller CON will give a high value to the quantization parameter QP. As mentioned above, this causes a block effect.

3 shows an example of a video encoding apparatus according to the present invention. The video encoding device retrieves the features shown in FIG. This provides an MPEG data stream MDS in response to a picture data stream PDS representing a series of pictures. The video encoding device comprises a pseudo video encoder PENC, a filter controller FCON, a delay circuit DEL, an adjustable filter FIL and a video encoder ENC. The video encoder ENC and pseudo video encoder PENC are similar to and detailed in the video encoder shown in FIG. However, the controller CON included in the pseudo video encoder does not control the quantization parameter in the same way as described above with reference to FIG. This will be explained in more detail below.

The video encoding apparatus shown in FIG. 3 basically works in the following manner. Said pseudo video encoder PENC, so to speak, codes the picture to derive a complexity indication CI indicating how difficult it is to code the picture. The filter controller FCON controls the adjustable filter FIL depending on the complexity indication CI. The delay circuit DEL compensates for the time used by the pseudo video encoder PENC to code the picture and the time used by the filter controller FCON to control the adjustable filter FIL. The adjustable filter FIL filters the picture to be coded with the cutoff frequency FC determined by the complexity indication CI. The video encoder ENC codes the filtered picture.

There are various ways in which the pseudo video encoder PENC provides a complexity indication CI. For example, the pseudo video encoder codes the picture with a quantization parameter QP with some fixed value. If the amount of coded data is relatively large, it can be said that the picture is difficult to code, and vice versa. Thus, the amount of coded data obtained by coding an image constitutes a complexity indication CI. The pseudo video encoder PENC also encodes the picture with the quantization parameter QP being controlled based on the target amount of coded data established at the encoder ENC. Given a relatively high value for the quantization parameter QP to reach the target, one can say that the picture is difficult to code, and vice versa. Thus, the quantization parameter QP constitutes a complexity indication CI. It is also possible to use the product of the quantization parameter QP and the amount of coded data obtained as a complexity indication. In this case it is not a question how the pseudo video encoder PENC establishes the value for the quantization parameter QP.

The complexity indication CI is effectively a distortion indication. If the complexity indication CI has a high value, which means that the image is difficult to code, the following occurs. The video encoder ENC gives a relatively high value to the quantization parameter QP, preventing a relatively large amount of coded data from being produced. In this case, block effects occur in the image of the end portion to be decoded as described above. The video encoder ENC gives the quantization parameter QP a value low enough to prevent such block effects. In this case, however, a relatively large amount of coded data will be produced, which will affect the quality of one or more subsequent pictures as described above. Thus, if the complexity indication CI has a high value, the picture itself or one or more subsequent pictures will be distorted, and both will be distorted if the encoder ENC attempts to code such a picture.

4 shows filter control characteristics applied by the filter controller FCON. 4 is a graph with a horizontal axis indicating the complexity indication CI and a vertical axis indicating the cutoff frequency FC of the adjustable filter FIL. The graph shows that if the complexity indication CI is below the threshold Cth, the cutoff frequency has a relatively high value that does not substantially change as a function of the complexity indication CI. Thus, if the picture is relatively easy to code, it will be filtered over a relatively small range, so that the details of the picture will be substantially preserved. However, if the complexity indication CI is above the threshold, the cutoff frequency decreases as the value of the complexity indication CI increases. Thus, if the picture is relatively difficult to code, it will be filtered to the extent that depends on how difficult the picture is to code. In other words, if the picture is relatively difficult to code, the details of the picture will be removed to the extent that depends on how difficult the picture is to code. This allows the video encoder ENC to code the filtered picture with a quantization parameter QP having a value low enough to prevent block effects in the picture itself, while the amount of coded data obtained allows satisfactory quality of subsequent pictures. Low enough to do.

5 shows another filter control characteristic to be applied at will in combination with the control characteristic shown in FIG. 4. 5 is a graph having a horizontal axis indicating an image number and a vertical axis indicating a cutoff frequency FC. The image number N indicates an image in which the complexity display CI is established as described above. The complexity indication CI of the picture is assumed to have a relatively high value, which means that the picture is difficult to code. As a result, the spatial filter FIL has a relatively low cutoff frequency FClow when picture number N is filtered. 5 shows that as a result, subsequent pictures with numbers N + 1, N + 2, M + 3 are also filtered with a cutoff frequency that gradually increases as the picture number increases.

The filter control characteristic shown in FIG. 5 is advantageous when a scene change occurs, for example. Assume that image number N is the first image of a new scene. Since the first picture of the new scene is filtered with a relatively low cutoff frequency FClow, relatively many details will be removed. In other words, at the end being decoded, the first picture of the new scene will have a relatively low resolution. As the cutoff frequency FC gradually increases with the picture number in the new scene, the resolution will be progressively better with each subsequent picture. The development of this resolution is consistent with the properties of human vision. Humans first pay attention to the basic shapes and colors of new objects and then to the details of the new objects. Thus, in the case of scene change, the filter control characteristic shown in FIG. 5 will affect the sensory resolution only in a relatively small range. Moreover, the control characteristic does not produce a relatively large amount of coded data and allows the first pictures of the new scene to be coded with a quantization parameter QP having a value high enough to avoid block effects. Thus, the control characteristic shown in FIG. 5 contributes to a satisfactory overall quality.

The drawings and the descriptions thereof illustrate rather than limit the invention. It will be apparent that there are numerous alternatives within the scope of the appended claims. In this case the following last comment is mentioned.

There are numerous ways of physically spreading functions and functional elements for various devices. In this case, the drawings are very schematic, each representing only one possible embodiment of the invention. Thus, while the figures show different functional elements as different blocks, this is not to exclude that any functional element or all functional elements are effective as a single physical device.

Although the video coding apparatus shown in FIG. 3 uses a pseudo video encoder PENC to provide a distortion indication in the form of a complexity indication CI, other circuits are used to provide the distortion indication. For example, a circuit is used, which establishes the sum of the absolute differences between the pixels of the picture to be coded and the corresponding pixels of the previous picture. If the sum of the absolute differences is high, the picture is difficult to code, and therefore the picture itself is distorted or one or more pictures, or both are distorted. So the sum of the absolute differences is a distortion indication.

In the video encoding apparatus shown in FIG. 3 the adjustable filter FIL filters pictures and controls on a picture-by-picture basis, but parts of pictures are also filtered and adjusted on a picture-by-part basis.

For the adjustable filter FIL shown in FIG. 5, the following is also noted. In principle, any type of filter can be used. For example, the adjustable filter FIL is a spatial filter or a horizontal filter. That is, it is a two-dimensional filter or a one-dimensional filter. The adjustable filter FIL is effective, for example, as an average filter, a Gaussian filter or an intermediate filter.

The adjustable filter FIL is also effective as a wavelet domain filter. The wavelet domain filter may be based on a de-blocking algorithm for JPEG compressed images. In that case, the filter performs edge detection and flattening to preserve the detected edges in the wavelet domain. The edge detection is based on the threshold determined by the pseudo video encoder PENC shown in FIG. The threshold is thus adapted to the complexity of the current picture to be encoded.

The wavelet domain filter calculates block effects by filtering only regions of the image where the effect occurs. Thus, the wavelet domain filter will reduce the resolution of the picture in a lesser range than the low pass filter. Since the wavelet domain filtering process is applied to picture-by-picture or field-by-field, the JPEG algorithm is more useful for calculating block effects. Such an algorithm provides satisfactory de-blocking performance and is relatively simple.

In view of the foregoing, it will be apparent that the term "filter" is not limited to signal processing characterized by any frequency response. The term “filter” should be interpreted broadly to encompass various forms of pretreatment that will reduce distortion that would otherwise be introduced by the processor.

No reference sign in the claims should be construed as limiting the related claim.

Claims (5)

In the method for processing the signal (S) by the signal processor (PRC), If the signal S is processed by the processor PRC (PRC [S]), the signal S to be processed to obtain a distortion indication IND indicating to what extent distortion DIST is to be introduced. EXAM step, Preprocessing (FIL) the signal (S) in dependence on the distortion indication (IND) to obtain a preprocessed signal (SF); Processing (PRC [SF]) the preprocessed signal SF by the processor PRC. In the signal processing apparatus comprising a signal processor (PRC), If the signal S is processed by the processor PRC (PRC [S]), the signal S to be processed to obtain a distortion indication IND indicating to what extent the distortion DIST is to be introduced. A test circuit (EXAM) for testing The signal processor PRC is coupled to a preprocessor FIL to process the preprocessed signal SF and to the distortion indication IND to obtain the preprocessed signal SF. And a preprocessor (FIL) for preprocessing the signal (S) in dependence. The method of claim 2, And the preprocessor (FIL) comprises an adjustable filter whose filter characteristics are adjusted depending on the distortion indication (IND). In the method of video encoding, Testing the video data to be encoded (PENC) to obtain a complexity indication (CI) indicating how difficult it is to encode video data, Filtering the video data (FIL) in dependence on the complexity indication (CI) to obtain filtered video data; Encoding the filtered video data (ENC). In the video encoding device, A test circuit (FIL) for testing the video data to be encoded to obtain a complexity indication (CI) indicating how difficult it is to encode the video data, An adjustable filter (FIL) for filtering the video data depending on the complexity indication (CI) to obtain filtered video data; And a video encoder (ENC) for encoding the filtered video data.