KR100602803B1 - Encoding block-organized data - Google Patents

Abstract

The information stream configured as a sequence of blocks is encoded according to a gradual degradation principle. The stream contains temporal non-uniform data processing requests. In particular, with respect to the received block, one or more blocky defined control parameters and associated blocky processing loads are detected (44). Under control of the processing load 46 for one or more previous blocks, one or more control parameters are adjusted 48 for one or more later blocks prior to processing. This lowers the expected load for later blocks in the case of excessive load, and vice versa when detecting sub-standard loads.

Information stream, encoder, block, processor, load

Description

Encoding block-organized data}

The present invention relates to block-structured audio / video data encoding.

Processing of digital audio and video for transmission or storage requires the use of various data compression techniques. Non-limiting examples include the MPEG standard with various versions of audio and video. Another standard is H.261. Such compression is realized in software by Ho-Chao Huang et al., New Generation of Real-Time Software-Based Video Codec: Popular Video Coder II, IEEE TR. Cons.EL. Vol. 42, No. 4, and P. 963-973. Compression and similar operations that run in a mixed software and hardware environment are feasible. The number of operations required for software encoding is difficult to predict. Embodiments herein will be primarily described with reference to video. Currently, compression is generally performed based on groups of pictures (GOPs). In this specification, the term "picture" will be used consistently. According to the actual video standard, the term "picture" may mean "field" as well as "frame". Currently, compression of framed audio or mixed audio / video information streams can be performed in a similar manner. This process must be performed in real time and the processor has to pay a high price by loosing images or parts thereof in case of overload. In order to protect basic system facilities, a gradual degradation that gives up some quality has been widely used in data processing.

As a result, the main object of the present invention is to provide an improved encoding method. Thus, according to one aspect of the invention, the invention features the subject matter described in the characterizing part of claim 1. The invention also relates to an encoder arrangement arranged to realize the method, advantageous aspects of the invention being described in the dependent claims.

The inventors of the present invention have recognized potential values of various control parameters defined for a portion of an image, such as an image or slice, to control the actual processing load. A suitable parameter for video is the amount of redundancy (Q), which indicates the amount of information that is treated unimportant within a block or the entire picture. According to the first aspect of the present invention, the actual processing load is used to predict future processing loads and consequently to adjust one or more control parameters to avoid overload-inflicted losses. .

These aspects and advantages of the present invention will be discussed in detail below with reference to the disclosure of the preferred embodiments.

1 is a block diagram of an apparatus according to the invention.

2 is a first enhanced profile chart.

3 is a second enhanced profile chart.

4 is a policy flowchart.

5 is a typical MPEG configuration diagram.

1 is a block diagram of devices according to the invention, in particular for use with video. First, a simplified version is discussed, where elements 62, 64, 68, 70, 72, 74, 76, 78, 80 are not discussed. The video received at the input 20 consists of pictures with a uniform number of pixels. In the DCT element 22, each picture is divided into a series of video blocks each composed of an array of 8 x 8 pixels. The middle segment of the image is slices consisting of two adjacent horizontal columns of blocks. Each block is discrete cosine transformed (DCT) to produce an array of 8 x 8 digital frequency coefficients.

In a two-dimensional DCT result block, each coefficient is associated with a wave frequency. The upper-left coefficient "00" is associated with an average value corresponding to zero spatial frequency in accordance with the two coordinates. To the right of this position (top-left), the wave is horizontal. Below the first position (top-left), the wave is vertical. In the oblique direction, the wave is directed in a corresponding way with respect to the coordinate directions. Subsequent decoding by inverse DCT will provide lossless reconstruction of the original image.

In Fig. 1, a weighting element 24 introduces weighting factors for each coefficient, taking into account the relatively low sensing capability for small details or high spatial frequencies. The purpose of this weighting is to reduce the data. The weighting factor for the coefficient "00" is one. The weighting factors decrease in all directions away from the coefficient "00". Weighting factors are generally fixed. For decoders, this technique will cause some loss of information that is virtually invisible to humans, even under any good conditions.

In quantizer 26, in order to further reduce the data, apart from coefficient 00, the coefficients are equal for that video block, even in a series of video blocks such as slices, or the entire video picture. Is divided by the equal overlap factor Q. Quotations are substantially clipped to an even threshold, which will drop coefficients that are not greater than the threshold. The processor load for encoding using software is applied to the elements 26 and 28 of FIG. 1 and can be mapped to a single high performance microprocessor such as INTEL Pentium. The Q-value and processor load are usually inversely proportional to each other.

Finally, in coder 28, the resulting coefficients are serialized and subjected to variable length encoding (VLE) in accordance with a Huffman code or similar code. The resulting bitstream is output to output 32. In the computing element 34, the actual processing load is calculated and recombined to the quantizer 26 along the line 30. The latter may adjust the value of Q to keep the processing load in blocks or pictures in the acceptable range.

In the above, the number of clock cycles depends on the content of the image. Differences may occur between various pictures as well as between blocks or slices within a single picture. Thus, the hardware facilities are greatly increased due to the need for handling worst case conditions. The present invention has recognized that throughput may be more dependent on factor Q and other control parameters. It is proposed to adjust Q to control channel bitrate and buffer occupancy. The present invention adjusts the processor load by adjusting Q and possibly other control parameters.

The following method is shown in the flowchart of FIG. At block 40, processing begins to configure and initialize the necessary hardware and software facilities. In block 42, processing begins for a new picture comprising a sequence of video blocks. In block 44, the actual amount of progress is updated. For example, the video block count or video line count is updated in accordance with the configuration of the various video blocks in the picture and also with the progress of time in the time interval assigned to that picture. In block 46 it is checked whether the progress is within the defined range. Initially, the progression is assumed to be linear in time, but depending on the sequence of images, the progression in each of the images may be nonlinear. Further, the progression profile may be non-uniform in sequence. If the progress is within the defined range, block 50 processes the next video block. The defined range may require continuous updating depending on the sequence of video blocks and / or pictures. In block 52, it is actually checked whether the picture is the last video block. If the picture is not the last block, the system returns to block 44. If the picture is the last block, the next picture starts at block 42. However, if the result at block 46 is negative, block 48 updates the amount of redundancy Q, which directly affects the processing load and thereby the next to be checked at block 46. It also affects progress.

In this regard, FIG. 2 shows a first enhanced profile chart, especially when linear. The lines shown represent nominal progression, indicating that all blocks take substantially the same time intervals to be processed. Lines hi and lo represent the upper and lower thresholds, respectively, considered in order to update the actual value of Q. If the process is too slow, the progress remains below the solid line so Q must rise and vice versa. If such a determination is taken solely on the basis of one image, only intercept of the upper edge of the figure is considered. The lines hi and lo have appreciable divergences from the nominal line to avoid too frequent modification of Q. The system must always be designed and controlled in such a way as to maintain a surplus capacity of 20%, in order to cope with unforeseen problems.

3 shows a second enhanced profile chart that is non-linear, which illustrates that the preceding blocks of the picture take more processing time than the later blocks. This has been determined by the sequence of preceding pictures, which can be caused by the non-uniform picture content of the preceding blocks, where an even image part will take less processing than a scene part with many items. Between successive images, both the slopes of the profile and the position of the edges in the profile are maintained. Such non-uniformity within one picture can lead to another way to quantize changes in Q and / or other control parameters. In general, quantization to change Q is a problem for artificial intelligence, and instability should be avoided. Sometimes this adjustment takes one or more images to be completely stable, in that correction steps may be on the smaller side. Also, the time-response to the measurements taken on the whole picture may be different from the measurements taken on a block-to-block basis.

The invention can be used, for example, in an MPEG environment. 5 illustrates an exemplary MPEG structure embodiment represented by a sequence of pictures. MPEG has three categories, namely I, B and P pictures. I pictures contain all the information for reproducing the image. P pictures contain less information than all the necessary information, but require an already processed picture to be reproduced in such a way that it can act on its own as a predecessor to later processed pictures. Finally, B pictures contain less information than all the necessary information, but require one or more already processed pictures to reproduce that image. However, it will still not get an image that can also act on its own as it is ahead of the image to be processed. Therefore, P pictures can be concatenated, but B pictures will always be the last pictures. Coherence is indicated by the arrow.

For MPEG design, Figure 1 is expanded. In particular, the inversely coupled loop includes an inverse quantizer 68 that is opposite of the quantizer 26. In addition, device 70 performs an inverse discontinuous cosine transform (IDCT) that is the opposite of DCT device 22. This result proceeds to adder 72 and is then stored in memory 74. For B and P pictures, the input 77 of element 76 receives one or more motion compensation vectors, so that the motion of the image can be effectively compensated. The resulting picture content is sent to subtractor 60, which operates exclusively for B and P pictures. For I pictures, the switch 62 is closed and the adder 60 is effectively shorted. For B and P pictures, the motion compensated picture is passed by the switch 64 to an adder 72 that adds to the pictures received from the IDCT 70.

Variable length coder 28 outputs a coded information stream to output 32 for storage and transmission. The signals may also go to the computing element 34 and send information about the output 32 of the output bit rate to the bit rate control block 80. The latter will check that the averaged bit rate for the applicable time interval does not exceed the processing and / or buffering capacity of the elements downstream from the output 32. The result is a control signal that can be output side by side with the output 32 in the downstream direction, and can also be combined inversely with the control signal from the calculation element 34 to the logic combination element 78. If the bit-rate load is not excessive, the computing element 34 determines. If the bit rate is too high, the combination element 78 invalidates the control by the calculation element 34.

In addition, modification of the control parameters may include the following. For MPEG streams, B-pictures can optionally be excluded. In addition, coefficient clipping in the weighting block 24 of FIG. 1 may be performed in a more precise or more generous manner. In addition, motion estimation may be performed in a less consuming manner, for example, taking fewer motion vector candidates that take three instead of five.

Further measurements according to the invention are as follows. Load complexity is monitored based on a group of consecutive pictures (GOP), all of which have a value approximately equal to Q, and in MPEG the nature of the various formats of the picture, such as I, P and B pictures, is individually appropriate. It is considered to assign the levels of Q.

The difference signal between the input and the compensated input may be forced to zero through coefficient clipping at the input of the DCT element 22, in any case where the association has a near zero difference. This may be signaled through the output signal of the motion estimation device. In this case, calculations in blocks DCT, Q, IQ, and IDCT may be preceded.
Another measure is to take the immediately preceding I picture along with the next non-I picture and divide the difference vector between the two pictures by two, even when a particular B or P picture cannot be recovered. Get

It should be noted that the above-mentioned embodiments somewhat limit the present invention, and that those skilled in the art can design numerous alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. "Include" does not exclude the presence of other elements or steps than those listed in the claims. The invention can be implemented in hardware means comprising several separate elements, and can be implemented in a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied in one and the same item of hardware.

Claims (11)

A method of encoding an input signal according to a gradual degradation principle to obtain an information stream, wherein the information stream is configured as a sequence of fields or pictures and there is a need for temporal non-uniform data processing. Determining 44 one or more control parameters and an information stream associated processing load, and Adjusting 48 one or more of the control parameters in the information stream before processing 50 under the control of the determined processing load; The information stream consists of fields or pictures each formed from an even sequence of blocks, The processing load is measured with respect to the progression in the sequences of blocks compared to the time progression in the actually assigned field or picture time, The one or more control parameters are adjusted for one or more subsequent blocks in the information stream, under the control of a processing load on one or more previous blocks, Wherein said one or more control parameters comprise an amount of redundancy (Q), and said step of adjusting for said amount of redundancy is performed by a quantizer (26). delete delete delete The method of claim 1, And said processing load is measured by comparing said progression with a corresponding progression for a preceding field or picture. The method of claim 1, Non-evenly temporal progression of the sequence of blocks is allowed. delete The method of claim 1, Monitoring load complexity based on a group of consecutive pictures (GOPs) having values of approximately uniform Q but with various different categories, and And considering the various categories to assign different levels for checking the processing load. The method of claim 1, Dynamically checking whether a difference between the input signal and the motion-compensated version is below a predetermined value, Forcing the difference to zero through coefficient clipping for the processing step waiting for the difference when the difference is less than or equal to the predetermined value;  Signaling previous calculations intended to find the motion-compensated version via a motion estimator. The method of claim 1, Checking whether a particular image can be recovered, and When the particular picture cannot be recovered, initiating finding an average picture between the immediately preceding picture and the next picture. An encoder for encoding an input signal according to a gradual degradation principle in order to obtain an information stream, wherein the information stream is configured as a sequence of fields or pictures and there is a need for temporal non-uniform data processing. Means (34) for determining one or more control parameters and an information stream associated processing load, and Means 26 for adjusting one or more of the control parameters in the information stream prior to processing under control 34 of the determined processing load, The information stream consists of fields or pictures each formed from an even sequence of blocks, Means for measuring the processing load with respect to progression in the sequences of blocks that are actually compared to time progression in the assigned field or picture time, And said adjusting means comprises a quantizer (26) for adjusting the amount of redundancy (Q) for one or more subsequent blocks in the sequence of blocks under the control of said processing load on one or more previous blocks.

Images