Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/177—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/124—Quantisation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/127—Prioritisation of hardware or computational resources
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/136—Incoming video signal characteristics or properties
  • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/146—Data rate or code amount at the encoder output
  • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
  • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

• the invention relates to encoding block-organized audio/video data.
• GOPs Groups of Pictures
• picture will be used consistently.
• the term “picture” may mean “frame” as well as “field”.
• framewise organized audio or mixed audio/video information streams may be effected in similar manner.
• Such processing must be done in real-time, and a high penalty must be paid in case of processor overload, by loosing pictures or parts thereof.
• Graceful degradation has been widely used in data processing, by surrendering a certain degree of quality in order to preserve basic system facilities.
• the invention is characterized as recited in the characterizing part of Claim 1.
• the invention also relates to an encoder device arranged for implementing the above method. Further advantageous aspects of the invention are recited in dependent Claims.
• the inventors have recognized the potential value of various control parameters that are defined over a picture or a part of a picture, such as a slice, for controlling the actual processing load.
• a suitable parameter for video is a redundancy quantity Q that indicates an amount of information to be treated as inconsequential within a block or within an overall picture.
• an actual processing load is used to predict future processing load, and in consequence, to adjust one or more control parameters for avoiding overload-inflicted losses of information.
• Figure 1 shows a block diagram of an inventive apparatus
• Figure 2 shows a first advance profile chart
• Figure 3 shows a second advance profile chart
• Figure 4 shows a policy flowchart
• Figure 5 shows an exemplary MPEG organization.
• FIG. 1 is a block diagram of an apparatus according to the invention, in particular for use with video.
• elements 62, 64, 68, 70, 72, 74, 76, 78, 80 have been left out.
• the video received on input 20 is organized in pictures that have uniform numbers of pixels.
• each picture is divided into a series of video blocks that are each composed from an array of 8X8 pixels.
• An intermediate division of the image is into slices, that consist of two adjacent horizontal rows of blocks.
• Each block is now subjected to a Discrete Cosine Transform to produce an array of 8X8 digital frequency coefficients.
• each coefficient relates to a wave frequency.
• the upper-left coefficient "00" relates to an average value corresponding to zero spatial frequency along both co-ordinates.
• waviness is horizontal. Below the first position the waviness is vertical. In slanted directions, the waviness is oriented in corresponding fashion with respect to the co-ordinate directions. Subsequent decoding by an inverse Discrete Cosine Transform will give a lossless reconstruction of the original image.
• weighting element 24 introduces weighting factors for the respective coefficients, taking into account the relatively lower sensitivity of human perception for smaller details or higher spatial frequencies.
• the object of this weighting is data reduction.
• the weighting factor for coefficient "00" is 1.
• the weighting factors decrease in all directions away from coefficient "00".
• the weighting factors are in general fixed. In a decoder, this technique will cause some loss of information, that is substantially invisible to a human person, even under favorable viewing conditions.
• the coefficients, apart from coefficient 00, are divided by a redundancy factor Q that is uniform for the video block in question, or even for a series of video blocks such as a slice, or for a whole video picture.
• the quotients are subsequently clipped with respect to a uniform threshold value: this will cause dropping coefficients that are not larger than the threshold.
• the processor load for encoding that uses software applies to elements 26, 28 in Figure 1, and may be mapped onto a single high-performance microprocessor such as an INTEL Pentium. Q-value and processor load are about inversely proportional to each other.
• coder 28 the resulting coefficients are serialized and subjected to Variable Length Encoding according to a Huffmann or similar type of code.
• the resulting bitstream is outputted on output 32.
• computing element 34 the actual processing load is calculated, and retrocoupled along line 30 to quantifier 26. The latter may adjust the value of Q to retain the processing load per block or per picture in an allowable range.
• the number of clock cycles depends on the image content. Differences may occur between various pictures, as well as between slices or between blocks within a single picture. A requirement to cope with worst case conditions will therefore cause overdimensioning of the hardware facilities.
• the invention has recognized that the amount of processing may furthermore depend on the factor Q and on other control parameters. It has been proposed to adjust Q for controlling channel bitrate and buffer occupancies. The invention allows to adjust processor load through adjusting Q and possibly other control parameters.
• a policy to be followed has been shown in the flow graph of Figure 4.
• processing is started and necessary hardware and software facilities are organized and initialized.
• processing begins for a new picture that comprises a sequence of video blocks.
• an actual progress quantity is updated. For example, a video block count or a video line count is updated in accordance with the configuration of the various video blocks within the picture in question, and also the advance of time within the time interval that has been assigned to the picture in question.
• Block 46 checks whether the progress is in a prescribed range. Initially it is supposed that progress should be linear in time ⁇ but along a sequence of pictures, the progress within each picture may be non-linear. Also, the progress profile may be non-uniform along the sequence.
• a next video block is processed.
• the range may need continual updating along a sequence of video blocks and/or pictures.
• block 52 it is checked whether it is actually the last video block of the picture in question. As long as it is not the last one, the system reverts to block 44. If it is the last video block, in block 42 the next picture is started. If the result in block 46 is negative however, in block 48 the quantity Q is updated, which immediately will influence the processing load, and therefore, also subsequent progress to be checked in block 46.
• Figure 2 shows a first advance profile chart, in particular a linear one.
• the drawn line indicates nominal progress, which indicates that all blocks take substantially equal time intervals for being processed.
• Lines high and low indicate the upper and lower thresholds, respectively, that are taken into account for updating the actual value of Q. If processing is too slow, progress remains below the drawn line, so that Q must be raised, and vice versa. If the decision is only taken on the basis of a picture as a whole, only the intercept with the upper edge of the Figure is taken into account.
• the lines hi and lo have appreciable divergences from the nominal line, to avoid all too frequent amending of Q.
• the system should always be designed and controlled in such manner that a certain surplus capacity remains, such as 20% or the like, to cope with unexpected events.
• Figure 3 shows a second advance profile chart, that is non-linear: the present case illustrates that earlier blocks of the picture took more processing time than later blocks. This has been concluded from earlier pictures of the sequence, and may be caused by a non- uniform picture content in the earlier blocks: a uniform image part will take less processing than a scene part with much detail. Between successive pictures, both the slopes of the profile, and also the position of the edge in the profile are retained. Such non-uniformity within a picture may cause a different policy in quantifying the changes in Q and/or other control parameters. In general, the quantifying of changing Q is a problem of cybernetics; instability should be avoided. Often the adjustment takes more than one picture to fully settle down, in that amending steps would be on the smaller side. Also, the time-response for measures taken on a whole picture could be different from measures taken on a block-to-block basis.
• the invention may, by way of example, be used in an MPEG environment.
• FIG. 5 shows an exemplary MPEG organization as expressed in a sequence of pictures.
• MPEG has three categories: I, B, and P pictures.
• I pictures contain all information to reproduce the image in question.
• P pictures contain less than all necessary information, but need an already processed picture to be reproduced in such manner that itself may act as predecessor for a picture processed still later.
• B pictures contain less than all necessary information, but need one or more already processed pictures to reproduce the image in question; however, such will not result in a picture that itself may act again as predecessor for an image to be processed yet.
• P pictures may therefore be chained, but B pictures will always be terminal pictures.
• Coherence has been indicated by arrows.
• Figure 1 has been extended.
• a retrocoupling loop comprises an inverse quantifier 68 that is the opposite of quantifier 26.
• element 70 executes an Inverse Discrete Cosine Transform as opposite to DCT element 22. The result is forwarded to adder 72 and subsequently stored in memory 74.
• input 77 of element 76 receives one or more motion compensation vectors, so that effectively, motion in the image may be compensated.
• the resulting picture content is sent to subtractor 60 that is exclusively operative for B and P pictures.
• switch 62 is closed, and adder 60 is effectively short-circuited.
• the motion- compensated picture is also sent by switch 64 to adder 72 for adding to received pictures from IDCT 70.
• Variable length coder 28 outputs the coded information stream on output 32 for storage or transmission. It furthermore signals progress to computing element 34 and can also send information pertaining to output bitrate 32 to bitrate control block 80. The latter will check whether the bitrate as averaged over an applicable time interval will not exceed processing and/or buffering capacities of elements that are downstream from output 32. The result is a control signal that may be outputted alongside with output 32 in a downstream direction, as well as be retrocoupled together with the control signal from computing element 34 to a logic combination element 78. If the bit-rate load is not excessive, computing element 34 is determining. If the bitrate is too high, combination element 78 will overrule the control by computing element 34.
• control parameters may include the following.
• B-pictures may be selectively left out.
• the coefficient clipping in weighting block 24 of Figure 1 may be executed in a more rigorous, or in a more relaxed manner.
• the motion estimation may be executed in a less exhaustive manner, such as by taking fewer motion vector candidates, e.g. three instead of five.
• the load complexity is monitored on the basis of a group of successive pictures (GOP) that all have approximately the same value of Q, and the nature of the various types of picture, such as I, P and B pictures in MPEG is taken into account for assigning respective appropriate levels of Q.
• GOP group of successive pictures
• the differential signal between input and compensated input may be forced to zero through coefficient clipping, on the input of DCT element 22, in case the associated have a near-zero difference anyway. This may be signalled through an output signal of a motion estimator device.
• the calculations in blocks DCT, Q, IQ, and IDCT may then be foregone.
• a still further measure when nevertheless a particular B or P picture could not be restored, is to take the immediately preceding I picture together with the next following non-I picture, and to divide-by-two the differential vector between the two pictures, so that in fact, an average picture will result.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computing Systems (AREA)
• Theoretical Computer Science (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Compression, Expansion, Code Conversion, And Decoders (AREA)

Priority Applications (1)

Applications Claiming Priority (4)

Publications (1)

Family

ID=8233742

Family Applications (1)

Country Status (6)

Families Citing this family (5)

Family Cites Families (9)

• 1999
  • 1999-05-06 EP EP99915979A patent/EP0998810A1/de not_active Withdrawn
  • 1999-05-06 JP JP2000550261A patent/JP2002516528A/ja not_active Withdrawn
  • 1999-05-06 WO PCT/IB1999/000812 patent/WO1999060772A2/en not_active Application Discontinuation
  • 1999-05-06 CN CNB998007935A patent/CN1201587C/zh not_active Expired - Fee Related
  • 1999-05-06 KR KR1020007000588A patent/KR100602803B1/ko not_active IP Right Cessation
  • 1999-05-19 US US09/315,081 patent/US6498812B1/en not_active Expired - Fee Related
• 2002
  • 2002-05-16 US US10/146,399 patent/US6999515B2/en not_active Expired - Fee Related

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events