TWI408965B - Method, device, and computer-readable medium of improved enhancement layer coding for scalable video coding - Google Patents

Abstract

This disclosure describes scalable video coding techniques. In particular, the techniques may be used to encode refinements of a video block for enhancement layer bit streams in a single coding pass, thereby reducing coding complexity, coding delay and memory requirements. In some instances, the techniques encode each nonzero coefficient of a coefficient vector of the enhancement layer without knowledge of any subsequent coefficients. Coding the enhancement layer in a single pass may eliminate the need to perform a first pass to analyze the coefficient vector and a second pass for coding the coefficient vector based on the analysis.

Description

Method, device and computer readable medium for improving enhanced coding of video coding

The present invention relates to digital video writing, and more particularly to adjustable video writing of video material.

The present application claims the benefit of U.S. Provisional Application Serial No. 60/979,919, filed on October 15, 2007, and U.S. Provisional Application Serial No. 60/940,214, filed on Oct. 16, 2007. The content of each of these applications is incorporated herein by reference.

Digital video capabilities can be incorporated into a wide range of devices, including digital TVs, digital live systems, wireless communication devices, wireless broadcast systems, personal digital assistants (PDAs), laptops or desktops, digital cameras, digital Recording devices, video game devices, video game consoles, cellular or satellite radiotelephones, and the like. Digital video equipment implements video compression technology, such as Moving Picture Experts Group (MPEG)-2, MPEG-4 or International Telecommunication Union Standardization Sector (ITU-T) H.264/MPEG-4 part 10 advanced video writing code (AVC) (hereinafter "H.264/MPEG-4 Part 10 AVC Standard") to transmit and receive digital video more efficiently. Video compression techniques perform spatial and temporal prediction to reduce or remove the inherent redundancy in the video sequence.

In video writing, video compression typically includes spatial prediction and/or motion estimation and motion compensation to produce a predictive video block. In-frame writing relies on spatial prediction to reduce or remove spatial redundancy between video blocks within a given code unit (eg, a picture frame or slice). In other words, the video encoder performs spatial prediction to compress the data based on other data within the same code unit. In contrast, inter-frame coding relies on temporal prediction to reduce or remove temporal redundancy between video blocks of successive video frames of a video sequence. Thus, for inter-frame coding, the video encoder performs motion estimation and motion compensation to track the movement of matching video blocks of two or more adjacent code units.

After spatial or temporal prediction, the block of residual coefficients (referred to as the remaining block or remaining information) is generated by subtracting the predicted video block from the original video block being coded. The remaining block may be a two-dimensional matrix that quantizes the coefficient values of the difference between the predicted video block and the original block. The video encoder can apply transform, quantization, and entropy write procedures to the remaining blocks to further reduce the bit rate associated with communication of the remaining blocks. Transform techniques may include discrete cosine transform (DCT), wavelet transform, integer transform, or other type of transform.

In DCT transform, for example, the transform process converts a set of pixel domain coefficients into transform coefficients that represent the energy of the pixel domain coefficients in the frequency domain or the transform domain. Quantization is applied to the transform coefficients to produce quantized transform coefficients. Quantization typically limits the number of bits associated with any given coefficient. The video encoder entropy encodes the quantized transform coefficients to further compress the quantized coefficient of variation. The video encoder may entropy encode the coefficients using variable length write code (VLC), arithmetic write code, fixed length write code, or a combination thereof. The video decoder can perform an inverse operation to reconstruct the video sequence.

Some video writing standard standards such as MPEG-2 encode video with relatively constant quality, bit rate or spatial resolution. This technique may be sufficient to provide a video application to a device having similar decoder capabilities (eg, memory or processing resources) and/or connection quality. However, most modern video transmission systems typically include devices with varying decoder capabilities and/or connection quality. In such systems, transmitting video encoded with a relatively constant quality, bit rate or spatial resolution results in the video application operating for devices with appropriate decoder capabilities and/or connection quality, and without proper decoder capability. And/or connected quality equipment does not work. In a wireless situation, for example, a device located near a video transmission source may have a higher quality connection than a device located away from the source. Likewise, devices that are located away from the source may not be able to receive encoded video transmitted at constant quality, bit rate, or spatial resolution.

Other video writing standards use adjustable code writing techniques to overcome these problems. (For example) an adjustable video code (SVC) according to an extension of ITU-T H.264/MPEG-4 Part 10 AVC, which refers to a video in which a video sequence is encoded as a base layer and one or more adjustable enhancement layers. Write code. For SVC, the base layer typically carries video material with a basic space, time, and/or quality level. One or more enhancement layers carry additional video material to support higher space, time and/or quality levels. The enhancement layer can, for example, add spatial resolution to the frame of the base layer, or additional frames can be added to increase the overall frame rate. In some examples, the base layer can be transmitted in a more reliable manner than the transmission of the enhancement layer. Likewise, devices that are away from the encoded video source location or have lower decoder capabilities may be able to receive the base layer and thus receive the video sequence even at the lowest space, time and/or quality level.

The present invention describes an adjustable video write code technique that allows entropy coding of enhancement layer bitstreams in a single write once operation. Conventionally, a plurality of write code operations are used to encode the enhancement layer bit stream. For each video block of the enhancement layer, for example, a first write once operation may collect statistics for the block used in the write code table (or codebook) selected for the entropy code block, and The second write once operation may entropy encode the block using the selected write code table. However, in accordance with the teachings of the present invention, the video blocks of the enhancement layer bitstream are entropy encoded without performing a first write once operation that collects statistical data used in the video codebook selection.

The reality is to encode the enhancement layer using a write code technique that encodes the coefficients of the enhancement layer on a coefficient-by-coefficient basis in a single write once operation. In one example, for each of the non-zero coefficients of the enhancement layer video block, the video encoder may encode an end of block (EOB) symbol, a run length, and a sign. The video encoder can encode the video blocks of the enhancement layer using only a single code table, thereby eliminating the need to perform a first write once operation for collecting statistics for selecting a code table.

In addition, the video encoder may not encode the magnitude of the non-zero coefficients in the enhancement layer. In this way, the magnitude of all non-zero coefficients of the enhancement layer can be limited to a magnitude of one. The magnitude of the coefficients that do not encode the enhancement layer can result in some loss in peak signal to noise ratio (PSNR), but reduces the number of bits used to encode the enhancement layer. The techniques of the present invention can provide several advantages. For example, such techniques can reduce write code complexity, write code latency, and memory requirements for encoding enhancement layer bitstreams while maintaining write code efficiency.

In one aspect, a method of encoding video data using an adjustable video write code includes encoding a video block with a first quality as part of a base layer bit stream. The method also includes refining the encoded video block as part of at least one enhanced layer bit stream, the refinement of the video blocks causing a greater than the first quality when combined with the video block encoded with the first quality Two quality video blocks. The method also includes encoding the refinement of the video block in a single coded one operation.

In another aspect, an apparatus for encoding video data using an adjustable video write code includes at least one encoder that encodes a video block as a portion of a base layer bit stream with a first quality, And refining the encoded video block as part of at least one enhanced layer bit stream, the refinement of the video blocks causing a second quality greater than the first quality when combined with the video block encoded with the first quality Video block. The refinement of the video block is encoded in a single coded one operation.

In another aspect, a computer readable medium comprising instructions for causing one or more processors to encode a video block as a portion of a base layer bit stream with a first quality; and encoding the video block As part of at least one enhanced layer bitstream, the refinement of the video blocks results in a video block having a second quality greater than the first quality when combined with the video block encoded with the first quality. The refinement of the video block is encoded in a single coded one operation.

In still another aspect, an apparatus for encoding video data using an adjustable video write code includes: a first component for encoding a video block as a portion of a base layer bit stream with a first quality; and Refining the encoded video block as a second component of at least one portion of the enhanced layer bitstream, the refinement of the video blocks resulting in a greater than the first quality when combined with the video block encoded with the first quality The second quality video block. The refinement of the video block is encoded in a single coded one operation.

In another aspect, a method of decoding video data using an adjustable video write code includes decoding a base layer bitstream to obtain a first quality video block, and decoding the enhancement layer bitstream to obtain a video block. Refinement, the refinement of the video blocks results in a second quality video block when combined with the first quality decoded video block. The decoding enhancement layer includes decoding, for each non-zero coefficient of the refinement of the video block, a symbol indicating the presence of at least one residual non-zero coefficient, a run length indicating the number of zero-valued coefficients before the non-zero coefficient, and a non-zero The sign of the coefficient.

In another aspect, an apparatus for decoding video data using an adjustable video write code includes at least one decoder that decodes a base layer bitstream to obtain a first quality video block, and Decoding the enhancement layer bitstream to obtain refinement of the video blocks, the refinement of the video blocks resulting in a second quality video block when combined with the first quality decoded video block. At least one decoder decodes, for each non-zero coefficient of the refinement of the video block, a symbol indicating the presence of at least one residual non-zero coefficient, a run length indicating a number of zero-valued coefficients before the non-zero coefficient, and a non-zero The sign of the coefficient.

In still another aspect, a computer readable medium comprising instructions for causing one or more processors to decode a base layer bitstream to obtain a first quality video block; and decoding the enhancement layer bitstream to The refinement of the video blocks is obtained, and the refinement of the video blocks results in a second quality video block when combined with the first quality decoded video block. The instructions cause the one or more processors to decode a symbol indicating the presence of at least one residual non-zero coefficient and a number of zero-value coefficients prior to the non-zero coefficient for each non-zero coefficient of refinement of the video block The length and the sign of the non-zero coefficient.

In another aspect, an apparatus for decoding video data using an adjustable video write code includes: a first component for decoding a base layer bitstream to obtain a first quality video block; and for decoding The layer bit stream is enhanced to obtain a second component of the refinement of the video block, and the refinement of the video blocks results in a second quality video block when combined with the first quality decoded video block. The second decoding component decodes, for each non-zero coefficient of the refinement of the video block, a run length indicating the presence of at least one residual non-zero coefficient, a run length indicating a number of zero-valued coefficients before the non-zero coefficient, and a non-zero coefficient The sign of the zero coefficient.

The techniques described in this disclosure can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the software may be embodied in a processor, which may be referred to as one or more processors, such as a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array ( FPGA) or digital signal processor (DSP), or other equivalent integrated or discrete logic circuits. Software containing instructions for performing such techniques may initially be stored on a computer readable medium and loaded and executed by a processor.

Accordingly, the present invention also contemplates a computer readable medium containing instructions that cause the processor to perform any of the various techniques as described in this disclosure. In some cases, a computer readable medium can form part of a computer program product that can be sold to a manufacturer and/or used in a device. The computer program product may include computer readable media and, in some cases, packaging materials.

The details of one or more aspects of the invention are set forth in the drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings.

1 is a block diagram showing a video transmission system 10 that supports video adjustability. In the example of FIG. 1, video transmission system 10 includes a source device 12 and a plurality of destination devices 14A, 14B (collectively "destination devices 14"). Source device 12 obtains digital video content from one or more sources and encodes the video content for transmission to destination device 14. Video content may, for example, be captured, archived (eg, pre-fetched), computer generated, or a combination thereof, on-the-fly or near-instant. In each case, the video content can be encoded by the source device 12 for transmission to the destination device 14 via the communication channel. Source device 12 can include or be coupled to a transmitter that includes appropriate radio frequency (RF) modulation, filtering, and an amplifier component to drive one or more antennas to deliver encoded video via a communication channel.

To support adjustable video, source device 12 encodes the source video as a base layer bit stream (or base layer) and one or more adjustable enhancement layer bit streams (or enhancement layers). The base layer bit stream typically carries video data with a base quality level. One or more enhancement layers carry additional video material referred to herein as refinement to support higher quality levels. Refinement encoded in the enhancement layer can gradually increase fidelity (eg, visual quality), for example, by providing additional higher frequency coefficients or further refining existing coefficients. In some examples, the base layer may be transmitted in a more reliable manner than the transmission of the enhancement layer (eg, at a lower packet error rate (PER)).

In the example illustrated in Figure 1, the base layer of a channel and a single enhancement layer are shown for simplicity. However, source device 12 may encode more than one enhancement layer of the channel carrying additional video material. In some examples, source device 12 may encode the source video in a separate bitstream to support different channels for selection by the user associated with destination device 14. Channels are typically transmitted simultaneously such that destination device 14 can select different channels for viewing at any time. Thus, much like the television viewing experience, the destination device 14 can select a channel to watch sports under user control and then select another channel to watch the news or some other scheduled broadcast program event. In general, each channel can be encoded as a base layer and one or more enhancement layers.

Moreover, for illustrative purposes, the techniques of the present invention are described in the context of quality adjustability (also known as signal to noise ratio (SNR) adjustability). However, the technology can be extended to spatial adjustability. In space-adjustability applications, the base layer carries video data at base space resolution, and the enhancement layer carries additional video data to support higher spatial resolution. In some examples, system 10 may utilize video adjustability that combines SNR, spatial and/or temporal adjustability.

Source device 12 may encode source video as a base layer, for example, according to the SVC extension of the ITU-T H.264/MPEG-4 Part 10 AVC standard, and encode the source video as a reinforcement layer in accordance with the techniques described in this disclosure. Likewise, the techniques as described in this disclosure may be applied in some aspects to implement video scalability scalability of the device, which additionally conforms to the H.264 standard. In fact, the techniques of this disclosure may represent potential modifications for future versions or extensions of the H.264 standard or other standards. However, such techniques may be used in conjunction with any of a variety of other video compression standards, such as those defined in MPEG-1 and MPEG-2, the ITU-T H.263 standard, and the Society of Motion Picture and Television Engineers. (SMPTE) 421M Video CODEC Standard (collectively referred to as "VC-1"), standard defined by the China Audio Video Writing Standards Working Group (collectively referred to as "AVS"), and defined by standard organization or developed by the organization Any other standard video writing standard.

Destination device 14 can support wired and/or wireless reception of encoded video. Destination device 14 may include any device, such as a wireless communication device, capable of receiving and decoding digital video material, such as a cellular or satellite wireless telephone, a wireless broadcast system, a personal digital assistant (PDA), a laptop, or a desktop computer. , digital cameras, digital recording devices, video game devices, video game consoles, digital TVs, digital live broadcast systems, and the like. In the example of Figure 1, two destination devices 14A, 14B are shown. However, system 10 can include any number of destination devices 14. Destination device 14 may also operate in accordance with any of the various video compression standards described above.

Figure 1 shows the location of destination device 14 relative to source device 12 that transmits encoded video. In particular, destination device 14A is closer to the transmission source (i.e., source device 12 in Figure 1), and destination device 14B is remote from the transmission source. In the case of encoding the base layer with a lower PER, the two destination devices 14A and 14B can reliably receive and decode the base layer. The destination device 14A positioned closer to the source device 12 can also reliably receive the reinforcement layer. However, destination device 14B located remotely from source device 12 may not be able to reliably receive the enhancement layer due to, for example, network or channel conditions.

Also, because both the base layer data and the enhancement layer data are available, the closer destination device 14A can have higher quality video; and the destination device 14B can only present the minimum quality level provided by the base layer material. Thus, the video obtained by destination device 14 is adjustable in the sense that the extra bits of the enhancement layer can be decoded and added to the base layer bitstream to increase the signal-to-noise ratio (SNR) of the decoded video. However, adjustability is possible only when reinforcement layer data is present. Thus, the term "quality" as used in the present invention may refer to objective and/or subjective visual qualities. In other words, enhancement layer refinement results in video material that is reproduced for higher quality of the original material. In this way, the fidelity of the video is increased by the enhancement layer.

In other examples, network or channel conditions may be sufficient for both destination devices 14A and 14B to receive the base layer and the enhancement layer. However, destination devices 14A and 14B may have different decoder capabilities that prevent one of destination devices 14A and 14B from using additional video material of the enhancement layer to produce higher quality video. If one of the destination devices 14 is a client device, such as a mobile handset or, for example, other small portable devices, there may be limitations due to computational complexity and memory requirements. Thus, destination device 14, which may have limited computation or memory resources, may design an adjustable video write code in a manner that only decodes the base layer. In this manner, destination device 14 with better network or channel conditions and/or higher decoder capabilities will be able to reconstruct video with higher video quality using additional video data of the enhancement layer.

The techniques described in this disclosure utilize an entropy write technique that facilitates efficient code writing of enhancement layer bitstreams. The entropy coding technique of the present invention can be enabled to write, for example, additional video data in a refined form in a reinforced layer bit stream in a single coded one operation, thereby reducing write code complexity, write code delay, and memory. Body requirements. As will be described in further detail, source device 12 may, in some examples, encode each of the coefficient vectors of the enhancement layer without knowing any subsequent coefficients (ie, any coefficients that are currently being written by the non-zero coefficients of the code). A non-zero coefficient. Writing the code enhancement layer in a single operation eliminates the need to perform a first operation to analyze the coefficient vector and a second operation to write the code coefficient vector based on the analysis.

For example, some conventional entropy encoders may perform a first coded one operation to generate symbols to represent coefficient vectors, at least some of which represent more than one non-zero coefficient. In other words, it is necessary to know the subsequent coefficients to encode the non-zero coefficients of the coefficient vector. Additionally or alternatively, some conventional entropy encoders may also select a VLC table for encoding symbols during a first or subsequent encoding operation. In one aspect, the VLC table can be selected based on the generated symbols. Alternatively, the statistical data may be collected by analyzing the coefficient vector during the first encoding one operation, and the VLC table may be selected based on the collected statistical data.

A second one-time encoding operation is then performed by a conventional entropy coder to entropy encode the coefficient vector based on the analysis performed during the first encoding one operation. As an example, some conventional entropy encoders may encode symbols generated during a first operation using a VLC table selected based on the generated symbols or other statistical data during a second one-time operation. Generating a symbol representing more than one non-zero coefficient and/or selecting a VLC table based on the generated symbol or other statistical data may allow for more efficient encoding of the coefficient vector.

The technique of the present invention not only eliminates the need to encode the enhancement layer using more than one encoding operation, but the entropy coding technique of the present invention can additionally result in code enhancement in the case of not storing and accessing coefficient information of the video material of the base layer. Layers to further reduce computational complexity and memory requirements.

Source device 12, destination device 14, or both may be wireless or wired communication devices as described above. Also, source device 12, destination device 14, or both may be implemented as an integrated circuit device (such as an integrated circuit die or wafer set) that may be incorporated into a wireless or wired communication device or to support digital Another type of device for video applications, such as digital media players, personal digital assistants (PDAs), digital televisions, or the like.

2 is a block diagram illustrating source device 12 and destination device 14 of write code system 10 in further detail. Destination device 14 can be, for example, any of destination device 14A or 14B of FIG. As shown in FIG. 2, the source device 12 can include a video source 18, a video encoder 20, and a transmitter 22. The video source 18 of the source device 12 can include a video capture device such as a video camera, a video archive containing previously captured video, or a video feed from a video content provider. As a further alternative, video source 18 may generate computer graphics based data as source video or a combination of live video and computer generated video. In some cases, source device 12 can be a so-called camera phone or video phone, in which case video source 18 can be a camera. In each case, the captured, pre-fetched or computer generated video can be encoded by video encoder 20 for transmission from source device 12 to destination device 14 via transmitter 22 and communication channel 16.

The video encoder 20 receives video data from the video source 18 and encodes the video data into a base layer bit stream and one or more enhancement layer bit streams. In the example illustrated in FIG. 2, video encoder 20 includes a base layer encoder 30 and a boost layer encoder 32. The base layer encoder 30 and the enhancement layer encoder 32 receive the shared video material from the video source 18. The base layer encoder 30 encodes the video material at a first bit rate to produce a base layer bit stream of the video at a first quality level. The enhancement layer encoder 32 encodes additional bits to produce one or more enhancement layers that enhance the video to a second, higher quality level when added to the base layer level. In other words, the reinforcement layer provides a second, higher bit rate when added to the base layer, the second higher bit rate providing a higher quality level. Thus, the enhancement layer can be considered as the coding refinement of the video material encoded in the base layer. Refinement can be, for example, an additional factor and/or refinement of existing coefficients. In the sense that the refinement in the enhancement layer gradually increases the quality of the video material as it is decoded, the refinement of the coding in the enhancement layer can be hierarchical. Thus, the decoding of the refinement of all enhancement layers will, for example, result in the highest bit rate and maximum quality, while only the decoding of the refinement of the first enhancement layer will produce a bit rate relative to the decoding of only the base layer. The quality is increasing incrementally.

The video data received from the video source 18 can be a series of video frames. The base layer encoder 30 and the enhancement layer encoder 32 divide the series of frames into code writing units and process the write code units to encode the series of video frames. The code unit can be, for example, an entire frame or a portion of a frame (such as a slice of a frame). Base layer encoder 30 and enhancement layer encoder 32 divide each code unit into pixel blocks (referred to herein as video blocks or blocks) and operate on video blocks within individual code units to encode video. data. Similarly, the video material may include multiple frames, one frame may include multiple slices, and one slice may include multiple video blocks.

Video blocks can have fixed or varying sizes and can vary in size depending on the particular code standard. As an example, ITU-T H.264/MPEG-4 Part 10 AVC supports intra-frame prediction of various block sizes, such as 16 by 16, 8 by 8 or 4 by 4 for luminance components, and for chrominance 8x8 component, and inter-frame prediction to support various block sizes, such as 16 by 16, 16 by 8, 8 by 16, 8 by 8, 8 by 4, 4 by 8, and 4 by 4 for the luminance component The corresponding adjustable size of the chrominance component. In H.264/MPEG-4 Part 10 AVC, each video block, commonly referred to as a macroblock (MB), can be subdivided into sub-blocks of fixed or varying size. That is, the code writing unit may contain sub-blocks having the same or different sizes. In general, MB and various sub-blocks can be considered as video blocks. Thus, the MB can be considered a video block, and in the case of split or sub-segmentation, the MB itself can be considered to define a set of video blocks.

The encoders 30, 32 perform in-frame writing and inter-frame writing of the video blocks of the frame. In-frame writing relies on spatial prediction to reduce or remove spatial redundancy in the video material within a given code unit (eg, a frame or slice). For in-frame writing, the encoders 30, 32 form spatial prediction blocks based on one or more previously encoded blocks within the same block as the block currently being coded. The prediction block can be a prediction pattern of the video block currently being coded. Base layer encoder 30 may perform interpolation (eg, according to the intra-frame write mode associated with the block), for example, by using pixel values of one or more previously encoded blocks within the base layer of the current frame. A prediction block is generated based on one or more previously encoded blocks within the frame. The enhancement layer encoder 32 may generate prediction blocks based on one or more previously encoded blocks within the frame. The enhancement layer encoder 32 may generate prediction blocks based, for example, on one or more previously encoded video blocks from the base layer and the enhancement layer within the frame. For example, enhancement layer encoder 32 may generate a prediction block using a weighted sum of pixel values from at least one previously encoded video block from the base layer and at least one previously encoded video block from the enhancement layer. .

Inter-frame coding relies on temporal prediction to reduce or remove temporal redundancy within adjacent frames of the video sequence. For inter-frame writing, the encoders 30, 32 perform motion estimation to track the movement of closely matched video blocks between two or more adjacent frames within the code unit. In the inter-frame prediction condition, the encoders 30, 32 may generate temporal prediction blocks based on one or more previously encoded blocks from other frames within the coding unit. The encoders 30, 32 can, for example, compare the current video block to the block in one or more adjacent video frames to identify the block in the adjacent frame that most closely matches the current video block, such as A block having a minimum mean square error (MSE), a squared difference sum (SSD), an absolute difference sum (SAD), or other difference measure in one or more adjacent frames. The encoders 30, 32 select the identified blocks in adjacent frames as prediction blocks. The base layer encoder 30 compares the blocks of one or more adjacent frames of the current video block and the base layer. The enhancement layer encoder 32 may compare the blocks of one or more adjacent frames in the current video block to the base layer and/or the enhancement layer.

After the intra-frame or inter-frame prediction of the video block, the encoder 30, 32 generates the remaining block by subtracting the generated prediction block from the original video block being coded. The remaining block thus indicates the difference between the predicted block and the current block being coded. The encoders 30, 32 may apply transform, quantization, and entropy write procedures to further reduce the bit rate associated with communication of the remaining blocks. A transform technique that may include discrete cosine transform (DCT), integer transform, wavelet transform, direction transform, or other transform operation changes a set of pixel differences to residual transform coefficients, which represent pixel differences in the frequency domain energy. The encoders 30, 32 apply quantization to the residual transform coefficients, which typically involves the process of limiting the number of bits associated with any given coefficient. The encoders 30, 32 scan the two-dimensional residual block to produce a one-dimensional vector of coefficients and entropy encode the coefficient vector to further compress the residual coefficients. Entropy coding may, for example, include variable length write code (VLC), arithmetic write code, fixed length write code, content adaptive VLC (CAVLC), content adaptive binary arithmetic write code (CABAC), and/or other entropy writes. Code technology.

SNR adjustability can be achieved by residual quantization. In particular, base layer encoder 30 may quantize the remaining transform coefficients using a first quantization parameter (QP), and enhancement layer encoder 32 may quantize the remaining transform coefficients using the second QP. In ITU-T H.264/MPEG-10 AVC, larger QPs typically result in the use of a smaller number of bits to encode video data at a lower quality, while smaller QPs result in a higher number of bits with higher quality. Encode video data. Thus, the base layer encoder 30, which encodes the video material with the minimum quality level, can quantize the coefficients of the base layer using a QP value that is greater than the QP value used by the enhancement layer encoder 32 to quantize the coefficients of the enhancement layer. As a result, the quantized residual transform coefficients from base layer encoder 30 represent the video sequence at the first quality, and the quantized residual transform coefficients from the enhancement layer encoder represent additional coefficients or refinement of existing coefficients of the video sequence. The additional coefficients or refinements increase the quality of the video sequence to a second, higher quality when combined with the base layer.

The encoders 30, 32 each receive a one-dimensional coefficient vector representing the quantized residual transform coefficients of the base layer and the enhancement layer, respectively. In other words, the base layer encoder 30 receives the coefficient vector of the base layer, and the enhancement layer encoder 32 receives the coefficient vector of the corresponding enhancement layer. Although the encoders 30, 32 receive the same raw video material, the coefficient vectors can be different. This may be due to the base layer encoder 30 and the enhancement layer encoder 32 that generate different prediction blocks, for example, the base layer encoder 30 generates prediction blocks from one or more previously encoded base layer blocks, and enhances The layer encoder 32 generates prediction blocks from one or more previously encoded base layer blocks and enhancement layer blocks.

The base layer encoder 30 and the enhancement layer encoder 32 each encode respective coefficient vectors to generate a base layer bit stream and at least one enhancement layer bit stream, respectively. In accordance with the teachings of the present invention, base layer encoder 30 and enhancement layer encoder 32 encode individual coefficient vectors using different write code techniques. The base layer encoder 30 may encode a coefficient vector using a plurality of encoding one-time operation processes, in which the base layer encoder 30 analyzes the coefficient vector during at least one encoding operation, and based on the analysis, at least A coefficient vector is encoded during a subsequent encoding operation. In an example, base layer encoder 30 may encode the quantized residual transform coefficients of the base layer coefficient vector according to CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard. A CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard may encode a base layer coefficient vector using a plurality of coding one operations.

During a first coded one operation, base layer encoder 30 may generate symbols to represent coefficient vectors, at least some of which represent more than one non-zero coefficient, and in some cases, all coefficients of the coefficient vector. The base layer encoder 30 may generate symbols, for example, according to CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard, which represent the total number of coefficients in the coefficient vector ("TotalCoeffs"), in the coefficient vector The number of trailing 1s ("T1s"), the sign of any trailing 1s, the magnitude (or level) of the non-zero coefficients other than the trailing 1th, the sum of all the races ("sumRuns"), and each The fortune before the non-zero coefficient. To generate some of the symbols, such as TotalCoeff and sumRuns, the base layer encoder 30 can analyze the entire coefficient vector.

During the first encoding once operation, base layer encoder 30 may also select a VLC table based on analysis of the coefficient vectors for use during subsequent encoding operations. In some examples, base layer encoder 30 may select a VLC table based on symbols generated during one operation of the first write code for use during a subsequent (eg, second) encoding one operation. For example, base layer encoder 30 may select a VLC table based on the total number of coefficients in the block (TotalCoeffs) to use when encoding the sumRuns symbol because there is a relationship between these two values. In particular, as TotalCoeffs increases, sumRuns decreases, and as TotalCoeffs decreases, sumRuns increases. Again, the VLC table is selected based on the total number of coefficients in the block (TotalCoeffs) to use the allowable base layer encoder 30 to select a VLC table that more efficiently encodes the sumRuns when encoding the sumRuns symbol. Similar VLC table selections can be performed for other symbols to be encoded or using other collected statistics.

The base layer encoder 30 encodes a symbol (TotalCoeff) representing the total number of non-zero coefficients in the coefficient vector and a symbol indicating the number of trailing 1s (referred to as T1s) during the second or other subsequent encoding one operation. The number of trailing 1s is the number of coefficients having a magnitude of one, and when the coefficient vectors are read in reverse order (ie, starting from the end of the coefficient vector), the coefficients appear in coefficients having magnitudes greater than one. It appeared before in the coefficient vector. The base layer encoder 30 may select the VLC table for use in encoding the TotalCoeff and T1 symbols based on the predicted number of non-zero coefficients, and encode the TotalCoeff and T1 symbols using the selected VLC table. The VLC table is selected based on the predicted number of non-zero coefficients to use the allowable base layer encoder 30 in the coded TotalCoeff and T1 symbols to select a VLC table that more efficiently encodes the TotalCoeff and T1 symbols. Thus, different VLC tables can be more efficient for different predicted numbers of non-zero coefficients. In an example, base layer encoder 30 may predict non-zero in the current block based on the number of non-zero coefficients of one or more previously encoded video blocks (eg, upper adjacent video blocks and left adjacent video blocks) The number of zero coefficients.

The base layer encoder 30 can encode the sign of any trailing ones. For example, for each of the trailing ones, base layer encoder 30 may encode "1" (if the trailing 1 has a positive sign) and encode "0" (if the trailing 1 is positive or negative) The number is negative). Thus, base layer encoder 30 may not need to perform VLC table selection for the sign. Base layer encoder 30 may encode the magnitude of the non-zero coefficients other than the trailing one. Base layer encoder 30 may encode the level of non-zero coefficients using a VLC table, a fixed length write code, or other type of entropy write code. For example, base layer encoder 30 may encode the level of non-zero coefficients using a binary write code.

The base layer encoder 30 may encode symbols (sumRuns) representing the number of zero-valued coefficients occurring in the coefficient vector before the last non-zero coefficient. As described above, base layer encoder 30 may select a VLC table based on the total number of coefficients in the block (TotalCoeffs) to use when encoding the sumRuns symbol because there is a relationship between these two values.

Base layer encoder 30 may encode a run (or run length) that occurs before each non-zero coefficient from the last non-zero coefficient of the coefficient vector. The length of the run length is the number of zero-valued coefficients before the non-zero coefficient. Therefore, the base layer encoder 30 may first encode the run length before the last non-zero coefficient of the coefficient vector (ie, the number of zero-valued coefficients), followed by the length of the run before the previous non-zero coefficient, etc., until the coding coefficient vector The length of the first run before the first non-zero coefficient.

The base layer encoder 30 may select a VLC table to separately encode each of the run lengths. The base layer encoder 30 may select a VLC table to encode the current run value based on the sum of the sumRuns symbols and the hitherto coded runes. As an example, if the coefficient vector has a runsum sum (sumRuns) and the run length encoded before the last non-zero coefficient of the code is six, then all residual runs must be zero, one, or two. Since the length of the possible path is gradually shortened as each additional process is encoded, the base layer encoder 30 can select a more efficient VLC table to reduce the number of bits used to represent the fortune.

In this manner, base layer encoder 30 performs a multi-time operation encoding to encode base layer coefficients, including a coefficient vector of the remaining blocks of the base layer, for example, to generate symbols and/or select a VLC table. One operation and one second coding one operation based on analyzing the coding coefficient vector. Although the base layer encoder 30 is described above as using the CAVLC coded quantized residual transform coefficients as defined in the H.264/MPEG-4 Part 10 AVC standard, the base layer encoder 30 may use other write code methods. The quantized residual transform coefficients are encoded.

The enhancement layer encoder 32 encodes the quantized residual transform coefficients of the enhancement layer, which may be in the form of a coefficient vector. The enhancement layer encoder 32 may generate quantized residual coefficients that are different from the quantized residual coefficients of the base layer. The quantized residual coefficients of the enhancement layer are due to the use of different QPs during quantization and may be different from the quantized residual coefficients of the base layer. In addition, the quantized residual transform coefficients may be different from the quantized residual transform coefficients of the base layer, because the remaining blocks represent the original video block and the predicted block generated by using the previous coded block forming the base layer and the enhancement layer. The difference between the two. The remaining blocks of the base layer are the difference between the original video block and the predicted block generated using only the previous coded block from the base layer. Thus, the reinforcement layer may include additional coefficients and/or refinement of existing coefficients. In this sense, the quantized residual transform coefficients of the video blocks in the enhancement layer represent refinements of the first quality coded video blocks in the base layer and provide higher quality video data when added to the base layer. .

The enhancement layer encoder 32 may discard one or more of the quantized residual coefficients of the coefficient vector during encoding, depending on the available bit rate. For example, when a coefficient scan is performed using a zigzag scan as illustrated in FIG. 3, the enhancement layer encoder 32 may discard coefficients corresponding to the high frequency transform basis function, for example, coefficients that are located near the end of the coefficient vector. Coding the quantized residual coefficients according to CAVLC encoding as defined in the H.264/MPEG-4 Part 10 AVC standard may not allow the enhancement layer encoder 32 to discard the coefficients, since at least some of the symbols to be encoded (eg, TotalCoeffs) And sumRuns) relate to all the coefficients in the block. If the enhancement layer encoder 32 discards one or more of the coefficients of the coefficient vector, the received information will be redundant, thus resulting in lower code efficiency. Furthermore, because the decoder must receive the motion of all non-zero coefficients in the block to be able to properly decode each in a zigzag scan when encoding using CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard The position of a coefficient, so the enhancement layer encoder 32 may not be able to discard the coefficients of the coefficient vector from the enhancement layer.

Thus, enhancement layer encoder 32 encodes the coefficients of the enhancement layer in accordance with the write code technique of the present invention. The enhancement layer encoder 32 encodes the quantized residual transform coefficients of the coefficient vectors in a single coded one operation. In other words, enhancement layer encoder 32 does not perform the first operation to analyze the coefficient vector, and does not subsequently encode the symbol during the second operation based on the analysis. Rather, the enhancement layer encoder 32 begins at the beginning of the coefficient vector and encodes each of the non-zero coefficients one by one in a single coded one operation. In this manner, enhancement layer encoder 32 may encode each of the non-zero coefficients without analyzing any subsequent coefficients in the coefficient vector (ie, without knowing any subsequent coefficients of the coefficient vector).

In one aspect, enhancement layer encoder 32 may encode, for each of the non-zero coefficients, a symbol indicating the presence of at least one residual non-zero coefficient in the coefficient vector. The symbol can be, for example, an end of block (EOB) symbol. The enhancement layer encoder 32 can encode symbols using a single bit. For example, enhancement layer encoder 32 may encode zero when there is at least one residual non-zero coefficient (eg, at least the current non-zero coefficient), and may encode one when there are no more residual non-zero coefficients.

After the EOB symbol for each coefficient, enhancement layer encoder 32 encodes the run before the current non-zero coefficient. As described above, the fortune represents the number of zero-valued coefficients that occur between the beginning of the previous non-zero coefficient or coefficient vector of the coefficient vector (in the case of the first non-zero coefficient) and the current non-zero coefficient. The enhancement layer encoder 32 can encode the motion using a single VLC table. In an example, when TotalCoeffs is equal to one, enhancement layer encoder 32 may encode the run to write code sumRuns using a VLC table used in CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard. In other words, the enhancement layer encoder 32 can reuse one of the VLC tables that have been maintained by the video encoder 20. In other examples, enhancement layer encoder 32 may encode the motion using one of the other VLC tables that have been maintained by video encoder 20. Alternatively, enhancement layer encoder 32 may maintain a separate VLC table that is specifically designed to encode the coefficients of the enhancement layer. In either case, enhancement layer encoder 32 may not need to adaptively select a VLC table for encoding the run. Instead, the enhancement layer encoder 32 can use a single VLC table, thus eliminating the need for a first operation to collect statistics for selecting a VLC table.

The enhancement layer encoder 32 encodes the sign of the non-zero coefficient after the encoded run of each coefficient. The enhancement layer encoder 32 may, for example, encode "1" (if the sign of the non-zero coefficient is positive) and encode "0" (if the sign of the non-zero coefficient is negative). The enhancement layer encoder 32 can adjust the magnitude of the non-zero coefficients by setting the magnitude of the non-zero coefficients to one. In some examples, enhancement layer encoder 32 may not encode magnitudes of non-zero coefficients. In this manner, enhancement layer encoder 32 may limit the magnitude of the non-zero coefficients to one. Destination device 14 is then configured to decode all non-zero coefficients identified in the refinement to have a magnitude equal to one. The magnitude of the coefficients that do not encode the enhancement layer can result in some loss in peak signal to noise ratio (PSNR), but reduces the number of bits used to encode the coefficients.

In this manner, enhancement layer encoder 32 may encode the coefficients of the enhancement layer bitstream in a single operation, for example, without knowing any subsequent coefficients in the coefficient vector. Since the enhancement layer encoder 32 does not need to analyze the coefficient vector (for example) to generate a symbol representing one or more non-zero coefficients of the vector or select a VLC table to encode the symbol, only one encoding once operation is performed. Conventional encoders typically perform at least two simultaneous operations; (1) analyzing the first operation of the coefficient vector, and (2) performing a second operation based on analyzing the coding coefficient vector. Additionally, enhancement layer encoder 32 may encode the coefficients of the enhancement layer using a single VLC table, thus eliminating the need to perform an encoding one operation to form a symbol for adaptively selecting a code table. In this manner, enhancement layer encoder 32 can reduce write code complexity, write code latency, and memory requirements. In addition, the entropy coding technique of the present invention can additionally result in writing coefficients of the code enhancement layer without storing and accessing coefficient information of the base layer, thereby further reducing computational complexity and memory requirements.

Source device 12 transmits the encoded video material to destination device 14 via transmitter 22. Destination device 14 may include a receiver 24, a video decoder 26, and a display device 28. Receiver 24 receives the encoded video bitstream from source device 12 via channel 16. As described above, the encoded video bitstream includes a base layer bitstream and one or more enhancement layer bitstreams. Video decoder 26 decodes the base layer and, if available, one or more enhancement layers to obtain video material.

In detail, video decoder 26 includes a base layer decoder 34 and an enhancement layer decoder 36. Base layer decoder 34 decodes the base layer bitstream received via channel 16 to produce video material of the first quality for presentation on display device 28. The enhancement layer decoder 36 decodes the bitstream of one or more enhancement layers to obtain additional video material (e.g., refinement) that increases the quality of the decoded video material to a second, higher quality. Again, the number of enhancement layers (eg, one, two, three or more) received by destination device 14 may be subject to visual channel conditions or other restrictions. Moreover, the number of enhancement layers received by the enhancement layer decoder 36 can be determined by the decoder limitations. In general, the encoding and decoding of the base layer in combination with the selected number of enhancement layers permits incremental improvements in the SNR quality of the decoded video.

The base layer decoder 34 decodes the base layer to obtain a symbol representing the vector of the quantized residual coefficients of the base layer. The base layer decoder 34 can decode the base layer to obtain the total number of non-zero coefficients in the block, the number of trailing 1s of the block, the trailing 1 sign, the magnitude of the coefficient other than the trailing 1 , all the fortune The sum of the sum and the non-zero coefficients before the fortune. Base layer decoder 34 may further decode the base layer bitstream to identify a VLC table for decoding base layer symbols. In other examples, base layer decoder 34 may select a VLC table for use based on previously decoded symbols. Using the decoded symbols, base layer decoder 34 may reconstruct the coefficient vectors of the base layer.

The enhancement layer decoder 36 decodes the bitstream of the enhancement layer to obtain, for example, a vector of additional residual coefficients or a refinement of the enhancement layer of the refined form of the existing residual coefficients. In particular, enhancement layer decoder 36 decodes the duration and sign of the enhancement layer coefficients using the same VLC table as the VLC table used by enhancement layer encoder 32 until the EOB symbol indicates that no non-zero coefficients are remaining. Using the decoded symbols, enhancement layer decoder 36 reconstructs the coefficient vector of the enhancement layer block.

The decoder 34, 36 reconstructs each of the blocks of the coded unit using the decoded quantized residual coefficients. After generating the coefficient vector, the decoders 34, 36 inversely scan the coefficient vector to produce a two-dimensional block of quantized residual coefficients. The decoders 34, 36 inverse quantize (ie, dequantize) the quantized residual coefficients and apply an inverse transform (eg, inverse DCT, inverse integer transform, inverse wavelet transform, or inverse direction transform) to the dequantized residual coefficients to produce pixels. The remaining block of values.

The decoders 34, 36 sum the prediction blocks generated by the decoders 34, 36 and the remaining blocks of pixel values to form reconstructed base layer video blocks and enhancement layer video blocks, respectively. The base layer video block and the enhancement layer video block are combined to form a video block having a higher resolution. The decoders 34, 36 generate prediction blocks in the same manner as described above with respect to the encoders 30, 32. The destination device 14 can display the reconstructed video block to the user via the display device 28. Display device 28 can include any of a variety of display devices, such as a cathode ray tube (CRT), liquid crystal display (LCD), plasma display, light emitting diode (LED) display, organic LED display, or another type of display. unit.

In some examples, video encoder 20 and video decoder 26 are configured to provide an adjustable boost bit stream that can be arbitrarily truncated. Thus, system 10 can avoid the use of discrete enhancement layers that must be coded as a whole to achieve adjustability. However, in some embodiments, system 10 can be configured to support adjustability using, for example, a generalized fine grain size adjustability (FGS) method or a discrete enhancement layer on a selective basis.

Source device 12 and destination device 14 can operate in a substantially symmetrical manner. For example, source device 12 and destination device 14 can each include a video encoding and decoding component. Thus, system 10 can support, for example, one-way or two-way video transmission between devices 12, 14 for video streaming, video broadcasting, or video telephony.

In some aspects, for video broadcasting, the techniques described in this disclosure can be applied to enhanced H.264 video writing code for use in the Forward Link Only (FLO) Empty Intermediary Specification" Direct Forward Video Intermediary Specification for Terrestrial Mobile Multimedia Multicast ("Technical Standard TIA-1099 ("FLO Specification") published in July 2007) Ground Mobile Multimedia Multicast (TM3) System Delivers Instant Video Services . That is, communication channel 16 may include a wireless information channel for broadcasting wireless video information in accordance with the FLO specification or the like. The FLO specification includes an example of bounding meta-stream syntax and semantics and a decoding process suitable for FLO null interfacing.

Alternatively, the video may be broadcast according to other standards such as DVB-H (Palm Digital Video Broadcasting), ISDB-T (Ground Integrated Services Digital Broadcasting), or DMB (Digital Media Broadcasting). Therefore, the source device 12 can be a mobile wireless terminal, a video streaming server, or a video broadcast server. However, the techniques described in this disclosure are not limited to any particular type of broadcast, multicast, or peer-to-peer system. In the broadcast condition, source device 12 may broadcast several channels of video material to a plurality of destination devices, each of which may be similar to destination device 14 of FIG. Thus, although a single destination device 14 is shown in FIG. 1, for video broadcasts, source device 12 will typically broadcast video content to many destination devices simultaneously.

In other examples, transmitter 22, communication channel 16 and receiver 24 may be configured to communicate in accordance with any wired or wireless communication system, including one or more of the following: Ethernet, telephone (eg, POTS), cable, power line and fiber optic system and/or wireless system, the wireless system comprising one or more of the following: a code division multi-directional proximity (CDMA or CDMA2000) communication system, frequency division multidirectional Proximity (FDMA) system, orthogonal frequency division multi-directional (OFDM) proximity system, time-division multi-directional proximity (TDMA) system (such as GSM (Global System for Mobile Communications), GPRS (General Packet Radio Service) or EDGE (Enhanced) Data GSM environment)), TETRA (terrestrial relay radio) mobile phone system, broadband coded multi-directional proximity (WCDMA) system, high data rate 1xEV-DO (first generation evolutionary data) or 1xEV-DO gold multicast system, IEEE 402.18 system, the MediaFLO ^(TM) system, the system DMB, DVB-H system, or another scheme for data communication between the two or more devices.

Video encoder 20 and video decoder 26 may each be implemented as one or more microprocessors, digital signal processors (DSPs), special application integrated circuits (ASICs), field programmable gate arrays (FPGAs), discrete logic, Software, hardware, firmware or any combination thereof. Each of video encoder 20 and video decoder 26 may be included in one or more encoders or decoders, any of which may be in a respective mobile device, user equipment, broadcast device, server, or the like. Integrated into a part of a combined encoder/decoder (CODEC). In addition, source device 12 and destination device 14 each may include appropriate modulation, demodulation, frequency conversion, filtering, and amplification components for transmitting and receiving encoded video, as applicable, including radio frequencies sufficient to support wireless communication. (RF) wireless components and antennas. However, for ease of illustration, such components are summarized as transmitter 22 of source device 12 and receiver 24 of destination device 14 in FIG.

3 is a block diagram illustrating an example base layer encoder 30 and enhancement layer encoder 32 in further detail. In the example of FIG. 3, the base layer encoder includes a prediction unit 33A, a frame storage 35A, a transform unit 38A, a quantization unit 40A, a coefficient scanning unit 41A, an inverse quantization unit 42A, an inverse transform unit 44A, and a base layer entropy coding. 46 and summers 48A and 48B ("summer 48"). The various features in FIG. 3 are depicted as elements intended to highlight different functional aspects of the illustrated device and do not necessarily imply that such elements must be implemented by separate hardware or software components. Rather, the functionality associated with one or more units can be integrated into a shared or separate hardware or software component.

The prediction unit 33A generates a prediction block using intra-frame prediction or inter-frame prediction. The prediction block can be a prediction pattern of the current video block being coded. As described above, prediction unit 33A may generate prediction blocks using in-frame prediction based on one or more previously encoded blocks of the base layer within the same frame as the block currently being coded. Alternatively, the prediction unit may generate the prediction block using inter-frame prediction based on one or more of the previous coding blocks in one or more adjacent frames of the base layer. Prediction unit 33A may extract the previously encoded block from frame storage 35A.

After the intra-frame or inter-frame prediction of the video block, the base layer encoder 30 generates the remaining block by subtracting the prediction block generated by the prediction unit 33A from the current video block at the summer 48A. . The remaining block includes a set of pixel difference values that quantify the difference between the pixel values of the current video block and the pixel values of the predicted block. The remaining blocks may be represented in a two-dimensional block format (eg, a two-dimensional matrix or array of pixel values). In other words, the remaining blocks are two-dimensional representations of pixel values.

Transform unit 38A applies a transform to the remaining blocks to produce residual transform coefficients. Transform unit 38A may, for example, apply DCT, integer transform, direction transform, wavelet transform, or a combination thereof. After applying the transform to the remaining blocks of pixel values, quantization unit 40A quantizes the transform coefficients to further reduce the bit rate. After quantization, inverse quantization unit 42A and inverse transform unit 44A may apply inverse quantization and inverse transform, respectively, to reconstruct the remaining blocks. The summer 48B adds the reconstructed residual block to the predicted block generated by the prediction unit 33A to produce a reconstructed video block for storage in the frame storage 35A. The reconstructed video blocks stored in the frame store 35A can be used by the prediction unit 32 of the base layer encoder 30 to write subsequent video blocks in-frame or inter-frame. In addition, as will be described in more detail below, the reconstructed video block stored in the frame store 35A can be used by the prediction unit 33B of the enhancement layer encoder 32 to block the video blocks in the in-frame or inter-frame code enhancement layer. Refinement.

After quantization, the coefficient scanning unit 41A scans the coefficients from the two-dimensional block format into a one-dimensional vector format, which is generally referred to as coefficient scanning. The coefficient scanning unit 41A can scan the two-dimensional block of coefficients, for example, using a zigzag scan order as described in further detail in FIG. After scanning, the base layer entropy encoder 46 entropy encodes the coefficients of the one-dimensional vector. Base layer encoder 46 may entropy encode coefficients of coefficient vectors, for example, using CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard and described above in detail with respect to FIG.

The enhancement layer encoder 32 includes a prediction unit 33B, a frame storage 35B, a transform unit 38B, a quantization unit 40B, a coefficient scanning unit 41B, an inverse quantization unit 42B, an inverse transform unit 44B, a boost layer entropy encoder 49, and summation. 48C and 48D ("Summer 48"). The elements of enhancement layer encoder 32 are substantially similar to similarly numbered units of base layer encoder 30. Thus, only the differences will be described.

The prediction unit 33B of the enhancement layer encoder 32 generates a prediction block that is a prediction pattern of the current video block. Unlike the prediction unit 33A of the base layer encoder 30 that uses only the previous coding block of the base layer to generate the prediction block, the prediction unit 33B of the enhancement layer encoder 32 may be based on one or more previous coding regions of the base layer and the enhancement layer. The block produces a prediction block. In other words, prediction unit 33B can generate predicted blocks using reconstructed video blocks from the base layer and the reconstructed video blocks of the enhancement layer. For example, prediction unit 33B may combine the reconstructed blocks of the reconstructed video block and the enhancement layer of the base layer to produce a prediction block that is at a second higher quality.

Since the prediction block generated by the prediction unit 33B is generated based on the reconstructed video blocks of both the base layer and the enhancement layer, the remaining blocks generated at the summer 48C represent the current video block and the self-base layer. And the difference between the previously coded blocks of the enhancement layer construction (ie, at the second higher visual quality).

Although operatively similar to the quantization unit 40A of the base layer encoder 30, the quantization unit 40B of the enhancement layer encoder 32 may use different QPs to quantize the transform coefficients. As described above with respect to Figure 2, SNR adjustability can be achieved by using different quantization parameters. For example, when base layer encoder 30 and enhancement layer encoder 32 operate in accordance with ITU-T H.264/MPEG-10 AVC, quantization unit 40A may encode using a QP value that is greater than the QP value used by quantization unit 40B. Video material. As a result, the quantized residual transform coefficients from the base layer encoder 30 represent the video sequence at the first quality, and the quantized residual transform coefficients from the enhancement layer encoder 32 represent additional coefficients or fine coefficients of the existing coefficients of the video sequence. The additional coefficients or refinements increase the quality of the video sequence to a second, higher visual quality when combined with the base layer. Moreover, as described in detail with respect to FIG. 2, enhancement layer entropy encoder 49 encodes the quantized residual transform coefficients in a single coded one operation. In other words, the enhancement layer entropy encoder 49 may encode each non-zero coefficient of the coefficient vector of the enhancement layer without knowing any subsequent coefficients of the coefficient vector. Writing the code enhancement layer in a single operation eliminates the need for a first operation of performing an analysis coefficient vector and a second operation for writing a code coefficient vector based on the analysis. The fact is that the enhancement layer entropy encoder 49 starts at the beginning of the coefficient vector and encodes each of the coefficients one by one in a single encoding one operation. More details regarding the entropy coding of the enhancement layer are described below with respect to FIG.

4 is a block diagram illustrating an example base layer entropy encoder 46 and enhancement layer entropy encoder 49 in further detail. The base layer entropy encoder 46 may include an analysis unit 50, a plurality of VLC tables 52A to 52N ("VLC table 52"), a total coefficient encoder 54, a trailing 1 (T1s) encoder 56, and a positive and negative code. The controller 58, a coefficient magnitude encoder 60, a and forward encoder 62, and a run length encoder 64. The enhancement layer entropy encoder 49 may include an EOB symbol encoder 66, a run length encoder 68, a positive and negative encoder 70, and a VLC table 69.

The base layer entropy encoder 46 encodes the coefficient vector representing the video block of the first quality by performing a plurality of encoding one operations. According to a CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard, for example, the base layer entropy encoder 46 may perform an analysis coefficient vector to, for example, generate a symbol representing a coefficient vector and/or select a VLC table. The first encoding one operation and the second encoding one operation based on analyzing the encoding coefficient vector.

As an example, analysis unit 50 of base layer entropy encoder 46 may analyze the coefficient vector to produce one or more symbols representing the coefficient block. Analysis unit 50 may, for example, determine the number of total coefficients (TotalCoeff), the number of trailing 1s (T1), the number of each trailing 1 in accordance with the H.264/MPEG-4 Part 10 AVC standard. The magnitude of each non-zero coefficient, the sum of the operations (sumRuns), and the length of the run before each non-zero coefficient. At least some of the symbols produced by analysis unit 50 (eg, TotalCoeff and sumRuns) may represent all coefficients of the coefficient vector. Analysis unit 50 may generate more symbols or fewer symbols in other examples.

Additionally or alternatively, analysis unit 50 may select a subset of VLC tables 52 for encoding symbols during a first or subsequent encoding operation. In one aspect, analysis unit 50 can select a subset of VLC tables 52 based on the generated symbols. Alternatively, analysis unit 50 may collect statistical data during analysis of the coefficient vector and select a subset of VLC table 52 based on the collected statistical data. For example, base layer encoder 30 may select a VLC table based on the total number of coefficients in the block (TotalCoeffs) to use when encoding the sumRuns symbol because there is a relationship between these two values. As will be described in greater detail below, selecting a subset of VLC tables 52 based on the generated symbols or other statistical data may enable more efficient encoding of the symbols representing the coefficient vectors.

Base layer entropy coder 46 encodes the coefficient vector during a second or other subsequent write code operation. In detail, the total coefficient encoder 54 encodes the total number of non-zero coefficients (TotalCoeff) in the coefficient vector. The total coefficient encoder 54 may encode TotalCoeff using one of the VLC tables 52 selected based on the prediction of the number of non-zero coefficients of the current coefficient vector. In an example, the number of non-zero coefficients for the current coefficient vector may be based on the number of non-zero coefficients of one or more previously encoded video blocks (eg, upper adjacent video blocks and left adjacent video blocks) Forecast. In this way, the base layer entropy decoder can select the same VLC table based on previously decoded blocks.

After the total coefficient encoder 54 encodes the total number of non-zero coefficients, the T1s encoder 56 encodes the T1s symbol. The T1s encoder 56 may encode the T1s symbol, for example, using one of the VLC tables 52 based on the predicted number of non-zero coefficients, in the same manner as described above with respect to the total coefficient encoder 54.

The sign encoder 58 encodes the sign of any trailing ones. For example, for each of the trailing ones, the sign encoder 58 may encode "1" (if the trailing 1 sign is positive) and encode "0" (if the trailing 1 is positive or negative) The number is negative). The system value encoder 60 encodes the level (e.g., magnitude) of the non-zero coefficient other than the trailing one. The system value encoder 60 may encode the level of non-zero coefficients using a VLC table, a fixed length write code, or other type of entropy write code.

The transport and encoder 62 may encode a symbol representing the number of zero-valued coefficients occurring in the coefficient vector before the last non-zero coefficient, i.e., the sumRuns symbol. The run and encoder 62 encodes the sumRuns symbol using one of the VLC tables 52 selected based on the total number of coefficients in the block (TotalCoeffs). Again, the VLC table is selected based on the total number of coefficients in the block (TotalCoeffs) to use the allowable run and encoder 62 to select a VLC table that more efficiently encodes the sumRuns when encoding the sumRuns symbol.

The run length encoder 64 encodes the run length of the coefficient vector. The run length encoder 64 may first encode the run length of the last non-zero coefficient of the coefficient vector, followed by the run length of the previous non-zero coefficient, etc., up to the run length before the first non-zero coefficient of the coding coefficient vector. In other words, the run length encoder can first start encoding the last run length. The run length encoder 64 may encode each of the run lengths using a sum of the sum of the coefficient vectors (sumRuns) and the VLC table 52 selected by the sum of the coded runs to date. As an example, if the coefficient vector has a runsum sum (sumRuns) and the run length encoded before the last non-zero coefficient of the code is six, then all residual runs must be zero, one, or two. Since the length of the possible path is gradually shortened as each additional process is encoded, the run length encoder 64 may select a more efficient VLC table to reduce the number of bits used to represent the run. In this manner, the VLC table 52 used by the run length encoder 64 is variable for each of the operational lengths.

The enhancement layer entropy coder 49 encodes a coefficient vector representing the refinement of the video block, for example, in the form of an additional coefficient or a refined form of the existing coefficients, in a single coded one operation to form a enhancement layer. As will be described in further detail, source device 12 may, in some examples, encode each non-zero coefficient in the coefficient vector of the enhancement layer without knowing any subsequent coefficients. The enhancement layer entropy coder 49 may begin at the beginning of the coefficient vector and encode each of the coefficients one by one in a single coded one operation. In this way, the enhancement layer encoder 49 encodes the coefficient vector on a coefficient-by-coefficient basis without analyzing the coefficients occurring later in the coefficient vector. Writing the code enhancement layer in a single operation eliminates the need for a first operation of performing an analysis coefficient vector and a second operation for writing a code coefficient vector based on the analysis.

For each of the non-zero coefficients, EOB symbol encoder 66 encodes an EOB symbol indicating the presence of at least one residual non-zero coefficient in the coefficient vector. For example, EOB symbol encoder 66 may encode zero when there is at least one residual non-zero coefficient (eg, at least the current non-zero coefficient), and may encode one when there are no more residual non-zero coefficients.

After encoding the EOB symbol for each coefficient, the run length encoder 68 encodes the run length before the non-zero coefficient. As described above, the run length represents the number of zero value coefficients prior to the current non-zero coefficient. The run length encoder 68 can encode the run length using a single VLC table 69. In an example, the VLC table 69 can be the same as one of the VLC tables 52 of the base layer entropy encoder 46. Alternatively, the run length encoder 68 may maintain a separate VLC table that is specifically designed to encode the coefficients of the enhancement layer. In either case, the run length encoder 68 may not need to adaptively select a VLC table for encoding the run. Rather, the run length encoder 68 can use a single VLC table, thus eliminating the need for a first operation to collect statistics for selecting a VLC table.

After the encoded run length of each coefficient, the sign encoder 70 encodes the sign of the non-zero coefficient. The sign encoder 70 can, for example, encode "1" (if the sign of the non-zero coefficient is positive) and encode "0" (if the sign of the non-zero coefficient is negative). The enhancement layer entropy coder 49 may not encode the magnitude of the non-zero coefficients of the enhancement layer, which may result in some loss in peak signal to noise ratio (PSNR), but reduce the number of bits used to encode the coefficients.

The entropy coding technique of the present invention may allow the enhancement layer entropy encoder 49 to encode the coefficients of the enhancement layer bitstream in a single operation. Since the enhancement layer entropy encoder 49 does not analyze the coefficient vector (for example) to generate symbols and/or select a VLC table, only one encoding operation is required. Conventional encoders typically perform at least two simultaneous operations; (1) analyzing the first operation of the coefficient vector, and (2) performing a second operation based on analyzing the coding coefficient vector. In addition, enhancement layer entropy encoder 49 may encode the coefficients of the enhancement layer using a single VLC table, thus eliminating the need to perform a coded one operation to make selections from various VLC tables. In this manner, the enhancement layer entropy encoder 49 can reduce write code complexity, write code latency, and memory requirements. In addition, the entropy coding technique of the present invention can additionally result in writing coefficients of the code enhancement layer without storing and accessing coefficient information of the base layer, thereby further reducing computational complexity and memory requirements.

FIG. 5 is a block diagram illustrating an example of the base layer decoder 34 and the enhancement layer decoder 36 in further detail. The base layer decoder 34 includes a base layer entropy decoder 72, a coefficient scanning unit 74A, an inverse quantization unit 76A, an inverse transform unit 78A, a prediction unit 80A, a frame storage 82A, and a summer 84A. The enhancement layer decoder 36 includes a enhancement layer entropy decoder 86, a coefficient scanning unit 74B, an inverse quantization unit 76B, an inverse transform unit 78B, a prediction unit 80B, a frame storage 82B, and a summer 84B.

The base layer entropy decoder 72 decodes the received base layer bitstream to generate video material of the first quality for presentation on the display device. The base layer entropy decoder 72 receives the base layer bitstream and decodes the base layer bitstream to obtain (eg, in the form of one of the quantized residual coefficients) the remaining information and (eg, in one or more header syntax) Header information in the form of features. The base layer entropy decoder 72 performs the reciprocal decoding function of the encoding performed by the base layer entropy encoder 46 of FIGS. 3 and 4.

In particular, the base layer entropy decoder 72 decodes the base layer to obtain a symbol representing the vector of the quantized residual coefficients of the base layer. When writing using CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard, for example, the base layer entropy decoder 72 may decode the base layer to obtain the total number of non-zero coefficients in the block (TotalCoeff). The number of trailing l (Tls) of the block, the sign of the trailing l, the magnitude of the coefficient other than the trailing l, the sum of all the operations (sumRuns), and the foreman before the non-zero coefficient . In some examples, the VLC table selected for decoding may be selected based on previously decoded blocks or previously decoded symbols of the current block. In other examples, base layer entropy decoder 34 may decode the base layer bitstream to identify a VLC table for decoding base layer symbols. Using the decoded symbols, base layer decoder 34 may reconstruct the coefficient vectors of the base layer.

After generating the coefficient vector, coefficient scanning unit 74A inversely scans the coefficient vector to produce a two-dimensional block of quantized residual coefficients. Inverse quantization unit 76A inverse quantizes (i.e., dequantizes) the quantized residual coefficients, and inverse transform unit 78A applies an inverse transform to the dequantized residual coefficients (e.g., inverse DCT, inverse integer transform, inverse wavelet transform, or inverse direction transform). ) to generate the remaining blocks of pixel values.

Prediction unit 80A uses one or more neighboring blocks within a common frame or one or more blocks within an adjacent frame to generate a predicted block in the case of intra-frame prediction or in the case of inter-frame prediction. . The prediction unit generates the prediction block using only the previous coding block from the base layer. Summer 84A sums the prediction blocks generated by prediction unit 80A and the remaining blocks of pixel values to form a reconstructed base layer video block. The base layer video block is stored in the frame store 82A for use in generating subsequent predicted blocks.

The enhancement layer decoder 36 decodes the bitstream of the enhancement layer to obtain, for example, a vector of additional residual coefficients or refinement of the video material in refined form of the existing residual coefficients. The enhancement layer entropy decoder 86 decodes the duration and sign of the enhancement layer coefficients using the same VLC table as the VLC table used by the enhancement layer entropy encoder 49 until the EOB symbol indicates that the non-zero coefficients are no longer residual. Using the decoded symbols, the enhancement layer entropy decoder 86 reconstructs the coefficient vectors of the enhancement layer blocks. The decoded coefficient vector represents additional bits that represent the refinement of the decoded video material quality to a second, higher quality when combined with the bits of the base layer.

After generating the coefficient vector, coefficient scanning unit 74B inversely scans the coefficient vector to produce a two-dimensional block of quantized residual coefficients. Inverse quantization unit 76B inverse quantizes (ie, dequantizes) the quantized residual coefficients, and inverse transform unit 78B applies an inverse transform to the dequantized residual coefficients (eg, inverse DCT, inverse integer transform, inverse wavelet transform, or inverse direction transform). ) to generate the remaining blocks of pixel values.

Prediction unit 80B uses one or more neighboring blocks within a common frame or one or more blocks within an adjacent frame to generate a predicted block in the case of intra-frame prediction or in the case of inter-frame prediction. . The prediction unit generates prediction blocks using previously encoded blocks from both the base layer and the enhancement layer. Summer 84B sums the prediction block generated by prediction unit 80B with the remaining blocks of pixel values to form a reconstructed enhancement layer video block. The enhancement layer video block is stored in the frame store 82B for use by the prediction unit 80B for generating subsequent prediction blocks. The reconstructed base layer video block and the reconstructed enhancement layer video block are combined at summer 84C to form a higher quality video block.

6 is a block diagram illustrating an example base layer entropy decoder 72 and enhancement layer entropy decoder 86 in further detail. Base layer entropy decoder 72 may include a plurality of VLC tables 52A through 52N ("VLC Table 52"), a total coefficient decoder 90, a trailing 1 (T1s) decoder 92, a sign decoder 94, a coefficient. A magnitude decoder 96, a and a range decoder 98, and a run length decoder 100. The enhancement layer entropy decoder 86 may include an EOB symbol decoder 102, a run length decoder 104, a sign decoder 106, and a VLC table 69.

The base layer entropy decoder 72 decodes the base layer bitstream to obtain symbols representing the coefficient vectors of the video blocks at the base quality level. The total coefficient decoder 90 decodes the bit stream using one of the VLC tables 52 to obtain the total number of non-zero coefficients (TotalCoeff) in the coefficient vector. The total coefficient decoder 90 may select the VLC table 52 for decoding TotalCoeff based on the prediction of the number of non-zero coefficients of the current coefficient vector (eg, based on the number of non-zero coefficients of the one or more previously decoded video blocks). . In this manner, total coefficient decoder 90 may select the same VLC table 52 as the VLC table used by total coefficient encoder 54 to encode the TotalCoeff symbol.

After the total coefficient decoder 90 decodes the total number of non-zero coefficients, the T1s decoder 92 decodes the T1s symbols. The T1s symbol represents the number of coefficients having a magnitude of one, and when the coefficient vector is read in reverse order, the coefficients are encountered before encountering coefficients having magnitudes greater than one. The T1s decoder 92 may decode the T1s symbol using one of the VLC tables 52 selected based on the predicted number of non-zero coefficients.

The sign decoder 94 decodes any trailing sign of 1. For example, the sign decoder 94 can determine that the sign of the coefficient is positive when receiving "1" for each of the trailing ones, and can determine the sign of the coefficient when "0" is received. Negative. The system value decoder 96 decodes the magnitude of the non-zero coefficient other than the trailing one. The system value decoder 116 may use a VLC table, a fixed length write code, or other type of entropy write code to decode the level of non-zero coefficients.

The transport and decoder 98 may decode the symbols representing the number of zero-valued coefficients occurring in the coefficient vector before the last non-zero coefficient, i.e., the sumRuns symbol. The transport and decoder 98 decodes the sumRuns symbol using one of the VLC tables 52 selected based on the total number of coefficients in the block (TotalCoeffs), and the total number of coefficients (TotalCoeff) in the block is previously performed by the total coefficient decoder 90. decoding. Again, the VLC table is selected based on the total number of coefficients in the block (TotalCoeffs) to use the allowable run and decoder 98 to select a VLC table that more efficiently decodes the sumRuns when decoding the sumRuns symbols.

The run length decoder 100 decodes the run length of the coefficient vector. The run length decoder 100 may first decode the run length of the last non-zero coefficient of the coefficient vector, followed by the run length of the previous non-zero coefficient, etc., until the run length before the first non-zero coefficient of the coefficient vector is decoded. In other words, the run length decoder 100 may first begin decoding the last run length. The run length decoder 100 may decode each of the run lengths using a sum of the sum of the coefficient vectors (sumRuns) and the VLC table 52 selected by the sum of the coded runs to date. The transport and decoder 98 previously decoded the sumRuns symbol. However, the run length decoder 100 may collect statistics on the sum of the decoded passes so far. Since the length of the possible run length becomes shorter as each additional run is decoded, the run length decoder 100 can select a more efficient VLC table to reduce the number of bits used to represent the run. In this manner, the VLC table 52 used by the run length decoder 100 is variable for each of the operational lengths.

The enhancement layer entropy decoder 86 decodes the bitstream of the enhancement layer to obtain, for example, a vector of additional coefficients or refinement of the video block for the refined form of the existing coefficients. The EOB symbol decoder 102 determines if the EOB symbol indicates whether there is at least one residual non-zero coefficient. The run length decoder 104 decodes the run length before the next non-zero coefficient when there is at least one residual non-zero coefficient. The run length decoder 104 can decode the run length of the next non-zero coefficient using the same VLC table 69 as the VLC table used by the run length encoder 68. The sign decoder 106 decodes the sign of the non-zero coefficient. For example, the sign decoder 106 can determine that the sign has a positive sign (when "1" is received) and a negative (when "0" is received). The enhancement layer entropy decoder 86 continues to decode the non-zero coefficients until the EOB symbol decoder 102 indicates that there are no residual non-zero coefficients.

FIG. 7 is a conceptual diagram illustrating a zigzag scan of a 4x4 coefficient block 40. The zigzag scan shown in Figure 7 can be implemented by the encoders 30, 32 of Figure 2. The scan order of this zigzag scan shown in Figure 7 follows the arrows passing through the video block 110 and is labeled with coefficients c1 through c16 in the scan order. In particular, the values shown in Figure 7 indicate the position of the coefficients within the sequential one-dimensional vector and do not represent the actual values of the coefficients. The result of the zigzag scan illustrated in Fig. 7 is a one-dimensional coefficient vector X , where X = [c1, c2, c3, c4, c5, c6, c7, c8, c9, c10, c11, c12, c13, c14, C15, c16] where c1 to c16 represent the position of the coefficients within the two-dimensional array of coefficients.

The techniques of this disclosure are not limited to any particular scanning order or technique. For example, the scan order used in the present invention may be the zigzag scan order shown in FIG. Alternatively, the scan order used in the present invention may be other scan orders, such as horizontal scan, vertical scan, or any other scanning technique.

FIG. 8 is a conceptual diagram of a hypothetical example of coefficient block 120 illustrating the coefficients of the enhancement layer. In this example, the values shown in Figure 8 indicate the actual values of the coefficients at the locations. The actual coefficient values of coefficient block 120 may represent quantized residual coefficients, unquantized transform coefficients, or other types of coefficients of the video block in the enhancement layer. In the example illustrated in Figure 8, coefficient block 120 is a 4x4 block. However, the techniques of this disclosure may be extended to apply to blocks of any size. After scanning the coefficient block 120 according to the zigzag scan illustrated in FIG. 3, the resulting coefficient vector V is:

V = [4,0,0,-2,0,1,0,0,0,0,0,0,0,0,0,0].

The enhancement layer encoder 32 encodes each of the coefficients of the coefficient vector V in accordance with the techniques described in this disclosure. As an example, for each of the non-zero coefficients of the coefficient vector V , the enhancement layer encoder 32 encodes the EOB symbol, the run length, and the sign. As described in detail above, the EOB symbol indicates whether there are any residual non-zero coefficients in the coefficient vector, the run length represents the number of zero-valued coefficients occurring before the current non-zero coefficient of the coefficient vector, and the sign indicates whether the coefficient value is positive or negative. .

According to one aspect of the invention, enhancement layer encoder 32 may not encode the magnitude of the coefficients. Rather, the enhancement layer encoder 32 may encode each of the non-zero coefficients as if all non-zero coefficients were equal to one. In this way, enhancement layer encoder 32 can be viewed as encoding the following coefficient vector V' instead of V.

V' =[1,0,0,-1,0,1,0,0,0,0,0,0,0,0,0,0]

32 may (e.g.) use of the EOB is equal to zero, the fortune zero codeword group and a number equal to the sign of the coefficient to encode a first enhancement layer encoder (i.e., coefficient vector V at 4 or coefficient vector V 'in 1), using the EOB is equal to zero, a code word of the group is equal to zero and the sign of two of fortune second encoded coefficients (i.e., coefficient vector V at -2 or coefficient vector V 'in -1), and the An EOB equal to zero, a codeword group of one fortune, and a sign equal to one followed by an EOB symbol equal to one encode a third non-zero coefficient (ie, a coefficient vector V or a coefficient vector V' ). As described above, the codeword group used to encode the run can be obtained from the VLC table defined in the H.264/MPEG-4 Part 10 AVC standard.

An example encoded bit stream is described for purposes of illustration. The enhancement layer encoder 32 can encode the coefficient vectors V , V' in different ways without departing from the scope of the invention. For example, the EOB symbol can be encoded as one to represent additional non-zero coefficients in the block, and coded to zero to indicate no residual non-zero coefficients. Similarly, the sign can be coded to zero to represent a positive non-zero coefficient, and encoded as a non-zero coefficient expressed as negative. As another example, the EOB symbol encoded for each non-zero coefficient may indicate whether the current coefficient is the last non-zero coefficient of the vector. Thus, there may be no EOB symbols at the end of the encoded bit stream. The fact is that when the EOB symbol indicates that the current coefficient is the last non-zero coefficient, the video decoder knows that there are no additional coefficients for the block after decoding the current coefficients and symbols.

9 is a flow diagram illustrating an example operation of a video encoder (such as video encoder 20 of FIG. 2) that implements the adjustable video coding technique of the present invention. The base layer encoder 30 and the enhancement layer encoder 32 of the video encoder 20 obtain video material (130) from the video source 18. As described above, the base layer encoder 30 and the enhancement layer encoder 32 obtain the same original video material. The video material obtained from the video source 18 can be, for example, a series of video frames.

For each video block, base layer encoder 30 encodes the base layer (132) using a write code technique that performs multiple coded one operations. The base layer encodes the video block with a first quality level. The base layer encoder 30 may generate a coefficient vector representing the video block of the first quality and encode the remaining transform coefficients of the block to generate the base layer. The base layer encoder 30 may generate a base layer according to a CAVLC coding coefficient vector as defined in the H.264/MPEG-4 Part 10 AVC standard. As described in detail above with respect to FIG. 2, base layer encoder 30 may perform a first one-time operation to analyze the coefficient vector and a second one to use to encode the coefficient vector based on the analysis.

For each video block, enhancement layer encoder 32 encodes the extra bits into a reinforcement layer (134) using a write code technique that performs a single code one operation. The extra bit coding refinement of the layer bit stream is enhanced, and the refinement enhances the video to a second higher quality level when added to the base layer bit stream. Although enhancement layer encoder 32 is described as encoding only a single enhancement layer in this example, enhancement layer encoder 32 may encode more than one enhancement layer bitstream. In this case, the reinforcement layer can be hierarchical in the sense that the reinforcement layer provides progressively higher quality as it is decoded.

The second entropy write technique used by enhancement layer encoder 32 may encode the EOB symbol, the run and the sign for each of the non-zero coefficients of the coefficient vector of the enhancement layer. As described in detail above, the EOB symbol can indicate whether there are any residual non-zero coefficients, the run length represents the number of zero-valued coefficients that occurred before the non-zero coefficient, and the sign indicates whether the coefficient value is positive or negative. After the sign of the last non-zero coefficient, enhancement layer encoder 32 may encode the EOB symbol to indicate that there are no residual non-zero coefficients.

The base layer encoder 30 and the enhancement layer encoder 32 respectively output the encoded base layer bit stream and the enhancement layer bit stream (136). The entropy coding technique used by enhancement layer encoder 32 may allow the residual coefficients of the enhancement layer to be encoded with lower computation and implementation complexity without significant loss of write efficiency. The entropy coding technique of the present invention can enhance the write code of the additional video data in the layered bit stream in a single coded operation, for example, in a single coded one operation, thereby reducing write code complexity, write code delay and Memory requirements. For example, enhancement layer encoder 32 may encode each non-zero coefficient of the coefficient vector of the enhancement layer without knowing any subsequent coefficients, thereby allowing the code coefficient vector to be written in a single operation and eliminating the execution of one to analyze the coefficients. The first operation of the vector and the need for a second operation based on analyzing the coded coefficient vector.

10 is a flow diagram illustrating an example operation of a enhancement layer encoder (such as enhancement layer encoder 32 of FIG. 2) of residual coefficients of a coding enhancement layer of an enhancement layer in accordance with an aspect of the present invention. The enhancement layer encoder 32 identifies the first non-zero coefficient (140) in the coefficient vector of the enhancement layer block. The enhancement layer encoder 32 encodes an EOB symbol (142) indicating that there is at least one residual non-zero coefficient in the coefficient vector of the enhancement layer block. The enhancement layer encoder 32 may encode the EOB symbol using a single bit, for example, encoding zero when there is at least one residual non-zero coefficient and encoding one when there are no residual non-zero coefficients.

The enhancement layer encoder 32 encodes a run (144) indicating the number of zero value coefficients preceding the non-zero coefficients. The enhancement layer encoder 32 may, in some examples, encode the run for a VLC table that has been stored as CAVLC as defined in the H.264/MPEG-4 Part 10 AVC Standard. For example, the enhancement layer encoder 32 may use a VLC table to encode the run when the total number of coefficients (TotalCoeffs) is equal to the sum of the total runs of the code (sumRuns). Alternatively, enhancement layer encoder 32 may maintain a separate VLC table that is specifically designed to encode the coefficients of the enhancement layer.

The enhancement layer encoder 32 may encode the sign of the non-zero coefficient (146). The enhancement layer encoder 32 may, for example, encode "1" (if the sign of the non-zero coefficient is positive) and encode "0" (if the sign of the non-zero coefficient is negative). In some examples, enhancement layer encoder 32 may not encode magnitudes of non-zero coefficients. In this manner, enhancement layer encoder 32 may limit the magnitude of the non-zero coefficients to one. Thus, any non-zero coefficient having a magnitude greater than one is set equal to one. The magnitude of the non-zero coefficients that do not encode the enhancement layer can result in some loss in peak-to-noise ratio (PSNR), but reduces the number of bits used to encode non-zero coefficients.

The enhancement layer encoder 32 determines if there are any residual non-zero coefficients in the enhancement layer block (148). When there is at least one residual non-zero coefficient in the enhancement layer block, enhancement layer encoder 32 continues to encode the EOB, the fort and the sign of each of the residual non-zero coefficients. When there are no residual non-zero coefficients in the enhancement layer block, enhancement layer encoder 32 encodes the EOB symbol to indicate that there are no residual non-zero coefficients in the coefficient vector of the enhancement layer block (149). As described above, the reinforcement layer is transported along with the base layer.

Since the enhancement layer write code technique described in FIG. 10 does not write code involving symbols of more than one coefficient, the enhancement layer write code technique may allow the enhancement layer encoder 32 to discard the coefficient vector during encoding, depending on the available bit rate. One or more of the remaining coefficients. In addition, enhanced layer write coding techniques reduce write code complexity and implementation.

11 is a flow diagram illustrating an example operation of an enhancement layer decoder (such as enhancement layer decoder 36 of FIG. 2) that decodes the enhancement layer bitstream to obtain a vector of residual transform coefficients. The enhancement layer decoder 36 obtains a enhancement layer bitstream (150). The enhancement layer decoder 36 analyzes the EOB symbols to determine if there are any residual non-zero coefficients (152). The enhancement layer decoder 36 may, for example, determine that there is at least one residual non-zero coefficient when the EOB symbol is equal to zero, and determine that there are no residual non-zero coefficients when the EOB symbol is equal to one.

When enhancement layer decoder 36 determines that there is at least one residual non-zero coefficient (e.g., the EOB symbol is equal to zero), enhancement layer decoder 36 decodes the motion associated with the next non-zero coefficient (154). The run associated with the next non-zero coefficient represents the number of zero-valued coefficients before the non-zero coefficient. The enhancement layer decoder 36 decodes the motion using the same VLC table as the VLC table used by the enhancement layer encoder 32. In an example, enhancement layer decoder 36 may be used for writing the total motion and sumRuns as defined in the CAVLC of the H.264/MPEG-4 Part 10 AVC Standard (when the total number of coefficients (TotalCoeffs) ) A VLC table equal to one hour) to decode the run. However, other VLC tables can be used as long as they are the same table used by the enhancement layer encoder 32. The enhancement layer decoder 36 sets the number of coefficients of the run length equal to the non-zero coefficient equal to zero (156). If the run length is equal to two, then for example, enhancement layer decoder 36 may set the two coefficients preceding the non-zero coefficients to be equal to zero.

The enhancement layer decoder 36 decodes the sign of the non-zero coefficient (158). The sign of the non-zero coefficient can be decoded as positive (when the sign is equal to one) and decoded as negative (when the sign is equal to zero). After decoding the sign of the non-zero coefficient, enhancement layer decoder 36 may set the non-zero coefficient equal to positive one or negative one (160) based on the decoded sign. As mentioned above, the enhancement layer may not encode the magnitude of the coefficients of the enhancement layer. Likewise, enhancement layer decoder 36 can be configured to set the magnitude of all non-zero coefficients equal to one.

The enhancement layer decoder 36 continues to decode the run and sign of the non-zero coefficients until the enhancement layer decoder 36 determines that there are no residual non-zero coefficients (e.g., the EOB symbol is equal to one). At this point, if any coefficients are left, the enhancement layer decoder 36 sets the residual coefficients of the vector equal to zero (162). As described in detail with respect to FIG. 2, enhancement layer decoder 36 uses coefficient vectors and other data in addition to the prediction blocks to reconstruct the video blocks for presentation to display 28.

12 through 15 are block diagrams illustrating different configurations of an encoder and/or decoder for use in an adjustable video write code. These example encoders and decoders are illustrative of encoder types that may utilize the techniques of the present invention. However, the example configuration should in no way limit the techniques as described. Technology can be used in any adjustable video encoder.

Each of the example video encoders and decoders illustrated in Figures 12 through 15 may utilize the entropy write code techniques described in this disclosure to facilitate efficient code writing of the enhancement layer bitstream. The entropy coding technique of the present invention can enable the writing of additional video data in the enhanced layer bit stream, for example, in a refined form, in a single encoding operation, thereby reducing write code complexity and write code delay. And memory requirements. As will be described in further detail, each non-zero coefficient in the coefficient vector of the enhancement layer can be encoded without any subsequent coefficients (ie, any coefficients that are currently being written by the non-zero coefficients of the code). Writing the code enhancement layer in a single operation eliminates the need for a first operation of performing an analysis coefficient vector and a second operation for writing a code coefficient vector based on the analysis.

FIG. 12 is a block diagram illustrating an example adjustable video encoder 170. The adjustable video encoder 170 can, for example, correspond to the video encoder 20 of FIG. In the example of FIG. 12, the adjustable video encoder 170 includes a base layer encoder 30, which includes a prediction unit 172, a frame storage 173, a transform unit 174, quantization units 175A and 175B, and an inverse. Quantization units 176A and 176B, inverse transform unit 177, multiplex module 178, and summers 179A through 179C. The various features in FIG. 3 are depicted as elements intended to highlight different functional aspects of the illustrated device and do not necessarily imply that such elements must be implemented by separate hardware or software components. Rather, the functionality associated with one or more units can be integrated into a common or separate hardware or software component.

Prediction unit 172 generates prediction blocks using in-frame prediction or inter-frame prediction. The prediction block can be a prediction pattern of the current video block being coded. As described above, prediction unit 172 can generate prediction blocks using in-frame prediction based on one or more previously encoded blocks of the base layer within the same frame as the block currently being coded. Alternatively, the prediction unit may generate the prediction block using inter-frame prediction based on one or more of the previous coding blocks in one or more adjacent frames of the base layer. Prediction unit 172 can extract the previously encoded block from frame storage 173.

After the intra-frame or inter-frame prediction of the video block, the base layer encoder 30 generates the remaining block by subtracting the prediction block generated by the prediction unit 172 from the current video block at the summer 179A. . The remaining block includes a set of pixel difference values that quantify the difference between the pixel values of the current video block and the pixel values of the predicted block. The remaining blocks may be represented in a two-dimensional block format (eg, a two-dimensional matrix or array of pixel values). In other words, the remaining blocks are two-dimensional representations of pixel values.

Transform unit 174 applies a transform to the remaining blocks to produce residual transform coefficients. Transform unit 174 can, for example, apply DCT, integer transform, direction transform, wavelet transform, or a combination thereof. After applying the transform to the remaining blocks of pixel values, quantization unit 175A quantizes the transform coefficients to further reduce the bit rate. The output of quantization unit 175A corresponding to the quantized coefficients associated with the base layer is provided to multiplex module 178.

After quantization, inverse quantization unit 176A applies inverse quantization to produce a reconstructed version of the remaining blocks of transform coefficients. The original residual block of the transform coefficients output by the summer transform unit 174 is subtracted from the reconstructed pattern of the remaining blocks of the transform coefficients output from the inverse quantization unit 176A. This block, referred to herein as a transform difference block, is provided to quantization unit 175B. Quantization unit 175B quantizes the transform coefficients to further reduce the bit rate. The output of quantization unit 175B corresponding to the quantized coefficients associated with the enhancement layer is provided to multiplex module 178. In an example, quantization unit 175A may quantize the residual coefficients using the first QP, and quantization unit 175B may quantize the residual coefficient differences using the second QP. The second QP may, for example, be half the value of the first QP, that is, QP/2.

After quantization by quantization unit 175B, inverse quantization unit 176B applies inverse quantization to produce a reconstructed version of the transformed difference block. The summer 179C sums the reconstructed version of the remaining block of the transform coefficients output from the inverse quantization unit 176A with the reconstructed version of the transformed difference block output by the inverse quantization unit 176B to produce a reconstructed residual block.

Inverse transform unit 177 applies an inverse transform to the reconstructed version of the video block. The reconstructed version of the video block is stored in the frame store 173 and can be used by the prediction unit 172 to write the subsequent video blocks in-frame or inter-frame. Prediction unit 172 can provide control data to multiplex module 178, such as motion vectors, split sizes, in-frame write patterns, or the like. The multiplex module 178 can combine base layer data and reinforcement layer data. In some examples, multiplex module 178 can include an entropy encoder for entropy encoding base layer data and enhancement layer data. In other examples, the base layer encoder and the enhancement layer encoder can be independent of the multiplex module.

FIG. 13 is a block diagram illustrating an example adjustable video decoder 180. Adjustable video decoder 180 may, for example, correspond to video decoder 26 of FIG. The adjustable video decoder 180 of FIG. 13 includes a demultiplexing module 181, inverse quantization units 182A and 182B, an inverse transform unit 183, a prediction unit 184, a frame memory 185, and summers 186A and 186B.

The demultiplexing module 181 receives the adjustable encoded video and demultiplexes the multiplexed signal. In some examples, the demultiplexing module 181 can include an entropy decoder for entropy decoding base layer data and enhancement layer data. In other examples, the base layer decoder and the enhancement layer decoder can be independent of the demultiplexing module.

Inverse quantization unit 182A inverse quantizes (i.e., dequantizes) the quantized residual coefficients associated with the base layer, and inverse quantization unit 182B dequantizes the quantized residual coefficients associated with the enhancement layer. In an example, inverse quantization unit 182A may quantize the residual coefficients using the first QP, and inverse quantization unit 182B may quantize the residual coefficient differences using the second QP. The second QP may, for example, be half the value of the first QP, that is, QP/2. The respective sets of dequantized transform coefficients output by inverse quantization units 182A and 182B are added at summer 186A to produce reconstructed residual transform blocks. As described above, the dequantized transform coefficients output by the inverse quantization unit 182A may correspond to the basic quality level, and the dequantized transform coefficients output by the inverse quantization unit 182B cause an increase when added to the output of the inverse quantization unit 182B. Quality level.

The inverse transform unit 183 applies an inverse transform (e.g., inverse DCT, inverse integer transform, inverse wavelet transform, or inverse direction transform) to the sum of the dequantized residual coefficient blocks to generate a remaining block of pixel values. Summer 186B adds the prediction block generated by prediction unit 184 to the remaining blocks of pixel values to form a reconstructed base layer video block. As described in detail above, the prediction unit 184 can use one or more adjacent blocks within the common frame or use one or more regions within the adjacent frame in the case of inter-frame prediction in the case of intra-frame prediction. The block produces a prediction block that can be stored in the frame store 185.

FIG. 14 is a block diagram illustrating another example adjustable video decoder 190. Adjustable video decoder 190 can, for example, correspond to video decoder 26 of FIG. The adjustable video decoder 190 of FIG. 14 includes a demultiplexing module 191, inverse quantization units 192A and 192B, inverse transform units 193A and 193B, a prediction unit 194, a frame storage 195, and summers 196A and 196B.

The demultiplexing module 191 receives the adjustable encoded video and demultiplexes the multiplexed signal. In some examples, the demultiplexing module 181 can include an entropy decoder for entropy decoding base layer data and enhancement layer data. In other examples, the base layer decoder and the enhancement layer decoder can be independent of the demultiplexing module.

Inverse quantization unit 192A and inverse transform unit 193A apply inverse quantization (i.e., dequantization) and inverse transform operations to the decoded residual coefficients associated with the base layer to obtain a reconstructed version of the remaining blocks of the base layer. Inverse quantization unit 192B and inverse transform unit 193B apply inverse quantization (i.e., dequantization) and inverse transform operations to the decoded residual coefficients associated with the enhancement layer to obtain a reconstructed version of the remaining blocks of the enhancement layer. In an example, inverse quantization unit 192A may quantize the residual coefficients using the first QP, and inverse quantization unit 192B may quantize the residual coefficient differences using the second QP. The second QP may, for example, be half the value of the first QP, that is, QP/2.

Prediction unit 194 may generate predicted blocks using one or more neighboring blocks within a common frame or one or more blocks within adjacent frames in the case of intra-frame prediction or in the case of inter-frame prediction. The prediction block can be stored in the frame storage 195. Summer 196A adds the prediction block generated by prediction unit 194 to the reconstructed residual block output from inverse transform unit 193A to produce decoded video material at a base quality level. The self-adjustable video decoder 190 outputs decoded video material having a basic quality level.

The decoded video material having the basic quality level is also provided to summer 196B. Summer 196B sums the output of summer 196A with the reconstructed version of the remaining blocks of the enhancement layer output from inverse transform unit 193B to produce decoded video material at a second, higher quality level. The self-adjustable video decoder 190 outputs decoded video material having a basic quality level.

FIG. 15 is a block diagram illustrating another example video encoder 200. In the example of FIG. 15, the base layer encoder 30 includes a prediction unit 33A, a frame storage 35A, a transform unit 38A, a quantization unit 40A, a coefficient scanning unit 41A, an inverse quantization unit 42A, an inverse transform unit 44A, and a base layer entropy. Encoder 46, summers 48A through 48C, and in-frame prediction unit 40A. The various features in FIG. 3 are depicted as elements intended to highlight different functional aspects of the illustrated device and do not necessarily imply that such elements must be implemented by separate hardware or software components. Rather, the functionality associated with one or more units can be integrated into a common or separate hardware or software component.

Prediction unit 33A generates prediction blocks using inter-frame prediction (e.g., motion compensated prediction). The prediction block can be a prediction pattern of the current video block being coded. As described above, prediction unit 33A may generate prediction blocks using inter-frame prediction based on one or more previously encoded blocks within one or more adjacent frames of the base layer. Prediction unit 33A may extract the previously encoded block from frame storage 35A.

After the inter-frame based prediction of the video block, the base layer encoder 30 generates the remaining block by subtracting the prediction block generated by the prediction unit 33A from the current video block at the summer 48A. The remaining block includes a set of pixel difference values that quantify the difference between the pixel values of the current video block and the pixel values of the predicted block. The remaining blocks may be represented in a two-dimensional block format (eg, a two-dimensional matrix or array of pixel values). In other words, the remaining blocks are two-dimensional representations of pixel values.

Transform unit 38A applies a transform to the remaining blocks to produce residual transform coefficients. Transform unit 38A may, for example, apply DCT, integer transform, direction transform, wavelet transform, or a combination thereof. After applying the transform to the remaining blocks of pixel values, quantization unit 40A quantizes the transform coefficients to further reduce the bit rate. After quantization, inverse quantization unit 42A and inverse transform unit 44A may apply inverse quantization and inverse transform, respectively, to reconstruct the remaining blocks. The summer 48B adds the reconstructed residual block to the predicted block generated by the prediction unit 33A to produce a reconstructed video block for storage in the frame storage 35A. The reconstructed video block stored in the frame store 35A can be used by the prediction unit 32 of the base layer encoder 30 to write coded subsequent video blocks in-frame or inter-frame. In addition, as will be described in more detail below, the reconstructed video block stored in the frame store 35A can be used by the prediction unit 33B of the enhancement layer encoder 32 to block the video blocks in the in-frame or inter-frame code enhancement layer. Refinement.

After quantization, summer 48C subtracts the intra-frame predicted blocks generated by in-frame prediction unit 40A from the quantized residual coefficients. In-frame prediction unit 40A may generate a prediction block using intra-frame prediction based on one or more previously encoded blocks within the same block as the block currently being coded. Base layer entropy encoder 46 entropy encodes the coefficients output from summer 48C, for example, using CAVLC as defined in the H.264/MPEG-4 Part 10 AVC standard and described above in detail with respect to FIG.

The enhancement layer encoder 32 includes a prediction unit 33B, a frame storage 35B, a transform unit 38B, a quantization unit 40B, a coefficient scanning unit 41B, an inverse quantization unit 42B, an inverse transform unit 44B, a boost layer entropy encoder 49, and a summer. 48D to 48F. The elements of enhancement layer encoder 32 are substantially similar to similarly numbered units of base layer encoder 30. Thus, only the differences will be described.

The prediction unit 33B of the enhancement layer encoder 32 generates a prediction block that is a prediction pattern of the current video block. Unlike the prediction unit 33A of the base layer encoder 30 that uses the previous coding block of the base layer to generate the prediction block, the prediction unit 33B of the enhancement layer encoder 32 may generate based on one or more previously coded blocks of the enhancement layer. Forecast block. The reconstructed video block of the enhancement layer may be at a second quality level that is higher than the quality level of the prediction block of the base layer.

An additional difference between the enhancement layer encoder 32 and the base layer encoder 30 is that the output of the inverse quantization unit 42B of the enhancement layer encoder 32 and the output of the inverse quantization unit 42A of the enhancement layer encoder 30 are at the summer 48F. combination. Adding the outputs of inverse quantization units 42A and 42B produces a higher quality reconstructed video block, thus allowing for better prediction by the prediction unit described above.

The techniques described in this disclosure can be implemented in hardware, software, firmware, or any combination thereof. Any feature described as a unit or component can be implemented together in an integrated logic device or separately as a discrete but interoperable logical device. If implemented in software, the techniques may be at least partially implemented by a computer-readable medium comprising instructions which, when executed, perform one or more of the methods described above. The computer readable medium can form part of a computer program product, which can include packaging materials. The computer readable medium can include random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable Programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage media, and the like. Additionally or alternatively, the techniques may be at least partially implemented by a computer readable communication medium carrying or transmitting code in the form of an instruction or data structure and accessible, readable, and/or executable by a computer.

The code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, special application integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrations. Or discrete logic circuits. Accordingly, the term "processor" as used herein may refer to any of the foregoing structures or any other structure suitable for implementation of the techniques described herein. Moreover, in some aspects, the functionality described herein may be provided within a dedicated software unit or hardware unit configured for encoding and decoding, or incorporated in a combined video encoder-decoder ( CODEC). The depiction of different features as a unit is intended to highlight different functional aspects of the illustrated device and does not necessarily imply that such elements must be implemented by separate hardware or software components. Rather, the functionality associated with one or more units can be integrated into a common or separate hardware or software component.

Various embodiments have been described. These and other embodiments are within the scope of the following patent claims.

10. . . Video transmission system

12. . . Source device

14. . . Destination device

14A. . . Multiple destination devices

14B. . . Multiple destination devices

16. . . Communication channel

18. . . Video source

20. . . Video encoder

twenty two. . . Transmitter

twenty four. . . receiver

26. . . Video decoder

28. . . display screen

30. . . Base layer encoder

32. . . Reinforcement layer encoder

33A. . . Forecasting unit

33B. . . Forecasting unit

34. . . Base layer decoder

35A. . . Frame storage

35B. . . Frame storage

36. . . Enhanced layer decoder

38A. . . Transform unit

38B. . . Transform unit

40A. . . Quantization unit

40B. . . Quantization unit

41A. . . Coefficient scanning unit

41B. . . Coefficient scanning unit

42A. . . Inverse quantization unit

42B. . . Inverse quantization unit

44A. . . Inverse transform unit

44B. . . Inverse transform unit

46. . . Base layer entropy encoder

48. . . Summer

48A. . . Summer

48B. . . Summer

48C. . . Summer

48D. . . Summer

48E. . . Summer

48F. . . Summer

49. . . Enhanced layer entropy encoder

50. . . Analysis unit

52. . . VLC table

52A~52N. . . VLC table

54. . . Total coefficient encoder

56. . . Trailing 1 (T1) encoder

58. . . Positive and negative encoder

60. . . Number value encoder

62. . . And forward encoder

64. . . Run length encoder

66. . . EOB symbol encoder

68. . . Run length encoder

69. . . VLC table

70. . . Positive and negative encoder

72. . . Base layer entropy decoder

74A. . . Coefficient scanning unit

74B. . . Coefficient scanning unit

76A. . . Inverse quantization unit

76B. . . Inverse quantization unit

78A. . . Inverse transform unit

78B. . . Inverse transform unit

80A. . . Forecasting unit

80B. . . Forecasting unit

82A. . . Frame storage

82B. . . Frame storage

84A. . . Summer

84B. . . Summer

86. . . Enhanced layer entropy decoder

90. . . Total coefficient decoder

92. . . Trailing 1 (T1) decoder

94. . . Positive and negative decoder

96. . . Number value decoder

98. . . And luck decoder

100. . . Run length decoder

102. . . EOB symbol decoder

104. . . Run length decoder

106. . . Positive and negative decoder

110. . . Video block

120. . . Coefficient block

170. . . Example adjustable video encoder

172. . . Forecasting unit

173. . . Frame storage

174. . . Transform unit

175A. . . Quantization unit

175B. . . Quantization unit

176A. . . Inverse quantization unit

176B. . . Inverse quantization unit

177. . . Inverse transform unit

178. . . Multiplex module

179A. . . Summer

179B. . . Summer

179C. . . Summer

180. . . Example adjustable video decoder

181. . . Multiplexed module

182A. . . Inverse quantization unit

182B. . . Inverse quantization unit

183. . . Inverse transform unit

184. . . Forecasting unit

185. . . Frame storage

186A. . . Summer

186B. . . Summer

190. . . Example adjustable video decoder

191. . . Multiplexed module

192A. . . Inverse quantization unit

192B. . . Inverse quantization unit

193A. . . Inverse transform unit

193B. . . Inverse transform unit

194. . . Forecasting unit

195. . . Frame storage

196A. . . Summer

196B. . . Summer

200. . . Example video encoder

FIG. 1 is a block diagram showing a video transmission system supporting video adjustability.

2 is a block diagram showing the source device and destination device of the code writing system of FIG. 1 in further detail.

3 is a block diagram illustrating an example base layer encoder and enhancement layer encoder in further detail.

4 is a block diagram illustrating an example base layer entropy encoder and a enhancement layer entropy encoder in further detail.

Figure 5 is a block diagram illustrating an example of a base layer decoder and an enhancement layer decoder in further detail.

6 is a block diagram illustrating an example base layer entropy decoder and a enhancement layer entropy decoder in further detail.

Figure 7 is a conceptual diagram illustrating a zigzag scan of a 4x4 coefficient block.

Figure 8 is a conceptual diagram illustrating a hypothetical example of coefficient blocks of a enhancement layer video block.

9 is a flow chart illustrating an example operation of a video encoder that performs the adjustable video coding technique of the present invention.

10 is a flow chart illustrating an example operation of a enhancement layer encoder that encodes residual coefficients of a enhancement layer video block in accordance with an aspect of the present invention.

11 is a flow diagram illustrating an example operation of an enhancement layer decoder that decodes a enhancement layer bitstream to obtain a vector of residual transform coefficients.

12 through 15 are block diagrams showing different configurations of an encoder and/or decoder for use in an adjustable video write code in accordance with the present invention.

(no component symbol description)

Claims (66)

A method for encoding video data using an adjustable video writing code, the method comprising: encoding, by a video writing device, a video block as a part of a base layer bit stream by a first quality; encoding by the video writing device Refining the video block as part of at least one enhancement layer bitstream, the refinement resulting in a second quality greater than the first quality when combined with the video block encoded with the first quality The video block, wherein the refinement of the video block is encoded without performing a first write operation that collects statistical data used in video code table selection, wherein the refinement is an additional coefficient And at least one of the refinements of the existing coefficients, and wherein encoding the refinements includes encoding, for each non-zero coefficient of the refinement of the video block, a symbol indicating the presence of at least one residual non-zero coefficient, a run length indicating a number of zero value coefficients preceding the non-zero coefficient and a sign of the non-zero coefficient, and the amount of the non-zero coefficients refined by the video writing device to the video block Adjustment is equal to one. The method of claim 1, wherein the refining the encoding of the video block comprises encoding each non-zero coefficient of the refinement without analyzing any subsequent coefficients. The method of claim 1, further comprising encoding a symbol to indicate that in the refinement of the video block after encoding a last non-zero coefficient There are residual non-zero coefficients. The method of claim 1, wherein the magnitude of each of the non-zero coefficients of the refinement of the video block is adjusted to be equal to a magnitude included in the non-zero coefficient The coefficients are encoded in the case. The method of claim 1, wherein the refinement encoding the video block as part of the enhancement layer bitstream includes the refinement encoding the video block, such that the video block is refined. The coefficients are decodable without accessing coefficient information of the video block that is encoded by the first quality as part of the base layer bit stream. The method of claim 1, further comprising encoding the refinement of the video block using only a single variable length code (VLC) table. The method of claim 1, wherein encoding the video block with the first quality as part of the base layer comprises encoding the video block with the first quality using a write code technique, the write code technology being first A coefficient vector of one of the video blocks is analyzed in one operation of the code, and the coefficient vector is encoded in the second write once operation based on the analysis. The method of claim 7, wherein: encoding the video block with the first quality comprises using a content adaptive variable length according to the ITU-T H.264/MPEG-4 Part 10 Advanced Video Recording (AVC) standard A code-writing (CAVLC) process encodes the video block with the first quality; and the refining of the video block includes encoding the video block using one of the VLC tables defined in the CAVLC process Such refinement. The method of claim 1, wherein the first quality and the second quality comprise one of a first signal to noise ratio (SNR) and a second signal to noise ratio (SNR) and a first spatial resolution and one One of the second spatial resolutions. An apparatus for encoding video data using an adjustable video write code, the apparatus comprising at least one encoder for encoding a video block as a portion of a base layer bit stream by a first quality; encoding the video block refinement As part of the at least one enhancement layer bitstream, the refinement results in a video block having a second quality greater than the first quality when combined with the video block encoded with the first quality, wherein Such refinement of the video block is encoded without performing a first write once operation of collecting statistical data used in video code table selection, and wherein such refinement is refined for additional coefficients and existing coefficients At least one of; for each non-zero coefficient of the refinement of the video block, encoding a number indicating a presence of at least one residual non-zero coefficient, and a number indicating a non-zero coefficient The length of the run length and a sign of the non-zero coefficient, and the magnitude of the non-zero coefficients of the refined portions of the video block are adjusted to be equal to one. The apparatus of claim 10, wherein the at least one encoder encodes each of the non-zero coefficients of the refinements without analyzing any subsequent coefficients. The device of claim 10, wherein the at least one encoder encodes a symbol to indicate that the video block is in the video block after encoding a last non-zero coefficient There are no residual non-zero coefficients in the chemistry. The device of claim 10, wherein the at least one encoder encodes the coefficients without encoding the magnitude of the non-zero coefficients. The device of claim 10, wherein the at least one encoder encodes the refinement of the video block such that the refined coefficients of the video block are encoded as the base layer without accessing the first quality. The coefficient information of the video block of the portion of the bit stream is decodable. The device of claim 10, wherein the at least one encoder encodes the refinements of the video block using only a single variable length code (VLC) table. The device of claim 10, wherein the at least one encoder encodes the video block with the first quality using a write code technique, and the write code technique analyzes one of the video blocks in a first write code operation A coefficient vector, and the coefficient vector is encoded in a second write once operation based on the analysis. The device of claim 16, wherein the at least one encoder: encoding the video block with the first quality comprises using a content according to the ITU-T H.264/MPEG-4 Part 10 Advanced Video Recording (AVC) standard An adaptive variable length write code (CAVLC) process encodes the video block with the first quality; and the refinement encoding the video block includes using one of the VLC tables defined in the CAVLC process Encoding such refinements of the video block. The device of claim 10, wherein the first quality and the second quality comprise one of a first signal to noise ratio (SNR) and a second signal to noise ratio (SNR) and a first spatial resolution and one One of the second spatial resolutions. The device of claim 10, wherein the at least one encoder comprises: a base layer encoder, the base layer encoder encoding the video block as a part of a base layer bit stream by the first quality, and a reinforcement layer An encoder, the enhancement layer encoder encoding the refinement of the video block as part of the at least one enhancement layer bitstream, the refinements causing greater than being combined with the video block encoded with the first quality The second quality of the first quality of the video block. The device of claim 10, wherein the device comprises a wireless communication device. The device of claim 10, wherein the device comprises an integrated circuit device. A computer readable medium containing instructions for causing one or more processors to: encode a video block as a portion of a base layer bit stream with a first quality; and encode the video block Refining as part of at least one enhancement layer bitstream, the refinement resulting in a video block having a second quality greater than the first quality when combined with the video block encoded with the first quality, The refinement of the video block is encoded without performing a first write operation of collecting the statistical data used in the video code table selection, wherein the refinement is an extra coefficient and an existing coefficient. At least one of the instructions, and wherein the instructions that cause the one or more processors to encode the refinements include a plurality of instructions that cause one or more processors to refine the video blocks Coded presence indication for each non-zero coefficient a symbol of at least one residual non-zero coefficient, a run length indicating a number of zero value coefficients before a non-zero coefficient, and a sign of the non-zero coefficient; and the non-refining of the video block The magnitude of the zero coefficient is adjusted to be equal to one. The computer readable medium of claim 22, wherein the instructions cause the one or more processors to encode each of the non-zero coefficients of the refinement without analyzing any subsequent coefficients. The computer readable medium of claim 22, wherein the instructions cause the one or more processors to encode a symbol to indicate that there is no residual non-zero in the refinement of the video block after encoding a last non-zero coefficient coefficient. The computer readable medium of claim 22, wherein the instructions cause the one or more processors to encode the coefficients without encoding the magnitude of the non-zero coefficients. The computer readable medium of claim 22, wherein the instructions cause the one or more processors to encode the refinement of the video block such that the refined coefficients of the video block are not accessed The first quality code is decodable in the case of coefficient information of the video block of the portion of the base layer bit stream. The computer readable medium of claim 22, wherein the instructions cause the one or more processors to encode the refinement of the video block using only a single variable length code (VLC) table. The computer readable medium of claim 22, wherein the instructions cause the one or more processors to encode the video zone with the first quality using a write code technique Block, the write code technique analyzes a coefficient vector of the video block in a first write once operation, and encodes the coefficient vector in a second write once operation based on the analysis. The computer readable medium of claim 28, wherein the instructions cause the one or more processors to: encode the video block with the first quality comprising 10 according to ITU-T H.264/MPEG-4 An audiovisual write code (AVC) standard uses a content adaptive variable length write code (CAVLC) process to encode the video block at the first quality; and the refinement encoding the video block includes use in the CAVLC One of the VLC tables defined in the process encodes the refinement of the video block. The computer readable medium of claim 22, wherein the first quality and the second quality comprise one of a first signal to noise ratio (SNR) and a second signal to noise ratio (SNR) and a first spatial resolution One of the degree and a second spatial resolution. An apparatus for encoding video data using an adjustable video write code, the apparatus comprising: a first component for encoding a video block as a portion of a base layer bit stream with a first quality; and for encoding the video zone Refining the block as a second component of at least one portion of the enhancement layer bitstream, the refinement resulting in a second quality greater than the first quality when combined with the video block encoded with the first quality The video block, wherein the refinement of the video block is not implemented Rows are encoded in a case where the first write code of the statistical data used in the selection of the video code table is encoded in one operation, wherein the refinement is at least one of the extra coefficient and the refinement of the existing coefficients, and wherein the second coding component Encoding, for each non-zero coefficient of the refinement of the video block, a symbol indicating the presence of at least one residual non-zero coefficient, a run length indicating the number of zero-value coefficients preceding the non-zero coefficient, and the non-zero a sign of the coefficient, and wherein the second encoding means adjusts the magnitudes of the non-zero coefficients refined by the video blocks to be equal to one. The apparatus of claim 31, wherein the second encoding component encodes each non-zero coefficient of the refinements without analyzing any subsequent coefficients. The apparatus of claim 32, wherein the second encoding component encodes a symbol to indicate that there are no residual non-zero coefficients in the refinement of the video block after encoding a last non-zero coefficient. The apparatus of claim 31, wherein the second encoding component encodes the coefficients without encoding the magnitude of the non-zero coefficients. The device of claim 31, wherein the second encoding component encodes the refinement of the video block such that the refined coefficients of the video block are encoded as the base layer without accessing the first quality. The coefficient information of the video block of the portion of the bit stream is decodable. The device of claim 31, wherein the second encoding component encodes the refinements of the video block using only a single variable length write code (VLC) table. The device of claim 31, wherein the first encoding component encodes the video block with the first quality as part of the base layer comprises using a code writing technique The video block is encoded by the first quality, and the code writing technique analyzes a coefficient vector of the video block in a first write operation, and encodes the second code in one operation based on the analysis. Coefficient vector. The device of claim 37, wherein the first encoding component performs the operation of encoding the video block with the first quality according to the ITU-T H.264/MPEG-4 Part 10 Advanced Video Recording (AVC) standard The video block is encoded with the first quality using a content adaptive variable length write code (CAVLC) process; and the refinement encoding the video block includes use in a VLC table defined in the CAVLC process One encodes the refinement of the video block. The device of claim 31, wherein the first quality and the second quality comprise one of a first signal to noise ratio (SNR) and a second signal to noise ratio (SNR) and a first spatial resolution and one One of the second spatial resolutions. A method for decoding video data using an adjustable video write code, the method comprising: decoding a base layer bit stream by a video writing device to obtain a video block of a first quality; decoding the video device by the video writing device Enhancing the layer bitstream to obtain refinement of the video block, the refinement causing the video block having a second quality when combined with the decoded first video block of the first quality, wherein the video blocks Refined to at least one of the additional coefficients and the refinement of the existing coefficients, and Decoding the enhancement layer includes decoding, for each non-zero coefficient of the refinement of the video block, a symbol indicating the presence of at least one residual non-zero coefficient, and a number indicating a zero-value coefficient before the non-zero coefficient The length of the run length and a sign of the non-zero coefficient; and the video code writing device sets a magnitude of each non-zero coefficient equal to one. The method of claim 40, further comprising decoding a symbol indicating that there are no residual non-zero coefficients in the refinements of the video block after a last non-zero coefficient. The method of claim 41, further comprising generating the refinement of the video block using the decoded run of each coefficient, the sign of each coefficient, and the symbol indicating the absence of residual non-zero coefficients One of the coefficients of the vector. The method of claim 40, wherein the refining the decoding of the video block comprises decoding the refinement of the video block without accessing coefficient information of the video block encoded by the first quality. . The method of claim 40, further comprising decoding the refinement of the video block using only a single variable length code (VLC) table. The method of claim 44, wherein the single VLC table comprises one of the VLC tables specified in a CAVLC as defined in the ITU-T H.264/MPEG-4 Part 10 Advanced Video Write Code (AVC) standard. By. An apparatus for decoding video data using an adjustable video write code, the apparatus comprising at least one decoder for: decoding a base layer bitstream to obtain a first quality video zone Decoding a enhancement layer bitstream to obtain refinement of the video block, the refinement causing the video block having a second quality when combined with the decoded first video block of the first quality And wherein the refinement is at least one of an additional coefficient and a refinement of the existing coefficients, and wherein the at least one decoder decodes an indication for each of the non-zero coefficients of the refinement of the video block. a sign of a residual non-zero coefficient, a run length indicating a number of zero values before the non-zero coefficient, and a sign of the non-zero coefficient; and setting a magnitude of each non-zero coefficient equal to one. The apparatus of claim 46, wherein the at least one decoder decodes a symbol indicating that there are no residual non-zero coefficients in the refinement of the video block after a last non-zero coefficient. The device of claim 47, wherein the at least one decoder generates the video block by using the decoded path of each coefficient, the sign of each coefficient, and the symbol indicating that there is no residual non-zero coefficient One of the coefficients of the refined coefficients. The device of claim 46, wherein the at least one decoder decodes the refinement of the video block without accessing coefficient information of the video block encoded by the first quality. The device of claim 46, wherein the at least one decoder decodes the refinement of the video block using only a single variable length write code (VLC) table. The device of claim 50, wherein the single VLC table is included as defined in One of the VLC tables specified in the CAVLC in the ITU-T H.264/MPEG-4 Part 10 Advanced Video Recording (AVC) standard. The device of claim 46, wherein the at least one decoder comprises: a base layer decoder that decodes the base layer bitstream with the first quality to obtain the video block; and an enhancement layer decoding The enhancement layer decoder decodes the enhancement layer bitstream to obtain such refinements of the video block. The device of claim 46, wherein the device comprises a wireless communication device. The device of claim 46, wherein the device comprises an integrated circuit device. A computer readable medium containing instructions for causing one or more processors to: decode a base layer bitstream to obtain a video block of a first quality; and decode an enhancement layer bitstream to Obtaining the refinement of the video block, the refinement, when combined with the decoded video block of the first quality, results in the video block having a second quality, wherein the refinement is an additional coefficient and At least one of the refinements of the existing coefficients, and wherein the instructions cause the one or more processors to decode one for each of the non-zero coefficients of the refinement of the video block indicating that there is at least one residual non-zero a sign of the coefficient, a run length indicating the number of one of the zero value coefficients before the non-zero coefficient, and a sign of the non-zero coefficient; and setting a magnitude of each non-zero coefficient equal to one. The computer readable medium of claim 55, wherein the instructions cause one or more The processor decodes a symbol indicating that there are no residual non-zero coefficients in the refinement of the video block after a final non-zero coefficient. The computer readable medium of claim 56, wherein the instructions cause the one or more processors to use the decoded fortra of each coefficient, the sign of each coefficient, and the symbol indicating that there are no residual non-zero coefficients Generating a vector of such refined coefficients of the video block. The computer readable medium of claim 55, wherein the instructions cause the one or more processors to decode the video block without accessing coefficient information of the video block encoded by the first quality Refined. The computer readable medium of claim 55, wherein the instructions cause the one or more processors to decode the refinement of the video block using only a single variable length write code (VLC) table. The computer readable medium of claim 59, wherein the single VLC table comprises a VLC table as defined in a CAVLC as defined in the ITU-T H.264/MPEG-4 Part 10 Advanced Video Code (AVC) standard One of them. An apparatus for decoding video data using an adjustable video write code, the apparatus comprising: a first component for decoding a base layer bitstream to obtain a first quality video block; and for decoding a reinforcement layer a stream stream to obtain a second component of the refinement of the video block, the refinement causing the video block having a second quality when combined with the decoded video block of the first quality, wherein the video stream Refining to at least one of the additional coefficients and the refinement of the existing coefficients, and The second decoding component decodes, for each of the non-zero coefficients of the refinement of the video block, a symbol indicating that there is at least one residual non-zero coefficient, and a number indicating a zero-value coefficient before the non-zero coefficient The length of the run length and a sign of the non-zero coefficient; and setting a magnitude of each non-zero coefficient to be equal to one. The apparatus of claim 61, wherein the second decoding component decodes a symbol indicating that there are no residual non-zero coefficients in the refinement of the video block after a last non-zero coefficient. The device of claim 62, further comprising: the decoded run of each coefficient, the sign of each coefficient, and the symbol indicating the absence of residual non-zero coefficients to generate the video block A component of the vector of refined coefficients. The device of claim 61, wherein the second decoding component decodes the refinement of the video block without accessing coefficient information of the video block encoded by the first quality. The device of claim 61, wherein the second decoding component decodes the refinement of the video block using only a single variable length write code (VLC) table. The device of claim 65, wherein the single VLC table comprises one of the VLC tables specified in a CAVLC as defined in the ITU-T H.264/MPEG-4 Part 10 Advanced Video Recording Code (AVC) standard. By.