JP2005519543A - Method and system for layer video coding - Google Patents

Abstract

Method for encoding a video image having a plurality of pixel blocks having at least one region determined to be important in a corresponding sublayer in a layer encoding system having at least one layer having a plurality of sublayers It is. The method includes associating an importance level with each block of known size in the at least one critical region, and the importance level of at least one block of known size included in a subsequent large block. Depending on at least one of the subsequent large blocks, and associating a level of importance with each of the associated importance levels. In another embodiment of the invention, an importance map is transmitted and the corresponding image layer is reconstructed using the importance map.

Description

  The present invention relates to video image encoding, and more particularly to partially encoding an enhancement layer of a layer encoded video image.

  Layer coding and wavelet coding, such as Fine Granular Scalar (FGS), are well known in video image coding technology. For example, FGS encoding encodes an image into a base layer and an enhancement layer. The base layer represents the smallest image that can be transmitted in a network with acceptable quality. The enhancement layer represents additional image detail that can be transmitted in the network when sufficient remaining bandwidth is available.

  The enhancement layer is encoded in a bit plane format, the most significant bit of each enhancement layer value is stored in the first bit plane, and each subsequent bit of each enhancement layer value is stored in the corresponding bit plane. During enhancement layer transmission, the value of each bit plane is transmitted continuously until the available bandwidth is occupied.

  The concept of partial bit planes was introduced in JPEG-2000 to distinguish the importance of various bits in the bit plane and to improve the efficiency of bit plane encoding in the bit plane. This concept does not exist in other layer coding methods such as FGS. Therefore, there is a need for an encoding method and apparatus in which regions of the video image that are determined to be superior are identified prior to encoding the enhancement layer.

  It should be understood that the accompanying drawings are merely intended to illustrate the concept of the invention and are not intended as a definition of the limits of the invention. The embodiment shown in FIGS. 1-4 and described in detail is to be used as an exemplary embodiment and should not be recognized as the only way of implementing the invention. Where appropriate, reference numbers that supplement reference characters as much as possible are used to identify similar elements.

  Method for encoding a video image having a plurality of pixel blocks having at least one region determined to be superior in a corresponding sublayer in a layer encoding system having at least one layer having a plurality of sublayers Are disclosed herein. The method includes associating an importance level with each known sized block in at least one upper region, and an importance level at least one of the known sized blocks included in the continuous layer block. Associating with each successive layer block depending on the importance level and mapping each associated importance level.

  In another embodiment of the invention, the importance map is transmitted and the corresponding image layer is reconstructed using the importance map.

  FIG. 1 shows a block diagram of an exemplary partial bit-plane encoder 100 in accordance with the principles of the present invention. In this figure, an input signal 110 is provided to an adder 115 and mixed with a motion compensated image, as further described below. The combined signal is then provided to a discrete coefficient transform (DCT) to convert pixel values to coefficients. The DCT coefficients are then provided to quantizer 125 for quantization. The quantized DCT coefficients are then provided to variable length encoder 130 and combiner 175.

  The quantized DCT coefficients are also provided to the inverse quantizer 135 to recover the DCT coefficients. It should be understood that the recovered DCT coefficients are not exactly the same as the original DCT values when some information is lost in the quantization process. The inverse quantized coefficients are then applied to the inverse DCT 140 to recover the original pixel elements after the DCT and quantization process. Similarly, there is a known difference between the original and recovered pixel elements because some information is lost in the quantization process. The recovered pixel elements are provided to a motion predictor / motion compensator 45. The motion prediction / compensation signal is then added to adder 115 and combined with the original image 110.

  The summed image 150 is also added to the adder 155 along with the recovered pixel elements output from the inverse DCT 140. The output of the adder is the remaining element between the original signal 110 and the recovered base layer image. The remaining images are added to enhancement layer encoder 160 and importance map encoder 165 simultaneously. The importance map encoder results are further applied to enhancement encoder 170 to map to the bit plane, as described in more detail below.

  The outputs of enhancement layer 170 and importance map 165 are provided to combiner 180 and the combined output is provided to combiner 175. The output 190 of the combiner 175 is then transmitted in the network or stored for the next transmission.

  FIG. 2a shows an image frame with important information such as changes in border, color or texture. Important image regions 210, 215, 220 are identified using known methods. In contrast, regions that show little or no change in texture can be identified as non-critical. Therefore, there is no or little information about these areas. Thus, in one embodiment of the present invention, the determination of critical areas can be made by examining each pixel element. In the preferred embodiment, the critical region determination can be made by examining the corresponding DCT coefficients.

  FIG. 2b shows another embodiment of the present invention, where an important image area, eg 210, is associated with a plurality of blocks corresponding to the macroblock and the upper macroblock. Although a specific segmentation of the image is shown, it will be understood that the image can be segmented according to other criteria, as described below. In this illustrative example, the image area 210 has upper macroblocks 222, 224, 226, 228, 230 and 232.

  Each upper macroblock can be divided into macroblocks. For clarity, the upper macroblock 222 is shown as being divided into macroblocks 240, 242, 244 and 246. Each macroblock 240, 242, 244 and 246 can be further divided into sub-macroblocks. Each sub macroblock can be further divided into blocks. For purposes of clarity, the lower macroblock is shown as being divided into blocks 260, 262, 264, and 266. As will be appreciated, each upper macroblock can be similarly divided, identified and associated with macroblocks, lower macroblocks and blocks.

  In a preferred embodiment, block 260 contains information related to the 8x8 configuration of image elements. Further, the lower macroblock 250 is associated with a 16x16 configuration of pixel elements, the macroblock 240 is associated with a 32x32 configuration of pixel elements, and the upper macroblock 222 is associated with a 64x64 configuration of pixel elements. In this preferred embodiment, block 260 is similar to DCT encoding of the corresponding block of pixel elements.

  FIG. 2c shows a bit plane mapping of the identified critical region 210 in the bit planes 272, 274 and 276, according to a preferred embodiment of the present invention. In this case, the enhancement layer is encoded using a bit plane of 3 bits. However, the depth of the bit plane can be any number and is not intended to limit the bit plane depth to the bit plane depth shown here. In this preferred embodiment, the DCT information is mapped to each bit plane so that the upper macroblock, macroblock, lower block and block associated with region 210 can be easily identified. .

  FIG. 3a shows a flowchart of an exemplary process 300 for importance mapping in accordance with the principles of the present invention. In this process, importance mapping is started at an arbitrarily selected bit plane associated with an image or picture. In the preferred embodiment shown, the bit plane associated with the most significant bit, ie bit plane 0, has been selected in block 305. At block 310, an importance map associated with the selected bit plane is determined. At block 315, the importance map associated with the bit plane is decoded. At block 320, the texture of the block identified as being superior is decoded and a bit-related representation of the importance map is generated. This bit-related indication in the importance map can be decoded on the receiving device to understand the importance map. At block 325, a determination is made whether the bit plane associated with the image has been processed. If the answer is negative, the next / following bit plane is selected at block 332 and the importance mapping process continues for the selected next / following bit plane.

  However, if the answer is affirmative, at block 330 a determination is made whether all the images have been processed. If the answer is negative, the next / following image or picture is selected at block 334. The importance mapping process then continues for each bit plane in the selected next / following image or picture.

  FIG. 3 b shows a flowchart of an exemplary importance mapping process 310. In this exemplary process, the initial block size and associated minimum and fine block sizes are determined at block 340. In this case, the initial block size associated with the preferred block size is shown. At block 345, it is determined whether the current block size is equal to the minimum block size. If the answer is positive, a determination is made at block 350 as to which non-zero coefficient the current block has. If the answer is positive, the relevant block is marked or identified as important at block 355.

  However, if the answer is negative, at block 370 the block is marked or identified as important.

  After the current thlock is identified as significant at block 355 or as unimportant at block 370, a determination is made at block 360 as to whether the last block has been reached. If the answer is negative, the next / following block in the bit plane is selected at block 365. Processing continues at the selected next / next block in block 345.

  However, if the answer at block 360 is positive, i.e., all blocks at the current size have been processed, a determination is made whether the current block size is greater than the maximum block size. If the answer is negative, at block 380 the current block size is increased, preferably doubled. Processing continues in each block associated with the increase in size at 345.

  Returning to the determination at block 345, if the answer is negative, a determination is made at block 385 as to whether the smaller block, i.e., the child block within the larger block, is significant. If the answer is negative, at block 355 the larger block is marked or identified as important. However, if the answer is negative, at block 370, the larger block is marked or identified as important.

  At this time, processing continues in each subsequent large block until the block size is greater than the maximum block size in block 375.

  FIG. 4 illustrates an exemplary embodiment of a system 400 that can be used to implement the principles of the present invention. System 400 is a television transmission or transmission system, desktop, laptop or palmtop computer, portable digital terminal (PDA), video / image storage device (VCR) such as a video cassette recorder, digital video recorder (DVR), TiVO. It is possible to represent devices, etc., and some or combinations of these and other devices. The system 400 can include one or more input / output devices 402, a processor 403, and memory 404, and can access one or more sources 401 having video images. . The source 401 can be stored on a permanent or semi-permanent medium such as a television receiver (SDTV or HDTV), VCR, RAM, ROM, hard disk drive, optical disk drive or other video image storage device. Source 401 may also be part of, for example, the Internet, a wide area network, a metropolitan area network, a narrow area network, a terrestrial broadcast system, a cable network, a satellite broadcast network, a wireless network, a telephone network, and other and other types of networks. It can be accessed over one or more network connections 410 to receive video from a server in a global computer communications network, such as a combined service.

  The input / output device 402, the processor 403, and the memory 404 can communicate over a communication medium 406. Communication medium 406 can represent, for example, a bus, a communication network, one or more circuits, an internal connection of a circuit card or other device, and some or a combination of these or other communication media. . Input data from source 401 is one or more software that can be executed by processor 403 and stored in memory 404 to provide a partially encoded video image to network 420. Processed according to the program. The partially encoded video image can be transmitted to a storage device or can be transmitted to a display system for viewing the encoded video image in real time.

  The processor 403 provides a known output in response to a known input, laptop computer, desktop computer, portable computer, dedicated logic circuit, integrated circuit, programmable array logic (PAL), application specific integrated circuit (ASIC). ) Etc., and can be a means such as a general purpose or special purpose computing system.

  In a preferred embodiment, encoding and decoding using the principles of the present invention can be performed by a computer readable code executed by the processor 403. The code can be executed by the memory 404 or read / downloaded from a storage medium such as a CD-ROM or floppy disk (not shown). In other embodiments, the hardware circuitry can be used in place of or in combination with software instructions to implement the present invention. For example, the elements shown here can also be implemented as separate hardware elements.

  In one aspect of the invention, the processor is in communication with at least one processing unit and other devices such as peripheral devices electrically connected thereto and one or more memories. Can represent a computing unit or one or more processing units. Further, the device is, for example, an ISA bus, a microchannel bus, or a PCI bus. One by an internal bus such as a PCMCIA bus, an internal connection of one or more circuits, a circuit card or other device, and their and other communication media or parts and combinations of external networks such as the Internet and Intranet Or it can be electrically connected to more processing units.

  Furthermore, the novel features of the present invention have been described with reference to the drawings and as applied to the preferred embodiments. Various deletions, additions and modifications may be made by those skilled in the art in the apparatus described, the details and construction of the apparatus disclosed, and their operation without departing from the scope and spirit of the invention. Need to be understood. For example, although the present invention has been described with respect to FGS coding, it should be understood that the present invention is also suitable for similarly developed layer coding systems. Similarly, although the upper macroblock has been described with respect to a 64x64 array or matrix, it is necessary to change the block size within the knowledge of those skilled in the art. Furthermore, although the upper macroblock boundaries have been shown to be fixed, the upper macroblock boundaries can be considered to be dynamically determined based on the initial identification of important data. .

  It is also expressly intended that all combinations of elements that perform substantially the same function in substantially the same way to achieve the same result are within the scope of the invention. Replacement of elements from one to the other of the described embodiments is also fully contemplated and contemplated.

FIG. 2 illustrates an FGS partial bit plane encoder in accordance with the principles of the present invention. It is a figure which shows the enhancement layer bit plane by which importance was mapped. FIG. 4 illustrates an exemplary block diagram flowchart for identifying important image regions within an image in accordance with the principles of the present invention. FIG. 5 shows a flowchart of an exemplary process for generating an importance map in accordance with the principles of the present invention. FIG. 2 illustrates a system for determining importance mapping layer bit planes in accordance with the principles of the present invention.

Claims (13)

Method for encoding a video image having a plurality of pixel blocks having at least one region determined to be important in a corresponding sublayer in a layer encoding system having at least one layer having a plurality of sublayers Because:
a. Associating an importance level with each block of known size within the at least one important region;
b. Associating an importance level with each of at least one of the subsequent large blocks depending on at least one of the importance levels of the block of known size contained within the subsequent large block; and c. Mapping each of the associated importance levels;
A method characterized by comprising: The method of claim 1, wherein:
Repeating steps a through c for each of the sublayers;
The method further comprising: The method of claim 1, wherein:
Transmitting the importance level mapped corresponding to the sublayer;
The method further comprising:   The method of claim 1, wherein the layer coding system is a fine grain scalable (FGS) system.   5. The method of claim 4, wherein the sublayer is a bit plane.   The method of claim 1, wherein the block size is selected from a set of predetermined sizes.   The method of claim 1, wherein the subsequent large block has a known maximum. A system for encoding a video image configured as a plurality of pixel blocks in at least one layer, wherein one of the layers has a plurality of sublayers, and the sublayer has at least one important region. The system is:
Means for associating an importance level with each block of known size within the at least one important region;
Means for identifying a level of importance for each of the at least one subsequent large block that depends on at least one of the importance levels of the block of known size contained within the subsequent large block; and Stage for mapping;
A method characterized by comprising:   9. The system of claim 8, wherein the mapping comprises information about the known size block and subsequent blocks having a known level.   9. The system of claim 8, wherein the known level represents a non-zero coefficient. A decoding system for decoding an image transmitted as a layer coded signal:
Means for receiving data corresponding to an importance map of at least one sublayer of the layer encoded signal;
Means for decoding the importance map; and means for reconstructing one corresponding to the sublayer from the importance map;
A decoding system comprising: 12. The decoding system according to claim 11, wherein:
Means for receiving said layer encoded signal transmitted in a network;
A decoding system characterized by comprising:   12. The decoding system according to claim 11, wherein the importance map includes information regarding blocks having important information.

Images