KR20110052527A - Method and apparatus for video encoding/decoding using adaptive codeword assignment to syntax element - Google Patents

Abstract

PURPOSE: A method and apparatus for video encoding/decoding using an adaptive codeword assignment to syntax elements are provided to enhance encoding performance under an image encoding environment having various encoding bit rates and various image sizes. CONSTITUTION: A feature analysis unit(111) analyzes the features of encoding data obtained from an input image, and a codeword assignment unit(112) allocates a codeword to a symbol value of a syntax element when using the input image. An encoding unit(113) encodes the syntax elements in the codeword to generate a bit stream.

Description

Image coding / decoding method and apparatus for adaptively allocating codewords to syntax elements {METHOD AND APPARATUS FOR VIDEO ENCODING / DECODING USING ADAPTIVE CODEWORD ASSIGNMENT TO SYNTAX ELEMENT}

The present invention relates to an image encoding and decoding apparatus. More particularly, the present invention relates to an image encoding performance by adaptively assigning a codeword to a syntax element in an image encoding environment having various resolutions and quality. It relates to a method and apparatus that can be improved.

The present invention is derived from the research conducted as part of the IT original technology development project of the Korea Communications Commission, the Korea Institute of Electronics and Telecommunications, and Kwangwoon University. [Task Management No .: 2008-F-011-01, Development of High Efficiency Video Codec Technology.

Recently, various devices having multimedia functions have been spread, and various multimedia services such as IPTV, DMB, and video telephony have been consumed through the various devices.

In this case, since the images for the services have various image quality and resolution and each transmission environment is different, the characteristics of the encoded image are also different.

In the conventional video encoding standard method, encoding and decoding are performed by assigning predetermined codewords to syntax elements.

However, the probabilistic statistics of the codewords defined in the individual syntax elements may not be appropriate at all coded bit rates of all images, and do not reflect the global or local characteristics of the image being coded, so that the codewords of predetermined syntax elements Cannot show optimal performance in multiple encoding environments.

Accordingly, there is a need for an efficient video encoding / decoding apparatus and method capable of adaptively assigning codewords to syntax elements in video services in various environments.

An embodiment of the present invention provides a method and apparatus for improving image encoding performance by adaptively assigning codewords to syntax elements in an image encoding environment having various resolutions and quality.

 An image encoding apparatus for adaptively allocating codewords according to a syntax element according to an embodiment of the present invention may include a feature analyzer configured to analyze a feature of an input image and a feature of encoded data encoding the input image; A codeword allocator configured to allocate a codeword to a symbol value of a syntax element used to encode the input image according to a feature of the encoded data; And an encoder configured to encode a syntax element of the input image using the codeword to generate a bitstream.

According to an embodiment of the present invention, by adaptively assigning codewords to syntax elements, encoding performance is improved in an image encoding environment having various image sizes and various encoding bit rates.

1 is a diagram illustrating a configuration of an image encoding apparatus and a decoding apparatus for adaptively allocating codewords according to syntax elements according to the present invention.
2 is a flowchart illustrating an example of an image encoding method for adaptively allocating codewords according to the present invention.
3 is a flowchart illustrating another example of an image encoding method for adaptively allocating codewords according to the present invention.
4 is a flowchart illustrating still another example of an image encoding method for adaptively allocating codewords according to the present invention.
5 is a flowchart illustrating a method of reallocating codewords and confirming whether the reallocated codewords are suitable according to the present invention.
6 is a flowchart illustrating an image decoding method of adaptively allocating codewords according to the present invention.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating a configuration of an image encoding apparatus and a decoding apparatus for adaptively allocating codewords according to syntax elements according to the present invention.

The present invention is a code for syntax elements in units of sequence, picture, slice, macroblock, and block, reflecting the characteristics of the encoded data according to the characteristics of the image and the encoding environment. The word is adaptively allocated to encode or decode the input image. In this case, the syntax element may mean a form in which information of a sequence, a picture, a slice, a macroblock, and a block is written in a bitstream.

The above-described macroblock refers to a unit of image encoding and decoding, and an encoding or decoding unit in image encoding and decoding refers to the divided unit when dividing an image into subdivided blocks to encode or decode the image. Therefore, it may be called a macroblock or a coding unit or a unit.

Terms used in the present specification are terms used to properly express preferred embodiments of the present invention, which may vary depending on the intention of a user, an operator, or customs in the field to which the present invention belongs. Therefore, the definitions of the terms should be made based on the contents throughout the specification.

The image encoding apparatus 110 according to the present invention may add and store codeword allocation information about syntax elements determined according to the characteristics of an image and encoding conditions to a bitstream. In addition, the image decoding apparatus 120 according to the present invention may decode the syntax element by using codeword assignment information on the syntax element indicated in the bitstream.

Referring to FIG. 1, the image encoding apparatus 110 adaptively allocating codewords according to syntax elements is an encoding apparatus allocating codewords for adaptive syntax elements in macroblock units in a slice unit lower layer. It consists of an analysis unit 111, a codeword assignment unit 112, and an encoding unit 113.

When encoding the input image, the image encoding apparatus 110 may analyze the characteristics of the input image and the encoded data according to the encoding environment in units of sequences, pictures, slices, macroblocks, and blocks.

The characteristic of the encoded data described above may mean a frequency of occurrence of a symbol value, a probability of occurrence, etc. for each syntax element.

The above-described encoded data may mean a form in which an input image is encoded, and may mean a bitstream that is an aggregated form of codewords.

Also, the image encoding apparatus 110 may determine and allocate an optimal codeword for a symbol value of a syntax element based on a feature of an input image and a feature of encoded data, and add the allocated codeword information to a bitstream. .

The feature analyzer 111 analyzes a feature of the input image and a feature of the encoded data encoding the input image. In detail, the feature analyzer 111 may encode the input image in a slice unit and analyze a feature of the encoded data for each slice unit.

In this case, the feature analyzer 111 may predict and analyze what features the input image has and what features the encoded data will have.

For example, the feature analyzer 111 may analyze the features of the encoded data by calculating statistics of syntax elements used to encode the input image and codewords allocated to the encoded data.

As another example, the feature analyzer 111 may extract a feature of the input image and predict a probability of generating a symbol value for each syntax element in the encoded data based on the feature of the input image.

In addition, the feature analyzer 111 may analyze the feature of the syntax element by calculating a statistic of a symbol value of an already encoded syntax element. In this case, the syntax element that is already encoded may be a syntax element encoded by a codeword assigned by the encoder 113 to the codeword allocator 112.

The codeword assignment unit 112 allocates a codeword suitable for a syntax element used when encoding the input image according to the feature analyzed by the feature analyzer 111.

In detail, the codeword allocator 112 may determine whether the basic codeword is suitable for the codeword allocated to the encoded data according to the input image, and if not, the syntax element used to encode the input image. Can be assigned a codeword suitable for.

In this case, whether or not the codeword is suitable can be used by using the bit amount generated when the codeword is encoded and the distortion degree compared to the original image of the encoding target image, and using the bit amount and the distortion with respect to the original image of the encoding target image. Distortion costs are available. In this case, it may be determined that the codeword is appropriate when the amount of bits below a certain threshold, the distortion below the threshold, and the rate-distortion cost below the threshold occur.

In this case, the basic codeword may be a codeword assigned by default regardless of the feature of the input image and the encoding environment according to the existing method.

The codeword allocator 112 may check whether the codewords allocated to the encoded data according to the input image are suitable by using the statistics of symbol values for each syntax element calculated by the feature analyzer 111.

In this case, whether or not the codeword is suitable may use the bit amount generated when the codeword is encoded, and the distortion of the original video of the encoding target video. Distortion costs can be used. In this case, it may be determined that the codeword is appropriate when the amount of bits below a certain threshold, the distortion below the threshold, and the rate-distortion cost below the threshold occur.

In addition, the codeword allocating unit 112 uses the codewords allocated to the coded data according to the basic codeword and the input image, and the amount of bits generated when the codewords are encoded using the respective codewords. It is possible to determine which codeword is more suitable through the rate-distortion cost using the bit amount and distortion compared to the original image of the encoding target image. At this time, it can be determined that a codeword representing a smaller amount of bits generated, a smaller image distortion, and a smaller rate-distortion cost is more suitable.

In addition, the codeword assignment unit 112 checks whether the basic codeword is suitable for the input image using the occurrence probability of the symbol value of the syntax element predicted by the feature analysis unit 111, and if it is not suitable, A codeword suitable for a symbol value of a syntax element according to the input image may be allocated based on the probability.

The codeword assignment unit 112 checks whether the allocated codeword is suitable for encoding the input image by using the statistics of symbol values of the already encoded syntax element, and if it is not suitable, the encoded syntax element. A codeword suitable for a syntax element used when encoding an input image may be reassigned based on the symbol value statistics of.

The codeword allocator 112 may store codeword information allocated by the codeword allocator 112 to symbol values of the syntax elements in a slice header of the input image.

For example, the codeword assignment unit 112 stores codeword assignment information indicating a codeword assigned to a symbol value of each syntax element in a slice header as shown in Table 1, and uses a codeword assignment scheme stored in a slice header. Commonly used in slices of.

In this case, information added to the slice header may include syntax_reordering_control_flag, changing_of_syntax_idc, syntax_element_idc, and diff_syntax_element_minus1.

In this case, syntax_reordering_control_flag may be stored as 1 when reassigning a flag indicating whether or not to reassign a codeword to a syntax element in a corresponding slice.

In addition, changing_of_syntax_idc may store a type of syntax element for changing a codeword in a corresponding slice.

 For example, changing_of_syntax_idc may store each type using fixed 3 bits for 8 types of syntax elements as shown in Table 2.

However, since the types of syntax elements that can be changed may increase or decrease according to embodiments, the method of changing_of_syntax_idc stores the types of syntax elements may also use a method of allocating bits with a variable length.

In addition, syntax_element_idc may store information about a symbol value change among syntax elements to change the codeword determined by changing_of_syntax_idc.

In addition, diff_syntax_element_minus1 may store to which codeword a specific symbol value is mapped in a codeword table when the codeword is reassigned.

For example, the codeword allocator 112 may store the P_slice_sub_MB type of H.264 / AVC in the form shown in Table 3.

When the codeword assignment unit 112 decides to change the codeword of P_L0_4x4 of P_MB_type among MB Sub-MB type for P slices, changing_of_syntax_idc becomes MB Sub-MB type for P slices, and syntax_element_idc indicates '3' representing P_L0_4x4. Can be.

In addition, when the shortest codeword of P_slice_sub_MB is assigned because the frequency of occurrence of the symbol value is the highest, the component has the highest priority and becomes the position of P_L0_8x8, so diff_syntax_element_ minus1 becomes '3-1' and becomes '2'. Can be

As another example, the codeword assignment unit 112 may store a codeword assigned to a symbol value of each syntax element in a slice header, as shown in Table 4.

Table 4 is a method used when a codeword for a symbol value of a syntax element indicated in a slice header is updateable in units of macroblocks in a slice, and Table 3 further includes update_of_syntax_MB_flag.

The method may be configured in such a manner that syntax elements marked in the slice header are updated and reflected in macroblock units.

In this method, the macroblock unit update may statistically use information of the previous macroblock in the slice, and whether to use the update may be encoded by storing update_of_syntax_MB_flag in the macroblock header.

In this case, the storage of whether the codeword update occurs and whether the update is used may be a block, slice, or picture unit as well as a macroblock.

The encoder 113 generates a bitstream by encoding an input image with a basic codeword or a codeword assigned by the codeword allocator 112.

The encoding apparatus 110 may improve encoding efficiency by analyzing a statistic reflecting local or global characteristics of an image with respect to a specific syntax element and assigning a codeword having a short length to frequently occurring symbol values.

The video decoding apparatus 120 adaptively assigning codewords according to syntax elements includes a bitstream analyzer 121, a codeword discriminator 122, and a decoder 123 as illustrated in FIG. 1. do.

The bitstream analyzer 121 extracts codeword assignment information from the slice header of the bitstream received from the video encoding apparatus 110.

In detail, the bitstream analyzer 121 may interpret the received bitstream information in the factor analysis step to check information on syntax elements and extract codeword allocation information stored in the slice header. In this case, the codeword assignment information may be information of a codeword allocated by the codeword assignment unit 112 for syntax elements of the bitstream.

The codeword determination unit 122 checks whether the bitstream is encoded as a basic codeword based on the codeword assignment information.

The decoder 123 decodes the bitstream using codeword allocation information extracted by the bitstream analyzer 121 when the bitstream is not encoded as a basic codeword.

In this case, the decoder 123 may decode the bitstream into the basic codeword when the bitstream is encoded into the basic codeword.

2 is a flowchart illustrating an example of an image encoding method for adaptively allocating codewords according to the present invention.

2 is an example of a method of performing encoding twice for statistics on syntax elements.

In operation S210, the encoding apparatus 110 receives an image to be encoded.

In operation S220, the encoder 113 generates encoded data by encoding the input image received in operation S210 in units of slices. In this case, the encoder 113 may encode an input image by using an existing encoding method. That is, the input image may be encoded by using the basic codeword.

In operation S230, the feature analyzer 111 calculates symbol value statistics of syntax elements included in the input image received in operation S220 or statistics of codewords allocated to encoded data encoded in operation S220. .

In step S240, the codeword assignment unit 112 checks whether the basic codeword is suitable as the codeword allocated for the syntax element encoding of the input image.

Whether or not the codeword is appropriate can be used by using the amount of bits generated when the codeword is encoded and the distortion compared to the original image of the encoding target image. Can be used. In this case, it may be determined that the codeword is appropriate when the amount of bits below a certain threshold, the distortion below the threshold, and the rate-distortion cost below the threshold occur.

For efficient encoding in general entropy encoding, short codewords are assigned to symbol values of statistically frequent syntax elements and relatively long codewords are assigned to symbol values of less frequently used syntax elements. .

In addition, when the probability of the symbol values of the syntax element is similar, the fixed length codeword may be allocated.

The codeword assignment unit 112 may determine whether statistics about the occurrence frequency of each syntax element are suitable for the assigned codeword using the rule.

In step S250, the codeword assignment unit 112 reassigns the appropriate codeword using the rule when it is determined in step S240 that the basic codeword is not suitable. That is, it can be reallocated using the characteristics of the input image and the statistics of the encoded data.

In this case, the codeword assignment unit 112 allocates a short codeword to a symbol having a high occurrence frequency in the codeword table according to the occurrence of a symbol value of a syntax element, and allocates a long codeword to a symbol having a low occurrence frequency. For example, when the probability of each symbol value generated for one syntax element is similar, a fixed length codeword may be allocated to the symbol value.

In step S260, the codeword assignment unit 112 stores the codeword assignment information for the codeword reassigned in step S250 in the slice header.

In operation S270, the encoder 113 encodes syntax elements of the input image by using the reallocated codeword in operation S250.

In operation S320, the encoder 113 encodes syntax elements of the input image using the basic codeword.

In operation S290, the encoder 113 may encode syntax elements of the input image using the basic codeword when it is determined that the basic codeword is more suitable than the codeword reassigned in operation S240. If it is determined at 250 that the reassigned codeword is appropriate, the syntax element of the input image may be encoded using the reassigned codeword.

3 is a flowchart illustrating another example of an image encoding method for adaptively allocating codewords according to the present invention.

3 is a method of calculating and encoding statistics on syntax elements without performing encoding twice.

In operation S310, the encoding apparatus 110 receives an image to be encoded.

In operation S320, the encoder 113 extracts a feature of a slice image by executing a preprocessing process for extracting an image feature on the input image received in operation S320.

In operation S330, the feature analyzer 111 predicts a probability of occurrence of a symbol value of each syntax element based on the feature extracted in operation S320.

In step S340, the codeword assignment unit 112 checks whether the basic codeword is suitable for encoding the syntax elements of the input image.

Whether or not the codeword is appropriate can be used by using the amount of bits generated when the codeword is encoded and the distortion compared to the original image of the encoding target image. Can be used. In this case, it may be determined that the codeword is appropriate when the amount of bits below a certain threshold, the distortion below the threshold, and the rate-distortion cost below the threshold occur.

For efficient encoding in general entropy encoding, short codewords are assigned to symbol values of statistically frequent syntax elements and relatively long codewords are assigned to symbol values of less frequently used syntax elements. .

In addition, when the probability of the symbol values of the syntax element is similar, the fixed length codeword may be allocated.

The codeword assignment unit 112 may determine whether statistics about the occurrence frequency of each syntax element are suitable for the assigned codeword using the rule.

In detail, the codeword allocator 112 may determine whether the basic codeword corresponds to a codeword of a symbol value of a syntax element predicted to have a high probability of occurrence in step S330 for encoding the input image.

In step S350, if it is determined in step S340 that the basic codeword is not suitable, the codeword assignment unit 112 uses the probability of the symbol value predicted in step S330 to apply the symbol value of the syntax element. Reallocate the appropriate codeword.

In this case, a codeword of a short length may be allocated to a symbol having a high probability of a symbol value, and a codeword of a long length may be allocated to a symbol having a low probability of a symbol value and reallocated a codeword suitable for the symbol value.

In addition, when the probability of the symbol values of a specific syntax element are all similar, a fixed length codeword may be allocated.

In step S360, the codeword assignment unit 112 stores codeword assignment information for the codeword reassigned in step S350 in the slice header.

In operation S370, the encoder 113 encodes syntax elements of the input image by using the reallocated codeword in operation S350.

In operation S380, the encoder 113 encodes a syntax element of the input image using the basic codeword.

In operation S390, the encoder 113 may encode syntax elements of the input image using the basic codeword when it is determined that the basic codeword is more suitable than the codeword reassigned in operation S340. If it is determined in step 350 that the reassigned codeword is appropriate, the syntax element of the input image may be encoded using the reassigned codeword.

4 is a flowchart illustrating still another example of an image encoding method for adaptively allocating codewords according to the present invention.

4 is an example of an encoding method in consideration of a case in which statistics on symbol values of syntax elements obtained through slice encoding are changed by codeword reallocation.

In operation S410, the encoding apparatus 110 receives an image to be encoded.

In operation S420, the encoder 113 generates encoded data by encoding the input image received in operation S410 in units of slices. In this case, the encoder 113 may encode an input image by using an existing encoding method. That is, the input image may be encoded by using the basic codeword.

In operation S430, the feature analyzer 111 calculates symbol value statistics of syntax elements included in the input image input in operation S420 or statistics of codewords allocated to encoded data encoded in operation S430. .

In step S440, the codeword assignment unit 112 checks whether the basic codeword is suitable as the codeword allocated for encoding the input image.

Whether or not the codeword is appropriate can be used by using the amount of bits generated when the codeword is encoded and the distortion compared to the original image of the encoding target image. Can be used. In this case, it may be determined that the codeword is appropriate when the amount of bits below a certain threshold, the distortion below the threshold, and the rate-distortion cost below the threshold occur.

For efficient encoding in general entropy encoding, short codewords are assigned to symbol values of statistically frequent syntax elements and relatively long codewords are assigned to symbol values of less frequently used syntax elements. .

In addition, when the probability of the symbol values of the syntax element is similar, the fixed length codeword may be allocated.

The codeword assignment unit 112 may determine whether statistics about the occurrence frequency of each syntax element are suitable for the assigned codeword using the rule.

In step S450, if it is determined in step S440 that the basic codeword is not suitable, the codeword assignment unit 112 reallocates the codeword suitable for encoding the input image using the above rule. That is, it can be reallocated using the characteristics of the input image and the statistics of the encoded data.

The reassignment and conformance check of the codeword will be described in detail with reference to FIG. 5.

In step S460, the codeword assignment unit 112 stores the codeword assignment information for the codeword reassigned in step S450 in the slice header.

In step S470, the encoder 113 encodes syntax elements of the input image using the reassigned codeword in step S450.

In operation S480, the encoder 113 encodes a syntax element of the input image using the basic codeword.

In operation S490, the encoder 113 may encode syntax elements of the input image using the basic codeword when it is determined that the basic codeword is more suitable than the codeword reassigned in operation S440. If it is determined at 450 that the reallocated codeword is appropriate, the syntax element of the input image may be encoded using the reallocated codeword.

5 is a flowchart illustrating a method of reallocating codewords and confirming whether the reallocated codewords are suitable according to the present invention. Here, steps S510 to S540 may be included in step S450 described with reference to FIG. 4.

In step S510, the codeword allocator 112 allocates a short length codeword to a symbol value of a frequently generated syntax element and a codeword of a relatively long length to a symbol value of a syntax element having a low frequency. The codeword suitable for encoding the input image is reassigned to the syntax element by using a rule for assigning a.

In operation S520, the encoder 113 encodes a syntax element of the input image using the codeword reassigned in operation S510.

In operation S530, the feature analyzer 111 calculates statistics on symbol values of syntax elements in the encoded data encoded in operation S520.

In step S540, the codeword assignment unit 112 checks whether the codeword reassigned in step S510 is suitable as the codeword allocated for symbol value encoding of syntax elements of the input image.

Specifically, the codeword assignment unit 112 checks whether the codeword reassigned in step S510 is suitable for the statistics calculated in step S530, and if not, executes step S510 again. You can repeat until the appropriate codeword is reallocated.

Whether or not the codeword is appropriate can be used by using the amount of bits generated when the codeword is encoded and the distortion compared to the original image of the encoding target image. Can be used. In this case, it may be determined that the codeword is appropriate when the amount of bits below a certain threshold, the distortion below the threshold, and the rate-distortion cost below the threshold occur.

For efficient encoding in general entropy encoding, short codewords are assigned to symbol values of statistically frequent syntax elements and relatively long codewords are assigned to symbol values of less frequently used syntax elements. .

In addition, when the probability of the symbol values of the syntax element is similar, the fixed length codeword may be allocated.

The codeword assignment unit 112 may determine whether statistics about the occurrence frequency of each syntax element are suitable for the assigned codeword using the rule.

6 is a flowchart illustrating an image decoding method of adaptively allocating codewords according to the present invention.

In operation S610, the bitstream analyzer 121 receives a bitstream from the image encoding apparatus 110.

In step S620, the bitstream analyzer 121 extracts slice feature and codeword assignment information from the slice header of the bitstream received in step S610.

In detail, the bitstream analyzer 121 may interpret the received bitstream information in the parsing step to check information on syntax elements and extract codeword allocation information stored in the slice header. In this case, the codeword assignment information is information of codewords allocated by the codeword assignment unit 112 for syntax elements of the bitstream.

In step S630, the codeword determination unit 122 checks whether the bitstream is encoded as a basic codeword based on the codeword assignment information extracted in step S620.

In step S640, the decoder 123 performs the bitstream based on the codeword allocation information extracted by the bitstream analyzer 121 when the bitstream is encoded with the reallocated codeword instead of the basic codeword. Decrypt

In step S650, the decoder 123 decodes the bitstream into the basic codeword when the bitstream is encoded into the basic codeword.

In operation S660, the decoder 123 outputs the image decoded in operation S640 or S650.

The present invention can improve coding performance in an image encoding environment having various image sizes and various encoding bit rates by adaptively assigning codewords to symbol values of syntax elements.

As described above, the present invention has been described by way of limited embodiments and drawings, but the present invention is not limited to the above embodiments, and those skilled in the art to which the present invention pertains various modifications and variations from such descriptions. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined not only by the claims below but also by the equivalents of the claims.

110: encoding device
111: feature analysis unit
112: codeword assignment unit
120: decryption device

Claims (1)

A feature analyzer configured to analyze a feature of an input image and a feature of encoded data encoding the input image;
A codeword allocator for allocating a codeword to a symbol value of a syntax element used to encode the input image according to a feature of the encoded data; And
An encoder which generates a bitstream by encoding a syntax element of the input image using the codeword
Image encoding apparatus comprising a.

Images