JP2011109364A - Image processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce capacity of a syntax element memory. <P>SOLUTION: In a CABAC decoder 100, an index generation part 114 determines one of an absolute value of an mvd syntax element of a reference submb stored in the syntax element memory 112, twice the absolute value, and 1/2 of the absolute value as an index value before generating a context index when a context index is generated. A syntax element replacement part 150 determines whether the index value becomes "0" irrespective of the value of the mvd syntax element when the mvd syntax element of the reference submb is stored in the syntax element memory 112, and when it is determined that the index value becomes "0", replaces the mvd syntax element of the reference submb with "0" to be stored in the syntax element memory 112. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to image processing, specifically CABAC encoding or decoding technology.

  International standard ITU-T Rec. H. H.264 | ISO / IEC 14496-10 (hereinafter abbreviated as H.264) is a wide range of video encoding / decoding methods from low bit rate applications such as mobile phones to high bit rate applications such as HDTV. (Non-patent Document 1, Patent Document 1, Patent Document 2). H. In H.264, as entropy coding methods, CAVLC (Context-based Adaptive Variable Length Coding) and CABAC (Context-based Adaptive Binary Arithmetic Coding) are selected. It can be done. When encoding, it is called a syntax element, and various data obtained from a moving image by a video encoder is encoded by CABAC or CAVLC to obtain a bit stream. When decoding, the CABAC decoder or CAVLC decoder obtains a syntax element from the bit stream in accordance with the encoding method (CABAC or CAVLC) indicated by the information included in the bit stream, outputs the syntax element to the video decoder, and the video decoder uses the syntax. A moving image is obtained from the element.

  Encoding and decoding are processes in reverse order. Here, processing at the time of decoding will be described as a representative.

  FIG. 1 shows a bit stream decoder 10 compliant with H.264. The bit stream decoder 10 includes an entropy decoder 20 and a video decoder 70. The entropy decoder 20 includes a CABAC decoder 30 and a CAVLC decoder 60.

  The entropy decoder 20 decodes the input bit stream by either the CABAC decoder 30 or the CAVLC decoder 60 to obtain a syntax element. The syntax element is data necessary for processing of the video decoder 70, such as an encoding mode, a motion vector, and pixel data.

  The video decoder 70 decodes using the syntax element to obtain an image of each frame constituting the moving image.

  H. In H.264, encoding and decoding are defined to be performed in units of macroblocks obtained by dividing a frame into a lattice shape, and each macroblock has a size of 16 pixels × 16 pixels when converted to luminance pixels. It is. For example, a frame of full HD size (1920 pixels × 1080 pixels) is divided into horizontal 120 macroblocks and vertical 68 macroblocks, and sequentially processed for each macroblock. Two modes of non-MBAFF and MBAFF are defined for the processing order of macroblocks.

  FIG. 11 shows the processing order of macroblocks when decoding in the non-MBAFF mode. In this mode, the upper line is processed earlier for each line, and the left macroblock is processed earlier in the same line. As shown in FIG. 11, first, macroblocks are processed in the order of progressing one by one in the horizontal direction from the left end of the first row, that is, the leftmost end of the frame. When the processing of the rightmost macroblock in the row is completed, the macroblocks are processed in the order of proceeding one by one in the horizontal direction from the leftmost again with respect to the next lower row.

  FIG. 12 shows the processing order of macroblocks when decoding in the MBAFF mode. In this mode, processing is performed every two adjacent rows. For convenience of explanation, these two lines are referred to as a group below. The upper group is processed first, and within the same group, the macroblock located in the upper row and the micro located on the left are processed earlier. As shown in FIG. 12, first, the leftmost macroblock in the first row is processed, and then the leftmost macroblock in the second row is processed. Then, the processing is performed in the order of the macroblock in the second column in the first row, the macroblock in the second column in the second row, the macroblock in the third column in the first row, and so on.

  FIG. 13 shows the CABAC decoder 30. The CABAC decoder 30 includes an arithmetic decoder 40 and a binary decoder 50. The arithmetic decoder 40 inputs a bit stream, performs arithmetic coding using a context assigned to each syntax element to be decoded, and obtains a binary data string called “bin”. The binary decoder 50 receives the bin string from the arithmetic decoder 40, obtains a syntax element, and outputs it. The syntax element obtained by the binary decoder 50 is output to the video decoder 70 and a part thereof is fed back to the arithmetic decoder 40. This part of syntax elements includes each syntax element for obtaining a motion vector of a macroblock to be decoded later.

  FIG. 14 shows the configuration of the arithmetic decoder 40. The arithmetic decoder 40 includes a syntax element memory 42, an index generator 44, a binary arithmetic decoder 46, and a context memory 48.

  The syntax element memory 42 stores each syntax element of the decoded macroblock fed back from the binary decoder 50.

  The context memory 48 stores contexts assigned to various syntax elements. The context is a probability or an output bit value used in arithmetic coding, and is necessary for decoding by the binary arithmetic decoder 46. The context stored in the context memory 48 is read from the context memory 48 every time one bin is obtained, used and updated by the binary arithmetic decoder 46 to obtain the next bin, and written back to the context memory 48. Made.

  The index generator 44 uses a decoded macroblock syntax element stored in the syntax element memory 42 to use a context index (hereinafter simply referred to as a context index) indicating the address of the context corresponding to the syntax element in the context memory 48. (Also referred to as an index) is generated and provided to the binary arithmetic decoder 46.

  The binary arithmetic decoder 46 reads the context stored at the address indicated by the index generated by the index generator 44 from the context memory 48, performs arithmetic decoding to obtain bin, and uses the context updated at the time of decoding as the context. Write back to the above address in the memory 48.

  H. H.264 specifies that syntax elements of two macroblocks adjacent to a decoding target macroblock among the already decoded macroblocks are used when decoding in the non-MBAFF mode shown in FIG. Yes. For example, when decoding the macroblock “Ck” shown in FIG. 11, the syntax elements of the macroblocks “Bk” and “C (k−1)” are used. In addition, when the macroblock “C (k + 1)” is decoded, the syntax elements of the macroblocks “B (k + 1)” and “Ck” are used. That is, when decoding by non-MBAFF, the syntax element memory 42 needs to store the syntax elements of one row of macroblocks. In the case of the example shown in FIG. 11, the syntax elements of w macroblocks are stored in the syntax element memory 42.

  In the following description, a macroblock to which a syntax element is referenced when a certain macroblock is decoded is referred to as a “reference MB”. A macroblock to be decoded is referred to as a “target macroblock”. For example, in the example illustrated in FIG. 11, for the target macroblock “C (k + 1)”, the macroblocks “B (k + 1)” and “Ck” are reference MBs.

  When decoding in the MBAFF mode shown in FIG. 12, the macroblocks “Ck” and “Dk” are decoded using the macroblocks “Ak”, “Bk”, “C (k−1)”, “D (k−1) ) ”Syntax element is required, and the macroblocks“ C (k + 1) ”,“ B (k + 1) ”,“ Ck ”,“ Ck ”, The “Dk” syntax element is required. That is, when decoding in the MBAFF mode, the syntax element memory 42 needs to store the syntax elements of macroblocks for two rows. In the case of the example illustrated in FIG. 12, “2 × w” macroblock syntax elements are stored in the syntax element memory 42.

  There are many types of syntax elements, and there are many syntax elements per macroblock. Here, attention is paid to a motion vector difference (mvd) which is one type of syntax element.

  In order to derive the motion vector of the macroblock, it is necessary to obtain the motion vector difference mvd of the macroblock. The syntax element representing the motion vector difference mvd is hereinafter referred to as mvd syntax element. The mvd syntax element is divided into mvd “vertical” and mvd “horizontal” representing the horizontal component and vertical component of the motion vector difference mvd, respectively. mvd “vertical” and mvd “horizontal” are further divided into two types, mvd_l0 and mvd_l1. It is divided into. These various mvd syntax elements are assigned to each sub-macroblock obtained by dividing a macroblock. H. In H.264, the maximum number of sub-macroblocks per macroblock is 16 (4 × 4). In the following description, each sub macroblock in the reference MB is referred to as a reference submb.

  The CABAC decoder 30 obtains various mvd syntax elements for each sub macroblock in order to decode the motion vector of the target macroblock. FIG. 15 shows syntax elements required for each reference macroblock when the CABAC decoder 30 decodes the motion vector of the target macroblock. As shown in the figure, for each reference block, in addition to the syntax element of each reference submb, another four syntax elements of the reference block (one mb_skip_flag, one mb_type, and two sub_mb_types) are necessary. The total number of bits of these syntax elements is 239 bits.

  The arithmetic decoder 40 performs decoding according to the magnitude relationship of the values of the three variables codIRANGE (hereinafter referred to as R), codIRrangeLPS (hereinafter referred to as L), and codIOoffset to obtain bin. R is a variable which is given an initial value at the start of CABAC decoding, L is a variable whose value is derived from the context selected from the context memory 48 with a specified expression, and codIOoffset is a variable whose value is given by the bitstream. is there. At the time of decoding, the arithmetic decoder 40 outputs “bin =! ValMPS” when codIOoffset is “RL” or more, and outputs “bin = valMPS” when codIOoffset is smaller than “RL”. The valMPS is a variable derived from the context selected from the context memory 48 and represents 0 or 1. In addition, the arithmetic decoder 40 writes back the values of the variable L and the variable valMPS derived from the context selected from the context memory 48 by a specified expression in the context memory 48 as updated values of the context.

  The processing of the index generator 44 that generates a context index used by the arithmetic decoder 40 to select a context from the context memory 48 will be described with reference to FIGS. 16 and 17. The index generator 44 obtains an index for each mvd syntax element. Hereinafter, mvd_l0 “vertical”, mvd_l1 “vertical”, mvd_10 “horizontal”, and mvd_l1 “horizontal” are represented by “mvd_lx”.

  In FIG. 16, a solid line frame indicates a macro block, and a dotted line frame indicates a sub macro block (sub_mb in the figure) obtained by dividing each macro block into 4 × 4. As described above, when the macroblock C is decoded, the macroblock A and the macroblock B become reference MBs.

  The index generator 44 obtains a context index for each sub-macroblock for the macroblock C. For calculating the context index of one sub macroblock, syntax elements of two reference submbs are used. For example, in the example shown in FIG. 16, when the context index of the sub macroblock c of the target macroblock (macroblock C) is obtained, the submacroblock a of the macroblock A that is the reference MB and the macroblock that is the reference MB B's sub-macroblock b becomes the reference submb, and their mvd syntax elements are used.

  FIG. 17 shows a flowchart when the index generator 44 obtains a context index for mvd_lx of the sub macroblock c. First, an index value A (“absMvdCompA” in FIG. 17) is obtained using a syntax element of one of the two reference submbs (block a in the figure) of the sub macroblock (S10). Next, the index value B is obtained using the syntax element of the other reference submb (block b in the figure) (S20). Then, the context index of mvd_lx of the sub macroblock c is determined by comparing the sum of the index value A and the index value B with a plurality of threshold values (S30).

  In step S30, specifically, the index generator 44 sets “ctxIdxInc = 0”, the index value A and the index value when the sum of the index value A and the index value B is less than the first threshold value (threshold value TH0). “CtxIdxInc = 2” when the sum of B is larger than the second threshold (threshold TH1), and “ctxIdxInc = 1” when the sum of the index value A and the index value B is greater than or equal to the threshold TH0 and less than or equal to the threshold TH1. Determine the value of ctxIdxInc. A different base address is assigned to each mvd_lx, and the sum of the base address of mvd_lx and the ctxIdxInc obtained for the mvd_lx is the context index of the mvd_lx. For convenience of explanation, ctxIdxInc and context index are used in the same meaning hereinafter.

  FIG. 18 shows details of step S10 and step S20 in FIG. In FIG. 18, the index value N (absMvdCompN) corresponds to the index value A and the index value B. In the figure, “block n” indicates a reference submb, and in the figure, “MB N” indicates a macroblock (that is, a reference MB) to which the reference submb belongs. Note that FIG. The process defined in H.264 is shown in a flowchart, and only the outline will be described here, and refer to the standard for details.

  As shown in FIG. 18, first, the processes of steps S100 to S126 are performed. These processes go through a plurality of determination steps to obtain a determination result as to whether or not absMvdCompN is “0” regardless of the value of mvd_lx of the reference submb. In the determination step S126, when the determination result is “Yes”, the index value N (absMvdCompN) of the reference submb is determined to be “0” (S128).

  In the determination step S126, when the determination result is “No”, the process proceeds to step S130. Then, through the processing of steps S130 to S144, the index value N is the absolute value of mvd_lx of the reference submb (S132, S144), twice the absolute value of mvd_lx of the reference submb (S138), and the absolute value of mvd_lx of the reference submb. 1/2 (S142) is determined.

US Pat. No. 7,286,710 US Pat. No. 7,469,070

ITU-T Rec. H. H.264 | ISO / IEC 14496-10

  As described above, there are many types of syntax elements. In order to generate a context index, the syntax element memory 42 stores macroblock syntax elements for one row or two rows in the non-MBAFF mode and the MBAFF mode, respectively. In order to decode the motion vector of the block of interest, as shown in FIG. 15, 239 bits of syntax elements are required for each reference MB. Therefore, in the non-MBAFF mode, “239 × w” bits (w: In the MBAFF mode, a “239 × 2 × w” -bit syntax element is stored in the syntax element memory 42. When the size of one frame of the moving image is 1920 pixels × 1080 pixels, the size of the syntax element of the reference block stored in the syntax element memory 42 is 7170 bytes for decoding the motion vector. In this case, the required capacity of the syntax element memory 42 is large.

  The present invention has been made in view of the above circumstances, and provides a technique capable of reducing the capacity of a syntax element memory that stores a syntax element of a reference macroblock.

  One aspect of the present invention is an image processing apparatus that sequentially encodes or decodes motion vectors for each macroblock by CABAC encoding or CABAC decoding. The image processing apparatus includes a syntax element memory, a context memory, an index generation unit, and a syntax element replacement unit.

  The syntax element memory stores a plurality of syntax elements of the reference MB. The reference MB is a macroblock to which the plurality of syntax elements are referred when the target macroblock is encoded or decoded. The plurality of syntax elements include various mvd syntax elements respectively representing various motion vector differences for each reference submb that is a sub macroblock obtained by dividing the reference MB.

  The context memory stores a context assigned to each mvd syntax element.

  The index generation unit generates a context index indicating an address at which a context corresponding to each mvd syntax element is stored in the context memory for each sub macroblock of the target macroblock. At that time, index values of two reference submbs corresponding to the sub-macroblock are obtained, respectively, and a context index is generated based on the sum of the obtained two index values. Further, the index generation unit is any one of an absolute value of the mvd syntax element of the reference submb stored in the syntax element memory, twice the absolute value, and 1/2 of the absolute value. Are determined as index values of the reference submb.

  When the syntax element replacement unit stores the mvd syntax element of the reference submb in the syntax element memory, does the index value of the reference submb become “0” regardless of the value of the mvd syntax element? In addition to determining whether the index value is “0”, the mvd syntax element is replaced with “0” and stored in the syntax element memory. Otherwise, the mvd syntax element is stored in the syntax element memory without replacement.

  Another aspect of the present invention is also an image processing apparatus that sequentially encodes or decodes a motion vector for each macroblock by CABAC encoding or CABAC decoding. The image processing apparatus includes a syntax element memory, a context memory, an index generation unit, and a syntax element replacement unit.

  The syntax element memory and the context memory are the same as those in the above-described image processing apparatus.

  When generating the context index of each mvd syntax element for each sub-macroblock of the target macroblock, the index generation unit obtains the index values of the two reference submbs corresponding to the sub-macroblock. A context index is generated based on the sum of two index values. At this time, context indexes having different values are generated when the sum of the two index values is less than the first threshold, when it is greater than the second threshold, and at other times. Note that the second threshold value is larger than the first threshold value.

The syntax element replacement unit has a value of the mvd syntax element of the reference submb that is equal to or greater than the third threshold indicated by the expression (1) smaller than the value of the mvd syntax element, and is equal to or greater than P indicated in the expression (1) Replace with an arbitrary value and store in the syntax element memory.
Third threshold ≧ P = (second threshold + 1) × 2 (1)

  Note that a representation in which the apparatus of the above aspect is replaced with a method or system, a program that causes a computer to be executed as a part of the apparatus, and the like are also effective as an aspect of the present invention.

  According to the technique according to the present invention, the capacity of the syntax element memory can be reduced.

It is a flowchart for demonstrating the 1st method concerning the technique of this invention (the 1). It is an equivalent flowchart of the flowchart shown in FIG. It is a flowchart for demonstrating the 2nd method concerning the technique of this invention (the 2). It is a figure for demonstrating the capacity reduction of the syntax element memory by the 1st method concerning the technique of this invention. It is a figure for demonstrating the 3rd method concerning the technique of this invention. It is a figure which shows the CABAC decoder concerning embodiment of this invention. 7 is a flowchart showing processing of a syntax element replacement unit in the CABAC decoder shown in FIG. 6. FIG. 7 is a flowchart showing a process for obtaining an index value N for the index generation unit in the CABAC decoder shown in FIG. 6 to generate a context index. FIG. 7 is a diagram for explaining capacity reduction of the syntax element memory by the CABAC decoder shown in FIG. 6. H. 1 is a diagram illustrating a bitstream decoder compliant with H.264. It is a figure for demonstrating non-MBAFF mode. It is a figure for demonstrating MBAFF mode. It is a figure which shows the CABAC decoder in the bit stream decoder shown in FIG. It is a figure which shows the arithmetic decoder in the CABAC decoder shown in FIG. It is a figure which shows the syntax element required per 1 reference macroblock in order to decode a motion vector. It is a figure for demonstrating calculation of the context index of a mvd syntax element (the 1). It is a figure for demonstrating calculation of the context index of a mvd syntax element (the 2). It is a figure which shows step S10 and S20 of FIG. 17 in detail.

  Embodiments of the present invention will be described below with reference to the drawings. For clarity of explanation, the following description and drawings are omitted and simplified as appropriate. Each element described in the drawings as a functional block for performing various processes can be configured by a CPU, a memory, and other circuits in terms of hardware, and a program loaded in the memory in terms of software. Etc. Note that, in each drawing, the same element is denoted by the same reference numeral, and redundant description is omitted as necessary.

Before describing specific embodiments of the present invention, first, the principle of the technology according to the present invention will be described.
The inventor of the present application The above-mentioned reference macroblock (reference MB) to which a plurality of syntax elements are referred when encoding or decoding a motion vector of a target macroblock when a moving image is encoded or decoded by a method defined in H.264. As a result of diligent research to reduce the capacity of the syntax element memory that stores multiple syntax elements, we have obtained multiple findings and established a method based on these findings. Hereinafter, these findings and methods will be described by taking the case of motion vector decoding as an example.

<First findings>
As described above, in order to obtain the context index of mvd_lx of the macro block of interest, as shown in FIG. 17, first, index values (index value A and index value B) are obtained for each of the two reference submbs. The inventor of the present application analyzed the processing for obtaining the index value N (absMvdCompN: index value A or index value B) shown in FIG.

  As illustrated in FIG. 18, the processes in steps S100 to S126 are processes for determining whether or not the index value N is set to “0” regardless of the value of mvd_lx of the reference submb. When the determination result of step S126 is “Yes”, the index value N is determined to be “0” (S128).

  In the processing from step S100 to S126, a maximum of six determination steps (S100, S102 or S104, S120, S122, S124, S126) are executed. These determination steps are processes in which the determination result can be understood when the reference MB is decoded.

  When the determination result of step S126 is “No”, the index value N is the absolute value of mvd_lx of the reference submb (S132, S144), twice the absolute value (S138), 1/2 (S142) is determined. Therefore, if mvd_lx of the reference submb is “0”, the index value N is determined to be “0” even if the determination result in step S126 is “No”.

  Furthermore, the mvd syntax elements (mb_skip_flag, mb_type, sub_mb_type) used for each determination step included in steps S100 to S126 are not used for each determination skip for determining the index value N later. That is, if the processing up to step S126 is completed before the mvd syntax element of the reference MB is stored in the syntax element memory, mb_skip_flag, mb_type, and sub_mb_type need not be stored in the syntax element memory.

Based on the above knowledge, the inventor of the present application has established a first method capable of reducing the capacity of a syntax element memory that stores a syntax element of a reference MB for decoding a motion vector.
<First method>
In this method, each syntax element obtained by decoding the macro block of interest is output to the subsequent processing (processing by the video decoder), and as each syntax element for decoding the motion vector of the subsequent macro block, Only the mvd syntax element of each sub macroblock is output to the syntax element memory. Further, when the mvd syntax element is output to the syntax element memory, the first mvd replacement process shown in FIG. 1 is performed on the mvd syntax element (mvd_lx).

  FIG. 1 shows a flowchart of the first mvd replacement process. In FIG. 1, “MB_N” is a currently decoded macroblock, and “block n” is a sub-macroblock included in the macroblock. As a result of the processing, the mvd syntax element for which the macroblock index value N is determined to be “0” is replaced with “0” regardless of the value of the mvd syntax element of the macroblock. Output to memory.

  For comparison with the conventional method, an equivalent flowchart of FIG. 1 is shown in FIG. In FIG. 2, in steps S100 to S126 surrounded by a dotted line, “MB N” is a currently decoded macroblock, “block n” is a sub macroblock included in the macroblock, and the processing is performed as block. The processing of each step is the same as the processing of the same number in FIG. 18 except that the processing is performed before n mvd_lx is stored in the syntax element memory. In FIG. 2, there is no processing of step S120 in FIG. This process determines whether the reference macroblock “MB N” has already been decoded and exists in the slice area when the context index is derived after being stored in the syntax element memory. This is because the determination cannot be made before it is stored in the syntax element memory. A slice is a CABAC processing unit in which a screen is divided into a plurality of macroblock groups.

  When the determination result in step S126 is “Yes”, the mvd_lx of the sub macroblock “block n” is replaced with “0” and output to the syntax element memory (S126: Yes, S200, S202). . On the other hand, when the determination result in step S126 is “No”, the mvd_lx is output to the syntax element memory as it is without being replaced with “0” (S126: No, S202).

  Further, in this method, when the context index is obtained for mvd_lx at the time of decoding the motion vector of the macro block of interest, processing is performed according to the procedure shown in the flowchart of FIG. In FIG. 3, “MB N” is a reference MB, and “block n” is a reference submb.

  As shown in FIG. 3, first, in step S120, it is determined whether or not a syntax element of the reference MB exists. If the reference MB has not been decoded yet or exists outside the slice area and the syntax element is not stored in the syntax element memory, the index value N is determined to be “0” ( S120: Yes, S128). These are the same as the processes with the same numbers in FIG.

  In step S120, if the result of the determination is “No”, that is, if the syntax element of the reference MB is already stored in the syntax element memory, the index value N is the absolute value of mvd_lx of the reference submb, and It is determined to be either twice the absolute value or 1/2 of the absolute value. These processes are the same as the processes with the same numbers in FIG.

  That is, in this method, when the mvd syntax element (mvd_lx) of each reference submb of the reference MB is stored in the syntax element memory, the reference value N should be “0” regardless of the value of mvd_lx. Replace mvd_lx of submb with “0”. Therefore, when the index value N is determined for the reference submb in the flowchart shown in FIG. 2 later when the target macroblock is decoded, the absolute value of mvd_lx, twice the absolute value, or 1/2 of the absolute value may be determined. Even so, the index value N is correctly obtained so as to be “0”.

  This eliminates the need to store mb_skip_flag, mb_type, and sub_mb_type in the syntax element memory. FIG. 4 shows mvd syntax elements for each reference MB stored in the syntax element memory in order to decode the motion vector of the macro block of interest when this method is applied. As shown in the figure, by applying this technique, the maximum total number of mvd syntax elements required per reference MB is 224 bits, which is 15 bits less than the conventionally required 239 bits shown in FIG. Can do. Therefore, the capacity of the syntax element memory can be reduced.

  Among the steps of the first mvd replacement process according to the present method, there are processes performed at the time of decoding the macroblock. For these processes, if the result of the process at the time of decoding the macroblock is used, the efficiency can be improved.

<Second finding>
The inventor of this application analyzed the process of step S30 in FIG. 17, that is, the process of calculating the context index using the index value A and the index value B obtained for each of the two reference submbs. As shown in FIG. 17, when the sum of the index value A and the index value B is larger than the threshold value TH1, the context index is determined to be “2”.

That is, if either one of the index value A and the index value B is larger than the threshold value TH1, the context index is determined to be “2”. Further, since the index value is determined to be one of the absolute value of mvd_lx of the reference submb, twice the absolute value, or 1/2 of the absolute value, mvd_lx of one of the two reference submbs is expressed by the formula (1). If the value of mvd_lx is equal to or greater than P shown in the above, the context index is correctly determined even if the value of mvd_lx is smaller than mvd_lx and replaced with any value greater than or equal to P.
Third threshold ≧ P = (second threshold + 1) × 2 (1)

  For example, when the threshold value TH1 is “32”, when the value of mvd_lx of the reference submbb “66” or more is replaced with “66”, the index value is 1/2 of the absolute value of mvd_lx (here, “33”). )), The context index is correctly determined to be “2”.

Therefore, based on this knowledge, the present inventor has established a second method capable of reducing the capacity of the syntax element memory for storing the syntax elements of the reference MB for decoding the motion vector.
<Second method>

  In this method, when the mvd syntax element (mvd_lx) of the reference submb is stored in the syntax element memory, the value of the mvd_lx of the reference submb that is equal to or greater than the third threshold indicated by the expression (1) is set to the value of the mvd_lx. It is smaller and is replaced with an arbitrary value equal to or larger than P shown in the expression (1) and stored in the syntax element memory.

By doing so, since mvd_lx equal to or greater than the third threshold value is replaced with a small value and stored in the syntax element memory, the capacity of the syntax element memory can be reduced.
Note that the smaller the third threshold value and the smaller the value with which mvd_lx is replaced, the greater the effect of reducing the capacity of the syntax element memory. It is preferable to replace them.
The replacement process by the second method described above is hereinafter referred to as “second mvd replacement process”.

<Third findings>
The inventor of the present application has obtained a third finding as a result of searching for research from the viewpoint of reducing the number of bits of the mvd syntax element of the reference submb.

  When the syntax element memory is connected by the system bus, when the mvd syntax element of the reference submb is transferred to the syntax element memory, the number of transfers can be reduced by reducing the number of bits of the mvd syntax element. Can be transferred efficiently. Further, since the data amount of the mvd syntax element is reduced by reducing the number of bits, the capacity of the syntax element memory can be reduced.

  As described above, if either one of the index value A and the index value B is larger than the threshold value TH1, the context index is determined to be “2”. Also, the index value is determined as one of the absolute value of mvd_lx of the reference submb, twice the absolute value, and 1/2 of the absolute value. If it is determined to be 1/2 of the absolute value, the fractional value is rounded down, that is, only the integer part of 1/2 of the absolute value is used.

Therefore, for example, when the threshold value TH1 is 32, two values of mvd syntax elements equal to or greater than “34” that have the same integer part when divided by 2 are the same value (for example, the smaller mvd Even if it is stored in the syntax element memory after being replaced with the syntax element), the context index can be correctly determined when the context index is obtained later using the mvd syntax element of the reference submb. .
Based on this knowledge, the present inventor has established a third method.

<Third method>
In this method, when the mvd_lx of the reference submb is stored in the syntax element memory, the value of mvd_lx that is equal to or greater than the fourth threshold indicated by the expression (2) is encoded by the following encoding rule and stored in the syntax element memory. To do. The index value of the reference submb is calculated after decoding the mvd syntax element of the reference submb stored in the syntax element memory that is equal to or greater than the fourth threshold value based on the encoding rule.
Fourth threshold ≧ Q = second threshold + 2 (2)

  The encoding rule is that “the value after encoding is not less than the value before encoding and not less than the fourth threshold value” and “two values having the same integer part after dividing by 2 are encoded to the same value. And “when the integer part increases by one, the value after encoding also increases by one”.

  The fourth threshold value may be Q or more, but is preferably Q. In this case, it is most effective in reducing the number of bits of the mvd syntax element of the reference submb.

  FIG. 5 shows an example of an encoding table and a decoding table according to the above encoding rule when the fourth threshold value is “60”.

  As can be seen from the encoding table shown in FIG. 5, a value smaller than “60” is directly encoded as a value equal to or greater than “60” to a value equal to or greater than 60 than the original value. Also, two values, for example “62” and “63”, that have the same integer part after dividing by 2, are encoded to the same value (61). Also, “60” and “61” whose integer part is “30” are encoded as “60”, and “62” and “63” whose integer part is “31” are encoded as “61”. Thus, encoding is performed so that the value after encoding is increased by one when the integer part is increased by one. The mvd syntax element of the reference submb is encoded in this way and stored in the syntax element memory.

  When the index value is calculated later using the mvd syntax element of the reference submb, the decoding is performed according to the decoding table shown in FIG.

The right part of FIG. 5 is encoded with the encoding table shown in the left part of FIG. 5, is decoded with the decoding table shown in the lower part of FIG. 5, and then becomes 1/2 of the absolute value of the mvd syntax element of the reference submb. The index value N determined when it is determined is shown. As shown in the figure, even when the mvd syntax element of the reference submbb is encoded by this method and then stored in the syntax element memory, the correct index value N can be obtained later.
In the following description, encoding by the third method is also referred to as “third mvd replacement process”.

  When the index value is calculated using the mvd syntax element of the reference submb stored in the syntax element memory, the value of the mvd syntax element of the reference submb is equal to or greater than the fourth threshold, and the index value Only when it is determined that becomes 1/2 of the absolute value of the mvd syntax element of the reference submb, it is preferable to perform decoding according to the encoding rule for the mvd syntax element of the reference submb. In other cases, the correct context index can be obtained using the mvd syntax element that has been encoded. By doing so, the efficiency of the process of calculating the context index can be improved.

  Each of the above-described methods can be effective in reducing the capacity of the syntax element memory, and by combining these methods, the capacity reduction effect of the syntax element memory can be further increased.

Based on the above principle, a specific embodiment of the present invention will be described.
FIG. 6 shows a CABAC decoder 100 according to an embodiment of the present invention. This CABAC decoder 100 corresponds to the CABAC decoder 30 in the bit stream decoder 10 shown in FIG. The bit stream compliant with H.264 is decoded for each macro block to sequentially obtain the syntax elements of each macro block.

  The CABAC decoder 100 includes an arithmetic decoder 110, a binary decoder 120, and a syntax element replacement unit 150.

  The arithmetic decoder 110 receives the bit stream and performs arithmetic coding using the context to obtain “bin”.

  The binary decoder 120 receives the bin string from the arithmetic decoder 110, obtains a syntax element, and outputs it. The syntax element obtained by the binary decoder 120 is output to a subsequent process such as a video decoder, and a part thereof is output to the syntax element replacement unit 150. This part of syntax elements includes each syntax element for obtaining a motion vector of a macroblock to be decoded later.

  The syntax element replacement unit 150 performs replacement for converting a part of the syntax element for obtaining the motion vector of the macroblock into another value, and outputs the result to the arithmetic decoder 110. The syntax elements that are not replaced are output to the arithmetic decoder 110 as they are.

  The arithmetic decoder 110 includes a syntax element memory 112, an index generation unit 114, a binary arithmetic decoder 116, and a context memory 118.

  The syntax element memory 112 stores the syntax element from the syntax element replacement unit 150.

  The context memory 118 stores contexts assigned to various syntax elements. The context stored in the context memory 118 is read from the context memory 118 every time one bin is obtained, used and updated by the binary arithmetic decoder 116 to obtain the next bin, and written back to the context memory 118. Made.

  For each sub macroblock of the macro block to be decoded (target macro block), the index generating unit 114 indicates a context in which contexts corresponding to various mvd syntax elements of the sub macro block are stored in the context memory 118. An index is generated and provided to the binary arithmetic decoder 116.

  The binary arithmetic decoder 116 reads the context stored at the address indicated by the index generated by the syntax element memory 112 from the context memory 118, performs arithmetic decoding to obtain bin, and updates the context updated at the time of arithmetic decoding. Is written back to the above address in the context memory 118.

  The CABAC decoder 100 includes a normal CABAC except that it includes a syntax element replacement unit 150 and the operation when the index generation unit 114 generates an index is different from the operation of the index generator in the normal CABAC decoder. Same as decoder. Therefore, only the syntax element replacement unit 150 and the index generation unit 114 will be described in detail below.

  FIG. 7 is a flowchart showing the flow of processing by the syntax element replacement unit 150. As illustrated, the syntax element replacement unit 150 first performs a first mvd replacement process on the mvd syntax element from the binary decoder 120 (the mvd syntax element of the sub-macroblock that will later become the reference submb). (S300). This process is a process up to step S200 in the flowchart shown in FIG. By this process, the mvd syntax element determined to have the index value N to be “0” is replaced with “0” regardless of the value of the mvd syntax element of the reference submb. Other mvd syntax elements are not replaced.

  In step S300, the syntax element replacement unit 150 outputs “0” to the syntax element memory 112 as the mvd syntax element for the reference submb that has been replaced with “0” (S320: Yes, S310). .

  On the other hand, in step S300, a second mvd replacement process is further performed on the mvd syntax element of the reference submb that has not been replaced with “0” (S302: No, S304).

In the second mvd replacement process in step S304, the value of the mvd syntax element equal to or greater than the third threshold indicated by the expression (1) is smaller than the value of the mvd syntax element and equal to or greater than P indicated in the expression (1). This is a process of replacing with a value such as P. Here, if the second threshold value is “32” and the third threshold value is P, in this step, mvd syntax elements of 66 or more are all replaced with “66”.
Third threshold ≧ P = (second threshold + 1) × 2 (1)

  It should be noted that the mvd syntax element whose third threshold value is smaller than 66, that is, 65 or less, is the value as it is without being replaced.

Next, the syntax element replacement unit 150 determines whether or not to perform the third mvd replacement process on the mvd syntax element after the second mvd replacement process of step S304 is performed (S306). If the mvd syntax element after the second mvd substitution process is smaller than the fourth threshold value shown in Expression (2) (S306: No), the third mvd substitution process is not performed and the mvd syntax element is used as the syntax. The data is output to the element memory 112 (S306: No, S310).
Fourth threshold ≧ Q = second threshold + 2 (2)

  Assuming that the second threshold is “32” and the fourth threshold is “60”, the mvd syntax element smaller than “60”, that is, “59” or less, is not subjected to the third mvd replacement process. Are output to the syntax element memory 112.

  On the other hand, the syntax element replacement unit 150 further performs a third mvd replacement process for mvd syntax elements that are equal to or greater than the fourth threshold (60 in this case) (S306: No, S308).

  The third mvd replacement process is a process for reducing the number of bits of the mvd syntax element, and is performed, for example, according to the encode table shown in FIG. In the present embodiment, since the third mvd substitution process is performed on the mvd syntax element after the second mvd substitution process, the maximum value before encoding is “66” in the encoding table. . In addition, since encoding is performed only for mvd syntax elements equal to or greater than the fourth threshold, the minimum value before encoding is “60” in the encoding table.

  The syntax element replacement unit 150 outputs the mvd syntax element obtained by performing the third mvd replacement processing to the syntax element memory 112 (S308, S310).

  The index generation unit 114 uses the mvd syntax elements of the two reference submbs stored in the syntax element memory 112 in order to generate the context of the mvd syntax element of the sub macroblock of the target macroblock, respectively. N is obtained. FIG. 8 is a flowchart illustrating a process in which the index generation unit 114 obtains the index value N. This flowchart is the same as the flowchart of FIG. 2 except that steps S330 to S332 are added.

  The index generation unit 114 determines the absolute value of the mvd syntax element of the reference submb, twice the absolute value, or 1/2 of the absolute value as the index value N. When it is determined that the index value N should be determined as the absolute value of the reference submb, the absolute value of the mvd syntax element stored in the syntax element memory 112 is determined as the index value N ((S130: No, S132) (S130: Yes, S134: No, S132) (S130: Yes, S134: Yes, S136: No, S140: No, S144) Also, the index value N is the absolute value of the mvd syntax element of the reference submb. Is determined to be twice the absolute value of the mvd syntax element stored in the syntax element memory 112 as the index value N (S130: Yes, S134: Yes, S136: Yes, S138).

  On the other hand, when the index generation unit 114 determines that the index value N should be determined to be 1/2 of the mvd syntax element of the reference submb (S130: Yes, S134: Yes, S136: No, S140: Yes) Further, it is confirmed whether or not the mvd syntax element of the reference submb stored in the syntax element memory 112 is equal to or greater than a fourth threshold (60 in this case) (S330). If the mvd syntax element is 60 or more, the index generation unit 114 decodes the mvd syntax element with reference to the decoding table shown in FIG. 5, and ½ of the absolute value of the mvd syntax element after decoding. Is determined as the index value N (S330: Yes, S332, S142). On the other hand, if the result of determination in step S330 is that the mvd syntax element is 59 or less, the index generation unit 114 sets 1/2 of the absolute value of the mvd syntax element stored in the syntax element memory 112 as the index value N. (S330: No, S142).

  The CABAC decoder 100 according to the present embodiment is a combination of the three methods described in the explanation of the principle of the present invention. The effects of these methods can be obtained, and the effects can be further increased by the combination. be able to.

  For example, as shown in FIG. 15, conventionally, the mvd syntax element has been expressed by 15 bits or 13 bits. On the other hand, the CABAC decoder 100 of the present embodiment suppresses the value of the mvd syntax element stored in the syntax element memory 112 to “63” or less by combining the second method and the third method. Therefore, the mvd syntax element can be represented by 6 bits. Further, by applying the first method, the syntax element of the reference MB stored in the syntax element memory 112 for decoding the motion vector of the later target macroblock is changed to only the mbd syntax element, and mb_skip_flag, mb_type, , Sub_mb_type need not be stored in the syntax element memory 112.

  Therefore, as shown in FIG. 9, in the CABAC decoder 100 of the present embodiment, the maximum number of mvd syntax elements stored in the syntax element memory 112 for each reference MB in order to decode the motion vector of the macroblock. The total number of bits is suppressed to 96 bits, which is about 60% reduction from the conventional 239 bits, and the capacity of the syntax element memory can be greatly reduced.

  Also, in the case of a system that transfers mvd syntax elements to the syntax element memory via the system bus, the number of transfers can be reduced and transfer efficiency can be improved. For example, when the bit width of the system bus is 128 bits, it is necessary to transfer twice in order to transfer the conventional 239 bits. According to the CABAC decoder 100, the transfer efficiency can be improved by 50% because only one transfer is possible. Similarly, in the case of a 64-bit bus width, two transfers can be reduced, and in the case of a 32-bit bus width, five transfers can be reduced, thereby improving efficiency by 50% and 62.5%, respectively.

  The present invention has been described above based on the embodiment. The embodiment is an exemplification, and various modifications, increases / decreases, and combinations may be made to the above-described embodiments without departing from the gist of the present invention. It will be understood by those skilled in the art that modifications in which these changes, increases / decreases, and combinations are also within the scope of the present invention.

  Moreover, although the embodiment in the case where the technique of the present invention is applied to CABAC decoding has been described, the technique of the present invention can be similarly applied to CABAC encoding. Moreover, by applying, the same effect as that at the time of decoding can be obtained.

10 bitstream decoder 20 entropy decoder 30 CABAC decoder 40 arithmetic decoder 42 syntax element memory 44 index generator 46 binary arithmetic decoder 48 context memory 50 binary decoder 60 CAVLC decoder 70 video decoder 100 CABAC decoder 110 arithmetic decoder 112 syntax element Memory 114 Index generation unit 116 Binary arithmetic decoder 118 Context memory 120 Binary decoder 150 Syntax element replacement unit

Claims (12)

An image processing apparatus that sequentially encodes or decodes a motion vector for each macroblock by CABAC encoding or CABAC decoding,
A plurality of syntax elements of a reference MB that is a macroblock to which a plurality of syntax elements are referred when encoding or decoding a motion vector of the target macroblock, and obtained by dividing the reference MB A syntax element memory for storing the plurality of syntax elements including various mvd syntax elements respectively representing various motion vector differences for each reference submb being a block;
A context memory for storing a context allocated for each mvd syntax element;
For each sub macroblock of the macro block of interest, when the context corresponding to each mvd syntax element generates a context index indicating the address stored in the context memory, the two macro macro blocks corresponding to the sub macro block are generated. An index generation unit that obtains an index value of each reference submb and generates the context index based on a sum of the obtained two index values, and the index subunit of the reference submb stored in the syntax element memory the index generation unit for obtaining any one of an absolute value of an mvd syntax element, twice the absolute value, and ½ of the absolute value as the index value;
When the mvd syntax element of the reference submb is stored in the syntax element memory, it is determined whether the index value of the reference submb is “0” regardless of the value of the mvd syntax element. When it is determined that the index value is “0”, the mvd syntax element is replaced with “0” and stored in the syntax element memory. Otherwise, the mvd syntax element is stored in the syntax element memory. An image processing apparatus comprising: a syntax element replacement unit for storing in the syntax element memory. The index generation unit may be context indexes having different values when the sum of the two index values is less than a first threshold, when the sum is greater than a second threshold greater than the first threshold, and at other times. Respectively,
The syntax element replacement unit further sets the value of the mvd syntax element of the reference submb that is equal to or greater than the third threshold indicated by the formula (1) to be smaller than the value of the mvd syntax element, and sets the formula (1) to The image processing apparatus according to claim 1, wherein the value is stored in the syntax element memory in place of an arbitrary value greater than or equal to P.
Third threshold ≧ P = (second threshold + 1) × 2 (1)   The syntax element replacement unit replaces the value of the mvd syntax element of the reference submb that is equal to or greater than P with P as the third threshold value, and stores it in the syntax element memory. The image processing apparatus according to claim 2. The syntax element replacement unit further encodes a value of the mvd syntax element of the reference submb that is equal to or greater than a fourth threshold indicated by the expression (2) according to a predetermined rule, and stores the encoded value in the syntax element memory.
The index generation unit decodes the mvd syntax element of the reference submb that is stored in the syntax element memory and is equal to or greater than the fourth threshold based on the predetermined rule, and then the index value of the reference submb To calculate
The predetermined rule is that two values whose encoded values are equal to or smaller than the pre-encoded value and equal to or greater than the fourth threshold and whose integer parts after dividing by 2 are the same are encoded into the same value, 4. The image processing apparatus according to claim 2, wherein when the integer part is increased by 1, the value after encoding is also increased by one.
Fourth threshold ≧ Q = second threshold + 2 (2)   The index generation unit determines that the index value of the reference submb stored in the syntax element memory is only when the index value of the reference submb is determined to be ½ of the absolute value of the mvd syntax element of the reference submb. The image processing apparatus according to claim 4, wherein the decoding is performed on the mvd syntax element based on the predetermined rule.   The image processing apparatus according to claim 4, wherein the fourth threshold value is Q shown in Expression (2). An image processing apparatus that sequentially encodes or decodes a motion vector for each macroblock by CABAC encoding or CABAC decoding,
The plurality of syntax elements of a reference MB, which is a macroblock to which a plurality of syntax elements are referred when encoding or decoding a target macroblock, and is a sub macroblock obtained by dividing the reference MB. A syntax element memory for storing the plurality of syntax elements including various mvd syntax elements respectively representing various motion vector differences for each reference submb;
A context memory for storing a context allocated for each mvd syntax element;
For each sub macroblock of the macro block of interest, when the context corresponding to each mvd syntax element generates a context index indicating the address stored in the context memory, the two macro macro blocks corresponding to the sub macro block are generated. An index generation unit that obtains each index value of a reference submb and generates the context index based on the sum of the two obtained index values, wherein the sum of the two index values is less than a first threshold value The index generation unit for generating context indexes of different values when, when greater than a second threshold greater than the first threshold, and at other times;
The value of the mvd syntax element of the reference submb that is equal to or larger than the third threshold indicated by the expression (3) is replaced with an arbitrary value that is smaller than the value of the mvd syntax element and is equal to or larger than P shown in the expression (3). An image processing apparatus comprising: a syntax element replacement unit for storing in the syntax element memory.
Third threshold ≧ P = (second threshold + 1) × 2 (3)   The syntax element replacement unit replaces the value of the mvd syntax element of the reference submb that is equal to or greater than P with P as the third threshold value, and stores it in the syntax element memory. The image processing apparatus according to claim 7. The syntax element replacement unit further encodes a value of the mvd syntax element of the reference submb that is equal to or greater than a fourth threshold indicated by Formula (4) according to a predetermined rule, and stores the encoded value in the syntax element memory.
The index generation unit decodes the mvd syntax element of the reference submb that is stored in the syntax element memory and is equal to or greater than the fourth threshold based on the predetermined rule, and then the index value of the reference submb To calculate
The predetermined rule is that two values whose encoded values are equal to or smaller than the pre-encoded value and equal to or greater than the fourth threshold and whose integer parts after dividing by 2 are the same are encoded into the same value, 9. The image processing apparatus according to claim 7, wherein when the integer part is increased by one, the encoded value is also increased by one.
Fourth threshold ≧ Q = second threshold + 2 (4)   The index generation unit determines that the index value of the reference submb stored in the syntax element memory is only when the index value of the reference submb is determined to be ½ of the absolute value of the mvd syntax element of the reference submb. The image processing apparatus according to claim 9, wherein the decoding is performed on the mvd syntax element based on the predetermined rule.   11. The image processing apparatus according to claim 9, wherein the fourth threshold value is Q shown in Expression (4). The syntax element replacement unit further stores the mvd syntax element of the reference submb in the syntax element memory when the index value of the reference submb is “regardless of the value of the mvd syntax element”. In addition to determining whether or not the index value is “0”, the mvd syntax element is replaced with “0” and stored in the syntax element memory. In the case of storing the mvd syntax element in the syntax element memory,
The index generation unit is one of an absolute value of the mvd syntax element of the reference submb stored in the syntax element memory, twice the absolute value, and 1/2 of the absolute value. The image processing apparatus according to claim 7, wherein one is obtained as the index value.

Images