JP2007312340A - Dynamic image encoding device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a dynamic image encoding device of a macro block pipeline structure wherein a processing result of an adjacent block is reflected in the process of the block being encoded. <P>SOLUTION: At least two rows of macro blocks continuous in the longitudinal direction are united into one set. One macro block is selected from each one row of macro blocks from the set of the macro block rows. The selected macro blocks are input into a macro block pipeline for each one macro block cycle and encoding of the macro blocks is started. For example, macro blocks of row-e column-10, row-f column-8, row-g column-6, row-h column-4 are input into the macro block pipeline for each macro block cycle and encoding is started. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a moving picture coding apparatus, a moving picture coding method, and an arithmetic coding apparatus for converting a fixed length code into a variable length code for compressing and reducing the data amount of the moving picture.

  Some moving images are composed of frames such as 720P and others are composed of fields such as 1080i, but the present invention is composed of fields even when applied to those composed of frames. Even if it is applied to things, the purpose can be achieved, and similar actions and effects are produced. Therefore, in the following, the term “picture” is used to indicate one image constituting a moving image. A picture shall represent both a frame and a field.

  MPEG-2 and H.264 In a moving image coding system such as H.264 (also referred to as MPEG-4 AVC) or VC-1, the screen is divided into macroblocks (hereinafter referred to as MB) of 16 pixels × 16 pixels or 8 pixels × 8 pixels. Motion detection, motion compensation, frequency conversion such as DCT or integer conversion, quantization, variable length coding, and the like are performed in MB units. In these moving image encoding systems, in order to improve the encoding efficiency, the result of the encoded peripheral MB is fed back, and adaptive processing for determining the processing content of the MB being encoded (hereinafter referred to as the current MB). Is done.

  FIG. 46 shows a block diagram of an MPEG-2 encoder. In MPEG-2, in order to prevent the generated code amount from fluctuating locally, the generated code amount is controlled by changing the quantization parameter of each MB. This is called rate control 92. In the rate control 92, the quantization parameter of the current MB is changed according to the code amount of the encoded MB generated as a result of the variable length encoding 91. For example, when the generated code amount of the left adjacent MB is large, the quantization parameter of the current MB is increased and the generated code amount of the current MB is decreased.

  FIG. 2 shows a block diagram of an H.264 encoder. H. H.264 can compress moving images with twice the efficiency of MPEG-2. In order to achieve this high compression efficiency, H.264 employs an increase in the number of reference pictures in motion compensation 100, variable block size motion compensation, and frequency conversion in units of 4 pixels × 4 pixels (hereinafter referred to as integer conversion 93), and in addition to rate control 103, The three adaptive processes are newly introduced. Here, variable block size motion compensation means that MB is divided into sub-blocks (hereinafter referred to as SB) such as 8 pixels × 16 pixels, 8 pixels × 8 pixels, 4 pixels × 4 pixels, etc., and motion compensation is performed for each SB. To do. Hereinafter, when simply referring to a block, both MB and SB are meant.

(1) Considering not only the code amount of a prediction error but also the code amount of a reference picture and the code amount of a motion vector, a combination of a reference picture, a motion vector, and a block that minimizes the total code amount thereof is selected. For the motion vector, a predicted motion vector (hereinafter referred to as PMV) predicted from the motion vectors of the left adjacent block, the upper adjacent block, and the upper right adjacent block is obtained, and the difference between the motion vector of the PMV and the current block is encoded. . In addition, the motion vectors of the left adjacent block, the upper adjacent block, and the upper right adjacent block are also referred to when obtaining the motion vector in the spatial direct mode.

(2) A prediction value is created from the reproduced pixels of the adjacent MB, and intra prediction 97 is performed. As for the luminance signal, there are an intra prediction mode of 4 pixels × 4 pixels SB (hereinafter referred to as 4 × 4 SB) and an intra prediction mode of 16 × 16 pixel blocks. In the 4 × 4SB intra prediction mode of the luminance signal, an intra prediction value of the current 4 × 4SB is generated from the reproduction pixels of the left adjacent 4 × 4SB, the upper adjacent 4 × 4SB, and the upper right adjacent 4 × 4SB, and this intra predicted value And the difference between the current 4 × 4 SB pixels. The difference pixel is subjected to integer transformation / quantization / inverse quantization / inverse integer transformation, and an intra prediction value is added to the result, thereby generating a reproduction pixel. In the 16 × 16 pixel block intra prediction mode, an intra prediction value of the current 16 × 16 pixel block is created from the reproduction pixels of the left adjacent MB and the upper adjacent MB, and the difference between the intra prediction value and the pixel of the current 16 × 16 pixel block is created. Is encoded.

(3) A deblocking filter 98 is applied to the boundary of the 4 pixel × 4 pixel block to remove block noise.

  On the other hand, a hardware moving image encoding apparatus employs a pipeline in MB units (referred to as a macroblock pipeline or an MB pipeline in the claims and the specification) in order to increase the operation rate of a circuit. (For example, see Non-Patent Document 1, pp. 126-128, FIG. 3). Further, in a hardware moving image encoding apparatus, encoding is generally performed from the left end MB toward the right end MB in a horizontal MB array (hereinafter referred to as MB row). When the encoding of the rightmost MB included in one MB row is completed, the MBs belonging to the lower MB row are encoded from the left end MB toward the right end MB. This is repeated from the top MB row of the picture to the bottom MB row of the picture. FIG. In the MB pipeline of the H.264 encoder, the manner in which the MB flows through each stage of the MB pipeline is shown. n-1, n, and n + 1 represent the (n-1) th, nth, and n + 1th MBs from the left in a certain MB row, respectively. The MB cycle is a unit of time required for processing one MB in each stage of the MB pipeline. In each stage, one MB is processed in one MB cycle.

  In FIG. 48, data necessary for encoding one MB is read out from the external memory over 1 MB cycle in stage 0. Stage 1 performs coarse motion detection for one MB over 1 MB cycle. Detailed motion detection is performed in a narrow range around the coarse motion candidate motion vector detected in stage 1 over 1 MB cycle in stage 2 and intra prediction is performed. In stage 3, integer conversion, quantization, inverse quantization, and inverse integer conversion are performed on one MB over 1 MB cycle. In stage 4, variable length coding and motion compensation are performed for one MB over one MB cycle. Then, stage 1 performs rate control and deblocking filtering on one MB over 1 MB cycle, and stage 6 writes a reproduction pixel or bit stream for one MB over 1 MB cycle to the external memory. Assuming that the MB cycle is 1000 cycles, the encoding process of one MB takes 7000 cycles when the MB pipeline is not performed, whereas the MB pipeline in FIG. 48 has one MB code every 1000 cycles. The conversion ends.

  However, in order to perform adaptive processing according to the processing result of the left adjacent MB in this MB pipeline, there are the following four problems.

(1) As shown in FIG. 48, when the rate control is finished for the (n-1) th MB, the quantization has already been finished up to the (n + 1) th MB. Therefore, the quantization parameter can be changed according to the generated code amount of the (n−1) th MB for the (n + 2) th and subsequent MBs. That is, 2 MB rate control is delayed.

(2) In FIG. 48, motion detection is executed in two stages of the MB pipeline, and intra prediction is executed simultaneously with high-precision motion vector detection in stage 2. An accurate PMV cannot be calculated unless a high-precision motion vector is determined and intra prediction or inter prediction is determined, but a high-precision motion vector and intra prediction or inter prediction are determined for the (n-1) th MB. Sometimes the motion vector detection with coarse accuracy in the stage 1 is finished for the nth MB. For this reason, in the MB pipeline of FIG. 48, a motion vector cannot be selected considering only a rough accuracy motion vector detection result for the left adjacent MB, and a highly accurate motion vector and intra prediction / A motion vector cannot be selected in consideration of the inter prediction determination result. In addition, H.C. In H.264, it is desirable to perform motion vector detection in consideration of the code amount of the motion vector even in the SB of 8 pixels × 16 pixels, 8 pixels × 8 pixels, 4 pixels × 4 pixels, and the like obtained by dividing the MB. In a pipeline, it is difficult to perform motion vector detection in consideration of the code amounts of the motion vectors of the left adjacent SB, the upper adjacent SB, and the upper right adjacent SB.

(3) The 16 pixel × 16 pixel block intra prediction mode is determined by evaluating the pixel value of the left adjacent 16 pixel × 16 pixel block and the upper adjacent 16 pixel × 16 pixel block and the pixel value of the current block. Here, it is desirable that the pixels of the adjacent 16 pixel × 16 pixel block used for evaluation are reproduction pixels. The reproduced pixel is obtained by performing integer conversion, quantization, inverse quantization, and inverse integer conversion on a prediction error obtained by intra prediction or inter prediction, and adding a prediction value. However, in the MB pipeline of FIG. 48, when the integer conversion of the (n−1) th MB is completed in stage 3 and the reproduction pixel of the (n−1) th MB is created by the motion compensation in stage 4, the n in the stage 2 The intra prediction of the second MB has been completed. For this reason, when the prediction mode determination of 16 pixel × 16 pixel block intra prediction is performed for the nth MB, the reproduction pixel of the (n−1) th MB adjacent to the left cannot be used. For this reason, in the prediction mode determination of the (n−1) th 16-pixel × 16-pixel block adjacent to the left, the pixel of the current picture being encoded must be used instead of the reproduction pixel. Similarly, H.M. In the 4 pixel × 4 pixel block intra prediction mode in H.264, it is desirable to determine the prediction mode using the reproduction pixels of the 4 pixels × 4 pixel block of the left adjacent, upper adjacent, and upper right adjacent, but the MB pipeline of FIG. In this case, it is impossible to use the pixels of the current picture being encoded as a suboptimal measure. For this reason, the intra prediction mode determined by the MB pipeline in FIG. 48 is suboptimal.

(4) Since the same intra prediction value must be used in the encoding device and the decoding device, it must be generated from the reproduced pixels. On the other hand, in the P picture and the B picture, when the n−1th MB adjacent to the left is determined to be in the inter prediction mode, a reproduced pixel cannot be obtained until the motion compensation is completed. However, in the MB pipeline of FIG. 48, the generation of the reproduction pixel in the stage 4 for the (n-1) th MB, the integer conversion in the stage 3 for the nth MB, and the like are executed simultaneously. For this reason, in the P picture and the B picture, it is difficult to create a prediction value using the reproduction pixel of the (n-1) th MB and perform integer conversion of the nth MB.

  Also, since the amount of picture data is enormous, the picture is generally stored in a memory external to the moving picture coding apparatus. FIG. 49 is a block diagram showing a configuration of a conventional motion vector detector 106 (see, for example, p2009 of Non-Patent Document 2 and FIG. 3). An internal memory 104 and a motion detection circuit 105 are included. In the conventional motion vector detector 106, a region where a motion vector may exist (hereinafter referred to as a reference region) is copied from the external memory 4 of the motion vector detector 106 to the internal memory 104 of the motion vector detector 106, An operation for detecting a motion vector is performed on the pixels in the internal memory 104. FIG. 50 shows an example of reference areas 109 and 110 of two adjacent MBs 107 and 108. The reference area 110 of the MB 108 substantially overlaps the reference area 109 of the MB 107, and the width of the non-overlapping area matches the number of pixels in the horizontal direction of the MB. The reference area 110 is only shifted from the reference area 109 to the right by the number of pixels in the horizontal direction of MB. Hereinafter, this shifted area is referred to as an update area 111. The conventional motion vector detector 106 uses this property to reduce the pixel transfer amount between the external memory 4 and the internal memory 104 of the motion vector detector 106. When only the update area 111 is transferred from the external memory 4 to the internal memory 104 and motion detection of the MB 108 is started, the origin of the reference area is changed from (0, 0) of the reference area 109 to (0, 0) ′ of the reference area 110. Replace with. As described above, the transfer amount from the external memory 4 to the internal memory 104 is reduced by reading only the update area 111 and updating the reference area. That is, conventionally, the amount of pixel transfer between the external memory 4 and the internal memory 104 has been reduced by utilizing the horizontal overlap of the reference areas. As a result, the conventional one-chip MPEG-2 codec LSI can encode a 1080i moving image by attaching two 32-bit DDR-SDRAMs operating at 200 MHz (for example, (See Non-Patent Document 4, pp. 12-13).

  However, the size of one picture according to the digital cinema standard defined by DCI (Digital Cinema Initiative) is 4096 pixels × 2160 pixels, which is four times as many as 1080i (1920 pixels × 1080 pixels). In addition, a CMOS sensor that outputs a picture having a number of pixels exceeding three times that of 1080i at a rate twice as high as that of 1080i (60 pictures / second) has been developed (see, for example, section 27.1 of Non-Patent Document 3). If the latest technology is used, it is predicted that a video camera that outputs a moving image of a digital cinema class can be realized. If the conventional technology is used, is a moving image encoding LSI provided with a 256-bit width input terminal? There is a problem that it is necessary to operate the external memory at a speed of about 800 MHz.

In addition, when the processing speed of the video encoding device is slow and the processing cannot be performed in time with one unit, parallel processing by a plurality of video encoding devices is performed. As the picture dividing method in this case, four types shown in FIG. 51 can be considered (for example, p.17 of Non-Patent Document 5, FIG.
1). That is, (a) horizontal division, (b) vertical division, (c) vertical and horizontal division, and (d) slice interleaving. Conventionally, (a) horizontal division has often been adopted because connection between a plurality of image encoding devices and data transfer are simplified (for example, see Non-Patent Document 4). However, the method of performing horizontal division and performing parallel processing with a plurality of moving image encoding apparatuses has a problem that the start of encoding is delayed until all pixels for one picture are read from the camera. The occurrence of this delay is undesirable in communication applications that require low delay such as videophones and remote monitoring.

H. In H.264, variable length coding can be performed by CABAC, which is a kind of arithmetic coding. FIG. 52 shows a block diagram of a CABAC encoder. In CABAC, first, multi-value syntax data (referred to as syntax data in H.264 but referred to as syntax data in this specification and claims) is converted to variable length 2 by a binarization circuit 112. Convert to a valued symbol. Next, the context calculation circuit 113 performs binarization symbol 1
Context calculation is performed for each bit (hereinafter referred to as symbol). Then, the arithmetic coding probability table 115 is selected according to the calculated context information, the symbol is arithmetically encoded by the arithmetic encoding circuit 114, the probability table 115 is updated, and the arithmetic symbol of the next symbol is calculated. The updated probability table 115 is used for conversion.

  FIG. 53 is a Biari_encode_symbol processing flow showing the processing contents of the arithmetic encoding circuit 114 (see, for example, Non-Patent Document 7). In CABAC, the coding state is represented by the width (hereinafter referred to as “range”) and the lower end (hereinafter referred to as “low”) of the existence sequence of the symbol sequence. The bit widths of range and low are 9 bits and 10 bits, respectively, and at the head of the slice, range is initialized to 0x1FE (0x indicates a hexadecimal number, hereinafter the same), and low is initialized to 0x000. Of 0 and 1, the lower occurrence probability is assigned to LPS (Least Probable Symbol), and the higher occurrence probability is assigned to MPS (Most Probable Symbol). The probability table 115 holds the occurrence probability of symbol for each context information. Each occurrence probability is indicated by information indicating whether 0 or 1 is assigned to the MPS and a number (hereinafter referred to as “state”) indicating an LPS occurrence probability (hereinafter referred to as “rLPS”). And range are used to obtain a table (rLPS_table_64x4) (step S30). On the other hand, the occurrence probability of MPS is calculated by subtracting rLPS from range (step S31).

  FIG. 54A shows changes in range and low when symbol is LPS. When symbol is LPS, low is updated by adding the occurrence probability of MPS, and rLPS is set in range (step S33). If state = 0, the occurrence probability of 0 and 1 in the next coding state is expected to be reversed, so 0 and 1 corresponding to MPS are switched (steps S34 and S35). Then, the state of the state is updated by drawing the transition table (AC_next_state_LPS_64) corresponding to the LPS (step S36).

  FIG. 54B shows changes in range and low when symbol is MPS. When symbol is MPS, low does not change, and MPS occurrence probability is set in range (step S31). Then, the state of the state is updated by drawing the transition table (AC_next_state_MPS_64) corresponding to the MPS (step S37).

  When range <0x100, renormalization is performed, and range and low are expanded (step S38). Due to renormalization, the range of range becomes 0x100 ≦ range ≦ 0x1FE.

  In this way, arithmetic coding must be performed for each symbol, and only one probability table 115 can be used in one slice despite the number of processing cycles. For this reason, the arithmetic encoder 114 cannot be parallelized in principle.

  Furthermore, a larger amount of code is allocated to the I picture than to the P picture and the B picture. For this reason, the amount of occurrence of binarized symbols in the I picture is larger than the average amount of binarized symbols in the entire picture. In addition, when the generated code amount suddenly increases, the generated code amount for each MB needs to be reflected in the rate control at an early stage in order to prevent the rate control from failing. In this way, the arithmetic coding circuit 114 is able to reduce the amount of binarized symbols generated suddenly in an I picture that cannot be parallelized and has a large amount of generated codes, despite the number of processing cycles. There is a problem that performance sufficient for processing is required.

Note that there has been proposed a method for estimating the generated code amount of arithmetic coding from the sum of the symbol lengths of binarized symbols without performing actual arithmetic coding for CABAC (for example, p. 20 of Non-Patent Document 6). FIG. 6). Specifically, binarized symbol length N
The generated code amount R = α × N is estimated using the ratio α between the binary symbol length of the immediately previous encoded picture and the generated code amount. Since it is possible to convert multi-level syntax data into binarized symbols with 1 CK using a lookup table, this method can be used to estimate the amount of code generated by arithmetic coding at 1 CK per syntax data. It can be carried out.
Hiroaki Watanabe, Hideaki Chaki, “Developed codec core for mobile devices compatible with both H.264 and MPEG-4, encoded VGA video with 36 mA”, Nikkei Electronics, September 27, 2004, no.883, pp123-133. Toshihiro Minami and Jiro Naganuma, “Proposal of Motion Vector Detector Configuration Suitable for Telescopic Search”, Theory of Science (D-2), vol.J87-D-2, no.11, pp.2007-2024, Nov .2004. S. Yoshihara, M. Kikuchi, Y. Ito, Y. Inada, S. Kuramochi, H. Wakabayashi, M. Okano, K. Koseki, H. Kuriyama, J. Inutsuka, A. Tajima, T. Nakajima, Y. Kudoh, F. Koga, Y. Kasagi, S. Watanabe, T. Nomoto, “A 1 / 1.8-inch 6.4MPixel 60frames / s CMOS Image Sensor with Seamless ModeChange”, ISSCC2006, session 27.1, Feb. 2006. Jiro Naganuma, Hiroe Iwasaki, Takahiro Nitta, Ken Nakamura, Ken Yoshidome, Mitsuo Ogura, Yasuyuki Nakajima, Yutaka Tashiro, Takayuki Onishi, Mitsuru Ikeda, Makoto Endo, Yoshiyuki Yashima, “One-chip HDTV MPEG-2 CODEC LSI Configuration Technology— VASA ", NTT Technical Journal, vol.15, No.9, pp.12-15, September 2003. Satoshi Nogaki, “Examination of coding control method in MPEG-2 MP @ HL encoder”, PCSJ98, P-2.04, pp.17-18, October 1998. Ikuo Yamakage, Takeshi Nakajo, Shinichiro Furuto, “Video coding technology for HD DVD”, Toshiba Review, vol.60, No.1, pp.17-29, 2005. H.264 / AVC reference software (JM 10.2).

  In view of the above circumstances, an object of the present invention is to enable feedback of the processing result of an adjacent block to the processing of a block being encoded in a moving picture encoding apparatus having an MB pipeline structure. Specifically, rate control of the current MB according to the generated code amount of the left adjacent MB, detection of the motion vector of the block being encoded in consideration of the code amount of the motion vector of the adjacent block, left adjacent Detecting the motion vector of the block being encoded in consideration of the MB intra prediction / inter prediction determination result, determining the intra prediction mode using the reproduced pixels of the adjacent block, and left P picture and B picture An object of the present invention is to perform intra prediction by creating a prediction value using a reproduction pixel of an adjacent MB.

  It is another object of the present invention to provide a moving picture coding apparatus that can reduce the amount of data transferred from an external memory that stores pictures to the internal memory of the moving picture coding apparatus.

  It is another object of the present invention to provide a parallel image encoding apparatus that requires a small delay for encoding.

  It is another object of the present invention to enable arithmetic coding even when the arithmetic coding circuit does not have a performance sufficient to handle a binarized symbol generation amount that suddenly increases.

  The moving picture coding apparatus according to the first aspect of the present invention is the first means for dividing the moving picture coding process into a plurality of stages and executing it in a pipeline, and m that is vertically continuous (m is an integer of 2 or more). A macroblock row, and for a macroblock belonging to the uppermost macroblock row in the group of macroblock rows, a macroblock for which the lower left adjacent macroblock has not yet been selected is selected; and For macroblocks belonging to macroblock rows other than the topmost in the group of macroblock rows, by selecting a macroblock in which the upper right adjacent macroblock has already been selected, the group of macroblock rows is changed to the group of macroblock rows. A second means for selecting one included macroblock per macroblock row; and the second means. For each selected macroblock, the macroblock belonging to the uppermost macroblock row in the group of macroblock rows to the macroblock belonging to the lowermost macroblock row takes m macroblock cycles. And, in turn, third means for setting in the first means the data necessary to encode individual macroblocks per macroblock cycle, and the third means When data necessary for encoding a macroblock belonging to the lowest macroblock row in the macroblock row is set in the first means, the second means sends the selected individual For each macroblock, the right adjacent macroblock is newly selected.

  A moving picture coding apparatus according to a second aspect of the present invention is a moving picture coding apparatus that combines two vertically adjacent macro blocks into a macro block pair, and switches between field coding and frame coding for each macro block pair. In this case, the first means for dividing the moving picture coding process into a plurality of stages and executing it in a pipeline, and 2n (n is an integer of 2 or more) continuous macroblock rows are grouped. And for a macroblock pair belonging to the top two macroblock rows in the group of macroblock rows, select a macroblock pair for which the lower left adjacent macroblock pair has not yet been selected, and A macroblock pair belonging to a macroblock row other than the uppermost two macroblock rows in the group of macroblock rows; A second means for selecting one macroblock pair included in the group of macroblock rows per two macroblock rows by selecting a macroblock pair whose upper right adjacent macroblock pair has already been selected; When field coding is selected for the macroblock pair selected by the second means, a top field macroblock is formed by an odd line of the macroblock pair, and a bottom field macro is formed by an even line of the macroblock pair. When a frame is selected, on the other hand, when the frame coding is selected, an upper frame macroblock is formed by the upper half line of the macroblock pair, and a lower frame macroblock is formed by the lower half line of the macroblock pair. According to the third means and the second means When field encoding is selected for the selected macroblock pair, first, from the macroblock pairs belonging to the top two macroblock rows in the group of macroblock rows, the Macros included in each macroblock pair in each macroblock cycle in order over n macroblock cycles up to the macroblock pair belonging to the two lowest macroblock rows in a group of macroblock rows Data necessary for encoding one of the blocks is set in the first means, and the second time, the macroblocks belonging to the two highest macroblock rows in the group of macroblock rows From the pair, the two bottommost macros in the group of macroblock rows The other one not set first among the macroblocks included in each macroblock pair is encoded in order of n macroblock cycles until the macroblock pair belonging to the blackblock row in order of n macroblock cycles. When the frame encoding is selected, the first two macroblock rows located at the top of the group of macroblock rows are set in the first means. From a macroblock pair belonging to a macroblock pair belonging to the lowest two macroblock rows in the group of macroblock rows to each macroblock cycle in order over n macroblock cycles. To encode the upper frame macroblock included in the macroblock pair of The necessary data is set in the first means, and from the macroblock pair belonging to the top two macroblock rows in the batch of macroblock rows, the batch of macroblocks is set a second time. The bottom frame macroblock included in each macroblock pair is encoded every one macroblock cycle in order over n macroblock cycles up to the macroblock pair belonging to the two lowest macroblock rows in the row And a fourth means for setting the data necessary for conversion into the first means, and the fourth means sets the lowest position in the group of macroblock rows by the fourth means. The data necessary for encoding the macroblocks included in the macroblock pairs belonging to the two macroblock rows If set to a first means, said second means is characterized by newly selecting the right neighboring macroblock pair for each individual macroblock pair being the selected.

  According to a third aspect of the present invention, there is provided the moving image encoding apparatus according to the first or second aspect, wherein the macroblock cycle is divided into sub-block cycles at some stages of the first means, The processing is performed for each sub-block.

  A parallel moving image encoding apparatus according to a fourth aspect of the present invention is a parallel moving image code including a number of moving image encoding apparatuses according to any one of claims 1 to 3 (where a is an integer of 2 or more). A picture is divided for each group of macroblock rows, and one moving picture coding apparatus is assigned to each divided divided area, and a vertically divided divided area includes Coding is performed in parallel by a number of moving image coding apparatuses.

  The moving picture encoding apparatus according to a fifth aspect of the present invention is the video encoding apparatus according to any one of the first to fourth aspects, wherein the binarizing means converts the syntax data into a binarized symbol, and the binarization. Binarized symbol storing means for storing the binarized symbol converted by the means, context information restoring means for restoring the context information corresponding to the binarized symbol read from the binarized symbol storing means, and the 2 Arithmetic coding means for arithmetically coding the binarized symbol read from the binarized symbol storage means based on the context information restored by the context information restoration means.

  A moving picture encoding apparatus according to a sixth aspect of the present invention is the provisional arithmetic encoding means according to claim 5, wherein the binary symbols converted by the binarizing means are provisionally arithmetically encoded to calculate a provisionally generated code amount. It is characterized by having.

  According to a seventh aspect of the present invention, there is provided the moving image encoding apparatus according to any one of the first to fourth aspects, wherein the syntax data compressing unit reversibly compresses the syntax data and the syntax data compressing unit. Compression syntax data storage means for storing compressed syntax data, syntax data restoration means for reading the compressed syntax data from the compression syntax data storage means and restoring the syntax data, and the syntax Binarizing means for converting the syntax data restored by the data restoring means into binarized symbols; and arithmetic coding means for arithmetically encoding the binarized symbols converted by the binarizing means. It is characterized by.

  According to an eighth aspect of the present invention, there is provided the moving image encoding apparatus according to any one of the first to fourth aspects, wherein the binarization means converts the syntax data into a binarized symbol, and the binarization. Generated by provisional arithmetic coding means for calculating a provisional generated code amount by provisional arithmetic coding of the binarized symbols converted by the means, CAVLC coding means for CAVLC coding the syntax data, and the CAVLC coding means CAVLC encoded bitstream storage means for storing the generated CAVLC encoded bitstream, CAVLC decoding means for reading the CAVLC encoded bitstream from the CAVLC encoded bitstream storage means and decoding the syntax data, and the CAVLC decoding The syntax data decoded by the means is converted into a binary symbol And second binarization means, and having a an arithmetic encoding means for arithmetic encoding of binary symbols converted by the second binarizing means.

  A moving picture coding method according to a ninth aspect of the present invention is a moving picture coding apparatus having coding means for performing moving picture coding processing in a plurality of stages and executing it in a pipeline manner. Is a group of macroblock rows of 2 or more), and for the macroblocks belonging to the uppermost macroblock row in the group of macroblock rows, the macroblock whose lower left adjacent macroblock has not yet been selected And for the macroblocks belonging to the macroblock row other than the topmost macroblock row in the group of macroblock rows, the macroblock whose upper right adjacent macroblock has already been selected is selected. Select one macroblock per macroblock row in a group of macroblock rows And the macros belonging to the lowermost macroblock row from the macroblocks belonging to the uppermost macroblock row in the group of macroblock rows for the individual macroblocks selected by the first step A second procedure for setting data necessary for encoding individual macroblocks for each macroblock cycle in the encoding means in order over m macroblock cycles until the block, When the data necessary for encoding the macroblock belonging to the lowest macroblock row in the group of macroblock rows is set in the encoding means by the procedure of 2, the first procedure is followed. Returning, the right adjacent macroblock is newly selected for each of the selected macroblocks. To.

  A moving picture coding method according to a tenth aspect of the present invention has coding means for dividing a moving picture coding process into a plurality of stages and executing it in a pipeline, and combining two macro blocks vertically adjacent to each other to create a macro In a moving image encoding apparatus that performs block encoding and switching between field encoding and frame encoding for each macroblock pair, 2n (n is an integer of 2 or more) continuous macroblock rows are grouped. , For a macroblock pair belonging to the top two macroblock rows in the group of macroblock rows, select a macroblock pair for which the lower left neighboring macroblock pair has not yet been selected, and A macroblock belonging to a macroblock line other than the topmost one in a group of macroblock lines For a pair, a first macroblock pair included in the group of macroblock rows is selected per two macroblock rows by selecting a macroblock pair whose upper right adjacent macroblock pair has already been selected. When field encoding is selected for the macroblock pair selected by the procedure and the first procedure, the top field macroblock is configured by the odd line of the macroblock pair and the bottom line by the even line of the macroblock pair. When a field macroblock is configured and frame encoding is selected, the upper frame macroblock is configured with the upper half line of the macroblock pair and the lower frame macroblock is configured with the lower half line of the macroblock pair. Second procedure to configure and the first procedure Thus, when field encoding is selected for the selected macroblock pair, the macroblock pair belonging to the two most macroblock rows in the group of macroblock rows is One of the macroblocks included in each macroblock pair in each macroblock cycle in order over n macroblock cycles up to the macroblock pair belonging to the two lowest macroblock rows in the macroblock row On the other hand, when the frame encoding is selected, the two macroblock rows located at the top of the group of macroblock rows are set in the encoding means. In the group of macroblock rows from the macroblock pairs belonging to In order to encode the upper frame macroblock included in each macroblock pair every n macroblock cycles in order over n macroblock cycles up to the macroblock pair belonging to the two lowest macroblock rows When the field encoding is selected for the macro block pair selected by the first procedure and the macro block pair selected by the first procedure, the data required for the group of macro block rows is set. N macroblock cycles from a macroblock pair belonging to the top two macroblock rows to a macroblock pair belonging to the bottommost two macroblock rows in the group of macroblock rows In turn, each macroblock every macroblock cycle Data necessary for encoding the other one not set by the third procedure among the macroblocks included in the block pair is set in the encoding means, while when frame encoding is selected , Belonging to the two lowest macroblock rows in the group of macroblock rows from the macroblock pairs belonging to the two uppermost macroblock rows in the group of macroblock rows Up to n macroblock cycles up to a macroblock pair, data necessary for encoding the lower frame macroblock included in each macroblock pair is set in the encoding means for each macroblock cycle. 4 in the group of macroblock rows by the fourth procedure. When data necessary for encoding a macroblock included in a macroblock pair belonging to two macroblock rows located below is set in the encoding means, the process returns to the first procedure, and the selection is performed. For each individual macroblock pair, the right adjacent macroblock pair is newly selected.

  According to the first invention, m MB rows (m is an integer of 2 or more) that are vertically continuous are taken as a group, and one MB is selected per MB row from the group of MB rows, Since these selected m MBs are continuously input into one MB pipeline, the processing result of the left adjacent MB can be fed back to the processing of the (m−1) stage fractional MB. In addition, since the reference areas of the selected m MBs are largely overlapped, if the reference area of these MBs is stored in the common reference area memory, the external memory storing the pictures There is no need to duplicate transfer for updating the overlapped portion. For this reason, by providing the reference area memory inside the moving picture coding apparatus, the amount of data transferred from the external memory storing the picture to the internal memory of the moving picture coding apparatus can be reduced.

  According to the second aspect of the present invention, 2n (n is an integer of 2 or more) MB rows that are vertically continuous are taken as a group, and one MB pair per two MB rows that are vertically continuous from the batch of MB rows. First, one MB included in the selected n MB pairs is continuously input to one MB pipeline, and the second time, the selected n MB pairs are transferred to the selected n MB pairs. Since the remaining one of the included MBs is continuously input to one MB pipeline, the processing result of the first MB to be input can be fed back to the processing of the remaining one MB for (n-1) stages. In addition, the processing result of the left adjacent MB pair can be fed back to the processing of the (n−1) stage partial MB pair. Further, since the reference areas of the selected 2n MBs are largely overlapped with each other, if the reference areas of these MBs are stored in a common reference area memory, the external memory storing the pictures This eliminates the need for duplicate transfer for updating the overlapped portion. For this reason, by providing the reference area memory inside the moving picture coding apparatus, the amount of data transferred from the external memory storing the picture to the internal memory of the moving picture coding apparatus can be reduced.

  According to the third invention, processing for each sub-block can be performed in a pipeline manner at some stages of the macroblock pipeline. According to the present invention, the processing result of the adjacent subblock can be fed back to the processing of the subblock being encoded.

  According to the fourth aspect of the invention, since the reference areas of the a moving image encoders operating in parallel overlap each other vertically, all the moving image encodings are performed from the external memory to the internal memory of the parallel moving image encoder. If the configuration is such that the reference area of the device is transferred and transferred from the internal memory to each moving picture encoding device, it is not necessary to transfer overlapping portions of the reference area in duplicate. For this reason, by providing an internal memory in the parallel video encoding device, it is possible to reduce the amount of data transferred from the external memory storing pictures to the internal memory of the parallel video encoding device. Further, unlike the conventional slice interleaving, it is not necessary to divide the slices for each area to be encoded by each moving picture coding apparatus, and the entire picture can be made into one slice. Furthermore, when the first region of a picture to be processed by the a number of moving image encoding devices is input, the moving image encoding device can start encoding, and thus encoding delay can be suppressed.

  According to a fifth aspect of the present invention, there is provided a video encoding apparatus to which the first to fourth aspects of the invention are applied. Even when arithmetic coding such as CABAC defined in H.264 is performed, rate control can be performed before the coding of all MBs included in a group of MB rows that are vertically continuous is completed. For this reason, when the generated code amount of a certain MB suddenly increases, the quantization parameter of the MB to be encoded next is quickly increased, and the code amount generated from the newly encoded MB is decreased. Rate control is possible. In addition, arithmetic coding can be performed even if the arithmetic coding circuit does not have a performance sufficient to process a binarized symbol generation amount that suddenly increases. In particular, if the binarized symbol storage circuit has sufficient storage capacity to store the binarized symbols of the I picture and the binarized symbols of the P picture and B picture before and after the I symbol, the performance of the arithmetic coding circuit can be improved. There is an effect that arithmetic coding can be performed even if the performance of the binarized symbol generation amount suddenly generated in the I picture is less than the capability of processing.

  According to the sixth invention, in addition to the same effect as the fifth invention, it is possible to perform rate control close to rate control based on the generated code amount by true arithmetic coding even when there is a scene change or the like. There is an effect that can be done.

  According to the seventh invention, as in the fifth invention, in the moving picture coding apparatus to which the first to fourth inventions are applied, the Even when arithmetic coding such as CABAC defined in H.264 is performed, rate control can be performed before the coding of all MBs included in a group of MB rows that are vertically continuous is completed. For this reason, when the generated code amount of a certain MB suddenly increases, the quantization parameter of the MB to be encoded next is quickly increased, and the code amount generated from the newly encoded MB is decreased. Rate control is possible. In addition, arithmetic coding can be performed even if the arithmetic coding circuit does not have a performance sufficient to process a binarized symbol generation amount that suddenly increases. In particular, if the binarized symbol storage circuit has sufficient storage capacity to store the binarized symbols of the I picture and the binarized symbols of the P picture and B picture before and after the I symbol, the performance of the arithmetic coding circuit can be improved. There is an effect that arithmetic coding can be performed even if the performance of the binarized symbol generation amount suddenly generated in the I picture is less than the capability of processing.

  According to the eighth invention, in addition to the same effects as in the seventh invention, the amount of binarized symbols generated in a certain MB suddenly increases, and as a result of provisional arithmetic coding, a true arithmetic coding circuit When it is predicted that the processing capacity of the MB will be exceeded, the entire picture including the MB is CAVLC encoded. On the other hand, when it is predicted that it can be processed by a true arithmetic encoding circuit as a result of provisional arithmetic encoding, the entire picture including the MB is encoded. CABAC encoding can be performed. As a result, it is possible to reliably avoid the situation where the performance of the true arithmetic coding circuit is unexpected. Furthermore, the CAVLC encoding circuit calculates the code amount of the CAVLC encoded bitstream, compares the temporarily generated code amount of CAVLC encoding and provisional arithmetic encoding, and entropy that is estimated to be small for each picture. An encoding mode can also be selected.

  According to the ninth aspect of the invention, as in the first aspect, m (m is an integer of 2 or more) MB rows that are vertically continuous are taken as a group, and one MB row is selected from the set of MB rows. Since the selected m MBs are continuously input to one MB pipeline, the processing result of the left adjacent MB is fed back to the processing of the (m-1) stage differential MB. it can. In addition, since the reference areas of the selected m MBs are largely overlapped, if the reference areas of these MBs are stored in the common reference area memory, the external memory storing the pictures This eliminates the need for duplicate transfer for updating the overlapped portion. For this reason, by providing the reference area memory inside the moving picture coding apparatus, the amount of data transferred from the external memory storing the picture to the internal memory of the moving picture coding apparatus can be reduced.

  According to the tenth invention, as in the second invention, 2n (n is an integer of 2 or more) MB rows that are continuous vertically are taken as a group, and two MBs that are vertically continuous are taken from the batch of MB rows. Select one MB pair per row, and first put one MB included in these selected n MB pairs into one MB pipeline in succession, and these are selected the second time Since the remaining one MB included in the n MB pairs is continuously input to one MB pipeline, the processing result of the first MB to be input first is (n-1) stages of the remaining one. It is possible to feed back to the processing of the MB, and it is possible to feed back the processing result of the left adjacent MB pair to the processing of the (n−1) stage partial MB pair. Further, since the reference areas of the selected 2n MBs are largely overlapped with each other, if the reference areas of these MBs are stored in a common reference area memory, the external memory storing the pictures This eliminates the need for duplicate transfer for updating the overlapped portion. For this reason, by providing the reference area memory inside the moving picture coding apparatus, the amount of data transferred from the external memory storing the picture to the internal memory of the moving picture coding apparatus can be reduced.

  In the present invention, a plurality of MB rows that are vertically continuous are grouped, one MB is selected per MB row from the group of MB rows, and the selected plurality of MBs are one MB. Put continuously into the pipeline.

  Hereinafter, a moving picture coding apparatus, a moving picture coding method, and an arithmetic coding apparatus according to embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is an explanatory diagram showing an example of MBs that are continuously input to the MB pipeline. The picture in FIG. 1 is composed of 12 MB rows from a row to l row, and each MB row is composed of 16 MBs from 0th to 15th. In the picture of FIG. 1, first, MBs included in the a-MB row, b-MB row, c-MB row, and d-MB row are encoded by one MB pipeline, and then the e-MB row, MBs included in the f-MB line, the g-MB line, and the h-MB line are encoded by one MB pipeline, and finally the i-MB line, the j-MB line, the k-MB line, and the l-MB. MB included in a row is encoded by one MB pipeline. The MB of e row 10 column, f row 8 column, g row 6 column, and h row 4 column in FIG. 1 is an example of an MB that is continuously input to one MB pipeline. The MB in f row and 8 columns is 2 MB away from the MB in e row and 10 columns in the left direction. Similarly, the MB of g row 6 column is 2 MB away from the MB of f row 8 column leftward, and the MB of h row 4 column is 2 MB away from the MB of g row 6 column leftward. These four MBs are not adjacent to each other in the left adjacent, the upper adjacent, or the upper right adjacent. In the following, it is said that these blocks are not adjacent to each other when the plurality of blocks are not adjacent to each other in the left adjacent, the upper adjacent, or the upper right adjacent.

  FIG. 2 shows a block diagram of a moving picture coding apparatus to which the present invention is applied. As shown in FIG. 1, the MB selection circuit 2 collects the e-MB row, the f-MB row, the g-MB row, and the h-MB row, and the MBs belonging to the e-MB row By selecting an MB that has not yet been selected and, for MBs belonging to f-MB, g-MB, and h-MB rows, by selecting an MB for which the upper right neighboring MB has already been selected, The MB of e row 10 column, f row 8 column, g row 6 column, h row 4 column is selected. The encoded data setting circuit 3 sends data necessary for encoding each MB to the MB pipeline 1 in the order of MB of e row 10 column, f row 8 column, g row 6 column, h row 4 column. Set. When data necessary for encoding an MB is set in the MB pipeline 1, the MB is input to the MB pipeline 1. When the data necessary for encoding the MB of h rows and 4 columns is set in the MB pipeline 1 and the MB of h rows and 4 columns is input to the MB pipeline 1, the MB selection circuit 2 A new MB of column, f row 9 column, g row 7 column, h row 5 column is selected.

  FIG. 3 shows a flowchart of the process of the moving picture encoding method to which the present invention is applied. e-MB line, f-MB line, g-MB line, and h-MB line are grouped, and for MBs belonging to the e-MB line, an MB for which the lower left neighboring MB is not yet selected is selected; and For MBs belonging to the f-MB row, the g-MB row, and the h-MB row, by selecting the MB for which the upper right adjacent MB has already been selected, the e row 10 column, the f row 8 column, the g row 6 An MB with 4 columns and h rows is selected (step S1). Next, data necessary for encoding each MB is set in the MB pipeline 1 in the order of MB of e row 10 column, f row 8 column, g row 6 column, h row 4 column, Input into the pipeline (steps S2 and S3). When data necessary for encoding an MB is set in the MB pipeline 1, the MB is input to the MB pipeline 1. When the data necessary for encoding the MB of h rows and 4 columns is set in the MB pipeline 1 and the MB of h rows and 4 columns is input to the MB pipeline 1, the process returns to step S1 (step S4). MBs of e row 11 column, f row 9 column, g row 7 column, h row 5 column are newly selected (step S1).

  FIG. 4 shows a state in which each MB flows through each stage of the MB pipeline 1 when these four MBs are continuously input to one MB pipeline 1. The processing contents of stage 0 to stage 6 are the same as those in FIG. FIG. 5 shows MB cycle numbers at which the MBs constituting the picture of FIG. 1 are input to the MB pipeline 1 and the stage 0 process is started. As shown in FIG. 4, four MBs of e row 10 column, f row 8 column, g row 6 column, h row 4 column are staged in order when the MB cycle numbers are 104, 105, 106, and 107, respectively. When the MB cycle numbers are 110, 111, 112, and 113, respectively, the stage 6 is reached in order and the encoding is completed. Thus, among a group of four MB rows, the MB belonging to the uppermost MB row, the MB belonging to the second MB row from the top, the MB belonging to the third MB row from the top, the fourth from the top The MBs that belong to the MB row are input to the MB pipeline 1 in the order of MBs, and then the right adjacent MBs are input to the MB pipeline 1 in the same order.

  According to the present invention, when the number of MB rows to be encoded as a group is four as in this embodiment, the processing result of the left adjacent MB can be fed back by three stages and reflected in the processing of the current MB. For this reason, as indicated by an arrow in FIG. 4, for example, the high-precision motion detection result and the intra prediction result of the f-row 7-column MB can be reflected in the coarse-precision motion detection of the f-row 8-column MB. Thus, unlike the conventional technique, the motion vector of the current MB can be selected in consideration of the high-precision motion detection result and intra prediction result of the MB adjacent to the left. Similarly, as indicated by an arrow, the result of motion compensation of the MB of f rows and 7 columns can be reflected in the intra prediction of MBs of f rows and 8 columns. Therefore, unlike the conventional technique, intra prediction of the current MB can be performed using the reconstructed pixel obtained by motion compensation of the MB adjacent to the left. Further, the rate control result of the MB of f rows and 7 columns can be reflected in the quantization of MBs of f rows and 8 columns. In this way, the quantization parameter of the current MB can be corrected according to the rate control result of the MB adjacent to the left.

  Here, in this embodiment, the case where the MB included in the MB row adjacent to the lower side is 2 MB away from the MB in the upper MB row in the left direction is shown. However, the object of the present invention can be achieved, and the same action and effect are produced. Also, taking four MB rows that are vertically continuous as one group, one MB per MB row is selected from these four MB rows, and these four MBs are selected as one MB pipeline. However, it is sufficient that the number of MB lines in a group is two or more. For example, eight MB lines that are vertically continuous are grouped, and one of these eight MB lines is used as one group. Even if one MB is selected per MB row, and the selected 8 MBs are continuously input to one MB pipeline, the object of the present invention can be achieved, and similar operations and effects can be achieved. Produce. However, when the number of MB rows in a group is 2, the processing result of the left adjacent MB can be fed back to the current MB processing for only one stage. Therefore, in the stage configuration of FIG. Cannot be fed back to the current MB intra prediction and integer transform / quantization / inverse quantization / inverse integer transform. When the number of MB lines in a group is 2, it is necessary to change the stage configuration from that shown in FIG.

  FIG. 6 shows an MB divided into four SBs of 8 pixels × 8 pixels (hereinafter referred to as 8 × 8SB). In the present embodiment, an example will be described in which a motion vector that minimizes the sum of the cost of a motion vector and the cost of a prediction error is detected for four 8 × 8 SBs. Generally, it is estimated that the cost of a motion vector is higher as the distance between the PMV and the motion vector is longer. For this reason, before starting motion detection for each 8 × 8 SB, the 8 × 8 SB PMV must be determined. For example, if PMV is determined based on the motion vector of the left adjacent 8 × 8SB, the upper adjacent 8 × 8SB, and the upper right adjacent 8 × 8SB, the numbers 0, 1, 2, and 3 in FIG. The motion vectors for every 8 × 8 SB must be detected in order. However, since the motion vector of the upper right 8 × 8SB does not exist when the third 8 × 8SB motion vector is detected, the third 8 × 8SB PMV is the first and second 8 × 8SB motion vectors MV1 and MV2. It is decided based on.

  FIG. 7 is an explanatory diagram showing an example of MBs continuously input into the MB pipeline. The picture in FIG. 7 starts from the a line, and there is an MB line below the r line. There are also MBs after the 16th column. In the present embodiment, the i-MB line, j-MB line, k-MB line, l-MB line, m-MB line, n-MB line, o-MB line, and p-MB line are grouped together. Select MB of i row 15 column, j row 13 column, k row 11 column, l row 9 column, m row 7 column, n row 5 column, o row 3 column, p row 1 column from a group of MB rows. An example is shown in which these eight selected MBs are continuously input into one MB pipeline.

  FIG. 8 shows how each MB flows through each stage when MBs included in 8 MB rows are continuously input to one MB pipeline in order to detect a motion vector every 8 × 8 SB. Show. FIG. 9 shows a block diagram of a motion vector detection circuit 12 to which the present invention is applied. The external memory 4 stores an image with single pixel accuracy. At stage 0, an image is read from the external memory 4. The image is reduced by the two-pixel precision image creation circuit 5 to create a two-pixel precision image and written in the two-pixel precision memory 6. While reading an image from the two-pixel accuracy memory 5 in the stage 1, the two-pixel accuracy motion detection circuit 7 detects a candidate motion vector with two-pixel accuracy. Then, in stage 2, the area around the candidate vector is read from the external memory 4 and written to the single pixel precision memory 8. Here, since an external large-capacity memory is generally low-speed, one stage is allocated for transfer from the external memory 4 to the single pixel precision memory 8. While reading the area around the two-pixel accuracy candidate vector from the single-pixel accuracy memory 8, the motion vector is detected by the single-pixel accuracy motion detection circuit 9 in the stage 3. Thereafter, the single pixel precision pixel is read from the single pixel precision memory 8 at the stage 4, and the quarter pixel precision motion vector detection circuit 11 is created while creating the quarter pixel precision image by the quarter pixel precision image creation circuit 10. Is used to perform 1/4 pixel precision motion vector detection. Since the single pixel accuracy memory 8 supplies a single pixel accuracy image to the single pixel accuracy motion detection circuit 9 and the ¼ pixel accuracy image creation circuit 10, the single pixel accuracy memory 8 has a double buffer configuration or more than two reads. It must be possible multi-port memory. In FIG. 8, the surrounding of the single-pixel accuracy motion vector is directly searched at stage 4 to detect the 1 / 4-pixel accuracy motion vector. However, after detecting the half-pixel accuracy motion vector at stage 4, A ¼ pixel accuracy motion vector may be detected by searching around the pixel accuracy motion vector. Alternatively, single-pixel precision motion vector detection and half-pixel precision motion vector detection may be performed in stage 3, and quarter-pixel precision motion vector detection may be performed in stage 4.

  In stages 1 to 5, the MB cycle is divided into four SB cycles, and each 8 × 8 SB process is performed in each SB cycle. Here, the SB cycle means a unit of time required for processing one SB, and when performing processing for each SB, the MB cycle is divided into SB cycles. Hereinafter, the stage group in which the MB cycle is divided into SB cycles and processing for each SB is performed is referred to as an SB pipeline (or sub-block pipeline). Stages 1 to 5 in FIG. 8 are SB pipelines. FIG. 10 shows the detailed operation of the SB pipeline. −0, −1, −2 and −3 mean 8 × 8SB numbered 0, 1, 2, 3 in FIG. For example, i15-2 represents the second 8 × 8SB included in the MB of i rows and 15 columns. In FIG. 10, an empty stage is inserted into the stage 5, and for example, a ¼ pixel precision motion vector detected in the stage 4 for i15-0SB can be reflected in the cost calculation of the 2-pixel precision motion vector in the stage 1 of i15-1SB. I am doing so. As described above, when the MB cycle is divided into the number of SB cycles equal to the number of SBs, the processing result of the previous 8 × 8 SB in the same MB is obtained by setting a pipeline of (SB number + 1) stages. Of 8 × 8SB.

  In stage 1 to stage 5, it is possible to provide a motion vector detection circuit for each MB that operates in parallel with a motion vector detection circuit for each 8 × 8 SB. Further, in stages 3 to 5, in the peripheral region of the candidate vectors for each 8 × 8SB detected in stage 1, motion vector detection for each 8 × 8SB is performed in parallel with the motion vector detection for each 8 × 8SB, for each 16 pixels × 8 pixels SB, for 8 pixels × It is also possible to detect a motion vector for every 16 pixels SB.

  After motion vectors are detected in stages 0 to 5, a prediction mode with the highest coding efficiency is selected from 16 × 16 intra prediction mode, 4 × 4 intra prediction mode, inter prediction mode, and the like in stage 6. The In stage 7, integer transformation, quantization, inverse quantization, and inverse integer transformation are performed. In stage 8, variable length coding and motion compensation are performed. In stage 9, rate control and deblocking filter processing are performed. In stage 10, the reproduced pixels and the encoded bit stream are written in the external memory 4.

  In this embodiment, two 1/4 pixel precision motion detection circuits 11 are provided, and the stage 3 and the stage 4 are combined to perform one 8 × 8SB 1/4 pixel precision motion detection over 2SB cycles. Indicates. Except stage 3 and stage 4, the stage configuration is the same as that of the MB pipeline of the second embodiment. FIG. 11 shows the detailed operation of the SB pipeline, and FIG. 12 shows a block diagram of the motion detection circuit 13 for realizing this operation. By operating the two quarter-pixel precision motion detection circuits 11 with a shift of 1 SB cycle, each 8 × 8 SB quarter-pixel precision motion detection can be performed over 2 SB cycles.

  In this embodiment, H. An example in which the present invention is applied to 4 pixel × 4 pixel intra prediction in H.264 (hereinafter referred to as 4 × 4 intra prediction) will be described. FIG. 13 shows an MB divided into 16 4 pixels × 4 pixels SB (hereinafter referred to as 4 × 4 SB). 4 × 4SB is encoded in the order of 0 to 15 in FIG. 13 and converted into a bit stream. For example, when 4 × 4 intra prediction is performed for the third 4 × 4SB, the pixels at the bottom 4 rows and the pixels at the right 4 × 4SB at the second 4 × 4SB are referred to. In addition, when performing 4 × 4 intra prediction for the 14th 4 × 4SB, the pixels at the bottom four rows in the 12th and thirteenth 4 × 4SB and the pixels at the rightmost four columns in the 11th 4 × 4SB are referred to. The As the pixels belonging to these adjacent 4 × 4 SBs, it is desirable to use reproduced pixels.

  FIG. 14 shows a state where each MB flows through each stage when MBs included in eight MB rows are continuously input to one MB pipeline for 4 × 4 intra prediction. Stage 4 performs 4 × 4 intra prediction using adjacent 4 × 4 SB reconstructed pixels, and stage 5 performs integer transformation, quantization, inverse integer transformation, and inverse quantization on the prediction error, and adds the intra prediction value Then, a reproduction pixel is generated. The 4 × 4 intra prediction circuit of stage 4 and the integer transform / quantization / inverse integer transform / inverse quantization circuit of stage 5 are in parallel with the motion detection circuits of stage 1 to stage 5 shown in the second or third embodiment. To work.

  In this embodiment, in stages 4 and 5, the MB cycle is divided into 16 SB cycles. For this reason, if the number of SBs is larger than the number of stages in the SB pipeline, and the number of SB pipelines is (SB number + 1), the number of empty stages increases, resulting in poor efficiency. FIG. 15 shows the detailed operation of the SB pipeline. −0, −1, −2,..., -15 means 4 × 4SB numbered 0, 1, 2,. For example, i15-8 represents the 8th 4 × 4SB included in the MB of i rows and 15 columns. As shown in FIG. 15, in this embodiment, 4 × 4 SB included in two MBs are alternately processed to process one MB over 2 MB cycles, that is, 32 SB cycles. When 4 × 4 SB belonging to one MB is subjected to 4 × 4 intra prediction, 4 × 4 SB belonging to another MB is simultaneously subjected to integer transform / quantization / inverse integer transform / inverse quantization. The processing of one MB is started every MB cycle, and the processing of another MB is finished, and matching with the other stages of the MB pipeline is performed. However, when operated in this way, as shown in FIG. 15, in the 4 × 4 intra prediction of stage 4, a deviation from the MB cycle occurs by 1 SB cycle, that is, 1/16 MB cycle every 2 MB cycles. Therefore, in FIG. 15, the effective period of stage 3 is set to 15/16 MB cycle, and the shift of 1/16 MB cycle in stage 4 is absorbed. When the stage 3 process is heavy and an invalid period cannot be provided, the valid period of the stage 6 may be set to 15/16 MB cycle, and the shift of 1/16 MB cycle may be absorbed. By operating in this way, the 4 × 4 intra prediction circuit and the 4 × 4 SB of the left adjacent, the upper adjacent, and the upper right adjacent are generated without generating a sky in the circuits that perform integer transform / quantization / inverse integer transform / inverse quantization. 4 × 4 intra prediction can be performed using the reconstructed pixels for the pixels belonging to.

  It should be noted that the MB pipeline shift can be absorbed by inserting an empty stage before or after the SB pipeline, instead of shortening the effective period of the stage before or after the SB pipeline.

  In the present embodiment, as indicated by an arrow in FIG. 14, the rate control result of the left adjacent MB can be reflected in quantization performed in association with 4 × 4 intra prediction.

  Further, 16 pixel × 16 pixel intra prediction of the luminance signal and 8 pixel × 8 pixel intra prediction of the color difference signal in the 4: 2: 0 format are processed for each MB. For this reason, it is only necessary to refer to pixels belonging to the left adjacent MB and the upper adjacent MB. In this case, since the reconstructed pixel obtained by the motion compensation in stage 8 can be referred to, there is no need to perform integer transformation / quantization / inverse integer transformation / inverse quantization in stage 5 as in 4 × 4 intra prediction. When the 4 × 4 intra prediction mode is selected in the intra / inter determination in stage 6, the reproduced pixels obtained by the integer transform, quantization, inverse integer transform, and inverse quantization in stage 5 are stored. Alternatively, the stage 7 may be configured to perform integer transformation / quantization / inverse integer transformation / inverse quantization again in the stage 7.

  If the mode determination of 4 × 4 intra prediction is performed by the input pixels, the integer transformation / quantization / inverse integer transformation / inverse quantization in stage 5 can be omitted. However, the image quality is deteriorated as compared with the case where the mode is determined using the reproduction pixels.

  In the present embodiment, in the MB pipeline having substantially the same stage configuration as in the second to fourth embodiments, the i-MB row, j-MB row, k-MB row, and l-MB row in FIG. From a group of MB rows, select MBs such as i rows 15 columns, j rows 13 columns, k rows 11 columns, l rows 9 columns, etc., and these selected 4 MBs are connected to one MB pipeline. An example is shown below. In the second to fourth embodiments, since eight MB rows are processed as a group, the processing result of each left adjacent MB can be fed back to the processing of the seven stage equivalent MB, but in this embodiment, only three stages can be fed back. For this reason, when the processing result of the left adjacent MB is reflected in the processing of the current MB, there is a restriction compared to the second to fourth embodiments.

  When MBs included in four MB rows are continuously input to one MB pipeline in order to detect a motion vector for each 8 × 8 SB, each MB flows through each stage of the MB pipeline. Is shown in FIG. In this case, for example, when the intra / inter determination of the MB of j rows and 12 columns ends in stage 5, the 2-pixel precision motion detection of the MB of j rows and 13 columns ends simultaneously in stage 1. For this reason, there is a limitation that the intra / inter determination result of the left adjacent MB cannot be reflected in the two-pixel precision motion detection of the current MB. However, as indicated by arrows in FIG. 16, the intra / inter determination result of the left adjacent MB can be reflected in the stage 2 and subsequent stages. Also, the intra / inter determination result determined in stage 5 for the upper left adjacent MB, the upper adjacent MB, the upper right adjacent MB, the MB one left in the left direction, and the like can be reflected in the two-pixel accuracy motion detection in stage 1. Therefore, in stage 1, a temporary PMV is created with reference to motion vectors such as the upper left adjacent MB, the upper adjacent MB, the upper right adjacent MB, and the MB separated by one in the left direction, and all four 8 × 8 SBs are common. The two-pixel precision motion detection is performed with reference to the temporary PMV. In stage 2, the peripheral area of the candidate motion vector of 2-pixel accuracy detected in stage 1 is set in the single-pixel accuracy memory 8, a true PMV is created in stage 3, and a true PMV is created in stage 3 and stage 4. The single-pixel precision motion detection and the quarter-pixel precision motion detection are performed with reference to FIG.

  FIG. 17 shows the detailed operation of the SB pipeline in stages 3 and 4. In single-pixel precision motion detection of stage 3 and quarter-pixel precision motion detection of stage 4, by processing 8 × 8SB included in two MBs alternately, one MB takes 2 MB cycles, that is, 8SB cycles. Process. When single pixel precision motion detection is performed for 8 × 8SB belonging to one MB, ¼ pixel precision motion detection is simultaneously performed for 8 × 8SB belonging to another MB. The processing of one MB is started for each MB cycle, and the processing is performed so as to finish the processing of another MB, so as to be consistent with the other stages of the MB pipeline. Then, the effective period of stage 2 is set to 3/4 MB cycle, and the shift of 1/4 MB cycle in stage 4 is absorbed. According to this method, four 8 × 8 SBs included in one MB can be serially processed from No. 0 to No. 3. For this reason, in the single-pixel precision motion detection and the quarter-pixel precision motion detection, it is possible to perform motion detection based on a true PMV in consideration of the adjacent 8 × 8 SB motion vector.

  However, in this embodiment, it is necessary to separately evaluate the space direct address. In stage 1, 2-pixel precision motion detection is performed based on the temporary PMV, and in stage 2, the peripheral area of the candidate motion vector detected in stage 1 is set in the single-pixel precision memory 8. When the temporary PMV of stage 1 and the true PMV of stage 3 are different, the motion detection areas in stages 3 and 4 do not include MB or 8 × 8 SB at the position corresponding to the spatial direct address. Therefore, it is necessary to perform processing for evaluating the MB of the position corresponding to the spatial direct address and the cost of 8 × 8 SB in parallel with stage 3 and stage 4.

  FIG. 18 shows how each MB flows through each stage of the MB pipeline when MBs included in four MB rows are continuously input to one MB pipeline for 4 × 4 intra prediction. Show. FIG. 19 shows the detailed operation of the SB pipeline in stages 3 and 4. In the 4 × 4 intra prediction in stage 4 and the integer transformation / quantization / inverse quantization / inverse integer transformation in stage 5, the same processing as in the SB pipeline in the fourth embodiment is performed.

  However, in this embodiment, since the processing result of the left adjacent MB can be fed back to the current MB processing for only three stages, if motion compensation is performed in stage 7, 4 × 4 intra prediction is performed using the reproduced pixels of the left adjacent MB. I can't. Therefore, there is a limitation that motion compensation must be performed at stage 6. The rate control result of stage 8 of the left adjacent MB can be reflected after stage 5 and cannot be reflected in the quantization of stage 4. However, there are cases where a large amount of bits are generated by variable length coding in stage 7 and it is necessary to perform rate control to increase the quantization parameter in stage 8. In this case, the 4 × 4 intra prediction mode is not selected in the intra / inter determination of stage 5, or when the 4 × 4 intra prediction mode is selected, the new quantization parameter is used in stage 6 It is necessary to redo integer conversion, quantization, inverse quantization, and inverse integer conversion.

  As shown in the first to fifth embodiments, in the present invention, a plurality of MB rows that are vertically continuous are taken as a group, and one MB per MB row is selected from the group of MB rows, These selected MBs are continuously input to one MB pipeline. Since these MBs are in a narrow image area, the image encoding device is used to display the reproduction pixels, motion vectors, and the like of the upper adjacent MB and the upper right adjacent MB until the encoding of the MBs belonging to the adjacent MB row below ends. Even if it is held inside, the scale of the memory circuit required for the holding is small. By providing this memory circuit, except for the MB included in the uppermost MB row of the plurality of MB rows, reproduced pixels, motion vectors, etc. of the upper adjacent MB and the upper right adjacent MB for encoding the current MB Can be solved.

  Note that the stage configuration of the MB pipeline shown in the first to fifth embodiments is an example, and other stage configurations may be used. If the stage configuration changes, the constraints in motion detection, intra prediction, and the like change even if the number of vertically continuous MB rows encoded as a group is the same.

  FIG. 20 is a block diagram of the video encoding device 23 having an internal memory. The moving image encoding device 23 reads all data necessary for MB encoding from the external memory 4 and stores the data in the internal memory. There are two types of internal memory: a reference area memory 14 and a buffer memory 15. The reference image to be referred to at the time of motion detection is stored in the reference area memory 14 and is supplied from the reference area memory 14 to the motion detection unit 16 and the motion compensation unit 17. Other data necessary for encoding is stored in the buffer memory 15, and includes a motion detection unit 16, a motion compensation unit 17, an intra prediction unit 18, an integer transform / quantization / inverse quantization / inverse integer transform unit 19, and a variable The long encoding unit 20 and the deblocking filter unit 21 transfer data to and from the buffer memory 15. The control unit 22 performs intra / inter determination and rate control. The control unit 22 is configured by a wired logic or a microprocessor. In FIG. 20, the reference area memory 14 and the buffer memory 15 are shown as separate ones. However, if the reference area address and the buffer address are separated, one common memory may be used.

  FIG. 21 shows reference areas 28, 29, 30, and 31 when four MBs 24, 25, 26, and 27 included in four vertically continuous MB rows are processed by one MB pipeline. Reference areas 28, 29, 30, and 31 are areas that are referred to when motions of MBs 24, 25, 26, and 27 are detected, respectively. The reference areas 28, 29, 30, and 31 are greatly overlapped, and the height of the non-overlapping portions in the reference areas of two MBs belonging to the vertically adjacent MB rows is equal to the number of pixels in the vertical direction of the MB. This overlapping portion becomes larger as the motion vector detection range becomes wider. In the present invention, all the pixels in the reference areas 28, 29, 30 and 31 are stored in the reference area memory 14. As in the prior art, in this embodiment, when encoding of the MB adjacent in the right direction is started, only the update areas 32, 33, 34, and 35 are transferred from the external memory 4 to the reference area memory 14, and the code is encoded so far. The origin of the reference area 28, 29, 30, 31 of the left adjacent MB that has been converted is changed to a new origin for the right adjacent MB to be encoded next. However, the update area 33 for the MB 25 is only a portion not included in the reference area 28 of the MB 24. Similarly, the update areas 34 and 35 of MB26 and MB27 are only portions not included in the reference areas 29 and 30 of MB25 and MB26, respectively. Therefore, the amount of pixel transfer between the external memory 4 and the reference area memory 14 can be greatly reduced as compared with the prior art.

  FIG. 22 shows another method for storing the reference areas 28, 29, 30, and 31 in the reference area memory 14. The reference area memory 14 stores a rectangular area 36 that includes all the reference areas 28, 29, 30, and 31. According to this method, the address generation of the reference area memory 14 is simplified. Further, since the update area 37 is also a group of rectangular areas, transfer of pixels from the external memory 4 to the reference area memory 14 is simplified.

  In the above, the case where four MBs included in four consecutive MB rows are simultaneously processed by one MB pipeline has been shown as an example. However, the method shown in the present embodiment may be performed on two or more consecutive MB rows. For example, it may be eight consecutive MB lines.

  The reference region memory 14 stores only a single pixel accuracy image, and the single pixel accuracy image read out from the reference region memory 14 is used to generate a half pixel accuracy image or ¼ inside the motion detection unit 16 and the motion compensation unit 17. It can be set as the structure which produces a pixel precision image. In this configuration, a half-pixel precision image or a quarter-pixel precision image is created in advance and stored in the external memory 4, and the half-pixel precision image or the quarter-pixel precision image is transferred from the external memory 4 to the reference area memory 14. The pixel transfer amount between the external memory 4 and the reference area memory 14 can be reduced as compared with the configuration to be performed.

  Further, there is a hierarchical search method in which a reduced image such as a two-pixel accuracy image is created, motion is first detected on the reduced image, and motion detection is performed on the high-accuracy image step by step. Even when the present invention is applied to this search method, only the single-pixel precision image is stored in the reference area memory 14, and a reduced image is created inside the motion detection unit 16 and the motion compensation unit 17. It can be configured. This configuration creates a reduced image in advance and stores it in the external memory 4, and transfers pixels between the external memory 4 and the reference region memory 14 rather than a configuration in which the reduced image is transferred from the external memory 4 to the reference region memory 14. The amount can be reduced.

  In addition, a telescopic search method has been proposed as a method for reducing the amount of calculation in motion detection. In the telescopic search method, an intermediate image between the encoding target image and the reference image is also required. In this case, the update method shown in this embodiment can be applied to a region necessary for motion detection in an intermediate image.

  Further, only the luminance signal may be transferred from the external memory 4 to the internal memory of the image encoding device 23 by the method shown in this embodiment, or the method shown in this embodiment is applied to both the luminance signal and the color difference signal. May be. If the color difference signal is not referred to during motion detection, the color difference signal may be stored in either the reference area memory 14 or the buffer memory 15.

  When HDTV video is to be encoded, the range of motion vectors needs to be about ± 200 pixels horizontally and ± 100 pixels vertically to obtain good image quality. The amount of computation for detecting a motion vector in this range is enormous, and a large number of cycles are required for motion detection. However, in a video encoding apparatus having an MB pipeline structure, the number of cycles per MB cycle is common to all stages. Only the stage for motion detection cannot increase the number of cycles per MB cycle. Also, in the MB pipeline to which the present invention is applied, there is a limit to the number of stages where the processing result can be fed back, so it is not preferable to increase the number of MB pipeline stages.

  FIG. 23 shows a combination of a motion detection device 38 capable of detecting motion in a wide range and a moving image encoding device 23 including an MB pipeline to which the present invention is applied. The motion detection device 38 detects motion vector candidates in consideration of only the encoding cost of the prediction error without considering the encoding cost of the motion vector. The video encoding device 23 receives the motion vector candidate detected by the motion detection device 38 and performs motion detection in consideration of the motion vector encoding cost for the peripheral region of the motion vector candidate. That is, motion is detected without referring to the PMV over a wide range, and motion detection is performed again with reference to the PMV for the peripheral region of the detected motion vector candidate. The motion detection again in the peripheral region of the received motion vector candidate can be performed at a motion detection stage of any accuracy such as 2-pixel accuracy motion detection or single pixel accuracy motion detection. In the moving image encoding device 23, in addition to the peripheral region of the motion vector candidate, motion detection may be performed simultaneously in another region where there is a high possibility that a motion vector with the minimum cost exists, for example, the peripheral region of the PMV. Since the motion detection device 38 operates independently of the MB pipeline of the video encoding device 23, motion detection can be performed over many cycles.

  In addition, the motion detection device 38 may be configured to perform motion detection on an input image that has not been encoded, but is not a reproduced image created by the video encoding device 23. With this configuration, it is not necessary to send a reproduced image from the moving image encoding device 23 to the motion detection device 38.

  It is possible to integrate the motion detection device 38 and the moving image encoding device 23 on the same LSI. In this case, the external memory 4 connected to the video encoding device 23 is a shared memory of the motion detection device 38 and the video encoding device 23, and the motion detection device 38 performs motion detection on the reproduced image. Also good.

  Also, MPEG-2 to H.264. When the already detected motion vector can be used as in the case of transcoding to H.264, the detected motion vector is directly input to the moving image encoding device 23 without using the motion detecting device 38, and this detected motion is detected. It is also possible to adopt a configuration in which motion detection is performed in consideration of the coding cost of the motion vector for the peripheral region of the vector.

  FIG. 24 shows a block diagram of a parallel video encoding device 42 having two video encoding devices 40 and 41 and an internal memory 39. FIG. 25 shows a picture area in which the video encoding devices 40 and 41 share the encoding. The moving image encoding device 40 divides and encodes i-MB line to l-MB line, q-MB line to t-MB line, etc., and e-MB line to h-MB line, m-MB line to p The video encoding device 41 shares MB rows and the like and encodes them. According to this picture division method, since the picture areas shared by the moving picture coding apparatuses 40 and 41 are vertically continuous, the reference areas of both are greatly overlapped vertically. If the configuration is such that the reference areas of the two moving image encoding devices 40 and 41 are transferred from the external memory 4 to the internal memory 39 and are transferred from the internal memory 39 to the individual moving image encoding devices 40 and 41, both references are made. Since there is no need to transfer the overlapping portions of the areas in an overlapping manner, the data transfer amount between the external memory 4 and the internal memory 39 can be reduced.

  FIG. 26 shows how the MB flows through each stage of the MB pipeline included in the moving image encoding devices 40 and 41. The moving image encoding devices 40 and 41 have the same stage configuration as the moving image encoding device of the first embodiment. However, for example, the reproduction pixel and motion vector of MB belonging to the l-MB row may be transferred from the moving image coding device 40 to the moving image coding device 41 and referred to in the coding of MB belonging to the m-MB row. Is possible. In this way, at the boundary between the regions shared by the moving image encoding devices 40 and 41, the moving image encoding device is at a timing at which data referred to by the current MB, such as reproduction pixels and motion vectors in the upper adjacent MB and the upper right adjacent MB, can be transferred. Operate 40 and 41. By operating in this way, unlike the conventional slice interleaving, according to the method of this embodiment, each of the i-MB line to l-MB line and the m-MB line to p-MB line in FIG. It is not necessary to divide the slice for each area to be encoded by the moving picture coding apparatuses 40 and 41, and the entire picture can be made into one slice. Further, when the first area of the picture to be processed by the moving image encoding devices 40 and 41 such as the a-MB row to the d-MB row and the e-MB row to the h-MB row is input, the moving image encoding device 40 is input. And 41 can start encoding, so that encoding delay can be suppressed.

  The parallel video encoding device 42 may include three or more video encoding devices.

  H. In H.264, CABAC can be used as a variable length coding method. CABAC continuously processes all MBs included in one slice using one arithmetic coding circuit. On the other hand, in the inventions described in the first to eighth embodiments, a plurality of vertically continuous MB rows are taken as a group, and one MB is selected per MB row from the group of MB rows. A plurality of MBs are continuously input to one MB pipeline. For this reason, when the present invention is used, a binary symbol cannot be arithmetically encoded unless encoding of all MBs included in a group of MBs that are vertically continuous is completed. However, when a certain MB is encoded, the generated code amount may be remarkably large. In such a case, it is necessary to increase the quantization parameter immediately and control the rate so as to reduce the amount of code generated from the newly encoded MB. In the present embodiment, the H.264 encoding apparatus to which the invention described in the first to eighth embodiments is applied. In the case of performing arithmetic coding such as CABAC defined by H.264, a method is shown in which rate control is performed before the coding of all MBs included in a group of MBs that are vertically continuous is completed.

  As described in the section of the background art, a method for estimating the generated code amount of arithmetic coding from the sum of the symbol lengths of binarized symbols has been proposed (for example, p.20 of Non-Patent Document 6, FIG. 6). reference). If rate control is performed based on the generated code amount estimated by this method, rate control can be performed without waiting for the completion of encoding of all MBs included in a group of MBs that are vertically continuous. FIG. 27 is a block diagram of a CABAC encoder that performs rate control based on the estimated generated code amount.

  First, the syntax data is binarized by the binarization circuit 43. The binarization circuit 43 includes a look-up table, and performs conversion into a binarized symbol in one cycle using the table look-up. Then, the generated code amount for each MB is estimated from the sum of the symbol lengths of the binarized symbols converted by the generated code amount estimation circuit 44, and the estimated generated code amount is sent to the control unit 22 for rate control. As a result, rate control can be performed before the encoding of all MBs included in a group of MBs that are vertically continuous is completed.

  Here, in the moving picture encoding apparatus to which the present invention is applied, when the first MB of the MB row is processed, the quantization parameter of the last MB of the MB row adjacent to it is not yet determined. However, the difference between the quantization parameter and the quantization parameter of the MB encoded immediately before is binarized, and arithmetic coding is performed. Thus, the quantization parameter of the first MB in the MB row is binarized from the temporary value (hereinafter referred to as temporary mb_qp_delta). The provisional value is desirably determined by referring to the quantization parameter of the already encoded MB and estimating the quantization parameter of the last MB in the MB row adjacent above. On the other hand, the quantization parameters of MBs other than the head in the MB row are H.264. As defined in H.264, the difference (hereinafter referred to as mb_qp_delta) from the left adjacent MB encoded immediately before is binarized.

  Next, the binarized symbols are temporarily stored in the binarized symbol storage circuit 45, and after the encoding of all MBs included in a group of MBs that are vertically continuous is completed, the binarized symbols are binarized. Read out from the symbol storage circuit 45. Then, the arithmetic coding circuit 46 performs arithmetic coding while restoring the context information corresponding to the context information restoring circuit 47. The processing of the context information restoration circuit 47 is almost the same as the context information restoration processing in the CABAC decoding circuit used in the video decoding device. The processing flow of the context information restoration circuit 47 is shown in FIG. Context information is initialized at the head of the slice (step S5). Next, the variable tmp is cleared (step S6), and one binarized symbol of syntax data is acquired (step S7). Thereafter, the symbol context information to be arithmetically encoded is acquired (step S8), and 1 bit of symbol is read from the binarized symbol storage circuit 45 (step S9). Then, the context information and symbol are output to the arithmetic encoding circuit 46 (step S10). After the variable tmp is logically shifted by 1 bit to the left, symbol is connected to LSB (step S11). If the new tmp is different from the binarized symbol acquired in step S7, the process returns to step S8 (step S12). If tmp matches the binarized symbol, it is determined whether or not it is the end of the slice. If it is not the end of the slice, the process returns to step S6 (step S13) to clear tmp and obtain a new binarized symbol. (Steps S6 and S7). If it is the end of the slice, the context information restoration process is terminated (step S13).

  However, when the temporary mb_qp_delta of the first MB in the MB row is stored in the binarized symbol storage circuit 45, the quantization parameter of the last MB in the MB row adjacent above is reproduced, and the difference between the parameters is calculated and the actual MB After obtaining mb_qp_delta, it must be binarized again and arithmetically encoded. At that time, it is necessary to decode mb_qp_delta for all MBs and reproduce the quantization parameter. In order to avoid this, the quantization parameter itself may be stored in the binary symbol storage circuit 45 together with mb_qp_delta for the last MB in the MB row. By storing the quantization parameter of the last MB in the MB row as a constant bit length, the context information restoration circuit 47 can identify the quantization parameter of the last MB in the MB row.

  Note that the binarized symbol storage circuit 45 may not be provided in the moving image encoding apparatus, and the binarized symbols may be stored in the external memory 4.

  The method of this embodiment has an advantage that arithmetic coding can be performed even if the arithmetic coding circuit does not have a performance sufficient to process a binarized symbol generation amount that suddenly increases. In particular, if the binarized symbol storage circuit 45 has a sufficient storage capacity to store the binarized symbols of the I picture and the binarized symbols of the P picture and B picture before and after the I symbol, the arithmetic coding circuit 46 is provided. There is an advantage that it is not necessary to have a performance sufficient to process the binarized symbol generation amount of an I picture, and it is sufficient to have a performance sufficient to process an average generation amount of binarized symbols of a plurality of pictures.

  In the present embodiment, a rate control method different from that of the ninth embodiment in the case where the present invention is applied to a moving picture coding apparatus using CABAC is shown. FIG. 29 is a block diagram of a CABAC encoder that performs rate control based on a provisional generated code amount. The binary symbol is temporarily stored in the binary symbol storage circuit 45, and is read out from the binary symbol storage circuit 45 to perform arithmetic coding while restoring the corresponding context information (hereinafter referred to as true arithmetic coding). This is the same as in the ninth embodiment. In the present embodiment, the binary symbols are provisionally arithmetically encoded by the provisional arithmetic encoding circuit 48 to calculate a provisional generated code amount, and all the MBs included in a group of MB rows that are vertically continuous are encoded. The difference is that the rate is controlled based on the provisional generated code amount before the end.

  In the first MB in the MB row, arithmetic coding is performed by taking over the range, low, and probability table when the arithmetic coding of the last MB in the adjacent MB row is finished. However, in the present invention, since encoding is performed with a plurality of vertically continuous MB rows as a group, when the first MB in the MB row is encoded, the last MB in the adjacent MB row is already encoded. Absent. For this reason, until encoding of a group of consecutive MB rows is completed, the arithmetic is performed by taking over the range, low, and probability table when the arithmetic encoding of the last MB in the adjacent MB row is finished. Encoding cannot be performed. Therefore, for the first MB in the MB row, provisional values are set in the provisional range, low, and probability table, and provisional arithmetic coding is started.

  FIG. 30 shows a block diagram of the provisional arithmetic encoding circuit 48 in the case where encoding is performed with a group of four consecutive MB rows as a group. The context calculation part 49 performs context calculation for each symbol. There are four portions 51 to 54 for storing a range, portions 55 to 58 for storing a low, and probability tables 59 to 62 corresponding to four MB rows. The range (n) storage portion 51, the low (n) storage portion 55, and the probability table (n) 59 correspond to the uppermost MB row of the four MB rows. The range (n + 1) storage portion 52, the low (n + 1) storage portion 56, and the probability table (n + 1) 60 correspond to the second MB row from the top. The range (n + 2) storage portion 53, the low (n + 2) storage portion 57, and the probability table (n + 2) 61 correspond to the third MB row from the top, the range (n + 3) storage portion 54, and the low (n + 3) storage. Portion 58 and probability table (n + 3) 62 correspond to the bottom MB row. The contents of the parts 51 to 54 for storing the range and the parts 55 to 58 for storing the low are read into the arithmetic encoding part 50 only when the arithmetic encoding of the individual MBs included in the four MB rows is started. When the arithmetic encoding of the MB is finished, it is written and updated from the arithmetic encoding portion 50.

  There are three possible methods for setting a temporary range, low, and probability table for the first MB in the MB row. The first method is described in H.W. This is a method of performing initialization defined in H.264. H. H.264 initializes the range, low, and probability tables at the beginning of the slice. This initialization is performed when provisional arithmetic coding is started for the first MB in the MB row.

  In the second method, the range, low, and probability tables when the last MB in the MB row has been arithmetically encoded for each MB row in a group of vertically contiguous MB rows are This is a method of taking over to the first MB in a corresponding MB row in a group of adjacent MBs. FIG. 31 shows a case where encoding is performed with a group of four vertically continuous MB rows. FIG. 31 shows a picture composed of 12 MB rows. For the top four batches of MB in the picture, the range, low, and probability table of the first MB in each MB row is H.264. It is initialized by the method specified in H.264. For the individual MB rows included in the four MB rows in the second batch from the top, the arithmetic encoding of the last MB in the individual MB rows included in the uppermost four MB rows has been completed. Take over the time range, low, and probability table. Similarly, for the 4 MB rows in the third batch from the top, the arithmetic encoding of the last MB in each MB row included in the 4 MB rows in the second batch from the top is finished. Take over the range, low, and probability tables.

  The third method is the same as the range, low, and probability table inherited by the MB at the same position in the picture that has already been subjected to true arithmetic coding, and the temporary range, low, And setting as a probability table.

  When a certain MB is encoded, the binarized symbols suddenly increase, and when the processing capacity of the provisional arithmetic encoding circuit 48 is exceeded, provisional arithmetic encoding is aborted on the way to the next MB. The provisional arithmetic coding may be started. When the generation amount of binarized symbols increases rapidly, the generated code amount is estimated from the generation amount of binarized symbols as in the ninth embodiment, and the quantization parameter of the MB to be encoded is increased from now on, Rate control can be performed by a method such as reducing the generation amount of binarized symbols. For this reason, it is not necessary to give the provisional arithmetic coding circuit 48 sufficient performance to handle the maximum amount of binarized symbols that occur unexpectedly. Also in the method of the present embodiment, as in the method of the ninth embodiment, if the binarized symbol storage circuit 45 has a sufficient storage capacity, the true arithmetic coding circuit 46 can average the binarized symbols of a plurality of pictures. There is an advantage that it is sufficient to have performance sufficient to process the generated amount.

  A plurality of provisional arithmetic encoding circuits 48 may be provided. FIG. 32 is a block diagram of the provisional arithmetic coding circuit 68 when provisional arithmetic coding is performed in two parallel. Two context calculation parts 49 and two arithmetic coding parts 50 are provided. One arithmetic encoding portion 50 includes a range (n) storage portion 51, a low (n) storage portion 55, a probability table (n) 59, a range (n + 2) storage portion 53, a low (n + 2) storage portion 57, and A probability table (n + 2) 61 is connected, and provisional arithmetic coding is performed on the MBs included in the uppermost MB row and the third MB row from the top among a group of four MB rows. The other arithmetic coding part 50 includes a range (n + 1) storage part 52, a low (n + 1) storage part 56, a probability table (n + 1) 60, a range (n + 3) storage part 54, and a low (n + 3) storage part. 58 and a probability table (n + 3) 62 are connected, and provisional arithmetic coding is performed on the MBs included in the second MB row from the top and the lowest MB row among a group of four MB rows. In the MB pipeline stage configuration similar to that of the first embodiment, a state in which the MB of e row 10 column, f row 8 column, g row 6 column, and h row 4 column in FIG. Shown in Thus, by performing provisional arithmetic coding in two parallels, provisional arithmetic coding of one MB can be performed over 2 MB cycles. Note that the memories 63 and 65 and the memories 64 and 66 in FIG. 32 are provided to hold binary symbols and syntax data, respectively, for 2 MB cycles.

  The method of the ninth embodiment requires a ratio α between the binary symbol length of the immediately previous encoded picture and the generated code amount. For this reason, when there is a scene change or the like, the estimation of the generated code amount becomes inaccurate. On the other hand, according to the method of the present embodiment, as the arithmetic coding of MBs in the same MB row is performed, the value of the probability table used for provisional arithmetic coding converges to an accurate one. For this reason, even when there is a scene change or the like, it is possible to perform rate control close to rate control based on the generated code amount by true arithmetic coding.

  After the syntax data is compressed by a method other than binarization and temporarily stored, the syntax data can be restored and binarized or arithmetically encoded. For example, H.M. In H.264, in addition to CABAC, CAVLC is defined as a variable-length encoding method, but CAVLC can variable-code one piece of syntax data with 1 CK. FIG. 34 shows a block diagram of a CABAC encoder when compressed by CAVLC. The CAVLC encoded bit stream generated by compressing the syntax data by the CAVLC encoding circuit 69 is temporarily stored in the CAVLC encoded bit stream storage circuit 70. Then, the CAVLC encoded bit stream read from the CAVLC encoded bit stream storage circuit 70 is decoded by the CAVLC decoding circuit 71, the restored syntax data is binarized by the binarizing circuit 43, and the arithmetic encoding circuit 46 Perform arithmetic coding. FIG. 34 shows a case where provisional arithmetic coding is performed and rate control is performed using the temporarily generated code amount. As shown in the ninth embodiment, arithmetic coding is performed based on the sum of the symbol lengths of the binarized symbols. The amount of generated codes may be estimated and the rate may be controlled.

  Here, as pointed out in the ninth embodiment, in the moving picture encoding apparatus to which the present invention is applied, when the first MB of the MB row is processed, the quantum of the last MB of the adjacent MB row is further processed. The parameter has not yet been determined. Thus, as in the ninth embodiment, the quantization parameter of the first MB in the MB row is encoded with a difference from the temporary value (hereinafter referred to as temporary mb_qp_delta) by CAVLC, and the quantum of the MB other than the first MB in the MB row. The encoding parameter is obtained by encoding the difference (hereinafter referred to as mb_qp_delta) with that of the left adjacent MB encoded immediately before by CAVLC. Then, with respect to the quantization parameter of the first MB in the MB row, the quantization parameter of the last MB in the upper adjacent MB row is reproduced, and the difference between them is calculated to obtain the actual mb_qp_delta, and then binarized. Perform arithmetic coding. At that time, in order to avoid decoding mb_qp_delta for all MBs and reproducing the quantization parameter, the quantization parameter itself is stored in the CAVLC encoded bitstream storage circuit 70 together with mb_qp_delta for the last MB in the MB row. It is also good.

  Any method other than CAVLC can be used as long as it is a reversible compression method capable of compressing and restoring syntax data. That is, syntax data compressed by lossless compression of syntax data is temporarily stored. Then, the compressed syntax data is read to restore the syntax data. The restored syntax data is converted into a binary symbol and arithmetically encoded.

  H. In H.264, CAVLC and CABAC can be switched for each picture by specifying entropy_coding_mode_flag. Therefore, when CAVLC is used as the lossless compression method, the amount of binarized symbols generated for a certain MB suddenly increases, and it is predicted that the processing capacity of the true arithmetic encoding circuit 46 will be exceeded as a result of provisional arithmetic encoding. The entire picture including the MB is CAVLC encoded, and when the result of provisional arithmetic encoding is predicted to be processed by the true arithmetic encoding circuit 46, the entire picture including the MB is CABAC encoded. be able to. As a result, a situation in which the performance of the true arithmetic coding circuit 46 is unexpected can be surely avoided.

  Further, the CAVLC encoding circuit 69 calculates the code amount of the CAVLC encoded bit stream, and compares the CAVLC encoding and the provisional generated code amount of the provisional arithmetic encoding, and it is estimated that the generated code amount is small for each picture. An entropy coding mode can also be selected.

  In the present embodiment, a method of generating one bit stream encoded by CABAC from the parallel moving image encoding device 42 including the two moving image encoding devices 40 and 41 described in the eighth embodiment will be described. . FIG. 35 shows a block diagram of a parallel moving image encoding device in which binarized symbols are output from the moving image encoding devices 40 and 41. The binarized symbols output from the moving image encoding devices 40 and 41 are temporarily stored in the common binarized symbol storage circuit 45. Then, the binary encoding symbol is read from the binary symbol storage circuit 45, and the arithmetic encoding circuit 46 performs arithmetic encoding while recovering the corresponding context information using the context information recovery circuit 47.

  FIG. 36 shows a block diagram of a parallel moving image encoding apparatus in which a CAVLC encoded bit stream is output from the moving image encoding apparatuses 40 and 41. The CAVLC encoded bit stream output from the moving image encoding devices 40 and 41 is temporarily stored in the common CAVLC encoded bit stream storage circuit 70. Then, the CAVLC encoded bit stream is read from the CAVLC encoded bit stream storage circuit 70, decoded by the CAVLC decoding circuit 71 and returned to syntax data, and then the syntax data is binarized and subjected to arithmetic encoding.

  The parallel video encoding device 42 may include three or more video encoding devices.

  H. In H.264 MBAFF (Macroblock Adaptive Frame Field Coding), two vertically adjacent MBs are paired (hereinafter referred to as MB pairs), and encoding is performed by switching between field encoding and frame encoding every 16 pixels × 32 lines. To do. An MB pair at the time of field coding and frame coding is shown in FIG. FIG. 37A shows an MB pair at the time of field coding. Of the 32 lines of pixels, odd lines constitute the top field MB72 and even lines constitute the bottom field MB73. FIG. 37B shows an MB pair at the time of frame coding. Of the 32 lines, the upper half 16 lines constitute the upper frame MB74, and the lower half 16 lines constitute the lower frame MB75. The top field MB72 and the bottom field MB73 are not in the relationship of the adjacent MB to each other, but the upper frame MB74 is in the relationship of the upper adjacent MB to the lower frame MB75. For this reason, when frame coding is selected, there is a limitation that the PMV of the lower frame MB75 must be determined in consideration of the intra / inter determination result of the upper frame MB74. In addition, since intra-prediction of the lower frame MB75 uses the reconstructed pixel of the upper frame MB74, if the motion compensation is performed for the upper frame MB74 and the reconstructed pixel is not created, the intra-prediction cannot be performed for the lower frame MB75. There is.

  When field coding or frame coding is selected for both the current MB pair and the left adjacent MB pair, the left adjacent MB of each MB included in the current MB pair is only the MB located on the left side. However, when the field encoding / frame encoding selection of the current MB pair and the left adjacent MB pair is different, both MBs included in the left adjacent MB pair are left adjacent MBs of individual MBs included in the current MB pair. . FIG. 38A shows a case where field encoding is selected for the current MB pair and frame encoding is selected for the left adjacent MB pair. The odd line 76 of the upper frame MB and the odd line 77 of the lower frame MB included in the left adjacent MB pair correspond to the left adjacent MB of the top field MB72, and the even number of the upper frame MB corresponds to the left adjacent MB of the bottom field MB73. A line corresponds to an even line of the lower frame MB. Further, FIG. 38B shows a case where frame coding is selected for the current MB pair and field coding is selected for the left adjacent MB pair. The upper half 16 lines 78 of the top field MB and the upper half 16 lines 79 of the bottom field MB correspond to the left adjacent MB of the upper frame MB74, and the lower half 16 lines of the top field MB correspond to the left adjacent MB of the lower frame MB75. The lower half 16 lines of the bottom field MB correspond. For this reason, if motion compensation is performed for both MBs included in the left adjacent MB pair and the reproduced pixel is not calculated, intra prediction cannot be performed for the current MB pair.

  In the present embodiment, in the case of MBAFF, eight MB rows that are vertically continuous are taken as a group, and one MB pair is selected from these batch of MB rows per two MB rows that are vertically continuous. An example will be shown in which the four MB pairs are input to one MB pipeline. FIG. 39 is a diagram showing an example of an MB pair to which the present invention is applied. The picture is the same as that shown in FIG. 7, but as described above, in MBAFF, line assignment to two MBs differs depending on whether the MB pair is field-coded or frame-coded. Therefore, an MB that constitutes an MB pair is shown using uppercase letters. In the field coding, I15, K13, M11, O9, etc. are the top field MB72, and J15, L13, N11, P9, etc. are the bottom field MB73. In frame encoding, I15, K13, M11, O9, etc. are the upper frame MB74, and J15, L13, N11, P9, etc. are the lower frame MB75.

  FIG. 40 is a block diagram of a moving picture encoding apparatus to which the present invention is applied. For example, as shown in FIG. 39, the MB pair selection circuit 81 includes an i-MB row, a j-MB row, a k-MB row, an l-MB row, an m-MB row, an n-MB row, an o-MB row, For the MB pairs belonging to the p-MB rows and belonging to the i-MB rows and the j-MB rows, the MB pairs for which the lower left neighboring MB pair has not yet been selected are selected, and the k-MB rows and the l- For MB pairs belonging to the MB row, the m-MB row and the n-MB row, the o-MB row and the p-MB row, by selecting the MB pair in which the upper right adjacent MB pair has already been selected, I15-J15 , K13-L13, M11-N11, O9-P9 MB pairs are selected. When field encoding is selected, the field MB pair / frame MB pair configuration circuit 82 configures the top field MB72 with the odd lines of the MB pair and configures the bottom field MB73 with the even lines of the MB pair. On the other hand, when frame coding is selected, the upper frame MB74 is configured by the upper half line of the MB pair, and the lower frame MB75 is configured by the lower half line of the MB pair. When field encoding is selected, the encoded data setting circuit 83 first takes 4 MB cycles for one of the MBs included in the MB pairs of I15-J15, K13-L13, M11-N11, and O9-P9. In order, data necessary for encoding individual MBs every 1 MB cycle is set in the MB pipeline 80. When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. Then, for the remaining one MB included in the MB pair of I15-J15, K13-L13, M11-N11, and O9-P9 for the second time, individual MBs are encoded every 1 MB cycle in order over 4 MB cycles. Data necessary for this is set in the MB pipeline 80. When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When frame encoding is selected, the data necessary for encoding each MB in each macroblock cycle in the order of I15MB, K13MB, M11MB, O9MB, J15MB, L13MB, N11MB, and P9MB Set to pipeline 80. When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When the data necessary for encoding the last MB selected by the MB pair selection circuit 81 is set in the MB pipeline 80 and the MB is input to the MB pipeline 80, the MB pair selection circuit 81 The MB pair of I16-J16, K14-L14, M12-N12, and O10-P10 is newly selected.

  FIG. 41 shows a flowchart of the process of the moving picture coding method to which the present invention is applied. For example, as shown in FIG. 39, i-MB line, j-MB line, k-MB line, l-MB line, m-MB line, n-MB line, o-MB line, and p-MB line are grouped. For the MB pairs belonging to the i-MB row and the j-MB row, the MB pair for which the lower left adjacent MB pair has not yet been selected is selected, and the k-MB row, the l-MB row, and the m-MB row For the MB pairs belonging to the n-MB row, the o-MB row, and the p-MB row, by selecting the MB pair for which the upper right adjacent MB pair has already been selected, I15-J15, K13-L13, M11- The MB pair of N11 and O9-P9 is selected (step S14). Next, when field coding is selected, the top field MB72 is composed of the odd lines of the MB pair, and the bottom field MB73 is composed of the even lines of the MB pair. On the other hand, when frame coding is selected, the upper frame MB74 is configured by the upper half line of the MB pair, and the lower frame MB75 is configured by the lower half line of the MB pair (steps S15, S16, and step S16). S23). When field coding is selected, first, one MB included in the MB pair of I15-J15, K13-L13, M11-N11, and O9-P9 is sequentially repeated every 4 MB cycles for every 1 MB cycle. Data necessary for encoding individual MBs is set in the MB pipeline 80 (step S17, step S18, step S19). When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. Next, in order to encode individual MBs for each 1 MB cycle in order over 4 MB cycles for the remaining one MB included in the MB pairs of I15-J15, K13-L13, M11-N11, and O9-P9 Necessary data is set in the MB pipeline 80 (step S20, step S21, step S22). When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When frame encoding is selected, the data necessary for encoding individual MBs every 1 MB cycle in the order of I15MB, K13MB, M11MB, O9MB, J15MB, L13MB, N11MB, and P9MB is stored in the MB pipeline. 80 (step S24, step S25, step S26, step S27, step S28, step S29). When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When data necessary for encoding the selected last MB is set in the MB pipeline 80, and the MB is input to the MB pipeline 80, the process returns to step S14 (steps S21 and S28), and I16. -MB pairs of J16, K14-L14, M12-N12, and O10-P10 are newly selected (step S14).

  FIG. 42 shows that when MB pairs included in 8 MB rows are input to one MB pipeline in order to detect a motion vector for each 8 × 8 SB, each MB pair flows through each stage of the MB pipeline. Show how it goes. Here, since the number of pixels in the vertical direction of the field image is half that of the frame image, when the pixel values of two pixels in the vertical direction are averaged to create a reduced image, the reduced image differs between the field image and the frame image. . On the other hand, in the reduced image of the vertical single pixel / horizontal two-pixel accuracy, when an odd line is extracted from the reduced image of the frame, a reduced image of the top field is obtained, and when an even line is extracted from the reduced image of the frame, a reduced image of the bottom field is obtained. Therefore, in this embodiment, as shown in FIG. 42, a reference area of vertical single pixel / horizontal two pixel accuracy is set in the memory at stage 0, and motion detection is performed at vertical single pixel / horizontal two pixel accuracy at stage 1. Do.

  Further, in this embodiment, after four MBs in one MB pair such as I15, K13, M11, and O9 are continuously inserted into the MB pipeline, the remaining MB pairs such as J15, L13, N11, and P9 are left. Four MBs on one side of are continuously inserted. Thus, since one MB of the MB pair is continuously input into the MB pipeline, when the motion vector reference area is updated in that order, the update area becomes discontinuous. Therefore, when setting the vertical single pixel / horizontal two-pixel accuracy memory at stage 0, as shown in FIG. 22, the update areas of eight MB rows are set as a group of rectangular areas.

  In addition, to select either field coding or frame coding, both intra prediction and inter prediction are performed for each of field coding and frame coding, and a combination of a coding mode and a prediction mode with a small prediction error. It is desirable to select. However, this method requires a huge amount of calculation. On the other hand, a simple selection method is also conceivable in which an encoding mode is determined only with pixels in the current MB pair, and intra prediction and inter prediction are performed only on the selected one. For example, the sum of absolute differences or the sum of squares of differences is calculated between adjacent lines for all 32 lines, and the sum of absolute differences or squares of differences is calculated for each line, that is, between adjacent lines of 16 odd lines and 16 even lines. When the sum is obtained and the value obtained for all 32 lines is smaller, frame coding is performed, and when the value obtained for each line is smaller, field coding is performed. Although it may not be an optimal selection, the amount of computation is small. In this embodiment, a simple selection method is used, and after performing field / frame determination at stage 0, inter prediction is performed at stages 1 to 4 as shown in FIG. 42 for the selected coding mode. In parallel with this, 4 × 4 intra prediction, 16 × 16 intra prediction, and the like are performed in stages 3 and 4.

  In the present embodiment, as in Embodiments 2 to 4, 8 MB rows that are vertically continuous are processed as a group. However, as pointed out in paragraph 0122 above, when frame encoding is selected, the upper frame MB74 is in the relationship of the upper adjacent MB with respect to the lower frame MB75. Also, as pointed out in paragraph 0123 above, when the field encoding / frame encoding selection of the current MB pair is different from that of the left adjacent MB pair, both MBs included in the left adjacent MB pair are included in the current MB pair. It becomes the left adjacent MB of each MB. For this reason, the processing result of the upper frame MB74 can be fed back to the processing of the lower frame MB75 only for three stages, and the processing result of the left adjacent MB pair can be fed back to the processing of the current MB pair only by three stages. The same limitation as in the fifth embodiment in which four consecutive MB rows are processed as a group occurs.

  Specifically, for example, when I15 MB intra / inter determination is completed in stage 5, J15MB vertical single pixel / horizontal two pixel accuracy motion detection is also ended in stage 1 at the same time. For this reason, there is a limitation that the intra / inter determination result of the upper frame MB74 cannot be reflected in the vertical single pixel / horizontal two-pixel accuracy motion detection of the lower frame MB75. However, as indicated by arrows in FIG. 42, the intra / inter determination result of the upper frame MB74 can be reflected in the stage 2 and subsequent stages. Also, the intra / inter determination results determined in stage 5 for the left adjacent MB pair, the upper adjacent MB pair, the upper right adjacent MB pair, and the like can be reflected in the vertical single pixel / horizontal two pixel accuracy motion detection in stage 1. Therefore, in stage 1, a temporary PMV is created with reference to motion vectors such as the left adjacent MB pair, the upper adjacent MB pair, and the upper right adjacent MB pair, and all four 8 × 8 SBs in the lower frame MB75 are shared. With reference to the provisional PMV, the vertical single pixel / horizontal two-pixel accuracy motion detection is performed. Then, in stage 2, the peripheral area of the candidate motion vector of vertical single pixel / horizontal two pixel precision detected in stage 1 is set in the single pixel precision memory, and a true PMV is created in stage 3, and stage 3 and stage 4 Then, single-pixel accuracy motion detection and quarter-pixel accuracy motion detection are performed with reference to the true PMV.

  FIG. 43 shows the detailed operation of the SB pipeline in stages 3 and 4. Four 8 × 8 SBs included in one MB are serially processed from No. 0 to No. 3. For this reason, in the single-pixel precision motion detection and the quarter-pixel precision motion detection, it is possible to perform motion detection based on a true PMV in consideration of the adjacent 8 × 8 SB motion vector.

  However, in this embodiment, it is necessary to separately evaluate the space direct address. In stage 1, vertical single-pixel / horizontal 2-pixel precision motion detection is performed based on the temporary PMV, and in stage 2, the peripheral area of the candidate motion vector detected in stage 1 is set in the single-pixel precision memory. When the temporary PMV of stage 1 and the true PMV of stage 3 are different, the area where motion is detected in stages 3 and 4 may not include the MB corresponding to the spatial direct address, 8 × 8 SB, or the like. Therefore, it is necessary to perform processing for evaluating the MB of the position corresponding to the spatial direct address and the cost of 8 × 8 SB in parallel with stage 3 and stage 4.

  FIG. 44 shows how each MB pair flows through each stage of the MB pipeline when an MB pair included in 8 MB rows is inserted into one MB pipeline for 4 × 4 intra prediction. Show. FIG. 45 shows the detailed operation of the SB pipeline in stages 3 and 4. Here, in order to satisfy the conditions pointed out in the above paragraphs 0122 and 0123, motion compensation is performed in the stage 6 as in the fifth embodiment. That is, intra prediction is performed for the lower frame MB75 after motion compensation is performed for the upper frame MB74, and intra prediction is performed for the current MB pair after motion compensation is performed for both MBs included in the left adjacent MB pair. In order to satisfy the condition that it is performed, motion compensation is performed at stage 6.

  In addition, in order to adjust fluctuations in the amount of generated code due to variable length coding in stage 7, it may be necessary to perform rate control in stage 8 to change the quantization parameter. In this case, as in the fifth embodiment, the 4 × 4 intra prediction mode is not selected in the intra / inter determination of stage 5, or a new quantum is determined in stage 6 when the 4 × 4 intra prediction mode is selected. It is necessary to redo integer conversion, quantization, inverse quantization, and inverse integer conversion using the quantization parameter.

  Note that the stage configuration of the MB pipeline shown in this embodiment is merely an example, and other stage configurations may be used. If the stage configuration changes, the constraints in motion detection, intra prediction, and the like change even if the number of vertically continuous MB rows encoded as a group is the same.

  In addition, the invention described in the present embodiment and the invention described in the sixth to twelfth embodiments can be simultaneously applied to a moving picture coding apparatus.

  The present invention has a high utility value in the field of hardware moving image encoding devices.

It is explanatory drawing which showed the example of MB thrown continuously into MB pipeline. Example 1 It is a block diagram of the moving image encoder to which the present invention is applied. Example 1 It is a flowchart of a process of the moving image encoding method to which this invention is applied. Example 1 FIG. 2 is an explanatory diagram showing a state in which each MB flows through each stage of the MB pipeline when the four MBs exemplified in FIG. 1 are continuously input into one MB pipeline. Example 1 FIG. 2 is an explanatory diagram showing MB cycle numbers at which the MBs constituting the picture of FIG. 1 are input to the MB pipeline and stage 0 processing is started. Example 1 It is explanatory drawing which showed MB divided | segmented into 4 SB of 8 pixels x 8 pixels. (Example 2) It is explanatory drawing which showed the example of MB thrown continuously into MB pipeline. (Example 2) In order to detect a motion vector for every 8 pixels × 8 pixels SB, when MBs included in 8 MB rows are continuously input to one MB pipeline, each MB flows through each stage. It is explanatory drawing shown. (Example 2) It is a block diagram of a motion vector detection circuit to which the present invention is applied. (Example 2) It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. (Example 2) It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. (Example 3) It is a block diagram of a motion vector detection circuit to which the present invention is applied. (Example 3) It is explanatory drawing which showed MB divided | segmented into 16 4 pixels x 4 pixels SB. (Example 4) An explanatory diagram showing how each MB flows through each stage when MBs included in 8 MB rows are continuously input to one MB pipeline for 4 pixel × 4 pixel intra prediction. is there. (Example 4) It is explanatory drawing which showed the detailed operation | movement of SB pipeline for 4 pixel x 4 pixel intra prediction. (Example 4) When MBs included in four MB rows are continuously input to one MB pipeline in order to detect a motion vector for each 8 × 8 SB, each MB flows through each stage of the MB pipeline. It is explanatory drawing which showed. (Example 5) It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. (Example 5) An explanatory diagram showing how each MB flows through each stage when MBs included in four MB rows are continuously input to one MB pipeline for 4 pixel × 4 pixel intra prediction. is there. (Example 5) It is explanatory drawing which showed the detailed operation | movement of SB pipeline for 4 pixel x 4 pixel intra prediction. (Example 5) It is a block diagram of the moving image encoder which has an internal memory. (Example 6) It is explanatory drawing which showed the reference area | region in the case of processing 4 MB contained in 4 vertical MB lines by one MB pipeline. (Example 6) It is explanatory drawing which showed another method which memorize | stores a reference area in the memory for reference areas. (Example 6) It is explanatory drawing which showed the combination of the motion detection apparatus which can detect a motion in a wide range, and the moving image encoding apparatus containing MB pipeline to which this invention was applied. (Example 7) It is a block diagram of a parallel moving image encoding device having two moving image encoding devices and an internal memory. (Example 8) It is explanatory drawing which showed the area | region of the picture which two moving image encoders share. (Example 8) It is explanatory drawing which showed a mode that MB flows through each stage of MB pipeline contained in two moving image encoders. (Example 8) It is a block diagram of a CABAC encoder that performs rate control based on an estimated generated code amount. Example 9 It is a figure which shows the processing flow of a context information restoration circuit. Example 9 It is a block diagram of a CABAC encoder that performs rate control based on a provisional generated code amount. (Example 10) It is a block diagram of a provisional arithmetic encoding circuit when encoding is performed with a group of four vertically continuous MB rows. (Example 10) It is a figure explaining one of the methods of setting a temporary range, low, and a probability table about the head MB in MB row in a temporary arithmetic coding circuit. (Example 10) It is a block diagram of a provisional arithmetic coding circuit when performing provisional arithmetic coding in two parallel. (Example 10) It is explanatory drawing which showed a mode that MB flows through each stage of MB pipeline, when provisional arithmetic coding is performed in 2 parallel. (Example 10) It is a block diagram of the CABAC encoder in the case of compressing by CAVLC. (Example 11) FIG. 2 is a block diagram of a parallel moving image encoding device that outputs a binary symbol from a parallel moving image encoding device configured by two moving image encoding devices and generates one bit stream encoded by CABAC. is there. (Example 12) Block diagram of a parallel video encoding device that outputs a CAVLC encoded bitstream from a parallel video encoding device composed of two video encoding devices and generates one bitstream encoded by CABAC It is. (Example 12) It is explanatory drawing which showed MB pair when a frame encoding is selected, and MB pair when field encoding is selected. (Example 13) Field coding is selected for the current MB pair, frame coding is selected for the left adjacent MB pair when the frame coding is selected for the left adjacent MB pair, frame coding is selected for the current MB pair, and field coding is performed for the left adjacent MB pair It is explanatory drawing which showed the left adjacent MB pair when is selected. (Example 13) It is explanatory drawing which showed the example of the MB pair thrown into MB pipeline. (Example 13) It is a block diagram of the moving image encoder to which the present invention is applied. (Example 13) It is a flowchart of a process of the moving image encoding method to which this invention is applied. (Example 13) In order to detect a motion vector for every 8 pixels × 8 pixels SB, when MB pairs included in 8 MB rows are input to one MB pipeline, each MB pair flows through each stage. FIG. (Example 13) It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. (Example 13) When MB pairs included in 8 MB rows are input to one MB pipeline for 4 pixel × 4 pixel intra prediction, each MB pair flows through each stage of the MB pipeline. It is explanatory drawing. (Example 13) It is explanatory drawing which showed the detailed operation | movement of SB pipeline for 4 pixel x 4 pixel intra prediction. (Example 13) It is a block diagram of an MPEG-2 encoder. (Background technology) H. 2 is a block diagram of an H.264 encoder. FIG. (Background technology) Conventional H.264. 2 is an explanatory diagram showing a state in which an MB flows through each stage of the MB pipeline in an MB pipeline of an H.264 encoder. FIG. (Background technology) It is the block diagram which showed the structure of the conventional motion vector detector. (Background technology) It is explanatory drawing which showed the example of the reference area of two adjacent MB. (Background technology) It is explanatory drawing which showed the picture division | segmentation method. (Background technology) It is a block diagram of a CABAC encoder. (Background technology) It is a Biari_encode_symbol processing flow showing the processing contents of the arithmetic encoding circuit. (Background technology) It is explanatory drawing which showed the change of range and low when symbol is LPS, and the change of range and low when symbol is MPS. (Background technology)

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 MB pipeline 2 MB selection circuit 3 Encoding data setting circuit 4 External memory 5 Two pixel precision image creation circuit 6 Two pixel precision memory 7 Two pixel precision motion detection circuit 8 Single pixel precision memory 9 Single pixel precision motion detection circuit 10 1 / 4 pixel precision image creation circuit 11 ¼ pixel precision motion detection circuit 12 motion vector detection circuit 13 motion vector detection circuit 14 reference area memory 15 buffer memory 16 motion detection unit 17 motion compensation unit 18 intra prediction unit 19 integer conversion / Quantization / inverse quantization / inverse integer transform unit 20 variable length coding unit 21 deblocking filter unit 22 control unit 23 moving image coding device 24, 25, 26, 27 MB
28, 29, 30, 31 Reference area 32, 33, 34, 35 Update area 36 Rectangular area including all reference areas 28, 29, 30, 34 34 Update area 38 Motion detector 39 Internal memory 40, 41 Video code 42 parallel video coding device 43 binarization circuit 44 generated code amount estimation circuit 45 binarized symbol storage circuit 46 arithmetic coding circuit 47 context information restoration circuit 48 temporary arithmetic coding circuit 49 in temporary arithmetic coding circuit 50 The arithmetic coding part in the provisional arithmetic coding circuit 51 to 54 The part storing the range in the temporary arithmetic coding circuit 55 to 58 The part storing the low in the temporary arithmetic coding circuit 59 to 62 Probability table 63, 65 in arithmetic coding circuit Memory 64, 66 for holding 2 MB cycles of binarized symbols Provisional arithmetic coding circuit 69 CAVLC encoding circuit 70 CAVLC coded bit stream storage circuit 71 CAVLC decoding circuit 72 the top field MB performing temporary arithmetic coding in the memory 67 the selector 68 2 parallel to the to 2MB cycle held Kussudeta
73 Bottom Field MB
74 Upper frame MB
75 Lower frame MB
76 Upper frame MB odd line 77 Lower frame MB odd line 78 Top field MB upper half 16 lines 79 Bottom field MB upper half 16 lines 80 MB pipeline 81 MB pair selection circuit 82 Field MB pair / frame MB pair configuration circuit 83 Coding Data setting circuit 84 DCT
85 Quantization 86 Inverse quantization 87 Inverse DCT
88 frame memory 89 motion compensation 90 motion detection 91 variable length coding 92 rate control 93 integer transform 94 quantization 95 inverse quantization 96 inverse integer transform 97 intra prediction 98 deblocking filter 99 frame memory 100 motion compensation 101 motion detection 102 variable length Encoding 103 Rate control 104 Internal memory 105 Motion detection circuit 106 Motion detector 107, 108 MB
109, 110 Reference area 111 Update area 112 Binary circuit 113 Context calculation circuit 114 Arithmetic coding circuit 115 Probability table

In the first MB in the MB row, arithmetic coding is performed by taking over the range, low, and probability table when the arithmetic coding of the last MB in the adjacent MB row is finished. However, in the present invention, since encoding is performed with a plurality of vertically continuous MB rows as a group, when the first MB in the MB row is encoded, the last MB in the upper adjacent MB row has been encoded. Absent. For this reason, until encoding of a group of consecutive MBs is completed, the arithmetic is performed by taking over the range, low, and probability table when arithmetic coding of the last MB in the adjacent MB row is completed. Encoding cannot be performed. Thus, for the first MB in the MB row, provisional values are set in the range, low, and probability tables, and provisional arithmetic coding is started.

The present invention relates to a moving image encoding apparatus and a moving image encoding method for compressing and reducing the amount of moving image data.

  Some moving images are composed of frames such as 720P and others are composed of fields such as 1080i, but the present invention is composed of fields even when applied to those composed of frames. Even if it is applied to things, the purpose can be achieved, and similar actions and effects are produced. Therefore, in the following, the term “picture” is used to indicate one image constituting a moving image. A picture shall represent both a frame and a field.

  MPEG-2 and H.264 In a moving image coding system such as H.264 (also referred to as MPEG-4 AVC) or VC-1, the screen is divided into macroblocks (hereinafter referred to as MB) of 16 pixels × 16 pixels or 8 pixels × 8 pixels. Motion detection, motion compensation, frequency conversion such as DCT or integer conversion, quantization, variable length coding, and the like are performed in MB units. In these moving image encoding systems, in order to improve the encoding efficiency, the result of the encoded peripheral MB is fed back, and adaptive processing for determining the processing content of the MB being encoded (hereinafter referred to as the current MB). Is done.

  FIG. 46 shows a block diagram of an MPEG-2 encoder. In MPEG-2, in order to prevent the generated code amount from fluctuating locally, the generated code amount is controlled by changing the quantization parameter of each MB. This is called rate control 92. In the rate control 92, the quantization parameter of the current MB is changed according to the code amount of the encoded MB generated as a result of the variable length encoding 91. For example, when the generated code amount of the left adjacent MB is large, the quantization parameter of the current MB is increased and the generated code amount of the current MB is decreased.

  FIG. 2 shows a block diagram of an H.264 encoder. H. H.264 can compress moving images with twice the efficiency of MPEG-2. In order to achieve this high compression efficiency, H.264 employs an increase in the number of reference pictures in motion compensation 100, variable block size motion compensation, and frequency conversion in units of 4 pixels × 4 pixels (hereinafter referred to as integer conversion 93), and in addition to rate control 103, The three adaptive processes are newly introduced. Here, variable block size motion compensation means that MB is divided into sub-blocks (hereinafter referred to as SB) such as 8 pixels × 16 pixels, 8 pixels × 8 pixels, 4 pixels × 4 pixels, etc., and motion compensation is performed for each SB. To do. Hereinafter, when simply referring to a block, both MB and SB are meant.

(1) Considering not only the code amount of a prediction error but also the code amount of a reference picture and the code amount of a motion vector, a combination of a reference picture, a motion vector, and a block that minimizes the total code amount thereof is selected. For the motion vector, a predicted motion vector (hereinafter referred to as PMV) predicted from the motion vectors of the left adjacent block, the upper adjacent block, and the upper right adjacent block is obtained, and the difference between the motion vector of the PMV and the current block is encoded. . In addition, the motion vectors of the left adjacent block, the upper adjacent block, and the upper right adjacent block are also referred to when obtaining the motion vector in the spatial direct mode.

(2) A prediction value is created from the reproduced pixels of the adjacent MB, and intra prediction 97 is performed. As for the luminance signal, there are an intra prediction mode of 4 pixels × 4 pixels SB (hereinafter referred to as 4 × 4 SB) and an intra prediction mode of 16 × 16 pixel blocks. In the 4 × 4SB intra prediction mode of the luminance signal, an intra prediction value of the current 4 × 4SB is generated from the reproduction pixels of the left adjacent 4 × 4SB, the upper adjacent 4 × 4SB, and the upper right adjacent 4 × 4SB, and this intra predicted value And the difference between the current 4 × 4 SB pixels. The difference pixel is subjected to integer transformation / quantization / inverse quantization / inverse integer transformation, and an intra prediction value is added to the result, thereby generating a reproduction pixel. In the 16 × 16 pixel block intra prediction mode, an intra prediction value of the current 16 × 16 pixel block is created from the reproduction pixels of the left adjacent MB and the upper adjacent MB, and the difference between the intra prediction value and the pixel of the current 16 × 16 pixel block is created. Is encoded.

(3) A deblocking filter 98 is applied to the boundary of the 4 pixel × 4 pixel block to remove block noise.

  On the other hand, a hardware moving image encoding apparatus employs a pipeline in MB units (referred to as a macroblock pipeline or an MB pipeline in the claims and the specification) in order to increase the operation rate of a circuit. (For example, see Non-Patent Document 1, pp. 126-128, FIG. 3). Further, in a hardware moving image encoding apparatus, encoding is generally performed from the left end MB toward the right end MB in a horizontal MB array (hereinafter referred to as MB row). When the encoding of the rightmost MB included in one MB row is completed, the MBs belonging to the lower MB row are encoded from the left end MB toward the right end MB. This is repeated from the top MB row of the picture to the bottom MB row of the picture. FIG. In the MB pipeline of the H.264 encoder, the manner in which the MB flows through each stage of the MB pipeline is shown. n-1, n, and n + 1 represent the (n-1) th, nth, and n + 1th MBs from the left in a certain MB row, respectively. The MB cycle is a unit of time required for processing one MB in each stage of the MB pipeline. In each stage, one MB is processed in one MB cycle.

  In FIG. 48, data necessary for encoding one MB is read out from the external memory over 1 MB cycle in stage 0. Stage 1 performs coarse motion detection for one MB over 1 MB cycle. Detailed motion detection is performed in a narrow range around the coarse motion candidate motion vector detected in stage 1 over 1 MB cycle in stage 2 and intra prediction is performed. In stage 3, integer conversion, quantization, inverse quantization, and inverse integer conversion are performed on one MB over 1 MB cycle. In stage 4, variable length coding and motion compensation are performed for one MB over one MB cycle. Then, stage 1 performs rate control and deblocking filtering on one MB over 1 MB cycle, and stage 6 writes a reproduction pixel or bit stream for one MB over 1 MB cycle to the external memory. Assuming that the MB cycle is 1000 cycles, the encoding process of one MB takes 7000 cycles when the MB pipeline is not performed, whereas the MB pipeline in FIG. 48 has one MB code every 1000 cycles. The conversion ends.

  However, in order to perform adaptive processing according to the processing result of the left adjacent MB in this MB pipeline, there are the following four problems.

(1) As shown in FIG. 48, when the rate control is finished for the (n-1) th MB, the quantization has already been finished up to the (n + 1) th MB. Therefore, the quantization parameter can be changed according to the generated code amount of the (n−1) th MB for the (n + 2) th and subsequent MBs. That is, 2 MB rate control is delayed.

(2) In FIG. 48, motion detection is executed in two stages of the MB pipeline, and intra prediction is executed simultaneously with high-precision motion vector detection in stage 2. An accurate PMV cannot be calculated unless a high-precision motion vector is determined and intra prediction or inter prediction is determined, but a high-precision motion vector and intra prediction or inter prediction are determined for the (n-1) th MB. Sometimes the motion vector detection with coarse accuracy in the stage 1 is finished for the nth MB. For this reason, in the MB pipeline of FIG. 48, a motion vector cannot be selected considering only a rough accuracy motion vector detection result for the left adjacent MB, and a highly accurate motion vector and intra prediction / A motion vector cannot be selected in consideration of the inter prediction determination result. In addition, H.C. In H.264, it is desirable to perform motion vector detection in consideration of the code amount of the motion vector even in the SB of 8 pixels × 16 pixels, 8 pixels × 8 pixels, 4 pixels × 4 pixels, and the like obtained by dividing the MB. In a pipeline, it is difficult to perform motion vector detection in consideration of the code amounts of the motion vectors of the left adjacent SB, the upper adjacent SB, and the upper right adjacent SB.

(3) The 16 pixel × 16 pixel block intra prediction mode is determined by evaluating the pixel value of the left adjacent 16 pixel × 16 pixel block and the upper adjacent 16 pixel × 16 pixel block and the pixel value of the current block. Here, it is desirable that the pixels of the adjacent 16 pixel × 16 pixel block used for evaluation are reproduction pixels. The reproduced pixel is obtained by performing integer conversion, quantization, inverse quantization, and inverse integer conversion on a prediction error obtained by intra prediction or inter prediction, and adding a prediction value. However, in the MB pipeline of FIG. 48, when the integer conversion of the (n−1) th MB is completed in stage 3 and the reproduction pixel of the (n−1) th MB is created by the motion compensation in stage 4, the n in the stage 2 The intra prediction of the second MB has been completed. For this reason, when the prediction mode determination of 16 pixel × 16 pixel block intra prediction is performed for the nth MB, the reproduction pixel of the (n−1) th MB adjacent to the left cannot be used. For this reason, in the prediction mode determination of the (n−1) th 16-pixel × 16-pixel block adjacent to the left, the pixel of the current picture being encoded must be used instead of the reproduction pixel. Similarly, H.M. In the 4 pixel × 4 pixel block intra prediction mode in H.264, it is desirable to determine the prediction mode using the reproduction pixels of the 4 pixels × 4 pixel block of the left adjacent, upper adjacent, and upper right adjacent, but the MB pipeline of FIG. In this case, it is impossible to use the pixels of the current picture being encoded as a suboptimal measure. For this reason, the intra prediction mode determined by the MB pipeline in FIG. 48 is suboptimal.

(4) Since the same intra prediction value must be used in the encoding device and the decoding device, it must be generated from the reproduced pixels. On the other hand, in the P picture and the B picture, when the n−1th MB adjacent to the left is determined to be in the inter prediction mode, a reproduced pixel cannot be obtained until the motion compensation is completed. However, in the MB pipeline of FIG. 48, the generation of the reproduction pixel in the stage 4 for the (n-1) th MB, the integer conversion in the stage 3 for the nth MB, and the like are executed simultaneously. For this reason, in the P picture and the B picture, it is difficult to create a prediction value using the reproduction pixel of the (n-1) th MB and perform integer conversion of the nth MB.

Also, since the amount of picture data is enormous, the picture is generally stored in a memory external to the moving picture coding apparatus. FIG. 49 is a block diagram showing a configuration of a conventional moving image encoding device (see, for example, p2009 of Non-Patent Document 2 and FIG. 3). The motion vector detector 106 includes an internal memory 104 and a motion detection circuit 105. In the conventional motion vector detector 106, a region where a motion vector may exist (hereinafter referred to as a reference region) is copied from the external memory 4 of the motion vector detector 106 to the internal memory 104 of the motion vector detector 106, An operation for detecting a motion vector is performed on the pixels in the internal memory 104. FIG. 50 shows an example of reference areas 109 and 110 of two adjacent MBs 107 and 108. The reference area 110 of the MB 108 substantially overlaps the reference area 109 of the MB 107, and the width of the non-overlapping area matches the number of pixels in the horizontal direction of the MB. The reference area 110 is only shifted from the reference area 109 to the right by the number of pixels in the horizontal direction of MB. Hereinafter, this shifted area is referred to as an update area 111. The conventional motion vector detector 106 uses this property to reduce the pixel transfer amount between the external memory 4 and the internal memory 104 of the motion vector detector 106. When only the update area 111 is transferred from the external memory 4 to the internal memory 104 and motion detection of the MB 108 is started, the origin of the reference area is changed from (0, 0) of the reference area 109 to (0, 0) ′ of the reference area 110. Replace with. As described above, the transfer amount from the external memory 4 to the internal memory 104 is reduced by reading only the update area 111 and updating the reference area. That is, conventionally, the amount of pixel transfer between the external memory 4 and the internal memory 104 has been reduced by utilizing the horizontal overlap of the reference areas. As a result, the conventional one-chip MPEG-2 codec LSI can encode a 1080i moving image by attaching two 32-bit DDR-SDRAMs operating at 200 MHz (for example, (See Non-Patent Document 3 , pp.12-13).

However, the size of one picture according to the digital cinema standard defined by DCI (Digital Cinema Initiative) is 4096 pixels × 2160 pixels, which is four times as many as 1080i (1920 pixels × 1080 pixels). In addition, a CMOS sensor has been developed that outputs a picture having a number of pixels exceeding three times that of 1080i at a rate twice that of 1080i (60 pictures / second) (for example, see section 27.1 of Non-Patent Document 4 ). If the latest technology is used, it is predicted that a video camera that outputs a moving image of a digital cinema class can be realized. If the conventional technology is used, is a moving image encoding LSI provided with a 256-bit width input terminal? There is a problem that it is necessary to operate the external memory at a speed of about 800 MHz.

In addition, when the processing speed of the video encoding device is slow and the processing cannot be performed in time with one unit, parallel processing by a plurality of video encoding devices is performed. As the picture dividing method in this case, four types shown in FIG. 51 can be considered (for example, p.17 of Non-Patent Document 5, FIG.
1). That is, (a) horizontal division, (b) vertical division, (c) vertical and horizontal division, and (d) slice interleaving. Conventionally, (a) horizontal division has been often employed because connection and data transfer between a plurality of image encoding devices are simplified (see, for example, Non-Patent Document 3 ). However, the method of performing horizontal division and performing parallel processing with a plurality of moving image encoding apparatuses has a problem that the start of encoding is delayed until all pixels for one picture are read from the camera. The occurrence of this delay is undesirable in communication applications that require low delay such as videophones and remote monitoring.

H. In H.264, variable length coding can be performed by CABAC, which is a kind of arithmetic coding. FIG. 52 shows a block diagram of a CABAC encoder. In CABAC, first, multi-value syntax data (referred to as syntax data in H.264 but referred to as syntax data in this specification and claims) is converted to variable length 2 by a binarization circuit 112. Convert to a valued symbol. Next, the context calculation circuit 113 performs binarization symbol 1
Context calculation is performed for each bit (hereinafter referred to as symbol). Then, the arithmetic coding probability table 115 is selected according to the calculated context information, the symbol is arithmetically encoded by the arithmetic encoding circuit 114, the probability table 115 is updated, and the arithmetic symbol of the next symbol is calculated. The updated probability table 115 is used for conversion.

FIG. 53 is a Biari_encode_symbol processing flow showing the processing contents of the arithmetic encoding circuit 114 (see Non-Patent Document 6 , for example). In CABAC, the coding state is represented by the width (hereinafter referred to as “range”) and the lower end (hereinafter referred to as “low”) of the existence sequence of the symbol sequence. The bit widths of range and low are 9 bits and 10 bits, respectively, and at the head of the slice, range is initialized to 0x1FE (0x indicates a hexadecimal number, the same applies hereinafter), and low is initialized to 0x000. Of 0 and 1, the lower occurrence probability is assigned to LPS (Least Probable Symbol), and the higher occurrence probability is assigned to MPS (Most Probable Symbol). The probability table 115 holds the occurrence probability of symbol for each context information. Each occurrence probability is indicated by information indicating whether 0 or 1 is assigned to the MPS and a number (hereinafter referred to as “state”) indicating an LPS occurrence probability (hereinafter referred to as “rLPS”). And a range by subtracting a table (rLPS_table_64x4) (step S30). On the other hand, the occurrence probability of MPS is calculated by subtracting rLPS from range (step S31).

  FIG. 54A shows changes in range and low when symbol is LPS. When symbol is LPS, low is updated by adding the occurrence probability of MPS, and rLPS is set in range (step S33). If state = 0, the occurrence probability of 0 and 1 in the next coding state is expected to be reversed, so 0 and 1 corresponding to MPS are switched (steps S34 and S35). Then, the state of the state is updated by drawing the transition table (AC_next_state_LPS_64) corresponding to the LPS (step S36).

  FIG. 54B shows changes in range and low when symbol is MPS. When symbol is MPS, low does not change, and MPS occurrence probability is set in range (step S31). Then, the state of the state is updated by drawing the transition table (AC_next_state_MPS_64) corresponding to the MPS (step S37).

  When range <0x100, renormalization is performed, and range and low are expanded (step S38). Due to renormalization, the range of range becomes 0x100 ≦ range ≦ 0x1FE.

  In this way, arithmetic coding must be performed for each symbol, and only one probability table 115 can be used in one slice despite the number of processing cycles. For this reason, the arithmetic encoder 114 cannot be parallelized in principle.

  Furthermore, a larger amount of code is allocated to the I picture than to the P picture and the B picture. For this reason, the amount of occurrence of binarized symbols in the I picture is larger than the average amount of binarized symbols in the entire picture. In addition, when the generated code amount suddenly increases, the generated code amount for each MB needs to be reflected in the rate control at an early stage in order to prevent the rate control from failing. In this way, the arithmetic coding circuit 114 is able to reduce the amount of binarized symbols generated suddenly in an I picture that cannot be parallelized and has a large amount of generated codes, despite the number of processing cycles. There is a problem that performance sufficient for processing is required.

The method for estimating the amount of code generated arithmetic coding has been proposed from the actual arithmetic coding sum of symbol length of the binary symbols without regard to CABAC (e.g., p.20 Non-Patent Document 7 FIG. 6). Specifically, binarized symbol length N
The generated code amount R = α × N is estimated using the ratio α between the binary symbol length of the immediately previous encoded picture and the generated code amount. Since it is possible to convert multi-level syntax data into binarized symbols with 1 CK using a lookup table, this method can be used to estimate the amount of code generated by arithmetic coding at 1 CK per syntax data. It can be carried out.
Hiroaki Watanabe, Hideaki Chaki, “Developed codec core for mobile devices compatible with both H.264 and MPEG-4, encoded VGA video with 36 mA”, Nikkei Electronics, September 27, 2004, no.883, pp123-133. Toshihiro Minami and Jiro Naganuma, “Proposal of Motion Vector Detector Configuration Suitable for Telescopic Search”, Theory of Science (D-2), vol.J87-D-2, no.11, pp.2007-2024, Nov .2004. Jiro Naganuma, Hiroe Iwasaki, Takahiro Nitta, Ken Nakamura, Ken Yoshidome, Mitsuo Ogura, Yasuyuki Nakajima, Yutaka Tashiro, Takayuki Onishi, Mitsuru Ikeda, Makoto Endo, Yoshiyuki Yashima, “One-chip HDTV MPEG-2 CODEC LSI Configuration Technology— VASA ", NTT Technical Journal, vol.15, No.9, pp.12-15, September 2003. S. Yoshihara, M. Kikuchi, Y. Ito, Y. Inada, S. Kuramochi, H. Wakabayashi, M. Okano, K. Koseki, H. Kuriyama, J. Inutsuka, A. Tajima, T. Nakajima, Y. Kudoh, F. Koga, Y. Kasagi, S. Watanabe, T. Nomoto, “A 1 / 1.8-inch 6.4MPixel 60frames / s CMOS Image Sensor with Seamless Mode Change”, ISSCC2006, session 27.1, Feb. 2006. Satoshi Nogaki, “Examination of coding control method in MPEG-2 MP @ HL encoder”, PCSJ98, P-2.04, pp.17-18, October 1998. H.264 / AVC reference software (JM 10.2). Ikuo Yamakage, Takeshi Nakajo, Shinichiro Furuto, “Video coding technology for HD DVD”, Toshiba Review, vol.60, No.1, pp.17-29, 2005.

  In view of the above circumstances, an object of the present invention is to enable feedback of the processing result of an adjacent block to the processing of a block being encoded in a moving picture encoding apparatus having an MB pipeline structure. Specifically, rate control of the current MB according to the generated code amount of the left adjacent MB, detection of the motion vector of the block being encoded in consideration of the code amount of the motion vector of the adjacent block, left adjacent Detecting the motion vector of the block being encoded in consideration of the MB intra prediction / inter prediction determination result, determining the intra prediction mode using the reproduced pixels of the adjacent block, and left P picture and B picture An object of the present invention is to perform intra prediction by creating a prediction value using a reproduction pixel of an adjacent MB.

  It is another object of the present invention to provide a moving picture coding apparatus that can reduce the amount of data transferred from an external memory that stores pictures to the internal memory of the moving picture coding apparatus.

Furthermore, an object of the present invention is to provide an image encoding apparatus that has a small delay required for encoding even when operating in parallel .

  It is another object of the present invention to enable arithmetic coding even when the arithmetic coding circuit does not have a performance sufficient to handle a binarized symbol generation amount that suddenly increases.

The moving picture coding apparatus according to the present invention is a moving picture coding apparatus that performs coding with a group of at least two macroblock rows that are continuous vertically, and performs a plurality of stages of moving picture coding processing. When the macroblock included in the uppermost macroblock row in the group of macroblock rows is input to the macroblock pipeline, the macroblock pipe A macroblock included in a lower adjacent macroblock row one line after a line starts encoding the macroblock and starts encoding a macroblock included in the upper adjacent macroblock row And one macroblock in the left direction from the macroblock included in the macroblock row adjacent above. The macroblocks that are separated from each other are input to the macroblock pipeline, and the macroblock pipeline starts encoding the macroblock. It repeats from the macroblock line to the bottom macroblock line.

In addition, the moving picture coding apparatus according to the present invention includes a macroblock pipeline that performs moving picture coding divided into a plurality of stages, and a macroblock of m rows (m is an integer of 2 or more) that are vertically continuous. For a macroblock belonging to the uppermost macroblock row in the group of macroblock rows, a macroblock for which the lower left adjacent macroblock has not yet been selected is selected, and the batch is selected. For macroblocks belonging to a macroblock row other than the topmost macroblock row, a macroblock included in the group of macroblock rows is selected by selecting a macroblock in which the upper right adjacent macroblock is already selected. and the macro block selector for selecting one per macroblock row block, the Ma For each macroblock selected by the B block selection unit, wherein the uppermost macroblocks belonging to the macro block rows in the macroblock row of batches and to the macroblock belonging to the macroblock line of the lowest, m macro An encoded data setting unit for setting data necessary for encoding individual macroblocks in each macroblock cycle in the macroblock pipeline in order over the block cycle, and setting the encoded data When the data necessary for encoding the macroblock is set by the unit, the macroblock pipeline starts encoding the macroblock, and the group of macroblock rows is started by the encoded data setting unit . Mac belonging to the bottom macroblock line in If the data needed to encode the block is set to the macro block pipeline, newly selects the right neighboring macroblocks for each individual macroblock the macroblock selector is the selected .

In addition, the moving picture encoding apparatus of the present invention collects macro block rows of 2n rows (where n is an integer equal to or greater than 2) as a group, and combines two adjacent macro blocks vertically to form a macro block pair. Is a moving picture coding apparatus that performs coding by switching between field coding and frame coding for each macroblock pair, and has a macroblock pipeline that divides moving picture coding processing into a plurality of stages. When one macroblock included in a macroblock pair belonging to the top two macroblock rows in the group of macroblock rows is input to the macroblock pipeline, the macroblock pipe The line starts encoding one of the macroblocks, and the macros belonging to the two adjacent macroblock rows are One macroblock included in a macroblock pair belonging to two adjacent macroblock rows after one macroblock cycle after the encoding of one of the macroblocks included in the block pair is started, One macroblock included in a macroblock pair that is one macroblock or more leftward from the macroblock pair belonging to the two adjacent macroblock rows is input to the macroblock pipeline, and the macroblock When the pipeline starts to encode one of the macroblocks, the macros in the two rows located at the bottom of the third and fourth macroblock rows from the top of the group of macroblock rows. It repeats to the block line, and is positioned at the top of the group of macro block lines. When the other one of the macroblocks that is not first encoded among the macroblocks included in the macroblock pair belonging to the two macroblock rows is input to the macroblock pipeline, the macroblock pipe One line after the line starts encoding the other one macroblock and the other one macroblock included in the macroblock pair belonging to the two adjacent macroblock rows starts encoding. After a block cycle, a macroblock pair belonging to the two adjacent macroblock rows below the macroblock pair belonging to the two adjacent macroblock rows above is left one macroblock or more in the left direction Encoding is not started first among the macroblocks included in the macroblock pair The other macroblock is put into the macroblock pipeline, and the macroblock pipeline starts encoding the other macroblock. Are repeated from the third and fourth macroblock rows to the two lowermost macroblock rows.

The moving picture encoding apparatus of the present invention, a macroblock pair combines the two macro blocks adjacent to the vertical, video encoding for encoding by switching the field coding and frame coding for each macroblock pair A macroblock pipeline that divides a video encoding process into a plurality of stages, and 2n rows (n is an integer greater than or equal to 2) macroblock rows as a group, wherein the most located on belonging to the macroblock line of the two rows macroblock pair in a macroblock line of human settlement, select the macroblock pair is the lower left neighboring macroblock pair is not already selected, and the macroblock pair belonging to the uppermost macroblock lines except two rows located in the macroblock row of human unity For, by selecting a macroblock pair that upper-right neighboring macroblock pair is already selected, the selection macroblock pair to select one per 2 macro block row macroblock pairs in the macroblocks line of human unity and parts, the macroblock pair selected by the macroblock pair selection unit, when the field coding is selected, the odd lines of the macroblock pair constitutes a top field macroblock, even lines of the macroblock pair When a bottom field macroblock is configured and frame coding is selected, the upper frame macroblock is configured with the upper half line of the macroblock pair, and the lower frame macroblock is configured with the lower half line of the macroblock pair. feel that A macroblock pair frame macroblock pair forming unit, the macroblock pair selected by the macroblock pair selection unit, when the field coding is selected, first, in a macro-block row of the human unity most from the macroblock pairs belonging to the macroblock row of two rows located above, until macroblock pair most belonging to the macroblock row of two rows located below in the macroblock row of the human unity, n macroblocks cycle In order, the data necessary for encoding one of the macroblocks included in each macroblock pair for each macroblock cycle is set in the macroblock pipeline , and the batch of The top two macros in the macro block line From the macroblock pairs belonging to the block row, until said macroblock pair most belonging to the macroblock row of two rows located below in the macroblock row of batches and sequentially over n macroblocks cycle, one macro block cycle For each of the macroblocks included in each macroblock pair, data necessary to encode the other one that was not initially set is set in the macroblock pipeline , while frame encoding is selected. when the initially from said uppermost belonging to the macroblock row of two rows located macroblock pair in the macroblock row of batches and located at the bottom in the macro-block row of the human unity until macroblock pairs belonging to the macro block rows in two rows, n macroblocks cycle Sequentially over, set the data required to encode the upper frame macroblocks contained in each macroblock pair for each macroblock cycle the macroblock pipeline, the second time, the human unity From the macroblock pair belonging to the two macroblock rows located at the top in the macroblock row, the macroblock pair belonging to the two macroblock rows located at the bottom in the group of macroblock rows until, in turn over n macroblocks cycle, the coded data for setting data required for encoding the lower frame macroblocks contained in each macroblock pair for each macroblock cycle the macroblock pipeline and a setting unit, macro by the encoded data setting unit When the data is set needed to encode the block, the macro block pipeline starts encoding of the macroblock by the encoded data setting unit, the second time, the macro of the human unity When data necessary for encoding a macroblock included in a macroblock pair belonging to the two macroblock rows located at the bottom of the block row is set in the macroblock pipeline , the macroblock The pair selection unit newly selects the right adjacent macroblock pair for each of the selected individual macroblock pairs.

Preferably, in the moving picture coding apparatus according to the present invention, at some stages in the macroblock pipeline , the macroblock cycle is divided into subblock cycles to form a subblock pipeline , and processing for each subblock is performed. Done.

Preferably, the moving picture encoding apparatus of the present invention has an internal memory for storing all data necessary for encoding one macroblock included in the group of macroblock rows for each macroblock row. Have.

Preferably, the video encoding apparatus of the present invention is a video encoding apparatus including the moving picture coding apparatus of two even less as described in any one of claims 1 to 6 Thus, a picture is divided into a group of macroblock rows, and each moving image encoding device is assigned to each of the divided divided areas, and at least two vertically divided divided areas are provided . It is encoded in parallel by the moving picture coding apparatus.

Preferably, the moving picture coding apparatus according to the present invention includes a binarization unit that converts syntax data into a binarized symbol, a binarized symbol storage unit that stores the binarized symbol, and the binarization. A context information restoration unit that restores context information corresponding to the binarized symbol read from the symbol storage unit; and a context in which the binary symbol read from the binarized symbol storage unit is restored by the context information restoration unit An arithmetic encoding unit that performs arithmetic encoding based on the information .

Preferably, the moving image encoding apparatus of the present invention includes a provisional arithmetic encoding unit that calculates provisionally generated code amount by provisionally arithmetically encoding the binarized symbols converted by the binarization unit .

Preferably, the moving image encoding apparatus of the present invention includes a syntax data compression unit that reversibly compresses syntax data, and a compression syntax data storage unit that stores syntax data compressed by the syntax data compression unit. , the compressed syntax data from the storage unit and the syntax data restoration unit for restoring the syntax data by reading the syntax data said compressed, the syntax data recovery restored the syntax data binary by section has a binarization unit for converting the encoded symbols, the arithmetic coding unit arithmetically codes the binary symbol which has been converted by the binarization unit.

Preferably, in the moving image encoding apparatus of the present invention, the syntax data compression unit compresses the syntax data by CAVLC.

In addition, the moving picture coding method of the present invention includes a macroblock pipeline that performs a moving picture coding process divided into a plurality of stages, and collects at least two macroblock rows that are vertically continuous. As a moving picture coding method in a moving picture coding apparatus that performs coding as described above, a macroblock included in an uppermost macroblock row in the group of macroblock rows is input to the macroblock pipeline. Then, the macroblock pipeline starts encoding the corresponding macroblock, and the macroblock included in the macroblock row adjacent above is started one macroblock cycle after the start of encoding of the macroblock. Macroblocks included in a macroblock row that are included in the adjacent macroblock row above When a macroblock that is one macroblock or more leftward from the macroblock is input to the macroblock pipeline, and the macroblock pipeline starts encoding the macroblock, the group of macroblock rows Repeating from the second macroblock row from the top to the bottommost macroblock row.

A moving picture coding method according to the present invention is a moving picture coding method in a moving picture coding apparatus including a macroblock pipeline that performs a moving picture coding process divided into a plurality of stages. A macro block row of continuous m rows (m is an integer of 2 or more) is grouped, and for the macro block belonging to the uppermost macro block row in the group of macro block rows, the lower left adjacent macro block is is still selected macroblock is not selected, and the for a macroblock most belonging to the macroblock row other than the above in the macroblock row of human unity, it macroblock upper right neighboring macroblocks have already been selected by Rukoto selected, macroblock 1 macros in the macro block row of the human unity A first step that will be selected per one row lock, for each macroblock selected by the first step, from the macroblock belonging to the most macroblock row of the upper in a macroblock row of the human unity until the macro blocks belonging to a macroblock line of the lowest, in order, over a period of m macro block cycle, the data required to encode each macroblock for each macroblock cycle is set to the macro block pipeline And a third step in which the macroblock pipeline starts encoding the macroblock when the data necessary for encoding the macroblock is set by the second step. If, macroblock of the human unity by said second step When data necessary for most coding the macroblocks belonging to the macroblock line below in the click row is set to the macro block pipeline, it returns to the first step, being said selected and a fourth step of the right adjacent macroblock for each individual macroblock Ru newly selected.

In addition, the moving picture coding method of the present invention includes a macroblock pipeline in which the moving picture coding process is divided into a plurality of stages. A moving picture code in a moving picture coding apparatus that performs coding by switching between field coding and frame coding for each macro block pair by combining two macro blocks vertically adjacent to each other as a group of block rows. A macroblock included in a macroblock pair belonging to the top two macroblock rows of the group of macroblock rows is input to the macroblock pipeline, The macroblock pipeline starts encoding one of the macroblocks and adjacent to the top One of the macroblocks included in the macroblock pair belonging to the macroblock row of the row, and one of the macroblocks included in the macroblock pair belonging to the two adjacent macroblock rows after one macroblock cycle from the start of encoding One macroblock included in a macroblock pair that is one macroblock or more leftward from a macroblock pair belonging to two macroblock rows adjacent to each other on the macroblock pipeline. When the macroblock pipeline starts encoding one of the macroblocks, the third and fourth macroblock rows from the top of the group of macroblock rows are the lowest. Repeating the steps up to two macroblock rows located at The other one of the macroblocks not encoded first among the macroblocks included in the macroblock pair belonging to the top two macroblock rows in the group of macroblock rows is the macroblock. A step of starting the encoding of the other macroblock by the macroblock pipeline, and one macroblock included in a macroblock pair belonging to the two adjacent macroblock rows; After one macroblock cycle from the start of encoding, a macroblock pair belonging to the two adjacent macroblock rows below the macroblock pair belonging to the two adjacent macroblock rows above Included in the macroblock pair that is one macroblock or more away in the left direction The other one of the macroblocks that has not been encoded first is input to the macroblock pipeline, and the macroblock pipeline starts to encode the other macroblock. , And a step repeated from the top of the group of macroblock rows to the third and fourth macroblock rows located at the bottom from the fourth macroblock row.

The moving picture coding method of the present invention includes a macroblock pipeline performed by dividing the process of the moving picture coding in a plurality of stages, vertically combines the two macro blocks adjacent to the macroblock pair A moving picture coding method in a moving picture coding apparatus that switches between field coding and frame coding for each macroblock pair, and is a macro of 2n rows (n is an integer of 2 or more) that are vertically continuous. block row is a human settlement, the the most belonging to the macroblock row of two rows located above macroblock pair in a macroblock line of human chunks, macroblocks lower left neighboring macroblock pair has not yet been selected pair is selected and macroblock other than two rows located uppermost in the macro block row of the human unity Tsu For macroblock pairs belonging to click the row, by Rukoto macroblock pair to the upper right neighboring macroblock pair is already selected is selected, the macroblock pair is 2 macroblock rows in the macroblock row of human unity a first step that will be selected per one, for the first macroblock pair elected by the step, when the field coding is selected, the top field macroblock in the odd lines of the macroblock pair There is constructed, the bottom field macroblock even lines of the macroblock pair is a configuration, when the frame coding is selected, the upper frame macroblock is composed of the upper half of the macroblock pair line, a macroblock pair Lower frame macro bro with lower half line A second step of click is Ru is configured, for the first macroblock pair selected by the step, when the field coding is selected, located uppermost in the macro block row of the human unity from the macroblock pairs belonging to the macro block rows in two rows, until said macroblock pair most belonging to the macroblock row of two rows located below in the macroblock row of batches and sequentially over n macroblocks cycle, 1 macro data required to encode the one of the macroblocks contained in each macroblock pair for each block cycle is set to the macro block pipeline, whereas, when frame coding is selected, the the most macroblock row of the two rows located above in the macroblock row of human unity From belonging macroblock pair, until said macroblock pair most belonging to the macroblock row of two rows located below in the macroblock row of batches and sequentially over n macroblocks cycle, every macroblock cycle a third step and the first macroblock pair selected by step data necessary Ru is set to the macro block pipeline to encode the upper frame macroblocks contained in each macroblock pair , when the field coding is selected, from said uppermost belonging to the macroblock row of two rows located macroblock pair in the macroblock row of batches and most in the macro block row of the human unity Makuroburo belonging to the macroblock row of two rows located below Until Kupea, sequentially over n macroblocks cycle, required to encode the other one that has not been set by the third step of the macro blocks included in each macroblock pair for each macroblock cycle such data is set to the macro block pipeline, whereas, when frame coding is selected, most belong to the macro-block row of two rows located above macroblock pair in a macroblock line of the human unity from until said macroblock pair most belonging to the macroblock row of two rows located below in the macroblock row of batches and sequentially over n macroblocks cycles, each macroblock pair for each macroblock cycle Necessary to encode lower frame macroblock contained in A fourth step of data Ru is set to the macro block pipeline, when the data is set needed to encode the macroblock by the third step or the fourth step, the macro-block fifth step and said fourth step by macroblock pair most belonging to the macroblock row of two rows located below in the macroblock row of the human unity pipeline starts encoding of the macroblock When the data necessary for encoding the macroblock included in the macroblock pipeline is set in the macroblock pipeline , the process returns to the first step , and the right adjacent macro is selected for each selected macroblock pair. And a sixth step in which the block pair is newly selected .

According to the present invention , m MB rows (m is an integer of 2 or more) that are vertically continuous are taken as a group, and one MB is selected per MB row from the group of MB rows, and these selections are made. Since the m MBs are continuously input to one MB pipeline, the processing result of the left adjacent MB can be fed back to the processing of the (m−1) stage fractional MB. In addition, since the reference areas of the selected m MBs are largely overlapped, if the reference area of these MBs is stored in the common reference area memory, the external memory storing the pictures There is no need to duplicate transfer for updating the overlapped portion. For this reason, by providing the reference area memory inside the moving picture coding apparatus, the amount of data transferred from the external memory storing the picture to the internal memory of the moving picture coding apparatus can be reduced.

In addition, according to the present invention , 2n (n is an integer of 2 or more) MB rows that are vertically continuous are taken as a group, and one MB pair per two MB rows that are vertically continuous from these MB rows. First, one MB included in the selected n MB pairs is continuously input to one MB pipeline, and the second time, the selected n MB pairs are transferred to the selected n MB pairs. Since the remaining one of the included MBs is continuously input to one MB pipeline, the processing result of the first MB to be input can be fed back to the processing of the remaining one MB for (n-1) stages. In addition, the processing result of the left adjacent MB pair can be fed back to the processing of the (n−1) stage partial MB pair. Further, since the reference areas of the selected 2n MBs are largely overlapped with each other, if the reference areas of these MBs are stored in a common reference area memory, the external memory storing the pictures This eliminates the need for duplicate transfer for updating the overlapping portion. For this reason, by providing the reference area memory inside the moving picture coding apparatus, the amount of data transferred from the external memory storing the picture to the internal memory of the moving picture coding apparatus can be reduced.

Furthermore, according to the present invention , at some stages of the macroblock pipeline, the macroblock cycle can be divided into subblock cycles to form a subblock pipeline, and processing for each subblock can be performed. For this reason, it becomes possible to feed back the processing result of the adjacent sub-block to the processing of the sub-block being encoded.

In addition, according to the present invention , since the reference areas of a plurality of video encoding devices that operate in parallel overlap each other vertically, all moving images are transferred from the external memory to the common internal memory of the video encoding device that operates in parallel. If the reference region of the image coding device is transferred and transferred from the common internal memory to each moving image coding device, there is no need to transfer overlapping portions of the reference region. For this reason, by providing a common internal memory in a moving image encoding apparatus that operates in parallel, the amount of data transferred from an external memory that stores pictures to the common internal memory of the moving image encoding apparatus that operates in parallel can be reduced. Further, unlike the conventional slice interleaving, it is not necessary to divide the slices into areas that are shared and encoded by the moving picture coding apparatuses operating in parallel , and the entire picture can be made into one slice. Furthermore, when the first region of a picture to be processed by a plurality of moving image encoding devices is input, the moving image encoding device can start encoding, and thus encoding delay can be suppressed.

Furthermore, according to the present invention, Even when arithmetic coding such as CABAC defined by H.264 is performed, rate control can be performed before the coding of all MBs included in a group of MBs that are vertically continuous is completed. For this reason, when the generated code amount of a certain MB suddenly increases, the quantization parameter of the MB to be encoded next is quickly increased, and the code amount generated from the newly encoded MB is decreased. Rate control is possible. In addition, arithmetic coding can be performed even if the arithmetic coding circuit does not have enough performance to handle the amount of binarized symbols that suddenly increases. In particular, if the binarized symbol storage circuit has sufficient storage capacity to store the binarized symbols of the I picture and the binarized symbols of the P picture and B picture before and after the I symbol, the performance of the arithmetic coding circuit can be improved. Arithmetic coding can be performed even if the performance of the binarized symbol generation amount that suddenly occurs in the I picture is less than the capability of processing .

Further, according to the present invention, even when there is a scene change or the like, it is possible to perform rate control close to rate control based on the generated code amount by true arithmetic coding .

Furthermore, according to the present invention, when the amount of binarized symbols generated in a certain MB suddenly increases and is predicted to exceed the processing capacity of a true arithmetic encoding circuit as a result of provisional arithmetic encoding, The entire picture including the MB can be CABAC-encoded if it is predicted that the entire picture including the CAVLC code is encoded by the provisional arithmetic encoding and can be processed by a true arithmetic encoding circuit. As a result, a situation in which the performance of the true arithmetic coding circuit is unexpected can be surely avoided . Furthermore, the CAVLC encoding circuit calculates the code amount of the CAVLC encoded bitstream, compares the temporarily generated code amount of CAVLC encoding and provisional arithmetic encoding, and entropy that is estimated to be small for each picture. An encoding mode can also be selected.

  In the present invention, a plurality of MB rows that are vertically continuous are grouped, one MB is selected per MB row from the group of MB rows, and the selected plurality of MBs are one MB. Put continuously into the pipeline.

Hereinafter, a moving picture coding apparatus and a moving picture coding method according to an embodiment of the present invention will be described with reference to the drawings.

(First embodiment)
FIG. 1 is an explanatory diagram showing an example of MBs that are continuously input to the MB pipeline. The picture in FIG. 1 is composed of 12 MB rows from a row to l row, and each MB row is composed of 16 MBs from 0th to 15th. In the picture of FIG. 1, first, MBs included in the a-MB row, b-MB row, c-MB row, and d-MB row are encoded by one MB pipeline, and then the e-MB row, MBs included in the f-MB line, the g-MB line, and the h-MB line are encoded by one MB pipeline, and finally the i-MB line, the j-MB line, the k-MB line, and the l-MB. MB included in a row is encoded by one MB pipeline. The MB of e row 10 column, f row 8 column, g row 6 column, and h row 4 column in FIG. 1 is an example of an MB that is continuously input to one MB pipeline. The MB in f row and 8 columns is 1 MB away from the MB in e row and 10 columns in the left direction. Similarly MB of g line 6 columns are spaced 1 MB to the left from the MB of f line 8 column, MB of h rows and four columns are spaced 1 MB to the left from the MB of g line 6 column. These four MBs are not adjacent to each other in the left adjacent, the upper adjacent, or the upper right adjacent. In the following, it is said that these blocks are not adjacent to each other, that the plurality of blocks are not adjacent to each other in the left adjacent, the upper adjacent, or the upper right adjacent.

  FIG. 2 shows a block diagram of a moving picture coding apparatus to which the present invention is applied. As shown in FIG. 1, the MB selection circuit 2 collects the e-MB row, the f-MB row, the g-MB row, and the h-MB row, and the MBs belonging to the e-MB row By selecting an MB that has not yet been selected and, for MBs belonging to f-MB, g-MB, and h-MB rows, by selecting an MB for which the upper right neighboring MB has already been selected, The MB of e row 10 column, f row 8 column, g row 6 column, h row 4 column is selected. The encoded data setting circuit 3 sends data necessary for encoding each MB to the MB pipeline 1 in the order of MB of e row 10 column, f row 8 column, g row 6 column, h row 4 column. Set. When data necessary for encoding an MB is set in the MB pipeline 1, the MB is input to the MB pipeline 1. When the data necessary for encoding the MB of h rows and 4 columns is set in the MB pipeline 1 and the MB of h rows and 4 columns is input to the MB pipeline 1, the MB selection circuit 2 A new MB of column, f row 9 column, g row 7 column, h row 5 column is selected.

  FIG. 3 shows a flowchart of the process of the moving picture encoding method to which the present invention is applied. e-MB line, f-MB line, g-MB line, and h-MB line are grouped, and for MBs belonging to the e-MB line, an MB for which the lower left neighboring MB is not yet selected is selected; and For MBs belonging to the f-MB row, the g-MB row, and the h-MB row, by selecting the MB for which the upper right adjacent MB has already been selected, the e row 10 column, the f row 8 column, the g row 6 An MB with 4 columns and h rows is selected (step S1). Next, data necessary for encoding each MB is set in the MB pipeline 1 in the order of MB of e row 10 column, f row 8 column, g row 6 column, h row 4 column, Input into the pipeline (steps S2 and S3). When data necessary for encoding an MB is set in the MB pipeline 1, the MB is input to the MB pipeline 1. When the data necessary for encoding the MB of h rows and 4 columns is set in the MB pipeline 1 and the MB of h rows and 4 columns is input to the MB pipeline 1, the process returns to step S1 (step S4). MBs of e row 11 column, f row 9 column, g row 7 column, h row 5 column are newly selected (step S1).

  FIG. 4 shows a state in which each MB flows through each stage of the MB pipeline 1 when these four MBs are continuously input to one MB pipeline 1. The processing contents of stage 0 to stage 6 are the same as those in FIG. FIG. 5 shows MB cycle numbers at which the MBs constituting the picture of FIG. 1 are input to the MB pipeline 1 and the stage 0 process is started. As shown in FIG. 4, four MBs of e row 10 column, f row 8 column, g row 6 column, h row 4 column are staged in order when the MB cycle numbers are 104, 105, 106, and 107, respectively. When the MB cycle numbers are 110, 111, 112, and 113, respectively, the stage 6 is reached in order and the encoding is completed. Thus, among a group of four MB rows, the MB belonging to the uppermost MB row, the MB belonging to the second MB row from the top, the MB belonging to the third MB row from the top, the fourth from the top The MBs that belong to the MB row are input to the MB pipeline 1 in the order of MBs, and then the right adjacent MBs are input to the MB pipeline 1 in the same order.

According to the present invention, as human unity as in the present embodiment when the number of MB rows to be encoded is 4, and three-stage partial feedback of the processing result of the left neighboring MB, it can be reflected in the processing of the current MB. Therefore, as indicated by an arrow in FIG. 4, for example, the high-precision motion detection result and the intra prediction result of the f-row 7-column MB can be reflected in the coarse-precision motion detection of the f-row 8-column MB. Thus, unlike the conventional technique, the motion vector of the current MB can be selected in consideration of the high-precision motion detection result and intra prediction result of the MB adjacent to the left. Similarly, as indicated by an arrow, the result of motion compensation of the MB of f rows and 7 columns can be reflected in the intra prediction of MBs of f rows and 8 columns. Therefore, unlike the conventional technique, intra prediction of the current MB can be performed using the reproduced pixels obtained by the motion compensation of the MB adjacent to the left. Further, the rate control result of the MB of f rows and 7 columns can be reflected in the quantization of MBs of f rows and 8 columns. In this manner, the quantization parameter of the current MB can be corrected according to the rate control result of the MB adjacent to the left.

Here, MB included in the MB row adjacent to the bottom in the present embodiment showed a case in which distant 1 MB to the left from the MB in MB line above, it is sufficient that apart at least 1 MB, for example, The object of the present invention can be achieved even at a distance of 2 MB, and the same actions and effects are produced. Also, taking four MB rows that are vertically continuous as a group, one MB per MB row is selected from these four MB rows, and these four MBs are selected as one MB pipeline. However, it is sufficient that the number of MB lines in a group is two or more. For example, eight MB lines that are vertically continuous are grouped, and one of these eight MB lines is used as one group. Even if one MB is selected per MB row, and the selected 8 MBs are continuously input to one MB pipeline, the object of the present invention can be achieved, and similar operations and effects are achieved. Produce. However, when the number of MB rows in a group is 2, the processing result of the left adjacent MB can be fed back to the current MB processing for only one stage. Therefore, in the stage configuration of FIG. Cannot be fed back to the current MB intra prediction and integer transform / quantization / inverse quantization / inverse integer transform. When the number of MB lines in a group is 2, it is necessary to change the stage configuration from that shown in FIG.

(Second Embodiment)
FIG. 6 shows an MB divided into four SBs of 8 pixels × 8 pixels (hereinafter referred to as 8 × 8SB). In the present embodiment , an example will be described in which a motion vector that minimizes the sum of the cost of a motion vector and the cost of a prediction error is detected for four 8 × 8 SBs. Generally, it is estimated that the cost of a motion vector is higher as the distance between the PMV and the motion vector is longer. For this reason, before starting motion detection for each 8 × 8 SB, the 8 × 8 SB PMV must be determined. For example, if PMV is determined based on the motion vector of the left adjacent 8 × 8SB, the upper adjacent 8 × 8SB, and the upper right adjacent 8 × 8SB, the numbers 0, 1, 2, and 3 in FIG. The motion vectors for every 8 × 8 SB must be detected in order. However, since the 8 × 8SB motion vector adjacent to the upper right side does not exist when the third 8 × 8SB motion vector is detected, the third 8 × 8SB PMV is the first and second 8 × 8SB motion vectors MV1 and MV2. It is decided based on.

FIG. 7 is an explanatory diagram showing an example of MBs continuously input to the MB pipeline. The picture in FIG. 7 starts from the a line, and there is an MB line below the r line. There are also MBs after the 16th column. In the present embodiment , the i-MB line, j-MB line, k-MB line, l-MB line, m-MB line, n-MB line, o-MB line, and p-MB line are grouped together. Select MB of i row 15 column, j row 13 column, k row 11 column, l row 9 column, m row 7 column, n row 5 column, o row 3 column, p row 1 column from a group of MB rows. An example is shown in which these eight selected MBs are continuously input into one MB pipeline.

FIG. 8 shows how each MB flows through each stage when MBs included in 8 MB rows are continuously input to one MB pipeline in order to detect a motion vector every 8 × 8 SB. Show. FIG. 9 shows a block diagram of a motion vector detection circuit 12 to which the present invention is applied. The external memory 4 stores an image with single pixel accuracy. At stage 0, an image is read from the external memory 4. The image is reduced by the two-pixel precision image creation circuit 5 to create a two-pixel precision image and written in the two-pixel precision memory 6. While reading an image from the two-pixel accuracy memory 6 at the stage 1, a two-pixel accuracy motion detection circuit 7 detects a candidate motion vector with two-pixel accuracy. Then, in stage 2, the area around the candidate vector is read from the external memory 4 and written to the single pixel precision memory 8. Here, since the external large-capacity memory is generally low speed, one stage is allocated for transfer from the external memory 4 to the single pixel precision memory 8. While reading the area around the two-pixel accuracy candidate vector from the single-pixel accuracy memory 8, the motion vector is detected by the single-pixel accuracy motion detection circuit 9 in the stage 3. Thereafter, the single pixel precision pixel is read from the single pixel precision memory 8 at the stage 4, and the quarter pixel precision motion vector detection circuit 11 is created while creating the quarter pixel precision image by the quarter pixel precision image creation circuit 10. Is used to perform 1/4 pixel precision motion vector detection. Since the single pixel accuracy memory 8 supplies a single pixel accuracy image to the single pixel accuracy motion detection circuit 9 and the ¼ pixel accuracy image creation circuit 10, the single pixel accuracy memory 8 has a double buffer configuration or more than two reads. It must be possible multi-port memory. In FIG. 8, the surrounding of the single-pixel accuracy motion vector is directly searched at stage 4 to detect the 1 / 4-pixel accuracy motion vector. However, after detecting the half-pixel accuracy motion vector at stage 4, A ¼ pixel accuracy motion vector may be detected by searching around the pixel accuracy motion vector. Further, single-pixel precision motion vector detection and half-pixel precision motion vector detection may be performed in stage 3, and quarter-pixel precision motion vector detection may be performed in stage 4.

  In stages 1 to 5, the MB cycle is divided into four SB cycles, and each 8 × 8 SB process is performed in each SB cycle. Here, the SB cycle means a unit of time required for processing one SB, and when performing processing for each SB, the MB cycle is divided into SB cycles. Hereinafter, the stage group in which the MB cycle is divided into SB cycles and processing for each SB is performed is referred to as an SB pipeline (or sub-block pipeline). Stages 1 to 5 in FIG. 8 are SB pipelines. FIG. 10 shows the detailed operation of the SB pipeline. −0, −1, −2 and −3 mean 8 × 8SB numbered 0, 1, 2, 3 in FIG. For example, i15-2 represents the second 8 × 8SB included in the MB of i rows and 15 columns. In FIG. 10, an empty stage is inserted into the stage 5, and for example, a ¼ pixel precision motion vector detected in the stage 4 for i15-0SB can be reflected in the cost calculation of the 2-pixel precision motion vector in the stage 1 of i15-1SB. I am doing so. As described above, when the MB cycle is divided into the number of SB cycles equal to the number of SBs, the processing result of the previous 8 × 8 SB in the same MB is obtained by setting a pipeline of (SB number + 1) stages. Of 8 × 8SB.

  In stage 1 to stage 5, it is possible to provide a motion vector detection circuit for each MB that operates in parallel with a motion vector detection circuit for each 8 × 8 SB. Further, in stages 3 to 5, in the peripheral region of the candidate vectors for each 8 × 8SB detected in stage 1, motion vector detection for each 8 × 8SB is performed in parallel with the motion vector detection for each 8 × 8SB, for each 16 pixels × 8 pixels SB, for 8 pixels × It is also possible to detect a motion vector for every 16 pixels SB.

  After motion vectors are detected in stages 0 to 5, a prediction mode with the highest coding efficiency is selected from 16 × 16 intra prediction mode, 4 × 4 intra prediction mode, inter prediction mode, and the like in stage 6. The In stage 7, integer transformation, quantization, inverse quantization, and inverse integer transformation are performed. In stage 8, variable length coding and motion compensation are performed. In stage 9, rate control and deblocking filter processing are performed. In stage 10, the reproduced pixels and the encoded bit stream are written in the external memory 4.

(Third embodiment)
In this embodiment, 1/4-pixel accuracy motion detection circuit 11 is provided two combines the stage 3 and stage 4 is performed by the quarter-pixel precision motion detection of a single 8 × 8SB over 2SB Cycle Example Indicates. Except for stage 3 and stage 4, the stage configuration is the same as that of the MB pipeline of the second embodiment . FIG. 11 shows the detailed operation of the SB pipeline, and FIG. 12 shows a block diagram of the motion detection circuit 13 for realizing this operation. By operating the two quarter-pixel precision motion detection circuits 11 with a shift of 1 SB cycle, individual 8 × 8 SB quarter-pixel precision motion detection can be performed over 2 SB cycles.

(Fourth embodiment)
In this embodiment, H. An example in which the present invention is applied to 4 pixel × 4 pixel intra prediction (hereinafter referred to as 4 × 4 intra prediction) in H.264 will be described. FIG. 13 shows an MB divided into 16 4 pixels × 4 pixels SB (hereinafter referred to as 4 × 4 SB). 4 × 4SB is encoded in the order of 0 to 15 in FIG. 13 and converted into a bit stream. For example, when 4 × 4 intra prediction is performed for the third 4 × 4SB, the pixels in the lowermost 4 rows of the first 4 × 4SB and the pixels in the fourth rightmost 4 columns of the second 4 × 4SB are referred to. In addition, when performing 4 × 4 intra prediction for the 14th 4 × 4SB, the pixels at the bottom four rows in the 12th and thirteenth 4 × 4SB and the pixels at the rightmost four columns in the 11th 4 × 4SB are referred to. The As the pixels belonging to these adjacent 4 × 4 SBs, it is desirable to use reproduction pixels.

FIG. 14 shows a state where each MB flows through each stage when MBs included in eight MB rows are continuously input to one MB pipeline for 4 × 4 intra prediction. Stage 4 performs 4 × 4 intra prediction using adjacent 4 × 4 SB reconstructed pixels, and stage 5 performs integer transformation, quantization, inverse integer transformation, and inverse quantization on the prediction error, and adds the intra prediction value Then, a reproduction pixel is generated. The 4 × 4 intra prediction circuit of stage 4 and the integer transform / quantization / inverse integer transform / inverse quantization circuit of stage 5 are the movements of stage 1 to stage 5 shown in the second embodiment or the third embodiment. Operates in parallel with the detection circuit.

In the present embodiment divides the MB cycle 16 of SB cycle at stage 4 and 5. For this reason, if the number of SBs is larger than the number of stages in the SB pipeline, and the number of SB pipelines is (SB number + 1), the number of empty stages increases and the efficiency is poor. FIG. 15 shows the detailed operation of the SB pipeline. −0, −1, −2,..., -15 means 4 × 4SB numbered 0, 1, 2,. For example, i15-8 represents the 8th 4 × 4SB included in the MB of i rows and 15 columns. As shown in FIG. 15, in this embodiment , by processing 4 × 4 SB included in two MBs alternately, one MB is processed over 2 MB cycles, that is, 32 SB cycles. When 4 × 4 SB belonging to one MB is subjected to 4 × 4 intra prediction, 4 × 4 SB belonging to another MB is simultaneously subjected to integer transformation / quantization / inverse integer transformation / inverse quantization. The processing of one MB is started every MB cycle, and the processing of another MB is finished, and matching with the other stages of the MB pipeline is performed. However, when operated in this way, as shown in FIG. 15, in the 4 × 4 intra prediction of stage 4, a deviation from the MB cycle occurs by 1 SB cycle, that is, 1/16 MB cycle every 2 MB cycles. Therefore, in FIG. 15, the effective period of stage 3 is set to 15/16 MB cycle, and the shift of 1/16 MB cycle in stage 4 is absorbed. If the stage 3 process is heavy and an invalid period cannot be provided, the stage 6 valid period may be set to 15/16 MB cycle to absorb the 1/16 MB cycle shift. By operating in this way, 4 × 4 SBs of the left adjacent, upper adjacent, and upper right adjacent can be generated without causing a sky in the 4 × 4 intra prediction circuit and the circuits that perform integer transform / quantization / inverse integer transform / inverse quantization. 4 × 4 intra prediction can be performed using the reconstructed pixels for the pixels belonging to.

  It should be noted that the MB pipeline shift can be absorbed by inserting an empty stage before or after the SB pipeline, instead of shortening the effective period of the stage before or after the SB pipeline.

In this embodiment, as shown by the arrows in FIG. 14, it can be reflected to the quantization performed concomitantly rate control results in the left neighboring MB to 4 × 4 intra prediction.

  Further, 16 pixel × 16 pixel intra prediction of the luminance signal and 8 pixel × 8 pixel intra prediction of the color difference signal in the 4: 2: 0 format are processed for each MB. For this reason, it is only necessary to refer to pixels belonging to the left adjacent MB and the upper adjacent MB. In this case, since the reconstructed pixel obtained by the motion compensation in stage 8 can be referred to, there is no need to perform integer transformation / quantization / inverse integer transformation / inverse quantization in stage 5 as in 4 × 4 intra prediction. When the 4 × 4 intra prediction mode is selected in the intra / inter determination in stage 6, the reproduced pixels obtained by the integer transform, quantization, inverse integer transform, and inverse quantization in stage 5 are stored. Alternatively, the stage 7 may be configured to perform integer transformation / quantization / inverse integer transformation / inverse quantization again in the stage 7.

  If the mode determination of 4 × 4 intra prediction is performed by the input pixels, the integer transformation / quantization / inverse integer transformation / inverse quantization in stage 5 can be omitted. However, the image quality is deteriorated as compared with the case where the mode is determined using the reproduction pixels.

(Fifth embodiment)
In the present embodiment , in the MB pipeline having substantially the same stage configuration as the second to fourth embodiments , the i-MB row, j-MB row, k-MB row, and l-MB row in FIG. Are selected, and MBs such as i rows 15 columns, j rows 13 columns, k rows 11 columns, l rows 9 columns, etc. are selected from these MB rows, and one of these four selected MBs is selected. An example of continuously charging to the MB pipeline is shown. In the second to fourth embodiments , since 8 MB rows are processed as a group, the processing result of each left adjacent MB can be fed back to the processing of 7 stages of MB. In this embodiment , 3 stages I can only give feedback. For this reason, when the processing result of the left adjacent MB is reflected in the processing of the current MB, there is a restriction compared to the second to fourth embodiments .

  When MBs included in four MB rows are continuously input to one MB pipeline in order to detect a motion vector for each 8 × 8 SB, each MB flows through each stage of the MB pipeline. Is shown in FIG. In this case, for example, when the intra / inter determination of the MB of j rows and 12 columns ends in stage 5, the 2-pixel precision motion detection of the MB of j rows and 13 columns ends simultaneously in stage 1. For this reason, there is a limitation that the intra / inter determination result of the left adjacent MB cannot be reflected in the two-pixel precision motion detection of the current MB. However, as indicated by arrows in FIG. 16, the intra / inter determination result of the left adjacent MB can be reflected in the stage 2 and subsequent stages. Also, the intra / inter determination result determined in stage 5 for the upper left adjacent MB, the upper adjacent MB, the upper right adjacent MB, the MB one left in the left direction, and the like can be reflected in the two-pixel accuracy motion detection in stage 1. Therefore, in stage 1, a temporary PMV is created with reference to motion vectors such as the upper left adjacent MB, the upper adjacent MB, the upper right adjacent MB, and the MB separated by one in the left direction, and all four 8 × 8 SBs are common. The two-pixel precision motion detection is performed with reference to the temporary PMV. In stage 2, the peripheral area of the candidate motion vector of 2-pixel accuracy detected in stage 1 is set in the single-pixel accuracy memory 8, a true PMV is created in stage 3, and a true PMV is created in stage 3 and stage 4. The single-pixel precision motion detection and the quarter-pixel precision motion detection are performed with reference to FIG.

  FIG. 17 shows the detailed operation of the SB pipeline in stages 3 and 4. In single-pixel precision motion detection of stage 3 and quarter-pixel precision motion detection of stage 4, by processing 8 × 8SB included in two MBs alternately, one MB takes 2 MB cycles, that is, 8SB cycles. Process. When single pixel precision motion detection is performed for 8 × 8SB belonging to one MB, ¼ pixel precision motion detection is simultaneously performed for 8 × 8SB belonging to another MB. The processing of one MB is started for each MB cycle, and the processing is performed so as to finish the processing of another MB, so as to be consistent with the other stages of the MB pipeline. Then, the effective period of stage 2 is set to 3/4 MB cycle, and the shift of 1/4 MB cycle in stage 4 is absorbed. According to this method, four 8 × 8 SBs included in one MB can be serially processed from No. 0 to No. 3. For this reason, in the single-pixel precision motion detection and the quarter-pixel precision motion detection, it is possible to perform motion detection based on a true PMV in consideration of the adjacent 8 × 8 SB motion vector.

However, in the present embodiment, the evaluation of the spatial direct address required separately. In stage 1, two-pixel precision motion detection is performed based on the temporary PMV, and in stage 2, the peripheral area of the candidate motion vector detected in stage 1 is set in the single-pixel precision memory 8. When the temporary PMV of stage 1 and the true PMV of stage 3 are different, the motion detection areas in stages 3 and 4 do not include MB or 8 × 8 SB at the position corresponding to the spatial direct address. Therefore, it is necessary to perform processing for evaluating the MB of the position corresponding to the spatial direct address and the cost of 8 × 8 SB in parallel with stage 3 and stage 4.

FIG. 18 shows how each MB flows through each stage of the MB pipeline when MBs included in four MB rows are continuously input to one MB pipeline for 4 × 4 intra prediction. Show. FIG. 19 shows the detailed operation of the SB pipeline in stages 3 and 4. In the 4 × 4 intra prediction in stage 4 and the integer transform / quantization / inverse quantization / inverse integer transform in stage 5, the same processing as in the SB pipeline of the fourth embodiment is performed.

However, in this embodiment , since the processing result of the left adjacent MB can be fed back to the current MB processing only for the three stages, if motion compensation is performed at stage 7, 4 × 4 intra prediction is performed using the reproduced pixels of the left adjacent MB. I can't. Therefore, there is a limitation that motion compensation must be performed at stage 6. The rate control result of stage 8 of the left adjacent MB can be reflected after stage 5 and cannot be reflected in the quantization of stage 4. However, there are cases where a large amount of bits are generated by variable length coding in stage 7 and it is necessary to perform rate control to increase the quantization parameter in stage 8. In this case, the 4 × 4 intra prediction mode is not selected in the intra / inter determination of stage 5, or the new quantization parameter is used in stage 6 when the 4 × 4 intra prediction mode is selected. It is necessary to redo integer conversion, quantization, inverse quantization, and inverse integer conversion.

As shown in the first to fifth embodiments , in the present invention, a plurality of MB rows that are vertically continuous are taken as a group, and one MB row is collected from these batches of MB rows. MBs are selected, and the selected MBs are continuously input to one MB pipeline. Since these MBs are in a narrow image area, the image encoding device is used to display the reproduction pixels, motion vectors, and the like of the MBs adjacent to the upper and right upper MBs until the encoding of the MBs belonging to the lower adjacent MB row is completed. Even if it is held inside, the scale of the memory circuit required for the holding is small. By providing this storage circuit, except for the MB included in the uppermost MB row of the plurality of MB rows, the reproduction pixels, motion vectors, etc. of the upper adjacent MB and the upper right adjacent MB for encoding the current MB, etc. Can be solved.

Note that the stage configuration of the MB pipeline shown in the first to fifth embodiments is an example, and other stage configurations may be used. If the stage configuration is changed, the constraints in motion detection, intra prediction, and the like change even if the number of vertically continuous MB rows encoded as a group is the same.

(Sixth embodiment)
FIG. 20 is a block diagram of a video encoding apparatus having an internal memory. The moving image encoding device 23 reads all data necessary for MB encoding from the external memory 4 and stores the data in the internal memory. There are two types of internal memory: a reference area memory 14 and a buffer memory 15. The reference image to be referred to at the time of motion detection is stored in the reference area memory 14 and is supplied from the reference area memory 14 to the motion detection unit 16 and the motion compensation unit 17. Other data necessary for encoding is stored in the buffer memory 15, and includes a motion detection unit 16, a motion compensation unit 17, an intra prediction unit 18, an integer transform / quantization / inverse quantization / inverse integer transform unit 19, a variable The long encoding unit 20 and the deblocking filter unit 21 transfer data to and from the buffer memory 15. The control unit 22 performs intra / inter determination and rate control. The control unit 22 is configured by a wired logic or a microprocessor. In FIG. 20, the reference area memory 14 and the buffer memory 15 are shown as separate ones. However, if the reference area address and the buffer address are separated, one common memory may be used.

FIG. 21 is an explanatory diagram showing a reference area when four MBs included in four vertically continuous MB rows are processed by one MB pipeline. Reference areas 28, 29, 30, and 31 are areas that are referred to when motions of MBs 24, 25, 26, and 27 are detected, respectively. The reference areas 28, 29, 30, and 31 are greatly overlapped, and the height of the non-overlapping portions in the reference areas of the two MBs belonging to the vertically adjacent MB rows matches the number of pixels in the vertical direction of the MB. This overlapping portion becomes larger as the motion vector detection range becomes wider. In the present embodiment , all the pixels in the reference areas 28, 29, 30 and 31 are stored in the reference area memory 14. As in the prior art, in this embodiment , when encoding of the MB adjacent in the right direction is started, only the update areas 32, 33, 34, and 35 are transferred from the external memory 4 to the reference area memory 14, and the code is encoded up to that point. The origin of the reference area 28, 29, 30, 31 of the left adjacent MB that has been converted is changed to a new origin for the right adjacent MB to be encoded next. However, the update area 33 for the MB 25 is only a portion not included in the reference area 28 of the MB 24. Similarly, the update areas 34 and 35 of MB26 and MB27 are only portions not included in the reference areas 29 and 30 of MB25 and MB26, respectively. Therefore, the amount of pixel transfer between the external memory 4 and the reference area memory 14 can be greatly reduced as compared with the prior art.

FIG. 22 is an explanatory diagram showing another method of storing the reference area in the reference area memory . The reference area memory 14 stores a rectangular area 36 that includes all the reference areas 28, 29, 30, and 31. According to this method, the address generation of the reference area memory 14 is simplified. Further, since the update area 37 is also a group of rectangular areas, transfer of pixels from the external memory 4 to the reference area memory 14 is simplified.

In the above, the case where four MBs included in four consecutive MB rows are simultaneously processed by one MB pipeline has been shown as an example. However, the method shown in the present embodiment may be performed on two or more consecutive MB rows. For example, it may be eight consecutive MB lines.

  The reference region memory 14 stores only a single pixel accuracy image, and the single pixel accuracy image read out from the reference region memory 14 is used to generate a half pixel accuracy image or ¼ inside the motion detection unit 16 and the motion compensation unit 17. It can be set as the structure which produces a pixel precision image. In this configuration, a half-pixel precision image or a quarter-pixel precision image is created in advance and stored in the external memory 4, and the half-pixel precision image or the quarter-pixel precision image is transferred from the external memory 4 to the reference area memory 14. The pixel transfer amount between the external memory 4 and the reference area memory 14 can be reduced as compared with the configuration to be performed.

  Further, there is a hierarchical search method in which a reduced image such as a two-pixel accuracy image is created, motion is first detected on the reduced image, and motion detection is performed on the high-accuracy image step by step. Even when the present invention is applied to this search method, only the single-pixel precision image is stored in the reference area memory 14, and a reduced image is created inside the motion detection unit 16 and the motion compensation unit 17. It can be configured. This configuration creates a reduced image in advance and stores it in the external memory 4, and transfers pixels between the external memory 4 and the reference region memory 14 rather than a configuration in which the reduced image is transferred from the external memory 4 to the reference region memory 14. The amount can be reduced.

In addition, a telescopic search method has been proposed as a method for reducing the amount of calculation in motion detection. In the telescopic search method, an intermediate image between the encoding target image and the reference image is also required. In this case, it is also possible to apply the update method shown in the present embodiment, on the space required for the motion detection in the middle of the image.

Also, it may be transferred from the external memory 4 into the internal memory of the image encoding device 23 in the manner shown only luminance signals in this embodiment, by applying the method shown in this embodiment for both the luminance signal and color difference signal May be. If the color difference signal is not referred to during motion detection, the color difference signal may be stored in either the reference area memory 14 or the buffer memory 15.

(Seventh embodiment)
When HDTV video is to be encoded, the motion vector range needs to be about ± 200 pixels horizontally and ± 100 pixels vertically to obtain good image quality. The amount of calculation for detecting a motion vector in this range is enormous, and a large number of cycles are required for motion detection. However, in a moving picture encoding apparatus having an MB pipeline structure, the number of cycles per MB cycle is common to all stages. Only the stage for motion detection cannot increase the number of cycles per MB cycle. Also, in the MB pipeline to which the present invention is applied, there is a limit to the number of stages where the processing result can be fed back, so it is not preferable to increase the number of stages in the MB pipeline.

  FIG. 23 shows a combination of a motion detection device 38 capable of detecting motion in a wide range and a moving image encoding device 23 including an MB pipeline to which the present invention is applied. The motion detection device 38 detects motion vector candidates in consideration of only the encoding cost of the prediction error without considering the encoding cost of the motion vector. The video encoding device 23 receives the motion vector candidate detected by the motion detection device 38 and performs motion detection in consideration of the motion vector encoding cost for the peripheral region of the motion vector candidate. That is, motion is detected without referring to the PMV over a wide range, and motion detection is performed again with reference to the PMV for the peripheral region of the detected motion vector candidate. The motion detection again in the peripheral region of the received motion vector candidate can be performed at a motion detection stage of any accuracy such as 2-pixel accuracy motion detection or single pixel accuracy motion detection. In the moving image encoding device 23, in addition to the peripheral region of the motion vector candidate, motion detection may be performed simultaneously in another region where there is a high possibility that a motion vector with the minimum cost exists, for example, the peripheral region of the PMV. Since the motion detection device 38 operates independently of the MB pipeline of the video encoding device 23, motion detection can be performed over many cycles.

  In addition, the motion detection device 38 may be configured to perform motion detection on an input image that has not been encoded, but is not a reproduced image created by the video encoding device 23. With this configuration, it is not necessary to send a reproduced image from the moving image encoding device 23 to the motion detection device 38.

  It is possible to integrate the motion detection device 38 and the moving image encoding device 23 on the same LSI. In this case, the external memory 4 connected to the video encoding device 23 is a shared memory of the motion detection device 38 and the video encoding device 23, and the motion detection device 38 performs motion detection on the reproduced image. Also good.

  Also, MPEG-2 to H.264. When the already detected motion vector can be used as in the case of transcoding to H.264, the detected motion vector is directly input to the moving image encoding device 23 without using the motion detecting device 38, and this detected motion is detected. It is also possible to adopt a configuration in which motion detection is performed in consideration of the coding cost of the motion vector for the peripheral region of the vector.

(Eighth embodiment)
FIG. 24 shows a block diagram of a parallel video encoding device 42 having two video encoding devices 40 and 41 and an internal memory 39. FIG. 25 shows a picture area in which the moving picture coding apparatuses 40 and 41 share the coding. The moving image encoding device 40 divides and encodes i-MB line to l-MB line, q-MB line to t-MB line, etc., and e-MB line to h-MB line, m-MB line to p The video encoding device 41 shares MB rows and the like and encodes them. According to this picture division method, since the picture areas shared by the moving picture coding apparatuses 40 and 41 are vertically continuous, the reference areas of both are greatly overlapped vertically. If the configuration is such that the reference areas of the two moving image encoding devices 40 and 41 are transferred from the external memory 4 to the internal memory 39 and are transferred from the internal memory 39 to the individual moving image encoding devices 40 and 41, both references are made. Since there is no need to transfer the overlapping portions of the areas in an overlapping manner, the amount of data transferred between the external memory 4 and the internal memory 39 can be reduced.

FIG. 26 shows how the MB flows through each stage of the MB pipeline included in the moving image coding apparatuses 40 and 41. The stage configuration of each of the video encoding devices 40 and 41 is the same as that of the video encoding device of the first embodiment . However, for example, the reproduction pixel and motion vector of MB belonging to the l-MB row may be transferred from the moving image coding device 40 to the moving image coding device 41 and referred to in the coding of MB belonging to the m-MB row. Is possible. In this way, at the boundary between the regions shared by the moving image encoding devices 40 and 41, the moving image encoding device is at a timing at which data referred to by the current MB, such as reproduction pixels and motion vectors in the upper adjacent MB and the upper right adjacent MB, can be transferred. Operate 40 and 41. By such operation, unlike the conventional slice interleaving, according to the method of this embodiment, as i-MB row to l-MB row and m-MB row ~p-MB line in FIG. 25 each It is not necessary to divide the slices for each area to be encoded by the moving image coding apparatuses 40 and 41, and the entire picture can be made into one slice. Further, when the first area of the picture to be processed by the moving image encoding devices 40 and 41 such as the a-MB row to the d-MB row and the e-MB row to the h-MB row is input, the moving image encoding device 40 is input. And 41 can start encoding, so that encoding delay can be suppressed.

  The parallel video encoding device 42 may include three or more video encoding devices.

(Ninth embodiment)
H. In H.264, CABAC can be used as a variable length coding method. CABAC continuously processes all MBs included in one slice by using one arithmetic coding circuit. On the other hand, in the inventions described in the first to eighth embodiments , a plurality of MB rows that are vertically continuous are taken as a group, and one MB per MB row is obtained from the batch of MB rows. A plurality of selected MBs are continuously input to one MB pipeline. For this reason, the binary symbols cannot be arithmetically encoded unless encoding of all MBs included in a group of MBs that are vertically continuous is completed. However, when a certain MB is encoded, the generated code amount may be remarkably large. In such a case, it is necessary to increase the quantization parameter immediately and control the rate so as to reduce the amount of code generated from the newly encoded MB. In the present embodiment, H.264 is applied to the moving picture coding apparatus to which the inventions described in the first to eighth embodiments are applied. In the case of performing arithmetic coding such as CABAC defined in H.264, a method is shown in which rate control is performed before the coding of all MBs included in a group of MBs that are vertically continuous is completed.

There is a method for estimating the generated code amount of arithmetic coding from the total symbol length of binary symbols . If rate control is performed based on the generated code amount estimated by this method, rate control can be performed without waiting for the completion of encoding of all MBs included in a group of MBs that are vertically continuous. FIG. 27 is a block diagram of a CABAC encoder that performs rate control based on the estimated generated code amount.

  First, the syntax data is binarized by the binarization circuit 43. The binarization circuit 43 includes a look-up table, and performs conversion into a binarized symbol in one cycle using the table look-up. Then, the generated code amount for each MB is estimated from the sum of the symbol lengths of the binarized symbols converted by the generated code amount estimation circuit 44, and the estimated generated code amount is sent to the control unit 22 for rate control. As a result, rate control can be performed before the encoding of all MBs included in a group of MBs that are vertically continuous is completed.

  Here, in the moving picture encoding apparatus to which the present invention is applied, when the first MB of the MB row is processed, the quantization parameter of the last MB of the MB row adjacent to it is not yet determined. However, the difference between the quantization parameter and the quantization parameter of the MB encoded immediately before is binarized, and arithmetic coding is performed. Thus, the quantization parameter of the first MB in the MB row is binarized from the temporary value (hereinafter referred to as temporary mb_qp_delta). The provisional value is desirably determined by referring to the quantization parameter of the already encoded MB and estimating the quantization parameter of the last MB in the MB row adjacent above. On the other hand, the quantization parameters of MBs other than the head in the MB row are H.264. As defined in H.264, the difference (hereinafter referred to as mb_qp_delta) from the left adjacent MB encoded immediately before is binarized.

  Next, the binarized symbols are temporarily stored in the binarized symbol storage circuit 45, and after the encoding of all MBs included in a group of MBs that are vertically continuous is completed, the binarized symbols are binarized. Read out from the symbol storage circuit 45. Then, the arithmetic coding circuit 46 performs arithmetic coding while restoring the context information corresponding to the context information restoring circuit 47. The processing of the context information restoration circuit 47 is almost the same as the context information restoration processing in the CABAC decoding circuit used in the video decoding device. The processing flow of the context information restoration circuit 47 is shown in FIG. Context information is initialized at the head of the slice (step S5). Next, the variable tmp is cleared (step S6), and one binarized symbol of syntax data is acquired (step S7). Thereafter, the symbol context information to be arithmetically encoded is acquired (step S8), and 1 bit of symbol is read from the binarized symbol storage circuit 45 (step S9). Then, the context information and symbol are output to the arithmetic encoding circuit 46 (step S10). After the variable tmp is logically shifted by 1 bit to the left, symbol is connected to LSB (step S11). If the new tmp is different from the binarized symbol acquired in step S7, the process returns to step S8 (step S12). If tmp matches the binarized symbol, it is determined whether or not it is the end of the slice. If it is not the end of the slice, the process returns to step S6 (step S13) to clear tmp and obtain a new binarized symbol. (Steps S6 and S7). If it is the end of the slice, the context information restoration process is terminated (step S13).

However, the temporary mb_qp_delta of the first MB in the MB row is stored in the binarized symbol storage circuit 45, and the quantization parameter of the first MB in the MB row is reproduced by the context information restoration circuit 47. After the quantization parameter of the last MB in the MB row is reproduced and the difference from the parameter is calculated to obtain the actual mb_qp_delta, it must be binarized again and arithmetically encoded. At that time, it is necessary to reproduce the quantization parameter by decoding mb_qp_delta for all MBs included in the MB row . In order to avoid this, the quantization parameter itself may be stored in the binarized symbol storage circuit 45 together with mb_qp_delta for the last MB in the MB row. By storing the quantization parameter of the last MB in the MB row as a constant bit length, the context information restoration circuit 47 can identify the quantization parameter of the last MB in the MB row.

  Note that the binarized symbol storage circuit 45 may not be provided in the moving image encoding apparatus, and the binarized symbols may be stored in the external memory 4.

The method of this embodiment has the advantage that it is possible to arithmetic coding without arithmetic coding circuit have a capability enough to handle binary symbol generation amount increases suddenly. In particular, if the binarized symbol storage circuit 45 has a sufficient storage capacity to store the binarized symbols of the I picture and the binarized symbols of the P picture and B picture before and after the I symbol, the arithmetic coding circuit 46 is provided. There is an advantage that it is not necessary to have a performance sufficient to process the binarized symbol generation amount of an I picture, and it is sufficient to have a performance sufficient to process an average generation amount of binarized symbols of a plurality of pictures.

(Tenth embodiment)
In the present embodiment, a rate control method different from that in the ninth embodiment in the case where the present invention is applied to a moving image encoding apparatus using CABAC is shown. FIG. 29 is a block diagram of a CABAC encoder that performs rate control based on a provisional generated code amount. The binary symbol is temporarily stored in the binary symbol storage circuit 45, and is read out from the binary symbol storage circuit 45 to perform arithmetic coding while restoring the corresponding context information (hereinafter referred to as true arithmetic coding). The point of performing is the same as in the ninth embodiment . In this embodiment, the binarized symbol is temporarily arithmetic coding by the temporary arithmetic coding circuit 48 calculates the generated code amount of temporary, vertical to the encoding of all the MB included in the MB row of human unity the successive The difference is that the rate is controlled based on the provisional generated code amount before the end.

  In the first MB in the MB row, arithmetic coding is performed by taking over the range, low, and probability table when the arithmetic coding of the last MB in the adjacent MB row is finished. However, in the present invention, since encoding is performed with a plurality of vertically continuous MB rows as a group, when the first MB in the MB row is encoded, the last MB in the adjacent MB row is already encoded. Absent. For this reason, until encoding of a group of consecutive MB rows is completed, the arithmetic is performed by taking over the range, low, and probability table when the arithmetic encoding of the last MB in the adjacent MB row is finished. Encoding cannot be performed. Therefore, for the first MB in the MB row, provisional values are set in the range, low, and probability tables, and provisional arithmetic coding is started.

FIG. 30 shows a block diagram of the provisional arithmetic encoding circuit 48 when encoding is performed with a group of four vertically continuous MB rows as a group. The context calculation unit 49 performs context calculation for each symbol. There are four portions 51 to 54 for storing a range, portions 55 to 58 for storing a low, and probability tables 59 to 62 corresponding to four MB rows. The range (n) storage portion 51, the low (n) storage portion 55, and the probability table (n) 59 correspond to the uppermost MB row of the four MB rows. The range (n + 1) storage portion 52, the low (n + 1) storage portion 56, and the probability table (n + 1) 60 correspond to the second MB row from the top. The range (n + 2) storage portion 53, the low (n + 2) storage portion 57, and the probability table (n + 2) 61 correspond to the third MB row from the top, the range (n + 3) storage portion 54, and the low (n + 3) storage. Portion 58 and probability table (n + 3) 62 correspond to the bottom MB row. The contents of the parts 51 to 54 for storing the range and the parts 55 to 58 for storing the low are read into the arithmetic encoding unit 50 only when the arithmetic encoding of individual MBs included in the four MB rows is started. When the arithmetic coding of MB is finished, it is written and updated from the arithmetic coding unit 50 .

  There are three possible methods for setting a temporary range, low, and probability table for the first MB in the MB row. The first method is described in H.W. This is a method of performing initialization defined in H.264. H. H.264 initializes the range, low, and probability tables at the beginning of the slice. This initialization is performed when provisional arithmetic coding is started for the first MB in the MB row.

  In the second method, the range, low, and probability tables when the last MB in the MB row has been arithmetically encoded for each MB row in a group of vertically contiguous MB rows are This is a method of taking over to the first MB in a corresponding MB row in a group of adjacent MBs. FIG. 31 shows a case where encoding is performed with a group of four vertically continuous MB rows. FIG. 31 shows a picture composed of 12 MB rows. For the top four batches of MB in the picture, the range, low, and probability table of the first MB in each MB row is H.264. It is initialized by the method specified in H.264. For the individual MB rows included in the four MB rows in the second batch from the top, the arithmetic encoding of the last MB in the individual MB rows included in the uppermost four MB rows has been completed. Take over the time range, low, and probability table. Similarly, for the 4 MB rows in the third batch from the top, the arithmetic encoding of the last MB in each MB row included in the 4 MB rows in the second batch from the top is finished. Take over the range, low, and probability tables.

  The third method is the same as the range, low, and probability table inherited by the MB at the same position in the picture that has already been subjected to true arithmetic coding, and the temporary range, low, And setting as a probability table.

When a certain MB is encoded, the binarized symbols suddenly increase, and when the processing capacity of the provisional arithmetic encoding circuit 48 is exceeded, provisional arithmetic encoding is aborted on the way to the next MB. The provisional arithmetic coding may be started. When the generation amount of binarized symbols increases rapidly, the generated code amount is estimated from the generation amount of binarized symbols as in the ninth embodiment, and the quantization parameter of the MB to be encoded is increased. In addition, rate control can be performed by a method such as reducing the amount of binarized symbols generated. For this reason, it is not necessary to give the provisional arithmetic coding circuit 48 enough performance to handle the maximum amount of binarized symbols that occur unexpectedly. Also in the method of the present embodiment, as in the method of the ninth embodiment , if the binarized symbol storage circuit 45 has a sufficient storage capacity, the true arithmetic coding circuit 46 has a plurality of picture binarized symbols. There is an advantage that it is sufficient to have a performance sufficient to process the average generation amount.

A plurality of provisional arithmetic encoding circuits 48 may be provided. FIG. 32 is a block diagram of the provisional arithmetic coding circuit 68 when provisional arithmetic coding is performed in two parallel. Two context calculation units 49 and two arithmetic coding units 50 are provided. One arithmetic encoding unit 50 includes a range (n) storage portion 51, a low (n) storage portion 55, a probability table (n) 59, a range (n + 2) storage portion 53, a low (n + 2) storage portion 57, and A probability table (n + 2) 61 is connected, and provisional arithmetic coding is performed on the MBs included in the uppermost MB row and the third MB row from the top among a group of four MB rows. The other arithmetic encoding unit 50 includes a range (n + 1) storage portion 52, a low (n + 1) storage portion 56, a probability table (n + 1) 60, a range (n + 3) storage portion 54, and a low (n + 3) storage portion. 58 and a probability table (n + 3) 62 are connected, and provisional arithmetic coding is performed on the MBs included in the second MB row from the top and the lowest MB row among a group of four MB rows. In the MB pipeline stage configuration similar to that of the first embodiment , a state in which the MB of e row 10 column, f row 8 column, g row 6 column, h row 4 column in FIG. 1 flows through each stage. As shown in FIG. Thus, by performing provisional arithmetic coding in two parallels, provisional arithmetic coding of one MB can be performed over 2 MB cycles. Note that the memories 63 and 65 and the memories 64 and 66 in FIG. 32 are provided to hold binary symbols and syntax data for 2 MB cycles, respectively.

The method of the ninth embodiment requires a ratio α between the binary symbol length of the immediately previous encoded picture and the generated code amount. For this reason, when there is a scene change or the like, the estimation of the generated code amount becomes inaccurate. On the other hand, the method of the present embodiment, the value of the probability table used for temporary arithmetic coding as arithmetic coding of the same MB row MB is made to converge to accurate. For this reason, even when there is a scene change or the like, it is possible to perform rate control close to rate control based on the generated code amount by true arithmetic coding.

(Eleventh embodiment)
After the syntax data is compressed by a method other than binarization and temporarily stored, the syntax data can be restored and binarized or arithmetically encoded. For example, H.M. In H.264, in addition to CABAC, CAVLC is defined as a variable-length encoding method, but CAVLC can variable-code one piece of syntax data with 1 CK. FIG. 34 shows a block diagram of a CABAC encoder when compressed by CAVLC. The CAVLC encoded bit stream generated by compressing the syntax data by the CAVLC encoding circuit 69 is temporarily stored in the CAVLC encoded bit stream storage circuit 70. Then, the CAVLC encoded bit stream read from the CAVLC encoded bit stream storage circuit 70 is decoded by the CAVLC decoding circuit 71, the restored syntax data is binarized by the binarizing circuit 43, and the arithmetic encoding circuit 46 Perform arithmetic coding. Although FIG. 34, the tentative arithmetic coding was carried out, as has been shown a case where the rate control in the tentative generated code amount shown in the ninth embodiment, arithmetic coding from the sum of the symbol length of the binary symbol The amount of generated code may be estimated and the rate may be controlled.

Here, as pointed out in the ninth embodiment , in the video encoding apparatus to which the present invention is applied, when processing the first MB of the MB row, the last MB of the adjacent MB row is processed. The quantization parameter of is not yet determined. Therefore, as in the ninth embodiment , the difference between the quantization parameter of the first MB in the MB row and the temporary value (hereinafter referred to as temporary mb_qp_delta) is encoded by CAVLC, and the MBs other than the first MB in the MB row are encoded. As for the quantization parameter of, the difference (hereinafter referred to as mb_qp_delta) from that of the left adjacent MB encoded immediately before is encoded by CAVLC. Then, with respect to the quantization parameter of the first MB in the MB row, the quantization parameter of the last MB in the upper adjacent MB row is reproduced, and the difference between them is calculated to obtain the actual mb_qp_delta, and then binarized. Perform arithmetic coding. At that time, in order to avoid decoding mb_qp_delta for all MBs and reproducing the quantization parameter, the quantization parameter itself is stored in the CAVLC encoded bitstream storage circuit 70 together with mb_qp_delta for the last MB in the MB row. It is also good.

  Any method other than CAVLC can be used as long as it is a reversible compression method capable of compressing and restoring syntax data. That is, syntax data compressed by lossless compression of syntax data is temporarily stored. Then, the compressed syntax data is read to restore the syntax data. The restored syntax data is converted into a binary symbol and arithmetically encoded.

  H. In H.264, CAVLC and CABAC can be switched for each picture by specifying entropy_coding_mode_flag. Therefore, when CAVLC is used as the lossless compression method, the amount of binarized symbols generated for a certain MB suddenly increases, and it is predicted that the processing capacity of the true arithmetic encoding circuit 46 will be exceeded as a result of provisional arithmetic encoding. The entire picture including the MB is CAVLC encoded, and when the result of provisional arithmetic encoding is predicted to be processed by the true arithmetic encoding circuit 46, the entire picture including the MB is CABAC encoded. be able to. As a result, a situation in which the performance of the true arithmetic coding circuit 46 is unexpected can be surely avoided.

  Further, the CAVLC encoding circuit 69 calculates the code amount of the CAVLC encoded bit stream, and compares the CAVLC encoding and the provisional generated code amount of the provisional arithmetic encoding, and it is estimated that the generated code amount is small for each picture. An entropy coding mode can also be selected.

(Twelfth embodiment)
In the present embodiment , a method of generating one bit stream encoded by CABAC from the parallel video encoding device 42 composed of the two video encoding devices 40 and 41 shown in the eighth embodiment. Indicates. FIG. 35 shows a block diagram of a parallel moving image encoding device in which binarized symbols are output from the moving image encoding devices 40 and 41. The binarized symbols output from the moving image encoding devices 40 and 41 are temporarily stored in the common binarized symbol storage circuit 45. A binary symbol is read from the binary symbol storage circuit 45, and arithmetic coding is performed by the arithmetic coding circuit 46 while restoring the corresponding context information using the context information restoration circuit 47.

  FIG. 36 shows a block diagram of a parallel moving image encoding apparatus in which a CAVLC encoded bit stream is output from the moving image encoding apparatuses 40 and 41. The CAVLC encoded bit stream output from the moving image encoding devices 40 and 41 is temporarily stored in the common CAVLC encoded bit stream storage circuit 70. Then, the CAVLC encoded bit stream is read from the CAVLC encoded bit stream storage circuit 70, decoded by the CAVLC decoding circuit 71 and returned to syntax data, and then the syntax data is binarized and subjected to arithmetic encoding.

  The parallel video encoding device 42 may include three or more video encoding devices.

(13th Embodiment)
H. In H.264 MBAFF (Macroblock Adaptive Frame Field Coding), two vertically adjacent MBs are paired (hereinafter referred to as MB pairs), and encoding is performed by switching between field encoding and frame encoding every 16 pixels × 32 lines. To do. An MB pair at the time of field coding and frame coding is shown in FIG. FIG. 37A shows an MB pair at the time of field coding. Of the 32 lines of pixels, odd lines constitute the top field MB72, and even lines constitute the bottom field MB73. FIG. 37B shows an MB pair at the time of frame coding. Of the 32 lines, the upper half 16 lines constitute the upper frame MB74, and the lower half 16 lines constitute the lower frame MB75. The top field MB72 and the bottom field MB73 are not in the relationship of the adjacent MB to each other, but the upper frame MB74 is in the relationship of the upper adjacent MB to the lower frame MB75. For this reason, when frame coding is selected, there is a restriction that the PMV of the lower frame MB75 must be determined in consideration of the intra / inter determination result of the upper frame MB74. In addition, since intra-prediction of the lower frame MB75 uses the reconstructed pixel of the upper frame MB74, if the motion compensation is performed for the upper frame MB74 and the reconstructed pixel is not created, the intra-prediction cannot be performed for the lower frame MB75. There is.

  When field coding or frame coding is selected for both the current MB pair and the left adjacent MB pair, the left adjacent MB of each MB included in the current MB pair is only the MB located on the left side. However, when the field encoding / frame encoding selection of the current MB pair and the left adjacent MB pair is different, both MBs included in the left adjacent MB pair are left adjacent MBs of individual MBs included in the current MB pair. . FIG. 38A shows a case where field encoding is selected for the current MB pair and frame encoding is selected for the left adjacent MB pair. The odd line 76 of the upper frame MB and the odd line 77 of the lower frame MB included in the left adjacent MB pair correspond to the left adjacent MB of the top field MB72, and the even number of the upper frame MB corresponds to the left adjacent MB of the bottom field MB73. A line corresponds to an even line of the lower frame MB. Further, FIG. 38B shows a case where frame coding is selected for the current MB pair and field coding is selected for the left adjacent MB pair. The upper half 16 lines 78 of the top field MB and the upper half 16 lines 79 of the bottom field MB correspond to the left adjacent MB of the upper frame MB74, and the lower half 16 lines of the top field MB correspond to the left adjacent MB of the lower frame MB75. The lower half 16 lines of the bottom field MB correspond. For this reason, if motion compensation is performed for both MBs included in the left adjacent MB pair and the reproduced pixel is not calculated, intra prediction cannot be performed for the current MB pair.

In the present embodiment , in the case of MBAFF, eight MB rows that are vertically continuous are taken as a group, and one MB pair is selected from these batch of MB rows for every two MB rows that are vertically continuous. An example will be shown in which the four MB pairs are input to one MB pipeline. FIG. 39 is an explanatory diagram showing an example of MB pairs to be input to the MB pipeline. The picture is the same as that shown in FIG. 7, but as described above, in MBAFF, line assignment to two MBs differs depending on whether the MB pair is field-coded or frame-coded. Therefore, an MB that constitutes an MB pair is shown using uppercase letters. In the field coding, I15, K13, M11, O9, etc. are the top field MB72, and J15, L13, N11, P9, etc. are the bottom field MB73. In frame encoding, I15, K13, M11, O9, etc. are the upper frame MB74, and J15, L13, N11, P9, etc. are the lower frame MB75.

FIG. 40 is a block diagram of a video encoding apparatus according to the thirteenth embodiment. For example, as shown in FIG. 39, the MB pair selection circuit 81 includes an i-MB row, a j-MB row, a k-MB row, an l-MB row, an m-MB row, an n-MB row, an o-MB row, For the MB pairs belonging to the p-MB rows and belonging to the i-MB rows and j-MB rows, the MB pairs for which the lower left neighboring MB pair has not yet been selected are selected, and the k-MB rows and the l- For MB pairs belonging to the MB row, the m-MB row and the n-MB row, the o-MB row and the p-MB row, by selecting the MB pair in which the upper right adjacent MB pair has already been selected, I15-J15 , K13-L13, M11-N11, O9-P9 MB pairs are selected. When field encoding is selected, the field MB pair / frame MB pair configuration circuit 82 configures the top field MB72 with the odd lines of the MB pair and configures the bottom field MB73 with the even lines of the MB pair. On the other hand, when frame encoding is selected, the upper frame MB74 is configured by the upper half line of the MB pair, and the lower frame MB75 is configured by the lower half line of the MB pair. When field encoding is selected, the encoded data setting circuit 83 first takes 4 MB cycles for one of the MBs included in the MB pairs of I15-J15, K13-L13, M11-N11, and O9-P9. In order, data necessary for encoding individual MBs every 1 MB cycle is set in the MB pipeline 80. When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. Secondly, for each remaining MB included in the MB pair of I15-J15, K13-L13, M11-N11, and O9-P9, individual MBs are encoded every 1 MB cycle in order over 4 MB cycles. Data necessary for this is set in the MB pipeline 80. When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When frame encoding is selected, the data necessary for encoding each MB in each macroblock cycle in the order of I15MB, K13MB, M11MB, O9MB, J15MB, L13MB, N11MB, and P9MB Set to pipeline 80. When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When data necessary for encoding the last MB selected by the MB pair selection circuit 81 is set in the MB pipeline 80, and the MB is input to the MB pipeline 80, the MB pair selection circuit 81 The MB pair of I16-J16, K14-L14, M12-N12, and O10-P10 is newly selected.

FIG. 41 is a diagram illustrating a flowchart of the process of the moving image coding method according to the thirteenth embodiment. For example, as shown in FIG. 39, i-MB line, j-MB line, k-MB line, l-MB line, m-MB line, n-MB line, o-MB line, and p-MB line are grouped. For the MB pairs belonging to the i-MB row and the j-MB row, the MB pair for which the lower left adjacent MB pair has not yet been selected is selected, and the k-MB row, the l-MB row, and the m-MB row For the MB pairs belonging to the n-MB row, the o-MB row, and the p-MB row, by selecting the MB pair for which the upper right adjacent MB pair has already been selected, I15-J15, K13-L13, M11- The MB pair of N11 and O9-P9 is selected (step S14). Next, when field coding is selected, the top field MB72 is composed of the odd lines of the MB pair, and the bottom field MB73 is composed of the even lines of the MB pair. On the other hand, when frame coding is selected, the upper frame MB74 is configured by the upper half line of the MB pair, and the lower frame MB75 is configured by the lower half line of the MB pair (steps S15, S16, and step S16). S23). When field coding is selected, first, one MB included in the MB pairs of I15-J15, K13-L13, M11-N11, and O9-P9 is sequentially repeated every 4 MB cycles for every 1 MB cycle. Data necessary for encoding individual MBs is set in the MB pipeline 80 (step S17, step S18, step S19). When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. Next, in order to encode each MB for every 1 MB cycle in order over the remaining one MB included in the MB pairs of I15-J15, K13-L13, M11-N11, and O9-P9 over 4 MB cycles. Necessary data is set in the MB pipeline 80 (step S20, step S21, step S22). When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When frame encoding is selected, the data necessary for encoding individual MBs every 1 MB cycle in the order of I15MB, K13MB, M11MB, O9MB, J15MB, L13MB, N11MB, and P9MB is stored in the MB pipeline. 80 (step S24, step S25, step S26, step S27, step S28, step S29). When data necessary for encoding an MB is set in the MB pipeline 80, the MB is input into the MB pipeline 80. When data necessary for encoding the selected last MB is set in the MB pipeline 80, and the MB is input to the MB pipeline 80, the process returns to step S14 (steps S21 and S28), and I16. -MB pairs of J16, K14-L14, M12-N12, and O10-P10 are newly selected (step S14).

FIG. 42 shows that when MB pairs included in 8 MB rows are input to one MB pipeline in order to detect a motion vector for each 8 × 8 SB, each MB pair flows through each stage of the MB pipeline. Show how it goes. Here, since the number of pixels in the vertical direction of the field image is half that of the frame image, when the pixel values of two pixels in the vertical direction are averaged to create a reduced image, the reduced image differs between the field image and the frame image. . On the other hand, in a reduced image with vertical / vertical two-pixel accuracy, if an odd line is extracted from the reduced image of the frame, it becomes a reduced image of the top field, and if an even line is extracted from the reduced image of the frame, it becomes a reduced image of the bottom field. Therefore, in the present embodiment, as shown in FIG. 42, to set the reference region of the vertical single pixel / horizontal half-pixel precision in the memory in stages 0, the motion detection in the vertical Stage 1 single pixel / horizontal half-pixel precision Do.

In this embodiment , after four MBs in one of the MB pairs such as I15, K13, M11, and O9 are continuously inserted into the MB pipeline, the remaining MB pairs such as J15, L13, N11, and P9 are left. 4 MBs are continuously inserted. Thus, since one MB of the MB pair is continuously input into the MB pipeline, when the motion vector reference area is updated in that order, the update area becomes discontinuous. Therefore, when setting the vertical single pixel / horizontal two-pixel accuracy memory at stage 0, as shown in FIG. 22, the update areas of eight MB rows are set as a group of rectangular areas.

In addition, to select either field coding or frame coding, both intra prediction and inter prediction are performed for each of field coding and frame coding, and a combination of a coding mode and a prediction mode with a small prediction error. It is desirable to select. However, this method requires a huge amount of calculation. On the other hand, a simple selection method is also conceivable in which an encoding mode is determined only with pixels in the current MB pair, and intra prediction and inter prediction are performed only on the selected one. For example, for all 32 lines, the sum of absolute differences or sum of squares of differences is calculated between adjacent lines, and the sum of absolute differences or squares of differences is calculated for each line, that is, between adjacent lines of 16 odd lines and 16 even lines. When the sum is obtained and the value obtained for all 32 lines is smaller, frame coding is performed, and when the value obtained for each line is smaller, field coding is performed. Although it may not be an optimal selection, the amount of computation is small. In this embodiment , a simple selection method is used, and after performing field / frame determination in stage 0, inter prediction is performed in stages 1-4 as shown in FIG. 42 for the selected coding mode. In parallel with this, 4 × 4 intra prediction, 16 × 16 intra prediction, and the like are performed in stages 3 and 4.

In the present embodiment , as in the second to fourth embodiments , eight MB rows that are vertically continuous are processed as a group. However, as pointed out in paragraph 0122 above, when frame coding is selected, the upper frame MB74 is in the relationship of the upper adjacent MB with respect to the lower frame MB75. Also, as pointed out in paragraph 0123 above, when the field encoding / frame encoding selection of the current MB pair is different from that of the left adjacent MB pair, both MBs included in the left adjacent MB pair are included in the current MB pair. It becomes the left adjacent MB of each MB. For this reason, the processing result of the upper frame MB74 can be fed back to the processing of the lower frame MB75 only for three stages, and the processing result of the left adjacent MB pair can be fed back to the processing of the current MB pair only by three stages. The same limitation as in the fifth embodiment in which four consecutive MB rows are processed as a group occurs.

  Specifically, for example, when I15 MB intra / inter determination is completed in stage 5, J15MB vertical single pixel / horizontal two pixel accuracy motion detection is also ended in stage 1 at the same time. For this reason, there is a limitation that the intra / inter determination result of the upper frame MB74 cannot be reflected in the vertical single pixel / horizontal two-pixel accuracy motion detection of the lower frame MB75. However, as indicated by arrows in FIG. 42, the intra / inter determination result of the upper frame MB74 can be reflected in the stage 2 and subsequent stages. Also, the intra / inter determination results determined in stage 5 for the left adjacent MB pair, the upper adjacent MB pair, the upper right adjacent MB pair, and the like can be reflected in the vertical single pixel / horizontal two pixel accuracy motion detection in stage 1. Therefore, in stage 1, a temporary PMV is created with reference to motion vectors such as the left adjacent MB pair, the upper adjacent MB pair, and the upper right adjacent MB pair, and all four 8 × 8 SBs in the lower frame MB75 are shared. With reference to the provisional PMV, the vertical single pixel / horizontal two-pixel accuracy motion detection is performed. Then, in stage 2, the peripheral area of the candidate motion vector of vertical single pixel / horizontal two pixel precision detected in stage 1 is set in the single pixel precision memory, and a true PMV is created in stage 3, and stage 3 and stage 4 Then, single-pixel accuracy motion detection and quarter-pixel accuracy motion detection are performed with reference to the true PMV.

  FIG. 43 shows the detailed operation of the SB pipeline in stages 3 and 4. Four 8 × 8 SBs included in one MB are serially processed from No. 0 to No. 3. For this reason, in the single-pixel precision motion detection and the quarter-pixel precision motion detection, it is possible to perform motion detection based on a true PMV in consideration of the adjacent 8 × 8 SB motion vector.

However, in the present embodiment, the evaluation of the spatial direct address required separately. In stage 1, vertical single pixel / horizontal 2 pixel precision motion detection is performed based on the provisional PMV, and in stage 2, the peripheral area of the candidate motion vector detected in stage 1 is set in the single pixel precision memory. When the temporary PMV of stage 1 and the true PMV of stage 3 are different, the area where motion is detected in stages 3 and 4 may not include the MB corresponding to the spatial direct address, 8 × 8 SB, or the like. Therefore, it is necessary to perform processing for evaluating the MB of the position corresponding to the spatial direct address and the cost of 8 × 8 SB in parallel with stage 3 and stage 4.

FIG. 44 shows how each MB pair flows through each stage of the MB pipeline when an MB pair included in 8 MB rows is inserted into one MB pipeline for 4 × 4 intra prediction. Show. FIG. 45 shows the detailed operation of the SB pipeline in stages 3 and 4. Here, in order to satisfy the conditions pointed out in the paragraphs 0122 and 0123, motion compensation is performed in the stage 6 as in the fifth embodiment . That is, intra prediction is performed for the lower frame MB75 after motion compensation is performed for the upper frame MB74, and intra prediction is performed for the current MB pair after motion compensation is performed for both MBs included in the left adjacent MB pair. In order to satisfy the condition that it is performed, motion compensation is performed at stage 6.

In addition, in order to adjust fluctuations in the generated code amount due to variable length coding in stage 7, it may be necessary to perform rate control in stage 8 to change the quantization parameter. In this case, as in the fifth embodiment , the 4 × 4 intra prediction mode is not selected in the intra / inter determination of stage 5 or when the 4 × 4 intra prediction mode is selected, in stage 6 It is necessary to redo integer transformation, quantization, inverse quantization, and inverse integer transformation using new quantization parameters.

Incidentally, the stage construction of the MB pipeline shown in this embodiment is an example, it is also possible to other stage arrangement. If the stage configuration is changed, the constraints in motion detection, intra prediction, and the like change even if the number of vertically continuous MB rows encoded as a group is the same.

It is also possible to apply simultaneously the video encoding apparatus of the invention described in has been the invention of the sixth to twelfth embodiments of the described embodiment.

It is explanatory drawing which showed the example of MB thrown continuously into MB pipeline. It is a block diagram of the moving image encoder to which the present invention is applied. It is a flowchart of a process of the moving image encoding method to which this invention is applied. FIG. 2 is an explanatory diagram showing a state in which each MB flows through each stage of the MB pipeline when the four MBs exemplified in FIG. 1 are continuously input into one MB pipeline. FIG. 2 is an explanatory diagram showing MB cycle numbers at which the MBs constituting the picture of FIG. 1 are input to the MB pipeline and stage 0 processing is started. It is explanatory drawing which showed MB divided | segmented into 4 SB of 8 pixels x 8 pixels. It is explanatory drawing which showed the example of MB thrown continuously into MB pipeline. In order to detect a motion vector for every 8 pixels × 8 pixels SB, when MBs included in 8 MB rows are continuously input to one MB pipeline, each MB flows through each stage. It is explanatory drawing shown. It is a block diagram of a motion vector detection circuit to which the present invention is applied. It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. It is a block diagram of a motion vector detection circuit to which the present invention is applied. It is explanatory drawing which showed MB divided | segmented into 16 4 pixels x 4 pixels SB. An explanatory diagram showing how each MB flows through each stage when MBs included in 8 MB rows are continuously input to one MB pipeline for 4 pixel × 4 pixel intra prediction. is there. It is explanatory drawing which showed the detailed operation | movement of SB pipeline for 4 pixel x 4 pixel intra prediction. When MBs included in four MB rows are continuously input to one MB pipeline in order to detect a motion vector for each 8 × 8 SB, each MB flows through each stage of the MB pipeline. It is explanatory drawing which showed. It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. An explanatory diagram showing how each MB flows through each stage when MBs included in four MB rows are continuously input to one MB pipeline for 4 pixel × 4 pixel intra prediction. is there. It is explanatory drawing which showed the detailed operation | movement of SB pipeline for 4 pixel x 4 pixel intra prediction. It is a block diagram of the moving image encoder which has an internal memory. It is explanatory drawing which showed the reference area | region in the case of processing 4 MB contained in 4 vertical MB lines by one MB pipeline. It is explanatory drawing which showed another method which memorize | stores a reference area in the memory for reference areas. It is explanatory drawing which showed the combination of the motion detection apparatus which can detect a motion in a wide range, and the moving image encoding apparatus containing MB pipeline to which this invention was applied. It is a block diagram of a parallel moving image encoding device having two moving image encoding devices and an internal memory. It is explanatory drawing which showed the area | region of the picture which two moving image encoders share. It is explanatory drawing which showed a mode that MB flows through each stage of MB pipeline contained in two moving image encoders. It is a block diagram of a CABAC encoder that performs rate control based on an estimated generated code amount. It is a figure which shows the processing flow of a context information restoration circuit. It is a block diagram of a CABAC encoder that performs rate control based on a provisional generated code amount. It is a block diagram of a provisional arithmetic encoding circuit when encoding is performed with a group of four vertically continuous MB rows. It is a figure explaining one of the methods of setting a temporary range, low, and a probability table about the head MB in MB row in a temporary arithmetic coding circuit. It is a block diagram of a provisional arithmetic coding circuit when performing provisional arithmetic coding in two parallel. It is explanatory drawing which showed a mode that MB flows through each stage of MB pipeline, when provisional arithmetic coding is performed in 2 parallel. It is a block diagram of the CABAC encoder in the case of compressing by CAVLC. FIG. 2 is a block diagram of a parallel moving image encoding device that outputs a binary symbol from a parallel moving image encoding device configured by two moving image encoding devices and generates one bit stream encoded by CABAC. is there. Block diagram of a parallel video encoding device that outputs a CAVLC encoded bitstream from a parallel video encoding device composed of two video encoding devices and generates one bitstream encoded by CABAC It is. It is explanatory drawing which showed MB pair when a frame encoding is selected, and MB pair when field encoding is selected. Field coding is selected for the current MB pair, frame coding is selected for the left adjacent MB pair when the frame coding is selected for the left adjacent MB pair, frame coding is selected for the current MB pair, and field coding is performed for the left adjacent MB pair It is explanatory drawing which showed the left adjacent MB pair when is selected. It is explanatory drawing which showed the example of the MB pair thrown into MB pipeline. It is a block diagram of the moving image encoder which concerns on 13th Embodiment . It is a figure which shows the flowchart of a process of the moving image encoding method which concerns on 13th Embodiment. In order to detect a motion vector for every 8 pixels × 8 pixels SB, when MB pairs included in 8 MB rows are input to one MB pipeline, each MB pair flows through each stage. FIG. It is explanatory drawing which showed the detailed operation | movement of SB pipeline for detecting the motion vector for every 8 pixel x 8 pixel SB. When MB pairs included in 8 MB rows are input to one MB pipeline for 4 pixel × 4 pixel intra prediction, each MB pair flows through each stage of the MB pipeline. It is explanatory drawing. It is explanatory drawing which showed the detailed operation | movement of SB pipeline for 4 pixel x 4 pixel intra prediction. It is a block diagram of an MPEG-2 encoder. H. 2 is a block diagram of an H.264 encoder. FIG. Conventional H.264. 2 is an explanatory diagram showing a state in which an MB flows through each stage of the MB pipeline in an MB pipeline of an H.264 encoder. FIG. It is the block diagram which showed the structure of the conventional moving image encoder . It is explanatory drawing which showed the example of the reference area of two adjacent MB. It is explanatory drawing which showed the picture division | segmentation method. It is a block diagram of a CABAC encoder. It is a Biari_encode_symbol processing flow showing the processing contents of the arithmetic encoding circuit. It is explanatory drawing which showed the change of range and low when symbol is LPS, and the change of range and low when symbol is MPS.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1,80 ... MB pipeline, 2 ... MB selection circuit, 3, 83 ... Encoded data setting circuit, 14 ... Memory for reference area, 15 ... Buffer memory, 20 ... Variable length encoding part, 23, 40, 41 ... Moving picture encoding device, 24, 25, 26, 27 ... MB, 28, 29, 30, 31, 36 ... Reference area, 32, 33, 34, 35, 37 ... Update area, 28, 29, 30, 34 Rectangular area including all, 39 ... internal memory, 42 ... parallel video encoding device, 43 ... binarization circuit, 44 ... generated code amount estimation circuit, 45 ... binary symbol storage circuit, 46 ... arithmetic encoding circuit 47 ... Context information restoration circuit, 48 ... Temporary arithmetic coding circuit, 68 ... Temporary arithmetic coding circuit for performing provisional arithmetic coding in parallel, 69 ... CAVLC coding circuit, 70 ... CAVLC coded bitstream storage circuit, 71 ... AVLC decoding circuit, 72 ... top field MB, 73 ... bottom field MB, 74 ... upper frame MB, 75 ... lower frame MB, 76 ... upper frame MB odd line, 77 ... lower frame MB odd line, 78 ... top field MB Half 16 lines, 79 ... Bottom field MB Upper half 16 lines, 81 ... MB pair selection circuit, 82 ... Field MB pair / Frame MB pair configuration circuit

The moving picture coding apparatus according to the present invention includes m rows (m is an integer equal to or greater than 2) that are macroblock rows that are a collection of macroblocks arranged in a row in the horizontal direction from the left end to the right end of a picture. It is a moving picture encoding apparatus that performs encoding as a unit, and in each macroblock row, starts encoding from the leftmost macroblock and sequentially encodes macroblocks adjacent to the right, A macroblock that divides the video encoding process into a plurality of stages, processes each macroblock by one macroblock cycle for each stage, and finishes encoding one macroblock every macroblock cycle It has a pipeline and is included in the top macroblock row of the group of macroblock rows every m macroblock cycles. A macroblock is input to the macroblock pipeline, the macroblock pipeline starts encoding the macroblock, and the lowest macroblock line from the top of the group of macroblock lines is the lowest. The macroblocks included in each macroblock row up to a macroblock row of the macroblock in order, and the macroblocks included in the macroblock row adjacent above the macroblock pipeline are inserted into the macroblock pipeline one macroblock cycle later. The macroblock pipeline starts to encode the macroblock.

Preferably , in the moving picture encoding apparatus according to the present invention, the macroblock included in a macroblock row from a second macroblock row to a lowermost macroblock row in the group of macroblock rows. When the macroblock is processed in the first stage included in the macroblock pipeline, the macroblock is input to the macroblock pipeline one macroblock cycle before the macroblock is input to the macroblock pipeline. Except for the processed macroblock, the macroblock included in the macroblock row adjacent above can be processed using the result processed in the second stage executed after the first stage.

Preferably , the moving picture encoding apparatus according to the present invention is configured such that the macroblock pipe in the macroblock row from the second macroblock row to the lowest macroblock row in the group of macroblock rows. The macroblock that is input to the line is located to the left of the macroblock that is input to the macroblock pipeline one macroblock cycle before the macroblock is input to the macroblock pipeline. The macroblock input to the macroblock pipeline one macroblock cycle input to the macroblock pipeline is a macroblock that is not the upper right adjacent macroblock.

Also, the moving picture encoding apparatus of the present invention converts a macro block row, which is a collection of macro blocks arranged in a row in the horizontal direction from the left end to the right end of a picture, into 2n rows (n is 2 or more). (Integer) are grouped together and a macroblock pair is created by combining two adjacent macroblocks in the vertical direction, and encoding is performed by switching between field encoding and frame encoding for each macroblock pair. A moving picture coding apparatus which starts coding from a leftmost macro block pair and sequentially codes right adjacent macro block pairs in two adjacent macro block lines, Is divided into a plurality of stages, and each macroblock is processed over one macroblock cycle for each stage, and every macroblock cycle is processed. A macroblock pipeline that finishes encoding one macroblock, and a macro that belongs to the two highest macroblock rows in the group of macroblock rows every 2n macroblock cycles One macroblock included in the block pair is input to the macroblock pipeline, the macroblock pipeline starts encoding the one macroblock, and the top three in the group of macroblock rows From the first and fourth macroblock rows to the two lowermost macroblock rows, one macroblock included in the macroblock pair belonging to the two macroblock rows adjacent in the vertical direction is sequentially , One of the macroblock pairs belonging to the two adjacent macroblock rows A macro block is inserted into the macro block pipeline one macro block cycle after the macro block is input into the macro block pipeline, the macro block pipeline starts encoding the one macro block, and the group of macro blocks One macroblock included in the macroblock pair belonging to the two lowest macroblock rows in the block row is input to the macroblock pipeline, and after one macroblock cycle, The other macroblock that is included in the macroblock pair belonging to the two topmost macroblock rows in the macroblock row and has not yet been put into the macroblock pipeline Is the macroblock pie The macroblock pipeline starts encoding the other macroblock, and the third and fourth macroblock rows from the top of the group of macroblock rows Other macroblocks included in a macroblock pair belonging to two macroblock rows adjacent in the vertical direction up to two macroblock rows positioned below and not yet put into the macroblock pipeline After one macroblock cycle, the other macroblock included in the macroblock pair belonging to the two macroblock rows adjacent to each other in order is put into the macroblock pipeline one macroblock cycle later. To the block pipeline, and the macroblock pipeline is connected to the other To start the encoding of black block.

Preferably, the moving picture coding apparatus according to the present invention includes a group of macroblock rows from the top to the third and fourth macroblock rows to the bottom two macroblock rows. One macroblock included in the macroblock pair belonging to two macroblock rows adjacent in the vertical direction is a first stage in which one macroblock included in the macroblock pair is included in the macroblock pipeline. Except for a macroblock pair that includes one macroblock that was input to the macroblock pipeline one macroblock cycle before the one macroblock was input to the macroblock pipeline. A macro block included in a macro block pair belonging to two macro block rows adjacent to Can be processed using the results processed in the second stage executed after the first stage, and the third and fourth rows from the top of the group of macroblock rows. The other macroblock included in the macroblock pair belonging to the two macroblock rows vertically adjacent from the first macroblock row to the two lowermost macroblock rows is the macroblock. When one other macroblock included in the pair is processed in the first stage included in the macroblock pipeline, one macroblock cycle in which the other macroblock is input to the macroblock pipeline A macro that contains another macroblock that was previously entered into the macroblock pipeline Except for the lock pair, the macroblocks included in the macroblock pairs belonging to the two adjacent macroblock rows are processed using the result processed in the second stage executed after the first stage. Can.

Preferably, the moving picture encoding apparatus according to the present invention includes the macroblock rows located at the bottom of the third and fourth macroblock rows from the top of the group of macroblock rows, In a macroblock pair including one macroblock and one other macroblock that are input to the macroblock pipeline, the one macroblock or the other macroblock is input to the macroblock pipeline. One macroblock cycle before the macroblock pair including the macroblock input to the macroblock pipeline, the one macroblock or the other macroblock is placed in the macroblock pipeline. One macroblock cycle before the input macroblock Macroblock pair that contains the macroblock is input to the pipeline is a macro block is not at the top right neighboring macroblock pair.

Preferably, the moving picture coding apparatus according to the present invention starts coding from the uppermost group of macroblock rows of a picture, and sequentially encodes the lowermost group of macroblock rows adjacent to each other.

Further , the moving picture coding apparatus of the present invention divides the moving picture coding process into a plurality of stages, processes each macro block for one macro block cycle for each stage, and 1 for each macro block cycle. A macroblock pipeline that finishes encoding one macroblock and a macroblock row that is a collection of macroblocks arranged in a row in the horizontal direction from the left end to the right end of the picture are at least 2 in the vertical direction. For each macroblock belonging to the uppermost macroblock row in the group of macroblock rows, a macroblock for which the lower left adjacent macroblock has not yet been selected is selected, and the batch is selected. For macroblocks belonging to macroblock lines other than the topmost A macroblock selection unit that selects one macroblock per macroblock row by selecting a macroblock in which the right upper adjacent macroblock has already been selected, and the macroblock For the individual macroblocks selected by the block selection unit, the macroblocks belonging to the uppermost macroblock row in the group of macroblock rows to the macroblocks belonging to the lowermost macroblock row are sequentially 1 An encoded data setting unit that sets data necessary for encoding individual macroblocks in each macroblock cycle in the macroblock pipeline, and encodes the macroblock by the encoded data setting unit. The data required for The macroblock pipeline starts encoding the macroblock, and the encoded data setting unit encodes a macroblock belonging to the lowest macroblock row in the group of macroblock rows. When the necessary data is set in the macroblock pipeline, the macroblock selection unit newly selects the right adjacent macroblock for each selected macroblock.

Preferably, in the moving image encoding apparatus of the present invention , the macroblock pipeline includes a subblock pipeline that performs processing for each subblock into which the macroblock is divided.

Preferably, the moving picture encoding apparatus of the present invention includes an internal memory for storing all data necessary for encoding one macroblock included in the group of macroblock rows, one for each macroblock row. Have.

Preferably, the moving picture coding apparatus according to the present invention includes at least two moving picture coding apparatuses according to any one of claims 1 to 10 and is continuous in the vertical direction. A group of macroblock rows is allocated to each of the moving picture encoding devices, and the group of macroblock rows continuous in the vertical direction is encoded in parallel.

Preferably, the moving picture coding apparatus according to the present invention includes a binarization unit that converts syntax data into a binarized symbol, a binarized symbol storage unit that stores the binarized symbol, and the binarization. A context information restoration unit that restores context information corresponding to the binarized symbol read from the symbol storage unit; and a context in which the binarized symbol read from the binarized symbol storage unit is restored by the context information restoration unit An arithmetic encoding unit that performs arithmetic encoding based on the information.

Preferably, the moving image encoding apparatus of the present invention includes a provisional arithmetic encoding unit that calculates provisionally generated code amounts by provisionally arithmetically encoding the binary symbols converted by the binarization unit.

Preferably, the moving image encoding apparatus of the present invention includes a syntax data compression unit that reversibly compresses syntax data, and a compression syntax data storage unit that stores syntax data compressed by the syntax data compression unit. A syntax data restoring unit that reads the compressed syntax data from the compressed syntax data storage unit and restores the syntax data; and the syntax data restored by the syntax data restoring unit is binary A binarizing unit that converts the binarized symbol, and an arithmetic coding unit that arithmetically codes the binarized symbol converted by the binarizing unit.

Preferably, in the moving image encoding apparatus of the present invention, the syntax data compression unit compresses the syntax data by CAVLC.

The present invention relates to a moving image encoding apparatus for compressing and reducing the amount of moving image data.

Arithmetic coding circuit enables arithmetic coding not have a performance sufficient to handle the binary symbol generation amount increases suddenly.

The moving image encoding apparatus of the present invention starts encoding from the leftmost macroblock in the macroblock row that is a collection of macroblocks arranged in the horizontal direction from the left end to the right end of the picture, A video encoding unit that sequentially encodes adjacent macroblocks;
The moving image coding process is divided into at least two processes, and at least two stages for performing the divided processes are included, and variable length coding for converting syntax elements into binary symbols in the stages is included. Including a macroblock pipeline that includes stages,
Selecting the macroblock rows as a group with at least two rows that are continuous in the vertical direction and having a number of rows smaller than the total number of the macroblock rows included in the entire picture;
Injecting the macroblock included in the top macroblock row in the group of macroblock rows into the macroblock pipeline;
Macros adjacent on each macroblock row in order from the macroblock included in the second macroblock row from the top to the macroblock included in the bottommost macroblock row in the group of macroblock rows. After the previous input macroblock included in the block row is input to the macroblock pipeline, it is included in each macroblock row and is located to the left of the previous input macroblock, and the previous input macroblock is adjacent to the upper right. Put macroblocks that are not macroblocks into the macroblock pipeline
A moving image encoding unit;
A binarized symbol storage unit in which the binarized symbols are written in the variable length coding stage;
The binarized symbols are read from the binarized symbol storage unit in order from the leftmost macroblock to the rightmost macroblock for the uppermost macroblock row in the group of macroblock rows, The binarized symbol storage unit sequentially from the top macroblock row to the bottom macroblock row from the top to the bottom macroblock row in order from the leftmost macroblock to the rightmost macroblock for each macroblock row A context information restoration unit that reads out the binarized symbol from and restores the context information corresponding to the read binarized symbol;
An arithmetic encoding unit that arithmetically encodes the read binary symbol based on the restored context information;
Have

Further , the moving image encoding apparatus of the present invention starts encoding from the leftmost macroblock in the macroblock row which is a collection of macroblocks arranged in a row in the horizontal direction from the left end to the right end of the picture, A video encoding unit that sequentially encodes macroblocks adjacent to the right;
The video encoding process is divided into two or more processes, and two or more stages for performing the divided processes are included, and the syntax elements compressed by reversibly compressing the syntax elements are included in the stages. A macroblock pipeline that includes a variable-length encoding stage to be generated;
Injecting the macroblock included in the top macroblock row in the group of macroblock rows into the macroblock pipeline;
Macros adjacent on each macroblock row in order from the macroblock included in the second macroblock row from the top to the macroblock included in the bottommost macroblock row in the group of macroblock rows. After the previous input macroblock included in the block row is input to the macroblock pipeline, it is included in each macroblock row and is located to the left of the previous input macroblock, and the previous input macroblock is adjacent to the upper right. Put macroblocks that are not macroblocks into the macroblock pipeline
A moving image encoding unit;
A compressed syntax element storage unit in which the compressed syntax element is written in the variable length encoding stage;
The compressed syntax elements are read from the compressed syntax element storage unit in order from the leftmost macroblock to the rightmost macroblock for the uppermost macroblock row in the group of macroblock rows, The compression syntax elements in a group of macroblock rows from the top to the bottom macroblock row from the top to the bottom macroblock row in order from the leftmost macroblock to the rightmost macroblock for each macroblock row A syntax element restoring unit that reads the compressed syntax element from the storage unit and restores the syntax element from the read compressed syntax element;
A binarization unit that converts the restored syntax element into a binarized symbol;
An arithmetic encoding unit for arithmetically encoding the converted binary symbol;
Have

Preferably, in the moving image encoding apparatus of the present invention, the moving image encoding process is performed according to the moving image encoding standard H.264. H.264,
The variable length encoding stage compresses the syntax element with CAVLC.

Preferably , the moving picture coding apparatus according to the present invention starts coding from the group of macroblock rows at the top of the picture, and sequentially codes the group of macroblock rows adjacent below.

The moving picture coding apparatus according to the present invention also includes a macroblock row in which macroblock rows, which are a set of macroblocks arranged in a row in the horizontal direction from the left end to the right end of a picture, are grouped into two adjacent rows in the vertical direction. A pair of macroblocks that are adjacent to each other in the macroblock row group and create a macroblock pair, and select frame coding and field coding for each macroblock pair. In the set of macroblock rows, encoding is started from the leftmost macroblock pair, and a moving picture encoding unit that sequentially encodes adjacent macroblock pairs on the right,
The moving image coding process is divided into four or more processes, and four or more stages for performing the divided processes are included, and variable length coding for converting syntax elements into binary symbols in the stages is included. A macroblock pipeline that includes stages,
A set of two or more sets of macroblock rows that are consecutive in the vertical direction, and a set of macroblock rows that is smaller than the total number of sets of macroblock rows included in the entire picture is selected as a group;
Select whether to encode the macroblock pair belonging to the topmost macroblock row set in the group of macroblock rows by frame encoding or field encoding, and select frame encoding When the frame encoding is selected, the upper frame macroblock included in the macroblock pair is input to the macroblock pipeline. When field coding is selected, one macroblock included in the macroblock pair is input to the macroblock pipeline. Throw in,
Macroblocks adjacent to each set of macroblock rows in order from the second set of macroblock rows from the top to the bottom set of macroblock rows in the set of macroblock rows After a previous input macroblock included in a macroblock pair belonging to a set of rows is input to the macroblock pipeline, the macroblock pair belonging to the set of each macroblock row and to the left of the macroblock pair including the previous input macroblock When a macro block pair that is located and the macro block pair including the immediately preceding input macro block is not the upper right adjacent macro block pair is to be encoded by frame encoding or field encoding, and frame encoding is selected Indicates the upper frame macroblock included in the macroblock pair as the macroblock. Tsu put into click pipeline, when you select the field coding charged with macroblocks of one included in the macroblock pair in the macroblock pipeline,
After a macroblock included in a macroblock pair belonging to the lowest set of macroblock rows in the set of macroblock rows is input to the macroblock pipeline, the set of macroblocks The macroblock pipeline belongs to each macroblock row set in order from the topmost macroblock row set to the bottommost macroblock row set in the row set, and only one macroblock belongs to the macroblock pipeline. The other one of the macroblocks included in the macroblock pair that has already been input but not yet input to the macroblock pipeline is input to the macroblock pipeline.
A moving image encoding unit;
A binarized symbol storage unit in which the binarized symbols are written in the variable length coding stage;
The binarized symbols are read from the binarized symbol storage unit in order from the leftmost macroblock pair to the rightmost macroblock pair for the uppermost macroblock row set in the group of macroblock rows. The leftmost macroblock for each set of macroblock rows in sequence, from the second set of macroblock rows to the lowest set of macroblock rows in the set of macroblock rows A context information restoring unit that reads the binarized symbols from the binarized symbol storage unit in order from the pair to the rightmost macroblock pair, and restores the context information corresponding to the read binarized symbols; ,
An arithmetic encoding unit that arithmetically encodes the read binary symbol based on the restored context information;
Have

The moving picture coding apparatus according to the present invention also includes a macroblock row in which macroblock rows, which are a set of macroblocks arranged in a row in the horizontal direction from the left end to the right end of a picture, are grouped into two adjacent rows in the vertical direction. A pair of macroblocks that are adjacent to each other in the macroblock row group and create a macroblock pair, and select frame coding and field coding for each macroblock pair. In the set of macroblock rows, encoding is started from the leftmost macroblock pair, and a moving picture encoding unit that sequentially encodes adjacent macroblock pairs on the right,
The video encoding process is divided into four or more processes, and four or more stages for performing the divided processes are included, and the syntax element compressed by reversibly compressing the syntax element is included in the stage. A macroblock pipeline that includes a variable-length encoding stage to be generated;
A set of two or more sets of macroblock rows that are consecutive in the vertical direction, and a set of macroblock rows that is smaller than the total number of sets of macroblock rows included in the entire picture is selected as a group;
Select whether to encode the macroblock pair belonging to the topmost macroblock row set in the group of macroblock rows by frame encoding or field encoding, and select frame encoding When the frame encoding is selected, the upper frame macroblock included in the macroblock pair is input to the macroblock pipeline. When field coding is selected, one macroblock included in the macroblock pair is input to the macroblock pipeline. Throw in,
Macroblocks adjacent to each set of macroblock rows in order from the second set of macroblock rows from the top to the bottom set of macroblock rows in the set of macroblock rows After a previous input macroblock included in a macroblock pair belonging to a set of rows is input to the macroblock pipeline, the macroblock pair belonging to the set of each macroblock row and to the left of the macroblock pair including the previous input macroblock When a macro block pair that is located and the macro block pair including the immediately preceding input macro block is not the upper right adjacent macro block pair is to be encoded by frame encoding or field encoding, and frame encoding is selected Indicates the upper frame macroblock included in the macroblock pair as the macroblock. Tsu put into click pipeline, when you select the field coding charged with macroblocks of one included in the macroblock pair in the macroblock pipeline,
After a macroblock included in a macroblock pair belonging to the lowest set of macroblock rows in the set of macroblock rows is input to the macroblock pipeline, the set of macroblocks The macroblock pipeline belongs to each macroblock row set in order from the topmost macroblock row set to the bottommost macroblock row set in the row set, and only one macroblock belongs to the macroblock pipeline. The other one of the macroblocks included in the macroblock pair that has already been input but not yet input to the macroblock pipeline is input to the macroblock pipeline.
A moving image encoding unit;
A compressed syntax element storage unit in which the compressed syntax element is written in the variable length encoding stage;
The compressed syntax element from the compressed syntax element storage unit in order from the leftmost macroblock pair to the rightmost macroblock pair for the uppermost macroblock row set in the set of macroblock rows. Are sequentially read from the second set of macroblock rows to the lowest set of macroblock rows in the set of macroblock rows, and the leftmost for each set of macroblock rows. The compressed syntax element is read from the compressed syntax element storage unit in order from the macro block pair to the rightmost macro block pair, and the syntax element is restored from the compressed syntax element that has been read. A syntax element restoration unit to perform,
A binarization unit that converts the restored syntax element into a binarized symbol;
An arithmetic encoding unit for arithmetically encoding the converted binary symbol;
Have

Preferably, in the moving image encoding apparatus of the present invention, the moving image encoding process is performed according to the moving image encoding standard H.264. H.264,
The variable length encoding stage compresses the syntax element with CAVLC.

Preferably, the moving picture coding apparatus according to the present invention starts coding from the set of macroblock rows at the top of a picture, and sequentially sets the set of macroblock rows adjacent below. Encode.

According to the present invention, arithmetic coding can be performed even if the arithmetic coding circuit does not have a performance sufficient to process a binarized symbol generation amount that suddenly increases. In particular, if the binarized symbol storage circuit has sufficient storage capacity to store the binarized symbols of the I picture and the binarized symbols of the P picture and B picture before and after the I symbol, the performance of the arithmetic coding circuit can be improved. Arithmetic coding can be performed even if the performance of the binarized symbol generation amount that suddenly occurs in the I picture is less than the capability of processing.

In addition , when the amount of binarized symbols generated in a certain MB suddenly increases and is predicted to exceed the processing capability of a true arithmetic encoding circuit as a result of provisional arithmetic encoding, the entire picture including that MB is encoded by the CAVLC code. On the other hand, when it is predicted that the result can be processed by a true arithmetic encoding circuit as a result of provisional arithmetic encoding, the entire picture including the MB can be CABAC encoded. As a result, a situation in which the performance of the true arithmetic coding circuit is unexpected can be surely avoided. Furthermore, the CAVLC encoding circuit calculates the code amount of the CAVLC encoded bitstream, compares the temporarily generated code amount of CAVLC encoding and provisional arithmetic encoding, and entropy that is estimated to be small for each picture. An encoding mode can also be selected.

Hereinafter, a moving picture coding apparatus according to an embodiment of the present invention will be described with reference to the drawings.

The moving picture encoding apparatus of the present invention starts encoding from the leftmost macroblock in the macroblock row which is a collection of macroblocks arranged in the horizontal direction from the left end to the right end of the picture, and A video encoding unit that sequentially encodes adjacent macroblocks;
The video encoding process is divided into at least two processes, and at least two stages for performing the divided processes are included, and a syntax element compressed by lossless compression of the syntax element is included in the stage. Including a macroblock pipeline that includes a variable length encoding stage to generate ,
Selecting the macroblock rows as a group at least two rows that are continuous in the vertical direction and having a number smaller than the total number of the macroblock rows included in the entire picture;
Injecting the macroblock included in the top macroblock row in the group of macroblock rows into the macroblock pipeline;
Macros adjacent on each macroblock row in order from the macroblock included in the second macroblock row from the top to the macroblock included in the lowest macroblock row in the group of macroblock rows. After the previous input macroblock included in the block row is input to the macroblock pipeline, it is included in each macroblock row and is located to the left of the previous input macroblock, and the immediately preceding macroblock is adjacent to the upper right. A video encoding unit that inputs macroblocks that are not macroblocks into the macroblock pipeline;
A compressed syntax element storage unit in which the compressed syntax element is written in the variable length encoding stage;
The compressed syntax elements are read from the compressed syntax element storage unit in order from the leftmost macroblock to the rightmost macroblock for the uppermost macroblock row in the group of macroblock rows, The compression syntax elements in a group of macroblock rows from the top to the bottom macroblock row from the top to the bottom macroblock row in order from the leftmost macroblock to the rightmost macroblock for each macroblock row A syntax element restoring unit that reads the compressed syntax element from the storage unit and restores the syntax element from the read compressed syntax element;
A binarization unit that converts the restored syntax element into a binarized symbol;
An arithmetic encoding unit that arithmetically encodes the converted binary symbol .

Preferably , in the moving image encoding apparatus of the present invention, the moving image encoding process is performed according to the moving image encoding standard H.264. H.264,
The variable length encoding stage compresses the syntax element with CAVLC.

Preferably, the moving picture coding apparatus according to the present invention starts coding from the uppermost group of macroblock rows of a picture, and sequentially encodes the lowermost group of macroblock rows adjacent to each other.

Preferably, the moving image encoding device of the present invention includes at least two moving image encoding units.

The moving picture coding apparatus according to the present invention also includes a macroblock row in which macroblock rows, which are a set of macroblocks arranged in a row in the horizontal direction from the left end to the right end of a picture, are grouped into two adjacent rows in the vertical direction. A pair of macroblocks that are adjacent to each other in the macroblock row group and create a macroblock pair, and select frame coding and field coding for each macroblock pair. In the set of macroblock rows, encoding is started from the leftmost macroblock pair, and a moving picture encoding unit that sequentially encodes adjacent macroblock pairs on the right,
The video encoding process is divided into four or more processes, and four or more stages for performing the divided processes are included, and the syntax element compressed by reversibly compressing the syntax element is included in the stage. Including a macroblock pipeline that includes a variable length encoding stage to generate,
A set of two or more sets of macroblock rows that are consecutive in the vertical direction, and a set of macroblock rows that is smaller than the total number of sets of macroblock rows included in the entire picture is selected as a group;
Select whether to encode the macroblock pair belonging to the topmost macroblock row set in the group of macroblock rows by frame encoding or field encoding, and select frame encoding When the frame encoding is selected, the upper frame macroblock included in the macroblock pair is input to the macroblock pipeline. When field coding is selected, one macroblock included in the macroblock pair is input to the macroblock pipeline. Throw in,
Macroblocks adjacent to each set of macroblock rows in order from the second set of macroblock rows from the top to the bottom set of macroblock rows in the set of macroblock rows After a previous input macroblock included in a macroblock pair belonging to a set of rows is input to the macroblock pipeline, the macroblock pair belonging to the set of each macroblock row and to the left of the macroblock pair including the previous input macroblock When a macro block pair that is located and the macro block pair including the immediately preceding input macro block is not the upper right adjacent macro block pair is to be encoded by frame encoding or field encoding, and frame encoding is selected Indicates the upper frame macroblock included in the macroblock pair as the macroblock. Tsu put into click pipeline, when you select the field coding charged with macroblocks of one included in the macroblock pair in the macroblock pipeline,
After a macroblock included in a macroblock pair belonging to the lowest set of macroblock rows in the set of macroblock rows is input to the macroblock pipeline, the set of macroblocks The macroblock pipeline belongs to each macroblock row set in order from the topmost macroblock row set to the bottommost macroblock row set in the row set, and only one macroblock belongs to the macroblock pipeline. A video encoding unit that inputs the other macroblock included in the macroblock pair that has already been input to the macroblock pipeline and that has not yet been input to the macroblock pipeline;
A compressed syntax element storage unit in which the compressed syntax element is written in the variable length encoding stage;
The compressed syntax element from the compressed syntax element storage unit in order from the leftmost macroblock pair to the rightmost macroblock pair for the uppermost macroblock row set in the set of macroblock rows. Are sequentially read from the second set of macroblock rows to the lowest set of macroblock rows in the set of macroblock rows, and the leftmost for each set of macroblock rows. The compressed syntax element is read from the compressed syntax element storage unit in order from the macro block pair to the rightmost macro block pair, and the syntax element is restored from the compressed syntax element that has been read. A syntax element restoration unit to perform,
A binarization unit that converts the restored syntax element into a binarized symbol;
An arithmetic encoding unit that arithmetically encodes the converted binary symbol .

Preferably , in the moving image encoding apparatus of the present invention, the moving image encoding process is performed according to the moving image encoding standard H.264. H.264,
The variable length encoding stage compresses the syntax element with CAVLC.

Preferably, the moving picture coding apparatus according to the present invention starts coding from the set of macroblock rows at the top of a picture, and sequentially sets the set of macroblock rows adjacent below. Encode.

Preferably, the moving image encoding device of the present invention includes at least two moving image encoding units.

According to the present invention, arithmetic coding can be performed even if the arithmetic coding unit does not have a performance sufficient to process a binarized symbol generation amount that suddenly increases. In particular, if have sufficient storage capacity to the compression syntax element storage unit and the compressed syntax elements of I-pictures can be stored compressed syntax elements before and after the P-picture and B-picture, the arithmetic coding part performance can be arithmetic coding even below the suddenly can handle binary symbol generation amount for generating performance in I-picture.

Claims (10)

A first means for dividing a moving image encoding process into a plurality of stages and executing it in a pipeline;
A macro block row of m (m is an integer of 2 or more) that is vertically continuous is taken as a group, and a macro block belonging to the uppermost macro block row in the group of macro block rows is a lower left adjacent macro block. For a macroblock belonging to a macroblock row other than the topmost macroblock row in the group of macroblock rows, a macroblock whose upper right adjacent macroblock has already been selected is selected. A second means for selecting one macroblock per macroblock row by selecting one macroblock included in the group of macroblock rows;
For each macroblock selected by the second means, m from the macroblock belonging to the uppermost macroblock row to the macroblock belonging to the lowermost macroblock row in the group of macroblock rows. A third means for setting data necessary for encoding individual macroblocks in each macroblock cycle in the first means in order over the macroblock cycles;
Have
When the data required for encoding the macroblock belonging to the lowest macroblock row in the group of macroblock rows is set in the first means by the third means, the second means Means for newly selecting the right adjacent macroblock for each of the selected macroblocks. A moving picture encoding apparatus that combines two vertically adjacent macroblocks into a macroblock pair, and switches between field encoding and frame encoding for each macroblock pair,
A first means for dividing a moving image encoding process into a plurality of stages and executing it in a pipeline;
A group of 2n (n is an integer greater than or equal to 2) macroblock rows that are vertically continuous, and a macroblock pair that belongs to the two highest macroblock rows in the group of macroblock rows, For a macroblock pair belonging to a macroblock row other than the top two macroblock rows in the group of macroblock rows, and a macroblock pair for which the lower left adjacent macroblock pair has not yet been selected is selected, A second means for selecting one macroblock pair included in the batch of macroblock rows per two macroblock rows by selecting a macroblock pair whose upper right adjacent macroblock pair has already been selected;
When field coding is selected for the macroblock pair selected by the second means, a top field macroblock is formed by an odd line of the macroblock pair, and a bottom field macroblock is formed by an even line of the macroblock pair. On the other hand, when frame encoding is selected, the upper frame macroblock is composed of the upper half line of the macroblock pair, and the lower frame macroblock is composed of the lower half line of the macroblock pair. 3 means,
When field coding is selected for the macroblock pair selected by the second means, first, the macros belonging to the top two macroblock rows in the group of macroblock rows From the block pair to the macroblock pair belonging to the two lowest-most macroblock rows in the group of macroblock rows, the individual macroblocks in each macroblock cycle in order over n macroblock cycles Data necessary for encoding one of the macroblocks included in the pair is set in the first means, and the second two macroblocks located at the top of the group of macroblock rows are set for the second time. From the macroblock pairs belonging to the row, the most in the group of macroblock rows Other macroblocks that are not set first among the macroblocks included in each macroblock pair in order of n macroblock cycles in order up to macroblock pairs belonging to two macroblock rows located at The data required to encode one is set in the first means, while when frame encoding is selected, it is first located at the top of the group of macroblock rows From one macroblock pair belonging to two macroblock rows to one macroblock pair belonging to the two lowermost macroblock rows in the group of macroblock rows, one macroblock in order over n macroblock cycles Upper frame macroblocks included in individual macroblock pairs in each block cycle Is set in the first means, and the second time, from the macroblock pair belonging to the top two macroblock rows in the group of macroblock rows, The macroblock pairs belonging to the two lowest macroblock rows in the group of macroblock rows are included in each macroblock pair in every macroblock cycle in order over n macroblock cycles. A fourth means for setting data necessary for encoding the lower frame macroblock in the first means;
Have
Data necessary for encoding a macroblock included in a macroblock pair belonging to the lowest two macroblock rows in the group of macroblock rows by the fourth means for the second time Is set in the first means, the second means newly selects a right adjacent macroblock pair for each of the selected macroblock pairs. Device. 3. The moving image according to claim 1, wherein a macroblock cycle is divided into sub-block cycles in a part of stages in the first means, and processing for each sub-block is performed in the sub-block cycle. 4. Image encoding device. A parallel video encoding device including a video encoding device according to any one of claims 1 to 3 (a is an integer of 2 or more),
A picture is divided for each group of macroblock rows, and one moving picture coding device is assigned to each divided divided area, and a vertically divided divided area is a moving picture code. A parallel moving image encoding device encoded in parallel by an encoding device. Binarization means for converting syntax data into binarized symbols;
Binary symbol storage means for storing the binary symbol converted by the binarization means;
Context information restoring means for restoring context information corresponding to the binarized symbol read from the binarized symbol storage means;
Arithmetic coding means for arithmetically coding the binarized symbol read from the binary symbol storage means based on the context information restored by the context information restoration means;
5. The moving picture encoding apparatus according to claim 1, further comprising: Provisional arithmetic coding means for calculating a provisional generated code amount by provisional arithmetic coding of the binarized symbol converted by the binarization means;
6. The moving picture coding apparatus according to claim 5, further comprising: Syntax data compression means for reversibly compressing syntax data;
Compression syntax data storage means for storing syntax data compressed by the syntax data compression means;
Syntax data restoring means for reading the compressed syntax data from the compressed syntax data storage means and restoring the syntax data;
Binarization means for converting the syntax data restored by the syntax data restoration means into binarized symbols;
Arithmetic coding means for arithmetically coding the binarized symbols converted by the binarizing means;
5. The moving picture encoding apparatus according to claim 1, further comprising: Binarization means for converting syntax data into binarized symbols;
Provisional arithmetic coding means for provisionally arithmetically coding the binarized symbol converted by the binarization means to calculate a provisionally generated code amount;
CAVLC encoding means for CAVLC encoding syntax data;
CAVLC encoded bitstream storage means for storing the CAVLC encoded bitstream generated by the CAVLC encoding means;
CAVLC decoding means for reading a CAVLC encoded bitstream from the CAVLC encoded bitstream storage means and decoding syntax data;
Second binarization means for converting the syntax data decoded by the CAVLC decoding means into binarized symbols;
Arithmetic coding means for arithmetically coding the binarized symbols converted by the second binarizing means;
5. The moving picture encoding apparatus according to claim 1, further comprising: In a moving image encoding apparatus having an encoding unit that executes a moving image encoding process into a plurality of stages and executes in a pipeline manner,
A macro block row of m (m is an integer of 2 or more) that is vertically continuous is taken as a group, and a macro block belonging to the uppermost macro block row in the group of macro block rows is a lower left adjacent macro block. For a macroblock belonging to a macroblock row other than the topmost macroblock row in the group of macroblock rows, a macroblock whose upper right adjacent macroblock has already been selected is selected. A first procedure for selecting one macroblock per macroblock row by selecting, the macroblock included in the group of macroblock rows;
For each macroblock selected by the first procedure, m from the macroblock belonging to the uppermost macroblock row in the group of macroblock rows to the macroblock belonging to the lowermost macroblock row. A second procedure for setting data necessary for encoding individual macroblocks in one macroblock cycle in the encoding means in order over the macroblock cycles;
Have
When data necessary for encoding a macroblock belonging to the lowest macroblock row in the group of macroblock rows is set in the encoding means by the second procedure, the first means Returning to the procedure, a video encoding method characterized by newly selecting a right adjacent macroblock for each selected macroblock. It has an encoding means that divides moving image encoding processing into a plurality of stages and executes it in a pipeline manner, and combines two vertically adjacent macroblocks into a macroblock pair, and a field code for each macroblock pair. In a video encoding device that performs encoding by switching between encoding and frame encoding,
A group of 2n (n is an integer greater than or equal to 2) macroblock rows that are vertically continuous, and a macroblock pair that belongs to the two highest macroblock rows in the group of macroblock rows, For a macroblock pair belonging to a macroblock row other than the top two macroblock rows in the group of macroblock rows, and a macroblock pair for which the lower left adjacent macroblock pair has not yet been selected is selected, A first procedure of selecting one macroblock pair included in the batch of macroblock rows per two macroblock rows by selecting a macroblock pair whose upper right adjacent macroblock pair has already been selected;
When field coding is selected for the macroblock pair selected by the first procedure, a top field macroblock is formed by an odd line of the macroblock pair, and a bottom field macroblock is formed by an even line of the macroblock pair. On the other hand, when frame encoding is selected, the upper frame macroblock is composed of the upper half line of the macroblock pair, and the lower frame macroblock is composed of the lower half line of the macroblock pair. 2 steps,
When field coding is selected for the macroblock pair selected by the first procedure, from the macroblock pairs belonging to the two highest macroblock rows in the group of macroblock rows. , Up to the macroblock pair belonging to the two lowermost macroblock rows in the group of macroblock rows, in order of n macroblock cycles, included in each macroblock pair in every macroblock cycle The data necessary for encoding one of the macroblocks to be encoded is set in the encoding means, while when frame encoding is selected, it is positioned at the top of the group of macroblock rows. From the macroblock pairs belonging to two macroblock rows, the set of macros Up to the macro block pair belonging to the two lowest macro block rows in the block row, the upper frame macro block included in each macro block pair is sequentially added every n macro block cycles over n macro block cycles. A third procedure for setting the data necessary for encoding in the encoding means;
When field coding is selected for the macroblock pair selected by the first procedure, from the macroblock pairs belonging to the two highest macroblock rows in the group of macroblock rows. , Up to the macroblock pair belonging to the two lowermost macroblock rows in the group of macroblock rows, in order of n macroblock cycles, included in each macroblock pair in every macroblock cycle Data necessary for encoding the other one of the macroblocks not set by the third procedure is set in the encoding means. On the other hand, when frame encoding is selected, the one of the macroblocks is set. The macros belonging to the top two macroblock rows in a group of macroblock rows From macroblock pairs to macroblock pairs belonging to the two lowest-most macroblock rows in the group of macroblock rows, each macroblock cycle in turn takes n macroblock cycles. A fourth procedure for setting, in the encoding means, data necessary for encoding the lower frame macroblock included in the block pair;
Have
According to the fourth procedure, data necessary for encoding a macroblock included in a macroblock pair belonging to the two lowest macroblock rows in the group of macroblock rows is encoded. If set to the means, the method returns to the first procedure, and the right adjacent macroblock pair is newly selected for each selected macroblock pair.

Images