JP4359273B2 - Coding mode selection method - Google Patents

Description

  The present invention relates to a moving picture (video) coding technique, and more particularly to a technique for adaptively selecting a coding mode. For example, the present invention relates to an MPEG moving image encoding apparatus.

(1) Coding technique of moving picture data Prediction / interpolation coding, motion compensation, DCT (discrete cosine transform), quantization, variable length coding (VLC), etc. are used in moving picture coding.

  In predictive coding, the current macroblock is compared with the reference macroblock and the difference is output to the DCT. The current macroblock is a 16 × 16 pixel block extracted from the current frame. The reference macroblock is a 16 × 16 pixel block extracted from the reference frame. A reference frame is a frame that precedes or follows the current frame. If the reference frame precedes, it is called forward predictive coding. If the reference frame follows, it is called backward predictive coding. When the average of the reference frame preceding and following the current frame is adopted, this is called interpolation coding.

  The reference macroblock extracted from the reference frame is desired to be similar to the current macroblock. For this reason, for example, a macroblock with the smallest prediction error is extracted. The position of the macroblock in the reference frame is generally different from the position of the current macroblock in the current frame. This position difference is specified by a motion vector. The difference between corresponding pixels of the current macroblock and the reference macroblock, which is specified by the motion vector, is output to the DCT. This is called motion compensation.

  In DCT, an 8 × 8 pixel current block is converted into an 8 × 8 coefficient matrix Cij by the DCT technique and output to a quantizer. The current block is obtained by dividing the macro block of the difference as shown in FIG.

  As shown in FIG. 4, the coefficient matrix Cij is divided by a certain divisor Qij (quantization step width q × appropriate constant Kij for each coefficient matrix Cij), and the remainder is rounded. The quantized coefficient matrix C'ij is zigzag scanned and output to the variable length encoder. The constant Kij is given by a quantization table.

  As the constants Kij and / or q increase, the quantized coefficient data C'ij output from the quantizer includes more "0", and the compression rate increases. In adaptive quantization, the bit rate of the bit stream output from the video encoder is monitored, and the quantization step width is set so that the bit rate matches the target value. That is, when the bit rate is smaller than the target value, the quantization step width q is controlled to be small, and when the bit rate is larger than the target value, the quantization step width q is controlled to be large.

  In variable length coding, for example, Huffman coding, a code having a length corresponding to the appearance frequency is assigned to each quantized coefficient data C′ij output from the quantizer.

(2) MPEG
One type of video coding system has been proposed by the Expert Committee for Video Standardization (MPEG) under the International Organization for Standardization (ISO). The MPEG1 standard is provided by ISO / IEC11172, and the MPEG2 standard is provided by ISO / IEC13818.

  In an MPEG system, many well-known data compression techniques are integrated into a single system. These include motion compensated prediction / interpolation coding, DCT, adaptive quantization, and VLC.

  As shown in FIG. 2, I, P, and B pictures are used in the MPEG standard. An I picture is composed of intra macroblocks encoded only by DCT, quantization, and VLC. That is, prediction / interpolation coding with motion compensation is not used. The I picture is decoded without a motion vector.

  A P picture is composed of an intra macroblock and a forward macroblock. A P picture is decoded using a motion vector from a preceding I or P picture. A B picture is composed of an intra macroblock, a forward prediction macroblock, a backward prediction macroblock, and an interpolation macroblock. B pictures are decoded using motion vectors from the preceding and following I or P pictures.

(3) Coding mode Six types of motion compensation, ie, frame MC, field MC, dual prime MC in the frame structure, and field MC, 16 × 8 MC, dual prime MC in the field structure are MPEG standards. Is acceptable. Three types of prediction directions are allowed in the MPEG standard: forward, backward, and bidirectional (forward and backward). Therefore, in the MPEG standard, there are a plurality of motion compensation modes. The number of motion vectors depends on the motion compensation mode. A predictive coding mode without motion compensation and an intra coding mode are allowed by the MPEG standard.

  Therefore, the MPEG standard has a plurality of types of encoding modes. In encoding, an optimal encoding mode is selected for each macroblock from among the allowable encoding modes. For example, in inter-picture prediction encoding, an encoding mode with the smallest prediction error is selected. In addition, when the minimum prediction error exceeds a predetermined threshold, the intra coding mode is selected. Here, the prediction error is given by, for example, the average value of the square error of the difference between the current macroblock and the reference macroblock, or the average value of the absolute value.

(4) Conventional Technology As conventional technologies related to motion vector detection, there are publications such as Patent Literature 1, Patent Literature 2, Patent Literature 3, Patent Literature 4, Patent Literature 5, Patent Literature 6, and the like. As conventional techniques related to a selection circuit that selects intra coding and inter coding, there are publications such as Patent Document 7 and Patent Document 8. As prior arts related to the amount of code output from a motion prediction / interpolation encoder, DCT, adaptive quantizer, and encoder having a VLC, Patent Document 9, Patent Document 10, Patent Document 11, etc. There is.
JP-A-4-145777 JP-A-4-79484 Japanese Patent Laid-Open No. 3-40687 JP-A-4-207790 JP-A-4-234276 JP-A-4-40193 JP-A-6-133301 Japanese Patent Laid-Open No. 5-137129 JP-A-4-215384 JP-A-2-29180 JP-A-2-222389

  The first problem of the present invention will be described. Conventionally, among the various motion compensation modes, the motion compensation mode that minimizes the prediction error is selected. However, as shown in Patent Document 9 (H04N7 / 13), the code amount when an image is actually encoded is not proportional to the prediction error. Further, as shown in Patent Document 10 (H04N7 / 137), it is known to select a mode in accordance with a code amount at the time of encoding. Also, as shown in Patent Document 11 (H04N7 / 137), it is known that when motion compensated predictive coding is performed, not only the code amount of the image itself but also the code amount of the motion vector is considered. .

  However, there is no conventional method for determining a coding mode by actually calculating a coding amount including a motion vector. A first object of the present application is to provide a motion compensation mode selection method considering an actual code amount including a motion vector. A second object of the present invention is to propose such a moving image compression coding apparatus.

  The third problem of the present invention will be described. Conventionally, among the various motion compensation modes, the motion compensation mode that minimizes the prediction error is selected. However, when the quantization step width (q) in the subsequent quantization circuit (118) is large (when the compression ratio is large), there is a high possibility that the quality of the reproduced image is deteriorated.

  In such a case, even if predictive coding is performed in the motion compensation mode in which the prediction error is the lowest, even if predictive coding is performed in the motion compensation mode in which the prediction error is second lowest, there is not much difference. The possibility is considered high. In such a case, even if the prediction vector is the motion vector with the lowest prediction error and the prediction vector is encoded with the motion vector with the second lowest prediction error, the quality of the decoded image is not much. It is likely that there will be no significant difference.

  However, in the past, there was no idea of connecting the compression rate and the motion compensation operation. The third object of the present application relates to this new recognition. That is, if the compression rate is large, even if the motion vector detection accuracy or the motion compensation mode selection accuracy is high, there is little meaning in terms of the image quality of the decoded image. Further, when the compression rate is large, a situation in which a small amount of code in actual encoding is desired.

  Therefore, it is only necessary that the motion compensation operation can be suitably changed according to the compression rate or the like. The third problem of the present application is to notify the motion compensation related circuit of the compression rate and the like. Also, the code amount of the entire bitstream is not necessarily determined only by the prediction error between images. That is, what is encoded is not only the pixel values of the difference block or processing block, but also motion compensation mode information, motion vectors, other parameters, etc., and the number of these varies depending on the motion compensation mode. For example, depending on the value of the motion vector, the code amount may be increased instead. Since the code amount control is indispensable in order to decode moving image compressed data in real time, if the code amount of one part of an image increases, the code amount of the other part must be reduced, and the overall image quality is reduced. May deteriorate. Therefore, even if the prediction error between some images is minimized, if the code amount increases, the overall image quality may be deteriorated. In particular, in encoding at a low bit rate, it is important to reduce the amount of code rather than image quality.

The invention of claim 1, at the coding mode of the current video data of the target to be encoded into corresponding current video coding in a method of selecting adaptively refers to the reference image data, said the current image data A prediction error is obtained for each coding mode, and the prediction error of each coding mode is compared with a threshold value that is changed according to the compression rate of the coding. Each of the encoding modes is a selection method in which a value related to the total code amount including the current video code and its encoding parameters is obtained, and the encoding mode with the smallest total code amount is selected.

  In the present invention, the code amount of the entire bit stream can be set to an optimum value in the compression, recording and transmission of moving images.

  In the following, described in MPEG video encoder terminology, it is understood that other types of video encoders may be used in which image frames are partially encoded based on motion compensated prediction or adaptive quantization. Intended.

(1) Typical MPEG encoder (FIG. 1).
FIG. 1 shows a typical MPEG encoder. In this system, a video signal describing an image is supplied to the screen rearrangement unit 111. The screen rearrangement device 111 rearranges the screen order. When the current frame is referred to by a temporally preceding frame, the current frame needs to be decoded and stored in the image memory 142 in advance. For this reason, the current frame and its temporally preceding frame are rearranged so that the current frame is processed first. For example, the current I or P picture referenced by the temporally preceding B picture is output to the macroblock converter 112 first.

  The video data rearranged by the screen rearranger 111 is input to the macroblock converter 112. For example, the macroblock converter 112 including a normal 2-port memory converts the signal from the raster scan format into a 16 × 16 pixel macroblock format and outputs the signal to the subtractor 114 and the motion detector 146. In the macroblock format, each frame of the image is represented as a collection of macroblocks having 256 pixels arranged in a 16 × 16 pixel matrix. The macroblock converter 112 supplies these pixel values to the subtracter 114 for each macroblock.

  In the non-intra coding mode, the subtractor 114 subtracts the reference macroblock supplied from the image memory 142 from the corresponding current macroblock supplied from the macroblock converter 112 to obtain a motion compensated difference macroblock. Is output to the block converter 116. In the intra coding mode, the subtractor 114 outputs the current macrobook supplied from the macroblock converter 112 to the block converter 116.

  As shown in FIG. 3, the block converter 116 converts the signal from a macro block format of 16 × 16 pixels into four block formats of 8 × 8 pixels and outputs them to the DCT 118. The block converter 116 supplies these pixel values to the DCT 118 for each block.

  The DCT processor 118 applies a DCT operation to the pixel values of each block to convert them into blocks of the DCT coefficient matrix Cij. Each block is arranged into a linear stream of 64 coefficients Cij using a zigzag scan as shown in FIG. In any block, the head of these coefficients Cij represents the direct current (DC) spatial frequency component of the pixel block. The remaining coefficient Cij is the next highest spatial frequency component.

  The coefficient values supplied by the DCT processor 118 are applied to the quantizer 120. The quantizer 120 converts each coefficient value Cij into a binary value having the allocated number of bits. In general, a higher number of bits is used for lower order coefficients than for higher order coefficients. This is because the human eye is less sensitive to high spatial frequency image components than to low spatial frequency image components. This operation can be performed, for example, by dividing each coefficient by a different value proportional to the spatial frequency.

  Further, the number of bits assigned to each coefficient value can be changed according to the quantization step width q supplied from the quantization controller 132. The quantization step width q is applied to divide each coefficient before or after being divided by the quantization matrix Kij. The quantizer 120 produces a stream of digital values that are input to the VLC 124 and the inverse quantizer 136. The quantization step width q for controlling the compression rate is variable.

  The VLC 124 encodes the data from the quantizer 120 using, for example, a run-length Huffman type code. Using the Huffman type code, the VLC 124 allocates a smaller number of bits for the combination of frequently occurring data values and for a sequence of zeros.

  There is a second VLC 134. In this case, the macroblock type data MBT and the motion vector data, both of which are data from the mode determiner 150, are variable-length encoded.

  The code generated by the VLC 124 and the code generated by the VLC 134 are input to an inserter (FIFO memory) 126. The inserter (FIFO memory) 126 combines them and outputs the bit stream to the buffer memory (FIFO memory) 128. This bit stream is stored in a buffer memory (FIFO memory) 128 and recorded on a recording medium 130 such as an optical disk.

  In the MPEG encoder, the amount of data in the buffer memory 128 is monitored, and the amount of data stored in the buffer memory of the MPEG decoder is simulated. As a result, the quantization step width q is controlled so that the buffer memory of the MPEG decoder does not overflow. That is, the quantization step width q is determined with reference to the buffer memory 128 and the capacity change of the buffer memory 128. As the quantization step width q, values 1 to 31 are usually adopted.

  In the B picture and the P picture, the difference value is DCTed and output, so that the data amount is smaller than that of the I picture. For this reason, in MPEG, the target data amount is assigned according to the picture type. Further, the generated data amount is monitored for each slice or macroblock. The amount of data is compared with the target value and evaluated by the quantization controller 132. For example, when the generated code amount is larger than the target value, the quantization step width q is increased and the quantization becomes coarse. This control is performed by the quantization controller 132. On the other hand, when the generated code amount is smaller than the target value, the quantization step width q is reduced and the quantization becomes finer. The buffer memory 128 mitigates fluctuations in the amount of generated code caused by the frame type, frame characteristics, and quantization step width.

  Note that the MPEG standard allows a variable bit rate in addition to a fixed bit rate, although it is not common. When the bit rate varies, the quantization step width q naturally varies.

  A local decoder composed of an inverse quantizer 136 and an inverse DCT 138 reproduces reference image data for a preceding or subsequent frame. The reproduced frame is stored in the image memory 142. Thereafter, it is output to the subtractor 114 as described above. The adder 140 adds the motion-compensated reference macroblock in the reference frame to the reproduction data when the reproduction data is difference data.

  The image memory 142 stores at least two image data. One of an I picture and an I picture, an I picture and a P picture, and a pair of a P picture and a P picture. The image memory 142 outputs each macro block for reference. Also output to the motion vector detector 146 for motion vector detection. Further, it is output to the mode determiner 150 for mode determination. In the motion vector detector 146, the region most similar to the current macroblock is found from the reference frame using, for example, a prediction error. The motion vector detector 146 includes a front detector 146F, a both-side detector 146M, and a rear detector 146B. The forward detector 146F detects the forward motion vector and outputs it to the motion compensation mode selector 148 together with the prediction error. The two-way detector 146M detects a motion vector in both directions and outputs it to the motion compensation mode selector 148 together with the prediction error. The backward detector 146B detects the backward motion vector and outputs it to the motion compensation mode selector 148 together with the prediction error.

  The motion compensation mode selection circuit 148 selects the one with the least prediction error. That is, if the prediction error from the backward motion vector detection circuit 146B is the smallest among the prediction error outputs from the three motion vector detection circuits 146F, 146M, and 146B, the motion compensation mode selection circuit 148 performs the backward motion compensation. In addition to outputting the designated macroblock type information, the motion vector from the backward motion vector detection circuit 146B is output.

  Similarly, if the prediction error of the bidirectional motion vector detection circuit 146M is the smallest among the prediction error outputs from the three motion vector detection circuits 146F, 146M, and 146B, the motion compensation mode selection circuit 148 In addition to outputting macroblock type information instructing compensation, a motion vector from the bidirectional motion vector detection circuit 146M is output.

  When the macroblock is subjected to motion compensation predictive coding (inter coding) by the motion compensation mode selection circuit 148 in the previous stage, the most appropriate motion compensation direction has been selected. However, depending on the image pattern, intra-frame coding (intra coding) may be more efficient when coding a macroblock. The mode determination circuit 150 performs the intra / inter determination. The mode determination circuit 150 is well known as shown in Patent Document 7 and Patent Document 8, for example.

  The mode determination circuit 150 obtains the variance value of the macroblock image from the macroblock converter 112. Also, the variance value of the difference screen when motion prediction encoding is performed based on the macroblock type information and the motion vector from the motion compensation mode selection circuit 148 is obtained. For this purpose, based on the macroblock type information and the motion vector from the motion compensation mode selection circuit 148, a prediction macroblock is read from the image memory 142 via the motion compensation read control circuit 144, and is sent to the mode determination circuit 150. input.

  The mode determination circuit 150 obtains a macro block of a difference screen between the predicted macro block and the macro block from the macro block converter 112. Then, the variance value of this macro block is obtained. The type of macroblock is determined by comparing the two variance values thus obtained. Based on this determination, macro block type information is output. Also, when inter coding is selected as the macroblock type information, motion vector information is also output.

  The operation of the MPEG encoder will be briefly described with reference to FIGS. First, the screen rearrangement circuit 111 sets an I picture in which one screen is compressed within a screen unit from several tens of screens, and other screens are set between screen units using motion compensation. Are compressed into B picture and P picture. The screen rearrangement circuit 111 rearranges the screen according to this setting.

  In an intra macroblock that performs compression within a screen unit, the block circuit 112 divides the screen into a plurality of regions, passes through the subtractor circuit 114, and the DCT circuit 118 performs two-dimensional discrete cosine transform (DCT) on each. Go to find the frequency component. The unit of this DCT processing is a block of 8 × 8 pixel units.

  The screen (FIG. 4 (a)) is subjected to DCT processing and converted into frequency components as shown in FIG. 4 (b). As a result, the upper left is a low frequency region and the lower right is a high frequency region. The obtained frequency component is divided by the value shown in FIG. This division is quantization. At the time of quantization, using the fact that human visual characteristics are insensitive to high frequencies, quantization is performed so that many codes are assigned to the low frequency side and the code amount on the high frequency side is reduced. That is, the above-described value Qij is a multiplication result of the quantization matrix Kij in which the value of the region corresponding to the high frequency is set large and the quantization step width q from the quantization circuit 120. The data thus obtained is zigzag from the low frequency side to the high frequency side as shown in FIG. 4C, and the result is variable length encoded.

  Further, in an inter macroblock that performs compression between screen units, the output (current screen) of the blocking circuit 112 and the prediction screen by motion compensation are input to the subtraction circuit 114 to obtain a difference. Encoding is equivalent to Thus, since the difference is transmitted between the B picture and the P picture, the data amount is small. Therefore, the data amount (bit amount) at the time of encoding differs depending on the screen as it is. However, in MPEG, the transfer bit rate is almost constant.

  For this reason, if a screen with a large amount of data continues, the buffer memory 128 may overflow. Therefore, by changing the quantization matrix (by controlling the compression ratio) by changing the value of the quantization matrix (quantization step width q, which is a multiplier of FIG. 4D), the amount of generated data is fed back. By controlling, overflow is prevented.

  The motion vector detection circuit 146 detects a motion vector. The motion compensation mode selection circuit 148 detects an appropriate encoding mode of this macroblock. The motion vector detection circuit 146 and the motion compensation mode selection circuit 148 select a motion compensation mode that minimizes the prediction error among all of several motion compensation prediction modes. The mode determination circuit 150 finally determines the macroblock type, and outputs this macroblock type information and a motion vector.

(2) First embodiment (FIGS. 5 to 8).
In FIG. 5, the same parts as those of FIG. In FIG. 5, 50 is a buffer for intra coding. 52, 54 and 56 are buffers for inter coding. Each buffer stores encoded data of an image including attached data such as a motion vector in each motion compensation mode.

  A buffer 52 temporarily stores codes generated when forward motion compensation prediction encoding is performed. A buffer 54 temporarily stores a code generated when bidirectional motion compensation predictive coding is performed. A buffer 56 temporarily stores codes generated when backward motion compensation prediction encoding is performed.

  58 is a mode selection circuit. The mode selection circuit 58 detects the code amount of the buffers 50, 52, 54, and 56, selects the mode with the smallest code amount, and outputs it to the mode determination circuit 60. This mode selection circuit 58 forms generated code amount detection means 58 for each mode for detecting a code amount including a motion vector that actually occurs when a moving image signal is encoded from a plurality of different motion compensation prediction encoding modes. is doing.

  The mode determination circuit 60 determines a mode for encoding. The mode determination circuit 60 forms at least adaptive motion compensation mode selection means 60 that selects a motion compensation mode to be encoded by the output of the mode selection circuit 58.

  The operation of the first embodiment will be described. When encoding a B picture, the mode must be determined for each macroblock. The motion detection circuit 146 obtains a motion vector of each mode for the macroblock to be encoded.

  The characteristics of the present application will now be described. As shown in S1 of FIG. 6, the mode determination circuit 60 first controls the encoder so that intra coding is performed. The code generated at this time is stored in the buffer 50.

  Next, as shown in S2 of FIG. 6, the mode determination circuit 60 first controls the encoder so that the forward prediction encoding of the inter encoding is performed, and the corresponding forward motion vector. Is output. The code generated at this time is stored in the buffer 52.

  As shown in S3 of FIG. 6, the mode determination circuit 60 controls the encoder so that bidirectional prediction encoding among inter encodings is performed, and outputs a corresponding bidirectional motion vector. The code generated at this time is stored in the buffer 54.

  As shown in S4 of FIG. 6, the mode determination circuit 60 controls the encoder so that backward prediction coding among inter coding is performed, and outputs a corresponding backward motion vector. The code generated at this time is stored in the buffer 56.

  The mode selection circuit 58 detects the code amount of the buffers 50, 52, 54, and 56, detects the buffer with the smallest code amount, and informs the mode decision circuit 60 as shown in S5 of FIG. That is, the mode selection circuit 58 detects the code amounts of the buffers 50, 52, 54, and 56, and the mode determination circuit 60 performs encoding in the mode corresponding to this smallest code amount as shown in S6 of FIG. To decide. From here, the normal encoding process is performed as in the conventional case.

  The determination of this mode will be described. For example, if the code amount of the buffer 50 is the smallest, the coding suitable for the macroblock at this time is intra coding. Therefore, when the mode selection circuit 58 detects this and communicates this to the mode determination circuit 60, the mode determination circuit 60 uses the macro indicating intra coding to control the encoder to perform intra coding. Output block type information.

  If the code amount of the buffer 56 is the smallest, the coding suitable for the macroblock at this time is backward motion compensation prediction coding. Therefore, when the mode selection circuit 58 detects this and communicates this to the mode decision circuit 60, the mode decision circuit 60 uses the latter to control the encoder to perform backward motion compensated predictive coding. The macro block type information indicating the direction prediction coding is output and the backward motion vector is output.

  Thus, according to the first embodiment, since the mode is selected based on the code amount including the actual motion vector, it is possible to select a motion compensation mode with a small generated code amount. Although the first embodiment has been described with reference to a hardware schematic circuit block diagram, the present application may naturally be adopted when MPEG encoding is performed by software. In the first embodiment, intra coding is included as a macroblock coding mode. However, the present application is not limited to this, and only a plurality of types of motion compensation modes may be used. In the first embodiment, three motion compensation modes have been described. However, the present application is naturally not limited to this. For example, the mode selection for frame prediction and field prediction in the MPEG2 frame structure is also possible. Available. It can also be used for 16 × 16 unit prediction and 16 × 8 unit prediction mode selection in the MPEG2 field structure. Moreover, you may use for a P picture.

  In the first embodiment, the mode is always selected from the actual code amount. However, the present application is not limited to this. For example, when the buffer 128 has a sufficient margin and the quantization step width q is minimum (when the compression rate is small), the mode may be determined in the same manner as in the conventional case. Then, for example, when the possibility that the buffer 128 will run out increases or the quantization step width q increases (when the compression ratio is large), the above-described processing is performed in order to reduce the generated code amount. It may be configured.

  In the first embodiment, all the three motion compensation modes are actually encoded. However, the present application is naturally not limited to this. For example, in the motion compensation mode, all of the top two motion compensation modes with small prediction errors may be actually encoded. In this way, the calculation amount in the program can be reduced, and the processing speed can be increased. As described above, when the prediction error is small, the code amount is not always small, but the possibility is high.

  Further, as described above, the upper two motion compensation modes with few prediction errors are not actually encoded, but the motion compensation modes with less prediction errors than a predetermined threshold are actually encoded as shown in FIG. It may be. In this way, the calculation amount in the program can be reduced, and the processing speed can be increased.

  Furthermore, the value of the predetermined threshold in FIG. 7 may be changed according to the quantization step width q. That is, as shown in FIG. 8, when the quantization step width q is large, a mode in which the actual code amount is detected as much as possible is selected. That is, when there is no room in the buffer 128, it is desired to reduce the amount of generated code. In such a case, this is performed to increase the possibility of reducing the amount of generated code as much as possible.

(3) Second embodiment (FIG. 9).
9, parts that are the same as those in FIGS. 1 and 5 are given the same reference numerals, and descriptions thereof are omitted. In this embodiment, the mode selection operation at the time of encoding is changed according to the value related to the compression rate or generated code amount in this encoder. Note that values related to the compression rate or generated code amount in this encoder include the generated code amount for each macroblock from the inserter 126, the remaining capacity of the buffer 128, and the quantization step width q.

  In this embodiment, the quantization step width q is used. Further, in this embodiment, when the mode selection operation is performed according to the generated code amount, the actual generated code amount is not detected, but the generated code amount is predicted from the variance value. In FIG. 9, reference numeral 62 denotes a motion compensation mode selection circuit. L is a signal line as a notification means for transmitting the quantization step width q to the motion compensation mode selection circuit 62.

  The motion compensation mode selection circuit 62 first detects a mode in which the prediction error is smaller than a predetermined threshold. The predetermined threshold is changed according to the quantization step width q. The quantization step width q is transmitted to the motion compensation mode selection circuit 62 through the signal line L. When the quantization step width q is large, the above-described threshold value is also greatly changed.

  Then, the prediction error is compared with the threshold value, and a prediction error mode smaller than the threshold value is detected. If there is no mode corresponding to this, the motion compensation mode selection circuit 62 selects the one with the smallest prediction error. That is, the macro block type information indicating this mode is output, and the motion vector of this mode is output. If there is only one mode corresponding to this, the motion compensation mode selection circuit 62 selects this mode. That is, the macro block type information indicating this mode is output, and the motion vector of this mode is output.

  If there are two or more modes corresponding to this, the motion compensation mode selection circuit 62 further performs a selection process. For this selection process, a process using a variance value is performed. The motion compensation mode selection circuit 62 obtains a variance value of an error screen when one of a plurality of modes is subjected to motion prediction encoding based on the macroblock type information and the motion vector.

  Therefore, the macro block type information and the motion vector corresponding to this mode are output to the motion compensation readout control circuit 144. As a result, the macro block of the corresponding prediction screen is output from the image memory 142 to the motion compensation mode selection circuit 62.

  The motion compensation mode selection circuit 62 obtains the difference between the current macroblock image from the macroblock converter 112 and the macroblock of the prediction screen from the image memory 142, and further obtains the variance value of this difference screen macroblock. The motion compensation mode selection circuit 62 performs the same processing for the remaining modes, and obtains a dispersion value in each mode.

  Then, the motion compensation mode selection circuit 62 compares the dispersion values and selects the motion compensation mode. Thus, the motion compensation mode selection circuit 62 selects the motion compensation mode with reference to the output of the motion vector detection circuit (prediction error detection means for each mode: 146) and the value related to the compression ratio. The motion compensation mode selection means 62 is formed. In addition, the motion compensation mode selection circuit 62 includes motion compensation mode selection means 62 that changes the process for selecting the motion compensation mode in accordance with at least the value related to the compression ratio. In this embodiment, the variance value of the difference screen in the motion compensation mode is obtained once again by the mode determination circuit 150. However, this may naturally be combined.

(4) Predictive coding with motion compensation (FIGS. 17 and 18).
An outline of a compression method using predictive coding with motion compensation, which is generally performed conventionally, will be described below. FIG. 17 is a block diagram of a compression operation based on the MPEG standard, and FIG. 18 is a block diagram of an operation for selecting a motion compensation mode. In FIG. 17, the intra image is first subjected to DCT and quantization in the DCT / quantization unit 101 for each block.

  At this time, the quantization step width and the like are determined according to the target code amount given from the code amount control unit 108. The generated data is sent to the VLC unit 102 where variable length coding (VLC) is performed. The encoded data is integrated with the quantum step width value to form one bit stream. On the other hand, the encoded data is also sent to the inverse quantization / inverse DCT (IDCT) unit 103 for decoding, and the decoded data (hereinafter, decoded image) is stored in the image memory 104.

  Next, in the inter image, first, the motion detection unit 105 detects a motion vector for the reference picture for each macroblock. Here, the reference picture is a decoded image held in the image memory 104. When a plurality of motion compensation modes are allowed, motion vector detection is performed for each motion compensation mode. Thereafter, the motion compensation unit 106 selects a motion compensation mode that minimizes the inter-picture prediction error or a mode that does not perform motion compensation.

  The difference block or processing block corresponding to the selected mode is sent to the DCT / quantization unit 101 and then to the VLC unit 102, where compression processing similar to that for the intra image is performed. On the other hand, the mode information selected by the motion compensation unit 106 and the corresponding motion vector information when motion compensation is performed are sent to the VLC unit 107 and encoded. Finally, in the data integration unit 110, each piece of encoded information is integrated into one bit stream and output. The generated bit stream amount is sent to the code amount control unit 108 and becomes a reference of the target code amount determined when the remaining image is encoded.

  FIG. 18 shows the details inside the motion compensation unit 106 of this configuration. Here, first, the block position designation unit 11 designates the reference block position in the picture using the detected motion vector data, and cuts out the reference block from the reference picture. Next, the difference block generation unit 12 calculates a difference between corresponding pixel values between the reference block and the processing block, and generates a difference block. The operations of 11 to 13 are performed for each of the plurality of motion compensation modes and for each of the inter-picture predictive coding without motion compensation. However, when motion compensation is not performed, the block position designating unit 11 designates the same coordinates as the processing macroblock without using the motion vector data. The minimum prediction error selection unit 14 selects a motion compensation mode that minimizes the prediction error between images based on a plurality of difference blocks corresponding to each mode.

  Finally, the intra / non-intra determination is performed by selecting either the inter-picture predictive coding with motion compensation (non-intra) with the selected motion compensation mode or the mode (intra) without inter-picture predictive coding. This is performed by the non-intra determination unit 15. In general, if the prediction error of inter-picture predictive coding exceeds a certain threshold, inter-picture predictive coding is not performed (intra determination).

  The selected motion compensation mode information and motion vector are sent to the VLC unit 107 of FIG. On the other hand, when a corresponding difference block or a mode in which motion compensation is not performed is selected, the processing block is sent to the DCT / quantization unit 101 in FIG.

(5) Third embodiment (FIGS. 10 and 11).
A third embodiment of the present invention will be described. FIG. 10 shows an example of the configuration of a moving image compression method according to the present invention, and FIG. 11 shows details of motion compensation mode selection according to the present invention.

  As shown in FIG. 10, the operation configuration of this embodiment includes a DCT / quantization unit 101 that performs DCT and quantization, a VLC 102 and VLC 107 that perform variable length coding, and an inverse quantization / IDCT that performs inverse quantization and IDCT. 103, image memory 104 for storing at least one picture data, motion detection unit 105 for motion detection, motion compensation unit 106 for motion compensation, code amount control unit 108 for code amount control, various compressed data Is composed of a data integration unit 110 that integrates a single bitstream.

  In the present invention, the difference from the prior art is the selection of the motion compensation mode in the inter image, so the compression operation of the intra image will not be described. In the case of an inter image, first, motion detection is performed on the reference image stored in the image memory by the motion detection unit 105, and motion compensation is performed by the motion compensation unit 106 using the motion vector obtained by the motion detection unit. Do.

  The motion compensation unit selects an optimal motion compensation mode from among a plurality of motion compensation modes using a target code amount obtained from the amount of a bit stream that has been encoded in the past. The DCT / quantizer 101 compresses the differential block data generated by motion compensation based on the selected motion compensation mode by DCT and quantization.

  Further, the VLC unit 102 performs variable length coding.

  On the other hand, the selected motion compensation mode information and the motion vector corresponding to the motion compensation mode are variable length encoded by the VLC unit 107. The data integration unit 110 integrates the compressed data into one bit stream and outputs it.

  FIG. 11 shows details of the motion compensation mode selection operation in this embodiment. First, the block position specifying unit 11 specifies the position of the reference block in the reference picture using the detected motion vector data. Next, the difference block generation unit 12 calculates a difference between corresponding pixel values between the processing block in the processing picture and the reference block in the reference picture based on the reference block position, and generates a difference block.

  The code amount calculation unit 23 obtains a code amount generated when the difference block is encoded. Here, in calculating the code amount, a table of code amounts corresponding to each element, pattern, and motion vector value of the difference block is held in advance, and the corresponding value is obtained by comparison. Alternatively, a code amount generated by actually encoding the difference block and other parameters may be obtained. In that case, the same operation as that of the DCT / quantization unit 101, the VLC unit 102, and the VLC unit 107 in FIG. 10 is performed for all modes to obtain the code amount. The recent code amount selection unit 34 selects a motion compensation mode that has a value closest to the target code amount from the generated code amount obtained for each motion compensation mode.

  When the selected motion compensation data and the corresponding difference block data or a mode in which motion compensation is not performed is selected, processing block data is output. Here, when encoding is performed at the time of code amount calculation, encoded data may be output instead of block data. In that case, the operations of DCT / quantization section 101, VLC section 102, and VLC section 107 in FIG. 10 are not performed.

(6) Fourth embodiment (FIG. 12).
Next, a fourth embodiment of the present invention will be described. The present embodiment is characterized in that a quantization step width is used for estimating a target code amount, and a motion compensation mode that becomes a recent code amount is selected as the target code amount. FIG. 12 shows details of the motion compensation mode selection operation in this embodiment.

  First, the block position specifying unit 11 specifies the position of the reference block in the reference picture using the detected motion vector data. Next, the difference block generation unit 12 calculates a difference between corresponding pixel values between the processing block in the processing picture and the reference block in the reference picture based on the reference block position, and generates a difference block.

  The code amount calculation unit 23 obtains a code amount generated when the difference block is encoded. Here, in calculating the code amount, a table of code amounts corresponding to each element, pattern, and motion vector value of the difference block is held in advance, and the corresponding value is obtained by comparison. Alternatively, a code amount generated by actually encoding the difference block and other parameters may be obtained. In that case, the same operation as that of the DCT / quantization unit 101, the VLC unit 102, and the VLC unit 107 in FIG. 10 is performed for all modes to obtain a code amount.

  When the quantization step width is determined according to the buffer remaining amount of the encoded bit stream, the quantization step width may be used instead of the target code amount. In this case, the target code amount estimation unit 45 in the figure holds a table of remaining buffer amounts corresponding to the quantization step width value in advance, and obtains an approximate target code amount by comparing with the table.

  The recent code amount selection unit 34 selects a motion compensation mode that has a value closest to the estimated target code amount from the generated code amount obtained for each motion compensation mode. When the selected motion compensation data and the corresponding difference block data or a mode in which motion compensation is not performed is selected, processing block data is output. Here, when encoding is performed at the time of code amount calculation, encoded data may be output instead of block data. In that case, the operations of DCT / quantization section 101, VLC section 102, and VLC section 107 in FIG. 10 are not performed.

(7) Fifth embodiment (FIG. 13).
Next, a fifth embodiment of the present invention will be described. The present embodiment is characterized in that a motion compensation mode that minimizes the amount of code after encoding is selected. FIG. 13 shows details of the motion compensation mode selection operation in this embodiment.

  First, the block position specifying unit 11 specifies the position of the reference block in the reference picture using the detected motion vector data. Next, the difference block generation unit 12 calculates a difference between corresponding pixel values between the processing block in the processing picture and the reference block in the reference picture based on the reference block position, and generates a difference block.

  The code amount calculation unit 23 obtains a code amount generated when the difference block is encoded. Here, in calculating the code amount, a table of code amounts corresponding to each element, pattern, and motion vector value of the difference block is held in advance, and the corresponding value is obtained by comparison. Alternatively, a code amount generated by actually encoding the difference block and other parameters may be obtained. In that case, the same operation as that of the DCT / quantization unit 101, the VLC unit 102, and the VLC unit 107 in FIG. 10 is performed for all modes to obtain a code amount.

  The minimum code amount selection unit 54 selects a mode having the minimum code amount from the generated code amount obtained for each mode, and the selected motion compensation data and corresponding difference block data, or a mode in which motion compensation is not performed. If selected, processing block data is output. When the selected motion compensation data and the corresponding difference block data or a mode in which motion compensation is not performed is selected, processing block data is output. Here, when encoding is performed at the time of code amount calculation, encoded data may be output instead of block data. In that case, the operations of DCT / quantization section 101, VLC section 102, and VLC section 107 in FIG. 10 are not performed.

(8) Sixth embodiment (FIG. 14).
Next, a sixth embodiment of the present invention will be described. The present embodiment is characterized in that a mode in which the encoded code amount is closest to the target code amount is selected from among the motion compensation mode in which the prediction error is minimized and the mode in which motion compensation is not performed.

  FIG. 14 shows details of the mode selection operation in this embodiment. First, the block position specifying unit 11 specifies the position of the reference block in the reference picture using the detected motion vector data. Next, the difference block generation unit 12 calculates a difference between corresponding pixel values between the processing block in the processing picture and the reference block in the reference picture based on the reference block position, and generates a difference block.

  The prediction error calculation unit 13 calculates a prediction error. The code amount calculation unit 23 obtains a code amount generated when the difference block is encoded. The minimum prediction error selection unit 64 selects the one having the smallest prediction error value obtained for each mode in which inter-picture prediction encoding is performed. The code amount calculation unit 23 calculates the code amount after compression for both the mode having the minimum prediction error and the mode for performing intra coding.

  The recent code amount selection unit 34 selects a mode in which the calculated code amount is closest to the target code amount. In this embodiment, means for selecting the minimum code amount may be used in place of the recent code amount selection unit 34. Further, instead of the target code amount, a means for estimating the target code amount using the quantization step width may be added.

(9) Seventh embodiment (FIG. 15).
Next, a seventh embodiment of the present invention will be described. The present embodiment is a compressed data recording method in which encoding is performed using a mode for obtaining an optimal code amount, and a generated bit stream is recorded. FIG. 15 shows an example of a compressed moving image recording method in this embodiment.

  As shown in FIG. 15, the operation configuration of this embodiment includes a DCT / quantization unit 101 that performs DCT and quantization, a VLC 102 and VLC 107 that perform variable-length coding, and an inverse quantization / IDCT unit that performs inverse quantization and IDCT. 103, an image memory 104 for storing at least one picture data, a motion detection unit 105 for motion detection, a motion compensation unit 106 for motion compensation, a code amount control unit 108 for code amount control, and various compressed data It comprises a data integration unit 110 that integrates into a single bitstream and a recording medium 611 that records compressed data.

  In the present invention, the difference from the prior art is the selection of the motion compensation mode in the inter image, so the compression operation of the intra image will not be described. In the case of an inter image, first, motion detection is performed on the reference image stored in the image memory by the motion detection unit 105, and motion compensation is performed by the motion compensation unit 106 using the motion vector obtained by the motion detection unit. Do.

  The motion compensation unit selects an optimal motion compensation mode from among a plurality of motion compensation modes using a target code amount obtained from the amount of a bit stream that has been encoded in the past. The DCT / quantizer 101 compresses the differential block data generated by motion compensation based on the selected motion compensation mode by DCT and quantization.

  Further, the VLC unit 102 performs variable length coding. On the other hand, the selected motion compensation mode information and the motion vector corresponding to the motion compensation mode are variable length encoded by the VLC unit 107. The data integration unit 110 integrates the compressed data into one bit stream and outputs it.

  The output bit stream is recorded on a recording medium 611 stored in the recording device. The internal configuration of the motion compensation unit 106 is the same as that of any one of the third to sixth embodiments.

(10) Eighth embodiment (FIG. 16).
Next, an eighth embodiment of the present invention will be described. The present embodiment is compressed data transmission means for performing encoding using a mode for obtaining an optimal code amount and transmitting a generated bit stream.

  FIG. 16 shows an example of the compressed moving image recording method in this embodiment. As shown in FIG. 16, the operation configuration of this embodiment includes a DCT / quantization unit 101 that performs DCT and quantization, a VLC 102 and VLC 107 that perform variable length coding, and an inverse quantization / IDCT that performs inverse quantization and IDCT. 103, image memory 104 for storing at least one picture data, motion detection unit 105 for motion detection, motion compensation unit 106 for motion compensation, code amount control unit 108 for code amount control, various compressed data Are integrated into a single bit stream, and a data integration unit 110 and transmission means 711 for transferring compressed data.

  In the present invention, the difference from the prior art is the selection of the motion compensation mode in the inter image, so the compression operation of the intra image will not be described. In the case of an inter image, first, motion detection is performed on the reference image stored in the image memory by motion detection · BR> · 05, and the motion compensation unit 106 uses the motion vector obtained by the motion detection unit. Perform motion compensation.

  The motion compensation unit selects an optimal motion compensation mode from among a plurality of motion compensation modes using a target code amount obtained from the amount of a bit stream that has been encoded in the past. The DCT / quantizer 101 compresses the differential block data generated by motion compensation based on the selected motion compensation mode by DCT and quantization. Further, the VLC unit 102 performs variable length coding. On the other hand, the selected motion compensation mode information and the motion vector corresponding to the motion compensation mode are variable length encoded by the VLC unit 107. The data integration unit 110 integrates the compressed data into one bit stream and outputs it.

  The bit stream of the moving image data compressed by each of the processing units 101 to 110 is transmitted by the transmission unit 711. The internal configuration of the motion compensation unit 106 is the same as that of any one of the third to sixth embodiments.

1 is a block diagram illustrating a typical moving image encoding apparatus. Explanatory drawing of a moving image encoding system. Explanatory drawing of a moving image encoding system. Explanatory drawing of a moving image encoding system. The block diagram of the moving image encoder of 1st Example. The flowchart which shows an example of the operation | movement procedure of the apparatus of FIG. The flowchart which shows an example of the operation | movement procedure of the apparatus of FIG. The flowchart which shows an example of the operation | movement procedure of the apparatus of FIG. The block diagram of the moving image encoder of 2nd Example. The functional block diagram of the moving image compression system of 3rd-6th Example. Operation | movement explanatory drawing of the moving image compression system of 3rd Example. Operation | movement explanatory drawing of the moving image compression system of 4th Example. Explanatory drawing of operation | movement of the moving image compression system of 5th Example. Operation | movement explanatory drawing of the moving image compression system of 6th Example. The functional block diagram of the moving image compression recording system of 7th Example. The functional block diagram of the moving image compression transmission system of 8th Example. The functional block diagram of a typical moving image compression encoding system. Explanatory drawing of operation | movement of the moving image compression system of FIG.

Explanation of symbols

q Quantization step width L Signal line 50 Buffer memory 52 Buffer memory 54 Buffer memory 56 Buffer memory 58 Mode selection circuit 60 Mode determination circuit 62 Motion compensation mode selection circuit

Claims (1)

In the coding mode of the current video data of the target to be encoded into corresponding current video coding to adaptively method of selecting,
With reference to the reference video data, the prediction error with the current video data is obtained for each encoding mode,
Comparing the threshold value is changed according to the prediction error compression ratio of the encoding of each encoding mode, the encoding mode of a small prediction error than the threshold value, each of the current video coding that Obtaining a value related to the amount of total codes including encoding parameters, and selecting an encoding mode with the least amount of total codes;
Selection method.

Images