JP2010283583A - Device and method for encoding moving image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To reduce possibility that underflow of a buffer occurs. <P>SOLUTION: Based on a generated code amount of encoded data generated by an entropy encoding circuit 115, an occupation amount of a virtual buffer for temporarily storing encoded data possessed by a decoding device when decoding the generated encoded data is verified. Whether motion retrieval carried out by an inter prediction circuit 103 is carried out by emphasizing a code amount or emphasizing image quality is controlled in accordance with the verified occupation amount of the virtual buffer. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a moving image encoding device, a moving image encoding method, a program, and a storage medium, and more particularly to a technique suitable for use in adjusting a generated code amount.

  2. Description of the Related Art Conventionally, a digital video camera is well known as a camera-integrated moving image recording apparatus that captures a subject and compresses and records moving image data obtained by the shooting. In recent years, recording media for recording moving image data have changed from conventional magnetic tapes to highly convenient disk media such as random accessibility and semiconductor memories. In addition, as a compression method, the MPEG2 method capable of compressing at a high compression rate by using motion prediction between frames is generally used, and in recent years, H.264 that can be compressed to a higher compression is used. The H.264 system is also used.

  In particular, H.C. In the H.264 system or the like, the buffer amount of encoded data that the decoding apparatus has is determined. Therefore, the encoding device virtually assumes the occupation amount of the encoded data buffer (virtual buffer) included in the decoding device, and must encode the video so as not to cause underflow or overflow.

  For example, Patent Document 1 discloses a technique for suppressing a buffer failure. In Patent Document 1, a maximum code amount that does not cause an underflow of a buffer is obtained in advance, and a quantization coefficient that does not exceed the maximum code amount is obtained. On the other hand, a quantization coefficient that achieves the target bit rate is obtained from the amount of code generated so far, and quantization is performed with a quantization coefficient having a larger quantization width. Yes.

JP 2002-51343 A

  However, in the prior art disclosed in the above-mentioned Patent Document 1, when the maximum code amount that does not cause the buffer underflow is extremely small, the generated code amount may be the maximum code amount depending on the input video. It may not be possible to encode to below. Therefore, there is a problem that it is not possible to sufficiently prevent the occurrence of buffer underflow.

  The present invention has been made in view of the above-described problems, and an object thereof is to further reduce the possibility of occurrence of buffer underflow.

  The moving image encoding apparatus of the present invention includes a motion search unit that performs a motion search with reference to a past or future image with respect to an encoding target block of moving image data, and the code according to a search result by the motion search unit. A case where the generated encoded data is decoded on the basis of the encoding means for encoding the block to be encoded and generating encoded data, and the generated code amount of the encoded data generated by the encoding means A virtual buffer verification unit that verifies an occupation amount of a virtual buffer that temporarily stores encoded data included in the decoding device, and an image quality according to the occupation amount of the virtual buffer verified by the virtual buffer verification unit Control means for controlling the operation of the motion search means so as to perform motion search by adjusting the specific gravity between the evaluation value that emphasizes the evaluation value and the evaluation value that emphasizes the generated code amount. I am characterized in.

  According to the present invention, it is possible to perform a motion search that suppresses the generated code amount based on the occupation amount of the virtual buffer. Thereby, the underflow of the buffer can be prevented more reliably.

It is a block diagram which shows the structural example of the moving image encoder which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the detailed structural example of an inter prediction circuit. It is a figure which shows the transition of a virtual buffer typically. It is a block diagram which shows the structural example of the moving image encoder which concerns on the 2nd Embodiment of this invention. It is a figure which shows the relationship between the number of encoding blocks, and code amount.

(First embodiment)
Hereinafter, an image coding apparatus according to the first embodiment of the present invention will be described.
FIG. 1 is a block diagram illustrating a configuration example of a moving image encoding apparatus 100 according to the present embodiment.

  In FIG. 1, a moving image encoding apparatus 100 includes an imaging unit 101 including a camera unit such as a lens or a CCD, a frame memory 102, an inter prediction circuit 103 that searches for a motion vector, and an intra prediction that selects an intra prediction method. Circuit 104. Furthermore, a prediction method selection circuit 105 that selects a prediction method of inter prediction or intra prediction, a prediction image generation circuit 106, a subtractor 107, an integer conversion circuit 108, and a quantization circuit 109 are provided. Yes. Further, an inverse quantization circuit 110, an inverse integer conversion circuit 111, an adder 112, an in-loop filter 113, an entropy encoding circuit 115, a quantization control circuit 116, a code amount control circuit 117, and a recording circuit 118. In addition, a recording medium 119 is mounted on the moving image encoding apparatus 100, and the frame memory 102 includes a reference image memory 114 that stores a reference image used for inter prediction.

  The moving image data obtained by imaging by the imaging unit 101 is sequentially stored in the frame memory 102 in the order of the first frame, the second frame, the third frame,. Image data is extracted from the frame memory 102 in the order of encoding, for example, the third frame, the first frame, the second frame,...

  Here, the encoding methods include intra encoding that encodes only image data within a frame, and inter encoding that encodes including prediction between frames. A picture to be inter-encoded is a P picture that predicts one reference frame for a motion compensation unit (MC block) and B that predicts up to two reference frames for an MC block. There is a picture. On the other hand, a picture to be subjected to intra coding is an I picture. Note that the order of frames to be encoded is different from the order of input frames in order to enable prediction (rearward prediction) with not only past frames but also temporally future frames.

  In the case of performing intra coding, image data of a coding target block serving as a coding unit is read from the frame memory 102 and input to the intra prediction circuit 104. In this embodiment, the unit of one encoding target block is 16 horizontal pixels × 16 vertical pixels. In addition, the pixel data adjacent to the read target block to be encoded is also read from the frame memory 102 and input to the intra prediction circuit 104.

  The intra prediction circuit 104 performs block matching between the encoding target block and a plurality of intra predicted image data generated from data of pixels adjacent to the encoding target block. Then, intra prediction image data having the highest correlation is selected and output to the prediction method selection circuit 105.

  When performing intra coding, the prediction method selection circuit 105 always selects intra prediction, and notifies the prediction image generation circuit 106 of the selection result. The predicted image generation circuit 106 generates intra predicted image data from reconstructed image data output from an adder 112 described later according to the prediction method selected by the prediction method selection circuit 105.

  The subtractor 107 receives the above-described intra prediction image data and the image data of the encoding target block read from the frame memory 102, and calculates the difference image data of the pixel values of the image of the encoding target block and the intra prediction image. Output to the integer conversion circuit 108. The integer conversion circuit 108 performs integer conversion on the input difference image data of the pixel value, and the quantization circuit 109 performs quantization processing on the signal converted by the integer conversion circuit 108.

  The entropy encoding circuit 115 entropy encodes the transform coefficient quantized by the quantization circuit 109 and outputs the result to the recording circuit 118 as a stream. Here, the quantization coefficient in the quantization circuit 109 is calculated by the quantization control circuit 116 from the code amount generated in the entropy encoding circuit 115, the target code amount set from the code amount control circuit 117, and the like. The recording circuit 118 records the stream output from the entropy encoding circuit 115 on the recording medium 119.

  Further, the transform coefficient quantized by the quantization circuit 109 is also input to the inverse quantization circuit 110. The inverse quantization circuit 110 inversely quantizes the input transform coefficient, and the inverse integer transform circuit 111 performs an inverse integer transform process on the inversely quantized signal.

  The adder 112 receives and adds the inverse integer-converted data and the intra predicted image data generated by the predicted image generation circuit 106. The data after the addition becomes decoded reconstructed image data, which is input to the above-described predicted image generation circuit 106 and used to generate intra predicted image data. Also, the reconstructed image data is subjected to encoding distortion reduction processing by the in-loop filter 113 and is stored in the reference image memory 114 as reference image data used in inter-encoding described later.

  On the other hand, when inter coding is performed, image data of an encoding target block serving as a coding unit is read from the frame memory 102 and input to the inter prediction circuit 103. Further, the inter prediction circuit 103 reads the reference image data from the reference image memory 114, detects a motion vector from the image data to be encoded and the reference image data, and notifies the prediction method selection circuit 105 of the motion vector.

  Depending on the frame, it is possible to select inter prediction or intra prediction for each encoding target block. When performing the intra prediction circuit 104, it operates as described above, and notifies the prediction method selection circuit 105 of the result of the intra prediction. The prediction method selection circuit 105 inputs the result of the inter prediction circuit 103 and the result of the intra prediction circuit 104, selects, for example, the prediction method with the smaller difference value, and notifies the prediction image generation circuit 106 of it. The subtractor 107 calculates the difference between the image to be encoded and the predicted image, and generates difference image data. The difference image data is output to the integer conversion circuit 108, and the subsequent processing is the same as in the case of the intra coding described above.

  Next, the operation of the code amount control circuit 117 will be described in detail. When the encoding of the picture is completed, the code amount control circuit 117 receives information on the code amount actually generated in the picture from the entropy encoding circuit 115 and updates the occupation amount of the virtual buffer. Here, the virtual buffer is assumed to be a buffer that the decoding apparatus temporarily stores encoded data, and is defined based on a generated code amount and an output code amount in entropy encoding. The

FIG. 3 is a diagram schematically showing the transition of the virtual buffer.
As shown in FIG. 3, the occupation amount of the virtual buffer increases with time according to the encoding bit rate, and decreases by the amount of code generated in the picture. In the present embodiment, for example, the code amount generated in the P (1) picture is defined as S (P (1)).

  The code amount control circuit 117 functions as a virtual buffer verification unit, updates the occupation amount of the virtual buffer by the operation as described above, and calculates a target code amount for a picture to be encoded from this. For example, when encoding up to a B (2) picture is finished and then encoding of a B (3) picture is performed, the maximum target code that can be generated in the B (3) picture from the virtual buffer occupation amount The amount Tmax (B (3)) is calculated.

  When the generated code amount of the B (3) picture exceeds the maximum target code amount Tmax (B (3)), the virtual buffer underflows, causing a buffer failure. Therefore, the code amount control circuit 117 notifies the quantization control circuit 116 of the target code amount of the picture to be encoded based on the compression rate that is the target of encoding under the condition that the maximum target code amount is not exceeded. . In addition, the inter prediction circuit 103 is notified of information on the virtual buffer occupation amount. For example, when encoding a B (3) picture, information on the maximum target code amount Tmax (B (3)) is notified to the inter prediction circuit 103.

Next, the operation of the inter prediction circuit 103 will be described in detail. FIG. 2 is a block diagram illustrating a detailed configuration example of the inter prediction circuit 103 according to the present embodiment.
In FIG. 2, an encoded image acquisition circuit 201 acquires image data of an encoding target block from the frame memory 102. In addition, the motion vector setting circuit 206 sequentially sets motion vectors from the plurality of motion vector candidates to the reference image acquisition circuit 202 and the code amount cost calculation circuit 209.

  The reference image acquisition circuit 202 acquires reference image data corresponding to the motion vector set by the motion vector setting circuit 206 from the reference image memory 114 of the frame memory 102. Then, the acquired image data and reference image data of the encoding target block are input to the encoding degradation cost calculation circuit 208 and the code amount cost calculation circuit 209, respectively.

  For example, the encoding deterioration cost calculation circuit 208 calculates the sum of absolute differences between the image of the encoding target block and the reference image, and calculates the encoding deterioration cost. The calculated coding degradation cost corresponds to an evaluation value of coding degradation (image quality).

  On the other hand, the code amount cost calculation circuit 209 generates a code amount corresponding to the code amount from the sum of absolute values of coefficients obtained by Hadamard transform of the difference between the image data of the encoding target block and the reference image data and the set motion vector. Calculate the cost. The calculated code amount cost corresponds to an evaluation value of the generated code amount.

  The first multiplier 203 multiplies the coding deterioration cost calculated by the coding deterioration cost calculation circuit 208 by the coefficient (Ka) for the coding deterioration cost generated in the coefficient generation circuit 205. On the other hand, the second multiplier 204 multiplies the code amount cost calculated by the code amount cost calculation circuit 209 by the coefficient (Kb) for the code amount cost generated in the coefficient generation circuit 205. Then, the adder 210 adds the result multiplied by the first multiplier 203 and the result multiplied by the second multiplier 204 to the cost comparison circuit 207 as a cost value for the set motion vector. Output.

  The cost comparison circuit 207 stores information on the cost value having the smallest value among the cost values calculated so far and the motion vector corresponding thereto as a search result. Then, the motion vector stored when the calculation of the cost values for all motion vector candidates is completed is determined as the motion vector for the encoding target block.

  In accordance with the occupation amount of the virtual buffer notified from the code amount control circuit 117, the coefficient generation circuit 205 increases the value of Kb with respect to Ka as the occupation amount decreases, and conversely as the occupation amount increases. The coefficient is generated so that the value of Kb becomes smaller. Thus, the evaluation value (coefficient Kb with respect to the code amount cost) in which the code amount is emphasized when determining the motion vector as the virtual buffer occupancy is small and the virtual buffer is likely to fail (underflow). Perform motion search with priority. Also, when the virtual buffer occupies a large amount and the virtual buffer is unlikely to fail, a motion search that prioritizes the evaluation value (coefficient Ka for the coding degradation cost) that places importance on image quality when determining the motion vector is performed. Do. As a result, the amount of code by encoding is suppressed, and the virtual buffer is less likely to fail.

  As described above, according to the present embodiment, the motion search is performed by adjusting the specific gravity of the code amount cost and the encoding degradation cost in accordance with the occupation amount of the virtual buffer. As a result, when the virtual buffer occupancy is small, a motion search can be performed with an emphasis on the code amount.

(Second Embodiment)
Hereinafter, an image coding apparatus according to the second embodiment of the present invention will be described.
FIG. 4 is a block diagram illustrating a configuration example of the video encoding device 400 according to the present embodiment. The basic configuration is the same as that of the first embodiment, and the same components as those in FIG. The difference is that instead of outputting information about the occupation amount of the virtual buffer from the code amount control circuit 417, information about the maximum code amount of the encoding target block is outputted from the quantization control circuit 416 to the inter prediction circuit 403. Is a point.

  Next, the operation of the quantization control circuit 416 will be described in detail. FIG. 5 is a diagram illustrating the relationship between the number of encoded blocks and the code amount. In FIG. 5, the horizontal axis indicates the number of encoded blocks to be processed from the beginning of the picture, and the vertical axis indicates the amount of code generated so far from the beginning of the picture.

  The quantization control circuit 416 functions as a virtual buffer verification unit, and receives from the code amount control circuit 417 information on the target code amount of a picture and information on the maximum code amount of a picture that does not cause a buffer failure. A point 501 in FIG. 5 indicates the maximum code amount of a picture that does not cause a buffer failure. Further, every time the encoding process of the encoding target block is completed, the information on the code amount actually generated in the encoding target block is received from the entropy encoding circuit 115, and the cumulative amount generated from the beginning of the picture up to now is received. Is calculated.

  Here, the code amount that is always generated by encoding the block to be encoded is set as the minimum code amount of the encoded block. Based on the minimum coding amount of the coding block and the maximum coding amount of the picture, in each coding target block, the maximum coding amount that does not cause the buffer to fail due to the cumulative coding amount generated from the beginning of the picture until then. Determined. A solid line 502 in FIG. 5 indicates the maximum code amount that does not cause a buffer failure when each of the encoding target blocks is encoded.

  A point 503 indicates the cumulative code amount actually generated from the beginning of the picture to the beginning. Therefore, a coding block in which the difference between the maximum code amount that does not cause a buffer failure and the cumulative code amount that has actually occurred from the beginning of the picture until the beginning of the picture does not cause a buffer failure in the encoding target block to be encoded. Maximum code amount. That is, it corresponds to the occupation amount of the virtual buffer. The quantization control circuit 416 outputs information on the maximum code amount of the encoded block to the inter prediction circuit 403.

  The coefficient generation circuit 205 in the inter prediction circuit 403 causes the value of Kb to increase with respect to Ka as the encoding block maximum code amount decreases, according to the encoding block maximum code amount calculated by the quantization control circuit 416. To generate a coefficient. Conversely, the coefficient is generated so that the value of Kb becomes smaller with respect to Ka as the coding block maximum code amount is larger.

  In other words, when the maximum code amount of the coding block is small and the virtual buffer is likely to fail, when determining the motion vector, the decision is made with an emphasis on the code amount. Failure is less likely to occur. The other configuration of the inter prediction circuit 403 is the same as the configuration of the inter prediction circuit 103 described in FIG. 2 of the first embodiment.

  As described above, according to the present embodiment, the motion search is performed by adjusting the specific gravity of the code amount cost and the encoding deterioration cost in accordance with the maximum code amount of the encoded block. As a result, when the virtual buffer occupancy is small, a motion search can be performed with an emphasis on the code amount.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

(Other embodiments according to the present invention)
Each means constituting the moving picture coding apparatus and each step of the moving picture coding method in the embodiment of the present invention described above can be realized by operating a program stored in a RAM or ROM of a computer. This program and a computer-readable storage medium storing the program are included in the present invention.

  In addition, the present invention can be implemented as, for example, a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  Note that the present invention includes a case where a software program that realizes the functions of the above-described embodiments is supplied directly or remotely to a system or apparatus. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the storage medium for supplying the program include a flexible disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. It can also be supplied by downloading the computer program itself of the present invention or a compressed file including an automatic installation function from a homepage to a storage medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, the present invention includes a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer.

  As another method, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, a program read from a storage medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

  103 inter prediction circuit, 115 entropy encoding circuit, 117 code amount control circuit

Claims (7)

A motion search means for performing a motion search with reference to a past or future image with respect to an encoding target block of moving image data;
Encoding means for encoding the block to be encoded in accordance with a search result by the motion search means, and generating encoded data;
Occupancy of a virtual buffer that temporarily stores the encoded data of the decoding device when the generated encoded data is decoded based on the generated code amount of the encoded data generated by the encoding means Virtual buffer verification means for verifying the amount;
The motion search is performed such that the motion search is performed by adjusting the specific gravity of the evaluation value emphasizing image quality and the evaluation value emphasizing the generated code amount according to the occupation amount of the virtual buffer verified by the virtual buffer verification unit. And a control means for controlling the operation of the means.   2. The moving image coding according to claim 1, wherein the control means controls the motion search means so as to perform a motion search with an emphasis on a generated code amount as the occupation amount of the virtual buffer decreases. apparatus.   The virtual buffer verification means calculates a maximum code amount that does not cause the virtual buffer to fail by encoding from the maximum generated code amount of the picture including the coding target block that does not fail the virtual buffer to the coding target block. The moving picture encoding apparatus according to claim 1, wherein the occupation amount of the virtual buffer is verified by calculation.   The control means has a difference between a maximum code amount at which the virtual buffer does not fail due to encoding up to the encoding target block and a code amount generated by encoding up to the encoding target block by the encoding means. 4. The moving picture coding apparatus according to claim 3, wherein the motion search means is controlled so as to perform a motion search with an emphasis on the amount of generated code as it decreases. A motion search step for performing motion search with reference to past or future images with respect to an encoding target block of moving image data;
An encoding step of encoding the block to be encoded according to a search result in the motion search step and generating encoded data;
Occupancy of a virtual buffer that temporarily stores the encoded data of the decoding device when the generated encoded data is decoded based on the generated code amount of the encoded data generated in the encoding step A virtual buffer verification process to verify the amount;
The motion search is performed such that the motion search is performed by adjusting the specific gravity of the evaluation value emphasizing image quality and the evaluation value emphasizing the generated code amount according to the virtual buffer occupation amount verified in the virtual buffer verification step. And a control process for controlling the operation in the process. A motion search step for performing motion search with reference to past or future images with respect to an encoding target block of moving image data;
An encoding step of encoding the block to be encoded according to a search result in the motion search step and generating encoded data;
Occupancy of a virtual buffer that temporarily stores the encoded data of the decoding device when the generated encoded data is decoded based on the generated code amount of the encoded data generated in the encoding step A virtual buffer verification process to verify the amount;
The motion search is performed such that the motion search is performed by adjusting the specific gravity of the evaluation value emphasizing image quality and the evaluation value emphasizing the generated code amount according to the virtual buffer occupation amount verified in the virtual buffer verification step. A program for causing a computer to execute a control process for controlling operations in the process.   A computer-readable storage medium storing the program according to claim 6.

Images