JP6943256B2 - Coding device, coding method and program - Google Patents

Description

The present invention relates to a coding device, a coding method and a program for coding a moving image. In particular, the present invention relates to a coding device, a coding method, and a program that divides an image to be coded into a plurality of image blocks and performs processing in units of image blocks.

Video compression coding technology is used in a wide range of applications such as digital broadcasting, distribution of video content using optical discs, and video distribution via the Internet. International standards have been established by international standardization bodies for technologies for encoding moving image signals with low bit rates, high compression rates, and high image quality, and for decoding encoded moving images. Representative standards bodies include the International Telecommunication Union, the International Organization for Standardization, and the American Society of Motion Picture and Television Engineers.

According to the International Telecommunication Union (ITU) standard, H. 261 and H. There are 263. International Organization for Standardization (ISO) standards include MPEG-1, MPEG-2, and MPEG-4 (MPEG: Moving Picture Experts Group). The standards of the Society of Motion Picture and Television Engineers (SMPTE) include VC-1.

H. 264 / MPEG-4 AVC (Advanced Video Coding) was standardized jointly by ITU and ISO (Non-Patent Document 1). In 2013, H.M. 265 / MPEG-H HEVC (High Efficiency Video Coding) has been standardized (Non-Patent Document 2). From this point onward, H. 264 / MPEG-4 AVC by H. Described as 264, H. 265 / MPEG-H HEVC It is described as 265. H. 265 is H. The data size can be compressed to about half with the same video quality as 264.

The above-mentioned moving image coding technology is composed of a combination of a plurality of elemental technologies such as motion compensation prediction, orthogonal transformation of a prediction error image, quantization of an orthogonal transformation coefficient, and entropy coding of a quantized orthogonal transformation coefficient. Therefore, it is called hybrid coding. Specific entropy coding methods are disclosed in, for example, Non-Patent Document 3 and Non-Patent Document 4.

In the above-mentioned moving image coding technology, high compression efficiency is achieved by performing in-frame prediction using image correlation in the spatial direction and inter-frame prediction using image correlation in the time axis direction. .. In general inter-frame prediction, motion compensation is used to correct the movement and positional deviation of a subject, background, etc. between images that are close in time to generate a predicted image.

Further, in a general moving image coding device, rate control processing for achieving high image quality is performed while suppressing the output code amount to a predetermined bit rate or less.

As an example of the rate control technology, the MPEG-2 Test Model 5 method (hereinafter referred to as TM-5 method) can be mentioned. The TM-5 method is composed of a combination of a plurality of controls. One is the code amount allocation control for each frame based on the complexity estimation that differs depending on the frame coding type such as I / P / B. One is the quantization parameter (QP) control in macroblock (MB) units within the frame. One is adaptive quantization control that improves subjective image quality. Of these, in the MB unit control, the QP of each MB is feedback-controlled based on the relationship between the code amount actually generated in the coded MB and the target code amount.

Further, Patent Document 1 discloses an image coding apparatus that performs coefficient truncation control as rate control. The apparatus of Patent Document 1 determines the censoring parameter based on the predicted value of the code amount and the target code amount for each frame or image block set separately. Then, the apparatus of Patent Document 1 replaces a part of the quantization coefficient with 0 (discontinuation) according to the determined termination parameter, and predicts the generated code amount from the distribution state of the quantization coefficient and the like.

In the TM-5 method and the method of Patent Document 1, since the code amount actually output in block units is fed back to control the generated code amount of the entire frame, it is possible to precisely control the target code amount.

Further, Non-Patent Document 5 and Patent Document 2 disclose a rate control method based on a model called an R-ρ model (R: bit rate, ρ: ratio of conversion coefficient to be zero after quantization). .. In the method using the R-ρ model, an appropriate QP is set in consideration of the occurrence status of the conversion coefficient of the entire frame, so that stable image quality can be achieved in the frame.

Further, Patent Document 3 discloses a data processing apparatus that evaluates the coding efficiency of input data and performs data processing according to the evaluation result. The device of Patent Document 3 is. Frequency conversion is performed on the input image data to generate a conversion coefficient. Then, the apparatus of Patent Document 3 evaluates the code amount based on the generated conversion coefficient, performs the quantization process according to the evaluation result, and encodes the quantized conversion coefficient with a Huffman code. ..

Japanese Unexamined Patent Publication No. 2010-087771 Japanese Patent Application Laid-Open No. 2013-532439 Japanese Unexamined Patent Publication No. 2011-142660

ITU-T Recommendation H.264 "Advanced video coding for generic audiovisual services", March 2010 ITU-T Recommendation H.265 "Advanced video coding for generic audiovisual services", April 2013 Joint Video Team (JVT) of ISO / IEC MPEG and ITU-T VCEG, Document JVT-O079, "Text Description of Joint Mode Reference Encoding Method and Decoding Concealment Method", April 2005 Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, Document JCTVC-S1002, "High Efficiency Video Coding (HEVC) Test Model 16 (HM 16) Improved Encoder Description" , October 2014 Yung-Lyul Lee et al., "Rate control using linear rate-ρ model for H.264", Signal Processing: Image Communication, Volume 19, Issue 4, P.341-352, April 2004

In the TM-5 method and the technique described in Patent Document 1, since control is performed by local feedback in block units, control parameters such as QP fluctuate or vibrate within the frame depending on the strength of feedback. Image quality may become unstable. This becomes remarkable when a region in which the pattern is complicated and difficult to code and a region in which the pattern is simple and easy to code coexist in one frame. Further, in the TM-5 method and the technique described in Patent Document 1, since the feedback control is performed in block units, sequential processing is performed in block units, and a plurality of blocks cannot be processed in parallel, resulting in high speed. There is a problem that it is difficult. Further, the method of performing the censoring process of Patent Document 1 has a problem that the control may not be stable.

In the techniques of Non-Patent Document 5 and Patent Document 2, a code amount model is constructed using the conversion coefficient of the entire frame, and code amount prediction corresponding to various QP settings is performed to correct the QP. In the techniques of Non-Patent Document 5 and Patent Document 2, since the conversion coefficient is the same as the number of pixels, a large number of conversion coefficient values are analyzed, which increases the processing amount. Further, in the techniques of Non-Patent Document 5 and Patent Document 2, since the control is performed based on the QP determination using the code amount model, there is a possibility that the difference from the target code amount when an error occurs in the code amount model becomes large. There is.

The apparatus of Patent Document 3 executes the quantization process twice or more. Quantization processing is a relatively heavy processing that performs operations including multiplication or division of individual conversion coefficients. Therefore, the apparatus of Patent Document 3 has a problem that the processing load of the entire apparatus increases when the quantization process is repeatedly executed.

An object of the present invention is to provide a coding device capable of reducing the processing amount of the entire device and realizing stable image quality in order to solve the above-mentioned problems.

In one aspect of the present invention, the coding apparatus receives a target code amount for each processing target block constituting the coding target image as an input, and sets a quantization control parameter for determining the quantization coefficient of the entire coding target image. Quantization control by inputting the quantization parameter control means generated based on the target code amount and the conversion coefficient generated based on the image to be encoded, and inputting the quantization control parameter from the quantization parameter control means. A quantization means that generates a quantization coefficient by performing a quantization process using parameters and conversion coefficients, a quantization coefficient memory that stores the quantization coefficient generated by the quantization means, and a quantization coefficient memory. A code amount model construction means for acquiring the quantization coefficient stored in the above and constructing a code amount model using the quantization coefficient obtained by quantizing the entire coded image, and a code amount model input from the code amount model construction means. At the same time, the target code amount is input, and the cutoff control means for determining the cutoff parameter for cutting off the quantization coefficient based on the target code amount and the code amount model, and the cutoff parameter are input from the cutoff control means. Quantization coefficient A quantization coefficient setting means for acquiring a quantization coefficient stored in a memory and having a coefficient cutoff means for cutting off the quantization coefficient according to a cutoff parameter is provided.

In one aspect of the present invention, in the coding method, the target code amount for each processing target block constituting the coding target image is input, and the quantization control parameter for determining the quantization coefficient of the entire coding target image is set. Generated based on the target code amount, input the conversion coefficient generated based on the image to be encoded, and generate the quantization coefficient by performing the quantization process using the quantization control parameter and the conversion coefficient. The generated quantization coefficient is stored in the quantization coefficient memory, a code amount model is constructed using the quantization coefficient obtained by quantizing the entire coded target image stored in the quantization coefficient memory, and the target code amount and code amount are used. Based on the model, the cutoff parameter for breaking the quantization coefficient is determined, and the quantization coefficient stored in the quantization coefficient memory is cut off according to the cutoff parameter.

In one aspect of the present invention, the program recording medium is a process of inputting a target code amount for each processing target block constituting the coding target image, and a quantization control for determining the quantization coefficient of the entire coding target image. Quantization by performing a process of generating parameters based on a target code amount, a process of inputting a conversion coefficient generated based on an image to be encoded, and a quantization process using a quantization control parameter and a conversion coefficient. A code amount model using the process of generating the quantization coefficient, the process of storing the generated quantization coefficient in the quantization coefficient memory, and the quantization coefficient obtained by quantizing the entire object to be encoded stored in the quantization coefficient memory. The process of constructing, the process of determining the terminating parameter for terminating the quantization coefficient based on the target code amount and the code amount model, and the process of terminating the quantization coefficient stored in the quantization coefficient memory according to the terminating parameter. Record the program that causes the computer to execute.

According to the present invention, it is possible to provide a coding device capable of realizing stable image quality while reducing the processing amount of the entire device.

It is a block diagram which shows the structure of the moving image coding apparatus which concerns on 1st Embodiment of this invention. It is a conceptual diagram which shows an example of the signal flow in the moving image coding apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structure of the quantization coefficient setting means of the moving image coding apparatus which concerns on 1st Embodiment of this invention. It is a conceptual diagram which shows an example of the signal flow in the quantization coefficient setting means of the moving image coding apparatus which concerns on 1st Embodiment of this invention. It is a flowchart which shows the operation of the moving image coding apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating the censoring target in the moving image coding apparatus which concerns on 1st Embodiment of this invention. It is a block diagram which shows the structure of the moving image coding apparatus which concerns on 2nd Embodiment of this invention. It is a conceptual diagram which shows an example of the signal flow in the moving image coding apparatus which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the operation of the moving image coding apparatus which concerns on 2nd Embodiment of this invention. It is a flowchart which shows the operation of the moving image coding apparatus which concerns on 3rd Embodiment of this invention. It is a block diagram which shows the structure of the moving image coding apparatus which concerns on 6th Embodiment of this invention. It is a conceptual diagram which shows an example of the signal flow in the moving image coding apparatus which concerns on 6th Embodiment of this invention. It is a flowchart which shows the operation of the moving image coding apparatus which concerns on 6th Embodiment of this invention. It is a block diagram which shows the structural example of the hardware which realizes the moving image coding apparatus which concerns on each embodiment of this invention. It is a block diagram which shows another structural example of the hardware which realizes the moving image coding apparatus which concerns on each embodiment of this invention. It is a conceptual diagram which shows the structural example of the GPU (Graphic Processing Unit) included in the hardware which realizes the moving image coding apparatus which concerns on each embodiment of this invention. It is a block diagram which shows the structure of the moving image coding apparatus of a related technique. It is a conceptual diagram which shows an example of the signal flow in the moving image coding apparatus of a related technique. It is a block diagram which shows the structure of the moving image coding apparatus which applies the TM-5 system. It is a conceptual diagram which shows an example of the signal flow in the moving image coding apparatus which applies the TM-5 system. It is a block diagram which shows the structure of the moving image coding apparatus of Patent Document 1. FIG. It is a conceptual diagram which shows an example of the signal flow in the moving image coding apparatus of Patent Document 1. FIG. It is a flowchart which shows the operation of the moving image coding apparatus of Patent Document 1. FIG. It is a block diagram which shows the structure of the moving image coding apparatus which applies the R-ρ model. It is a conceptual diagram which shows an example of the signal flow in the moving image coding apparatus which applies the R-ρ model. It is a flowchart which shows the operation of the moving image coding apparatus which applies an R-ρ model.

Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. However, although the embodiments described below have technically preferable limitations for carrying out the present invention, the scope of the invention is not limited to the following. In all the drawings used in the following embodiments, the same reference numerals are given to the same parts unless there is a specific reason. Further, in the following embodiments, repeated explanations may be omitted for similar configurations and operations. Further, the direction of the arrow in the drawing shows an example, and does not limit the direction of the signal between the blocks.

(First Embodiment)
〔composition〕
First, the configuration of the moving image coding device (also referred to as a coding device) according to the first embodiment of the present invention will be described with reference to the drawings.

FIG. 1 is an example of the configuration of the moving image coding device 1 according to the present embodiment. FIG. 2 is a conceptual diagram showing an example of a signal flow in the moving image coding device 1 of the present embodiment.

When a new image is input, the moving image coding device 1 performs coding processing in units of image blocks of a predetermined size. For example, H. In the 264 method, a block of 16 × 16 pixels called a macroblock (MB: Macroblock) is used. In addition, H. In the 265 method, blocks such as 16 × 16 pixels, 32 × 32 pixels, and 64 × 64 pixels, which are called Coding Tree Units (CTUs), are used. Note that in FIGS. 1 and 2, the function of acquiring the input image and the function of dividing the input image are omitted.

The moving image coding device 1 includes a prediction means 10, a subtraction / conversion means 11, a quantization coefficient setting means 12, a coding means 13, an inverse quantization means 14, an inverse conversion / addition means 15, a filter 16, and a frame buffer 17. Be prepared.

The prediction means 10 takes an input image (encoded image) as an input, and acquires an encoded image (image data of a frame encoded in the past) corresponding to the input image from the frame buffer 17. The prediction means 10 generates a prediction image by performing prediction processing within or between frames using the coded image and the input image. The prediction means 10 outputs the generated predicted image to the subtraction / conversion means 11 and the inverse conversion / addition means 15.

The subtraction / conversion means 11 takes an input image as an input, and acquires a prediction image corresponding to the input image from the prediction means 10. The subtraction / conversion means 11 subtracts the prediction image from the input image to generate a prediction error image. The subtraction / conversion means 11 performs an orthogonal transform process similar to DCT (Discrete Cosine Transform) on the generated prediction error image, and outputs a conversion coefficient.

For example, H. In the case of the 264 method, the subtraction / conversion means 11 performs orthogonal conversion processing in block units of 4 × 4 pixels or 8 × 8 pixels. Also, for example, H. In the case of the 265 method, the subtraction / conversion means 11 performs orthogonal conversion processing in block units of a predetermined size from 4 × 4 pixels to 32 × 32 pixels.

The quantization coefficient setting means 12 inputs a conversion coefficient from the subtraction / conversion means 11. Further, the quantization coefficient setting means 12 inputs a target code amount for each frame and image block set by an external system such as a host system.

The quantization coefficient setting means 12 determines a quantization control parameter (QP: Quantization Parameter) based on the code amount of the coding result and the target code amount. The quantization coefficient setting means 12 performs rate control processing for achieving high image quality while suppressing the amount of code output from the moving image coding device 1 to a predetermined bit rate or less.

Further, the quantization coefficient setting means 12 performs a quantization process using the conversion coefficient input from the subtraction / conversion means 11 and the determined quantization control parameter, and the quantization coefficient (hereinafter referred to as quantization) is quantized. Coefficient) is output.

The coding means 13 inputs the quantization coefficient from the quantization coefficient setting means 12. The coding means 13 entropy-codes the quantization coefficient according to a predetermined rule, and outputs a bit stream of the coding result.

For example, the coding means 13 is H.I. In the 265 method, the quantization coefficient is entropy-coded using Context-based Adaptive Binary Arithmetic Coding (CABAC). Further, the coding means 13 is described by H.I. In the 264 method, the quantization coefficient is entropy-coded using Context-based Adaptive variable-length coding (CAVLC) and CABAC.

The inverse quantization means 14 inputs the quantization coefficient from the quantization coefficient setting means 12. The dequantization means 103 dequantizes the quantization coefficient and outputs the dequantized quantization coefficient to the inverse conversion / addition means 15.

The inverse transformation / addition means 15 inputs the inverse quantization coefficient from the inverse quantization means 14. Further, the inverse conversion / addition means 15 inputs a prediction image from the prediction means 10. The inverse transformation / addition means 15 adds the inverse quantized quantization coefficient and the predicted image to generate a reconstructed image. The inverse transformation / addition means 15 outputs the reconstructed image to the filter 16.

The filter 16 inputs the reconstructed image from the inverse transformation / addition means 15. The filter 16 filters the reconstructed image and outputs the filtered reconstructed image. The filtering process reduces the distortion generated in the image due to the coding. For example, H. In the case of the 264 method, the filter 16 filters the reconstructed image by using the deblock filter. In addition, H. In the case of the 265 method, the filter 16 filters the reconstructed image by using the SAO (Sample Adaptive Offset) and the deblock filter.

The frame buffer 17 stores the reconstructed image after the filter processing output from the filter 16. The filtered reconstructed image stored in the frame buffer 17 is used as an encoded image in the coding of subsequent frames.

[Quantization coefficient setting means]
Next, the detailed configuration of the quantization coefficient setting means 12 will be described with reference to the drawings. FIG. 3 is a block diagram showing the configuration of the quantization coefficient setting means 12. FIG. 4 is a conceptual diagram showing an example of signal flow in the quantization coefficient setting means 12.

The quantization coefficient setting means 12 includes a QP control means 21, a quantization means 22, a quantization coefficient memory 23, a code amount model construction means 24, a cutoff control means 25, and a coefficient cutoff means 26.

The QP control means 21 (also referred to as a quantization parameter control means) inputs a target code amount from an external host system or the like. The QP control means 21 is a quantization control parameter for determining the quantization coefficient (also referred to as QP) of the entire frame based on the code amount of the coding result and the target code amount for each frame or image block set separately. To generate. The QP control means 21 outputs the generated quantization control parameter to the quantization means 22.

For example, the QP control means 21 generates quantization control parameters (MPEG: Moving Picture Experts Group) by using the frame level control method of the Test Model 5 method (hereinafter referred to as TM-5 method) of MPEG-2. When the TM-5 method is applied, the QP control means 21 calculates the QP based on the code amount allocation for each frame based on the complexity estimation that differs depending on the frame coding type and the past coding history. Further, the QP control means 21 may calculate the QP based on the statistical property of the difference between the input image and the predicted image.

The quantization means 22 inputs the conversion coefficient from the subtraction / conversion means 11, and also inputs the quantization control parameter from the QP control means 21. The quantization means 22 performs a quantization process using a quantization control parameter and a conversion coefficient to generate a quantized conversion coefficient (hereinafter referred to as a quantization coefficient). The quantization means 22 stores the quantization coefficient in the generated quantization coefficient memory 23.

The quantization coefficient memory 23 stores the quantization coefficient for one frame.

The code amount model construction means 24 acquires the quantization coefficient stored in the quantization coefficient memory 23, and constructs the code amount model using the quantization coefficient obtained by quantizing the entire frame. The code amount model construction means 24 outputs the constructed code amount model to the censoring control means 25.

The censoring control means 25 inputs the code amount model from the code amount model construction means 24 and also inputs the target code amount. The censoring control means 25 determines the censoring parameter for censoring the quantization coefficient based on the predicted value of the code amount, the target code amount for each frame or image block set separately, and the code amount model. The censoring control means 25 outputs the determined censoring parameter to the coefficient censoring means 26. Note that discontinuing the quantization coefficient means forcibly replacing at least a part of the quantization coefficient with 0.

The coefficient truncation means 26 inputs the truncation parameter from the truncation control means 25 and acquires the quantization coefficient stored in the quantization coefficient memory 23. The coefficient truncation means 26 truncates the quantization coefficient according to the truncation parameter. The coefficient truncation means 26 outputs the processed quantization coefficient to the coding means 13 and the inverse quantization means 14.

The above is a description of the configuration of the moving image coding device 1 of the present embodiment.

〔motion〕
Next, the operation of the moving image coding device 1 according to the present embodiment will be described with reference to the drawings.

FIG. 5 is a flowchart showing the operation of the moving image coding device 1 of the present embodiment. In the following description of the flowchart of FIG. 5, the components of the moving image coding device 1 are described as the moving main body of each step, but the overall moving main body is the moving image coding device 1.

The quantization coefficient setting means 12 determines the QP to be applied to the entire frame (step S101).

The subtraction / conversion means 11 performs a predetermined orthogonal transformation process on the block, and outputs the conversion coefficient to the quantization coefficient setting means 12 (step S102).

The quantization coefficient setting means 12 performs the quantization processing of the conversion coefficient based on the determined QP, outputs the quantization coefficient to the coding means 13 and the inverse quantization means 14, and stores the quantization coefficient of the entire frame. (Step S103).

The moving image coding device 1 determines whether or not the processing of all the blocks constituting the frame is completed (step S104). If there is a block that has not been completed (No in step S104), the process returns to step S102.

The quantization coefficient setting means 12 constructs a code amount model such as an R-ρ model based on the generation status of the quantization coefficient of the quantization result for all the blocks constituting the frame, and calculates the model parameters (step S105). ).

The quantization coefficient setting means 12 determines the coefficient truncation parameter so as to match the generated code amount of the frame with the target code amount by using the code amount model (step S106).

The quantization coefficient setting means 12 performs the quantization coefficient truncation process based on the coefficient truncation parameter (step S107).

The coding means 13 performs a variable-length coding process on the quantization coefficient and outputs a code (step S108).

The inverse quantization means 14 performs an inverse quantization process of the quantization coefficient (step S109).

The inverse transformation / addition means 15 performs an inverse transformation process to complete the process of one block (step S110).

The moving image coding device 1 determines whether or not the processing of all the blocks constituting the frame is completed (step S111). When the processing of all the blocks is completed (Yes in step S110), the processing according to the flowchart of FIG. 5 is completed. If there is a block for which processing has not been completed (No in step S111), the process returns to step S107.

In a general moving image coding device to which the R-ρ model is applied, the QP is determined based on the code amount model constructed from the conversion coefficients before quantization. On the other hand, in the present embodiment, the coefficient cutoff parameter is determined based on the code quantity model constructed by using the quantization coefficient obtained by quantizing the entire frame.

[Determination of coefficient censoring parameters]
Here, a method for determining the coefficient truncation parameter will be described with an example. FIG. 6 is a diagram illustrating an example of determining a coefficient truncation parameter in the present embodiment.

FIG. 6 is a histogram in which the quantization coefficients are classified according to the order and the number of non-zero coefficients is arranged in order from the DC component and the AC low-order component to the AC high-order component (DC: Direct Current, AC: Alternating Current). .. The order of arranging the non-zero coefficients may be, for example, the coding order of the quantization coefficients such as ZigZag scan and Digital scan.

Using the histogram of FIG. 6, when a certain order is set as the cutoff threshold and the quantization coefficient higher than that order is cut off, the coefficient of what proportion of the non-zero quantization coefficient is deleted. You can ask for a coefficient. By applying the non-zero quantization coefficient when the truncation threshold is set as shown in FIG. 6 to a code amount model such as the R-ρ model, it is possible to calculate how much the bit rate is reduced. By using this relationship, it is possible to determine how to set the censoring threshold to achieve the target code amount.

As a method for determining the coefficient parameter, various methods can be considered in addition to the above example. For example, it is possible to create a histogram in which the quantization coefficients are classified by absolute value, and determine the threshold value from the relationship between the non-zero coefficient reduction rate when the coefficient whose absolute value is smaller than a certain threshold value is cut off and the bit rate reduction rate. can. Further, a coefficient truncation condition that combines both the order of the coefficient and the absolute value can be used. That is, the cutoff threshold value in the absolute value of the coefficient is set to a different value for each coefficient order, and the higher the order of the coefficient, the higher the cutoff threshold value, or the higher the coefficient of both the horizontal frequency and the vertical frequency, the higher the cutoff threshold value. Such control can be performed. It is also possible to set the relationship between the order and the censoring threshold based on the properties of the quantization matrix. Further, the coefficient truncation parameter can be set differently according to the block size for orthogonal transformation.

As described above, in the present embodiment, an appropriate coefficient truncation parameter is set in consideration of the generation state of the quantization coefficient of the entire frame. Therefore, according to the present embodiment, stable image quality can be achieved within the frame. That is, according to the present embodiment, it is possible to reduce the processing amount of the entire device and realize stable image quality.

Further, in the present embodiment, the entire frame is quantized by the QP set at the beginning to perform rate control with coarse accuracy, and then the coefficient is truncated based on the actual quantization result to achieve a more detailed target. Perform rate control to match the code amount. Therefore, according to the present embodiment, even when an error occurs in the QP control or the code amount model, the magnitude of the influence can be suppressed and the difference from the target code amount can be controlled to be small.

Further, in the present embodiment, the code amount model is constructed and the truncation control is performed using the quantization coefficient after the quantization process. Therefore, the processing amount is smaller than that of the technique described in Patent Document 2 (Japanese Patent Laid-Open No. 2013-532439), and high-speed processing is possible. This is because in a normal image, most of the conversion coefficients are zeroed by the quantization process, so the number of quantization coefficients processed in the code quantity model construction is much smaller than the number of conversion coefficients before quantization. be.

Further, according to the present embodiment, the quantization processing and the coefficient truncation processing can be further speeded up by performing parallel processing in block units.

In the flowchart of FIG. 5, step S102 (conversion process) and step S103 (quantization process) are sequentially processed in block units, but the conversion process and the quantization process can be processed in parallel for a plurality of blocks. Further, in the flowchart of FIG. 5, step S107 (coefficient truncation processing), step S109 (inverse quantization processing), and step S110 (inverse transformation processing) are also sequentially processed in block units, but these processes are also performed in a plurality of blocks. Can be processed in parallel.

(Second Embodiment)
〔composition〕
Next, the moving image coding apparatus according to the second embodiment of the present invention will be described with reference to the drawings. The configuration of the moving image coding device of this embodiment is the same as that of the first embodiment (FIG. 1), but the functions of the quantization coefficient setting means 12 are different.

FIG. 7 is a block diagram showing the configuration of the quantization coefficient setting means 12-2 included in the moving image coding apparatus of the present embodiment. FIG. 8 is a conceptual diagram showing an example of the signal flow in the quantization coefficient setting means 12-2. In the quantization coefficient setting means 12-2, in the quantization coefficient setting means 12 (FIG. 3) of the first embodiment, the quantization coefficient analysis means 27 is added between the quantization means 22 and the quantization coefficient memory 23. Has a quantized configuration. Since the configuration of the quantization coefficient setting means 12-2 other than the quantization coefficient analysis means 27 is the same as that of the quantization coefficient setting means 12 of the first embodiment, detailed description thereof will be omitted.

In the present embodiment, the quantization means 22 outputs the generated quantization coefficient to the quantization coefficient analysis means 27 instead of the quantization coefficient memory 23.

The quantization coefficient analysis means 27 inputs the quantization coefficient from the quantization means 22. The quantization coefficient analysis means 27 analyzes the quantization coefficient and verifies whether or not there is a quantization coefficient that becomes non-zero after quantization in the block to be processed (quantization coefficient analysis). The quantization coefficient analysis means 27 stores information regarding whether or not a non-zero quantization coefficient exists (referred to as non-zero quantization coefficient presence / absence information) in the quantization coefficient memory 23 in association with the quantization coefficient.

The moving image coding apparatus of the present embodiment performs a skip determination of subsequent processing based on the non-zero quantization coefficient presence / absence information of the block to be processed. When the non-zero quantization coefficient of the block to be processed does not exist, the coefficient truncation means 26, the coding means 13, the inverse quantization means 14, and the inverse conversion / addition means 15 do not perform processing. On the other hand, when the non-zero quantization coefficient of the block to be processed exists, the coefficient truncation means 26, the coding means 13, the inverse quantization means 14, and the inverse conversion / addition means 15 execute the processing.

The above is a description of the configuration of the moving image coding device of the present embodiment.

〔motion〕
Next, the operation of the moving image coding apparatus of this embodiment will be described with reference to the drawings.

FIG. 9 is a flowchart showing the operation of data conversion in the present embodiment. In the following description of the flowchart of FIG. 9, the components of the moving image coding device are described as the moving main body of each step, but the overall moving main body is the moving image coding device. In the following, detailed description of the same operation as that of the first embodiment will be omitted, and different operations will be described.

In the flowchart of the present embodiment (FIG. 9), a quantization coefficient analysis process (step S204) and a process skip determination process (step S208) are added to the flowchart of the first embodiment (FIG. 5). That is, in the flowchart of FIG. 9, in the flowchart of FIG. 5, a quantization coefficient analysis process is added between steps S103 and S104, and a process skip determination process is added between steps S106 and S107. ..

In addition, steps S101 to S103 of FIG. 5 correspond to steps S201 to S203 of FIG. 9, steps S104 to S106 of FIG. 5 correspond to steps S205 to S207 of FIG. 9, and steps S107 to S111 of FIG. 5 correspond to FIG. Corresponds to steps S209 to S213 of.

The differences between the flowchart of FIG. 5 and the flowchart of FIG. 9 are as follows.

That is, in step S205, the quantization coefficient analysis means 27 analyzes the quantization coefficient, and information on whether or not there is a quantization coefficient that becomes non-zero after quantization in the block to be processed (non-zero quantization coefficient information). ) Is output.

In step S208, the moving image coding apparatus makes a skip determination of the subsequent processing based on the non-zero quantization coefficient presence / absence information output by the quantization coefficient analysis means 27.

That is, when there is no non-zero quantization coefficient presence / absence information (Yes in step S208), the coefficient truncation means 26, the coding means 13, the inverse quantization means 14, and the inverse conversion / addition means 15 perform processing related to the block. No (proceed to step S213).

On the other hand, when the non-zero quantization coefficient presence / absence information exists (No in step S208), the coefficient truncation means 26, the coding means 13, the inverse quantization means 14, and the inverse conversion / addition means 15 perform processing related to the block. (Proceed to step S209).

In the present embodiment, the processing is skipped based on the non-zero quantization coefficient presence / absence information of the block to be processed. Therefore, according to the present embodiment, the processing amount can be reduced as compared with the first embodiment, and higher speed processing is possible. This is because, in a normal image, the quantization coefficient is all zero in many blocks, so the amount of processing can be reduced by skipping the processing after the coefficient cutoff in the block in which the quantization coefficient is all zero.

(Third Embodiment)
Next, a third embodiment of the present invention will be described with reference to the drawings. The configuration of this embodiment is the same as that of the second embodiment (FIG. 7), but the operation of the quantization coefficient analysis means 27 is different.

FIG. 10 is a flowchart showing the operation of data conversion in the present embodiment. In the following description of the flowchart of FIG. 10, the components of the moving image coding device are described as the moving main body of each step, but the overall moving main body is the moving image coding device. In the following, detailed description of the same operation as in the second embodiment will be omitted, and different operations will be described.

The flowchart of the present embodiment (FIG. 10) is different from the flowchart of the second embodiment (FIG. 9) in that the operation of step S304 (quantization coefficient analysis) and step S309 (censoring skip determination) are included. ..

In step S304, the quantization coefficient analysis means 27 adds the information on the presence / absence of the non-zero quantization coefficient, and if the non-zero quantization coefficient exists, the coefficient distribution information regarding the distribution status of the non-zero coefficient in the processing target block. To generate. The quantization coefficient analysis means 27 outputs the generated coefficient distribution information.

For example, the quantization coefficient analysis means 27 divides a block into finer subblocks such as 4 × 4, and outputs a flag sequence indicating whether or not each of the subblocks included in the block contains a non-zero coefficient. ..

In step S309, the moving image coding apparatus determines whether or not to skip the coefficient censoring process by using the coefficient censoring parameter and the coefficient distribution information. At this time, when the moving image coding device is determined to cut off the coefficient of the order higher than a certain threshold as the coefficient cutoff parameter, the target subblock corresponding to the position of the corresponding order or higher is not included in the processing target block. Check the flag with or without zero coefficient.

When the non-zero coefficient does not exist in all the target subblocks (Yes in step S309), the coefficient censoring means 26 skips the coefficient censoring process based on the determination result. On the other hand, when a non-zero coefficient exists in any of the target subblocks (No in step S309), the coefficient censoring means 26 executes the coefficient censoring process based on the determination result (step S310).

As the coefficient distribution information, various information other than the above examples can be considered. For example, the order information of the non-zero coefficient located at the highest order in the block, the number of non-zero coefficients having a predetermined order or more, and the like can be used as the coefficient distribution information.

According to the present embodiment, as compared with the second embodiment, the processing amount can be reduced and higher speed processing can be performed. This is because the coefficient truncation processing is not performed for all the non-zero quantizations contained in the block, but the processing is performed by skipping the discontinuing processing in the block that does not require the discontinuing processing by the judgment using the coefficient distribution information. This is because the amount can be reduced.

(Fourth Embodiment)
Next, a fourth embodiment of the present invention will be described with reference to the drawings. The configuration and operation of this embodiment are the same as those of FIGS. 7 and 10, respectively, and the operation of the QP control means 21 is different.

In the first embodiment, the QP control means 21 determines the QP by a method such as frame level control of the TM-5. On the other hand, the QP control means 21 of the present embodiment modifies the QP determined as in the first embodiment by a predetermined method to make the QP smaller. For example, the QP control means 21 modifies the QP so that the quantization step width determined by the QP is reduced by a predetermined ratio. The QP control means 21 may set the reduction ratio of the quantization step width to be different for each frame coding type such as I / P / B.

When the QP control means 21 modifies the QP, the probability that the generated code amount after quantization will be slightly larger than the target code amount increases, but after that, the coefficient cutoff process is performed to reduce the code amount to the target code amount. Can be controlled.

As described above, according to the present embodiment, it is possible to control the code amount to be closer to the target code amount. In the first embodiment, when the code amount is less than the target code amount in the coarsely accurate rate control that quantizes the entire frame, the code amount cannot be brought closer to the target code amount. On the other hand, in the present embodiment, the probability of falling below the target code amount can be reduced at this stage by modifying the QP in the coarsely accurate rate control that quantizes the entire frame to a small value. Therefore, it is possible to get closer to the target code amount by the operation combined with the detailed rate control by the rate reduction at the coefficient truncation.

(Fifth Embodiment)
Next, a fifth embodiment of the present invention will be described with reference to the drawings. The configuration and operation of this embodiment are the same as those in FIGS. 7 and 10, respectively, and the operation of the censoring control means 25 is different. The present embodiment is different from the first to third embodiments in that different censoring parameters are set for each block when the QP for each block is different.

For example, in a block having a small QP, the threshold value of the order of censoring is increased to make it difficult for coefficient censoring to occur. On the contrary, in the block having a large QP, the parameter is set so that the coefficient can be easily terminated.

For example, when the quantization step obtained from the QP distribution of the entire frame is QSave and the quantization step corresponding to the QP of a certain block is QScur, the threshold value of the order for performing coefficient truncation is as follows according to the value of QScur / QSave. Adjust as follows. Under the following conditions, 0.83 is a value corresponding to 1 / 1.2, and 0.67 is a value corresponding to 1 / 1.5.
If 0.67> QScur / QSave, the threshold order is set to -2.
If 0.83> QScur / QSave ≥ 0.67, the threshold order is set to -1.
If 1.2 ≧ QScur / QSave ≧ 0.83, the threshold order is not changed.
If 1.5 ≧ QScur / QSave> 1.2, the threshold order is set to +1.
If QScur / QSave> 1.5, the threshold order is set to +2.

The method of setting the censoring parameter is not limited to the above, and for example, when the QP of the block is in the range of a specific small value, the censoring may not be performed.

As described above, in the present embodiment, the image quality of the coding result can be improved. This is because, according to the present embodiment, when different QPs are set for each block in order to improve subjective image quality such as adaptive quantization control, the difference in QP value is reflected even in the coefficient truncation, so that the subjectivity is subjective. This is because rate control that matches the purpose of improving image quality is possible.

(Sixth Embodiment)
〔composition〕
Next, the moving image coding apparatus according to the sixth embodiment of the present invention will be described with reference to the drawings. The configuration of the moving image coding device of this embodiment is the same as that of the first embodiment (FIG. 1), but the functions of the quantization coefficient setting means 12 are different.

FIG. 11 is a block diagram showing the configuration of the quantization coefficient setting means 12-6 included in the moving image coding apparatus of the present embodiment. FIG. 12 is a conceptual diagram showing an example of signal flow in the quantization coefficient setting means 12-6. In the quantization coefficient setting means 12-2 (FIG. 8) of the second embodiment, the quantization coefficient setting means 12-6 is a second quantization coefficient memory 28 (additional quantization coefficient memory) after the coefficient truncation means 26. Also called) has an additional configuration. Since the configuration of the quantization coefficient setting means 12-6 other than the second quantization coefficient memory 28 is the same as that of the quantization coefficient setting means 12-2 of the second embodiment, detailed description thereof will be omitted.

In the present embodiment, the coefficient truncation means 26 stores the processed quantization coefficient in the second quantization coefficient memory 28.

The coding means 13 and the inverse quantization means 14 in the subsequent stage of the quantization coefficient setting means 12-6 acquire the quantization coefficient stored in the second quantization coefficient memory 28.

The above is a description of the configuration of the moving image coding device of the present embodiment.

〔motion〕
Next, the operation of the moving image coding apparatus of this embodiment will be described with reference to the drawings.

FIG. 13 is a flowchart showing the operation of data conversion in the present embodiment. In the following description of the flowchart of FIG. 13, the components of the moving image coding device are described as the moving main body of each step, but the overall moving main body is the moving image coding device. In the following, detailed description of the same operation as in the first to fifth embodiments will be omitted, and different operations will be described.

The flowchart of the present embodiment (FIG. 13) has two differences from the flowchart of the third embodiment (FIG. 10). One point is that the quantization coefficient output in step S609 is stored for one frame. Another point is that the variable length coding process (step S614) is executed after the coefficient truncation process (step S610), the inverse quantization process (step S611), and the inverse transformation process (step S612) are completed for all the blocks. Is. That is, in the flowchart of FIG. 13, the variable length coding process (step S311) in the flowchart of FIG. 10 is performed after step S314, and the variable length coding process (step S614-S615) is performed for all the blocks.

In addition, steps S301 to S310 of FIG. 10 correspond to steps S601 to S611 of FIG. 13, and steps S321 to S314 of FIG. 10 correspond to steps S610 to S613 of FIG.

The differences between the flowchart of FIG. 10 and the flowchart of FIG. 13 are as follows.

That is, after the coefficient truncation processing (step S610), the inverse quantization processing (step S611), and the inverse transformation processing (step S612) of all the blocks are completed, the coding means 13 encodes the variable length for any of the blocks. The process is performed (step S614).

Then, when the variable-length coding process is completed for all the blocks (Yes in step S615), the process according to the flowchart of FIG. 13 is terminated. On the other hand, when the variable length coding process is not completed for all the blocks (No in step S615), the process returns to step S614, and the coding means 13 performs the variable length coding process for another block.

As described above, in the present embodiment, the processing of the processing group 1 (steps S602 to S604) and the processing of the processing group 2 (steps S608 to S613) can be independently processed in block units. Therefore, although the flowchart of FIG. 13 shows an example of sequentially processing in block units, it is also possible to process a plurality of blocks in parallel.

Further, the processing group 1, the processing group 2, and the processing group 3 (steps S614 to S615) input and output data from the quantization coefficient memory 23 and the second quantization coefficient memory 28 for processing. Therefore, each processing group can be configured to be processed by a different processor or arithmetic circuit. For example, a configuration can be applied in which the processing group 1 and the processing group 2 are processed by a manycore processor or GPU which is a processor suitable for parallel processing, and the processing group 3 where parallel processing is difficult is processed by a general-purpose processor. The processing configuration is not limited to the above. For example, the processing group 1 which is a relatively fixed processing is processed by a GPU for the processing group 2 which may change the processing contents such as a specially designed arithmetic circuit and the update of the control contents, and the processing group 3 is processed by a general-purpose processor. , Etc. can also be applied.

As described above, in the present embodiment, higher speed processing is possible by configuring the processing with a processor or the like suitable for parallel processing.

In the above, the embodiment of the present invention is described in H.I. 264 and H. Although the description has been made according to a coding method such as 265, the method of the embodiment of the present invention is not limited to the application to these coding methods. For example, the method of the embodiment of the present invention is described in H.I. 264 and H. It is also possible to apply it to a coding method such as VC-1 which is different from 265, or a coding method which is not included in the international standard moving image coding method or the like. Further, regarding the rate control used in the embodiment of the present invention, the operation to which the TM-5 method is applied has been described in detail with specific examples, but it can be applied to various methods other than the illustrated operation. can.

(hardware)
Here, the hardware 90 for realizing the moving image coding apparatus according to the embodiment of the present invention will be described with reference to FIG. The hardware 90 is an example for realizing the moving image coding device of the present embodiment, and does not limit the scope of the present invention.

As shown in FIG. 14, the hardware 90 includes a processor 91, a main storage device 92, an auxiliary storage device 93, an input / output interface 95, and a network adapter 96. The processor 91, the main storage device 92, the auxiliary storage device 93, the input / output interface 95, and the network adapter 96 are connected to each other via the bus 99. Further, the processor 91, the main storage device 92, the auxiliary storage device 93, and the input / output interface 95 are connected to a network such as an intranet or the Internet via a network adapter 96. Each of the components of the hardware 90 may be single or may be plural.

The processor 91 is an arithmetic unit that expands the program stored in the auxiliary storage device 93 or the like into the main storage device 92 and executes the expanded program. In the present embodiment, the software program installed in the hardware 90 may be used. The processor 91 executes various arithmetic processes and control processes. For example, the processor 91 is realized by a CPU (Central Processing Unit). Further, the GPU (Graphics Processing Unit) may be made to bear a part of the functions of the processor 91. For example, as shown in the hardware 90-2 of FIG. 15, the configuration can include the CPU 911 and the GPU 912.

The main storage device 92 has an area in which the program is developed. The main storage device 92 may be, for example, a volatile memory such as a DRAM (Dynamic Random Access Memory). Further, a non-volatile memory such as an MRAM (Magnetoresistive Random Access Memory) may be configured and added as the main storage device 92.

The auxiliary storage device 93 is a storage device for storing various data. The auxiliary storage device 93 is configured as a local disk such as a hard disk or a flash memory.

The input / output interface 95 is an interface (I / F: Interface) that connects the hardware 90 and peripheral devices based on the connection standard.

If necessary, input devices such as keyboards, mice, and touch panels, and output devices such as displays and printing devices may be connected to the hardware 90. These input devices and output devices can be used for inputting information and settings, outputting moving image information, and the like. Data transfer between the processor 91 and the input device may be mediated by the input / output interface 95.

The network adapter 96 is an interface for connecting to a network based on a standard or a specification. The input / output interface 95 and the network adapter 96 may be shared as an interface for connecting to an external device.

Further, the hardware 90 may be provided with a reader / writer, if necessary. The reader / writer is connected to the bus 99. The reader / writer mediates between the processor 91 and a recording medium (program recording medium) (not shown), reading a data program from the recording medium, writing the processing result of the hardware 90 to the recording medium, and the like. The recording medium can be realized by, for example, a semiconductor recording medium such as an SD (Secure Digital) card or a USB (Universal Serial Bus) memory. Further, the recording medium may be realized by a magnetic recording medium such as a flexible disk, an optical recording medium such as a CD (Compact Disc) or a DVD (Digital Versatile Disc), or another recording medium.

It is preferable that the moving image coding apparatus according to the embodiment of the present invention can execute parallel processing. FIG. 16 is a configuration example of the GPU 912 that executes parallel processing by the moving image coding apparatus according to the present embodiment.

The GPU 912 of FIG. 16 includes an interface 921, a cache 922, and a plurality of streaming multiprocessors 923 (hereinafter, SM: Streaming Multiprocessor). The configuration of FIG. 16 is a conceptual configuration example, and the actual GPU 912 includes components other than the interface 921, the cache 922, and the SM923.

The interface 921 is a connection unit for exchanging data with a main storage device 92, a CPU 911, or the like realized by a DRAM or the like.

The cache 922 is a high-speed memory shared between SMs.

The SM923 is a calculation unit for processing the moving image coding process. For example, the SM923 includes a plurality of cores (arithmetic units) called a streaming processor, shared memory, and memory such as registers.

The above is an example of hardware for realizing the moving image coding device according to the embodiment of the present invention. The components of each embodiment of the present invention can be realized as a circuit including at least one of the components included in the hardware of FIGS. 14 to 16. In addition, the components of each embodiment of the present invention can be realized as software that operates on a computer having the hardware configurations shown in FIGS. 14 to 16.

The hardware configurations shown in FIGS. 14 to 16 are examples of hardware configurations for enabling the moving image coding apparatus of the present embodiment, and do not limit the scope of the present invention. Further, in each embodiment of the present invention, an example of speeding up by parallel processing using a manycore or GPU has been described, but a plurality of CPUs or PC clusters may be used (PC: Personal Computer). Further, the processing according to each embodiment of the present invention may be realized by using an FPGA (Field-Programmable Gate Array), a dedicated LSI (Large-Scale Integration) circuit, or the like.

Further, the scope of the present invention also includes a program for causing a computer to execute the processing by the moving image coding apparatus of the present embodiment. Further, a program recording medium on which a program according to an embodiment of the present invention is recorded is also included in the scope of the present invention.

(Related technology)
Here, the related technology of the moving image coding apparatus according to each embodiment of the present invention will be described.

[Video coding device]
FIG. 17 is an example of the configuration of the moving image coding device 1000 using the intra-frame prediction and the inter-frame prediction. FIG. 18 is a conceptual diagram showing an example of a signal flow in the moving image coding device 1000.

The moving image coding device 1000 includes a prediction means 100, a subtraction / conversion means 101, a quantization means 102, an inverse quantization means 103, an inverse conversion / addition means 104, a filter 105, a coding means 106, a rate control means 107, and a frame. A buffer 108 is provided.

The prediction means 100 predicts by performing prediction processing within or between frames using the coded image (image data of the frame encoded in the past) and the input image stored in the frame buffer 108. Generate an image. The prediction means 100 outputs the generated prediction image to the subtraction / conversion means 101 and the inverse conversion / addition means 104.

The subtraction / conversion means 101 subtracts the prediction image input from the prediction means 100 from the input image to create a prediction error image. The subtraction / conversion means 101 performs an orthogonal transform process similar to DCT (Discrete Cosine Transform) on the created prediction error image, and outputs a conversion coefficient. For example, H. In the 264 method, orthogonal transformation processing is performed in block units of 4 × 4 pixels or 8 × 8 pixels. In addition, H. In the 265 method, orthogonal conversion processing is performed in block units of a predetermined size from 4 × 4 pixels to 32 × 32 pixels.

The quantization means 102 performs a quantization process using the conversion coefficient input from the subtraction / conversion means 101 and the quantization control parameter input from the rate control means 107, and the quantized conversion coefficient (hereinafter referred to as “)”. Quantization coefficient) is generated. The quantization means 102 outputs the generated quantization coefficient to the inverse quantization means 103 and the coding means 106.

The dequantization means 103 inputs a quantization coefficient from the quantization means 102, and dequantizes the input quantization coefficient. The inverse quantization means 103 outputs the inverse quantization coefficient to the inverse transformation / addition means 104.

The inverse conversion / addition means 104 inputs the quantization coefficient inversely quantized by the inverse quantization means 103 and the prediction image generated by the prediction means 100. The inverse transformation / addition means 104 generates a reconstructed image by adding the inverse quantized quantization coefficient and the predicted image. The inverse transformation / addition means 104 outputs the generated reconstructed image to the filter 105.

The filter 105 inputs the reconstructed image generated by the inverse transformation / addition means 104, and filters the reconstructed image. The filter 105 outputs the filtered reconstructed image to the frame buffer 108. The filtering process is a process of reducing the distortion generated in the image due to the coding. For example, H. 264 method and H. In the 265 method, a deblock filter is used. In addition, H. In the 265 method, SAO (Sample Adaptive Offset) is used.

The frame buffer 108 stores the reconstructed image after the filter processing output from the filter 105. The filtered reconstructed image stored in the frame buffer 108 is used as an encoded image in the coding of subsequent frames.

The coding means 106 inputs the quantization coefficient from the quantization means 102. The coding means 106 entropy-codes the quantization coefficient according to a predetermined rule, and outputs a bit stream of the coding result. For example, H. 264 method and H. When applying the 265 method, the coding means 106 entropy-codes the quantization coefficient using Context-based Adaptive Binary Arithmetic Coding (CABAC). In addition, H. When applying the 264 method, the coding means 106 entropy-codes the quantization coefficient using Context-based Adaptive variable-length coding (CAVLC).

The rate control means 107 determines the quantization control parameter (QP: Quantization Parameter) based on the code amount of the coding result, the frame separately set, and the target code amount for each image block. The rate control means 107 controls the quantization means 102 by outputting a QP to the quantization means 102. The rate control means 107 performs rate control processing for achieving high image quality while suppressing the output code amount to a predetermined bit rate or less.

[QP control]
FIG. 19 is a block diagram showing a configuration of a moving image coding device that performs rate control by the TM-5 method. FIG. 20 is a conceptual diagram showing an example of signal flow in the configuration of FIG. In the block diagram of FIG. 19, the portion related to rate control is shown in detail in FIG. 17, and the peripheral configuration is omitted.

The quantization means 202 inputs a conversion coefficient from a subtraction / conversion means (not shown). The quantization means 202 performs the quantization processing of the conversion coefficient according to the QP input from the rate control means 107, and generates the quantization coefficient. The quantization means 202 outputs the generated quantization coefficient to the inverse quantization means 203 and the coding means 205.

The inverse quantization means 203 inputs the quantization coefficient from the quantization means 202. The dequantization means 203 dequantizes the quantization coefficient and outputs the dequantized quantization coefficient to an inverse conversion / addition means (not shown).

The coding means 205 inputs the quantization coefficient from the quantization means 202. The coding means 205 entropy-codes the quantization coefficient, outputs a bit stream of the coding result, and outputs information on the amount of generated code.

The QP control means 206 determines the QP based on the coding amount of the coding result and the target code amount for each frame or image block set separately. The QP control means 206 controls the quantization means 202 based on the determined QP.

[Coefficient censoring control]
Further, a method (coefficient censoring control) for realizing rate control by a method other than the setting of QP is disclosed in Patent Document 1 (Japanese Unexamined Patent Publication No. 2010-087771). Patent Document 1 describes a coefficient that estimates the generated code amount from the distribution of the quantization coefficient of a macroblock (MB: Macroblock) and forcibly sets the quantization coefficient to zero from the relationship between the estimated code amount and the target code amount. A technique for controlling a position and a cutoff threshold is disclosed.

FIG. 21 is a block diagram showing a configuration of a moving image coding device that performs rate control by the method (coefficient censoring control) disclosed in Patent Document 1. FIG. 22 is a conceptual diagram showing an example of signal flow in the configuration of FIG. 21. In FIG. 21, the quantization means 202, the dequantization means 203, and the coding means 205 operate in the same manner as in the configuration of FIG.

The coefficient censoring means 301 cuts off the quantization coefficient according to the censoring parameter input from the censoring control means 303.

The code amount estimation means 302 predicts the generated code amount from the distribution status of the quantization coefficient and the like.

The censoring control means 303 determines the censoring parameter based on the predicted value of the code amount and the target code amount for each frame or image block set separately.

FIG. 23 is a flowchart showing the operation of the TM-5 method and the rate control of Patent Document 1. In these methods, the operation of performing the coding process in a predetermined order such as raster scan in the image block unit obtained by dividing the frame into a predetermined size and performing the rate control in the block unit is common. For example, MPEG-2 and H.M. In 264, a block of 16 × 16 pixels called a macroblock (MB: Macroblock) is used. In addition, H. In the 265 method, blocks such as 16 × 16 pixels, 32 × 32 pixels, and 64 × 64 pixels, which are called Coding Tree Units (CTUs), are used.

The censoring control means 303 determines control parameters such as QP and quantization coefficient censoring used for quantization of the coded target block based on the relationship between the code amount actually generated in the coded block and the target code amount ( Step S1001).

A subtraction / conversion means (not shown) performs a predetermined orthogonal transformation process on the block and outputs a conversion coefficient (step S1002).

The quantization means 202 performs the quantization processing of the conversion coefficient based on the QP, and outputs the quantization coefficient (step S1003).

The coefficient truncation means 301 performs a quantization coefficient truncation process (step S1004).

The coding means 205 performs a variable-length coding process on the quantization coefficient and outputs a code (step S1005).

The inverse quantization means 203 performs an inverse quantization process of the quantization coefficient (step S1006).

An inverse transformation / addition means (not shown) performs an inverse transformation process to complete the process for one block (step S1007).

The moving image coding device determines whether or not the processing for all the blocks in the frame is completed (step S1008). In step S1008, if there is a block for which the coding process has not been completed (No in step S1008), the process returns to step S1001. On the other hand, in step S1008, when there is no block for which the coding process has not been completed (Yes in step S1008), the process according to the flowchart of FIG. 23 is completed.

Then, when returning to step S1001, the moving image coding apparatus determines the quantization control parameter of the next block after reflecting the code amount actually generated in the block encoded immediately before.

In the above method (coefficient censoring control), since the code amount actually output in block units is fed back to control the generated code amount of the entire frame, it is possible to precisely control the target code amount.

[R-ρ model]
Further, Patent Document 2 (Japanese Patent Laid-Open No. 2013-532439) discloses a rate control method based on a model called an R-ρ model (R: bit rate, ρ: conversion to be zero after quantization). Coefficient ratio).

Generally, in the R-ρ model, R and ρ have a nearly linear relationship, and are formulated as in Equation 1 (θ is a coefficient).
R = θ (1-ρ) ・ ・ ・ (1)
In the rate control based on the R-ρ model, first, the QP set tentatively is used to process the prediction, conversion, and quantization of the entire frame, and the relationship between R and ρ is obtained from the occurrence status of the quantization coefficient. At this time, the coefficient θ is obtained. Next, the QP that realizes the target code amount is calculated using the R-ρ model. Then, the calculated QP is used for quantization to generate a code.

FIG. 24 is a block diagram of a moving image coding device that performs rate control based on the R-ρ model. FIG. 25 is a conceptual diagram showing an example of signal flow in the configuration of FIG. 24. The quantization means 202, the dequantization means 203, and the coding means 205 of FIG. 24 operate in the same manner as the configuration of FIG.

The code quantity model building means 501 builds an R-ρ model based on the result of the conversion process.

The QP control means 502 determines the QP so as to match the generated code amount of the frame with the target code amount by using the R-ρ model.

The conversion coefficient memory 503 stores the conversion coefficient for one frame.

FIG. 26 is a flowchart showing the operation in rate control based on the R-ρ model.

The QP control means 502 determines the QP to be applied to the entire frame (step S2001). For example, in rate control based on the R-ρ model, like the frame level control of TM-5, the code amount allocation for each frame based on the complexity estimation that differs depending on the frame coding type and the past coding history It can be determined by QP calculation based on the above. Further, in the rate control based on the R-ρ model, QP calculation or the like may be used based on the statistical property of the difference between the input image and the predicted image.

The subtraction / conversion means (not shown) performs a predetermined orthogonal transformation process on the block, outputs the conversion coefficient, and stores the conversion coefficient of the entire frame (step S2002).

The moving image coding device determines whether or not the processing relating to all the blocks in the frame is completed (step S2003). If there is a block for which processing has not been completed (No in step S2003), the process returns to step S2002. On the other hand, in step S2003, if there is no block for which processing has not been completed (Yes in step S2003), the process proceeds to step S2004.

The code quantity model construction means 501 constructs a code quantity model such as an R-ρ model and calculates model parameters based on the occurrence status of the conversion coefficient of the conversion result in all the blocks in the frame (step S2004).

The QP control means 502 uses the R-ρ model to update the QP so as to match the generated code amount of the frame with the target code amount (step S2005). For example, in the technique of Patent Document 2, a histogram of conversion coefficients is constructed, the relationship between the QP value and the non-zero coefficient ratio is calculated, and the QP is changed based on the calculated value.

The quantization means 202 performs the quantization processing of the conversion coefficient based on the changed QP, and outputs the quantization coefficient (step S2006).

The coding means 205 performs a variable-length coding process on the quantization coefficient and outputs a code (step S2007).

The inverse quantization means 203 performs an inverse quantization process of the quantization coefficient (step S2008).

An inverse transformation / addition means (not shown) performs an inverse transformation process to complete the process of one block (step S2009).

The moving image coding device determines whether or not the processing relating to all the blocks in the frame is completed (step S2010). If there is a block for which processing has not been completed (No in step S2010), the process returns to step S2006. On the other hand, when the processing is completed in all the blocks (Yes in step S2010), the processing according to the flowchart of FIG. 26 is completed.

In the method using the R-ρ model, an appropriate QP is set in consideration of the occurrence status of the conversion coefficient of the entire frame, so that stable image quality can be achieved in the frame.

In the TM-5 method and coefficient censoring control, control is performed by local feedback in block units, so the image quality is unstable due to fluctuations or vibrations of control parameters such as QP within the frame depending on the strength of the feedback. May become. This becomes remarkable when a region in which the pattern is complicated and difficult to code and a region in which the pattern is simple and easy to code coexist in one frame. In addition, the TM-5 method and the coefficient truncation control have a problem that it is difficult to speed up because the feedback control is performed in block units, so that the sequential processing is performed in block units, and multiple blocks cannot be processed in parallel. There is. Further, the method of performing the coefficient truncation process of Patent Document 1 has a problem that the control may not be stable.

In the R-ρ model, the code amount model is constructed using the conversion coefficient of the entire frame, and the code amount prediction corresponding to various QP settings is performed to correct the QP. In the R-ρ model, since the conversion coefficient is the same as the number of pixels, a large number of conversion coefficient values are analyzed, which increases the amount of processing. Further, since the R-ρ model is controlled based on the QP determination using the code amount model, there is a possibility that the difference from the target code amount when an error occurs in the code amount model becomes large.

The apparatus of Patent Document 3 (Japanese Unexamined Patent Publication No. 2011-142660) executes the quantization process twice or more. Quantization processing is a relatively heavy processing that performs operations including multiplication or division of individual conversion coefficients. Therefore, the apparatus of Patent Document 3 has a problem that the processing load of the entire apparatus increases when the quantization process is repeatedly executed.

With respect to the above related techniques, according to each embodiment of the present invention, the processing load can be reduced by configuring the first coding process to perform the quantization process and the second coding process to perform the coefficient truncation process. Both stable control can be achieved. Further, according to each embodiment of the present invention, unnecessary processing can be skipped in block units and the processing load can be further reduced based on the information on the presence / absence of the non-zero coefficient of the quantization result and the distribution status.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the structure and details of the present invention within the scope of the present invention.

This application claims priority on the basis of Japanese application Japanese Patent Application No. 2016-207469 filed on October 24, 2016, the entire disclosure of which is incorporated herein by reference.

1 Video coding device 10 Prediction means 11 Subtraction / conversion means 12 Quantization coefficient setting means 13 Coding means 14 Inverse quantization means 15 Inverse conversion / addition means 16 Filter 17 Frame buffer 21 QP control means 22 Quantization means 23 Quantum Quantization coefficient memory 24 Code amount model construction means 25 termination control means 26 Quantization censoring means 27 Quantization coefficient analysis means 28 Second quantization coefficient memory

Claims (10)

Quantization in which the target code amount for each processing target block constituting the coded target image is input, and the quantization control parameter for determining the quantization coefficient of the entire coded target image is generated based on the target code amount. Parameter control means and
The conversion coefficient generated based on the coded image is input, the quantization control parameter is input from the quantization parameter control means, and the quantization control parameter and the conversion coefficient are used for quantization. A quantization means that generates the quantization coefficient by performing processing, and
A quantization coefficient memory in which the quantization coefficient generated by the quantization means is stored, and
A code amount model construction means for acquiring the quantization coefficient stored in the quantization coefficient memory and constructing a code amount model using the quantization coefficient obtained by quantizing the entire coded target image.
The code amount model is input from the code amount model construction means, and the target code amount is input, and the cutoff parameter for cutting off the quantization coefficient is determined based on the target code amount and the code amount model. Discontinuation control means and
A quantization coefficient setting having a coefficient cutoff means for inputting the cutoff parameter from the cutoff control means, acquiring the quantization coefficient stored in the quantization coefficient memory, and cutting off the quantization coefficient according to the cutoff parameter. A coding device comprising means. A frame buffer for storing the coded image used when encoding the coded image, and a frame buffer.
Using the coded target image as an input, the coded image corresponding to the coded target image is acquired from the frame buffer, and prediction processing is performed using the coded target image and the coded image. A prediction means that generates a prediction image by
With the coded target image as an input, the predicted image corresponding to the coded target image is acquired from the prediction means, and the prediction error image generated by subtracting the predicted image from the coded target image is subjected to orthogonal conversion processing. To generate the conversion coefficient by performing the subtraction / conversion means,
An inverse quantization means that takes the quantization coefficient as an input from the quantization coefficient setting means and dequantizes the quantization coefficient,
The quantization coefficient dequantized from the dequantization means is input, and the prediction image is input from the prediction means, and the dequantized quantization coefficient and the prediction image are added. Inverse conversion / addition means to generate a reconstructed image
A filter that takes the reconstructed image generated by the inverse transformation / addition means as an input, filters the reconstructed image, and stores the filtered reconstructed image in the frame buffer.
It is provided with a coding means for inputting the quantization coefficient from the quantization coefficient setting means and outputting a bit stream as a result of entropy coding the quantization coefficient.
The quantization coefficient setting means is
The conversion coefficient is input from the subtraction / conversion means, the target code amount for each processing target block is input, the quantization control parameter is determined based on the target code amount, and the determined quantization is performed. The coding apparatus according to claim 1, wherein the quantization coefficient is generated by using the control parameter and the conversion coefficient. The quantization coefficient setting means is
The quantization coefficient is input from the quantization means, the quantization coefficient is analyzed, and it is verified whether or not the quantization coefficient that becomes non-zero after quantization exists in the processing target block, and the quantization coefficient becomes non-zero. It has a quantization coefficient analysis means for storing non-zero quantization coefficient presence / absence information regarding whether or not the quantization coefficient exists in the quantization coefficient memory in association with the quantization coefficient.
The coefficient censoring means is
If the processing target block does not have the non-zero quantization coefficient presence / absence information, the quantization coefficient is not cut off.
The coding apparatus according to claim 2, wherein when the processing target block has the non-zero quantization coefficient presence / absence information, the quantization coefficient is cut off. The quantization coefficient analysis means is
When the non-zero quantization coefficient exists in any of the processing target blocks, coefficient distribution information indicating the distribution status of the non-zero quantization coefficient in the processing target block is generated.
The coefficient censoring means is
Based on the determination result using the censoring parameter and the coefficient distribution information
If at least one of the target sub-blocks constituting the processing target block does not have the non-zero quantization coefficient, the quantization coefficient is not cut off.
The coding apparatus according to claim 3, wherein when at least one of the target sub-blocks has the quantization coefficient that is non-zero, the quantization coefficient is cut off. The quantization parameter control means is
The coding apparatus according to claim 3 or 4, wherein the quantization control parameter is modified so that the quantization step width determined by the determined quantization control parameter is reduced by a predetermined ratio. The censoring control means
The coding apparatus according to any one of claims 3 to 5, wherein when the quantization control parameter is different for each processing target block, the different termination parameter is set for each processing target block. The quantization coefficient setting means is
The coding apparatus according to any one of claims 3 to 6, further comprising an additional quantization coefficient memory for storing the quantization coefficient whose coefficient has been terminated by the coefficient termination means. The coding according to any one of claims 2 to 7, wherein at least one of the subtraction / conversion means, the quantization means, and the coefficient censoring means executes processing for a plurality of the processing target blocks in parallel. Device. Enter the target code amount for each processing target block that constitutes the coded target image,
A quantization control parameter for determining the quantization coefficient of the entire image to be encoded is generated based on the target code amount.
Enter the conversion coefficient generated based on the image to be encoded.
The quantization coefficient is generated by performing a quantization process using the quantization control parameter and the conversion coefficient.
The generated quantization coefficient is stored in the quantization coefficient memory, and is stored in the quantization coefficient memory.
A code amount model is constructed using the quantization coefficient obtained by quantizing the entire coded image stored in the quantization coefficient memory.
Based on the target code amount and the code amount model, the cutoff parameter for cutting off the quantization coefficient is determined.
Quantization coefficient A coding method for censoring the quantization coefficient stored in the memory according to the censoring parameter. Processing to input the target code amount for each processing target block that constitutes the coded target image, and
A process of generating a quantization control parameter for determining the quantization coefficient of the entire image to be encoded based on the target code amount, and a process of generating the quantization control parameter.
The process of inputting the conversion coefficient generated based on the image to be encoded, and
A process of generating the quantization coefficient by performing a quantization process using the quantization control parameter and the conversion coefficient, and a process of generating the quantization coefficient.
The process of storing the generated quantization coefficient in the quantization coefficient memory, and
A process of constructing a code amount model using the quantization coefficient obtained by quantizing the entire coded image stored in the quantization coefficient memory.
A process of determining a truncation parameter for truncating the quantization coefficient based on the target code amount and the code amount model, and
Program for executing a process of aborting the quantized coefficients stored in the quantized coefficient memory in accordance with the abort parameter into the computer.

Images