JP7450078B1 - Video encoding device and video encoding method - Google Patents

Abstract

An object of the present invention is to improve the possibility of buffer failure occurring when encoding video data. A video encoding device 1 encodes a video in block units for each image based on combinations of blocks of different sizes, and calculates cost values for each of a plurality of combination candidates obtained by arbitrarily combining the blocks. a buffer circuit 12 that stores encoded data of the image encoded in units of blocks; and a buffer circuit 12 that stores encoded data of the image encoded in block units; an offset calculation circuit 13 that calculates an offset value based on a corresponding index; and a block size determination circuit that determines a combination for encoding from among the plurality of combination candidates based on each of the cost values including the offset value. 14. [Selection diagram] Figure 1

Description

The present disclosure relates to a video encoding device and a video encoding method.

Video signals are encoded on a block-by-block basis. H. In H.264, encoding processing was performed in units of 16×16 pixels called MB (Macro Block).

On the other hand, in HEVC (High Efficiency Video Coding), encoding processing is performed using four types of block sizes called CU (Coding Unit): 64 x 64 pixels, 32 x 32 pixels, 16 x 16 pixels, and 8 x 8 pixels. It will be done. Specifically, the video signal is divided into CTUs (Coding Tree Units), and each CTU is divided into variable-sized CUs. A CTU is expressed as a combination of CUs.

As for the combination of CUs, it is necessary to select the optimal CU based on the amount of calculation during encoding, the amount of code after encoding, the image quality after encoding, and the like. Patent Documents 1 to 4 propose various CU selection methods.

Patent Document 1 discloses a method in which a variance value is calculated from the activity of an index indicating the variation in luminance value of each pixel in order from a large block size to a small block size of a CU, and a CU is selected by comparing the variance value and a threshold value. is disclosed.

In Patent Document 2, when calculating the RD (Rate Distortion) cost (the sum of the amount of generated information and the flag information), the RD cost was conventionally calculated for all block sizes, but the pre-processing unit The RD cost is calculated only for the block size determined after performing the processing operations. As a result, the number of times of cost calculation, which was conventionally required multiple times, can be reduced to one, thereby suppressing the amount of encoding calculations.

In Patent Document 3, the prediction result of the prediction process is received, the encoding cost is calculated, and the combination of the prediction mode and block size that minimizes the encoding cost is selected. For the encoding cost, a value of "D+λ·R" (D is the amount of distortion, λ is a predetermined value, and R is the amount of code) is used. If it is difficult to calculate the value, an absolute difference value or the like is used instead of D, and a tentative estimated code amount is used for R.

Furthermore, in Patent Document 3, by setting an offset value in the encoding cost, when regions with different image feature amounts coexist in a CTU, a complex region and a flat region are included in one CU. It discloses that it is possible to suppress the mixture of In this case, the offset value is indicated to be added to suppress large block sizes and increase the occurrence ratio of smaller block sizes, or to promote division into CUs. .

In Patent Document 4, in inter-picture prediction processing, cost comparison of block size candidates is performed based on the layer depth (Temporal ID) in the reference structure of the picture to be encoded and the statistical value information of the motion vector of the encoded picture. It is disclosed that the amount of calculation required for motion search can be reduced by determining an offset value and appropriately narrowing down block size candidates through cost comparison.

In Patent Document 4, by setting a negative offset value only for the 64×64 block size, a larger block size is more likely to be selected. It has been shown that it is possible to prevent the amount of code from becoming too small and to reduce the amount of code by performing motion prediction using larger blocks for pictures with large Temporal IDs. The offset value is determined using Temporal ID.

Japanese Patent Application Publication No. 2015-109586 Japanese Patent Application Publication No. 2016-187140 Japanese Patent Application Publication No. 2017-005505 JP2017-028337A

However, Patent Document 1 aims to reduce the amount of calculation in the CU selection process, and the reduction in the amount of code is insufficient. In particular, in scenes where detailed pictures are encoded, the block size becomes smaller and the number of blocks to be encoded increases, which increases the overhead of the encoded data, making it impossible to suppress the increase in the amount of code and reducing the amount of encoded data. There was a risk that a buffer failure would occur, where the storage buffer would not be able to store the supplied data.

Patent Document 2 aims to reduce the number of times of cost calculation processing for selecting a CU. Even if there is a possibility of buffer failure, the CU of the block size tentatively determined in pre-processing is maintained and processed, so it may not necessarily lead to a reduction in the amount of information, and there is a possibility of buffer failure. .

Patent Document 3 aims to suppress the coexistence of a complex area and a flat area in one CU. To achieve this purpose, by setting an offset value in cost calculation, the occurrence ratio of small block sizes is increased and division into CUs is promoted. However, when the number of CUs is large, it is not always possible to Since this may not lead to a reduction in the amount, there is a possibility that the buffer will collapse.

Patent Document 4 aims to reduce the amount of processing required for inter-screen prediction processing without reducing encoding efficiency by appropriately narrowing down block size candidates and performing motion search processing. It is said that by setting a negative offset value only for 64x64 block size when calculating the cost, a large block size will be more likely to be selected and a reduction in code amount can be expected, but the increase in code amount due to the reduction in processing amount will be suppressed. This is thought to be due to the fact that the amount of code is not reduced sufficiently (such as the amount of processing required for intra-screen prediction processing), and there is a risk of buffer failure. Further, in calculating the offset value, since it is calculated from the Temporal ID and the motion vector statistical value, complicated calculation is required.

In other words, the conventional CU selection method in video signal encoding processing does not reduce the amount of code sufficiently, so there is a possibility that buffer failure will occur in the buffer circuit that temporarily stores encoded data when outputting encoded data. The problem was that it was not small.

The present disclosure has been made in view of the above circumstances, and aims to provide a technique that can improve the possibility of buffer failure occurring during encoding of video data.

A video encoding device according to an aspect of the present disclosure encodes a video in units of blocks for each image based on a combination of blocks of different sizes, and calculates a cost value for each of a plurality of combination candidates obtained by arbitrarily combining the blocks. an encoding circuit, a buffer circuit for accumulating encoded data of the image encoded in units of blocks, and a buffer remaining amount of the buffer circuit and the number of blocks in one image or equivalent to the number of blocks in the one image. an offset calculation circuit that calculates an offset value based on an index; and a block size determination circuit that determines a combination for encoding from among the plurality of combination candidates based on each of the cost values including the offset value. Equipped with

A video encoding method according to an aspect of the present disclosure is a video encoding method performed by a video encoding device, in which an encoding circuit encodes a video in units of blocks for each image based on a combination of blocks of different sizes; A cost value for each of a plurality of combination candidates obtained by arbitrarily combining blocks is calculated, a buffer circuit accumulates coded data of the image encoded in units of blocks, and an offset calculation circuit calculates a cost value for a plurality of combination candidates in which blocks are arbitrarily combined. An offset value is calculated based on the remaining amount and the number of blocks in one image or an index corresponding to the number of blocks in the one image, and the block size determination circuit calculates the offset value based on the respective cost values including the offset value. A combination for encoding is determined from among a plurality of combination candidates.

According to the present disclosure, it is possible to provide a technique that can reduce the possibility that a buffer failure will occur when encoding video data.

1 is a diagram illustrating a configuration example of a video encoding device. FIG. 2 is a diagram illustrating a configuration example of an intra/inter prediction circuit. FIG. 6 is a diagram illustrating an example of comparison of code amounts due to differences in block size. FIG. 2 is a diagram illustrating a configuration example of a block size determination circuit. FIG. 3 is a diagram showing a processing flow of a video encoding method. FIG. 3 is a diagram illustrating a process flow for calculating an offset value. FIG. 7 is a diagram showing a processing flow of cost value comparison and block size combination selection. 8 is a diagram showing a processing image of step S301 shown in FIG. 7. FIG. 8 is a diagram showing a processing image of step S302 shown in FIG. 7. FIG. 8 is a diagram showing a processing image of step S303 shown in FIG. 7. FIG. FIG. 3 is a diagram illustrating the concept of how offset values are applied. 1 is a diagram showing an example of a video encoding device. FIG. 7 is a diagram showing simulation results of the selection ratio of each block size number before and after application of the present disclosure. FIG. 2 is a diagram showing the SN ratio before and after application of the present disclosure. FIG. 7 is a diagram showing the remaining buffer amount before and after the present disclosure is applied.

Embodiments of the present disclosure will be described below with reference to the drawings. In the description of the drawings, the same parts are denoted by the same reference numerals and explanations will be omitted.

[Summary of this disclosure]
In Patent Document 3, an offset value is set in the encoding cost in order to increase the occurrence ratio of small block sizes (increase the number of CUs). On the contrary, the present disclosure is characterized in that an offset value is set in the encoding cost in order to increase the occurrence ratio of large block sizes (reduce the number of CUs). In this regard, Patent Document 4 increases the occurrence ratio of large block sizes, but the present disclosure is characterized in that an offset value is set to prevent buffer failure in the buffer circuit that stores encoded data. There is.

Hereinafter, a video encoding device according to this embodiment will be explained. The video encoding device according to this embodiment is a video encoding device based on HEVC or a standard similar to HEVC. In this embodiment, a case where video encoding is performed based on HEVC will be described as an example. However, like the CU in HEVC, the basic unit of encoding is defined by multiple block sizes, and video encoding devices and video encoding devices that combine the multiple block sizes and perform video encoding for each picture (image). Applicable to methods.

[Configuration example of video encoding device]
FIG. 1 is a diagram showing a configuration example of a video encoding device 1 according to the present embodiment. The video encoding device 1 includes an encoding circuit 11, a buffer circuit 12, an offset calculation circuit 13, and a block size determination circuit 14.

[Encoding circuit 11]
The encoding circuit 11 is a circuit that encodes image data input to the video encoding device 1 for each picture and further for each CTU obtained by dividing the image data in units of CU blocks. The encoding circuit 11 includes a prediction residual generation section 111, a DCT (Discrete Cosine Transform) transformation section 112, a quantization section 113, a variable length encoding section 114, an inverse quantization section 115, and an inverse DCT section 116. , a decoding unit 117 , and an intra/inter prediction circuit 118 .

The prediction residual generation unit 111 generates the difference between the image data input to the video encoding device 1 and the prediction signal output from the intra/inter prediction circuit 118 as a prediction residual signal. Thereafter, the DCT transform section 112, the quantization section 113, and the variable length encoding section 114 encode the prediction residual signal and store the encoded encoded data in the buffer circuit 12.

At this time, the decoding unit 117 calculates the actual prediction based on the prediction residual signal restored by the inverse quantization unit 115 and the inverse DCT unit 116 and the prediction signal predicted in the past by the intra/inter prediction circuit 118. A decoded signal representing the video data is generated. At this time, a decoded signal for one picture is generated. Thereafter, the intra/inter prediction circuit 118 generates the next predicted signal again based on the decoded signals for one picture or multiple pictures.

Note that "prediction" refers to predicting the image frame at time t from the image frame at time _tr . The "prediction signal" is an image frame at time t predicted based on the image frame at time _tr . Time _tr is a time that is earlier than time t in the encoding order. The "prediction residual signal" is a signal of the difference between the predicted image frame at time t and the image frame at time t. The more accurate the prediction, the smaller the prediction residual signal.

The above operation is the basic encoding operation performed in HEVC. The encoding circuit 11 shown in FIG. 1 is a typical example for encoding in HEVC. The encoding circuit 11 is not limited to the configuration shown in FIG. 1, and may have any other configuration as long as it can perform HEVC encoding.

[Buffer circuit 12]
The buffer circuit 12 temporarily stores the encoded data for each CTU in the encoding circuit 11 for one picture or multiple pictures, and converts the stored encoded data for one picture or multiple pictures into a video code. This is a circuit for outputting from the conversion device 1 to the outside.

[Intra/inter prediction circuit 118 of encoding circuit 11]
The intra/inter prediction circuit 118 that constitutes the encoding circuit 11 will be explained. As described above, the intra/inter prediction circuit 118 is a circuit that generates the next prediction signal again based on the decoded signal for one picture.

Encoding in the encoding circuit 11 is performed for each CTU obtained by dividing one picture into CTUs. In HEVC, the CTU is further divided into CUs, and encoding is performed for each CU. The encoded data generated by the encoding for each CU is combined to become the encoded data of the CTU.

There are four types of CU block sizes defined by HEVC: 64 x 64 pixels, 32 x 32 pixels, 16 x 16 pixels, and 8 x 8 pixels. Hereinafter, each will be expressed as 64×64, 32×32, 16×16, and 8×8. Division of CTUs into CUs is performed based on the result of block size determination by the block size determination circuit 14.

FIG. 2 is a diagram showing a configuration example of the intra/inter prediction circuit 118. The intra/inter prediction circuit 118 includes a predicted signal candidate generation section 1181 and a predicted signal determination section 1182.

The predicted signal candidate generation unit 1181 generates predicted signal candidates for combinations of CU block sizes that can be candidates at the time of prediction by the intra/inter prediction circuit 118, and supplies them to the predicted signal determination unit 1182. At this time, the predicted signal candidate generation unit 1181 calculates cost values corresponding to the plurality of generated predicted signal candidates, and supplies the calculated cost values of each predicted signal candidate to the block size determination circuit 14.

The predicted signal determining unit 1182 selects a predicted signal candidate corresponding to the determined block size determined by the block size determining circuit 14 from among the plurality of predicted signal candidates supplied from the predicted signal candidate generating unit 1181. The predicted signal candidate is output as a predicted signal to prediction residual generation section 111 and decoding section 117.

[Cost value]
The cost value is an index/evaluation value representing the video quality, image quality, and code amount of encoded data. This is a cost related to encoding, and is also called encoding cost.

For example, as shown in Patent Document 3, the cost value is calculated by "D+λ·R" (D is the amount of distortion, λ is a predetermined value, and R is the amount of code). Instead of D, a Sum of Absolute Difference (SAD value) or a Sum of Absolute Transformed Difference (SATD value) after Hadamard transformation may be used. A tentative estimated code amount may be used for R.

The cost value is not limited to a value that directly indicates the amount of deterioration in video quality or the amount of code, as long as it corresponds to the amount of deterioration in video quality or the amount of code. Other parameters associated with them are also available.

A smaller cost value indicates that encoding can be performed with a smaller amount of code and less amount of image deterioration (higher image quality). Generally, the smaller the block size, the smaller the cost value. Therefore, when determining the block size of a CU performed by the block size determining circuit 14, a combination of block sizes is basically selected such that the cost value is small (the block size is small).

[How to select block size]
On the other hand, the larger the block size (the smaller the number of CUs), the smaller the overall code amount may be. FIG. 3 shows a comparison example of the amount of code depending on the block size (number of CUs). FIG. 3(a) is a diagram showing four 8×8 code amounts. FIG. 3(b) is a diagram showing the amount of one 16×16 code. FIG. 3(a) shows a case where one 16×16 is divided into four 8×8.

Each code amount is mainly composed of coeff and overhead (OH). Coeff is the code amount of encoded data obtained by encoding the prediction residual, and directly contributes to the image. OH is auxiliary information that does not directly contribute to the image, and is required for each CU. OH is a substantially constant amount regardless of the size of the CU.

The right side of FIG. 3 also shows a case where the coeff is reduced to, for example, 1/2 by reducing the amount of code. In this case, OH does not contribute to reducing the amount of code. Note that the code amount can be reduced by, for example, increasing the QP (Quantization Parameter) value of the step in the quantization unit 113.

Here, in general, the smaller the block size, the smaller the prediction residual, so the coeff tends to be smaller. Therefore, the total code amount is small.

However, if the code amount of the coeff is reduced to, for example, 1/2 for each, the total code amount becomes smaller in 16×16. This is because when the block size is small (the number of CUs is large), the number of OHs that do not contribute to the reduction of the code amount increases accordingly, and the overall effect of reducing the code amount becomes smaller. In this case, the larger the block size, the greater the effect of reducing the amount of code.

Therefore, when the amount of code is reduced, it can be said that the larger the block size, the smaller the overall amount of code may be. Therefore, in a situation where a buffer failure occurs in the buffer circuit 12, by selecting a larger block size, it is possible to suppress the buffer failure. Although some image quality deterioration is expected due to the selection of a large block size, it is considered to be within an acceptable range compared to image quality deterioration due to buffer failure.

Specifically, the offset calculation circuit 13 calculates an offset value that induces an increase in the occurrence ratio of larger block sizes (reduces the number of CUs). Then, the block size determination circuit 14 selects a combination of block sizes that minimizes the cost value after adding the offset.

Hereinafter, the offset calculation circuit 13 and the block size determination circuit 14 will be explained.

[Offset calculation circuit 13]
The offset calculation circuit 13 is a circuit that calculates an offset value. Specifically, the offset calculation circuit 13 guides the selection of a larger block size (smaller number of CUs) based on the number of CUs in one picture and the buffer occupancy (buffer remaining amount) of the buffer circuit 12. Calculate the offset value for each block size.

Instead of the number of CUs in one picture, the overhead ratio (OH ratio) of the amount of code in one picture may be used. This is because the OH ratio is associated with the number of CUs, as shown in FIG.

Supplementary information (examples) will be given regarding the "number of CUs in one picture" and the "overhead ratio (OH ratio) of code amount in one picture." The number of CUs and the OH ratio are calculated for each picture. For example, the number of CUs in four 8×8 code amounts is 4. The OH ratio in the four 8×8 code amounts is calculated as “data amount of four OH/{data amount of (4 coeff+4 OH) (=total code amount)}”.

Note that a specific method for calculating the offset value will be described later. This offset value is supplied to the block size determination circuit 14.

[Block size determination circuit 14]
The block size determining circuit 14 is a circuit that determines a combination of divisions from CTU to CU block size. The combination of CU block sizes is determined based on the cost value of each combination supplied from the intra/inter prediction circuit 118 and the offset value supplied from the offset calculation circuit 13.

As mentioned above, the cost value is an evaluation value that indicates the amount of code and image quality (amount of distortion, etc.) when encoding.The smaller the cost value, the lower the amount of code and the lower distortion. It is said to be efficient.

FIG. 4 is a diagram showing a configuration example of the block size determination circuit 14. The block size determination circuit 14 includes an addition section 141 and a size selection section 142.

The adding unit 141 adds the offset value from the offset calculation circuit 13 to the cost value from the intra/inter prediction circuit 118. Note that in addition to addition, four arithmetic operations such as multiplication or multiplication by a coefficient may be used.

The size selection unit 142 compares the cost values after addition of the offset value among a plurality of combinations, and selects a combination of block sizes that provides the minimum cost value after addition of the offset value. The size selection unit 142 supplies the selected combination of block sizes to the predicted signal determination unit 1182 as a determined block size. A specific method of comparing cost values and a method of selecting a combination of block sizes will be described later.

By adding an offset value to the cost value, direction is given to the selection of block size combinations. In this embodiment, a larger block size is more likely to be selected. As described above, the offset value is determined based on the number of CUs in one picture and the buffer occupation amount (buffer remaining amount) of the buffer circuit 12. If the number of CUs cannot be determined, it can be estimated from the OH ratio of the code amount for one picture.

By selecting a larger block size in this way, it is possible to suppress two points: an increase in the number of CUs in one picture, and an increase in the amount of code due to the resulting increase in the number of OHs. .

Generally, the encoding circuit 11 reduces the amount of code during encoding. Examples of code amount reduction include a method of increasing the QP value of the step in the quantization unit 113 described above, a method of adjusting the scaling list of the quantization unit 113 to cut high frequency components, and the like. However, even if these efforts are made to reduce the amount of code, the amount of code may exceed the storage capacity of the buffer circuit 12 and a buffer failure may occur in which all encoded data is not transmitted from the video encoding device 1.

On the other hand, in this embodiment, when performing encoding after intra/inter prediction, a larger block size is selected when selecting a combination of block sizes of CUs, so that the code is It becomes possible to suppress the increase in the amount, and it becomes possible to eliminate buffer failure. The video encoding device 1 according to this embodiment can be applied to a case where a buffer failure occurs even if the amount of code is reduced.

[Operation of video encoding device 1]
FIG. 5 is a diagram showing a processing flow of a video encoding method performed by the video encoding device 1.

Step S101;
First, the encoding circuit 11 encodes video data in units of blocks for each picture based on the determined block size. Then, the buffer circuit 12 temporarily stores the encoded data of each picture encoded in units of blocks.

At this time, the encoding circuit 11 generates a plurality of predicted signal candidates corresponding to a plurality of combination candidates obtained by arbitrarily combining the blocks, calculates a cost value corresponding to each of the generated plurality of predicted signal candidates, and calculates The cost value of each predicted signal candidate is supplied to the block size determination circuit 14.

Step S102;
Next, the offset calculation circuit 13 calculates, based on the number of CUs in one picture and the buffer occupancy (buffer remaining amount) of the buffer circuit 12, as described later, the larger the CU number, the larger the buffer occupancy (buffer remaining amount). The smaller the offset value is, the larger the offset value set is determined, and the determined offset value is supplied to the block size determining circuit 14.

Step S103;
Next, the block size determination circuit 14 adds an offset value to the cost value of each predicted signal candidate, compares the cost value after adding the offset value among the plurality of predicted signal candidates, and selects the predicted signal with the minimum cost value. A candidate is selected, and the selected predicted signal candidate is determined as a determined block size (block size combination for encoding) and is supplied to the encoding circuit 11.

Step S104;
Finally, the encoding circuit 11 selects a predicted signal candidate corresponding to the determined block size, uses the selected predicted signal candidate as a predicted signal, and calculates the difference between the predicted signal and the image data input to the video encoding device 1. The prediction residual signal is generated and encoded, and the encoded data is stored in the buffer circuit 12.

[Operation of offset calculation circuit 13]
FIG. 6 is a diagram showing the flow of the offset value calculation process performed by the offset calculation circuit 13. In the figure, offset 8, offset 16, offset 32, and offset 64 are offset values to be added to each cost value of block size 8×8, block size 16×16, block size 32×32, and block size 64×64, respectively. shows. Within one picture, the offset of the same block size can be the same regardless of CTU or CU.

Furthermore, four offsets (0, a, b, c) are set for each block size depending on the storage status of the buffer circuit 12 and the status of the number of CUs per picture. The sizes of the offsets (0, a, b, c) satisfy 0<a<b<c in the same block size.

According to the following algorithm, the offset calculation circuit 13 calculates an offset value for each picture from the failure status of the buffer circuit 12 and the status of the number of CUs.

Step S201;
First, the offset calculation circuit 13 determines whether a buffer failure has occurred in the buffer circuit 12.

Step S202;
Next, if a buffer failure has not occurred (remaining buffer amount≧α), the offset calculation circuit 13 determines whether there is a possibility of a buffer failure after several seconds.

Step S203;
Next, the offset calculation circuit 13 determines whether the number of CUs is large if there is a possibility that the buffer will fail after several seconds (remaining buffer amount <β).

Step S204;
The offset calculation circuit 13 sets c8 to c64 to offset 8 to offset 64 (offset c setting), respectively, when a buffer failure occurs (buffer remaining amount<α), regardless of the number of CUs.

Step S205;
The offset calculation circuit 13 calculates that if a buffer failure has not occurred (buffer remaining amount ≧ α), but there is a possibility that the buffer failure will occur after a few seconds (buffer remaining amount < β) and the number of CUs is large (OH ratio ≧ γ), Set b8 to b64 in offset 8 to offset 64, respectively (offset b setting).

Step S206;
The offset calculation circuit 13 calculates that if a buffer failure has not occurred (buffer remaining amount ≧ α), but there is a possibility that the buffer failure will occur after a few seconds (buffer remaining amount < β), and the number of CUs is small (OH ratio < γ), Set a8 to a64 in offset 8 to offset 64, respectively (offset a setting).

Step S207;
If there is no possibility that the buffer will fail after several seconds (buffer remaining amount≧β), the offset calculation circuit 13 sets each of offset 8 to offset 64 to 0 (offset 0 setting), regardless of the number of CUs. Offset 0 setting is the case where no offset is set.

Note that α, β, and γ are values determined empirically. For example, α can be set to 0 or a value close to 0 (negative values are also possible), β can be set to a value relatively close to 0, and γ can be set to about several tens of percent.

Further, the number of CUs per picture can be estimated from the ratio of the OH code amount to the overall code amount. Since there is a correlation between the number of CUs and the ratio of OH, the condition of the number of CUs can be replaced with the condition of the ratio of OH. The OH ratio is the ratio of the OH code amount to the total code amount. Since the OH code amount is almost constant regardless of the block size, it can be seen that, for example, when the OH ratio is high, the number of CUs tends to be large. From this, it is possible to infer the trend of the number of CUs. Note that if there is a parameter that has a correlation with the number of CUs in addition to the OH ratio, it can also be used.

In the flowchart shown in FIG. 6, the larger the number of CUs and the higher the possibility of buffer failure, the larger the offset value is set (0<a<b<c). The smaller the number of CUs and the lower the possibility of buffer failure, the smaller the offset value is set. That is, when the number of CUs is large (that is, when the block size is small), the offset value is set relatively large.

That is, the block size determining circuit 14 adds the offset value to the cost value and makes the offset of the smaller block size relatively larger, thereby making it easier to select a larger block size. This reduces the possibility of buffer failure.

Note that offsets a8 to a64, b8 to b64, and c8 to c64 are set to values smaller than the existing cost values. If a value larger than the existing cost value is set, the offset becomes more dominant than the cost value, exceeding the scope of fine adjustment, which is the original role of the offset.

[Operation of size selection unit 142 of block size determination circuit 14]
FIG. 7 is a diagram showing a processing flow of cost value comparison and block size combination selection performed by the size selection unit 142 of the block size determination circuit 14.

The size selection unit 142 applies the offset determined by the offset calculation circuit 13 to all cost values of the prediction signal candidates of the combination of CU block sizes generated by the prediction signal candidate generation unit 1181 of the intra/inter prediction circuit 118. The cost values after adding the offset are compared between the predicted signal candidates, and the optimal predicted signal candidate is selected.

The comparison and selection operations in the size selection unit 142 are performed in the following steps. Here, a case where the CTU is 64×64 and is divided into 16 16×16 pieces will be explained.

Step S301;
First, the size selection unit 142 determines the cost value for each of the 16 16×16 patterns in the case of a pattern divided into four 8×8 patterns and in the case of a single 16×16 pattern without division. + Compare offset values. The four 8×8 cost values are the total value of each cost value+offset. Then, the CU division pattern with the smaller value is selected. This is done for 16 16×16 pieces.

Step S302;
Next, the size selection unit 142 compares the cost value+offset between the 16 CU division patterns selected in step S301 and the 32×32 non-division pattern. The cost value of the 16 CU division patterns is the total value of the cost value of the pattern selected in step S301+the offset. Then, the CU division pattern with the smaller value is selected. This is done for four 32×32 pieces.

Step S303;
Finally, the size selection unit 142 compares the cost value+offset between the four CU division patterns selected in step S302 and the 64×64 non-division pattern. The cost value of the four CU division patterns is the total value of the cost value of the pattern selected in step S302+offset. Then, the CU division pattern with the smaller value is selected.

(Operation of step S301)
FIG. 8 is a diagram showing a processing image of step S301.

Block A is the upper left block, block B is the upper right block, block C is the lower left block, and block D is the lower right block when the upper size is divided into four parts.

Furthermore, the cost n_M is defined as the cost value of n×n in block M (=A to D) when the upper size is divided into four. An offset n is defined as an offset for a size n×n in a picture.

However, even if the cost n_M has the same name, the value will be different if the upper block is different. For example, cost 8_A is the 8x8 cost value in 16x16 block A, and offset 8 is the offset for size 8x8. However, although 8x8 in 16x16 block A exists in 32x32 block A and block B, the values of their costs 8_A are not the same. However, offset 8 is common.

As shown in FIG. 8(a), among the four 16×16 blocks, block A is divided into four 8×8 patterns and a 16×16 pattern without the blocks. , the cost values are compared. The four 8×8 cost values are the total value of cost 8_A+offset 8, cost 8_B+offset 8, cost 8_C+offset 8, and cost 8_D+offset 8. On the other hand, the cost value of 16×16 without division is cost 16_A+offset 16. Here, the cost values of both are compared and the smaller size pattern is selected.

8(b), (c), and (d), similarly, the sum of the cost values of the four 8×8 patterns in blocks B, C, and D among the four 16×16 and the cost value of the undivided 16×16 pattern, and select the smaller size pattern. This selection situation changes depending on the image situation.

(Operation of step S302)
FIG. 9 is a diagram showing a processing image of step S302.

In the operation of step S301 shown in FIG. 7, it was selected whether to use an 8×8 pattern divided into four parts or a 16x16 pattern without division. In step S302 shown in FIG. 9, it is selected whether to use the four divided 16×16 patterns as the pattern selected in step S301 or the undivided 32×32 pattern. Note that the definitions of cost n_M and offset n are the same as those shown in the explanation of FIG.

In FIG. 9(a), for block A among four 32×32 blocks, the cost values are compared between a pattern in which it is divided into four 16×16 blocks and a 32×32 pattern in which it is not divided. . The four 16×16 cost values are the total cost of the block A pattern selected in step S301, the total cost of the block B pattern selected in step S301, the total cost of the block C pattern selected in step S301, This is the total value of the total costs of the patterns of block D selected in step S301. On the other hand, the cost value of a 32×32 pattern that is not divided is cost 32_A+offset 32. Here, the cost values of both are compared and the smaller size pattern is selected.

9(b), (c), and (d), similarly, the sum of the cost values of four 16×16 patterns in blocks B, C, and D among the four 32×32 and the cost value of the undivided 32×32 pattern, and select the smaller size pattern.

(Operation of step S303)
FIG. 10 is a diagram showing a processing image of step S303.

In the operation of step S302 shown in FIG. 9, it was selected whether to use a 16x16 pattern divided into four parts or a 32x32 pattern without division. In step S303 shown in FIG. 10, it is selected whether to make each of the four divided 32×32 patterns selected in step S302 or to leave the undivided 64×64 pattern as it is. Note that the cost n_M and the offset n are the same as shown in the explanation of FIG.

In FIG. 10A, the cost values are compared between a pattern in which 64×64 is divided into four 32×32 patterns and a pattern in which 64×64 is not divided. The four 32×32 cost values are the total cost of the block A pattern selected in step S302, the total cost of the block B pattern selected in step S302, the total cost of the block C pattern selected in step S302, This is the total value of the total costs of the patterns of block D selected in step S302. On the other hand, the cost value of a 64×64 pattern that is not divided is 64+offset 64. Here, the cost values of both are compared and the smaller size pattern is selected. Note that "_M" is not attached to the cost 64 because there is no upper size.

FIG. 10(b) shows an example of the finally selected CU division pattern. In this example, in step S301, four 8x8 patterns are selected in the 64x64 block A and its 32x32 block A, and the 64x64 block A and its 32x32 block B, A 16×16 pattern is selected in C and D, and in step S302, the 64×64 block A is selected as is, and the 32×32 pattern is selected in each of the 64×64 blocks B, C, and D. , shows a case where 64×64 blocks A, B, C, and D are selected as they are in step S303. As a result, smaller CU block sizes are selected toward the upper left in 64×64.

[How to apply offset value]
FIG. 11 is a diagram showing the concept of how offset values are applied.

When comparing cost values of block sizes within 64x64, if there are multiple 8x8, 16x16, and 32x32 blocks, a common offset value is added to each block size. Furthermore, even in one picture, the same offset value is added to the same block size.

[Example]
FIG. 12 is a diagram showing an example of the video encoding device 1 according to this embodiment.

FIG. 12 illustrates a case where specific cost values and offsets in step S301 shown in FIG. 8 are set. In step S301, the cost value when the 32x32 block A, 16x16, is divided into four 8x8 blocks (blocks A to D) is compared with the cost value when the 16x16 block A is not divided. Ru.

FIG. 12A shows a case where only cost values are compared without adding offset values. FIG. 12(b) shows a comparison of cost values after addition of offset values. Note that the 8×8 cost value, cost 8_A to cost 8_D, is calculated as 100, which is common to blocks A to D, and the 16×16 cost value of block A, cost 16_A, is calculated as 410. do. Further, it is assumed that offset 8, which is an 8×8 offset, is calculated to be 5, and offset 16, which is a 16×16 offset, is calculated to be 3.

In FIG. 12(a), the total of four 8×8 cost values is 100×4=400. Comparing this with the 16×16 cost value 410, the former is smaller. Therefore, in this case, four 8x8 patterns are selected.

On the other hand, in FIG. 12(b), the total of the four 8×8 cost values+offset is (100+5)×4=420. Comparing this with 16×16 cost value+offset of 410+3=413, the latter is smaller. Therefore, in this case, a 16×16 pattern is selected.

In this way, by adding the offset value to the cost value, a larger block size is more likely to be selected. Note that the offset value is set according to the algorithm shown in FIG. 6, but the value is determined empirically. Offset values are similarly set in other steps. The larger the number of CUs and the higher the possibility of failure of the buffer circuit 12, the larger the offset value is added to the cost value, so that a larger block size is more likely to be selected.

[Simulation results and effects of embodiment]
FIG. 13 is a diagram showing simulation results of selection ratios for each block size number before and after application of the present disclosure. FIG. 13(a) shows the block size selection ratio in a conventional video encoding device to which the present disclosure is not applied. FIG. 13(b) shows the block size selection ratio in the video encoding device to which the present disclosure is applied.

Both horizontal axes indicate the number of pictures. The vertical axis indicates the ratio of the number of each block size per picture. The number of pictures corresponds to the passage of time. The higher the value on the vertical axis, the more block sizes are selected in the target video scene.

This simulation was performed for the video scene where the marathon started. In particular, the second half of the video scene P1 is a scene in which confetti is thrown at the start of a marathon. In a scene where confetti is falling, the block size tends to be small to express the detailed patterns in the scene.

In FIG. 13A to which the present disclosure is not applied, in this video scene P1, 16×16 is noticeable in addition to 32×32, and 64×64 is decreasing. In addition, 8×8 also exists to some extent. On the other hand, in FIG. 13B to which the present disclosure is applied, in the similar video scene P1, 16×16 and 8×8 are decreased, and 64×64 and 32×32 are increased.

Based on this result, in the video encoding device 1 to which the present disclosure is applied, in a scene where there are many small block sizes, when the possibility of failure of the buffer circuit 12 is estimated, a large block size is likely to be selected. I understand that. This reduces the amount of code and eliminates the failure of the buffer circuit 12.

FIG. 14 is a diagram showing the SN ratio of the image encoding device before and after the present disclosure is applied.

FIG. 14(a) is a case where the present disclosure is not applied. FIG. 14(b) shows a case where the present disclosure is applied. The horizontal axis indicates the number of pictures for each video scene, that is, the elapsed time of the video. The vertical axis shows the simulation result of the SN ratio of the encoded video. The SN ratio indicates signal versus distortion, and the lower the SN ratio, the greater the amount of deterioration in image quality.

In the case where the present disclosure is not applied, the S/N ratio is greatly degraded due to the buffer failure of the buffer circuit 12 in the video scene P2 of the second half of the marathon and the video scene P3 of the water polo, but by applying the present disclosure, the S/N ratio is greatly degraded in the video scene P2 of the second half of the marathon and the video scene P3 of the water polo. It can be seen that the SN ratio has greatly improved in scenes P2 and P3.

FIG. 15 is a diagram showing the remaining buffer capacity of the image encoding device before and after the present disclosure is applied. The simulation results of the buffer circuit situation are shown. FIG. 15(a) is a case where the present disclosure is not applied. FIG. 15(b) shows a case where the present disclosure is applied. The horizontal axis indicates the number of pictures, that is, the elapsed time of the video. The vertical axis indicates the remaining buffer capacity of the buffer circuit 12. In particular, in this example, the portion where the remaining buffer capacity is zero or less is assumed to be a situation in which the buffer circuit 12 has failed.

If the present disclosure is not applied, as shown in FIG. 15(a), in the second half of the video scene P4, the remaining buffer capacity becomes zero or less, and the buffer is broken. On the other hand, when the present disclosure is applied, as shown in FIG. 15(b), in the video scene P4 in which the buffer is corrupted in FIG. There is.

[Means, actions, and effects of this embodiment]
The video encoding device 1 according to the present disclosure calculates an offset value according to the accumulation status of encoded data in the buffer circuit 12 and the status of the number of CUs in one picture, and uses the calculated offset value as a combination of each block size of the CU. These are added to the cost values of the candidates, and the combination of block sizes that minimizes the cost value after adding the offset value is selected.

For the offset value, the larger the number of CUs and the higher the possibility of buffer failure, the larger the offset value is set, and the smaller the number of CUs and the lower the possibility of buffer failure, the smaller the offset value is set. . That is, when the number of CUs is large (that is, when the block size is small), the offset value is set relatively large.

Basically, the combination of block sizes is determined so that the cost value becomes smaller (the block size becomes smaller), but the above offset value is added to the cost value and the offset of the one with the smaller block size is made relatively larger. Therefore, larger block sizes are more likely to be selected.

As a result, when there is a high possibility of buffer failure in the buffer circuit 12, a larger block size can be easily selected, thereby reducing the overall code amount and suppressing the buffer failure in the buffer circuit 12.

[Effects of this disclosure]
According to the present disclosure, the video encoding device 1 encodes a video in units of blocks for each image based on combinations of blocks of different sizes, and calculates each cost value of a plurality of combination candidates arbitrarily combining the blocks. a buffer circuit 12 that stores encoded data of the image encoded in block units, and a buffer remaining amount of the buffer circuit and the number of blocks in one image or the blocks in the one image. an offset calculation circuit 13 that calculates an offset value based on an index corresponding to a number; and a block size that determines a combination for encoding from among the plurality of combination candidates based on each of the cost values including the offset value. Since the determination circuit 14 is provided, the possibility of buffer failure occurring during encoding of video data can be reduced.

According to the present disclosure, the index corresponding to the number of blocks in one image is determined based on the total amount of encoded data and the total overhead data amount of all encoded data in the one image. Since this is the ratio of the total overhead data amount, it is possible to reduce the possibility that a buffer failure will occur when encoding video data.

According to the present disclosure, the offset value is a first value when the buffer remaining amount of the buffer circuit is less than the first threshold, and the offset value is the first value when the buffer remaining amount of the buffer circuit is equal to or greater than the first threshold. 2 and the number of blocks in the one image is large, the second value is set, and the remaining buffer capacity of the buffer circuit is equal to or greater than the first threshold and less than the second threshold, and the When the number of blocks in one image is small, it is a third value, and when the buffer remaining amount of the buffer circuit is equal to or more than the second threshold, it is a zero value, and the first value is the second value. The second value is a value larger than the third value, and the third value is a value larger than the zero, so that encoding of video data is performed. This can reduce the possibility of buffer failure occurring at times.

According to the present disclosure, since the cost value is a value related to at least one of the code amount and video quality, it is possible to reduce the possibility that a buffer failure will occur during encoding of video data.

According to the present disclosure, the cost value is a value related to at least one of the amount of code and the amount of video distortion, and the block size determining circuit selects a combination candidate with the smallest cost value including the offset value. Since the combination for encoding is determined, the possibility of buffer failure occurring during encoding of video data can be reduced.

1 Video encoding device 11 Encoding circuit 12 Buffer circuit 13 Offset calculation circuit 14 Block size determination circuit 111 Prediction residual generation section 112 DCT transformation section 113 Quantization section 114 Variable length encoding section 115 Inverse quantization section 116 Inverse DCT section 117 Decoding unit 118 Intra/inter prediction circuit 1181 Prediction signal candidate generation unit 1182 Prediction signal determination unit 141 Addition unit 142 Size selection unit

Claims (6)

an encoding circuit that encodes a video image by block based on a combination of blocks of different sizes, and calculates a cost value for each of a plurality of combination candidates obtained by arbitrarily combining the blocks;
a buffer circuit that stores encoded data of the image encoded in units of blocks;
an offset calculation circuit that calculates an offset value based on the remaining buffer amount of the buffer circuit and the number of blocks in one image or an index corresponding to the number of blocks in the one image;
a block size determining circuit that determines a combination for encoding from among the plurality of combination candidates based on each of the cost values including the offset value;
A video encoding device comprising: The index corresponding to the number of blocks in one image is
2. The video encoding according to claim 1, wherein the ratio is the amount of total overhead of the entire encoded data to the amount of total encoded data and the total overhead of the entire encoded data in the one image. Device. The offset value is a first value when the remaining buffer amount of the buffer circuit is less than a first threshold, and the offset value is a first value when the remaining buffer amount of the buffer circuit is greater than or equal to the first threshold and less than a second threshold, and , if the number of blocks in the one image is large, the second value is the buffer remaining capacity of the buffer circuit is greater than or equal to the first threshold and less than the second threshold, and the number of blocks in the one image is a second value; is a third value when the buffer capacity of the buffer circuit is small, and is a zero value when the remaining buffer capacity of the buffer circuit is equal to or greater than the second threshold value,
the first value is larger than the second value,
The second value is larger than the third value,
The video encoding device according to claim 1, wherein the third value is a value larger than the zero. The cost value is
The video encoding device according to claim 1, wherein the value is related to at least one of code amount and video quality. The cost value is a value related to at least one of the amount of code and the amount of distortion of the video,
The block size determining circuit is
The video encoding apparatus according to claim 1, wherein a candidate combination having the smallest cost value including the offset value is determined as the combination for encoding. In a video encoding method performed by a video encoding device,
an encoding circuit encodes the video in block units for each image based on combinations of blocks of different sizes, calculates each cost value of a plurality of combination candidates arbitrarily combining the blocks,
a buffer circuit accumulates encoded data of the image encoded in units of blocks;
an offset calculation circuit calculates an offset value based on the remaining buffer amount of the buffer circuit and the number of blocks in one image or an index corresponding to the number of blocks in the one image,
a block size determining circuit determines a combination for encoding from among the plurality of combination candidates based on each of the cost values including the offset value;
Video encoding method.

Images