JP6470191B2 - Video encoding method, video encoding apparatus, and video encoding program - Google Patents

Description

  The present invention relates to a video encoding method, a video encoding device, and a video encoding program that appropriately calculate the number of parallel processes in parallel processing of video encoding.

  As a method of encoding video at high speed, there is a method of performing encoding processing of a plurality of pictures in parallel. FIG. 4 is a diagram showing the relationship (hereinafter referred to as “reference structure”) of each picture with other pictures referenced at the time of decoding in the order of display. A picture is one frame (or one screen) of an encoding target video. This figure shows a reference structure called M = 3 in which one bidirectional prediction picture (hereinafter referred to as “P picture”) has two bidirectional prediction pictures (hereinafter referred to as “B pictures”). It is a thing. An arrow drawn above or below each picture represents a destination picture to which each picture refers.

In this figure, the encoding order is the order of I ₁ → P ₁ → B ₁ → B ₂ → P ₂ → B ₃ → B _{4 as} indicated by numbers below each picture. However, according to the reference structure, when encoding of P ₁ is completed, not only B ₁ but also B ₂ and P ₂ can be encoded. Therefore, in a system that can perform encoding processing in parallel, these three pictures can be encoded simultaneously.

  Next, HEVC and H.C. An apparatus configuration of an encoding unit in a general video encoding standard such as H.264 will be described. FIG. 5 is a block diagram illustrating a device configuration of an encoding unit in a general video encoding standard. The apparatus shown in FIG. 5 includes an original image buffer 1, a decoded image buffer 10, and an encoding unit 12.

  The encoding unit 12 includes a subtracter 2, an adder 3, a DCT (discrete cosine transform) calculator ("DCT" in the figure) 4, a quantizer ("Q" in the figure) 5, and an inverse quantizer (in the figure). “IQ”) 6, an IDCT (Inverse Discrete Cosine Transform) calculator (“IDCT” in the figure) 7, a prediction mode selection unit 8, a loop filter 9 and an entropy coding unit 11.

  The original image buffer 1 holds the original images input in the display order, rearranges them in the encoding order, and outputs them in order. The subtractor 2 calculates a prediction residual signal by taking the difference between the original image sent from the original image buffer 1 and the predicted image sent from the prediction mode selection unit 8 and outputs the prediction residual signal to the DCT calculator 4.

  The adder 3 calculates the sum of the prediction image output from the prediction mode selection unit 8 and the quantized prediction residual signal sent from the IDCT calculator 7, and sends the sum to the loop filter 9 as a decoded image before filtering. . The DCT calculator 4 performs discrete cosine transform on the prediction residual signal and sends the calculated DCT coefficient to the quantizer 5.

  The quantizer 5 quantizes the DCT coefficient sent from the DCT calculator 4 using the quantization parameter QP given from the outside, and uses the entropy encoding unit 11 and the inverse quantizer as a quantized value of the DCT coefficient. Send to 6. The inverse quantizer 6 inversely quantizes the quantized DCT coefficient obtained by the quantizer 5 using the quantization parameter QP given from the outside, and sends it to the IDCT calculator 7 as the DCT coefficient after quantization. send.

  The IDCT calculator 7 performs inverse discrete cosine transform on the quantized DCT coefficient sent from the inverse quantizer 6 to obtain a quantized prediction residual signal. The prediction mode selection unit 8 creates and outputs a prediction image closest to the input original image from the input reference image, and sends the prediction mode information to the entropy encoder 11.

  The loop filter 9 performs a filtering process on the decoded image before the filter sent from the adder 3 and sends it to the decoded image buffer 10 as a decoded image. The decoded image buffer 10 stores the decoded image output from the loop filter 9 and outputs the decoded image to the prediction mode selection unit 8 as a reference image.

  The entropy encoding unit 11 performs variable length encoding on the quantized value of the DCT coefficient sent from the quantizer 5 and the prediction mode information sent from the prediction mode selection unit 8, and outputs the result as an encoded stream.

  Next, the processing operation of the apparatus shown in FIG. 5 will be described with reference to FIG. 6 shows HEVC or H.264 shown in FIG. 2 is a flowchart showing an operation of an encoding device in a general video encoding standard such as H.264. When the process starts, first, the original image buffer 1 rearranges the inputted pictures in the encoding order (step S21).

  Thereafter, the encoding unit 12 processes each picture in the encoding order. Each picture is divided into rectangular blocks. The prediction mode selection unit 8 determines a prediction mode for each block (step S22). The subtracter 2 acquires the difference between the predicted image corresponding to the prediction mode and the original image. The subtracter 2 outputs a prediction residual signal based on the acquired difference (step S23).

  Next, the DCT calculator 4 performs DCT (step S24) on the output prediction residual signal. The quantizer 5 performs quantization (step S25) on the DCT coefficient. The inverse quantizer 6 performs inverse quantization (step S26). The IDCT calculator 7 calculates IDCT (step S27).

  Next, the adder 3 adds the predicted image and the predicted residual signal after quantization to generate a decoded image (step S28). The loop filter 9 applies a loop filter to the generated decoded image (step S29) and stores it as a decoded image in the decoded image buffer 1 (step S30). The stored decoded image is used for subsequent prediction image generation.

  On the other hand, the entropy encoding unit 11 performs variable-length encoding on the DCT and quantized DCT coefficients that have been subjected to quantization and corresponding prediction mode information, and outputs the result as an encoded stream (step S31). .

  When the processing shown in FIG. 6 (steps S22 to S31) is performed, the quantization parameter QP needs to be given to each picture. The quantization parameter QP represents the roughness of quantization. As the quantization parameter QP is larger, the quantization is coarser and coding noise increases. On the other hand, the larger the quantization parameter QP, the higher the compression rate, and the smaller the amount of code generated. The quantization parameter QP is calculated based on a predetermined bit rate and buffer size so as to comply with the buffer model of the decoder.

  Here, the buffer model of the decoder will be described. The video decoder has a reception buffer for storing the encoded stream, and the encoded stream received from the outside is stored in the reception buffer. In decoding, the encoded stream is extracted from the reception buffer one frame at a time and decoded.

  Since the size of this reception buffer is finite, the speed at which the received encoded stream is accumulated in the reception buffer and the speed at which the encoded stream accumulated in the reception buffer is extracted from the buffer must be balanced.

  For example, when the former speed is high, the reception buffer becomes full and a "buffer overflow" (hereinafter referred to as "overflow") that prevents receiving the encoded stream occurs. Conversely, when the latter speed is high, reception occurs. A “buffer underflow” (hereinafter referred to as “underflow”) occurs in which the buffer becomes empty and decoding of the picture stops.

  Therefore, a general video encoder incorporates a mechanism called a rate control unit that appropriately controls the amount of code generated in each picture and thus the quantization parameter QP so as not to cause overflow or underflow of the reception buffer as described above. It is.

  Here, in the decoding order, the decoding time of the nth picture is expressed as t (n), and the data accumulation amount in the reception buffer immediately after decoding the nth picture is expressed as Bt_after (t (n)). The time t (n + 1) at which the next n + 1-th picture is decoded can be expressed as t (n + 1) = t (n) + 1 / FPS using the frame rate FPS [frame / second].

Therefore, in the case of a general CBR model (constant bit rate model), the data accumulation amount Bt_before (t (n + 1)) of the reception buffer immediately before decoding the (n + 1) th picture is the bit rate b [bit / second]. Using Bt_before (t (n + 1)) = Bt_after (t (n)) + b / FPS
It can be expressed.

If the generated code amount of the (n + 1) th picture is G (n + 1), the data accumulation amount Bt_after (t (n + 1)) in the reception buffer immediately after decoding the (n + 1) th picture is Bt_after (t (n + 1)) = Bt_before (t (n + 1))-G (n + 1)
It can be expressed.

  Here, if G (n + 1)> Bt_before (t (n + 1)), Bt_after (t (n + 1)) <0 and a buffer underflow occurs. On the other hand, when the reception buffer size is set to S, if G (n + 1) <Bt_before (t (n + 1)) + b / FPS-S, Bt_after (t (n + 1))> S and buffer overflow occurs. .

  For this reason, the above-described rate control unit calculates an appropriate value of the generated code amount G of each picture so that the buffer underflow or overflow does not occur in this way, and determines the QP so as to be such a generated code amount. It becomes a mechanism.

  The relationship between the generated code amount G and the quantization parameter QP. When the quantization width corresponding to a certain quantization parameter QP is expressed as Qstep (QP), the quantization width Qstep (QP) and the generated code amount at that time There is a roughly inverse relationship between G. The product of these two is called the complexity index X = G × Qstep (QP) of this picture. Since this complexity index X is approximately the same as that of the picture encoded immediately before, X can be obtained from the product of the result of encoding immediately before and used for calculating G and QP of the next picture. Many.

  Also, as a method for calculating an appropriate generated code amount G for each picture, the MPEG-5 TM5 model is often used. This sets the amount of code that can be used for GOP (Group of Pictures), which is a series of pictures starting from I, according to the complexity indices Xi, Xp, and Xb for each picture type I, P, and B. The code amount is distributed to each picture in the GOP.

  Each time one picture is encoded, an error between the allocated code amount and the actual generated code amount is fed back, and the code amount distribution of each picture is distributed to achieve the target bit rate b while suppressing buffer underflow and overflow. decide.

  Here, a processing example of the rate control unit is shown. The processing of the rate control unit is roughly divided into two parts, pre-processing and post-processing. First, preprocessing will be described. When the process starts, first, the buffer position Bt_before is initialized with a predetermined initial value. Also, the complexity indices Xi, Xp, and Xb for each picture type are initialized with predetermined constants.

  Next, an initial value of the code amount R assigned to 1 GOP is set. For example, if the number of pictures included in one GOP is N, the initial value of R is calculated as R = b × N / FPS. Also, the number of pictures of each type included in one GOP is substituted into Ni (= number of I pictures = 1), Np (= number of P pictures), and Nb (= number of B pictures), respectively.

  Next, the allocated code amount T is calculated for each picture to be encoded by the following process. When the encoding target picture is an I picture, T is calculated as T = (Xi × R) / (Xi × Ni + Xp × Np + Xb × Nb). Thereafter, the value of Ni is decremented by 1. Similarly, when the encoding target picture is a P picture, after calculating as T = (Xp × R) / (Xi × Ni + Xp × Np + Xb × Nb), the value of Np is decremented by 1. In the case of a B picture, after calculating as T = (Xb × R) / (Xi × Ni + Xp × Np + Xb × Nb), the value of Nb is decremented by 1.

For the code amount T obtained above, the encoding buffer position estimated value Bt_after is set to Bt_after = Bt_before-T
Calculate as At this time, T is clipped to a range of Bt_before + b / FPS−S ≦ T ≦ Bt_before so that Bt_after + b / FPS> S (buffer overflow) when Bt_after <0 (buffer underflow) or when the buffer size is S is set.

Next, a quantization parameter QP corresponding to the allocated code amount T is calculated. This quantization parameter QP is named target QP. As described above, the following relationship is established between the complexity index X and the generated code amount G.
X = G × Qstep (QP)

  Therefore, for example, in the case of an I picture, a QP in which the product of the code amount T and Qstep (QP) is closest to Xi may be set as the target QP. Similarly, the target QP may be the QP whose product is closest to Xp in the case of a P picture, and the QP that is closest to Xb in the case of a B picture.

  This is the preprocessing. After actually performing encoding using the target QP thus obtained, the rate control unit performs post-processing. Specifically, this processing is as follows. First, the value of the complexity index is updated from the encoding result. If the generated code amount of the encoded picture is G, and the quantization width corresponding to the target QP at that time is Qstep (target QP), the complexity index can be obtained by the product of both.

  Therefore, Xi is updated to the value of G × Qstep (target QP) if the picture type of the encoded picture is I picture, Xp if it is a P picture, and Xb if it is a B picture. Also, the value of the code amount R is updated. Specifically, a value obtained by subtracting the generated code amount G from R is newly set as the code amount R.

Next, the value of the buffer position Bt_before is updated using the generated code amount G. this is,
Bt_before = Bt_before−G + b / FPS
Is calculated as The above is the post-processing of the rate control unit.

  Thereafter, returning to the preprocessing of the rate control unit, the process of calculating the target QP for the next coded picture is repeated. When encoding for 1 GOP is completed, the value of R is updated, and the same processing is repeated for the next GOP. Specifically, for the value of R, b × N / FPS is added to the value held at that time. Further, the values of Ni, Np, and Nb are returned to the number of pictures of each picture type included in 1 GOP.

  By performing such processing, it is possible to generate a bit stream in accordance with the target bit rate while preventing the buffer from failing (see Non-Patent Document 1, for example).

  Next, an apparatus configuration of an apparatus for simultaneously encoding a plurality of pictures as described above will be described. FIG. 7 is a block diagram showing a device configuration of a device for simultaneously encoding a plurality of pictures as described above. This apparatus includes an original image buffer 1, a decoded image buffer 10, N (N is a natural number) encoding units 12-1 to N, a stream buffer 13, a parallel processing allocation unit 14, a buffer calculation unit 15, and an allocated code amount calculation. The unit 16, the QP calculation unit 17, and the complexity calculation unit 18 are configured. A rate control unit 19 is configured by the buffer calculation unit 15, the allocated code amount calculation unit 16, the QP calculation unit 17, and the complexity calculation unit 18.

  Similar to the above, the original image buffer 1 rearranges the original images in the encoding order and sends them to the encoding units 12-1 to 12 -N. However, since there are a plurality of encoding units 12-1 to 12 -N in this configuration, N encoding units 12 of N pictures in the encoding order based on the original image allocation information sent from the parallel processing allocation unit 14. -1 to N, respectively.

  Similar to the above, the decoded image buffer 10 stores the decoded images sent from the encoding units 12-1 to N, and sends them to the encoding units 12-1 to 12-N as reference images as necessary. However, since there are a plurality of encoding units 12-1 to 12 -N in this configuration, this buffer also supports simultaneous storage of a plurality of decoded images and simultaneous transmission of a plurality of reference images.

  The encoding units 12-1 to 12 -N have the same function as the encoding unit 12 surrounded by the dotted line in FIG. 5, and N units are provided in parallel in this configuration example. The stream buffer 13 stores the encoded streams sent from the encoding units 12-1 to 12 -N, and uses the data size as the generated code amount for the buffer calculation unit 15, the assigned code amount calculation unit 16, and the complexity calculation unit 8. Send to.

  The parallel processing allocation unit 14 determines which picture in the original image buffer is sent to which of N encoding units (any of the encoding units 12-1 to N) based on the input parallel processing number, It is transmitted to the original image buffer 1 as allocation information.

  When the process starts, the buffer calculation unit 15 calculates and holds an initial value Bt_before (0) of the buffer position based on the rate control setting information. Further, every time the generated code amount of each picture is obtained from the stream buffer 13, the value of Bt_before is calculated and sent to the allocated code amount calculation unit 16.

  The allocation code amount calculation unit 16 allocates code amounts T (1) to T (1) to T (1) to T (pictures) for the next N pictures based on the rate control setting and the parallel number, buffer position, and the generated code amount of each picture. N) is obtained and sent to the QP calculation unit 17. The QP calculation unit 17 calculates a target QP for N pictures based on the complexity index of each picture type sent from the complexity calculation unit 18 and the assigned code amount of each picture, and corresponding N encoding units. Send to 12-1 to N.

  The complexity calculation unit 18 calculates a complexity index for each picture type from the quantization parameter QP of each picture sent from the QP calculation unit 17 and the generated code amount of the target picture sent from the stream buffer 13, and the assigned code amount The data is sent to the calculation unit 16 and the QP calculation unit 17.

  Next, the processing operation of the apparatus shown in FIG. 7 will be described with reference to FIG. FIG. 8 is a flowchart showing processing operations performed by the apparatus for simultaneously encoding a plurality of pictures as described above. When encoding is started, the original image buffer 1 first rearranges the pictures in the encoding order (step S41), as in the flow of FIG.

  Next, based on the processing of the rate control unit 19 described above, the allocation code amount calculation unit 16 calculates the allocation code amount for the next N pictures (step S42). At this time, the allocation code amount calculation of the first picture out of N pictures may be performed as described above. However, for the second and subsequent pictures, the allocated code amount cannot be calculated with the above formula.

  For example, in order to calculate the second picture, the generated code quantity obtained from the encoding result of the first picture is used, and the buffer position Bt_before, the code quantity R, and the complexity index of the corresponding picture type are updated. This is necessary.

  Similarly, the encoding result of the first and second pictures is required for the third picture, and the generated code amount of the first to third pictures is required for the fourth picture. However, since there is no generated code amount of these pictures at this time, the allocated code amount is used as a substitute for the generated code amount.

Therefore, regarding the second picture, the complexity index is the same as that of the first picture, and the buffer position Bt_before (2) is the buffer position Bt_before (1) related to the first picture and the assigned code amount T ( From 1) Bt_before ′ (2) = Bt_before (1) −T (1) + b / FPS
It becomes.

The code amount R (2) is obtained by subtracting the code amount T (1) from the code amount R (1) at the time of calculating the first picture. R (2) = R (1) −T (1)
Is used. Using these values, the allocated code amount T (2) for the second picture may be calculated. Similarly, the complexity index for the third picture is the same as that for calculating the first picture, and the buffer position Bt_before (3) is Bt_before (3) = Bt_before (2) −T (2) + b / FPS.
= Bt_before (1)-(T (1) + T (2)) + 2 × b / FPS

The code amount R (3) is R (3) = R (2) −T (2) = R (1) − (T (1) + T (2)).
The allocated code amount T (3) may be calculated from these.

  After calculating the allocated code amounts T (1) to T (N) for N pictures by the above calculation, the QP calculating unit 17 calculates the quantization parameter QP of each picture corresponding to these allocated code amounts (step). S43). For this purpose, a QP having a quantization width closest to a value obtained by dividing the complexity index Xi, Xp, or Xb for the picture type of each picture by the allocated code amount of each picture may be obtained. For example, if the picture type of the first picture is an I picture, the quantization width can be calculated by Xi / T (1), and the quantization parameter QP closest to the quantization width becomes the target QP of the first picture. .

  Similarly, after calculating the target QP for all N pictures, the encoding units 12-1 to 12-N encode these N pictures in parallel at the same time (step S44). Specifically, this processing corresponds to the simultaneous encoding of each picture in accordance with the “core flow of encoding processing” surrounded by a broken line in the flowchart shown in FIG.

  When encoding for N pictures is completed, the generated code amounts G (1) to G (N) corresponding to each picture are obtained, and the complexity calculation unit 18 updates the complexity index based on this result (step Then, the buffer calculation unit 15 updates the buffer position (step S46). Regarding the update of the complexity index, since the complexity index of the picture is obtained by the product of the generated code amount and Qstep corresponding to the target QP, the complexity index of the picture is classified for each picture type, and an average value is obtained to obtain a new complexity of the picture type. For example, the index may be used.

On the other hand, for updating the buffer position, the buffer position Bt_before (N + 1) is obtained as follows based on the generated code amounts G (1) to G (N) for N pictures.
Bt_before (N + 1) = Bt_before (1) − (T (1) + T (2) +... T (N)) + N × b / FPS

  By performing the above process on all the pictures (step S47), it is possible to perform the encoding process in parallel with a plurality of pictures.

MPEG-2, Test Model5 (TM5), Doc.ISO / IECJTC1 / SC29 / WG11 / NO400, Test Model Editing Committee, Apr.1993

  By the way, in the above-described method of encoding a plurality of pictures at the same time, the allocation code amount of each picture cannot always be calculated based on accurate information. Specifically, in the above, the code amount T (1) can be calculated based on the exact buffer position Bt_before (1) and the code amount R (1). The buffer positions Bt_before (2) and R (2) that are the origins of are temporary values.

  In general, in video coding, a divergence occurs between the allocated code amount T and the actual generated code amount G. For this reason, even if there is no buffer failure at the time of calculation of the allocated code amount, there is a possibility that a buffer failure occurs when N pictures are actually encoded.

  In particular, it can be understood that the larger the value of the parallel number N, the greater the influence of the difference between the allocated code amount T of each picture and the actual generated code amount G, and thus the higher the possibility of buffer failure. Therefore, the simultaneous parallel processing of a plurality of pictures can shorten the processing time as the number of parallel processes increases, but there is a problem that the risk of buffer failure increases as the number of parallel processes increases.

  Note that there is a means for avoiding this buffer failure by inserting empty data (filler data) in the case of buffer overflow. On the other hand, since there is no such avoidance means for buffer underflow where the buffer position is less than 0, avoiding this buffer underflow is a big problem particularly in the multi-picture parallel coding.

  The present invention has been made in view of such circumstances, and it is possible to perform high-speed encoding by increasing the number of parallel processes as much as possible while keeping the risk of buffer underflow low. It is an object of the present invention to provide a video encoding method, a video encoding device, and a video encoding program capable of calculating the number of processes.

  One aspect of the present invention is an encoding unit that encodes a plurality of pictures up to N (N is a natural number of 2 or more) in parallel, and an allocation code amount calculation unit that calculates an allocation code amount for each of the N pictures. And a QP calculation unit that calculates a target QP that is a quantization parameter corresponding to the allocated code amount, and is a video encoding method performed by a video encoding device that encodes video, each of the N sheets An allocation code amount recalculation step for recalculating the allocation code amount based on an error of the allocation code amount for a picture, and data storage in a buffer after encoding each picture from the allocation code amount for each of N pictures A buffer position estimated value calculating step for calculating a buffer position estimated value indicating an estimated amount of the quantity, and a buffer position estimated value after the encoding of each calculated picture A comparison step for comparing the magnitude with a predetermined threshold, a number calculation step for obtaining the number of pictures that are equal to or greater than the threshold based on the result of the magnitude comparison, and a number based on the number of pictures. And a parallel number calculating step of calculating a parallel number that is the number of pictures to be encoded in parallel by the encoding unit.

  One aspect of the present invention is the video encoding method, wherein in the allocation code amount recalculation step, the allocation code amount of each picture is multiplied by one or more predetermined coefficients.

  One aspect of the present invention is the video encoding method, wherein in the allocation code amount recalculation step, a predetermined fixed value is added to the allocation code amount of each picture.

  One aspect of the present invention is the video encoding method, wherein in the allocation code amount recalculation step, a value calculated by substituting the allocation code amount of each picture into a predetermined equation based on four arithmetic operations is used. The allocated code amount based on the error is obtained.

  One aspect of the present invention is the video encoding method, wherein in the buffer position estimation value calculation step, an initial buffer position input as the buffer position of the buffer is used as a starting point for the number of pictures defined by the parallel number. The buffer position estimation value of the buffer after encoding is calculated from the allocated code amount based on the error for each picture.

  One aspect of the present invention is a video encoding device that encodes video, an encoding unit that encodes a plurality of pictures up to N (N is a natural number of 2 or more) in parallel, and each of the N images An allocation code amount calculation unit that calculates an allocation code amount for a picture, a QP calculation unit that calculates a target QP that is a quantization parameter corresponding to the allocation code amount, and an error in the allocation code amount for each of the N pictures An allocation code amount re-recalculation unit that recalculates the allocation code amount based on the above, and a buffer that indicates an estimated amount of data accumulation in a buffer after encoding each picture from the allocation code amount for each of N pictures A buffer position estimated value calculation unit for calculating a position estimated value, and a ratio between the calculated buffer position estimated value of each picture after encoding and a predetermined threshold value are compared. A comparison unit, a number calculation unit for determining the number of pictures that are equal to or greater than the threshold based on the result of the size comparison, and a picture that is encoded in parallel by the encoding unit based on the number of pictures It is a video coding apparatus provided with the parallel number calculation part which calculates the parallel number which is the number of sheets.

  One aspect of the present invention is a video encoding program for causing a computer to execute the video encoding method.

  According to the present invention, it is possible to calculate the number of parallel processes for enabling high-speed encoding by increasing the number of parallel processes as much as possible while suppressing the risk of buffer underflow. Is obtained.

It is a block diagram which shows the structure of the video coding apparatus by one Embodiment of this invention. It is a block diagram which shows the structure of the parallel number calculation part 20 shown in FIG. It is a flowchart which shows operation | movement of the video coding apparatus shown in FIG. It is the figure which showed the reference structure of the picture in display order. HEVC and H.C. 1 is a block diagram illustrating a device configuration of an encoding unit in a general video encoding standard such as H.264. HEVC and H.264 shown in FIG. 2 is a flowchart showing processing operations of an encoding unit in general video encoding standards such as H.264. It is a block diagram which shows the apparatus structure of the apparatus for encoding a some picture simultaneously. It is a flowchart which shows the processing operation which the apparatus which codes a some picture simultaneously performs.

  Hereinafter, a video encoding apparatus according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the embodiment. The apparatus shown in this figure is different from the conventional apparatus shown in FIG. 7 in that a parallel number calculation unit 20 is newly provided.

  As for the input to the parallel processing allocation unit 14, the parallel number given from the outside is given as a fixed value in the configuration shown in FIG. 7, but the parallel number output from the parallel number calculation unit 20 in the configuration shown in FIG. The processing number m is an input.

  The apparatus shown in FIG. 1 is the same as the apparatus shown in FIG. 7. The original image buffer 1, the decoded image buffer 10, N (N is a natural number) encoding units 12-1 to N, the stream buffer 13, and parallel processing allocation A unit 14, a buffer calculation unit 15, an allocated code amount calculation unit 16, a QP calculation unit 17, and a complexity calculation unit 18. A rate control unit 19 is configured by the buffer calculation unit 15, the allocated code amount calculation unit 16, the QP calculation unit 17, and the complexity calculation unit 18. And the newly provided parallel number calculation part 20 is provided. The apparatus shown in FIG. 1 has the same configuration as that shown in FIG. 7 except for the parallel number calculation unit 20, and therefore will be described briefly here.

  The original image buffer 1 rearranges the original images in the encoding order and sends them to the encoding units 12-1 to 12 -N. However, since there are a plurality of encoding units 12-1 to 12 -N in this configuration, N encoding units for m pictures in the encoding order based on the original image allocation information sent from the parallel processing allocation unit 14. Transmission is performed for any m of 12-1 to 12-N.

  The decoded image buffer 10 stores the decoded images sent from the encoding units 12-1 to 12 -N, and sends them to the encoding units 12-1 to 12 -N as reference images as necessary. However, since there are a plurality of encoding units 12-1 to 12 -N in this configuration, the decoded image buffer 10 also supports simultaneous storage of a plurality of decoded images and simultaneous transmission of a plurality of reference images.

  The encoding units 12-1 to 12-N have the same function as the encoding unit 12 shown in FIG. 5, and in this configuration example, N encoding units 12 are provided in parallel. The stream buffer 13 stores the encoded streams sent from the encoding units 12-1 to 12 -N, and uses the data size as the generated code amount for the buffer calculation unit 15, the assigned code amount calculation unit 16, and the complexity calculation unit 8. Send to.

  The parallel processing allocation unit 14 determines which picture in the original image buffer 1 is sent to which N encoding units (any of the encoding units 12-1 to N) based on the input parallel processing number m. Obtained and transmitted to the original image buffer 1 as allocation information. As a method for assigning the pictures in the original image buffer 1 to the encoding units 12-1 to 12-N, for example, m pictures may be assigned in a left-justified manner.

  When the process starts, the buffer calculation unit 15 calculates and holds an initial value Bt_before (0) of the buffer position based on the rate control setting information. Further, every time the generated code amount of each picture is obtained from the stream buffer 13, the value of Bt_before is calculated and sent to the allocated code amount calculation unit 16 and the parallel number calculation unit 20.

  The allocation code amount calculation unit 16 allocates code amounts T (1) to T (1) to T (1) to T (pictures) for the next N pictures based on the rate control setting and the parallel number given from the outside, the buffer position, and the generated code amount of each picture. N) is obtained and sent to the QP calculation unit 17 and the parallel number calculation unit 20. The QP calculation unit 17 calculates a target QP for N pictures based on the complexity index of each picture type sent from the complexity calculation unit 18 and the assigned code amount of each picture, and corresponding N encoding units. Send to 12-1 to N.

  The complexity calculation unit 18 calculates a complexity index for each picture type from the quantization parameter QP of each picture sent from the QP calculation unit 17 and the generated code amount of the target picture sent from the stream buffer 13, and the assigned code amount The data is sent to the calculation unit 16 and the QP calculation unit 17.

  The parallel number calculating unit 20 encodes the encoding unit 12-1 based on the allocated code amount of each picture output from the allocated code amount calculating unit 16, the buffer position output from the buffer calculating unit 15, and the parallel number designated from the outside. Output the number m of parallel processes in .about.N. Here, the number m of parallel processes to be output is required to satisfy N ≧ m.

  Next, the configuration of the parallel number calculation unit 20 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a block diagram showing a configuration of the parallel number calculation unit 20 shown in FIG. The parallel number calculation unit 20 includes an error-considered code amount calculation unit 21, a buffer transition estimation unit 22, a threshold determination unit 23, and a parallel processing number determination unit 24. The parallel number calculation unit 20 receives the assigned code amount of each picture, the parallel number, and the buffer position.

  The error-considered code amount calculation unit 21 receives the assigned code amount for the number of pictures specified by the input parallel number, and multiplies them by a predetermined error coefficient k = 1.2 for each picture. Is sent to the buffer transition estimation unit 22 as an error-considered code amount. Here, the error coefficient k is multiplied by 1.2, but is not limited to this value, and any value can be used.

  The buffer transition estimation unit 22 uses the initial buffer position input as the buffer position as a starting point and the encoded buffer position for the number of pictures defined by the parallel number (the amount of data stored in the reception buffer immediately after decoding the nth picture) Bt_after) is calculated from the error-considered allocated code amount for each picture and sent to the threshold value determination unit 23.

  The threshold determination unit 23 compares the buffer positions after encoding (Bt_after of each picture) for the number of pictures defined by the parallel number with a predetermined threshold Th and compares the result with the number of parallel processes. The data is sent to the determination unit 24.

  The parallel processing number determination unit 24 receives the output of the threshold determination unit 23, obtains the maximum picture number for which the threshold determination result is always greater than or equal to the threshold Th within a range equal to or less than the value defined by the parallel number, and parallelizes it. Output as the processing number m.

  Next, the operation of the video encoding device shown in FIG. 1 including the parallel number calculation unit 20 shown in FIG. 2 will be described with reference to FIG. FIG. 3 is a flowchart showing the operation of the video encoding apparatus shown in FIG.

  First, when encoding starts, as in the conventional method, the original image buffer 1 rearranges the input images in the encoding order (step S1). Subsequently, the allocated code amount calculation unit 16 calculates the allocated code amount for N pictures (N is a natural number) beyond (step S2). The calculation method is as described above.

  Next, the error-considered code amount calculation unit 21 adds an error to the calculated assigned code amounts T (1) to T (N) of each picture (step S3). Specifically, T (1) to T (N) are multiplied by an error coefficient k = 1.2, and error-considered code amounts T ′ (1) to T ′ (N) are calculated.

Next, the buffer transition estimation unit 22 estimates buffer transition (step S4). Specifically, the estimated buffer positions Bt_after (1) to Bt_after (N) at the end of encoding of each picture are obtained as follows.
Bt_after (1) = Bt_before (1) −T ′ (1)
Bt_after (2) = Bt_before (2) −T ′ (2)
= Bt_before (1)-(T '(1) + T' (2)) + b / FPS
...
Bt_after (N) = Bt_before (N) −T ′ (N)
= Bt_before (1) − (T ′ (1) + T ′ (2) +... T ′ (N)) + (N−1) × b / FPS

  Then, the threshold determination unit 23 compares Bt_after (1) to Bt_after (N) with a predetermined threshold Th (step S5). Based on the comparison result, the parallel processing number determination unit 24 finds the picture number that falls below the threshold Th the earliest in the coding order, and outputs the immediately preceding picture number as the parallel processing number m (step S6). For example, if Bt_after (1) and Bt_after (2) are above the threshold Th, and Bt_after (3) is below the threshold Th, the parallel processing count m is 2.

  After that, the QP calculation unit 17 calculates the quantization parameter QP for m (the number of parallel processing) pictures beyond (step S7). In the calculation of the quantization parameter QP at this time, the quantization parameter QP is calculated based on the values of the code amounts T (1) to T (m) before being multiplied by the error coefficient k (for example, 1.2).

  Next, when target QPs for m pictures beyond are obtained, the encoding units 12-1 to 12-N encode and process m (parallel processing number) pictures in parallel as in the conventional method. Is performed (step S8). Not all of the encoding units 12-1 to 12 -N are necessarily used for encoding m sheets, but only m encoding units are used. Then, the complexity calculation unit 18 updates the complexity index based on the obtained generated code amount (step S9).

  Further, the buffer calculation unit 15 updates the buffer position based on the obtained generated code amount (step S10). If the encoded frame remains, the process returns to the allocated code amount calculation process for the next N pictures and repeats the encoding process to the end (step S11).

  In the above description, an example in which the error coefficient k = 1.2 is multiplied as a method for adding an error to the allocated code amount, but the present invention is not limited to multiplication by the error coefficient k. For example, a fixed value may be added. Further, not only multiplication and addition, but an allocation code amount based on a value obtained by substituting the allocation code amount of each picture into an equation based on four arithmetic operations for calculating a predetermined error coefficient k may be used.

  Further, the error coefficient k when multiplying by the error coefficient k is set to a safe side by multiplying the assigned code amount by one or more predetermined coefficients, so that it should be one or more coefficients. That's fine. In particular, in the present embodiment, there is a divergence between the allocated code amount T (N) and the actual generated code amount G (N). Is multiplied by the error coefficient. Therefore, for example, encoding is performed in advance in a state where parallelization is not performed, and an allocated code amount T (N) and an actual generated code amount G (N) are calculated. It may be determined.

  For example, a method is used in which a large amount of all images are encoded in advance to obtain an average value of the allocated code amount T (N) and the actual generated code amount G (N), and the ratio is “one or more coefficients”. An error coefficient k can be set.

  Further, the error coefficient k is not necessarily a fixed value. When encoding is actually performed, the assigned code amount T (N) and the actual generated code amount G (N) of each frame are obtained sequentially, and are updated sequentially with the average value for the latest M frames. It is also possible to use an average value of values continuously accumulated from the beginning.

  As described above, when encoding a plurality of pictures at the same time, the video encoding apparatus according to the embodiment performs buffer transition estimation in consideration of an error in the allocated code amount. The video encoding apparatus sets an appropriate number of pictures to be encoded at the same time. According to this configuration, it is possible to suppress the occurrence of buffer overflow and buffer underflow at the time of decoding, and it is possible to increase the encoding speed.

  In particular, when the buffer position is lowered, it is possible to reduce the risk of buffer underflow instead of reducing the processing speed by reducing the number of parallel processes by the parallel number calculation unit 20. Conversely, when the risk of buffer underflow is low, such as when the buffer position is raised, the parallel number calculation unit 20 can increase the processing speed by increasing the parallel number. As a result, it is possible to perform high-speed encoding by increasing the number of parallel processes as much as possible while keeping the risk of buffer underflow low.

  You may make it implement | achieve all or one part of the video coding apparatus in embodiment mentioned above with a computer. In that case, a program for realizing this function may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be read into a computer system and executed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Furthermore, the “computer-readable recording medium” dynamically holds a program for a short time like a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line. In this case, a volatile memory inside a computer system serving as a server or a client in that case may be included and a program held for a certain period of time. Further, the program may be a program for realizing a part of the above-described functions, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system. It may be realized using hardware such as PLD (Programmable Logic Device) or FPGA (Field Programmable Gate Array).

  As mentioned above, although embodiment of this invention has been described with reference to drawings, the said embodiment is only the illustration of this invention, and it is clear that this invention is not limited to the said embodiment. is there. Therefore, additions, omissions, substitutions, and other modifications of the components may be made without departing from the technical idea and scope of the present invention.

  It can be applied to applications where it is essential to perform high-speed encoding by increasing the number of parallel processes as much as possible while keeping the risk of buffer underflow low.

  DESCRIPTION OF SYMBOLS 1 ... Original image buffer, 10 ... Decoded image buffer, 12-1 to N ... Encoding part, 13 ... Stream buffer, 14 ... Parallel processing allocation part, 15 ... Buffer calculation , 16 ... Allocation code amount calculation unit, 17 ... QP calculation unit, 18 ... Complexity calculation unit, 19 ... Rate control unit, 20 ... Parallel number calculation unit

Claims (7)

An encoding unit that encodes a plurality of pictures up to N (N is a natural number of 2 or more) in parallel, an allocation code amount calculation unit that calculates an allocation code amount for each of the N pictures, and the allocation code amount A video encoding method performed by a video encoding device that encodes video, comprising a QP calculation unit that calculates a target QP that is a corresponding quantization parameter,
An allocation code amount recalculation step of recalculating the allocation code amount based on an error of the allocation code amount for each of the N pictures;
A buffer position estimated value calculating step for calculating a buffer position estimated value indicating an estimated amount of data accumulation in the buffer after encoding each picture from the allocated code amount for each of N pictures;
A comparison step for comparing the calculated buffer position estimated value of each picture with a predetermined threshold value;
A number-of-sheets calculating step for determining the number of pictures that are equal to or greater than the threshold based on the result of the comparison of the magnitudes;
And a parallel number calculating step of calculating a parallel number that is the number of pictures to be encoded in parallel by the encoding unit based on the number of pictures.   2. The video encoding method according to claim 1, wherein in the allocation code amount recalculation step, the allocation code amount of each picture is multiplied by one or more predetermined coefficients.   The video encoding method according to claim 1, wherein in the allocation code amount recalculation step, a predetermined fixed value is added to the allocation code amount of each picture.   2. The allocation code amount recalculation step determines the allocation code amount based on the error by using a value calculated by substituting the allocation code amount of each picture into an equation based on predetermined four arithmetic operations. Video encoding method.   In the buffer position estimated value calculation step, the buffer position estimated value of the buffer after encoding for the number of pictures defined by the parallel number starting from the initial buffer position input as the buffer position of the buffer, The video encoding method according to claim 1, wherein calculation is performed from the allocated code amount based on the error with respect to a picture. A video encoding device for encoding video,
An encoding unit for encoding a plurality of pictures up to N (N is a natural number of 2 or more) in parallel;
An allocation code amount calculation unit for calculating an allocation code amount for each of the N pictures;
A QP calculation unit for calculating a target QP that is a quantization parameter corresponding to the allocated code amount;
An allocation code amount re-recalculation unit that recalculates the allocation code amount based on an error of the allocation code amount for each of the N pictures;
A buffer position estimated value calculating unit that calculates a buffer position estimated value indicating an estimated amount of data accumulation in a buffer after encoding each picture from the allocated code amount for each of N pictures;
A comparing unit that compares the calculated buffer position estimated value of each picture after the encoding with a predetermined threshold;
A number calculation unit for determining the number of pictures that are equal to or greater than the threshold based on the result of the magnitude comparison;
A video encoding apparatus comprising: a parallel number calculating unit that calculates a parallel number that is the number of pictures to be encoded in parallel by the encoding unit based on the number of pictures.   A video encoding program for causing a computer to execute the video encoding method according to any one of claims 1 to 5.

Images