JP5706759B2 - Image coding apparatus and image coding method - Google Patents

Description

  The present invention relates to an image encoding device and an image encoding method.

  In the encoding of video frames, in order to improve the processing speed of the video encoder, the encoding of video frames is shared by a plurality of encoders and processed in parallel. As such a technique, Patent Document 1 is disclosed.

  Patent Document 1 discloses a technique for performing control for optimizing a quantization parameter based on a bit amount generated during encoding of each video frame when encoding video frames in parallel. ing. The information processing apparatus disclosed in Patent Document 1 calculates and predicts the generated bit amount of the remaining partition from the bit amount Bi of the encoded partition in one frame and the number of unencoded partitions Ri using an equation. To do. This information processing apparatus compares the sum of the generated bit amount Bi and the calculation result by the mathematical expression with the target bit amount. The information processing apparatus performs control to optimize the quantization parameter by increasing or decreasing the value of the quantization parameter according to the comparison result.

  However, with this control method, the bit amount of the remaining partition is basically calculated using only the bit amount being encoded, so the bit amount of the remaining partition can be accurately predicted. There is no problem. Therefore, an inappropriate quantization parameter is determined based on the prediction result. As a result, there is a problem that encoding is performed with an inappropriate quantization parameter, which causes a deterioration in image quality of the video frame.

  Here, Patent Document 2 discloses a technique for the purpose of suppressing the total code amount of the entire image to a certain value or less while suppressing image quality deterioration. In the encoding method disclosed in Patent Document 2, the ratio of the code amount allocated to each frame within the same code amount control range is changed according to the absolute value sum of the difference data of the motion detection result between frames. In this encoding method, a prediction model of the generated code amount is generated from the activity indicating the feature of the image, and encoding is performed according to the prediction model. Specifically, the reference scaling factor is controlled based on the difference between the generated code amount and the prediction model.

  However, Patent Document 2 does not disclose a technique for estimating a code amount of an unencoded area of a picture that is being encoded and determining a code amount control parameter of the unencoded area. Furthermore, in both documents 1 and 2, the buffer occupation amount of the generated bitstream is not taken into consideration, and there is a possibility of causing a buffer overflow or a buffer underflow.

JP 2007-166192 A JP-A-10-313463

  As described in the background art, the technique disclosed in Patent Document 1 has a problem that the image quality of a frame is deteriorated when encoding is performed.

  An image encoding apparatus according to a first aspect of the present invention is an image encoding apparatus that encodes moving image data including a plurality of frames, and an encoding unit that encodes the frames; A code amount storage unit that stores a code amount of the encoded frame and a code amount of the encoded region included in the encoding frame that is being encoded by the encoding unit; and the code amount storage unit A code amount estimation unit that estimates a code amount of an unencoded area included in the frame being encoded based on the code amount of the encoded frame and the code amount of the frame that is being encoded, A code amount control unit that determines a code amount control parameter of the uncoded region based on a code amount of the uncoded region and a code amount that can be assigned to the uncoded region, The encoding unit includes the code An uncoded region included in a middle frame is coded based on the determined code amount control parameter, and the code amount estimation unit performs coding close in time to the frame being coded. Based on the ratio of the first code amount and the second code amount that are the code amounts of the regions corresponding to the encoded region and the unencoded region included in the encoded frame in the completed frame, The code amount of the uncoded region is estimated from the code amount of the encoded region.

  An image encoding method according to a second aspect of the present invention is an image encoding method for encoding moving image data including a plurality of frames, the encoding step for encoding the frames, and the encoding A storage step of storing in the code amount storage unit the code amount of the encoded frame and the code amount of the encoded region included in the encoding frame that is being encoded; A code amount estimating step of estimating a code amount of an unencoded area included in the frame being encoded based on a code amount of the encoded frame and a code amount of the frame being encoded; and the estimated uncoded A parameter determining step for determining a code amount control parameter for the unencoded region based on a code amount of the encoded region and a code amount that can be assigned to the uncoded region, and the code amount estimating step Then, the mark A first code amount that is a code amount of each of a region corresponding to an encoded region and an unencoded region included in the encoded frame in an encoded frame that is temporally adjacent to the encoded frame; 2, the code amount of the unencoded region is estimated from the code amount of the encoded region, and in the encoding step, the unencoded region included in the frame being encoded is The encoding is performed based on the determined code amount control parameter.

  According to the first and second aspects of the present invention described above, the code amount of the uncoded region is estimated with high accuracy with reference to the code amount distribution of the encoded frame having a high correlation with the frame being encoded. It becomes possible. Therefore, an appropriate code amount control parameter can be determined based on the code amount of the uncoded region estimated with high accuracy.

  An image encoding apparatus according to a third aspect of the present invention is an image encoding apparatus that encodes moving image data composed of a plurality of frames or still image data composed of a single frame, and encodes the encoded image data. Stores a feature amount detection unit that detects a feature amount of a frame, an encoding unit that encodes the frame, and a code amount of an encoded area included in a frame that is being encoded by the encoding unit. A code amount storage unit, a code amount estimation unit that estimates a code amount of a coding area included in the frame being encoded, based on the detected feature amount and the stored code amount, A code amount control unit that determines a code amount control parameter of the uncoded region based on the estimated code amount of the uncoded region and the code amount that can be assigned to the uncoded region; The encoding unit includes the A non-encoded area included in a frame being encoded is encoded based on the determined code amount control parameter, and the code amount estimation unit includes an encoded portion included in the frame being encoded Estimating the code amount of the unencoded region from the code amount of the encoded region based on the ratio between the feature amount of the region and the feature amount of the unencoded region included in the frame being encoded It is.

  An image encoding device according to a fourth aspect of the present invention is an image encoding device that encodes moving image data including a plurality of frames, and a feature amount detection unit that detects a feature amount of a frame to be encoded. An encoding unit that encodes the frame, a code amount storage unit that stores a code amount of an encoded region included in the encoding frame that is being encoded by the encoding unit, and the detected A code amount estimation unit that estimates a code amount of an encoded region included in the frame being encoded based on a feature amount and the stored code amount; and a code amount of the estimated unencoded region; A code amount control unit that determines a code amount control parameter of the uncoded region based on a code amount that can be assigned to the uncoded region, and the coding unit The included uncoded area The encoding is performed based on the determined code amount control parameter, and the code amount estimation unit is encoded in a frame being encoded in a frame that is temporally close to the frame being encoded Based on the ratio of the first feature value and the second feature value, which are the feature values of the region corresponding to each of the region and the uncoded region, the code of the uncoded region from the code amount of the coded region The amount is to be estimated.

  An image encoding method according to a fifth aspect of the present invention is an image encoding method for encoding moving image data consisting of a plurality of frames or still image data consisting of a single frame. A code amount storage unit that stores a feature amount detection step of detecting a feature amount of a frame, an encoding step of encoding the frame, and a code amount of an encoded area included in a frame being encoded that is a frame in the middle of encoding Based on the storage step stored in the feature amount, the feature amount detected in the feature amount detection step, and the code amount stored in the code amount storage unit. A code amount estimation step for estimating the amount, a code amount control parameter for the uncoded region based on the estimated code amount of the uncoded region and a code amount that can be assigned to the uncoded region. Para to decide And determining the feature amount of the encoded region included in the frame being encoded and the feature amount of the unencoded region included in the frame being encoded. Based on the ratio, the code amount of the unencoded region is estimated from the code amount of the encoded region, and in the encoding step, the uncoded region included in the frame being encoded is the parameter determining step. Is encoded based on the code amount control parameter determined in (1).

  According to the third to fifth aspects of the present invention described above, unencoded with reference to the feature amount distribution of the encoded frame or the encoded frame that has a high correlation with the encoded amount distribution of the encoded frame. It is possible to estimate the code amount of the region with high accuracy. Therefore, an appropriate code amount control parameter can be determined based on the code amount of the uncoded region estimated with high accuracy.

  According to each aspect of the present invention described above, it is possible to provide an image encoding device and an image encoding method capable of suppressing deterioration in image quality of a frame when encoded.

It is a block diagram of the image coding apparatus concerning Embodiment 1 of this invention. It is a figure which shows the image coding process timing by the image coding apparatus concerning Embodiment 1 of this invention. FIG. 3 is a diagram illustrating a picture encoding process status at time t in FIG. 2. It is a figure which shows the structure of a picture. It is the figure which showed typically an example of distribution of the generated code amount in 1 picture concerning Embodiment 1 of this invention. It is a figure for demonstrating the code amount estimation method of the uncoded area | region concerning Embodiment 1 of this invention. It is a flowchart which shows the process of the image coding apparatus concerning Embodiment 1 of this invention. It is a block diagram of the image coding apparatus concerning Embodiment 2 of this invention. It is a figure for demonstrating the code amount estimation method of the uncoded area | region concerning Embodiment 2 of this invention. It is a flowchart which shows the process of the image coding apparatus concerning Embodiment 2 of this invention. It is a block diagram of the image coding apparatus concerning Embodiment 3 of this invention. It is a flowchart which shows the process of the image coding apparatus concerning Embodiment 3 of this invention. It is a block diagram of the image coding apparatus concerning Embodiment 4 of this invention. It is a figure for demonstrating the code amount estimation method of an unencoded area | region when there exists a motion concerning Embodiment 4 of this invention. It is a figure for demonstrating the code amount estimation method of an unencoded area | region when there exists a motion concerning Embodiment 4 of this invention. It is a flowchart which shows the process of the image coding apparatus concerning Embodiment 4 of this invention. It is a figure for demonstrating the code amount estimation method of an unencoded area | region when there exists a motion concerning Embodiment 4 of this invention. It is a figure for demonstrating the code amount estimation method of an unencoded area | region when there exists a motion concerning Embodiment 4 of this invention. It is a block diagram of the image coding apparatus concerning Embodiment 5 of this invention. It is a figure which shows the buffer occupation amount of the virtual buffer by the side of the decoder estimated by the estimation method of the buffer occupation amount concerning Embodiment 5 of this invention. It is a block diagram of the image coding apparatus concerning Embodiment 6 of this invention. It is a figure which shows the structure of the bit stream of 1 picture in a moving image encoding standard. It is a figure which shows transition of the buffer occupancy of the estimated virtual buffer of the decoder side, and transition of the buffer occupancy of the actual virtual buffer of the decoder side. It is a block diagram of the image coding apparatus concerning Embodiment 7 of this invention. It is a figure for demonstrating the code amount estimation method of the uncoded area | region concerning other embodiment of this invention. It is a figure for demonstrating the code amount estimation method of the uncoded area | region concerning other embodiment of this invention.

Embodiment 1 of the Invention
With reference to FIG. 1, the structure of the image coding apparatus 1 concerning Embodiment 1 of this invention is demonstrated. FIG. 1 is a configuration diagram of an image encoding device 1 according to the first embodiment of the present invention.

  The image encoding device 1 includes a frame buffer 10, a generated code amount storage device 11, a generated code amount prediction device 12, a code amount control device 13, a stream combination device 14, encoders 30 to 33, and stream buffers 40 to 43. The image encoding device 1 encodes moving image data including a plurality of frames. Each of a plurality of frames constituting moving image data is sequentially input to the image encoding device 1 as an input image. Hereinafter, the frame is also referred to as “picture” or “image”.

  The image encoding device 1 is, for example, a recorder that encodes a moving image and stores it in a storage medium, a digital broadcast transmission device that encodes and transmits a moving image of a terrestrial digital broadcast program, and encodes a moving image to another television set. Included in video-phone calls that are sent to the phone. The storage medium is, for example, a memory, a hard disk, an optical disk, or the like.

  The frame buffer 10 temporarily stores an input image input to the image encoding device 1. The frame buffer 10 includes an arbitrary storage device for storing an input image. Here, the storage device is, for example, a memory and a hard disk.

  The generated code amount storage device 11 stores the generated code amount of the encoded image and the generated code amount of the encoded region of the image being encoded. The generated code amount is stored for each unit obtained by dividing one image into a predetermined size. The generated code amount storage device 11 includes an arbitrary storage device for storing the generated code amount.

  The generated code amount prediction device 12 generates an unencoded area of the image being encoded based on the generated code amount of the encoded image and the generated code amount of the encoded area of the image being encoded. Estimate the code amount.

  The code amount control device 13 calculates a code amount control parameter designated for the encoders 30 to 33 so as to achieve a bit rate designated in advance based on the estimated generated code amount of the uncoded region. The code amount control device 13 specifies the calculated code amount control parameter for the encoders 30 to 33.

  The stream combining device 14 combines the bit streams stored in the stream buffers 40 to 43 in a specified order to generate one bit stream. The stream combining device 14 outputs the generated bit stream to the outside of the image encoding device 1.

  Each of the encoders 30 to 33 reads an input image designated as an encoding target from the frame buffer 10 and encodes it. Each of the encoders 30 to 33 encodes an input image with a code amount corresponding to a code amount control parameter designated by the code amount control device 13. Each of the encoders 30 to 33 generates a bit stream including data obtained by encoding an input image. Each of the encoders 30 to 33 stores the bit stream in each of the stream buffers 40 to 43. Each of the encoders 30 to 33 is, for example, MPEG-2 or H.264. An input image is encoded by an encoding method defined in a moving image encoding standard such as H.264.

  Each of the stream buffers 40 to 43 temporarily stores a bit stream output from each of the encoders 30 to 33. Specifically, the stream buffer 40 stores the bit stream generated by the encoder 30, the stream buffer 41 stores the bit stream generated by the encoder 31, and the stream buffer 42 stores the bit stream generated by the encoder 32. The stream buffer 43 stores the bit stream generated by the encoder 33.

  Next, with reference to FIG. 2, the image encoding processing timing by the image encoding device 1 according to the first embodiment of the present invention will be described. FIG. 2 is a diagram illustrating image encoding processing timing by the image encoding device 1 according to the first embodiment of the present invention. FIG. 2 shows a case where the four encoders 30 to 33 perform input image encoding processing in parallel with time division.

  As shown in FIG. 2, in the first embodiment, each of the plurality of pictures 1 to 9 included in the moving image is sequentially assigned to each of the encoders 30 to 33 in the encoding order. Here, the encoding order of pictures 1 to 9 is assumed to be the order of picture 1, picture 2,. Further, illustration and description of a picture subsequent to the picture 9 are omitted. Specifically, pictures 1, 5, and 9 are assigned to the encoder 30, pictures 2 and 6 are assigned to the encoder 31, pictures 3 and 7 are assigned to the encoder 32, and pictures 4 and 6 are assigned to the encoder 33. 8 is assigned. Each of the encoders 30 to 33 encodes the assigned picture. In other words, the assigned picture is a picture designated as an encoding target.

  Hereinafter, FIG. 2 will be described in detail. At time t0, the encoder 30 acquires picture 1 from the frame buffer 10 and starts encoding the acquired picture 1. Hereafter, the times when the picture input interval elapses from time t0 are indicated as t1 to t8. Note that the picture input interval is arbitrarily determined in advance.

  At time t1, the encoder 31 acquires picture 2 from the frame buffer 10 and starts encoding of the acquired picture 2. At time t2, the encoder 32 acquires the picture 3 from the frame buffer 10 and starts encoding the acquired picture 3. At time t3, the encoder 33 acquires the picture 4 from the frame buffer 10 and starts encoding the acquired picture 4.

  Here, it is assumed that the time for encoding one picture with one encoder takes less than four times the picture input interval. Therefore, at time t4, encoding of picture 1 by the encoder 30 started from time t0 is completed.

  Therefore, at time t4, the encoder 30 acquires the picture 5 from the frame buffer 10 and starts encoding the acquired picture 5. Similarly, every time the picture input interval elapses, encoding of picture 6 by the encoder 31 is started at time 5, encoding of picture 7 by the encoder 32 is started at time 6, and encoder 33 is started at time 7. Encoding of picture 8 is started, and encoding of picture 9 by encoder 30 is started at time t8. In this way, the encoders 30 to 33 encode each of the pictures 1 to 9 in parallel.

  Note that the order in which pictures are assigned to the encoders 30 to 33 is not limited to the order illustrated in FIG. In the first embodiment, as illustrated in FIG. 2, pictures are cyclically assigned in the order of encoders 30, 31, 32, and 33. However, pictures are assigned to encoders 30 to 33 in other orders. You may make it allocate.

  Further, as described above, the configuration in which the picture to be encoded is assigned to each of the encoders 30 to 33 and the picture to which each of the encoders 30 to 33 is assigned may be realized in any way. As an example, the input images are input to the image encoding device 1 in the encoding order, and the input images are queued in the frame buffer 10 in the input order. Then, each of the encoders 30 to 33 sequentially takes out the input image from the frame buffer 10.

  Next, with reference to FIG. 3, the picture encoding process status at time t in FIG. 2 will be described. FIG. 3 is a diagram illustrating a picture encoding process status at time t in FIG. The time t is a time after the encoding of the picture 5 by the encoder 30 is finished and before the encoding of the picture 9 by the encoder 30 is started.

  FIG. 3 shows the encoding processing status of each of the pictures 5 to 9 at time t in FIG. In FIG. 3, a hatched area in a picture indicates an encoded area, and a plain area not hatched in a picture indicates an uncoded area. In this way, at a certain moment in the middle of encoding, the size of the encoded region and the unencoded region are different in each of the pictures 5 to 8 being encoded.

  Next, the picture configuration will be described with reference to FIG. FIG. 4 is a diagram illustrating the configuration of a picture.

  As shown in FIG. 4, one picture is divided into a plurality of macroblocks and encoded in units of macroblocks when encoding is performed. Note that H.264 of the moving image coding standard. In H.264, a picture may be encoded in units of a macroblock pair in which two macroblocks adjacent in the vertical direction are paired. Therefore, encoding may be performed in units of macroblock pairs.

  Here, in the first embodiment, it is assumed that the generated code amount for each column in the horizontal direction of the macroblock is stored in the generated code amount storage device 11. Hereinafter, a unit of one column in the horizontal direction of the macroblock is referred to as a “macroblock line”. That is, each time the encoders 30 to 33 encode a macroblock line, the generated code amount in the macroblock line is stored in the generated code amount storage device 11. Each of the encoders 30 to 33, for example, every time a macroblock is encoded, the code amount per macroblock generated by encoding is cumulatively added, and the generated code amount per macroblock line is calculated. .

  Further, the generated code amount for each horizontal column of the macroblock pair may be stored in the generated code amount storage device 11. Hereinafter, a unit of one column in the horizontal direction of the macroblock pair is referred to as a “macroblock pair line”. In this case, each of the encoders 30 to 33 stores the generated code amount in the macroblock pair line in the generated code amount storage device 11 every time the macroblock pair line is encoded. Each of the encoders 30 to 33, for example, every time a macroblock pair is encoded, the code amount per macroblock pair generated by encoding is cumulatively added, and the generated code amount per macroblock pair line, for example. Is calculated. In the present embodiment, as shown in FIG. 4, a case where there are N lines of macroblocks in one picture is illustrated. Here, N is an arbitrary positive integer.

  Next, an example of the distribution of the generated code amount in one picture will be described with reference to FIG. FIG. 5 is a diagram schematically illustrating an example of a generated code amount distribution in one picture according to the first embodiment of the present invention.

  In general, pixel values in one picture are not uniform and do not change monotonously. Therefore, the code amount in the encoded picture is not uniform and does not change monotonously. As illustrated in FIG. 5, there is a case where the amount of code generated by encoding differs greatly between the first half macroblock line and the second half macroblock line within one picture. In such an image, for example, even if the encoding has been completed up to the i-th macroblock line and the generated code amount up to the i-th macroblock line is known, the i + 1-th macroblock line to be encoded thereafter It is difficult to predict the generated code amount in the Nth macroblock line only from the generated code amount in the encoded area up to the i-th macroblock line. Here, i is a positive integer less than or equal to N.

  Next, with reference to FIG. 6, a code amount estimation method for an uncoded region according to the first embodiment of the present invention will be described. FIG. 6 is a diagram for explaining the code amount estimation method for an uncoded region according to the first embodiment of the present invention.

  In general, it is known that if an image constituting a moving image is an image close in time, the correlation is high, and the generated code amount distribution in the image is similar. Therefore, the generated code amount prediction apparatus 12 according to the first embodiment determines the generated code amount in the unencoded area of the picture being encoded, the generated code amount in the encoded area of the picture being encoded, and the immediately preceding code amount. Is estimated from the generated code amount of the encoded picture that has been encoded. Referring to FIG. 2, for example, when the generated code amount prediction device 12 encodes picture 5, the generated code amount prediction device 12 calculates the uncoded region of picture 5 from the code generation amount of the immediately previous encoded picture 1. Estimate the code amount. Hereinafter, a picture in the middle of encoding is referred to as a “picture in progress”, and a picture that has been encoded is referred to as a “coded picture”.

  For example, as illustrated in FIG. 6, it is assumed that the encoding has progressed to the range indicated by “a” in the middle of encoding. B shown in FIG. 6 is a length obtained by subtracting the vertical length a of the encoded region from the vertical length N of the entire picture being encoded. That is, N = a + b is established. Here, a is a positive integer less than or equal to N, and b is a positive integer less than or equal to N. Therefore, the range indicated by a in the picture being encoded is an encoded area, and the range indicated by b in the picture being encoded is an uncoded area. In other words, the macroblock line included in the range indicated by a in the mid-coding picture is encoded, and the macroblock line included in the range indicated by b in the mid-encoding picture is not encoded. Since N is the number of macroblock lines in one picture, a is the number of macroblock lines included in the encoded area, and b is the number of macroblock lines included in the uncoded area.

  At this time, when the generated code amount of the encoded area of the picture being encoded is Sa, the generated code quantity Sb of the uncoded area being encoded is estimated by the following equation (1). Here, S′a is the generated code amount in the range indicated by a of the encoded picture, and S′b is the generated code amount in the range indicated by b of the encoded picture. That is, S′a is the generated code amount of the area corresponding to the encoded area of the halfway encoded picture among the encoded pictures, and S′b is the halfway encoded picture among the encoded pictures. This is a generated code amount in a region corresponding to the uncoded region. Here, the encoded region and the region corresponding to the encoded region are regions of the same area at the same position in the picture, and the uncoded region and the region corresponding to the uncoded region are also included in the picture. It becomes the area | region of the same area in the same position.

  As described above, if the pictures are close in time, the generated code amount distributions in the pictures are similar to each other. Here, an encoded picture that has been encoded immediately before is used as the encoded picture. In other words, the encoded picture and the encoded picture are pictures close in time. Therefore, the ratio between the generated code amount Sa of the encoded area of the picture being encoded and the generated code amount Sb of the uncoded area of the encoded picture is the code of the encoded picture of the encoded pictures. Similar to the ratio between the generated code amount S′a of the region corresponding to the encoded region and the generated code amount S′b of the region corresponding to the uncoded region of the encoded picture among the encoded pictures become. Therefore, by calculating Sb from Sa so that the ratio of Sa and Sb is the same as the ratio of S′a and S′b by Equation (1), the unencoded area of the encoded picture The generated code amount can be estimated with high accuracy. Hereinafter, Sb is also referred to as “estimated code amount”.

  Here, in the first embodiment, a case where an encoded picture that has been encoded immediately before is used as an encoded picture used for estimation of the generated code amount is exemplified, but the present invention is not limited to this. As described above, an encoded picture that is close in time to an encoding-in-progress picture has a high correlation with the encoding-in-progress picture. For this reason, an encoded picture that is temporally close to an in-encoding picture other than the encoded picture that has been encoded immediately before may be used.

  For example, any of the encoded pictures included in the range up to a picture started to be encoded a predetermined number of times before the in-encoding picture is used as an encoded picture that is temporally close to the in-encoding picture. You may do it. Also, MPEG-2 or H.264. In some cases, like the video coding standard such as H.264, the picture coding order does not always match the picture playback order. In such a case, if the reproduction order of pictures is known, an encoded picture to be used may be determined based on the picture reproduction order. For example, any of the encoded pictures included in the range up to a picture to be reproduced a predetermined number of times before or after the in-encoding picture is used as an encoded picture that is temporally close to the in-encoding picture. May be. Even in such a moving picture coding standard, the coding order and the playback order do not differ greatly, so that the coded picture to be used should be determined based on which order. Also good.

  In addition, as the encoded picture, an encoded picture of the same picture type as the in-encoding picture and temporally close encoded pictures may be used. Here, for example, MPEG-2 or H.264 is used. In the moving picture coding standard such as H.264, the predictive coding method differs depending on the picture type. Intra-frame prediction (intra prediction) is used for I pictures, forward motion compensated interframe prediction is used for P pictures, and forward, backward, or bidirectional motion for B pictures. Compensated interframe prediction is used. That is, the characteristics of the generated code amount are close between pictures of the same picture type. Therefore, by using a coded picture of the same picture type as the picture being coded, it is possible to improve the estimation accuracy of the generated code amount in the uncoded area.

  Here, there is no encoded picture immediately after the start of encoding of moving image data. In addition, if the scene property changes between an encoded picture and an encoded picture due to a scene change, even if the encoded picture is temporally close to the encoded picture, There is a high possibility that the generated code amount distribution is not similar to the picture being processed. Therefore, in these cases, the generated code amount prediction device 12 does not use the encoded picture, and responds to the ratio between the encoded area of the encoding intermediate picture and the unencoded area of the encoding intermediate picture. The generated code amount Sb of the unencoded area of the picture being encoded may be estimated by proportional calculation. That is, the generated code amount in the unencoded area of the picture being encoded is estimated by the following equation (2).

  The code amount control device 13 determines the code amount control parameter based on the difference between the estimated code amount of the uncoded region estimated by the method described above and the target code amount that can be allocated to the uncoded region. To do. As described with reference to FIG. 3, since the progress of encoding is different for each of the encoders 30 to 33, different code amount control parameters are determined for each of the encoders 30 to 33. become. Then, the code amount control device 13 gives the determined code amount control parameter to each of the encoders 30 to 33. Each of the encoders 30 to 33 encodes an unencoded area of a picture being encoded based on a code amount control parameter given from the code amount control device 13. The code amount control parameter is, for example, a quantization step. For example, when the estimated code amount is larger than the target code amount, the code amount generated by suppressing the quantization step is suppressed. Conversely, when the estimated code amount is smaller than the target code amount, the generated code amount is increased by reducing the quantization step.

  It should be noted that since the encoded region of the picture does not yet exist at the start of picture encoding, the code amount control device 13 may determine the code amount control parameter in any way. As an example, a code amount control parameter that is considered appropriate is determined in advance according to the target code amount of the picture. Specifically, information capable of deriving an appropriate code amount control parameter from each target code amount that a picture can take is prepared in advance in an arbitrary storage device included in the image encoding device 1. This information may be, for example, a table or a function that can obtain a code amount control parameter suitable for the target code amount. Then, the code amount control device 13 refers to this information so that the code amount control parameter can be determined from the target code amount of the picture.

  Further, the target code amount of the picture is determined in advance so as to achieve a predetermined bit rate. The target code amount of a picture may be determined by weighting differently for each picture type based on the bit rate. Specifically, the compression rate in encoding increases in the order of I picture, P picture, and B picture. Therefore, for example, weighting is performed so that the target code amount increases in the order of B picture, P picture, and I picture. For example, the target code amount of a picture being encoded is stored in advance in an arbitrary storage device included in the image encoding device 1 so that the code amount control device 13 can refer to the target code amount.

  Next, with reference to FIG. 7, processing of the image encoding device 1 according to the first embodiment of the present invention will be described. FIG. 7 is a flowchart showing processing of the image encoding device 1 according to the first embodiment of the present invention. Here, optimization of the code amount control parameter for the encoder 30 will be described as an example, but the same processing is performed in parallel for each of the encoders 31 to 33.

  The encoder 30 encodes the macroblock line included in the picture (S1). The encoder 30 stores the encoded data in the stream buffer 40 sequentially. As a result, a bit stream is formed in the stream buffer 40. The encoder 30 stores the generated code amount of the encoded macroblock line in the generated code amount storage device 11. The encoder 30 determines whether all macroblock lines included in the picture have been encoded (S2).

  When all the macroblock lines are encoded (S2: YES), the encoder 30 ends the encoding of the picture.

  When not all the macroblock lines are encoded (S2: NO), from the generated code amount Sa based on the ratio of the generated code amounts S′a and S′b according to the above-described formula (1), The generated code amount Sb is estimated (S3).

  Here, the generated code amount storage device 11 stores the generated code amount in each of the macroblock lines included in the encoded area of the picture being encoded and the generated code in each of the macroblock lines included in the encoded picture. The amount and the stored. Therefore, the generated code amount prediction device 12 calculates each generated code amount Sa, S′a, and S′b based on the generated code amount for each macroblock line stored in the generated code amount storage device 11.

  Specifically, the generated code amount prediction device 12 calculates the sum of the generated code amounts of the macroblock lines in the encoded region of the picture being encoded as the generated code amount Sa of the encoded region of the picture being encoded. To do. The generated code amount prediction device 12 calculates the sum of the generated code amounts of the macroblock lines in the region corresponding to the encoded region of the halfway encoded picture among the encoded pictures in the region corresponding to the encoded region. Calculated as the generated code amount S′a. The generated code amount prediction device 12 calculates the sum of the generated code amounts of the macroblock lines in the region corresponding to the uncoded region of the picture being encoded among the encoded pictures in the region corresponding to the uncoded region. The generated code amount S′b is calculated.

  The generated code amount prediction device 12 outputs the estimated generated code amount Sb of the uncoded region to the code amount control device 13.

  The code amount control device 13 compares the estimated code amount Sb of the unencoded area of the picture being encoded output from the generated code amount prediction apparatus 12 with the target code amount of the uncoded area of the picture being encoded. (S4). For example, the code amount control device 13 calculates the target code amount of the uncoded region by subtracting the generated code amount of the encoded region of the halfway encoded picture from the target code amount of the halfway encoded picture. It should be noted that the code amount control device 13 may acquire the generated code amount of the encoded region of the picture being encoded from the generated code amount storage device 11 in the same manner as the generated code amount prediction device 12, or the generated code amount You may make it acquire with the estimated code amount Sb from the quantity prediction apparatus 12. FIG.

  When the estimated code amount of the uncoded region is larger than the target code amount of the uncoded region (S4: large), the code amount control device 13 reduces the code amount generated by the encoding by the encoder 30. The code amount control parameter is changed (S5). That is, the quantization step is increased. The code amount control device 13 outputs the changed code amount control parameter to the encoder 20.

  When the estimated code amount of the uncoded region is smaller than the target code amount of the uncoded region (S4: small), the code amount control device 13 increases the code amount generated by the encoding by the encoder 30. The code amount control parameter is changed (S6). That is, the quantization step is reduced. The code amount control device 13 outputs the changed code amount control parameter to the encoder 20.

  When the estimated code amount of the uncoded region is equal to the target code amount of the uncoded region (S4: equal), the code amount control parameter is not changed.

  The encoder 30 encodes the next macroblock line based on the code amount control parameter output from the code amount control device 13 (S1). Similarly, by repeating the processes of steps S1 to S6, the picture 5 can be encoded while optimizing the encoding control parameter.

  As described above, since the encoding process according to the first embodiment uses the encoding result of a highly correlated picture, a highly accurate code is generated according to the distribution of the code amount of the picture. Quantity estimation is possible. In particular, as illustrated in FIG. 5, it is possible to estimate the amount of code with high accuracy even for a picture in which the amount of code generated by encoding differs greatly between the first half and the latter half of the picture.

  As described above, in the first embodiment, in the encoded frame that is temporally close to the encoded frame, the regions corresponding to the encoded region and the unencoded region included in the encoded frame The code amount of the uncoded region is estimated from the code amount of the encoded region based on the ratio of the respective code amounts.

  According to this, it is possible to estimate the code amount of the uncoded region with high accuracy by referring to the code amount distribution of the encoded frame having a high correlation with the frame being encoded. Therefore, since an appropriate code amount control parameter can be determined based on the code amount of the unencoded area estimated with high accuracy, it is possible to suppress deterioration of the image quality of the frame when encoded.

Embodiment 2 of the Invention
Next, the configuration of the image encoding device 2 according to the second embodiment of the present invention will be described with reference to FIG. FIG. 8 is a configuration diagram of the image encoding device 2 according to the second embodiment of the present invention. In addition, about the component similar to Embodiment 1, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The image encoding device 2 includes a frame buffer 10, a generated code amount storage device 11, a code amount control device 13, a stream combination device 14, a pre-analysis device 15, an analysis result storage device 16, a generated code amount prediction device 17, and encoders 30 to 30. 33 and stream buffers 40-43. In the second embodiment, the input image input to the image encoding device 2 may be a frame constituting moving image data or a single frame constituting still image data.

  The pre-analysis device 15 receives an input image from the outside of the image encoding device 2. The pre-analysis device 15 analyzes the input image and calculates a feature amount for each unit obtained by dividing one image into a predetermined size. The pre-analysis device 15 stores the calculated feature quantity in the analysis result storage device 16 as a pre-analysis result. The pre-analysis device 15 stores the input image after the pre-analysis in the frame buffer 10.

  The analysis result storage device 16 temporarily stores the pre-analysis result of the input image. The feature amount as the pre-analysis result is stored for each unit obtained by dividing one image into a predetermined size. The analysis result storage device 16 is configured by an arbitrary storage device for storing feature quantities.

  The generated code amount prediction device 17 generates the generated code of the unencoded area of the image being encoded based on the generated code amount of the encoded area of the image being encoded and the feature amount of the image being encoded. Estimate the amount.

  Next, with reference to FIG. 9, a code amount estimation method for an uncoded region according to the second embodiment of the present invention will be described. FIG. 9 is a diagram for explaining a code amount estimation method for an unencoded area according to the second embodiment of the present invention.

  In general, there is a correlation between a specific feature amount in an image and a generated code amount in the image, and the specific feature amount distribution in the image is similar to the generated code amount distribution in the image. It is known. Therefore, the generated code amount prediction apparatus 17 according to the second embodiment determines the generated code amount in the unencoded area of the picture in the middle of encoding, the generated code quantity in the encoded area of the picture in the middle of encoding, It is estimated from the feature quantity of the picture.

  For example, as illustrated in FIG. 9, it is assumed that the encoding has progressed to the range indicated by “a” in the middle of encoding. That is, in FIG. 9, as in the right diagram of FIG. 6, the range indicated by a in the middle picture is an encoded area, and the range indicated by b in the middle picture is an uncoded area. . At this time, when the generated code amount of the encoded region of the picture in the middle of encoding is Sa, the generated code amount Sb of the uncoded region of the picture in the middle of encoding is estimated by the following equation (3). Here, Ca is the feature amount of the encoded area of the picture in the middle of encoding, and Cb is the feature quantity of the uncoded area of the picture in the middle of encoding.

  As described above, the specific feature amount distribution in the picture and the generated code amount distribution in the picture are similar to each other. Therefore, the ratio between the generated code amount Sa of the encoded region of the picture being encoded and the generated code amount Sb of the unencoded region of the image being encoded is the generated code amount of the encoded region of the picture being encoded This is similar to the ratio between Ca and the unencoded code amount Cb of the picture being encoded. Therefore, by calculating Sb from Sa so that the ratio of Sa and Sb is the same as the ratio of Ca and Cb according to Equation (3), the generated code amount of the uncoded area of the encoded picture can be calculated. It can be estimated with high accuracy.

  Here, the pre-analysis device 15 uses, as the feature quantity, for example, a statistical quantity correlated with the generated code quantity such as the sum of absolute difference values between adjacent pixels in the picture, the sum of absolute difference values between pictures, and the intra-picture pixel variance. Is calculated. Here, the difference absolute value between adjacent pixels in a picture is an absolute value of a difference value of pixel values between adjacent pixels in the picture. The inter-picture difference absolute value is an absolute value of a difference value of pixel values at the same position between a picture being coded and a picture input immediately before the picture being coded. Note that the pre-analysis device 15 is not limited to the picture input immediately before, but refers to a picture input in the past and stored in the frame buffer 10, so that the absolute value of the inter-picture difference from the picture input this time May be calculated. The intra-picture pixel variance value is a variance value of pixel values within a picture. The pre-analysis device 15 calculates a feature amount for each macroblock line and stores it in the analysis result storage device 16.

  As the feature amount, any one statistic may be used among the sum of absolute differences between adjacent pixels in a picture, the sum of absolute differences between pictures, and the pixel dispersion value in a picture. Or two or more statistics may be used. In the case of using any two or more statistics, a value obtained by multiplying each of the statistics by a predetermined weight and then adding them may be used as the feature.

  Here, when a picture in the middle of encoding is encoded by intra-frame prediction (intra prediction) like an I picture, the generated code amount is the sum of absolute values of differences between adjacent pixels in a picture or a pixel dispersion value in a picture It grows according to. Therefore, in this case, it is preferable to use at least one of the sum of absolute differences between adjacent pixels in the picture and the pixel dispersion value in the picture as the feature amount.

  Further, when a picture in the middle of encoding is encoded by motion compensated interframe prediction, such as a P picture or a B picture, the generated code amount increases in accordance with the sum of absolute differences between pictures. Therefore, in this case, it is preferable to use the sum of absolute differences between pictures as the feature amount.

  In other words, all the different types of statistics described above may be calculated, and the statistics may be selectively used as the feature value from the calculated statistics according to the picture type.

  Next, with reference to FIG. 10, processing of the image encoding device 2 according to the second exemplary embodiment of the present invention will be described. FIG. 10 is a flowchart showing processing of the image encoding device 2 according to the second embodiment of the present invention. Here, optimization of the code amount control parameter for the encoder 30 will be described as an example, but the same processing is performed in parallel for each of the encoders 31 to 33.

  The pre-analysis device 15 calculates the feature amount of the picture input as the input image for each macroblock line (S11). The prior analysis device 15 stores the calculated feature amount for each macroblock line in the analysis result storage device 16. Since steps S12 and S13 are the same as steps S1 and S2 in the first embodiment, the description thereof is omitted.

  When all the macroblock lines are not encoded (S13: NO), from the generated code amount Sa of the encoded region based on the ratio of the feature amounts Ca and Cb, the uncoded region is calculated based on the ratio of the feature amounts Ca and Cb. The generated code amount Sb is estimated (S14).

  Here, the analysis result storage device 16 stores the feature values of each of the macroblock lines included in the picture being encoded. Therefore, the generated code amount prediction device 17 calculates each feature amount Ca and Cb based on the feature amount for each macroblock line stored in the analysis result storage device 16.

  Specifically, the generated code amount prediction apparatus 17 calculates the sum of the feature amounts of the macroblock lines in the encoded region of the picture being encoded as the feature amount Ca of the encoded region of the picture being encoded. The generated code amount prediction device 17 calculates the sum of the feature amounts of the macroblock lines in the unencoded area of the picture being coded as the feature quantity Cb of the uncoded area of the picture being coded. The generated code amount predicting device 17 encodes the halfway encoded picture as the generated code amount Sa of the encoded region of the intermediate encoded picture, similarly to the generated code amount predicting device 12 according to the first embodiment. The sum of the generated code amounts of the macro block lines in the area is calculated.

  The generated code amount prediction device 17 outputs estimated code amount information indicating the estimated generated code amount Sb of the uncoded region to the code amount control device 13. Henceforth, about the process of step S15-S17, since it is the same as that of step S4-S6 in Embodiment 1, description is abbreviate | omitted.

  As described above, the second embodiment is based on the ratio between the feature amount of the encoded region included in the frame being encoded and the feature amount of the unencoded region included in the frame being encoded. The code amount of the uncoded region is estimated from the code amount of the encoded region.

  According to this, it is possible to estimate the code amount of the uncoded region with high accuracy by referring to the feature amount distribution of the encoded frame having a high correlation with the code amount distribution of the encoded frame. Therefore, since the code amount control parameter can be determined based on the code amount of the unencoded area estimated with high accuracy, it is possible to suppress deterioration of the image quality of the frame when it is encoded.

  In the second embodiment described above, the case where the generated code amount prediction device 17 estimates the generated code amount of the unencoded area of the coding intermediate picture based on the feature amount of the coding intermediate picture is illustrated. However, it is not limited to this. The generated code amount prediction device 17 may estimate the generated code amount of an unencoded area of the picture being encoded based on the feature value of the encoded picture that is temporally close to the picture being encoded. . Specifically, the generated code amount prediction device 17 determines that the ratio between the generated code amount of the encoded region of the picture being encoded and the generated code amount of the uncoded region of the picture being encoded is that of the encoded picture. Among them, the ratio between the feature amount of the region corresponding to the encoded region of the picture being encoded and the feature amount of the region corresponding to the uncoded region of the encoded picture of the encoded picture is the same. In this way, the generated code amount of the uncoded region may be calculated from the generated code amount of the encoded region.

  This is because, in general, an encoded picture that is temporally close to an in-encoding picture has a high correlation with the in-encoding picture, and the feature amount distribution is similar to the in-encoding picture. For this reason, even if the feature amount of the encoded picture that is temporally close to the picture that is being encoded is used, the generated code amount of the uncoded area of the picture that is being encoded can be estimated with high accuracy.

Embodiment 3 of the Invention
Next, the configuration of the image encoding device 3 according to the third embodiment of the present invention will be described with reference to FIG. FIG. 11 is a configuration diagram of the image encoding device 3 according to the third embodiment of the present invention. In addition, about the component similar to Embodiment 1 and Embodiment 2, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The image encoding device 3 includes a frame buffer 10, a generated code amount storage device 11, a code amount control device 13, a stream combination device 14, a pre-analysis device 15, an analysis result storage device 16, a generated code amount prediction device 18, and encoders 30 to 30. 33 and stream buffers 40-43.

  The generated code amount prediction device 18 normally performs the unencoding of the image being encoded based on the generated code amount of the encoded image and the generated code amount of the encoded region of the image being encoded. Estimate the amount of generated code in a region. In other words, the generated code amount prediction device 18 estimates the generated code amount of an unencoded area of an image that is being encoded using Equation (1) in the same way as in the first embodiment.

  The generated code amount prediction device 18 generates an encoded region of an image being encoded when a scene change causes a scene change between the encoded image and the image being encoded. Based on the code amount and the feature amount of the image being encoded, the generated code amount of the unencoded area of the image being encoded is estimated. In other words, the generated code amount prediction device 18 uses the expression (3) in the same manner as in the second embodiment to calculate the generated code amount of the unencoded area of the image being encoded when the scene property changes. presume. As described above, when the nature of the scene changes, even if the encoded picture is temporally close to the encoding-in-progress picture, the generated code amount distribution of the encoded picture and the encoding-in-progress picture This is because there is a high possibility that the generated code amount distribution is not similar.

  Next, with reference to FIG. 12, processing of the image encoding device 3 according to the third exemplary embodiment of the present invention will be described. FIG. 12 is a flowchart showing processing of the image encoding device 3 according to the third embodiment of the present invention. Here, the optimization of the code amount control parameter in the encoder 30 will be described as an example.

  In addition, about step S21-23, since it is the same as that of step S11-13 concerning Embodiment 2, description is abbreviate | omitted.

  When all the macroblock lines have not been encoded (S23: NO), the generated code amount prediction device 18 determines whether or not the scene property has changed between the encoded picture and the intermediate picture. (S24). Note that whether or not the nature of the scene has changed is determined based on the difference value of the pixel value between the encoded picture and the current encoding picture.

  For example, the pre-analysis device 15 calculates the sum of the absolute values of inter-picture differences between the encoded picture and the half-encoded picture and stores it in the analysis result storage device 16. The generated code amount prediction device 18 acquires the sum total of the inter-picture difference absolute values stored in the analysis result storage device 16. The generated code amount prediction device 18 determines that the nature of the scene has changed when the sum of the absolute values of inter-picture differences in the entire picture exceeds a predetermined threshold value, and determines the inter-picture difference in the entire picture. When the sum of absolute values does not exceed a predetermined threshold value, it is determined that the scene property has not changed.

  When it is determined that the nature of the scene has changed (S24: Yes), the generated code amount prediction device 18 uses the feature value of the coding intermediate picture based on the feature amount of the coding middle picture as in step S14 in the second embodiment. The estimated code amount of the uncoded area is estimated (S25).

  If it is determined that the nature of the scene has not changed (S24: No), the generated code amount prediction device 18 uses the generated code amount of the encoded picture, as in step S3 according to the first embodiment. The estimated code amount of the unencoded area of the picture being encoded is estimated (S26).

  The generated code amount prediction device 18 outputs estimated code amount information indicating the estimated generated code amount Sb of the uncoded region to the code amount control device 13. Henceforth, about the process of step S27-S29, since it is the same as that of step S4-S6 in Embodiment 1, description is abbreviate | omitted.

  As described above, in the third embodiment, when there is a predetermined change between the encoded frame and the encoded frame, the feature amount of the encoded area included in the encoded frame is The code amount of the unencoded region is estimated from the code amount of the encoded region based on the ratio to the feature amount of the unencoded region included in the frame being encoded. In the third embodiment, if there is no predetermined change between the encoded frame and the encoded frame, the encoded frame in the encoded frame that is temporally adjacent to the encoded frame is being encoded. The code amount of the uncoded region is estimated from the code amount of the encoded region based on the ratio of the code amount of each of the encoded region and the region corresponding to the uncoded region included in the frame. .

  According to this, when there is a predetermined change such as a scene change between the encoded frame and the encoded frame, and it is considered that the correlation between the encoded frame and the encoded frame is not high. Thus, it is possible to estimate the code amount of the uncoded region with high accuracy by referring to the feature amount distribution of the encoded frame having a high correlation with the code amount distribution of the encoded frame. In addition, if there is no predetermined change such as a scene change between the encoded frame and the encoded frame, and the correlation between the encoded frame and the encoded frame is considered high, With reference to the code amount distribution of the encoded frame having a high correlation with the frame, the code amount of the uncoded area can be estimated with high accuracy.

  Therefore, an appropriate code amount control parameter can be determined based on the code amount of the uncoded region estimated with high accuracy regardless of whether or not the nature of the scene is changed. Degradation of the image quality of the frame can be suppressed.

  In the third embodiment described above, only the feature amount of the picture in the middle of encoding is used when the scene is changed. However, the present invention is not limited to this. For example, when a scene change occurs, as shown in the following equation (4), an uncoded region is generated by using an average value of the feature amount of a picture being encoded and the generated code amount of a coded picture. The code amount Sb may be calculated.

  In addition, when a scene change occurs, as shown in the following equation (5), the weighted average value obtained by weighting the feature amount of the picture being encoded and the code amount of the encoded picture is used. The generated code amount Sb in the coding area may be calculated. Here, W is a predetermined arbitrary value between 0 and 1.

Embodiment 4 of the Invention
Next, the configuration of the image encoding device 4 according to the fourth embodiment of the present invention will be described with reference to FIG. FIG. 13 is a configuration diagram of the image encoding device 4 according to the fourth embodiment of the present invention. In addition, about the component similar to Embodiment 1-3, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The image encoding device 4 includes a frame buffer 10, a generated code amount storage device 11, a code amount control device 13, a stream combination device 14, a pre-analysis device 19, an analysis result storage device 20, a generated code amount prediction device 21, and encoders 30 to 30. 33 and stream buffers 40-43.

  The pre-analysis device 19 receives an input image from the outside of the image encoding device 4. The pre-analysis device 19 analyzes the input image and calculates a motion amount for each unit obtained by dividing one image into a predetermined size. The amount of motion is, for example, a motion vector. The prior analysis device 19 stores the calculated amount of motion in the analysis result storage device 20 as a prior analysis result. The pre-analysis device 19 stores the input image after the pre-analysis in the frame buffer 10. Here, the pre-analysis device 19 calculates the amount of motion in the image input this time by referring to the previously input image stored in the frame buffer 10 without being limited to the image input immediately before. You may do it.

  The analysis result storage device 20 temporarily stores the pre-analysis result of the input image. The amount of movement as a result of the prior analysis is stored for each unit obtained by dividing one screen into predetermined sizes.

  The generated code amount prediction device 21 performs encoding based on the generated code amount of the encoded image, the generated code amount of the encoded region of the image being encoded, and the motion amount of the image being encoded. A generated code amount of an unencoded area of an intermediate image is estimated.

  Next, with reference to FIGS. 14A and 14B, a code amount estimation method for an uncoded region when there is a motion according to the fourth embodiment of the present invention will be described. FIGS. 14A and 14B are diagrams for explaining a code amount estimation method for an uncoded region when there is a motion according to the fourth embodiment of the present invention.

  In the fourth embodiment, when there is a motion in an unencoded area of a picture in the middle of encoding, it has been encoded at a position corrected in accordance with the motion of the unencoded area of the picture in the middle of encoding detected by the pre-analysis device 19 Using the generated code amount of a picture, the generated code amount of an unencoded area of a picture being encoded is estimated.

  As shown in FIG. 14A, the pre-analysis device 19 calculates the amount of motion in the vertical direction for each macroblock line of a picture. The prior analysis device 19 stores the calculated amount of motion in the analysis result storage device 20 for each macroblock line. When the generated code amount prediction device 21 estimates the generated code amount of the uncoded region of the picture being encoded, the generated code amount prediction device 21 calculates the motion amount of the uncoded region based on the motion amount stored in the analysis result storage device 20. calculate. Here, FIG. 14A illustrates a case where there are N macroblock lines in one picture, as in FIG. Further, in FIG. 14, as in the right diagram of FIG. 6, the range indicated by a in the middle picture is an encoded area, and the range indicated by b in the middle picture is an uncoded area. .

  For example, as illustrated in FIG. 14A, the generated code amount prediction device 21 calculates the average value of the motion amounts of the macroblock lines included in the unencoded area of the in-encoding picture in the unencoded area of the in-encoding picture. Calculated as the amount of movement. Note that the method for calculating the motion amount of the uncoded region is not limited to this. For example, the sum of the motion amounts of the macroblock lines included in the unencoded area of the picture that is being encoded may be calculated as the motion amount of the uncoded area. For example, instead of all the macroblock lines included in the unencoded area of the picture being encoded, the average value or sum of two or more macroblocks arbitrarily selected from them is used as the motion amount of the uncoded area It may be calculated. For example, the motion amount of a macroblock line located in the middle of all the macroblock lines included in the unencoded region of the picture that is being encoded may be calculated as the motion amount of the unencoded region.

  As shown in FIG. 14A, when there is a vertical downward motion in the uncoded area, as shown in FIG. 14B, the image in the uncoded area of the picture being coded is It will not be in the same position. That is, among the encoded pictures, the area corresponding to the uncoded area of the picture being coded is located at a position shifted vertically upward from the uncoded area of the picture being coded. Strictly speaking, this relationship is established when the picture being encoded is a picture displayed after the encoded picture.

  The same can be said between the encoded region of the in-encoding picture and the region corresponding to the encoded region of the in-encoding picture among the encoded pictures. This is because the unencoded area of the picture being encoded and the area corresponding thereto are adjacent to the encoded area of the picture being encoded and the areas corresponding thereto. In this case, as shown in FIG. 14B, all the images in the encoded area of the picture in the middle of encoding are not included in the encoded picture.

  Here, in FIG. 14B, the range indicated by a ′ of the encoded picture is an area corresponding to the range indicated by a that is the encoded area of the picture in the middle of the encoded picture. The range indicated by b ′ of the completed picture is an area corresponding to the range indicated by b, which is an unencoded area of the picture being encoded, among the encoded pictures. Then, a ′ is shorter than a, and b ′ is the same length as b. That is, the uncoded area and the area corresponding to the uncoded area are areas of the same area at different positions in the picture, but the encoded area and the area corresponding to the encoded area are a picture. It becomes the area | region of a different area in a different position inside.

  Therefore, when the generated code amount prediction device 21 determines that there is a motion in the unencoded area of the halfway encoded picture, the generated code amount prediction apparatus 21 corrects the generated code amount of the encoded picture, Estimate the amount of generated code in a region.

  Specifically, when the generated code amount of the encoded region of the picture in the middle of encoding is Sa, the generated code amount Sb of the uncoded region of the picture in the middle of encoding is estimated by the following equation (6). Here, S′a is the generated code amount in the range indicated by a ′ of the encoded picture, and S′b is the generated code amount in the range indicated by b ′ of the encoded picture.

  First, in this way, when it is considered that the range indicated by b of the picture being encoded exists in the range indicated by b ′ in the encoded picture, the generated code amount in the range indicated by b ′ of the encoded picture Is used as the generated code amount of the area corresponding to the range indicated by b which is an uncoded area of the picture in the middle of encoding. That is, the position corrected according to the amount of motion of the uncoded area is determined as the position of the area corresponding to the uncoded area of the picture being coded. The same applies to the position corresponding to the encoded area of the picture being encoded.

  However, as described above, the range indicated by a in the picture being encoded is only within the range indicated by a ′ in the encoded picture. For this reason, the estimated code amount Sb is calculated by correcting the generated code amount S′a per range indicated by a ′ of the encoded picture to the generated code amount per range indicated by a.

  It should be noted that the above equation (6) calculates the estimated code amount Sb by correcting the generated code amount Sa per range indicated by a of the picture being encoded to the generated code amount per range indicated by a ′. Equivalent to equation (7).

  Therefore, the estimated code amount Sb of the uncoded area when there is a motion in the uncoded area of the picture in the middle of coding may be estimated by either expression (6) or expression (7). That is, when there is a motion in an unencoded area of a picture in the middle of encoding, one of the generated code amount Sa and the generated code amount S′a is generated as in Expression (6) or Expression (7) described above. The amount is corrected so that the other generated code amount becomes the generated code amount per the same area as the calculated area, and the generated code amount Sb of the uncoded region is estimated.

  In this way, by using the pre-analysis result, even if there is a motion between the coded picture and the halfway coded picture, it is possible to use the generated code amount in an appropriate region and The generated code amount of the area can be estimated. Also, by using the pre-analysis result, even if there is a motion between the encoded picture and the intermediate picture, the generated code amount to be compared is corrected to the generated code amount per the same area. Thus, it is possible to estimate the generated code amount of the uncoded area. Therefore, according to the fourth embodiment, it is possible to estimate the generated code amount while suppressing a decrease in accuracy even when there is a motion.

  Next, with reference to FIG. 15, processing of the image encoding device 4 according to the fourth embodiment of the present invention will be described. FIG. 15 is a flowchart showing processing of the image encoding device 4 according to the fourth embodiment of the present invention. Here, the optimization of the code amount control parameter in the encoder 30 will be described as an example.

  The pre-analysis device 19 calculates the motion amount of the picture input as the input image for each macroblock line (S31). Since steps S32 and 33 are the same as steps S1 and S2 in the first embodiment, description thereof is omitted.

  When all the macroblock lines are not encoded (S33: NO), the generated code amount prediction device 21 determines whether or not there is a motion in the unencoded area of the picture being encoded (S34). Here, the analysis result storage device 20 stores the amount of motion in each of the macroblock lines included in the picture being encoded. Therefore, the generated code amount prediction device 21 calculates the motion amount of the unencoded area of the picture being encoded based on the motion amount for each macroblock line stored in the analysis result storage device 20. Specifically, the generated code amount prediction device 21 calculates the average value of the motion amounts of the macroblock lines in the uncoded region of the picture being coded as the motion amount of the uncoded region of the picture being coded. The generated code amount prediction device 21 determines that there is a motion in the uncoded region when the calculated motion amount in the uncoded region indicates a motion.

  When it is determined that there is no motion in the uncoded region (S34: No), the generated code amount prediction device 21 is based on the generated code amount of the encoded picture, similarly to step S3 according to the first embodiment. The estimated code amount of the unencoded area of the picture being encoded is estimated (S35).

  If it is determined that there is motion in the unencoded area (S34: Yes), the generated code amount prediction device 21 selects the code among the encoded pictures according to the amount of motion in the unencoded area of the in-encoding picture. The position of the region corresponding to each of the encoded region and the unencoded region of the picture in process of correction is corrected. The generated code amount prediction device 21 corrects the generated code amount S′a and the generated code amount Sa so as to be the generated code amount per the same area by the above-described equation (6) or (7). Based on the ratio between the generated code amounts S′a and S′b, the generated code amount Sb in the uncoded region is estimated from the generated code amount Sa (S36).

  Here, in the case where there is motion in the unencoded area, what is the method for specifying the position corresponding to the motion amount of the area corresponding to the unencoded area of the halfway encoded picture among the encoded pictures? It may be a method. As an example, based on the amount of motion of an unencoded area of a picture that is being encoded, information that can be used to derive how many macroblock lines the uncoded area and the area that should correspond to it are image-encoded A storage device (not shown) included in the device 4 is prepared in advance. This information may be, for example, a table or function that can obtain the number of macroblock lines from the amount of motion. Then, the generated code amount prediction device 21 refers to this information and can specify the position according to the amount of motion of the region corresponding to the uncoded region from the amount of motion of the uncoded region. Note that the generated code amount can be calculated by the sum of the generated code amounts of the macroblock lines in the target region as in the first embodiment.

  The generated code amount prediction device 21 outputs estimated code amount information indicating the estimated generated code amount Sb of the uncoded region to the code amount control device 13. Henceforth, about the process of step S37-S39, since it is the same as that of step S4-S6 in Embodiment 1, description is abbreviate | omitted.

  As described above, in the fourth embodiment, when it is determined that there is motion in the frame being encoded based on the motion amount of the frame being encoded, each of the encoded region and the unencoded region is displayed. The position of the corresponding region is corrected according to the amount of motion in the frame being encoded.

  According to this, even if there is a motion between the encoded frame and the encoded frame having a high correlation with the encoded frame, the appropriate position of the code amount distribution of the encoded frame is referred to. Thus, it is possible to estimate the code amount of the uncoded area with high accuracy. Therefore, since an appropriate code amount control parameter can be determined based on the code amount of the unencoded area estimated with high accuracy, it is possible to suppress deterioration of the image quality of the frame when encoded.

  In the fourth embodiment described above, the case where there is a movement in the vertically downward direction in the uncoded area is exemplified, but as shown in FIG. 16A, the case where there is a movement in the vertically upward direction in the uncoded area. Similarly, the generated code amount of the encoded picture may be corrected to estimate the generated code amount of the unencoded area of the picture being encoded. As shown in FIG. 16A, when there is a vertical upward motion in the unencoded area, the range indicated by b of the halfway encoded picture reaches the range indicated by b ′ in the encoded picture as shown in FIG. 16B. It will only fit. Here, since N, a, b, a ′, and b ′ are the same as those in FIGS. 14A and 14B, description thereof is omitted. However, in FIG. 16B, a ′ has the same length as a, and b ′ is shorter than b.

  In such a case, the generated code amount Sb of the unencoded area of the picture being encoded is estimated by the following equation (8). Since Sa, S′a, and S′b are the same as in equations (6) and (7), description thereof is omitted.

  In this manner, the estimated code amount Sb may be calculated by correcting the generated code amount S′b per range indicated by b ′ of the encoded picture to the generated code amount per range indicated by b. it can.

  That is, according to the fourth embodiment, the code amount estimation method for an unencoded area when there is a motion is performed on the generated code amount of the encoded area of the picture being encoded and the encoded area of the picture being encoded. It is not limited to correcting one of the generated code amounts so that the generated code amount of the corresponding region is the generated code amount per the same area. The coding intermediate picture so that the generated code amount of the uncoded area of the coding intermediate picture and the generated code quantity of the area corresponding to the uncoded area of the coding intermediate picture become the generated code quantity per the same area. The generated code amount in the area corresponding to the uncoded area may be corrected.

  Further, these corrections are not limited to correction to the generated code amount per the same area as in the above-described fourth embodiment, but are substantially the same area to the extent that it is considered that the code amount control is not affected. You may make it correct | amend to the amount of generated codes per hit. As an example, the generated code amount per substantially the same area may be corrected so that the difference between the areas is within a few percent of any one area.

  In the fourth embodiment described above, the motion amount of the unencoded area of the picture being encoded is calculated and used, but the present invention is not limited to this. For example, in the same manner, the motion amount of the entire picture being encoded or the motion amount of the encoded picture of the intermediate picture may be calculated and used.

  The fourth embodiment may be implemented in combination with the third embodiment. For example, when it is determined that the scene property has changed (S24: No), the generated code amount prediction device 21 determines whether or not there is motion in the uncoded part (S34), Accordingly, the above-described processing is executed (S35, S36). In this case, the prior analysis device 19 calculates the motion amount together with the feature amount.

Embodiment 5 of the Invention
Subsequently, the configuration of the image encoding device 5 according to the fifth embodiment of the present invention will be described with reference to FIG. FIG. 17 is a configuration diagram of the image encoding device 5 according to the fifth embodiment of the present invention. In addition, about the component similar to Embodiment 1-4, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The image encoding device 5 includes a frame buffer 10, a generated code amount storage device 11, a stream combination device 14, a pre-analysis device 19, an analysis result storage device 20, a generated code amount prediction device 22, a buffer occupation amount prediction device 23, a code amount. A control device 24, encoders 30 to 33, and stream buffers 40 to 43 are included.

  The generated code amount prediction device 22 estimates the generated code amount of the unencoded area of the coding intermediate picture in the same manner as the generated code amount prediction device 18 according to the third embodiment. The generated code amount of the encoded region and the generated code amount and estimated code amount of each of the encoded region and unencoded region of the picture being encoded are output to the buffer occupancy prediction device 23. Here, the generated code amount prediction device 22 acquires the generated code amount necessary for predicting the buffer occupancy amount from the generated code amount storage device 11 and outputs it to the buffer occupancy amount prediction device 23.

  The buffer occupation amount prediction device 23 predicts the transition of the buffer occupation amount of the virtual buffer on the decoder side based on the generated code amount and the estimated code amount output from the generated code amount prediction device 22. Here, the virtual buffer on the decoder side is, for example, MPEG-2 and H.264. It is a buffer in a virtual decoder model defined in a moving picture coding standard such as H.264. Note that the buffer occupation amount prediction device 23 may acquire the generated code amount directly from the generated code amount storage device 11.

  In addition to the difference between the estimated code amount of the uncoded region estimated by the generated code amount prediction device 22 and the target code amount that can be assigned to the uncoded region, the code amount control device 24 further predicts the buffer occupancy amount. Based on the transition of the buffer occupation amount of the decoder-side virtual buffer predicted by the device 23, the code amount control parameter is determined.

  Next, a buffer occupancy estimation method according to Embodiment 5 of the present invention will be described with reference to FIG. FIG. 18 is a diagram illustrating the buffer occupancy of the decoder-side virtual buffer estimated by the buffer occupancy estimation method according to the fifth embodiment of the present invention. FIG. 18 illustrates a case where the buffer occupation amount of the virtual buffer on the decoder side is estimated at the end of encoding of picture 5 in FIG.

  The buffer occupancy prediction device 23 calculates the buffer occupancy of the virtual buffer on the decoder side at the end of encoding of each of the pictures 6 to 9 which are pictures being encoded at the end of encoding of the picture 5 in FIG. It estimates by Formula (9)-(12). Here, B (x) is the buffer occupation amount of the virtual buffer on the decoder side at the end of the encoding of the picture x. In other words, B (x) is the buffer occupancy before the picture x is extracted from the virtual buffer on the decoder side. S (x) is the generated code amount of picture x. Sa (x) is the generated code amount of the encoded region of picture x, and Sb (x) is the generated code amount of the uncoded region of picture x. Rt is a value obtained by multiplying a predetermined bit rate R and the picture encoding interval Δt.

  At the end of encoding of picture 5 in FIG. 2, as shown in FIG. 3, each of pictures 6 to 8 is in the middle of encoding. Therefore, as in the above-described equations (9) to (12), the estimated code amounts Sb (6), Sb (7), and Sb (8) of the unencoded areas of the pictures 6 to 8, and the pictures 5 to 5, respectively. 8 based on the generated code amounts S (5), Sa (6), Sa (7), Sa (8) of each of the encoded areas of 8 and the next encoding The transition of the buffer occupancy B (6) to B (9) at the end of encoding of each picture 9 that starts is estimated.

  Furthermore, Formula (9)-(12) can be represented as following Formula (13). The expression (13) is obtained when the encoding of a picture n is completed, or the picture i that is in the middle of encoding after the picture n or the picture i that is the next picture to be encoded is ended. This is an equation for calculating the buffer occupation amount of the virtual buffer on the decoder side at the time. In addition, when the case of the above-mentioned formula (9) is applied to the formula (13), (Sa (5) + Sb (5)) = S (5). This is because the picture 5 has been encoded and therefore Sa (5) = S (5) and Sb (5) = 0. B (1) is set to an arbitrary value up to the size of the virtual buffer size. For example, a value obtained by subtracting an appropriate margin from the virtual buffer size.

  The code amount control device 24 performs code amount control based on the buffer occupancy estimated by the buffer occupancy prediction device 23 so that the virtual buffer does not fail. For example, if the estimated buffer occupancy is large and an overflow may occur, control is performed to increase the generated code amount. Conversely, if the estimated buffer occupancy is small and underflow may occur, control is performed to reduce the amount of generated code. As described above, the amount of generated code is controlled by adjusting the quantization parameter.

  The code amount control device 24 determines whether or not there is a possibility of overflow, for example, whether or not B (i) is larger than the buffer size of a predetermined virtual buffer. That is, the code amount control device 24 determines that an overflow may occur when B (i) is larger than the buffer size. Further, whether or not there is a possibility of overflow may be determined based on whether or not the difference between B (i) and the buffer size is smaller than a predetermined threshold. That is, when the difference between B (i) and the buffer size is smaller than a predetermined threshold value, it is determined that there is a possibility of overflow.

  Further, the code amount control device 24 determines whether or not underflow may occur, for example, based on whether or not the buffer occupancy after extracting a picture from the virtual buffer is less than zero. That is, the code amount control device 24 determines that underflow may occur when the buffer occupation amount after extracting a picture from the virtual buffer is smaller than the buffer size. Further, whether or not there is a possibility of underflow may be determined based on whether or not the buffer occupation amount after the picture is extracted from the virtual buffer is smaller than a predetermined threshold. That is, it is determined that there is a possibility that underflow may occur when the buffer occupancy after extracting a picture from the virtual buffer is smaller than a predetermined threshold.

  Here, the buffer occupation amount after the picture is extracted from the virtual buffer is B (i) − (Sa (i) + Sb (i)). That is, the buffer occupancy after drawing a picture from the virtual buffer can be expressed by the following equation (14).

  Therefore, the code amount control device 24 generates, for example, when there is a possibility of overflow even when the estimated code amount of the uncoded region is larger than the target code amount of the uncoded region. Control is performed so as not to reduce the code amount. Also, the code amount control device 24, for example, when underflow may occur even when the estimated code amount of the uncoded region is smaller than the target code amount of the uncoded region, Control may be made so as not to increase the generated code amount.

  The processing contents in other encodings in the image encoding device 5 according to the fifth embodiment are the same as those of the image encoding device 3 according to the third embodiment described with reference to FIG. Therefore, the description is omitted.

  As described above, in the fifth embodiment, in the transition of the buffer occupancy, the buffer occupancy when the encoded frame is extracted from the buffer is used as the code of the encoded area of the encoded frame. It is estimated by the code amount of the frame being encoded that is calculated based on the amount and the code amount of the uncoded region estimated from the code amount of the encoded region by the generated code amount prediction device 22.

  According to this, based on the generated code amount of the encoded region for which the generated code amount is fixed and the estimated code amount of the uncoded region estimated with high accuracy by the generated code amount prediction device 22, Can be calculated with high accuracy. Therefore, the transition of the buffer occupancy in the virtual decoder can be estimated with high accuracy. Therefore, the code amount control can be performed more accurately based on the estimated transition of the buffer occupancy so that the buffer in the virtual decoder does not fail.

  That is, MPEG-2 and H.264. A bit stream corresponding to the standard can be generated in consideration of a virtual decoder buffer restriction defined in a moving picture coding standard such as H.264. In other words, appropriate code amount control can be performed while complying with the restrictions of the standard.

  In the fifth embodiment described above, the generated code amount prediction device 22 determines the generated code amount of the uncoded area of the picture being encoded in the same manner as the generated code amount prediction device 18 according to the third embodiment. Although illustrated about the case where it estimates, it is not restricted to this. For example, similarly to the generated code amount prediction device 12 according to the first exemplary embodiment, the generated code amount prediction device 22 generates the generated code amount of the encoded picture, and the generated code amount of the encoded region of the intermediate picture. Based on the above, the generated code amount of the unencoded area of the picture being encoded may be estimated. Similarly to the generated code amount prediction device 17 according to the second exemplary embodiment, the generated code amount prediction device 22 converts the generated code amount in the encoded region of the in-encoding picture and the feature amount of the in-encoding picture. Based on this, the generated code amount of the unencoded area of the picture being encoded may be estimated. Similarly to the generated code amount prediction apparatus 21 according to the fourth embodiment, the generated code amount prediction apparatus 22 generates the generated code amount of the encoded picture, and the generated code amount of the encoded area of the intermediate picture. The generated code amount of the unencoded area of the picture being coded may be estimated based on the amount of motion of the picture being coded.

Embodiment 6 of the Invention
Subsequently, the configuration of the image encoding device 6 according to the sixth embodiment of the present invention will be described with reference to FIG. FIG. 19 is a configuration diagram of the image encoding device 6 according to the sixth embodiment of the present invention. In addition, about the component similar to Embodiment 1-5, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The image encoding device 6 includes a frame buffer 10, a generated code amount storage device 11, a pre-analysis device 19, an analysis result storage device 20, a generated code amount prediction device 22, a buffer occupation amount prediction device 23, a code amount control device 24, an initial value The apparatus includes a delay amount calculation device 25, a stream correction position storage device 26, a stream combination device 27, encoders 30 to 33, and stream buffers 40 to 43.

  The initial delay amount calculation device 25 calculates the determined initial delay amount of the correct picture after the initial delay amount of the picture in the virtual buffer on the decoder side is determined. The initial delay amount calculation device 25 outputs the calculated correct initial delay amount to the stream combination device 27.

  In the stream correction position storage device 26, the initial delay amount data start position data indicating the position where the recording of the initial delay amount data is started on the stream buffers 40 to 43, and the maximum bit amount that the initial delay amount data can take. Is stored as an initial delay amount data bit amount. The stream correction position storage device 26 includes an arbitrary storage device for storing the initial delay amount data start position data and the initial delay amount data bit amount.

  The stream combiner 27 generates a single bitstream by combining the bitstreams stored in the stream buffers 40 to 43 as in the stream combiner 14 in each of the first to fifth embodiments. The initial delay amount data in the bitstream is replaced with the correct initial delay amount output from the initial delay amount calculation device 25. The stream combining device 27 may include an arbitrary storage device for working for combining the bit streams.

  Subsequently, referring to FIG. 20 and FIG. Problems in generating a bitstream in the moving picture coding standard such as H.264 will be described. FIG. 20 is a diagram showing the configuration of a bit stream of one picture in the moving picture coding standard such as MPEG-2. FIG. 21 shows the transition of the buffer occupancy of the virtual buffer estimated at the end of encoding of picture 5 and the buffer occupancy of the actual virtual buffer determined by the end of encoding of picture 8 in the case shown in FIG. It is a figure which shows transition.

  As shown in FIG. 20, in MPEG-2, a bit stream of one picture includes a picture header and a macroblock code area. The picture header includes an initial delay data area in which initial delay amount data is stored. The initial delay amount data indicates the initial delay amount of the picture in the virtual decoder model. The macro block code area stores encoded macro block data. That is, data obtained by coding a picture is stored in the macroblock code area. H. Similarly, in H.264, a data area including initial delay amount data as auxiliary additional information is arranged in front of the macroblock code area. In the following description, it is arranged before the macroblock code area. A data area including data in which information relating to a picture layer is encoded, such as a picture parameter set defined in H.264 and auxiliary additional information, is referred to as a picture header.

  Thus, MPEG-2 and H.264 In a moving picture coding standard such as H.264, initial delay amount data is included in a picture header. In addition, in such a moving picture coding standard, the initial delay amount data is expressed by a fixed-length data expressed by a predetermined number of bits or a minimum number of bits that can express the initial delay amount. Variable length data. When the initial delay amount data is expressed as variable length data, the initial delay amount data area has a size corresponding to the initial delay amount data. Therefore, the picture header including the initial delay amount data area also has a size corresponding to the initial delay amount data.

  Here, in the standard, picture headers are aligned in units of 1 byte. That is, the bit amount of the picture header is a multiple of 8 bits (1 byte). For example, when the data included in the picture header such as the information on the picture layer described above can be expressed by 87 bits, the remaining 1 bit in the picture header is filled with stuffing bits. In other words, the data excluding the stuffing bits in the picture header is effective data used for decoding and reproduction of the picture. Hereinafter, valid data in the picture header is referred to as “picture header data”. Of course, the size of the picture header may be equal to the size of the picture header data. In this case, the stuffing bit is not included in the picture header. Note that the start position of the initial delay amount data area is not necessarily a position in units of 8 bits (1 byte). In other words, the start position of the initial delay amount data area is not necessarily an alignment boundary.

  Here, when the bit stream is generated by the encoder, the processing for generating the bit stream can be simplified by generating the data in the bit stream in the output order. Specifically, first, a picture header is created, and after the picture header, data obtained by encoding a macroblock is connected every time the macroblock is encoded. If the picture header is created first in this way, the bit stream can be generated only by connecting the encoded macro blocks.

  However, as shown in FIG. 2, when the encoding of the picture 5 in the encoder 30 is completed and before the encoding of the picture 9 is started, the buffer occupation of the virtual buffer at the end of the encoding of the picture 9 is performed. The amount is not fixed. In other words, the buffer occupation amount of the virtual buffer on the decoder side before the picture 9 is extracted from the virtual buffer and decoded on the decoder side is not fixed. Therefore, at this time, the determined initial delay amount of the picture 9 cannot be stored in the picture header. Here, the buffer occupation amount of the virtual buffer at the end of the encoding of the picture 9 is determined at the end of the encoding of the picture 8. Specifically, as shown in FIG. 21, at the end of encoding of picture 8, the buffer occupancy B (8) at the end of encoding of picture 8 is known, and the generated code amount S (8) of picture 8 is obtained. Has also been confirmed. Therefore, as will be described later, the buffer occupancy of the virtual buffer at the end of encoding of the fixed picture 9 can be calculated at a predetermined bit rate.

  Therefore, at the time when encoding of picture 5 is finished and before encoding of picture 9 is started, the initial delay amount of picture 9 is estimated and stored in the picture header of picture 9, and encoding of picture 8 is performed. It is also conceivable to replace the estimated initial delay amount stored in the picture header of the picture 9 with the determined correct initial delay amount at the time when the process is completed.

  However, as shown in FIG. 21, the estimated initial delay amount and the correct initial delay amount are not necessarily the same value. In other words, the initial delay amount data indicating the estimated initial delay amount may be different from the initial delay amount data indicating the correct initial delay amount. In this case, since the size of the picture header changes in the above-described method, it is necessary to reconfigure the arrangement of the picture header and the macroblock code area in the bit stream and change the size of the bit stream accordingly. There is a problem of end. Therefore, when bit streams are generated in parallel by a plurality of encoders, one bit stream cannot be generated by simply combining the bit streams. That is, there is a problem that the process of generating the bitstream becomes complicated.

  Therefore, in the sixth embodiment, the initial delay amount data area is secured with the maximum initial delay amount data bit amount that the initial delay amount data can take. That is, the picture header is set to the maximum size that the picture header can take. Then, after the correct initial delay amount is determined, the initial delay amount data indicating the correct initial delay amount is replaced. At this time, stuffing bits are additionally included in the picture header as necessary, so that the size of the picture header is maintained to be the same before and after replacement of the initial delay amount data. Hereinafter, the bit stream of the picture 9 generated in the encoder 30 will be described as an example, but the same processing is performed on the bit streams of other pictures generated in the encoders 30 to 33, respectively.

  Referring to FIG. 21, the encoder 30 reserves the initial delay amount data area of the picture header of the picture 9 with the maximum size that the initial delay amount data can take when the encoding of the picture 9 is started. That is, when storing the picture header in the stream buffer 40, the encoder 30 stores temporary initial delay amount data of the maximum size that can be taken by the initial delay amount data. As the temporary initial delay amount data, for example, data filled with stuffing bits is used. The encoder 30 sequentially stores data obtained by encoding a frame following the picture header in the stream buffer 40 according to the encoding of the frame.

  Thereafter, when the encoding of picture 8 is completed, the buffer occupation amount of the decoder-side virtual buffer at the end of encoding of picture 9 is determined. Therefore, when the encoding of the picture 8 is completed, the initial delay amount calculation device 25 calculates the correct initial delay amount of the picture 9.

  Note that the initial delay amount D (9) of the picture 9 at this time is calculated by the following equation (15). Also, the buffer occupancy B (9) at the end of encoding of picture 9 in equation (16) is calculated by the following equation (16). Here, R is a predetermined bit rate, S (x) is a generated code amount of picture x, and Rt is a value obtained by multiplying bit rate R and picture encoding end interval Δt. is there. The initial delay amount calculation device 25 acquires the generated code amount S (8) of the picture 8 from the generated code amount storage device 11.

  Furthermore, Formula (15) can be represented by the following Formula (17), and Formula (16) can be represented by the following Formula (18). i is a picture number. For example, D (i) indicates the initial delay amount of picture i. Note that B (i-1) is calculated when the encoding of the picture i-2 is completed, so that it may be used. B (1) is set to an arbitrary value up to the size of the virtual buffer size. For example, a value obtained by subtracting an appropriate margin from the virtual buffer size.

  The initial delay amount calculation device 25 outputs the calculated correct initial delay amount of the picture 9 to the stream combination device 27.

  On the other hand, the stream correction position storage device 26 calculates the initial delay amount data start position of the picture 9 in the stream buffer 40 based on the generated code amount stored in the generated code amount storage device 11. Here, it is assumed that each of the stream buffers 40 to 43 is a buffer in which the bit stream is cyclically stored in the generation order. In this case, if the generated code amount of the picture 5 encoded immediately before the picture 9 is known, the start position of the picture header of the picture 9 in the stream buffer 40 can be known. The reason is that if the generated code amount of picture 5 is known, the size of the macroblock code area of the bit stream of picture 5 can be found, and the size of the bit stream of picture 5 can be found by adding the size of the picture header to it. is there. That is, the end position of the bit stream of picture 5 that is the start position of the picture header of picture 9 is known. Note that the size of the picture header may be stored in the stream correction position storage device 26 in advance.

  In this way, the stream correction position storage device 26 calculates the start position of the picture header of the picture 9 by referring to the generated code amount of the picture 5 stored in the generated code amount storage device 11. Then, the stream correction position storage device 26 calculates the initial delay amount data start position of the picture 9 based on the calculated start position of the picture header of the picture 9. The stream correction position storage device 26 stores initial delay amount data start position data indicating the calculated initial delay amount data start position of the picture 9. Note that the initial delay amount data start position can be calculated by storing the relative position from the head of the picture header of the initial delay amount data start position in the stream correction position storage device 26 in advance. Further, the initial delay amount data bit amount is also fixed in advance as the maximum value that the initial delay amount data can take, and therefore may be stored in the stream correction position storage device 26 in advance.

  Then, the stream combination device 27 acquires the initial delay amount data start position data and the initial delay amount data bit amount of the picture 9 stored in the stream correction position storage device 26 at the time of stream combination. The stream combining device 27 specifies the data of the area corresponding to the initial delay amount data bit amount from the initial delay amount data start position of the picture 9 indicated by the initial delay amount data start position data as the initial delay amount data area of the picture 9. . The stream combination device 27 replaces the initial delay amount data of the picture 9 in the specified area with the initial delay amount data indicating the correct initial delay amount of the picture 9 output from the initial delay amount calculation device 25.

  Here, the stream combination device 27 arranges the correct initial delay amount data from the initial delay amount data start position so that the head of the correct initial delay amount data is positioned at the initial delay amount data start position. At this time, if the size of the initial delay amount data after the replacement is smaller than the size of the initial delay amount data before the replacement, simply replacing only the initial delay amount data, the initial delay amount data and the initial delay amount A gap is generated between the data following the amount data. Therefore, in this case, in order to prevent the occurrence of a gap between the data, the stream combining device 27 moves the subsequent data so that it immediately follows the initial delay amount data.

  However, when doing so, the end of the picture header data moves forward. For this reason, the stream combining device 27 moves at least data from the end of the new picture header data in order to prevent a part of the data before the movement from remaining after the end of the new picture header data in the picture header. Bits corresponding to the number of bits are replaced with stuffing bits. That is, if the size of the initial delay amount data after replacement is smaller than the initial delay amount data before replacement, the data from the end of the initial delay amount data after replacement to the end of the picture header data before replacement is also replaced. There is a need.

  The stream combining device 27 replaces the temporary initial delay amount data with the correct initial delay amount data in the alignment unit. Here, if the size of the initial delay amount data does not change before and after the replacement, the data replacement range may be any range as long as it is the size of the alignment unit and includes the initial delay amount data.

  For example, if the initial delay amount data size is 16 bits (2 bytes) and the start position of the initial delay amount data area is located at the alignment boundary, if the initial delay amount data is replaced with a minimum size, 16 It is only necessary to replace the initial delay amount data of bits (2 bytes). If the initial delay amount data size is 16 bits (2 bytes), but the start position of the initial delay amount data area is not located at the alignment boundary, the initial delay amount data can be replaced with a minimum size. The 24-bit (3-byte) data including the initial delay amount data may be replaced. This is because the initial delay amount data extends over three alignments. Further, in this case, the data for a total of 8 bits (1 byte) excluding the initial delay amount data may be the same data as before the replacement.

  Also, in these cases, if the initial delay amount data is replaced with a size other than the minimum size, for example, a range including from the start position of the initial delay amount data to the end of the picture header data is to be replaced. Of the picture headers, the entire picture header data may be the replacement target, or the entire picture header may be the replacement target. When the entire picture header data or the entire picture header is to be replaced, for example, by storing data indicating the picture header start position instead of the initial delay amount data start position in the stream correction position storage device 26, The stream combination device 27 identifies an area to be replaced.

  As described above, when the size of the initial delay amount data becomes small after replacement, it is necessary to replace the data up to the end of the picture header data before replacement. Therefore, in this case, the data replacement range is the size of the alignment unit, and any size is acceptable as long as it includes data from the beginning of the initial delay amount data to the end of the picture header data before replacement. Good.

  Here, whether the size of the initial delay amount data becomes small after replacement or not, the range after the end of the picture header data before replacement remains filled with stuffing bits. However, this is limited to the case where the size of the picture header is not equal to the size of the picture header data. When the size of the initial delay amount data becomes small after replacement, replacement with stuffing bits is performed so that the picture header data before replacement does not remain as described above. Therefore, picture header data in a range excluding the range filled with stuffing bits in the picture header can be acquired as data including correct initial delay amount data.

  The processing contents in other encodings in the image encoding device 6 according to the sixth embodiment are the same as the processing of the image encoding device 3 according to the third embodiment described with reference to FIG. Therefore, the description is omitted.

  As described above, according to the sixth embodiment, each of the encoders 30 to 33, when encoding a frame and generating stream data, has a temporary initial of the maximum size that the initial delay amount data can take. After the picture header including the delay amount data is stored in each of the stream buffers 40 to 43, the encoded data of the frame is stored in each of the stream buffers 40 to 43 in accordance with the encoding of the frame. I am going to go. Then, when the stream combining device 27 replaces the temporary initial delay amount data with the initial delay amount data indicating the determined initial delay amount after the determination of the initial delay amount of the frame, the size of the initial delay amount data after the replacement. Is smaller than the size of the provisional initial delay amount data before replacement, the header data size is included before and after replacement of the initial delay amount data by including stuffing bits in the header data instead of the reduced size data. Are the same.

  According to this, it is possible to prevent the size of the picture header from changing even when the data is replaced with the determined correct initial delay amount data. Therefore, it is not necessary to reconfigure the arrangement of the picture header and the macroblock code area in the bit stream to change the size of the bit stream, and the process of generating the bit stream can be simplified. That is, when bit streams are generated in parallel by a plurality of encoders, one bit stream can be generated by simply combining the bit streams.

  In the above-described sixth embodiment, the case where data filled with stuffing bits is used as the temporary initial delay amount data is exemplified, but the present invention is not limited to this. For example, data of the maximum size that can be taken by the initial delay amount data including the estimated initial delay amount data may be used as the temporary initial delay amount data. At this time, if the estimated size of the initial delay amount data is less than the size of the temporary initial delay amount data, the missing amount may be compensated by stuffing bits.

  In the sixth embodiment described above, the initial delay amount calculation device 25 and the stream correction position storage device 26 are further added to the configuration of the fifth embodiment, and the stream combining device is replaced with the stream combining device 14. Although illustrated about the case where it has 27, it is not restricted to this. You may make it apply similarly with respect to either of Embodiment 1-5, or the combination of two or more arbitrary embodiments among them.

Embodiment 7 of the Invention
Next, the configuration of the image encoding device 7 according to the seventh embodiment of the present invention will be described with reference to FIG. FIG. 22 is a configuration diagram of the image encoding device 7 according to the seventh embodiment of the present invention. In addition, about the component similar to Embodiment 1-6, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In this Embodiment 7, the case where the image coding apparatus 7 is a transcoder is illustrated. That is, Embodiment 7 exemplifies a case where the input image input to the image encoding device 7 is an encoded image. Input images are, for example, MPEG-2 and H.264. It is encoded according to a moving image encoding standard such as H.264. In the seventh embodiment, information acquired in decoding in transcoding is used as a preliminary analysis result.

  The image encoding device 7 includes a frame buffer 10, a generated code amount storage device 11, a generated code amount prediction device 22, a buffer occupancy amount prediction device 23, a code amount control device 24, an initial delay amount calculation device 25, and a stream correction position storage device. 26, a stream combination device 27, a decoder 28, a decode information storage device 29, encoders 30 to 33, and stream buffers 40 to 43.

  The decoder 28 decodes the input image. The decoder 28 stores the code amount for each decoding unit of the input image in the decode information storage device 29 as a feature amount before decoding the input image. Here, the decoding unit is, for example, a macroblock line or a macroblock pair line. The input image is MPEG-2 and H.264. When encoded in accordance with a moving image encoding standard such as H.264, the input image includes information indicating the amount of motion for each macroblock. This information is, for example, a motion vector used for compensation inter-frame prediction in the decoding process. Based on the information, the decoder 28 calculates a motion amount for each unit obtained by dividing one image into a predetermined size, and stores it in the decode information storage device 29. Specifically, for example, the amount of motion for each macroblock line unit or macroblock pair line unit is stored.

  In the decode information storage device 29, the feature amount of the input image is temporarily stored as a pre-analysis result. The decode information storage device 29 temporarily stores the amount of motion of the input image. The decode information storage device 29 includes an arbitrary storage device for storing feature amounts and motion amounts.

  Therefore, the generated code amount prediction device 22 uses the feature amount stored in the decode information storage device 29 to estimate the generated code amount of the unencoded area of the picture being encoded. The processing contents in other encodings in the image encoding device 7 according to the seventh embodiment are the same as those of the image encoding device 3 according to the third embodiment described with reference to FIG. Therefore, the description is omitted.

  Note that, here, the configuration of the sixth embodiment has been illustrated for the case where the decoder 28 and the decode information storage device 29 are provided instead of the pre-analysis device 19 and the analysis result storage device 20, but the present invention is not limited thereto. . You may make it apply similarly to either of Embodiment 2-5, or the combination of arbitrary 2 or more of those embodiments.

  At that time, for example, in the case where the present invention is applied to the one that does not require the use of the motion amount in the code amount estimation as in the second and third embodiments, the decoder 28 may not detect the motion amount. In the case where the present invention is applied to the case where the feature amount does not need to be used in the code amount estimation as in the fourth embodiment, the decoder 28 may not detect the feature amount. For example, when applied to the fourth embodiment, the generated code amount prediction device 21 uses the motion amount stored in the decode information storage device 29 to estimate the generated code amount of the uncoded region of the picture being encoded. Will do.

  As described above, in the seventh embodiment, when moving image data including a plurality of frames is re-encoded, the code amount of the frame before decoding is used as the feature amount of the frame used for the code amount estimation. I try to detect it. Further, the amount of motion of the frame calculated by the decoding process is detected as the amount of motion of the frame used for code amount estimation.

  According to this, since the information acquired when re-encoding the moving image data is the feature amount and the motion amount used for the estimation of the generated code amount of the uncoded region, a process for separately detecting them is performed. It becomes unnecessary and the processing can be simplified.

Another embodiment of the invention.
In each of the above-described embodiments, as an example, a case has been described where four pictures are encoded in parallel with four encoders, but the present invention is not limited thereto. For example, a picture may be encoded in two parallels with two encoders, or a picture may be encoded in three parallels with three encoders. Further, four or more pictures may be encoded in parallel by having four or more encoders.

  In each embodiment described above, the calculation of the estimated code amount and the update of the code amount control parameter corresponding to the calculation are exemplified for each encoding of the macroblock line or the macroblock pair line. I can't. The timing for calculating the estimated code amount and updating the code amount control parameter may be every time a macroblock is encoded, or every time a macroblock group including a plurality of macroblocks is encoded. Moreover, it is good whenever it encodes the macroblock line group which consists of a some macroblock line. As long as the progress of encoding is substantially the same in each unit obtained by dividing a picture, a unit obtained by arbitrarily equally dividing a picture may be encoded. Moreover, it is not restricted to the division | segmentation which codes such a picture, It is good also as arbitrary timings.

  The unit for storing the pre-analysis result and the generated code amount of the encoded image may not coincide with the encoding processing unit. That is, it does not have to be stored in units of macroblock lines or macroblock pair lines. For example, the number of screen divisions may be smaller than these units, and the divided areas may be larger. In this case, proportional distribution is made according to the processing unit.

  For example, as shown in FIG. 23, the generated code amount of an encoded picture may be stored for each of three areas indicated by x, y, and z, respectively. In this case, when the generated code amount in the range indicated by x is Sx, the generated code amount in the range indicated by y is Sy, and the generated code amount in the range indicated by z is z, the following equation (19) and By (20), the generated code amount S′a in the range indicated by a and the generated code amount S′b in the range indicated by b are calculated. That is, S′a calculated by Expression (19) and S′b calculated by Expression (20) are used as S′a and S′b in Expression (1), respectively.

  According to this, it is only necessary to store three pieces of generated code amount data for one encoded picture. That is, it is not necessary to store the generated code amount data for each macroblock line or each macroblock pair line. Therefore, since the amount of data to be stored can be reduced, the size of the storage device can be reduced, and the overall size of the image encoding device can be reduced.

  This may be similarly applied to the feature amount that is the result of the prior analysis. Further, by applying the feature amount, it is possible to cope with the case where the image is scaled and the image size when pre-analysis is different from the image size when encoding. In other words, after the feature amount is calculated, the image is upscaled, and then the image is encoded. The generated code amount in the corresponding region can be accurately calculated.

  Also, as shown in FIG. 24, in equation (1), the range indicated by a is a partial area of the entire encoded area of the in-encoding picture, and the range indicated by b is the unencoded area of the in-encoding picture. It is good also as a partial area | region of the whole encoding area | region. In this case, the target code amount to be compared with the estimated code amount Sb is compared with the code amount per range indicated by b among the target code amounts that can be assigned to the uncoded area. That is, when it is referred to as an unencoded area included in the frame being encoded, a part of the entire unencoded area included in the frame being encoded is also included, and the encoding included in the frame being encoded is included. When it is referred to as a completed area, a part of the entire encoded area included in the frame being encoded is also included.

  Note that the present invention is not limited to the above-described embodiment, and can be changed as appropriate without departing from the spirit of the present invention.

1, 2, 3, 4, 5, 6, 7 Image encoding device 10 Frame buffer 11 Generated code amount storage device 12, 17, 18, 21, 22 Generated code amount prediction device 13, 24 Code amount control devices 14, 27 Stream combiners 15 and 19 Pre-analyzers 16 and 20 Analysis result storage device 23 Buffer occupancy prediction device 25 Initial delay amount calculation device 26 Stream correction position storage device 28 Decoder 29 Decode information storage devices 30, 31, 32, and 33 Encoder 40 , 41, 42, 43 Stream buffer

Claims (25)

An image encoding device for encoding moving image data composed of a plurality of frames,
An encoding unit for encoding the frame;
A code amount storage unit that stores a code amount of the encoded frame that has been encoded and a code amount of an encoded region that is included in a frame that is being encoded by the encoding unit;
A code amount estimation unit that estimates the code amount of an unencoded area included in the encoding frame based on the encoding amount of the encoded frame and the encoding amount of the encoding frame stored in the code amount storage unit When,
A code amount control unit that determines a code amount control parameter of the uncoded region based on the estimated code amount of the uncoded region and a code amount that can be assigned to the uncoded region; Prepared,
The encoding unit encodes an unencoded area included in the frame being encoded based on the determined code amount control parameter,
The code amount estimation unit includes a code amount of a region corresponding to each of an encoded region and an unencoded region included in the encoded frame in an encoded frame that is temporally adjacent to the encoded frame. Based on the ratio of the first code amount and the second code amount, the code amount of the uncoded region is estimated from the code amount of the encoded region.
Image encoding device. The image encoding device includes a feature amount detection unit that detects a feature amount of the frame to be encoded,
The code amount estimation unit further estimates a code amount of the uncoded region based on the detected feature amount,
The code amount estimation unit, when there is a predetermined change between the encoded frame and the encoded frame, a feature amount of an encoded area included in the encoded frame, and the encoding Based on the ratio with the feature amount of the unencoded area included in the middle frame, the code amount of the unencoded area is estimated from the code amount of the encoded area, and the encoded frame and the encoded frame When there is no predetermined change between frames, the code amount of the uncoded region is calculated from the code amount of the encoded region based on the ratio of the first code amount and the second code amount. Estimate the quantity,
The image encoding device according to claim 1. The image coding apparatus includes a motion amount detecting section for detecting the motion amount of the frame in the coding,
When the code amount estimation unit determines that there is motion in the encoded frame based on the detected motion amount of the encoded frame, each of the encoded region and the unencoded region The position of the region corresponding to is corrected according to the amount of motion in the frame being encoded,
The image coding apparatus according to claim 1 or 2. When the code amount estimation unit determines that there is a motion in an unencoded area of the frame being encoded, the code amount estimation unit determines the position of an area corresponding to each of the encoded area and the uncoded area. Correct according to the amount of motion in the uncoded area of the middle frame,
The image encoding device according to claim 3. Said code amount estimation unit, if it is determined that there is motion in the frame the encoding, and the code amount of the encoded region, of either the sign-weight area corresponding to the encoded region the code amount of area, corrects the amount of codes and such so that the area and per approximately the same area of the other regions, or, marks of the the code amount of uncoded regions, regions corresponding to the non-coding region the code amount of one of the regions of the issue amount to compensation such that the code amount of approximately per same area as the area of the other regions,
The substantially same area is an area where the difference from the area of the other region is within a few percent of the area of the other region.
The image encoding device according to claim 3 or 4. The image encoding device estimates transition of buffer occupancy in a virtual decoder model based on a code amount stored in the code amount storage unit and a code amount estimated by the code amount estimation unit. A buffer occupancy estimation unit;
The buffer occupancy estimation unit is configured to calculate the buffer occupancy when the encoded frame is extracted from the buffer in the transition of the buffer occupancy, and the code of the encoded region included in the encoded frame. Estimated based on the amount of code and the amount of code of the unencoded region estimated from the amount of code of the encoded region,
The code amount control unit determines the code amount control parameter based on a transition of the buffer occupancy estimated by the buffer occupancy estimation unit so that the buffer does not fail.
The image encoding device according to any one of claims 1 to 5. The image encoding device, as an encoding format of the frame, indicates initial delay amount of the frame in a virtual decoder model, and includes header data including initial delay amount data having a size corresponding to the initial delay amount; Encoding the frame according to an encoding standard that defines stream data including the header data and the encoded data obtained by encoding the frame, and generating the stream data,
The image encoding device includes a data storage unit in which the stream data is stored,
When the encoding unit generates the stream data by encoding the frame, the encoding unit stores header data including temporary initial delay amount data having a maximum size that the initial delay amount data can take in the data storage unit. Then, according to the encoding of the frame, the encoded data of the frame is stored in the data storage unit,
The image encoding device includes a replacement unit that replaces the temporary initial delay amount data with initial delay amount data indicating the determined initial delay amount after the initial delay amount of the frame is determined,
When the size of the initial delay amount data after the replacement is smaller than the size of the temporary initial delay amount data before the replacement, the replacement unit includes stuffing bits in the header data instead of the data of the reduced size. Thus, the header data size is the same before and after replacement of the initial delay amount data.
The image encoding device according to any one of claims 1 to 5. The image encoding apparatus calculates the initial delay amount of the frame by dividing a predetermined bit rate from the buffer occupation amount of the virtual decoder model when the encoding of the frame is completed. With an initial delay amount calculation unit
The image encoding device according to claim 7. Each of the plurality of frames included in the moving image data is encoded,
The image encoding device is a transcoder that re-encodes the moving image data,
The feature amount detection unit decodes the frame and detects a code amount of the frame before decoding as a feature amount of the frame,
The encoding unit encodes the frame decoded by the feature amount detection unit;
The image encoding device according to claim 2. Each of the plurality of frames included in the moving image data is encoded,
The image encoding device is a transcoder that re-encodes the moving image data,
The motion amount detection unit decodes the frame and detects a motion amount calculated in decoding the frame as a motion amount of the frame,
The encoding unit encodes the frame decoded by the motion amount detection unit;
The image encoding device according to any one of claims 3 to 5. The encoded frame is a frame of the same picture type as the encoded frame.
The image encoding device according to any one of claims 1 to 10. An image encoding method in which moving image data consisting of a plurality of frames is encoded by an image encoding device ,
An encoding step in which the image encoding device encodes the frame;
Storing said image encoding apparatus, which stores the code amount of the encoded encoded frame, and a code amount of encoded area included in the frame during the encoding in the middle encoded code amount storage unit Process,
Based on the code amount of the encoded frame and the code amount of the frame being encoded, which is stored in the code amount storage unit by the image encoding device, the code amount of the unencoded area included in the frame being encoded A code amount estimation step for estimating
The image encoding device determines a code amount control parameter for the uncoded region based on the estimated code amount of the uncoded region and the code amount that can be assigned to the uncoded region. A determination step,
In the code amount estimation step, the code amount of an area corresponding to each of an encoded area and an unencoded area included in the encoded frame in an encoded frame that is temporally close to the encoded frame in it based on the ratio of the first code amount and the second code amount, the code amount of the uncoded region from the code amount of the encoded region image coding apparatus estimates,
In the encoding process, the non-coding region contained in the frame during the coding, the image coding apparatus for encoding based on the determined code amount control parameter,
Image coding method. An image encoding device for encoding moving image data consisting of a plurality of frames or still image data consisting of a single frame,
A feature amount detection unit for detecting a feature amount of a frame to be encoded;
An encoding unit for encoding the frame;
A code amount storage unit for storing a code amount of an encoded area included in a frame being encoded that is being encoded by the encoding unit;
And the detected feature quantity, and the based on the stored code amount, the code amount estimator for estimating the code amount of unencoded area included in the frame in the coding,
A code amount control unit that determines a code amount control parameter of the uncoded region based on the estimated code amount of the uncoded region and a code amount that can be assigned to the uncoded region. ,
The encoding unit encodes an unencoded area included in the frame being encoded based on the determined code amount control parameter,
The code amount estimation unit is based on a ratio between a feature amount of an encoded region included in the frame being encoded and a feature amount of an unencoded region included in the frame being encoded. Estimating the code amount of the uncoded region from the code amount of the region,
Image encoding device. The image encoding device encodes the moving image data out of the moving image data and the still image data,
The code amount storage unit further stores a code amount of the encoded frame that has been encoded,
When there is a predetermined change between the encoded frame and the encoded frame, the code amount estimation unit calculates the uncoded amount from the code amount of the encoded region based on the feature amount ratio. An encoded frame that estimates the code amount of the encoding area and is temporally adjacent to the encoding frame if there is no predetermined change between the encoded frame and the encoding frame , Based on the ratio of the first code amount and the second code amount, which are the code amounts of the regions corresponding to each of the encoded region and the unencoded region included in the encoded frame, The code amount of the uncoded region is estimated from the code amount of the completed region,
The image encoding device according to claim 13. The image coding apparatus includes a motion amount detecting section for detecting the motion amount of the frame in the coding,
When the code amount estimation unit determines that there is motion in the encoded frame based on the detected motion amount of the encoded frame, each of the encoded region and the unencoded region The position of the region corresponding to is corrected according to the amount of motion in the frame being encoded,
The image encoding device according to claim 14. When the code amount estimation unit determines that there is a motion in the unencoded area of the frame being encoded, the code amount estimation unit determines the position of the area corresponding to each of the encoded area and the uncoded area. Correct according to the amount of motion in the uncoded area of the middle frame,
The image encoding device according to claim 15. Said code amount estimation unit, if it is determined that there is motion in the frame the encoding, and the code amount of the encoded region, of either the sign-weight area corresponding to the encoded region the code amount of area, corrects the amount of codes and such so that the area and per approximately the same area of the other regions, or, marks of the the code amount of uncoded regions, regions corresponding to the non-coding region the code amount of one of the regions of the issue amount to compensation such that the code amount of approximately per same area as the area of the other regions,
The substantially same area is an area where the difference from the area of the other region is within a few percent of the area of the other region.
The image encoding device according to claim 15 or 16. The image encoding device encodes the moving image data out of the moving image data and the still image data,
The image encoding device estimates transition of buffer occupancy in a virtual decoder model based on a code amount stored in the code amount storage unit and a code amount estimated by the code amount estimation unit. A buffer occupancy estimation unit;
The buffer occupancy estimation unit is configured to calculate the buffer occupancy when the encoded frame is extracted from the buffer in the transition of the buffer occupancy, and the code of the encoded region included in the encoded frame. Estimated based on the amount of code and the amount of code of the unencoded region estimated from the amount of code of the encoded region,
The code amount control unit determines the code amount control parameter based on a transition of the buffer occupancy estimated by the buffer occupancy estimation unit so that the buffer does not fail.
The image encoding device according to claim 13. The image encoding device encodes the moving image data out of the moving image data and the still image data, and an initial delay of the frame in a virtual decoder model as an encoding format of the frame This defines stream data having header data including initial delay amount data indicating the amount and having a size corresponding to the initial delay amount, and encoded data obtained by encoding the frame following the header data. According to an encoding standard, the frame is encoded to generate the stream data,
The image encoding device includes a data storage unit in which the stream data is stored,
When the encoding unit generates the stream data by encoding the frame, the encoding unit stores header data including temporary initial delay amount data having a maximum size that the initial delay amount data can take in the data storage unit. Then, according to the encoding of the frame, the encoded data of the frame is stored in the data storage unit,
The image encoding device includes a replacement unit that replaces the temporary initial delay amount data with initial delay amount data indicating the determined initial delay amount after the initial delay amount of the frame is determined,
When the size of the initial delay amount data after the replacement is smaller than the size of the temporary initial delay amount data before the replacement, the replacement unit includes stuffing bits in the header data instead of the data of the reduced size. Thus, the header data size is the same before and after replacement of the initial delay amount data.
The image encoding device according to claim 13. The image encoding apparatus calculates the initial delay amount of the frame by dividing a predetermined bit rate from the buffer occupation amount of the virtual decoder model when the encoding of the frame is completed. With an initial delay amount calculation unit
The image encoding device according to claim 19. Each of the plurality of frames included in the moving image data is encoded,
The image encoding device is a transcoder that re-encodes the moving image data out of the moving image data and the still image data,
The feature amount detection unit decodes the frame and detects a code amount of the frame before decoding as a feature amount of the frame,
The encoding unit encodes the frame decoded by the feature amount detection unit;
The image encoding device according to any one of claims 13 to 20. Each of the plurality of frames included in the moving image data is encoded,
The image encoding device is a transcoder that re-encodes the moving image data out of the moving image data and the still image data,
The motion amount detection unit decodes the frame and detects a motion amount calculated in decoding the frame as a motion amount of the frame,
The encoding unit encodes the frame decoded by the motion amount detection unit;
The image encoding device according to any one of claims 15 to 17. The feature amount detection unit detects a plurality of feature amounts of different types,
The code amount estimation unit uses a feature amount corresponding to a picture type of the frame being encoded among the plurality of feature amounts.
The image encoding device according to any one of claims 13 to 22. An image encoding device for encoding moving image data composed of a plurality of frames,
A feature amount detection unit for detecting a feature amount of a frame to be encoded;
An encoding unit for encoding the frame;
A code amount storage unit for storing a code amount of an encoded area included in a frame being encoded that is being encoded by the encoding unit;
And the detected feature quantity, and the based on the stored code amount, the code amount estimator for estimating the code amount of unencoded area included in the frame in the coding,
A code amount control unit that determines a code amount control parameter of the uncoded region based on the estimated code amount of the uncoded region and a code amount that can be assigned to the uncoded region. ,
The encoding unit encodes an unencoded area included in the frame being encoded based on the determined code amount control parameter,
The code amount estimation unit is a feature amount of a region corresponding to each of an encoded region and an unencoded region included in the frame being encoded in a frame that is temporally adjacent to the frame being encoded. Based on the ratio of the feature quantity of 1 and the second feature quantity, the code quantity of the uncoded area is estimated from the code quantity of the coded area.
Image encoding device. An image encoding method in which an image encoding device encodes moving image data consisting of a plurality of frames or still image data consisting of a single frame,
A feature amount detecting step in which the image encoding device detects a feature amount of a frame to be encoded;
An encoding step in which the image encoding device encodes the frame;
A storage step in which the image encoding device stores in the code amount storage unit the code amount of the encoded region included in the encoding frame that is being encoded;
A code amount estimating step in which the image encoding device estimates a code amount of an unencoded area included in the frame being encoded based on the detected feature amount and the stored code amount;
The image encoding device determines a code amount control parameter for the uncoded region based on the estimated code amount of the uncoded region and the code amount that can be assigned to the uncoded region. A determination step,
In the code amount estimation step, the encoded amount is calculated based on the ratio between the feature amount of the encoded region included in the encoded frame and the feature amount of the unencoded region included in the encoded frame. the code amount of the uncoded region the image encoding apparatus estimates the code amount of area,
In the encoding process, the non-coding region contained in the frame during the coding, the image coding apparatus for encoding based on the determined code amount control parameter,
Image coding method.

Images