JP2006108785A - Image encoder - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To realize stable code amount control in image encoding, and to form an encoded image of good quality where deviation from a predetermined code amount to be controlled is small. <P>SOLUTION: At the time of encoding an image signal constituted of a plurality of frames, image signals for the plurality of frames being encoded are stored in an input image memory 404, and a re-encoding control circuit 402 judges whether a frame encoded in a predictive encoding circuit 150 is re-encoded. Consequently, encoding is set quicker than input of a frame to an input image memory, and re-encoding of a frame employing proper encoding control parameters is carried out in units of frame (picture) according to various conditions such as an encoding error, a degree of fill-up of frame in the input image memory, and the like. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image encoding apparatus for performing image encoding of moving images such as MPEG2 (Moving Picture Experts Group Phase 2), and in particular, to a predetermined code amount while suppressing visual image quality degradation. The present invention relates to an image encoding device that can be controlled to perform image encoding at a timing.

  In recent years, services that distribute information through various transmission paths such as satellite waves, terrestrial waves, and telephone lines using information compressed by high-efficiency coding for digitized image signals have been put into practical use. Yes. In such a service, when distributing information such as moving images / sounds, MPEG2 (Moving Picture Experts Group Phase 2), which is an international standard, is used as a high-efficiency encoding method for moving images / sounds. MPEG2 is an encoding method that compresses the information amount of an image signal by using a correlation between adjacent pixels (spatial direction) of an image signal and a correlation between adjacent frames or adjacent fields (time direction).

  Image encoding in the MPEG2 standard is processed by the following algorithm. First, temporally continuous image frames are divided into a reference frame and a prediction frame. By encoding the reference frame using only the spatial correlation, the original image can be restored using only the encoded data of the frame. On the other hand, the prediction frame is encoded using the correlation in the time direction and the correlation in the spatial direction from the reference frame, thereby realizing higher encoding efficiency than the reference frame. Note that the encoded data of the prediction frame is recovered by the recovered reference frame and the encoded data of the prediction frame.

  Next, a specific encoding system used in MPEG2 image encoding will be described with reference to FIG. In FIG. 5, numbers are assigned to the picture types as necessary so that they can be identified. An I picture (I frame), which is a reference frame indicated as “I” in FIG. 5A, is periodically present and is information serving as a reference for decoding processing. On the other hand, in the prediction frame, a P picture (P frame) encoded by only prediction from a temporally previous (past) reference frame, indicated by “P” in FIG. There is a B picture (B frame) indicated by “B” in FIG. 5A, which is predictively encoded from two temporally preceding and following (past and future) reference frames. Note that arrows in FIG. 5A indicate prediction directions related to the P picture and the B picture. The P picture itself is a prediction frame and is also used as a reference frame for other P pictures and B pictures.

  An I-picture image signal is divided into processing units called macroblocks of horizontal 16 pixels × vertical 16 pixels based on the luminance signal. The divided macroblock data is further divided into two-dimensional blocks of 8 pixels × 8 pixels and subjected to DCT (Discrete Cosine Transform) processing which is a kind of orthogonal transform.

  Since the signal after the DCT processing shows a value according to the frequency component of the two-dimensional block, the component is concentrated in a low band in a general image. In addition, information degradation of high frequency components has a property that is visually less noticeable than information degradation of low frequency components. Therefore, the amount of information is compressed by finely quantizing the low frequency components and coarsely quantizing the high frequency components, and variable length coding the coefficient component and the continuous length of coefficient 0 having no component.

  Similarly to the I picture, the P picture image signal is also divided into units of macroblocks of horizontal 16 pixels × vertical 16 pixels based on the luminance signal. In the P picture, a motion vector between the reference frame and each macroblock is calculated. Motion vector detection is generally obtained by block matching. In this block matching, the difference between each pixel of the macroblock and each pixel obtained by blocking the reference frame where the horizontal / vertical position where the macroblock exists by the motion vector value is moved into 16 horizontal pixels × 16 vertical pixels. The absolute value sum (or the sum of squared differences) is obtained, and the value of the motion vector taking the minimum value is output as the detected motion vector.

  Each pixel of the macro block is subjected to a difference from each pixel of the two-dimensional block cut out by the motion vector. When an accurate motion vector is detected, the information amount of the difference block is significantly smaller than the information amount of the original macroblock, so that coarser quantization processing than that of the I picture is possible. Actually, it is selected whether to encode a differential block or a non-differential block (intra block) (prediction mode determination), and DCT similar to an I picture is selected for the selected block. A variable length encoding process is performed to compress the amount of information.

  In addition, the same processing as that for the P picture is performed for the B picture, but the I and P pictures that are the reference frames exist before and after the time frame, and motion vectors are detected between the reference frames. Is called. In the B picture, prediction options are prediction from a previous reference frame (forward prediction), prediction from a subsequent reference frame (backward prediction), and an average value (average (average (2)) of two prediction blocks. There are three types of (average) prediction), and prediction mode determination is performed from among four types of schemes including a scheme for decoding only by intra blocks.

  B pictures can be predicted from temporally preceding and following reference frames, so that prediction efficiency is further improved than P pictures. Therefore, in general, a B picture is quantized more coarsely than a P picture. The block selected as the B picture is subjected to the same encoding process as the I and P pictures.

  In the decoding process of a B picture, a prediction process from a later reference frame is also performed, and thus this reference frame needs to be encoded before the B picture. Therefore, in the encoding process, as shown in FIG. 5B, the recorded image signal is arranged such that the B picture is placed after the I picture or P picture which is the reference frame of the B picture. Then, the order is rearranged and encoded. That is, during the encoding process, the order of the input order of the original image is rearranged in view of the encoding order during the decoding process. On the other hand, in the decoding process, as shown in FIG. 5C, the decoded image is reproduced in the order of the input image signals by performing the reverse permutation to the order of FIG. 5B and outputting the result. It becomes possible.

  Next, a general image encoding device and decoding device for realizing MPEG2 image encoding will be described. First, a general image coding apparatus in the prior art will be described. FIG. 6 is a block diagram illustrating an example of a general image encoding device according to the related art. In FIG. 6, the digital image signal (input image signal) input from the input terminal 201 is supplied to and stored in the input image memory 202 and is delayed in order to be rearranged in the encoding order according to the encoding syntax. Is done. The digital image signal output from the input image memory 202 is subjected to a macroblock cutout process in the two-dimensional block conversion circuit 203.

  Macroblock data related to the reference frame is supplied to the orthogonal transform circuit 205 via the subtractor 204, where DCT conversion is performed in units of horizontal 8 pixels × vertical 8 pixels to calculate DCT coefficients. The DCT coefficients are further collected in units of macroblocks of horizontal 16 pixels × vertical 16 pixels based on the luminance signal, and sent to the quantization circuit 206. In the quantization circuit 206, for example, the quantization process is performed by dividing by a different value for each DCT coefficient by a quantization matrix having a different value for each frequency component. The quantized DCT coefficient is sent to the encoding circuit 214, and variable length or fixed length encoding is performed by referring to an address corresponding to the coefficient of the encoding table 215. Then, the multiplexer 216 multiplexes the encoded data after the processing in the encoding circuit 214 and the additional information indicating the location of the macroblock in the screen from the two-dimensional block conversion circuit 203, and the image stream After being stored in the buffer 218 once, it is output from the output terminal 219 as a bit stream (output image bit stream).

  Also, the DCT coefficients quantized by the quantization circuit 206 are subjected to inverse quantization processing and inverse DCT processing by the inverse quantization circuit 212 and the inverse orthogonal transform circuit 213, and the quantized DCT coefficients are decoded. The data is supplied to and stored in the reference image memory 209 via the adder 210 and the deblocking circuit 211. The image stored in the reference image memory 209 is used at the time of predictive frame encoding processing.

  On the other hand, for the predicted frame, a motion vector between images is obtained by the motion vector detection circuit 207 between the macroblock data cut out from the input image memory 202 and the image stored in the reference image memory 209. The motion vector obtained in the motion vector detection circuit 207 is supplied to the motion compensation prediction circuit 208, where a prediction block is cut out from the reference image from the reference image memory 209. The motion compensated prediction circuit 208 selects an optimal prediction mode according to the plurality of extracted prediction blocks, and sends a difference signal from the input image block to be encoded to the orthogonal transformation circuit 205. For this difference signal, the same processing as that of each block of the reference frame described above is performed, the DCT coefficient is quantized, and a bit stream together with the motion vector and the prediction mode is sent from the multiplexer 216 to the image stream buffer 218, Output from the output terminal 219.

  Regarding the control of the code amount, the code amount control circuit 217 compares the code amount of the bit stream output from the multiplexer 216 with the target code amount (target code amount) to obtain the target code amount. In order to make it closer, the quantization fineness (quantization scale) of the quantization circuit 206 is controlled. Then, for the above-described three types of picture types (frame types) having different information amounts, the target code amount for each frame is calculated using the nature and appearance frequency of each picture type for the set encoding bit rate. .

  Also, the target code amount is limited so that a buffer overflow / underflow does not occur in a stream buffer (called a VBV (Video Buffer Verifier) buffer) virtually simulated by a decoding device. In addition, the quantization scale is calculated by calculating the quantization scale value for the target code amount for each frame type by utilizing the fact that the scale and the output code amount are generally inversely proportional. Done. Then, by controlling the quantization scale in a direction approaching the target code amount for each block, control is performed to suppress the encoded stream within the target code amount.

  Next, a general decoding device in the prior art will be described. FIG. 7 is a block diagram illustrating an example of a general decoding device according to the related art. In FIG. 7, first, an image bit stream (image stream) input from the input terminal 101 is stored in the image stream buffer 102. Note that a virtually simulated buffer value is written in the image bit stream, and the following decoding process is performed after the image bit stream is stored in the image stream buffer 102 by the buffer value. By doing so, it is possible to prevent the decoding process from stopping due to the failure of the buffer. In the image bit stream output from the image stream buffer 102, the variable length decoding circuit 103 separates additional information such as a quantization scale, a prediction mode, and a motion vector, and decodes the quantized DCT coefficient. .

  The decoded DCT coefficients are processed in the same manner as the inverse quantization circuit 212 and the inverse orthogonal transform circuit 213 in the encoding circuit (encoding apparatus shown in FIG. 6), and the inverse quantization circuit 105 and the inverse orthogonal transform are performed. In the circuit 111, inverse quantization processing and inverse DCT processing are performed, and an intra block or a difference block is decoded and supplied to the adder 107. In the case of a prediction block, a reference image signal read from the reference image memory 109 in the motion compensated prediction circuit 106 based on the prediction mode and the motion vector value decoded by the variable length decoding circuit 103 (of the process). A prediction block is cut out from a previously stored image signal of an I picture or a P picture. As a result, the decoded intra block or difference block and the prediction block cut out by the motion compensation prediction circuit 106 are added by the adder 107, and the image signal of the macro block is restored.

  The macroblock data (macroblock image signal) restored by the addition processing in the adder 107 is supplied to the deblocking circuit 108 and returned to the image signal in the order of image scanning. At this time, in the case of an I or P picture, it is written in the reference image memory 109, and in the case of a B picture, it is once stored in the output frame memory 110 and then output as an image signal (output image signal). Note that the I or P picture image data stored in the reference image memory 109 is once stored in the output frame memory 110 in accordance with the image output timing as shown in FIGS. In the same manner as described above, an image signal (output image signal) is output.

  Further, in a system that distributes image information using the above-described image encoding technology, there is a need to perform encoding processing again after decoding a once encoded bitstream. For example, when transmitting information from a place where information is collected / recorded to a place where information is distributed, a wired / wireless communication line or recording medium may be considered as a transmission path, but information is distributed from the place where information is collected / recorded. If the bandwidth of the transmission path for transmitting to the location is different from the bandwidth of the transmission path of the system that distributes information, it is necessary to re-encode and change the bit rate of the bit stream There is. For example, such a condition applies to a case where a broadcast station conducts interviewing and edits image data recorded on a VTR and then broadcasts. In addition, when it is desired to record an encoded stream sent by broadcasting or the like on a predetermined recording medium, it is necessary to re-encode the image stream according to the recording capacity and recording rate of the recording medium. There is also. An apparatus that performs the re-encoding process as described above is called a transcoder (image stream conversion apparatus). The basic configuration of the decoding device and the encoding device in the transcoder includes, for example, an image output (for example, an output terminal 112 shown in FIG. 7) of the decoding device and an image input (for example, an input terminal 201 shown in FIG. 6) of the encoding device. Can be realized by a directly connected configuration.

  In such a transcoder, in order to reduce encoding degradation at the time of re-encoding, the picture type at the time of encoding on the image signal (decoded baseband signal) transmitted by the transcoder A scheme is considered in which frame information and macroblock additional information (hereinafter referred to as encoded information) are transmitted in a superimposed manner. For example, in MPEG2 encoding, the quality of an image signal generally deteriorates in the order of I picture, P picture, and B picture. Since the I picture is a reference frame, it is quantized more finely than other frames and is not affected by reference frame deterioration because there is no reference from other images. On the other hand, the P picture and the B picture are roughly quantized in the order of the P picture and the B picture, and are easily affected by reference frame deterioration.

  In the transcoder, it is possible to improve the quality of encoded data after the re-encoding process and to efficiently perform the re-encoding process by effectively using the above-described encoded information. For example, at the time of re-encoding in the transcoder, referring to the encoding information, the picture type in the encoded data before decoding in the transcoder and the picture type at the time of re-encoding are matched, thereby Factors can be reduced. Also, by referring to the macroblock additional information, the processing amount for motion vector detection can be reduced, and by referring to the quantization scale and the number of encoded bits, a guideline for the information amount possessed by the macroblock can be obtained. As a result, good code amount control can be performed at the time of bit rate conversion.

  Here, the configuration of the decoding apparatus and the encoding apparatus in the transcoder that transmits the encoding information as described above will be described. FIG. 8 is a block diagram showing an example of a general stream conversion recording apparatus according to the prior art. The stream conversion recording apparatus shown in FIG. 8 basically has a configuration in which the encoding apparatus shown in FIG. 6 and the decoding apparatus shown in FIG. 7 are connected. Only the functional blocks added in the stream conversion recording apparatus shown in FIG. 8 will be described, and only the part related to image coding will be described.

  The variable-length decoding circuit 2 (corresponding to the variable-length decoding circuit 103 shown in FIG. 7) of the decoding device 100, which is a component of the image stream conversion device, performs frame information, macroblock additional information, and macroblock when the image stream is decoded. Are calculated, and the calculated data is supplied to the encoded information generation circuit 7. The encoded information generation circuit 7 formats the frame information and macroblock additional information for each frame and stores them in the encoded information memory 8.

  The encoded information memory 8 has a storage capacity for a plurality of frames stored in the output frame memory 3 (corresponding to the output frame memory 110 shown in FIG. 7) so as to correspond to output rearrangement of I, P, and B pictures. Yes. The encoded information composed of the frame information and the macroblock additional information stored in the encoded information memory 8 is supplied to the encoded information superimposing circuit 10, and the order of the encoded information extracted according to the picture type in the encoded information superimposing circuit 10. Are changed and output in synchronization with the image data (output image signal) output from the output frame memory 3 in the decoding apparatus 100. That is, the data output from the encoding information superimposing circuit 10 is encoding information attached to the image data output from the output frame memory 3 in the decoding device 100. The encoded information output from the encoded information superimposing circuit 10 is stored in the encoded information memory 11. At this time, in the encoding device 200 which is a component of the image stream conversion device, the image data (image signal) output from the output frame memory 3 corresponds to the input frame memory 14 (the input image memory 202 shown in FIG. 6). ).

  The encoded information stored in the encoded information memory 11 is read by the encoded information separation circuit 12. Then, the encoded information separation circuit 12 separates the frame information and the macroblock additional information from the encoded information, and sends them to the encoding syntax control circuit 13 and the macroblock information generation circuit 15. The encoding syntax control circuit 13 performs control to detect the frame type from the received frame information and rearrange the input images in the encoding order. When the encoded information separation circuit 12 cannot extract the encoded information of the frame, a non-detection signal is sent from the encoded information separation circuit 12 to the encoding syntax control circuit 13, and the encoding information is separated. In the syntax control circuit 13, an encoding syntax is configured in the encoding device 200 in the same manner as a normal encoding process, and the encoding process is performed. On the other hand, the macroblock information generation circuit 15 receives from the encoding information separation circuit 12 the macroblock additional information whose order has been changed in accordance with the encoding syntax in the encoding information separation circuit 12.

  The macroblock information generation circuit 15 converts the encoded information received from the encoded information separation circuit 12 into a motion compensation prediction circuit 16 (corresponding to the motion compensation prediction circuit 208 shown in FIG. 6) and code amount control in the encoding device 200. This is supplied to the circuit 17 (corresponding to the code amount control circuit 217 shown in FIG. 6). The motion compensation prediction circuit 16 cuts out a prediction block from the reference image memory 18 (corresponding to the reference image memory 209 shown in FIG. 6) using the motion vector and the prediction mode existing in the encoded information, and subtracter 19 (FIG. 6). The difference signal with the input image block from the input frame memory 14 to be encoded is generated, and the difference signal is generated by the orthogonal transformation circuit 20 (orthogonal shown in FIG. 6). Corresponding to the conversion circuit 205). If the encoded information separation circuit 12 cannot extract the encoded information of the picture or the encoded information for each macroblock, the encoded information is not detected by the macroblock information generation circuit 15 from the encoded information separation circuit 12. The signal is sent, and the motion vector detection and prediction mode selection processing similar to the normal encoding processing is performed in the encoding device 200.

  Further, the code amount control circuit 17 is supplied with the quantization scale for each macroblock at the time of encoding and the number of bits required for this processing as encoded information, and the supplied information, the current control target, Comparison processing with the set code amount at the encoding bit rate is performed. As a result, the code amount control circuit 17 determines an appropriate quantization scale and performs the encoding process. In the above description, the encoded information memories 8 and 11 are provided separately. However, in a transcoder in which the decoding apparatus 100 and the encoding apparatus 200 are integrated, these encoded information memories 8 and 11 are provided. , 11 can be combined into one.

  Also, unlike the process in the transcoder described above, encoding information is not extracted, and the picture type of the input image stream is always inherited as the picture type at the time of re-encoding. There is also a method for reducing the rearrangement processing of the decoding circuit and the encoding circuit. This method is disclosed, for example, in Patent Document 1 below.

  In addition, in an image encoding technique that allows variation in the code amount for each predetermined frame as in the MPEG2 standard, as a method for controlling an output image bitstream to be a target code amount, generally, for example, the following Such a method is adopted.

For example, a target code amount for each frame is assigned to a target code amount for a certain period of time according to the information amount of each picture type encoded image. Specifically, when the code amount required for each previously encoded picture type is Bits and the average value of the quantization scale value of each picture type is AvgQ, an approximation of the complexity of each picture type C can be calculated by the product of the code amount and the average quantization scale value.
C (T) = Bits (T) * AvgQ (T)
(T: Picture type)

At this time, using the approximate value C of the complexity of each picture type described above, each picture type has a target code amount TotalBits given within a predetermined time (for example, F frame) assumed in coding control. When the number of frames used is Fnum, the target code amounts Budget (I), Budget (P), and Budget (B) for frames when the picture types to be encoded are I, P, and B pictures, respectively,
Budget (I) = {TotalBits * C (I)} / {Fnum (I) * C (I) + Fnum (P) * C (P) + Fnum (B) * C (B)}
Budget (P) = {TotalBits * C (P)} / {Fnum (I) * C (I) + Fnum (P) * C (P) + Fnum (B) * C (B)}
Budget (B) = {TotalBits * C (B)} / {Fnum (I) * C (I) + Fnum (P) * C (P) + Fnum (B) * C (B)}
Is calculated.

  As a method for determining the quantization scale of each macroblock in a picture and controlling the code amount related to quantization / coding with respect to the target code amount Budget thus calculated, generally feedback Control is performed. Hereinafter, an example of this feedback control will be described.

  In feedback control, a quantization control buffer (QB buffer) for determining the quantization scale is provided, and the quantization scale is determined by the position of the QB buffer. Note that, as an initial value of the QB buffer, a buffer point QB_Point indicating a final quantization scale immediately before encoding with the same picture type is set. Further, it is assumed that the target code amount MB_Budget scheduled for each macroblock is uniformly allocated to each macroblock in the picture.

Here, when the number of bits generated by encoding the macroblock is used code amount MB_Used [N], the buffer point QB_Point [N + 1] indicating the quantization scale of the next macroblock is, for example, the target code The buffer point QB_Point [N] can be corrected by the difference value between the amount MB_Budget [N] and the used code amount MB_Used [N].
QB_Point [N + 1] = QB_Point [N] + MB_Used [N] -MB_Budget [N]
However, N represents a macroblock position.

  In this case, when the used code amount MB_Used [N] increases with respect to the target code amount MB_Budget [N], the buffer point QB_Point [N + 1] at the next macroblock position rises, and the quantization scale increases. When the used code amount MB_Used [N] decreases, QB_Point [N + 1] decreases and the quantization scale decreases. That is, as the used code amount MB_Used [N] increases / decreases, the buffer point QB_Point [N + 1] at the next macroblock position increases / decreases, and the quantization scale is determined, and as a result, approaches the constant code amount. Control will be performed.

  Further, when an image stream is transmitted in real time by digital broadcasting or the like, output from a plurality of encoders may be switched in order to switch programs with different resolutions or formats. In such a case, it is necessary to control the virtual buffer positions of a plurality of encoders to the same position at the program switching timing in order not to temporarily stop the reproduced image from the decoder. In order to align the virtual buffer position at a predetermined timing, it is necessary to generate a stream with a fixed code amount for encoding a certain frame. In addition, when editing an image stream, it is possible to easily insert an image stream by completely matching the total code bit amount up to a predetermined frame. It is necessary to make it. If the code amount is suppressed below a predetermined code amount to be controlled, it is possible to adjust to a fixed code amount by packing invalid information called stuffing bits. This wastes information and degrades the image quality of the encoded image stream.

  In addition, even when the above conditions are not met, if the characteristics of the input image change when the scene changes, or if the code amount control in the picture is disturbed, the whole or part of the screen May be encoded at a high quantization scale, resulting in visual degradation.

In addition, for the image data for a certain period of time, it can be controlled so that the entire encoded stream has a constant code amount, and good encoding processing is realized in response to changes in characteristics when the scene changes. As a method for doing this, for example, a technique disclosed in Patent Document 2 below is known. In Patent Document 2, provisional encoding of an image signal is performed to obtain a provisional code amount, and a target transfer rate is set so that the total sum of the provisional code amounts becomes a predetermined value. A method for improving control is disclosed. In addition, when transcoding (image stream conversion) of an image stream to be re-encoded after decoding, the amount of code control is stabilized by analyzing the input image stream at the time of re-encoding. It is also possible to plan.
Japanese Unexamined Patent Publication No. 2000-92497 (paragraphs 0019 to 0025, FIG. 1) JP-A-6-141298 (paragraphs 0015 to 0018, FIG. 1)

  However, when encoding processing is performed in real time or when image stream conversion processing is performed, a highly accurate encoding control method for controlling the code amount to a constant value at a certain timing is required. Furthermore, when the encoding cannot be performed with high accuracy, it is necessary to realize an encoding control method that enables recovery.

  The technique disclosed in Patent Document 2 described above cannot control a predetermined code amount in real time at a certain timing, and it is necessary to control the code amount to a constant value at a certain timing. Under such conditions, it cannot be used as a useful technique.

  In addition, when performing image stream conversion, the method of stabilizing the code amount control by analyzing the input image stream at the time of re-encoding, the re-encoding is performed on the input image stream. If the conditions such as the coding bit rate and the resolution are greatly different, the code amount control is instantaneously disturbed, and as a result, visual image quality degradation may occur.

  In view of the above problems, the present invention makes it possible to realize stable code amount control in an image encoding process, and to produce an encoded image with a small error from a predetermined code amount to be controlled and a good image quality. An object of the present invention is to provide an image encoding device that can be generated.

In order to achieve the above object, according to the present invention, a frame buffer for storing a predetermined number of frames among a plurality of frames to be encoded in an image signal;
Encodes the frame stored in the frame buffer and, upon receiving predetermined instruction information related to the reencode process, re-encodes the frame once encoded Encoding processing means;
A stream buffer for storing the frame after encoding by the encoding processing means;
Controls the number of output frames output from the encoding processing means to the stream buffer, measures errors related to the encoding processing by the encoding processing means, and further relates to the re-encoding processing. A code amount control unit that performs an update process of an encoding control parameter related to the re-encoding process performed in the encoding process unit based on predetermined instruction information;
Encoding control means for instructing the start and end of the encoding process and the timing of the scene switching point;
The re-encoding process based on at least one information of a buffer fullness in the frame buffer, an error measurement result by the code amount control unit, the instruction from the encoding control unit, and a picture type of the frame If the re-encoding process is determined to be performed, predetermined instruction information for performing the re-encoding process is output in units of frames, and the re-encoding process is performed. Re-encoding control means for controlling
An image encoding device is provided.

In order to achieve the above object, according to the present invention, a frame buffer for storing a predetermined number of frames among a plurality of frames to be encoded in an image signal;
Encodes the frame stored in the frame buffer and, upon receiving predetermined instruction information related to the reencode process, re-encodes the frame once encoded Encoding processing means;
A stream buffer for storing the frames after encoding by the encoding processing means for the predetermined number of frames;
Controls the number of output frames output from the encoding processing means to the stream buffer, measures errors related to the encoding processing by the encoding processing means, and further relates to the re-encoding processing. A code amount control unit that performs an update process of an encoding control parameter related to the re-encoding process performed in the encoding process unit based on predetermined instruction information;
Determining whether or not to perform the re-encoding process based on at least one of the buffer fullness in the frame buffer, the error measurement result by the code amount control means, and the picture type of the frame; Re-encoding control means for controlling the re-encoding process by outputting predetermined instruction information for performing the re-encoding process in units of frames when it is determined that the re-encoding process is performed; The
An image encoding device is provided.

  The image encoding device according to the present invention can realize stable code amount control in an image encoding process, and can generate an encoded image with good image quality with a small error from a predetermined code amount to be controlled. It has an effect that it can be generated.

  Hereinafter, first to third embodiments of the present invention will be described with reference to the drawings.

<First Embodiment>
First, a first embodiment of the present invention will be described. FIG. 1 is a block diagram showing an example of an image coding apparatus according to the first embodiment of the present invention. Note that the image encoding device shown in FIG. 1 has components common to the above-described conventional image encoding device (see FIG. 6), and here, description of these common components is provided. Is omitted. Further, the predictive coding circuit 150 in the image coding apparatus shown in FIG. 1 has components that are completely identical to those of the conventional image coding apparatus (see FIG. 6).

  The image encoding apparatus according to the present invention shown in FIG. 1 has an encoding control circuit 401 and a re-encoding control circuit 402 in addition to the conventional image encoding apparatus shown in FIG. In addition to the function of the code amount control circuit 217 shown in FIG. 6, the code amount control circuit 403 further has a function for performing processing according to the re-encoding control information ReEnc from the re-encoding control circuit 402. Have. Further, the input image memory 404 has a delay for rearranging the input image memory (for example, the input image memory 202 in FIG. 6) of the conventional image encoding device in the order of encoding according to the encoding syntax. Compared to having a frame storage capacity of about one minute (two frames in the case of the period M = 3), an additional memory capable of further storing N frames is provided. . Similarly to the input image memory 404, the image stream buffer 405 has a buffer size that can store an image bit stream corresponding to the time of N frames in addition to the amount assumed in the stream buffer of the decoding device. In addition, it is desirable that an image bit stream can be output via the output terminal 219 in a form that matches the simulation of the virtual buffer of the decoding apparatus after the N frames have been accumulated.

  An input image signal (digital image signal) input from the input terminal 201 of the image encoding device is supplied to the input image memory 404. The image signal stored in the input image memory 404 is output from the input image memory 404 to the two-dimensional block conversion circuit 203 in accordance with a request from the predictive encoding circuit 150. Similarly, an image encoding process is performed.

  On the other hand, the re-encoding control circuit 402 outputs the re-encoding control information ReEnc to the predictive encoding circuit 150, so that the predictive encoding circuit 150 updates the input image signal and encodes a new picture. It is configured to be able to control whether to perform processing or to perform processing to re-encode a picture that is currently encoded. Further, control related to erasure of the encoded image encoded in the predictive encoding circuit 150 is also performed by the reencoding control information ReEnc output from the reencoding control circuit 402 to the predictive encoding circuit 150. It is configured to be

  Thus, when the predictive encoding circuit 150 encodes a new picture under the control of the re-encoding control circuit 402, the previously encoded picture is erased from the memory, while the predictive encoding circuit 150 However, when the picture is re-encoded, the information related to the picture is held until the re-encoding is performed. Note that the re-encoding control information ReEnc output from the re-encoding control circuit 402 is also supplied to the code amount control circuit 403 in the same manner as the predictive encoding circuit 150.

  The code amount control circuit 403 controls the bit amount to be encoded in the same manner as the conventional processing, and further, an error Budget-BitUsed between the assumed target code amount Budget and the generated used code amount BitUsed, A difference value QB_Point_max-QB_Point [0] between QB_Point [0] at the start of encoding of one picture and the maximum QB buffer point QB_Point_max at the time of encoding in each macroblock is calculated. Note that the error Budget-BitUsed is a parameter indicating the difference between the target code amount and the actually used code amount, and the difference value QB_Point_max-QB_Point [0] indicates a change in quantization scale (disturbance in image quality). It can be said that it is a parameter.

Then, from this calculation result, control response accuracy information α representing the accuracy related to the encoding process and used as an index as to whether or not to perform re-encoding is calculated, and the calculated control response accuracy information α is re-encoded. Supplied to the control circuit 402. An example of the calculation formula for the control response accuracy information α is as follows, for example.
α = β * abs (Budget-BitUsed) / Budget + γ * (QB_Point_max-QB_Point [0])
Where β and γ are positive constants

  The re-encoding control circuit 402 is configured to control the write pointer of the image stream buffer 405 along with the generation of the re-encoding control information ReEnc described above. Thereby, for example, when the predictive encoding circuit 150 is controlled to encode a new picture, the write pointer updated by the predictive encoding circuit 150 is simultaneously sent to the image stream buffer 405. On the other hand, when control is performed so that the prediction encoding circuit 150 performs re-encoding, at the same time, a pointer at the end of encoding of the previous picture is stored in the image stream buffer 405. Control is performed to return the write pointer to the value.

  As described above, the re-encoding control circuit 402 re-encodes the image signal by generating and outputting the re-encoding control information ReEnc. The re-encoding control circuit 402 generates the re-encoding control information ReEnc, for example, the buffer fullness (the number of stored pictures) of the input image memory 404, the control response accuracy information α from the code amount control circuit 403, This is performed based on a determination result of re-encoding conditions such as trigger information from the encoding control circuit 401.

  Hereinafter, an algorithm for determining whether or not to perform re-encoding in the re-encoding control circuit 402 will be described with reference to FIG. FIG. 2 is a flowchart showing a determination algorithm of the re-encoding control circuit of the image encoding device according to the first embodiment of the present invention. The re-encoding control circuit 402 first checks the buffer fullness of the input image memory 404 (step S1). When the free buffer amount (number of pictures) C = 0 in the input image memory 404 (“Yes” in step S1), the re-encoding control circuit 402 outputs re-encoding control information ReEnc = off (OFF). (Step S6), control is performed so as not to perform the re-encoding process.

  On the other hand, when the free buffer amount C> 0 of the input image memory 404 (“No” in step S1), the trigger information from the encoding control circuit 401 is confirmed (step S2). The encoding control circuit 401 starts and ends the encoding process (for example, the final picture), a scene change point (scene switching point), and a GOP indicating the final picture when the total number of bits is to be fixed in a fixed time unit. (Group of Pictures) can be set. For example, the encoding control circuit 401 outputs timing information for designating a start picture and a final picture of such encoding processing to the re-encoding control circuit 402 using information requesting re-encoding as trigger information.

  When the re-encoding control circuit 402 receives trigger information requesting re-encoding from the encoding control circuit 401 ("Yes" in step S2), the control response accuracy supplied from the code amount control circuit 403 The information α is compared with a predetermined threshold value ThreshA (step S4). When the comparison result is α> ThreshA (“Yes” in step S4), the re-encoding control circuit 402 outputs re-encoding control information ReEnc = on (ON) (step S5). On the other hand, if α ≦ ThreshA (“No” in step S4), the re-encoding control information ReEnc = off is output (step S6).

  When the trigger information is not received from the encoding control circuit 401 (“No” in step S2), the re-encoding control information α is used to compensate for the deterioration of the response accuracy in the normal image encoding process. Is compared with a threshold value ThreshB [T] [C] (> ThreshA) (step S3). Note that the threshold value ThreshB [T] [C] varies depending on the picture type T of the encoded picture and the free buffer amount C of the input image memory 404. For example, in order to give priority to the re-encoding process for the encoding accuracy of the reference frame, ThreshB [T] [C] is set low so that I picture <P picture <B picture, and the input image memory As the free buffer amount C 404 increases, the margin for the re-encoding process increases, so it is desirable to set the threshold value low. As a result, re-encoding is easily performed when there is a margin in the reference frame and the input image memory 404.

  When the comparison result in step S3 is α> ThreshB [T] [C] (“Yes” in step S3), the re-encoding control circuit 402 outputs re-encoding control information ReEnc = ON (step S5). On the other hand, if α ≦ ThreshB [T] [C] (“NO” in step S3), the re-encoding control information ReEnc = OFF is output (step S6).

  Next, an operation of the code amount control circuit 403 when receiving the value (on / off) of the re-encoding control information ReEnc determined as described above and output from the re-encoding control circuit 402 will be described. In the code amount control circuit 403, when the re-encoding control information ReEnc is on, the encoding control parameter update process for performing re-encoding is performed. Hereinafter, an example of the update process of the encoding control parameter in the code amount control circuit 403 will be described.

When receiving the re-encoding control information ReEnc = ON from the re-encoding control circuit 402, the code amount control circuit 403 first calculates the complexity C of the last encoded picture. Specifically, when the average value of the quantization scale of the last encoded picture is AvgQ and the code amount required for this picture is Bits, the complexity C of this picture is C = Bits * AvgQ Calculated. Further, the code amount control circuit 403 is configured to be able to hold the complexity C ₋₁ of the previous same picture type, and is the same as the complexity C of the last encoded picture. A difference value ΔC = abs | CC ₋₁ | from the complexity C ₋₁ of the picture type is calculated, and ΔC is compared with a predetermined threshold value ThreshC.

In the case of ΔC <ThreshC, it is determined that the complexity has not changed greatly in comparison with the previous same picture type, and the previously set target code amount Budget remains unchanged when re-encoding. The target code amount (re-encoding target code amount NewBudget) is set. On the other hand, when C ≧ ThreshC, for example, the re-encoding target code amount NewBudget is newly calculated using the following equation.
NewBudget = {TotalBits * C} / {Fnum (I) * C (I) + Fnum (P) * C (P) + Fnum (B) * C (B)}
Where T is the picture type

In this way, the re-encoding target code amount NewBudget set in the code amount control circuit 403 is supplied to the predictive encoding circuit 150, and the re-encoding code amount control based on the re-encoding target code amount NewBudget is performed. Done. Further, the initial point of the QB buffer when performing the re-encoding process is updated in the same manner. The initial point of the QB buffer is obtained by converting the assumed initial quantization scale value Qini using the buffer point conversion table QBTable [Qini]. The Qini calculation method is as follows, for example.
Qini = Bits * AvgQ / NewBudget

  As described above, in the above-described first embodiment of the present invention, when encoding an image signal, an empty buffer amount of the input image memory is provided with a margin (the number of frames). ) And the control accuracy of the encoding process, it is determined whether or not to perform the re-encoding process (two-pass process) for each frame, and the encoding process / re-encoding process is performed based on the determination result. It is configured. In other words, when there is a sufficient free buffer amount in the input image memory, for example, when the start and end of the encoding process, a scene switching point, etc. are specified, the control stability of the code amount control (target code amount and use Whether or not to re-encode pictures that meet these conditions is determined according to various conditions such as when the deviation from the code amount is large or when the quantization scale is large) When re-encoding is performed, the encoding control parameter is updated (target code amount and quantization scale are reset) based on the encoding information once encoded. As a result, it is possible to realize stable code amount control in the image encoding process, and it is possible to generate an encoded image having a small error from the predetermined code amount to be controlled and having good image quality. .

<Second Embodiment>
Next, a second embodiment of the present invention will be described. FIG. 3 is a block diagram showing an example of an image coding apparatus according to the second embodiment of the present invention. 3 is provided with an HDD (Hard Disk Drive) 500 as an input source of an input image signal and an output destination of an output image bitstream in the image encoding device shown in FIG. It has a feature in that. In FIG. 3, the supply source of the input image signal and the output destination of the output image bit stream are the same HDD 500, but different HDDs 500 may be used. Further, instead of the HDD 500, other data storing / writing means capable of storing and writing digital image data can be used.

  The HDD 500 is an example of data storage / writing means capable of storing and writing digital image data. The image encoding apparatus can read the digital image data stored in the HDD 500 in non-real time. The output image bit stream finally output from the image stream buffer 405 can be written to the HDD 500.

  The image coding apparatus shown in FIG. 1 has a system configuration that realizes constant speed processing for input / output and internal coding processing at a speed higher than 1 × speed, whereas the image coding apparatus shown in FIG. The apparatus has a system configuration that realizes processing at a speed higher than 1 × speed for input / output / internal encoding processing. In the second embodiment, the input frame speed depends on the reading performance from the HDD 500.

  Next, an overview of the image encoding device shown in FIG. 3 will be described. In the input image memory 404, an N-frame additional memory is prepared as in the first embodiment. It is desirable that the encoding process is started when the input image signal read from the HDD 500 is stored in the input image memory 404 for N-1 frames. At the start of this encoding process, the fullness of the frame memory is supplied from the input image memory 404 to the re-encoding control circuit 402. Note that the frame memory sufficiency determination method in the re-encoding control circuit 402 is the same as that in the first embodiment described above. However, at the start of the encoding process, C = In contrast to N, C = 1 in the second embodiment.

  In the image encoding device shown in FIG. 3, the input image signal reading speed (input image signal input cycle) from the HDD 500 and the speed related to the encoding process can be set independently. Therefore, for example, by controlling the speed related to the encoding process to be higher than the reading speed of the input image signal from the HDD 500, the encoding process can be preceded. The performed surplus time can be assigned to the re-encoding process. As a result, the re-encoding process can be performed in the surplus time generated in the encoding process, and unnecessary re-encoding process can be avoided at the start of the encoding process.

  In addition, when reading of the input image signal from the HDD 500 is slower than the encoding process (this depends on the data transfer speed between the HDD 500 and the image encoding device, for example), the input image memory 404 is used. By starting the encoding process in a state where the frames are stored in the frame, it is possible to avoid unnecessary re-encoding process from being performed at the start of the encoding process. In the second embodiment, since the output of the output image bit stream is also non-real time, the image stream buffer 405 may be prepared with a buffer having a size corresponding to the output speed to the HDD 500. When necessary (for example, when the writing speed to the HDD 500 is slower than the encoding process), control for temporarily stopping the encoding process is performed. Therefore, in the second embodiment, it is not necessary to provide an additional stream buffer for N frames in the image stream buffer 405 as in the first embodiment described above.

  As described above, in the above-described second embodiment of the present invention, the encoding process can be controlled independently of the input cycle of the input image signal. It is possible to allocate the surplus time generated when the encoding process is performed to the re-encoding process. Thereby, it is possible to realize an encoding process in accordance with the speed of the input image signal, and to efficiently perform the re-encoding process.

  Further, whether or not the re-encoding amount control circuit 402 performs the re-encoding process according to, for example, the control specification of the code amount control or the picture designation that requires the accuracy based on the trigger information commanded from the outside. By making the determination, even when the total number of pictures that can be re-encoded is not allocated so much, the re-encoding process can be effectively allocated to a necessary part (necessary picture). Therefore, it is possible to prevent deterioration in image quality due to the fact that a sufficient amount of information cannot be given, and to realize a good encoding process.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. FIG. 4 is a block diagram showing an example of an image stream conversion apparatus according to the third embodiment of the present invention. Note that the image stream conversion apparatus shown in FIG. 4 has components that are common to the above-described conventional image stream conversion apparatus (see FIG. 8), and here, description of these common components is provided. Is omitted.

  4 basically includes the decoding apparatus (decoding apparatus 100) illustrated in FIG. 7 and the image encoding apparatus (encoding apparatus 200) illustrated in FIG. It is a connected structure. That is, the image stream conversion apparatus shown in FIG. 4 has a margin in the free buffer amount of the input image memory when performing the re-encoding process related to the transcoding based on the output frame generated by the decoding process and the encoding information. Therefore, based on the margin (the number of frames) and the control accuracy of the encoding process, it is possible to determine whether or not to perform the re-encoding process (2-pass process) for each frame. An encoding control circuit 31 (corresponding to the encoding control circuit 401 shown in FIG. 1) and a re-encoding control circuit 32 (corresponding to the re-encoding control circuit 402 shown in FIG. 1) are provided.

  With the above configuration, the image stream conversion apparatus shown in FIG. 4 is independent of the input cycle of the image signal generated in the image stream conversion process to the encoding apparatus 200 when transcoding the image stream. The re-encoding process can be controlled, and as a result, the surplus time generated when the re-encoding process related to transcoding is performed in advance can be controlled by stabilizing the code amount control and stream conversion accuracy. It is possible to assign to re-encoding processing for the purpose of improving the above.

  In the first to third embodiments described above, hardware elements such as circuits and schematic blocks are illustrated as examples of components of the image stream conversion apparatus according to the present invention. Similarly to the conventional image stream conversion apparatus, these hardware elements can be realized by software (programs) that can be executed by a computer. In the first to third embodiments described above, re-encoding processing is described in units of one frame. However, if the buffer amount C is sufficiently large, re-encoding in units of two frames or more is performed. Processing can also be realized by the same determination.

  The image encoding device according to the present invention can realize stable code amount control in image encoding processing, and can generate an encoded image having a small error from a predetermined code amount to be controlled and having good image quality. It has the effect of making it possible to generate, and can be applied to moving picture image coding techniques such as MPEG2.

It is a block diagram which shows an example of the image coding apparatus in the 1st Embodiment of this invention. It is a flowchart which shows the determination algorithm of the re-encoding control circuit of the image coding apparatus in the 1st Embodiment of this invention. It is a block diagram which shows an example of the image coding apparatus in the 2nd Embodiment of this invention. It is a block diagram which shows an example of the image stream converter in the 3rd Embodiment of this invention. It is a figure which shows typically the arrangement | sequence of the image at the time of the process in MPEG2 image coding based on the prior art, and an output, (A) is a figure which shows the encoding system used by MPEG2 image coding, ) Is a diagram showing rearrangement of the encoding order at the time of MPEG2 image encoding, and (C) is a diagram showing the stream arrival order and the decoded image output order at the time of MPEG2 image decoding. It is a block diagram which shows an example of the general image coding apparatus which concerns on a prior art. It is a block diagram which shows an example of the general decoding apparatus concerning a prior art. It is a block diagram which shows an example of the general stream conversion recording device based on a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1,102,218,405 Image stream buffer 2,103 Variable length decoding circuit 3,110 Output frame memory 4 Header extraction circuit 5 Syntax evaluation circuit 6 Syntax reconstruction circuit 7 Encoding information generation circuit 8, 11 Encoding information Memory 9 Encoding information conversion circuit 10 Encoding information superposition circuit 12 Encoding information separation circuit 13 Encoding syntax control circuit 15 Macroblock information generation circuit 16, 106, 208 Motion compensation prediction circuit 17, 217, 403 Code amount control circuit 18, 109, 209 Reference image memory 19, 204 Subtractor 20, 205 Orthogonal transformation circuit 31, 401 Encoding control circuit 32, 402 Re-encoding control circuit 100 Decoding device 101, 201 Input terminal 104, 215 Encoding table 105, 212 Inverse quantization circuit 107, 210 Adder 108, 211 Deblock circuit 111, 213 Inverse orthogonal transform circuit 112, 219 Output terminal 150 Predictive coding circuit 200 Coding device 202, 404 Input image memory 203 Two-dimensional block transform circuit 206 Quantization circuit 207 Motion vector detection circuit 214 Coding circuit 216 Multiplexer 500 HDD (Hard disk drive)

Claims (2)

A frame buffer for storing a predetermined number of frames among a plurality of frames to be encoded in an image signal;
Encodes the frame stored in the frame buffer and, upon receiving predetermined instruction information related to the reencode process, re-encodes the frame once encoded Encoding processing means;
A stream buffer for storing the frame after encoding by the encoding processing means;
Controls the number of output frames output from the encoding processing means to the stream buffer, measures errors related to the encoding processing by the encoding processing means, and further relates to the re-encoding processing. A code amount control unit that performs an update process of an encoding control parameter related to the re-encoding process performed in the encoding process unit based on predetermined instruction information;
Encoding control means for instructing the start and end of the encoding process and the timing of the scene switching point;
The re-encoding process based on at least one information of a buffer fullness in the frame buffer, an error measurement result by the code amount control unit, the instruction from the encoding control unit, and a picture type of the frame If the re-encoding process is determined to be performed, predetermined instruction information for performing the re-encoding process is output in units of frames, and the re-encoding process is performed. Re-encoding control means for controlling
Image coding apparatus having the same. A frame buffer for storing a predetermined number of frames among a plurality of frames to be encoded in an image signal;
Encodes the frame stored in the frame buffer and, upon receiving predetermined instruction information related to the reencode process, re-encodes the frame once encoded Encoding processing means;
A stream buffer for storing the frames after encoding by the encoding processing means for the predetermined number of frames;
Controls the number of output frames output from the encoding processing means to the stream buffer, measures errors related to the encoding processing by the encoding processing means, and further relates to the re-encoding processing. A code amount control unit that performs an update process of an encoding control parameter related to the re-encoding process performed in the encoding process unit based on predetermined instruction information;
Determining whether or not to perform the re-encoding process based on at least one of the buffer fullness in the frame buffer, the error measurement result by the code amount control means, and the picture type of the frame; Re-encoding control means for controlling the re-encoding process by outputting predetermined instruction information for performing the re-encoding process in units of frames when it is determined that the re-encoding process is performed; The
Image coding apparatus having the same.

Images