JP2018101907A - Image encoder, method for controlling the same, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To control the amount of encoding while suppressing variations in quantization parameter and maintaining image quality.SOLUTION: An image encoder comprises: a quantization part that quantizes image data of a frame of interest in moving image data in a preset block unit according to a quantization parameter determined according to a target encoding amount; an encoding part that encodes the data after quantization obtained in the quantization part in the block unit; and an encoding amount control part that controls the encoding part so that the amount of encoding performed by the encoding part becomes a set block target encoding amount. The encoding amount control part obtains the degree of correlation between an image of the frame of interest and an image of a frame immediately before the frame of interest, and sets the block target encoding amount on the basis of the degree of correlation.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a moving image data encoding technique.

  Currently, digital devices that record moving images, such as digital video cameras, are in widespread use. In such a digital device, moving image data is compressed and encoded so that a moving image for a predetermined time can be recorded on a predetermined recording medium.

  H.264 (H.264 / MPEG-4 Part 10: Advanced Video Coding) is known as a typical conventional compression coding method. Such a compression encoding method compresses the amount of data using temporal redundancy and spatial redundancy of a moving image for each block composed of a predetermined number of pixels within one frame. .

  In the above H.264, motion detection and motion compensation for temporal redundancy, discrete cosine transform (DCT) as frequency transform for spatial redundancy, and compression coding by combining techniques such as quantization and entropy coding Has been realized.

  However, if the compression rate is increased to a certain level or more, block distortion peculiar to DCT conversion becomes remarkable and image degradation becomes conspicuous. Therefore, in JPEG2000, etc., discrete wavelet transform (Discrete Wavelet Transform; which is decomposed into frequency bands called multiple subbands by applying low-pass filtering and high-pass filtering in the horizontal and vertical directions respectively as frequency conversion. DWT) is adopted. Subband coding using DWT has characteristics such that block distortion is less likely to occur and compression characteristics at high compression are better than coding techniques using DCT.

  In general code amount control, a target code amount of a frame to be encoded next is determined based on a generated code amount of a frame that has been encoded. In order to converge the generated code amount to the target code amount per frame, the control of the code amount is performed by performing the quantization control for changing the quantization parameter Qp used for quantization for each predetermined region of the image. Do. Note that Qp is a parameter that can reduce the amount of code as the value increases. On the other hand, Qp is preferably as small as possible and constant in the screen because it causes image quality degradation. It is possible to compress image data to a desired code amount by quantization control.

  As described above, Qp is preferably constant in the screen from the viewpoint of image quality. In order to realize this, it is necessary to set a target code amount such that Qp is constant in the screen for each predetermined region of the image. Since the generated code amount becomes smaller as Qp is larger, the target code amount in the region where the generated code amount is difficult in the image is large, and the target code amount in the region where the generated code amount is not easily generated is allocated to be small in the screen. Thus, it is possible to make Qp constant. However, since the real-time property is required for the compression encoding of the moving image, the difficulty in the image needs to be fed back from the image one frame before.

  However, it is necessary to compress and encode moving images in real time. For this reason, for example, if a scene change occurs, such as when a flash is fired during video recording, the code amount control is performed with a set value on the assumption that the scene has not changed, so the target code amount per frame is greatly exceeded. There is a possibility that the generated code amount will be output. Therefore, in some cases, it becomes impossible to write to the recording medium.

  Therefore, the amount of generated code is integrated for each predetermined area, and when the amount of generated code is larger than the upper limit, the amount of code controllability can be improved by changing the quantization parameter of the next area so as to suppress the amount of code. A technique to improve is described in Patent Document 1.

  According to Patent Document 1, since it is possible to prevent the generated code amount from being a predetermined value or more, it is possible to avoid the breakdown that cannot be written on the recording medium.

Japanese Patent No. 5451487

  However, the technique described in Patent Document 1 uniquely determines the upper limit value of the generated code amount regardless of the image, and thus changes the quantization parameter even in an image that does not require suppression of the generated code amount. May end up.

  For example, consider the case where the upper part of the image is extremely difficult and the code amount is generated, and the lower part is extremely easy and the code amount is not generated. Normally, encoding is performed when compression encoding is performed in the order of raster scanning from the top to the bottom of the image. In the case of the above image, when the upper limit value of the generated code amount is uniquely determined, the quantization parameter is set from the middle of the image. It may change and cause image quality degradation.

  In view of the above problems, the present invention provides a technique for controlling the amount of code while maintaining the image quality by suppressing the fluctuation of the quantization parameter.

In order to solve this problem, for example, an image encoding device of the present invention has the following configuration. That is,
An image encoding device that encodes moving image data captured by an imaging means,
Quantizing means for quantizing the image data of the frame of interest in the moving image data in a preset block unit according to a quantization parameter determined according to a target code amount;
Encoding means for encoding the quantized data obtained by the quantization means in units of the blocks;
Code amount control means for controlling the encoding means so that the code amount generated by the encoding means becomes a set block target code amount;
The code amount control means obtains the degree of correlation between the image of the frame of interest and the image of the immediately preceding frame, and sets the block target code amount based on the degree of correlation.

  According to the present invention, it is possible to control the code amount while maintaining the image quality by suppressing the variation of the quantization parameter.

The block block diagram of the image coding apparatus which concerns on 1st Embodiment. The figure which shows the input order of the pixel data which concern on 1st Embodiment. The figure which shows the moving image pattern A and B of the scene change which concerns on 1st Embodiment. The figure for demonstrating transition of the line unit target code amount before and behind the scene change which concerns on 1st Embodiment. The figure for demonstrating the transition of the target code amount of the line unit in the pattern A at the time of setting the code amount upper limit which concerns on 1st Embodiment, and transition of Qp. The figure for demonstrating transition of the target code amount of a line unit in case the code amount upper limit is not set in the pattern B which concerns on 1st Embodiment, and transition of Qp. The figure for demonstrating transition of the target code amount of a line unit at the time of setting the code amount upper limit in the pattern B which concerns on 1st Embodiment, and transition of Qp. The processing flowchart of the block target code amount change part which concerns on 1st Embodiment. The figure for demonstrating the difference in the inclination of the code amount upper limit according to the image correlation degree which concerns on 1st Embodiment. The figure which shows the moving image pattern explaining the effect which concerns on 1st Embodiment. FIG. 11 is a diagram for explaining a transition of a target code amount in units of lines and a transition of Qp for each frame in the moving image pattern of FIG. 10 according to the first embodiment. The block block diagram of the image coding apparatus which concerns on 2nd Embodiment. The figure for demonstrating the subband formation which performed discrete wavelet transformation which concerns on 2nd Embodiment 3 times each horizontally and vertically. The block block diagram of the image coding apparatus which concerns on 3rd Embodiment. The block block diagram of the image coding apparatus which concerns on 5th Embodiment. The figure which shows the relationship of the same pixel position with respect to the image before discrete wavelet transformation of each subband at the time of performing discrete wavelet transformation 3 times.

  Preferred embodiments of the present invention will be described below with reference to the drawings. Note that the image encoding apparatus described in the embodiment will be described as being mounted on an imaging apparatus typified by a digital video camera. Therefore, the encoding target is image data obtained by the imaging unit imaging at a predetermined frame rate.

[First Embodiment]
FIG. 1 is a block configuration diagram of an image encoding device according to the first embodiment. FIG. 2 is a diagram illustrating that the image data input to the image encoding unit 100 in FIG. 1 is in raster scan order. As shown in FIG. 2, the image data is supplied to the image encoding unit 100 in a raster order in units of pixel lines, and is compressed and encoded. In the present embodiment, an example will be described in which the quantization parameter is changed (updated) in units of one pixel line and the code amount control is performed. Note that the image data may be a pixel value itself input to the imaging apparatus or data obtained by performing some processing on the pixel value.

  The outline of the encoding process in the image encoding apparatus according to this embodiment will be described below with reference to FIG.

  The image encoding unit 100 performs code amount control by the code amount control unit 101. The code amount control unit 101 includes a quantization parameter serving as a reference of an image set by the quantization setting unit 102, an image target code amount of the image set by the target code amount setting unit 103, and a complexity setting unit 111. The image data is compression-encoded according to the image complexity set in step (1). Note that a quantization parameter that is set by the quantization setting unit 102 and serves as a reference of an image is simply referred to as a reference quantization parameter.

  The quantization unit 104 uses the quantization parameter Qp output from the quantization control unit 105 and quantizes input image data for each pixel. As the value of Qp increases, the amount of code can be reduced. On the other hand, as the value of Qp increases, image quality deterioration becomes more significant. Therefore, Qp can also be said to be an index value representing the degree of image degradation.

  The encoding unit 106 encodes the image data quantized by the quantization unit 104 and outputs the generated encoded data. At this time, the amount of generated encoded data (code amount) is held in the generated code amount holding unit 107.

  The quantization parameter holding unit 108 holds a reference quantization parameter Qpbase and a quantization parameter Qp [i] determined by the quantization control unit 105 and used for a line to be encoded next. Hereinafter, the variable i indicates the line number to be encoded, and Qp [i] means Qp of the i-th line of the frame.

  The complexity calculation unit 109 calculates the complexity X [i-1] of the previous line for the i-th line to be encoded next (details will be described later).

  The target code amount calculation unit 110 includes the reference target code amount Tn of the image set by the target code amount setting unit 103, the image complexity Xn-1 set by the complexity setting unit 111, and the complexity X [i− 1], the target code amount T [i] for the i-th line to be encoded next is calculated. Here, n represents the encoding target image number, for example, Tn represents the image target code amount of the nth frame, and the image complexity Xn-1 is the complexity of the image immediately before the target image to be encoded. Represents degrees.

  The target code amount changing unit 112, when a certain condition is satisfied from the generated code amount up to the encoded line and the image correlation degree of the correlation determining unit 115, the target code amount calculated by the target code amount calculating unit 110 The amount is changed (details will be described later).

  The difference calculating unit 113, for each pixel line, the line generated code amount S [i-1] held by the generated code amount holding unit 107 and the target code amount T ′ [i output from the target code amount changing unit 113. ] Is calculated, and the calculation result is output to the difference holding unit 114. The difference holding unit 114 holds the integrated difference value ΣE [i−1] from the first line of the image.

  The quantization control unit 105 uses the integrated difference value ΣE [i−1] held by the difference holding unit 114 and the reference quantization parameter Qpbase held by the quantization parameter holding unit 108 to quantize the next line. The parameter Qp [i] is determined. The quantization parameter is controlled such that the generated code amount integrated amount up to the line of the image approaches the target code amount integrated amount.

The quantization control unit 105 calculates the quantization parameter Qp [i] of the line to be encoded next so that the absolute value of the integrated difference value ΣE [i−1] becomes small. The detailed calculation method will be described later.
The code amount is controlled as described above.

<Complexity calculation>
The complexity is an index indicating the degree of difficulty, and is a value that increases as the image becomes more complex and decreases as the image becomes flatter.

  The complexity calculation unit 109 calculates the complexity of the previously encoded line in order to calculate the target code amount of the line of interest.

Using the quantization parameter Qp [i-1] used for encoding one line before and the generated code amount S [i-1], the complexity X [i-1] one line before is Calculated according to equation (1).
X [i-1] = Qp [i-1] × S [i-1] (1)

<Calculation of target code amount>
By assigning a larger target code amount to a difficult line in an image and assigning a smaller target code amount to an easy line, the variation in Qp, which means the degree of image quality degradation, can be minimized within the screen. Therefore, the target code amount calculation unit 110 in the embodiment includes the reference target code amount Tn of the image set by the target code amount setting unit 103, the image complexity Xn-1 [i] set by the complexity setting unit 111, and Then, according to the complexity X [i-1], the target code amount T [i] of the target line to be encoded next is calculated. The calculation formula of the target code amount is as the following formula (2).
T [i] = Tn × X [i-1] / Xn-1 [i] (2)
Since the complexity is a feedback amount that cannot be calculated unless it is encoded as in equation (1), the target code amount of the next line to be encoded is the complexity of the previous line and the i-th line of the previous frame. Calculate complexity feedback.

<Quantization control>
One of the quantization parameter calculation methods is a known technique shown in MPEG2 Test Model 5. With reference to this Test Model 5, in the embodiment, the integrated difference value ΣE [i−1] held by the difference holding unit 114 and the reference quantization parameter Qpbase held by the quantization parameter holding unit 108 are used to The line quantization parameter Qp [i] is calculated as the following equation (3).
Qp [i] = Qpbase + r × ΣE [i−1] (3)
In the equation, r represents the control sensitivity, and the larger the parameter, the sharper the Qp [i] is changed in accordance with the integrated difference value, thereby improving the controllability of the code amount.

  By using Test Model 5, it is possible to set the quantization parameter to be larger if the generated code amount is larger than the target code amount, and to control the code amount by setting the quantization parameter to be smaller if it is smaller.

<Operation during scene change>
The scene change in the present embodiment indicates a relationship between frames in which the image difficulty level of the frame of interest deviates from the image one frame before. For example, when a moving image is taken in the dark and a flash is cooked in the Nth frame. The previous N-1 frame is an image in which the object cannot be visually recognized in the dark, whereas the N frame is an image in which the object can be visually recognized by flash, and the image difficulty of the N frame is higher than that of the N-1 frame. High enough.

  Such scene changes in moving images frequently occur during normal shooting. Therefore, the operation of the code amount control unit 101 of the embodiment at the time of a scene change will be described below.

  The code amount control unit 101 performs code amount control according to the set values of the quantization setting unit 102, the target code amount setting unit 103, and the complexity setting unit 111 as described above. In compression encoding of moving images, generally, the set value uses a feedback amount for real-time processing. Also, within the code amount control unit 101, as shown in Expression (2), the feedback amount is used to calculate the target code amount of the target line to be encoded. From the above, when a scene change occurs, there is a problem that the code amount controllability deteriorates.

  Here, the problem at the time of a scene change is demonstrated. FIGS. 3A and 3B show two types of moving image patterns A and B used for explanation of operations at the time of a scene change. FIG. 4 shows the transition of the target code amount in line units before and after the scene change in the moving image pattern A.

  As can be seen from FIG. 4, when an object suddenly enters the image as in pattern A of FIG. 3A, Xn-1 shown in equation (2) is set to the value of the image before the scene change. Therefore, the target code amount integrated amount becomes equal to or larger than the screen target code amount, and the generated code amount is generated largely following the target code amount. For simplicity, it is assumed that the line generation code amount matches the line target code amount. Therefore, in order to converge the code amount, it is effective to set an upper limit value of the code amount.

  FIG. 5A shows the transition of the target code amount in units of lines in the pattern A of FIG. 3A when the code amount upper limit value is set, and FIG. 5B shows the transition of Qp. As shown in FIGS. 5A and 5B, by setting the target code amount in units of lines after reaching the code amount upper limit value so as to become the code amount upper limit value, the generated code amount becomes the screen target. The code amount can be converged. However, Qp increases in order to suppress the generated code amount.

  Next, a problem caused by setting the code amount upper limit value will be described. 6 (a), 6 (b), and 7 (a) and 7 (b) show the transition of the target code amount in units of lines and the transition of Qp in the pattern B of Fig. 3 (b). 7B shows transitions when the code amount upper limit value is not set, and FIGS. 7A and 7B show transitions when the code amount upper limit value is set.

  As shown in FIGS. 6A and 6B, if the code amount upper limit value is not set, the pattern B has no scene change and the feedback amount used for the code amount control is correct. Converge to.

  On the other hand, when the code amount upper limit value is set as in the case of the pattern A, as shown in FIGS. 7A and 7B, there is a possibility that Qp may fluctuate despite no scene change. is there.

  Looking at the pattern B in FIG. 3B, since the objects are concentrated on the upper part of the screen, as shown in FIGS. 6A and 6B, the line target code amount is set large in the first half line. It is desirable to do. However, since a large amount of target code is allocated to the first half line, the generated code amount exceeds the code amount upper limit value, and Qp is excessively increased in the subsequent lines. cause. From the above, it can be understood that it is not preferable to set the code amount upper limit value fixedly regardless of the image.

<Target code amount changing unit>
For the above reason, the target code amount changing unit 112 in the embodiment makes the code amount controllability and the suppression of image quality degradation compatible by adaptively changing the code amount upper limit value according to the degree of image correlation. Specifically, the target code amount changing unit 112 changes the target code amount Tn serving as a reference per line in the equation (2) shown above according to the degree of scene change.

  The correlation degree determination unit 115 obtains the total value of the luminance components of the target frame to be encoded. Correlation degree determination section 115 compares the luminance total value of the frame of interest with the luminance total value of the immediately previous frame before encoding the frame of interest, and determines the image correlation degree. The image correlation degree in the embodiment is closer to 1 as the difference between the luminance total values of the frame of interest and the immediately preceding frame is smaller, and closer to 0 as the difference is larger. For this reason, the image encoding unit 100 has a buffer 116 for delaying the moving image to be encoded by one frame, and the correlation degree determination unit 115 inputs the frame from the imaging unit without delay. Prior to frame coding, the luminance total value of the frame of interest is calculated. Note that, from the above description, the correlation degree determination unit 115 includes a storage unit (memory or register) that holds the luminance total value of the immediately preceding frame.

  FIG. 8 shows a processing flow per frame of the target code amount changing unit 112. Hereinafter, processing of the target code amount changing unit 112 of the embodiment will be described with reference to FIG.

  In S801, the target code amount changing unit 112 acquires, from the correlation degree determining unit 115, the image correlation degree between the target frame to be encoded and the frame immediately before the encoding is completed.

  In S802, the target code amount changing unit 112 applies the inclination of the line segment representing the code amount upper limit described above to the target frame so that the inclination of the line segment is small when the image correlation degree is large and is large when the image correlation degree is small. To set.

  Since the number of lines whose code amount can be controlled decreases as the line is just before the end of encoding, the code amount upper limit value is smaller as the first line is closer to the last line. In the present embodiment, the code amount upper limit value of the final line is made to coincide with the image target code amount, but may be set to a value larger than the image target code amount from the viewpoint of image quality.

  In S803, the target code amount changing unit 112 branches to S804 if it is not the last line. In S804, the target code amount changing unit 112 branches to S805 if the generated code amount integrated amount is larger than the code amount upper limit value, and otherwise branches to S803. In S805, the target code amount changing unit 112 changes the target code amount of the target line so that the target code amount integrated amount becomes the code amount upper limit value. With the above processing, even when the feedback amount is not appropriate due to a scene change or the like, the generated code amount can be brought close to the screen target code amount.

<Effect>
FIG. 9 shows the difference in the slope of the code amount upper limit value according to the image correlation degree. In the coordinate space shown in FIG. 9, the horizontal axis indicates the position of the line, and the vertical axis indicates the total code amount. As shown in FIG. 9, the code amount upper limit value is set so that the target code amount in units of lines is easily changed from the early stage as the image correlation degree is small, and is difficult to change as the image correlation degree is large.

  In the figure, each line segment represented by the code amount upper limit value matches the target code amount of the frame at the last line position, and the difference is its inclination. Note that the target code amount with the last line position may be different for each line segment. In this case, the inclination and intercept may be set in advance.

  The target code amount changing unit 112 approaches “1” as the difference between the luminance total values of the frame of interest and the immediately preceding frame is smaller (the degree of scene change is smaller), and as the difference is larger (the degree of scene change is larger). It was explained that the degree of correlation approaching “0” is obtained. The target code amount changing unit 112 uses this degree of correlation as the gradient of the code amount upper limit value in FIG.

  Then, the target code amount changing unit 112 in the embodiment determines the target code amount of the i-th line to be encoded next according to Equation (2) described above, but from the head to the immediately preceding i-th line. When the total of the actual code amount exceeds the line segment of the code amount upper limit value set for the scene, T [i] is changed to conform to the set code amount upper limit value.

  FIG. 10 shows a moving image pattern C for explaining the effect of carrying out this embodiment. FIG. 11 shows the transition of the target code amount in units of lines and the transition of Qp in the moving image pattern C for each frame.

  In the N frame and N + 1 frame, since no scene change has occurred and is stable, the code amount upper limit value is set high, the generated code amount converges to the image target code amount, and Qp is also constant. Transition to.

  In the N + 2 frame, an object that did not exist in the previous N + 1 frame appears in the upper part of the image. Therefore, if the code amount upper limit is not set, the generated code amount integrated amount is generated more than the image target code amount. To do.

  On the other hand, when the code amount upper limit value is set smaller than the N + 1 frame because the correlation with the N + 1 frame is small, when the generated code amount integrated amount exceeds the code amount upper limit value, the target code amount integrated amount Therefore, the generated code amount integrated amount converges to the image target code amount.

  In the (N + 3) th frame, the lower object disappears from the (N + 2) th frame, and the degree of image correlation is further reduced. For this reason, the code amount upper limit value is set smaller than the (N + 2) th frame. Since the upper part of the N + 3 frame is a complex image, a large amount of generated code occurs in the first half line, and when the code amount upper limit is reached, the target code amount integrated amount is changed to the upper limit value for code, and the image target code Although the generated code amount converges to the amount, Qp increases to suppress the generated code amount.

  The N + 4 frame is an image having a very high image correlation with the N + 3 frame, so the code amount upper limit value is set high. Since the code amount upper limit value is high, the target code amount integrated amount is not changed as in the case of the N + 3 frame, and the generated code amount integrated amount converges to the image target code amount while Qp is constant.

  When the code amount upper limit is constant regardless of the image, the N + 4 frame causes Qp to increase despite the fact that the image has not changed. By changing it accordingly, the change in Qp can be reduced and image quality deterioration can be suppressed except for the frame immediately after the scene change.

  By doing as described above, it is possible to provide an image encoding device that suppresses deterioration in image quality by reducing the change in Qp and has high code amount controllability.

  Note that the correlation degree determination unit may determine the image correlation degree between the frame and the previous frame using another parameter. For example, it is also within the scope of the present invention to determine the degree of image correlation by comparing histograms that are parameters indicating image characteristics.

  In the above-described embodiment, the description has been made assuming that quantization and encoding are performed in units of lines. However, the quantization and coding unit may be a block unit composed of m × n pixels. In this case, the target code amount calculation unit 110 calculates the block target code amount of the target block to be encoded, and the target code amount change unit 112 changes the block target code amount for the target block. For example, quantization may be performed in units of blocks such as 8 × 8 pixels and encoded into the quantized blocks. It can be said that the line of the encoding unit in the above embodiment is a block whose height is one pixel and whose horizontal direction is the number of pixels in the horizontal direction of the image. This point is the same in each embodiment described below.

[Second Embodiment]
FIG. 12 is a block configuration diagram of an image encoding device according to the second embodiment. In the second embodiment, the object to be controlled by the code amount control unit is transform coefficient data obtained by performing discrete wavelet transform on image data, and the code amount control unit 101 is a subband generated by discrete wavelet transform. It is assumed that code amount control is performed every time. Reference numerals 100 to 115 are the same as those in the first embodiment, and a description thereof is omitted.

  In FIG. 12, a discrete wavelet transform unit 1216 transforms image data into a frequency domain signal and outputs the transform coefficient to the quantization unit 104 of the code amount control unit 101 for each subband.

<Discrete wavelet transform>
FIG. 13 shows subbands obtained when vertical and horizontal filtering according to the discrete wavelet transform (DWT) is executed three times.

  In the discrete wavelet transform, the frequency band of the image data is decomposed into a plurality of subbands by filtering each vertically and horizontally. Then, by applying DWT recursively to the low-frequency subband LL generated by this conversion, the decomposition level can be increased, and the granularity of frequency decomposition can be reduced as shown in FIG. Note that “L” and “H” in FIG. 3 indicate a low band and a high band, respectively, and the order indicates a band obtained as a result of performing horizontal filtering on the front side, and a band obtained as a result of performing vertical filtering on the rear side. The number after “Lv” indicates the decomposition level of DWT. It should be noted that the subband LL shown in the figure does not have “Lv” because when the DWT is performed a plurality of times, the subband LL obtained by the immediately preceding DWT becomes the conversion target, and the subband LL is always one. This is because it is not necessary to distinguish at the decomposition level.

  In the code amount control unit 101, image data that is difficult to visually recognize due to human visual characteristics by setting the quantization parameter to be larger for the high frequency sub-band and setting the quantization parameter to be smaller for the low frequency sub-band. It is common to compress the amount of generated code and improve the encoding efficiency as the frequency increases.

<Operation of Code Quantity Control Unit>
The code amount control unit 101 in the second embodiment performs code amount control for each subband generated by the discrete wavelet transform. That is, the quantization setting unit 102 sets the reference quantization parameter for each subband. Further, the target code amount setting unit 103 sets a subband target code amount. The complexity setting unit 111 sets the image complexity for each subband. The code amount control unit 101 performs code amount control for each line (or block) based on the set value for each subband.

  By doing as described above, it is possible to independently control the code amount of each subband, to reduce the change in Qp, to suppress image quality deterioration, and to provide an image encoding device with high code amount controllability. can do.

[Third Embodiment]
FIG. 14 is a block configuration diagram of an image encoding device according to the third embodiment. Reference numerals 100 to 115 in the figure are the same as those in the first embodiment, and a description thereof will be omitted.

  In the third embodiment, the encoding process of the target subband of the target frame is performed as follows.

  When encoding the i-th line of the target subband, the code amount from the first line to the (i-1) -th line, the same decomposition level of the previous frame (encoded), and the head of the same type of subband The difference from the code amount from the line to the (i-1) th line is obtained. When the difference exceeds the code amount difference threshold, the target code amount of the target line (i-th line) to be encoded is changed. Further, the target code amount is corrected in accordance with the degree of image correlation by reducing the threshold values of other subbands in accordance with the number of subbands exceeding the code amount difference threshold.

<Target code amount changing unit>
The generated code amount holding unit 107 holds the generated code amount integrated amount for each subband of the immediately preceding frame. When attention is paid to a certain subband, the generated code amount integrated amount of the subband means the integrated code amount at a predetermined line interval. For example, the line interval (number of blocks) is “8”, and the first line is defined as the first line. In this case, the generated code amount holding unit 107 includes an integrated code amount that is a total amount of encoded data of the first to eighth lines of the target subband, an integrated code amount of the first to 16th lines, and an integrated amount of the first to 24th lines. The code amount is held. Hereinafter, the line held as the integrated code amount is referred to as a code amount comparison line.

  The following is a case where attention is paid to one subband, and it should be noted that, when simply expressed as a line, the line means a line within the attention subband.

The target code amount changing unit 112, when the target line to be encoded is the code amount comparison line + 1 (= 1, 9, 17,...), The code amount generated from the head to the immediately preceding line (hereinafter simply referred to as the code amount comparison line + 1). The current code amount) is compared with the integrated code amount up to the corresponding line in the corresponding subband of the immediately preceding frame (hereinafter simply referred to as the previous code amount). When the current code amount is larger than the previous code amount, the target code amount changing unit 112 uses the target code amount T ′ [i] of the lines after the target line as the generated code amount integrated amount with respect to the target code amount Tn of the subband. The difference of ΣS [i] is changed to a value divided by the number of remaining lines that have not yet been encoded. The changed line target code amount is given by the following equation (4).
T '[i] = (Tn-ΣS [i]) / (itotal-icheck) (4)
Here, icheck is the number of lines up to the line where the line target code amount starts to be changed.

<Image correlation degree change unit>
The image correlation change unit 1417 changes the image correlation set by the correlation determination unit 115 according to the number of subbands whose target code amount has been changed. In the third embodiment, the image correlation degree indicates a code amount difference threshold.

  Since the code amount control unit 101 performs code amount control independently for each subband, there is a possibility that the target code amount of another subband is changed and the target code amount of the target subband is not changed. As the target code amount is changed on the line just before the end of encoding, the amount of change to increase Qp increases to suppress the generated code amount.

  Accordingly, if the accumulated amount of generated code of the frame of interest differs from that of the previous frame in other subbands, it can be determined that the image correlation is not high, so the image correlation changing unit 1417 changes the target code amount. The code amount difference threshold value of each subband is set small according to the number of subbands. As a result, the code amount controllability is improved by changing the target code amount for each line at an early stage after the start of encoding.

  As described above, in the code amount control for each subband, by changing the code amount difference threshold according to the degree of correlation between the images of each subband, the change in Qp is reduced, image quality deterioration is suppressed, and It is possible to provide an image encoding device with high amount controllability.

  In this embodiment, the subbands are divided into eight equal parts and the subband generation code amount integration amount is compared. However, the present invention is not limited to this. Category.

  In addition to the equal division, a configuration in which only a line immediately after the start of encoding is compared finely is also in the category of this embodiment.

  Furthermore, an image encoding apparatus having a code amount control unit having a plurality of cores is also within the scope of the present embodiment, and the correlation degree of the core can be changed including the number of changes in the target code amount by subbands of other cores. Is possible.

[Fourth Embodiment]
In the fourth embodiment, the target code amount of a line is changed by changing the complexity that is a feedback parameter. The apparatus configuration in the fourth embodiment is the same as that in the first embodiment (FIG. 1).

  As shown in equation (2), since the complexity is a parameter that cannot be acquired unless encoding is completed, the target code amount calculation unit 110 uses the complexity Xn-1 of the previous frame. Since the line target code amount is calculated, the target code amount integrated amount may not converge to the target code amount to be controlled by the code amount.

<Target code amount changing unit>
Therefore, the target code amount changing unit 112 changes the target code amount by weighting Xn−1 when changing the target code amount of the target line. The changed target code amount T ′ [i] is shown in the following equation (5). Α (≧ 1) is a weighting coefficient for performing the weighting.
T ′ [i] = Tn × X [i-1] / (Xn-1 × α) (5)
By using the above formula (5), the screen target code amount does not necessarily converge, but by changing the target code amount of the line, the generated code amount integrated amount is suppressed and complicated compared to the case where it is not changed. Since the degree is expressed by the equation (1), the magnitude of Qp can be adjusted according to the value of α. Since the magnitude of Qp can be adjusted, code amount control with higher priority on image quality can be performed.

  By doing as described above, it is possible to provide an image encoding device that arbitrarily adjusts Qp, suppresses image quality deterioration, and has high code amount controllability.

[Fifth Embodiment]
FIG. 15 is a block configuration diagram of an image encoding device according to the fifth embodiment. In the drawing, reference numerals 100 to 115 and 1216 are the same as those in the second embodiment, and thus the description thereof is omitted.

  The subband generated code amount integrating unit 1518 integrates the generated code amount of each subband for each line corresponding to the same pixel position. In the present embodiment, it is assumed that the target code amount for each line is changed at the same time for all subbands, and all subbands are encoded in parallel.

<Subband same pixel position>
FIG. 16 shows the relationship between the same pixel positions in the space represented by the image before the discrete wavelet transform of each subband when vertical and horizontal filtering of the discrete wavelet transform (DWT) is performed three times.

  In DWT, conversion coefficients for one pixel line are generated for two pixel lines of an image before conversion. In addition, since DWT is performed recursively on the subband LL, the conversion coefficient for one line of decomposition level 2 is generated from the conversion coefficient of two pixel lines of decomposition level 1. Even if the decomposition level is increased thereafter, the same relationship is obtained.

  From the above relationship, the N / 2 line of Lv2 and the N / 4 line of Lv3 correspond to the same pixel position with respect to the N line of Lv1 which is the highest range. When the DWT is executed three times, as shown in FIG. 16, the lines that can be regarded as the same pixel position are two lines of Lv2 and one line of Lv3 with respect to four lines of Lv1.

<Target code amount changing unit>
The target code amount changing unit 112 sets the code amount upper limit value as the subband total code amount upper limit value for the generated code amount integrated amount of all the subbands, and the subband generated by the subband generated code amount integrating unit 1518 The generated code amount is compared with the integrated amount. If the subband generation code amount integration amount is larger than the subband total code amount upper limit value, the target code amount T ′ [i] of the target line of each subband is changed using Equation (5).

  In this way, Qp is also changed at the same time for all subbands by changing the target code amount at the same time for all the subbands. Qp is generally set larger as the high frequency component of the image data and smaller as the low frequency component. However, if the target code amount is changed independently of the subband, the above relationship may be lost. For example, when the Qp of the subband HL is much larger than the Qp of the subband LH, only the vertical edge of the image is extremely deteriorated, and the deterioration of the subjective image quality is conspicuous.

  On the other hand, if this embodiment is applied, α in the equation (5) can be changed to a predetermined value all at once, and thus it is possible to suppress extreme deterioration in image quality.

  As described above, by changing the target code amount of each subband from the same pixel position, Qp is also changed from the same pixel position, and the code amount controllability is suppressed while suppressing image quality deterioration. A high image encoding device can be provided.

  Note that it is not always necessary to change the target code amount in all subbands. For example, a configuration in which the subband LL which is the lowest frequency component gives priority to image quality and the target code amount is not changed is also within the scope of the present invention.

  Each embodiment has been described in detail above. However, the present invention is not limited to the specific embodiment, and various modifications and changes can be made within the scope described in the claims. It is also possible to combine all or a plurality of the constituent elements of the above-described embodiment.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 100 ... Image encoding part 101 ... Code amount control part 102 ... Quantization setting part 103 ... Target code amount setting part 104 ... Quantization part 105 ... Quantization control part 106 ... Encoding part 107 ... Generated code amount holding unit, 108 ... quantization parameter holding unit, 109 ... complexity calculating unit, 110 ... target code amount calculating unit, 111 ... complexity setting unit, 112 ... target code amount changing unit, 113 ... difference calculating unit, 114: Difference holding unit, 115 ... Correlation degree determination unit, 116 ... Buffer, 1216 ... Discrete wavelet transform unit, 1417 ... Image correlation degree changing unit, 1518 ... Subband generation code amount integration unit

Claims (11)

An image encoding device that encodes moving image data captured by an imaging means,
Quantizing means for quantizing the image data of the frame of interest in the moving image data in a preset block unit according to a quantization parameter determined according to a target code amount;
Encoding means for encoding the quantized data obtained by the quantization means in units of the blocks;
Code amount control means for controlling the encoding means so that the code amount generated by the encoding means becomes a set block target code amount;
The code amount control means obtains the degree of correlation between the image of the frame of interest and the image of the immediately preceding frame, and sets the block target code amount based on the degree of correlation. The code amount control means includes:
A coordinate represented by a position indicated by a predetermined frame target code amount in a coordinate space having a horizontal axis as an axis from the first block to the last block in one frame and a vertical axis as an axis that combines the integrated amounts of encoded data. First determination means for determining a slope of a line segment representing a code amount upper limit value to pass based on the degree of correlation;
Second determination means for determining the block target code amount of the block of interest based on a preset reference target code amount and the complexity of the encoded block immediately before the block of interest;
When the total code amount from the head block of the target frame to the immediately preceding block exceeds the code amount indicated by the line segment determined by the first determining unit, the block target code amount of the target block Changing means for changing the line to be along the line segment;
The image encoding apparatus according to claim 1, comprising: The first determining means includes
The inclination is determined based on any one of a difference between the luminance value of the frame of interest and the previous frame, a difference of a histogram, or a difference of code amount up to the block of interest. Image coding apparatus. The second determining means includes
The block of interest is the i-th block, the code amount of the encoded data of the immediately preceding i-1th block is S [i-1], and the quantization parameter used when the i-th block is encoded is When defined as Qp [i-1], X [i-1] representing the complexity is obtained by the following equation: X [i-1] = Qp [i-1] × S [i-1]
The image coding apparatus according to claim 2 or 3, wherein The second determining means includes
When the integrated amount of the generated code amount up to the target block reaches the code amount upper limit value at the position of the target block, the block target code amount is determined by correcting to reduce the complexity. The image encoding device according to claim 2 or 3. Furthermore, it has conversion means for performing discrete wavelet transform on the image data of the frame of interest in the moving image data,
The image coding apparatus according to claim 3, wherein the quantization unit quantizes each subband obtained by the conversion unit. The code amount control means includes image correlation degree changing means,
The image correlation degree changing means changes the image correlation degree smaller as the number of subbands exceeding the code amount difference threshold among a plurality of subbands is larger, and larger as the number is smaller. Image coding apparatus. The image coding apparatus according to claim 5, wherein the second determination unit determines the block target code amount each time encoding of a preset number of blocks is completed. The encoding means encodes blocks in the same position in the space represented by the image in all subbands in parallel,
The code amount control means performs quantization control on one or more subbands, determines a block target code amount for each of the one or more subbands, and is generated before the target block in the one or more subbands. The image coding apparatus according to claim 6, wherein the block target code amount in each subband is determined based on the total code amount of the obtained code amount. A control method of an image encoding device that encodes moving image data captured by an imaging means,
A quantization step of quantizing the image data of the frame of interest in the moving image data in a predetermined block unit according to a quantization parameter determined according to a target code amount;
An encoding step for encoding the quantized data obtained in the quantization step in units of the blocks;
A code amount control step for controlling the encoding step so that a code amount generated by the encoding step becomes a set block target code amount,
The code amount control step obtains the degree of correlation between the image of the frame of interest and the image of the immediately preceding frame, and sets the block target code amount based on the degree of correlation. .   A program for causing a computer to execute each step of the method according to claim 10 by being read and executed by the computer.