JP6916618B2 - Image coding device and its control method and program - Google Patents

Description

The present invention relates to a technique for encoding moving image data.

Currently, digital devices such as digital video cameras that record moving images are in widespread use. In such a digital device, the moving image data is compressed and coded so that the moving image for a certain period of time can be recorded on a predetermined recording medium.

H.264 (H.264 / MPEG-4 Part10: Advanced Video Coding) is known as a conventional typical compression coding method. In such a compression coding method, the amount of data is compressed by utilizing the time redundancy and spatial redundancy of the moving image for each block composed of a predetermined number of pixels in one frame. ..

In the above H.264, motion detection and motion compensation for time redundancy, discrete cosine transform (DCT) as frequency conversion for spatial redundancy, and compression code by combining technologies such as quantization and entropy coding. Has been realized.

However, if the compression rate is increased to a certain extent or more, the block distortion peculiar to the DCT transform becomes remarkable and the image deterioration becomes conspicuous. Therefore, in the JPEG2000 system, etc., the discrete wavelet transform (Discrete Wavelet Transform;) that decomposes into multiple frequency bands by applying low-frequency filtering and high-frequency filtering in the horizontal and vertical directions as frequency transforms. DWT) is adopted. Subband coding using DWT has features that block distortion is less likely to occur and compression characteristics at high compression are better than coding technology using DCT.

In general code amount control, the target code amount of the frame to be coded next is determined based on the generated code amount of the frame for which coding is completed. Then, in order to converge the generated code amount to the target code amount per frame, the code amount is controlled by performing quantization control in which the quantization parameter Qp used for quantization is changed for each predetermined region of the image. conduct. The larger the value of Qp, the more the code amount can be reduced. On the other hand, it causes deterioration of image quality. Therefore, it is desirable that Qp is as small as possible and constant in the screen. The quantization control makes it possible to compress the image data to a desired code amount.

As described above, it is desirable that Qp is constant in the screen from the viewpoint of image quality. In order to realize this, it is necessary to set a target code amount so that Qp becomes constant in the screen for each predetermined area of the image. Since the larger the Qp, the smaller the generated code amount, the target code amount in the area where the generated code amount is difficult to be generated is large in the image, and the target code amount in the area where the generated code amount is not easily generated is assigned small in the screen. It is possible to make Qp constant with. However, since real-time performance is required for compression coding of moving images, it is necessary to feed back the difficulty level in the image from the image one frame before.

However, video compression coding needs to be done in real time. For this reason, for example, if a scene change occurs during video recording such that the flash is cooked, the code amount is controlled with the set value assuming that the scene has not changed, which greatly exceeds the target code amount per frame. There is a possibility that the amount of generated code will be output. Therefore, in some cases, it becomes impossible to write to the recording medium.

Therefore, the code amount controllability is improved by integrating the generated code amount for each predetermined area and changing the quantization parameter of the next area so as to suppress the code amount when the generated code amount is larger than the upper limit value. The technique to be improved is described in Patent Document 1.

According to Patent Document 1, since it is possible to prevent the generated code amount from exceeding a predetermined value, it is possible to avoid a bankruptcy that cannot be written on a recording medium.

Japanese Patent No. 5451487

However, the technique described in Patent Document 1 uniquely defines the upper limit of the generated code amount regardless of the image, so that the quantization parameter can be changed even in an image in which the generated code amount does not need to be suppressed. It may end up.

For example, consider the case of an image in which the upper part of the image is extremely difficult to generate a code amount and the lower part is extremely easy to generate a code amount. Normally, coding is performed when compression coding is performed in the order of raster scan from the top to the bottom of the image. Therefore, in the case of the above image, when the upper limit of the generated code amount is uniquely determined, the quantization parameter is from the middle of the image. It may change and cause deterioration of image quality.

In view of the above problems, the present invention provides a technique for controlling the amount of code while suppressing fluctuations in quantization parameters and maintaining image quality.

In order to solve this problem, for example, the image coding apparatus of the present invention has the following configuration. That is,
An image coding device that encodes moving image data captured by an imaging means.
A quantization means for quantizing the image data of the frame of interest in the moving image data in preset block units according to a quantization parameter determined according to a target code amount.
A coding means that encodes the quantized data obtained by the quantization means in block units, and
It has a code amount control means for controlling the quantization means so that the code amount generated by the coding means becomes a set block target code amount.
The code amount control means is
Acquisition means for acquiring a pre-Symbol correlation of the image of the image and the frame immediately before the frame of interest,
In a coordinate space where the axis from the first block to the last block in one frame is the horizontal axis and the axis representing the integrated amount of coded data is the vertical axis, the coordinates represented by the position indicated by the preset frame target code amount are set. It is the first determination means for determining the slope of the line segment representing the upper limit of the amount of code to pass based on the correlation degree. If the correlation degree is large, the slope of the line segment is small, and if the correlation degree is small, the slope is small. A first determining means for determining the inclination of the line segment so that the inclination of the line segment becomes large,
It is a second determination means for determining the block target code amount of the block of interest based on a preset reference target code amount and the complexity of the coded block immediately before the block of interest, and is the focus. When the complexity of the coded block immediately before the block is high, the block target code amount of the block of interest is large, and when the complexity of the coded block immediately before the block of interest is low, the block of interest is said. A second determining means for determining the block target code amount of the block of interest so that the block target code amount becomes small,
When the total code amount from the first block of the focus frame to the immediately preceding block exceeds the code amount indicated by the line segment determined by the first determination means, the block target code amount of the focus block Is characterized by having a changing means for changing the line segment so as to follow the line segment.

According to the present invention, it is possible to control the code amount while suppressing fluctuations in the quantization parameter and maintaining image quality.

The block block diagram of the image coding apparatus which concerns on 1st Embodiment. The figure which shows the input order of the pixel data which concerns on 1st Embodiment. The figure which shows the moving image pattern A, B of the scene change which concerns on 1st Embodiment. The figure for demonstrating the transition of the line unit target code amount before and after the scene change which concerns on 1st Embodiment. The figure for demonstrating the transition of the target code amount in line unit, and the transition of Qp in the pattern A when the code amount upper limit value which concerns on 1st Embodiment is set. The figure for demonstrating the transition of the target code amount in line unit, and the transition of Qp in the pattern B which concerns on 1st Embodiment when the code amount upper limit value is not set. The figure for demonstrating the transition of the target code amount and the transition of Qp in a line unit when the code amount upper limit value is set in the pattern B which concerns on 1st Embodiment. The processing flowchart of the block target code amount change part which concerns on 1st Embodiment. The figure for demonstrating the difference in the inclination of the code amount upper limit value according to the image correlation degree which concerns on 1st Embodiment. The figure which shows the moving image pattern explaining the effect which concerns on 1st Embodiment. The figure for demonstrating the transition of the target code amount and the transition of Qp for each line in the moving image pattern of FIG. 10 which concerns on 1st Embodiment. The block block diagram of the image coding apparatus which concerns on 2nd Embodiment. The figure for demonstrating the subband formation which performed the discrete wavelet transform which concerns on 2nd Embodiment three times each horizontally and vertically. The block block diagram of the image coding apparatus which concerns on 3rd Embodiment. The block block diagram of the image coding apparatus which concerns on 5th Embodiment. The figure which shows the relationship of the same pixel position with respect to the image before the discrete wavelet transform of each subband when the discrete wavelet transform is performed three times.

A preferred embodiment of the present invention will be described below with reference to the drawings. The image coding device described in the embodiment will be described as being mounted on an image pickup device typified by a digital video camera. Therefore, the coding target is the image data obtained by the imaging unit imaging at a predetermined frame rate.

[First Embodiment]
FIG. 1 is a block configuration diagram of an image coding device according to the first embodiment. FIG. 2 is a diagram showing that the image data input to the image coding unit 100 of FIG. 1 is in the raster scan order. As shown in FIG. 2, the image data is supplied to the image coding unit 100 in raster order in pixel line units and is compressed and encoded. In this embodiment, an example in which the quantization parameter is changed (updated) for each pixel line to control the code amount will be described. The image data may be the pixel value itself input to the image pickup apparatus or data obtained by subjecting the pixel value to some processing.

Hereinafter, an outline of the coding process in the image coding apparatus according to the present embodiment will be described with reference to FIG.

The image coding unit 100 controls the code amount by the code amount control unit 101. The code amount control unit 101 includes a quantization parameter that serves as a reference for an image set by the quantization setting unit 102, an image target code amount of the image set by the target code amount setting unit 103, and a complexity setting unit 111. The image data is compressed and encoded according to the image complexity set in. The quantization parameter set by the quantization setting unit 102, which is the reference of the image, is simply referred to as the reference quantization parameter.

The quantization unit 104 uses the quantization parameter Qp output by the quantization control unit 105 to quantize the input image data pixel by pixel. The larger the value of Qp, the more the code amount can be reduced, while the larger the value, the more remarkable the deterioration of image quality. Therefore, Qp can also be said to be an index value representing the degree of image deterioration.

The coding unit 106 encodes the image data quantized by the quantization unit 104 and outputs the generated coded data. At this time, the amount of generated coded data (code amount) is held in the generated code amount holding unit 107.

The quantization parameter holding unit 108 holds the quantization parameter Qp [i] used for the next coding line determined by the reference quantization parameter Qpbase and the quantization control unit 105. Hereinafter, the variable i indicates the coded line number, and Qp [i] means the Qp of the i-th line of the frame.

The complexity calculation unit 109 calculates the complexity X [i-1] of the line immediately before the i-th line to be encoded next (details will be described later).

The target code amount calculation unit 110 has a reference target code amount Tn of the image set by the target code amount setting unit 103, an image complexity Xn-1 set by the complexity setting unit 111, and a complexity level X [i-. According to 1], the target code amount T [i] of the i-th line to be encoded next is calculated. Here, n indicates an image number to be encoded, for example, Tn represents an image target code amount of the nth frame, and image complexity Xn-1 is the complexity of the image immediately before the image of interest to be encoded. Represents the degree.

When a certain condition is satisfied from the generated code amount up to the line where the coding is completed and the image correlation degree of the correlation degree determination unit 115, the target code amount changing unit 112 calculates the target code by the target code amount calculation unit 110. Change the amount (details below).

The difference calculation unit 113 has a line generation code amount S [i-1] held by the generation code amount holding unit 107 and a target code amount T'[i] output from the target code amount change unit 113 for each pixel line. ] Difference E [i-1] is calculated, and the calculation result is output to the difference holding unit 114. The difference holding unit 114 holds the integrated difference value ΣE [i-1] from the first line of the image.

The quantization control unit 105 uses the integrated difference value ΣE [i-1] held by the difference holding unit 114 and the reference quantization parameter Qpbase held by the quantization parameter holding unit 108 to quantum the next line. The quantization parameter Qp [i] is determined. The quantization parameter is controlled so that the integrated code amount generated up to the line of the image approaches the target code amount integrated amount.

The quantization control unit 105 calculates the quantization parameter Qp [i] of the line to be encoded next so that the absolute value of the integrated difference value ΣE [i-1] becomes small. The detailed calculation method will be described later.
The code amount is controlled by the above.

<Calculation of complexity>
The complexity is an index indicating the difficulty level, and is a value that increases as the image becomes more complex and decreases as the image becomes flat.

The complexity calculation unit 109 calculates the complexity of the previously encoded line in order to calculate the target code amount of the line of interest.

Using the quantization parameter Qp [i-1] used for coding one line before and the code amount S [i-1] generated at that time, the complexity X [i-1] one line before is calculated as follows. Calculated by equation (1).
X [i-1] = Qp [i-1] × S [i-1]… (1)

<Calculation of target code amount>
By allocating a larger target code amount to a difficult line in an image and allocating a smaller target code amount to an easy line, the fluctuation of Qp, which means the degree of image quality deterioration, can be minimized on the screen. Therefore, the target code amount calculation unit 110 in the embodiment includes the reference target code amount Tn of the image set by the target code amount setting unit 103, the image complexity Xn-1 [i] set by the complexity setting unit 111, and the image complexity Xn-1 [i]. , The target code amount T [i] of the line of interest to be encoded next is calculated according to the complexity X [i-1]. The formula for calculating the target code amount is as shown in the following formula (2).
T [i] = Tn × X [i-1] / Xn-1 [i]… (2)
Since the complexity is a feedback amount that cannot be calculated without coding as in equation (1), the target code amount of the line to be encoded next is the complexity of one line before and the i-th line of one frame before. Feedback calculation of complexity.

<Quantization control>
As one of the quantization parameter calculation methods, there is a known technique shown in MPEG2 Test Model 5. With reference to this Test Model 5, in the embodiment, the integrated difference value ΣE [i-1] held by the difference holding unit 114 and the reference quantization parameter Qpbase held by the quantization parameter holding unit 108 are used as follows. The quantization parameter Qp [i] of the line of is calculated as the following equation (3).
Qp [i] = Qpbase + r × ΣE [i-1]… (3)
In the equation, r indicates the control sensitivity, and the larger it is, the more steeply the Qp [i] is changed according to the integrated difference value, and the better the controllability of the code amount is.

By using Test Model 5, it is possible to control the code amount by setting the quantization parameter to be large if the generated code amount is large with respect to the target code amount and to set the quantization parameter to be small if it is small.

<Operation at the time of scene change>
The scene change in the present embodiment refers to the relationship between frames in which the image difficulty of the frame of interest deviates from the image one frame before. For example, when a moving image is taken in the dark and the flash is cooked in the Nth frame. In the previous N-1 frame, the image was such that the object could not be seen in the dark, but in the N frame, the object became visible by the flash, and the image difficulty of the N frame was higher than that of the N-1 frame. It will be high enough.

Scene changes in moving images as described above frequently occur during normal shooting. Therefore, the operation of the code amount control unit 101 of the embodiment at the time of a scene change will be described below.

As described above, the code amount control unit 101 controls the code amount according to the set values of the quantization setting unit 102, the target code amount setting unit 103, and the complexity setting unit 111. In the compression coding of moving images, the above set values generally utilize the feedback amount for real-time processing. Further, even inside the code amount control unit 101, as shown in the equation (2), the feedback amount is used to calculate the target code amount of the line of interest to be coded. From the above, when a scene change occurs, there is a problem that the code amount controllability deteriorates.

Here, the problem at the time of scene change will be explained. FIGS. 3A and 3B show two types of moving image patterns A and B used for explaining the operation at the time of a scene change. FIG. 4 shows the transition of the target code amount for each line before and after the scene change in the moving image pattern A.

As can be seen from FIG. 4, when an object suddenly enters the image as in pattern A of FIG. 3A, the value of the image before the scene change is set for Xn-1 shown in the equation (2). Therefore, the integrated amount of the target code amount becomes equal to or larger than the screen target code amount, and the generated code amount is greatly generated following the target code amount. For the sake of simplicity, it is assumed that the line generation code amount matches the line target code amount. Therefore, in order to converge the code amount, it is effective to set the upper limit value of the code amount.

FIG. 5A shows the transition of the target code amount for each line in the pattern A of FIG. 3A when the upper limit value of the code amount is set, and FIG. 5B shows the transition of Qp. As shown in FIGS. 5A and 5B, by setting the target code amount for each line after reaching the code amount upper limit value to be the code amount upper limit value, the generated code amount is the screen target. It can be converged to a code amount. However, Qp is increased in order to suppress the generated code amount.

Next, the problem caused by setting the upper limit of the code amount will be described. 6 (a), 6 (b) and 7 (a, (b) show the transition of the target code amount in line units and the transition of Qp in the pattern B of FIG. 3 (b). (B) shows the transition when the code amount upper limit value is not set, and FIGS. 7 (a) and 7 (b) show the transition when the code amount upper limit value is set.

As shown in FIGS. 6A and 6B, if the upper limit of the code amount is not set, the scene change does not occur in the pattern B and the feedback amount used for the code amount control is correct, so that the screen target code amount is displayed. Converges to.

On the other hand, when the upper limit of the code amount is set as in the case of pattern A, as shown in FIGS. 7A and 7B, there is a possibility that Qp fluctuates even though no scene change has occurred. be.

Looking at the pattern B in FIG. 3 (b), since the objects are concentrated on the upper part of the screen, the line target code amount is set large in the first half of the line as shown in FIGS. 6 (a) and 6 (b). It is desirable to do. However, since a large amount of target code is allocated to the first half of the line, the generated code exceeds the upper limit of the code amount, and Qp is excessively increased in the subsequent lines, resulting in deterioration of image quality due to the increase in Qp. cause. From the above, it can be understood that it is not preferable to set the upper limit of the code amount fixedly regardless of the image.

<Target code amount change unit>
For the above reason, the target code amount changing unit 112 in the embodiment adaptively changes the code amount upper limit value according to the image correlation degree, thereby achieving both code amount controllability and suppression of image quality deterioration. Specifically, the target code amount changing unit 112 changes the target code amount Tn, which is a reference per line in the above-mentioned equation (2), according to the degree of the scene change.

The correlation degree determination unit 115 obtains the total value of the luminance components of the frame of interest to be encoded. Then, the correlation degree determination unit 115 compares the total brightness value of the frame of interest with the total brightness value of the immediately preceding frame before coding the frame of interest, and determines the image correlation degree. The image correlation degree in the embodiment is set so that the smaller the difference between the total luminance values of the frame of interest and the immediately preceding frame, the closer to 1, and the larger the difference, the closer to 0. Therefore, the image coding unit 100 has a buffer 116 for delaying the moving image to be encoded by one frame, and the correlation degree determination unit 115 inputs the frame from the imaging unit without delay and pays attention to it. Prior to the coding of the frame, the total brightness value of the frame of interest is calculated. From the above description, the correlation degree determination unit 115 has a storage unit (memory or register) that holds the total luminance value of the immediately preceding frame.

FIG. 8 shows the processing flow per frame of the target code amount changing unit 112. Hereinafter, the processing of the target code amount changing unit 112 of the embodiment will be described with reference to the figure.

In S801, the target code amount changing unit 112 acquires the image correlation degree between the frame of interest to be encoded and the frame immediately before the completion of coding from the correlation degree determination unit 115.

In S802, the target code amount changing unit 112 sets the slope of the line segment representing the code amount upper limit value described above with respect to the frame of interest so that it becomes smaller when the image correlation is large and increases when the image correlation is small. And set.

Since the number of lines for which the code amount can be controlled decreases as the line near the end of coding decreases, the code amount upper limit value is smaller as the head line and increases as it approaches the final line. In the present embodiment, the upper limit of the code amount of the final line is matched with the image target code amount, but from the viewpoint of image quality, a value larger than the image target code amount may be set.

In S803, the target code amount changing unit 112 branches to S804 if it is not the final line. In S804, the target code amount changing unit 112 branches to S805 if the generated code amount integrated amount is larger than the code amount upper limit value, and branches to S803 otherwise. In S805, the target code amount changing unit 112 changes the target code amount of the line of interest so that the target code amount integrated amount becomes the code amount upper limit value. By the above processing, even when the feedback amount is not appropriate due to a scene change or the like, the generated code amount can be brought close to the screen target code amount.

<Effect>
FIG. 9 shows the difference in the slope of the upper limit of the code amount according to the degree of image correlation. In the coordinate space shown in FIG. 9, the horizontal axis indicates the position of the line, and the vertical axis indicates the total code amount. As shown in FIG. 9, the lower the image correlation degree is, the easier it is to change the target code amount for each line from an early stage, and the larger the image correlation degree is, the more difficult it is to change the code amount upper limit value.

In the figure, each line segment represented by the code amount upper limit value matches the target code amount of the frame at the last line position, and what is different is the slope thereof. The target code amount from the last line position may be different for each line segment. In this case, the slope and intercept may be preset.

The target code amount changing unit 112 approaches "1" as the difference between the total brightness values of the frame of interest and the immediately preceding frame is smaller (the smaller the degree of scene change), and the larger the difference (the greater the degree of scene change). It was explained that the degree of correlation approaching "0" is to be obtained. The target code amount changing unit 112 uses this correlation degree as the slope of the code amount upper limit value in FIG.

Then, the target code amount changing unit 112 in the embodiment determines the target code amount of the i-th line to be encoded next according to the equation (2) described above, from the beginning to the immediately preceding i-1 line. When the total of the actual code amounts of is exceeded the line segment of the code amount upper limit value set for the scene, T [i] is changed so as to follow the set code amount upper limit value.

FIG. 10 shows a moving image pattern C for explaining the effect of implementing the present embodiment. FIG. 11 shows the transition of the target code amount and the transition of Qp for each line in the moving image pattern C for each frame.

In the N frame and N + 1 frame, scene changes have not occurred and are stable, so the upper limit of the code amount is set high, the generated code amount converges to the image target code amount, and the Qp is also constant. Transition to.

In the N + 2 frame, an object that did not exist in the previous N + 1 frame appears at the top of the image, so if the upper limit of the code amount is not set, the generated code amount integration amount will be larger than the image target code amount. do.

On the other hand, since the correlation between the upper limit of the code amount and the N + 1 frame is small, when the upper limit of the code amount is set smaller than the N + 1 frame, the target code amount integration amount is reached when the accumulated code amount exceeds the upper limit value of the code amount. Is changed to the upper limit value of the code amount, so that the integrated amount of the generated code amount converges on the image target code amount.

In the N + 3 frame, the lower object disappears with respect to the N + 2nd frame, and the image correlation becomes even lower. Therefore, the upper limit of the code amount is set smaller than the N + 2nd frame. Since the upper part of the N + 3 frame is a complicated image, a large amount of code is generated in the first half line, and when the code amount upper limit is reached, the target code amount integration amount is changed to the code upper limit, and the image target code. Although the generated code amount converges on the quantity, Qp increases in an attempt to suppress the generated code amount.

In the N + 4 frame, since the image has a very high image correlation with the N + 3 frame, the upper limit of the code amount is set high. Since the upper limit of the code amount is high, the target code amount integration amount is not changed as in the N + 3 frame, and the generated code amount integration amount converges on the image target code amount while the Qp remains constant.

When the upper limit of the code amount is constant regardless of the image, the Qp of the N + 4 frame increases even though the image has not changed, which causes image deterioration, while the upper limit of the code amount is used as the image correlation degree. By changing accordingly, the change in Qp can be made small and the deterioration of image quality can be suppressed except for the frame immediately after the scene change.

By doing so, it is possible to provide an image coding apparatus that can reduce the change in Qp, suppress the deterioration of image quality, and have high code amount controllability.

The correlation degree determination unit may determine the image correlation degree between the frame and the previous frame by using other parameters. For example, it is also within the scope of the present invention to determine the degree of image correlation by comparing histograms, which are parameters indicating the characteristics of images.

Further, in the above embodiment, it has been described that the quantization and coding are performed in line units. However, the quantization and coding unit may be a block unit composed of m × n pixels. In this case, the target code amount calculation unit 110 calculates the block target code amount of the block of interest to be encoded, and the target code amount changing unit 112 changes the block target code amount for the block of interest. For example, it may be quantized in block units such as 8 × 8 pixels and encoded in the quantized block. It can be said that the line of the coding unit in the above embodiment is a block in which the height is one pixel and the horizontal direction is the number of pixels in the horizontal direction of the image. This point is the same in each of the embodiments described below.

[Second Embodiment]
FIG. 12 is a block configuration diagram of the image coding device according to the second embodiment. In the second embodiment, the object to be controlled by the code amount control unit is the conversion coefficient data obtained by discrete wavelet transform on the image data, and the code amount control unit 101 is a subband generated by the discrete wavelet transform. It is assumed that the code amount is controlled every time. Reference numerals 100 to 115 are the same as those in the first embodiment, and the description thereof will be omitted.

In FIG. 12, the discrete wavelet transform unit 1216 converts the image data into a signal in the frequency domain, and outputs the conversion coefficient to the quantization unit 104 of the code amount control unit 101 for each subband.

<Discrete wavelet transform>
FIG. 13 shows the subbands obtained when the vertical and horizontal filtering related to the discrete wavelet transform (DWT) is executed three times each.

The discrete wavelet transform decomposes the frequency band of image data into a plurality of subbands by filtering vertically and horizontally respectively. Then, by recursively applying DWT to the low frequency subband LL generated by this conversion, the decomposition level can be increased, and as shown in FIG. 3, the particle size of frequency decomposition can be made finer. In addition, "L" and "H" in FIG. 3 indicate a low region and a high region, respectively, and the order thereof indicates a band resulting from horizontal filtering on the front side and a band resulting from vertical filtering on the rear side. The number after "Lv" indicates the decomposition level of DWT. The reason why "Lv" is not attached to the illustrated sub-band LL is that when the DWT is performed a plurality of times, the sub-band LL obtained in the immediately preceding DWT is the conversion target, and the sub-band LL is always one. This is because it is not necessary to distinguish by the decomposition level.

In the code amount control unit 101, the quantization parameter is set larger for the high frequency subband and the quantization parameter is set smaller for the low frequency subband, and the code amount control is performed, so that the image data is difficult to see due to human visual characteristics. Generally, the higher the frequency of, the more the generated code amount is compressed to improve the coding efficiency.

<Operation of code amount control unit>
The code amount control unit 101 in the second embodiment performs code amount control for each subband generated by the discrete wavelet transform. That is, the quantization setting unit 102 sets the reference quantization parameter of each subband. Further, the target code amount setting unit 103 sets the subband target code amount. Then, the complexity setting unit 111 sets the image complexity for each subband. Then, the code amount control unit 101 controls the code amount for each line (or block) based on the above set value for each subband.

By doing so, it is possible to independently control the code amount of each subband, provide an image coding device that reduces the change in Qp, suppresses image quality deterioration, and has high code amount controllability. can do.

[Third Embodiment]
FIG. 14 is a block configuration diagram of the image coding device according to the third embodiment. Reference numerals 100 to 115 in the figure are the same as those in the first embodiment, and the description thereof will be omitted.

In the third embodiment, the coding process of the focused subband of the focused frame is performed as follows.

When encoding the i-th line of the sub-band of interest, the amount of coding from the first line to the i-1 line, the same decomposition level of the previous frame (encoded), and the beginning of the same type of sub-band The difference from the code amount from the line to the i-1th line is obtained. Then, when the difference exceeds the code amount difference threshold value, the target code amount of the line of interest (the i-th line) to be encoded is changed. Further, the target code amount is corrected according to the image correlation degree by reducing the threshold value of the other subbands according to the number of subbands exceeding the code amount difference threshold value.

<Target code amount change unit>
The generated code amount holding unit 107 holds the generated code amount integrated amount for each subband of the immediately preceding frame. When focusing on one sub-band, the integrated code amount of the generated code amount of the sub-band means the integrated code amount at a predetermined line interval. For example, the line interval (number of blocks) is set to "8", and the first line is defined as the first line. In this case, the generated code amount holding unit 107 integrates the integrated code amount, which is the total amount of the coded data of the 1st to 8th lines of the subband of interest, the integrated code amount of the 1st to 16th lines, and the integrated code amount of the 1st to 24th lines. Holds the code amount, ... Hereinafter, the line held as the integrated code amount is referred to as a code amount comparison line.

It should be noted that the following is the case of focusing on one subband, and when it is simply expressed as a line, that line means a line in the subband of interest.

The target code amount changing unit 112 has a code amount generated from the beginning to the immediately preceding line when the line of interest to be encoded becomes the code amount comparison line +1 (= 1, 9, 17 ...) (hereinafter, simply The current code amount) and the code amount integration amount up to the corresponding line in the corresponding subband of the immediately preceding frame (hereinafter, simply referred to as the previous code amount) are compared. Then, when the current code amount is larger than the previous code amount, the target code amount changing unit 112 sets the target code amount T'[i] of the line after the line of interest to the generated code amount integration amount with respect to the target code amount Tn of the subband. The difference in ΣS [i] is changed to a value divided by the number of remaining lines that have not yet been encoded. The changed line target code amount is given by the following equation (4).
T'[i] = (Tn-ΣS [i]) / (itotal-icheck)… (4)
Here, icheck is the number of lines up to the line where the line target code amount starts to be changed.

<Image correlation change part>
The image correlation degree changing unit 1417 changes the image correlation degree set by the correlation degree determination unit 115 according to the number of subbands in which the target code amount has been changed. In the third embodiment, the image correlation degree indicates a code amount difference threshold value.

Since the code amount control unit 101 controls the code amount independently for each subband, there is a possibility that the target code amount of the other subband is changed and the target code amount of the subband of interest is not changed. As the target code amount is changed on the line just before the end of coding, the amount of change that increases Qp in order to suppress the generated code amount increases.

Therefore, if the integrated amount of generated code amount of the frame of interest is different from that of the frame of interest in the other subbands, it can be determined that the image correlation degree is not high. Therefore, the target code amount of the image correlation degree changing unit 1417 is changed. The code amount difference threshold of each subband is set small according to the number of subbands. As a result, the code amount controllability is improved by changing the target code amount for each line at an early stage after the start of coding.

As described above, in the code amount control for each subband, by changing the code amount difference threshold value according to the degree of correlation of the images of each subband, the change in Qp is reduced, the image quality deterioration is suppressed, and the code An image coding apparatus having high quantity controllability can be provided.

In the present embodiment, the sub-bands are divided into eight equal parts and the amount of integrated code amount generated by the sub-bands is compared. It is a category.

Further, not only the equal division but also the configuration in which only the lines immediately after the start of coding are compared in detail is also included in the scope of the present embodiment.

Further, an image coding device in which the code amount control unit has a plurality of cores is also within the scope of the present embodiment, and the correlation degree of the core can be changed including the number of subbands of other cores changing the target code amount. It is possible.

[Fourth Embodiment]
In the fourth embodiment, the target code amount of the line is changed by changing the complexity which is a feedback parameter. The apparatus configuration in the fourth embodiment is the same as that in the first embodiment (FIG. 1).

As shown in the equation (2), since the complexity is a parameter that cannot be acquired until the coding is completed, the target code amount calculation unit 110 uses the complexity Xn-1 of the previous frame. Since the line target code amount is calculated, the target code amount integrated amount may not converge to the target code amount of the code amount control target.

<Target code amount change unit>
Therefore, when the target code amount of the line of interest is changed, the target code amount changing unit 112 changes the target code amount by weighting Xn-1. The changed target code amount T'[i] is shown in the following equation (5). In addition, α (≧ 1) is a weighting coefficient for performing the above weighting.
T'[i] = Tn x X [i-1] / (Xn-1 x α)… (5)
By using the above equation (5), the screen target code amount does not necessarily converge, but the generated code amount integration amount is suppressed and complicated by changing the line target code amount as compared with the case where it is not changed. Since the degree is represented by the equation (1), the magnitude of Qp can be adjusted according to the value of α. Since the magnitude of Qp can be adjusted, it is possible to control the code amount with higher priority on image quality.

By doing so, it is possible to provide an image coding apparatus that can arbitrarily adjust Qp, suppress image quality deterioration, and have high code amount controllability.

[Fifth Embodiment]
FIG. 15 is a block configuration diagram of the image coding device according to the fifth embodiment. In the illustration, reference numerals 100 to 115, 1216 are the same as those in the second embodiment, and thus the description thereof will be omitted.

The sub-band generation code amount integrating unit 1518 integrates the generation code amount of each sub-band for each line corresponding to the same pixel position. In the present embodiment, it is assumed that the target code amount for each line is changed at the same time for all subbands, and the coding of all subbands is performed in parallel.

<Same pixel position of subband>
FIG. 16 shows the relationship of the same pixel positions in the space represented by the image before the discrete wavelet transform of each subband when the vertical and horizontal filtering of the discrete wavelet transform (DWT) is performed three times each.

In DWT, a conversion coefficient for one pixel line is generated for two pixel lines of the image before conversion. Further, since the DWT is performed recursively with respect to the subband LL, the conversion coefficient for one line at the decomposition level 2 is generated from the conversion coefficient for the two pixel lines at the decomposition level 1. The same relationship is obtained even if the decomposition level is increased thereafter.

From the above relationship, the N / 2 line of Lv2 and the N / 4 line of Lv3 correspond to the same pixel position with respect to the N line of Lv1 which is the highest region. When DWT is executed three times, as shown in FIG. 16, the lines that can be regarded as the same pixel position are two lines of Lv2 and one line of Lv3 with respect to four lines of Lv1.

<Target code amount change unit>
The target code amount changing unit 112 sets the code amount upper limit value as the subband total code amount upper limit value with respect to the generated code amount integrated amount of all subbands, and the subbands integrated by the subband generated code amount integrated unit 1518. Compare with the accumulated code amount. If the integrated amount of code amount generated by the subband is larger than the upper limit of the total code amount of the subband, the target code amount T'[i] of the line of interest of each subband is changed by using Eq. (5).

By changing the target code amount for all subbands at the same time in this way, Qp is also changed for all subbands at the same time. Generally, Qp is set larger for the high frequency component and smaller for the low frequency component of the image data, but if the target code amount is changed independently for each subband, the above relationship may be broken. For example, when the Qp of the sub-band HL is much larger than the Qp of the sub-band LH, only the vertical edge of the image is extremely deteriorated, and the deterioration of the subjective image quality is conspicuous.

On the other hand, if the present embodiment is applied, α of the equation (5) can be changed to a predetermined value all at once, so that extreme deterioration of image quality can be suppressed.

By doing the above, by changing the target code amount of each subband from the same pixel position, the Qp can also be changed from the same pixel position, while suppressing the deterioration of image quality and the code amount controllability. It is possible to provide a high-quality image coding apparatus.

It is not always necessary to change the target code amount in all subbands. For example, the subband LL, which is the lowest frequency component, gives priority to image quality and does not change the target code amount, which is also within the scope of the present invention.

Although each embodiment has been described in detail above, the present invention is not limited to a specific embodiment, and various modifications and changes can be made within the scope of the claims. It is also possible to combine all or a plurality of the components of the above-described embodiment.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by the processing to be performed. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

100 ... image coding unit, 101 ... coding amount control unit, 102 ... quantization setting unit, 103 ... target coding amount setting unit, 104 ... quantization unit, 105 ... quantization control unit, 106 ... coding unit, 107 ... Generated code amount holding unit, 108 ... Quantization parameter holding unit, 109 ... complexity calculation unit, 110 ... target code amount calculation unit, 111 ... complexity setting unit, 112 ... target code amount changing unit, 113 ... difference calculation unit, 114 ... Difference holding unit, 115 ... Correlation degree determination unit, 116 ... Buffer, 1216 ... Discrete wavelet transforming unit, 1417 ... Image correlation degree changing unit, 1518 ... Subband generation code amount integrating unit

Claims (10)

An image coding device that encodes moving image data captured by an imaging means.
A quantization means for quantizing the image data of the frame of interest in the moving image data in preset block units according to a quantization parameter determined according to a target code amount.
A coding means that encodes the quantized data obtained by the quantization means in block units, and
It has a code amount control means for controlling the quantization means so that the code amount generated by the coding means becomes a set block target code amount.
The code amount control means is
Acquisition means for acquiring a pre-Symbol correlation of the image of the image and the frame immediately before the frame of interest,
In a coordinate space where the axis from the first block to the last block in one frame is the horizontal axis and the axis representing the integrated amount of coded data is the vertical axis, the coordinates represented by the position indicated by the preset frame target code amount are set. It is the first determination means for determining the slope of the line segment representing the upper limit of the amount of code to pass based on the correlation degree. If the correlation degree is large, the slope of the line segment is small, and if the correlation degree is small, the slope is small. A first determining means for determining the inclination of the line segment so that the inclination of the line segment becomes large,
It is a second determination means for determining the block target code amount of the block of interest based on a preset reference target code amount and the complexity of the coded block immediately before the block of interest, and is the focus. When the complexity of the coded block immediately before the block is high, the block target code amount of the block of interest is large, and when the complexity of the coded block immediately before the block of interest is low, the block of interest is said. A second determining means for determining the block target code amount of the block of interest so that the block target code amount becomes small,
When the total code amount from the first block of the focus frame to the immediately preceding block exceeds the code amount indicated by the line segment determined by the first determination means, the block target code amount of the focus block With a changing means for changing the line segment so as to follow the line segment.
An image coding device comprising. The first determination means is
The slope is determined based on either the difference between the total brightness of the frame of interest and the total brightness of the immediately preceding frame, the difference in the histogram, or the difference in the amount of code up to the block of interest. The inclination is determined so that the inclination becomes large and the difference becomes small.
The image coding apparatus according to claim 1. The second determination means is
The block of interest is the i-th block, the coding amount of the coding data of the immediately preceding i-1 block is S [i-1], and the quantization parameter used when encoding the i-th block is When defined as Qp [i-1], X [i-1] representing the complexity is obtained by the following equation. X [i-1] = Qp [i-1] × S [i-1]
The image coding apparatus according to claim 1 or 2. The second determination means is
When the integrated amount of generated code amounts up to the block of interest reaches the upper limit of the code amount at the position of the block of interest, the block target code amount is determined by modifying the complexity so as to be small. The image coding apparatus according to claim 1 or 2. Further, it has a conversion means for performing a discrete wavelet transform on the image data of the frame of interest in the moving image data.
The image coding apparatus according to claim 2 , wherein the quantization means quantizes each subband obtained by the conversion means. The code amount control means has an image correlation degree changing means.
The fifth aspect of claim 5 is the image correlation degree changing means, wherein the larger the number of subbands exceeding the code amount difference threshold value, the smaller the image correlation degree among the plurality of subbands, and the smaller the number, the larger the image correlation degree is changed. Image encoding device. The image coding apparatus according to claim 4 , wherein the second determining means determines the block target code amount each time the coding of a preset number of blocks is completed. The coding means encodes blocks at the same position in the space represented by the image in parallel in all subbands.
The code amount control means performs quantization control on one or more subbands, determines a block target code amount for each of the one or more subbands, and generates up to the block of interest in the one or more subbands. The image coding apparatus according to claim 5 , wherein the block target code amount in each subband is determined based on the total code amount of the code amounts. It is a control method of an image coding device that encodes moving image data captured by an imaging means.
A quantization step in which the image data of the frame of interest in the moving image data is quantized in preset block units according to a quantization parameter determined according to a target code amount.
A coding step of encoding the quantized data obtained in the quantization step in block units, and
It has a code amount control step of controlling the quantization step so that the code amount generated by the coding step becomes a set block target code amount.
The code amount control step is
An acquisition step of acquiring pre Symbol correlation of the image of the image and the frame immediately before the frame of interest,
In a coordinate space where the axis from the first block to the last block in one frame is the horizontal axis and the axis representing the integrated amount of coded data is the vertical axis, the coordinates represented by the position indicated by the preset frame target code amount are set. In the first determination step of determining the slope of the line segment representing the upper limit of the amount of code to pass based on the correlation degree, if the correlation degree is large, the slope of the line segment is small, and if the correlation degree is small, the slope is small. The first determination step of determining the inclination of the line segment so that the inclination of the line segment becomes large, and
This is a second determination step of determining the block target code amount of the block of interest based on a preset reference target code amount and the complexity of the coded block immediately before the block of interest. When the complexity of the coded block immediately before the block is high, the block target code amount of the block of interest is large, and when the complexity of the coded block immediately before the block of interest is low, the block of interest is said. A second determination step of determining the block target code amount of the block of interest so that the block target code amount becomes small, and
When the total code amount from the first block of the focus frame to the immediately preceding block exceeds the code amount indicated by the line segment determined in the first determination step, the block target code amount of the focus block With the change process of changing to follow the line segment
A method for controlling an image coding apparatus, which comprises. A program for causing the computer to execute each step of the method according to claim 9 , which is read and executed by the computer.

Images