JP5626728B2 - Image coding method, image coding apparatus, and image coding program - Google Patents

Description

  The present invention relates to a high-efficiency image signal encoding method, and more particularly to an image encoding method for improving energy compaction and improving encoding efficiency in an alternating projection method.

  One of the important elemental techniques in image coding is transform coding represented by Discrete Cosine Transform (DCT). The role of transform coding in image coding is to remove spatial correlation between pixels. The overall position of the encoder is as follows: information is concentrated on a small number of transform coefficients by transform coding, and the transform coefficients with low information concentration are rounded down by quantizing the transform coefficients, thereby reducing the amount of information for the signal to be encoded. Contributes to the reduction of

  FIG. 7 shows an example of a general encoding apparatus 100 that encodes a video signal. When the video signal is input, the encoding apparatus 100 obtains a prediction residual signal from the difference from the prediction signal predicted by the prediction unit 106, and the transform unit 101 performs orthogonal transform on the prediction residual signal. The output transform coefficient is quantized by the quantization unit 102, and the quantized value is variable-length encoded by the entropy encoding unit 107 and output as an encoded stream. On the other hand, the output of the quantization unit 102 is inversely quantized by the inverse quantization unit 103 and further inversely orthogonally transformed by the inverse transform unit 104. A decoded signal is generated by adding a prediction signal to the conversion result. The decoded signal is subjected to noise removal processing by the distortion removal filter 105 and input to the prediction unit 106 as a reference decoded signal. The prediction unit 106 generates a prediction signal for encoding the next video signal by motion search or the like.

  Up to now, in application to image coding, many transform coding schemes such as discrete cosine transform (DCT) and overlapping orthogonal transform discrete wavelet transform (DWT) have been studied. For example, as transform coding, JPEG employs discrete cosine transform (DCT), and JPEG 2000 employs discrete wavelet transform (DWT). In addition, since orthogonal transformation uses a complete basis, the number of data before and after the transformation is unchanged. For this reason, the orthogonal transform is a non-redundant transform. In the moving image encoding apparatus, the conversion unit 101 in FIG. 7 corresponds to the above technique.

  On the other hand, there is a transform called a redundant transform using an overcomplete basis in which the number of bases is larger than the number of samples of the original signal. For this reason, the redundant transform cannot be an orthogonal transform, but by giving redundancy to the converted data, it is possible to have characteristics that cannot be realized by the non-redundant transform. For example, a discrete stationary wavelet transform (SWT) that is a DWT that does not perform downsampling processing can establish shift invariance lost in the DWT due to redundancy after the transformation.

  In the field of image processing, “conversion with direction separation characteristics” has attracted attention. Such conversion is generally redundant conversion, and a typical example is Curvelet conversion. A parallel tree complex wavelet transform (DTCWT) is a transform having similar characteristics. The conversion having the direction separation characteristic is a conversion in which a curve such as an edge included in the image signal is expressed using a direction base defined in two dimensions. Since the two-dimensional structure is approximated with high accuracy using the direction base, it is more effective for noise removal and feature extraction than a transformation with poor direction separation characteristics such as DWT.

  However, conversion with direction separation characteristics has a problem that the number of data after conversion increases. When x is an input signal to transform coding and Ψ is a transform matrix, a transform coefficient y obtained by the transform is expressed as follows.

y = Ψx (1)
On the other hand, the process of decoding the value in the signal domain from the transform coefficient is represented by the following equation, where Φ is a matrix representing the inverse transform of the overcomplete basis set.

x = Φy (2)
In the case of DTCWT, if x is an n-dimensional vector, the conversion coefficient y obtained by the conversion is a 2n-dimensional vector. For this reason, when the transformation is applied to image coding, it is necessary to appropriately select a transformation coefficient from the viewpoint of reducing the number of data. This selection of transform coefficients can be formulated as the following constrained minimization problem.

min _y ‖y‖ ₀ subject to Φy = x (3)
Here, ‖ · ‖ ₀ is the L ⁰ norm and represents the number of non-zero coefficients. The above minimization problem with constraints is reduced to the following minimization problem by Lagrange's undetermined multiplier method.

min _y ‖y‖ ₀ + λ‖Φy-x‖ ₂ ² (4)
Here, λ is a weight parameter given from the outside. The first term is the number of selected transform coefficients, which is an approximate value of the information amount of transform coefficients. The second term represents the reconstruction error associated with the selection of the transform coefficient and represents the coding distortion. ‖ · ‖ ₂ ² is the square value of the L ² norm and represents the sum of squares. However, since the above minimization problem is difficult to NP, conventionally, the following L ^{1 is used.} A method of approximating the norm as a minimization problem has been taken.

min _y ‖y‖ ₁ + λ‖Φy−x‖ ₂ ² (5)
Where ‖ and ‖ are L ¹ Norm, which represents the sum of absolute values of vector elements.

  As a method for giving a sub-optimal solution of the minimization problem of Equation (5), a method called an alternate projection method has been proposed (see Non-Patent Document 1).

  Equation (3) expresses the following conditional minimization problem

The above equation can be interpreted as looking for y having K non-zero coefficients that minimizes the Euclidean distance from x. In order to find a solution that realizes the minimization of the above equation, the alternating orthogonal projection between the set (affine set) S (Φ, x) and C (K) as the solution of Equation (2) is repeated. The orthogonal projection from C (K) to S (Φ, x) is

Given in. Where y _LS is based on the norm of square error minimization,
y _LS = Φ ^T [ΦΦ ^T ] ⁻¹ x
Is a coefficient vector obtained by The orthogonal projection from S (Φ, x) to C (K) is given by ht (y, K). Here, ht (y, K) is a projection that minimizes the Euclidean distance to C (K), and corresponds to an operation of selecting K coefficients from the largest value.

The initial vector of coefficients is set to y ⁽⁰⁾ , and iterative processing is performed according to the following recurrence formula.

  This iterative process is repeated until the following termination condition is satisfied.

^{‖Y (n + 1) -y (} n) || _{2 <ε}
Such conversion using an overcomplete system is a kind of sparse representation in which an image signal is assumed to be sparseness and an image is expressed with a small number of conversion coefficients.

L. Mancera and J. Portilla, “L0-norm-based sparse representation through alternate projections”, IEEE ICIP, Oct., 2006.

  Image signals have spatial locality, and their properties are not uniform. In other words, the number of transform coefficients necessary to represent an image signal with a certain accuracy, or the representation accuracy of an image signal that can be represented with a certain number of transform coefficients varies depending on the nature of the region in the image. . However, the alternating projection method is applied to the entire image. For this reason, locality in the image cannot be considered, and there is room for improvement in improving the coding efficiency.

  The present invention has been made in view of such circumstances, and is an image segmentation that realizes an improvement in coding efficiency in an alternate projection method that eliminates redundancy between transform coefficients in a redundant transform using an overcomplete basis. It aims at establishing the image coding method based on a technique.

  In order to solve the above-described problems, the present invention performs a conversion using a redundant conversion base on an input video signal, and narrows down the coefficients used for encoding for the obtained conversion coefficients. In image coding, an area for minimizing the total sum of distortion in all areas under the condition that the total number of coefficients in the screen is constant (the total number of coefficients in all divided areas is constant) according to the local properties of the image. It is characterized in that the division and the conversion coefficient for each divided area are selected.

  The present invention can improve coding efficiency by dividing the screen into regions of similar nature and selecting transform coefficients that reduce the cost for each divided region.

In addition, the present invention performs the conversion using the redundant conversion base on the input video signal and narrows down the coefficients used for the encoding on the obtained conversion coefficients. Depending on the local nature of the image, the total number of distortions in the entire area and a predetermined width in contact with the boundary of the divided area under the condition that the total number of coefficients in the screen is constant (the total number of coefficients in all the divided areas is constant) . It is characterized in that region division for minimizing the weighted sum with the degree of deviation of the approximation error based on narrowing down of the transformation coefficients in the two boundary regions , and selection of transformation coefficients for each divided region are performed.

  Further, the present invention is characterized in that, in the above invention, the number of coefficients of the divided area is set according to the number of pixels of the divided area.

  Also, the present invention recursively repeats the process of selecting area division and transform coefficients until a predetermined division termination condition is satisfied for each divided area in the selected area division. Features. The condition for terminating the division is, for example, that the number of pixels in one divided area is equal to or smaller than a predetermined threshold value, or that the number of transform coefficients in one divided area is equal to or smaller than the predetermined threshold value. In addition, when it is optimal not to divide, it becomes a condition for ending the division.

  According to the present invention, by selecting a coefficient in consideration of the locality of an image, energy compaction in the alternate projection method is improved, and information is integrated into a specific component, thereby improving encoding efficiency. Can do.

It is a figure which shows the structural example of the image coding apparatus which concerns on embodiment of this invention. It is a figure which shows the coefficient selection process flow which a coefficient selection process part performs. It is a figure which shows the optimal division process flow which an optimal division process part performs. It is a figure which shows the optimal division process flow which an optimal division process part performs. It is a figure which shows the cost function calculation process flow which a cost calculation part performs. It is a system block diagram when implement | achieving from a software program. It is a block diagram of a general encoding apparatus.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  The basic policy of the present invention is to divide the screen into regions having similar properties and apply sparse representation to each divided region. Hereinafter, four types of processing are shown.

  [Area division method 1] shows a division and parameter selection method for minimizing the total distortion amount in all areas under the condition that the total number of coefficients in the screen is constant (the total number of coefficients in all divided areas is constant). .

  [Region division method 2] indicates a division and parameter selection method for minimizing the amount of distortion in each region under the condition that the number of coefficients per pixel is constant in each division region.

  [Area division method 3] is a modification of the cost function in [Area division method 1] that takes into account the effect of discontinuity of the area boundary.

  Similarly, [Area division method 4] is a modification of the cost function in [Area division method 2] that takes into account the influence of discontinuity of the area boundary.

Assume that an image signal existence area is 0 ≦ s ≦ W and 0 ≦ t ≦ H, and pixel values in the area are rearranged as a one-dimensional vector. Further, from the same area, the pixel values in the rectangular area having the upper left corner coordinate value (s ₀ , t ₀ ) and the horizontal and vertical side lengths w and h are extracted and arranged as a one-dimensional vector. Let the changed vector be x (s ₀ , t ₀ , w, h).

For x (s ₀ , t ₀ , w, h), k transform coefficients {c ₀ ,..., c _k−1 } are selected using the alternate projection method, and the selected base signal is used. An approximation error when x (s ₀ , t ₀ , w, h) is expressed as E [x, s ₀ , t ₀ , w, h, k, C]. Here, C represents k conversion coefficients C = {c ₀ ,..., C _k−1 }. For example, in the case of DTCWT, k coefficients are selected from a total of 2n coefficients, and an alternate projection method can be used for this selection.

[Area Division Method 1]
Consider a division for minimizing the total distortion amount in all areas under the condition that the total number of coefficients in the screen is constant (the total number of coefficients in all divided areas is constant). Further, the results of dividing vertically into two and horizontally dividing are compared, and either vertical division or horizontal division is selected on the basis of the above-described distortion amount minimization rule.

  The following approximation error is used as a measure of the amount of distortion.

_{Ξ hor [x, 0, h} , W, H, k h, k-k h, C u, C d] = E [x, 0,0, W, h, k h, C d] + E [x, 0, h, W, H- h, k-k h, C u]
The above formula changes the region 0 ≦ s ≦ W, 0 ≦ t ≦ H to the upper region 0 ≦ s ≦ W, 0 ≦ t ≦ h−1, and the lower region 0 ≦ s ≦ W, h ≦ t ≦ H. divided, respectively upper region and a lower region, represents an approximation error sum when expressed in k _h number of coefficients and k-k _h number of coefficients.

Ξ _ver [x, w, 0, W, H, k _w , k−k _w , C _l , C _r ] = E [x, 0, 0, w, H, k _w , C _l ] + E [x, w, 0, W−w, H, k−k _w , C _r ]
The above expression divides the region 0 ≦ s ≦ W and 0 ≦ t ≦ H into the left region 0 ≦ s ≦ w−1, 0 ≦ t ≦ H and the right region w ≦ s ≦ W, 0 ≦ t ≦ H. and, respectively left area and right area, it represents an approximation error sum when expressed in k _w pieces of coefficients and k-k _w pieces of coefficients.

When dividing into two in the vertical direction under the condition that the sum of transform coefficients in the two areas after division is kept at k, the division position and the number of coefficients assigned to each area after division are determined based on the following equations: . The division position h ₀ when dividing into two in the vertical direction and the number k _{0, h} of transform coefficients used in the upper divided area are given by the following equations (the number of transform coefficients used in the lower divided area is k− k _{0, h} ).

When dividing horizontally into two under the condition that the sum of transform coefficients in the two areas after division is kept at k, the division position and the number of coefficients assigned to each area after division are determined based on the following equations: . The division position w ₀ in the case of dividing into two in the horizontal direction and the number k _{0, w} of transform coefficients used in the left divided area are given by the following equations (note that the number of transform coefficients used in the right divided area is k −k _{0, w} ).

At this time,
Hor _hor [x, 0, h ₀ , W, H, k _{0, h} , k−k _{0, h} , C _u , C _d ]
≦ Ξ _ver [x, w ₀ , 0, W, H, k _{0, w} , k−k _{0, w} , C _l , C _r ]
In such a case, division is performed in the vertical direction at the position t = h ₀ , and in other cases, division is performed in the horizontal direction at the position s = w ₀ . Note that when h ₀ = 0 or w ₀ = 0, it is more optimal not to perform the division, so the division is not performed.

  The above equation finds the division position and the number of transform coefficients assigned to each division area.

  For the region with the largest approximation error (division target region) among the rectangular regions obtained by the division, the number of coefficients included in the division target region is regarded as k in the above equation, and the same division process as above is performed. Apply. If the selected area has not been divided, the same dividing process as described above is performed on the area having the next largest approximation error after the previously selected area. This repetition is continued until all the divided areas are not divided. Alternatively, it is possible to set a lower limit value for the number of pixels in the divided area and to limit generation so that an area below the lower limit value is not generated as a result of the division.

[Area division method 2]
Consider a division for minimizing the amount of distortion in each region under the condition that the number of coefficients per pixel is constant in the two divided regions. Further, the results of dividing vertically into two and horizontally dividing are compared, and either vertical division or horizontal division is selected based on the above weighted sum minimization rule.

  The difference from the [region dividing method 1] is that the coefficient allocation to the divided region is set according to the number of pixels in the divided region. For this reason, when viewed from the viewpoint of minimizing approximate error, if the solution obtained by [Area Division Method 1] is the optimal solution, the method obtained by this method is a suboptimal solution, but the calculation cost for obtaining the solution is reduced. can do.

  The following approximation error is used as a measure of the amount of distortion.

  In this formula, the value (k · h / H) between the L-shaped symbol and the left-right inverted L-shaped symbol represents the maximum integer that does not exceed the value (hereinafter the same). In the following description, it is also expressed as int (x). That is, int (x) means the largest integer that does not exceed the real number x.

  The above formula changes the region 0 ≦ s ≦ W, 0 ≦ t ≦ H to the upper region 0 ≦ s ≦ W, 0 ≦ t ≦ h−1, and the lower region 0 ≦ s ≦ W, h ≦ t ≦ H. This represents an approximate error sum when the upper region and the lower region are expressed by int (k · h / H) coefficients and k−int (k · h / H) coefficients, respectively.

  The above expression divides the region 0 ≦ s ≦ W and 0 ≦ t ≦ H into the left region 0 ≦ s ≦ w−1, 0 ≦ t ≦ H and the right region w ≦ s ≦ W, 0 ≦ t ≦ H. The left side region and the right side region represent the approximate error sum when expressed by int (k · w / W) coefficients and k-int (k · w / W) coefficients, respectively.

  When dividing into two in the vertical direction under the condition that the sum of the transform coefficients in the two regions after division is kept at k, the division position is determined based on the following equation. The following equation takes an approach that allocates coefficients according to the number of pixels in the area.

  When dividing into two in the horizontal direction under the condition that the sum of the transform coefficients in the two regions after division is kept at k, the division position is determined based on the following equation. The following equation takes an approach that allocates coefficients according to the number of pixels in the area.

Here, M _h and M _w are natural numbers given in advance as the minimum change width. At this time,

In such a case, division is performed in the vertical direction at the position t = h ₀ , and in other cases, division is performed in the horizontal direction at the position s = w ₀ . Note that when h ₀ = 0 or w ₀ = 0, it is more optimal not to perform the division, so the division is not performed.

  The above equation finds the optimal division position in the framework of allocating coefficients according to the number of pixels in the area.

  For the region with the largest approximation error (division target region) among the rectangular regions obtained by the division, the number of coefficients included in the division target region is regarded as k in the above equation, and the same division process as above is performed. Apply. If the selected area has not been divided, the same dividing process as described above is performed on the area having the next largest approximation error after the previously selected area. This repetition is continued until all the divided areas are not divided. Alternatively, it is possible to set a lower limit value for the number of pixels in the divided area and to limit generation so that an area below the lower limit value is not generated as a result of the division.

[Region division method 3]
Under the condition that the total number of coefficients in the screen is constant (the total number of coefficients in all the divided areas is constant), the sum of the distortion amounts after division and the weighted sum of the evaluation criteria for discontinuity of the division boundary (described later) (called division cost) Consider a partition that minimizes. Further, the result of dividing vertically into two and horizontally dividing into two is compared, and either vertical division or horizontal division is selected based on the above-mentioned norm of division cost minimization.

  The following approximation error is used as a measure of the amount of distortion.

The above formula changes the region 0 ≦ s ≦ W, 0 ≦ t ≦ H to the upper region 0 ≦ s ≦ W, 0 ≦ t ≦ h−1, and the lower region 0 ≦ s ≦ W, h ≦ t ≦ H. divided, respectively upper region and a lower region, represents an approximation error sum when expressed in k _h number of coefficients and k-k _h number of coefficients.

The above expression divides the region 0 ≦ s ≦ W and 0 ≦ t ≦ H into the left region 0 ≦ s ≦ w−1, 0 ≦ t ≦ H and the right region w ≦ s ≦ W, 0 ≦ t ≦ H. and, respectively left area and right area, it represents an approximation error sum when expressed in k _w pieces of coefficients and k-k _w pieces of coefficients.

The number of significant coefficients and the evaluation measure (discontinuity measure) of the discontinuity of the dividing boundary are used as a measure of region division. The discontinuity scale is defined for the following boundary regions: For vertical division that divides into two regions of the upper and lower, the lower 0 ≦ s ≦ W in the upper region (expressed in k _h pieces of coefficient _{C u), h-L ≦} t ≦ h-1 and the lower side A rectangular region 0 ≦ s ≦ W, h−L ≦ t ≦ h + L consisting of the upper part 0 ≦ s ≦ W and h ≦ t ≦ h + L in the region (expressed by k−k _h coefficients C _d ) The discontinuity scale for a rectangular area is expressed as follows.

Here, the second and third arguments indicate the lower and upper limits of the horizontal existence area 0 ≦ s ≦ W with respect to the areas before and after the division. The fourth and fifth arguments indicate the lower limit h−L of the existence region in the vertical direction with respect to the boundary region and the vertical width 2L + 1 of the region. The 6,7 argument k _h, k-k _h is the number of coefficients in the upper region and a lower region. The eighth and ninth arguments C _u and C _d are coefficients in the upper region and the lower region.

In the case of horizontal division into two areas on the left and right, the right end area w−L ≦ s ≦ w−1, 0 ≦ t ≦ H and the right area in the left area (represented by k _w coefficients C ₁ ) A rectangular region w-L≤s≤w + L, 0≤t≤H consisting of left end w≤s≤w + L, 0≤t≤H in (represented by k-k _w coefficients _Cr ), The discontinuity scale for the rectangular area is expressed as follows.

Here, the second and third arguments indicate the lower and upper limits of the existence area 0 ≦ t ≦ H in the vertical direction with respect to the areas before and after the division. The fourth and fifth arguments indicate the lower limit w−L of the horizontal existence area with respect to the boundary area and the horizontal width 2L + 1 of the same area. The sixth and seventh arguments k _w and k−k _w are the numbers of coefficients in the left region and the right region. The eighth and ninth arguments C _l and C _r are coefficients in the left region and the right region.

As the discontinuity scale, for example, the deviation degree of the approximation error is used. This is because, in the case of vertical division, lower 0 ≦ s ≦ W in the upper region (expressed in _{k h} pieces of coefficient _{C u),} the approximation error in h-L ≦ t ≦ h- 1, the lower region (k- The difference value of the approximation error in the upper part 0 ≦ s ≦ W and h ≦ t ≦ h + L in k _h coefficients C _d ), and in the case of horizontal division, the left region (expressed by k _w coefficients C ₁₎ ) In the right end w−L ≦ s ≦ w−1, 0 ≦ t ≦ H, and the left end w ≦ s ≦ w + L in the right region (represented by k−k _w coefficients C _r ), It is a difference value from the approximation error when 0 ≦ t ≦ H. Alternatively, it is possible to use a value obtained by multiplying the difference value of the approximation error by a constant weight.

It is also possible to use a value obtained by inverting the sign of SSIM (see Reference 1) in the divided boundary region.
[Reference 1]: Z. Wang and EP Simoncelli, “Translation insensitive image similarity in complex wavelet domain”, IEEE International Conference on Acoustics, Speech & Signal Processing, vol. II, pp. 573-576, Philadelphia, PA, March 2005.

When dividing into two in the vertical direction under the condition that the sum of transform coefficients in the two areas after division is kept at k, the division position and the number of coefficients assigned to each area after division are determined based on the following equations: . The dividing position h ₀ and the number k _{0, h} of transform coefficients used in the upper divided area when dividing into two in the vertical direction are given by the following equation (the number of transform coefficients used in the lower divided area is k−k _{0, h} ).

When dividing horizontally into two under the condition that the sum of transform coefficients in the two areas after division is kept at k, the division position and the number of coefficients assigned to each area after division are determined based on the following equations: . The division position w ₀ in the case of dividing into two in the horizontal direction and the number k _{0, w} of transform coefficients used in the left divided area are given by the following equations (note that the number of transform coefficients used in the right divided area is k −k _{0, w} ).

At this time,
Hor _hor [x, 0, h ₀ , W, H, k _{0, h} , k−k _{0, h} , C _u , C _d ] +
λΘ _hor [x, 0, W, h ₀ −L, 2L + 1, k _{0, h} , k−k _{0, h} , C _u , C _d ]
≦ Ξ _ver [x, w ₀ , 0, W, H, k _{0, w} , k−k _{0, w} , C _l , C _r ] +
λΘ _ver [x, 0, H, w ₀ −L, 2L + 1, k _{0, w} , k−k _{0, w} , C _l , C _r ]
In such a case, division is performed in the vertical direction at the position t = h ₀ , and in other cases, division is performed in the horizontal direction at the position s = w ₀ . Note that when h ₀ = 0 or w ₀ = 0, it is more optimal not to perform the division, so the division is not performed.

  The above equation finds the division position and the number of transform coefficients assigned to each division area.

  For the region with the largest approximation error (division target region) among the rectangular regions obtained by the division, the number of coefficients included in the division target region is regarded as k in the above equation, and the same division process as above is performed. Apply. If the selected area has not been divided, the same dividing process as described above is performed on the area having the next largest approximation error after the previously selected area. This repetition is continued until all the divided areas are not divided. Alternatively, it is possible to set a lower limit value for the number of pixels in the divided area and to limit generation so that an area below the lower limit value is not generated as a result of the division.

[Area Division Method 4]
Minimize the weighted sum (referred to as the division cost) of the sum of the amount of distortion after division and the evaluation measure of discontinuity of the division boundary (described later) under the condition that the number of coefficients per pixel is constant in the two division regions Think of the division to be. Further, the results of dividing vertically into two and horizontally dividing are compared, and either vertical division or horizontal division is selected based on the above weighted sum minimization rule. The difference from the [region dividing method 3] is that the coefficient allocation to the divided regions is set according to the number of pixels in the divided regions. Therefore, from the viewpoint of minimizing the division cost, if the solution obtained by [Area division method 3] is the optimum solution, the method obtained by this method is a suboptimal solution, but the calculation cost for obtaining the solution is reduced. can do.

  The following approximation error is used as a measure of the amount of distortion.

  The above formula changes the region 0 ≦ s ≦ W, 0 ≦ t ≦ H to the upper region 0 ≦ s ≦ W, 0 ≦ t ≦ h−1, and the lower region 0 ≦ s ≦ W, h ≦ t ≦ H. This represents an approximate error sum when the upper region and the lower region are expressed by int (k · h / H) coefficients and k−int (k · h / H) coefficients, respectively.

  The above expression divides the region 0 ≦ s ≦ W and 0 ≦ t ≦ H into the left region 0 ≦ s ≦ w−1, 0 ≦ t ≦ H and the right region w ≦ s ≦ W, 0 ≦ t ≦ H. The left side region and the right side region represent the approximate error sum when expressed by int (k · w / W) coefficients and k-int (k · w / W) coefficients, respectively.

  As the discontinuity measure of the division boundary, for example, SSIM in the division boundary region is used. A rectangular region 0 ≦ s ≦ W, h− that is composed of a lower portion 0 ≦ s ≦ W, h−L ≦ t ≦ h−1 in the upper region and an upper portion 0 ≦ s ≦ W, h ≦ t ≦ h + L in the lower region. The discontinuity scale for L ≦ t ≦ h + L is expressed as follows.

  A rectangular region w−L ≦ s ≦ w + L composed of a right end portion w−L ≦ s ≦ w−1, 0 ≦ t ≦ H in the left region and a left end portion w ≦ s ≦ w + L, 0 ≦ t ≦ H in the right region. , 0 ≦ t ≦ H, the discontinuity scale is expressed as follows.

  When dividing into two in the vertical direction under the condition that the sum of the transform coefficients in the two regions after division is kept at k, the division position is determined based on the following equation. The following equation takes an approach that allocates coefficients according to the number of pixels in the area.

  When dividing into two in the horizontal direction under the condition that the sum of the transform coefficients in the two regions after division is kept at k, the division position is determined based on the following equation. The following equation takes an approach that allocates coefficients according to the number of pixels in the area.

M _h and M _w are natural numbers given in advance as the minimum change width. At this time,

In such a case, division is performed in the vertical direction at the position t = h ₀ , and in other cases, division is performed in the horizontal direction at the position s = w ₀ . Note that when h ₀ = 0 or w ₀ = 0, it is more optimal not to perform the division, so the division is not performed.

  The above equation finds the optimal division position in the framework of allocating coefficients according to the number of pixels in the area.

  For the rectangular area obtained by the division with the maximum division cost (division target area), the number of coefficients included in the division target area is regarded as k in the above equation, and the same division process as above is performed. Apply. If the selected area has not been divided, the same dividing process as described above is performed on the area having the next highest division cost after the previously selected area. This repetition is continued until all the divided areas are not divided. Alternatively, it is possible to set a lower limit value for the number of pixels in the divided area and to limit generation so that an area below the lower limit value is not generated as a result of the division.

[Configuration Example of Image Encoding Device]
FIG. 1 shows a configuration example of an image coding apparatus according to an embodiment of the present invention. When the image encoding device 10 receives the video signal, the image encoding device 10 obtains a prediction residual signal from the difference from the prediction signal predicted by the prediction unit 14, and the prediction residual signal is converted by the conversion unit 11 using the conversion base of the redundant system. Convert. The coefficient selection processing unit 20 divides the screen into regions of similar properties in order to narrow down the transform coefficients that are the output of the transform unit 11, and sets the transform coefficients that have a lower cost for each divided region as encoding targets. select. The entropy encoding unit 15 performs variable length encoding on the selected transform coefficient and outputs it as an encoded stream.

  On the other hand, the output of the coefficient selection processing unit 20 is inversely transformed by the inverse transformation unit 12, and a prediction signal is added to the transformation result to generate a decoded signal. The decoded signal is subjected to noise removal processing by the distortion removal filter 13 and input to the prediction unit 14 as a reference decoded signal. The prediction unit 14 generates a prediction signal for encoding the next video signal.

  Selection of the conversion coefficient in the coefficient selection processing unit 20 is performed as follows. First, the optimum division processing unit 22 sets a division position for each of a plurality of division candidate positions for dividing an image region, performs region division, and the coefficient selection unit 221 calculates the total number of coefficients of all divided regions. Find the optimal area division that minimizes the sum of the distortion amounts of all the divided areas under the constant condition, or the area division that minimizes the discontinuity of the boundary between the total distortion amount and the divided areas. Select the conversion coefficient for the region. The cost calculation unit 222 calculates the encoding cost according to the region division, that is, the total sum of distortion amounts of all the divided regions (sum of errors caused by truncation of transform coefficients) or the discontinuity at the boundary portion of the divided regions. When considering it as a cost, the cost of the discontinuity is calculated. Based on the result, the optimum division processing unit 22 selects the area division with the smallest cost as the optimum division.

  The region division shape setting unit 21 uses the information indicating the region division determined by the optimum division processing unit 22 and the transform coefficient selected by the coefficient selection unit 221 for the region division as the encoding target information, and uses the entropy code. To the conversion unit 15.

[Coefficient selection process flow]
FIG. 2 is a diagram illustrating a coefficient selection processing flow executed by the coefficient selection processing unit 20. Here, an embodiment in the case of performing region division mainly by [region division method 4] will be described, but this method includes the methods of [region division method 1] to [region division method 3] From the following description, it is clear that the coefficient selection processing by area division of [area division method 1] to [area division method 3] can be similarly performed.

In this example, the coefficient selection processing unit 20 is called by the Segment function having as arguments the total number K of coefficients in the division target area and the area of the processing target image (initially all areas of the input image signal). I do.
[Step S1]: When the region is divided into the region 1 and the region 2, the number k1 of the coefficients of the region 1 is set as a loop index, the initial value is k1 = 0, the increment is 1, and k1 <K. The processes of S1 to S3 are repeated. Note that the number k2 of the coefficients in the region 2 is k2 = K−k1.
[Step S2]: The Opt_sub_div function (optimal division processing function) is called with the number of coefficients k1 in region 1, the number k2 of coefficients in region 2 (= K−k1), and the input region as arguments. With this function, the optimum division processing unit 22 executes the optimum division processing shown in FIGS.
[Step S3]: 1 is added to the loop index k1, and the processing after step S1 is repeated. When k1 becomes K or more, the loop is terminated.
[Step S4]: Calculate the optimum number of coefficients for the regions 1 and 2 from the processing result of the step S2.
[Step S5]: It is determined whether or not there is an area having no undivided label. If there is an area having no undivided label, the process proceeds to step S6. If all the regions are regions having no division labels, the process proceeds to step S8.
[Step S6]: A region having the maximum cost function value is identified. Let K ′ be the number of coefficients in this region.
[Step S7]: The Segment function is called with the number of coefficients K ′ and the area having the maximum cost function value as arguments, and the process by the coefficient selection processing unit 20 is recursively performed until there is no area with no division label. Repeat.
[Step S8]: The area having the maximum cost function value is identified, and the process returns to the caller of the Segment function.

[Optimal split processing flow]
FIG. 3 and FIG. 4 show the optimum division processing flow executed by the optimum division processing unit 22. The optimum division processing unit 22 is called by the Opt_sub_div function (optimum division processing function) in step S2 shown in FIG. The inputs are the number of coefficients k1 in region 1, the number k2 of coefficients in region 2, and the region to be optimally divided.
[Step S10]: The vertical division position (position at which the region is divided into two in the vertical direction) is set as a loop index, the initial value is 0, the increment is M, and the processing of steps S10 to S12 is performed while the vertical division position <the screen vertical width. repeat.
[Step S11]: The cost function calculation function is called with the number of coefficients k1, k2 and the vertical division position as arguments. With this function, the cost calculation unit 222 executes the cost function calculation process shown in FIG.
[Step S12]: M is added to the vertical division position of the loop index, and the processing after step S10 is repeated. If the vertical division position exceeds the screen vertical width, the loop is terminated.
[Step S13]: The optimum vertical division position is calculated from the processing result of the cost function calculation, and the process proceeds to step S20 in FIG.
[Step S14]: The horizontal division position (position at which the region is divided into two in the horizontal direction) is set as a loop index, the initial value is 0, the increment is M, and the processing of steps S14 to S16 is performed while the horizontal division position <the horizontal width of the screen. repeat.
[Step S15]: The cost function calculation function is called with the number of coefficients k1 and k2 and the horizontal division position as arguments. With this function, the cost calculation unit 222 executes the cost function calculation process shown in FIG.
[Step S16]: M is added to the horizontal division position of the loop index, and the processes in and after step S14 are repeated. If the horizontal division position exceeds the screen horizontal width, the loop is terminated.
[Step S17]: The optimum horizontal division position is calculated from the processing result of the cost function calculation, and the process proceeds to Step S20 in FIG. Note that the processes of steps S10 to S13 and steps S14 to S17 may be executed in parallel or serially.
[Step S20]: The cost using the optimum vertical division position is compared with the cost using the optimum horizontal division position. If the cost using the optimum vertical division position is smaller, the process proceeds to step S21. If not, the process proceeds to step S25.
[Step S21]: If the number of pixels in one divided area is 0 or the number of transform coefficients in one divided area is 0, the process proceeds to step S22, and if not, the process proceeds to step S24.
[Step S22]: A non-division label indicating that the area is not divided is assigned to the input area.
[Step S23]: A non-divided label is returned to the caller of the Opt_sub_div function, and the process ends.
[Step S24]: The information of the optimal vertical division position, the conversion coefficient of each divided area, the approximate error of each divided area, and the cost function value are returned to the caller of the Opt_sub_div function, and the process is terminated.
[Step S25]: If the number of pixels in one divided area is 0 or the number of transform coefficients in one divided area is 0, the process proceeds to step S26, and if not, the process proceeds to step S28.
[Step S26]: A non-division label indicating that the area is not divided is assigned to the input area.
[Step S27]: A label without division is returned to the caller of the Opt_sub_div function, and the process is terminated.
[Step S28]: The information of the optimal horizontal division position, the conversion coefficient of each divided area, the approximate error of each divided area, and the cost function value are returned to the caller of the Opt_sub_div function, and the process is terminated.

[Cost function calculation processing flow]
FIG. 5 shows a cost function calculation process flow executed by the cost calculation unit 222. The cost calculation unit 222 is called by the cost function calculation function in steps S11 and S15 shown in FIG. The input is the number of coefficients k1 in region 1, the number k2 of coefficients in region 2, and the vertical division position or horizontal division position.
[Step S30]: A coefficient selection process for selecting k1 coefficients is performed for the region 1.
[Step S31]: An approximation error is calculated when the region 1 is approximated by k1 coefficients selected by the coefficient selection process.
[Step S32]: A decoded signal is obtained when the region 1 is approximated by k1 coefficients selected by the coefficient selection process. Thereafter, the process proceeds to step S36.
[Step S33]: A coefficient selection process for selecting k2 coefficients for the region 2 is performed.
[Step S34]: An approximation error is approximated when the region 2 is approximated by k2 coefficients selected by the coefficient selection process described above.
[Step S35]: A decoded signal is obtained when the region 2 is approximated by k2 coefficients selected by the coefficient selection process. Note that steps S30 to S32 and steps S33 to S35 may be executed in parallel or serially.
[Step S36]: A discontinuity measure is calculated for the boundary region of the decoded signals in regions 1 and 2.
[Step S37]: λ × discontinuity scale is added to the approximate error sum of regions 1 and 2 to calculate a cost function value. The calculated cost function value is returned to the function caller, and the process ends.

  The above-described image encoding processing can be realized by a computer and a software program, and the program can be recorded on a computer-readable recording medium or provided through a network.

  FIG. 6 shows an example of the system configuration when the present invention is implemented using a software program. The memory 52 stores an image encoding program 53 for performing the image encoding process of the present invention. The CPU 50 sequentially fetches and executes the instructions of the image encoding program 53 stored in the memory 52. The video storage device 51 is a device that stores a video signal to be encoded. The video signal may be input from a camera or the like (not shown). The encoded stream generated by the image encoding program 53 is stored in the encoded stream storage device 54. Alternatively, the encoded stream may be output to an external device via an interface such as a network adapter. The system bus 55 is a bus that connects the CPU 50, the video storage device 51, the memory 52, and the encoded stream storage device 54.

DESCRIPTION OF SYMBOLS 10 Image coding apparatus 11 Conversion part 12 Inverse conversion part 13 Distortion removal filter 14 Prediction part 15 Entropy encoding part 20 Coefficient selection process part 21 Area division shape setting part 22 Optimal division process part 221 Coefficient selection part 222 Cost calculation part

Claims (8)

In an image encoding method for performing conversion using a redundant conversion basis on an input image signal, narrowing down the obtained transform coefficients, and coding the transform coefficients obtained as a result of the narrowing down,
A process of setting a division position for each of a plurality of division candidate positions for dividing an image region;
Dividing the image area at the set division position and selecting transform coefficients for each divided area under the condition that the total number of coefficients of all the divided areas is constant;
Among all the region divisions respectively divided at the plurality of division candidate positions, the region division that minimizes the total amount of distortion due to the selection of transform coefficients for all regions is selected, and encoding in each division region is performed. And a process for selecting a transform coefficient to be used. In an image encoding method for performing conversion using a redundant conversion basis on an input image signal, narrowing down the obtained transform coefficients, and coding the transform coefficients obtained as a result of the narrowing down,
A process of setting a division position for each of a plurality of division candidate positions for dividing an image region;
Dividing the image area at the set division position and selecting transform coefficients for each divided area under the condition that the total number of coefficients of all the divided areas is constant;
Among all the area divisions divided at the positions to be the plurality of division candidates, the total distortion amount by selection of the conversion coefficient of all areas and the conversion in two boundary areas having a predetermined width in contact with the boundary of the divided areas An image coding method comprising: selecting a region division that minimizes a weighted sum with a divergence degree of an approximation error based on a narrowing down of coefficients, and selecting a transform coefficient used for encoding in each divided region Method. The image encoding method according to claim 1 or 2,
An image coding method characterized in that the number of coefficients in a divided area is set according to the number of pixels in the divided area. In the image coding method of any one of Claim 1- Claim 3,
An image coding method comprising: recursively repeating the process of selecting the area division and transform coefficients until a predetermined division end condition is satisfied for each divided area in the selected area division. In an image encoding apparatus that performs conversion using a conversion base of a redundant system on an input image signal, narrows down the obtained conversion coefficients, and encodes the conversion coefficients obtained as a result of the reduction,
Means for setting a division position for each of a plurality of division candidate positions for dividing an image area;
Means for dividing an image area at the set division position and selecting transform coefficients for each divided area under a condition that the total number of coefficients of all the divided areas is constant;
Among all the region divisions respectively divided at the plurality of division candidate positions, the region division that minimizes the total amount of distortion due to the selection of transform coefficients for all regions is selected, and encoding in each division region is performed. An image encoding device comprising: means for selecting a transform coefficient to be used. In an image encoding apparatus that performs conversion using a conversion base of a redundant system on an input image signal, narrows down the obtained conversion coefficients, and encodes the conversion coefficients obtained as a result of the reduction,
Means for setting a division position for each of a plurality of division candidate positions for dividing an image area;
Means for dividing an image area at the set division position and selecting transform coefficients for each divided area under a condition that the total number of coefficients of all the divided areas is constant;
Among all the area divisions divided at the positions to be the plurality of division candidates, the total distortion amount by selection of the conversion coefficient of all areas and the conversion in two boundary areas having a predetermined width in contact with the boundary of the divided areas An image code comprising: selecting a region division that minimizes a weighted sum with a divergence degree of approximation error based on a coefficient narrowing down; and selecting a transform coefficient used for encoding in each divided region Device. In the image coding device according to claim 5 or 6,
An image coding apparatus characterized in that the number of coefficients in a divided area is set according to the number of pixels in the divided area.   An image encoding program for causing a computer to execute the image encoding method according to any one of claims 1 to 4.

Images