JP4819855B2 - Moving picture quantization method, moving picture quantization apparatus, moving picture quantization program, and computer-readable recording medium recording the program - Google Patents

Description

  The present invention relates to a moving picture quantization method and apparatus used in moving picture coding for coding a moving picture by performing information compression by transform coding and quantization on a prediction error signal, and the moving picture quantum The present invention relates to a moving picture quantization program used for realizing the conversion method and a computer-readable recording medium on which the program is recorded.

[Quantization for orthogonal transform coefficients]
H. In the case of an encoding method based on intra-frame / inter-frame prediction represented by H.264, orthogonal transform / quantization is performed on the prediction error for each rectangular region (referred to as a block).

H. The flow of encoding processing up to quantization in H.264 is shown below. In the following, S is an original signal, q is a quantization parameter, m is a number representing a prediction mode, ^ S _{m, q} is predicted for the original signal S using the prediction mode m, and the quantization parameter q is It is a decoded signal when quantized by using. The symbols used for the explanation are summarized in the following table.

Let P _m be the prediction signal when the prediction method of mode number m is used for the original signal S. H. In the H.264 encoding process, the orthogonal transformation using the transformation matrix to Φ is applied to the prediction error signal R _m (= S−P _m ) when the prediction method of the mode number m is used as follows: Is done. In the following notation, the symbol “˜X” (where X is a letter) indicates a symbol attached to “X”.

Here, ~ Φ ^t represents a transposed matrix for the transformation matrix ~ Φ. Note that the transformation matrix to Φ is an orthogonal matrix of integer elements expressed by the following equation.

  Next, since the matrices Φ are non-normal matrices, processing corresponding to matrix normalization as shown below is performed.

  This is equivalent to using Φ of the following expression instead of ˜Φ in the expression (1).

Further, the quantization using the quantization step width Δ is performed on C _m as follows.

  Here, round (•) is a function for rounding a real value to the nearest integer value.

On the other hand, H. In the H.264 decoding process, V is inversely quantized as in the following equation to obtain a decoded coefficient ^ C _{m, q} of the transform coefficient.

Here, {circumflex over (C _{)} m, q} is a decoded value of a coefficient obtained by quantizing / inverse-quantizing C _m with the quantization parameter q.

Next, inverse transform is applied to this ^ C _{m, q} as shown in the following equation to obtain a prediction error decoded signal.

  Finally, a decoded signal of the encoding target image is obtained by the following equation.

[Quantization using quantization weight coefficient]
MPEG-2 and H.264 In H.264, a mechanism for setting a quantization step width for each element (k and l components) of a transform coefficient is prepared as in the following equation.

  Here, W [k, l] is a coefficient determined corresponding to the frequency component (k, l). Hereinafter, W [k, l] is referred to as a quantization weight coefficient.

  For example, changes in high frequency components are less likely to be visually detected than changes in low frequency components. Therefore, quantization is performed in which the high-frequency component W [k, l] is set larger than that of the low-frequency component, and the high-frequency component is processed with a relatively large quantization step width.

Further, considering the visual sensitivity in the time direction, studies have been made to adaptively set the quantization step width in accordance with the shift amount (see Patent Document 1).
JP-A-5-236444

  In the conventional setting of the quantization weight coefficient, weighting by a sensitivity function is performed independently for each block, so that discontinuity (block distortion) at the block boundary is not reflected in the weighted distortion amount.

  In a block-based coding scheme (for example, H.264) that combines inter-frame prediction based on motion compensation and orthogonal transform, block distortion is a characteristic coding distortion. For this reason, if the quantization step width is set without considering the dependency relationship with the adjacent blocks, there is a risk that block distortion becomes obvious and subjective image quality is greatly impaired.

  For example, taking the one-dimensional signal shown in FIG. 6 as an example, discontinuity between blocks is performed in a process closed to blocks that weight the DCT coefficients of each block (block k-1, block k, block k + 1). (Discontinuity between block k-1 and block k, or discontinuity between block k and block k + 1) is not known.

  The present invention has been made in view of such circumstances, and as a quantization step width for a block-based encoding scheme (for example, H.264) that combines inter-frame prediction by motion compensation and orthogonal transform, It is an object of the present invention to provide a new moving image quantization technique that enables setting of a reflection of subjective image quality including block distortion.

  In the present invention, frequency analysis is performed in consideration of discontinuity between adjacent blocks together with the waveform in the block, and the quantization step width is set based on the contrast sensitivity function.

[Calculation of quantization weight coefficient]
In the present invention, the distortion amount in each block is weighted based on the spatiotemporal visual sensitivity function. In the calculation of the weighting coefficient, input is a conversion coefficient and a shift amount (indicating temporal movement of an image signal such as a motion vector). Here, also for a frame for which intra prediction is performed, a shift amount can be obtained by obtaining temporal movement of an image signal between frames.

  In the following, a case where an image having a vertical width H is observed at a viewing distance rH is considered. r is called a viewing distance parameter. Also, in the following, in order to simplify the expression, the subscript indicating the distinction between Y, U, and V is omitted, and the Y component will be discussed. The U and V components can be discussed in the same manner as described below.

The transformation of a two-dimensional signal by orthogonal transformation is to express a signal using a base image of the orthogonal transformation. When the k-th column vector (N-dimensional vector) of the transformation matrix Φ (N × N matrix) is φ _k , the base image for the matrix is
f _{k, l} (x, y) = φ _k [y] φ _l [x] ^t (0 ≦ x, y ≦ N−1)
It can be obtained from the formula H. In the case of H.264, the possible value for N is either 4 or 8. Here, φ _l ^t is a transposed vector of φ _l .

Predictive error signal R _m [x, y] (Ni _x ≦ x ≦ Ni _x + N−1, Ni _y ≦ y ≦ Ni _y + N−1) in the region of N × N [pixel] is ^expressed as R _m ^{(ix, iy).} And abbreviated as “reference block” hereinafter. Here, i _{x and} i _y are integer values indicating the position of the reference block. Further, assuming that the corresponding orthogonal transform coefficient is C _m ^{(ix, iy)} [k, l] (0 ≦ k, l ≦ N−1), the prediction error signal R _m ^{(ix, iy)} is given by It can be expressed as

M _x × M _y-number of reference block ^{_{(R m (ix, iy)}} ) composed of M _x N × M _y prediction including N pixel error signal _{R m [x, y] (} Ni x0 ≦ x ≦ N ( For i _x0 + M _x ) −1, Ni _y0 ≦ y ≦ N (i _y0 + M _y ) −1) (referred to as an analysis target block), the subjective image quality considering block distortion is considered.

Here, since the analyzed block is intended to be composed of M _x × M _y-number of reference blocks, when Fourier transform the analyte block, discontinuity between adjacent reference blocks not only the reference block It becomes possible to evaluate the property (block distortion).

To represent the relationship between the analyzed block and the reference block, as shown in the following ^formula, R _m ^{(ix, iy)} with respect to, the zero padded, obtaining a signal of M _x N × M _y N pixels .

Similarly, as shown in the following equation, the base image f _k, with respect to l (x, y) (0 ≦ x, y ≦ N-1), the zero padded, the M _x N × M _y N pixels Get a signal.

Zero padding the resulting ^{_{~f k, l (p) (}} x, y) (x = 0, ...., M x N-1; y = 0, ...., M y N-1; p = 0, ...., referred to as a M _x M _y -1) a modified base image. For example, in the case of _{N = 4, M x = 2} , M y = 2 , as shown in FIG. 1, is arranged the base image in 4 × 4 pixels shaded portion, are the zero value to the other position Padded.

Here, FIG. 1A corresponds to i _x = i _x0 +1, i _y = i _y0 , and FIG. 1B corresponds to i _x = i _x0 , i _y = i _y0 , and FIG. ) Corresponds to i _x = i _x0 +1, i _y = i _y0 +1, and FIG. 1D corresponds to i _x = i _x0 , i _y = i _y0 +1.

  At this time, the analysis target block can be expressed as follows using the corrected base image.

  The Fourier transform (expressed by F [•]) on both sides of the equation (13) can be expressed as the following equation by the linearity of the Fourier transform.

  Here, this equation (14) is obtained by the Fourier transform coefficient (frequency component resulting from discontinuity between adjacent reference blocks as well as the frequency component in the reference block) obtained by performing Fourier transform on the analysis target block. This means that the corrected base image is represented by a linear sum of Fourier transform coefficients obtained by performing Fourier transform.

^{_{F [~f k, l (ix}} , iy)] is a complex vector of M _x N × M _y N-dimensional, the first (u _x, u _y) elements Fourier transform is expressed by the following equation It is a coefficient. In the following, it is assumed that N = 2 ^m .

Here, j is an imaginary unit. U _{x and} u _y are referred to as spatial frequency indexes.

In this way, the frequency component due to the discontinuity of the block is also taken into account by setting the target of the Fourier transform to ˜f _{k, l} (x, y) instead of f _{k, l} (x, y). Frequency analysis can be performed.

Fourier transform coefficients of the F _k, relative to _{l (u x, u y)} (0 ≦ u x ≦ M x N-1,0 ≦ u y ≦ M y N-1), performs the weighting of the following. Note that (d _{x, dy} ) is a transition amount estimated for the analysis target block.

Hereinafter, ~ F _{k, l} (ux _, u _y ) will be described. Here, {circumflex over (g)} (η, d) is a function set based on visual system characteristics such as contrast sensitivity, and is called a visual sensitivity function. As for the setting of the visual sensitivity function, for example, there is a method shown in [Visual sensitivity function setting 1] or [Visual sensitivity function setting 2] described later.

The quantization weight coefficient W (d _{x, dy} ) [k, l] for the (k, l) basis component of the prediction error signal R _m is defined as the power ratio of the following equation.

  At this time, it is also possible to change the model parameter with the luminance component and the color difference component.

If explained the meaning of the formula (17), since the analyzed block is intended to be composed of M _x × M _y-number of reference blocks, if the analyte block Fourier transform, not only in the block It also becomes possible to evaluate discontinuity between adjacent blocks. On the other hand, as shown in Expression (14), Fourier transform of the analysis target block means that a linear sum of Fourier transform coefficients obtained by Fourier transform of the corrected base image (linear sum using orthogonal transform coefficients as coefficients). Is equivalent to calculating.

Therefore, in the present invention, in order to be able to evaluate not only the block but also the discontinuity between adjacent blocks, the sum of squares of the part described in {•} on the right side of Expression (14) (Expression (Corresponding to the numerator of (17)) and calculating the sum of squares corresponding to the weighted expression (16) (corresponding to the denominator of (17)), The quantization weight coefficient W (d _{x, dy} ) [k, l] for the (k, l) basis component of the prediction error signal R _m is calculated according to the ratio (17). is there.

  Incidentally, the sum of the calculated values of the numerator of Equation (17) for k and l corresponds to the sum of squares of Equation (13), and from this, the numerator of Equation (17) corresponds to the power of the prediction error signal. To be.

Since the quantization weight coefficient W (d _{x, dy} ) [k, l] determined in this way indicates a value according to the spatial frequency and the time frequency, the quantization reflecting the subjective image quality is performed. Processing is realized. Moreover, since it is determined by performing frequency analysis considering discontinuity between adjacent blocks together with the waveform in the block, quantization processing reflecting subjective image quality including block distortion is realized. ing.

That is, with respect to the quantization step width of the spatio-temporal frequency component that is difficult to detect visually, the quantization weight coefficient W (d _{x, dy} ) [k, l] becomes relatively large due to the relatively large value. On the other hand, the quantization step width of the spatio-temporal frequency component that is easily detected visually has a relatively small quantization weight coefficient W (d _x, d _y ) [k, l]. Therefore, the quantization processing that reflects the subjective image quality is realized. In addition, the value of the quantization weight coefficient W (dx _{, dy} ) [k, l] is determined by performing frequency analysis in consideration of discontinuity between adjacent blocks together with the waveform in the block. For this reason, quantization processing reflecting subjective image quality including block distortion is realized.

[Quantization of quantization weight coefficient]
The quantization weight coefficient obtained by Expression (17) is N ² elements W (d _x, d _y ) [k, l] (0 ≦ k, l ≦ N) for each shift amount (d _x, d _y ). it is N ^{2-dimensional} vector composed of -1). In the following, this N ^two- dimensional vector is expressed as <W (dx _{, dy} )> = {W (dx _{, dy} ) [k, l] (0≤k, l≤N-1)}.
Represent as

  If all the quantization weight coefficient information is encoded for each analysis target block, the additional information of the same coefficient becomes excessive, and the encoding efficiency is lowered. Therefore, in order to reduce the amount of information that expresses the quantization weight coefficient, the following method based on clustering of multidimensional information is applied.

First, clustering based on the shift amount is performed. L _x × L _y classes (b (i _x −1) ≦ | d _x | <b (i _x ), b (i _y −1) ≦ | d _{y using the} amount of change of each analysis target block as input. | <B (i _y )). Parameters (b (i _x ), b (i _y ) (i _x = 0,..., L _x, i _y = 0,..., L _y )) defining the boundaries of each class are externally applied. Shall be given. For example, there is a method of setting the same parameter so that the frequencies in each class are almost equal.

Next, the quantization weight coefficient belonging to each class <W (d _x, d _y )> (b (i _x ) ≦ | d _x | <b (i _x +1), b (i _y ) ≦ | d _y | Considering <b (i _y +1)) as a set of N ^two- dimensional vectors, L _w representative vectors are calculated. That is, the purpose is to aggregate this vector set into L _w representative vectors to reduce the amount of information necessary for the quantization weight coefficient.

The calculation method of the representative vector is given separately from the outside. For example, K-means clustering method is divided into L _w class by LBG algorithm, the centroid vector of each class has a method of the representative vector of each class. Alternatively, there is a method in which principal component analysis is applied to a set of N ^two- dimensional vectors, and eigenvectors corresponding to the top L _w principal components having a large contribution ratio are used as representative vectors. Note that the parameter _Lw is given from the outside. The parameter can be set individually for each class.

  Next, [visual sensitivity function setting 1] and [visual sensitivity function setting 2], which are examples of the visual sensitivity function setting method, will be described.

[Visual Sensitivity Function Setting 1]
Consider a contrast sensitivity function such as:

Here, a ₁ , a ₂ , a ₃ , and a ₄ are parameters (hereinafter referred to as model parameters) that define the function form of the visual sensitivity function.
_{(A 1, a 2, a} 3, a 4) = (6.1, 7.31, 2, 45.9)
Such a value is used.

  Also, η is a spatial frequency [cycle / degree] representing the number of light-dark pairs within the unit viewing angle. Note that η is a one-dimensional spatial frequency.

At this time, between η and the spatial frequency index (either u _x or u _y ),
η (u, r) = θ (r, H) u / 2MN Equation (19)
There is a relationship.

In the case of u = u _x, a M = M _x, the case of u = u _y, is M = M _y. Θ (r, H) is an angle [degrees / pixel] per pixel when an image having a vertical width H is observed at a viewing distance rH.
θ (r, H) = (1 / H) × arctan (1 / r) × (180 / π)
... Formula (20)
Is given by the expression

ω is the angle change per unit time [degrees / sec]. At this time, between the ω the displacement amount d (either d _x or d _y) is
^{ω (d) = tan -1 (} f r d / rH) ··· formula (21)
There is a relationship. Here, _fr is a frame rate.

Expression (19) and Expression (21) are substituted into Expression (18), and the contrast sensitivity function g (η, ω) is expressed as a function of u and d by the following expression ^ g (u, d)
^ G (u, d) = g (η (u), ω (d)) (22)
Is a visual sensitivity function.

[Visual sensitivity function setting 2]
Consider the following contrast sensitivity function.

Here, b ₁ , b ₂ , b ₃ , and b ₄ are parameters (hereinafter referred to as model parameters) that define the function form of the visual sensitivity function.
_{(B 1, b 2, b} 3, b 4) = (0.4992, 0.2964, -0.114, 1.1)
_{(B 1, b 2, b} 3, b 4) = (0.2, 0.45, -0.18, 1)
_{(B 1, b 2, b} 3, b 4) = (0.31, 0.69, -0.29, 1)
_{(B 1, b 2, b} 3, b 4) = (0.246, 0.615, -0.25, 1)
It takes such a value.

  Also, η is a spatial frequency [cycle / degree] representing the number of light-dark pairs within the unit viewing angle. Note that η is a one-dimensional spatial frequency.

At this time, between η and the spatial frequency index (either u _x or u _y ),
η (u, r) = θ (r, H) u / 2MN Equation (24)
There is a relationship.

In the case of u = u _x, a M = M _x, the case of u = u _y, is M = M _y. Θ (r, H) is an angle [degrees / pixel] per pixel when an image having a vertical width H is observed at a viewing distance rH.
θ (r, H) = (1 / H) × arctan (1 / r) × (180 / π)
... Formula (25)
Is given by the expression

Further, r is adaptively changed according to the magnitude of the shift amount d. For example, the setting method is as follows. Here, A is a threshold value, and r ₁ > r ₂ .

Substituting Equation (24) and Equation (25) into Equation (23), and representing the contrast sensitivity function g (η) as a function of u and d, ^ g (u, d)
^ G (u, d) = g (η (u, r (d)) (27)
Is a visual sensitivity function.

  In accordance with the configuration described above, according to the present invention, it is possible to introduce a quantization weighting factor that appropriately evaluates subjective image quality including block distortion, thereby reducing block distortion as a quantization step width. It becomes possible to set what can realize the quantization process reflecting the subjective image quality including the high-efficiency encoding, and the code amount can be reduced.

  Next, the configuration of the moving picture coding apparatus provided with the present invention will be described.

A moving image coding apparatus including the present invention performs moving image coding by performing information compression by transform coding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. (A) is defined in association with an analysis target block composed of a plurality of blocks to be converted matrix, and a base image of the conversion matrix is arranged in one block, (B) temporal movement of the image signal of the block to be analyzed; and (b) storage means for storing spatial frequency components calculated for the corrected base images for the number of blocks configured by embedding zero values in other blocks (C) reading out the spatial frequency component of the corrected base image from the storage means, and estimating the spatial frequency index of the spatial frequency component A visual sensitivity value assigned to the spatial frequency component based on the estimated shift amount of the means and weighting the spatial frequency component; (d) a spatial frequency component weighted by the weighting means; , A calculation for calculating a weighting factor for the quantization step width set for each base component of the transformation matrix based on the spatial frequency component without weighting and the transformation factor of the block constituting the analysis target block by using the means, the calculated weighting coefficient (e) calculation means, to modify the quantization step width is set pre Me, and a determining means for determining a quantization step width used in the quantization process, (F) The calculation means includes a multiplication value of the square sum of the transform coefficients and the weighted square norm sum of the spatial frequency components, and a spatial frequency not weighted with the square sum of the transform coefficients. Seeking a multiplication value of the square norm sum of the number components, configured to calculate a weight coefficient according quotient of the two multiplied values.

  Here, the storage means for storing the spatial frequency component calculated for the corrected base image is provided because the spatial frequency component of the corrected base image can be obtained regardless of the moving image to be encoded. This is because it is not necessary to calculate each time.

  When adopting this configuration, the weighting means calculates the visual sensitivity value in the horizontal direction based on the horizontal spatial frequency index and the horizontal component of the displacement estimated by the estimating means, and the vertical spatial frequency index and The visual sensitivity value assigned to the spatial frequency component of the corrected base image read from the storage means may be calculated by calculating the visual sensitivity value in the vertical direction based on the vertical component of the shift amount estimated by the estimation means. .

  The reason why the spatial frequency component and the direction dependency of the motion are considered in this way is because the detection mechanism of the visual system in the spatio-temporal region depends on the spatial edge direction and the direction of motion. For example, when the vertical stripe moves, only the horizontal component of the movement can be recognized as the movement. That is, when evaluating the visual sensitivity with respect to the spatio-temporal frequency, it is necessary to consider the edge direction and the transition direction of the spatial frequency component. In consideration of this, the weighting means may calculate the visual sensitivity value in consideration of the spatial frequency component and the direction dependency of motion.

  The moving image encoding method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is provided by being recorded on a suitable computer-readable recording medium. Alternatively, the present invention is realized by being provided via a network, installed when the present invention is carried out, and operating on a control means such as a CPU.

  Thus, in the present invention, a moving image is encoded by performing information compression by transform coding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. In determining the quantization step width for each orthogonal transform coefficient component in the moving image encoding, the spatial frequency component in the block and the spatial frequency component related to the discontinuity between adjacent blocks are measured and further obtained. The temporal frequency component is estimated based on the spatial frequency component obtained and the motion vector obtained by the motion estimation, and a weighting factor is calculated for each spatiotemporal frequency component based on the visual sensitivity function. The quantization step width used for the quantization process is determined by correcting the set quantization step width.

  In the present invention, for a block-based coding scheme that combines inter-frame prediction by motion compensation and orthogonal transform, the spatio-temporal frequency is estimated by considering the direction of the spatial frequency component and the direction of the shift amount, Introduce a quantization weighting factor that appropriately evaluates subjective image quality including distortion.

  Thus, according to the present invention, it is possible to set a quantization step width that can realize a quantization process that reflects subjective image quality including block distortion, thereby realizing a highly efficient encoding. At the same time, the amount of code can be reduced.

  Hereinafter, the present invention will be described in detail according to embodiments.

  FIG. 2 illustrates an embodiment of the moving picture encoding apparatus 1 including the present invention.

  As shown in this figure, the moving picture coding apparatus 1 including the present invention includes (1) an encoding unit 10 that encodes moving picture data, and (2) the encoding unit 10 executes a quantization process. A quantization step width determination unit 11 that determines a quantization step width to be used sometimes; (3) a base image storage unit 12 that stores a base image required by the quantization step width determination unit 11; and (4) quantization. And a shift amount storage unit 13 that stores an estimated value of the shift amount of the analysis target block required by the step width determination unit 11.

  The encoding unit 10 performs (1) a quantization step for storing a quantization step width used for the quantization process in order to execute a quantization process (DCT coefficient quantization) necessary for encoding moving image data. Width storage unit 101, (2) quantization step width initial value setting unit 102 that sets an initial value of the quantization step width and stores it in quantization step width storage unit 101, and (3) quantization step width storage unit And a quantization unit 103 that quantizes the DCT coefficient using the quantization step width stored in 101.

  Here, in the following description, it is assumed that the quantization step width initial value setting unit 102 sets Δ, which is a value independent of the basis component of the transformation matrix, as an initial value as the initial value of the quantization step width. ing.

The quantization step width determination unit 11 inputs the DCT coefficient of the analysis target block, the base image stored in the base image storage unit 12, the shift amount of the analysis target block stored in the shift amount storage unit 13, and the contrast function. The quantization weight coefficient W (d _{x, dy} ) [k, l] for the (k, l) basis component of the prediction error signal R _m is calculated according to the calculation formula of the equation (17), and the calculation is performed. The quantization unit 103 performs quantization by multiplying the quantization weight coefficient W (d _x, d _y ) [k, l] by the quantization step width initial value Δ stored in the quantization step width storage unit 101. A process of determining “Δ × W (d _x, d _y ) [k, l]”, which is a quantization step width used in the process, is stored in the quantization step width storage unit 101.

In order to execute this process, the quantization step width determination unit 11 calculates the quantization weight coefficient W (d _{x, dy} ) [k, l] according to the calculation formula of the expression (1) (17). A weighting factor calculation unit 111; (2) a quantization weighting factor storage unit 112 that stores the quantization weighting factor W (dx _{, dy} ) [k, l] calculated by the quantization weighting factor calculation unit 111; 3) The quantization weight coefficient W (d _{x, dy} ) [k, l] stored in the quantization weight coefficient storage unit 112 and the initial value Δ of the quantization step width stored in the quantization step width storage unit 101 A quantization weight coefficient multiplication unit 113 that calculates a multiplication value and stores it in the quantization step width storage unit 101.

Then, the quantization weight coefficient calculation unit 111 performs (4) a process 4 execution unit 1111 that executes “quantization weight coefficient calculation process 4” that performs the process of calculating the DFT coefficient of the modified base image using the base image as an input. (2) a modified base image DFT coefficient storage unit 1112 that stores the DFT coefficient of the modified base image calculated by the process 4 execution unit 1111; and (3) a modified base image stored in the modified base image DFT coefficient storage unit 1112. The process of calculating the prediction error power (the numerator of Expression (17)) for the analysis target block is performed using the DFT coefficient of the above and the DCT coefficient calculated by the encoding unit 10 as inputs. The processing 2 execution unit 1113 for executing “4”, (4) the DFT coefficient of the modified base image stored in the modified base image DFT coefficient storage unit 1112, and the encoding unit 10 calculate “Quantization weight coefficient calculation process for performing a process of calculating prediction error power (denominator of Expression (17)) for the analysis target block with the input DCT coefficient, the motion vector of the analysis target block, and the contrast function as inputs. 3 ”, the prediction error power obtained by the process 2 execution unit 1113 (numerator of Expression (17)), and the prediction error power obtained by the process 3 execution part 1114 (formula ( 17) is used as an input, and “quantization weight coefficient calculation process 1” is executed in which a quantization weight coefficient W (d _x, d _y ) [k, l] is calculated according to equation (17). And a processing 1 execution unit 1115 for performing the processing.

  Next, processing executed by the quantization weight coefficient calculation unit 111 configured as described above will be described according to the flowchart shown in FIG.

  When there is an instruction to calculate the quantization weight coefficient, the quantization weight coefficient calculation unit 111 first receives the base image stored in the base image storage unit 12 in step S101 as shown in the flowchart of FIG. Then, the “quantization weight coefficient calculation process 4” is executed to perform the process of calculating and outputting the DFT coefficient of the modified base image.

  The DFT coefficients of the corrected base image calculated at this time are stored in the corrected base image DFT coefficient storage unit 1112 in consideration of being reused. When the DFT coefficient of the image is stored, it is not necessary to execute the process of step S101.

  Subsequently, in step S102, the DFT coefficient of the modified base image obtained by the “quantization weight coefficient calculation process 4” and the DCT coefficient calculated by the encoding unit 10 are input, and the prediction error power ( A “quantization weight coefficient calculation process 2” is executed to perform a process of calculating and outputting the numerator of Expression (17).

  Subsequently, in step S103, the DFT coefficient of the modified base image obtained by the “quantization weight coefficient calculation process 4”, the DCT coefficient calculated by the encoding unit 10, and the analysis target block stored in the shift amount storage unit 13 Then, a “quantization weight coefficient calculation process 3” is performed in which the prediction error power (the denominator of Expression (17)) for the analysis target block is calculated and output using the motion vector and the contrast function as inputs.

Subsequently, in step S104, the prediction error power (numerator of Expression (17)) obtained by “quantization weight coefficient calculation process 2” and the prediction error power (expression ( 17) is used as an input, and the quantization weight coefficient W (d _{x, dy} ) [k, l] is calculated and output according to the equation (17). ”Is executed.

  Next, detailed flowcharts of “quantization weight coefficient calculation process 1”, “quantization weight coefficient calculation process 2”, “quantization weight coefficient calculation process 3”, and “quantization weight coefficient calculation process 4” will be described.

[1] “Quantization weight coefficient calculation process 4”
Input: k-th, l-th base image (k, l = 0,..., N−1)
Output: DFT coefficient for modified base image
(1) Reading indexes i _{x and} i _y indicating position information (2) l = 0
(3) k = 0
(4) Read the k-th and l-th base images f _{k, l} ^{(ix, iy)} from the base image storage unit 12 (5) For the base image of (4), zero corresponding to the position information of (1) padding by, generating a M _x N × M _x image ~f _k of ^{_{N, l (ix, iy)}} . The image obtained here is called the modified base image. The specific generation method is the equation (12).
(6) DFT is performed on the modified base image of (5), and the distribution of frequency components in the modified base image is calculated. The specific calculation method is formula (15).
(7) k = k + 1
(8) If k = N, go to (9), otherwise go to (4) (9) l = l + 1
(10) If k = N, the process ends; otherwise, go to (3). According to this flowchart, the quantization weight coefficient calculation unit 111 performs the base image f _{k, l} ^{(ix as} input ^iy), it is performed DFT coefficient F _k modified base ^{_{image, l (ix, iy) (}} u x, the process of calculating the u _y).

[2] “Quantization weight coefficient calculation process 2”
Input: DFT coefficient of modified base image (k, l = 0,..., N−1,
i _x = 0, ..., M _x -1, i _y = 0, ..., M _y -1)
: DCT coefficient Output: Prediction error power E [k, l] for the analysis target block (numerator of equation (17))
Where E [k, l] is an array process:
(1) l = 0
(2) k = 0
(3) E [k, l] = 0
(4) i _y = 0
(5) i _x = 0
(6) S = 0
(7) u _y = 0
(8) u _x = 0
(9) Read the u _{x and} u _y components of the DFT coefficients of the k th and l corrected base images of the position index i _{x and} i _y from the corrected base image DFT coefficient storage unit 1112. (10) Square of complex number read immediately before Calculate the norm F _{k, l} ^{(ix, iy)} ( ^ux _, u _y ) ² (11) S = S + F _{k, l} ^{(ix, iy)} ( ^ux _, u _y ) ²
(12) u _x = u _x +1
(13) If u _x = NM _x , go to the next step, otherwise go to (9) (14) u _y = u _y +1
(15) If u _y = NM _y , go to the next step, otherwise go to (8). (16) From the encoding unit 10, the DCT coefficient C ^{(ix, iy)} [k, l] of the k th and l th basis is ^calculated . Read (17) E [k, l] = E [k, l] + C ^{(ix, iy)} [k, l] ² S
(18) i _x = i _x +1
(19) If i _x = M _x , go to the next step, otherwise go to (6) (20) i _y = i _y +1
(21) i _y = M _y if to the next, if not to (5) (22) k = k + 1
(23) If k = N, go to the next, otherwise go to (3) (24) l = l + 1
(25) If l = N, end; otherwise, go to (2) According to this flowchart, the quantization weight coefficient calculation unit 111 performs the “quantization weight coefficient calculation process 4” in the “quantization weight coefficient calculation process 2”. The DFT coefficient F _{k, l} ^{(ix, iy)} ( ^ux _, u _y ) and the DCT coefficient C ^{(ix, iy)} [k, l] of the modified base image obtained in “ The process of calculating the prediction error power E [k, l] (numerator of Expression (17)) is performed.

[3] “Quantization weight coefficient calculation process 3”
Input: DFT coefficient of modified base image (k, l = 0,..., N−1,
i _x = 0, ..., M _x -1, i _y = 0, ..., M _y -1)
: DCT coefficient: Motion vector (d _{x, dy} ) of the analysis target block
: Contrast sensitivity function ^ g (η, ω)
Output: Prediction error power ^ E (d _{x, dy} ) [k, l] for the analysis target block (denominator of equation (17))
Here, ^ E (d _x, d _y ) [k, l] is an array process:
(0) The motion vector (d _{x, dy} ) is read from the shift amount storage unit 13 (1) l = 0
(2) k = 0
(3) ^ E (d _x, d _y ) [k, l] = 0
(4) i _y = 0
(5) i _x = 0
(6) ^ S = 0
(7) u _y = 0
(8) u _x = 0
(9) Read the u _{x and} u _y components of the DFT coefficients of the k th and l corrected base images of the position index i _{x and} i _y from the corrected base image DFT coefficient storage unit 1112. (10) Square of complex number read immediately before Calculate norm F _{k, l} ^{(ix, iy)} ( ^ux _, u _y ) ² (11) Calculate _{circumflex over (g) _} (η _x, d _x ) and _{circumflex over (g) _} (η _y, d _y ). The specific calculation is obtained by, for example, Expression (18). (12) ^ S = ^ S + F _{k, l} ^{(ix, iy)} (u _x, u _y ) ² ^ g (η _x, d _x ) ² ^ g ( η _y, d _y ) ²
(13) u _x = u _x +1
(14) If u _x = NM _x , go to the next step, otherwise go to (9) (15) u _y = u _y +1
(16) If u _y = NM _y , go to the next step, otherwise go to (8). (17) From the encoding unit 10, the DCT coefficients C ^{(ix, iy)} [k, l] of the kth and ^lth bases are obtained. Read (18) ^ E (dx _{, dy} ) [k, l] = ^ E (dx _{, dy} ) [k, l] + C ^{(ix, iy)} [k, l] ² ^ S
(19) i _x = i _x +1
(20) If i _x = M _x , go to the next step, otherwise go to (6) (21) i _y = i _y +1
(22) If i _y = M _y , go to the next step, otherwise go to (5) (23) k = k + 1
(24) If k = N, go to the next step, otherwise go to (3) (25) l = l + 1
(26) If l = N, end; otherwise, go to (2). According to this flowchart, the quantization weight coefficient calculation unit 111 performs the “quantization weight coefficient calculation process 4” in the “quantization weight coefficient calculation process 3”. DFT coefficient F _{k, l} ^{(ix, iy)} ( ^ux _, u _y ), DCT coefficient C ^{(ix, iy)} [k, l], and the motion vector of the analysis target block ( d _x, d _y ) and the contrast sensitivity function ^ g (η, ω) as inputs, and the prediction error power ^ E (d _x, d _y ) [k, l] (equation (17) The process of calculating the denominator) is performed.

[4] “Quantization weight coefficient calculation process 1”
Input: E [k, l] (k, l = 0,..., N−1) obtained by “quantization weight coefficient calculation processing 2”
: ^ E (d _{x, dy} ) [k, l] (k, l = 0,..., N−1) obtained in “quantization weight coefficient calculation process 3”
Output: Quantization weight coefficient for the analysis target block
(1) l = 0
(2) k = 0
(3) Read E [k, l] (4) Read ^ E (d _{x, dy} ) [k, l] (5) Calculate quantization weight coefficient W (d _{x, dy} ) [k , L] = E [k, l] / ^ E (d _{x, dy} ) [k, l]
(6) k = k + 1
(7) If k = N, go to the next, otherwise go to (3) (8) l = l + 1
(9) If l = N, the process ends; otherwise, the process proceeds to (2). According to this flowchart, the quantization weight coefficient calculation unit 111 performs the “quantization weight coefficient calculation process 2” in the “quantization weight coefficient calculation process 1”. Analytical object obtained by considering the prediction error power E [k, l] (numerator of equation (17)) for the analytical object block obtained in “and the quantization weighting factor calculation process 3 in consideration of the spatiotemporal visual sensitivity. The prediction error power ^ E (d _{x, dy} ) [k, l] (the denominator of equation (17)) for the block is input,
W (dx _{, dy} ) [k, l] = E [k, l] / ^ E (dx _{, dy} ) [k, l]
According to the calculation formula (formula (17)), a process of calculating the quantization weight coefficient W (d _{x, dy} ) [k, l] for the analysis target block is performed.

In the configuration shown in FIG. 2, the quantization weight coefficient calculation unit 111 adopts a configuration in which the above-described “quantization of the quantization weight coefficient” is not executed, but this “quantization of the quantization weight coefficient” 4, the processing 5 execution unit 1116 is provided as shown in FIG. 4, and as shown in the flowchart of FIG. 5, the number L of representative vectors to be output in step S <b> 105 following step S <b> 104. _A “quantization weight coefficient calculation process 5” is performed in which _w is input and the quantization weight coefficient for the analysis target block is quantized and output.

  Next, a detailed flowchart of the “quantization weight coefficient calculation process 5” will be described.

[5] “Quantization weight coefficient calculation process 5”
Input: Motion vector (d _{x, dy} ) of the analysis target block
: Quantization weight coefficient of analysis target block <W (d _{x, dy} )>
: Number of blocks of analysis target block: Parameters (b (i _x ), b (i _y ) (i _x = 0,..., L _x, i _y = 0,. , L _y ))
: Number of representative vectors to be output L _w
Output: Quantization weight coefficient for the analysis target block
(1) i _y = 0
(2) i _x = 0
(3) The motion vectors d _{x and dy} are (b (i _x −1) ≦ | d _x | <b (i _x ), b (i _y −1) ≦ | d _y | <b (i _y )) quantization weighting coefficient that satisfies _{<W (d x, d y} )> for the set of read vector reads a (4) (3), the LBG algorithm, calculates a representative vector of L _w present. The L _w representative vectors are used as quantization weight coefficients of the (ix _, i _y ) th class. (5) i _x = i _x +1
(6) Next if i _x = L _x , otherwise go to (3) (7) i _y = i _y +1
(8) If i _y = L _y , go to the next, otherwise go to (2) According to this flowchart, the quantization weight coefficient calculation unit 111 performs “quantization weight coefficient calculation” in “quantization weight coefficient calculation processing 5”. A process of quantizing the quantization weight coefficient W (d _{x, dy} ) [k, l] for the analysis target block obtained in the calculation process 1 ″ is performed.

  Finally, an experiment conducted to verify the effectiveness of the present invention will be described.

In this experiment, the following equation (28), which is the reciprocal of equation (17), is used as the sensitivity coefficient W _{k, l} ^s (d _x, d _y ) for the (k, l) basis component of the prediction error signal R _m. Define. In other words, it is necessary to evaluate the distortion amount of the spatio-temporal frequency component that is difficult to detect visually as a relatively small value, and evaluate the distortion amount of the spatio-temporal frequency component that is easily detected as a relatively large value. Therefore, the following equation (28), which is the reciprocal of equation (17), is expressed as a sensitivity coefficient W _{k, l} ^s (d _x, d _y ) for the (k, l) basis component of the prediction error signal R _m. Define as

Then, according to the sensitivity coefficient W _{k, l} ^s (d _x, d _y ), the distortion amount is weighted as shown in the following equation.

Then, the optimum prediction mode is selected using the reference software JSVM (version 8.0. [3]) modified with the RD cost function having the weighting distortion amount, and such weighting is not performed. By selecting the optimal prediction mode using the modified JSVM and considering this sensitivity coefficient W _{k, l} ^s (d _x, d _y ) based on the comparison, the block distortion can be reduced. And confirming that the code amount can be reduced.

The following table shows the experimental conditions. The encoding target sequence has a size of 352 × 288 [pixels] and a frame rate of 30 [fps]. The encoding process was performed on the first 120 frames. The parameters in equation (26) were r ₁ = 8, r ₂ = 6, A = 5. Further, the parameter that gives the size of the reference block is N = 4, and the parameter that gives the size of the analysis target block is M _x = M _y = 4.

The following table shows the comparison results of the code amount. In any sequence and QP value, it can be confirmed that the code amount is reduced by the present invention. It has been confirmed that there is no subjective difference in image quality between the decoded images of both methods. Further, in order to evaluate the relative code amount reduction rate of the present invention with respect to JSVM, the code amount of JSVM and the code amount of the present invention are respectively represented as R _JSVM and R _Ours .
{(R _Ours -R _JSVM ) / R _JSVM } × 100%
The code amount reduction rate represented by the formula is shown in parentheses in the third column of the following table. As a result, it has been confirmed that the present invention achieves an average code amount reduction of 5.3% with respect to JSVM.

  The following table shows the ratio of the encoding mode. Here, the SKIP column indicates the skip mode ratio, the INTER column indicates the inter prediction ratio excluding the skip mode, and the INTRA column indicates the selected ratio of all intra prediction modes. Is shown. As shown in this table, it can be seen that the reduction of the bit rate according to the present invention is realized by selecting many skip modes with a small amount of generated codes.

Furthermore, the evaluation result about block distortion is shown. The decoded pixel value at the position (x, y) of each frame (X × Y [pixel]) is S (x, y), and the inter-pixel difference values in the horizontal and vertical directions are respectively δ _h (x, y) = S. (X + 1, y) -S (x, y)
δ _v (x, y) = S (x, y + 1) −S (x, y)
And At this time,
Δ = {Σ ₁ Σ ₂ | δ _h (Ni _x, i _y ) |} / {2Y (X / N−1)}
+ {Σ ₃ Σ ₄ | δ _v (ix _, Ni _y ) |} / {2X (Y / N−1)}
Where Σ ₁ is the sum of i _x = 1 to (X / N−1)
Σ ₂ is the sum of i _y = 0 to (Y−1)
Σ ₃ is the sum of i _y = 1 to (Y / N−1)
Σ ₄ evaluates the discontinuity between adjacent blocks by using the average value Δ of the inter-pixel difference value at the block boundary expressed by the formula of the sum of i _x = 0 to (X−1).

The table below shows the total frame average of Δ for JSVM and the present invention. In addition, the value in the second column and the value in the third column of each row are set as Δ _JSVM and Δ _Ours respectively.
{(Δ _Ours −Δ _JSVM ) / Δ _JSVM } × 100%
The block distortion reduction rate represented by the formula is shown in parentheses in the third column of the following table. As a result, it was confirmed that the present invention achieved an average 1.2% block distortion reduction with respect to JSVM.

  From this experiment, it was confirmed that according to the present invention, the code amount can be reduced with respect to JSVM, and further, block distortion can be reduced. In the present invention, frequency analysis is performed in consideration of the dependency between adjacent blocks. For this reason, the discontinuity between adjacent blocks is analyzed as a frequency component to which the contrast sensitivity function gives a large weight as horizontal and vertical edges. That is, a large sensitivity coefficient value is set. Since a distortion measure weighted by this sensitivity coefficient is used in the cost function, distortion of components that cause discontinuity between blocks is avoided, and it is considered that block distortion is reduced as a result.

  From this experimental result, it was possible to indirectly verify the effectiveness of the present invention in which the process of calculating the quantization weight coefficient according to the equation (17) was performed.

  The present invention can be applied to a case where a moving image is encoded by performing information compression by transform coding and quantization on a prediction error signal obtained by intra-frame prediction or inter-frame prediction. By applying the invention, it is possible to set a quantization step width that can realize quantization processing that reflects subjective image quality including block distortion, so that highly efficient coding can be realized. At the same time, the amount of code can be reduced.

It is explanatory drawing of a correction base image. It is an apparatus block diagram of the moving image encoder which comprises this invention. It is a flowchart which a quantization weighting coefficient calculation part performs. It is an apparatus block diagram of the moving image encoder which comprises this invention. It is a flowchart which a quantization weighting coefficient calculation part performs. It is explanatory drawing of the discontinuity between blocks.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Moving image encoding apparatus 10 Encoding part 11 Quantization step width determination part 12 Base image memory | storage part 13 Transition amount memory | storage part 101 Quantization step width memory | storage part 102 Quantization step width initial value setting part 103 Quantization part 111 Quantization Weight coefficient calculation unit 112 Quantization weight coefficient storage unit 113 Quantization weight coefficient multiplication unit 1111 Processing 4 execution unit 1112 Modified base image DFT coefficient storage unit 1113 Processing 2 execution unit 1114 Processing 3 execution unit 1115 Processing 1 execution unit 1116 Processing 5 execution Part

Claims (8)

A moving image used in moving picture coding that encodes a moving picture by performing information compression by transform coding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. An image quantization method comprising:
It is defined by associating with the analysis target block consisting of multiple blocks that are the target of the transformation matrix, and is configured by placing the base image of the transformation matrix in one block and embedding zero values in other blocks A process of reading the spatial frequency components of the corrected base image from the storage means for storing the spatial frequency components calculated for the corrected base images for the number of blocks
Estimating a shift amount indicating temporal movement of the image signal of the analysis target block;
Calculating a visual sensitivity value assigned to the spatial frequency component based on the spatial frequency index of the read spatial frequency component and the shift amount, and weighting the spatial frequency component;
Based on the weighted spatial frequency component, the non-weighted spatial frequency component, and the transform coefficient of the block constituting the analysis target block, the quantization step width set for each base component of the transform matrix The process of calculating the weighting factor for
A process of determining a quantization step width used for quantization processing by modifying a preset quantization step width using the weighting factor ,
In the process of calculating, a multiplication value of the square sum of the transform coefficients and the square norm sum of the weighted spatial frequency components, and a square sum of the square sum of the transform coefficients and the square frequency norm sum of the non-weighted spatial frequency components. Obtaining a multiplication value and calculating the weighting factor according to a division value of the two multiplication values ;
A moving image quantization method as a feature. The moving image quantization method according to claim 1,
In the weighting process, a visual sensitivity value in the horizontal direction is calculated based on the horizontal spatial frequency index and the horizontal component of the shift amount, and based on the vertical spatial frequency index and the vertical component of the shift amount. Calculating a visual sensitivity value assigned to the read spatial frequency component by calculating a vertical visual sensitivity value,
A moving image quantization method as a feature. In the moving image quantization method according to claim 1 or 2 ,
In the process of determining, determining a quantization step width used for a quantization process by multiplying the weighting factor and a predetermined quantization step width that does not depend on a base component of a transformation matrix set in advance.
A moving image quantization method as a feature. A moving image used in moving picture coding that encodes a moving picture by performing information compression by transform coding and quantization on an image signal or a prediction error signal obtained by intra-frame prediction and inter-frame prediction. An image quantization device,
It is defined by associating with the analysis target block consisting of multiple blocks that are the target of the transformation matrix, and is configured by placing the base image of the transformation matrix in one block and embedding zero values in other blocks Storage means for storing spatial frequency components calculated for the corrected base images for the number of blocks
Estimating means for estimating a shift amount indicating temporal movement of the image signal of the analysis target block;
The spatial frequency component of the corrected base image is read from the storage means, and the visual sensitivity value assigned to the spatial frequency component is calculated based on the spatial frequency index of the spatial frequency component and the shift amount, and the spatial frequency is calculated. A weighting means for weighting the components;
Based on the weighted spatial frequency component, the non-weighted spatial frequency component, and the transform coefficient of the block constituting the analysis target block, the quantization step width set for each base component of the transform matrix Calculating means for calculating a weighting factor for
Using the weighting factor, to modify the quantization step width is set in advance, e Bei and determining means for determining the quantization step width used in the quantization process,
The calculating means multiplies a multiplication value of a square sum of the transform coefficients and a square norm sum of the weighted spatial frequency components, and a multiplication of a square sum of the transform coefficients and a square norm sum of the non-weighted spatial frequency components. And calculating the weighting factor according to a division value of the two multiplication values ,
A moving image quantizing device. The moving picture quantization apparatus according to claim 4 , wherein
The weighting means calculates a visual sensitivity value in the horizontal direction based on a horizontal spatial frequency index and a horizontal component of the shift amount, and based on a vertical spatial frequency index and the vertical component of the shift amount. By calculating the visual sensitivity value in the vertical direction, calculating the visual sensitivity value assigned to the read spatial frequency component,
A moving image quantizing device. The moving image quantization apparatus according to claim 4 or 5 ,
The determining means determines the quantization step width used for the quantization process by multiplying the weighting factor by a predetermined quantization step width that does not depend on a base component of a transformation matrix set in advance.
A moving image quantizing device. A moving picture quantization program for causing a computer to execute the moving picture quantization method according to any one of claims 1 to 3 . A computer-readable recording medium on which a moving image quantization program for causing a computer to execute the moving image quantization method according to any one of claims 1 to 3 is recorded.

Images