JP4681011B2 - Moving picture coding method, moving picture coding apparatus, moving picture coding program, and computer-readable recording medium recording the program - Google Patents

Description

  The present invention relates to a moving picture coding method and apparatus for coding a moving picture by performing information compression by transform coding and quantization on a prediction error signal obtained by inter-frame prediction, and the moving picture thereof. The present invention relates to a moving image encoding program used for realizing an encoding method and a computer-readable recording medium storing the program.

[Encoding method using cost function of square error criterion]
H. In H.264, with the introduction of intra prediction and variable shape motion compensation, the types of prediction modes are increasing compared to the conventional standardized method. For this reason, in order to reduce the amount of codes while maintaining a constant subjective image quality, it is necessary to select an appropriate prediction mode. H. In the H.264 reference software JM (see Non-Patent Document 1), the following prediction mode that minimizes the RD cost is selected. In the following notation, the symbol ^ in “^ X” (where X is a letter) indicates a symbol attached to “X”.

Here, S is an original signal, q is a quantization parameter, m is a number representing a prediction mode, ^ S _{m, q} is predicted for the original signal S using the prediction mode m, and the quantization parameter q is It is a decoded signal when quantized by using. Further, λ is a Lagrange's undetermined multiplier used for mode selection. Further, D (S, ^ S _{m, q} ) is a sum of square errors shown in the following equation.

S ^Y , S ^U , and S ^V are Y, U, and V components of the original signal, respectively, and ^ S ^Y _{m, q} , ^ S ^U _{m, q} , and ^ S ^V _{m, q} are Y of the decoded signal, respectively. , U and V components. Further, x ₀ and y ₀ are coordinate values at positions closest to the origin in the block.

  H. The calculation method of the decoded signal in H.264 is shown below. The symbols used for the explanation are summarized in the following table.

Let P _m be the prediction signal when the prediction method of mode number m is used for the original signal S. H. In the H.264 encoding process, the orthogonal transformation using the transformation matrix to Φ is applied to the prediction error signal R _m (= S−P _m ) when the prediction method of the mode number m is used as follows: Is done. In the following notation, the symbol “˜X” (where X is a letter) indicates a symbol attached to “X”.

Here, ~ Φ ^t represents a transposed matrix for the transformation matrix ~ Φ. Note that the transformation matrix to Φ is an orthogonal matrix of integer elements expressed by the following equation.

  Next, since the matrices Φ are non-normal matrices, processing corresponding to matrix normalization as shown below is performed.

  This is equivalent to using Φ of the following expression instead of ˜Φ in the expression (3).

Further, the quantization using the quantization parameter q is performed on C _m as follows. H. In the H.264 reference software JM, normalization is built into quantization.

  On the other hand, H. In the H.264 decoding process, V is inversely quantized as in the following equation to obtain a decoded value of a transform coefficient.

Next, inverse transformation is applied to { _circumflex over (C _{)} m, q as} in the following equation to obtain a prediction error decoded signal.

  Finally, a decoded signal of the encoding target image is obtained by the following equation.

KPLim and G. Sullivan and T. Wiegand, Text Description of Joint Model Reference Encoding Methods and Decoding Concealment Methods.Joint Video Team (JVT) of ISO / IEC MPEG and ITU-T VCEG, JVT-R95, Jan., 2006. http://ftp3.itu.ch/av-arch/jvt-site/2006_01_Bangkok/JVT-R095.zip

[Weighting distortion amount considering subjective image quality]
As described above, H.P. The measure of subjective image quality used in the H.264 reference software JM is a square error. However, this square error is not necessarily a distortion amount reflecting subjective image quality degradation. For example, a change in a high frequency component is less visually detected than a change in a low frequency component.

  However, an encoder (for example, JM) that does not use such visual characteristics still has room for improvement in terms of efficient code amount reduction.

  In view of this, there is a demand for a study utilizing the difference in visual sensitivity with respect to spatio-temporal frequency components. A distortion amount corresponding to the subjective image quality is defined by weighting the distortion amount for each spatial frequency component in accordance with the visual sensitivity with respect to the orthogonal transform coefficient. Further, in consideration of the visual sensitivity in the time direction, the above-described weighted distortion amount is further weighted according to the shift amount. Thus, it is necessary to use a distortion amount weighted on the basis of spatio-temporal visual sensitivity in a cost function for selecting an encoding parameter.

  The weighting based on the visual sensitivity for the quantization error signal will be described below. In the present invention, it is assumed that the RD cost of the following equation is used.

Here, C _m is a transform coefficient for the prediction residual signal R _m when the mode number m is used, and ^ C _{m, q} is a transform obtained by quantizing and dequantizing C _m with the quantization parameter q. This is the decoded value of the coefficient. The following weighted distortion amount is used as the distortion amount used for calculating the RD cost.

Here, C _m ^{s (i)} [k, l] (s = Y, U, V) is an element of C _m , and a macroblock (16 × 16 [pixel], U, V in the case of Y component) In the case of components, it is a conversion coefficient included in the i-th sub-block scanned in the raster scanning among the sub-blocks (N × N [pixels]) in 8 × 8 [pixels]). ^ C _{m, q} ^{s (i)} [k, l] (s = Y, U, V) is an element of ^ C _{m, q} , and a macroblock (16 × 16 [pixels in the case of Y component) ], U, and V components, the decoding transform coefficients included in the i-th sub-block scanned in the raster scan among the sub-blocks (N × N [pixels]) in 8 × 8 [pixels]). is there.

Further, W _{k, l} ^s (s = Y, U, V) is a weighting coefficient set to 1 or less, and is hereinafter referred to as a sensitivity coefficient. In Expression (12), setting W _{k, l} ^s to a small value corresponds to estimating the quantization distortion to D (C _m , ^ C _{m, q} ) to be small.

From the normality of orthogonal transformation, if W _{k, l} ^s = 1 (∀k, l; s = Y, U, V), the above-described weighted distortion amount is equivalent to the square error sum.

In order to define the distortion amount corresponding to the subjective image quality, it is necessary to determine the sensitivity coefficient W _{k, l} ^s (s = Y, U, V) in consideration of the spatial frequency and the temporal frequency.

However, this point has not been studied in the prior art. The higher the spatial frequency and the temporal frequency, the smaller the value of the sensitivity coefficient W _{k, l} ^s must be. However, the fact is that this point has not been studied.

Further, in determining the sensitivity coefficient W _{k, l} ^s , it is necessary to consider the direction of the spatial frequency component and the direction of the motion vector.

  That is, the spatio-temporal frequency obtained based on the spatial frequency component and the motion vector in the block depends on the direction of the spatial frequency component and the direction of the motion vector. This is because the movement can be recognized by the movement in the direction perpendicular to the edge change direction. For example, as shown in FIG. 13, when an edge that changes in the vertical direction moves, only the direction caused by the vertical component of the motion vector can be recognized as a motion.

  However, in the prior art, modeling considering such a direction is not naturally performed. For this reason, there is a case where the obtained weighted distortion amount cannot correctly reflect the subjective image quality.

  The present invention has been made in view of such circumstances, and is directed to a spatial frequency component direction and a block-based encoding scheme (for example, H.264) that combines inter-frame prediction based on motion compensation and orthogonal transformation. Estimating spatio-temporal frequency by considering the direction of the motion vector, and realizing the introduction of a measure of coding distortion that appropriately evaluates subjective image quality, The object is to provide a new moving picture coding technique that can realize what reflects subjective image quality.

  In the present invention, the distortion amount in each block is weighted based on the spatiotemporal visual sensitivity function. In the calculation of the weighting coefficient, the input is a transformation matrix and a shift amount (indicating temporal motion of an image signal such as a motion vector).

  In the following, a case where an image having a vertical width H is observed at a viewing distance rH is considered. r is called a viewing distance parameter. Also, in the following, in order to simplify the expression, the subscript indicating the distinction between Y, U, and V is omitted, and the Y component will be discussed. The U and V components can be discussed in the same manner as described below.

[I] Sensitivity coefficient calculation method (1)
When the k-th column vector (N-dimensional vector) of the transformation matrix Φ (N × N matrix) is φ _k , the base image for the matrix is
f _{k, l} (x, y) = φ _k [y] φ _l [x] ^t (0 ≦ x, y ≦ N−1)
It can be obtained from the formula H. In the case of H.264, the possible value for N is either 4 or 8. Here, φ _l ^t is a transposed vector of φ _l .

For each base image f _{k, l} (x, y) (0 ≦ x, y ≦ N−1), the size is doubled by zero padding as shown in the following equation. This block expansion is for performing a spatial frequency analysis in consideration of a spatial frequency component resulting from the discontinuity of the block.

˜f _{k, l} (x, y) (x = 0,..., 2N−1; y = 0,..., 2N−1) obtained as a result of zero padding is called a corrected base image.

The modified base image is subjected to a discrete Fourier transform represented by the following equation, and Fourier coefficients F _{k, l} (u _x , u _y ) (0 ≦ u _x ≦ 2N−1, 0 ≦ u _y ≦ 2N−1) are obtained. obtain. In the following, it is assumed that N = 2 ^m . Here, j is an imaginary unit. U _x and u _y are referred to as spatial frequency indexes.

The following weighting based on visual sensitivity is performed on the obtained Fourier coefficient F _{k, l} (u _x , u _y ) (0 ≦ u _x ≦ 2N−1, 0 ≦ u _y ≦ 2N−1).

Hereinafter, ~ F _{k, l} (u _x , u _y , ω _x , ω _y ) will be described. Here, {circumflex over (g)} (η, ω) is a function set based on the visual sensitivity function g (η, ω). First, the visual sensitivity function g (η, ω) will be described, and then the relationship between g (η, ω) and ^ g (η, ω) will be shown.

  The visual sensitivity function g (η, ω) is as follows. When η and ω are increased according to this function form, the visual sensitivity function g (η, ω) indicates a small value.

Here, a, b, c, and d are parameters (hereinafter referred to as model parameters) that define the function form of the visual sensitivity function g (η, ω).
(A, b, c, d) = (6.1, 7.31, 2, 45.9)
A value such as is used.

(A) η η is a spatial frequency (cycle per degree) representing the number of light-dark pairs within a unit viewing angle. In the equation (15), η _x and η _y are horizontal and vertical spatial frequencies, respectively.
η _x = θ (r, H) u _x / 4N
η _y = θ (r, H) u _y / 4N
It is expressed as

Here, θ (r, H) is an angle per pixel (degree per pixel) when an image having a vertical width H is observed at a viewing distance rH.
θ (r, H) = (1 / H) × arctan (1 / r) × (180 / π) [degrees / pixel]
Is given by the expression

(B) About ω ω is an angle change per unit time [degrees / sec]. In equation (15), ω _x and ω _y are the amount of change in angle per unit time in the horizontal and vertical directions, respectively.

When the macroblock displacement is estimated as (d _x , d _y ) and an image with a vertical width H is observed at a viewing distance rH, the amount of change in the angle per unit time in the horizontal direction and the vertical direction is
^{_{ω x = tan -1 (f r}} d x / rH)
^{_{ω y = tan -1 (f r}} d y / rH)
Is given by the expression Here, _fr is a frame rate.

Here, if it is noted that the upper limit of the number of cycles per unit time (cycle rate) is equal to or less than the frame rate, ω _x × η _x , ω _y × η _y ([degrees / sec] × [cycles / degree] ≡ [cycles / sec]) must be set to a value below the frame rate.

  Therefore, a function obtained by correcting g (η, ω) according to the magnitude relationship between the cycle rate and the frame rate is defined as ^ g (η, ω) as follows.

In the sensitivity coefficient calculation method (part 1), the Fourier coefficient F _{k, l} (u _x , u _y ) and the Fourier coefficient F _{k, l} (u _x , u _y ) are weighted using equation (15). was _{~F k, l (u x,} u y, ω x, ω y) and uses, (k, l) sensitivity coefficient W _k of the ^{_{basis, l Y (ω x, ω}} y) , and the following equation power Define as a ratio.

Note that W _{k, l} ^U (ω _x , ω _y ) and W _{k, l} ^V (ω _x , ω _y ) can be obtained in the same manner. At this time, it is also possible to change the model parameter with the luminance component and the color difference component.

The sensitivity coefficient W _{k, l} ^s (ω _x , ω _y ) (s = Y, U, V) determined in this way becomes smaller as the spatial frequency and temporal frequency are higher. It becomes possible to define the amount of distortion corresponding to.

  The sensitivity coefficient can be obtained independently of the encoding target image. For this reason, the sensitivity coefficient can be obtained prior to encoding. If this sensitivity coefficient is stored in the lookup table, the calculation for calculating the sensitivity coefficient at the time of encoding can be omitted.

[Ii] Sensitivity coefficient calculation method (2)
Next, another method for calculating the sensitivity coefficient will be described.

In the sensitivity coefficient calculation method (part 2), the Fourier coefficient obtained by the equation (14) is also weighted based on visual sensitivity. In the sensitivity coefficient calculation method (part 2), the weighting is performed. G (η) = (0.31 + 0.69η) exp (−0.29η) (Equation (a))
The visual sensitivity function is used.

References: KNNgan, KSLeong, and H. Singh. Cosine transform coding incorp orating human visual system moodel. SPIE Fiber, pp.165-171, Sept. 1986.
Here, η is a spatial frequency (cycle per degree) representing the number of light-dark pairs within a unit viewing angle. Note that η is a one-dimensional spatial frequency.

At this time, the η and the spatial frequency index u (either u _x or u _y) is
η (u, r) = θ (r, H) u / 4N Expression (b)
There is a relationship.

Note that θ (r, H) is an angle per pixel (degree per pixel) when an image having a vertical width H is observed at a viewing distance rH.
θ (r, H) = (1 / H) × arctan (1 / r) × (180 / π) Equation (c)
Is given by the expression Here, r is called a viewing distance parameter.

  The expression (b) and the expression (c) are substituted into the expression (a), and g (η) expressed as a function of u and r is denoted by g (u, r).

In the sensitivity coefficient calculation method (part 2), this ^ g (u, r) is used to calculate the Fourier coefficient F _{k, l} (u _x , u _y ) (0 ≦ u _x ≦ 2N−1, 0 ≦ u _y). For ≦ 2N−1), the following weighting is performed.

Here, _{ circumflex over (g) _} (u _x , r _x ) and _{ circumflex over (g) _} (u _y , r _y ) are weighting functions in the horizontal direction and the vertical direction, respectively. In addition, r _x, will be described later method of deriving the r _y.

In the sensitivity coefficient calculation method (part 2), the Fourier coefficient F _{k, l} (u _x , u _y ) and the Fourier coefficient F _{k, l} (u _x , u _y ) are weighted using equation (19). to F _k, using a _{l (u x, u y,} r x, r y) were, first (k, l) sensitivity coefficient of the basis of the components ^{_{W k, l Y (r x}} , r y) the following It is defined as the power ratio of the equation.

  The sensitivity coefficient can be obtained independently of the encoding target image. For this reason, the sensitivity coefficient can be obtained prior to encoding. If this sensitivity coefficient is stored in the lookup table, the calculation for calculating the sensitivity coefficient at the time of encoding can be omitted.

Next, a method for deriving the viewing distance parameters r _x and r _y described in Expression (19) will be described.

  The visual sensitivity with respect to the spatial frequency depends on the amount of change in the time axis direction. For example, in a frame in which a large change in the time axis direction (high-speed camera pan / tilt, scene change, etc.) has occurred, the sensitivity of the frame to image quality degradation decreases. For this reason, for a frame in which a large change in the time axis direction has occurred, a further reduction in code amount can be expected by controlling the sensitivity coefficient to be a small value.

Here, attention is focused on the relationship between the sensitivity coefficient and the viewing distance parameter. It can be seen that W _{k, l} (r _x , r _y ) is a decreasing function with respect to r _x , r _y . This is because _{{circumflex over} (g) _} (u, r) constituting ˜F _{k, l} (u _x , u _y , r _x , r _y ) is a decreasing function with respect to r.

The nature of this sensitivity coefficient matches the visual characteristic that visual sensitivity decreases with viewing distance. W _{k, l} (r _x , r _y ) is
W _{k, l} (r _x , r _y ) ≦ 1
Satisfy the relationship. This is because ^ g (u, r) takes a value of 1 or less.

Therefore, in the sensitivity coefficient calculation method (part 2), the viewing distance parameter is used to control the sensitivity coefficient. This control is performed for each macro block in accordance with the magnitude of the shift amount (the one indicating the temporal movement of the image signal such as a motion vector). The shift amount d = (d _x , d _y ) used at this time is obtained with the block size of the shift amount estimation being 16 × 16.

Then, the viewing distance parameter is adaptively changed as shown in the following equation in accordance with the obtained shift amount d. Here, A is a threshold value, and r ₁ > r ₂ .

In this equation (21), r _x and r _y are independently set as parameters for the horizontal direction and the vertical direction. This is based on the fact that the detection function of the visual system reacts only to movement in a direction perpendicular to the spatial edge direction. That is, the horizontal component of motion is not detected in a texture region composed of horizontal edges, and similarly, the vertical component of motion is not detected in a texture region composed of vertical edges. For this reason, the viewing distance parameter as the control parameter for the sensitivity coefficient is set for each of the horizontal and vertical directions.

On the other hand, the present inventor disclosed an invention for determining the sensitivity coefficient W _{k, l} ^s (ω) (s = Y, U, V) according to a similar technique in the following literature.

References: Yukihiro Bando, Kazuya Hayase, Masayuki Takamura, Hitoshi Uekura, Yoshiyuki Yashima, H. Based on spatio-temporal visual sensitivity characteristics. H.264 / AVC mode selection method The Institute of Image Information and Television Engineers 2007 Annual Conference, 7-4, 2007
However, in the invention disclosed in this document, ~ F _{k, l} (u, v, ω) corresponding to ~ F _{k, l} (u _x , u _y , ω _x , ω _y ) described in the equation (15). As
˜F _{k, l} (u, v, ω) = F _{k, l} (u, v) ^ g (η, ω)
Without any distinction between η _x and η _y for η,
η = θ (r, H) (u ² + v ² ) ^1/2 / 2N
And without distinguishing the direction of ω _x , ω _y for ω,
ω = tan ⁻¹ (f _r (d _x ² + d _y ² ) ^1/2 / rH)
It is determined according to the formula.

As described above, in the invention disclosed in this document, the sensitivity coefficient W _{k, l} ^s (ω) is determined in consideration of the spatial frequency and the time frequency (variation amount), but the spatial frequency component is determined. However, the sensitivity coefficient W _{k, l} ^s (ω) is not determined in consideration of the direction and the amount of displacement, and there remains room for improvement.

That is, in the invention disclosed in this document, the sensitivity coefficient W _{k, l} ^s (ω) is not determined in consideration of the direction of the spatial frequency component and the direction of the shift amount. There is a problem that the image quality of the decoded pixel is deteriorated only by securing the excessive weight with respect to the distortion amount.

Therefore, in the present invention, when the sensitivity coefficient calculation method (part 1) is used, ~ g (η, ω) is decomposed into the x direction and the y direction, and ~ F _{k, l} (ux ₎ , U _y , ω _x , ω _y ) are defined as in equation (15), and are decomposed into η _x , η _{y in} order to consider the direction of η, and these η _x , η _y are respectively Independently
η _x = θ (r, H) u _x / 4N
η _y = θ (r, H) u _y / 4N
Defined and so, further, in order to consider the direction of the omega omega _x, decompose so as omega _y, these omega _x, omega _y and independently,
^{_{ω x = tan -1 (f r}} d x / rH)
^{_{ω y = tan -1 (f r}} d y / rH)
In this way, the sensitivity coefficient W _{k, l} (ω _x , ω _y ) is determined in consideration of the direction of the spatial frequency component and the direction of the shift amount.

And in this invention, when it comprises with the calculation method (the 2) of a sensitivity coefficient, g ((eta)) is set to
η (u, r) = θ (r, H) u / 4N
θ (r, H) = (1 / H) × arctan (1 / r) × (180 / π)
Is extended to u, r function ^ g (u, r), and this ^ g (u, r) is decomposed into x-direction and y-direction, ~ F _{k, l} (u _x , u _The sensitivity coefficient W _{k, l} (r _x , r _y ) is determined in consideration of the direction of the spatial frequency component and the direction of the shift amount by defining _y , r _x , r _y ) as in equation (19). The composition of doing is taken.

  In accordance with this configuration, according to the present invention, it is possible to introduce a measure of coding distortion that appropriately evaluates subjective image quality, thereby reflecting subjective image quality as a cost function used for coding parameter selection. Will be realized.

  Next, the configuration of the moving picture coding apparatus constructed according to the present invention will be described.

  When the moving picture coding apparatus according to the present invention is configured by the sensitivity coefficient calculation method (part 1) described above, the prediction error signal obtained by the inter-frame prediction is transformed by transform coding and quantization. When adopting a configuration for encoding a moving image by performing information compression, (1) a first calculation means for calculating a spatial frequency component of a base image of a transformation matrix, and (2) an image of an encoding target block Estimating means for estimating a shift amount indicating temporal movement of the signal; (3) a horizontal component of the spatial frequency of the spatial frequency component calculated by the first calculating means, and a horizontal component of the shift amount estimated by the estimating means; Based on the vertical direction based on the vertical component of the spatial frequency of the spatial frequency component calculated by the first calculation means and the vertical component of the displacement estimated by the estimation means. Direction A second calculating means for calculating a sensitivity function; and (4) weighting the spatial frequency component calculated by the first calculating means based on the visual sensitivity functions in the horizontal direction and the vertical direction calculated by the second calculating means. And (5) importance calculated by the third calculation means based on the weighted spatial frequency components, and (5) importance calculated by the third calculation means. And determining means for determining a parameter to be used for encoding the block to be encoded by evaluating the encoding cost by using the amount of encoding distortion weighted using.

  In addition, when the moving picture coding apparatus according to the present invention is configured by the sensitivity coefficient calculation method (part 2) described above, transform coding and quantization are performed on a prediction error signal obtained by interframe prediction. (1) First calculation means for calculating a spatial frequency component of a base image of a transformation matrix, and (2) an encoding target block, when adopting a configuration for encoding a moving image by performing information compression by encoding , An estimation means for estimating a shift amount indicating temporal movement of the image signal, and (3) a view necessary for calculating a visual sensitivity function in the horizontal direction based on the horizontal component of the shift amount estimated by the estimation means. A distance parameter is determined, a visual sensitivity function in the horizontal direction is calculated based on the horizontal component of the spatial frequency of the spatial frequency component calculated by the first calculation means and the determined viewing distance parameter, and the estimation means of Based on the vertical component of the determined shift amount, a visual distance parameter necessary for calculating the visual sensitivity function in the vertical direction is determined, and the vertical component of the spatial frequency of the spatial frequency component calculated by the first calculation means And a second calculation means for calculating a visual sensitivity function in the vertical direction based on the determined visual distance parameter, and (4) based on the visual sensitivity functions in the horizontal direction and the vertical direction calculated by the second calculation means. The third calculation for weighting the spatial frequency component calculated by the first calculation means and calculating the importance of each spatial frequency component calculated by the first calculation means based on the weighted spatial frequency component And (5) an encoding cost is evaluated using an amount of encoding distortion weighted by using the importance calculated by the third calculating means, so that the parameter used for encoding the encoding target block is calculated. Configured to and a determination means for determining.

  Here, the third calculation means calculates a ratio value of the sum of squares for each spatial frequency of the weighted spatial frequency component and the sum of squares for each spatial frequency of the spatial frequency component that is not weighted, The importance of each spatial frequency component calculated by the first calculation means may be calculated.

  The first calculation unit may calculate a spatial frequency component using a corrected base image in which a zero value is embedded around the base image.

  The moving image encoding method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is provided by being recorded on a suitable computer-readable recording medium. Alternatively, the present invention is realized by being provided via a network, installed when the present invention is carried out, and operating on a control means such as a CPU.

  The present invention estimates the spatio-temporal frequency by taking into account the direction of the spatial frequency component and the direction of the shift amount for the block-based coding method combining inter-frame prediction by motion compensation and orthogonal transform. Introduce a measure of coding distortion with appropriate evaluation of image quality.

  Thus, according to the present invention, it is possible to realize a cost function that reflects subjective image quality as a cost function used for selecting an encoding parameter, thereby realizing highly efficient encoding.

  Hereinafter, the present invention will be described in detail according to embodiments.

  First, the present invention will be described in detail according to an embodiment constituted by a sensitivity coefficient calculation method (part 1).

  FIG. 1 illustrates a device configuration of a moving image encoding device 1 to which the present invention is applied.

  The moving image encoding apparatus 1 to which the present invention is applied is an H.264 image taking into account spatiotemporal visual sensitivity. H.264 encoding a moving image, and as shown in this figure, a coding parameter selection unit 10 that selects a coding parameter used for coding a coding target macroblock, and a coding parameter selection And an encoding unit 20 that encodes the encoding target macroblock using the encoding parameter selected by the unit 10.

  2 to 4 show examples of flowcharts executed by the encoding parameter selection unit 10. Here, in this flowchart, it is assumed that the encoding parameter selection unit 10 selects an optimal prediction mode as an encoding parameter.

  When selecting the optimal prediction mode used for encoding the encoding target macroblock, the encoding parameter selection unit 10 firstly sets the initial prediction mode in step S101 as shown in the flowchart of FIG. Set the value (prediction mode to be the initial value).

  Subsequently, in step S102, an initial cost indicating a large value is stored in a register for storing a minimum cost (hereinafter also referred to as a minimum cost register), and a register for storing an optimal prediction mode (hereinafter, referred to as a minimum cost register). These registers are initialized by storing meaningless values for (sometimes referred to as optimal mode registers).

Subsequently, in step S103, the shift amount (the above-mentioned (d _x , _dy )) of the encoding target macro block is estimated. This estimation method is given from the outside. For example, H.M. It is also possible to use a motion vector calculated by the H.264 reference software JM as an estimated value of the shift amount used below. Alternatively, the estimated shift amount can be calculated according to a standard that minimizes the sum of absolute value errors between the encoding target macroblock and the reference macroblock.

  Subsequently, in step S104, the set prediction mode of the processing target, the prediction vector based on the prediction mode, the quantization parameter, the encoding target frame signal, and the reference frame signal are input, and encoding is performed using the prediction mode. The code amount in the case is calculated. The specific calculation method is as follows. According to the H.264 reference software JM method.

  Subsequently, in step S105, first, a weight considering the spatiotemporal visual sensitivity is determined based on the amount of transition estimated in step S103, and then, the prediction mode to be processed and the prediction based on the prediction mode are determined. Calculates the weighted distortion amount when encoding using the prediction mode based on the input signal and the determined weight based on the vector, quantization parameter, encoding target frame signal, and reference frame signal. To do. A specific calculation method will be described with reference to the flowcharts of FIGS.

  Subsequently, in step S106, the prediction mode to be processed and the quantization parameter that have been set are input, and an undetermined multiplier for encoding using the prediction mode is calculated.

  Subsequently, in step S107, based on the code amount calculated in step S104, the weighted distortion amount calculated in step S105, and the undetermined multiplier calculated in step S106, the RD cost represented by Expression (11) is used. Is calculated.

  Subsequently, in step S108, the calculated RD cost is compared with the cost stored in the minimum cost register, and the calculated RD cost is greater than the cost stored in the minimum cost register. If it is determined that the value is smaller, the process proceeds to step S109, where the calculated RD cost is stored in the minimum cost register, and in step S110, the identification information of the prediction mode to be processed is set in the optimum mode register. To store. On the other hand, when it is determined in step S108 that the calculated RD cost is higher than the cost stored in the minimum cost register, the processes in steps S109 and 110 are omitted.

  Subsequently, in step S111, it is determined whether or not processing has been performed for all prediction modes, and when it is determined that processing has not been performed for all prediction modes, the process proceeds to step S112, and unprocessed according to a predetermined order. After the prediction mode to be processed is updated by selecting one prediction mode from among the prediction modes, the process returns to step S104.

  On the other hand, when it is determined in step S112 that all the prediction modes have been processed, the process proceeds to step S113, the prediction mode stored in the optimum mode register is output as the optimum prediction mode, and the process ends.

  Next, according to the flowchart of FIG. 3, the weighted distortion amount calculation process executed in step S105 of the flowchart of FIG. 2 will be described.

  The calculation processing of the weighted distortion amount is executed by calculating the calculation formula of Formula (12). In the following flowchart, for convenience of explanation, the transformation coefficient summation process for sub-block i described in Expression (12) is omitted as being included in the macroblock transformation coefficient summation process. .

When the encoding parameter selection unit 10 enters the process of step S105 in the flowchart of FIG. 2, first, as shown in the flowchart of FIG. 3, in step S201, ^ C _{m, q} ^{s in} equation (12). ⁽ⁱ⁾ Each transform coefficient represented by [k, l] (s = Y, U, V) is obtained by decoding.

Subsequently, in step S202, the shift amount (the aforementioned (d _x , _dy )) of the encoding target macroblock estimated in step S103 of the flowchart of FIG. 2 is read. This amount of change is necessary for executing the flowchart of FIG. 4, and is read at this stage.

  Subsequently, in step S203, a counter k designated for the kth and lth component value k of the transform coefficient is initialized to 0, and a counter l designated for the kth and lth component value l of the transform coefficient is initialized to 0. Further, the register S is initialized to 0.

Subsequently, in step S204, | C _m ^{s (i)} [k, l] − ^ C _{m, q} ^{s (i)} [k, l] | ² described in equation (12)
However, s = Y, U, V
The amount of coding distortion of the kth and lth components of the transform coefficient is calculated according to

Subsequently, in step S205, sensitivity coefficients W _{k, l} ^s (s = Y, U, V) of the k-th and l-th components of the conversion coefficient described in Expression (12) are set. A specific setting method will be described with reference to the flowchart of FIG.

Subsequently, in step S206, the transform coefficient is calculated by multiplying the amount of coding distortion calculated in step S204 by the sensitivity coefficient W _{k, l} ^s (s = Y, U, V) set in step S205. The weighted distortion amount for the kth and lth components is calculated.

  Subsequently, the multiplication value calculated in step S206 is added to the register S in step S207.

  Subsequently, in step S208, it is determined whether or not all components of the transform coefficient have been processed, and when it is determined that all components of the transform coefficient have not been processed, the process proceeds to step S209, where all components of the transform coefficient are processed. 1 is added to the counter k or the counter l, and the process returns to step S204.

  On the other hand, when it is determined in step S208 that all components of the transform coefficient have been processed, it is determined that the weighted distortion amount calculation processing has been completed, and the process proceeds to step S210 to store the weight stored in the register S. The attached distortion amount is output and the process is terminated.

Next, the sensitivity coefficient W _{k, l} ^s setting process executed in step S205 of the flowchart of FIG. 3 will be described with reference to the flowchart of FIG.

The coding parameter selection unit 10 sets the sensitivity coefficient W _{k, l} ^s (s = Y, U, V) of the k-th and l-th components of the transform coefficient by entering the process of step S205 in the flowchart of FIG. If necessary, as shown in the flowchart of FIG. 4, first, in step S301, the shift amount of the encoding target macroblock read in step S202 of the flowchart of FIG. 3 (the above-mentioned (d _x , d _y )). ). Subsequently, in step S302, the size of the transformation matrix is read.

Subsequently, in step S303, the kth and lth base images (f _{k, l} (x, y) described above) are calculated, and in step S304, zeros are embedded in the calculated base images. A modified base image having a size twice that of the base image is generated. That is, the modified base image represented by the equation (13) is generated.

Subsequently, in step S305, spatial frequency analysis is performed by performing discrete Fourier transform on the generated corrected base image. That is, the Fourier coefficient F _{k, l} (u _x, u _y ) is calculated by applying the discrete Fourier transform expressed by the equation (14) to the generated corrected base image.

  Subsequently, in step S306, the visual sensitivity function g (η, ω) represented by the equation (16) is read.

Subsequently, in step S307, the counter ux specifying the values u _x of the u _x component of the spatial frequency component of the corrected base image is initialized to 0, the u _y component values u _y of the spatial frequency component of the modified base image Is initialized to 0, the register W1 [k, l] is initialized to 0, and the register W2 [k, l] is initialized to 0.

Then, in step S308-1, relative to the u _x component and displacement amount of d _x of the spatial frequency component of the modified base image, and calculates the corresponding value of visual sensitivity function, in step S308-2, modified respect to the first u _y component and displacement amount of d _y spatial frequency components of the base image, and calculates the corresponding value of visual sensitivity function.

That is, as mentioned above,
_{η x = θ (r, H} ) u x / 4N, ω x = tan -1 (f r d x / rH)
Is calculated and substituted into the visual sensitivity function g (η _, ω) to calculate g (η _x, ω _x ) _, which is the corresponding value of the visual sensitivity function,
_{η y = θ (r, H} ) u y / 4N, ω y = tan -1 (f r d y / rH)
Is calculated and substituted into the visual sensitivity function g (η _, ω) to calculate g (η _y, ω _y ) _, which is a corresponding value of the visual sensitivity function. In calculating g (η _x, ω _x ) and g (η _y, ω _y ), equation (17) is considered.

Subsequently, in step S309, the multiplying calculation value of visual sensitivity function calculated in step S308-1,2 first u _x of the spatial frequency component of the modified base image, the u _y component. That is, as shown in equation (15), the Fourier coefficient _{F k, l (u x,} u y) with respect to, is the multiplying _{g (η x, ω x)} × g (η y, ω y) of .

Subsequently, in step S310, the square value of the multiplication value obtained in step S309 is calculated, and the calculated value is added to the register W1 [k, l]. That is, the square value of F _{k, l} (ux _, u _y ) × g (η _x, ω _x ) × g (η _y, ω _y ) is obtained in the register W1 in order to obtain the numerator value of the equation (18). It is added to [k, l].

Subsequently, in step S311, a square value of the u _{x and} u _y components of the spatial frequency component of the corrected base image is calculated, and the calculated value is added to the register W2 [k, l]. That is, the square value of F _{k, l} (u _x, u _y ) is added to the register W2 [k, l] in order to obtain the value of the denominator of equation (18).

  Subsequently, in step S312, when it is determined whether or not all spatial frequency components of the modified base image have been processed, and when it is determined that all spatial frequency components of the transform coefficient have not been processed, the process proceeds to step S313. In order to process all the spatial frequency components of the corrected base image, 1 is added to the counter ux or the counter ui, and then the process returns to the processing of steps S308-1 and S308-2.

On the other hand, when it is determined in step S312 that all spatial frequency components of the corrected base image have been processed, the process proceeds to step S314, and the value of the register W1 [k, l] is divided by the value of the register W2 [k, l]. By calculating the sensitivity coefficient W _{k, l} ^s and outputting it, the process is terminated.

  In this way, the encoding parameter selection unit 10 selects the optimum prediction mode used for encoding the encoding target macroblock in consideration of the spatiotemporal visual sensitivity by executing the flowcharts of FIGS. It is processed like this.

  The processing described above is summarized as follows.

(1) Read the size of DCT (N × N)
(2) DCT base image is calculated (3) A 2N × 2N image is generated by zero padding for the base image of (2). The image obtained here is called a corrected base image. A specific generation method follows Formula (13): (4) DFT is performed on the modified base image of (3), and the distribution of spatial frequency components in the modified base image is calculated. A specific calculation method follows equation (14). (5) Read visual sensitivity function (CSF) (6) The spatial frequency distribution of (4) is weighted with the CSF of (5). The specific calculation method follows equation (15). The weighted spatial frequency components obtained in (7) and (6) are summed for all spatial frequencies. (8) Obtained by the processing in (7) The weighted sum for each spatial frequency component is used for weighting the amount of distortion for each spatial frequency component. (6) above is a novel process in the present invention. Specifically, it is as follows.

Input: spatial frequency F _k for each base _{image, l [u x, u y} ] (u x, u y = 0,1, ..., 2N-1), the motion vector _{d x, d y, CSFg (} η , Ω)
Output: Weight for block Procedure:
(1) Read a motion vector (d _x , d _y ) (2) u _x = 0
(3) u _y = 0
_{(4) (u x, d} x) is referred to as argument CSF value g (u _x, d _x) calculated _{(5) (u y, d} y) CSF value g to an argument (u _y, d _y) (6) to F [u _x , u _y ] = F [u _x , u _y ] g (u _x , d _x ) g (u _y , _dy )
(7) u _x ++
(8) If u _x = 2N, go to the next. Otherwise, go to (4) (9) u _y ++
(10) If u _y = 2N, go to the next. Otherwise, go to (3) (11) Calculate weighting factor W = | ~ E | / | E |
Where | E | = Σ _{ux Σuy} E [u _x ] [u _y ] ²
| _˜E | = Σ _{ux Σuy˜E} [u _x ] [u _y ] ²
5 to 7 show the apparatus configuration of the coding parameter selection unit 10 configured to execute the flowcharts of FIGS. 2 to 4.

  Next, the apparatus configuration of the encoding parameter selection unit 10 will be described with reference to FIGS.

  In order to execute the flowchart of FIG. 2, the encoding parameter selection unit 10, as illustrated in FIG. 5, includes a transition amount storage unit 101 that stores an estimated value of the transition amount of the encoding target block, and an encoding target block. A prediction vector calculation unit 102 that calculates a prediction vector; a prediction vector storage unit 103 that stores a prediction vector calculated by the prediction vector calculation unit 102; an initial mode setting unit 104 that sets an initial value of a prediction mode; A mode setting unit 105 that sets a prediction mode, a mode storage unit 106 that stores a prediction mode set by the initial mode setting unit 104 or the mode setting unit 105, and a prediction mode that is stored in the mode storage unit 106 The code amount calculation unit 107 that calculates the code amount of the code amount, and the code amount storage unit 108 that stores the code amount calculated by the code amount calculation unit 107 , A weighted distortion amount calculation unit 109 that calculates a weighted distortion amount when encoded in the prediction mode stored in the mode storage unit 106, and a weight that stores the weighted distortion amount calculated by the weighted distortion amount calculation unit 109 Attached distortion amount storage unit 110, an unknown multiplier calculation unit 111 that calculates an unknown multiplier when encoded in the prediction mode stored in the mode storage unit 106, and an unknown multiplier that stores the unknown multiplier calculated by the unknown multiplier calculation unit 111 Based on the storage unit 112, the code amount, the weighted distortion amount, and the undetermined multiplier, the cost calculation unit 113 that calculates the cost in the case of encoding in the prediction mode stored in the mode storage unit 106, and the cost calculation unit 113 The cost storage unit 114 that stores the calculated cost, the minimum cost storage unit 115 that stores the minimum cost obtained so far, and the minimum cost storage unit 115 A minimum cost determination unit 116 that determines whether the cost calculated by the cost calculation unit 113 is the minimum cost obtained so far, and an optimal mode storage unit that stores an optimal prediction mode 117, and an optimum mode update unit 118 that updates the prediction mode stored in the optimum mode storage unit 117 according to the prediction mode stored in the mode storage unit 106 when the minimum cost determination unit 116 determines that the cost is the minimum cost; When the minimum cost determination unit 116 determines that it is not the minimum cost, it immediately determines whether or not all of the prediction modes have been processed. When the minimum cost determination unit 116 determines that the minimum cost has been reached, the optimum mode update unit 118 In response to the instruction from, it is determined whether or not all of the prediction modes have been processed, and it is determined that it is not the final prediction mode. When determining, the final mode determination unit 119 that instructs the mode setting unit 105 to set the next prediction mode, and the optimal mode storage unit when determining that the final mode determination unit 119 is the final prediction mode. And an optimal mode output unit 120 that outputs the prediction mode stored in 117 as the optimal prediction mode.

  The encoding parameter selection unit 10 executes the flowchart of FIG. 2 according to this apparatus configuration.

  FIG. 6 illustrates an apparatus configuration of the weighted distortion amount calculation unit 109 that executes the flowchart of FIG. 3.

The weighted distortion amount calculation unit 109 stores the transform coefficient decoding unit 201 that decodes the transform coefficient and the transform coefficient decoded by the transform coefficient decoding unit 201, as illustrated in FIG. 6, in order to execute the flowchart of FIG. Encoding distortion of the kth and lth components (k and l are updated) of the transform coefficient based on the decoded transform coefficient storage unit 202 to be performed and the transform coefficient and the transform coefficient stored in the decoded transform coefficient storage unit 202 A distortion amount calculation unit 203 that calculates the distortion amount of the distortion, a distortion amount storage unit 204 that stores the distortion amount calculated by the distortion amount calculation unit 203, and k, l component values k, The transform coefficient index storage unit 205 that stores l, the transition amount storage unit 206 that stores the estimated value of the transition amount of the encoding target macroblock, and the correction obtained by the discrete Fourier transform performed on the corrected base image Based on the base image DFT storage unit 207 storing the spatial frequency component of the bottom image, the shift amount stored in the shift amount storage unit 206, and the spatial frequency component of the modified base image stored in the base image DFT storage unit 207 A sensitivity coefficient calculation unit 208 that calculates the sensitivity coefficient W _{k, l} of the k-th and l-th components of the coefficient, a sensitivity coefficient storage unit 209 that stores the sensitivity coefficient calculated by the sensitivity coefficient calculation unit 208, and a distortion amount storage unit 204 By multiplying the stored distortion amount by the sensitivity coefficient stored in the sensitivity coefficient storage unit 209, the sensitivity coefficient multiplication unit 210 that calculates the weighted distortion amount, and the distortion that stores the distortion amount calculated by the sensitivity coefficient multiplication unit 210. An amount storage unit 211; and a distortion amount sum calculation unit 212 that calculates a weighted distortion amount by calculating a sum of distortion amounts sequentially stored in the distortion amount storage unit 211.

  The weighted distortion amount calculation unit 109 executes the flowchart of FIG. 3 according to this device configuration.

  FIG. 7 illustrates an apparatus configuration of the sensitivity coefficient calculation unit 208 that executes the flowchart of FIG.

  In order to execute the flowchart of FIG. 4, the sensitivity coefficient calculation unit 208 stores a shift amount storage unit 301 that stores an estimated value of the shift amount of the encoding target block and a visual sensitivity function, as shown in FIG. Visual sensitivity function storage unit 302, conversion matrix storage unit 303 for storing the conversion matrix, conversion matrix size storage unit 304 for storing the size of the conversion matrix, and conversion matrix and conversion matrix size storage stored in the conversion matrix storage unit 303 Based on the size of the transformation matrix stored in the unit 304, the base image calculation unit 305 that calculates the kth and lth base images, and the base image storage unit 306 that stores the base images calculated by the base image calculation unit 305 A corrected base image calculation unit 307 that calculates a corrected base image based on a base image stored in the base image storage unit 306, and a correction base calculated by the correction base image calculation unit 307. A modified base image storage unit 308 that stores images, a DFT processing unit 309 that performs a discrete Fourier transform on the modified base image stored in the modified base image storage unit 308, and a space of the modified base image obtained by the DFT processing unit 309 A spatial frequency component storage unit 310 that stores frequency components, a spatial frequency index storage unit 311 that stores an index (ux, uy: updated) of the spatial frequency currently being processed, a visual sensitivity function, and a shift amount A horizontal visual sensitivity function calculation unit 312 that calculates a horizontal visual sensitivity function based on the spatial frequency index, and a horizontal direction that stores the horizontal visual sensitivity function calculated by the horizontal visual sensitivity function calculation unit 312. Based on the visual sensitivity function storage unit 313, the visual sensitivity function, the shift amount, and the spatial frequency index, the visual sense in the vertical direction Vertical direction visual sensitivity function calculation unit 314 that calculates a function, vertical direction visual sensitivity function storage unit 315 that stores the vertical direction visual sensitivity function calculated by the vertical direction visual sensitivity function calculation unit 314, and the spatial frequency of the corrected base image A multiplication unit 316 that multiplies the component, the horizontal sensitivity function, and the vertical sensitivity function, a square value cumulative addition unit 317 that cumulatively adds the squares of the multiplication values of the multiplication unit 316, and a space of the corrected base image A square value cumulative addition unit 318 that cumulatively adds square values of frequency components; a division unit 319 that divides the cumulative addition value of the square value cumulative addition unit 317 and the cumulative addition value of the square value cumulative addition unit 318; Is provided.

  The sensitivity coefficient calculation unit 208 executes the flowchart of FIG. 4 according to this device configuration.

  In the embodiment described above, the encoding parameter selection unit 10 has been described as selecting the optimum prediction mode as the encoding parameter. However, the present invention is also applicable to the case of selecting an encoding parameter other than the prediction mode. It can be applied as it is.

  For example, when the encoding parameter selection unit 10 selects an optimal quantization parameter as the encoding parameter, the encoding parameter selection unit 10 executes the flowchart of FIG. 8 instead of the flowchart of FIG. Become.

  That is, when selecting the optimal quantization parameter used for encoding the macroblock to be encoded, the encoding parameter selection unit 10 firstly, in step S401, the quantum parameter is selected as shown in the flowchart of FIG. The initial value of the quantization parameter (quantization parameter as the initial value) is set.

  Subsequently, in step S402, an initial cost indicating a large value is stored with respect to a register (minimum cost register) for storing a minimum cost, and a register for storing an optimal quantization parameter (hereinafter referred to as an optimal quantization parameter register). These registers are initialized by storing meaningless values.

Subsequently, in step S403, the shift amount (the aforementioned (d _x , _dy )) of the encoding target macro block is estimated.

  Subsequently, in step S404, the set quantization parameter, the prediction mode used for encoding, the prediction vector based on the prediction mode, the encoding target frame signal, and the reference frame signal are input, and the encoding is performed using the quantization parameter. The amount of code for conversion is calculated.

  Subsequently, in step S405, first, a weight considering the spatiotemporal visual sensitivity is determined based on the amount of transition estimated in step S403, and then the set quantization parameter, the prediction mode used for encoding, Weighted distortion amount when encoding using the prediction parameter, the encoding target frame signal, and the reference frame signal in the prediction mode, and encoding using the quantization parameter based on the input signal and the determined weight Is calculated. A specific calculation method is as described in the flowcharts of FIGS.

  Subsequently, in step S406, the set quantization parameter and the prediction mode used for encoding are input, and an undetermined multiplier when encoding using the quantization parameter is calculated.

  Subsequently, in step S407, based on the code amount calculated in step S404, the weighted distortion amount calculated in step S405, and the undetermined multiplier calculated in step S406, the RD cost represented by Expression (11) is used. Is calculated.

  Subsequently, in step S408, the calculated RD cost is compared with the cost stored in the minimum cost register, and the calculated RD cost is greater than the cost stored in the minimum cost register. If it is determined that the value is smaller, the process proceeds to step S409, where the calculated RD cost is stored in the minimum cost register, and in step S410, the set quantization parameter is stored in the optimum quantization parameter register. . On the other hand, when it is determined in step S408 that the calculated RD cost is higher than the cost stored in the minimum cost register, the processes in steps S409 and 410 are omitted.

  Subsequently, in step S411, it is determined whether or not all the quantization parameters have been processed, and when it is determined that all the quantization parameters have not been processed, the process proceeds to step S412 in accordance with a predetermined order. After the quantization parameter to be processed is updated by selecting one quantization parameter from the unprocessed quantization parameters, the process returns to step S404.

  On the other hand, when it is determined in step S411 that all the quantization parameters have been processed, the process proceeds to step S413, and the quantization parameter stored in the optimum quantization parameter register is output as the optimum quantization parameter. The process ends.

  FIG. 9 illustrates a device configuration of the encoding parameter selection unit 10 configured to execute the flowchart of FIG. Here, the same components as those shown in FIG. 5 are indicated by the same symbols.

  As shown in FIG. 9, the encoding parameter selection unit 10 includes an initial quantization parameter setting unit 404 instead of the above-described initial mode setting unit 104 in order to execute the flowchart of FIG. A quantization parameter setting unit 405 instead of the unit 105, a quantization parameter storage unit 406 instead of the mode storage unit 106, and an optimal quantization parameter storage unit 417 instead of the optimal mode storage unit 117. An optimal quantization parameter update unit 418 instead of the optimal mode update unit 118 described above, a final quantization parameter determination unit 419 instead of the final mode determination unit 119 described above, and the optimal mode output unit 120 described above. Instead, an optimum quantization parameter output unit 420 is provided.

  Next, a case where the sensitivity coefficient calculation method (part 2) is used will be described.

  In the case of the sensitivity coefficient calculation method (part 2), the encoding parameter selection unit 10 executes the flowcharts of FIGS. 10 and 11 instead of the flowchart of FIG.

The difference between the flowcharts of FIGS. 10 and 11 and the flowchart of FIG. 4 is that, in the flowcharts of FIGS. 10 and 11, in step S301, the amount of change of the macroblock to be encoded (the aforementioned (d _x , _dy )) is changed. read When, in the subsequent step S301arufa, displacement amount d _x, vision corresponding to d _y distance parameter r _x, from seeking r _y, is that so that the process proceeds to step S302.

Here, the method of obtaining the viewing distance parameters r _x and r _y is as described in the above-described equation (21).

Then, in the flowchart of FIG. 10 and FIG. 11, in step S308α-1 for executing the processing corresponding to step S308-1 in the flowchart of FIG. 4, the first u _x component and a viewing distance parameter of the spatial frequency component of the modified base image against r _x, to calculate the corresponding value of visual sensitivity function, at step S308α-2 to perform the processing corresponding to step S308-2 in the flowchart of FIG. 4, first u _y of the spatial frequency component of the modified base image The corresponding value of the visual sensitivity function is calculated for the component and the viewing distance parameter r _y .

That is, in step S308α-1, as described above,
_{η (u x, r x)} = θ (r x, H) u x / 4N
θ (r _x , H) = (1 / H) × arctan (1 / r _x ) × (180 / π)
Η (u _x , r _x ) is calculated and the corresponding value of the visual sensitivity function is calculated by substituting it into the visual sensitivity function g (η). In step S308α-2, as described above,
η (u _y , r _y ) = θ (r _y , H) u _y / 4N
_{θ (r y, H) =} (1 / H) × arctan (1 / r y) × (180 / π)
Accordingly, η (u _y , r _y ) is obtained and substituted for the visual sensitivity function g (η) to calculate the corresponding value of the visual sensitivity function.

  FIG. 12 illustrates a device configuration of the sensitivity coefficient calculation unit 208 that executes the flowcharts of FIGS. 10 and 11.

The difference between the sensitivity coefficient calculation unit 208 shown in FIG. 12 and the sensitivity coefficient calculation unit 208 shown in FIG. 7 that executes the flowchart of FIG. 4 is that the sensitivity coefficient calculation unit 208 shown in FIG. to displacement amount d _x, d viewing distance parameter corresponding to the _y r _x, so as to comprise a viewing distance parameter calculating section 500 for calculating a r _y, the horizontal visual sensitivity function calculating section 312, the visual sensitivity function and viewing distance A visual sensitivity function in the horizontal direction is calculated based on the visual distance parameter r _x calculated by the parameter calculation unit 500 and the spatial frequency index, and the vertical visual sensitivity function calculation unit 314 calculates the visual sensitivity function and the visual distance parameter. based on the index of the calculated viewing distance parameter r _y and the spatial frequency parts 500, a point that is to calculate the visual sensitivity function in the vertical direction.

  The present invention can be applied to the case of encoding a moving image by performing information compression by transform coding and quantization on a prediction error signal obtained by inter-frame prediction, and the present invention is applied. As a result, the cost function used for selecting the encoding parameter can be realized by reflecting the subjective image quality, so that highly efficient encoding can be realized.

It is an apparatus block diagram of the moving image encoding apparatus with which this invention is applied. It is a flowchart which an encoding parameter selection part performs. It is a flowchart which an encoding parameter selection part performs. It is a flowchart which an encoding parameter selection part performs. It is an apparatus block diagram of an encoding parameter selection part. It is an apparatus block diagram of an encoding parameter selection part. It is an apparatus block diagram of an encoding parameter selection part. It is a flowchart which an encoding parameter selection part performs. It is an apparatus block diagram of an encoding parameter selection part. It is a flowchart which an encoding parameter selection part performs. It is a flowchart which an encoding parameter selection part performs. It is an apparatus block diagram of an encoding parameter selection part. It is explanatory drawing of the problem which a prior art has.

Explanation of symbols

301 Transition amount storage unit 302 Visual sensitivity function storage unit 303 Transformation matrix storage unit 304 Transformation matrix size storage unit 305 Base image calculation unit 306 Base image storage unit 307 Correction base image calculation unit 308 Correction base image storage unit 309 DFT processing unit 310 Space Frequency component storage unit 311 Spatial frequency index storage unit 312 Horizontal visual sensitivity function calculation unit 313 Horizontal visual sensitivity function storage unit 314 Vertical visual sensitivity function calculation unit 315 Vertical visual sensitivity function storage unit 316 Multiplication unit 317 Accumulated square value Adder 318 Square value cumulative adder 319 Divider

Claims (10)

A video encoding method for encoding a video by performing information compression by transform coding and quantization on a prediction error signal obtained by inter-frame prediction,
Calculating the spatial frequency component of the base image of the transformation matrix;
For the encoding target block, a process of estimating a shift amount indicating temporal movement of the image signal,
Based on the horizontal component of the spatial frequency of the calculated spatial frequency component and the horizontal component of the estimated shift amount, a visual sensitivity function in the horizontal direction is calculated, and the vertical of the spatial frequency of the calculated spatial frequency component is calculated. Calculating a vertical visual sensitivity function based on the component and the vertical component of the estimated shift amount;
Based on the calculated horizontal and vertical visual sensitivity functions, the calculated spatial frequency component is weighted, and the importance of each calculated spatial frequency component is calculated based on the weighted spatial frequency component. Process,
A step of determining a parameter to be used for encoding a block to be encoded by evaluating a coding cost using a distortion amount of coding weighted using the calculated importance.
A moving image encoding method as a feature. A video encoding method for encoding a video by performing information compression by transform coding and quantization on a prediction error signal obtained by inter-frame prediction,
Calculating the spatial frequency component of the base image of the transformation matrix;
For the encoding target block, a process of estimating a shift amount indicating temporal movement of the image signal,
Based on the estimated horizontal component of the shift amount, a viewing distance parameter necessary for calculating a visual sensitivity function in the horizontal direction is determined, and the horizontal component of the spatial frequency of the calculated spatial frequency component and the determined A visual sensitivity function in the horizontal direction is calculated based on the visual distance parameter, and a visual distance parameter necessary for calculating the visual sensitivity function in the vertical direction is determined based on the vertical component of the estimated shift amount. Calculating a vertical visual sensitivity function based on the vertical component of the spatial frequency of the calculated spatial frequency component and the determined viewing distance parameter;
Based on the calculated horizontal and vertical visual sensitivity functions, the calculated spatial frequency component is weighted, and the importance of each calculated spatial frequency component is calculated based on the weighted spatial frequency component. Process,
A step of determining a parameter to be used for encoding a block to be encoded by evaluating a coding cost using a distortion amount of coding weighted using the calculated importance.
A moving image encoding method as a feature. In the moving image encoding method according to claim 1 or 2,
In the process of calculating the importance, by calculating a ratio value of the sum of squares for each spatial frequency of the weighted spatial frequency component and the sum of squares for each spatial frequency of the spatial frequency component not weighted, Calculating the importance,
A moving image encoding method as a feature. The moving image encoding method according to any one of claims 1 to 3,
In the process of calculating the spatial frequency component of the base image, calculating the spatial frequency component using a corrected base image in which a zero value is embedded around the base image,
A moving image encoding method as a feature. A video encoding device that encodes a video by performing information compression by transform coding and quantization on a prediction error signal obtained by inter-frame prediction,
Means for calculating a spatial frequency component of the base image of the transformation matrix;
Means for estimating a shift amount indicating temporal movement of the image signal for the encoding target block;
Based on the horizontal component of the spatial frequency of the calculated spatial frequency component and the horizontal component of the estimated shift amount, a visual sensitivity function in the horizontal direction is calculated, and the vertical of the spatial frequency of the calculated spatial frequency component is calculated. Means for calculating a visual sensitivity function in the vertical direction based on the component and the vertical component of the estimated shift amount;
Based on the calculated horizontal and vertical visual sensitivity functions, the calculated spatial frequency component is weighted, and the importance of each calculated spatial frequency component is calculated based on the weighted spatial frequency component. Means,
Means for evaluating a coding cost using a coding distortion weighted using the calculated importance to determine a parameter to be used for coding the block to be coded.
A moving image encoding device. A video encoding device that encodes a video by performing information compression by transform coding and quantization on a prediction error signal obtained by inter-frame prediction,
Means for calculating a spatial frequency component of the base image of the transformation matrix;
Means for estimating a shift amount indicating temporal movement of the image signal for the encoding target block;
Based on the estimated horizontal component of the shift amount, a viewing distance parameter necessary for calculating a visual sensitivity function in the horizontal direction is determined, and the horizontal component of the spatial frequency of the calculated spatial frequency component and the determined A visual sensitivity function in the horizontal direction is calculated based on the visual distance parameter, and a visual distance parameter necessary for calculating the visual sensitivity function in the vertical direction is determined based on the vertical component of the estimated shift amount. Means for calculating a visual sensitivity function in the vertical direction based on the vertical component of the spatial frequency of the calculated spatial frequency component and the determined viewing distance parameter;
Based on the calculated horizontal and vertical visual sensitivity functions, the calculated spatial frequency component is weighted, and the importance of each calculated spatial frequency component is calculated based on the weighted spatial frequency component. Means,
Means for evaluating a coding cost using a coding distortion weighted using the calculated importance to determine a parameter to be used for coding the block to be coded.
A moving image encoding device. In the moving image encoding device according to claim 5 or 6,
The means for calculating the importance degree calculates a ratio value of a sum of squares for each spatial frequency of the weighted spatial frequency component and a sum of squares for each spatial frequency of the spatial frequency component not weighted, Calculating the importance,
A moving image encoding device. In the moving image encoder according to any one of claims 5 to 7,
The means for calculating the spatial frequency component of the base image calculates the spatial frequency component using a corrected base image in which a zero value is embedded around the base image.
A moving image encoding device.   A moving picture coding program for causing a computer to execute the moving picture coding method according to claim 1.   A computer-readable recording medium on which a moving image encoding program for causing a computer to execute the moving image encoding method according to any one of claims 1 to 4 is recorded.

Images