JP2009207071A - Method, apparatus and program for estimating motion estimation accuracy and computer-readable recording medium with the program recorded thereon - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a new technology for achieving efficient encoding to a high frame rate video. <P>SOLUTION: Four model parameters are input which are described in a function to be used for estimating prediction error power in inter-frame prediction when performing encoding at a certain frame rate and on the basis of these model parameters, a secondary function is determined, with the inverse of the frame rate as a parameter, that is used for estimating motion estimation accuracy to minimize the amount of codes to be generated by encoding for fixing a distortion amount. A value of the frame rate to be processed is then substituted to the secondary function, thereby estimating motion estimation accuracy. An analytic theoretical model is constructed which represents a relationship between the frame rate and motion estimation accuracy, and suitable motion estimation accuracy for each frame rate is estimated using the model, thereby achieving efficient encoding to a high frame rate video. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a motion estimation accuracy estimation method and apparatus used in video encoding that realizes efficient encoding for high frame rate video, a motion estimation accuracy estimation program used to realize the motion estimation accuracy estimation method, and The present invention relates to a computer-readable recording medium on which the program is recorded.

  In recent years, there are growing expectations for super-high-quality images such as large-screen sports images and digital cinema that are full of realism. In response to this, research on high-quality video has been vigorously conducted.

  The following four elements are necessary to realize super high-quality video. That is, spatial resolution, pixel value depth, color reproducibility, and temporal resolution. In response, the former three elements are being studied in applications such as digital cinema and natural vision projects.

  However, sufficient studies have not been made on improving the temporal resolution, that is, increasing the frame rate of an image, which is indispensable for expressing the natural movement of the subject.

  According to Spillmann et al., Physiological knowledge is shown that the upper limit of the number of pulses output by ganglion cells, which are output cells of the retina, is about 300 to 400 pulses per second. For this reason, it is presumed that the human visual system is the shortest and can perceive the difference in light emission up to 1/150 to 1/200 second. This means that the perceivable frame rate detection limit is 150 to 200 [frames / second]. The current video frame rate of 30, 60 [frame / second] is determined from the flicker detection limit, and is not a sufficient value to express natural motion.

  On the other hand, since the super high image quality of the video causes an explosive increase in the amount of data, an efficient encoding method is required when the code amount is limited due to network bandwidth limitations or the like.

  In the case of a high frame rate video, since inter-frame correlation increases, inter-frame prediction by motion compensation is effective. One parameter of motion compensation is motion estimation accuracy. For example, the motion estimation accuracy in an MPEG-2 compliant encoder is ½ pixel accuracy.

  In the case of a high frame rate video, since the frame interval is shortened, high accuracy is required for the shift amount. Therefore, it is important to grasp the relationship between the frame rate and the motion estimation accuracy. Intuitively, it is expected that higher motion estimation accuracy is required as the frame rate increases.

  However, in the conventional research, the influence of the frame rate is not considered, and the quantitative property regarding the relationship between the frame rate and the motion estimation accuracy is not grasped (for example, refer to Non-Patent Document 1). For this reason, the motion estimation accuracy is set by an ad hoc method. When the motion estimation accuracy is insufficient, the coding efficiency is lowered.

On the other hand, there is also a method in which the motion estimation accuracy in a settable range is encoded with brute force and the motion estimation accuracy with the highest encoding efficiency is selected. However, in this case, the encoding processing time increases.
J.Ribas-Corbera and DLNeuhoff.Optimizing motion-vector accuracy in block-based video coding.IEEE CSVT, Vol.11, No.4, pp.497-511, 2001.

  As described above, in order to express the natural movement of the subject, it is necessary to realize a high frame rate of the video.

  In order to realize a high frame rate of video, since the frame interval is shortened, high accuracy is required for the shift amount, and therefore it is important to grasp the relationship between the frame rate and the motion estimation accuracy.

  However, in the conventional research, the quantitative property about the relationship between the frame rate and the motion estimation accuracy is not grasped, and the actual situation is that the motion estimation accuracy is set by an ad hoc method.

  Thus, according to the prior art, when encoding a high frame rate video, if the motion estimation accuracy is insufficient, there is a problem that the encoding efficiency decreases.

  The present invention has been made in view of such circumstances. In an encoding process for a high frame rate video, an analytical theoretical model representing the relationship between the frame rate and motion estimation accuracy is constructed, and the theoretical model is It is an object of the present invention to establish a new motion estimation accuracy estimation technique that realizes efficient encoding for a high frame rate video by estimating an appropriate motion estimation accuracy for each frame rate.

  In order to achieve this object, the motion estimation accuracy estimation apparatus of the present invention is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. In order to realize the estimation of motion estimation accuracy that minimizes the amount of code that will be generated in the encoding under the condition that the encoding is constant, (a) the frame rate input to input the frame rate And (b) model parameter input means for inputting four model parameters described in a function used for estimation of prediction error power of inter-frame prediction when encoding is performed at a certain frame rate, and (c) Based on the four model parameters input by the model parameter input means, a code that is generated by the encoding under the condition of the encoding with a constant distortion amount Determining means for determining a quadratic function using a reciprocal of the frame rate as a variable, and (d) a frame rate input means for the quadratic function determined by the determining means. Substituting the value of the input frame rate to estimate the motion estimation accuracy for minimizing the amount of code that will occur when encoding is performed on the frame input by the frame rate input means; And (e) output means for outputting the motion estimation accuracy estimated by the estimation means.

  The motion estimation accuracy estimation method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is provided by being recorded on an appropriate computer-readable recording medium. Alternatively, the present invention is realized by being provided via a network, installed when the present invention is carried out, and operating on a control means such as a CPU.

In the motion estimation accuracy estimation apparatus of the present invention configured as described above, when the frame rate to be processed is input, subsequently, for example, (1) derived from the sum of squares of the difference values between adjacent pixels in the reference frame. Model parameter α ₁ , (2) model parameter α ₂ derived from the sum of the product of the approximation error of Taylor expansion for inter-frame prediction error and the difference value between adjacent pixels in the reference frame, and (3) superposition on the original signal Model parameter α ₃ derived from the approximate value of the sum of squares of the sum of the noise and the interpolation error in the reference frame, and (4) a model derived from the approximate sum of squares of the Taylor expansion approximation error for the inter-frame prediction error to enter the four model parameters of that parameter α _4.

  Here, after the four model parameters are input, the frame rate to be processed may be input.

Subsequently, if the frame rate is represented by F, the number of segments as a unit of motion compensation is represented by M, and the coefficient is represented by κ, the input four model parameters α _1, α _2, α _3, α ₄ For example, based on
(Α ₁ F ⁻² + α ₂ F ⁻¹ + α ₃ + α ₄ ) / κα ₁ (M−1)
By determining the function, the reciprocal of the frame rate used to estimate the motion estimation accuracy that minimizes the amount of code that will be generated by the coding under the condition of coding with a constant amount of distortion. Determine a quadratic function as a variable.

  This quadratic function is a model that describes the relationship between the frame rate and the motion estimation accuracy that minimizes the amount of code that occurs when encoding is performed at that frame rate.

  Next, motion estimation that minimizes the amount of code that will occur when encoding is performed on the input frame by substituting the value of the input frame rate for the determined quadratic function. The accuracy is estimated, and the estimated motion estimation accuracy is output.

  According to the present invention, it is possible to estimate the motion estimation accuracy that minimizes the code amount when the distortion amount is constant (quantization step width is fixed) when the frame rate is given.

  That is, according to the present invention, it is possible to set an appropriate motion estimation accuracy according to the frame rate, and it is possible to avoid a decrease in coding efficiency due to a lack of motion estimation accuracy. In general, in encoding of a high frame rate video that requires high motion estimation accuracy, it is possible to estimate appropriate motion estimation accuracy in advance, and it is expected that the encoding process can be speeded up.

  Hereinafter, the present invention will be described in detail according to embodiments.

  As described above, in the present invention, in an encoding process for a high frame rate video, an analytical theoretical model representing the relationship between the frame rate and the motion estimation accuracy is constructed, and the frame model is used for each frame rate using the theoretical model. By estimating the appropriate motion estimation accuracy, efficient coding for a high frame rate video is realized.

  Next, first, “[1] Relationship between frame rate and inter-frame prediction error power” explains the relationship between frame rate and inter-frame prediction error power, followed by “[2] Parameter estimation”. The parameter estimation described in the relationship between the frame rate and the inter-frame prediction error power will be described, and then, “[3] Relationship between the displacement estimation error and the frame rate” will be described. The relationship between the shift amount estimation error and the frame rate necessary for deriving the relationship with the error power will be explained. Finally, in [4] Optimization of motion estimation accuracy, the relationship between the frame rate and motion estimation accuracy. A configuration is described in which an analytical theoretical model representing the above is constructed and appropriate motion estimation accuracy for each frame rate is estimated using the theoretical model.

[1] Relationship between frame rate and inter-frame prediction error power The relationship between frame rate and inter-frame prediction error power is analytically derived. Here, for simplicity, a one-dimensional signal will be described as an example.

An observation signal at position x at time t having X samples is represented by g _t (x) (0 ≦ x ≦ X−1), and segment B [i] (i = 1, 2,. / L) as a unit, motion compensation (estimated displacement ^ d [i]) is performed, and the prediction error after motion compensation in the segment is evaluated.

Here, it is assumed that the observation signal g _t (x) is configured by superimposing noise n _t (x) on the image signal f _t (x). That is,
g _t (x) = _ft (x) + _nt (x)
And

  In the following notation, the symbol ^ in “^ X” (where X is a letter) indicates a symbol attached to “X”. In the following, the true displacement amount at the position x is d (x).

When the estimated shift amount ^ d [i] has decimal pixel accuracy, a reference pixel value is generated by interpolation processing. When a signal to be interpolated and s _t (x), the interpolation processing is filtering represented by the following formula (1). In the following, the operator on the left side of the following formula (1) may be expressed as “I”.

Here, δ is the accuracy of motion estimation. For example, in the case of an MPEG-2 compliant encoder, δ = 1/2 is set. The interpolation coefficient ψ _j [k] satisfies the following expression (2).

Using the above-described interpolation filter, the pixel value at the reference position is approximated as I (˜gt ₁ (x + ^ d [i]) ^(q) ).

  In the following notation, the symbol “˜X” (where X is a letter) indicates a symbol attached to “X”.

Here, ~ gt _-1 (x + ^ d [i]) ^(q) is a case where quantization with a quantization step width q is performed on gt _-1 (x + ^ d [i]). And is expressed as the following equation (3).

In this equation (3), v _t-1 (x + ^ d [i]) q represents an error caused by quantization, and v _t-1 (x + ^ d [i]) represents the signal f _{t- 1 This} parameter depends on (x + ^ d [i]).

The difference between the pixel value f _t-1 (x + ^ d [i]) at the reference position and the value Ι (˜gt ₁ (x + ^ d [i]) ^(q) ) generated by interpolation is the above-described equation. Using (3), it can be expressed as the following equation (4).

Here, e _t-1 ^int (x + ^ d [i]) is
e _t-1 ^int (x + ^ d [i])
= _Ft-1 (x + ^ d [i])-I ( _ft-1 (x + ^ d [i]))
It is as follows.

In this case, the prediction error after motion compensation in a _{segment, "g t (x) =} f t (x) + n t (x)" and the above equations (3) and by using the equation below (5 ).

  Here, φ (x) is the second and subsequent terms of Taylor expansion.

N (x) is the sum of the interpolation error N _t-1 ^ref (x) in the reference frame and the noise n _t (x) superimposed on the original signal,
N (x) = _nt (x) -I ( _nt-1 (x + ^ d [i]))
+ _Et-1 ^int (x + ^ d [i])-I (vt _-1 (x + ^ d [i])) q
It can be expressed as

Further, ζ [i] (x) is an estimation error between the estimated shift amount ^ d [i] and the true shift amount d [i] (x) at x.
ζ [i] (x) = d [i] (x) − ^ d [i]
It can be expressed as

  As will be described later in “[3] Relationship between shift amount estimation error and frame rate”, the shift amount estimation error can be approximated as a function of frame rate F as shown in the following equations (6) and (7).

Here, κ ₁ , κ ₁ ′ and κ ₂ are constants depending on the video signal. Also, δ is the accuracy of motion estimation. For example, in the case of an MPEG-2 compliant encoder, δ = 1/2 is set.

The sum of σ [i] ² is calculated for all segments (B [i] (i = 1, 2,..., X / L)). At this time, by assuming the independence of each term of N (x), an approximate expression as shown in the following expression (8) is obtained.

Here, ~ N (x) ² is as shown in the following formula (9).

  Further, assuming that the displacement estimation error ζ [i] (x) and the differential value of the pixel value are statistically independent, substituting Equation (6) and Equation (7) into Equation (8) gives the prediction For the error power, an approximate expression shown in the following expression (10) is obtained.

Here, the model parameters α _1, α _2, α _3, α _{4, and} α ₅ are respectively expressed by the following formula (11).

As can be seen from this equation, the model parameter α ₁ is derived from the sum of squares of the difference values between adjacent pixels in the reference frame. Further, the model parameter α ₂ is derived from the sum of products of the approximation error of Taylor expansion with respect to the inter-frame prediction error and the difference value between adjacent pixels in the reference frame. The model parameter α ₃ is a square value of N (x) defined as the sum of the noise n _t (x) superimposed on the original signal and the interpolation error N _t-1 ^ref (x) in the reference frame. It will be derived as an approximate value. The model parameter α ₄ is derived from the sum of squares of approximation errors of Taylor expansion with respect to interframe prediction errors.

[2] Parameter Estimation Next, estimation of the above-described equation (9) that defines the model parameter α ₃ will be described. Each term of the above equation (9) is estimated as follows.

N _t (x) ² in the first term of the above equation (9) is the power of the noise component due to the thermal noise of the camera.

For this estimation, first, a still region in a frame is identified. The method for identifying the static region is given from the outside. For example, a frame is divided into blocks, motion estimation is performed for each block to obtain a motion vector, and a region where the motion vector becomes a zero vector is set as a still region. An inter-frame difference is obtained for a block identified as a still region. The square sum of the inter-frame difference signal is obtained, and the pixel average value n _t (x) ² of the square sum is used as the estimated value.

I (n _t-1 (x + ^ d [i])) ² of the second term of the above-described equation (9) is the power of the noise component after passing through the interpolation filter.

Assuming that the noise component n _t (x) ² is uncorrelated, the noise power after passing through the interpolation filter at jδ (j = 1,..., Δ ⁻¹ −1) pixel position is expressed by the following equation (12): ).

Further, assuming that the interpolation filter processing at the jδ (j = 1,..., Δ ⁻¹ −1) pixel position is performed at substantially the same frequency, I ( _nt−1 (x + ^ d [ i])) ² can be approximated by the following equation (13).

The third term e _t-1 ^int (x + ^ d [i]) ² in the above equation (9) is an estimation error accompanying interpolation of the pixel value at the decimal pixel position.

A downsampling process is performed on f _t (x) to obtain an image signal h _t (x) with a resolution reduced to δ. It is assumed that the filter used for the downsampling process is given from the outside. For example, an inverse filter of the above-described interpolation filter or a band limiting filter based on the sinc function can be considered. For this h _t (x), a signal f _t (x) ′ whose resolution is enlarged δ ⁻¹ times is obtained by the interpolation filter described above. The estimated value of the interpolation error associated with the interpolation filter is approximated by the following equation (14).

I (v _t-1 (x + ^ d [i])) ² q ² in the fourth term of the above equation (9) is an error accompanying quantization of the reference signal.

The difference value (pixel average) between the prediction error power when the original signal is used as the reference signal and the prediction error power when the decoded signal obtained by quantization / inverse quantization is used as the reference signal is represented by v _t−1. Let (x + ^ d [i]) ² q ^{2 be} an estimated value “−ε _q ”. Here, - symbols in "ε _q" "-" indicates the symbols attached to the top of the "epsilon _q". Further, using this estimated value “−ε _q ”, I (v _t−1 (x + ^ d [i])) ² q ² is approximated as in the following equation (15).

[3] Relationship between Transition Amount Estimation Error and Frame Rate Next, the fact that the above-described equations (6) and (7) are established will be described.

[3-1] Derivation of Equation (6) A discrete value when the amount of change ^ d ^F [i] expressed as a continuous value is discrete is denoted by ˜d ^F [i]. Assuming that the discretization error at this time is ε [i], ˜d ^F [i] is expressed by the following equation (16).

It is assumed that ε [i] follows a uniform distribution in which the mean is zero and the variance is proportional to the square value of motion estimation accuracy. In block matching based on the square error criterion, the estimated shift amount for each block can be approximately expressed using the shift amount of the pixel at the block center position. Assuming that x _c [i] is the coordinates of the center position of the segment B [i], the estimated shift amount when the frame rate F = F ₀ can be approximated as in the following equation (17).

  Further, the difference between the shift amount at ^ x and the shift amount at ^ x 'can be modeled as a normal distribution in which the average is zero and the variance is proportional to the difference value between the two coordinate positions. Here, {circumflex over (x)} and {circumflex over (x)} ′ denote coordinate positions in the segment B [i] (i = 1, 2,..., X / L). That is, approximation as shown in the following equation (18) is possible.

Here, E [·] denotes the expectation operation, ^ c _h is a parameter of the signal-dependent. The model is the difference value (d ^F0 [i] (^ x) −d ^F0 [i] (^ x) of the transition amount in all segments B [i] (i = 1, 2,..., X / L). Give a good approximation of the set average for ')).

  The following equation (19) is obtained from the equations (17) and (18).

Here, c _h ² = (X / L) ^ c _h ² . ξ is a coordinate value in a coordinate system with the center position of the segment B [i] as the origin, and ξ can be expressed as xx ₀ [i]. That is, ξ is a distance from the center position of the segment B [i].

  Deriving the relationship between two shifts at different frame rates. It is assumed that the movement of the object between adjacent frames follows a uniform motion. In this case, the amount of displacement of the object is proportional to the frame interval. That is, it can be said that the shift amount is inversely proportional to the frame rate. Thereby, the relationship of the following formula | equation (20) is obtained.

  Using equation (16), equation (20) and the assumption that ε [i] does not depend on the frame rate, the following equation (21) is obtained.

  By using the assumption that the equation (21) and ε [i] are statistically independent of the shift amount, the following equation (22) is obtained.

Here, ξ = x−x _c [i], and δ represents the motion estimation accuracy. For example, in the case of an MPEG-2 encoder, δ is set to 1/2, and in the case of an MPEG-4 ASP (Advanced Simple Profile) encoder, δ is set to ¼.

In the formula (22), it represents a _{^{F 0 2 c h 2 Σξ 2}} κ 1 and, by the XY / 12 expressed as kappa ₁ ', to obtain a formula (6).

[3-2] Derivation of Equation (7) Assuming that ε [i] follows a zero-average uniform distribution, the following Equation (23) is obtained.

ζ ^F [i] (x)
ζ ^F [i] (x) = d ^F [i] (x) − ^ d ^F [i]
It is assumed that the deviation estimation error is defined as follows.

At this time, {ΣΣζ ^F [i] (x)} ² can be expressed as the following equation (24).

  The first term of equation (24) can be approximated by equation (22). The second term satisfies the following equation (25) from the Schwarz inequality.

Here, ξ = x−x _c [i] and ξ ′ = x′−x _c [i]. The approximate expression of the following expression (26) is obtained from the inequality of the expression (25).

  Here, β is a parameter that takes a value in the range of −1 to 1. By substituting Equation (26) and Equation (22) into Equation (24), the following Equation (27) is obtained.

  Therefore, the following equation (28) is obtained.

Here, γ is 1 or -1. Formula (7) is obtained by setting the following formula (29) described in formula (28) as κ ₂ .

[4] Optimization of motion estimation accuracy Consider the code amount when the distortion amount is constant (quantization step width is fixed) as an evaluation function. Hereinafter, a method of calculating motion estimation accuracy that minimizes the evaluation function when a quantization step width is given will be described in detail.

When the quantization step width q, the frame rate F, and the motion estimation accuracy δ are given, the code amount of the inter-frame prediction error is
R _c (q, F, δ) = (1/2) log ₂ γ _c (P (F, δ) / q ² )
... Formula (30)
Can be modeled as follows.

Here, γ _c is a signal-dependent constant parameter. P (F, δ) is
P (F, δ) = Σσ [i] ² where Σ is the sum for i = 1 to X / L and represents the prediction error power sum in the frame.

On the other hand, when the frame rate F and the motion estimation accuracy δ are given, the code amount of the shift amount is
R _mv (F, δ) = (X / L) log ₂ (2U (F) / δ) (31)
Can be modeled as follows.

  Here, U (F) represents a motion estimation search range when the frame rate is F, and the shift amount ^ d can take a range of -U (F) ≤ ^ d≤U (F) -1. Shall. Therefore, 2U (F) / δ represents the number of candidates for the estimated shift amount within the search range.

Since the shift amount decreases in proportion to the frame interval, U (F) is inversely proportional to F from the viewpoint of code amount reduction.
U (F) = U ₀ F ₀ / F
And so on. Here, U ₀ is a parameter representing the range of motion estimation when the frame rate is F ₀ .

At this time, the sum of the code amount of the prediction error signal and the shift amount is
R (q, F, δ) = R _c (q, F, δ) + R _mv (F, δ) (32)
It becomes.

  This equation (32) gives the code amount when the distortion amount is constant (quantization step width q is fixed).

In this way, when the frame rate F and the quantization step width q are given, the motion estimation accuracy δ ^* (F) that minimizes R (q, F, δ) is calculated. Δ ^* (F) to be obtained is
∂R (q, F, δ) / ∂δ
= ∂R _c (q, F, δ) / ∂δ + ∂R _mv (F, δ) / ∂δ = 0
... Formula (33)
The following expression is satisfied.

Δ ^* (F) satisfying the expression (33) can be expressed as the following expression (34).

  Here, the value of the equation (34) satisfies the relationship of the following equations (35) and (36).

Equation (35) indicates that the motion estimation accuracy needs to be reduced as the frame rate increases. Furthermore, equation (36) indicates that the decrease in motion estimation accuracy converges when the frame rate increases to the limit. Here, it is considered that the model parameter α ₄ includes a fourth-order or higher term of F ⁻¹ .

~ N (x) in the model parameter α ₃ is caused by an error accompanying generation of a pixel value at an interpolation position required for reference, thermal noise of the imaging system, etc., when the estimated shift amount ^ d is decimal pixel accuracy. It is an error term consisting of components. When the frame rate increases to the limit and the frame interval decreases to the limit, the component due to the error term becomes dominant in the inter-frame prediction error. In such a situation, even if the motion estimation accuracy is increased, a reduction in prediction error corresponding to the accuracy cannot be obtained. This is shown in equation (36), which matches the physical nature of the video signal.

In this way, according to the present invention, when the frame rate F is given, the movement of minimizing the code amount when the distortion amount is constant (the quantization step width is fixed) according to the equation (34). The estimated accuracy δ ^* (F) can be calculated.

  Next, examples of the present invention will be described.

  FIG. 1 shows a device configuration of a video encoding device 1 having the present invention.

  As shown in this figure, the video encoding device 1 of the present invention uses a video file 10 that stores video to be encoded and inter-frame prediction that performs motion estimation with non-integer pixel accuracy, and uses the video file 10. An encoding unit 11 that encodes the video stored in the video, a motion estimation accuracy calculation unit 12 that calculates the motion estimation accuracy necessary for the encoding unit 11 to perform inter-frame prediction, and provides the motion estimation accuracy to the encoding unit 11; And an encoded data file 13 for storing the encoded data encoded by the encoding unit 11.

  2 and 3 are flowcharts executed by the motion estimation accuracy calculation unit 12. Next, processing executed by the present invention will be described in detail according to this flowchart.

  As shown in the flowchart of FIG. 2, the motion estimation accuracy calculation unit 12 first reads the encoding target frame in step S101, and then reads the reference frame in step S102.

Subsequently, in step S103, a difference value between adjacent pixels in the reference frame is calculated. That is, “d / dx (f _t−1 (x))” described in the definition formula of α ₁ in Expression (11) is calculated.

Subsequently, in step S104, the sum of squares of the difference values between adjacent pixels calculated in step S103 is calculated. That is, “ΣΣ (d / dx (ft ₋₁ (x))) ² ” described in the definition formula of α ₁ in Expression (11) is calculated.

Subsequently, in step S105, an approximation error of Taylor expansion with respect to the inter-frame prediction error is calculated. That is, “φ (x)” described in the definition formula of α _{2 and} α _{4 in} the equation (11) is calculated.

Subsequently, in step S106, the sum of the products of the approximation error of Taylor expansion calculated in step S105 and the difference value between adjacent pixels calculated in step S103 is calculated. That is, “ΣΣ (φ (x) × d / dx (ft _-1 (x)))” described in the definition formula of α ₂ in Expression (11) is calculated.

Subsequently, at step S107, it calculates a model parameter alpha ₄ by calculating the square sum of the approximation error of the Taylor expansion calculated in step S105. That is, the model parameter α ₄ is calculated by calculating “ΣΣ (φ (x) ² )” described in the definition formula of α _{4 in} the equation (11).

Subsequently, at step S108, it calculates a model parameter alpha ₃ by calculating the "ΣΣ (~N (x) ^2)" described in the alpha ₃ defining equation of the formula (11). Details of the calculation process of the model parameter α ₃ executed in step S108 will be described later with reference to the flowchart of FIG.

On the other hand, in step S109, the frame rate F is read. In the subsequent step S110, the reciprocal number F ⁻¹ of the read frame rate is calculated. In the subsequent step S111, the square value F ⁻² of the reciprocal number of the calculated frame rate is calculated. .

  Subsequently, in step S112, the number of pixels X in the frame is read, and in step S113, the number of pixels L in the segment is read. That is, the X value of the X sample and the L value of the L sample described in “[1] Relationship between frame rate and inter-frame prediction error power” are read.

  Subsequently, in step S114, the number of segments (X / L) is calculated by dividing the number of pixels X read in step S112 by the number of pixels L read in step S113.

When the processing of step S101 to step S114 is completed in this way, subsequently, in step S115, a value obtained by multiplying the calculated value of step S104 by F ⁻² and the constant parameter κ ₁ is calculated, thereby obtaining the equation (34 Α ₁ F ^-2 described in ( ₁ ) is calculated.

That is, by multiplying the calculated value in step S104 by the constant parameter κ ₁ , the model parameter α ₁ is calculated in accordance with the definition formula of α _{1 in} equation (11), and multiplied by F ⁻² to obtain equation (34). Α ₁ F ^-2 described in ( ₁₎ is calculated.

Subsequently, in step S116, by calculating a value obtained by multiplying the calculated value of step S106 by F ⁻¹ and the double value (2 × κ ₂ ) of the constant parameter κ ₂ , α described in the equation (34) is obtained. ₂ Calculate F- ¹ .

That is, the model parameter α ₂ is calculated according to the definition formula of α ₂ in the equation (11) by multiplying the calculated value in step S106 by the double value (2 × κ ₂ ) of the constant parameter κ ₁ , and F ⁻ By multiplying by ¹ , α ₂ F ⁻¹ described in the equation (34) is calculated.

  Subsequently, in step S117, the numerator value of the equation (34) is calculated by calculating the sum of the calculated values in steps S107, S108, S115, and S116.

That is, α ₄ that is the calculated value in step S107, α ₃ that is the calculated value in step S108, α ₁ F ⁻² that is the calculated value in step S115, and α ₂ F ⁻¹ that is the calculated value in step S116. By calculating the sum of the numerator value α ₁ F ⁻² + α ₂ F ⁻¹ + α ₃ + α _{4 in} equation (34).
Is calculated.

Subsequently, in step S118, a value obtained by subtracting 1 from the calculated value in step S114 is multiplied by the calculated value in step S104 and the constant parameters κ _{1 and} κ ₁ ′, thereby calculating the denominator of the equation (34). Calculate the value.

That is, a value obtained by subtracting 1 from the calculated value in step S114 (X / L−1) is added to “ΣΣ (d / dx (ft _-1 (x)) ² ” that is the calculated value in step S104. By multiplying the model parameter α ₁ defined as the product of the parameter κ ₁ and further multiplying it by the constant parameter κ ₁ ′, the value of the denominator (X / L−1) α ₁ κ ₁ in the equation (34) '
Is calculated.

Subsequently, in step S119, by calculating a value obtained by dividing the calculated value in step S117 by the calculated value in step S118, the code amount when the distortion amount is constant (quantization step width is fixed) is minimized. The motion estimation accuracy δ ^* (F) is calculated.

That is, in accordance with the definition of Expression (34), α ₁ F ⁻² + α ₂ F ⁻¹ + α ₃ + α ₄ that is the calculated value in step S117 is (X / L−1) α ₁ κ that is the calculated value in step S118. _By dividing by ₁ ′, the motion estimation accuracy δ ^* (F) that minimizes the code amount when the distortion amount is constant (quantization step width is fixed) is calculated.

In this way, the motion estimation accuracy calculation unit 12 minimizes the code amount when the distortion amount is constant (the quantization step width is fixed) according to the equation (34) when the frame rate is given. The motion estimation accuracy δ ^* (F) is calculated.

Next, details of the calculation process of the model parameter α ₃ executed in step S108 will be described according to the flowchart of FIG.

Here, the model parameter α ₃ is defined as a sum of ˜N (x) ² defined by the equation (9) as described in the equation (11).

  When entering the process of step S108 in the flowchart of FIG. 2, the motion estimation accuracy calculation unit 12 first reads the encoding target frame in step S201 as shown in the flowchart of FIG. Read a reference frame.

Subsequently, in step S203, a transfer function of the interpolation filter (interpolation coefficient ψ _j [k] described in the above equation (2)) is calculated.

  Subsequently, in step S204, the noise power superimposed on the original signal (the first term on the right side of Equation (9)) is estimated.

  Subsequently, in step S205, noise power (second term on the right side of Equation (9)) to be superimposed on the original signal after passing through the interpolation filter is estimated.

  Subsequently, in step S206, the estimated error power of the pixel value at the interpolation position (the third term on the right side of Equation (9)) is estimated.

  On the other hand, in step S207, the quantization parameter is read, and in the subsequent step S208, the quantization step width corresponding to the read quantization parameter is calculated.

  Subsequently, in step S209, the quantization error power of the reference image after passing through the interpolation filter (the fourth term on the right side of Equation (9)) is estimated.

When the processing of step S201 to step S209 is completed in this way, finally, in step S210, the sum of the powers obtained in steps S204, S205, S206, and S209 is calculated, so that equations (11) and ( The model parameter α ₃ defined in 9) is calculated, and the process of step S108 in the flowchart of FIG.

  FIG. 4 illustrates a device configuration of the motion estimation accuracy calculation unit 12 that executes the processing described above.

  As shown in this figure, the motion estimation accuracy calculation unit 12 includes an encoding target frame storage unit 101, a reference frame storage unit 102, an inter-pixel difference value calculation unit 103, an inter-pixel difference value storage unit 104, and a square. Sum calculation unit 105, Taylor expansion approximation error calculation unit 106, Taylor expansion approximation error storage unit 107, integrated value addition processing unit 108, modified Taylor expansion approximation error storage unit 109, sum of squares calculation unit 110, Taylor Expanded approximation error sum of squares storage unit 111, residual component calculation unit 112, residual component storage unit 113, frame rate storage unit 114, reciprocal number calculation unit 115, frame rate reciprocal number storage unit 116, and square value calculation Unit 117, frame rate reciprocal square value storage unit 118, segment number calculation unit 119, segment number storage unit 120, subtraction processing unit 121, segment Number storage unit 122, parameter storage unit 123, parameter storage unit 124, integration processing unit 125, inter-pixel difference value square sum storage unit 126, integration processing unit 127, integration processing unit 128, and addition processing unit 129, an addition value storage unit 130, an integration processing unit 131, an integration value storage unit 132, and a division processing unit 133.

  In the motion estimation accuracy calculation unit 12 configured as described above, the encoding target frame storage unit 101 stores the encoding target frame. The reference frame storage unit 102 stores a reference frame.

  The inter-pixel difference value calculation unit 103 executes the process of step S103 in the flowchart of FIG. 2, and calculates a difference value between adjacent pixels in the reference frame. The inter-pixel difference value storage unit 104 stores the adjacent inter-pixel difference value calculated by the inter-pixel difference value calculation unit 103. The square sum calculation unit 105 executes the process of step S104 in the flowchart of FIG. 2 and calculates the square sum of the difference values between adjacent pixels calculated by the inter-pixel difference value calculation unit 103.

  The Taylor expansion approximation error calculation unit 106 executes the processing of step S105 in the flowchart of FIG. 2 and calculates an approximation error of Taylor expansion with respect to the inter-frame prediction error. The Taylor expansion approximation error storage unit 107 stores the Taylor expansion approximation error calculated by the Taylor expansion approximation error calculation unit 106.

  The integrated value addition processing unit 108 executes the process of step S106 in the flowchart of FIG. 2, and calculates the Taylor expansion approximation error calculated by the Taylor expansion approximation error calculation unit 106 and the inter-pixel difference value calculation unit 103. The sum of the product with the difference value between adjacent pixels is calculated. The corrected Taylor expansion approximation error storage unit 109 stores the calculated value of the integrated value addition processing unit 108 (referred to as a corrected Taylor expansion approximation error).

The square sum calculation unit 110 executes the process of step S107 in the flowchart of FIG. 2 and calculates the model parameter α ₄ by calculating the square sum of the Taylor expansion approximation error calculated by the Taylor expansion approximation error calculation unit 106. Is calculated. The Taylor expansion approximate error square sum storage unit 111 stores the model parameter α ₄ calculated by the square sum calculation unit 110.

Residual component calculating section 112 is for executing the process of step S108 in the flowchart of FIG. 2, by executing the processing in the flowchart of FIG. 3, model parameters are calculated alpha _3. The residual component storage unit 113 stores the model parameter α ₃ calculated by the residual component calculation unit 112.

The frame rate storage unit 114 stores the frame rate F. The reciprocal calculation unit 115 executes the process of step S110 in the flowchart of FIG. 2 and calculates a reciprocal F ^{−1 of the} frame rate. The frame rate reciprocal storage unit 116 stores the reciprocal F ⁻¹ of the frame rate calculated by the reciprocal calculation unit 115. The square value calculation unit 117 performs the process of step S111 in the flowchart of FIG. 2, and calculates a square value F ⁻² of the reciprocal of the frame rate. The frame rate reciprocal square value storage unit 118 stores the square value F ⁻² of the reciprocal of the frame rate calculated by the square value calculation unit 117.

  The segment number calculation unit 119 executes the process of step S114 in the flowchart of FIG. 2 and calculates the number of segments X / L. The segment number storage unit 120 stores the segment number X / L calculated by the segment number calculation unit 119. The subtraction processing unit 121 calculates a value (X / L−1) obtained by subtracting 1 from the number of segments calculated by the segment number calculation unit 119. The segment number storage unit 122 stores the number of segments (X / L−1) calculated by the subtraction processing unit 121.

The parameter storage unit 123 stores constant parameters κ _1, κ ₁ ′. Parameter storage unit 124 stores a constant parameter kappa _2.

The integration processing unit 125 calculates the model parameter α ₁ defined by Expression (11) by multiplying the square sum of the difference value between adjacent pixels calculated by the square sum calculation unit 105 and the constant parameter κ ₁ . The inter-pixel difference value square sum storage unit 126 stores the model parameter α ₁ calculated by the integration processing unit 125.

The integration processing unit 127 executes processing corresponding to the processing in step S115 in the flowchart of FIG. 2, and the model parameter α ₁ calculated by the integration processing unit 125 and the frame rate calculated by the square value calculation unit 117 are calculated. Multiplying the reciprocal square value F ^-2 calculates α ₁ F ^-2 described in the equation (34).

The integration processing unit 128 executes the process of step S116 in the flowchart of FIG. 2, and includes the calculated value of the integrated value addition processing unit 108, the double value of the constant parameter κ ₂ , and the frame calculated by the reciprocal calculation unit 115. Multiplying by the reciprocal rate F ⁻¹ , α ₂ F ⁻¹ described in the equation (34) is calculated.

The addition processing unit 129 executes the process of step S117 in the flowchart of FIG. ² and includes α ₁ F ⁻² calculated by the integration processing unit 127 and α ₂ F ⁻¹ calculated by the integration processing unit 128. Then, by adding the model parameter α ₄ calculated by the sum of squares calculation unit 110 and the model parameter α ₃ calculated by the residual component calculation unit 112, the numerator value of the equation (34) is calculated. The addition value storage unit 130 stores the value calculated by the addition processing unit 129.

The integration processing unit 131 executes the process of step S118 in the flowchart of FIG. 2. The integration processing unit 131 calculates the model parameter α ₁ calculated by the integration processing unit 125, the constant parameter κ ₁ ′, and the subtraction processing unit 121. By multiplying the number of segments (X / L-1), the value of the denominator of Equation (34) is calculated. The integrated value storage unit 132 stores the calculated value of the integrated processing unit 131.

The division processing unit 133 executes the process of step S119 in the flowchart of FIG. 2, and is defined by Expression (34) by dividing the calculated value of the addition processing unit 129 by the calculated value of the integration processing unit 131. The calculated motion estimation accuracy δ ^* (F) is calculated.

In this way, the motion estimation accuracy calculation unit 12 executes the flowcharts of FIGS. 2 and 3 according to the apparatus configuration shown in FIG. 4, and when the frame rate is given, according to the equation (34), The motion estimation accuracy δ ^* (F) for minimizing the code amount when the distortion amount is constant (quantization step width is fixed) is calculated.

  The present invention can be applied to video coding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. According to the present invention, when the frame rate is given, the distortion amount is constant. It is possible to estimate the motion estimation accuracy that minimizes the code amount.

It is an apparatus block diagram of the video coding apparatus of this invention. It is a flowchart which a motion estimation precision calculation part performs. It is a flowchart which a motion estimation precision calculation part performs. It is an apparatus block diagram of a motion estimation precision calculation part.

Explanation of symbols

101 encoding target frame storage unit 102 reference frame storage unit 103 inter-pixel difference value calculation unit 104 inter-pixel difference value storage unit 105 square sum calculation unit 106 Taylor expansion approximation error calculation unit 107 Taylor expansion approximation error storage unit 108 integrated value addition processing 109 Modified Taylor expansion approximate error storage unit 110 Sum of squares calculation unit 111 Taylor expansion approximate error sum of squares storage unit 112 Residual component calculation unit 113 Residual component storage unit 114 Frame rate storage unit 115 Reciprocal number calculation unit 116 Frame rate reciprocal number storage unit 117 square value calculation unit 118 frame rate reciprocal square value storage unit 119 segment number calculation unit 120 segment number storage unit 121 subtraction processing unit 122 segment number storage unit 123 parameter storage unit 124 parameter storage unit 125 integration processing unit 126 inter-pixel difference value 2 Sum storage unit 127 integration processing unit 128 integration processing unit 129 adding unit 130 adds value storage unit 131 integration processing unit 132 integrated value storage unit 133 division processing unit

Claims (10)

A motion estimation accuracy estimation method used in video encoding with inter-frame prediction,
The process of entering the frame rate;
Occurs when encoding with the input frame using a model that describes the relationship between the frame rate and the motion estimation accuracy that minimizes the amount of code that will occur when encoding at that frame rate. Estimating the motion estimation accuracy that minimizes the amount of code to be performed,
Outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation method. It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy, and occurs when encoding video under encoding conditions with a constant amount of distortion. A motion estimation accuracy estimation method for estimating motion estimation accuracy that minimizes the amount of code to be
The process of entering the frame rate;
A process of inputting four model parameters described in a function used to estimate prediction error power of inter-frame prediction when encoding at a certain frame rate;
A frame used for estimation of motion estimation accuracy that minimizes the amount of code that will be generated by the encoding under the condition of encoding with a constant amount of distortion based on the four model parameters that are input. Determining a quadratic function with the inverse of the rate as a variable;
By substituting the value of the input frame rate for the determined quadratic function, motion estimation accuracy is estimated to minimize the amount of code that will be generated when encoding is performed on the input frame. The process of
Outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation method. The motion estimation accuracy estimation method according to claim 2,
In the input process, (1) model parameters derived from the sum of squares of the difference values between adjacent pixels in the reference frame, and (2) an approximation error of Taylor expansion with respect to the inter-frame prediction error and the difference between adjacent pixels in the reference frame (3) a model parameter derived from an approximate value of a sum of squares of the sum of the noise superimposed on the original signal and the interpolation error in the reference frame; ) Inputting four model parameters, model parameters derived from the sum of squares of approximation errors of Taylor expansion for inter-frame prediction errors,
A characteristic motion estimation accuracy estimation method. The motion estimation accuracy estimation method according to claim 2 or 3,
In the process of the determining, represents the frame rate at F, the four model parameters _{α 1, α 2, α 3} , expressed as alpha _4, it represents the number of segments as a unit of motion compensation M, the coefficients If expressed by κ, the quadratic function is
(Α ₁ F ⁻² + α ₂ F ⁻¹ + α ₃ + α ₄ ) / κα ₁ (M−1)
To determine the function
A characteristic motion estimation accuracy estimation method. A motion estimation accuracy estimation device used in video encoding with inter-frame prediction,
An input means for inputting a frame rate;
Using a model describing the relationship between the frame rate and the motion estimation accuracy that minimizes the amount of code that will occur when encoding at that frame rate, encoding is performed with the frame input by the input means Estimating means for estimating motion estimation accuracy that minimizes the amount of code that will occur in the case;
Output means for outputting the motion estimation accuracy estimated by the estimation means,
A characteristic motion estimation accuracy estimation apparatus. It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy, and occurs when encoding video under encoding conditions with a constant amount of distortion. A motion estimation accuracy estimation device that estimates motion estimation accuracy that minimizes the amount of code to be
A frame rate input means for inputting a frame rate;
Model parameter input means for inputting four model parameters described in a function used for estimating prediction error power of inter-frame prediction when encoding at a certain frame rate;
Based on the four model parameters input by the model parameter input means, the estimation of motion estimation accuracy that minimizes the amount of code that will occur in the encoding under the condition of the encoding with a constant distortion amount Determining means for determining a quadratic function using a reciprocal of the frame rate as a variable,
Occurs when encoding is performed with the frame input by the frame rate input means by substituting the value of the frame rate input by the frame rate input means for the quadratic function determined by the determining means. Estimating means for estimating the motion estimation accuracy that minimizes the amount of code to become,
Output means for outputting the motion estimation accuracy estimated by the estimation means,
A characteristic motion estimation accuracy estimation apparatus. The motion estimation accuracy estimation apparatus according to claim 6,
The model parameter input means includes (1) a model parameter derived from a sum of squares of a difference value between adjacent pixels in a reference frame, (2) an approximation error of Taylor expansion with respect to an inter-frame prediction error, and an adjacent pixel in a reference frame. (3) model parameters derived from the sum of squares of the sum of the noise superimposed on the original signal and the interpolation error in the reference frame; 4) Input four model parameters, model parameters derived from the sum of squares of approximation errors of Taylor expansion with respect to inter-frame prediction errors,
A characteristic motion estimation accuracy estimation apparatus. The motion estimation accuracy estimation apparatus according to claim 6 or 7,
The determination means represents the frame rate as F, the four model parameters as α _1, α _2, α _3, α ₄ , the number of segments as a unit of motion compensation as M, and the coefficient as κ. Is expressed as follows:
(Α ₁ F ⁻² + α ₂ F ⁻¹ + α ₃ + α ₄ ) / κα ₁ (M−1)
To determine the function
A characteristic motion estimation accuracy estimation apparatus.   A motion estimation accuracy estimation program for causing a computer to execute the motion estimation accuracy estimation method according to any one of claims 1 to 4.   A computer-readable recording medium having recorded thereon a motion estimation accuracy estimation program for causing a computer to execute the motion estimation accuracy estimation method according to claim 1.

Images