JP4785890B2 - Motion estimation accuracy estimation method, motion estimation accuracy estimation device, motion estimation accuracy estimation program, and computer-readable recording medium recording the program - Google Patents

Description

  The present invention relates to a motion estimation accuracy estimation method and apparatus used in video encoding that realizes efficient encoding for high frame rate video, a motion estimation accuracy estimation program used to realize the motion estimation accuracy estimation method, and The present invention relates to a computer-readable recording medium on which the program is recorded.

  In recent years, there are growing expectations for super-high-quality images such as large-screen sports images and digital cinema that are full of realism. In response to this, research on high-quality video has been vigorously conducted.

  The following four elements are necessary to realize super high-quality video. That is, spatial resolution, pixel value depth, color reproducibility, and temporal resolution. In response, the former three elements are being studied in applications such as digital cinema and natural vision projects.

  However, sufficient studies have not been made on the improvement of time resolution, that is, the increase of the video frame rate, which is indispensable for expressing the natural movement of the subject.

  According to Spillmann et al., Physiological knowledge is shown that the upper limit of the number of pulses output by ganglion cells, which are output cells of the retina, is about 300 to 400 pulses per second. For this reason, it is presumed that the human visual system is the shortest and can perceive the difference in light emission up to 1/150 to 1/200 second. This means that the perceivable frame rate detection limit is 150 to 200 [frames / second]. The current video frame rate of 30, 60 [frame / second] is determined from the flicker detection limit, and is not a sufficient value to express natural motion.

  On the other hand, since the super high image quality of the video causes an explosive increase in the amount of data, an efficient encoding method is required when the code amount is limited due to network bandwidth limitations or the like.

  In the case of a high frame rate video, since inter-frame correlation increases, inter-frame prediction by motion compensation is effective. One parameter of motion compensation is motion estimation accuracy. For example, the motion estimation accuracy in an MPEG-2 compliant encoder is ½ pixel accuracy.

  In the case of a high frame rate video, since the frame interval is shortened, high accuracy is required for the shift amount. Therefore, it is important to grasp the relationship between the frame rate and the motion estimation accuracy. Intuitively, it is expected that higher motion estimation accuracy is required as the frame rate increases.

  However, in the conventional research, the influence of the frame rate is not considered, and the quantitative property regarding the relationship between the frame rate and the motion estimation accuracy is not grasped (for example, refer to Non-Patent Document 1). For this reason, the motion estimation accuracy is set by an ad hoc method. When the motion estimation accuracy is insufficient, the coding efficiency is lowered.

  On the other hand, there is also a method in which the motion estimation accuracy in a settable range is encoded with brute force and the motion estimation accuracy with the highest encoding efficiency is selected. However, in this case, the encoding processing time increases.

  Further, as a parameter necessary for explaining the characteristics of the video signal in the time axis direction, there is an aperture ratio of the imaging system in addition to the frame rate. In order to secure the amount of light incident on the imaging device, a certain aperture ratio is required. The increase in the aperture ratio is effective in reducing jerkiness (continuation of the same video frame) due to frame skipping. On the other hand, an increase in the aperture ratio causes motion blur due to an integration effect when accumulating the imaging device.

Thus, the trade-off between jerkiness and motion blur can be controlled by the aperture ratio. Further, the integration effect accompanying the increase in the aperture ratio shows a low-pass characteristic. Such low-pass characteristics of the imaging system change the properties of the video signal. Therefore, the aperture ratio is an important factor when considering the influence of the frame rate on the video signal.
J.Ribas-Corbera and DLNeuhoff.Optimizing motion-vector accuracy in block-based video coding.IEEE CSVT, Vol.11, No.4, pp.497-511, 2001.

  As described above, in order to express the natural movement of the subject, it is necessary to realize a high frame rate of the video.

  In order to realize a high frame rate of video, since the frame interval is shortened, high accuracy is required for the shift amount, and therefore it is important to grasp the relationship between the frame rate and the motion estimation accuracy.

  However, in the conventional research, the quantitative property about the relationship between the frame rate and the motion estimation accuracy is not grasped, and the actual situation is that the motion estimation accuracy is set by an ad hoc method. Moreover, the fact is that the aperture ratio is not considered at all in this setting.

  Thus, according to the prior art, when encoding a high frame rate video, if the motion estimation accuracy is insufficient, there is a problem that the encoding efficiency decreases.

  The present invention has been made in view of such circumstances, and in the encoding process for a high frame rate video, the frame rate and the motion estimation are performed in consideration of the influence of the integration effect generated according to the aperture time of the imaging system. By constructing an analytical theoretical model that expresses the relationship with accuracy and estimating the appropriate motion estimation accuracy for each frame rate using the theoretical model, efficient coding for high frame rate video has been realized. The purpose is to establish a new motion estimation accuracy estimation technique.

[1] Configuration in which frame rate and aperture ratio of imaging system can be set arbitrarily In order to achieve the above object, a motion estimation accuracy estimation apparatus of the present invention performs a frame estimation with non-integer pixel accuracy. Motion estimation that is used in video encoding with inter-prediction and minimizes the amount of code that will be generated in the encoding under the condition of constant distortion when encoding video In order to realize accuracy estimation, (b) input means for inputting frame rate and aperture ratio values, and (b) an estimation function using the frame rate and aperture ratio as variables, with a constant amount of distortion. Determining means for determining an estimation function used for estimation of motion estimation accuracy for minimizing the amount of code that will be generated under the encoding condition, and (c) the estimation determined by the determining means For functions The operation of minimizing the amount of code that will occur when encoding is performed with the frame rate and aperture ratio values input by the input means by substituting the frame rate and aperture ratio values input by the input means An estimation means for estimating the estimation accuracy and (d) an output means for outputting the motion estimation accuracy estimated by the estimation means are provided.

  The motion estimation accuracy estimation method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is provided by being recorded on an appropriate computer-readable recording medium. Alternatively, the present invention is realized by being provided via a network, installed when the present invention is carried out, and operating on a control means such as a CPU.

[2] Configuration in which the aperture ratio of the imaging system is fixed and the frame rate can be arbitrarily set To achieve the above object, the motion estimation accuracy estimation apparatus of the present invention performs motion estimation with non-integer pixel accuracy. Used in video encoding with inter-frame prediction, and when encoding video, minimizes the amount of code that will be generated in the encoding under constant encoding In order to realize the estimation accuracy of motion estimation, (b) input means for inputting a frame rate value, and (b) an estimation function having a preset aperture ratio as a constant and a frame rate as a variable. And (c) determining means for determining an estimation function used for estimating motion estimation accuracy that minimizes the amount of code that will be generated in the encoding under the condition of encoding with a constant distortion amount. Deciding on the decision means This occurs when encoding is performed using the frame rate value input by the input means and the preset aperture ratio value by substituting the frame rate value input by the input means for the estimated function. And (d) an output unit that outputs the motion estimation accuracy estimated by the estimation unit.

  The motion estimation accuracy estimation method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is provided by being recorded on an appropriate computer-readable recording medium. Alternatively, the present invention is realized by being provided via a network, installed when the present invention is carried out, and operating on a control means such as a CPU.

[3] Configuration in which the frame rate is fixed and the aperture ratio of the imaging system can be arbitrarily set To achieve the above object, the motion estimation accuracy estimation apparatus of the present invention performs motion estimation with non-integer pixel accuracy. Used in video encoding with inter-frame prediction, and when encoding video, minimizes the amount of code that will be generated in the encoding under constant encoding In order to realize the estimation accuracy of motion estimation, (a) an input means for inputting an aperture ratio value, and (b) an estimation function having a preset frame rate as a constant and an aperture ratio as a variable. And (c) determining means for determining an estimation function used for estimating motion estimation accuracy that minimizes the amount of code that will be generated in the encoding under the condition of encoding with a constant distortion amount. Estimating function determined by the determining means This occurs when encoding is performed with the numerical aperture value input by the input means and the preset frame rate value by substituting the numerical aperture value input by the input means for the number. An estimation unit that estimates the motion estimation accuracy that minimizes the code amount and (d) an output unit that outputs the motion estimation accuracy estimated by the estimation unit are provided.

  The motion estimation accuracy estimation method of the present invention realized by the operation of each of the above processing means can also be realized by a computer program, and this computer program is provided by being recorded on an appropriate computer-readable recording medium. Alternatively, the present invention is realized by being provided via a network, installed when the present invention is carried out, and operating on a control means such as a CPU.

[4] Processing of Motion Estimation Accuracy Estimating Device of the Present Invention In the motion estimation accuracy estimating device of the present invention configured as described above, according to an input value or a preset value, a frame rate to be estimated for motion estimation accuracy and When the value of the aperture ratio is set, an estimation function used to estimate the motion estimation accuracy that minimizes the amount of code that will be generated by the encoding is determined under the encoding condition in which the distortion amount is constant.

  As this estimation function, for example, it is described in a function used for estimation of prediction error power of inter-frame prediction when encoding is performed at a frame rate and a certain frame rate and a certain aperture ratio. The model parameter to be changed is defined by the function type.

For example, if the frame rate is represented by F, the aperture ratio is represented by κ, and the model parameters are represented by α ₁ ′, α ₂ ′, α ₃ ′, α ₄ , α ₅ ,
((Α ₁ 'F ^-1 + α ₂ ' F ^-1/2 + α ₃ 'F + α ₄ + α ₅ ) / α ₁ ') ^1/2
However, α ₁ '∝1 / κ, α ₂ ' ∝1 / κ ^1/2 , α ₃ '∝1 / κ
An estimation function proportional to is used.

Here, (1) the model parameter α ₁ ′ that depends on the aperture ratio derived from the square sum of the difference values between adjacent pixels in the reference frame is used, and (2) the model parameter α ₂ ′ Use the one that depends on the aperture ratio derived from the sum of the product of the approximation error of Taylor expansion for the prediction error and the difference value between adjacent pixels in the reference frame, and (3) superimpose it on the original signal as the model parameter α ₃ ′ Using a value that depends on the aperture ratio derived from the sum of the estimated value of noise power and the estimated value of noise power superimposed on the original signal after passing through the interpolation filter, (4) interpolation of pixel values as model parameter α ₄ used does not depend on the aperture ratio derived from the sum of the estimated value of the error power due to the quantization of the reference signal and the estimated value of the error power due to (5) as a model parameter alpha _5, frame It may be used which does not depend on the aperture ratio derived from the sum of the squares of the approximation error of the Taylor expansion for between prediction error.

  In this way, when the estimation function used to estimate the motion estimation accuracy that minimizes the amount of code that will occur in the encoding under the condition of the encoding with a constant amount of distortion is determined, the estimation function On the other hand, by substituting the set frame rate and aperture ratio values, the motion estimation accuracy that minimizes the amount of code that occurs when encoding is performed with the set frame rate and aperture ratio values. The estimated motion estimation accuracy is output.

  According to the present invention, when a frame rate and an aperture ratio are given, it is possible to estimate a motion estimation accuracy that minimizes a code amount when a distortion amount is constant (a quantization step width is fixed). become.

  That is, according to the present invention, it is possible to set an appropriate motion estimation accuracy according to the frame rate and the aperture ratio, and it is possible to avoid a decrease in coding efficiency due to a lack of motion estimation accuracy. In general, in encoding of a high frame rate video that requires high motion estimation accuracy, it is possible to estimate appropriate motion estimation accuracy in advance, and it is expected that the encoding process can be speeded up.

  Hereinafter, the present invention will be described in detail according to embodiments.

  As described above, in the present invention, the relationship between the frame rate and the motion estimation accuracy is expressed in consideration of the influence of the integration effect generated according to the aperture time of the imaging system in the encoding process for the high frame rate video. By constructing an analytical theoretical model and estimating an appropriate motion estimation accuracy for each frame rate using the theoretical model, efficient encoding for a high frame rate video is realized.

  Next, firstly, “[1] Relationship between frame rate and inter-frame prediction error power” is used to analytically derive the relationship between inter-frame prediction error power with respect to the frame rate and the aperture ratio. 2] Relationship between shift amount estimation error and frame rate "explains the relationship between the shift amount estimation error and the frame rate necessary for deriving the relationship between the frame rate and the aperture ratio and the inter-frame prediction error power. Then, in [3] Optimization of motion estimation accuracy, an analytical theoretical model that represents the relationship between the frame rate and aperture ratio and motion estimation accuracy is constructed, and the frame rate and aperture are calculated using the theoretical model. The configuration for estimating the appropriate motion estimation accuracy according to the rate will be described. Finally, in [4] Model parameter estimation, the relationship between the frame rate and the aperture ratio and the motion estimation accuracy will be described. Estimation of the model parameters is described which constitutes the analytical theoretical model representing the.

[1] Relationship between frame rate and inter-frame prediction error power The relationship of inter-frame prediction error power to the frame rate and the aperture ratio is derived analytically. Here, for simplicity, a one-dimensional signal will be described as an example.

A signal obtained by opening the shutter from time t to t + Δ at the position x is taken as g _t (x, Δ) (0 ≦ x ≦ X−1). The signal has X samples. Further, it is assumed that the observation signal g _t (x, Δ) is configured as follows by superimposing noise n _t (x, Δ) on the image signal f _t (x, Δ).

g _t (x, Δ) = f _t (x, Δ) + n _t (x, Δ)
At this time, Emuderuta the shutter opening time (m is a natural number) signal obtained when stretched ^- g _t (x, mΔ) can be expressed as follows using g _t (x, Δ). In the following notation, ^"- Z" symbol in (Z character) ^- shows the symbols attached on the "Z".

  In this case, the frame interval matches the opening time of the shutter, and this corresponds to the frame when the shutter is fully opened. On the other hand, a frame obtained by partially opening the shutter between times mΔ ≦ t ≦ mΔ + to mΔ, where the shutter opening time is −mΔ, can be expressed as follows. In the following notation, the symbol “˜Z” (Z is a letter) indicates a symbol attached on “Z”.

  Here, ~ m takes a value of 1 ≦ m ≦ m. That is, when the change in the shutter opening time is accompanied with the conversion of the frame rate, it is necessary to consider the influence of the low-pass filter shown in the equation (2). In the following, each term of the formula (2) is expressed as the following formula.

Observed signal ^- g _t relative (x, mΔ) (0 ≦ x ≦ X-1), a segment B composed of L samples [i] (i = 1,2, ..., X / L) as a unit Motion compensation (estimated displacement ^ d [i]) is performed, and the prediction error after motion compensation in the segment is evaluated. In the following notation, the symbol “^” in “^ Z” (Z is a letter) indicates a symbol attached to “Z”. In the following, the true displacement amount at the position x is d (x).

When the estimated shift amount ^ d [i] has decimal pixel accuracy, a reference pixel value is generated by interpolation processing. Assuming that the signal to be interpolated is _st (x), the interpolation processing is filtering represented by the following equation (5). In the following, the operator on the left side of the following formula (5) may be expressed as “I”.

  Here, δ is the accuracy of motion estimation. For example, in the case of an MPEG-2 compliant encoder, δ = 1/2 is set. The interpolation coefficient ψ [k] satisfies the following formula (6).

Using the interpolation filter, the pixel values of the reference position, ^I _- is approximated as (~ g t-1 (x + ^ d [i], ~mΔ) (q)).

_{^{Here, ~ - g t-1 (}} x + ^ d [i], ~mΔ) (q) _{^{is, - g t-1 (x}} + ^ d [i], ~mΔ) relative to the quantization step size q This is a decoded signal when quantization is performed at, and can be expressed as in the following equation (7).

Here, v _t-1 (x + ^ d [i], ~ mΔ) q represents an error caused by quantization, and v _t-1 (x + ^ d [i], ~ mΔ) is a signal. _{^{- f t-1 (x +}} ^ d [i], ~mΔ) is a parameter that depends on.

Pixel values of the reference position _{^{- f t-1 (x +}} ^ d [i], ~mΔ) and the value generated by interpolation _{^{Ι (~ - g t-1}} (x + ^ d [i], ~mΔ) (q)) Can be expressed as the following formula (8).

_{^{Here, - e t-1 int (}} x + ^ d [i], ~mΔ) is as Equation (9) below.

  At this time, the prediction error after motion compensation in the segment can be expressed by the following equation (10).

  Here, φ (x) in Equation (10) is a second-order term or later of Taylor expansion.

Further, N in equation (10) (x, ~mΔ), the interpolation error ^{_{N t-1 ref (x,}} ~mΔ) in the reference frame as the noise superimposed on the original signal ^- n _t (x, ^~mΔ) And can be expressed as the following equation (11).

In addition, ζ [i] (x) in equation (10) is an estimation error between the estimated shift amount ^ d [i] and the true shift amount d [i] (x) in x.
ζ [i] (x) = d [i] (x) − ^ d [i]
It can be expressed as

  As will be described later in “[2] Relationship between Transition Amount Estimation Error and Frame Rate”, the following formulas (12) and (13) are used with the transition amount estimation error ζ [i] (x) as a function of the frame rate F. ).

Here, κ ₁ , κ ₁ ′ and κ ₂ are constants depending on the video signal. Also, δ is the accuracy of motion estimation. For example, in the case of an MPEG-2 compliant encoder, δ = 1/2 is set.

The sum of σ [i] ² is calculated for all segments (B [i] (i = 1, 2,..., X / L)). At this time, by assuming the independence of each term of N (x, ˜mΔ), an approximate expression as shown in the following expression (14) is obtained.

Here, ˜N (x, ˜mΔ) ² is as shown in the following formula (15).

The sum of the first term and the second term in Equation (15) is defined as ~ N ₁ (x, ~ mΔ) ² , and the sum of the third term and the fourth term in Equation (15) is defined as ~ N ₂ ( By defining x, ˜mΔ) ^2, they are defined as the following formulas (16) and (17) as ˜N ₁ (x, ˜mΔ) ² and ˜N ₂ (x, ˜mΔ) ² .

  Further, assuming that the displacement estimation error ζ [i] (x) and the differential value of the pixel value are statistically independent, substituting Equation (12) and Equation (13) into Equation (14) yields a pixel The approximate formula shown in the following formula (18) is obtained for the predicted error power per hit.

Here, the model parameters α ₁ , α ₂ , α ₃ , α ₄ , α ₅ , α ₆ are as shown in the following equation (19).

Further, when the model parameter α ₁ is calculated, the following equation (20) is obtained.

  here,

Further, for the sake of simplicity, when f _t (x, Δ) is abbreviated as f _t (x), an approximate relationship of the following equation (22) is obtained.

  Here, in the approximation of the equation (22), the following homogeneous model was used.

  Furthermore, the following approximation was used.

Here, ^- d _i and ^- d _j are each an average value of d _i (x) and _{d j (x) (x∈L)} . _Further, α _{i, j} is the average displacement amount is a correction parameter for approximation using the _{^{(- d j - d i,}} ).

ρ is 1 or less because of autocorrelation of the image signal, and can be regarded as a value close to 1. For this reason,
(1-ρ) / ρ << 1
To get the inequality.

  By using this inequality, equation (22) can be approximated as follows.

Let the shutter aperture ratio κ be the ratio of m to ~ m,
~ M / m = κ (≦ 1)
Define as follows. Further, considering that m is the ratio of the downsampled frame rate F to the maximum frame rate F ₀ , ~ m is
~ M = κm = κF ₀ / F
It becomes.

Using these equations, an approximate equation of the following equation (27) is obtained for the model parameter α ₁ from the equation (20).

The calculation for the model parameter α ₂ is as follows.

  Here, the following approximate relationship is obtained.

Here, γ is 1 or −1. In the above formula, f _t (x, Δ) is abbreviated as f _t (x) for simplicity.

Thereby, an approximate expression of the following expression (30) is obtained for the model parameter α ₂ .

When the model parameter α ₃ is calculated, an approximate expression of the following expression (31) is obtained.

Here, n ₀ ² represents noise power defined as the following equation (32).

By substituting the model parameters α ₁ , α ₂ , and α ₃ obtained as described above into the equation (18), the following equation (33) is obtained.

Here, the model parameters α ₁ ′, α ₂ ′, α ₃ ′ are as shown in the following formula (34), respectively.

Note that between the model parameters α ₁ ′, α ₂ ′, α ₃ ′ and the model parameters α ₁ , α ₂ , α ₃ ,
α ₁ ′ = (1 / F) × α ₁ , α ₂ ′ = (1 / F) ^1/2 × α ₂ , α ₃ ′ = (1 / F) × α ₃
The relationship is established.

[2] Relationship between Transition Amount Estimation Error and Frame Rate Next, it will be described that the above equations (12) and (13) hold.

[2-1] Formula discrete values in the case of discrete (12) displacement amount when expressed as derived successive values of ^ d ^F [i] and to d ^F [i]. Assuming that the discretization error at this time is ε [i], ˜d ^F [i] is expressed by the following equation (35).

It is assumed that ε [i] follows a uniform distribution in which the mean is zero and the variance is proportional to the square value of motion estimation accuracy. In block matching based on the square error criterion, the estimated shift amount for each block can be approximately expressed using the shift amount of the pixel at the block center position. Assuming that x _c [i] is the coordinates of the center position of the segment B [i], the estimated shift amount when the frame rate F = F ₀ can be approximated as in the following equation (36).

  Further, the difference between the shift amount at ^ x and the shift amount at ^ x 'can be modeled as a normal distribution in which the average is zero and the variance is proportional to the difference value between the two coordinate positions. Here, {circumflex over (x)} and {circumflex over (x)} ′ indicate coordinate positions in the segment B [i] (i = 1, 2,..., X / L). That is, approximation as shown in the following equation (37) is possible.

Here, E [·] denotes the expectation operation, ^ c _h is a parameter of the signal-dependent. The model is the difference value (d ^F0 [i] (^ x) −d ^F0 [i] (^ x) of the transition amount in all segments B [i] (i = 1, 2,..., X / L). Give a good approximation of the set average for ')).

  The following equation (38) is obtained from the equations (36) and (37).

Here, c _h ² = (X / L) ^ c _h ² . ξ is a coordinate value in a coordinate system with the center position of the segment B [i] as the origin, and ξ can be expressed as xx ₀ [i]. That is, ξ is a distance from the center position of the segment B [i].

  Deriving the relationship between two shifts at different frame rates. It is assumed that the movement of the object between adjacent frames follows a uniform motion. In this case, the amount of displacement of the object is proportional to the frame interval. That is, it can be said that the shift amount is inversely proportional to the frame rate. Thereby, the relationship of the following formula | equation (39) is obtained.

  Using Equation (35), Equation (39) and the assumption that ε [i] does not depend on the frame rate, Equation (40) below is obtained.

  By using the assumption that the equation (40) and ε [i] are statistically independent of the shift amount, the following equation (41) is obtained.

Here, ξ = x−x _c [i], and δ represents the motion estimation accuracy. For example, in the case of an MPEG-2 encoder, δ is set to 1/2, and in the case of an MPEG-4 ASP (Advanced Simple Profile) encoder, δ is set to ¼.

In formula (41),
κ ₁ = F ₀ ² c _h ² Σξ ² where Σ is the sum of ξ Equation (42)
κ ₁ ′ = XY / 12 Formula (43)
Is expressed as (12).

[2-2] Derivation of Equation (13) Assuming that ε [i] follows a zero-average uniform distribution, the following Equation (44) is obtained.

ζ ^F [i] (x)
ζ ^F [i] (x) = d ^F [i] (x) − ^ d ^F [i]
It is assumed that the deviation estimation error is defined as follows.

At this time, {ΣΣζ ^F [i] (x)} ² can be expressed as the following equation (45).

  The first term of equation (45) can be approximated by equation (41). The second term satisfies the following equation (46) from the Schwarz inequality.

Here, ξ = x−x _c [i] and ξ ′ = x′−x _c [i]. The approximate expression of the following expression (47) is obtained from the inequality of the expression (46).

  Here, β is a parameter that takes a value in the range of −1 to 1. By substituting Equation (47) and Equation (41) into Equation (45), the following Equation (48) is obtained.

  Therefore, the following formula (49) is obtained.

Here, γ is 1 or -1. Formula (13) is obtained by setting the following formula (50) described in formula (49) as κ ₂ .

[3] Optimization of motion estimation accuracy Consider the code amount when the distortion amount is constant (quantization step width is fixed) as an evaluation function. Hereinafter, a method of calculating motion estimation accuracy that minimizes the evaluation function when a quantization step width is given will be described in detail.

When the quantization step width q, the frame rate F, and the motion estimation accuracy δ are given, the code amount of the inter-frame prediction error is
R _c (q, F, δ) = (1/2) log ₂ γ _c (P (F, δ) / q ² )
... Formula (51)
Can be modeled as follows.

Here, γ _c is a signal-dependent constant parameter. P (F, δ) is
P (F, δ) = Σσ [i] ² where Σ is the sum for i = 1 to X / L and represents the prediction error power sum in the frame.

On the other hand, when the frame rate F and the motion estimation accuracy δ are given, the code amount of the shift amount is
R _mv (F, δ) = (X / L) log ₂ (2U (F) / δ) Expression (52)
Can be modeled as follows.

  Here, U (F) represents a motion estimation search range when the frame rate is F, and the shift amount ^ d can take a range of -U (F) ≤ ^ d≤U (F) -1. Shall. Therefore, 2U (F) / δ represents the number of candidates for the estimated shift amount within the search range.

Since the shift amount decreases in proportion to the frame interval, U (F) is inversely proportional to F from the viewpoint of code amount reduction.
U (F) = U ₀ F ₀ / F
And so on. Here, U ₀ is a parameter representing the range of motion estimation when the frame rate is F ₀ .

At this time, the sum of the code amount of the prediction error signal and the shift amount is
R (q, F, δ) = R _c (q, F, δ) + R _mv (F, δ) (53)
It becomes.

  This equation (53) gives the code amount when the distortion amount is constant (quantization step width q is fixed).

In this way, when the frame rate F and the quantization step width q are given, the motion estimation accuracy δ ^* (F) that minimizes R (q, F, δ) is calculated. Δ ^* (F) to be obtained is
∂R (q, F, δ) / ∂δ
= ∂R _c (q, F, δ) / ∂δ + ∂R _mv (F, δ) / ∂δ = 0
... Formula (54)
The following expression is satisfied.

Δ ^* (F) satisfying the equation (54) can be expressed as the following equation (55).

The model parameters α ₁ ′, α ₂ ′, α ₃ ′ described in the equation (55) are as described in the equation (34):
α ₁ '∝1 / κ, α ₂ ' ∝1 / κ ^1/2 , α ₃ '∝1 / κ
In this way, it has the property of changing its value by the aperture ratio κ. That is, δ ^* (F) shown in the equation (55) has a property that its function form is defined by the frame rate F and the aperture ratio κ, and δ ^* (F , Κ).

From this, when the aperture ratio κ is given in advance as a fixed value, and the frame rate F is given as a variable, the aperture ratio κ given as the fixed value and the frame rate F given as the variable value Is substituted into the equation (55), the motion estimation accuracy δ ^* (F, κ) that minimizes the code amount can be calculated at the frame rate and the aperture ratio.

When the frame rate F is given as a fixed value in advance, and the aperture ratio κ is given as a variable, the frame rate F given as the fixed value and the aperture rate κ given as the variable value Is substituted into the equation (55), the motion estimation accuracy δ ^* (F, κ) that minimizes the code amount can be calculated at the frame rate and the aperture ratio.

When the frame rate F and the aperture ratio are given as variables, the frame rate F given as the variable value and the aperture ratio given as the variable value are substituted into the equation (55), so that the frame In the rate and the aperture ratio, the motion estimation accuracy δ ^* (F, κ) that minimizes the code amount can be calculated.

[4] Estimation of model parameters Next, estimation of model parameters α ₁ ′, α ₂ ′, α ₃ ′, α ₄ and α ₅ will be described.

[4-1] 'estimated model parameters alpha _1' of the model parameter alpha ₁ is intended to be defined by the definition expression of the model parameters alpha ₁ 'of formula (34).

As for 2σ _s ² (1-ρ) constituting the model parameter α ₁ ′, it is assumed that an estimation means is given from the outside. For example, considering that σ _s ² is the sum of squares of pixel values in a frame and ρ is a correlation coefficient between pixels, 2σ _s ² (1−ρ) is the sum of squares of the difference values between adjacent pixels. It is an approximate value. Therefore, in the encoding target frame, a sum of squares of the difference values between adjacent pixels is obtained and set as an approximate value for 2σ _s ² (1−ρ).

Further, L, F ₀ , and κ constituting the model parameter α ₁ ′ are set as parameters at the time of encoding (κ can be handled as a variable), and therefore can be appropriately read. .

Assume that an estimation means is given from the outside for κ ₁ constituting the model parameter α ₁ ′. For example, a motion vector is obtained for each pixel, and the relationship between the square norm of the difference vector between the two motion vectors and the square norm of the distance between the pixels is measured, and the ratio between the ^{two is used as} an estimated value of ^ c _h ² . Since the components of κ ₁ ^ c _h ² are set as parameters at the time of encoding, an estimated value of κ ₁ is obtained based on the equation (42).

Therefore, using the above values, an estimated value of the model parameter α ₁ ′ is obtained based on the definition formula of the model parameter α ₁ ′ in Expression (34).

[4-2] 'estimated model parameters alpha _3' of the model parameter alpha ₃ is intended to be defined by the definition expression of the model parameters alpha ₃ 'of formula (34).

Assume that an estimation means is given from the outside for n ₀ ² constituting the model parameter α ₃ ′. For example, it is performed as follows. n _t (x, δ) ² is the power of the noise component due to the thermal noise of the camera. First, a still area in a frame is identified. A method for identifying a static region is given from the outside. For example, a frame is divided into blocks, motion estimation is performed for each block, and a region where the motion vector becomes a zero vector is set as a still region. An inter-frame difference is obtained for a block identified as a still region. The sum of squares of the inter-frame difference signal is obtained, and the sum of squares is set as an estimated value of n ₀ ² .

Since κ, F ₀ , and ψ [k] constituting the model parameter α ₃ ′ are set as parameters at the time of encoding (κ can be handled as a variable), and can be appropriately read. it can.

Sum So, using the above values, based on the definition formula of the model parameters alpha ₃ 'of formula (34), an estimate of the noise power n ₀ ² and the estimated value n ₀ ² of the noise power after passing through the interpolation filter To obtain an estimate of the model parameter α ₃ ′.

[4-3] Estimation Model parameter alpha ₄ model parameters alpha ₄ are those defined by the formula (19) the model parameters alpha ₄ defining equation and equation (17).

Constitute the model parameters _{^{α 4 - e t-1 int}} (x + ^ d [i], ~mΔ) 2 is the estimated error due to interpolation of the pixel value of the decimal pixel position. ^- obtain f _t (x, ~mΔ) and ^{_{(δ) - f t (x}} , ~mΔ) against performs down-sampling processing, the image signals obtained by reducing the resolution to [delta]. It is assumed that the filter used for the downsampling process is given from outside. For example, an inverse filter of the above-described interpolation filter or a band limiting filter based on the sinc function can be considered. The ^- get f _t (x, ~mΔ) a ^{_{'- f t (x, ~mΔ}} ) relative ([delta]), by the aforementioned interpolation filter, enlarged signal the resolution to [delta] ^-1 times. The estimated value of the interpolation error associated with the interpolation filter is approximated by the following equation (56).

I (v _t−1 (x + ^ d [i], ˜mΔ)) ² q ² constituting the model parameter α ₄ is an error accompanying quantization of the reference signal. The difference value (pixel average) between the prediction error power when the original signal is used as the reference signal and the prediction error power when the decoded signal obtained by quantization / inverse quantization is used as the reference signal is represented by v _t−1. (x + ^ d [i] , ~mΔ) estimate of the ² q ^{2 -} and epsilon _q. Moreover, the estimate ^- using _{ε q, I (v t-} 1 (x + ^ d [i], ~mΔ)) a ² q ^2, approximated as the following equation (57).

Therefore, it is given by equation _{^{(56) - e t-1}} int (x + ^ d [i], ~mΔ) and approximation of ^2, I obtained by the formula _{(57) (v t-1} (x + ^ d [ i], ˜mΔ)) ² Estimate value of model parameter α ₄ is obtained according to the sum of q ² and approximate value.

Estimating model parameters alpha ₅ in the 4-4 model parameter alpha ₅ is intended to be defined by the definition expression of the model parameters alpha ₅ in the formula (19). In addition, although not explicitly described in Expression (19), φ (x) in Expression (19) includes noise n (˜m).

Σ _x {| φ (x) + n (˜m) | ² } ≈Σ _x {φ (x) ² + n (˜m) ² } constituting the model parameter α ₅ is a motion compensation frame from the equation (14). It can be seen that the error power sum for the inter prediction.

Therefore, the sum of squares of the error signal is obtained for the sample frame. The average of the square sum per segment is taken as an estimated value of Σ _x {φ (x) ² + ˜N (x, ˜mΔ) ² }. This estimated value is called interframe prediction error power. The value obtained by subtracting the inter-frame prediction error power from the inter-frame prediction error power is set as the estimated value of the model parameter α ₅ .

[4-5] 'estimation of the model parameters alpha _2' model parameter alpha ₂ is intended to be defined by the definition expression of the model parameter alpha ₂ 'of formula (34).

Assuming that φ (x) constituting the model parameter α ₂ ′ is an error signal having a property close to white noise, {Σ _x φ (x)} ² ≈Σ _x {| φ (x) | ² } Considering this, Σ _x φ (x) ≈ [Σ _x {| φ (x) | ² }] ^1/2 = (α ₅ ) ^1/2 .

The average of the sum of φ (x) per segment is regarded as an estimated value of Σ _x φ (x), and this estimated value is called a Taylor expansion approximation error. The absolute value of the value obtained by multiplying this Taylor expansion approximation error by α ₁ ′ is equal to | α ₂ ′ |. However, γ is undetermined, and uncertainty remains regarding the sign. Regarding the sign of α ₂ ′, if the value obtained by the equation (55) is a negative value or a value of 1 or more, the corresponding α ₂ ′ is not used. In other cases, two α ₂ ′ and the corresponding two ME precisions obtained by the equation (55) are substituted into the equation (53), and the calculated encoding cost value becomes smaller. _{Use 2} '.

Thus, according to the present invention, when the frame rate F and the aperture ratio κ are given, the code amount when the distortion amount is constant (the quantization step width is fixed) according to the equation (55). The motion estimation accuracy to be minimized δ ^* (F) can be calculated.

  Next, examples of the present invention will be described.

  FIG. 1 shows a device configuration of a video encoding device 1 having the present invention.

  As shown in this figure, the video encoding device 1 of the present invention uses a video file 10 that stores video to be encoded and inter-frame prediction that performs motion estimation with non-integer pixel accuracy, and uses the video file 10. An encoding unit 11 that encodes the video stored in the video, a motion estimation accuracy calculation unit 12 that calculates the motion estimation accuracy necessary for the encoding unit 11 to perform inter-frame prediction, and provides the motion estimation accuracy to the encoding unit 11; And an encoded data file 13 for storing the encoded data encoded by the encoding unit 11.

  2 to 4 show examples of flowcharts executed by the motion estimation accuracy calculation unit 12. Next, processing executed by the present invention will be described in detail according to these flowcharts. In the following description, it is assumed that the aperture ratio κ is a preset value for convenience of description.

  As shown in the flowchart of FIG. 2, the motion estimation accuracy calculation unit 12 first reads the shutter aperture ratio κ in step S101, and then reads the number of pixels L in the segment in step S102.

  Subsequently, in step S103, the encoding target frame is read, and in step S104, the reference frame is read.

Subsequently, in step S105, a difference value between adjacent pixels in the reference frame is calculated, and in a subsequent step S106, a square sum of the calculated adjacent pixel difference values is calculated. That is, “2σ _s ² (1−ρ)” described in the definition formula of the model parameter α ₁ ′ shown in the equation (34) is calculated.

Subsequently, in step S107, an approximation error of Taylor expansion with respect to the inter-frame prediction error is calculated. That is, “φ (x)” described in the definition equation of the model parameter α ₂ ′ shown in the equation (34) and the definition equation of the model parameter α ₅ shown in the equation (19) is calculated.

Subsequently, in step S108, a product of the approximation error of Taylor expansion calculated in step S107 and the square root of the square sum of the difference value between adjacent pixels calculated in step S106 is calculated. That is, “φ (x) (2σ _s ² (1−ρ)) ^1/2 ” described in the definition formula of the model parameter α ₂ ′ shown in Expression (34) is calculated.

Subsequently, at step S109, it calculates a model parameter alpha ₅ by calculating the square sum of the approximation error of the Taylor expansion calculated in step S107. That is, the model parameter α ₅ is calculated according to the definition equation shown in equation (19).

Subsequently, in step S110, a model parameter α ₃ ′ that is an error component due to the noise signal is calculated. Details of the calculation process of the model parameter α ₃ ′ executed in step S110 will be described later with reference to the flowchart of FIG.

Subsequently, in step S111, the interpolation filter, to calculate a model parameter alpha ₄ is an error component due to quantization. Details of the calculation process of the model parameter α ₄ executed in step S111 will be described later with reference to the flowchart of FIG.

On the other hand, in step S112, the frame rate F is read. In the subsequent step S113, the reciprocal number F ⁻¹ of the read frame rate is calculated. In the subsequent step S114, the square root F ^−1/2 of the reciprocal number of the read frame rate is calculated. To do.

In this way, when the processing of step S101 to step S111 and step S112 to step S114 is completed, subsequently, in step S115, the calculated value of step S106 is set to F ⁻¹ and the constant parameter κ ₁ / L · κ · F _0. Α ₁ 'F ^-1 described in Expression (55) is calculated.

That is, by multiplying the constant parameters _{κ 1 / L · κ · F} 0 to the calculated value of the step S106, to calculate the 'model parameters alpha ₁ according to the definition formula of' model parameters alpha ₁ shown in formula (34), it By multiplying by F ⁻¹ , α ₁ ′ F ⁻¹ described in the equation (55) is calculated.

Subsequently, in step S116, a value obtained by multiplying the calculated value of step S108 by F ^−1/2 and the constant parameter 2κ ₂ · 2γ / L · (κ · F ₀ ) ^1/2 is calculated. Α ₂ ′ F ^−1/2 described in ( ¹⁾ is calculated.

That is, by multiplying the calculated value in step S108 by the constant parameter 2κ ₂ · 2γ / L · (κ · F ₀ ) ^1/2 , the model parameter α ₂ according to the definition formula of the model parameter α ₂ ′ shown in the equation (34). By calculating 'and multiplying it by F ^-1/2 , α ₂ ' F ^-1/2 described in the equation (55) is calculated.

Subsequently, in step S117, α ₃ 'F described in the equation (55) is calculated by calculating a value obtained by multiplying the calculated value in step S110 by F.

That is, 'since the calculated, the calculated alpha _3' model parameter alpha ₃ in step S110 by calculating the value obtained by multiplying F to, to calculate the alpha ₃ 'F described in the equation (55) It is.

  Subsequently, in step S118, by calculating the sum of the calculated values of step S115, step S116, step S117, step S111, and step S109, the value of the molecule in the square root of the calculation formula shown in formula (55) is calculated. .

That is, α ₁ ′ F ⁻¹ that is the calculated value of step S115, α ₂ ′ F ^−1/2 that is the calculated value of step S116, α ₃ ′ F that is the calculated value of step S117, and By calculating the sum of α ₄ that is the calculated value and α ₅ that is the calculated value in step S109, the value of the molecule α ₁ 'F ⁻¹ + α ₂ ' in the square root of the calculation formula shown in Formula (55) F ^-1/2 + α ₃ 'F + α ₄ + α ₅
Is calculated.

Subsequently, in step S119, (1-1 / L) is calculated, and the calculated value, the calculated value in step S106, the multiplication value of the constant parameter κ ₁ / L · κ · F ₀ and the constant parameter κ ₁ ′ are calculated. By calculating, the value of the denominator in the square root of the calculation formula shown in Formula (55) is calculated.

That is, (1-1 / L) is calculated, and the model parameter α ₁ ′ is calculated by multiplying the calculated value in step S106 by the constant parameter κ ₁ / L · κ · F ₀ to calculate (1 −1 / L) and the calculated α ₁ ′ multiplied by the constant parameter κ ₁ ′, the denominator value (1-1 / L) in the square root of the equation (55) is calculated. ) ・ Α ₁ '・ κ ₁ '
Is calculated.

Subsequently, in step S120, a value obtained by dividing the calculated value in step S118 by the calculated value in step S119 is (α ₁ ′ F ⁻¹ + α ₂ ′ F ^−1/2 + α ₃ ′ F + α ₄ + α ₅ ) / (( 1-1 / L) ・ α ₁ '・ κ ₁ ')
Is calculated.

Subsequently, in step S121, by calculating the square root of the calculated value in step S120, the motion estimation accuracy δ ^* (F) that minimizes the code amount when the distortion amount is constant (quantization step width is fixed). Is calculated.

That is, according to the definition of Expression (55), by calculating the square root of the calculated value in step S120, the motion estimation accuracy δ ^* that minimizes the code amount when the distortion amount is constant (the quantization step width is fixed) ^. (F) is calculated.

In this way, the motion estimation accuracy calculation unit 12 uses the code when the distortion amount is constant (the quantization step width is fixed) according to the equation (55) when the frame rate F and the aperture ratio κ are given. The motion estimation accuracy δ ^* (F) that minimizes the amount is calculated.

Next, details of the calculation process of the model parameter α ₃ ′ executed in step S110 of the flowchart of FIG. 2 will be described according to the flowchart of FIG.

Here, the model parameter α ₃ ′ is defined by the definition formula of the model parameter α ₃ ′ shown in the equation (34).

  When entering the process of step S110 in the flowchart of FIG. 2, the motion estimation accuracy calculation unit 12 first reads the encoding target frame in step S201 as shown in the flowchart of FIG. Read a reference frame.

  Subsequently, in step S203, motion estimation with an adjacent frame is performed, and a still region is identified by specifying a region where the motion vector is a zero vector.

Subsequently, in step S204, the interframe differential power in the still region is obtained and used as an estimated value of the noise power superimposed on the original signal. That is, the noise power n ₀ ² described in the definition equation of the model parameter α ₃ ′ shown in the equation (34) is estimated.

Subsequently, in step S205, a transfer function of the interpolation filter is calculated. That is, Σψ [k] ² described in the definition equation of the model parameter α ₃ ′ shown in the equation (34) is calculated.

Subsequently, in step S206, the band pass rate obtained from the transfer function of the interpolation filter is multiplied by the noise power obtained in step S204 to obtain an estimated value of noise power to be superimposed on the original signal after passing through the interpolation filter. That is, n ₀ ² · Σψ [k] ² described in the definition formula of the model parameter α ₃ ′ shown in the equation (34) is calculated.

Subsequently, in step S207, the sum of the estimated value of the noise power superimposed on the original signal obtained in step S204 and the estimated value of the noise power superimposed on the original signal after passing through the interpolation filter obtained in step S206 is calculated. . That is, (1 + Σψ [k] ² ) n ₀ ² described in the definition formula of the model parameter α ₃ ′ shown in the equation (34) is calculated.

Subsequently, in step S208, a multiplication value F ₀ · κ · X of the maximum frame rate F ₀ , the aperture ratio κ, and the number of pixels X is calculated.

Finally, in step S209, a value obtained by multiplying the power value obtained in step S207 by the calculated value in step S208 is calculated, thereby being defined by the definition equation of the model parameter α ₃ ′ shown in equation (34). The model parameter α ₃ ′ is calculated, and the process of step S110 in the flowchart of FIG.

Next, the details of the calculation process of the model parameter α ₄ executed in step S111 of the flowchart of FIG. 2 will be described according to the flowchart of FIG.

Here, the model parameter alpha ₄ are those defined by the formula (19) the model parameters alpha ₄ defining equation and equation (17).

  When the process of step S111 in the flowchart of FIG. 2 starts, the motion estimation accuracy calculation unit 12 first reads a reference frame in step S301 as shown in the flowchart of FIG.

  Subsequently, in step S302, the filter coefficient of the interpolation filter (interpolation coefficient ψ [k] described in the above equation (6)) is read.

  Subsequently, in step S306, the quantization parameter is read. In subsequent step S307, the quantization step width q is calculated based on the read quantization parameter.

Subsequently, in step S308, the quantization error power is calculated based on the calculated quantization step width. In other words, which are described in formulas (57) ^- it is to calculate the epsilon _q ^2.

Subsequently, in step S309, the transfer function of the interpolation filter is calculated, and the band pass rate is calculated. That is, Σψ [k] ² described in the equation (57) is calculated.

In step S310, the calculated value in step S308 and the calculated value in step S309 are multiplied to obtain an estimated value for the quantization error of the reference image after passing through the interpolation filter. That is, the estimated value of I (v _t-1 (x + ^ d [i],... MΔ)) ² q ² described on the left side of the equation (57) is used.

  On the other hand, when the process of step S302 is completed, the original signal of the reference frame is reduced in step S303, and the reduced image is enlarged in the subsequent step S304.

Subsequently, in step S305, the power of the difference signal between the calculated value in step S310 and the calculated value in step S304 is obtained and used as an estimated value for the estimated error power of the pixel value at the interpolation position. That is, as described in the left side of the equation _{^{(56) - e t-1}} int (x + ^ d [i], ~mΔ) is taken as the estimated value of ^2.

And, finally, defined at step S311, by calculating the sum of the power obtained by the power and the step S310 obtained in step S305, the model parameters alpha ₄ defining equation and equation (17) in equation (19) The calculated model parameter α ₄ is calculated, and the process of step S111 in the flowchart of FIG.

  FIG. 5 illustrates an example of a device configuration of the motion estimation accuracy calculation unit 12 that performs the processing described above.

  As shown in this figure, the motion estimation accuracy calculation unit 12 includes an encoding target frame storage unit 101, a reference frame storage unit 102, a frame rate storage unit 103, an inverse number calculation unit 104, and a frame rate inverse number storage unit 105. A square root calculation unit 106, a frame rate reciprocal square root storage unit 107, a segment number reciprocal number calculation unit 108, a segment number reciprocal number storage unit 109, an addition / subtraction processing unit 110, a multiplier parameter storage unit 111, and an inter-pixel The difference value calculation unit 112, the inter-pixel difference value storage unit 113, the square sum calculation unit 114, the integration processing unit 115, the parameter storage unit 116, the inter-pixel difference value square sum storage unit 117, and the integration processing unit 118. Taylor expansion approximation error calculation unit 119, Taylor expansion approximation error storage unit 120, integrated value addition processing unit 121, modified Taylor expansion Similar error sum storage unit 122, integration processing unit 123, parameter storage unit 124, frame superimposed noise power estimation unit 125, frame superimposed noise power storage unit 126, integration processing unit 127, and integrated value storage unit 128 Frame superimposed noise power estimation unit 129, frame superimposed noise power storage unit 130, sum of squares calculation unit 131, Taylor expansion approximate error sum of squares storage unit 132, addition processing unit 133, addition value storage unit 134, An integration processing unit 135, an integration value storage unit 136, a division processing unit 137, and a square root calculation unit 138 are provided.

  In the motion estimation accuracy calculation unit 12 configured as described above, the encoding target frame storage unit 101 stores the encoding target frame. The reference frame storage unit 102 stores a reference frame.

The frame rate storage unit 103 stores the frame rate F. The reciprocal calculation unit 104 calculates the reciprocal F ^{−1 of the} frame rate. The frame rate reciprocal storage unit 105 stores the reciprocal F ⁻¹ of the frame rate calculated by the reciprocal calculation unit 104. The square root calculator 106 calculates the square root F ^−1/2 of the reciprocal of the frame rate. The frame rate reciprocal square root storage unit 107 stores the square root F ^−1/2 of the reciprocal of the frame rate calculated by the square root calculation unit 106.

  The reciprocal number calculation unit 108 calculates a reciprocal 1 / L of the number L of pixels in the segment. The segment number reciprocal storage unit 109 stores 1 / L calculated by the segment number reciprocal calculation unit 108. The addition / subtraction processing unit 110 calculates a value (1-1 / L) obtained by subtracting 1 / L calculated by the segment reciprocal number calculation unit 108 from 1. The multiplier parameter storage unit 111 stores the multiplier parameter (1-1 / L) calculated by the addition / subtraction processing unit 110.

  The inter-pixel difference value calculation unit 112 calculates a difference value between adjacent pixels in the reference frame. The inter-pixel difference value storage unit 113 stores the inter-pixel difference value calculated by the inter-pixel difference value calculation unit 112. The sum of squares calculation unit 114 calculates the sum of squares of the difference values between adjacent pixels calculated by the inter-pixel difference value calculation unit 112.

The integration processing unit 115 calculates the model parameter α ₁ ′ defined by Expression (34) by multiplying the square sum of the difference value between adjacent pixels calculated by the square sum calculation unit 114 and the constant parameter. The parameter storage unit 116 stores constant parameters required by the integration processing unit 115 and the like. The inter-pixel difference value square sum storage unit 117 stores the model parameter α ₁ ′ calculated by the integration processing unit 115.

The integration processing unit 118 multiplies the model parameter α ₁ ′ calculated by the integration processing unit 115 by the reciprocal F ^{−1 of the} frame rate calculated by the reciprocal calculation unit 104, thereby α ₁ described in the equation (55). 'F ^-1 is calculated.

  The Taylor expansion approximation error calculation unit 119 calculates an approximation error of Taylor expansion with respect to the inter-frame prediction error. The Taylor expansion approximation error storage unit 120 stores the Taylor expansion approximation error calculated by the Taylor expansion approximation error calculation unit 119.

  The integrated value addition processing unit 121 calculates the sum of products of the Taylor expansion approximation error calculated by the Taylor expansion approximation error calculation unit 119 and the inter-pixel difference value calculated by the inter-pixel difference value calculation unit 112. The corrected Taylor expansion approximate error sum storage unit 122 stores the calculated value of the integrated value addition processing unit 121 (referred to as a corrected Taylor expansion approximate error).

The integration processing unit 123 is described in Expression (55) by multiplying the modified Taylor expansion approximation error calculated by the integration value addition processing unit 121 by the constant parameter and F ^−1/2 calculated by the square root calculation unit 106. Α ₂ 'F ^-1/2 is calculated. The parameter storage unit 124 stores constant parameters required by the integration processing unit 123.

The frame superimposed noise power estimation unit 125 receives the encoding target frame and the reference frame as input, and sums the estimated noise power value superimposed on the original signal and the estimated noise power value superimposed on the original signal after passing through the interpolation filter. The model parameter α ₃ ′ derived from is estimated. The frame superimposed noise power storage unit 126 stores the model parameter α ₃ ′ estimated by the frame superimposed noise power estimation unit 125.

The integration processing unit 127 multiplies the model parameter α ₃ ′ estimated by the frame superimposed noise power estimation unit 125 by the frame rate F stored in the frame rate storage unit 103, thereby α ₃ described in Expression (55). 'F is calculated. The integrated value storage unit 128 stores α ₃ ′ F calculated by the integration processing unit 127.

The frame superimposed noise power estimation unit 129 receives the encoding target frame and the reference frame as input, and derives from the sum of the error power estimation value accompanying pixel value interpolation and the error power estimation value accompanying reference signal quantization. The model parameter α ₄ to be estimated is estimated. The frame superimposed noise power storage unit 130 stores the model parameter α ₄ estimated by the frame superimposed noise power estimation unit 129.

The square sum calculation unit 131 calculates the model parameter α ₅ by calculating the square sum of the Taylor expansion approximation error calculated by the Taylor expansion approximation error calculation unit 119. The Taylor expansion approximate error square sum storage unit 132 stores the model parameter α ₅ calculated by the square sum calculation unit 131.

The addition processing unit 133 includes α ₁ ′ F ⁻¹ calculated by the integration processing unit 118, α ₂ ′ F ^−1/2 calculated by the integration processing unit 123, and α ₃ ′ F calculated by the integration processing unit 127. , and the calculated alpha ₄ frames superimposed noise power estimation unit 129, by adding the calculated alpha ₅ of square sum calculator 131 is a value of molecules in the square root calculation formula shown in equation (55) alpha ^{_{1 'F -1 + α 2'}} F -1/2 + α 3 'F + α 4 + α 5 is calculated. The addition value storage unit 134 stores α ₁ ′ F ⁻¹ + α ₂ ′ F ^−1/2 + α ₃ ′ F + α ₄ + α ₅ calculated by the addition processing unit 133.

The integration processing unit 135 stores α ₁ ′ stored in the inter-pixel difference value square sum storage unit 117, constant parameter κ ₁ ′ stored in the parameter storage unit 116, and the multiplier parameter storage unit 111 (1-1 / L) is multiplied to calculate (1-1 / L) α ₁ 'κ ₁ ', which is the value of the denominator in the square root of the calculation formula shown in Formula (55). The integrated value storage unit 136 stores (1-1 / L) α ₁ 'κ ₁ ' calculated by the integration processing unit 135.

The division processing unit 137 divides the calculated value of the addition processing unit 133 by the calculated value of the integration processing unit 135,
(Α ₁ 'F ^-1 + α ₂ ' F ^-1/2 + α ₃ 'F + α ₄ + α ₅ ) / ((1-1 / L) α ₁ ' κ ₁ ')
Is calculated.

The square root calculation unit 138 calculates (α ₁ ′ F ⁻¹ + α ₂ ′ F ^−1/2 + α ₃ ′ F + α ₄ + α ₅ ) / ((1-1 / L) α ₁ ′ κ ₁ calculated by the division processing unit 137. ')
The motion estimation accuracy δ ^* (F) that minimizes the code amount when the distortion amount is constant (quantization step width is fixed) is calculated according to the equation (55).

In this way, the motion estimation accuracy calculation unit 12 executes the flowcharts of FIGS. 2 to 4 according to the apparatus configuration shown in FIG. 5 to obtain the equation when the frame rate F and the aperture ratio κ are given. According to (55), the motion estimation accuracy δ ^* (F) that minimizes the code amount when the distortion amount is constant (quantization step width is fixed) is calculated.

  The present invention can be applied to video coding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. According to the present invention, when a frame rate and an aperture ratio are given, the amount of distortion is constant. It is possible to estimate the motion estimation accuracy that minimizes the code amount.

It is an apparatus block diagram of the video coding apparatus of this invention. It is a flowchart which a motion estimation precision calculation part performs. It is a flowchart which a motion estimation precision calculation part performs. It is a flowchart which a motion estimation precision calculation part performs. It is an apparatus block diagram of a motion estimation precision calculation part.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Encoding object frame memory | storage part 102 Reference frame memory | storage part 103 Frame rate memory | storage part 104 Reciprocal number calculation part 105 Frame rate reciprocal number memory | storage part 106 Square root calculation part 107 Frame rate reciprocal square root memory | storage part 108 Segment number reciprocal number calculation part 109 Segment number reciprocal number memory | storage part 110 Addition / Subtraction Processing Unit 111 Multiplier Parameter Storage Unit 112 Interpixel Difference Value Calculation Unit 113 Interpixel Difference Value Storage Unit 114 Square Sum Calculation Unit 115 Integration Processing Unit 116 Parameter Storage Unit 117 Interpixel Difference Value Square Sum Storage Unit 118 Integration Processing Unit 119 Taylor expansion approximation error calculation unit 120 Taylor expansion approximation error storage unit 121 integrated value addition processing unit 122 modified Taylor expansion approximation error sum storage unit 123 integration processing unit 124 parameter storage unit 125 frame superimposed noise power estimation unit 12 Frame superimposed noise power storage unit 127 Integration processing unit 128 Integrated value storage unit 129 Frame superimposed noise power estimation unit 130 Frame superimposed noise power storage unit 131 Sum of squares calculation unit 132 Taylor expansion approximate error square sum storage unit 133 Addition processing unit 134 Addition value Storage unit 135 Integration processing unit 136 Integration value storage unit 137 Division processing unit 138 Square root calculation unit

Claims (12)

It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. When video is encoded, it occurs in the encoding under the condition of constant distortion. A motion estimation accuracy estimation method for estimating motion estimation accuracy that minimizes the amount of code,
Entering frame rate and aperture ratio values;
An estimation function that uses the frame rate and aperture ratio as variables, and is used to estimate the motion estimation accuracy that minimizes the amount of code that will be generated by the encoding under the condition of encoding with a constant amount of distortion. Determining the estimated function to be obtained,
By substituting the input frame rate and aperture ratio values for the determined estimation function, the amount of code that will be generated when encoding is performed using the input frame rate and aperture ratio values. Estimating the motion estimation accuracy to be minimized;
Outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation method. It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. When video is encoded, it occurs in the encoding under the condition of constant distortion. A motion estimation accuracy estimation method for estimating motion estimation accuracy that minimizes the amount of code,
The process of entering the frame rate value,
An estimation function that uses a preset aperture ratio as a constant and a frame rate as a variable, and is a motion that minimizes the amount of code that will be generated by the encoding under the encoding conditions with a constant amount of distortion A process of determining an estimation function used to estimate the estimation accuracy;
It occurs when encoding is performed with the input frame rate value and the preset aperture ratio value by substituting the input frame rate value into the determined estimation function. Estimating the motion estimation accuracy to minimize the amount of code
Outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation method. It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. When video is encoded, it occurs in the encoding under the condition of constant distortion. A motion estimation accuracy estimation method for estimating motion estimation accuracy that minimizes the amount of code,
A process of inputting an aperture ratio value;
An estimation function that uses a preset frame rate as a constant and an aperture ratio as a variable, and is a motion that minimizes the amount of code that will be generated by the encoding under the encoding conditions with a constant amount of distortion. A process of determining an estimation function used to estimate the estimation accuracy;
It occurs when encoding is performed with the input aperture ratio value and the preset frame rate value by substituting the input aperture ratio value into the determined estimation function. Estimating the motion estimation accuracy to minimize the amount of code
Outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation method. The motion estimation accuracy estimation method according to any one of claims 1 to 3,
The estimation function is described in a function used for estimation of prediction error power of inter-frame prediction when encoding is performed at a frame rate and a certain frame rate and a certain aperture ratio, and a model parameter that changes its value depending on the aperture ratio. That the function form is defined by
A characteristic motion estimation accuracy estimation method. The motion estimation accuracy estimation method according to claim 4,
As the model parameters, (1) model parameters depending on the aperture ratio derived from the sum of squares of the difference values between adjacent pixels in the reference frame, (2) an approximation error of Taylor expansion with respect to the inter-frame prediction error, and the reference frame Model parameters that depend on the aperture ratio derived from the sum of the products of the difference values between adjacent pixels, (3) the estimated noise power superimposed on the original signal, and the noise power superimposed on the original signal after passing through the interpolation filter Derived from the sum of the model parameter depending on the aperture ratio derived from the sum of the estimated value and (4) the estimated value of the error power accompanying the interpolation of the pixel value and the estimated value of the error power accompanying the quantization of the reference signal Model parameters that do not depend on the aperture ratio, and (5) a model that does not depend on the aperture ratio derived from the sum of squares of approximation errors of Taylor expansion for inter-frame prediction errors. The use of five model parameters as parameters,
A characteristic motion estimation accuracy estimation method. The motion estimation accuracy estimation method according to claim 4 or 5,
If the frame rate is represented by F, the aperture ratio is represented by κ, and the model parameters are represented by α ₁ ′, α ₂ ′, α ₃ ′, α ₄ , α ₅ ,
((Α ₁ 'F ^-1 + α ₂ ' F ^-1/2 + α ₃ 'F + α ₄ + α ₅ ) / α ₁ ') ^1/2
However, α ₁ '∝1 / κ, α ₂ ' ∝1 / κ ^1/2 , α ₃ '∝1 / κ
To use a function proportional to
A characteristic motion estimation accuracy estimation method. It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. When video is encoded, it occurs in the encoding under the condition of constant distortion. A motion estimation accuracy estimation device that estimates motion estimation accuracy that minimizes the amount of code,
Means for inputting values of frame rate and aperture ratio;
An estimation function that uses the frame rate and aperture ratio as variables, and is used to estimate the motion estimation accuracy that minimizes the amount of code that will be generated by the encoding under the condition of encoding with a constant amount of distortion. Means for determining the estimated function to be obtained;
By substituting the input frame rate and aperture ratio values for the determined estimation function, the amount of code that will be generated when encoding is performed using the input frame rate and aperture ratio values. Means for estimating the motion estimation accuracy to be minimized;
Means for outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation apparatus. It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. When video is encoded, it occurs in the encoding under the condition of constant distortion. A motion estimation accuracy estimation device that estimates motion estimation accuracy that minimizes the amount of code,
Means for inputting a frame rate value;
An estimation function that uses a preset aperture ratio as a constant and a frame rate as a variable, and is a motion that minimizes the amount of code that will be generated by the encoding under the encoding conditions with a constant amount of distortion Means for determining an estimation function used to estimate the estimation accuracy;
It occurs when encoding is performed with the input frame rate value and the preset aperture ratio value by substituting the input frame rate value into the determined estimation function. Means for estimating motion estimation accuracy to minimize the amount of code
Means for outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation apparatus. It is used in video encoding with inter-frame prediction that performs motion estimation with non-integer pixel accuracy. When video is encoded, it occurs in the encoding under the condition of constant distortion. A motion estimation accuracy estimation device that estimates motion estimation accuracy that minimizes the amount of code,
Means for inputting the value of the aperture ratio;
An estimation function that uses a preset frame rate as a constant and an aperture ratio as a variable, and is a motion that minimizes the amount of code that will be generated by the encoding under the encoding conditions with a constant amount of distortion. Means for determining an estimation function used to estimate the estimation accuracy;
It occurs when encoding is performed with the input aperture ratio value and the preset frame rate value by substituting the input aperture ratio value into the determined estimation function. Means for estimating motion estimation accuracy to minimize the amount of code
Means for outputting the estimated motion estimation accuracy,
A characteristic motion estimation accuracy estimation apparatus. The motion estimation accuracy estimation apparatus according to any one of claims 7 to 9,
The estimation function is described in a function used for estimation of prediction error power of inter-frame prediction when encoding is performed at a frame rate and a certain frame rate and a certain aperture ratio, and a model parameter that changes its value depending on the aperture ratio. That the function form is defined by
A characteristic motion estimation accuracy estimation apparatus.   A motion estimation accuracy estimation program for causing a computer to execute the motion estimation accuracy estimation method according to any one of claims 1 to 6.   A computer-readable recording medium in which a motion estimation accuracy estimation program for causing a computer to execute the motion estimation accuracy estimation method according to claim 1 is recorded.

Images