JP4538426B2 - Movement vector detection apparatus, movement vector detection method, and movement vector detection program - Google Patents

Description

  The present invention relates to a technique for stably and robustly estimating a movement vector of a movement target from continuous time-series images with significant luminance fluctuations.

  At present, research for estimating object motion (optical flow) from time-series images occupies an important position in the field of computer vision, and research is actively conducted every day.

  The most basic method for detecting motion is a cross-correlation method (CC method) that uses high similarity between two consecutive images, and is well known in the field of coding represented by MPEG. . Specifically, on the premise that there is no luminance change in two time-series images, the similarity between images is evaluated by a cross-correlation function or the like, and a similar region is regarded as a movement vector between images. In order to improve estimation accuracy, a method is also used in which an image is divided into a plurality of blocks and a movement vector is obtained in units of sub-blocks.

  On the other hand, in the computer vision field, a method for detecting motion using an optical flow method is known. This optical flow method is one of the methods to determine the movement of each point from a moving image. It is a method to check the spatial change and temporal change of the image brightness and determine how fast each point has moved. is there. Specifically, it is a method for estimating the motion speed of a target from a sequence of two or more images in a framework called regularization, and a variational method based on an expression defined as an objective function based on two time-series images. A nonlinear simultaneous equation is obtained according to (or Euler-Lagrange method). This operation of calculating the nonlinear simultaneous equations is repeated until the iteration error is reduced by the relaxation method. The solution converged in this way becomes a velocity vector to be moved between the two images. It should be noted that the meanings of the luminance value, intensity value, and gray value of the image are all equivalent, and the image data used in the calculation is the gray value of the image.

In the conventional optical flow method, equation (1) is derived on the assumption that the flow rate in and out of a certain region of the image is preserved, that is, the luminance fluctuation is constant, and this equation (1) is calculated. Therefore, the velocity vector was obtained.

  In Expression (1), E represents the luminance (lightness / darkness) of the two-dimensional image (x, y) at time t. v is a velocity vector. The first term on the left side represents a function of luminance change in a minute time, and the second term represents a function of the amount of luminance divergence, and represents that the luminance variation is constant by setting the right side to zero.

Next, Formula (2) can be obtained by expanding Formula (1) according to the rules of differential operation.

Furthermore, the moving displacement vector d of the fluid in the image can be defined by Equation (3) by the discrete time width Δt and the velocity vector v.

Expression (4) and Expression (5) can be easily obtained by performing integral calculation with respect to time using Expression (2) and Expression (3).

  Equation (4) is a first-order differential equation with respect to time, and equation (5), which is a general solution, can be obtained by analytically solving equation (4). Note that the equation (∀uε (t, t + Δt)) described on the right side of the equation (4) represents that the velocity u in the x-axis direction is all included in the discrete time Δt.

For this equation (5), a luminance variation equation relating to luminance is derived as shown in equation (6) through Taylor expansion. The velocity vector is estimated by performing convergence calculation using Equation (6) as a minimization problem.

However, since the solution cannot be converged by calculation using only equation (6), a certain condition is introduced into equation (6) for calculation. Here, as a hydrodynamic constraint condition, a constraint condition considering the divergence and vorticity is defined by Equation (7).

  The first term and the second term on the right side are divergence equations, and the first term ξ and the second term consider errors related to the divergence for performing the convergence calculation. The third term and the fourth term on the right side are vorticity equations, and the third term ζ and the fourth term consider errors related to vorticity for performing convergence calculation.

Equation (8) is derived from the equations (6) and (7) as the objective function. α is a regularization parameter. In the equation d = (u, v), d on the left side is a moving speed vector, and (u, v) on the right side are x-axis and y-axis speed vectors per unit time.

  Accordingly, the velocity vector (u, v) can be calculated by performing convergence calculation until the divergence error ξ and the vorticity error ζ in Equation (8) become small.

Specifically, since Equation (8) is a functional, a nonlinear simultaneous equation for three variables (Δd, ξ, ζ) can be easily obtained via the Euler-Lagrange equation by the framework of the variational method. it can. Here, when the position of the pixel of the two-dimensional image is defined by (i, j) as a discretized calculation grid, the divergence and vorticity are described by the equation (9) discretized by the finite difference method. can do.

Further, a Leclerc function is used as a nonlinear robust function used when calculating the nonlinear simultaneous equations. This is to suppress unstable calculation due to a steep luminance change. σ represents dispersion.

  Therefore, the velocity vector (u, v) is obtained by using the above-described nonlinear simultaneous equations, the equations (9), (10), and the gray value of the image at the position (i, j) of the two-dimensional image. It can be calculated by performing convergence calculation until ξ and ζ become small. In terms of calculation efficiency, the initial values of the three variables (Δd, ξ, ζ) are all set to zero, and one of these three is regarded as a variable and the remaining two are regarded as fixed values. Estimate the velocity vector (u, v).

The formulation used here is in accordance with Non-Patent Documents 1 to 3.
JL Barron, two others, "Performance of Optical Flow Techniques", IJCV, 1994, vol.12, no.1, p.43-77 Richard P. Wildes, 3 others, "Recovering Estimates of Fluid Flow from Image Sequence Data", CVIU, 2000, vol.80, p.246-266 Thomas Corpetti, 2 others, “Dense Estimation of Fluid Flows”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, March 2002, vol.24, no.3, p.365-381

  When motion is detected using the optical flow method, as described above, in general, in many cases, a precondition that the luminance variation is constant is considered. In other words, it is assumed that the motion of the image is always smooth without considering the sudden luminance change, and the motion is set by setting the first and second derivatives of velocity to approximately zero. Estimated.

  However, in the actual environment around us, local brightness fluctuations due to occlusion and shadows, steep developments such as clouds and rain observed from weather radar patterns and satellite image patterns, decline, division, fusion, Complex brightness fluctuations such as disappearance are always occurring.

  In other words, in the conventional optical flow method, an equation formulated in the field of numerical fluid dynamics called vorticity and divergence is applied as a constraint, but it has the original characteristics of fluid such as vortex development and decay. Since luminance fluctuations are not taken into account, there is a problem that the movement of clouds or the like estimated from the weather radar pattern is detected as an unnatural disturbance and the accuracy of estimating the movement is low.

  Also, fluid-like patterns such as satellite images are not clear in the edge structure of fluids compared to weather radar patterns, and brightness fluctuations between images are relatively small, so even if vortices are observed, they are constant. It is possible to obtain a certain degree of accuracy with respect to the movement of the image, but it is assumed that the luminance fluctuation does not always occur because a large luminance change occurs when the period of the convection phenomenon is extremely short such as torrential rain or thundercloud If the fluid motion is estimated, it becomes unnatural as described above, resulting in a problem that the estimation accuracy is low.

  The present invention has been made in view of the above, and it is an object of the present invention to stabilize and improve the accuracy of estimating a motion (speed) vector of a moving target from a video that causes luminance fluctuations.

According to a first aspect of the present invention, there is provided a moving vector detection device, an image input unit for inputting an image including a moving object, an image storage unit for storing the input image in time series, and reading the stored image. Applying a luminance value at a position on the image to an autoregressive model or a polynomial regression model, generating a hierarchical image having a different spatial resolution for each image and storing the hierarchical image in the hierarchical image accumulating unit; Luminance fluctuation amount between images (= (∂E (x) / ∂t) + ∇E (x) · Δt · v (x, t)) (where v is a velocity vector at time t at position x, E the luminance value of the position x, Delta] t derives a luminance variation equation F ₁ which discretized time width) has to follow the autoregressive model or the polynomial regression model, the position on the image with respect to the formula F ₁ Constraining brightness divergence and vorticity And the objective function deriving unit that stores the objective function storing means to define an expression F _2, to derive the objective function aims to minimize the sum of the formula F ₁ and the formula F ₂ used for, Reading the stored objective function, substituting the objective function for the brightness value of the two hierarchical images having a low spatial resolution among the hierarchical images for the two consecutive images, and the divergence of the objective function Speed vector temporary estimation means for temporarily estimating the speed vector of the moving object by repeatedly calculating until the error of the degree and the error of the vorticity converge, and storing the speed vector in the speed vector storage means, and the stored objective function reads said velocity vector, by substituting the luminance value of the velocity vector and the next higher spatial resolution two of the hierarchical image to the objective function, the divergence of the error and the vortex of the objective function Error is characterized by having a a velocity vector estimation means for storing the velocity vector storage means to estimate the velocity vector of the moving object by calculating repeatedly until convergence.

  In the present invention, since the velocity vector is estimated by deriving the luminance variation equation from the equation in which the luminance variation amount is an autoregressive model or a polynomial regression model, a strong vortex or the like observed when sensing the real environment with an image. Even when there is a non-linear motion and a large luminance variation, the motion of the moving object can be reliably followed, and the motion (speed) vector can be stabilized and highly accurate.

  In addition, a plurality of hierarchical images having different spatial resolutions are respectively generated for two consecutive images, and a velocity vector is temporarily calculated using two hierarchical images having a low spatial resolution among the generated hierarchical images. The velocity vector is estimated using the velocity vector and two hierarchical images having the next highest spatial resolution. As a result, the area and spatial resolution of the hierarchical image can be reduced to reduce the amount of calculation, so that the speed vector convergence calculation speed can be improved and the convergence can be ensured.

In the movement vector detection apparatus, the speed vector estimation means includes two images having the next highest spatial resolution after the speed vector stored by the speed vector estimation means and the two hierarchical images used by the speed vector estimation means. Substituting the brightness value of the hierarchical image in the objective function into the objective function to repeatedly estimate the velocity vector, and storing the velocity vector in the velocity vector storage means.

In the present invention, the velocity vector is estimated using the velocity vector estimated by the velocity vector estimating means and the luminance values of the two hierarchical images having the next highest spatial resolution after the two hierarchical images used by the velocity vector estimating means. By repeating the estimation of the vector, it is possible to estimate the velocity vector of the moving object between the images, and similarly to the above, the area and spatial resolution of the hierarchical image can be reduced to reduce the calculation amount, The speed of the speed vector convergence calculation can be improved.

  In the movement vector detection apparatus, the velocity vector temporary estimation unit and the velocity vector estimation unit use a Lorentz function as a nonlinear robust function used when calculating the objective function.

  In the present invention, by applying the Lorentz function as a nonlinear robust function, the speed vector convergence calculation speed can be increased.

The movement vector detection method according to the second aspect of the present invention includes a first step of inputting an image including a moving object by an image input unit, a second step of storing the input image in time series by an image storage unit, reading the stored image; a third step of storing the hierarchical image storage unit by the hierarchical image generating means to generate a hierarchical image having different spatial resolutions for each of the images, the luminance values of the position on the image self Applied to regression model or polynomial regression model, luminance fluctuation amount between time series images (= (∂E (x) / ∂t) + ∇E (x) · Δt · v (x, t)) (where v Is the velocity vector at time t at position x, E is the luminance value at position x, Δt is the discretized time width) and derives the luminance variation equation F ₁ assuming that it follows the autoregressive model or the polynomial regression model, in the equation _{F 1} Purpose of the divergence and vorticity of the luminance of the position on the image to define an expression F ₂ used for the constraint conditions, aimed at minimizing the sum of the formula F ₁ and the formula F ₂ A fourth step of deriving a function and storing the objective function in the objective function storage means by the objective function deriving means; and reading out the stored objective function to obtain a low spatial resolution 2 of the hierarchical images for the two consecutive images Substituting the brightness values of the layer images into the objective function, the velocity vector of the moving object is temporarily estimated by repeatedly calculating until the divergence error and the vorticity error of the objective function converge The fifth step of storing in the speed vector storage means by the speed vector temporary estimation means, and reading out the stored objective function and the speed vector, and the speed vector and the next two images having the next higher spatial resolution Substituting the luminance value of the hierarchical image to the objective function, to estimate the velocity vector of the moving object in the repeated computation that until the error of the divergence of the error and the vorticity of the objective function converges And a sixth step of storing in the speed vector storing means by the speed vector estimating means.

In the movement vector detection method, the sixth step includes the two vectors having the next highest spatial resolution after the velocity vector stored by the velocity vector estimation means and the two hierarchical images used in the sixth step. Substituting the brightness value of the hierarchical image into the objective function to repeatedly estimate the speed vector, and storing the speed vector in the speed vector storage means by the speed vector estimation means.

  In the movement vector detection method, the fifth step and the sixth step use a Lorentz function as a nonlinear robust function used when calculating the objective function.

  A moving vector detection program according to a third aspect of the present invention causes a computer to execute each step in the moving vector detection method described in the moving vector detection method.

  According to the present invention, it is possible to stabilize and improve the accuracy of estimating a motion (speed) vector of a moving object from a video that causes luminance fluctuations.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a configuration diagram illustrating a movement vector detection apparatus according to an embodiment of the present invention. As shown in FIG. 1, the movement vector detection apparatus 10 generates an image input unit 100 that inputs an image, an image storage unit 110 that stores an image as a time-series image, and a plurality of hierarchical images having different spatial resolutions for each image. A hierarchical image generating unit 120, an objective function deriving unit 130 for deriving an objective function, a velocity vector estimating unit 140 for estimating a velocity vector, and a movement vector display unit 150.

  The moving vector detection device 10 is realized by a CPU, a memory, a hard disk, and the like constituting a computer main body. In addition, operation processing of each step (S1 to S6) described below is executed by a program, and the image storage unit 110, the hierarchical image storage unit, the objective function storage unit, and the velocity vector storage unit are configured by a memory or a hard disk. The

  An operation process of the movement vector detection device 10 described above will be described.

  First, a plurality of images in which movement targets are recorded are input to the image input unit 100 (S1). This moving object indicates, for example, a fluid such as a cloud photographed by a weather radar or the like.

  Next, a plurality of images input by the image input unit 100 are stored in the image storage unit 110 in time series (S2). The time series means a sequence in which observed values of phenomena that change with time are recorded with time.

  Subsequently, the hierarchical image generation unit 120 reads two consecutive stored images from the plurality of time-series images stored by the image storage unit 110 from the image storage unit 110, and stores each of the two stored images. On the other hand, a plurality of hierarchical images having a lower (rougher) spatial resolution than the stored image are generated and stored in the hierarchical image storage unit (S3).

  This hierarchical image generation will be described with reference to FIG. FIG. 2 is a schematic diagram showing a hierarchical image in the accumulated image 200 at a certain time t and the accumulated image 210 at a time t + 1 that is continuous with the time t. The hierarchical image generation unit 120 uses the stored image 200 to generate a reduced hierarchical image 201 having a resolution lower than that of the stored image 200, and further to a resolution lower than that of the hierarchical image 201. A reduced hierarchical image 202 is generated. Similarly for the accumulated image 210, the hierarchical image generation unit 120 generates a reduced hierarchical image 211 having a lower spatial resolution than the accumulated image 210, and further has a lower spatial resolution than the hierarchical image 211. To generate a reduced hierarchical image 212. It is also possible to generate three or more hierarchical images from one stored image.

  Then, the objective function deriving unit 130 derives a luminance variation equation from an equation having the luminance variation amount as an autoregressive model or a polynomial regression model, and from the luminance variation equation, the divergence equation related to divergence, and the vorticity equation related to vorticity. An objective function related to the velocity vector, the divergence error, and the vorticity error is derived and stored in the objective function storage unit (S4). A specific method for deriving the objective function will be described later. Note that S4 described here is not limited to the operation after S3, and may be between S1 to S3 or before S1.

  Thereafter, the velocity vector estimation unit 140 reads the objective function from the objective function storage unit, substitutes a plurality of hierarchical images generated by the hierarchical image generation unit 120 for the objective function, estimates the velocity vector, and Store in the storage unit (S5).

  Specifically, the velocity vector is temporarily estimated using the velocity vector whose initial value is zero and the hierarchical image 202 and the hierarchical image 212 having the lowest spatial resolution among the plurality of hierarchical images. Thus, the first velocity vector is estimated using the temporarily estimated velocity vector and the hierarchical image 201 and the hierarchical image 211 having the next highest spatial resolution. Then, the second velocity vector is estimated using the first velocity vector and the accumulated image 200 and the accumulated image 210 which are the original images for generating the hierarchical images. This second main velocity vector becomes a velocity vector to be moved between successive images.

  Even when three or more hierarchical images are generated from one stored image, the two original hierarchical images generated in the same manner as described above in order from the two hierarchical images having the lowest spatial resolution. By repeating the same calculation until the velocity vector is estimated using the images (accumulated image 200 and accumulated image 210 in the above case), the velocity vector of the moving object between the two consecutive images is estimated. be able to. A specific velocity vector estimation method will be described later.

  Finally, the movement vector display unit 150 displays the movement vector to be moved on the screen as an arrow symbol using the velocity vector estimated by the velocity vector estimation unit 140 (S6).

  Next, a velocity vector estimation method using the optical flow method in the present embodiment will be described.

  Originally, the right side of the equation (1) described in the background art is not zero but is defined as a unit flow rate Q. In other words, in the case of the conventional estimation method, considering the development and decline as seen in the convection cell, it is unnatural assuming that the flow rate in and out of a certain area is preserved, It was not modeled when the amount of steep development and decline were different.

Here, it can be assumed that the steep development and decline of the fluid correspond to the luminance change of the moving object on the time-sequential images. In other words, the conventional estimation method is a constant luminance change model in which the right side of Equation (1) is zero, but this embodiment replaces Equation (1) with Equation (11) that applies an autoregressive model. ) Or the velocity vector is estimated by clearly expressing the luminance fluctuation between images using the equation (12) applying the polynomial model.

α is an autoregressive constant, β is a polynomial regression constant, and ε is a residual. For equation (11), a luminance variation equation regarding luminance is derived as in equation (13) through Taylor expansion. By analyzing the equation (13) as a minimization problem, the velocity vector is estimated.

Expression (13) corresponds to conventional Expression (6), and the objective function of Expression (8) is derived from the constraint condition expressions of Expression (13) and Expression (7). The velocity vector (u, v) can be calculated by performing convergence calculation until the divergence error ξ and the vorticity error ζ in Equation (8) become small. Since the specific calculation method using Expression (8) is the same as the method described in the background art, the description is omitted here. However, the convergence calculation is performed instead of the equation (10) of the Leclerc function as the nonlinear robust function used in the past, instead of the equation (14) of the Lorentz function.

  Next, the change of the image pattern in two continuous time-series images will be described. FIG. 3 is a diagram showing a time-series image of the reflection intensity over the country taken by the weather radar. The image pattern of the image 300 at a certain time t and the image pattern of the image 310 at a time t + 1 subsequent to the time t both have a fluid-like pattern, and as the time elapses from the time t to the time t + 1, the contour and The pattern is remarkably changed to an irregular shape. Note that the arrows shown in the image 310 are obtained by estimating the movement vector when changing from the image 300 to the image 310 by the movement vector detection apparatus 10 in the present embodiment.

  FIG. 4 is a comparison diagram showing a fluid movement vector when the conventional velocity vector estimation method is used (a) and when the velocity vector estimation method of the present embodiment is used (b). Originally, the fluid near the center of both images fades and rotates to the right. However, when the conventional estimation method shown in FIG. 4A is used, the movement vector of the vortex component is not detected. Therefore, it is unnatural to estimate a component that decays outward from the center of the fluid as a movement vector. On the other hand, when the estimation method of the present embodiment shown in FIG. 4B is used, the rotation vector of the right rotation due to the vortex is correctly detected, and the fluid near the center rotates outward while rotating right. The moving vector that declines is displayed more accurately.

  FIG. 5 is a diagram showing an evaluation result of the estimation accuracy of the movement vector when the conventional speed vector estimation method is used (broken line) and when the speed vector estimation method of the present embodiment is used (solid line). is there. FIG. 5A shows the measurement result with the horizontal axis as the luminance fluctuation amount and the vertical axis as the movement error of the fluid. FIG. 5B shows the vorticity and divergence as the horizontal axis. The measurement results with the vertical axis representing the movement error of the fluid are shown. FIG. 5 shows a case where a conventional velocity vector estimation method is used and a case where the velocity vector estimation method of the present embodiment is used, using time-sequential satellite images observed by a weather radar. The movement vector with respect to the luminance fluctuation amount or vorticity and divergence is measured, and the difference between the measurement result and the actual movement vector obtained from the artificial satellite image is displayed as a movement error. From FIG. 5 (a) and FIG. 5 (b), it can be recognized that the motion error in the present embodiment is small compared to the conventional case. Therefore, based on the measurement result, the estimation accuracy in the velocity vector estimation method in the present embodiment is higher than that in the conventional method, and the moving vector detection device 10 in the present embodiment is effective. It is shown.

  FIG. 6 is a diagram showing a movement vector of a blood flow flowing inside the blood vessel when this embodiment is used. According to the movement vector detection device 10 according to the present embodiment, when a small thrombus is present inside a blood vessel, it is possible to detect that blood flow disturbance or vorticity is generated around the thrombus. .

  According to the present embodiment, the velocity vector is estimated by deriving the luminance variation equation from the equation in which the luminance variation amount is an autoregressive model or a polynomial regression model, so that a strong vortex or the like observed when sensing the real environment with video Even when there is a non-linear motion and a large luminance variation, the motion of the moving object can be reliably followed, and the motion (speed) vector can be stabilized and highly accurate.

  As a result, for example, by visualizing the flow of blood flow as a movement vector and automatically analyzing singular points such as vorticity and divergence, it is possible to easily grasp the irregular flow of blood flow due to thrombus, etc. It is possible to prevent oversight of the lesion at the time of diagnosis. In the medical field, visualization of blood flow using ultrasound images has been performed, but it is difficult to instantly determine the irregular blood flow around the thrombus from ultrasound images. However, even a doctor who has relatively little practical experience can easily grasp the lesion. In addition, it can be applied to the fields of fluid dynamics, medicine, and aerodynamics that require motion analysis through flow visualization by flowing particles in fluid experiments.

  In addition, a plurality of hierarchical images having different spatial resolutions are respectively generated for two consecutive images, and a velocity vector is temporarily calculated using two hierarchical images having a low spatial resolution among the generated hierarchical images. Then, the velocity vector is estimated using a hierarchical image having a higher spatial resolution. By repeating this estimation of the velocity vector until the two accumulated images that are the original images from which the hierarchical images are generated are used, the velocity vector to be moved between the two consecutive images is estimated. As a result, the area and spatial resolution of the hierarchical image can be reduced to reduce the amount of calculation, so that the speed vector convergence calculation speed can be improved and the convergence can be ensured. For example, the amount of calculation when calculating the nonlinear simultaneous equations is almost proportional to the area of the image. Therefore, when one side of the hierarchical image is half of the accumulated image, the calculation is about 4 (2 × 2) times faster. It becomes possible to do.

  Furthermore, by using the Lorentz function as the nonlinear robust function used when calculating the objective function, the speed vector convergence calculation speed can be increased.

It is the block diagram which showed the movement vector detection apparatus which concerns on embodiment of this invention. It is the schematic diagram which showed the hierarchical image in the accumulation image in a certain time t, and the accumulation image in the time t + 1 following the time t. It is the figure which showed the time-sequential image of the reflection intensity of the domestic sky image | photographed with the weather radar. It is the comparison figure which showed the movement vector of the fluid at the time of using the estimation method of the conventional velocity vector, and the estimation method of the velocity vector of this Embodiment. It is the figure which showed the evaluation result of the estimation accuracy of the movement vector at the time of using the estimation method of the conventional velocity vector, and the velocity vector estimation method of this Embodiment. It is the figure which showed the movement vector of the flow of the blood flow which flows into the blood vessel at the time of using this Embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Moving vector detection apparatus 100 ... Image input part 110 ... Image storage part 120 ... Hierarchical image generation part 130 ... Objective function derivation part 140 ... Speed vector estimation part 150 ... Display vector estimation part 200,210 ... Accumulated image 201,202, 211, 212 ... Hierarchical image 300, 310 ... Image S1-S6 ... Step

Claims (7)

An image input means for inputting an image including a moving object;
Image storage means for storing the input image in time series;
A hierarchical image generation unit that reads out the stored image, generates a hierarchical image having a different spatial resolution for each image, and stores the hierarchical image in the hierarchical image storage unit;
The luminance value at the position on the image is applied to the autoregressive model or the polynomial regression model, and the amount of luminance fluctuation between time series images (= (∂E (x) / ∂t) + ∇E (x) · Δt · v ( x, t)) (where v is the velocity vector at time t at position x, E is the luminance value at position x, Δt is the discretized time width) and the luminance that the autoregressive model or the polynomial regression model follows deriving a fluctuation equation F _1, the divergence and vorticity of the luminance of the position on the image to define an expression F ₂ used for the constraint condition with respect to the expression F _1, the formula F ₁ in the formula F ₂ An objective function deriving means for deriving an objective function for minimizing the sum of the objective function and storing the objective function in the objective function storing means;
Reading the stored objective function, substituting the objective function for the brightness value of the two hierarchical images having a low spatial resolution among the hierarchical images for the two consecutive images, and the divergence of the objective function Speed vector temporary estimation means for temporarily estimating the speed vector of the moving object by repeatedly calculating until the error of the degree and the error of the vorticity converge, and storing them in the speed vector storage means,
Reading the stored objective function and the velocity vector, substituting the velocity vector and the luminance values of the next two hierarchical images with the highest spatial resolution into the objective function, and the divergence of the objective function Speed vector estimation means for estimating the speed vector of the moving object by repeatedly calculating until the error of the vorticity and the error of the vorticity converge, and storing the speed vector in the speed vector storage means,
A moving vector detection apparatus comprising: The velocity vector estimation means includes the velocity vector stored in the velocity vector estimation means and the luminance values of the two hierarchical images having the next highest spatial resolution after the two hierarchical images used in the velocity vector estimation means. 2. The movement vector detecting apparatus according to claim 1, wherein the speed vector is repeatedly estimated by substituting and into the objective function, and the speed vector is stored in the speed vector storage means.   The movement vector detection apparatus according to claim 1, wherein the temporary velocity vector estimation unit and the velocity vector estimation unit use a Lorentz function as a nonlinear robust function used when calculating the objective function. A first step of inputting an image including a moving object by an image input means;
A second step of storing the input image in time series by the image storage means;
A third step of reading the stored image, generating a hierarchical image having a different spatial resolution for each image, and storing it in the hierarchical image storage means by the hierarchical image generation means;
The luminance value at the position on the image is applied to the autoregressive model or the polynomial regression model, and the amount of luminance fluctuation between time series images (= (∂E (x) / ∂t) + ∇E (x) · Δt · v ( x, t)) (where v is the velocity vector at time t at position x, E is the luminance value at position x, Δt is the discretized time width) and the luminance that the autoregressive model or the polynomial regression model follows deriving a fluctuation equation F _1, the divergence and vorticity of the luminance of the position on the image to define an expression F ₂ used for the constraint condition with respect to the expression F _1, the formula F ₁ in the formula F ₂ A fourth step of deriving an objective function for the purpose of minimizing the sum of and the objective function deriving means and storing the objective function in the objective function storage means;
Reading the stored objective function, substituting the objective function for the brightness value of the two hierarchical images having a low spatial resolution among the hierarchical images for the two consecutive images, and the divergence of the objective function A fifth step of temporarily estimating the velocity vector of the moving object by repeatedly calculating until the error of degree and the error of vorticity converge, and storing the velocity vector in the velocity vector storage means by the velocity vector temporary estimation means;
Reading the stored objective function and the velocity vector, substituting the velocity vector and the luminance values of the next two hierarchical images with the highest spatial resolution into the objective function, and the divergence of the objective function A sixth step of estimating the velocity vector of the moving object by repeatedly calculating until the error of vorticity and the error of the vorticity converge and storing the velocity vector in the velocity vector storage means by the velocity vector estimation means;
A moving vector detection method characterized by comprising: In the sixth step, the velocity vector stored by the velocity vector estimation means and the luminance values of the two layer images having the next highest spatial resolution after the two layer images used in the sixth step are used. 5. The movement vector detection method according to claim 4, wherein the velocity vector is repeatedly estimated by substituting in the objective function, and the velocity vector is stored in the velocity vector storage unit by the velocity vector estimation unit. .   6. The movement vector detection method according to claim 4, wherein the fifth step and the sixth step use a Lorentz function as a nonlinear robust function used when calculating the objective function.   A motion vector detection program that causes a computer to execute each step in the motion vector detection method according to any one of claims 4 to 6.