JP2007272461A - Motion estimating device, method, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate an accurate motion state from an image without a cost. <P>SOLUTION: A camera 10 photographs an image I<SB>0</SB>at a time T and an image I<SB>1</SB>at a time (T+ΔT). An operation circuit 20 generates a virtual image I'<SB>0</SB>at the time T on the basis of the image I<SB>1</SB>at the time (T+ΔT), a rotation component R, and a translation component T. The operation circuit 20 three-dimensionally moves the virtual image I'<SB>0</SB>minutely while changing the rotation component R and the translation component T and calculates an evaluation value SS showing the amount of deviation between the image I<SB>0</SB>at the time T and the virtual image I'<SB>0</SB>. In the calculation of the evaluation value SS, a weight of the amount of deviation is increased in the case of the presence of a vertical edge. The operation circuit 20 estimates the rotation component R and the translation component T, which minimize the evaluation value SS, as a momentum at the time T. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a motion estimation device, method, and program, and more particularly to a motion estimation device that estimates a motion state from a change in an image.

  In order to realize driver assistance for safe driving of automobiles and automatic driving of autonomous mobile vehicles, vehicle-mounted stereo cameras can be used for all road areas such as road plane areas, preceding cars, oncoming cars, parked vehicles, and pedestrians. A technique for detecting an obstacle is disclosed.

  In Patent Document 1, in an egomotion determination system for generating a prediction regarding egomotion (self-motion) of a transportation means moving along a road, a series of records recorded when the transportation means moves along a road. An image information receiver configured to receive image information associated with the at least two images, and configured to process the image information received by the image information receiver and generate an egomotion prediction of the vehicle And a processor including the same processor. In the technique of Patent Document 1, the road surface area represented by the image information is divided for each area, and the motion of the host vehicle is estimated by setting a weight for each area. As the area weighting, a weight α regarded as a road projection, a gradient strength weight β, and a non-road weight λ are used.

  Patent Document 2 discloses a method for recognizing an object reflected in an image obtained by photographing a peripheral area with a camera by image processing. Specifically, the technique of Patent Document 2 acquires a series of image sequences captured while moving by a single camera, identifies object candidates reflected in the image, and identifies the identified object candidates as described above. A recognition processing step of tracking in a series of image sequences and obtaining three-dimensional information about the object from change information of the object candidate image obtained by the tracking.

In Patent Document 3, using only a stereo moving image composed of a base image and a reference image, which is taken by an imaging means mounted on an automobile, all the road plane regions and road surfaces that can be traveled are used. A road plane area and an obstacle detection method using a stereo image that can detect an obstacle are disclosed. Specifically, the technique of Patent Document 3 uses a first step of dynamically estimating a projection transformation matrix for a road surface, and a plane area that can be traveled using the projection transformation matrix obtained in the first step. A second step of extracting the road surface, a third step of calculating a slope of the road surface by decomposing the projective transformation matrix obtained in the first step, and a step of calculating the road surface of the road surface calculated in the third step A fourth step of generating a virtual projection plane image when the road surface is viewed from above on the basis of the inclination; and the plane area on which the vehicle can travel and the vehicle on the virtual projection plane image generated in the fourth step A fifth step of calculating the position and direction of
Japanese translation of PCT publication No. 2003-515827 (formula (19)) JP 2001-266160 A JP-A-2005-217883

  However, in the technique of Patent Document 1, only α out of the three parameters described above uses the change in the density value of the region, but is only used for road projection determination, and the vehicle motion The state cannot be estimated specifically. The technique of Patent Document 2 calculates an optical flow from two images, but uses vehicle sensor information for the motion of the host vehicle, and does not estimate the motion state from the images. Moreover, since the technique of patent document 3 produces | generates a stereo image using two cameras, when mounting in a vehicle, there exists a problem of taking cost and taking a space further.

  The present invention has been proposed to solve the above-described problems, and an object thereof is to provide a motion estimation device, method, and program for estimating an accurate motion state from an image without cost.

  The motion estimation apparatus according to the present invention is input by the image input means for inputting the first image at the first time, the second image after a predetermined time has elapsed from the first time, and the image input means. Virtual image generation means for generating a virtual image at the first time based on a second image and a momentum parameter representing the virtual momentum at the first time, and the image for each predetermined divided region The density difference between the first image input by the input unit and the virtual image generated by the virtual image generation unit is calculated, and the weighting according to the magnitude of the density difference and the weighting according to the direction of the density difference are performed. An evaluation value calculating means for calculating an evaluation value representing a deviation amount between the first image and the virtual image, and an evaluation value calculated by the evaluation value calculating means while changing the momentum parameter. The smallest virtual momentum the indicating momentum parameter when made, and includes a motion estimation means for estimating a motion amount in the first time, the.

  The image input means inputs the first image at the first time and the second image after a predetermined time has elapsed from the first time. The first and second images may be captured in real time by the imaging unit, or may be images prepared in advance.

  The virtual image generation means generates a virtual image at the first time based on the second image and the momentum parameter representing the virtual momentum at the first time. That is, the virtual image generation unit generates a virtual image that is traced back from the second image for a predetermined time by using the momentum parameter.

  The evaluation value calculation means calculates an evaluation value for evaluating a deviation amount between the first image and the virtual image for each predetermined divided region of the image. At this time, the evaluation value is weighted according to the magnitude of the density difference between the first image and the virtual image, and weighted according to the direction of the density difference. The reason why the weighting according to the direction of the density difference is performed is that there is a relation between the direction of the density difference and the direction of movement, and the momentum is reflected in the direction of the predetermined density difference.

  The momentum estimation means changes, as the momentum at the first time, the virtual momentum indicated by the momentum parameter when the evaluation value is smallest while changing the momentum parameter, that is, when the first image and the virtual image are closest to each other. presume.

  Therefore, the motion estimation device calculates the evaluation value by performing weighting according to the magnitude of the density difference and weighting according to the direction of the density difference using the density difference between the first image and the virtual image. By estimating the virtual momentum indicated by the momentum parameter when the evaluation value is the smallest as the momentum at the first time, the exercise state can be accurately estimated from the image. The present invention is also applicable to a motion estimation method and program.

  The motion estimation apparatus, method, and program according to the present invention can estimate an accurate motion state from an image without cost.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing the configuration of the vehicle motion estimation apparatus according to the first embodiment of the present invention. The vehicle motion estimation device includes, for example, a camera 10 that is mounted on a vehicle and captures the front of the vehicle, and an arithmetic circuit 20 that performs an operation of estimating the motion state of the vehicle based on an image captured by the camera 10. .

  The camera 10 includes a CCD image sensor 11 that generates an image corresponding to imaging light from a subject, and an A / D converter 12 that converts the image generated by the CCD image sensor 11 from an analog signal to a digital signal. ing. The arithmetic circuit 20 estimates the motion state of the vehicle based on the image supplied from the A / D converter 12.

  FIG. 2 is a diagram showing a general camera coordinate system. The camera coordinate system is a coordinate system having a translation component T and a rotation component R which are momentum parameters. The translation component T (Tx, Ty, Tz) is the amount of movement of the Xc axis, Yc axis, and Zc axis of the camera coordinate system. The rotation component R (Wx, Wy, Wz) is a rotation amount (pitch, yaw, roll) around the Xc axis, Yc axis, and Zc axis of the camera coordinate system. Here, the road surface equation is defined as equation (1).

  At this time, when the normal vector N of the road surface is (a, b, c), the equation (2) is obtained.

  Here, x and y are image coordinates. Further, when the focal length of the lens of the camera 10 is f, a = A / f, b = B / f, and c = C.

  The arithmetic circuit 20 of the vehicle motion estimation device configured as described above executes the following motion estimation routine. FIG. 3 is a flowchart showing a motion estimation routine by the arithmetic circuit 20.

In step S1, the arithmetic circuit 20 inputs the image I ₀ at time T from the camera 10, inputs the image I ₁ at time (T + ΔT) after the time ΔT has elapsed, and proceeds to step S2. Here, FIG. 4 is a diagram showing an image I ₀ at time T. FIG. 5 is a diagram showing an image I ₁ at time (T + ΔT).

In step S2, the arithmetic circuit 20 sets initial values R ₀ and T ₀ for the rotation component R and the translation component T, which are momentum parameters indicating the virtual momentum at time T, and proceeds to step S3.

In step S3, the arithmetic circuit 20 generates a virtual image I ′ ₀ at time T based on the image I _{1 at} time (T + ΔT) and the initial values R ₀ and T ₀ . That is, the arithmetic circuit 20, by modifying the image I ₁ by using the initial value R _0, T _0, and generates a virtual image I _'0 predated by the time [Delta] T, the process proceeds to step S4.

FIG. 6 is a diagram showing a virtual image I ′ ₀ at time T. Comparing FIG. 4 and FIG. 6, it can be seen that even images at the same time T are slightly different. These images are compared below.

In step S4, the arithmetic circuit 20 divides the area at the lower center of the image I ₀ and the virtual image I ′ ₀ into a plurality of divided areas, and proceeds to step S5.

FIG. 7 is a diagram illustrating a state in which the division target area is divided into a plurality of divided areas. Here, in consideration of the residual calculation described later, the divided areas are displayed in the difference image between the image I ₀ and the virtual image I ′ ₀ . Here, the reason why the lower central area of the image is used as the division target area is that the change is most likely to appear in the area.

In step S5, the arithmetic circuit 20 calculates an evaluation value that represents the amount of deviation between the image I ₀ and the virtual image I ′ ₀ . Specifically, the following evaluation routine is executed.

  FIG. 8 is a flowchart showing the evaluation routine. That is, in step S5, the processing after the next step S11 is executed.

In step S11, the arithmetic circuit 20 sets initial values R ₀ and T ₀ , sets an initial value count value to 0 (Count = 0), and proceeds to step S12. Here, the count value is the number of times that the evaluation value is considered to be almost the minimum.

  In step S12, the arithmetic circuit 20 sets the division area number i = 0 and the residual integrated value SS = 0, and proceeds to step S13.

In step S13, the arithmetic circuit 20 calculates a residual Si between the image I ₀ and the virtual image I ′ ₀ in the divided area with the divided area number i. Specifically, the following equation (3) is calculated.

f is the concentration value at the coordinates of the image I ₀ (x, y) (the luminance value), g is the density value in the virtual image I _'0 of the coordinate (x, y). And the arithmetic circuit 20 calculates Formula (3), and progresses to step S14.

  In step S14, the arithmetic circuit 20 calculates Equation (4) in order to calculate the evaluation value SS by integrating the residual Si in the divided region number i.

  Expression (4) is an evaluation function for calculating an evaluation value. αi is a weighting coefficient corresponding to the intensity (magnitude) of change in density value. βi is a weighting coefficient corresponding to the direction of the density gradient.

For example, αi increases as the difference in density value between the image I ₀ and the virtual image I ′ ₀ increases, and decreases as the difference decreases.

βi takes a large value when there is a vertical edge (horizontal edge) perpendicular to the traveling direction (when the change in density value in the vertical direction is larger than the horizontal direction), and a small value when there is no vertical edge. Become. That is, βi increases the weight of the shift amount between the image I ₀ and the virtual image I ′ ₀ when a vertical edge exists in the divided area. The reason is that the vertical edge is an edge perpendicular to the vehicle traveling direction, and thus it is easy to detect the amount of deviation in the traveling direction.

  FIG. 9 is a diagram illustrating an example of a method of calculating αi. For example, the arithmetic circuit 20 may calculate αi according to the total value of the residuals at each point in the divided area, or may calculate αi using the standard deviation in the divided area. Further, the edge strength in Sobel or Canny may be used in the edge filter. Further, βi is obtained as follows, for example.

  FIG. 10 is a diagram illustrating a vertical edge on a traveling path. On the travel path, a vertical edge caused by a shadow may occur in a direction perpendicular to the travel lane. In order to detect this vertical edge, the arithmetic circuit 20 performs the following processing.

  FIG. 11 is a diagram illustrating an example of a βi calculation method. The intensity (magnitude of change) in the x direction (horizontal direction) is dx, the intensity in the y direction (vertical direction) is dy, and the angle formed by the intensity direction with respect to the x direction is θ. At this time, the arithmetic circuit 20 calculates dy / dx (= tan θ).

  Here, in the case of a vertical edge, the change in the y direction is larger than the change in the x direction. In the case of a traveling direction edge (vertical edge) parallel to the traveling direction, the opposite is true. Therefore, the value of tanθ is large for a vertical edge, and the value of tanθ is small for a traveling edge.

  FIG. 12 is a diagram illustrating the division of βi on the xy plane. The value of tan θ represents the inclination in the xy plane of FIG. Region (a) shows a vertical edge because the change in the y direction is larger than the change in the x direction. Conversely, the area (b) shows an area other than the vertical edge. Further, βi of the region (a) is set to a value that is, for example, 10 times that of βi of the region (b). Therefore, the arithmetic circuit 20 determines whether the value of tan θ belongs to the region (a) or (b), and determines βi based on the determination result.

  In step S15, the arithmetic circuit 20 determines whether or not processing has been performed on the last divided area, that is, whether or not the divided area number i is the last number. If the determination is affirmative, the process proceeds to step S17. If it is determined, the process proceeds to step S16.

  In step S16, the arithmetic circuit 20 reads data of the next divided area (i = i + 1), and returns to step S13. As a result, the arithmetic circuit 20 repeats the processing from step S13 to step S16 again to execute the processing for all the divided areas, and then proceeds to step S17.

In step S17, the arithmetic circuit 20 determines whether or not the rotation component R and the translation component T are the initial values R ₀ and T ₀ , respectively. If the determination is affirmative, the operation proceeds to step S22. Proceed to S18.

  In step S18, the arithmetic circuit 20 calculates the absolute value ΔSS of the difference between the currently calculated evaluation value SS and the previously calculated evaluation value prevSS (ΔSS = | SS−prevSS |), and proceeds to step S19.

  In step S19, the arithmetic circuit 20 determines whether or not ΔSS is equal to or less than a preset threshold value A. If the determination is affirmative, the process proceeds to step S20. If the determination is negative, the process proceeds to step S21. Here, the threshold A is a sufficiently small value. Therefore, when ΔSS is equal to or less than the threshold value A, the evaluation value obtained by the evaluation function is almost at the minimum value.

  In step S20, the arithmetic circuit 20 updates the count value so as to increase the count value by 1 (Count = Count + 1), and proceeds to step S21.

  In step S21, the arithmetic circuit 20 determines whether or not the count value is equal to or greater than the set value B. If the determination is affirmative, the processing of this routine is terminated, and the process proceeds to step S6 shown in FIG. In this case, the process proceeds to step S22. When the count value is equal to or greater than the set value B, it is considered that the evaluation value has converged to reach a substantially minimum value, and thus the process is terminated.

In step S22, the arithmetic circuit 20 gives a minute change ΔR to the rotation component R and a minute change ΔT to the translation component T (R → R + ΔR, T → T + ΔT). Then, the arithmetic circuit 20 generates a new virtual image I ′ ₀ using the new rotation component R and translation component T. That is, the virtual image I ′ ₀ is three-dimensionally moved. Further, the arithmetic circuit 20 assigns the evaluation value SS to the prevSS that indicates the previous evaluation value and is a parameter, and returns to step S12.

As described above, the arithmetic circuit 20 three-dimensionally moves the virtual image I ′ ₀ while changing the rotation component R and the translation component T, and the evaluation value indicating the shift amount between the image I ₀ and the virtual image I ′ _0. SS is calculated, and a state where the evaluation value SS is the minimum value is detected.

  On the other hand, in step S6 shown in FIG. 4, the arithmetic circuit 20 extracts the rotation component R and the translation component T when the evaluation value SS becomes the minimum value, and uses the rotation component R and the translation component T as the time T of the own vehicle. This routine is terminated by estimating the momentum at.

FIG. 13 is a diagram illustrating an example of the image I ₀ generated by the camera 10 and its divided areas. This image I ₀ includes a white line parallel to the traveling direction, a shadow perpendicular to the traveling direction, and the like.

  FIG. 14 is a diagram illustrating a state in which the divided areas are weighted using a conventional method. FIG. 15 is a diagram illustrating a state in which the vehicle motion estimation device according to the present embodiment weights divided areas. Comparing FIG. 14 and FIG. 15, it can be seen that the weights of the four divided regions where the vertical edges exist are increased.

As described above, the vehicle motion estimation apparatus according to the first embodiment is based on the image I _{1 at} time (T + ΔT) and the rotation component R and the translation component T that are virtual momentums at time T. A virtual image I ′ ₀ is generated. Then, the vehicle motion estimation device performs the three-dimensional minute movement of the virtual image I ′ ₀ while changing the rotation component R and the translation component T, and evaluates the shift amount between the image I ₀ and the virtual image I ′ _{0 at} time T. The value SS is calculated, and the rotation component R and the translation component T in a state where the evaluation value SS is the minimum value are estimated as the momentum at the time T.

  In particular, when there is a vertical edge, the vehicle motion estimation device estimates the accurate amount of motion by accurately reflecting the amount of displacement in the vehicle traveling direction by calculating the evaluation value SS by weighting the amount of displacement above. can do.

[Second Embodiment]
Next, a second embodiment of the present invention will be described. In addition, the same code | symbol is attached | subjected to the site | part same as 1st Embodiment, and the overlapping description is abbreviate | omitted. Moreover, the vehicle motion estimation apparatus according to the second embodiment is configured as shown in FIG. 1 as in the first embodiment.

  The camera coordinate system shown in FIG. 2 has a large number of parameters. For this reason, in the second embodiment, the number of parameters is reduced to reduce the calculation load.

  FIG. 16 is a diagram illustrating a camera coordinate system according to the second embodiment. In this camera coordinate system, the camera 10 is installed so that the road surface and the optical axis of the camera 10 are orthogonal to each other. That is, the normal vector N of the road surface is orthogonal to the optical axis of the camera 10. As a result, the translation component (Tx, Ty) becomes (0, 0), and the rotation component (Wz) becomes (0). The normal vector N of the road surface is (0, b, 0). That is, only Wx, Wy, and Tz can be used as the momentum parameters. Thus, the parameters can be reduced depending on the constraint conditions between the camera and the road surface.

That is, the arithmetic circuit 20 of the vehicle motion estimation device executes the motion estimation routine shown in FIG. 3 using the above three motion amount parameters. Therefore, the initial value R ₀ of the rotation component R is (0, 0, Wz), and the initial value T ₀ of the translation component T is (Tx, Ty, 0). However, in step S5 of FIG. 3, the following evaluation routine is executed.

  17 and 18 are flowcharts showing an evaluation routine according to the second embodiment. Here, only steps different from FIG. 8 will be mainly described. Hereinafter, FIG. 17 is referred to as a first routine, and FIG. 18 is referred to as a second routine.

  The first routine sets the weight of the divided region belonging to the region (a) shown in FIG. 12 to 0, the weight of the divided region belonging to the region (b) to 1, and Wx when the evaluation value becomes the minimum value, Find Wy.

Specifically, steps S31 to S42 in FIG. 17 substantially correspond to steps S11 to S22 in FIG. However, in step S31 of FIG. 17, the arithmetic circuit 20 sets Wx, Wy, and Tz as initial values R ₀ and T ₀ of the rotation component R and translation component T, respectively, and sets the count value to zero.

  In step S33, the arithmetic circuit 20 calculates the residual Si between the divided areas in which no vertical edge exists (including the traveling direction edge) according to the above-described equation (3).

In step S42, the arithmetic circuit 20 gives the minute change ΔWx to Wx of the rotation component R and gives the minute change ΔWy to Wy (Wx → Wx + ΔWx, Wy → Wy + ΔWy). Then, the arithmetic circuit 20 generates a new virtual image I ′ ₀ using the new rotation component R. Further, the arithmetic circuit 20 substitutes the evaluation value SS for the previous evaluation value, which is a parameter, prevSS.

  FIG. 19 is a diagram showing the weight of each divided region used in the first routine. Here, the weight of the divided region having a large density change is increased, and the weight of the region having a small density change is decreased.

  Next, the second routine fixes Wx and Wy obtained in the first routine, sets the weight of the divided area belonging to the area (a) shown in FIG. 12 to 1, and the weight of the divided area belonging to the area (b). Is set to 0, and Tz when the evaluation value is minimized is obtained. At this time, Tz is obtained by using the region belonging to the region (a) in FIG. 12 so that the evaluation value SS becomes the minimum value.

Specifically, steps S51 to S62 in FIG. 18 substantially correspond to steps S11 to S22 in FIG. However, in step S51 in FIG. 18, the arithmetic circuit 20 sets Wx and Wy determined in the first routine as initial values R ₀ and T ₀ of the rotation component R and translation component T, and further sets a predetermined Tz. Set the count value to 0.

  In step S53, the arithmetic circuit 20 calculates the residual Si between the divided regions where the vertical edge exists and between the divided regions where the vertical edge does not exist according to the above-described equation (3).

Further, in step S62, the arithmetic circuit 20 gives a minute change ΔTz to Tz of the translation component T (Tz → Tz + ΔTz). Then, the arithmetic circuit 20 generates a new virtual image I ′ ₀ using the new translation component T. Further, the arithmetic circuit 20 substitutes the evaluation value SS for the previous evaluation value, which is a parameter, prevSS.

  FIG. 20 is a diagram showing the weight of each divided region used in the second routine. Here, the weight of the divided area where the vertical edge exists is increased, and the weight of the divided area where the vertical edge does not exist is decreased.

  And the arithmetic circuit 20 will perform step S6 shown in FIG. 3, after complete | finishing a 1st and 2nd routine. In the second embodiment, the arithmetic circuit 20 executes the second routine after the first routine. However, the arithmetic circuit 20 may execute the second routine first and then execute the first routine.

  As described above, the vehicle motion estimation apparatus according to the second embodiment sets the camera coordinate system so that the road surface and the optical axis of the camera 10 are orthogonal to each other, so that the three momentum parameters Wx, Wy, Tz are set. Therefore, the calculation load can be greatly reduced.

  Note that the present invention is not limited to the above-described embodiment, and it is needless to say that the present invention can also be applied to a design modified within the scope of the claims.

  For example, in the above-described embodiment, βi is divided into two stages whether or not a vertical edge exists, but the present invention is not limited to this. For example, βi may increase as the edge direction approaches the vehicle traveling direction (vertical direction), and may decrease as the edge direction approaches a direction perpendicular to the vehicle traveling direction.

  In the above-described embodiment, the vehicle motion estimation device mounted on the vehicle has been described as an example. However, the present invention is not limited to the vehicle, and can naturally estimate the motion state of other moving objects. is there.

It is a block diagram which shows the structure of the vehicle motion estimation apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows a general camera coordinate system. It is a flowchart which shows the exercise | movement estimation routine by an arithmetic circuit. Is a diagram showing an image I ₀ at time T. Is a diagram illustrating an image I ₁ at time (T + ΔT). It is a diagram showing a virtual image I _'0 time T. It is a figure which shows the state which divided | segmented the division | segmentation object area | region into the several division area. It is a flowchart which shows an evaluation routine. It is a figure which shows an example of the calculation method of (alpha) i. It is a figure which shows the vertical edge which exists on a travel path. It is a figure which shows an example of the calculation method of (beta) i. It is a figure which shows the division of (beta) i on xy plane. It is a diagram illustrating an example of a generated image I ₀ and the divided regions by the camera. It is a figure which shows the state which weighted the division area using the conventional method. It is a figure which shows the state which the vehicle motion estimation apparatus weighted the division area. It is a figure which shows the camera coordinate system of 2nd Embodiment. It is a flowchart which shows the evaluation routine which concerns on 2nd Embodiment. It is a flowchart which shows an evaluation routine. It is a figure which shows the weight of each division area used by the 1st routine. It is a figure which shows the weight of each division area used by the 2nd routine.

Explanation of symbols

10 Camera 11 CCD Image Sensor 12 A / D Converter 20 Arithmetic Circuit

Claims (5)

An image input means for inputting a first image at a first time and a second image after a predetermined time has elapsed from the first time;
Virtual image generation means for generating a virtual image at the first time based on the second image input by the image input means and the momentum parameter representing the virtual momentum at the first time;
For each predetermined divided region, a density difference between the first image input by the image input unit and the virtual image generated by the virtual image generation unit is calculated, and weighting according to the magnitude of the density difference is performed. An evaluation value calculating means for calculating an evaluation value representing a deviation amount between the first image and the virtual image by performing weighting according to a direction of density difference;
Momentum estimation means for estimating the virtual momentum indicated by the momentum parameter when the evaluation value calculated by the evaluation value calculation means is smallest while changing the momentum parameter, as the momentum at the first time;
A motion estimation device comprising: The motion estimation apparatus according to claim 1, wherein the weighting according to the direction of the density difference increases the weight in the horizontal direction of the image compared to the vertical direction of the image. The motion estimation device according to claim 1, wherein the momentum parameters are a movement amount in each of the three-dimensional directions and a rotation amount around each rotation axis in the three-dimensional direction. Input a first image at a first time and a second image after a predetermined time has elapsed from the first time,
Based on the input second image and a momentum parameter representing a virtual momentum at the first time, a virtual image at the first time is generated,
By calculating a density difference between the first image and the virtual image for each predetermined divided region, and performing weighting according to the magnitude of the density difference and weighting according to the direction of the density difference, Calculating an evaluation value representing the amount of deviation between the image of 1 and the virtual image;
A motion estimation method for estimating a virtual momentum indicated by the momentum parameter when the evaluation value is smallest while changing the momentum parameter as a momentum at a first time. On the computer,
Input a first image at a first time and a second image after a predetermined time has elapsed from the first time;
Based on the input second image and a momentum parameter representing the virtual momentum at the first time, a virtual image at the first time is generated,
By calculating a density difference between the first image and the virtual image for each predetermined divided region, and performing weighting according to the magnitude of the density difference and weighting according to the direction of the density difference, An evaluation value representing the amount of deviation between the image of 1 and the virtual image is calculated;
A motion estimation program for estimating a virtual momentum indicated by the momentum parameter when the evaluation value is smallest while changing the momentum parameter as a momentum at a first time.