JPH0981217A - Method for controlling motion of mobile object by image - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a mobile object motion controlling method by image processing which can continuously controlling the motion of a mobile object even when the partial acquisition of image features is failed. SOLUTION: A target track for an object in the three-dimensional space of a camera is set (step 101). An image obtained at the time of moving the camera along the target track is found, image feature points are extracted from the image, a reference image feature group vector is found from the image feature points, and a parameter matrix is found from the feature group vector (steps 102 to 105). Then the reference image feature group vector and the parameter matrix are inputted (step 106). Time is set up to '0' (step 107). An image feature vector at control time is found, correspondence between the reference image feature group vector and control time image feature vector is found, a feedback matrix at control time is found, a motion controlling vector is found, and operation for controlling the motion of the camera is repeated up to the end time T in each time ΔT in accordance with these vectors (steps 108 to 113).

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention fixes an image pickup device on a movable object and uses an image taken by the image pickup device to follow a target trajectory preset for an object taken by the image pickup device. The present invention relates to a method for controlling a movable object to move.

The present invention is used for moving a fixed object along a preset target trajectory, such as an end effector of a moving body such as an automobile or a robot manipulator. For example, it is said that a car runs automatically along a road and avoids obstacles, and the robot manipulator is moved along a target trajectory set for the target object. It is possible to do such things.

[0003]

2. Description of the Related Art Japanese Patent Application No. 6-34806 discloses a method for controlling a movable object to move along a target trajectory set with respect to a target object using an image captured by an image pickup device fixed to the movable object. "Movement control method of movable object by image" (Endo, Takekawa).

This method will be described as a method of controlling the motion of a camera by using a video camera (hereinafter abbreviated as a camera) as an imaging device and fixing the camera to a movable object.

A flow chart of this method is shown in FIG.
This method is divided into teaching processing and control processing. The first process is a teaching process, which is a process of converting a target trajectory set for a target object into parameters for camera motion control. The subsequent process is a control process, and the process for actually controlling the camera motion is performed based on the camera motion control parameter obtained in the teaching process.

First, the teaching process will be described. First,
What kind of movement the camera makes with respect to the target object in the three-dimensional space is set as the target trajectory (step 20).
1). Next, the camera is moved along the target trajectory set in step 201, and the time j · ΔT (j =
0, ..., N), N + 1 images taken by the camera are obtained (step 202). Next, image features are extracted from each image obtained in step 202 by image processing or the like (step 203). Next, from the image features obtained in step 203, the reference image feature vector

[0007]

[Outside 1] Is calculated (step 204). Finally, step 204
Reference image feature vector obtained in

[0008]

[Outside 2] And weight matrix Q _j (j = 1, ..., N), H _j = (j =
0, ..., N−1) to the feedback matrix R _j , S
_j (j = 1, ..., N) is calculated (step 20)
5).

Next, the control process will be described. First,
Reference image feature vector obtained by teaching process

[0010]

[Outside 3] And feedback matrix R _j , S _j (j = 1, ...,
N) is read and stored (step 206). Next, the time is reset to 0 (step 207). Next, image features are extracted from the image captured by the camera by image processing, and the image feature vector is extracted from the image features.

[0011]

[Outside 4] Is calculated (step 208). Next, in step 208

[0012]

[Outside 5] The control law

[0013]

[Equation 1] Substituting into, the motion control vector of the camera

[0014]

[Outside 6] Is calculated (step 209). Next, the motion control vector of the camera obtained in step 209.

[0015]

[Outside 7] The camera is moved according to (step 210).
Next, the time is advanced by ΔT (step 211). Next, it is determined whether or not the time is T (step 212). If the time is T, the process ends. If it is not T, the process returns to step 208 to continue the process.

In order to control the movement of the camera, the target trajectory of the movement of the camera is expressed by the time-series reference image feature vector extracted from the image determined by the positional relationship between the object and the camera by the above processing. , It becomes possible to move the camera along the trajectory set for the target object. As a result, during motion control, even if the position and orientation of the target object with respect to the camera are different, and even if the position and orientation of the movable object with the camera attached are different in the initial state, the trajectory is set to the target object. The camera can be controlled to move.

[0017]

However, the above-mentioned conventional method causes the image processing to fail due to noise included in the image as shown in FIG.
Due to the fact that the feature points are not visible and do not appear in the image due to occlusion by another object as shown in (3) or due to occlusion by itself as shown in FIG. If the extraction of the image feature points as shown in FIG. 5 (1) fails in step 208 of the control process shown in FIG. 5, there is a problem that the subsequent processes cannot be continued.

The object of the present invention is to solve the above-mentioned drawbacks and to provide a method for controlling the motion of a movable object by means of an image, in which the motion control of the movable object can be continued even if acquisition of some image features fails. That is.

[0019]

In order to achieve the above object, a motion control method for moving a moving object by an image according to the present invention is to move the moving object along a target trajectory preset for the target object, A first process of obtaining a reference image feature group and a parameter matrix from a time-series image captured by the image capturing device at that time, and a second process of extracting a control image feature group from the image captured by the image capturing device during motion control Processing of
On the basis of the third process for obtaining the correspondence between the reference image feature group obtained by the first process and the control-time image feature group extracted by the second process, and the correspondence obtained by the third process. A fourth step of obtaining a correspondence matrix and a control time feedback matrix based on the correspondence matrix, the reference image feature group and the parameter matrix obtained in the first process, and the control time image feature group obtained in the second process By substituting the processing, the reference image feature group obtained in the first process, the control-time image feature group obtained in the second process, and the control-time feedback matrix obtained in the fourth process into the control law It has a fifth process of obtaining a motion control vector and controlling the motion of the movable object.

The first processing is based on the shape information of the target object, various parameters of the image pickup apparatus, and the arrangement information of the target object and the image pickup apparatus, and is a time-series image pickup by the image pickup apparatus when the movable object moves on the target trajectory. The image is estimated and obtained by calculation, and the reference image feature group is obtained from the images in time series.

The first processing is to actually move the movable object along the target trajectory, and obtain the reference image feature group from the actual time-series images captured by the image pickup apparatus.

In the first process, the relational expression between the image feature group and the motion control vector obtained from the image from the image pickup device attached to the movable object and the degree of coincidence between the reference image feature group and the control-time image feature group are determined. The parameter matrix is obtained based on the expressed evaluation function.

The first to fifth processes use the coordinate values of the image feature points obtained by projecting a plurality of feature points on the target object on the image as the reference image feature group and the control-time reference image feature group. is there.

[0024]

BEST MODE FOR CARRYING OUT THE INVENTION First, the first processing will be described. This process is a process of generating a parameter matrix used for control before controlling the motion of the movable object.

It is assumed that the movement of the movable object with respect to the target object is given in advance as a target trajectory in a three-dimensional space. This trajectory includes not only the position of the movable object, but also the posture of the movable object indicating in which direction the movable object faces. For example, the trajectory of the movable object is expressed by the following formula.

[0026]

[Equation 2] This method assumes a discrete-time system. The start time of the movement along the target trajectory is set to 0, the end time is set to T, and the sampling interval is set to ΔT. However, T = N · ΔT (N is a positive integer).

When the movable object is moved along the target trajectory, the image pickup device fixed to the movable object moves at each time j.
・ Take one image for each ΔT (j = 0, ..., N). As a result, N + 1 images are obtained at the end of the exercise.

Image characteristics are obtained from each of the obtained images by image processing or the like. Examples of image features are points, edges,
There are regions, etc., and which image feature in the image to use is determined in advance. Image feature vector consisting of image features

[0029]

[Outside 8] It is represented by. For example, when a point is used as the image feature, the image feature vector is defined by the coordinates (X, Y) of the point on the image.

[0030]

[Outside 9] It can be expressed as. Here, T on the right shoulder represents transposition.
Generally, a plurality of image features are obtained from one image. Therefore, if there are m image features in one image,

[0031]

[Outside 10] Express.

Further, in order to collectively handle a plurality of image features obtained from one image, the following image feature vector

[0033]

[Outside 11] Is defined.

[0034]

(Equation 3) By obtaining the image feature group vector from each image at time 0, ΔT, 2 · ΔT, ..., N · ΔT, a time-series image feature group vector

[0035]

[Outside 12] Is found. Since this image feature vector is used for reference when controlling the movable object, it is called a reference image feature vector group,

[0036]

[Outside 13] Express.

Motion control vector for the motion of a movable object

[0038]

[Outside 14] It is represented by. The motion control vector is, for example, as follows.

[0039]

(Equation 4) Here, v _x , v _y and v _z are translational velocity components with respect to coordinate axes x, y and z of a coordinate system centered on the movable object,

[0040]

[Outside 15] Are rotational angular velocity components around the x, y, and z axes, respectively.

Image feature group vector

[0042]

[Outside 16] And moving object motion control vector

[0043]

[Outside 17] The relationship with is expressed by the following equation.

[0044]

(Equation 5) By the way, the time j when the exercise is actually controlled
An image feature group vector obtained from an image captured by the imaging device at ΔT is referred to as a control-time image feature group vector,

[0045]

[Outside 18] If we express that, in order for the movable object to move along the target trajectory preset for the target object,
Control-time image feature group vector obtained after control time j

[0046]

[Outside 19] Is a reference image feature group vector obtained from an image captured after the time j · ΔT during which the movable object is moving on the previously obtained target trajectory.

[0047]

[Outside 20] Must match. Therefore, these image feature group vectors

[0048]

[Outside 21] Motion control vector so that

[0049]

[Outside 22] And its motion control vector

[0050]

[Outside 23] It is considered that if the movement of the movable object is controlled in accordance with the above, the movable object is controlled to move along the target trajectory set in advance for the target object. However, since there is a limit to the motion ability such as the speed of the movable object, it is necessary to suppress the magnitude of the motion control vector so as not to become excessive. Considering these points, this control is performed at time j.
The motion control vector that minimizes the evaluation function σ _j of

[0051]

[Outside 24] Results in the problem of seeking.

[0052]

(Equation 6) Here, Q _i and H _i−1 are positive-value symmetric matrices and represent weights that determine the characteristics of control. The first term in the above equation is the time i
.Image feature group vector during control in ΔT

[0053]

[Outside 25] And the target reference image feature group vector

[0054]

[Outside 26] And the second term represents the kinetic energy of the movable object. By minimizing this evaluation function, the control-time image feature group vector after the first term to time j

[0055]

[Outside 27] Target image feature group vector

[0056]

[Outside 28] It is expected that the motion control vector will be close to

[0057]

[Outside 29] Is restricted so that it does not exceed the movement ability such as the maximum speed of the movable object.

From the above preparations, the motion control vector that minimizes the evaluation function given by equation (5)

[0059]

[Outside 30] The matrix required to find is obtained as follows (reference “Modern Control Theory”, pp. 405-411, Julius T.To.
u, Kahei Nakamura, Masami Ito, Tsuyoshi Matsuo co-translation, Corona Publishing Co.).

First, the matrices Q _j , H _j (j = 0, 1, ...
., N) are determined appropriately from the conditions required for control. Further, E ₀ is a zero matrix in which all the elements are 0, and D ₀ = Q
_{Let N} , F ₀ = −2Q _N.

Next, according to equations (6) and (7), R
₁ and S ₁ are calculated. That is, equation (8). It becomes (9).

[0062]

(Equation 7) Next, according to equations (10), (11), and (12), D
Calculate ₁ , E ₁ , F ₁ . That is, equations (13) and (1
4) and (15).

[0063]

(Equation 8) Similarly, equations (6) and (7) and equations (10) and (1
By repeatedly using (1) and (12), R ₂ , S
₂ , D ₂ , E ₂ , F ₂ ..., _RN , _SN , _DN , E
_{Find N} and F _N.

D _j , F _j obtained as described above
(J = 1, ..., N) is called a parameter matrix. Save these matrices for use in subsequent processing.

Next, the second processing will be described. Subsequent processing is processing performed during execution of control of the movement of the movable object.

Before performing the second process, the parameter matrix D _j , F _j (j = 1, ..., N) obtained in the first process and the reference image feature group vector

[0067]

[Outside 31] Is read and stored in advance.

The control-time image feature is extracted from the image captured by the image pickup device at the control time j · ΔT by image processing or the like.

Next, the third processing will be described.

Which of the reference image features obtained in the first process the control-time image feature extracted in the second process corresponds to is determined.

For example, as shown in FIG. 3B, the control-time image feature vector acquired at time j

[0072]

[Outside 32] Is the reference image feature vector set in the first process

[0073]

[Outside 33] And so on. Since there may be image features that have failed to be extracted during control, there are not necessarily control-time image features corresponding to all reference image features.

Next, the fourth processing will be described.

First, let m be the number of reference image feature points.
Next, the reference image feature vector

[0076]

[Outside 34] Let p (i) be the number of elements of. Also,

[0077]

[Outside 35] And

Similarly, the number of control-time image feature points at time j is

[0079]

[Outside 36] , Control image feature vector

[0080]

[Outside 37] The number of elements in

[0081]

[Outside 38] And

Next, the matrix

[0083]

[Outside 39] Ask for. First,

[0084]

[Outside 40] Is set as a q (m) × q (m) matrix in which all elements are 0.

Next, the control-time image feature vector

[0086]

[Outside 41] The reference image feature vector corresponding to

[0087]

[Outside 42] When

[0088]

[Outside 43] of

[0089]

[Outside 44] Set the element to 1.

As an example, there are six reference image feature vectors as shown in FIG. 3 (1), and four of them are control-time image feature vectors as shown in FIG. 3 (2).

[0091]

[Outside 45] I was able to ask. And each is the reference image feature vector

[0092]

[Outside 46] It corresponds to.

All the vectors have two elements. Then

[0094]

[Outside 47] Is as follows.

[0095]

[Equation 9] This matrix

[0096]

[Outside 48] The meaning of is explained. This matrix

[0097]

[Outside 49] Represents the correspondence between the control-time image feature vector and the reference image feature vector. For example, let a 2 × 2 identity matrix be I
_{If the 2} and 2 × 2 zero matrix is O ₂ , then

[0098]

[Outside 50] Is represented as follows.

[0099]

(Equation 10) The rows of this matrix are the reference image feature vector from the top.

[0100]

[Outside 51] , The columns are from left to right

[0101]

[Outside 52] Is represented. That is, I _{2 in the} 1st row and 4th column is the reference image feature vector

[0102]

[Outside 53] Is the reference image feature vector during control

[0103]

[Outside 54] I _{2 in} 3 rows and 1 column is

[0104]

[Outside 55] But

[0105]

[Outside 56] It corresponds to. Also, the elements in the 2nd row are all O _2, but this is

[0106]

[Outside 57] It means that there is no control-time image feature vector corresponding to.

This

[0108]

[Outside 58] Using

[0109]

[Outside 59] Ask for.

[0110]

[Equation 11] Next, the control time feedback matrix is given by

[0111]

[Outside 60] Ask for. here,

[0112]

[Outside 61] And

[0113]

(Equation 12) In addition, the control image feature group vector

[0114]

[Outside 62] , Reference image feature group vector

[0115]

[Outside 63] Ask for.

[0116]

(Equation 13) Finally, the fifth process will be described.

The motion control vector of the movable object according to the following equation

[0118]

[Outside 64] Ask for.

[0119]

[Equation 14] Motion control vector obtained as above

[0120]

[Outside 65] To control the movement of the movable object.

As a result, even when only a part of the set image features can be extracted, the control-time image feature group vector

[0122]

[Outside 66] To obtain the motion control vector

[0123]

[Outside 67] Then, it becomes possible to continue the motion control of the movable object.

[0124]

Next, embodiments of the present invention will be described with reference to the drawings.

First, the setting of this embodiment will be described.

In the following description, a video camera (hereinafter simply referred to as "camera") is used as the image pickup device. Further, since the imaging device is fixed to the movable object, the following description will be given assuming that the motion of the movable object and the motion of the imaging device are the same, and the motion of the camera is controlled.

Further, a case will be described as an example in which the feature points of the target object photographed by the camera are used as the image features, and the points reflected in the image from the camera are used. The points shown in this image are called image feature points. As the feature points set on the target object, the vertices of the target object, protrusions or holes on the target object, drawn points, and the like can be used.

Four or more characteristic points of n points which are not on the same straight line on the plane where the target object is photographed by the camera

[0129]

[Outside 68] Set.

Next, the camera model is set as shown in FIG. As shown in FIG. 2, the z axis is set along the optical axis of the camera, and the focal length of the camera is l. Next, the feature points on the target object captured by the camera

[0131]

[Outside 69] Image feature points in the image

[0132]

[Outside 70] And the coordinates in the image plane coordinate system are expressed as (Xi, Yi).

At this time, the image feature group vector is

[0134]

[Outside 71] , The motion control vector of the camera

[0135]

[Outside 72] And

V _x , V _y and V _z are translational velocities along the x, y and z coordinate axes of the camera,

[0137]

[Outside 73] Is the rotational angular velocity about each of the x, y, and z coordinate axes.

Then, the matrix A _j of the equation (4) is expressed by the following equation.

[0139]

(Equation 15) Further, the matrix G _j of Expression (4) is expressed by an approximate expression such as the following expression.

[0140]

(Equation 16) FIG. 1 is a flow chart showing the processing of this embodiment.

The processing of this embodiment is divided into teaching processing (FIG. 1 (1)) which is preparation for performing control and control processing (FIG. 1 (2)) for performing actual control.

First, the teaching process will be described.

First, a target trajectory in a three-dimensional space that represents how the camera moves with respect to the target object is set (step 101).

This is a method of generating a trajectory by calculation based on the shape information of the target object and the arrangement information indicating the geometrical relationship between the target object and the camera, or actually moving the movable object in a three-dimensional space. There is a method of setting a trajectory by exercising and measuring the trajectory of the movement. In addition, combining these two, a temporary target trajectory is first generated by calculation based on the shape information of the target object and the arrangement information indicating the geometric relationship between the target object and the camera. In addition, there is a method of setting a target trajectory by correcting it while moving the camera along the temporary target trajectory and measuring the trajectory of the corrected motion.

Next, a time series image taken by the camera when the camera is actually moved along the set target trajectory is obtained (step 102).

Next, image feature points are extracted from each of the time-series images by image processing (step 103).

Next, the time-series reference image feature group vector is calculated from the coordinate values of the image feature points.

[0148]

[Outside 74] Is calculated (step 104).

Further, the above steps 102 to 104 do not use the actual image, but the image which may be obtained at each time from the shape information of the target object, the camera parameters such as the focal length, and the target trajectory for the camera to move. It is also possible to calculate and estimate the coordinates of the feature points, and then obtain the time-series reference image feature vector.

Next,

[0151]

[Outside 75] Q _j (j = 1, ..., N) given in advance,
H _j (j = 0, ..., N−1) and G _j (j = 0, ..., N) given by equation (26) are used according to equations (6) to (11). Parameter matrix D _j , F _j (j =
1, ..., N) are obtained (step 105).

The matrices Q _j and H _{j in the} equation (5) are determined appropriately from the conditions required for motion control.

For example,

[0154]

[Equation 17] Etc. However, I is an identity matrix.

When calculating G _j (j = 0, ..., N),
The value of the z coordinate, which is the depth value of the feature point in the camera center coordinate system, is required. Since the target trajectory of the camera in the three-dimensional space for the target object is given in advance,
It can be determined based on the geometric model of the target object and the target trajectory that the camera moves. Further, it is possible to calculate the z-coordinate value of the depth value of each feature point from the image feature points in which the minimum of 6 feature points set in the target object or the minimum of 4 feature points on the same plane are reflected in the image. Yes,
An example of this is the reference “A new technique for fully
autonomous 3Drobotics hand / eye caribration ”, R.
T. Tsai and RKLenz, IEEE Transactions of Robotics
and Automation, Vol. 5, No. 3, pp.345-358.

Next, the control processing will be described.

First, it is obtained by the teaching process.

[0158]

[Outside 76] Is read and stored (step 106).

Next, j = 0, that is, the time is set to 0 (step 107).

Next, an image is obtained from the camera at time j · ΔT, and image feature points are extracted from the image by image processing or the like (step 108). Then, from the coordinate value of the image feature point, the image feature vector during control

[0161]

[Outside 77] Ask for. However,

[0162]

[Outside 78] Is the number of acquired control-time image feature vectors.

Next, the control-time image feature vector

[0164]

[Outside 79] Which reference image feature vector

[0165]

[Outside 80] (Step 109).

As a method of obtaining this correspondence, the reference image feature vector

[0167]

[Outside 81] There is a method in which an image around the image feature point P _i corresponding to is stored as a reference image at the time of the teaching process, and at the time of control, a region matching the reference image is found by matching from the camera image and the correspondence is obtained.

Next, according to the correspondence obtained in step 109, using equations (20) and (21)

[0169]

[Outer 82] Is calculated (step 110).

Next, the control-time image feature group vector

[0171]

[Outside 83] Substituting into equation (24), the camera motion control vector

[0172]

[Outside 84] The motion control vector

[0173]

[Outside 85] The camera is controlled according to (step 111).

Next, the time is advanced by ΔT (step 11
2), it is determined whether or not the time reaches the end time T (step 113), and if the time reaches the end time T, the end,
Otherwise, return to step 108.

[0175]

As described above, according to the present invention, a trajectory of a movable object is obtained in a time-series image feature group obtained from an image captured by an imaging device when the movable object moves along the trajectory. In the method of controlling the motion of the movable object by the image feature group obtained from the image captured by the imaging device at the time of control, it is possible to obtain some image features when obtaining the image feature from the image at the time of control. Even if there is not, it is possible to control the motion of the possible object using only the remaining image features that can be obtained.

As a result, even when some image features are not obtained due to noise contained in the image captured by the image pickup apparatus, the target object is hidden by another object under control and some image features are hidden. Even if the image disappears, or even if some of the image features disappear on the back side of the target object depending on the shooting direction of the imaging device, the motion control of the movable object can be continued. It will be possible.

[Brief description of drawings]

FIG. 1 is a flowchart showing a method of controlling motion of a movable object by an image according to an embodiment of the present invention, FIG. 1 (1) shows a teaching process, and FIG. 1 (2) shows a control process.

FIG. 2 is a diagram showing a camera model.

FIG. 3 is a diagram showing an example of correspondence between a reference image feature (FIG. 3 (1)) and a control image feature (FIG. 3 (2)).

FIG. 4 is a flowchart showing a conventional method.
(1) shows a teaching process, and FIG. 4 (2) shows a control process.

FIG. 5 shows a case where image feature points are normally extracted (FIG. 5).
(1)) and when extraction of image feature points fails (FIG. 5).
It is a figure which shows (2), (3), (4).

[Explanation of symbols]

 101-113, 211-222 steps

Continuation of front page (51) Int.Cl. ⁶ Identification code Office reference number FI Technical display location H04N 7/18 G06F 15/70 455Z

Claims (5)

[Claims] 1. Image information of a target object obtained from an image pickup device attached to a movable object, and a reference image determined based on a target trajectory of the position and orientation of the movable object with respect to the target object given in advance. Is a motion control method of a movable object by an image, in which the movable object is moved along the target trajectory, wherein the movable object is moved along the target trajectory previously set for the target object. A first process of deriving a reference image feature group and a parameter matrix from a time-series image captured by the image capturing device at that time, and a control-time image feature group from an image captured by the image capturing device during motion control. And a third process for determining the correspondence between the reference image feature group obtained in the first process and the control-time image feature group extracted in the second process. And a correspondence matrix is obtained based on the correspondence obtained in the third processing, and the correspondence matrix, the reference image feature group obtained in the first processing, the parameter matrix, and the second processing are obtained. Fourth processing for obtaining a control-time feedback matrix based on the control-time image feature group, the reference image feature group obtained in the first process, and the control-time image feature group obtained in the second process Controlling the motion of a movable object by an image, which has a fifth process of calculating a motion control vector by substituting the control-time feedback matrix obtained in the fourth process into a control law, and controlling the motion of the movable object. Method. 2. The first process, when the movable object moves along a target trajectory based on the shape information of the target object, various parameters of the imaging device, and arrangement information of the target object and the imaging device. 2. The method for controlling the motion of a movable object by an image according to claim 1, wherein the time-series image captured by the imaging device is obtained by calculation, and the reference image feature group is obtained from the time-series image. 3. The first processing is to actually move the movable object along a target trajectory, and obtain the reference image feature group from an actual time-series image captured by the imaging device. Item 2. A method for controlling the movement of a movable object by the image according to item 1. 4. A relational expression between an image feature group obtained from an image from the image pickup device attached to the movable object and the motion control vector, the reference image feature group, and the control time. 4. The method of controlling the motion of a movable object by an image according to claim 1, wherein the parameter matrix is obtained based on an evaluation function expressing the degree of coincidence with the image feature group. 5. The first to fifth processes are performed for image feature points obtained by projecting a plurality of feature points on the target object on the image as the reference image feature group and the control-time reference image feature group. The method for controlling the motion of a movable object by an image according to claim 1, wherein coordinate values are used.