JP2920352B2 - Traveling control method and apparatus for mobile object - Google Patents

Abstract

PURPOSE: To perform travel control over a teaching type autonomous travel vehicle which is of simple constitution and easily operated. CONSTITUTION: The travel control over the teaching type travel vehicle 1 which uses visual servo is performed. The moving body is provided with one CCD camera 10 so that it can angularly be displaced around a longitudinal axis through a tripod; when teaching, two landmarks 16 and 17 whose positions are unknown are picked up in one visual field while the camera 10 is fixed to the moving body to make a travel along a predetermined course, and the landmarks are stored in a memory. For a playback travel, the land marks 16 and 17 are picked up by the camera 10, the tripod of the camera 10 is brought under angular displacement control, and the direction of the camera 10 is matched with the teaching state. Then steering control is so performed that the position shift from the teaching state becomes zero. This control is repeated to make the travel vehicle 1 gradually return to a taught course and travels along the taught course 18.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for controlling the traveling of a moving body which performs navigation by using an image.
In particular, for example, autonomous vehicles equipped with a traveling drive source, such as automobiles, and even moving objects not equipped with such a traveling drive source, and moving objects such as ships and airplanes,
The present invention relates to a traveling control method and apparatus for a moving body that moves while steering along a predetermined path taught (teached).

[0002]

2. Description of the Related Art In a control device of an autonomous traveling vehicle which is being considered to be applied to an automatic guided vehicle or an unmanned inspection vehicle, a visual servo which performs traveling control only by a visual sensor without using an inner field sensor has attracted attention. I have. Objects that exist in the traveling environment are recorded as traveling marks or landmarks by a camera, which is a visual sensor attached to the traveling vehicle at the time of teaching, and the trajectory of these landmarks on the image screen during traveling (playback) is determined in advance. A so-called teaching-type autonomous traveling system (teaching / playback) has been proposed in which traveling of a traveling vehicle is controlled so as to match a locus of a corresponding traveling landmark recorded.

[0003] In the visual servo, a visual sensor such as a camera is attached to a traveling vehicle, and the traveling vehicle is controlled so that the visual sensor data during traveling approaches the target visual sensor data (teaching data). This method uses other visual sensors such as look and move, which does not cause a self-position measurement error due to traveling as in the case of using an internal sensor, does not need to create a map relating to the traveling environment in advance. And is less susceptible to camera calibration errors.

In order to perform the teaching-type autonomous traveling by the visual servo, a change in the visual sensor information is associated with a change in the position and orientation of the traveling vehicle by some method, and the obtained visual sensor information during the autonomous traveling is used during the teaching. The input to the traveling vehicle is determined so as to approach the visual sensor information. That is, in order to control the traveling vehicle by the visual servo, it is necessary to obtain a relationship between a change in the visual sensor data and a variation in the motion of the traveling vehicle.

Conventionally, as a method of the association, for example, “Control of Autonomous Mobile Robot Using Visual Servo” by Mikawa et al., Proceedings of the 11th Annual Conference of the Robotics Society of Japan, As can be seen from 1309-1310 (1993), there is a method of describing the relationship between the change in the characteristic point position and the change in the position and orientation of the traveling vehicle using an artificial target mark using the geometric relationship of the target mark.

[0006] In this prior art, an artificial target mark must be set in advance in the traveling environment, so that preparations for traveling are required, and autonomous traveling is impossible in a place where no target mark exists. There is a disadvantage that the degree of freedom of the traveling route is limited.

[0007] Another prior art is "Construction of Visual Servo System without Using Knowledge about Structure and Parameters" by Hosoda et al., The 4th Robot Symposium, pp. 1-35. 37-4
2 (1994), and this prior art is a method of estimating the relationship between a change in the image feature amount of an arbitrary object and a change in the state of the robot by using an internal sensor attached to the robot.
It is intended for robot manipulators.

In this prior art, an internal sensor must be attached to the traveling vehicle, and there is a drawback that the configuration of the equipment becomes complicated. These hinder the application of autonomous vehicles to the industry where simple operation procedures and easy equipment maintenance are required.

Still another prior art is “Study of hyper scooter: autonomous mobile robot capable of teaching driving” by Shibata et al., Proceedings of the 11th Annual Conference of the Robotics Society of Japan,
pp. 1301-1304 (1993), which is a technique in which an operator indicates a relationship between a change in position of an arbitrary target on an image and a change in position and orientation of a traveling vehicle.

[0010] This prior art has a drawback that, when operating an autonomous vehicle, an operator having knowledge of autonomous driving control is required. Therefore,
The teaching operation procedure is complicated because the manual operation of the operator is required for the autonomous traveling.

Further, as another conventional method, there is a method of calculating the position and orientation of a traveling vehicle from the principle of triangulation using two cameras, which are visual sensors, and one traveling landmark. It cannot be used for visual servoing due to the need for careful calibration and the long processing time.

Still another prior art is disclosed in JP-A-63-10 / 1988.
No. 9585, which is an example in which visual servo is applied to control of a robot arm, and does not refer to control of an autonomous vehicle.

[0013]

SUMMARY OF THE INVENTION An object of the present invention is to provide a method and an apparatus for controlling the traveling of a moving body which have a simple structure and are easy to implement.

[0014]

SUMMARY OF THE INVENTION According to the present invention, at the time of teaching, at least two landmarks provided at a fixed position are moved on a moving body along a predetermined route so that the moving body can be angularly displaced. With the provided camera fixed,
The camera captures an image in a single field of view, and for each landmark, associates each coordinate value of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera with a combination of coordinate values. A memory in which the landmark is imaged in a single field of view by a camera during traveling, and one of the landmarks obtained by imaging has the same coordinate value v1 in the one-axis V direction. Coordinate value v of the store contents of
t1 is read and searched, and the coordinate value vt2 in the one axis V direction of the other landmark in the combination including the searched coordinate value vt1 of one landmark is searched, and the coordinate value of the searched other landmark is searched. The camera is angularly displaced so that vt2 and the coordinate value v2 in the one-axis V direction of the other landmark imaged by the camera coincide with each other, and then the coordinates of the one landmark imaged in the other axis H direction. The value h1 is equal to the coordinate value ht1 of one landmark of the combination in the direction of the other axis H, and the coordinate value h of the captured other landmark in the direction of the other axis H.
2 is the coordinate value ht2 of the other landmark in the direction of the other axis H
This is a traveling control method for a moving body, characterized in that the vehicle is steered so as to coincide with the following. Further, according to the present invention, at the time of the steering, the coordinate value h1 of the one landmark imaged in the other axis H direction is taken.
, A center point of the imaged other landmark with the coordinate value h2 in the other axis H direction, the coordinate value ht1 of one landmark in the other axis H direction in the combination, and the other of the other landmarks in the combination. Coordinate value ht in axis H direction
The steering is performed so that the difference Δh from the center point of the steering wheel 2 becomes zero. Further, the present invention is characterized in that the moving body is provided with a plurality of sensors for detecting obstacles in or around the teaching path, and the vehicle is steered in a predetermined steering mode individually corresponding to a combination pattern of outputs of the sensors. And Further, according to the present invention, at the time of teaching, the mobile body travels along a predetermined route, and at least two landmarks provided at a fixed position are imaged in a single field of view by a camera fixed to the mobile body. Then, a combination of the coordinate values of each landmark, which is made by associating each coordinate value of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera for each landmark, is set along the path. When the vehicle is running, the landmark is imaged in a single field of view by a camera, and the coordinate value v1 in the one-axis V direction of one of the landmarks obtained by imaging is the same. The coordinate value vt1 of the contents stored in the memory is read and searched, and the coordinate value vt2 in the one-axis V direction of the other landmark in the combination including the searched coordinate value vt1 of the one landmark is obtained. The coordinates of the other landmark are searched from the store contents of the memory, and the coordinate value vt2 of the other landmark obtained by the camera is coincident with the coordinate value v2 of the other landmark in the one-axis V direction taken by the camera. The direction deviation is corrected by steering, and the coordinate value h1 of the one landmark imaged in the direction of the other axis H is then corrected.
Is coordinated with the coordinate value ht1 of the one landmark in the other axis H direction in the combination stored in the memory, and the coordinate value of the captured other landmark in the other axis H direction is a moving body that corrects the displacement of the moving body by steering such that h2 coincides with a coordinate value ht2 of the other landmark in the other axis H direction in the combination stored in the memory; Is a traveling control method. Also, the present invention provides (a) a moving body, (b) a steering means provided on the moving body for steering, and (c) a camera mounted on the moving body and capable of being angularly displaced around a vertical axis. (D) angular displacement driving means for driving the camera angularly displaced around the vertical axis; and (e) a memory, wherein the camera is fixed to the movable body and the movable body is moved along a predetermined path. When the vehicle travels, a combination of coordinate values obtained by imaging at least two landmarks provided at fixed positions in a single field of view by a camera is stored, and the combination of coordinate values is stored for each landmark. A memory in which coordinate values of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera are associated with each other; and (f) control means, which responds to the output of the camera and captures an image by the camera. One land obtained The coordinate value vt1 of the stored content of the memory having the same coordinate value v1 in the one-axis V direction of the mark is read and searched, and the other landmark in the combination including the searched coordinate value vt1 of one landmark is read. The coordinate value vt2 in the one-axis V direction
Is searched, and the coordinate value v of the other searched landmark is
The angular displacement driving means is operated to angularly displace the camera so that t2 and the coordinate value v2 of the other landmark imaged by the camera in the one-axis V direction coincide with each other. Coordinate value h1 of mark in other axis H direction
Is equal to the coordinate value ht1 of one landmark in the other axis H direction in the combination, and the coordinate value h2 of the other landmark in the other axis H direction is the other axis H direction of the other landmark. And control means for operating and steering the steering means so as to match the coordinate value ht2 of the moving body. Further, according to the present invention, the control means controls the coordinate value h1 of the one landmark imaged in the other axis H direction and the coordinate value h2 of the other landmark imaged in the other axis H direction during the steering by the steering means. And the difference Δh between the center point of the coordinate value ht1 of one landmark in the combination in the direction of the other axis H and the coordinate value ht2 of the other landmark in the combination in the direction of the other axis H becomes zero. It is characterized by steering. Further, according to the present invention, a first sensor M1 for detecting a left front obstacle on an axis passing through the center of rotation of the moving body and a front obstacle detecting on an axis passing through the center of rotation of the moving body are provided. And a third sensor M3 for detecting a right front obstacle on an axis passing through the center of rotation of the moving body.
In response to the output of the third sensor, when the first or third sensor detects an obstacle, the steering is steered by the steering means to change the direction to the right or left by a predetermined first angle, respectively, and the second sensor is When an obstacle is detected and the first and third sensors do not detect an obstacle, the steering is performed by the steering means so as to turn right or left by a second angle larger than the first angle. And The present invention also provides: (a) a moving body; (b) steering means provided on the moving body for steering; (c) a camera fixed to the moving body; Each landmark obtained by imaging at least two landmarks provided at a fixed position in a single field of view by a camera when the mobile body travels along a predetermined route with the camera fixed to the body. A combination of coordinate values for each of the landmarks is stored along the path, and the combination of the coordinate values is stored for each landmark in each of the coordinates of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera. And (e) a control means, wherein the coordinate value v1 in the one-axis V direction of one of the landmarks obtained by imaging by the camera in response to the output of the camera is the same. The coordinates of the store contents of a memory The value vt1 is read and searched, and the coordinate value vt2 in the one axis V direction of the other landmark in the combination including the searched coordinate value vt1 of the one landmark is searched from the stored contents of the memory. The steering means is operated so that the coordinate value vt2 of the other landmark coincides with the coordinate value v2 in the direction of the one axis V of the other landmark imaged by a camera. The displacement is corrected, and then the coordinate value h1 of the one landmark imaged in the other axis H direction is the coordinate of the one landmark in the other axis H direction in the combination stored in the memory. Value ht
1 and the coordinate value h2 in the other axis H direction of the captured other landmark in the other axis H direction in the other axis H direction of the other landmark in the combination stored in the memory. And a control means for correcting the positional deviation of the moving body by operating and steering the steering means so as to coincide with the coordinate value ht2.

[0015]

In order to solve the above-mentioned problems, the present invention uses a camera as a single visual sensor, and further uses at least two landmarks, which are running marks, whose positional relationship may not be known. The position and the attitude can be controlled. Accordingly, the present invention uses a plurality of objects existing in the traveling environment as landmarks, and uses the geometric relationship between the imaging positions of the landmarks during teaching and autonomous traveling along the teaching path of the traveling vehicle. A new method and apparatus for guiding is provided.

In the present invention, the control for simultaneously returning the position / posture of the moving body to the teaching state is performed by the following three properties 1 between the position / posture of the traveling vehicle and the imaging position of the traveling mark.
The positions and postures of the traveling vehicles are individually controlled by using 〜 to ３ to approach the teaching state.

According to the present invention, at the time of teaching,
The vehicle travels along a predetermined route, and in a state in which a camera is fixed to the mobile body, at least two landmarks provided near the route in a single field of view by the camera are singulated. And stores a combination of each coordinate value of two axes V and H in the orthogonal coordinate system of the camera for each landmark. Based on the stored contents of this memory, when playing back, that is, when traveling, in the initial state, since the position and orientation of the moving body and the direction of the camera are different from the teaching state, first, the camera is angularly displaced. Then, the direction of the camera is set equal to the teaching state. For this purpose, the coordinate value vt1 of the stored content of the memory having the same coordinate value v1 in the one-axis V direction of one landmark obtained by imaging with the camera is read and searched, and the coordinate value of the coordinate value thus found is obtained. The coordinate value vt2 of the other landmark in the combination in the one-axis V direction is searched, and the searched coordinate value vt2 of the other landmark is searched.
And the difference Δv between the coordinate value v2 of the other landmark imaged by the camera and the one axis V direction is zero, that is, the coordinate values vt2 and v2 match.
The camera is angularly displaced about a vertical axis.

According to the present invention, after the movement of the camera becomes equal to the teaching state as described above, the steering, that is, the steering control is performed so that the positional deviation from the camera becomes zero. By repeating these controls, the moving body can gradually return to the teaching path and move on the predetermined path. According to the present invention, since the displacement becomes zero, the coordinate value h1 of the one landmark imaged in the other axis H direction matches the coordinate value ht1 of the combination stored in the memory and the other landmark. The same applies to the landmark, that is, the steering by the steering means is performed such that the coordinate value h2 of the other landmark imaged coincides with the coordinate value ht2 of the combination.

According to the present invention, in order to reduce the displacement to zero, a center point C1 between the coordinate value h1 of the one landmark imaged and the coordinate value h2 of the other landmark imaged is obtained. A center point C2 between the coordinate value ht1 of one landmark in the combination and the coordinate value ht2 of the other landmark in the combination is obtained, and the steering is performed so that the difference Δh between these two center points C1 and C2 becomes zero. .

According to the present invention, the moving body is provided with a plurality of sensors, detects obstacles in or near the teaching path, and determines a predetermined steering individually corresponding to a combination pattern of outputs from the sensors. Steering is performed in this manner, so that the moving body does not collide with an obstacle while the moving body is traveling, and can move along a predetermined route.

According to the present invention, FIGS.
As described below in connection with 6, the camera may remain fixed to the mobile, in which case the orientation of the camera, and thus the orientation of the mobile, may correspond to the teaching state.
Steer to correct the misalignment of the moving body, and then the camera, and thus the moving body, so that the misalignment is zero.
The displacement of the moving body is corrected by steering.

In the present invention, the position and orientation of a traveling vehicle can be controlled in principle by using two landmarks.
If this method is applied to three or more landmarks, it is expected that the reliability and accuracy of the position and orientation control of the traveling vehicle will be improved.

In the present invention, it is also possible to implement the present invention by connecting a memory in which a combination of coordinate values is stored in advance in a removable manner and connecting it to the control means of the present invention. For example, the traveling control of a plurality of moving bodies can be performed by one teaching.

The present invention can be implemented not only in relation to a moving body provided with a driving source for traveling, but also with respect to a moving body not having such a driving source.

[0025]

FIG. 1 is a side view of a moving body 1 according to one embodiment of the present invention, and FIG. 2 is a simplified plan view of the moving body 1. The moving body 2 is provided with a pair of front wheels 3, 4 and a pair of rear wheels 5, 6, and the front wheels 3, 4 can be steered by steering means 23 (see FIG. 4 described later), and the rear wheels are provided. The wheels 5 and 6 are not steered, and are driven to rotate by a drive source to be able to travel autonomously. A camera 10 is mounted on a front end 7 of the moving body 2 by a camera platform 9 that is rotatable around a vertical rotation axis 8. The camera platform 9 and thus the camera 10 are driven by a turning drive source 22 (see FIG. 4 described later). The optical axis of the camera 10 is as indicated by reference numeral 11, and is set to a depression angle θ1 from the horizontal axis 13, for example, −20 degrees when the moving body 1 is placed on a horizontal path 12.
The camera 10 may have a configuration in which an image is formed by an objective lens on an image sensor including a matrix CCD (charge storage device), for example. The axis of the moving body 2 in the straight traveling direction is indicated by reference numeral 14, and this axis 14 and the camera 1
The rotation axis 8 of the camera platform 9 of 0 is orthogonal to the plane within one plane.
The turning axis 8 is perpendicular to a plane that is in contact with the traveling road surface of the four wheels 3 to 6. This one plane is, for example, the sea surface or the water surface when the moving body 1 is a ship. The center of rotation of the vehicle is designated by the reference numeral 15, and the distance DS1 from the center 15 to the front wheel axle and the distance DS2 from the rear wheel axle are equal.

FIG. 3 is a simplified plan view showing a state in which the moving body 1 moves along a predetermined route 18 as a taught target according to landmarks 16 and 17 provided at fixed positions. In order to control the autonomous traveling vehicle by the visual servo, a relationship between a change in output data of the camera 10 and a change in the position / posture of the traveling vehicle must be expressed. Therefore, two objects existing in the traveling environment are used as landmarks 16 and 17, and the geometrical properties between the image capturing positions during teaching and traveling obtained by the camera 10 mounted on the traveling vehicle are used. Express the leverage relationship. Then, using this relationship, a visual servo is individually applied to the pan head on which the camera 10 is mounted and the steering of the traveling vehicle to realize autonomous traveling along the target orbit route 18.

In the present embodiment, the following conditions 1 to 5 are set.
is necessary. Regarding the traveling vehicle, (1) having a steering mechanism for controlling movement, (2) camera 1
A visual sensor, such as 0, is installed on a pan that can rotate (pan) about an axis perpendicular to the traveling plane of the vehicle, and (3) the steering and the pan can be individually controlled. thing. Regarding the traveling environment, (4) the traveling vehicle moves on the leveling ground (therefore, the height H1 of the camera in FIG. 1 is constant), and (5) the landmark 1
There are two or more objects that can be used as 6,17. However, that a certain object can be used as the landmarks 16 and 17 means that the object can be extracted by image processing. Therefore, if an object recognition technique based on color image processing or pattern matching is applied, more objects can be landmarks 16 and 1.
7 can be used.

FIG. 4 is a block diagram showing an electrical configuration of one embodiment of the present invention. The two-dimensional captured image from the camera 10 is provided to the image processing circuit 19. The teaching data is stored in the memory 20, and the contents of the memory 20 are controlled by a processing circuit 21 for writing and reading. The turning of the camera 10 around the vertical axis 8 by the camera platform 9 is performed by a turning drive source 22. Also front wheels 3, 4
Is performed by the steering means 23. The configuration related to FIG. 4 will be described in connection with the later-described playback traveling.

At the time of teaching, the camera 10 is directed in the straight direction of the moving body 1 as a traveling vehicle, that is, in the forward direction along the axis 14, in other words, the axis 14 of the moving body direction 2 and the optical axis of the camera 10 are aligned. Is in one plane, the head 9,
Therefore, teaching is performed with the camera 10 fixed to the moving body 2, and the landmarks 16 and 17 are imaged in a single field of view and recorded in the memory 20.

In running vehicle control by visual servo,
First, teaching work is performed. In a state where the camera 10 by the camera platform 9 is fixed in the traveling direction of the traveling vehicle, the operator teaches the traveling vehicle by actually traveling the traveling vehicle along a desired route. Then, from the image captured by the camera 10 mounted on the traveling vehicle at that time, objects whose image characteristic parameters such as luminance, shape, and color are significantly different from the background are identified as landmarks 16 and 1.
7, two are extracted by image processing.

Next, the trajectories on the images are obtained to create teaching data as shown in Table 1, and used for subsequent autonomous traveling. These image processes are automated. Further, when two or more objects are extracted by the image processing, it is automatically determined at which point in time which object should be used as a landmark by using a production system for planning a landmark use plan. be able to. Therefore, the teaching operation necessary for the operator in this method is only to make the traveling vehicle actually travel along the desired route.

FIG. 5 is a diagram showing the definition of a rectangular coordinate system. In FIG. 5, a symbol Σc is a camera coordinate system, Σi is an image plane coordinate system, cPf is a coordinate of a feature, that is, a landmark 16 or 17, viewed from Σc, iPf is a coordinate of a feature viewed from Σi, and f is a lens focal point. Distance. Camera 1
Σi is determined such that the focal position of 0 coincides with the origin of Σc and the optical axis coincides with the X axis.

The suffix w is a world coordinate system, c is a camera coordinate system, i is an image coordinate system, f indicates a point of a landmark, t indicates teaching data, and r indicates a travel time. Data is shown, and superscripts 1 and 2 are numbers representing landmarks 16 and 17 which are features.

To further describe the symbols, Σ indicates a coordinate system, P indicates a coordinate, cPf indicates the coordinates of a landmark which is a feature in the coordinate system Σc, x, y, z indicate three-dimensional coordinates, h, v Indicates image plane coordinates, and α indicates a coordinate system Σc
Is the rotation angle around the wZ axis, the direction of the camera 10 is positive in the counterclockwise direction, β is the rotation angle of the coordinate system Σc around the wY axis, indicates the depression angle of the camera 10, and the downward direction is positive. , Θ indicate angles, L indicates the distance between the camera 10 and the landmarks 16 and 17 at wZ, d indicates the amount of displacement of the traveling vehicle during teaching and during traveling, and symbol f indicates the focal point of the lens of the camera 10. Indicates the distance.

At the time of teaching, the coordinate values of each of the landmarks 16 and 17 in one vertical axis V direction and the other horizontal axis H direction in the orthogonal coordinate system 25 of the image, which is the image plane of the camera 10, correspond to each other. Then, the combination of the coordinate values is stored in the memory 20. The axis V direction is parallel to the pivot axis 8. Image numbers 1 to n in Table 1 represent a combination of a total of n coordinate values.

[0036]

[Table 1]

The method of the visual servo according to the present invention will be described. In the present invention, the geometric properties of the imaging positions of the two landmarks 16 and 17 obtained by the camera 10 during teaching and during traveling are used. In FIG. 6, FIG.
Is the image P of the landmarks 16 and 17 by the camera 10
FIG. 3 is a diagram showing images on the image forming plane of the camera 10 in the directions of the corresponding points Pt1 and Pt2 stored by the memory 1 and P2 and the memory 20. FIG. 6B shows points P1 and P2 captured by the camera 10 and points P1 and P2 stored in the memory 20.
It is a figure for explaining displacement of position of t1 and Pt2.
6 (1) and 6 (2), the broken lines indicate
4 shows a teaching path of the landmarks 16 and 17. Pt1 (ht1, vt1) and Pt2 (ht2, vt2) are the imaging positions at time t when the two landmarks 16 and 17 are taught.
Assuming that (ht1> ht2) and the imaging positions during traveling are P1 (h1, v1) and P2 (t2, v2), these properties 1 to 3 can be expressed as follows.

(Properties 1) Landmarks 16, 1 running
If the vertical imaging position v1 of 7 is equal to the vertical imaging position vt1 of the landmark at time t in the teaching data, the distance between the landmark and the camera 10 at that time is equal to the distance at time t of the teaching data. That is, if the given vertical imaging position v of the landmark 16 is equal to the vertical imaging position vt of the landmark at time t in the teaching data, the distance between the given landmark and the camera is the teaching data time. equal to that at t.

The proof of the geometric property 1 of the two landmarks 16 and 17 used in the present method during teaching and during traveling is shown below.

First, a description will be given of derivation of a basic expression that is the basis of properties 1 to 3. From FIG. 5, the coordinate system 定 め る i is determined so that the focal length of the camera 10 coincides with the origin of the coordinate system Σc and the optical axis coincides with the X axis (cX). At this time,

[0041]

(Equation 1)

Rotation angle of Σc around wZ axis: α (camera direction) Rotation angle of Σc around wY axis: β (elevation angle of camera) wPf = wRc · cPf + wPc (3) ∴cPf = wRc ⁻¹ (wPf) −wPc) (4) Here, also referring to FIG. 7, Σw: the world coordinate system, wPc: the position of Σc (camera) in Σw. However,

[0043]

(Equation 2)

Therefore,

[0045]

(Equation 3)

For simplicity, if the elevation angle of the camera is β = 0,

[0047]

(Equation 4)

For simplicity, the camera elevation angle β
= 0, the image plane coordinates (hf (t), v
f (t)) ^T , and the image plane coordinates (h′f (t ′), v′f (t ′)) ^T of the feature at the time t ′ during traveling
By using equations 1 and 2, respectively.

[0049]

(Equation 5)

Here, (wXf, wYf, wZf) ^T : coordinates of the feature f viewed from Σw, (wXc (t), wYc (t), wZc (t)) ^T : time at the time of teaching viewed from Σw camera coordinates at t, (wX'c (t '), wY'c (t'), wZ'c (t ')) ^T : the coordinates of the camera at time t' when traveling as viewed from Σw, α : The rotation angle of the camera around the wZ axis at the time t during teaching α ': The rotation angle of the camera around the wZ axis at the time t' during traveling In this way, the basic formula was derived.

Next, the above (property 1) will be proved. Referring to FIG. 8, states 1 and 2 on image forming plane 25 of camera 10 are as follows. [State 1] Position of traveling vehicle in state t during teaching [State 2] Position of traveling vehicle in state t 'during traveling In these states 1 and 2, when the distance between feature f and camera 10 is equal, According to 2, the image plane v coordinate of the feature f is c.
It is determined by Zf and cXf.

Now, assuming that cZf = wZf-wZc = const (10) (where wZf = const is clear and wZc = const), vf is determined by cXf.

Accordingly, the fact that the distance between the feature f and the camera 10 is equal in the states 1 and 2 means that cXf (t) = cX'f (t) = cXf (11). Are the same in the image plane v coordinate position of the feature f.

Therefore, the above property 1 has been proved.

In the present invention, referring to FIG.
The following property 2 is used.

(Property 2) When the property 1 is satisfied, the difference Δv Δv = vt2−v2 between the vertical imaging position v2 during the travel of the landmark and the vertical imaging position vt2 at time t in the teaching data (12) ) Represents the deviation of the direction from the teaching state of the camera 10 (however, only when the deviation of the direction is within 45 °). That is, focusing on the two landmarks f1 and f2, when the distance between the camera 10 and the landmark f1 is equal to that at the time t in the teaching data in a given state, the teaching data of the vertical imaging position of the landmark f2 is Difference Δv
Reference numeral 2 denotes a deviation from the teaching state.

Regarding the features f1 and f2 in FIG.
The difference between the vertical imaging coordinates v between the teaching time (state 1) and the traveling time (state 2) is given by Equation 2.

[0058]

(Equation 6)

Since the distance between the camera and f1 is equal in states 1 and 2, Δv1 = 0 (15) cxr1 = cXt1 (16) where cxt1 = (wx1-wxtc) cosαt- (wy1-wytc) sinαt (17) cxr1 = (wx1-wxrc) cosαr- (wy1-wyrc) sinαr (18) For simplicity, in order to refer to the direction of the traveling vehicle at the time of teaching, αt = 0 0 (19) Δα = Αr−αt (= αr) (20), cxr1-wxtc = (wx1-wxrc) cosΔα− (wy1-wyrc) sinΔα (21) ∴wxtc = wx1- (wx1-wxrc) cosΔα + (wy1- wyrc) sinΔα (22) Therefore, Expression 14 uses Expression 22. (Numerator of Expression 14) = cxr2-cxt2 = (wx2-wxrc) cosΔα- (wy2-wyrc) s inΔα- (wx2-wxtc) = (wx2-wxrc) cosΔα- (wy2-wyrc) sinΔα-wx2 + wx1- (wx1-wxrc) cosΔα + (wy1-wyrc) sinΔα = (ΔY1-ΔY2) sinΔα- (ΔX1-) cosΔα + (ΔX1−ΔX2) (23) (denominator of equation 14) = cxrt2 · cx2 = [(wx2-wxrc) cosΔα− (wy2-wyrc) sinΔα] × [wx2-wx + (wx1-wxrc) cosΔα- (wy1) −wyrc) sinΔα] = [ΔX2 · cosΔα−ΔY2 · sinΔα] × [ΔX1 · cosΔα−ΔY1 · sinΔα + ΔX2-ΔX1] (24)

[0060]

(Equation 7)

Where ΔX = wx1−wx2 ΔY = wy1−wy2 ΔX1 = wx1−wxrc ΔY1 = wy1−wyrc ΔX2 = wx2−wxrc ΔY2 = wy2−wyrc Δv2 monotonically increases as Δα increases, and the property 2 holds. The condition is obtained by numerical analysis of Expression 25.

As an example, the angle of view and the image size are each 9
0 °, 512 × 512 pixels, and when the distance between the camera and the landmark is in the range of 1 to 10 m, the captured image plane h coordinates of the cameras 1 and 2 are 275 to 275, respectively.
If the objects at 511 to 0 to 235 are used, the property 2 is satisfied in the range of −5 ° ≦ Δα ≦ 5 ° (26). Thus, Property 2 was proved.

The amount of deviation of the direction of the camera 10 between the teaching state and the actual state can be determined by referring to FIGS. 10 (1) and 10 (2).
As can be seen from (L2−Lt2), that is, Δv2. The shift amount Δv2 indicates a shift of the position of the landmark 16 or 17 on the imaging screen 25 of the camera 10 along the vertical axis v. Referring to FIG. 10A, in a certain state P during traveling, landmark 16
When attention is paid to the teaching state T in which the distance L1 to
The difference in distance between the other landmarks 17 (L2-Lt
2), that is, the difference Δ between the image vertical positions of the landmarks 17
v2 represents the magnitude of the direction shift amount θ.

Property 3 will be described as the last property used in the present invention. The property 3 is that, when the orientation of the camera 10 is not shifted between the teaching state and the current state of imaging by the camera 10 with reference to FIGS. That is, it is proportional to the displacement amount of 1. That is, FIG.
In the equation, Δh is proportional to the displacement d.

(Property 3) When property 2 is satisfied, the horizontal imaging position h of the landmark 1 (or 2) during traveling
The difference between 1 (or h2) and the horizontal imaging position ht1 (or ht2) at time t in the teaching data Δh1 = Δht1-h1 (27) or Δh2 = Δht2-h2 (28) Represents a positional deviation from That is, when there is no misalignment between the given state and the state at time t in the teaching data, the difference Δh1, Δh2 between the landmark horizontal imaging positions at that time indicates the misalignment with the teaching state.

Next, it will be proved that the property 3 is satisfied.

In states 1 and 2, there is no misalignment (α =
α ′), when the distance to the feature f is equal, P1: (wXc, wYc, wZc) (29) P2: (wX′c, wY′c, wZ′c) (30)

[0068]

(Equation 8)

From Expression 1, the h position hf of the feature f on the coordinates of the image plane 25 is determined by cXf and cYf.
Since the distance to the feature f is equal according to the condition, cXf (t) = cX'f (t ') = cXf (32), so that hf is determined by cYf.

Further, hf (t) −h′f (t ′) ∝cYf (t) −cY′f (t ′) ∝ [(wXf−wXc (t)) sin α + (wYf−wYc (t)) ) Cos α] − [(wXf−wXc (t) + d sin α) sin α + (wYf−wYc−d cos α) cos α] ∝ d (33) Accordingly, the displacement of the feature f in the image plane h coordinate position The quantity hf (t) -h'f (t ') is proportional to the displacement d of the traveling vehicle. Property 3 was thus proved.

In the present invention, the pan head 9 is set so that Δv = 0.
And the steering is controlled so that Δh = 0 so that the traveling vehicle travels along the teaching path. Head 9
The steering is controlled using PID (proportional, integral, differential) control or the like. When only the amount Δh relating to the deviation of the position of the traveling vehicle is used for the steering control, there is a property that the motion of the traveling vehicle hardly converges. Therefore, in order to provide the effect of suppressing the change in the attitude of the traveling vehicle, the visual servo system as shown in FIG. 4 is configured by adding the operation amount of the camera platform to the operation amount of the steering wheel. In the above description, the landmarks 16, 1
7 may be indicated by features f1, f2, and the like.

FIG. 13 is a flowchart for explaining the operation of the electric circuit of one embodiment of the present invention shown in FIG. During playback running, the operation shown in FIG. 13 is performed based on the teaching data stored in the memory 20. Step b1 to step b6,
The control is performed by operating the turning drive means 22 by the camera platform 9.
In step b7, the steering control by the steering means 23 is performed by utilizing the above-described property 3. That is, in Steps b1 to b6, the turning drive unit 22 by the camera platform 9 is operated and controlled using the above-described properties 1 and 2 so that the direction of the camera 10 is always substantially the same as the teaching state. . In step b7, since the deviation between the teaching state and the direction of the camera 10 is almost eliminated as described above, the above-mentioned property 3 is used, and the steering unit 23 is operated so that the positional deviation of the camera 10 becomes almost zero. Perform steering control. This steering control is the same as performing steering control so that the displacement of the moving body 1 that is a traveling vehicle becomes zero.

First, reference is made to FIG. 14 for the description of steps b1 to b6. In FIG. 14, a broken line indicates a teaching path, and the process moves from step b1 to step b2 in the image forming plane 25 of the camera 10 to obtain image data. As a result, in step b3, the points P1 and P2 in the VH plane are
Is obtained. Therefore, one landmark 16 is considered as a reference. For the landmark 16, the teaching point Pt1 having the same coordinate value of the vertical axis V is searched for from the teaching data in the memory 20 in Table 1 in step b4 using the above-described property 1. The coordinate value vt1 stored in the memory 20 having the same coordinate value v1 in the axis V direction of the point P1 is read and searched, and the image number, that is, the combination of the coordinate values is found.

At step b5, a point Pt2 on the teaching path of another landmark 17 corresponding to the point Pt1 is searched. This search is obtained from within the above-described combination of the same coordinate values.

Thus, at step b4, the teaching point Pt1
In the next step b5, the teaching point Pt2 is searched.

In step b6, the turning of the camera platform 9 is controlled. The turning control amount E1 of the camera platform 9 is determined, for example, by the method shown in FIG. Δv in FIG. 4 is as described in Expression 12 above.

At step b7, steering control is performed. In this step b7, the point P on the horizontal axis H shown in FIG.
1, P2 center point c11 and teaching data points Pt1, Pt
The center point c21 of 2 is obtained by Expressions 34 and 35.

[0078]

(Equation 9)

These values ht1 and ht2 are stored in the memory 20
Are used in which the combination of the above-mentioned coordinate values is the same.

The difference Δh is obtained by equation (36). Δh
Is used in the following Expression 36.

Δh = c11−c21 (36) The steering control amount E2 is determined, for example, by the method shown in FIG.

At step b8, it is determined whether or not all the teaching paths have been completed.
Move on to If it is determined in step b8 that the operation has been completed, the process proceeds to step b9, and all operations are completed.
By selecting the constants F1 and F2 to appropriate values in this way, the traveling vehicle can follow the teaching route almost in agreement.

Referring back to FIG. 4, the arithmetic circuit 26 calculates the above equation (3).
4 and the operation circuit 27 performs the operation of the above-described equation 35. The subtractor 28 calculates the direction shift Δv.
Another subtractor 29 calculates the displacement Δh. In order to control the turning drive source 22, circuits 30 and 31 for proportionality and integration and an adder 32 are provided, and a counter 33 is further provided. The turning drive source 22 is provided to the drive circuit 35 and thus the turning drive source 22 is operated.

For steering of the steering means 23, circuits 37 and 38 for proportionality and integration and an adder 39 are provided. Further, counters 40 and 42 are provided. Adder 4
In 3, the signal for turning the camera 10 through the counter 44 is given and added, and the effect of suppressing the change in the posture of the moving body 1 is achieved as described above, and the movement of the moving body 1 is easily converged. I do. The output of the adder 43 is supplied to a steering limiting circuit 44.
And the steering angle is limited, and the drive circuit 45
Drives the steering means 23.

FIG. 16 is a flowchart for explaining the operation of the electric circuit of another embodiment of the present invention shown in FIG. During playback running, the operation shown in FIG. 16 is performed based on the teaching data stored in the memory 20. Step a1 to step a6
Uses the properties 1 and 2 described above to
Is controlled by operating the swivel driving means 22 by the camera platform 9 so that the direction is always the same as the teaching state. In steps a7 to a9, since the deviation between the teaching state and the direction of the camera 10 has been eliminated as described above, the above-described property 3 is used, and the steering unit 23 is operated so that the positional deviation of the camera 10 becomes zero. Perform steering control. This steering control is the same as performing steering control so that the displacement of the moving body 1 that is a traveling vehicle becomes zero.
First, regarding the description of steps a1 to a6, FIG.
Refer to FIG. In FIG. 14, a broken line indicates a teaching path, and the process moves from step a1 to step a2 in the image forming plane 25 of the camera 10 to obtain image data. Thereby, with respect to the landmarks 16 and 17, at step a2-1, points P1 and P2 are obtained in the VH plane. Therefore, one landmark 16 is considered as a reference. For the landmark 16, the teaching point Pt1 having the same coordinate value of the vertical axis V is searched from the teaching data in the memory 20 in Table 1 described above using the above-described property 1. The coordinate value vt1 stored in the memory 20 having the same coordinate value v1 in the axis V direction of the point P1 is read and searched, and the image number, that is, the combination of the coordinate values is found.

At step a3, a search is made for a point Pt2 on the teaching path of another landmark 17 corresponding to the point Pt1. This search is obtained from within the above-described combination of the same coordinate values.

In step a4, the turning drive source 22 by the pan head 9 is moved to step a so that the difference Δv (= vt2−v2) between the coordinate values of the vertical axis v between the points P2 and Pt2 becomes zero.
Drive control is performed by 5 and a6. Thus, when Δv = 0, the direction of the camera 10 becomes the same as the teaching state, which means that the property 2 is used.

In the next step a7, the center point c11 of the points P1 and P2 on the horizontal axis H and the center point c21 of the points Pt1 and Pt2 of the teaching data shown in FIG.

These values ht1 and ht2 are stored in the memory 20
Are used in which the combination of the above-mentioned coordinate values is the same.

The difference Δh is obtained by Expression 36.

Then, the steering means 2 is set so that Δh = 0.
The steering control by 3 is performed in step a8. As a result, the position of the camera 10, that is, the position of the traveling vehicle of the moving body 1 becomes the same as the teaching state, which means that the property 3 is used. By repeating the processing operations of steps a1 to a9, the traveling vehicle can autonomously travel along the teaching route while the camera 10 is always controlled to the direction at the time of teaching.

FIG. 17 shows the experimental results of the present inventor.
The landmarks 16 and 17 are arranged in front of the teaching path 47, and the moving body 1 has started automatic traveling from the start position 48. As a result, it was confirmed that the target position 51 could be reached by following the route 50.

Although the teaching path 47 is a straight path in FIG. 17 described above, according to another experiment result of the present inventor shown in FIG. 18, the teaching path 52 is a curved path, and the moving object is moved from the start position 53 to the starting position 53. It was confirmed that the user started moving and could follow the route 55 to reach the target position 56.

In another embodiment of the present invention, autonomous traveling over a long distance becomes possible by sequentially using more landmarks. FIG. 19 is an example of verifying long-distance autonomous traveling by simulation. Here, the teaching records the trajectories on the image of all the objects extracted when the traveling vehicle travels along the desired route. During autonomous driving, two objects located close to the traveling vehicle (the vertical imaging position is below) are sequentially selected from the observed objects as landmarks, and visual servoing is performed using the teaching data of those objects. Do. Therefore, in this embodiment, autonomous traveling over a long distance is possible only by the operator actually driving the traveling vehicle. The landmarks 81 to 84 are sequentially switched and used for each of the areas A, B, and C. In addition, by using a production system that incorporates knowledge about the selection of a landmark that is optimal for a visual servo for the selection of a landmark, more reliable and efficient autonomous driving becomes possible.

The results of another experiment of the present inventors will be described. FIG. 20 is a simplified plan view of a moving body 1 according to another embodiment of the present invention using the above-described embodiment of the present invention.
FIG. 2 is a simplified plan view showing a more specific configuration of the moving body 1. The structure relating to the moving body 2 is the same as that of the above-described embodiment, and such a structure is not described in detail in FIGS. It should be noted that in this embodiment, the moving body 1 is configured to avoid obstacles. A first sensor M1 that detects an obstacle on the left front on an axis 59 passing through the rotation center 15 is attached to the main body 2. Further, a second sensor M2 for detecting an obstacle ahead on the axis 14 passing through the rotation center 15 is fixed to the main body 2. The third sensor M3 detects a right front obstacle on an axis 60 passing through the center of rotation 15. The axes 59 and 60 correspond to the axis 1 of the moving body 2.
4, the angles θ2 and θ3 may be, for example, 30 degrees.

FIG. 22 is a block diagram showing the overall electrical configuration of the embodiment shown in FIGS. 20 and 21. Processing circuit 6 realized by microcomputer or the like
2 includes a camera 10, a memory 20, drive circuits 35 and 45,
Are connected, and this configuration is similar to that of the above-described embodiment. In particular, in this embodiment, the outputs of the sensors M1 to M3 are provided to the processing circuit 62.

FIG. 23 is a flow chart for explaining the operation of the processing circuit 62. The operation moves from step b1 to step b2, and the operation of traveling on the teaching route is performed. The operation in step b2 is the same as the operation described with reference to FIG. In particular, in this embodiment, it is determined in step b3 whether there is an obstacle by the sensors M1 to M3. When an obstacle is detected by the sensors M1 to M3, the process moves from step b3 to step b4, and a control operation by the steering means 23 for avoiding the obstacle is performed. Each of the sensors M1 to M3 may be an ultrasonic sensor, that is, has a configuration in which ultrasonic waves are radiated outward and the distance between each of the sensors M1 to M3 and an obstacle is measured by a reflected wave. When the detected distance is less than a predetermined value, it is determined that there is an obstacle, and ON is determined.
A signal is derived, and when the detection distance is equal to or greater than a predetermined value, an OFF signal is derived. Processing circuit 62
In step b4 of FIG. 23, each sensor M
1 to M3, the steering means 2 shown in Table 2
3 for steering angle control.

[0098]

[Table 2]

When the first or third sensor M1, M3 detects an obstacle and derives an ON signal, the steering angle γ is shifted to the right.
The steering means 23 is controlled so as to change the direction by R / 2 or the steering angle γL / 2 to the left. Further, the second sensor M2 detects an obstacle and derives an ON signal, and the first and third sensors M1 and M3 do not detect an obstacle and are turned OFF.
When the signal is derived, in this embodiment, the steering angle γ
The steering means 23 is controlled by R, and is turned rightward at the maximum steering angle γR. As another embodiment, at this time, the steering means 23 may be steered so that the direction can be changed by γL.

ΓR is the maximum rightward steering angle of the steering unit 23, and γL is the maximum leftward steering angle of the steering unit 23. In this embodiment, for example, γR = γL.

If the obstacle detecting sensor is mounted on the traveling vehicle as described above, the obstacle avoidance function can be realized by the above-described embodiment. As an example, three obstacle detection sensors M1 to M3 were installed on a traveling vehicle as shown in FIG. 24, and an obstacle avoidance simulation was performed. This sensor M1
It is assumed that M3 detects an object existing within 80 cm on the mounting direction axis and outputs a detection signal. Here, the pan head 9 is controlled so that the teaching state and the orientation are always the same regardless of the presence or absence of the obstacle 63.
Only when M3 detects an obstacle, only the steering of the traveling vehicle is controlled to a constant value in a direction to avoid the obstacle. In this embodiment, when only the sensor M2 outputs the detection ON signal, the steering is set to turn left at the maximum. That is, the three obstacle detection sensors M1 to M3 were mounted on a traveling vehicle, and the obstacle avoidance control according to the present invention was simulated by a computer. These sensors output an obstacle detection signal when an obstacle is detected within a certain distance, and the relationship shown in Table 2 is used between the output signal of each sensor and the travel control.

FIG. 24 shows the running result at this time. Thus, it is understood that obstacle avoidance control can be performed according to the present invention. The simulation verifies the autonomous driving when an obstacle is placed on the straight teaching path,
Columnar object (cross section 40cm x 40) as obstacle
cm, height infinity). FIG. 24 shows the simulation results. This shows that the obstacle avoidance function can be realized by this method. In this simulation, hiding of a landmark by an obstacle is not considered, but in such a case, autonomous traveling can be continued by using another object as a landmark.

The traveling speed of the moving body 1 is, for example, 10 cm.
/ Sec and may be constant.

FIG. 25 is a simplified plan view for explaining another embodiment of the present invention. Although this embodiment is similar to the above-described embodiment shown in FIGS. 1 to 19, it should be noted that in this embodiment, the camera platform 10 is not provided, and the camera 10 is directly fixed to the moving body 2. It is configured to be unable to turn. The optical axis 11 of the camera 10 coincides with the rectilinear axis 14 of the moving body 2.

When the landmarks 16 and 17 are present and the moving body is taught to travel from the start position 64 to the target position 65 via the teaching path 66, the moving body is indicated by a reference numeral 67. 26, first, the direction of the moving body 67 is corrected by the steering means 23, and then the displacement is measured in step e2. In step e3, the steering means 23 is operated so that the amount of displacement becomes zero. Thereby, the deviation of the direction is corrected in step e4. By repeating steps e1 to e4 in this way, the moving body travels so as to approach the teaching path 66.
In step e1, the moving object moves from 67 to 68, in step e2, the moving object moves from 69 to 69, and in step e3, the moving object moves from 69 to 70. In step e4, the moving object advances from 70 to 71. In this manner, the moving body moves as indicated by reference numeral 72 and returns to the teaching path 66.

In this embodiment, when the mobile unit is moved along a predetermined route while the camera 10 is fixed to the mobile unit 1 as in the above-described embodiment, at least two lands provided at the fixed positions are provided. Marks 16 and 17 on camera 10
The combination of the coordinates of the landmark 16 and the coordinates of the landmark 17 obtained by imaging in a single field of view is stored in the memory 20. This combination of coordinate values is such that the coordinate values of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera 10 correspond to each landmark 16 and 17.

The control means responds to the output of the camera 10,
The coordinate value vt1 of the stored content of the memory having the same coordinate value v1 in the one-axis V direction of the one landmark 16 obtained by imaging with the camera 10 is read and searched, and the one landmark 16 is searched. The coordinate value vt2 in the one-axis V direction of the other landmark 17 in the combination including the coordinate value vt1 is searched. The moving means is operated by operating the steering means 23 so that the coordinate value vt2 of the other landmark 17 thus retrieved matches the coordinate value v2 of the other landmark 17 captured by the camera 10 in the one-axis V direction. The direction of 1 is corrected by angular displacement.

Next, the coordinate value h1 of one of the landmarks 16 in the other axis H direction is coincident with the coordinate value ht1 of the one landmark 16 in the other axis H direction in the combination, and the image was picked up. The position of the moving body is controlled by operating the steering means 23 so that the coordinate value h2 of the other landmark 17 in the other axis H direction coincides with the coordinate value ht2 of the other landmark 17 in the other axis H direction. Correct the misalignment.

The moving body 1 is not limited to the configuration shown in FIGS. 1 and 2, but may have another configuration. That is, the moving body 1 is a four-wheel drive and / or a four-wheel drive.
The wheel steering system may be used. Further, as shown in FIG. 27, a three-wheeled vehicle having front wheels 3, 4 or rear wheels 9
The steering wheel may be configured to be steered by a differential turning method in such a three-wheeled vehicle as shown in FIG. , 4 or the rear wheel 98. Furthermore, according to the present invention, the configuration of the driving and steering includes various configurations,
It is easily conceived by those skilled in the art. The invention may also be practiced in connection with ships or airplanes and the like.

[0110]

As described above, according to the present invention, since the position of a landmark used for traveling control, that is, the position of a landmark is completely unknown, many objects existing in the traveling environment are used as the traveling landmark. can do. For this reason, it is not necessary to maintain the traveling environment such as newly installing traveling marks, and the degree of freedom of the traveling route is high. Also, there is no need for an internal sensor or operator's instruction.

Thus, according to the present invention, the 2
An excellent effect is obtained in that two objects are landmarks, the positions of those objects need not be known, and only one camera is used, and further, no operator instruction is required. According to the present invention, since an object existing in the traveling environment is used as a landmark, there is no need to newly install a target mark or the like that is intentionally designed, and if there are at least two objects to be used as landmarks, Frequently, their positional relationship may not be known, only one camera is used, and no internal sensor is required.

Thus, according to the present invention, the teaching is completed only by causing the traveling vehicle to actually travel along the desired route, and thereafter, it is possible to autonomously travel the route without the intervention of the operator.

Further, according to the present invention, a plurality of sensors are provided on the moving body, and the steering is performed in a predetermined steering mode by the steering means in correspondence with the combination pattern of the outputs of the respective sensors. It is possible to prevent the moving body from colliding with an obstacle during traveling and move to the destination on the teaching route.

[Brief description of the drawings]

FIG. 1 is a side view of a moving body 1 according to an embodiment of the present invention.

FIG. 2 is a simplified plan view of the moving body 1 of the embodiment shown in FIG.

FIG. 3 is a plan view showing a state in which the moving body 1 moves.

FIG. 4 is a block diagram showing an electrical configuration of the moving body 1.

FIG. 5 is a diagram showing a definition of a coordinate system.

FIG. 6 is a diagram for explaining the principle of visual servo using two landmarks 16 and 17;

FIG. 7 is a diagram for explaining a coordinate system.

FIG. 8 is a diagram for explaining the principle underlying the present invention.

FIG. 9 is a diagram for explaining property 1 of the present invention.

FIG. 10 is a diagram for explaining property 2 of the present invention.

FIG. 11 is a diagram for explaining property 3 in the present invention.

FIG. 12 is a diagram for explaining property 3 in a state different from FIG. 11;

FIG. 13 is a flowchart for explaining an operation in the embodiment shown in FIGS. 1 to 12;

FIG. 14 is a diagram for explaining a specific method of visual servo using two landmarks 16 and 17 and explaining an operation for eliminating a deviation in the direction of the camera 10;

FIG. 15 is a view for explaining a specific method of visual servo using two landmarks 16 and 17,
FIG. 4 is a diagram for explaining an operation for eliminating a displacement of the moving body 1.

FIG. 16 is a flowchart for explaining the operation of another embodiment of the present invention.

FIG. 17 is a diagram showing a state in which a moving object follows a straight teaching path 47 in an experiment of the present inventor of one embodiment of the present invention.

FIG. 18 is a view showing another experimental result of the present inventor and showing a state when the teaching path 52 is a curved path.

FIG. 19 is a diagram showing a result of a long-distance autonomous traveling simulation of the present inventor when the moving route is a long distance.

FIG. 20 is a simplified plan view of a moving body 1 according to another embodiment of the present invention.

21 is a simplified plan view showing sensors M1 to M3 of the moving body 1 of the embodiment shown in FIG.

FIG. 22 is a block diagram showing an electrical configuration of the embodiment shown in FIGS. 20 and 21.

FIG. 23 is a flowchart illustrating an operation of the processing circuit 62 of FIG. 22;

24 is a diagram showing a result of an obstacle avoidance simulation as a result of an experiment performed by the present inventor in the examples of FIGS. 20 to 23. FIG.

FIG. 25 is a diagram for explaining a simplified configuration and a route thereof according to still another embodiment of the present invention.

FIG. 26 is a flowchart for explaining the operation of the embodiment in FIG. 27;

FIG. 27 is a simplified plan view of a moving body 1 according to another embodiment of the present invention.

[Explanation of symbols]

 Reference Signs List 1 moving body 2 moving body main body 3, 4 front wheel 5, 6 rear wheel 7 front end 8 turning axis 9 pan head 10 camera 15 vehicle rotation center 16, 17 landmark 18 route 19 image processing circuit 20 memory 21 processing circuit 22 turning drive Source 23 Steering means 63 Obstacle M1-M3 Obstacle detection sensor

──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Yasuo Otsuki 118 Futatsuka, Noda City, Chiba Prefecture, Kawasaki Heavy Industries, Ltd. Noda Plant (72) Inventor Nobuo Nagai 1-1-1, Kawasaki-cho, Akashi-shi, Hyogo Prefecture Kawasaki Heavy Industries (56) References JP-A-2-244206 (JP, A) JP-A-59-74905 (JP, A) (58) Fields investigated (Int. Cl. ⁶ , DB name) G05D 1/02 G06T 1/00

Claims (8)

(57) [Claims] At the time of teaching, a moving body is moved along a predetermined route, and a camera provided to be capable of angularly displacing at least two landmarks provided at a fixed position is fixed to the moving body. In this state, an image is taken by the camera within a single field of view, and for each landmark, the respective coordinate values of one axis V direction and the other axis H direction in the camera's orthogonal coordinate system are associated with each other,
The combination of the coordinate values is stored in a memory, and when the vehicle is running, the landmark is imaged in a single field of view by a camera, and the coordinate value of one of the landmarks obtained by imaging in the one-axis V direction is obtained. The coordinate value vt1 of the store content of the memory having the same v1 is read and searched, and the coordinate value vt2 in the one axis V direction of the other landmark in the combination including the searched coordinate value vt1 of one landmark is searched. Then, the camera is angularly displaced so that the coordinate value vt2 of the other landmark retrieved matches the coordinate value v2 of the other landmark imaged by the camera in the one-axis V direction. The image is taken such that the coordinate value h1 of the one landmark in the other axis H direction matches the coordinate value ht1 of the one landmark in the other axis H direction in the combination. A traveling control method for a moving body, wherein the vehicle is steered such that a coordinate value h2 of the other landmark in the other axis H direction coincides with a coordinate value ht2 of the other landmark in the other axis H direction. 2. A coordinate value h in the direction of the other axis H of one of the landmarks imaged during the steering.
1, a center point between the imaged other landmark and the coordinate value h2 in the other axis H direction, the one landmark in the combination, the coordinate value ht1 in the other axis H direction, and the other landmark in the combination. The difference Δ from the center value with the coordinate value ht2 in the other axis H direction
The traveling control method for a moving body according to claim 1, wherein the steering is performed so that h becomes zero. 3. The moving body is provided with a plurality of sensors for detecting obstacles in or near the teaching path, and the moving body is steered in a predetermined steering mode individually corresponding to a combination pattern of outputs of the sensors. Claim 1
Or the traveling control method of the moving object according to 2. 4. During teaching, the mobile unit travels along a predetermined route and images at least two landmarks provided at fixed positions in a single field of view by a camera fixed to the mobile unit. Then, a combination of coordinate values for each landmark, which is formed by associating each coordinate value of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera for each landmark, is set along the path. When the vehicle is running, the landmark is imaged in a single field of view by a camera, and the coordinate value v1 in the one-axis V direction of one of the landmarks obtained by imaging is the same. The coordinate value vt1 of the contents stored in the memory is read and searched, and the one axis V of the other landmark in the combination including the searched coordinate value vt1 of the one landmark is read.
The direction coordinate value vt2 is retrieved from the stored contents of the memory, and the retrieved coordinate value vt2 of the other landmark is retrieved.
And correcting the deviation of the direction of the moving body by steering so as to match the coordinate value v2 of the other landmark imaged by the camera in the one-axis V direction. Next, the one landmark imaged The coordinate value h1 of the other land H in the other axis H direction of the one landmark in the combination stored in the memory so as to match the coordinate value ht1 of the other land mark in the other axis H direction. Coordinate value h2 of the mark in the other axis H direction
Is adapted to correct the displacement of the moving body by steering so as to coincide with the coordinate value ht2 of the other landmark in the other axis H direction in the combination stored in the memory. Travel control method. 5. A moving body, (b) steering means provided on the moving body for steering, and (c) a camera mounted on the moving body and capable of being angularly displaced around a vertical axis. (D) angular displacement driving means for driving the camera angularly displaced around the vertical axis; and (e) a memory, wherein the camera is fixed to the movable body and the movable body is moved along a predetermined path. When the vehicle travels, a combination of coordinate values obtained by imaging at least two landmarks provided at fixed positions in a single field of view by a camera is stored, and the combination of coordinate values is stored for each landmark. A memory in which coordinate values of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera are associated with each other; and (f) control means, which responds to the output of the camera and captures an image by the camera. One land obtained The coordinate value vt1 of the store content of the memory having the same coordinate value v1 in the one-axis V direction of the mark is read and searched, and the other landmark in the combination including the searched coordinate value vt1 of one landmark is read. The coordinate value vt2 in the one-axis V direction is searched, and the coordinate value vt2 of the other landmark searched for is made coincident with the coordinate value v2 in the one-axis V direction of the other landmark captured by the camera. Then, the camera is angularly displaced by operating the angular displacement driving means. Next, the coordinate value h1 of one of the landmarks in the other axis H direction in the combination is the coordinate value ht1 of one of the landmarks in the other axis H direction in the combination. And the coordinate value h2 of the other captured landmark in the other axis H direction.
And a control means for operating the steering means so as to make the same as the coordinate value ht2 of the other landmark in the other axis H direction. 6. The control means controls the coordinate value h in the direction of the other axis H of one of the landmarks imaged by the steering means during the steering.
1, a center point between the imaged other landmark and the coordinate value h2 in the other axis H direction, the one landmark in the combination, the coordinate value ht1 in the other axis H direction, and the other landmark in the combination. The difference Δ from the center value with the coordinate value ht2 in the other axis H direction
6. The travel control device for a moving body according to claim 5, wherein the steering is performed so that h becomes zero. 7. A first sensor M1 for detecting a left front obstacle on an axis passing through the rotation center of the moving body, and detecting a front obstacle on an axis passing through the rotation center of the moving body. And a third sensor M3 for detecting a right front obstacle on an axis passing through the center of rotation of the moving body. The obstacle avoidance control means includes an output of the first to third sensors. When the first or third sensor detects an obstacle, steering is performed by the steering means so as to change the direction to the right or left by a predetermined first angle, respectively, and the second sensor detects the obstacle, 6. The steering device according to claim 5, wherein when the first and third sensors do not detect an obstacle, the steering unit steers to the right or left by a second angle larger than the first angle. 6
A travel control device for a moving object according to claim 1. 8. (a) a moving body, (b) steering means provided on the moving body for steering, (c) a camera fixed to the moving body, and (d) a memory, Each landmark obtained by imaging at least two landmarks provided at a fixed position in a single field of view by a camera when the mobile body travels along a predetermined route with the camera fixed to the body. A combination of coordinate values for each of the landmarks is stored along the path, and the combination of the coordinate values is stored in each of the landmarks in the coordinate system of one axis V direction and the other axis H direction in the orthogonal coordinate system of the camera. And (e) control means for responding to the output of the camera, wherein one of the landmarks obtained by imaging with the camera has the same coordinate value v1 in the one-axis V direction. Store contents of a certain memory The value vt1 is read and searched, and the coordinate value vt2 in the one-axis V direction of the other landmark in the combination including the searched coordinate value vt1 of the one landmark is searched from the contents stored in the memory. Coordinate value vt2 of the other landmark
And the coordinate value v2 in the direction of the one axis V of the other landmark imaged by the camera matches,
The steering unit is operated to correct the deviation of the direction of the moving body. Next, the coordinate value h1 of the one landmark imaged in the direction of the other axis H in the one of the combinations in the combination stored in the memory is stored. The coordinate value h in the direction of the other axis H of the other landmark imaged so as to match the coordinate value ht1 in the direction of the other axis H of the landmark.
2 is a coordinate value ht2 of the other landmark in the other axis H direction in the combination stored in the memory.
Control means for correcting the displacement of the moving body by operating and steering the steering means so as to coincide with the following.