JPS63186308A - Method and device for guiding mobile object - Google Patents

Abstract

PURPOSE:To produce an accurate map from a map using the position information on an encoder, etc., in order to obtain the shift value of a mobile object, by utilizing the matching result among a distance picture and an original picture of the current time point and the original picture set previously by a certain time point. CONSTITUTION:A distance picture generating part 14 calculates the distance information at a current time point (t) from the right and left picture PRt and PLt received from a TV camera 11. Then an inter-picture matching part 13 searches the specific picture elements of the original picture PRt of the time point (t) which each picture element of the original picture PRt-1 corresponds. In this searching, an optical floor algorithm is used for calculation of a specific direction in which each picture element moved. This calculated optical floor pattern shown the picture element positions of the picture PRt to which each picture element of the picture PRt-1 corresponds by means of arrow heads. Thus the matching is secured between picture together with production of the distance picture. Then a shift value calculation part 15 calculates the shift value of a mobile object. A map replacement part 16 uses said shift for conversion of the space position coordinate and adds this converted coordinates to a map storage part 17. While a mobile object guidance controller 18 calculates a guiding route of the mobile object.

Description

[Detailed Description of the Invention] [Industrial Application Field] The present invention relates to a method and device for guiding a moving body using distance information generated based on images input from a TV camera, etc. The present invention relates to a method and apparatus for guiding a moving object, which generates a map necessary for guidance from distance information that changes as the distance changes.

[Conventional technology]

The function of capturing three-dimensional positional coordinate information of the environment is essential for viewing devices such as industrial robots, and the robot's eyes interpret images based on brightness information obtained from TV cameras. A device that determines the shape of an object based on the degree of bending of a slit light source, or
There are devices that measure the distance to a target object using a laser beam and recognize the object based on the distance information.

For example, ``information processing in cartographic visual systems.

The method described in Journal of the Robotics Society of Japan, June 1.984, pp. 15-23'' creates a map based on ranging data obtained from external sensors for the purpose of guiding moving objects. are doing.

This method uses distance measurement data obtained from external sensors such as laser beams, interpolation between distance measurement points using a spline function, and past topographical information (map) to create an ever-precise map. It will be updated as follows.

In this case, what is required is the amount of conversion (amount of rotation/translation) required to add the current distance measurement data to the past topographical information, and in this method, this amount of conversion can be accurately obtained using a sensor such as an encoder. It is also assumed that

However, in reality, an error amount is always added to this converted amount, so as time passes, the value becomes far different from the actual amount, and the accumulated map becomes inaccurate. Problem] In the above-mentioned conventional technology, no consideration was given to the amount of error in the position information of the moving body by the encoder or the like.

That is, based on the assumption that position information of a moving object can be obtained accurately, the ability to correspond between distance measurement data obtained over time and past accumulated map information has been utilized.

For this reason, the accuracy of the obtained maps varied.

The purpose of the present invention is to improve such problems, to obtain a more accurate movement amount of a moving object without depending on the position information of the moving object including the error amount of the encoder, and to calculate the amount of movement of the moving object in a time-series manner. It is an object of the present invention to provide a method and device for guiding a moving body that can generate a map that is resistant to instability in the accuracy of correspondence between a plurality of images obtained.

[Means for solving problems]

In order to achieve the above object, the moving object guidance method of the present invention uses distance information generated based on images input from a TV camera and matching results between images at each time to guide the moving object. By calculating the amount of movement (rotation, translation) using the least squares method and considering the correspondence between multiple images, we improve the accuracy and add distance information at each time to the stored map. There are characteristics.

The moving body guidance device of the present invention also includes means for matching an original image inputted from a TV camera at the current time with an original image inputted one time ago (image matching section), and an input at each time. A means for calculating the amount of movement of the moving object by the least squares method using distance information calculated based on the image and the matching result of the original image; It is characterized by having means (map update section) for updating the map using map information from one hour ago.

[For production]

In the present invention, the amount of movement is calculated using a distance image obtained as a three-dimensional coordinate position with the viewpoint of the moving object as the origin, and a matching processing result between an image at the current time and an image one time ago, and Maps are generated by considering the correspondence between multiple images obtained in chronological order.

For this reason, it is possible to generate a more accurate map than conventional map updating using sensors such as encoders whose errors increase with time.

〔Example〕

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a configuration diagram of a moving body guiding device in an embodiment of the present invention.

Transfer of this example! The IJ body guidance device includes a TV camera 11, an image input unit 12, an image matching unit 13, a distance image generation unit 14, a movement amount calculation unit 15, a map update unit 1G, a map storage unit I7, and a moving body guidance control unit. - Equipped with rollers 18, each part has the following configuration and function.

Two TV cameras are equipped to capture the real environment.
Further, the image input section 12 is an A/D converter.

Equipped with image memory, etc.

The distance image generation unit 14 calculates distance information based on the image input from the TV right camera 1. This distance information is distance information for each pixel of the original image, and a two-dimensional array of this distance information is a distance image.

Regarding the second calculation method, please refer to the patent application No. 59-237G.
This method is similar to the method described in No. OQ, and the inter-image matching unit 13 generates a distance from two images input from the left and right cameras, and takes correspondence between images input at each time.

The movement amount calculating section 15 calculates the movement amount of the moving object using the output results of the distance image generation section 14 and the inter-image makonoting section 13.

The map update unit 16 converts the distance information obtained from the distance image crowd at each time using the movement amount calculated by the movement amount calculation unit 15, that is, the spatial position coordinates, and updates the map stored in the map storage unit 17. Add to.

The mobile object guidance controller 18 uses the generated map information to calculate a guidance route for the mobile object.

The moving object guidance device configured as described above obtains the accurate movement distance of the moving object, updates and stores the map, and guides the moving object.

FIG. 3 is an explanatory diagram of mobile environment settings in one embodiment of the present invention.

In the moving object guidance device of this embodiment, when setting the operating environment of the moving object at time t, as shown in the plan view (i), there are a long rope, a shelf at the back right of the hallway, a shelf, and an obstacle. The space where the object box is placed is set as the operating environment. Note that the dot , the point T--1, and the arrow indicate the position of the moving object at the current time, the position one time ago, and the line of sight, respectively.

In addition, the original image obtained by dotting will be shown as shown in (11).Furthermore, find the correspondence between each pixel at the current time and each pixel at one time ago, and form each pixel like (a mountain). Calculate in which direction the pixel has moved.

FIG. 2 is a map update processing flowchart in one embodiment of the present invention, FIG. 4 is an explanatory diagram showing the coordinate system at each time in one embodiment of the present invention, and FIG. 5 is a movement diagram in one embodiment of the present invention. As shown in FIG. 2 and FIG. 3, which are side views of the map generated by the body guidance device, LI!
Regarding the process of update processing in Figure 1, first, at the current time t, the TV
Left and right images PL(t) input from right camera 1, and P
Using R(t) (200), distance information is calculated in the distance image generation unit 14 (201).

In this case, the measurement standard is the input left and right images,
Calculate centering on the right image.

The calculation result represents the three-dimensional position in real space indicated by each pixel using a coordinate system with the viewpoint as the origin.

That is, for each pixel P(it J) in the original image PR(t), the distance information 2 for each pixel is
Using the distance image D(t), which is a dimensional array, it is possible to obtain the spatial coordinate (x+'It 7) indicated by the pixel, P
R(J:9P (l, J)l−+(x, y r z
)εD(t).

Next, in the inter-image matching unit 13, the original image PR (t-1) at time t-1, which is one hour ago, and the current time t
The correspondence with the original picture PR(t) is determined (202).

That is, a process is performed to search to which pixel in the original picture PR(t) at the current time each pixel of the original picture PR(t-1) corresponds.

Regarding the search, please refer to "Artificial Intelligence 17. 1981, pp. 185-203.
Page (Artificjal Tntelligenc
e, Vnl.

17.1981. , pp. 185-203)" is used. In this method, in the two original images, each pixel is Calculate which direction it moved.

The optical flow pattern calculated in this way is the third
As shown in Figure (iii), each arrow indicates which pixel position in PR(t) each pixel of PR(t-1) corresponds to.

When these two processes, that is, distance image generation and matching between images, are executed, the movement amount calculation unit 15
At 203L, a movement amount calculation process is performed (203L). In this case, a point P' in the original image PR (t-1) at time t-1
−” (+ l J) is the three-dimensional position @ p 'Cr − in the coordinate system whose origin is the viewpoint at this time.
1(x, y'+z) corresponds.

Also, P'-1(i l J) is the original picture PR(t
) The distance to which the point corresponds depends on the result of the inter-image matching process.
For example, (ul, (i, j L vt, l, j
)), so Pt(i
＋＋＋6 (+ l JLj+V, (+ ; j)) Moreover, the position coordinates of this point pt in real space are P(xl, y' ,y
′) is obtained.

Therefore, based on this position information, we calculate the amount of movement (amount of rotation).

#L base quantity) is obtained by L7 as follows.

That is, as shown in FIG. 4, for the coordinate system 0t-1-Xcar1, -1, z'o-1 at time t-1, X, +, axis, Y6-1I! It, Zb-II
If the rotation amounts for each of II are α, β, and γ, and the translation amounts are ΔX, Δy, and ΔL, in a minute time, P
−” +(Q, x ptl −” +V) Niyo IJ
This is the point corresponding to the point * RO A KA P”.

However, V = (ΔX, Δy, Δl), and to = (α, β
, γ).

Therefore, the unknown quantities Q and V are determined by the method of least squares as follows.

First, in calculating the amount of movement 0 and V, the target point is a point that has good accuracy in distance image calculation and high reliability of optical flow, that is, a point with a brightness of 2. A point where the order differential value is 0 is used.

From those points, the set of points targeted at time t-1 is
(PH-' (x, y, z): 4.= 1....
・.

n), and the set of points at the corresponding time C is (P4(x'
Iy'Hz'):='r...+n), then E= u 1+ p stem-(P+:'+Ω×PR-
1+v) Find a and ■ when the quantity E shown by the phase equation I2 is minimum.

Note that ■・U is the norm, and its formula can be written as Pλ(x'
, y' , y,' ), P4 (X4 v ykl 7
4), Q, = (α, β, γ), V = (△X, Δy, △
Specifically expressed as Z), E=Σ[(x4-(x4+β・vh-γ・y++ΔX)
) 2 circles + (to y - (gap + γ・X eichi α・7 Haruka + Δy)) 2 + (
Yi(唖+α・f−β・輛+△z))2].

Based on Q and V obtained in this way, the past accumulated map information, that is, the point coordinates, can be converted to the current coordinate system, or conversely, the coordinate system at the initial time 1 = 0 is used as the world coordinate system, and each time Transform the point coordinates in .

In other words, if we set At-t=(Ω, V) as the amount of conversion Ω from time t-1 to Mu, and ■, then smell=Heu-1-P
f/! , (danger = 1...., n), the map M at the current time t is Mf, = At, -
1 ・Defined as M car 1 + D (hi).

The meaning of this formula is that all points stored in the map M6-1 at time t-1 are transformed by transformation aAf,-1, and the set D(t) of spatial points obtained at time t is added. This indicates that the set of points obtained is the map M at the current time t.

If the conversion amount A is -0, a certain function value δ is determined, and il PA −At, −I P, e, l > δ(i=
1. ..., n) Naru vs. Mindfulness P! , and X-t
Accuracy may be increased by recalculating by excluding .

Furthermore, in order to improve the accuracy of the conversion amount At, for example, excluding sudden changes such as changing the moving direction of the moving object (TV force, direction of the camera) to the right, the corresponding points are calculated over n images. It is also possible to modify the amount of transformation using the set, that is, by determining a certain f direction δ′,
The conversion amount A is reconstructed by excluding the corresponding selling points P and pt-n that satisfy the condition "pt-nll>δ'.

However, also=1. ..., n', and A'mi A, -□
.

At-2,...+At+-n.

In this way, even if the accuracy of the range image and the accuracy of matching between images are unstable, the three-dimensional map necessary for guiding moving objects can be created by the reconstructed transformation amount as shown in Figure 5. , is constructed and stored as a set of points (204,
205).

〔Effect of the invention〕

According to the present invention, the amount of movement of a moving object is determined by using a distance image at the current time and the matching result between the original image at the current time and the original image one time ago, so an encoder that always has an error amount is used. It is possible to generate more accurate maps compared to map generation using positional information such as can be described in terms of

Furthermore, by correcting the amount of movement gathered at each time across multiple images and generating a map, we can generate a map that is resistant to instability in the accuracy of distance images obtained at each time and the accuracy of the correspondence between images. It can be carried out.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of a moving object guidance device in an embodiment of the present invention, FIG. 2 is a flowchart of a map update process in an embodiment of the present invention, and FIG. 3 is a flowchart of a moving environment setting in an embodiment of the present invention. An explanatory diagram, FIG. 4 is an explanatory diagram showing a coordinate system at each time in an embodiment of the present invention, and FIG. 5 is a map generated by a moving body guiding device according to an embodiment of the present invention. +]: TV camera, 12: Image input unit, 13: Image matching unit, 14: Distance image generation unit, 15: Movement amount calculation unit 216: Map update unit, 17: Map storage unit, 18: For guiding moving objects Controller, r-ca. T-t, -t: position of moving object, P: pixel, rt, β, γ
: Rotation amount, △X, Δy, ΔZ: Translation amount, O, Of, , O
g-1: Viewpoint of moving object (origin L
zt l Xt −1+”+, −1t Z+J−,,
x, v, Z: coordinate axes. Figure 1 Figure 2 Figure 3 (iiJ
(iii) Figure 4 tl

Claims (1)

[Claims] 1. Measure distance and calculate distance information based on input images from a TV camera without having prior knowledge of the moving environment, and map the moving environment based on the distance information. generate as,
In the method for guiding a mobile object that stores the generated map, an original image input at the current time is matched with an original image input one hour ago, and the matching processing result is used from the stored map information one hour ago. , the amount of movement of the moving object is determined using the least squares method on distance information between times, and the map is updated by comparing the amount of movement at each time between multiple images. A method for guiding a moving object. 2. A plurality of TV cameras and means for calculating distance information by measuring distance based on input images from the TV cameras;
In a guidance device for a moving object having no prior knowledge of the moving environment, the device includes means for generating a map of the moving environment based on the distance information, and means for accumulating the generated map, and inputs the current time. Means for matching the original image inputted one time ago with the original image input one time ago, and using the distance information obtained at each time and the matching result, the amount of movement of the moving object is determined by distance information between times. A mobile object characterized in that it is equipped with means for calculating by least squares method, and means for updating a map using the amount of movement and map information stored one time ago in the storage means. Guidance device.