JPS6359670A - Off-position measuring instrument - Google Patents

Abstract

PURPOSE:To simply obtain traveling control information by comparing the three-dimensional position of an object obtained from a stereoscopic picture at the time of traveling a moving robot with the three-dimensional position of the object obtained from a preset spot, obtaining a position coordinate converting matrix and obtaining an off-position from the set spot. CONSTITUTION:A three-dimensional position detection part 2 stereoscopically calculates and processes the three-dimensional position from the position of the moving robot on the object extracted in an object extracting part 1 according to the stereoscopic picture. A quantity of off-position calculating part 3 compares the three-dimensional position of the object obtained from the stereoscopic picture with the three dimensional position of object groups to the preset spot, respectively, and the quantity of the misregistration from the preset spot of the position of the moving robot at the time of inputting said streoscopic picture is calculated from the quantity of the off-position of the three- dimensional position on the same object. The quantity of the off-position obtained in the quantity of off-position calculating part 3 is applied to the movement control part of the moving robot and the traveling thereof is controlled.

Description

[Detailed Description of the Invention] [Object of the Invention] (Industrial Application Field) The present invention is directed to inputting, for example, a positional deviation from a travel trajectory set for a mobile robot using a stereo camera mounted on the mobile robot. The present invention relates to a highly practical positional deviation measuring device that can be effectively measured from stereo images and used for driving control.

(Prior Art) In recent years, as cameras have become lighter and smaller and visual information processing technology has developed, the development of mobile robots with visual V1 ability is actively underway. Compared to conventional mobile robots that are close-sensory and cooperative, this type of mobile robot with visual ability can be equipped with advanced abilities such as being able to recognize only the moving VJ environment and setting its own VJ route. It is possible. Therefore, for example, in order to carry out maintenance inspection work at a nuclear power plant,
It is expected to be applied in fields where autonomous driving is required.

As a mobile robot equipped with a visual function, for example, one has been reported that controls its traveling route by comparing and comparing an image inputted by a camera with a preset feature model of the mobile environment. Furthermore, there have been reports of methods in which a stereo image input using a stereo camera is processed three-dimensionally to set a movement route without using the feature model of the movement environment.

However, the former type of mobile robot that uses a feature model of the moving environment can be applied to a simple environment where the feature model can be easily modeled, but in an environment that includes a complex background image, for example, the feature model There were problems such as unnecessarily complicating the process of converting images and comparing and matching input images.

On the other hand, with the latter type of mobile robot that inputs stereo images and processes them in 3D, there is no need to create an environment model in advance. Requires manual processing. Furthermore, it takes a considerable amount of time to complete the three-dimensional calculation of a stereo image that inputs various environmental information. For this reason, there has been a problem in that it is difficult to control the mobile robot to move quickly and smoothly.

(Problems to be Solved by the Invention) As mentioned above, the present invention solves problems such as the visual function built into conventional mobile robots being only compatible with simple environments or requiring a large amount of time to process. This was done in view of the fact that it is impossible to expect smooth running control of a mobile robot, and at the same time, it is extremely rare for a mobile robot to run in a completely free environment. We provide a displacement measurement table that uses electric functions to quickly detect the information necessary for travel control, that is, positional deviation from the set travel trajectory, and use it effectively for travel control. It is intended to.

[Structure of the Invention] (Means for Solving the Problems) The present invention inputs a stereo image of an environment including a target object using a stereo camera mounted on a running body such as a mobile robot, and uses the input stereo image. A plurality of pre-registered targets are each extracted from among them. The three-dimensional position of each of these targets from the local point, that is, the three-dimensional position of the target group from the position indicating the travel trajectory set when the stereo image was input, is then compared, and the comparison results are calculated. The present invention is characterized in that a positional shift between a point where the stereo image of the traveling object is input and a predetermined point is calculated according to the following.

(Operation) According to the present invention, the three-dimensional position of the target from the above-mentioned local point, which is determined from the stereo image input at the position of the mobile robot, and the three-dimensional position of the target from the preset point of the mobile robot. By comparing the three-dimensional position of the target object, the deviation of the three-dimensional position of the target object due to the difference in the viewpoint position is determined. This makes it possible to quickly determine the deviation of the current position of the self-propelled robot with respect to the travel route that has been taken. Therefore, according to this positional deviation information, the traveling position of the mobile robot is corrected with respect to the traveling route.
In other words, it becomes possible to carry out the driving control effectively.

(Example of Solid Lines) Hereinafter, an embodiment of the present invention will be described and roughly divided into positional deviation measuring units 3 with reference to the drawings.

The target object extraction unit 1 extracts a target object, such as a curved corner or a corner of a working space, in the environment in which the mobile robot runs from a stereo image inputted by a stereo camera mounted on the mobile robot. It is something to extract. The three-dimensional position detection unit 2 performs stereo calculation processing on the target object extracted by the target object extraction unit 1 from the position of the mobile robot at the time when the stereo image is input, according to the stereo image; Specifically, the needle ring is set according to the principle of triangulation method. Then, the calculation unit 3 compares the three-dimensional position of the target obtained from the stereo image with the three-dimensional position of the target object group with respect to a preset point, and calculates the positional deviation by
The displacement of the position of the mobile robot from a preset point at the time when the stereo image is input from the dimensional position displacement is calculated.

The amount of positional deviation determined by the calculating section 3 is given to the movement control section of the mobile robot, and the movement of the robot is controlled.

Now, FIG. 2 is a diagram showing an example of the configuration of the target object extraction section 1. As shown in FIG. The stereo camera mounted on the mobile robot consists of, for example, two cameras installed at a predetermined distance apart in the horizontal direction, that is, a left camera 5a and a right camera 5b, and a switch circuit 7 controlled by a control unit 6. Each scene is inputted to the image input section 8 via. The image input unit 8 receives the two left and right images ii! A stereo image consisting of an i-image is stored in an image memory prepared inside and subjected to the image processing described below.

The preprocessing unit 11 then processes the stereo ii! stored in the image memory! For example, random M sound removal by spatial filtering, shading correction processing, etc. are performed on the J image. The feature extraction unit 12 extracts image features corresponding to the type of target object from the preprocessed stereo image.

For example, if the input stereo image consists of an environment showing a corridor (passageway) as shown in Fig. 3(a), the target object A may be the intersection between the wall and the floor, the intersection between the corner of a pillar and the floor, etc. Set. For such 24 cases, if line segments are extracted as features of the image, then the input image is subjected to differential processing,
The differential output is 2)! The line segment image shown in FIG. 3(1)) may be obtained by

Feature 1 of images that can be obtained in this way! The target object position detection unit 13 detects a preset target object from the N-minute images as shown in FIG. 3(C).

To detect this target object, for example, a characteristic pattern related to the target object as shown in FIG. 4 may be prepared, and matching processing may be performed on the line segment image shown in the above-mentioned M3 diagram (1)). Specifically, a partial image is cut out from the line segment image shown in FIG. 3(b), and it is examined whether this partial image conforms to the characteristic pattern shown in FIG. 4. This process is repeated while sequentially changing the cutout position of the partial images, and when the patterns match, this is determined as target object detection,
Moreover, the cutout position of the partial image at that time may be used as the pressure detection position of the target object.

The target object detection information detected from the stereo image in this way is stored in the target object position memory 14. Furthermore, the number of targets is not limited to one. ! There may be an arithmetic class. In this case, the detected position information for each target object may be stored in the target object position memory 14, respectively.

On the other hand, the three-dimensional position detecting section 2 is configured as shown in FIG. 5, for example. This three-dimensional position Rfft-J section 2
(The block shown in Fig. 5) creates calibration data using a reference target (mark) whose three-dimensional position is known in advance, and calculates the three-dimensional position of target A detected as described above according to this calibration data. It is to be calculated.

That is, the mark position detection section 2) detects preset reference targets (marks) from the two left and right stereo images stored in the image memory of the image input section 8. This mark is defined, for example, at a plurality of locations as an easily identifiable object in the input image. Then, the marks are correlated based on the 11 yen relationship between the marks in the left and right images, and the calibration section 22 calculates the stereo parameters necessary for the three-dimensional position calculation described in 1) as correction data. ing.

In other words, from the correspondence of each mark in the stereo image, stereo parameters are determined that specify the three-dimensional positions between the corresponding points in the stereo image. This calibration data (parameters) is stored in the calibration data memory 23.

The stereo calculation unit 24 uses the stereo parameters to calculate the three-dimensional position of the pixel target from the position of the target in the stereo image stored in the target object memory 14, and calculates the three-dimensional position of the pixel target. The three-dimensional position of the target object is stored in the three-dimensional coordinate memory 25. Therefore, the three-dimensional position of the target object stored in the three-dimensional coordinate memory 25 is the three-dimensional position (the current position of the mobile robot at the time when approximately 2 stereo images were input) as the 14th standard.
X, V, Z).

Now, the positional deviation calculating section 3 is configured as shown in FIG. 6, for example.

The target object model coordinate memory 31 stores in advance three-dimensional position data of the target object as seen from each point on a preset traveling route of the mobile robot.

The three-dimensional position data of the target stored in the target object model coordinate memory 31 can be obtained by, for example, making a mobile robot run on the above-mentioned travel route, and calculating the three-dimensional position data (X, Y, Z) of the target that is obtained at that time. ) etc.

The conversion matrix calculator n section 32 stores the target object model coordinate memory 3.
The three-dimensional position [(X, Y, '; Based on the three-dimensional position (x, y, z) of the object, a coordinate transformation matrix between the windowed point and the local point is calculated.

That is, as shown in FIG. 7, the point on the travel path P determined for the mobile robot moving on a plane is Q, and the local point where the mobile robot moves on its own and obtained the stereo image is R. , the translation between them is V (p
.

Then, the following relationship is established between the coordinates of the three-dimensional position determined for the same target object. However, α=−pcosθ−q sinθβ=pSinθ−qcosθ. A matrix indicating the relationship between these coordinate values is the coordinate transformation matrix T.

The transformation matrix calculation unit 32 calculates the coordinate transformation matrix T by combining the above-mentioned equations from the three-dimensional position information regarding at least two targets obtained from the stereo image and solving the equations. Then, the positional deviation output unit 33 calculates the positional deviation (p, q, r, θ) of the local point R of the mobile robot with respect to the windowed point Q from the coordinate transformation matrix T, and calculates the positional deviation (p, q, r, θ) It is output to the motion control section 34 of the mobile robot.

In addition, when two or more sets of targets are extracted from a stereo image, even if the principle of least squares method is adopted, for example, the least squares method is applied, even if the target objects are out of the field of view of the stereo camera, at least two sets of targets are extracted from the above field of view. Needless to say, if it remains within the range, it is possible to find the positional deviation as described above.

As described above, according to the present invention, from stereo images input by a stereo camera mounted on a mobile robot,
The displacement of the mobile robot from its original traveling position can be determined very easily. Moreover, since the positional deviation is calculated only by stereo calculations for several targets, there is no need for the corresponding numbering of each part of the stereo image or the stereo needle calculation for all of these parts, which is required with conventional devices. . As a result, the calculation process can be executed efficiently and at high speed. Furthermore, by selecting only objects whose features can be easily extracted as targets, it is possible to prevent the image processing from becoming unnecessarily complicated, improving the total Ll 24 degrees for the target, and increasing the reliability of positional deviation detection. It will be possible to improve the level of performance.

Therefore, it can be effectively applied to the traveling control of a mobile robot, and it can be appropriately employed to fully utilize its visual abilities. In addition, the present invention can be modified and practiced in various ways without departing from the gist thereof.

C. Effects of the Invention] As explained above, according to the present invention, the three-dimensional position of the target obtained from the stereo image during the actual running of the mobile robot and the three-dimensional position of the target obtained from the preset point , calculate the position coordinate transformation matrix, and convert the above stereo image to Information can be obtained easily, quickly, and with high precision.Therefore, a great effect can be achieved as a visual function of a mobile robot.

[Brief explanation of drawings]

Fig. 1 is a schematic diagram of the construction of a device for producing 7J! ! Figures 3 and 4 show the concept of the target object extraction process, and Figure 5 shows the three-dimensional position detection unit in the example device. FIG. 6 is a diagram showing an example of the configuration of the positional deviation agricultural bill fraud section in the embodiment. 1...Target extraction section, 2...Three-dimensional position detection Part, 3
... Position shifter St calculation section, 5a, 5b... Stereo camera, 8... Image input section, 11... Preprocessing section,
12... waiting microextraction section, 13... target position detection section,
14... Target position memory, 2)... Mark position detection section, 22... Configuration [sieve section, 23... Calibration data memory, 24... Stereo calculation section, 25... 3 Dimensional coordinate memory, 31...Target model coordinate memory, 32
. . . Transformation matrix total n parts, 33 . . . Position shift output part. Taketachi Applicant Director of the Agency of Industrial Science and Technology Tatsu Todoroki Figure 1, 7, 2, 4, 5/-Figure 7 Procedures Amendment (Method) 1988
lσ Month 9 Motion '1 Director-General of the Office 1, Indication of the case Japanese Patent Application No. 61-109577 2, Name of the invention positional deviation measuring device 3, Person making the amendment Relationship with the case Patent applicant's address Same as above ゛ Name Agency of Industrial Science and Technology Research and Development Miyake Telephone 03(501)1511 Extensions 4611-24
, Date of the amendment order: September 22, 1988 (shipment date) 5. Column 6 for a brief explanation of the drawings in the specification subject to the amendment, Page 16, lines 9 to 11 of the statement of contents of the amendment. , ``Figure 6 is a diagram showing an example of the configuration of the positional deviation amount calculating section in the embodiment device.'' , FIG. 7 is a diagram for explaining the state of positional deviation.''that's all

Claims (2)

[Claims] (1) Means for inputting a stereo image of an environment including a target object using a stereo camera mounted on a traveling body; means for extracting a plurality of pre-registered targets from the input stereo image; means for determining the three-dimensional position of each target object from a point where the stereo image of the body is inputted by stereo calculation; means for comparing the three-dimensional positions of the respective targets, and means for calculating a positional deviation between a point where the stereo image of the traveling object is input and a predetermined point according to the comparison result. A positional deviation measuring device characterized by comprising: (2) The positional deviation measuring device according to claim 1, wherein the traveling object is a mobile robot, and the predetermined point indicates a point on a traveling trajectory set for the mobile robot. .