JP2002288637A - Environmental information forming method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an environmental information forming method, capable of simplifying formation of environmental information of a robot and displaying it to an operator in easy to understand manner. SOLUTION: This environmental information forming method includes a process of find a distance data from a camera of each point of an object surface in a front image photographed by a camera loaded on a moving body, a process to extract the object by clustering-processing the distance date, a process to form an obstruction data base by memorizing a data of an object cluster in time series and a processing process for displaying the positional relation of the object in the visual field of the camera, the object out of the visual field of the camera and the moving body in accordance with information of the object accumulated in the obstruction data base on the same environment display screen. Consequently, since it is possible to control past history of the object information, including the distance information provided from the image information from a CCD camera, etc., and visually display them on the screen of the display device, it is possible for the operator to know also obstruction information out of the visual field of the camera to photograph the front of the robot.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method for remotely controlling a moving object such as a robot based on image information displayed on an image display device such as a CRT. The present invention relates to a method for creating environmental information for providing.

[0002]

2. Description of the Related Art A video camera mounted on a robot displays an image of the front of the robot on an image display device such as a CRT or a liquid crystal panel. The narrow viewing angle of the camera, the difficulty in grasping the distance, and the like are raised. In other words, it is extremely difficult to accurately judge the situation around the robot only from the image on the image display device using the video camera, which has been one factor that hinders the practical use of the remote control robot.

It is conceivable to use distance information obtained by using an ultrasonic sensor or a laser radar, etc.
The distance information obtained by ultrasonic sensors or laser radar is basically only point information on obstacles. Even if this is displayed on the monitor as it is, the operator can grasp the surrounding situation of the robot from the image It is extremely difficult to do.

Also, the ultrasonic sensor has a relatively high directivity, and in order to recognize the entire circumference of the robot, a large number of sensors are mounted around the robot so as to emit sound waves in a radial manner, or the sensors are moved so as to emit sound waves radially. Since scanning must be performed, the processing is complicated and the system tends to be expensive.

[0005]

SUMMARY OF THE INVENTION The present invention has been devised to solve such problems of the prior art.
A main object of the present invention is to provide a method for creating environment information that simplifies creation of environment information of a robot and that can be displayed in a manner that is easy for a pilot to understand.

[0006]

In order to achieve the above object, in the present invention, distance data from a camera for each point on the surface of an object in a forward image picked up by a camera mounted on a moving object is obtained. Obtaining the object, extracting the object by performing a clustering process on the distance data, storing the object cluster data in chronological order to create an obstacle database, and acquiring information on the object stored in the obstacle database. Displaying a positional relationship between an object in the field of view of the camera, an object outside the field of view of the camera, and the moving object on the same environment display screen based on the information processing method. I decided that.

In this manner, the past history of object information including the distance information obtained from the image information from the CCD camera or the like can be managed and visually displayed on the screen of the display device. The person can also know the obstacle information outside the field of view of the camera that images the front of the robot.

In particular, the method may include a process of stereoscopically displaying each object on an environment display screen, a process of mapping brightness information obtained from a camera onto a three-dimensional model of the object, and a plurality of processes. By including a process of displaying the same object displayed in a different form on the display screen in the same color, the situation around the robot can be more easily understood to the operator.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described below in detail with reference to the accompanying drawings.

As shown in FIG. 1, an apparatus to which an environmental information processing method according to the present invention is applied is an image data output apparatus 1 such as a CCD camera mounted on a robot as a moving body.
And an obstacle data processing device 2 that extracts and stores obstacle data from input image data, and an environment display device 3 that visually indicates the position of the obstacle.

The processing steps in this apparatus will be described below with reference to FIG. First, a brightness image obtained by capturing the obstacles 10a to 10e as shown in FIG. 3 is obtained in time series by a binocular stereoscopic vision system or the like that captures images of two cameras (step 1), and a three-dimensional image for each pixel is obtained. Obtain distance data (step 2). And this distance data,
For example, clustering of obstacles is performed by performing histogram processing (step 3). The image of the camera in this case is represented by a perspective view in which one point at infinity is a vanishing point. As the robot moves forward, the image shown in FIG. Obstacles 10a to 10c that have changed from the viewpoint and are out of the field of view of the camera are not displayed on the screen.

Next, pixels within the range of the obstacle cluster are extracted from the image (step 4), and three-dimensional distance data for each pixel is made to correspond to the extracted pixels, thereby obtaining three-dimensional data of each obstacle (step 5). ).

Further, an obstacle database formed by storing the average position and distribution of each obstacle cluster, rectangular data on an image, three-dimensional data, and the like in a display storage device is provided.
The map is sequentially updated according to the progress of the robot, and thereby an obstacle map on the path of the robot is created (step 6).

Based on the data, image processing is performed so that the operator can easily grasp the surroundings of the robot, and the result is displayed on an environmental image display device such as a CRT (step 7).

At this time, the obstacles 10d and 10e within the viewing angle of the image photographed by the camera mounted on the robot and the three-dimensional contour images of the obstacles 10b and 10c located outside the field of view are simultaneously displayed on one screen. By displaying, the pilot can
The presence of an object outside the viewing angle range that can be captured by the camera can be easily grasped (see FIG. 4). In addition, by simultaneously displaying the image of the raw external environment captured by the camera and the image that has been subjected to the image processing to the operator, remote control can be performed more easily. In FIG. 4, the outline of the obstacle is displayed only by a line surrounded by a rectangle in order to simplify the processing, but not only such an outline but also a camera is displayed. Since the brightness information is simultaneously obtained from the actually photographed image, by adding the brightness information to the three-dimensional distance data for each pixel (texture mapping), a more intuitive three-dimensional display can be performed.

Naturally, since the two-dimensional position of the obstacle is also extracted, the data obtained on the path through which the robot has passed based on its own visual information is stored as a history on an environment map. Thus, the route map of the robot can be created together with the movement and displayed as a planar map.

[0017] Further, by performing color coding according to the distance based on the distance data, for example, red for a short distance and blue for a long distance, the sense of distance can be easily grasped. By aligning the display color and the display color of the environmental image, it is possible to easily recognize the sameness of the two, and it is also possible to easily grasp the relative positional relationship between the robot and the obstacle.

Next, an example of the extent to which the data of an obstacle is to be retained as a history as a history will be described with reference to a method of obtaining the distance accuracy of a binocular stereoscopic system as a reference.

The distance accuracy Δr of the binocular stereoscopic system is generally obtained by the following equation.

Δr = Δd (ur ² / bf)

Here, Δd: parallax resolution (for example, 0.1 when calculating parallax up to 1/10 sub-pixel of one pixel), u: size of one pixel, r: distance from viewpoint, b: 2 The distance between the two cameras (the lines of sight of the two cameras are parallel to each other) and f: focal length.

For example, u is 9.9 μm, b is 70 mm,
If f is 6 mm and Δd is 1, the relationship between r and Δr is as shown in FIG.

Therefore, for example, when the allowable error is assumed to be 50 cm, r is about 4.6 m. Therefore, it is practically set to be about 4.6 m in radius with the center of the robot as the history range. Become. Then, based on this value, the history range may be determined in consideration of the traveling speed and acceleration ability of the robot and the processing speed of the CPU.

In order to obtain the distance data, not only the binocular stereoscopic system but also, for example, Japanese Patent Application Laid-Open No. 60-85897.
The technology disclosed in Japanese Patent Application Laid-Open Publication No. H10-26095 can also be applied. Needless to say, the present invention can be applied outdoors.

[0025]

As described above, according to the present invention, it is possible to visually display an obstacle outside the field of view of a camera for imaging the front, and to easily and accurately remotely control a situation around the robot. Therefore, the remote control of the robot can be further facilitated, and a great effect can be achieved.

[Brief description of the drawings]

FIG. 1 is a block diagram of a device configuration to which the present invention is applied.

FIG. 2 is a processing flowchart of the present invention.

FIG. 3 is a conceptual diagram of a screen displaying an image captured by a camera.

FIG. 4 is a conceptual diagram of a screen displaying a result of data processing of an image.

FIG. 5 is a relationship diagram between the distance accuracy Δr of the binocular stereoscopic vision system and the distance from the lens.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Image data processing device 2 Obstacle data processing device 3 Display device 10a-10e Obstacle

Claims (4)

[Claims] 1. A process of obtaining distance data from a camera for each point on the surface of an object in a forward image picked up by a camera mounted on a moving object, and extracting an object in the image by performing a clustering process on the distance data. Creating an obstacle database by chronologically storing the data of the object cluster, and creating an obstacle database based on the information of the objects stored in the obstacle database. Displaying a positional relationship between the object and the moving object on the same environment display screen. 2. The environment information creating method according to claim 1, further comprising a process of stereoscopically displaying each object on an environment display screen. 3. The method according to claim 1, further comprising the step of mapping brightness information obtained from the camera to a three-dimensional model of the object. 4. The environment information creating method according to claim 1, further comprising a process of displaying the same object displayed in a different form on a plurality of display screens in the same color. .