JPS6371604A - System for detecting road boarder and obstacle by using area-divided color image - Google Patents

Abstract

PURPOSE:To detect an obstacle on a road by utilizing a blind person guide robot and an outdoor moving robot for road and obstacle detection by detecting the road borders and obstacle by using an area-divided color image process. CONSTITUTION:Three image planes of R, G and B of a color TV camera 3 are digitized and differential operators are put in operation for the respective image planes of R, G, and B to form differentiated images. The differentiated images of R, G, and B are added together to obtain a monochromatic differentiated image. Then the original image is divided into areas where the differentiated image exceeds a threshold value. Then the means values of the center coordiantes, length, and width RGB value of each area are calculated and an area which has a gray RGV value and its center coordiantes in the lower half of the image plane and is large in area is regarded as a road candidate area. The contour of the road candidate area is approximated with a directed segment to obtain a road border observation straight line 4. Then an area which straddles the road border observation straight line and differs in RGB value from a road area is extracted as an obstacle part area 5.

Description

DETAILED DESCRIPTION OF THE INVENTION [Field of Application of the Invention] The present invention is used to detect roads and obstacles for blind walking guidance robots and outdoor mobile robots using a TV camera.

[Object of the Invention] It is known that when a robot moves on a general road or sidewalk, it can be controlled so that it moves a certain distance away from the road boundary and parallel to the road boundary. but,
- Road boundaries on ship roads vary, including grass shoulders, white lines, fences, and steps, so there has been no good method for detecting them. The object of the present invention is to invent this method and to invent a method for detecting obstacles on the road.

[Summary of the invention] 1) Inverse transformation of road boundary image As shown in FIG. 1, a mobile robot is placed near the road boundary. Assume that a color TV camera with a lens of focal length f is attached to the head of a mobile robot at a height of h meters in the direction of the robot's front principal axis, and that the optical axis of the camera is tilted downward by φ degrees from the horizontal plane. The image of the road boundary on the TV screen is a straight line that crosses the upper left corner of the screen, as shown in FIG.

Coordinate system x whose origin is the point on the bottom of the car body directly below the TV camera
Think Oyozo. However, yO is a coordinate axis taken in the front principal axis direction of the robot, and xO and ZO are coordinate axes perpendicular to yo in the horizontal direction and perpendicular to the vertical direction. The xoyQzO coordinate is translated in the zO force direction h, and then rotated downward by φ around the XO axis to obtain a new coordinate system xlylzl. The origin of this coordinate system coincides with the center of the TV camera lens, and yl@ coincides with the optical axis. The coordinate transformation from xOyozO to xlylzl is as follows.

Consider a two-dimensional coordinate system uv whose origin is the center on the image plane. However, the U axis is taken in the direction of the -x1 axis, and the V axis is taken in the direction of the -21 axis. If the focal length of the TV camera lens is f, then from the lens formula, the image of point (xi, yl, zl) is (
U, V) are expressed as follows.

v/f = zl/yl (2
) u/f = xl/yl (
3) From equations (1), (2), and (3), (xO, yO,
When zO) is given, (u, v) can be found immediately.

When the distance from the robot to the road boundary is a, and the angle between the front principal axis direction and the road boundary is θ, the road boundary is expressed as follows.

xo cosθ+yOsinθ= a (
4) zO=O(5) The image of this road boundary can be found from equations (1), (2), and (3) as follows.

hcoseu/f+(h sir+φ sinθ+a
cosφ) V/f=asinφ −hcosφ sin
θ (6) Conversely, if the image of the road boundary is determined on the screen, the angle θ between the distance aa from the road boundary in real space and the front principal axis direction is determined.

2) Extraction of road candidate area The RGB 3 screen of the color TV camera is digitized. R
A differential image is created by applying a differential operation to each screen of GB. A monochrome differential image is obtained by adding the RGB differential images. Next, the original image is divided into regions where the value of the differential image exceeds the threshold value. Next, the average value of the area, center coordinates, length, width, and RGB values of each region is determined. gray R
A region having a GB value, whose center coordinates are in the lower half of the screen, and which has a large area is defined as a road candidate region.

3) Calculation of predicted road boundary area Let us assume that the robot runs parallel to the road boundary, that is, θ=O, while maintaining the distance @a. A straight line representing the road boundary image is found from equation (6).

htj/f +aCO5tlJV/f=asinφ
(7) This equation is transformed as follows.

u/fcosω+v/fsinω=b (
8) However, tan(a) = acosφ/h b=asinφcosω/h The area surrounded by two straight lines obtained by moving straight line (8) vertically by Δb is defined as a road boundary prediction area.

4) Road boundary observation Straight road candidate areas may become one with areas outside the road due to discontinuities in road boundaries such as white lines, obstacles, etc. Therefore, the outline of the road candidate area is approximated by directed line segments, and within that,
Straight line u/fcosΩ+v/fsinΩ=B made by connecting directed line segments within the predicted area of the road boundary
Let (9) be the road boundary observation straight line. See Figure 2.

5) Obstacle Detection As shown in Figure 3, detect obstacles across the road boundary observation straight line and in the RG
A region whose B value is different from the road region is extracted and set as an obstacle partial region. Obstacles such as cars and people are not uniform in color, so they are divided into several areas on the image fff.

Therefore, if there is a region adjacent to the obstacle partial region whose U coordinate values at both ends match that of the region, the obstacle partial region and its adjacent region are merged. This process satisfies the conditions! Repeat until there are no more a prefix areas. The combined area obtained in this way is defined as an obstacle area. Given the coordinates (υ, V) of the lower right corner of the obstacle area on the screen, (1), (2)
, (3) is inversely transformed to obtain the coordinates (X
O, YO, ZO) can be found.

[Example] 1) The horizontal angle of view of the lens is 40Q, the vertical angle of view is 29-゛, and the number of pixels of the image memory is 320 horizontally and 240 vertically.
Then, both v/f and u/f are 0.002, which corresponds to one pixel.

When h=1250mm, a=1000mm, and φ・8″, ω and b in equation (8) representing the road boundary are ω=38.
4°, b=0.0873.

2) If the road boundary observation straight line in equation (9) is given, then from equation (6) tanΩ=(hsinφsinθ+a cosφ)/h
cosθB=(asinφ − hcosφ sinθ)
cosΩ/hcosθ Furthermore, Ω=45.6, B=
0.225, h=1250, and φ=8, the distance from the robot to the road boundary is a=1300+nm, and the angle between the robot and the road boundary is θ·-10.

3) As a differential operator, the four operators shown in FIG. 4 are used to obtain the absolute value, and the maximum value among them is taken as the value of the differential operator. This differential operator is applied to each RGB surface, and the sum is calculated to form a differential image. When the density level of a pixel was set to 64 levels, good results were obtained when dividing into regions using 23 levels as a threshold.

[Brief explanation of the drawing]

Figure 1 shows the robot, xoyOzo coordinate system, and xlyl
A diagram showing the relationship between the zl coordinate system and the uv coordinate system, Figure 2 is 7
FIG. 3 is a diagram showing the relationship between road boundary images and obstacles, and FIG. 4 is a diagram showing a differential operator. 1... Robot, 2... Road boundary, 3... TV camera, 4... Road boundary image, 5... Obstacle

Claims (1)

[Claims] A method for detecting road boundaries and obstacles using region-divided color image processing.