JP2578120B2 - Road shape recognition device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial application field) The present invention detects a white line of a road as a vehicle guide line, for example, from image information in a traveling direction of a vehicle. A road shape recognition device that recognizes the shape of a road by calculating the curvature of the white line.

(Prior Art) Conventionally, a technique described in Japanese Patent Application Laid-Open No. Sho 60-170715 is known as a technique for determining a road shape in a traveling direction of a vehicle. This is to determine whether the vehicle is currently traveling on a curved road using a steering angle sensor.

(Problems to be Solved by the Invention) However, although the speed of a curved road can be determined at the present time, the shape of a road to be driven, for example, the degree of a curve, cannot be accurately grasped. When used for controlling the imaging direction of a camera or the like, there is a problem that control is delayed on a curved road.

Therefore, the present invention has been made in view of the above, and an object of the present invention is to provide a road shape recognition device capable of recognizing a road shape in the traveling direction of a vehicle with high reliability.

[Configuration of the Invention] (Means for Solving the Problems) In order to solve the above problems, the road shape recognition device of the present invention captures a perspective image in the traveling direction of the vehicle as shown in FIG. Imaging means, image conversion means for converting perspective image information captured by the imaging means into plane image information, and a vehicle for guiding a vehicle extending in the traveling direction of the vehicle from the plane image information from the image conversion means Vehicle guide line detecting means for detecting a guide line, and tangent calculating means for obtaining respective tangents to the vehicle guide line at first and second points on the vehicle guide line at first and second predetermined distances in the traveling direction of the vehicle When,
Angle calculating means for calculating each angle of each tangent calculated by the tangent calculating means with respect to the traveling direction of the vehicle, and detecting the curvature of the vehicle guide line in the traveling direction of the vehicle based on each angle calculated by the angle calculating means. A curvature detecting means for recognizing, as a road shape, an angle of each tangent to the traveling direction of the vehicle calculated by the angle calculating means and a curvature of the vehicle guide line detected by the curvature detecting means. And

(Operation) In the road shape recognition device of the present invention, a perspective image captured in the traveling direction of the vehicle is converted into a planar image, and a vehicle guide line for guiding a vehicle extending in the traveling direction of the vehicle is converted from the planar image. Detecting the tangents of the first and second points on the vehicle guide line at the first and second predetermined distances in the traveling direction of the vehicle, and detecting the vehicle guide line from each angle of the tangents in the traveling direction of the vehicle. , The calculated angles of the respective tangents with respect to the traveling direction of the vehicle and the detected curvature of the vehicle guide line are recognized as the road shape.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings.

FIG. 2 is a block diagram showing an overall configuration of an embodiment of an autonomous traveling vehicle control device to which the road shape recognition device of the present invention is applied.

The autonomous traveling vehicle control device shown in the figure performs control for making a vehicle heading to a set destination unmanned and autonomous while appropriately judging a road condition in a traveling direction detected by a camera, a sensor, or the like. An image information processing unit 10 that captures an image in the traveling direction of the vehicle and performs image processing.
0, an ultrasonic sensor, a detection processing unit 200 that detects an object such as a traveling direction or a side by a laser radar, for example, a preceding vehicle, a guardrail, an obstacle, and the like, and also detects and processes a wheel speed and the like, An actuator control unit 300 that has actuators for operating the steering, accelerator, brake, blinker, etc. of the vehicle for driving the vehicle unmanned, and an actuator control unit 300 that controls these, and map information storage that stores map information to the destination A travel control unit 500 that controls the actuator control unit 300 and the like to cause the vehicle to travel toward the destination according to information from each unit so as not to collide with an obstacle or the like; While inputting information on the destination, the image information processing unit 100
A man-machine interface unit 600 that displays images and other information from the vehicle, an auxiliary control unit 700 that has functions such as applying an emergency brake and limiting the maximum speed, and a collision state such as a flight recorder of an airplane. And a data recording unit 800 for recording the status of each part of the vehicle in the case of emergency braking or the like.

The image information processing unit 100 has a set of two cameras 101 and 103. The cameras 101 and 103 are provided, for example, on the left and right of the front part of the vehicle, and capture a stereo image in the traveling direction of the vehicle, that is, a stereo perspective image. The oblique images captured by the cameras 101 and 103 are supplied to an image processing computer 105 and subjected to image processing, whereby parallax is determined to determine the presence or absence of an obstacle, and the distance and direction to the obstacle are measured. The image processing computer 105 converts the oblique image into a planar image, that is, the image is inverted. The image processing computer 105 detects, for example, a white line or a road speed zone, a center line, or the like, which is a pair of vehicle guide lines drawn on the road, from the planar image, and measures the position.

More specifically, by detecting a white line or the like on the road, the relative relationship between the road and the vehicle, that is, the distance XL from the vehicle to the left white line, the distance XR from the right white line, the traveling direction of the vehicle and the road Is calculated, and the direction and curvature of the curve of the road are obtained from these values. Also,
Before the intersection, the distance Y to the intersection is determined by measuring how the white line is cut.

The distances XL, XR, the angle θ, the distance Y, and the like obtained by the image processing computer 105 are supplied to the local vehicle position estimating unit 107, whereby the positional relationship between the road and the vehicle, ie, the local vehicle The position can be estimated.
It should be noted that the local vehicle position refers to the vehicle position locally narrowly obtained by the image processing in the image processing computer 105 as described above. In addition, the host vehicle position roughly obtained from the local host vehicle position as described later is referred to as a global host vehicle position.

In addition, in order to take a wide angle of view, about three sets of cameras are installed and switched to be used, so that the right front, the left front,
The above parameters can be obtained from the image in front.

The detection processing unit 200 uses an ultrasonic sensor, a laser radar, and the like, and detects an object such as a traveling direction or a side, such as a preceding vehicle, a guardrail, and an obstacle, and also detects a wheel speed and the like. However, this makes it possible to run to some extent even when there is no image information from the image information processing unit 100, thereby trying to fulfill a fail-safe role.

The detection processing unit 200 includes four ultrasonic sensors 201, 203, 205, and 207 provided at four locations on the side, front, rear, left, and right sides of the vehicle.
The outputs of these ultrasonic sensors 201 are supplied to a fail-safe local host vehicle position detection unit 215, which measures the distance between the vehicle and the guardrail on the road by using these sensors. The same parameters as those obtained at 100, that is, the distances XL, XR, the angle θ, etc. can be measured. Also, by determining the break of the guardrail, for example, if the break of the guardrail is regularly formed at a position Y meters before the intersection, the distance to the intersection can be known. That is,
With these information, the same local vehicle position information as obtained by the image information processing unit 100 can be obtained.

The detection processing unit 200 is provided at the front of the vehicle, and detects an obstacle or the like existing in front of the vehicle.
And a forward ultrasonic sensor 210, and the outputs thereof are supplied to a fail-safe obstacle determining unit 217. The laser radar 209 and the front ultrasonic sensor 210 also calculate the distance to the obstacle, and even when the cameras 101 and 103 of the image information processing unit 100 cannot recognize the distance, the laser radar 2
When 09 detects an obstacle, the running of the vehicle is temporarily stopped or decelerated.

Further, the detection processing unit 200 has a pair of wheel speed sensors 211 and 213 provided on the left and right sides of the rear wheel, and outputs of these sensors are supplied to a wheel speed data processing unit 218. From global vehicle position estimator 219
Is supplied to The wheel speed sensors 211 and 213 detect the rotation of the wheel, and generate thousands (specifically, 1000 to 4000) of pulses for each of the right and left wheels for each rotation of the wheel in synchronization with the rotation. Therefore, if the difference between the numbers of the two pulses generated for each of the left and right wheels is calculated, the difference in the traveling distance is obtained. From this difference, it can be determined whether or not the vehicle is traveling in a curve. The traveling distance of the left and right wheels is the traveling distance of the vehicle as it is. Therefore, the displacement (ΔX, ΔY, Δθ) of the vehicle can be obtained by calculating these information every moment. Specifically, the attitude of the vehicle at a certain point in time,
That is, the relative vehicle position information based on the position, that is, the relative position of the X, Y coordinates and information such as θ can be obtained, and if the position of the vehicle before traveling is known, The current position of the running vehicle can be constantly detected by sequentially performing the wheel speed processing.
However, since errors are accumulated, if the mileage becomes longer,
Measurement error increases. The approximate vehicle position obtained in this way is the global vehicle position (X, Y).

Actuator control unit 300 has various actuators necessary for running the vehicle unmanned, that is, steering actuator 301 for steering, throttle actuator 303 corresponding to the accelerator, brake actuator 305, blinker drive unit 307, these The actuator control unit 309 controls each of the actuators based on control information from the travel control unit 500. When the vehicle is an AT car and only runs forward, only the above-described actuator may be used, but when controlling an MT car or a reverse movement, an actuator for operation such as a clutch or a shift lever is also required. . The actuator control unit 309 receives an acceleration / deceleration command or a target vehicle speed command from the traveling steering control unit 505, and controls an accelerator, a brake, and the like. The steering control is similarly operated upon receiving a rotation command to the right or left or a target steering angle command.

The map information storage unit 400 includes a map data storage unit 401 storing map information on the destination, map information to the destination, for example, map information such as an intersection position existing on a road to the destination, an intersection distance, and the like. A map information access control unit 403 controls access to the map data storage unit 401 from the travel control unit 500.

The traveling control unit 500 determines the road information of the traveling direction detected by the image information processing unit 100 and the detection processing unit 200 as appropriate and refers to the map information from the map information storage unit 400, and outputs the information from the man-machine interface unit 600. The actuator control unit 300 is driven and controlled to drive the vehicle toward the input destination, and obstacle data is supplied from the image processing computer 105 of the image information processing unit 100. Obstacle avoiding direction determining unit 501 that determines the avoiding direction of the obstacle based on the map information from the map information storage unit 400, and the global own vehicle position information from the global own vehicle position estimating unit 219 of the detection processing unit 200 ( X, Y), local vehicle position estimation unit of image information processing unit 100
Correction data from 107, man-machine interface 600
Is supplied with information such as destination information from, and based on this information, comprehensive driving strategy information including a route to the destination etc. is planned, and straight ahead, right / left turn, deceleration, acceleration,
A travel command unit 503 which is a planner that outputs control information such as information on a travel operation such as a stop and a distance to an intersection.
And the control information from the traveling command unit 503, the image information processing unit 1
00 local vehicle position information including the distance, angle, etc. from the road edge from the local vehicle position estimating unit 107, obstacle avoiding direction information from the obstacle avoiding direction determining unit 501, the detection processing unit 200
Of the vehicle (ΔX, ΔY,
Δθ), local self-vehicle position information including the distance and angle from the road edge from the fail-safe local self-vehicle position detection unit 215 of the detection processing unit 200, and the fail-safe obstacle determination unit 217. Information on the distance from the vehicle to the obstacle, information from the auxiliary control unit 700, etc. are supplied, and various control signals necessary for controlling the actuator control unit 300 based on the information, such as target vehicle speed, target steering angle information, etc. A travel steering control unit 505 that supplies information to the actuator control unit 300 and thereby performs steering control and the like.

More specifically, when destination information is input from the man-machine interface unit 600, the travel command unit 503 searches for a route to the destination while accessing the map information storage unit 400, and determines the shortest route. . Then, travel control information is created while comparing the determined information on the shortest route with the global host vehicle position information calculated based on the information detected by the wheel speed sensors 211 and 213. For example, when approaching an intersection, the vehicle outputs an approximate deceleration command or outputs information such as "how many meters to turn left" to the traveling steering control unit 505. Control of the actuator in the traveling steering control unit 505 is performed by intelligent control such as fuzzy control. That is, it is controlled in accordance with the production rules described in the format of "if ... then ...". In addition, the direction of the obstacle avoidance is determined based on the distance and the direction to the obstacle by the image information processing unit 100.

The man-machine interface unit 600 has a keyboard 601 for inputting destination information and the like, and a CRT display 603 for displaying a map to the destination and displaying various other information such as information to an intersection. Note that the keyboard 601 may be a digitizer or the like instead. Also,
The man-machine interface unit 600 may include a voice recognition or synthesis device as a man-machine interface.

The auxiliary control unit 700 has an emergency brake actuator 701, and the emergency brake actuator 701 operates in parallel with the brake actuator 305 for normal traveling,
Improves safety. The emergency brake actuator 701 is supplied with an external emergency brake signal received by the antenna 705 via the receiver 707 and the control unit 703, or an emergency brake switch 709 provided inside the vehicle.
When the operation signal is supplied from the control unit 703 via the control unit 703, it operates regardless of the control of the travel control unit 500, and stops the vehicle. Further, the auxiliary control unit 700 includes a maximum vehicle speed limiter 711, a speed setting unit 713 for setting the maximum vehicle speed for the maximum vehicle speed limiter 711, and a vehicle speed sensor 714 for supplying the actual vehicle speed information to the maximum vehicle speed limiter 711. And can travel at the maximum vehicle speed set by the speed setting unit 713. This maximum vehicle speed limiter 711 is for setting the maximum vehicle speed when the occupant of the vehicle wants to run slowly, and the set maximum vehicle speed information is supplied to the traveling steering control unit 505, and the traveling steering control unit At 505, control is performed so as not to exceed this speed. The maximum vehicle speed is determined by the driving command section.
503 can be set for each road. If the vehicle speed exceeds the set maximum vehicle speed, the vehicle speed sensor 714 detects the vehicle speed.
The emergency brake actuator 701 is controlled by operating the emergency brake actuator 701 so as to operate the emergency brake.

The data recording unit 800 includes a data recording unit 801 and a G that include a memory for recording the status of each part of the vehicle at the time of a collision, an emergency brake operation, and the like, such as a flight recorder.
It has a sensor 803. It is used later to clarify the cause or the like based on the data recorded in the data recording unit 801.

 Next, the operation will be described with reference to FIG.

This action is performed during the autonomous traveling control in the autonomous traveling vehicle control device, in particular, the calculation of the local own vehicle position of the vehicle based on the image information captured by the image information processing unit 100, specifically, the degree of curve of the curved road and This is for obtaining the positional relationship between the vehicle and the curved road.

The image in front of the vehicle captured by the cameras 101 and 103 of the image information processing unit 100 is a perspective image as shown in FIG. This oblique image is supplied from the cameras 101 and 103 to the image processing computer 105 and subjected to reverse oblique conversion, and the third
A planar image in the world coordinate system as shown in FIG. In the plane image, a pair of white lines L, which are a pair of vehicle guide lines, are shown as shown in the figure, and the white lines L are extracted by the image processing computer 105.

With respect to the white line L thus extracted, FIG. 3 (c)
, The X coordinate and the Y coordinate are set with the position of the vehicle 900 as the origin, and one white line at two positions Y1 and Y2 meters ahead of the vehicle 900, the first and the second on the right white line L in the drawing. The position X1, of the X coordinate of the second point P1, P2
X2, that is, distances X1 and X2 from the center with the vehicle 900 as the origin to the points P1 and P2 are obtained. Then, tangents T1 and T2 to the white line L at the points P1 and P2 are obtained, and each angle θ between the tangents T1 and T2 and the traveling direction of the vehicle 900, that is, the direction of the Y axis.
1. Calculate θ2.

The angles θ1 and θ2 calculated as described above are supplied to the local host vehicle position estimating unit 107, so that the signs of the angles θ1 and θ2 and the difference between the two angles (θ2−θ1) are as shown in the following table. It is determined whether the road is curved in the traveling direction of the vehicle, that is, whether it is a right curve or a left curve. Note that the angle θ is positive in the counterclockwise direction as shown in FIG. 3 (c).

From this table, for example, θ1 <0, θ2 <0, (θ2−θ1)
When <0, θ1> 0, θ2 <0, (θ2−θ1) <0,
When θ1> 0, θ1> 0, (θ2−θ1) <0, the curve is a right curve. When θ1 <0, θ2 <0, (θ2−θ1)> 0, θ1 <0, θ2> 0. , (Θ2−θ1)> 0, and θ1> 0, θ2> 0, (θ2−θ1)> 0, the left curve is determined, and when θ1 = θ2, the road is determined to be a straight road. it can.

The vehicle attitude is, for example, θ1 = θ2 on a straight road.
And have the same value with the opposite sign to θ1 or θ2. That is, the sign of θ1 corresponding to the near side Y1 is reversed, and the angle between the road and the vehicle is set. Since θ1 is the angle at the position of Y1 ahead of the vehicle, if it is necessary to obtain the distance and angle from the road edge at the current position, the instantaneous θ and X are obtained and stored, and the previous values of θ and X are obtained. Should be used.

Further, the angle difference (θ2-θ1) is (sin θ2-sin θ
Although 1) is more desirable, the greater the angle difference, the greater the degree of curve, the sharper the curve, and the smaller the angle difference, the closer to a straight line. At the entrance of the curve, the degree of the curve is small and gradually increases.

FIG. 4 illustrates the case where the curvature radius of the curve is obtained as the degree of the curve. The curvature radius R at each point P1 and P2 of the white line L is expressed by the following equation.

R = (Y1−Y2) / (sin θ2−sin θ1) Therefore, in this case, the reciprocal of the curve degree is given by the following equation.

1 / R = (sin θ2−sin θ1) / (Y2−Y1) The reciprocal of the radius of curvature of the white line L obtained in this manner may be used as the degree of the curve. Become.

Information such as the distances X1, X2, angles θ1, θ2, etc. obtained as described above and the degree of curve of the white line L, right curve or left curve obtained from these data is sent from the image information processing unit 100 to the travel control unit 500. Supplied. Thereby, the autonomous traveling control of the vehicle is appropriately performed.

Note that the above processing is performed at regular intervals, for example, at intervals of 0.1 seconds, while controlling the traveling steering.

In the above embodiment, the right white line has been described. However, the same can be applied to the left white line. Therefore, by combining information from the white lines on both the left and right sides, it is possible to control to run the key presto and the center even on a road where the width of the left and right white lines, that is, the road width changes.

Further, if the road width can be assumed to be constant, it is possible to improve the certainty by averaging information from the left and right white lines. If the left and right information are different, it can be determined that the image information is incorrect. In addition, it is possible to obtain the degree of reliability of the image information by obtaining the variation in the value of each angle of the left and right white lines. Since the camera angle of view is limited, it is recommended that the vehicle be driven with the white line on the left side as a guide during a right curve and the white line on the right side as a guide during a left curve.

In the above-described embodiment, the case where the road shape recognition device of the present invention is applied to the autonomous traveling vehicle control device has been described as an example. However, the present invention may be applied to other cases such as the radar scanning direction according to the curvature of a curved road. Also, the present invention can be applied to a device that controls the imaging direction of a camera.

[Effects of the Invention] As described above, according to the present invention, a perspective image captured in the traveling direction of a vehicle is converted into a planar image, and a vehicle extending from the planar image in the traveling direction of the vehicle is guided. And the first and second vehicle traveling directions are detected.
Detecting each tangent at the first and second points on the vehicle guide line at a predetermined distance, and detecting the curvature of the vehicle guide line from each angle in the traveling direction of the vehicle of each tangent,
The road shape can be recognized accurately and with high reliability from the detected curvature and the angles of the tangents.

[Brief description of the drawings]

FIG. 1 is a diagram corresponding to claims of the present invention, FIG. 2 is a block diagram showing an overall configuration of an autonomous traveling vehicle control apparatus according to an embodiment of the present invention, and FIG. 3 is an autonomous traveling vehicle control apparatus shown in FIG. FIG. 4 is an explanatory diagram for obtaining the curvature of the white line of the road in the autonomous traveling vehicle control device of FIG. DESCRIPTION OF SYMBOLS 1 ... Image pick-up means, 3 ... Image conversion means, 5 ... Guide line detection means 7 ... Tangent line calculation means, 9 ... Angle calculation means, 11 ... Curvature detection means

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Akira Hattori, Inventor Nissan Motor Co., Ltd., 2 Takaracho, Kanagawa-ku, Yokohama-shi (56) References JP-A-63-52213 (JP, A) JP-A-58-181180 (JP) , A) JP-A-51-12530 (JP, A)

Claims (1)

(57) [Claims] 1. An image pickup means for picking up a perspective image in a traveling direction of a vehicle, an image conversion means for converting perspective image information picked up by the image pickup means into plane image information, and a plane image information from the image conversion means. Vehicle guide line detecting means for detecting a vehicle guide line for guiding a vehicle extending in the traveling direction of the vehicle; first and second positions on the vehicle guide line at first and second predetermined distances in the traveling direction of the vehicle; A tangent calculating means for calculating each tangent to the vehicle guide line at point 2; an angle calculating means for calculating each angle of the tangent calculated by the tangent calculating means with respect to the traveling direction of the vehicle; and each of the angles calculated by the angle calculating means. And a curvature detecting means for detecting a curvature of the vehicle guide line in the traveling direction of the vehicle based on the angle, each angle of the tangent line calculated by the angle calculating means with respect to the traveling direction of the vehicle, and the curvature detection Road shape recognition device and recognizes the curvature of the detected vehicle guide line by stages as the road shape.