JP3402075B2 - Vehicle lane recognition device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle lane recognition device capable of recognizing a shape of a lane from image information obtained by capturing an image of a region in the vicinity of a vehicle with an on-vehicle camera.

[0002]

2. Description of the Related Art Conventionally, as a device for detecting an object from image information obtained by picking up an image of the vicinity of a vehicle with a camera mounted on the side of the vehicle, for example, Japanese Patent Laid-Open No. 63-38 is known.
The one described in Japanese Patent No. 056 is known. With this device, the right and left cameras installed on the right and left sides of the front of the vehicle capture the right and left sides of the vehicle, respectively, and then determine the distance to the object from each image. Then, it is determined whether or not the own vehicle comes into contact with the object when the lane is changed. However, it is necessary to detect the adjacent lane in advance in order to determine whether or not the contact target object is on the adjacent lane when the lane is changed.

Normally, when the image is taken from the camera on the side of the vehicle, the right camera shows the right adjacent lane and the left camera shows the left adjacent lane, and the driving lane of the vehicle is divided into the lanes where the cameras are installed. A lane mark for it appears. In this case, in order to detect the lane shape from the image information obtained from the left and right cameras, the lane shape is separately obtained using the left and right road shape models.
That is, as shown in FIG. 6, the parameter estimation for the image information on the right side and the parameter estimation for the parameter on the left side are performed independently, so that separate left and right road models are required.

[0004]

However, in such a conventional technique, since the processing is performed independently on the right side and the left side, for example, the processing result corresponding to the image information from the right camera is the left side. It had nothing to do with the image information of the camera. Therefore, if an error occurs in the left or right lane detection processing due to the hiding of the lane mark by the vehicle, etc., this error cannot be removed, and as a result, this error is also included in the processing result. Will end up.

Furthermore, if concealment occurs on both lanes, an error will occur in both lane detection results, so it is extremely difficult to remove an error that has occurred once in post-processing. The present invention has been made in view of the above, and an object thereof is to provide a vehicle lane recognition installation that can contribute to improvement in lane detection accuracy.

[0006]

The invention according to claim 1 is
In order to solve the above problems, two image pickup means for picking up an image of the vicinity of the host vehicle, a road shape storage means for storing the shape of a road as a road model, and coordinates of one road model stored in the road shape storage means. Coordinate conversion means for converting the system using the image capturing attitude parameters of the two image capturing means to calculate the image coordinate values of the road model corresponding to each image capturing means, and scanning using the image coordinate values of the road model. Feature point coordinate detection means for defining a range, scanning the image information captured by the two image capturing means, and detecting the position coordinates of the feature points representing the lane from each image information, and the feature point coordinate detection Lane extracting means for extracting a lane constituted by connecting pixels representing each characteristic point so as to form a straight line or a curved line with reference to the coordinates of the characteristic point detected by the means; Change amount calculating means for comparing the coordinate values of the characteristic points detected by the coordinate point detecting means with the image coordinate values of the road model to calculate the change components of the image capturing attitude parameters of the two image capturing means; The image capturing attitude parameters of the image capturing means are updated so that the change components of the image capturing attitude parameters of the two image capturing means calculated by the change amount calculating means are equal to each other, and the approximate expression of the lane extracted by the lane extracting means is calculated. The gist of the present invention is to include a parameter updating unit for storing the new road model in the road shape storage unit.

In order to solve the above-mentioned problems, the invention according to claim 2 is characterized in that the two image pickup means are respectively mounted on door mirror portions of the own vehicle and pick up images of adjacent lane regions on the left and right of the own vehicle's driving lane. Use as a summary.

In order to solve the above-mentioned problems, the invention according to claim 3 is characterized in that the two image pickup means respectively image the rear regions of the vehicle.

[0009]

[0010]

[0011]

According to the first aspect of the present invention, when the white lines of a plurality of lanes are present in parallel, the image information captured separately on the left and right sides is used at the same time so that the lanes on both sides are adjacent to each other. Even if a vehicle exists, at least two white lines can be detected, so that highly accurate lane extraction can be performed, which contributes to improvement in lane detection accuracy.

Further, since the shape of the lane is detected by using one road model from the image information captured separately on the left and right sides,
Even if the lane mark is hidden by an obstacle in one of the images, the shape of the lane can be detected by the image information captured in the other, so accurate road shape detection can be performed without relying on post-processing. Therefore, it is possible to contribute to improvement in lane detection accuracy.

Further, by making the left and right road models common, the storage capacity for storing the road models can be small, and further, high-speed processing can be performed using, for example, one CPU.

Further, since the scanning range of the image can be reduced, the processing can be performed at high speed.

According to the second aspect of the present invention, since the image pickup position can be set at a position distant from the host vehicle, the obtained image information can include information behind the host vehicle, and only the lane adjacent to the host vehicle. Not only that, but also behind the driving lane can be monitored.

Further, by embedding the image pickup means in the door mirror portion, it does not bother the appearance and has a design advantage.

According to the third aspect of the present invention, it is possible to monitor an area such as a rear area of the own vehicle, which is likely to become a blind spot of a person. Therefore, when a lane is changed, an area for detecting a following vehicle on an adjacent lane to be monitored. Can be set on the adjacent lane.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing a system configuration of a vehicle lane recognition device according to a first embodiment of the present invention. As shown in the figure, the vehicle lane recognition device includes an image pickup unit 1, a processing unit 8 and a memory 18, and
The processing unit 8 includes an image processing unit 9 and a determination unit 10.
Further, the determination unit 10 exchanges signals between the vehicle state sensor 15, the vehicle motion control device 16 and the alarm device 17.

Next, each component of the vehicle lane recognition device will be described.

First, the image pickup section 1 will be described. In the right image pickup device 2, the CCD camera 4 obtains image information by picking up an image of a right rear side region of the vehicle as a region near the vehicle, for example, and converts image information of a subject in the rear region into an analog image signal. Convert. Similarly, in the left image pickup device 3, the CCD camera 6 also picks up an image of the left rear side region of the vehicle, for example. The CCD camera 4,
When 6 is mounted in a vehicle, it may be installed in a place with a wide field of view such as a door mirror section of the vehicle. The A / D converter 5 converts the analog image signal output from the CCD camera 4 into a digital image signal. In addition, the A / D converter 7
Similarly, the analog image signal output from the CCD camera 6 is converted into a digital image signal.

Next, the function of each component of the processing unit 8 will be briefly described.

First, the components of the image processing unit 9 will be described. The edge extraction processing circuit 11 uses, for example, a spatial differentiation filter or the like as edge information on the digital image signals output from the A / D converter 5 and the A / D converter 7 as an example of feature points representing lanes. Create an edge image that includes. Next, the image coordinates of the detection point are obtained based on the road shape and the camera posture, which are the previous processing results stored in the memory 18, and the edge image is scanned within a preset range using these coordinate values. Next, the pixels corresponding to the preset extraction conditions are extracted, and the edge coordinate data, which is the position coordinate data for the extracted pixels, is transferred to the lane recognition unit 12.

The lane recognition unit 12 includes an edge extraction processing circuit 1
Based on the edge coordinate data transferred from 1, the edge coordinate data of adjacent lanes on the left and right of the traveling lane in which the vehicle is traveling, or edge coordinate data for a following vehicle or an obstacle, for example, is extracted, and based on the extracted edge coordinate data. The vehicle recognizes a right lane, a left lane, a following vehicle, an obstacle, etc., and stores the recognition result in the memory 18 and transfers it to the determination unit 14.

Next, the components of the judgment unit 10 will be described. The vehicle state determination unit 13 determines the traveling state of the host vehicle based on the signal sent from the vehicle state sensor 15, and transfers the determination result to the approach degree determination unit 14. Here, the vehicle state sensor 15 detects the momentum of the own vehicle, the driver's intention to operate, and the like. For example, from a vehicle speed sensor that measures the speed of the own vehicle, a direction indicator, a steering angle sensor, or the like. Composed.

In the approaching degree determining unit 14, information such as presence / absence information and shape information of the left and right adjacent lanes with respect to the traveling lane of the own vehicle sent from the lane recognizing unit 12 and information of other vehicles, obstacles and the like are stored. Based on the information transferred from the vehicle state determination unit 13, the vehicle proximity determination degree indicating the lane departure or lane change of the own vehicle is determined. For example, when the relative speed, which is the difference between the speed of the vehicle detected by the vehicle speed sensor and the speed of the other vehicle measured by a sensor such as a laser radar, is greater than or equal to a predetermined value, and the direction indicator is operated. Yes, and in the image information on the rear side of the operated side of the turn signal, when the presence of a following vehicle or the like is recognized at a specific position on an adjacent lane,
This is determined to be the connected state, and the vehicle motion control device 16 and the alarm device 17 are driven.

The vehicle motion control device 16 is a device for controlling a drive system, a control system, a steering system, etc., and automatically when, for example, the approach determination unit 14 determines that there is a possibility of approach. It is possible to use an automatic brake that applies the brake to the. Further, as the alarm device 17, the driver's hearing, visual sense,
Anything that can appeal to the five senses such as the sense of touch and call attention to the driver may be used. For example, driving a chime or LE.
Display at D, vibration of the seat, etc. are conceivable. As described above, various forms can be considered for the configuration example of the determination unit 10.

The memory 18 stores initial conditions of images and processed data, road models, parameters representing camera attitudes, and the like. The memory 18 also functions as a work area when the lane recognition unit 12 and the like perform processing.

Next, the operation of the vehicle lane recognition device will be described with reference to FIGS. 1 and 2. First, in step S2, when the device power is turned on by the operation of the operator,
In step S4, initial setting processing is performed. here,
The initial setting is a process of clearing the work area of the memory 18. The operations performed by the operator include initial adjustment of the image pickup unit 1, setting of initial values of road model parameters indicating the shape of the lane, and setting of initial values of camera attitude parameters indicating the mounting position and direction of the CCD camera. is there.

Next, in step S6, the CCD camera 4
A signal corresponding to the image information of the rear side area imaged by the CCD camera 6 or the CCD camera 6 is converted into an analog image signal. Next, in step S8, the A / D converter 5 and the A / D converter 7 convert the analog image signal into a digital signal. Images obtained at this time are shown in FIGS. 3 (b) and 3 (c).

Next, in step S10, the lane recognition unit 1
As a lane recognition process 2, a lane is estimated by obtaining an approximate straight line or an approximate curve based on the edge coordinate data sent from the edge extraction processing circuit 11. Next, the amount of change in the road shape parameter and the camera posture parameter from the previous time is obtained as each parameter, a new road model is created from this amount of change, and the result is stored in the memory 18.

Next, in step S12, the approaching degree determining unit 14 performs traveling such as changing the vehicle speed or the lane of the own vehicle in response to a signal from the vehicle state sensor 15 such as a vehicle speed sensor or a direction indicator as an approaching determination process. The vehicle is in an approaching state based on the result of the determination and the relationship between the result of the determination, the road surface that can be driven by the lane, the existing position of the following vehicle on the adjacent lane, and the relative speed of the vehicle. Or not.

In step S14, the approaching state of the own vehicle and the other vehicle is determined. That is, for example, if the relative speed between the own vehicle and another vehicle obtained by the vehicle state sensor 15 is a predetermined value or more,
If the following vehicle on the adjacent lane is located within the predetermined range and the direction indicator is operated to the side where the following vehicle is present, it is determined as an approaching state, and step S1
Go to 6. On the other hand, if not, the process returns to step S6 to obtain the next image information. In step S16,
As an alarm process, the warning device 17 may approach the following vehicle when the driver of the own vehicle deviates from the lane or changes the lane when the approach degree determination unit 14 determines that the vehicle is approaching. The notification is made by driving the indicated chime, displaying with an LED, or vibration of the seat. Even if it is judged that the driver's operation was insufficient for the approach warning,
The vehicle motion control device 16 may be configured to be driven.

In step S18, the vehicle motion control system 1
6 is an avoidance process, for example, when the approach degree determination unit 14 determines that the vehicle is in the approaching state, the brake is automatically applied. Next, returning to step S6, a series of these processes is repeated.

Now, with reference to the processing diagram shown in FIG. 5, the lane recognition processing in step S10 corresponding to the processing content of the image processing unit 9 will be described in detail with reference to the flowchart shown in FIG. Note that the step numbers shown in FIG. 5 correspond to those shown in FIG. In step S1002, the edge extraction processing circuit 11 first creates a horizontal edge image and a vertical edge image by an edge image creation process using a spatial differential filter of 3 pixels × 3 pixels, for example. As the spatial differential filter, for example, a SOBEL filter may be used.

Next, in step S1004, by the coordinate conversion process, in the road model stored in the memory 18, the image coordinates of the target point which is separated by a predetermined distance in the road coordinate system by using the camera posture parameter or the like. Image coordinate values of the target point (hereinafter referred to as model coordinate values) are calculated by performing coordinate conversion to the system. Next, in step S1006, the number of pixels corresponding to a certain distance in the road coordinate system is calculated by the matching process using, for example, the focal length of the CCD camera, and the calculated number of pixels is centered around this model coordinate value. The edge image is scanned left or right or up and down by a minute. At this time, the scanning direction and the type of edge image to be scanned depend on the shape of the lane.If the white line on the image is close to vertical, the vertical edge image is scanned horizontally, and if it is close to horizontal,
Scan the horizontal edge image vertically. Next, a combination of plus edges above a certain value and minus edges below a certain value separated by the number of pixels corresponding to the width of the white line is extracted,
Coordinate value of either edge (hereinafter, calculated coordinate value)
Ask for. At this time, which edge coordinate value is used may be the same throughout the processing. Further, the calculated coordinate values and the model coordinate values, which are the edge coordinate data obtained in this way, are transferred to the lane recognition unit 12.

Next, in step S1008, the lane recognition unit 12 first calculates the amount of change of each parameter from the calculated coordinate value and the model coordinate value sent by the edge extraction processing circuit 11 by parameter estimation processing. Therefore, by using the constraint conditions such as the road width and the camera setting position to adjust the variation amount of the parameter, as shown in FIG.
The left and right road models can be used.

Here, the symbols used for the explanation are defined as follows. Dx-r (Dx-1): Deviation from the reference position of the right (left) camera (for example, the center of the vehicle lane) Dy-r (Dy-1): Reference position of the right (left) camera (e.g. , Height from the road surface) θr (θ1): Right (left) camera yaw angle φr (φ1): Right (left) camera pitch angle ψr (ψ1): Right (left) camera roll angle αr (α1) ): Yaw angle βr (β1) in the vehicle coordinate system of the right (left) camera: Pitch angle γr (γ1) in the vehicle coordinate system of the right (left) camera: Roll angle in the vehicle coordinate system of the right (left) camera , Camera pose parameters.

ΔDx-r (ΔDx-1): Dx-r (D
x-1) change amount ΔDy-r (ΔDy-1): Dy-r (Dy-1) change amount Δαr (Δα1): αr (α1) change amount Δβr (Δβ1): βr (β1) change amount Amount Δγr (Δγ1): Amount of change in γr (γ1) D1: Installation distance between the right camera and the left camera Dw: lane width Here, if Dw is constant as a constraint condition, D
The relationship of Dw = | Dx-r | + | Dx-1 | + D1 is established between x-r and Dx-1.

Further, by adjusting the left and right cameras at the time of installation, Dy-r = Dy-1, and it is possible to make the other camera attitude parameters equal on the left and right as well. Further, even if the directions of the left and right cameras are not equal, that state may be given at the time of initial setting.

On the other hand, the amount of change has no relation to the initial setting, and Δ
A relationship of | ΔDx-r | = | ΔDx-1 | holds between Dx-r and ΔDx-1. In addition, the amount of change in each of the other parameters is substantially the same on the left and right.

Next, based on the edge coordinate data of the calculated coordinate values representing the lane sent from the edge extraction processing circuit 11, an approximate straight line or an approximate curve is obtained by, for example, the method of least squares, and this is used as the recognition lane. . Next, step S
In 1010, each parameter is updated by the parameter update processing so that the shapes of the recognized lanes are equal on the left and right sides, and the approximate expression of the recognized lanes is set as a new road model.
It is stored in the memory 18.

In this way, the same lane shape is obtained on the left and right from the image information obtained from the left and right side cameras. Therefore, the recognition shape between the left and right lanes can be determined with a simple structure while ensuring a wide field of view. It is possible to realize a device that can recognize lanes with little variation and with high accuracy. Further, even if the lane is partially invisible due to concealment by the vehicle, it is possible to accurately detect the position of the own vehicle and the shape of the road in the traveling environment. An area for detecting a following vehicle can be set on the adjacent lane.

According to the present invention, the edge extraction processing circuit 11 first scans the image information picked up by the right side image pickup device 2 and the left side image pickup device 3 to detect the position coordinates of the characteristic points representing the lane. . Next, the lane recognition unit 12 refers to the coordinates of the detected feature points and extracts a lane formed by connecting pixels representing the feature points so as to form a straight line or a curved line. Next, the lane recognizing unit 12 compares the coordinate value of the extracted feature point with the image coordinate value obtained by converting the coordinate system of the road model stored in the memory 18 by the coordinate conversion process to determine the road shape. Road shape parameters and right imager 2
And a change amount calculation process for calculating at least one change component among the image pickup posture parameters representing the posture of the left image pickup device 3. Next, each parameter of the road model and the coordinate conversion process stored in the memory 18 is updated based on the calculated change amount and the extracted lane shape. As a result, the amount of information in the lane extraction process can be increased by simultaneously referring to the plurality of pieces of image information obtained from the right side image pickup device 2 and the left side image pickup device 3.

[Brief description of drawings]

FIG. 1 is a diagram showing a system configuration of a vehicle lane recognition device according to a first embodiment of the present invention.

FIG. 2 is a flowchart for explaining the operation of the vehicle lane recognition device.

FIG. 3 shows an imaging range (a) of a rear side area, an image model (b) obtained by the right side image pickup device, and an image model (c) obtained by the left side image pickup device.

FIG. 4 is a flowchart for explaining the operation of lane recognition processing.

FIG. 5 is a diagram for explaining lane recognition processing.

FIG. 6 is a diagram for explaining a conventional lane recognition process.

[Explanation of symbols]

1 Imaging unit 2 Right imager 3 Left imager 4 CCD camera 5 A / D converter 6 CCD camera 7 A / D converter 8 processing units 9 Image processing section 10 Judgment section 11 Edge extraction processing circuit 12 lane recognition 13 Vehicle status determination unit 14 Proximity judgment part 15 Vehicle status sensor 16 Vehicle motion control device 17 Alarm device 18 memory

─────────────────────────────────────────────────── ─── Continuation of the front page (56) Reference JP-A-7-93693 (JP, A) JP-A-5-265547 (JP, A) JP-A-6-20189 (JP, A) JP-A-6- 266828 (JP, A) (58) Fields surveyed (Int.Cl. ⁷ , DB name) G06T ⁷ /00-7/60 B60R 21/00 G06T 1/00 G08G 1/00-1/16 H04N 7/18

Claims (3)

(57) [Claims] 1. Two image pickup means for picking up an image of the vicinity of the own vehicle, a road shape storage means for storing a road shape as a road model, and one road model stored in the road shape storage means .
The coordinate system uses the imaging posture parameters of the two imaging means.
Image of the road model corresponding to each imaging means
A coordinate conversion means for calculating coordinate values respectively, and a scanning range is determined using the image coordinate values of the road model,
Image information picked up by the two image pickup means respectively
The position of the feature point that represents the lane from each image information
It is configured by connecting feature point coordinate detection means for detecting respective coordinates and pixels representing each feature point so as to form a straight line or a curve with reference to the coordinates of the feature points detected by the feature point coordinate detection means. Lane extracting means for extracting the lanes, and the coordinates of the characteristic points detected by the characteristic point coordinate detecting means.
The value and the image coordinate value of the road model are compared,
Calculates the change component of the imaging posture parameters of the two imaging means
Change amount calculating means and image pickup of the two image pickup means calculated by the change amount calculating means
Make sure that the change components of the posture parameters are equal to each other.
The image pickup attitude parameter of the image pickup means is updated to extract the lane.
Lane approximation formula extracted by means of a new road model
Parameter change stored in the road shape storage means as
A vehicle lane recognition device comprising: a new means . 2. The two image pickup means are respectively mounted on a door mirror portion of the own vehicle to drive the own vehicle.
The vehicle lane recognition device according to claim 1 , wherein images of adjacent lane regions on the left and right of the lane are imaged . 3. The vehicle lane recognition device according to claim 1, wherein the two image pickup means respectively image a rear area of the vehicle.