JP2003346163A - Track recognition device for vehicle - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To integrate an input image without fade-out of a white line for not only fluctuation of a horizontal-direction position but also vertical-direction vibration of a vehicle, and to detect a cruising lane highly accurately. <P>SOLUTION: The white line detection regions L, R having the same distance as a mileage of the vehicle in a period wherein both images are taken with the same length in an actual coordinate system relative to the longitudinal-direction width of each white line detection region are set on each track image. In the track image (a) at the time t0, representative points G on white line positions are detected by white line detection. When integrating an image in the white line detection region relative to the image of (b) at the time t1 after one period together with an integrated image of the previous period, the image in a farther region by one in the previous period than the aimed region is corrected to have the size of the aimed region, and the corrected image is compared with the image in the aimed region, and the horizontal-direction position of the integrated image in the previous period is corrected based on the comparison result, and then integration operation is performed together with the image in the aimed region. Hereby, integration can be performed without fade-out even when the vertical-direction vibration of the vehicle or the like is generated. <P>COPYRIGHT: (C)2004,JPO

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle travel path recognition device that detects a white line from a travel path image and recognizes the travel path.

[0002]

2. Description of the Related Art When performing automatic traveling control of a vehicle, it is indispensable to detect a position of the own vehicle in a traveling lane. In order to detect the position of the vehicle in the traveling lane with high accuracy, it is necessary to accurately recognize the traveling lane itself. Various methods have been studied for detecting the driving lane. Particularly, the detection method using image processing can directly recognize the driving environment such as the driving lane separated by the white line and the vehicle in front, and provides information necessary for driving. It is noted because most can be obtained. In running lane detection by image processing, in order to remove noise components such as shadows and road markings, the traveling road images captured at each time were integrated, and transient components such as shadows and road markings were removed. Then, a white line is detected.

[0003] In the integration, if the captured traveling road image is integrated as it is, for example, when the vehicle moves in the left-right direction, the integration is performed at a position where the white line is shifted, and the boundary between the white line and the road surface is deviated. It blurs. For this reason, the “vehicle traveling environment recognition device” described in Japanese Patent Application Laid-Open No. 10-208047 calculates the lateral position of the host vehicle with respect to the traveling road and uses the lateral position relative to the traveling lane position to calculate the lateral position of the image. After changing the direction position, an integration operation is performed on the newly input image and the boundary between the white line and the road surface is prevented from being blurred.

[0004]

However, in the above-described conventional apparatus, the lateral shift amount of the image to be integrated is calculated based on the relative lateral position with respect to the traveling lane. Although the correction is effective when the vehicle moves in the left-right direction, it cannot be corrected for the positional deviation of the white line due to the vertical vibration of the vehicle because the lateral position of the own vehicle does not change. There was a problem.

In calculating the lateral position shift,
Since the displacement is detected by comparing the images in the same detection area, the data to be compared is different both temporally and spatially, and partial blurring of the white line and noise other than the white line If there is, it is not possible to calculate the lateral displacement accurately. In view of the above-mentioned conventional problems, the present invention is resistant to blurring and noise, and can integrate not only when moving in the lateral direction but also when the vehicle vibrates in the vertical direction without deviation of the white line. It is an object of the present invention to provide a vehicle travel path recognition device that can detect a vehicle.

[0006]

According to the present invention, there is provided an image pickup means mounted on a vehicle for picking up an image of a traveling road, and a method for periodically acquiring a traveling road image from the image pickup means at every predetermined moving distance of the vehicle. Image capturing means for capturing the captured travel road image; area setting means for setting a plurality of white line detection areas having a width corresponding to a predetermined moving distance of the vehicle in the vertical direction on the captured travel road image; An integration operation means for performing an integration operation on the road image in the white line detection area to create an integration image; and
White line detection means for detecting a white line and recognizing a traveling road, and image correction means for correcting an image in the white line detection area to a size corresponding to a predetermined white line detection area to be formed in the next cycle,
Comparing means for comparing the corrected corrected image with an actual input image in the predetermined white line detection area; and detecting a lateral displacement from a previous cycle of the white line position based on a result of the comparison. A position shift detecting unit, wherein the integration operation unit performs an integration operation with the input image after correcting a horizontal position of the integrated image based on the detected lateral position shift amount. It was taken.

According to a second aspect of the present invention, the image correcting means generates a plurality of templates by shifting positions in a horizontal direction and a rotating direction based on the corrected image, and the comparing means generates the plurality of templates. The input image was compared with the input image using a template.

[0008]

According to the first aspect of the present invention, the vertical width of the white line detection area is set in accordance with the vehicle movement distance during the period of capturing the road image, so that the vehicle travels in accordance with the traveling of the vehicle. The same white line portion in the vertical direction can be sequentially detected from each white line detection area. For this reason, the image in the white line detection area is corrected to the size of the white line detection area to be brought to the next cycle, and the corrected image is compared with the actual input image, so that the white line position is shifted from the previous cycle in the horizontal direction. Can be calculated. As described above, since the image of the same white line portion is used for calculating the horizontal position shift of the white line position, the shift amount can be accurately calculated even if the white line has noise or partial blurring.

Further, since the change in the appearance of the white line to be detected is directly detected as a shift amount, the shift amount can be reliably calculated not only in the case of horizontal movement of the vehicle but also in the case of vertical vibration. Then, by performing the integration process with the input image after changing the horizontal position of the integrated image using the calculated shift amount, even when the vehicle moves in the horizontal direction or vibrates in the vertical direction, the white line is formed. The image can be integrated without shifting in the lateral direction, and the traveling lane can be detected with high accuracy.

According to the second aspect of the present invention, the template is used for comparing the corrected image with the input image, and the template includes a template whose inclination is rotated with respect to the corrected image. For white lines having different inclinations, it is possible to calculate the quantity with good accuracy. As a result, an effect is obtained that an image can be integrated without a white line being shifted in a horizontal direction even on a curved road or the like.

[0011]

Embodiments of the present invention will be described below with reference to examples. FIG. 1 is a diagram illustrating the configuration of the embodiment. Microcomputer (microcomputer) 2 and camera 1
And a memory 3 and a wheel speed sensor 4 are connected. The camera 1 is attached to a vehicle (not shown) so as to capture an image of a traveling road from the front of the vehicle. Microcomputer 2
Detects the lane by detecting the white line by performing image processing on the road image captured by the camera 1 and detects the own vehicle position.

Image processing data such as a road image is stored in a memory 3.
Is stored. Image capture from the camera 1 is performed every time the vehicle travels a predetermined distance, and vehicle speed information from the wheel speed sensor 4 is used to calculate the travel distance. The microcomputer 2 is connected to a display 7 as a subsequent device, a vehicle control device 5 and an alarm device 6, and outputs information such as a running lane to these devices.
Thus, for example, when the vehicle departs from the traveling lane, an alarm is issued and control for applying the brake can be automatically performed. In addition, the driver can check the traveling lane from the display 7.

FIG. 2 is a functional block diagram showing a traveling path detection function in the microcomputer 2. The microcomputer 2 includes a moving distance calculating unit 11, an image capturing unit 1,
3, an area setting unit 15, a correction unit 16, a comparison unit 17, a displacement detection unit 18, a calculation unit 19, and a white line detection unit 20. The camera 1 and the memory 3 are connected to the image capture unit 13. . The wheel speed sensor 4 is connected to the moving distance calculator 11.

The moving distance calculator 11 calculates the moving distance of the vehicle based on the vehicle speed information from the wheel speed sensor 4. The image capturing unit 13 periodically updates the camera 1 for each predetermined moving distance.
From the road. The memory 3 stores the acquired traveling road image. The area setting unit 15 is configured to detect a white line having the same length (width) in the real coordinate system in the vertical direction on the traveling road image based on white line position information from a white line detection unit 20 described later. A plurality of white line detection areas having the same distance are set. The vertical distance of each white line detection area is set to the same value in accordance with the predetermined moving distance of the vehicle during the period for capturing the road image.

The correction unit 16 converts the image in the white line detection area into
A correction image is created by correcting the size of the predetermined white line detection area to be formed in the next capture cycle. The comparing unit 17 compares the corrected image with the actual input image. Position shift detector 18
Detects a lateral displacement between a white line position detected by the white line detection unit 20 and a white line position in the input image based on a comparison result between the corrected image and the input image.

The calculation unit 19 performs an integration operation on the road image captured at each time in a predetermined white line detection area to create an integrated image. At the time of the integration operation, the horizontal position of the integration white line is corrected based on the horizontal position shift detected by the position shift detection unit 18, and then the integration operation with the newly input image is performed. The white line detection unit 20 performs image processing on the calculated integrated image to detect a white line, and recognizes the traveling lane and the own vehicle position at each time based on the white line position. The coordinates of the detected representative point G of the white line position are output to the area setting unit 15 and the displacement detection unit 18.

Next, the flow of white line detection in the above configuration will be described with reference to the flowchart of FIG. First, in step 110, initialization is performed. By this initial setting, the cumulative traveling distance parameter in the moving distance calculation unit 11 becomes zero, and the coordinates for setting the white line detection region are set in the region setting unit 15. The coordinates of the white line detection area are updated when a white line is detected.

In step 120, the moving distance calculating section 11 calculates the traveling distance of the vehicle based on the vehicle speed information from the wheel speed sensor 4 and the processing speed of the microcomputer 2. In step 130, the image capturing unit 13 determines whether the traveling distance of the vehicle has exceeded the set value Tu. If the traveling distance is equal to or less than the set value Tu, it is determined that it is not time to capture an image, and the process returns to step S120. If the traveling distance exceeds the set value Tu, it is determined that it is time to capture an image, and the process proceeds to step 140. In step 140, the image capturing unit 13 captures the traveling road image from the camera 1, and the captured traveling road image is stored in the memory 3.

In this embodiment, since the camera 1 has a high-speed frame rate function and the images are sequentially updated in a short period in the present embodiment, the image at the time of the capture timing is selected and captured. Shall be.
If a normal camera without the high-speed frame rate function is used, the camera 1
, An image may be taken directly from the camera 1 into the memory 3. Set value Tu is camera 1
Is set to, for example, 3 m in accordance with the imaging range of. In this case, every time the vehicle travels 3 m, the traveling road image is captured.

In step 150, the area setting unit 1
Reference numeral 5 sets a plurality of white line detection areas on the captured traveling road image to detect white lines. FIG. 4 is a traveling road image in which a white line detection area is set. Here, the white line S
The white line detection areas L (L1, L1,
L2, L3) and R (R1, R2, R3) are set three by three. Note that the number of white line detection areas can be increased or decreased as needed. The vertical position (Y axis) of the white line detection areas L and R is a position where a position that is a predetermined distance ahead of the own vehicle that is set in advance is projected on the image coordinate system. For example, assuming that the setting position of the farthest white line detection area is 20 m away from the host vehicle, the upper sides of the white line detection areas L1 and R1 are set in accordance with the point D on the traveling road image.

As shown in FIG. 5, each white line detection area detects a white line having the same length HR in an area U which is separated from the white line S in real coordinates. As shown in FIG.
The vertical position of each white line detection area is set while dividing the traveling road image by a width corresponding to. Here, the length HR of the white line
Is set to the same value as the predetermined value Tu. For example, Tu
Is set to 3 m, HR also becomes 3 m.

The horizontal position (X-axis) and the size of the white line detection areas L and R are determined based on the X coordinate value of the image coordinate of the representative point of the white line position as the center of the white line detection result of the previous cycle in each white line detection area. The size in the direction is, for example, the number of pixels corresponding to 1 m in the real coordinate system.

In step 160, the correction unit 16
Reads the traveling road image from the memory 3 and
The image in the white line detection area set in step 5 is corrected to create a corrected image. This corrected image is used to detect a lateral displacement from the previous period of the white line position in the white line detection area. FIG. 6 is a diagram for explaining detection of a lateral displacement using a corrected image. In the drawing, (b) is a traveling road image at an arbitrary time t1, and (a) is one cycle before the time t0.
It is a traveling road image of. White line detection areas L and R are set on each traveling road image. In the traveling road image shown in FIG. 7A, the representative point G of the white line position is detected by the white line detection.

The vertical distance between the white line detection areas L and R is
As described above, in the real coordinate system, the traveling distance of the vehicle during the period in which both images are captured is the same as the traveling distance of the vehicle. For example, at time t0, the image in the white line detection area L2 becomes At t1, it is just included in the white line detection area L3 'of the traveling road image (b).

Therefore, as shown in FIG. 6C, the image in the white line detection area L2 at the time t0 one cycle before is set to the angle of view, the focal length, the number of pixels and the mounting position, which are the characteristic parameters of the camera 1. If the correction is made to the size at the time of reaching the white line detection area L3 'based on the white line detection area L3', the corrected image in the area H2 is compared with the actual input image in the white line detection area L3 '. It is possible to detect a lateral displacement from the previous period of the image in the image.

The correction section 16 reads the road image from the memory 3 and corrects the image in the white line detection area set by the area setting section 15 to the size detected at the next cycle. Create an image. In the present embodiment, since a template is used for comparing images, the template is created by shifting the horizontal position and the inclination of the corrected image. The correction image and the template are created in the white line detection area L1,
This is performed only for L2, R1, and R2.

In step 170, the comparison unit 17
The correction unit 16 compares the template created with the input image in the corresponding white line detection area to detect a lateral displacement of the input image. For example, in the case of the white line detection area L3 ′ shown in FIG. 6, as shown in FIG.
In order to make a comparison with the corrected image in the area H2 corrected based on the input image, the input image in the white line detection area L3 'is compared with a template based on the corrected image in the area H2.

Thus, although the comparison differs in time,
Since an image including the same white line portion is used, the correlation value at the matching position is very large, and the lateral displacement can be accurately detected. Further, as shown in FIG. 6, for example, when the blur 26 exists in the white line portion in the white line detection area L3 ', the corrected image calculated based on the image of the white line detection area L2 in the previous cycle also has the blur 25. There is a corresponding blur 25 ', so that both images can be compared exactly,
It is possible to prevent undetectable or detection errors due to changes on the image such as blurring or noise.

Although the detection of the positional deviation can be performed at an arbitrary point on the corrected image, in the present embodiment, it is performed at the representative point G of the white line position detected by the white line detection. For example, when comparing the input image of the white line detection area L3 'shown in FIG. 6B, the position information of the point g corresponding to the representative point G of the detected white line position is obtained on the corrected image of the area H2. The comparison unit 17 detects the X coordinate of the point B having the largest correlation value with the point g from the input image of the white line detection area L3 ′.

In step 180, the displacement detector 18 compares the X coordinate value of this point B with the X coordinate value of the representative point G of the white line position in the same white line detection area in the previous cycle, thereby obtaining the input image. In the horizontal direction from the previous cycle. For example, in the case of the white line detection area L3 ′ shown in FIG. 6, the X coordinate of the point B is compared with the X coordinate of the representative point G of the white line position detected in the previous cycle white line detection area L3, and the input image is compared. Detects lateral displacement from the previous cycle.

In step 190, the calculating section 19
In the white line detection areas L2, L3, R2, and R3, an integration operation of the input image input at each time is performed. At the time of the calculation, the horizontal position of the integrated image is corrected using the detected horizontal position shift amount, and then the integration operation is performed with the newly input image based on the following equation. H = H × (1−β) + F × β (1) where H is the integral image and F is the input image. β is a constant (0 <β <1).

Finally, in step 200, the white line detection unit 20 performs a white line detection process from the integrated image after the integration. As a detection method, for example, a Sobel filter operation or the like is performed on the integral image in the white line detection area to extract a boundary between the white line and the road surface, and the position of the boundary is detected in consideration of the linearity. At this time, the selected boundary only needs to be consistent in the processing, and either the boundary of the white line or the road surface may be used.

The intersection of this boundary and, for example, the upper side of the white line detection area is used as a representative point G of the white line position to obtain a coordinate value.
The coordinates of the representative point G of the white line position are output to the region setting unit 15 and the positional deviation detecting unit 18 and serve as a reference point for the white line detection region setting and positional deviation calculating processing in the next processing. After the coordinate value of the representative point G of the white line position in each white line detection area is obtained, if necessary, a road alignment is obtained by approximating each point with a curve, or the lateral position of the own vehicle in the lane. Is calculated.

Next, the case where the traveling road is a curved road will be described. FIG. 7 is a flowchart showing the flow of the image correction process in step 160 of FIG. 3 in more detail.
In step 61, the correction unit 16 determines the input image that is one distance away in the previous cycle as an image for matching with the white line detection area currently focused on, based on the mounting angle of the camera 1, the lens parameters, and the like. Make corrections to create corrected images.

The reference position for this processing is the representative point G of the white line position. For example, as shown in FIG. 6B, when the target area is the white line detection area L3 'in FIG. 6, the area one distant from the white line detection area L3', which is the target area, is the white line detection area L2 '. White line detection area L
The input image one cycle before in 2 ′ is a white line detection area L2 in (a), and point G is a representative point of the white line position at that time. The corrected image of the white line detection area L2 is the corrected image in the area H2 shown in (c), and the point g on the corrected image is the reference position.

In step 62, a laterally shifted image is created by shifting the corrected image created in step 61 by a predetermined number of pixels in the horizontal direction. For example, if the shift pitch is 1 pixel and the amount is 5 pixels on the left and right, the total is 11 pixels.
One laterally shifted image is created.

In step 63, a rotated image is formed by rotating the image created in step 61 by a predetermined angle around the reference position g, including the corrected image. For example, if the rotation pitch is 1 pixel and the amount is ± 5 pixels on the lower side of the corrected image, a total of 11 rotated images are created. The laterally shifted image and the rotated image created in this way are used as templates.

FIG. 8 is a diagram showing an example of a template. (B1) of (b) is a corrected image and is a reference template. (A1) and (c1) are templates whose horizontal positions are shifted in the left-right direction from the reference template.
(A) (a2) and (a3) are templates obtained by rotating the inclination θ with respect to the template (a1).
(B) (b2), (b3), (c) (c2), (c)
3) is a template obtained by rotating the inclination θ with respect to the templates (b1) and (c1).

As described above, the template includes not only the template whose horizontal position is shifted from the corrected image but also the template whose inclination is rotated with respect to each template. Even when the inclination of the white line in the input image is different, comparison can be made. When the traveling road is a curved road, the inclination of the white line in the white line detection area varies depending on the distance from the host vehicle. By using the template whose inclination is rotated as described above, even if the inclination of the white line is different between the corrected image and the input image, it is possible to make a comparison, and it is possible to calculate the lateral displacement.

FIG. 9 is a diagram for explaining the detection of a lateral displacement when the traveling road is a curved road. (A) in the figure
Is a traveling road image captured at an arbitrary time t0, and (b)
Is a traveling road image at time t1 after one cycle. White line detection areas L and R are set on each traveling road image. (A)
In the traveling road image, the representative point G of the white line position is detected by the white line detection in each white line detection area.

For example, when correcting the lateral displacement of the white line detection area L3 ', as shown in (c), at t0 one cycle before, the white line detection area L2 which is one distance away from the white line detection area L3' is used. Is corrected to create a corrected image in the area H2 shown in (c). (D) is a white line detection area L
3 'is an input image. Since the travel path is a curved road, the corrected image and the input image in the white line detection area L3 'have different inclinations θ of the white line.
Position cannot be detected. In this case, the position of the point B can be detected by applying the template whose inclination is rotated. By comparing the coordinate value of this point B with the representative point G of the white line position of the white line detection area L3 at the time t0 one cycle before, it is possible to detect a lateral displacement from the previous cycle of the input image.

Finally, details of the integration processing in step 190 of FIG. 3 will be described with reference to the flowchart of FIG. In step 91, the horizontal position of the integral image is corrected in the white line detection area using the detected horizontal position shift amount. This is because the image X coordinate value of the representative point G of the white line position in the white line detection area detected in the previous cycle in the white line detection area of interest at present is aligned with the position of the point B in the horizontal direction. It is to perform parallel movement conversion.

Then, in step 92, step 91
The integrated image and the input image whose positions have been transformed by
Is used to perform integration processing to create a new integrated image.
Based on the integrated image created here, detection processing of the representative point G of the white line position is performed in the following processing.

According to the present embodiment configured as described above, when calculating the shift amount of the input image, the accuracy of detecting the shift amount is improved by using the input image including the same white line portion. Since the integration process is performed after correcting the position of the image to be integrated based on the white line, the white line can be integrated without shifting in the horizontal direction even when the host vehicle moves left and right.

Further, when calculating the shift amount of the image, the shift amount between the previous cycle and the current input image is directly obtained. After correcting the position of the integral image based on the shift amount, the new input image and the new input image are corrected. By performing the integration processing, even when the host vehicle vibrates up and down, the road white line can be integrated without shifting in the lateral direction.

Further, a white line image having a different inclination is generated as a template when correcting an image used for calculating a shift amount of an image, and accurate shift amount detection can be performed even when the road alignment differs from the time of the previous cycle. By performing the integration process after correcting the position of the image to be integrated based on the amount, even when the traveling road is curved, the white line can be integrated without shifting in the horizontal direction.

In the present embodiment, the description has been made based on the imaging of the front area of the vehicle. However, for example, the rear area of the vehicle may be processed. In this case, the image integrated in the white line detection area becomes an image of an area closer to the vehicle than the attention area. Further, the description of the embodiments is provided for facilitating the understanding of the present invention, and is not for the purpose of limiting the present invention, and various modifications are possible.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an embodiment.

FIG. 2 is a functional block diagram showing a traveling path detection function in the microcomputer.

FIG. 3 is a flowchart illustrating a flow of white line detection.

FIG. 4 is a diagram illustrating a traveling road image in which a white line detection area is set.

FIG. 5 is a diagram for explaining a vertical distance of each white line detection area in real coordinates.

FIG. 6 is a diagram for describing detection of a lateral displacement using a corrected image.

FIG. 7 is a flowchart illustrating a flow of an image correction process.

FIG. 8 is a diagram illustrating an example of a template.

FIG. 9 is a diagram for explaining detection of a lateral position shift when the traveling road is a curved road.

FIG. 10 is a flowchart illustrating a flow of an integration process.

[Explanation of symbols]

1 camera (imaging means) 2 Microcomputer 3 memory 4 Wheel speed sensor 5 Vehicle control device 6 Alarm device 7 Display 11 Moving distance calculation unit 13. Image capture unit (image capture means) 15 area setting unit (area setting means) 16 Correction unit (image correction means) 17 Comparison section (comparison means) 18 Position shift detection unit (position shift detection means) 19 Calculation part (integral calculation means) 20 White line detection unit (White line detection means) 25, 25 ' Point B G White line position representative point g G corresponding point L, R White line detection area

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G06T 1/00 330 G06T 1/00 330A 7/00 300 7/00 300D G08G 1/16 G08G 1/16 C F term (reference) 5B057 AA16 CA08 CA12 CA16 CB12 CB16 CD03 DA07 DA08 DB02 DC33 5H180 AA01 BB15 CC04 CC24 CC30 FF32 LL07 LL15 5L096 BA04 CA04 FA03 FA15 FA69 GA09 GA17 JA09

Claims (2)

[Claims] 1. An image pickup means mounted on a vehicle for picking up an image of a traveling road, an image capturing means for periodically capturing a traveling road image from the imaging means for each predetermined moving distance of the vehicle, and An area setting means for setting a plurality of white line detection areas having a width corresponding to a predetermined moving distance of the vehicle in the vertical direction; and integrating the acquired traveling road image in the white line detection area to calculate an integrated image. Integral calculating means to be created, and a white line detecting means for detecting a white line and recognizing a traveling road from the calculated integrated image,
Image correction means for correcting the image in the white line detection area to a size corresponding to a predetermined white line detection area to be formed in the next cycle, the corrected corrected image, and an actual image in the predetermined white line detection area. Comparing means for comparing the input image with the input image; and positional deviation detecting means for detecting a lateral positional deviation from a previous period of the white line position based on a result of the comparison. A traveling road recognition device for a vehicle, wherein the lateral position of the integral image is corrected based on the lateral displacement amount obtained, and then the integral operation is performed with the input image. 2. The image correction means generates a plurality of templates by shifting positions in a horizontal direction and a rotation direction based on the corrected image, and the comparing means generates the plurality of templates by using the generated templates. 2. The vehicle travel path recognition device according to claim 1, wherein the comparison is performed with an image.