JP3341664B2 - Vehicle line detecting device, road line detecting method, and medium recording program - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicle line detecting device, a road line detecting method, and a medium on which a program is recorded, and more particularly to a method for processing a road image to detect a line in the image.

[0002]

2. Description of the Related Art Conventionally, there has been developed a technology for detecting a relative displacement of a vehicle with respect to a road (line) by photographing a road image and detecting a line in the image from the obtained road image.

For example, Japanese Patent Application Laid-Open No. 7-141592 discloses a technique for detecting a white line on a road by performing a correlation operation between an obtained road image and a template cut out from an asphalt surface.

[0004]

According to the operation (template matching method) using such a template,
In many cases, it is possible to detect a line on a road, but it may be difficult to detect a line from a road image depending on a shooting environment or a traveling environment.

For example, when a vehicle enters a tunnel,
Since the brightness inside the tunnel is significantly different from the brightness outside the tunnel, a clear road image may not be obtained with a camera having a finite dynamic range (for example, a CCD camera).

Also, a line (for example, a white line) in a road image
Does not always have a constant inclination (for example, 45 °). For example, when a lane is changed, the inclination of a line in the image greatly changes, and a constant inclination exists in the image. If the template matching method is used on the assumption, there is a problem that it is difficult to detect a line.

Further, when there is a puddle on the road due to rainy weather, sunlight or the like may be specularly reflected by the puddle and become as bright as a line on the road. The line and the puddle portion cannot be distinguished, and the line may be erroneously recognized.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and has as its object to provide a vehicle line detection system capable of reliably detecting a line on a road under various photographing environments or running environments. It is an object to provide a device, a road line detection method, and a medium on which a program is recorded.

[0009]

In order to achieve the above object, the present invention provides a two-dimensional CCD sensor and the two-dimensional CCD sensor.
An optical system for projecting a road image with a first exposure amount on a first area of the D sensor and projecting the road image with a second exposure amount on a second area of the two-dimensional CCD sensor;
Detecting means for detecting a line in the road image based on a first image output from a first area of the CD sensor and a second image output from a second area of the two-dimensional CCD sensor; A first region, the second region, and the first region;
Wherein the exposure amount second exposure amount varies from each other, said detecting means
Are the corresponding pluralities in the first image and the second image
Set multiple search windows for line detection at the position
Setting means and the corresponding search windows
And the processing result in any of the search windows
And selecting means for selecting .

The setting means includes an upper part and a lower part of the first area.
Part, and an upper portion of the second region corresponding to the first region
Top and bottom search windows for line detection
And a lower search window, and the selecting means
The processing result in the upper search window of the first area and the second
One of the processing results in the search window at the top of the region
Selected and in the lower search window of the first area
Processing result in the lower search window in the second area
It is preferable to select any of the processing results .

Further, the present invention detects a line on a road.
Different exposure areas on a single CCD.
Projecting the same road image,
For line detection on multiple corresponding parts of the same road image
Setting multiple search windows for
In the corresponding search window,
Select a processing result in the search window to select a line.
And a step of detecting

Further, the present invention detects a line on a road.
Medium for recording a program for
The computer to different areas of a single CCD
Input the same road image projected with different exposure,
For multiple corresponding parts of the same road image with different exposures
Set multiple search windows for line detection, and
In the search window corresponding to
By selecting the processing result in the search window of
Is detected .

[0013]

[0014]

[0015]

[0016]

[0017]

[0018]

[0019]

[0020]

[0021]

[0022]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present invention will be described with reference to the drawings, taking a case where a white line is detected as a line as an example.

<First Embodiment> FIG. 1 is a block diagram showing the configuration of the present embodiment. In this embodiment,
This is effective in situations where the brightness fluctuates greatly in the image, such as when a vehicle enters a tunnel, and therefore the white line image cannot be clearly captured with the normal dynamic range of a CCD camera. is there.

In the figure, an optical system 1 is for projecting the same road images having different exposure amounts to different regions of a single CCD sensor 24. Specifically, the optical system 1 includes a lens (L 1) 10, a lens (L 2) 18, a neutral density filter 16 that is disposed in front of the lens (L 2) 18 and reduces brightness by a predetermined amount, and a lens (L 1). A mirror (M1) 12, a mirror (M2) 14, a neutral density filter 16, a mirror (M3) 20 for reflecting light passing through a lens (L2) 18, and a mirror (M4) The light from the road is incident on the CCD sensor 24 through two optical paths, an optical path without the neutral density filter 16 and an optical path with the neutral density filter 16.

The CCD sensor 24 is a two-dimensional CCD sensor in which a plurality of CCD elements are arranged two-dimensionally, and two images (a first image and a second image) projected from the optical system 1 and having different exposure amounts from each other. (Referred to as a second image) is converted into an electric signal and output to the amplifier 26.

The amplifier 26 amplifies the first image signal and the second image signal from the CCD sensor 24, and
8 is output.

The image processing device 28 includes an A / D converter 28
a, a frame memory 28b, and an image recognition processing unit 28c. The first image signal and the second image signal converted into digital signals by the A / D converter 28a are stored in the frame memory 28b. The first image signal and the second image signal stored in the frame memory 28b are further supplied to an image recognition processing unit 28c.

The image recognition processing unit 28c is specifically constituted by a microcomputer, processes the first image signal and the second image signal, detects a white line in the image, and displays the detection result on the display unit 30. At the same time, the power is supplied to the vehicle control unit 32.

The display unit 30 is composed of a CRT, a liquid crystal or the like, and displays the detected white line in a form that is easy for the driver to visually recognize. The vehicle control unit 32 determines the lateral displacement of the vehicle with respect to the white line based on the detected white line. Calculate and control the steering angle and the like of the vehicle.

As the neutral density filter 16 for controlling the amount of exposure, for example, a filter for reducing the incident light to 1/10 ³ can be used, which has a normal dynamic range of the CCD sensor 24. 10 ³ to 1
0 ⁶ can be enlarged. Therefore, when the CCD sensor 24 is set to be saturated at 10 ⁵ cd / m ² in the area of the CCD sensor 24 that is not dimmed by the neutral density filter 16 (having a large amount of exposure), the detection range of the CCD sensor 24 as a whole is 10 ⁻ a ^{1 ~10 5} cd / m ^2. Brightness 10 ⁰ cd / m ² in the tunnel, the brightness of outdoor Hinata 10 ⁴ c
Since d / m ² , the value is sufficient to obtain a clear image.

FIG. 2 schematically shows two images having different exposure amounts projected on the CCD sensor 24 according to the configuration of FIG. 1, that is, a first image 100a and a second image 100b. The first image 100a is an image that is not dimmed by the dimming filter 16 and has a large exposure amount. On the other hand, the second image 100b is an image that has been dimmed by the dimming filter 16 and has a small exposure amount.
In the figure, “bright” and “dark” respectively indicate the magnitude of the exposure amount.

As described above, the two images 100a, 100
Since the exposure amounts b are different from each other, for example, when the vehicle is traveling on a road in fine weather, the first image 100a is slightly overexposed and becomes saturated, while the second image 100b has an approximately moderate exposure amount. While it is difficult to detect a white line from the first image 100a, the second image 100b
Makes it easy to detect a white line. In cloudy weather, in a tunnel, or the like, the first image 100a has an appropriate exposure amount, whereas the second image 100b tends to be underexposure, and a white line can be detected from the first image 100a. It is difficult to detect a white line from the second image 100b.

Assuming that the vehicle enters a tunnel, the first image 100a or the second image 100a
The image upper region b is darker than the image lower region because of the image in the tunnel. However, in the first image 100a, since the light is not dimmed by the dimming filter 16, even in the tunnel image at the top of the image, a sufficient exposure amount for detecting the white line is secured. On the other hand, the second image 100
In b, since the light is dimmed by the dimming filter 16,
In the tunnel image at the top of the image, white lines cannot be detected due to insufficient exposure.

On the other hand, the first image 100a and the second image 10
The lower area of the image 0b has normal brightness because it is outside the tunnel, and therefore it is difficult to detect a white line in the first image 100a because it tends to be overexposed. Since the light is attenuated by the optical filter 16, the amount of exposure becomes appropriate, and a white line can be detected.

As described above, the white line can be detected by selectively using the first image 100a and the second image 100b even in various environments where the brightness changes in the image. The image recognition processing unit 28c detects a white line based on the principle described above.

FIG. 3 schematically shows processing for specifically detecting a white line. Image recognition processing unit 28c
Sets a search window in each of the first image 100a and the second image 100b when detecting a white line. The search window is a window for matching a white line template prepared in advance in the window with an image, and performs a correlation operation while sequentially moving the template in the search window.

In the figure, search windows 200a, 200b, 300a, and 300b are set in a first image 100a, and search windows 202a, 202b, and 3b are set in regions of the second image 100b corresponding to the first image 100a.
02a and 302b are set respectively. The corresponding set of search windows is
a and 202a, 200b and 202b, 300a and 302
a, 300b and 302b. Image recognition processing unit 28c
Thus, the first image 100a and the second image 100b
, A plurality of search windows are set, and pattern matching (correlation calculation) is performed in each search window to detect a white line. Then, the detection results in each search window are compared, and one of the detection results in the corresponding search window is selected and output as the detection result. Specifically, a processing result in which a white line can be detected with high accuracy by the pattern matching method is selected and output.

For example, assuming that the vehicle enters the tunnel as described above, the search window 200a set in the tunnel image in the first image 100a,
While the white line can be detected in 300a, the white line cannot be detected in the search windows 202a and 302a that are the corresponding search windows of the second image 100b because the exposure amount is insufficient. Accordingly, the image recognition processing unit 28c compares the processing results in the search window 200a and the search window 202a, and outputs a white line detection result by using the processing result in the search window 200a. Similarly, the processing results of the search window 300a and the search window 302a are compared, and the processing result of the search window 300a is selected and output.

The search windows 200b, 300b, 202b, and 302b located outside the tunnel are located in the search window 200 in the first image 100a.
b and 300b, it is difficult to detect a white line due to overexposure, while the search window 202b and 302b in the second image 100b have the darkening filter 16b.
Therefore, the white line can be detected with an appropriate exposure amount. Therefore, the image recognition processing unit 28c compares the processing results of the search windows 200b and 202b, and selects and outputs the processing result from the search window 202b. Also, search windows 300b and 302
b, and selects and outputs the processing result from the search window 302b.

As described above, a plurality of search windows are provided for each of the first image 100a and the second image 100b,
By selecting and outputting one of the corresponding search windows, it is possible to reliably detect a white line even in a situation where the luminance rapidly changes in one image.

In this embodiment, the case where four search windows are set in one image is shown for convenience of explanation, but the number of search windows can be set arbitrarily.

<Second Embodiment> FIG. 4 is a block diagram showing the configuration of the second embodiment. This embodiment is particularly effective when the inclination of the white line in the image deviates from an expected value (for example, 45 °).

The CCD sensor 24 functions as photographing means for photographing a road image, and converts the obtained road image into an amplifier 26.
Output to

The amplifier 26 amplifies the input image signal and outputs it to the image processing device 28. Image processing device 28
Has an A / D converter 28a, a frame memory 28b, and an image recognition processing unit 28c.

The A / D converter 28a converts the image signal supplied from the amplifier 26 into a digital signal and stores the digital signal in the frame memory 28b.

The frame memory 28b supplies the stored image signal to the image recognition processing section 28c as appropriate.

The image recognition processing section 28c is constituted by a microcomputer and has a frame memory 28 as described later.
b), a matching operation is performed between the image signal read from b and a template prepared in advance, and a white line is detected and the display unit 3
0 or output to the vehicle control unit 32.

The display unit 30 and the vehicle control unit 32 display the detected white line, detect the relative displacement of the vehicle with respect to the white line, and control the steering angle and the like of the vehicle, as in the first embodiment.

FIG. 5 schematically shows the processing of the template matching method in the present embodiment. CCD
A search window 400 for detecting a white line is set in the image obtained by the sensor 24. Then, a predetermined calculation is performed while sequentially moving the template 500 prepared in advance in the search window 400. Here, the predetermined operation means a product-sum operation of the luminance of the obtained image and the luminance template 500 when the template 500 is a luminance template obtained by digitizing the luminance. This product-sum operation is
This indicates the correlation between the luminance pattern of the image and the luminance pattern of the luminance template. The larger the value of the product-sum operation, that is, the larger the product-sum value, the higher the correlation between the two. Therefore, when a product-sum operation is performed and the product-sum value is larger than a predetermined threshold value, it is possible to detect that a white line exists at that position.

FIG. 6 shows an example of a conventionally used luminance template 500. In the normal running state of the vehicle, in consideration of the fact that the white line in the image appears at an angle of about 45 °, the upper part of the upper side of the 45 ° boundary line has a brightness value of 1.
(White), and the lower part is a luminance value -1 (black). That is, the region 500a of the luminance value 1 with the boundary of 45 ° as a boundary
And a region 500b having a luminance value of -1 constitutes a luminance template 500. With such a luminance template 500, the right edge of the left white line can be detected.

However, the inclination of the white line in the image is 45
When the angle deviates from °, the sum of products becomes small when the conventional luminance template 500 shown in FIG. 6 is used, and when there is another object other than the white line having a contrast near 45 °. In some cases, the product-sum operation value with this object may be larger than the product-sum operation value with the white line, and the object may be erroneously recognized as a white line.

Therefore, in this embodiment, the two regions 500a, 500a having the luminance values 1 and -1 are
Instead of using the luminance template consisting of b, first weighting of the high luminance part (weight 1 of the white line part in FIG. 6)
And a third weight whose absolute value is sufficiently smaller than the second weighting of the low-luminance portion (weight-1 of the asphalt portion in FIG. 6).
Template matching with a road image is performed using a luminance template having a weighting of a predetermined range.

FIG. 7 shows an example of a luminance template according to the present embodiment. 6 is different from the luminance template of FIG.
A region 500c with a weight of 0 is placed at the boundary of the region 500b with a 1
And 500d. These areas 500
c and 500d are provided in a predetermined range of the boundary between the area 500a of weight 1 and the area 500b of weight-1. Specifically, as shown in the figure, the angle is set to a region which is increased or decreased by a predetermined angle with respect to the boundary line of 45 °. As the predetermined angle, for example, the angle α can be set to 63 °.

The difference between the product-sum operations when the luminance templates shown in FIGS. 6 and 7 are used will be described below.

FIG. 8 schematically shows an example of a road image obtained by the CCD sensor 24. (A) is a white line image with an inclination of 63 °, and (B) is an object image other than a white line with an inclination of 45 °. In (A), the brightness of the white line portion is 245 and the brightness of the asphalt portion is quantified to 10, indicating that the contrast is large. Also,
In (B), the bright part of the object is 230 and the dark part is 2
The numerical value is 5 and the contrast is lower than that of the white line image. When such two images exist in the same road image, in order to correctly recognize the white line, the product-sum operation value shown in FIG. It must be larger than the calculated product-sum operation value. If the product-sum operation value of the image shown in (B) is larger than the product-sum operation value of the white line image shown in (A), the object image shown in (B) is regarded as a white line. The white line image shown in (A) is erroneously recognized as a non-white line.

Now, the image shown in FIG. 8 is divided into a predetermined number of areas, for example, 16 areas, and the area of all the images is normalized to 1 and the product sum with the luminance template shown in FIGS. The operation is as follows. That is, (A)
The product-sum operation value S1 of the image shown in FIG. 6 and the conventional luminance template shown in FIG.

S1 = 245 × 7/16 + 10 × 1/16 + (− 245) × 1/16 + (− 10) × 7/16 = 88.125 (1) Further, it is shown in (B). The product-sum operation value S2 of the image and the conventional luminance template is

S2 = 230 × 8/16 + (− 25) × 8/16 = 102.5 (2) Therefore, since S1 <S2, when the conventional luminance template 500 shown in FIG. 6 is used,
The object image shown in (B) is erroneously recognized as a white line image.

On the other hand, the product-sum operation value S of the image shown in FIG. 7A and the luminance template of this embodiment shown in FIG.
1 'is

S1 ′ = 245 × 6/16 + 0 × 2/16 + 0 × 2/16 + (− 10) × 6/16 = 88.125 (3) Further, the image and figure shown in (B) 7, the product-sum operation value S2 ′ with the luminance template of the present embodiment is

S2 ′ = 230 × 6/16 + 0 × 2/16 + 0 × 2/16 + 0 × 2/16 + (− 25) × 6/16 = 76.875 (4) Therefore, since S1 ′> S2 ′, in the product-sum operation using the luminance template of the present embodiment,
The object image shown in (B) is not erroneously recognized as a white line, and the white line image with an inclination of 63 ° shown in (A) can be correctly recognized as a white line. This is because, by setting an area with a weight of 0 near 45 °, even if the inclination of the white line slightly changes from 45 °, it does not significantly affect the product-sum operation. That is, the area with the weight of 0 functions as a dead zone for the product-sum operation.

Although the case where the right edge of the left white line is detected has been particularly described above, the third case is similarly applied to the case where the left edge of the left white line or the left edge or the right edge of the right white line is detected. It can be detected using a weighted luminance template.

By devising a luminance template, both edges of a left white line or a right white line can be detected at the same time. FIG. 9 shows an example of the luminance template 500 when the left and right edges of the left white line are simultaneously detected. The white line is the area 500g with weight 1,
The asphalt portion is a region 500e and a region 500i with a weight of −1, and regions 500f and 500h with a weight of 0 are provided at the boundary between the white line portion and the asphalt portion. The left edge of the left white line is detected in the regions 500e to 500g, and the right edge of the left white line is detected in the regions 500g to 500i.

As described above, by providing the area of weight 0 at the boundary between the white line portion (high-luminance portion) and the asphalt portion (low-luminance portion), the white line is not affected by the variation of the white line width and the variation of the inclination. Can be detected.

It is also possible to set a plurality of third weighting values to be set for the high luminance portion and the low luminance portion.

FIG. 10 shows an example in which a plurality of (three in the figure) third weights are set. In addition to the area 500a having a weight of 1 and the area 500b having a weight of -1, an area 50 having a weight of 0.25 is located within a predetermined range of the boundary.
0j, area 500k with a weight of -0.75, area 500m with a weight of 0.75, area 5 with a weight of -0.25
00n is provided.

As described above, by setting a plurality of third weights, it is possible to preferentially detect the inclination of the white line in a specific direction. FIG. 10 shows a luminance template that can preferentially detect a white line with a particularly large inclination. As a specific running state of the vehicle, a white line when a lane is changed from a certain lane to an adjacent lane is detected. It is suitable for. Hereinafter, a specific product-sum operation will be described.

FIG. 11 schematically shows white line images having various inclinations. (A) is a white line image when the inclination is 63 °, the luminance value of the white line portion is 245, and the luminance value of the asphalt portion is 10. (B) shows a slope of 27.
(C) is a white line image when the inclination is 45 °. Normally, a white line image having an inclination of 45 ° is obtained as shown in (C), but when the vehicle changes lanes, the inclination of the white line in that direction becomes large as shown in (A). On the other hand, the slope of the opposite lane becomes small as shown in FIG. The purpose of the luminance template 500 shown in FIG. 10 is to preferentially detect a white line on the side where the lane change is performed, that is, a white line image with an inclination larger than 45 ° shown in FIG. It is in.

When the product-sum operation is performed on each of FIGS. 11A, 11B, and 11C using the luminance template shown in FIG. 10, the following results are obtained. That is, (A)
-Sum operation value S3 'of the luminance template 500 and

S3 ′ = 245 × {1 × 6/16 + 0.75 × 1/16 + (− 0.25) × 1/16} + 10 × Δ−1 × 6/16 + (− 0.75) × 1 / 16 + 0.25 × 1/16｝ = 95.46875 (5) Further, the product-sum operation value S4 ′ of the image of FIG.

S4 ′ = 245 × {1 × 6/16 + 0.25 × 1/16 + (− 0.75) × 1/16} + 10 × Δ−1 × 6/16 + (− 0.25) × 1 / 16 + 0.75 × 1/16｝ = 80.78125 (6) Further, the product-sum operation value S5 ′ of the image of FIG.

S5 ′ = 245 × {1 × 6/16 + 0.25 × 1/16 + 0.75 × 1/16} + 10 × ｛(− 1) × 6/16 + (− 0.25) × 1/16 + ( −0.75) × 1/16｝ = 102.8125 (7) Accordingly, S3 ′> S4 ′, so that the image shown in FIG. Is more easily detected as a white line.

On the other hand, when the conventional luminance template shown in FIG. 6 is used, the product-sum operation values S3, S4, and S5 with the images shown in (A) to (C) are as follows. become.

[0067]

S3 = 245 × 1 × 6/16 + 10 × (−1) × 6/16 = 88.125 (8)

S4 = 245 × 1 × 6/16 + 10 × (−1) × 6/16 = 88.125 (9)

S5 = 245 × 1 × 6/16 + 10 × (−1) × 6/16 = 88.125 (10) That is, S3, S4, and S5 all have the same value, and the slope of the white line It can be seen that it is impossible to detect the image shown in (A) larger than 45 ° as a white line with higher priority than the image shown in (B). From this, the effectiveness of the luminance template shown in FIG. 10 will be clear.

Although the present embodiment has been described above, the third weighting value in this embodiment is not necessarily 0 (in the case of FIG. 7), 0.25, -0.75 or the like (in the case of FIG. 10).
However, any value can be used as long as its absolute value is sufficiently small as compared with 1 as the first weight and −1 as the second weight. Also, the value of the first weight and the value of the second weight are
Any value can be used without being limited to 1, -1.

The predetermined range having the third weight can be determined as appropriate, but can be determined as follows, for example. That is, the horizontal direction of the image is X, the vertical direction is Y, and the distance from the CCD sensor 24 to the left white line and the right white line in the direction perpendicular to the axle is BL and B, respectively.
R, the height of the CCD sensor 24 from the road surface is H, and the vertical and horizontal size of one pixel on the CCD sensor 24 is pd.
When v and pdh, the yaw angle of the vehicle with respect to the white line is yaw, and the angle of the optical axis with respect to the road surface is th, the inclination PL of the left white line and the inclination PR of the right white line on the image are:

PL = (BL × cos (th) / (− H) + yaw × sin (th)) / pdh × pdv (11)

## EQU12 ## PR = (BR × cos (th) / (− H) + yaw × sin (th)) / pdh × pdv (12) Here, since the height H of the CDD and the sizes pdh and pdv of one pixel are known and constant, BL, B
P when R, th, yaw are changed within the expected range
The angle α in a predetermined range may be calculated from the maximum and minimum values of L and PR.

<Third Embodiment> FIG. 12 is a block diagram showing the configuration of the third embodiment. This embodiment is particularly effective when a puddle exists on the road surface and a high-luminance portion other than a white line appears.

In the present embodiment, the photographing means has two systems for vertical polarization and horizontal polarization. The vertical polarization filter 34 transmits only the vertical polarization component of the incident light, and projects the transmitted vertical polarization light onto the vertical polarization CCD sensor 24a.

The horizontal polarization filter transmits only the horizontal polarization of the incident light, and projects the transmitted horizontal polarization light onto the horizontal polarization CCD sensor 24b.

The vertical polarization CCD sensor 24a and the horizontal polarization CCD sensor 24b convert the incident light into electric signals and output the electric signals to the synchronous output amplifiers 26a and 26b.

The synchronous output amplifiers 26a and 26b amplify the input electric signal and output the amplified electric signal to the image processing device 28, respectively.

The image processing device 28 includes A / D converters 28a and 28d for vertical polarization and horizontal polarization, frame memories 28b and 28e, and an image recognition processor 28.
c, and converts the input image signal into a digital signal and stores it in the frame memories 28b and 28e.

The image recognition processing section 28c is constituted by a microcomputer, reads out the image data stored in the frame memories 28b and 28e as appropriate, removes a specular reflection component due to a puddle, which will be described later, and scatters a light image on a road surface other than a puddle. Only the white lines in the image are detected from the scattered light image. The detected white line is supplied to the display unit 30 and to the vehicle control unit 32. The display unit 30 and the vehicle control unit 32 display the detected white line on a CRT, a liquid crystal, or the like, and detect the relative displacement of the vehicle with respect to the white line, similarly to the first and second embodiments described above, to detect the steering angle of the vehicle. And so on.

As described above, the present embodiment is characterized in that the specular reflection component included in the input image is removed by the image recognition processing unit 28c, and an image including only the scattered light component is obtained to detect a white line. .

Conventionally, there has been proposed a method of removing a specular reflection component due to a puddle or the like in order to prevent the sunlight from being specularly reflected due to a puddle on the road or the like and erroneously recognizing the puddle as a white line on the road. . For example, JP-A-7-
128059 discloses a technique for detecting a white line from a horizontally polarized image, paying attention to the fact that the horizontal polarization component of specular reflection becomes substantially zero at Brewster's angle. According to this technique, the specular reflection component in the vicinity of the Brewster angle is 0, so that only the scattered light component can be extracted and a white line can be detected. In the case where a camera is installed in front of the camera and a front image is photographed), the horizontal polarization image gradually increases from 0, so that the horizontal polarization image also includes the specular reflection component. There is a problem that it is difficult to detect only a certain white line.

Therefore, in the present embodiment, in order to solve the problem when only the horizontally polarized image is used, the vertical and horizontal polarized light components are skillfully used to perform specular reflection regardless of the incident angle. The white line is reliably detected by removing the component.

FIG. 13 shows a case of specular reflection (a puddle).
The relationship of the reflectance with respect to the incident angle is shown. In the figure, the horizontal axis is the incident angle (deg), that is, the angle with respect to the normal to the reflecting surface, and the vertical axis is the reflectance. Curve a is the vertical polarization component, curve b is the horizontal polarization component, curve c is the average of the mixed light of the vertical polarization component and the horizontal polarization component (that is, (a + b) / 2), and curve d is the vertical polarization component a and the horizontal polarization component. The difference (polarization difference) of the component b is shown. When the incident angle is the Brewster angle, the horizontal polarization component becomes 0 as shown in the figure. On the other hand, when the incident angle becomes larger than the Brewster angle, the horizontal polarization component gradually increases as does the vertical polarization component. The polarization difference indicated by the curve d shows a characteristic that it gradually increases up to an incident angle near 80 °, and gradually decreases when it exceeds 80 °.

Here, in the case of scattered light, since the vertical polarization component and the horizontal polarization component are contained in substantially equal amounts, the polarization difference which is the difference between the vertical polarization component and the horizontal polarization component is substantially independent of the incident angle. It becomes 0. Therefore, by detecting the polarization difference value, it is possible to identify whether the incident light is scattered light or specular reflected light. The image recognition processing unit 28c specifies the specular reflection light based on the above principle, and removes the pseudo white line image due to the puddle by removing only the specular reflection component from the road image.

FIG. 14 shows a specific calculation process in the image recognition processing section 28c. First, the image recognition processing unit 28c acquires the vertically polarized image A (S10).
1). Next, a horizontally polarized image B is obtained (S102), and a difference between the two images is calculated to calculate a difference image D = | AB | (S103). This difference image reflects the specular reflection component as described above.

However, when only the scattered light component is extracted by subtracting the difference image from the horizontal polarization image B, for example, the horizontal polarization image itself also includes a specular reflection component corresponding to the incident angle (see FIG. 13). ). Therefore, as a correction coefficient corresponding to the incident angle, a value obtained by dividing the polarization difference at the incident angle θ by the horizontal polarization component reflectivity at the incident angle θ is defined as a coefficient I. The specular reflection component E is calculated (S104).
That is, the correction coefficient I is a ratio of the horizontal polarization component to the polarization component difference, and when a polarization difference exists at a certain incident angle, D is calculated by the reflection characteristic at the boundary between water and air.
A coefficient derived from the fact that only I horizontal polarization components are included. For example, when the incident angle is 80 °, the polarization reflectance difference value is 0.24, and the horizontal polarization component reflectance is 0.2.
Since it is 22, the correction coefficient I = 0.24 / 0.22.
Since the horizontal polarization reflectance is 0 at the Brewster angle, the correction coefficient is set to 0.

After calculating the specular reflection component E at the incident angle θ as described above, the image recognition processing unit 28c calculates the scattered light image F from the difference between the horizontal polarization image B and the specular reflection component E (S105). ). Then, a white line in the image is detected from the scattered light image F and output to the display unit 30 and the vehicle control unit 32 (S106).

The white line recognition in S106 can use, for example, the template matching method in the second embodiment. The difference calculation or the like is specifically a matrix calculation using the pixel value of each image as a matrix component. It can be carried out.

The correction coefficient I in S104 is determined as a function of the incident angle. The incident angle in the image can be calculated from the position in the image.
That is, the smaller the incident angle, the lower the position in the image,
The larger the angle of incidence, the higher the position of the image. Therefore, the incident angle may be estimated based on the vertical position in the image, and the correction coefficient I may be calculated for each incident angle based on the incident angle. Specifically, since the mounting position and angle of the CCD camera 24 on the vehicle are known, the incident angle can be calculated from the position on the image, assuming that the road surface is constant from directly below the vehicle. That is, the angle of the optical axis with respect to the road surface of the camera is a
If the number of pixels in the vertical direction (vertical direction) from the image center of the target pixel is b (downward is positive) and the shooting angle for one pixel in the vertical direction is c, the incident angle d is d
= 90− (a + bc).

FIGS. 5 to 19 schematically show the processing described above. FIG. 15 is a road image when the vertical polarization filter and the horizontal polarization filter 36 are not used. Both the scattered light component and the specular reflection component are included.For example, in rainy weather, there is a puddle portion in addition to the white line portion as a high luminance portion, and it is difficult to detect only the white line by a pattern matching method or the like. It is.

FIG. 16 is a horizontal polarization image B transmitted through the horizontal polarization filter 36. In the upper part of the image (part farther away) where the incident angle is large, particularly a large amount of specular reflection components appear (see FIG. 13). ). Near the Brewster angle, the specular reflection component is almost zero.

FIG. 17 shows a vertical polarization image A transmitted through the vertical polarization filter 34. Almost all regions including the white line portion become high luminance portions, and it is almost impossible to extract only the white line portion. This is because the vertical polarization component of the specular reflection component has a finite value at all incident angles.

FIG. 18 shows a difference image D obtained by subtracting the horizontal polarization image B from the vertical polarization image A. Since the scattered light contains substantially equal amounts of the horizontal polarization component and the vertical polarization component, the white line image portion due to the scattered light is almost zero, and many specular reflection components other than the scattered light appear.

FIG. 19 schematically shows an image obtained by subtracting the difference image D corrected by the correction coefficient I from the horizontal polarization image B which is the final result. Only the white line image due to the scattered light is extracted, the asphalt portion including the puddle is removed, and the specular reflection component is not included. Therefore, only the white line can be easily detected from the image shown in FIG.

In this embodiment, the scattered light image is obtained by subtracting the difference image from the horizontal polarization image. However, instead of the horizontal polarization image, the vertical polarization image A or the horizontal polarization image and the vertical polarization image are obtained. It is also possible to obtain a scattered light image by performing a difference operation between the mixed image (the average of the horizontal polarization image and the vertical polarization image or an image that does not pass through the polarization filter) and the difference image. At this time, for example, when a vertically polarized image is used, the correction coefficient I
The polarization difference at the incident angle θ may be divided by the vertical polarization component reflectance at the incident angle θ, and the same applies when a mixed image is used.

Although the embodiment of the present invention has been described above, the processing in the first to third embodiments is the same as that of the image processing apparatus 2.
8 can be executed by installing a processing program in the image recognition processing unit 28c. Such a program can be supplied from a medium on which the program is recorded, and the medium may be a CD-ROM or DVD-RO
It is possible to use any medium such as M, a magnetic disk such as a hard disk and a floppy disk, and a semiconductor memory.

[0094]

As described above, according to the present invention,
Lines on the road can be reliably detected irrespective of the traveling environment or the photographing environment.

[Brief description of the drawings]

FIG. 1 is a configuration block diagram of a first embodiment of the present invention.

FIG. 2 is an explanatory diagram of a first image and a second image in the embodiment.

FIG. 3 is an explanatory diagram of setting a search window in the embodiment.

FIG. 4 is a configuration block diagram according to a second embodiment of the present invention.

FIG. 5 is an explanatory diagram of a pattern matching method in the embodiment.

FIG. 6 is an explanatory diagram of a conventional luminance template.

FIG. 7 is an explanatory diagram of a luminance template in the embodiment.

FIG. 8 is an explanatory diagram showing an example of a road image in the embodiment.

FIG. 9 is an explanatory diagram of another luminance template in the embodiment.

FIG. 10 is an explanatory diagram of still another luminance template in the embodiment.

FIG. 11 is an explanatory diagram showing another example of the road image in the embodiment.

FIG. 12 is a configuration block diagram according to a third embodiment of the present invention.

FIG. 13 is a graph showing the relationship between the incident angle and the reflectance.

FIG. 14 is a processing flowchart in the embodiment.

FIG. 15 is an explanatory diagram of an image when the light does not pass through the polarizing filter in the embodiment.

FIG. 16 is an explanatory diagram of a horizontally polarized image.

FIG. 17 is an explanatory diagram of a vertically polarized image.

FIG. 18 is an explanatory diagram of a difference image obtained by subtracting a horizontal polarization image from a vertical polarization image.

FIG. 19 is an explanatory diagram of an image obtained by subtracting a difference image from a horizontal polarization image.

[Explanation of symbols]

1 optical system, 24 CCD sensor, 26 amplifier, 28
Image processing device, 30 display unit, 32 Vehicle control unit.

Continuation of the front page (56) References JP-A-7-221924 (JP, A) JP-A-6-119040 (JP, A) JP-A-6-341821 (JP, A) JP-A-3-118612 (JP) , A) Japanese Patent Application Laid-Open No. 8-13989 (JP, A) Japanese Patent Application Laid-Open No. 9-149314 (JP, A) Japanese Patent Application Laid-Open No. 8-167623 (JP, A) Japanese Patent Application Laid-Open No. 7-105486 (JP, A) 7-141592 (JP, A) JP-A-5-303625 (JP, A) JP-A-7-239996 (JP, A) (58) Fields investigated (Int. Cl. ⁷ , DB name) G06T 1/00 330 G06T 1/00 420 B60R 21/00 624 G06T 7/60 200

Claims (4)

(57) [Claims] 1. A two-dimensional CCD sensor, and a road image is projected on a first area of the two-dimensional CCD sensor with a first exposure amount, and the road image is projected on a second area of the two-dimensional CCD sensor with a second exposure amount. An optical system for projecting an image, and a first output from a first area of the two-dimensional CCD sensor.
Detecting means for detecting a line in the road image based on an image and a second image output from a second area of the two-dimensional CCD sensor, wherein the first area, the second area, said second exposure amount and the first exposure amount varies from each other, said detecting means, a corresponding plurality of positions of the first image and the second image
Setting multiple search windows for line detection
In the search window that corresponds to the
A selection method to select the processing result in any of the search windows
A line detection device for a vehicle , comprising: a step ; 2. The apparatus according to claim 1, wherein said setting means includes an upper part and a lower part of said first area and a front part.
The upper and lower portions of the second region corresponding to the first region;
Upper search window and lower search for line detection
Setting a window, wherein the selection means includes an upper search window of the first area.
Processing results in the upper search window of the second area
One of the results, and the lower part of the first area
The processing result in the search window and the lower server in the second area
A line detection device for a vehicle , wherein any one of the processing results in the window is selected . 3. A method of detecting the line of the road, the exposure amount of different identical road image in a single CCD different areas
Projecting, and a plurality of corresponding portions of the same road image having different exposure amounts
Stearyl setting multiple service switch window for line detection
And in the search windows that correspond to each other,
By selecting the processing result in one of the search windows
Detecting a line. 4. A program for detecting a line on a road.
A medium on which the program is stored.
Projected onto different areas of a single CCD with different exposures.
One road image is input, and a plurality of corresponding portions of the same road image having different exposure amounts
In the search window for line detection is set a plurality in the corresponding search window from each other, Izu respectively
By selecting the processing result in one of the search windows
Record a program characterized by detecting lines
Medium.