JPH07141486A - Picture sensor for vehicle - Google Patents

Abstract

PURPOSE:To easily and accurately detect the vehicle pitch angle based on the relation between the pitch angle and the width of the white lines on the road on a picture sensor for vehicle which detects the vehicle pitch angle based on the width of the white lines on the road or traffic lens photographed by a camera fixed on the vehicle. CONSTITUTION:When the pitch angle of a vehicle is '0', the position where the white line separated by the prescribed distance is photographed, is set as Y=Ya. When a specific pitch angle occurs in the vehicle, templates with some kinds of width are set corresponding to the width of the white line on the road to be photographed at the position of Y=Ya. The comparative picture with the same size is segmented from the area of Y=Ya during the camera is picking up the image. By performing the correlation arithmetic operation by template matching, the template with the highest correlation is selected. The pitch angle to be set on the corresponding template is detected as the pitch angle of the vehicle.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a vehicular image sensor, and more particularly to a vehicular front side photographed by a camera fixed to the vehicular vehicle and based on the width of a road white line or a traveling lane in the obtained picked-up image. The present invention relates to a vehicular image sensor that detects a pitch angle occurring in a vehicle.

[0002]

2. Description of the Related Art Conventionally, for example, Japanese Patent Laid-Open No. 64-1560.
As disclosed in Japanese Patent Laid-Open No. 5 (1994), there is known a device that images the front of the vehicle with a camera and estimates the pitch angle generated in the vehicle based on the state of the road white line in the obtained captured image.

In this device, when the camera is fixed to the vehicle, the elevation angle of the camera changes according to the pitch angle of the vehicle, and as a result, the position of the white line in the captured image changes according to the pitch angle. However, a method of estimating the pitch angle based on the coordinates of the vanishing points of the left and right white lines in the captured image is adopted.

That is, when the camera is installed in the center of the vehicle, the road white line drawn on the left side of the vehicle is imaged as a curve (or straight line) rising to the right in the captured image, and the road white line drawn on the left side of the vehicle. Is imaged as a curve (or a straight line) rising to the left in the captured image. Therefore, the two photographed white lines always have a relationship of intersecting with each other, and the vanishing point mentioned above is a point indicating an intersection of these two white lines in the image.

In this case, if tangents are obtained for each of the left and right white lines and the points where they intersect are grasped as the points of intersection,
The intersection will move depending on the pitch of the vehicle, that is, if the vehicle is moving downwards, it will move upwards in the captured image, and if the vehicle moves upwards, it will move downwards in the captured image.
By monitoring the change in the coordinates, the pitch angle of the vehicle can be detected, and the conventional device described above performs the pitch angle detection using this principle.

[0006]

However, since the above-mentioned conventional apparatus has an essential requirement to detect vanishing points of two road white lines, at least two points are provided for each of the left and right white lines.
It is necessary to detect the white line position at a total of four points and to detect the tangent line at each of these points, which causes a problem that the processing takes a long time.

Two tangent lines are detected for each of the left and right white lines in order to properly estimate the vanishing point in the case where the road is curved. It is difficult to perfectly match and grasp the vanishing point in the curve and the vanishing point in the curve, and there is room for improvement in the detection accuracy of each pitch.

The present invention has been made in view of the above points, and when a pitch angle occurs in a vehicle and the elevation angle of the camera changes, a white line or a lane imaged in a specific area in the captured image is accordingly recorded. Focusing on the fact that the width of the vehicle changes, the above problem is solved by detecting the pitch angle of the vehicle by comparing and considering the reference width and the actual road white line or lane width when the pitch angle is “0”. An object is to provide an obtained vehicle image sensor.

[0009]

FIG. 1 shows the principle configuration of an image sensor for a vehicle which achieves the above object. That is, the above-mentioned purpose is, as shown in FIG. 1 (A), when the camera M1 is fixed to the vehicle to photograph the front of the vehicle, and the image is taken by the camera M1 in a situation where no pitch angle is generated in the vehicle. In the reference width setting means M2 for setting the width of the road white line or lane in the captured image as a reference width at a position separated by a predetermined distance from the vehicle, and a pitch angle is generated in the vehicle in the image captured by the camera M1. In a reference area where the white line or the lane is imaged with the reference width in a situation where there is no such situation, the horizontal width detection means M3 that detects the width occupied by the road white line or the lane actually photographed in the captured image, and the reference width setting Based on the reference width set by the means M2, a pitch-corresponding width setting for setting the width of a road white line or lane estimated to be imaged in the reference area when a specific pitch angle occurs in the vehicle The pitch angle occurring in the vehicle is detected based on the means 4, the actual width detected by the horizontal width detecting means 3, and the pitch corresponding width set by the pitch corresponding width setting means 4 corresponding to the pitch angle. This is achieved by the vehicle image sensor including the pitch angle detecting means 5.

Further, in the vehicular image sensor having the above structure, the reference width setting means M2 sets a plurality of reference widths corresponding to a plurality of different predetermined distances, and the horizontal width detection means M is set.
3 detects the widths of road white lines or lanes in a plurality of reference areas set corresponding to the plurality of reference widths, and the pitch corresponding width setting means M4 detects a plurality of pitches corresponding to the plurality of reference widths. A corresponding angle is set, and the pitch angle detecting means M5 causes the vehicle based on the plurality of widths detected by the horizontal width detecting means M3 and the plurality of pitch corresponding widths set by the pitch corresponding width setting means M4. The configuration for detecting the pitch angle that is present is effective in the sense of improving the pitch angle detection accuracy.

Further, a camera M1 fixed to the vehicle to photograph the front of the vehicle, and a road white line at a position separated from the vehicle by a predetermined distance when the camera M1 photographs the vehicle in a situation where no pitch angle is generated. Alternatively, reference width setting means M for setting the width occupied by the lane in the captured image as the reference width.
2, and in the image captured by the camera M1, a reference area in which the white line or the lane is imaged with the reference width and a plurality of areas set above and below the reference area in a situation where the vehicle does not have a pitch angle. A plurality of horizontal width detecting means M6 for detecting the width occupied by a road white line or a lane actually photographed in the captured image, an actual width detected by the plurality of horizontal width detecting means M6, and the reference width setting means M2. A vehicle image sensor including a pitch angle detecting means M7 for detecting a pitch angle occurring in the vehicle based on the reference width set by is also effective.

[0012]

In the vehicular image sensor according to the present invention, the camera M1 photographs road white lines and lanes in front of the vehicle.
At this time, the elevation angle of the camera M1 changes according to the pitch angle of the vehicle, and when the vehicle pitches forward and downward, the road white line and the lane at a position separated from the vehicle by a predetermined distance move upward in the captured image, If is pitched forward and upward, it moves downward in the captured image.

In this case, the width of the road white line and the width of the lane are
The width of the road white line and the width of the lane in the reference area monitored by the horizontal width detection means M3 are wider when the vehicle pitches downward to the front and wider when the vehicle pitches upward to the front. Are narrow and change depending on the pitch angle.

On the other hand, the reference width setting means M2 sets the width of a road white line or a lane that will be imaged in the reference area when the vehicle is not pitching, as the reference width, and the pitch width setting means M4. Based on this reference width, sets the width of the road white line or lane that will be imaged in the reference area when a pitch occurs in the vehicle in association with the pitch angle.

Then, the pitch angle detecting means M5 and the road width or lane width in the reference area detected by the horizontal width detecting means M3 and the pitch corresponding width setting means M4.
The pitch angle generated in the vehicle is detected by associating with the pitch corresponding width detected by.

Further, the reference width setting means M2, the horizontal width detecting means M3, the pitch corresponding width setting means M4, and the pitch angle detecting means M5 respectively detect the road white line or the lane at a plurality of different positions separated by a predetermined distance from the vehicle. When the pitch angle is detected based on the width by the above method, higher detection accuracy can be obtained than when the pitch angle is detected based on the width of the road white line or the lane at a single position.

By the way, the plural horizontal width detecting means M6
Detects the road white line width or lane width not only in the reference region in the image captured by the camera M1, but also in a plurality of regions above and below the reference region. Therefore, when the vehicle pitches forward or downward, if the pitch does not occur, the road white lines and lanes to be imaged in the reference area shift to the upper or lower area of the reference area, and the road in the area is changed. It is detected as the white line width or the lane width.

Then, the pitch angle detecting means M7 determines which one of the plurality of horizontal widths detected by the plurality of horizontal width detecting means M6 corresponds to the reference width set by the reference width setting means M2, and then from the vehicle. While recognizing that the white line and the lane at positions separated by a predetermined distance are photographed in the detected area of the horizontal width, the pitch angle is detected based on the position of the area in the captured image.

[0019]

FIG. 2 is a block diagram showing the overall structure of a road white line detecting apparatus according to an embodiment of the present invention. In the figure, the camera 10 corresponds to the above-mentioned camera M1, and is an A / D
It is connected to the image frame memory 14 via the conversion device 12. This gives the 1 obtained by the camera 10.
The image for the screen is stored in the image frame memory 14 as digital data.

In the image frame memory 14, a CPU 16 for realizing the above-mentioned reference width setting means M2, horizontal width detecting means M3, pitch corresponding width setting means M4, pitch angle detecting means M5, M7, and plural horizontal width detecting means M6 is provided. It is connected. The CPU 16 realizes these functions by reading out image data of a predetermined portion of the image frame memory 14 and using the correlation calculation device 18 and executing a program described later to perform various calculation processes. It is an essential part of this embodiment.

Data relating to the template set by the CPU 16 and data of the comparative image cut out from the image data of the image frame memory 14 by the CPU 16 are supplied to the correlation calculation device 18. Then, the correlation calculation device 18
Performs the correlation calculation by template matching based on these data.

The correlation calculation method by template matching will be briefly described below with reference to FIG.

FIG. 3A shows a template of m × n pixels used for template matching, and FIG. 3B shows a picked-up image in which a part of the image in the template is picked up. When performing template matching, first see FIG.
A search window (S) is roughly set in the captured image shown in (B). Then, an image of the same size as the template is cut out from this search window as a comparison image (C), the difference in luminance between the corresponding pixels is obtained, and the sum of the absolute values thereof is obtained as the correlation value D.

In this case, as shown in FIGS. 3A and 3B, when the image in the template and the comparison image do not match, the correlation value D is large, and the image in the template and the comparison image are When the values match completely, the correlation value D = “0”
Becomes

Therefore, when the comparison image is set by shifting one pixel in order within the set search window and the correlation value D with the template is obtained for each comparison image, the position where the correlation value D becomes the minimum is found. It should be a position where the comparative image with the highest correlation can be cut out. Therefore, the position can be specified as the existing position of the template image.

As described above, the correlation calculation by template matching is an effective method for detecting an image close to the template from within the search window. The vehicle image sensor of the present embodiment utilizes the above principle,
Coordinates in which the width of the road white line is equal to the width of the template in the captured image are detected, and the pitch angle of the vehicle is detected based on the coordinates.

Then, the CPU 16 outputs the pitch angle thus detected to a predetermined vehicle control device (not shown) via the pitch angle output device 20. A CRT 24 is connected to the output side of the image frame memory 14 via the D / A converter 22.
The images stored in the CRT 24 are sequentially displayed on the CRT 24.

The contents of the processing executed by the CPU 16 to detect the pitch angle of the vehicle in the vehicle image sensor of this embodiment will be described below.

FIG. 4 is a flow chart of an example of a main routine executed by the CPU 16 to realize such a function and to implement the invention described in claim 1. That is, FIG.
In the routine shown in FIG. 5A, as the road white line at a position where the distance from the vehicle is short, the width is photographed thicker, as shown in FIG. 5A, for example, at the position Y = Ya (when the pitch angle is “0”,
When the width Wa of the road white line (corresponding to the position (m)) is focused, it is a routine for detecting the pitch angle by utilizing the fact that Wa changes according to the pitch angle.

More specifically, in the storage device connected to the CPU 16, as shown in FIG. 5B, the number of pixels (corresponding to m pixels) × 1 pixel in which the pitch angle and the white line width are associated with each other. By storing the seed template Tn, the reference width setting means M2 and the pitch corresponding width setting means M described above are stored.
4 is realized and the pitch angle is detected by searching for the template Tn having the highest correlation with the width of the road white line detected in the search window set at the position Y = Ya in the captured image.

When the routine shown in FIG. 4 is started, the initial process is first executed in step 100. This initial processing is a step of detecting the X coordinate Xa of the position where the road white line exists from within the search window set at the position Y = Ya, and is implemented by the initial processing routine shown in FIG.

The position of the road white line (X
a, Ya), the white line width detection process is performed in step 200 to detect the width Wa of the road white line at the specified position. This step corresponds to the horizontal width detecting means M3 described above, and is realized by the white line width detection processing routine shown in FIG. 7 as described later.

In step 300, the white line width Wa at (Xa, Ya) detected in this way is shown in FIG.
In light of the preset map as shown in (B),
This is a step of detecting the pitch angle currently occurring in the vehicle. This step corresponds to the pitch angle detection means M5 described above, and is specifically realized by the pitch angle detection routine shown in FIG.

In the routine shown in FIG. 4, the reference width setting means M2 and the pitch corresponding width setting means M4 store the template Tn shown in FIG. 5 (B) and the pitch angle corresponding thereto as described above. Although implemented by a storage device, the reference width setting means M2 is provided with a function for learning the road white line width based on an image taken when the vehicle is in the reference state, and the pitch angle is determined based on the learned value. The pitch corresponding width setting means M4 may be realized by providing a function of appropriately calculating the width of the white line.

The contents of the subroutine for realizing the above steps 100 to 300 will be described in detail below.

When the initial processing routine shown in FIG. 6 is started, first, in step 102, a template Ti corresponding to the width of a standard road white line is selected from the templates Tn stored in the storage device.

Next, at step 104, initial values are set to start the correlation calculation by template matching. Here, the initial coordinates (x, y) of the comparison image cut out when executing the template matching are set to the coordinates (Xf / 2, Ya) at the center of the image, and the lowest correlation value Dx, y.
Dbuf for storing (corresponding to the correlation value D at the position where the highest correlation is obtained) is set to the upper limit value 10000, and Dbuf
Xbuf that stores the X coordinate of the obtained position of is set to 0, respectively.

When this processing is completed, the routine proceeds to step 106, where the coordinates of the left end portion are set to (x, y) m ×.
Correlation value Dx between the comparison image of one pixel and the template Ti,
y is obtained as the total sum of the absolute values of the brightness differences of the corresponding pixels.

Then, in step 108, the current Dbu
The value stored in f and the value of Dx, y calculated this time are compared to see if Dx, y <Dbuf holds.
The above condition is satisfied when the highest correlation is obtained this time. In this case, the process proceeds to step 110 and Dbuf, Xbuf are set to the current value Dx, to update the history.
Processing for rewriting y and X is performed. On the other hand, if the condition is not satisfied in step 108, it is not necessary to rewrite the history of the correlation value, and therefore, step 11 is performed as it is.
Go to 2.

In step 112, the comparison image is newly set to 1
X-coordinate value x of the left end of the pixel to cut out
Decrement. In step 114, it is determined whether x thus decremented has reached "0", that is, whether the scanning of the comparative image started from the center of the captured image has reached the left end of the captured image. Where x = "0"
Is not established, the process returns to step 106, and x =
Repeat the above steps 106 to 1 until "0" is established.
Process 4 is executed.

As a result, in step 114, x
At the time when it is determined that = "0" holds, the correlation value Dx, y at the position where the highest correlation is obtained from the center x = Xf / 2 of the captured image to the left end x = 0 is always Dbuf,
The x at that time is stored in Xbuf. Step 116 is a step of outputting the values of these Xbuf and Dbuf as Xa and Da, respectively, and this processing is executed and this routine ends.

The white line width detection processing routine shown in FIG. 7 takes an image (Xa, Ya) by searching for the template Tn having the highest correlation with the comparative image having Xa thus specified as the left end. It is a routine that realizes the width of the road white line being displayed.

That is, when the routine shown in FIG. 7 is started, initial values are set in step 202,
In order to specify the position of the comparative image, the settings X = Xa and Y = Ya are set. Further, the white line width (the number of white line pixels) Wi in the template Ti selected in the initial processing routine is substituted into K as the template number.

After completing the initial setting, step 20
See the value of flag in 4. This flag is a flag indicating the execution status of the white line detection processing, and is cleared to "0" immediately after the start of this routine. Therefore, if this time is the first processing, in step 204, f
It is determined that lag = “0”, and thereafter the process proceeds to step 206.

In step 206, the template number J for correlation calculation is calculated. Here, the number obtained by subtracting 1 from the white line width Wi is set as the template number J. This is because the correlation calculation is executed with a template that is narrower by one pixel than the template used for calculating the correlation value in the initial processing routine.

Step 208 is a step of performing the correlation calculation using the template of the template number J set as described above. Here, in this embodiment, the calculation is performed by the correlation calculation routine shown in FIG.

In the correlation calculation routine shown in FIG.
When the routine is activated, first, in step 250, the template TJ is selected. Here, the template TJ is the template with the template number J.

Next, at step 252, the initial values required for the correlation calculation are set. In this correlation calculation routine, an area having a predetermined width W centered on Xa specified as the position of the white line in the initial processing routine is set as the search area. In this step, X is set to specify the scanning start position of the comparison image. = Xa + W, Y = Y
The initial setting "a" is performed. Also, in the process of correlation calculation, the upper limit value 1 is set to Dbuf that stores the minimum correlation value Dx, y.
0000 and "0" are assigned to the Xbuf that stores the X coordinate at that time.

In step 254, the correlation value Dx, y is calculated from the difference in brightness between the corresponding pixel of the comparison image and the template TJ set in this way, and in step 256, the minimum value of Dx, y is updated. This is a step of rewriting Dbuf and Xbuf only when the above is performed. Further, step 258 is a step of decrementing the X coordinate value x of the comparative image so as to move it by one pixel.

Then, step 262 is a step of determining whether or not the result of decrementing, x <Xa-W, has been established, and the processing of steps 254 to 260 is performed until it is determined that the condition of this step is established. If it is determined that x <Xa-W is satisfied by repeated execution,
After that, the processing of step 264 is executed, and this routine ends.

In this case, the condition of step 262 is satisfied when the scanning of the comparison image from x = Xa + W to x = Xa-W is completed. Therefore, at this point, Dbuf has the smallest correlation value D calculated in the meantime.
The values of x and y should be stored in Xbuf, and x at that time should be stored in Xbuf. In step 264, this Dbuf,
And Xbuf are the template TJ of the template number J
Is a step of outputting as the correlation value DJ in X and the X coordinate XJ at that time.

When step 208 in FIG. 7 is executed for the first time, the correlation calculation routine shown in FIG. 8 is executed with J = Wi-1, and as a result, DWi-1 and XWi-1.
Will be calculated.

Step 210 is a step of determining whether the template number J used in the current correlation calculation is Wi + 1 with respect to the template number K = Wi used in the initial processing routine. If this is the first processing, J = Wi− as described above.
1 and J ≧ Wi + 1 is not established. Therefore, in this case, the process of step 208 is further performed after step 212.

Here, step 212 is a step of incrementing J. Therefore, last time J = Wi-1
If it is, it is rewritten to J = Wi and step 20
8, the correlation calculation is performed for the template with the template number J = Wi, and thereafter in step 210, J
Repeat steps 208 to 2 until ≧ Wi + 1 holds.
Twelve processes are performed.

That is, in step 210, it is determined that the conditions are satisfied when the correlation value DJ and the X coordinate XJ are calculated for J = Wi-1, Wi, Wi + 1, and these three adjacent templates are used. When the correlation calculation of is completed, the process of step 214 is performed thereafter.

In step 214, J calculated as described above is used.
= Wi−1, Wi, Wi + 1 is a step of determining whether the correlation value DWi for Wi is the smallest among the correlation values DJ. Here, if DWi is the minimum,
It can be determined that Ti has the highest correlation among the three templates.

Therefore, when such a condition is satisfied, the routine proceeds to step 216, the value of Wi is grasped as it is as the white line width Wi, and the flag is flagged at step 218.
Is set to "0", and in step 220 W
i, Xi, and Di are output, and the processing of this time is ended. still,
Here, Xi and Di are the X coordinate of the comparative image obtained corresponding to Wi and the correlation value (corresponding to the minimum correlation value) obtained for the comparative image.

On the other hand, if it is determined in step 214 that DWi is not the minimum value, the process proceeds to step 222, and DWi-1 calculated in step 210 above.
And DWi + 1, it is determined whether or not DWi-1 <DWi + 1 holds. In other words, by determining which correlation value is smaller, the template Ti-1, Ti + 1
Which of the two has a higher correlation with the comparison image is determined.

Here, DWi-1 <DWi + 1 holds when the template Ti-1 has a higher correlation. In this case, the process proceeds to step 224 to display such a situation, and the flag is flagged. Set “1” to. On the other hand, if the above conditions are not satisfied, the template T
i + 1 is the case where a higher correlation can be obtained,
In this case, the process proceeds to step 226 and "2" is set to the flag to display such a situation.

Upon completion of any of these processes, the process of step 204 is performed again. This time, flag =
Since it is not "0", it is determined that the condition is not satisfied, and subsequently, the process of step 228 is executed. This step 2
Reference numeral 28 is a step for checking whether "1" is set in the flag, and when flag = "1", the processing from step 230 onward is executed thereafter.

The flag becomes "1" when the width of the road white line photographed in the Y = Ya area is narrow with respect to the template Ti selected in the initial processing routine. In this case, the template Ti-1 is not always the template with the highest correlation,
In some cases, the narrower template Ti-2, TI-3, ... Has a higher correlation.

Therefore, when the processing from step 230 onward is executed, first, rewriting J = Wi-2 is performed in step 230, and this J = W in step 232.
The correlation value DJ is calculated for the i-2 template. The correlation value DJ is calculated by the correlation calculation routine shown in FIG.

Then, the correlation value D for J = Wi-2
When J is calculated, in step 234, DJ <DJ
+1 (DWi− calculated in step 210 above)
It is determined whether 1) is established. Where DJ <DJ
+1 holds when the template with J = Wi-2 has a higher correlation than the template with J = Wi-1. Therefore, in this case, step 236 is performed to calculate the correlation value for the narrower template.
Then, 1 is subtracted from J and the processing of steps 232 and 234 is performed.

On the other hand, DJ <DJ + 1 is not satisfied when the template TJ + 1 used in the previous correlation value calculation can obtain a higher correlation than the template TJ used in the latest correlation value calculation, that is, TJ + 1
This is the case where the highest correlation can be obtained with respect to the captured road white line.

Therefore, when such a determination is made,
In step 238, J + is added to Wi representing the width of the road white line.
1 is substituted, and the processing of steps 218 and 220 is executed thereafter, and the processing of this time is ended.

On the other hand, "2" is set in the flag, and flag =
If it is determined that “1” is not established, step 2
In step 40, the process of J = Wi + 2 is executed. fla
“2” is set in g when a template wider than the template Ti used in the initial processing routine can secure a higher correlation.
This is to compare the correlation values DJ for +1, Ti + 2, ...

That is, after the processing of step 240 is completed, the correlation calculation is performed for J = Wi + 2 in step 242, and DJ-1 <in step 244.
A determination is made as to the feasibility of DJ. If the above condition is not satisfied, that is, if the newly calculated DJ is less than or equal to the correlation value DJ-1 obtained for the narrower template TJ-1, then in step 246,
To further widen the width of the template, 1 is added to J, and the processing from step 242 above is repeated.

If it is determined that the condition of step 244 is satisfied, it is possible to determine that the template TJ-1 for which the correlation calculation has been performed last time is the template with the highest correlation, and in this case, step 248. In J, J-1 is substituted for the width Wi of the road white line, and the processes of steps 218 and 220 are executed thereafter, and the process of this time is ended.

As described above, in the vehicle image sensor of this embodiment, the CPU 16 executes the white line width detection processing, that is, the white line width detection routine shown in FIG. The template Tn having the highest correlation with the width of the road white line photographed in the region of = Ya is detected, and the value set as the white line width in the template Tn can be obtained as the white line width Wi. Here, in the present embodiment, as shown in FIG. 5B, the pitch angle is set for each template Tn, and Y in the captured image is set as described above.
When the white line width Wa in the region of = Ya is found to be Wi, the Pucci angle detection routine shown in FIG. 9 is started,
Wa obtained by searching the map shown in FIG.
The pitch angle corresponding to (= Wi) is specified.

As described above, in the vehicular image sensor of this embodiment, the pitch angle is detected based on the change in the photographed road white line width without obtaining the vanishing points of the left and right white lines drawn on the road. It can be carried out. For this reason, the processing contents are simplified and the execution speed is increased as compared with the conventional device, and the detection error is not emphasized even in the curve etc., and the pitch angle detection accuracy is significantly improved. You can

In the main routine shown in FIG. 4, attention is paid to the fact that the width of the road white line photographed in the area Y = Ya in the picked-up image changes according to the pitch angle of the vehicle. The pitch angle is detected by detecting the change in. Therefore, it is impossible to detect a minute change in which the width Wa of the road white line does not change in the captured image.

On the other hand, as shown in FIG. 10 (A), Y = Y
Assuming that Y = Yb and Yc are also watched in addition to the point a, as shown in FIG. 10B in which Wa at each point is displayed in correspondence with the pitch angle, the point Wa = Ya is Wa. Even if the area does not change (for example, in FIG.
In (B), Wa may change at a point of Y = Yb, Yc at a pitch angle of −2 ° to 1 °.

Therefore, if the road white line widths Wa, Wb, and Wc for three points of Y = Ya, Yb, and Yc are considered in combination, a minute pitch angle that cannot be detected by a configuration in which a single point is focused. Can also be detected.

Focusing on this point, FIG. 10 shows a flowchart of a main routine executed by the CPU 16 in order to realize the invention described in claim 2.

That is, when the routine shown in FIG. 10 is started, first, at step 100a, an initial process corresponding to step 100 in FIG. 4 is executed. Then, in steps 100b and 100c, Y = Yb,
The initial process for the point of Yc is executed. As a result, the X coordinate values Xa, Xb, Xc of the position where the road white line is photographed in each area are detected.

Then, based on these values Xa, Xb, Xc, in steps 200a, 200b, 200c,
The white line width detection process corresponding to step 200 in FIG. 4 is executed. As a result, the data necessary to refer to the map shown in FIG. 10B, that is, Y = Ya, Yb,
Road white line widths Wa, Wb, and Wc for Yc are obtained.

In step 310, these Y = Ya, Y
12 is a step of detecting the pitch angle of the vehicle based on the road white line widths Wa, Wb, and Wc for b and Yc.
This is realized by the pitch angle detection routine shown in. That is, in the pitch angle detection routine, the map search shown in FIG. 10B is performed using Wa, Wb, and Wc obtained in steps 200a to 200c (step 31).
2), the result is output as the pitch angle. When the detection of the pitch angle is completed in this way, the above step 2
The processing after 00a is repeatedly executed to continue the pitch angle detection processing.

In this case, a minute pitch angle change can be detected as compared with the case where the routine shown in FIG. 4 is executed, and the pitch angle detection accuracy can be further improved.

By the way, the device for recognizing the road white line in front of the vehicle, such as the vehicle image sensor of this embodiment, can be applied to the automatic steering using the road white line as a guideline. In this case, the data to be detected by the vehicular image sensor is the position of the road white line at a position separated from the vehicle by a predetermined distance, and if the X coordinate of the road white line in the captured image can be captured, this request is satisfied. become.

However, the camera 10 is a member that is fixed to the vehicle to the last, and since the vehicle pitches during traveling, the positions where the road white lines at the positions separated by a predetermined distance are photographed are not constant, but simply a specific Y value. Searching for the white line position in the coordinate area does not mean that the road white line at a position separated by a predetermined distance is monitored.

In this case, if the pitch angle of the vehicle can be detected as shown in the above embodiment, various corrections are made based on the detected pitch angle, and the position of the road white line in the picked-up image corresponds to the pitch angle. Despite moving up and down, it is possible to specify at what coordinate the road white line at a position separated by a predetermined distance is captured in the captured image. That is, the vehicular image sensor shown in the above-mentioned embodiment is a particularly effective device when realizing automatic piloting using the road white line as a guideline.

By the way, as shown in FIG. 13 (A), when the vehicle is in a steady state with a pitch angle of "0", the road white line at a position separated from the vehicle by a predetermined distance is Y in the captured image.
It is assumed that an image having a width Wa is captured at a position of = Ya. In this state, an appropriate pitch is generated in the vehicle, and the captured image is shown in FIG.
If the road white line as a whole shifts upward in the captured image as shown in (B), a road white line thicker than Wa is imaged at a position of Y = Ya, and a road white line of width Wa is above Y = Ya. There are cases where the image is taken at the position of Y = Yd.

At this time, since the width of the road white line has not actually changed, it is supposed that the road white line at a position separated from the vehicle by a predetermined distance should be photographed with the width Wa in the captured image. Considering this, in the state shown in FIG. 13B, the road white line captured at the position Y = Yd can be regarded as the road white line at a position separated from the vehicle by a predetermined distance.

That is, as shown in FIG. 13, the width of the road white line is monitored in advance for a plurality of points (Y = Ya to Ye) in the picked-up image, and the processing for detecting the position closest to Wa is performed. In this case, the vehicle pitch angle can be detected based on the Y coordinate of the detected position, and the road white line at a position separated by a predetermined distance from the vehicle is captured at any position in the captured image without detecting the pitch angle. Can be detected, which is advantageous for speeding up the process.

FIG. 14 is a flow chart of an example of a main routine executed by the CPU 16 to execute the pitch angle detection based on such a principle. The CPU 16 executes this routine to realize the invention of claim 3. Will be done.

When the routine shown in FIG. 14 is started, the white line width is first learned in step 400. This step 400 is a step for realizing the reference width setting means M2 described above in the present embodiment, and is realized by the white line width learning routine shown in FIG.

That is, when the execution command of step 400 is generated, the routine shown in FIG.
At 2, it is determined whether or not the differential value of the vehicle speed is "0". Here, the differential value of the vehicle speed does not become "0" when the vehicle is in the acceleration / deceleration state. Therefore, when such a condition is satisfied, it can be estimated that a pitch occurs in the front-rear direction, and in that case, the process of step 402 is executed again.

On the other hand, when it is determined that the differential value of the vehicle speed is "0", it can be predicted that the vehicle is in the state of the pitch angle "0", and therefore the process proceeds to step 404 in order to proceed the learning of the white line width. .

This step 404 is a step for checking whether the steering angle is "0". If it is determined that the steering angle is not “0”, that is, the vehicle is turning, it is predicted that the vehicle is rolling, so the above steps are performed from the beginning to confirm that the white line learning condition is satisfied. Four
Return to 02.

On the other hand, when the steering angle = "0" is detected, it can be predicted that the vehicle is traveling straight ahead and its posture is stable. Therefore, it is necessary to learn the width of the road white line. Proceed to 406.

Step 406 is step 2 in FIG.
00, that is, the step of detecting the white line width in the area Y = Ya in the captured image by the white line width detection routine shown in FIG. In this case, since the vehicle does not have a pitch angle, the width of the white line detected in the region of Y = Ya can be treated as it is as the width of the road white line at a position separated from the vehicle by a predetermined distance.

Therefore, when the processing of the above step 406 is completed, the routine proceeds to step 408, where the width Wa of the road white line detected this time is stored as the width Wr of the road white line at the position separated by the predetermined distance in the steady state, and this routine is executed. finish.

Instead of executing this routine, it is possible to store the reference road white line width Wa as in the routines shown in FIGS. 4 and 11. However, if the configuration that sets the reference white line width by executing this white line width learning routine is adopted, appropriate detection accuracy is maintained even if the width of the white line drawn on the road changes. The effect that it can be obtained can be obtained.

In this way, when the reference width Wr of the road white line in the steady state is learned, the next steps 500a-50
At 0e, the initial processing for the positions of Y = Ya to Ye is sequentially executed. These initial processes are realized by the initial process routine shown in FIG. 16, whereby the X coordinate of the white line in the area of Y = Ya to Ye is detected.

That is, when the learning of the white line width is completed, the routine shown in FIG.
In 2, the template TWr having the width Wr is selected. Then, in order to perform correlation calculation by template matching using the template TWr, step 50
In step 4, necessary initial value setting is performed.

In this routine, the comparative image is scanned from the center of the picked-up image, and (Xf / 2, Yi) is set as the scanning start coordinate (X, Y). If the processing this time corresponds to step 500a in FIG. 14, Ya is set for Yi, and then step 500b.
˜500e, Yi to Yb to Y, respectively.
Substitute e. Then, Dbuf that stores the minimum correlation value
At the position where the upper limit value of 10000 and the minimum correlation value are realized
Substituting "0" into Xbuf that stores the coordinates, step 5
Proceed to 06.

Steps 506 to 514 are the same as steps 106 to 114 shown in FIG. 6 above. The minimum correlation value is detected while scanning the comparison image pixel by pixel toward the left end of the image, and that value is Dbuf. And the X coordinate at that time is Xb
This is the step of storing in uf.

When the scanning of the comparative image is completed, the process proceeds to step 516, where the X coordinate of the position where the minimum correlation value is obtained in the region of Y = Yi, that is, the value stored in Xbuf as the white line position Xi. Output. As a result, the white line positions Xa to Xe in the respective areas are detected at the time when the processing of steps 500a to 500e is completed.

After this processing is completed, white line width detection processing is performed in steps 200a to 200e as shown in FIG. These steps 200a to 200e are realized by the white line width detection routine shown in FIG. 7, and Xa to Xe detected as described above are used as the central coordinates ±.
Correlation calculation by template matching is performed with the width of W as the white line width detection region, and the white line widths Wa to We in each region are detected.

By performing these processes, the above-mentioned FIG.
3, Y = Ya to Y set arbitrarily in the captured image.
In the area e, it is detected what width the road white line is actually photographed. In this sense, in this embodiment, the steps 200a to 200e realize the plural horizontal width detecting means M6.

In step 600, among Wa to We detected in this way, the one that most approximates Wr obtained as the learning value of the white line width is selected, and the white line of that width is detected. A white line data output process is executed to output the position as the position of the road white line at a position separated from the vehicle by a predetermined distance.

This processing is realized by the white line data output processing routine shown in FIG. In the routine shown in FIG. 17, first in Step 602, Wa to We are set.
Area Yi in which the white line closest to Wr is detected
Is specified as a region where the road white line at a position separated from the front of the vehicle by a predetermined distance is photographed.

Next, at step 604, output processing is performed at the position Xi where the white line is detected in Yi. In this case, for example, if Xi is output to the control device for automatic steering as the steering data of the vehicle, the positional relationship between the vehicle and the road white line at a position separated by a predetermined distance can be specified without calculating the pitch angle. Therefore, the time required to identify the steering data required for automatic piloting is shown in FIGS.
This has the effect of being able to shorten the time as compared with the case of executing the routine shown in FIG.

Step 700 in FIG. 14 is a step for detecting the pitch angle based on Yi or Wa to We thus specified. In this case, since Yi moves up and down according to the pitch angle, for example, if a map relating the area selected as Yi and the pitch angle is set, the map is searched by Yi as shown in FIG. By (step 702), the pitch angle can be detected.

Further, as in the routine shown in FIG. 11, the processing of steps 200a to 200e is nothing but detecting the white line width by gazing at a plurality of regions in the picked-up image. The same map as that set for Ya to Yc in B) is set for Ya to Ye,
As shown in FIG. 19, a configuration may be used in which the pitch angle is detected by searching the map with Wa to We.

As described above, according to the vehicle image sensor of the present embodiment, it is possible to specify the positional relationship between the vehicle and the position of the road white line at a position separated from the front of the vehicle by a predetermined distance without obtaining the pitch angle. This is advantageous in increasing the processing speed, and has the effect that the pitch angle generated in the vehicle can be detected with high accuracy as in the routines shown in FIGS. 4 and 11.

In each of the above embodiments, the pitch angle of the vehicle is detected by focusing on the width of the white line drawn on the road. However, the width between the left and right white lines, that is, the width of the lane The pitch width may be detected by focusing attention.
In this case, the lane width is wider than the white line width, and the amount of change generated according to the pitch angle is large, so that higher detection accuracy can be ensured.

[0108]

As described above, according to the first aspect of the invention, the pitch angle is detected by focusing on the fact that the width of the road white line in front of the vehicle or the width of the lane changes according to the pitch angle of the vehicle. Therefore, it is not necessary to obtain the vanishing point of the road white line as in the conventional device, and the processing speed can be increased. Also,
According to the vehicle image sensor of the present invention, even if the road curves, the road white line width and the lane width do not change, so that the accuracy of the pitch angle detection does not deteriorate due to the influence of the curve. Therefore, it has a feature that it can exert an excellent effect as compared with the conventional device.

Further, according to the invention of claim 2, the pitch angle detection according to the invention of claim 1 is performed at a plurality of positions in front of the vehicle. It is possible to detect a minute pitch angle change that does not appear. That is, the vehicular image sensor according to the present invention has the effect of being able to detect the pitch angle with higher detection accuracy than the invention according to claim 1 above.

On the other hand, according to the third aspect of the present invention, the plural horizontal width detecting means detects the width of the road white line or the lane for a plurality of regions in the image photographed by the camera. Means, the road white line or the lane at a position separated by a predetermined distance from the vehicle only by finding one that matches the reference width of the road white line or the lane set by the reference width setting means from among the plurality of detected horizontal widths. The area being photographed can be specified.

Therefore, the pitch angle detecting means detects the pitch angle based on the specified region, so that the pitch angle can be detected accurately and promptly, as in the case of the above-mentioned invention. . Further, as described above, since the area where the road white line or the lane at a position separated from the vehicle by a predetermined distance is photographed is specified, if this area is continuously monitored, regardless of the pitch angle of the vehicle. It also has the feature of being able to continuously monitor the state of the road white line or lane at a position that is a predetermined distance away from the vehicle.

[Brief description of drawings]

FIG. 1 is a principle diagram of an image sensor for a vehicle according to the present invention.

FIG. 2 is a block diagram showing the overall configuration of an embodiment of a vehicle image sensor according to the present invention.

FIG. 3 is a diagram for explaining the principle of correlation calculation by template matching.

FIG. 4 is a flowchart of an example of a main routine executed by a CPU in the vehicle image sensor according to the present embodiment.

FIG. 5 is a diagram for explaining a white line width detection method in the vehicle image sensor according to the present embodiment.

FIG. 6 is a flowchart of an example of an initial processing routine executed by a CPU in the vehicle image sensor according to the present embodiment.

FIG. 7 is a flowchart of an example of a white line width detection processing routine executed by a CPU in the vehicle image sensor according to the present embodiment.

FIG. 8 is a flowchart of an example of a correlation calculation routine executed by a CPU in the vehicle image sensor of this embodiment.

FIG. 9 is a flowchart of an example of a pitch angle detection routine executed by a CPU in the vehicle image sensor according to the present embodiment.

FIG. 10 is a diagram for explaining a pitch angle detecting method in the vehicle image sensor of the present embodiment.

FIG. 11 is a CPU of the vehicle image sensor according to the present embodiment.
6 is a flowchart of a second example of a main routine executed by the.

FIG. 12 is a CPU in the vehicular image sensor of this embodiment.
5 is a flowchart of a second example of a pitch angle detection routine executed by the.

FIG. 13 is a diagram illustrating a relationship between a width of a road white line and a pitch angle in a captured image.

FIG. 14 is a CPU in the vehicular image sensor of this embodiment.
11 is a flowchart of a third example of a main routine executed by.

FIG. 15 is a CPU of the vehicle image sensor according to the present embodiment.
5 is a flow chart of an example of a white line width learning routine executed by.

FIG. 16 is a CPU of the vehicle image sensor according to the present embodiment.
8 is a flowchart of a second example of an initial processing routine executed by the.

FIG. 17 is a CPU in the vehicular image sensor of the present embodiment.
5 is a flow chart of an example of a white line data output processing routine executed by.

FIG. 18 is a CPU in the vehicular image sensor of the present embodiment.
9 is a flowchart of a third example of a pitch angle detection routine executed by the.

FIG. 19 is a CPU in the vehicular image sensor of the present embodiment.
9 is a flowchart of a fourth example of a pitch angle detection routine executed by.

[Explanation of symbols]

 M1, 10 camera M2 reference width setting means M3 horizontal width detecting means M4 pitch corresponding width setting means M5, M7 pitch angle detecting means M6 plural horizontal width detecting means 16 CPU 18 correlation calculation device

Continuation of front page (51) Int.Cl. ⁶ Identification code Office reference number FI Technical display area H04N 7/18 C

Claims (3)

[Claims] 1. A camera fixed to the vehicle to photograph the front of the vehicle, and a road white line or a road white line at a position separated from the vehicle by a predetermined distance when the image is photographed by the camera in a situation where the vehicle does not have a pitch angle. Reference width setting means for setting a width occupied by a lane in a captured image as a reference width, and in the image captured by the camera, the white line or the lane is captured with the reference width in a situation where no pitch angle is generated in the vehicle. Based on the reference width set by the reference width setting means, the horizontal width detection means for detecting the width occupied by the actually photographed road white line or lane in the captured image in the reference area If a width corresponding to a road white line or a lane estimated to be imaged in the reference area is set, a width corresponding to a pitch corresponding to the width, an actual width detected by the horizontal width detecting means, and Vehicular image sensor, characterized in that the pitch corresponding width setting means comprises a pitch angle detecting means for detecting a pitch angle that occurs in the vehicle based on the pitch corresponding width set in response to the pitch angle. 2. The reference width setting means sets a plurality of reference widths corresponding to a plurality of different predetermined distances, and the horizontal width detection means sets a plurality of reference widths set corresponding to the plurality of reference widths. Detecting the width of a road white line or a lane in a region, the pitch corresponding width setting means sets a plurality of pitch corresponding angles corresponding to the plurality of reference widths, the pitch angle detecting means, the horizontal width detecting means 2. The vehicle image sensor according to claim 1, wherein the pitch angle occurring in the vehicle is detected based on the plurality of detected widths and the plurality of pitch corresponding widths set by the pitch corresponding width setting means. 3. A camera fixed to the vehicle so as to photograph the front of the vehicle, and a road white line at a position separated from the vehicle by a predetermined distance when the image is photographed by the camera under the condition that the vehicle does not have a pitch angle. Reference width setting means for setting a width occupied by a lane in a captured image as a reference width, and in the image captured by the camera, the white line or the lane is captured with the reference width in a situation where no pitch angle is generated in the vehicle. A plurality of horizontal width detection means for detecting the width occupied by a road white line or a lane actually photographed in a captured image in a reference area and a plurality of areas set above and below the reference area; A vehicle characterized by comprising pitch angle detecting means for detecting a pitch angle occurring in the vehicle based on the detected actual width and the reference width set by the reference width setting means. Image sensor.