JP3697435B2 - Outside monitoring device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an out-of-vehicle monitoring device that recognizes an out-of-vehicle situation by processing an image obtained by imaging an object outside the vehicle, and more particularly to an out-of-vehicle monitoring device that detects a side wall as a continuous three-dimensional object that becomes a road boundary such as a guardrail. .
[0002]
[Prior art]
These days, TV cameras and laser radars are installed in automobiles to detect vehicles and obstacles ahead, determine the risk of collision with them, issue warnings to the driver, and automatically activate the brakes. Development of technologies related to ASV (Advanced Safety Vehicle) such as stopping and stopping, or automatically increasing or decreasing the traveling speed to keep the distance between the vehicle and the preceding vehicle safe Yes.
[0003]
As a technique for detecting a front object from a TV camera image, there is a technique disclosed in Japanese Patent Laid-Open No. 5-265547 previously filed by the present applicant. In this technique, two stereos mounted on the left and right sides of the vehicle are used. A camera image is converted into a distance image, the distance image is divided into grid-like regions at predetermined intervals, and a three-dimensional object is detected for each division. In addition, the present applicant similarly extracts a solid object data for each category, processes the data by Hough transform, and detects a solid object (side wall) along a road such as a guardrail. This is proposed in Kaihei 6-266828.
[0004]
[Patent Document 1]
JP-A-5-265547
[0005]
[Patent Document 2]
JP-A-6-266828
[0006]
[Problems to be solved by the invention]
However, in the conventional technology, since the data of the three-dimensional object is processed by Hough transform etc., the guardrail along the curved road only recognizes the part in a relatively close range as a straight line, and recognizes it far away It was difficult to do.
[0007]
The present invention has been made in view of the above circumstances, and an object thereof is to provide an out-of-vehicle monitoring device capable of detecting a series of three-dimensional objects constituting a road boundary as a wall surface even when the road is curved. It is said.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, a first vehicle exterior monitoring device according to the present invention is a wall surface classified as a wall surface constituting a road boundary in a vehicle exterior monitoring device that detects the position of a three-dimensional object outside the vehicle and recognizes the situation outside the vehicle. Means for extracting a position data group, means for sequentially setting a plurality of node points representing the wall surface shape based on the previously set node points, and the wall surface position data group in the predetermined area including the set node points. A means for specifying the node point based on the wall surface position obtained by performing matching processing with the wall surface pattern on the plurality of nodes by sequentially setting the node point and specifying the node point based on the wall surface position. And a means for forming a wall surface model representing the wall surface shape by points.
[0009]
The second vehicle exterior monitoring device according to the present invention is classified into the wall surfaces constituting the road boundary in the vehicle exterior monitoring device that detects the position of the solid object outside the vehicle and sets the position of the solid object indicating the road boundary as the wall surface position. For the plurality of three-dimensional objects classified into the wall surface position data group and the plurality of three-dimensional objects classified as the wall surface position data group, the position that is most similar to the wall pattern that is preset with the three-dimensional object is set as the wall surface position. And means for setting.
[0010]
That is, when the first exterior monitoring device of the present invention extracts a wall surface position data group classified as a wall surface constituting a road boundary, a plurality of node points representing the wall surface shape are sequentially set based on the previously set node points. Then, the node point is specified based on the wall surface position obtained by performing the matching process with the wall surface pattern on the wall surface position data group in the predetermined area including the set node point. Then, by sequentially setting the node points and specifying the node points based on the wall surface position, a wall surface model representing the wall surface shape is formed by the plurality of node points.
[0011]
At that time, the degree of coincidence is evaluated by the sum of the weights corresponding to the deviation between the X coordinate of each wall surface position data and the pattern center X coordinate of the wall surface pattern, and the wall surface pattern is moved in the X coordinate direction with respect to the wall surface position data group. It is desirable to specify the position of the pattern center when the degree of coincidence becomes maximum as a node point, and the coordinates of the specified node point specified in a direction asymptotic to a straight line connecting the preceding and following node points It is desirable to correct.
[0012]
Further, the second vehicle exterior monitoring device of the present invention extracts a wall surface position data group classified as a wall surface constituting a road boundary, and a plurality of solid objects classified into the wall surface position data group The position most similar to the wall surface pattern set in advance is set as the wall surface position.
[0013]
At that time, a wall pattern matching process is applied to the three-dimensional objects that make up the wall surface position data group within the predetermined area, and multiple wall surface positions are set by sequentially setting the areas and performing the matching process for each area. It is desirable to do. As the wall surface pattern, a pattern having a vertical axis with a weighting factor set on one side relatively high with respect to the pattern center point and set on the other side relatively low in the horizontal axis direction of the wall surface pattern is adopted. It is desirable.
[0014]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. 1 to 20 relate to an embodiment of the present invention, FIG. 1 is an overall configuration diagram of the outside monitoring apparatus, FIG. 2 is a circuit block diagram of the outside monitoring apparatus, and FIGS. 3 to 5 are three-dimensional object / side wall group detections. 6 is a flowchart of the wall surface detection process, FIGS. 7 and 8 are flowcharts of the wall surface position correction process, FIG. 9 is an explanatory diagram illustrating an example of an image captured by an in-vehicle camera, and FIG. 10 is an example of a distance image. FIG. 11 is an explanatory diagram showing the position of the three-dimensional object detected for each section, FIG. 12 is an explanatory diagram showing the detection result of the side wall, and FIG. 13 is an explanation showing the detection result of the side wall in the XZ plane. FIG. 14 is an explanatory diagram of a wall surface model, FIG. 15 is an explanatory diagram showing a method for searching for a wall surface pattern, FIG. 16 is an explanatory diagram showing a weighting coefficient pattern, FIG. 17 is an explanatory diagram showing a calculation result of coincidence, FIG. Is an explanatory diagram showing the connection of node points, FIG. 19 is a wall Explanatory view showing a detection result of FIG. 20 is an explanatory view showing the detection result of the wall in the X-Z plane.
[0015]
In FIG. 1, reference numeral 1 denotes a vehicle such as an automobile, and the vehicle 1 is mounted with an out-of-vehicle monitoring device 2 that images an object outside the vehicle and recognizes and monitors the situation outside the vehicle from the captured image. The vehicle exterior monitoring device 2 includes a stereo optical system 10 for imaging an object outside the vehicle from different positions, an image processor 20 that processes an image captured by the stereo optical system 10 and calculates three-dimensional distance distribution information, The distance information from the image processor 20 is input, the road shape and the three-dimensional position of a plurality of three-dimensional objects are detected at high speed from the distance information, and the preceding vehicle and the obstacle are identified based on the detection result, and the collision occurs. The computer includes a recognition processing computer 30 that performs alarm judgment processing and the like.
[0016]
The recognition processing computer 30 is connected to sensors for detecting the current traveling state of the vehicle such as the vehicle speed sensor 4 and the steering angle sensor 5, and the recognized object becomes an obstacle of the host vehicle 1. In this case, a warning is given to the driver by displaying it on the display 9 installed in front of the driver, and automatic collision avoidance control of the vehicle body can be performed by connecting an external device that controls actuators (not shown). Yes.
[0017]
The stereo optical system 10 is composed of a pair of left and right CCD cameras 10a and 10b using a solid-state imaging device such as a charge coupled device (CCD), for example. In the image processor 20, a pair of images captured by the CCD cameras 10a and 10b. A correlation between images is obtained, and a distance is obtained from the parallax with respect to the same object by a triangulation principle. A three-dimensional distance distribution over the entire image is calculated by a so-called stereo method.
[0018]
The recognition processing computer 30 reads the distance distribution information from the image processor 20 to detect the road shape and the three-dimensional position of a plurality of three-dimensional objects (vehicles, obstacles, etc.) at high speed, and collides with or touches the detected object. The possibility is judged based on the traveling state of the host vehicle detected by the vehicle speed sensor 4, the steering angle sensor 5, etc., and the result is displayed on the display 9 to notify the driver.
[0019]
In detail, the image processor 20 and the recognition processing computer 30 have a hardware configuration shown in FIG. The image processor 20 searches a portion of the pair of stereo images captured by the CCD cameras 10a and 10b for the same object in every predetermined small area, and obtains the amount of displacement of the corresponding position. And a distance image memory 20b for storing the distance distribution information output from the distance detection circuit 20a.
[0020]
The distance distribution information output from the distance detection circuit 20a is a pseudo image (distance image) shaped like an image, and is an image taken with the two left and right CCD cameras 11a and 11b, for example, as shown in FIG. When such an image (FIG. 9 schematically shows an image captured by one camera) is processed by the distance detection circuit 20a, a distance image as shown in FIG. 10 is obtained.
[0021]
In the example of the distance image shown in FIG. 10, the image size is 600 pixels wide × 200 pixels long, and the distance data has a white dot portion in the drawing, which is among the pixels of the image of FIG. This is a portion where a change in brightness between pixels adjacent in the left-right direction is large. The distance detection circuit 20a treats this distance image as an image composed of blocks of horizontal 150 × vertical 50 with a small area of 4 × 4 pixels, and calculates a distance (number of pixel shifts) for each block.
[0022]
On the other hand, the recognition processing computer 30 detects a microprocessor 30a mainly for detecting road shapes and the like, and a microprocessor 30b mainly for detecting individual solid objects based on the detected road shapes. A multi-microprocessor in which a preceding vehicle and an obstacle are identified based on the position information of a three-dimensional object, and a microprocessor 30c, which mainly processes collision and contact risk determination, is connected in parallel via a system bus 31. System configuration.
[0023]
The system bus 31 includes an interface circuit 32 connected to the distance image memory 20b, a ROM 33 for storing a control program, a RAM 34 for storing various parameters during calculation processing, and an output for storing parameters of processing results. Memory 35, a display controller (DISP. CONT.) 36 for controlling the display (DISP) 9, and an I / O interface circuit 37 for inputting signals from the vehicle speed sensor 4, the steering angle sensor 5, and the like. It is connected.
[0024]
The recognition processing computer 30 treats the coordinate system on the distance image in units of pixels as shown in FIG. 9 with the lower left corner as the origin, the horizontal direction as the i coordinate axis, and the vertical direction as the j coordinate axis, and the number of pixel shifts. A point (i, j, dp) on the distance image as dp is converted into a coordinate system in the real space, and processing such as road shape recognition and solid object position detection is performed.
[0025]
That is, the three-dimensional coordinate system of the real space is a coordinate system fixed to the own vehicle (vehicle 1), the X axis is the right side of the traveling direction of the vehicle 1, the Y axis is above the vehicle 1, and the Z axis is the vehicle 1 Assuming that the front and the origin are the road surface directly below the center of the two CCD cameras 10a and 10b, the XZ plane (Y = 0) coincides with the road surface when the road is flat. By the equations (1) to (3), the point (i, j, dp) on the distance image can be coordinate-converted to the point (x, y, z) on the real space.
x = CD / 2 + z · PW · (i-IV) (1)
y = CH + Z · PW · (j−JV) (2)
z = KS / dp (3)
However, CD: interval between CCD cameras 10a and 10b
PW: Viewing angle per pixel
CH: Mounting height of the CCD cameras 10a and 10b
IV, JV: Coordinates (pixels) on the image of the infinity point directly in front of the vehicle 1
KS: Distance coefficient (KS = CD / PW)
The equation for calculating the point (i, j, dp) on the image from the point (x, y, z) in the real space is obtained by modifying the above equations (1) to (3) as follows. .
[0026]
i = (x−CD / 2) / (z · PW) + IV (4)
j = (y−CH) / (z · PW) + JV (5)
dp = KS / z (6)
[0027]
Next, each process in the recognition processing computer 30 will be described. First, in the road detection process by the microprocessor 30a, the three-dimensional position information from the distance image stored in the distance image memory 20b is used, and the built-in road model parameters are corrected to match the actual road shape. Change to recognize the road shape.
[0028]
The above road model divides the lane of the road up to the recognition target range into a plurality of sections according to the set distance and approximates the left and right white lines and guard rails etc. with a three-dimensional linear expression for each section into a broken line shape These are connected, and the horizontal linear parameters a and b and the vertical linear parameters c and d in the real space coordinate system are obtained, and the horizontal straight line shown in the following formula (7) is obtained. The vertical straight line equation shown in the following equation (8) is obtained.
x = a · z + b (7)
y = c · z + d (8)
[0029]
Further, in the three-dimensional object detection processing by the microprocessor 30b, the distance image is divided into a grid pattern at predetermined intervals, the three-dimensional object data is extracted for each of the sections, a histogram is created, and each section is represented from this histogram. Find the location of a solid object and its distance. Next, the distance for each section is sequentially compared from the left to the right of the image, and the distances in the front-rear direction (Z-axis direction) and the horizontal direction (X-axis direction) are gathered together as a group, For each group, the data arrangement direction is checked, and the group is divided at a portion where the direction greatly changes.
[0030]
Then, each group is classified into a three-dimensional object or a side wall from the arrangement direction (inclination with respect to the Z-axis) of the distance data as a whole group. By calculating parameters such as the X coordinate of the right end, and by calculating parameters such as the alignment direction (tilt with the Z axis) and the position of the front and rear ends (Z, X coordinates) for the group classified as the side wall, Processing such as detection of a structure along the road, such as a rear part or a side part of a three-dimensional object, a guardrail, that is, a side wall is performed.
[0031]
The generation of the distance image, the process of detecting the road shape from the distance image, and the collision / contact determination process are disclosed in Japanese Patent Laid-Open Nos. 5-265547 and 6-266828 previously filed by the present applicant. Is described in detail.
[0032]
In this case, a side wall such as a guardrail can recognize a curved wall surface along the road to a distant place even when the road is curved. Hereinafter, processing related to wall surface detection by the microprocessor 30b will be described with reference to the flowcharts shown in FIGS.
[0033]
FIGS. 3 to 5 are programs for processing the distance data obtained from the distance image to classify the object into a three-dimensional object group and a side wall group. The existence of a three-dimensional object and its distance are calculated for each section divided in a grid pattern. That is, in step S101, the distance image is divided into a grid pattern at a predetermined interval (for example, an interval of 8 to 20 pixels), and in step S102, three-dimensional object data is extracted for each division, and the detection distance is calculated. Read the data of the first section.
[0034]
Next, when the process proceeds to step S103 and the first data in the section is set, the three-dimensional position (x, y, z) of the subject is obtained by the above-described equations (1) to (3) in step S104, and in step S105. Then, the height yr of the road surface at the distance z is calculated using the linear equations (7) and (8) of the road shape described above. When the road shape cannot be recognized at first such as a road without a white line, the road surface height yr is set on the assumption that the road surface is horizontal to the vehicle 1.
[0035]
Next, the process proceeds to step S106, and data above the road surface is extracted as three-dimensional object data based on the height H of the subject from the road surface calculated by the following equation (9). In this case, a subject whose height H is about 0.1 m or less is considered to be a white line, dirt, shadow, or the like on the road, so the data of this subject is rejected. An object above the height of the host vehicle 1 is also considered to be a pedestrian bridge, a sign, etc., so it is rejected and only the data estimated as a three-dimensional object on the road is selected.
H = y-yr (9)
[0036]
Thereafter, the process proceeds to step S107 to check whether or not the data is the final data. If the final data is not the final data, the next data in the section is set in step S108, and the process returns to the above-described step S104. Extract the data in When the processing of the final data is completed within one section, the process proceeds from step S107 to step S109, and the number of data included in the section of the preset distance z is counted with respect to the extracted three-dimensional object data. Create a histogram with z as the horizontal axis.
[0037]
In the subsequent step S110, a section where the frequency of the histogram is equal to or greater than the determination value and the maximum value is detected, and if there is a corresponding section, in step S111, it is determined that a three-dimensional object exists in the section, and the three-dimensional object is reached. Detect the distance. In the above histogram, there are some erroneously detected values in the distance data in the input distance image, and some data actually appears at positions where no object is present. However, when there is an object of a certain size, the frequency at that position shows a large value, whereas when there is no object, the frequency generated only by erroneous distance data becomes a small value.
[0038]
Therefore, if there is a section in which the frequency of the created histogram is greater than or equal to the preset judgment value and takes the maximum value, it is determined that an object exists in that section. If the maximum value of the frequency is less than the judgment value, the object exists. If the image data contains some noise, the object can be detected while minimizing the influence of the noise.
[0039]
Thereafter, the process proceeds from step S111 to step S112 to check whether the final section has been reached. Then, when the final section has not been reached, the process proceeds from step S112 to step S113 and the next section of data is read, and the process returns to the above-described step S103 to extract data above the road surface, create a histogram, and The solid object is detected and the distance is calculated in each section. The above process is repeated, and when the final section is reached, the process proceeds from step S112 to step S114.
[0040]
FIG. 11 shows the positions of the three-dimensional objects detected for each section from the original image of FIG. 9, and the distance data of these three-dimensional objects is divided into groups that are close in distance by the processing of step S114 to step S120. . In this process, the detection distance of the three-dimensional object in each section is examined, and if the difference in the detection distance to the three-dimensional object in the adjacent section is equal to or less than the judgment value, it is regarded as the same three-dimensional object, while the judgment value is exceeded Are grouped as if they were separate solid objects.
[0041]
Therefore, in step S114, first, the first section (for example, the left end) is examined. If a three-dimensional object is detected, the distance data is read, and this section R1 is classified into group G1 and distance Z1. Next, proceeding to step S115, the right-side section R2 is examined, and if a three-dimensional object is not detected, the group G1 exists in and near the section R1, and the distance is determined to be Z1, If a three-dimensional object is detected in the section R2 and the detected distance is Z2, the difference between the distance Z1 in the section R1 and the distance Z2 in the right adjacent section R2 is calculated.
[0042]
Thereafter, the process proceeds to step S116 to check whether or not the difference in distance from the right adjacent section is equal to or smaller than the determination value. When the distance difference is equal to or smaller than the determination value and is approaching each other, in step S117, it is detected in the section R2. The determined three-dimensional object is determined to belong to the previously detected group G1, and the same group is labeled. The distance is set as the average value of Z1 and Z2, and the process proceeds to step S119.
[0043]
On the other hand, when the difference in distance from the right adjacent section exceeds the determination value, the process proceeds from step S116 to step S118, and the three-dimensional object detected in the section R2 is different from the group G1 detected previously. Determination is made to label the new group (group G2, distance Z2), and the process proceeds to step S119.
[0044]
In step S119, it is checked whether or not the final section has been reached. If the final section has not been reached, the distance of the next section is read in step S120, the process returns to step S115, and the area on the right is further checked. . When the final section is reached, the process proceeds from step S119 to step S121 and thereafter.
[0045]
In the above processing, for example, in the situation where the vehicle is parked beside the guard rail, the distance data of the guard rail and the distance data on the vehicle parked beside the guard rail are processed as the same group. May end up. Therefore, the arrangement direction of the distance data on the XZ plane is checked in the processing in the next steps S121 to S131, and the group is divided into a portion where the arrangement direction is parallel to the Z axis and a portion parallel to the X axis.
[0046]
In the process of dividing the group, in step S121, the data of the first group is read, and in step S122, the arrangement direction of each section in the group is calculated. In step S123, “object” and “side wall” are added to each section. Label. Specifically, assuming that the position of the leftmost section K1 in the group is Z1 and X1 and the positions of the rightmost sections K are Zp and Xp, two points of points X1 and Z1 and points Xp and Zp are obtained. An inclination A1 of the connecting straight line with respect to the Z axis is calculated, and the inclination A1 of the straight line is compared with a set value (for example, about 45 °). If the slope A1 of the straight line is equal to or less than the set value and the data sequence is substantially in the Z-axis direction, the section K1 is labeled “side wall”, the slope A1 of the straight line exceeds the set value, and the data sequence is In the case of a substantially X-axis direction, it is labeled “object”.
[0047]
The interval N between the divisions for labeling is about N = 2 to 4 divisions. This is because in N = 1, that is, in the right adjacent section, the arrangement direction greatly varies due to variations in the detection distance, and it is difficult to determine the division. By using the arrangement direction with a slightly separated section, , To stabilize the direction. Then, the labeling of the “side wall” or “object” is performed in order from the left end segment in the group to the N left segments from the right end, and each segment is labeled.
[0048]
As described above, when the labeling of each section is completed, the process proceeds from step S123 to step S124 to read the label of the leftmost section, and further, in step S125, the label of the next adjacent section is read. Next, the process proceeds to step S126, and it is checked whether the leftmost label is different from the label on the right. As a result, when the labels are the same, the process jumps from step S126 to step S128, and when the labels are different, the process proceeds from step S126 to step S127, and the section labeled "side wall" and the section labeled "object" Divide into another group and go to step S128. The position of the segment to be divided is on the right side by N / 2 segments where the label changes from “side wall” ← → “object”.
[0049]
In this case, in order to deal with the situation where the label partially changes due to variations in distance data, etc., the division is performed only when the same label is continuously arranged more than the judgment value (for example, three or more categories), If it is less than the judgment value, no division is performed.
[0050]
In step S128, it is checked whether or not it is the final division. If it is not the final division, the label of the next division is read in step S129, the flow returns to step S125, and the same processing is repeated. When the final section is reached, the process proceeds from step S128 to step S130 to check whether the final group has been reached. As a result, when the final group has not been reached, the data of the next group is read in step S131, and the process of dividing the group in the same manner is performed on the next group. This process is repeated, and when the final group is reached, the group division process is completed and the process proceeds from step S130 to step S132 and subsequent steps.
[0051]
The next steps S132 to S137 are processes for calculating the parameters of each group by classifying each divided group as a side wall or an object. When the first group data is read in step S132, In S133, an approximate straight line is obtained from the position (Xi, Zi) of each section in the group by the Hough transform or the least square method, and the inclination of the entire group is calculated.
[0052]
Next, proceeding to step S134, classifying the group having the inclination in the X-axis direction as an object and the group having the inclination in the Z-axis direction as side walls from the inclination of the entire group, and calculating parameters of each group in step S135. To do. This parameter is a parameter such as an average distance calculated from the distance data in the group and the X coordinate of the left end and the right end in a group classified as an object. In a group classified as a side wall, the alignment direction (Z Parameters such as the inclination with respect to the axis and the positions of the front and rear ends (Z, X coordinates). The group classification may be performed based on the label of “side wall” or “object” of each section attached in the group division process described above.
[0053]
Then, the process proceeds from step S135 to step S136 to check whether or not the final group has been reached. If the final group is not reached, the next group data is read in step S137 and the process returns to step S133. , Exit the program.
[0054]
In the above processing, the detection result of the side wall as shown in FIG. 12 is obtained, and when the distance for each section is shown on the XZ plane, it is regarded as a straight side wall group as shown in FIG. The part along is not recognized. Therefore, in the wall surface detection processing program of FIG. 6, the wall surface along the curved road is recognized using the data of the side wall group obtained as described above.
[0055]
In this program, first, in step S201, a group that is estimated to detect a wall surface such as a guard rail is selected from the group of side walls by examining the positional relationship with the host vehicle (one or less each for left and right), After step S202, the wall surface is searched and expanded using the wall surface model by using the data (position and direction) of the selected side wall group as a clue.
[0056]
As shown in FIG. 14, this wall surface model expresses the contour of a wall surface by connecting a predetermined range in front of the host vehicle with node points provided for each set interval. These 41 node points form a contour, and each node point is managed by being sequentially numbered from the own vehicle side. The Z coordinate of each node point is fixed, and the X coordinate is determined by a procedure described later.
[0057]
Therefore, first, in step S202, the corresponding node point Ns is obtained from the Z coordinate of the end point on the near side of the side wall group selected in step S201, and the X coordinate of this node point Ns is set in accordance with the position of the side wall. Then, the process proceeds to step S203, and the position (X coordinate) of the next node point Ns + i is set. When the node point Ns + i is set to i = 1 and the node point Ns and the adjacent node point Ns + 1 are set, the node point Ns + i is set to a position along the inclination direction of the side wall group. When setting the second and subsequent node points, the positions are along the direction determined by the two previous node points.
[0058]
Next, the process proceeds to step S204, and as shown in FIG. 15, the position of the wall surface is determined by pattern matching within a predetermined range around the coordinates (Xns + i, Zns + i) of the node point Ns + i set in step S203. Explore. The search range is, for example, about Xns + i ± 3 to 5 m in the X direction and about Zns + i ± 1 m in the Z direction, and the three-dimensional object Pi for each section within the search range is extracted.
[0059]
Then, wall surface pattern matching is performed on the three-dimensional object Pi within the search range. FIG. 16 shows an example of a wall surface pattern (weight coefficient pattern) used in this pattern matching (FIG. 16 shows a pattern corresponding to the left wall surface, and a pattern symmetrical to the left side is used for detecting the right wall surface. Use). The horizontal axis of the wall pattern indicates the distance in the X coordinate direction, and the vertical axis indicates the weighting coefficient. A position with the highest matching degree is searched while shifting the center point of the wall pattern in the X coordinate direction within the search range. Specifically, the weight Wi corresponding to the deviation in the X coordinate direction from the center of the wall pattern of each solid object Pi is obtained based on the wall pattern shown in FIG. 16, and the sum of the weights Wi corresponding to each solid object Pi is obtained. Calculated as the matching degree F. The center point position of the wall pattern when the matching degree F is maximized is recognized as the wall surface position. However, when the maximum value of the matching degree F is equal to or less than the determination value, it is determined that there is no wall surface.
[0060]
When the processing in step S204 is completed, the process proceeds to step S205, and the X coordinate (Xpw) of the center point of the wall surface pattern when the matching degree F is maximized is set as the X coordinate of the wall surface position corresponding to the node point Ns + i. Determine.
[0061]
Then, the process proceeds to step S206 to check whether or not it is the last node point in the selected side wall group, and when it is not the last node point, the process returns to step S203 to set the next node point and repeat the same processing. When the last node point is reached, the process proceeds to step S207, and among the node points determined in steps S202 to S205, the node number having the smallest number (the one closest to the host vehicle) and the node number having the largest number (the most own Searching for objects far from the vehicle), and exiting the program with these as the start point Ns and end point Ne of the detection range. This program is executed for the left side wall group and then for the right side wall group. In the example of FIG. 14, the wall surface is detected from the ninth node to the twenty-sixth node on the right side of the host vehicle, and each node point having the ninth node as the start point Ns and the twenty-sixth node as the end point Ne is managed as an effective node point. .
[0062]
The position of the wall surface detected by the above processing is further corrected by the program shown in FIGS. 7 and 8 using the new data of the side wall group obtained by the program of FIGS.
[0063]
In this wall surface position correction program, in step S301, it is checked whether or not the effective node point starting point Ns obtained in the previous processing is larger than the first node point N1 of the wall surface model. When the wall surface has already been detected up to the first node point N1, the process jumps to step S306, and when Ns> N1, the process proceeds to step S302 to return to the near side to set the node point Ns-i, and in step S303 When the wall surface pattern is searched, the X coordinate of the wall surface is determined in step S304 according to the search result.
[0064]
Next, the process proceeds from step S304 to step S305 to check whether or not the first node point has been reached. If the first node point N1 has not been reached, steps S302 to S304 are repeated to determine the wall surface position up to the node point N1. When the search to the first node point N1 is completed, the process proceeds to step S306, where the effective node point end point Ne is the last node point Nse of the wall surface model (for example, the wall surface model is converted into 41 node points). In the case of comprising, it is checked whether it is smaller than the node point N41).
[0065]
As a result, when the wall surface has already been detected up to the last node point with Ne = Nse, the process jumps from step S306 to step S311. When Ne <Nse, the process proceeds from step S306 to step S307, and the node points after the end point Ne. When Ne + i is sequentially set and wall surface pattern matching is performed in step S308, the X coordinate of the wall surface is determined in step S309 according to the matching result. Then, in step S310, it is checked whether or not the last node point Nse has been reached, the matching of the wall surface position to the last node point Nse is continued, and when the processing up to the last node point Nse is completed, the process proceeds to step S311. .
[0066]
The above-described steps S302 to S304, and node point setting, wall surface pattern matching, and determination of the wall surface X coordinate in accordance with the search result in steps S307 to S309 are performed in steps S203, S204, This is the same as the processing of S205.
[0067]
The processing after step S311 is processing for correcting the position (X coordinate) of each node point from the first node point N1 to the last node point Nse. First, in step S311, the data of the first node point N1 is obtained. Set, and proceed to step S312. The processing from step S312 to step S321 is repeatedly executed from the first node point N1 to the last node point Nse by sequentially setting node point data in step S322.
[0068]
First, in step S312, a wall surface at the node point Ni is searched, and in step S313, it is checked whether or not a wall surface is detected by pattern matching. When the wall surface is detected by pattern matching, the process proceeds from step S313 to step S314, and the difference between the wall surface position Xpw read out and the node point position Xni is within a set value (for example, within ± 1 m). If the difference is within the set value, the node point is moved to the wall surface position in step S315 (Xni ← Xpw). If the difference exceeds the set value, the node point is moved in the direction of the wall surface in step S316. It moves by a predetermined amount (for example, about ± 0.3 m) (Xni ← Xni ± 0.3 m).
[0069]
On the other hand, when the wall surface is not detected by pattern matching, the process branches from step S313 to step S317, and the number C0 of the solid object data Xpi (Xni <Xpi) existing on the left side relative to the node point position Xni, The number C1 of the three-dimensional object data Xpi (Xni> Xpi) existing on the right side is counted, and in step S318, the number of three-dimensional object data is large and the node point is set to a predetermined amount (on the side where the three-dimensional object is offset. For example, it moves 0.8 m) (Xni ← Xni ± 0.8).
[0070]
That is, when a wall pattern is not detected near the node point, the position of the node point is greatly separated from the wall surface by moving the node point by a predetermined amount in the direction in which the three-dimensional object is detected in the left-right direction of the node point. Even in the case of failure, the position of the node point can be brought closer to the direction in which the wall surface may exist.
[0071]
When the position of the node point is moved by any one of the above steps S315, S316, S318, the process proceeds to step S319, and the node point Ni + 1, which is one distant from the current node point Ni, The position (X coordinate) Xc of the midpoint of the straight line connecting the one neighboring node point Ni-1 with respect to the node point Ni is obtained, and in step S320, as shown in FIG. 18, the current node point Ni and Two straight lines connecting the far-side node point Ni + 1 and the straight line connecting the current node point Ni and the nearby node point Ni + 1 are connected by springs, and are straight. The node point is moved in the direction of the midpoint as if trying to do so. The amount of movement in this case is limited to about 1/2 to 1/5 of the difference between the midpoint position Xc and the node point position Xni.
[0072]
In other words, there are irregularities in the position of the wall surface detected by pattern matching etc. due to the influence of data variation, etc., but in many cases the wall surface of the guard rail or the like that actually forms the road boundary curves smoothly. Yes. For this reason, the fine unevenness | corrugation can be smoothed by adding the effect | action of the above springs, and a smooth curve shape can be obtained. In this case, as a general method for smoothing the entire shape of the node point, there is a least square method or the like. However, the above-described method using a spring action has a simple calculation process and can improve the processing speed.
[0073]
Thereafter, the process proceeds to step S321 to check whether or not the final node point Nse has been reached. If the final node point Nse has not been reached, the data of the next node point is set in step S322 and the process returns to step S312. The above processing is repeated. When the processing up to the last node point Nse is completed, the process proceeds from step S321 to step S323, and it is checked whether or not the movement amount is within a determination value (for example, ± 0.1 m) at all the node points.
[0074]
If there is even one node point whose movement amount exceeds the judgment value, the process returns to step S311 and repeats the correction processing of the above positions from the first node point to the last node point. In step S324, the start point Ns and end point Ne of the node point detection range are obtained and the program exits. Thereby, partial misdetection etc. are corrected in repetition, and the shape of the most suitable wall surface as a whole is obtained. This program is also executed for the right side wall group after being executed for the left side wall group.
[0075]
FIG. 19 shows a detection result obtained by capturing a wall surface along a curved road far away from the original image of FIG. 9, and on the XZ plane, the correction result of the wall model is as shown in FIG. In the detection result of the curved wall surface by this wall surface model, it is possible to detect the wall surface as far as, for example, about twice as far as when the side wall is captured as a straight line as shown in FIG.
[0076]
In addition, the above wall model can support various shapes, and not only a guardrail, but also a series of three-dimensional objects that constitute the boundary of the road by connecting various objects such as planting around the road and housing fences. It can be detected as a wall surface. Therefore, by using the above wall surface detection results, the road shape can be recognized even on roads that do not have white lines or roads that cannot recognize white lines such as snowy roads.
[0077]
【The invention's effect】
As described above, according to the present invention, even when the road is curved, a series of three-dimensional objects constituting the boundary of the road can be detected as a wall surface, and a white line is generated by using this wall surface detection result. The road shape can be recognized even on a road that cannot recognize a white line such as a road without snow or a snowy road, and an excellent effect such as being able to accurately grasp the situation outside the vehicle can be obtained.
[Brief description of the drawings]
FIG. 1 is an overall configuration diagram of a vehicle exterior monitoring device
FIG. 2 is a circuit block diagram of the outside monitoring device.
FIG. 3 is a flowchart of solid object / side wall group detection processing (part 1).
FIG. 4 is a flowchart of solid object / side wall group detection processing (part 2).
FIG. 5 is a flowchart of a three-dimensional object / side wall group detection process (part 3);
FIG. 6 is a flowchart of wall surface detection processing.
FIG. 7 is a flowchart of wall surface position correction processing (part 1).
FIG. 8 is a flowchart of wall surface position correction processing (part 2).
FIG. 9 is an explanatory diagram illustrating an example of an image captured by an in-vehicle camera.
FIG. 10 is an explanatory diagram illustrating an example of a distance image.
FIG. 11 is an explanatory diagram showing the position of a three-dimensional object detected for each section
FIG. 12 is an explanatory view showing the detection result of the side wall.
FIG. 13 is an explanatory diagram showing the detection result of the side wall in the XZ plane.
FIG. 14 is an explanatory diagram of a wall surface model
FIG. 15 is an explanatory diagram showing a wall pattern search method
FIG. 16 is an explanatory diagram showing a weighting coefficient pattern.
FIG. 17 is an explanatory diagram showing the calculation result of the degree of coincidence
FIG. 18 is an explanatory diagram showing connection of node points;
FIG. 19 is an explanatory diagram showing wall surface detection results
FIG. 20 is an explanatory diagram showing wall surface detection results in the XZ plane.
[Explanation of symbols]
1 ... Outside monitoring device
10a, 10b ... CCD camera
20 Image processor
30 ... Recognition processing computer
Ni ... Node point

Claims (6)

In a vehicle outside monitoring device that detects the position of a three-dimensional object outside the vehicle and recognizes the situation outside the vehicle,
Means for extracting a wall surface position data group classified into wall surfaces constituting a road boundary;
Means for sequentially setting a plurality of node points representing the wall shape based on the previously set node points;
Means for identifying the node point based on a wall surface position obtained by performing a matching process with a wall surface pattern on the wall surface position data group in a predetermined region including the set node point;
A vehicle exterior monitoring apparatus comprising: means for forming a wall surface model representing a wall surface shape by a plurality of node points by sequentially setting the node points and specifying the node points based on the wall surface position. The means for specifying the node point evaluates the degree of coincidence based on a sum of weights corresponding to a deviation between the X coordinate of each wall surface position data and the X coordinate of the pattern center of the wall surface pattern, and the wall surface pattern is determined as the wall surface position data. 2. The out-of-vehicle monitoring apparatus according to claim 1, wherein the position of the pattern center when the coincidence is maximized by moving the group in the X coordinate direction is specified as the node point. The vehicle exterior monitoring device according to claim 1 or 2, wherein the means for forming the wall surface model corrects the coordinates of the specified predetermined node point in a direction asymptotic to a straight line connecting the preceding and following node points. In the vehicle outside monitoring device that detects the position of a three-dimensional object outside the vehicle and sets the position of the three-dimensional object indicating the road boundary as the wall surface position,
Means for extracting a wall surface position data group classified into wall surfaces constituting a road boundary;
Means for setting, as a wall surface position, a position most similar to a wall surface pattern whose positional relationship with the three-dimensional object is preset for a plurality of three-dimensional objects classified into the wall surface position data group. A vehicle exterior monitoring device. A plurality of wall surface positions are set by applying matching processing by the wall surface pattern to the three-dimensional objects constituting the wall surface position data group in a predetermined region, sequentially setting regions and performing matching processing for each region. The vehicle exterior monitoring device according to claim 4. The wall surface pattern is a pattern having, on the vertical axis, a weighting factor that is set such that one side is set relatively high with respect to the pattern center point in the horizontal axis direction of the wall surface pattern and the other side is set relatively low. The vehicle exterior monitoring device according to claim 4.

Images