JP5502448B2 - Road surface shape recognition device - Google Patents

Description

  The present invention relates to a road surface shape recognition device, and more particularly to a road surface shape recognition device that generates a road surface shape model based on position data obtained by a distance image generation means.

  In recent years, an imaging device such as a CCD (Charge Coupled Device) camera or a CMOS (Complementary Metal Oxide Semiconductor) camera or a laser radar distance measuring device has been mounted on a vehicle. Development of a road surface shape recognition device for recognizing a road surface shape on which a vehicle equipped with such a device travels is performed by analyzing reflected light of emitted radio waves or laser beams (see, for example, Patent Documents 1 and 2). .

  In Patent Document 1, out of a pair of images captured by an imaging device including a stereo camera, each pixel corresponding to a lane on the side of the host vehicle is detected from the image based on the luminance of each pixel of one image. A technique for calculating a model of a road surface shape in the horizontal direction and the vertical direction by linearly approximating the position data of each pixel corresponding to the detected lane for each section set in front of the host vehicle is disclosed. The lane refers to a continuous line or a broken line marked on a road surface such as an overtaking prohibition line or a lane marking that divides a roadside zone and a roadway.

  Patent Document 2 discloses a technique for projecting position data obtained by measuring a distance to an object onto three-dimensional coordinates and determining a road surface based on the distribution thereof.

JP 2001-92970 A Japanese Patent Laid-Open No. 10-143659

  However, in the technique described in Patent Document 1, it is possible to detect a lane on a normal road in which a lane is marked on the road surface, and to detect a road surface shape based on the detected lane. The shape could not be detected on the road surface where no lane was marked on the road surface.

  Moreover, in the technique described in Patent Document 2, it is necessary that the position data detected using a stereo camera or the like is accurate, but in the stereo matching process for a pair of images captured by the stereo camera, There are many cases where mismatches occur. Since position data varies, it is not always easy to determine a road surface from a three-dimensional distribution of position data.

  Moreover, in the technique described in Patent Document 2, the effectiveness can be expected on a road surface with many textures such as an unpaved road or a mountain road, but when the texture is relatively small like a normal road, The number of data points of the position data corresponding to the road surface is reduced. If there are other three-dimensional objects or the like, data at a few positions corresponding to the road surface are mixed with data at positions corresponding to the three-dimensional object, and there is a possibility that the road surface shape cannot be detected stably.

  The present invention has been made in view of the circumstances as described above, and it is possible to accurately detect the actual road surface shape. Of course, when the lane is marked on the road surface, the lane is marked. An object of the present invention is to provide a road surface shape recognition device capable of accurately detecting the road surface shape even when the road surface is not.

In order to solve the above problem, the first invention is a road surface shape recognition apparatus,
A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Equipped with a,
The approximate straight line calculation means, for all the plots that remain on the virtual plane without being excluded by the evaluation means, based on the representative distance, the plots on the side closer to the host vehicle. Each time the plot of the boundary part of two groups is transferred from one group to the other group, an approximate straight line that approximates each plot for each group is divided into a group and a group on the far side. Respectively,
The road surface shape model generation means calculates a statistical value based on the approximate line for each group in which the plot is transferred, and based on the calculated statistical value based on the approximate lines of the two groups. One of the combinations of the approximate lines in the two groups is selected, and the road surface shape model is generated using the selected combination of the approximate lines .

2nd invention is the road surface shape recognition apparatus of 1st invention,
The approximate straight line calculating means calculates the approximate straight line that approximates each plot for each group by a least square method,
The road surface shape model generation means calculates, for each group, a variance or a standard deviation of the plots belonging to each group with respect to the approximate line as a statistical value based on the approximate line, and the calculated two groups of the two groups A combination of the approximate lines that minimizes a total value of statistical values based on the approximate lines is selected .

According to a third aspect of the present invention, in the road surface shape recognition apparatus according to the second aspect , the approximate straight line calculating means represents the representative distance as z and the position of the horizontal line in the distance image as j, and the boundary portion to be transferred. When the representative distance in the plot is represented by za, the position of the horizontal line is represented by ja, and the sum for each group is represented by Σ, the plot of the boundary portion is expressed from one group to the other group. each time transferring the, Shigumaz in said one group, Σj, Σz ^2, respectively, from Σzj za, ^ja, subtracts za 2, zaja, Σz in the other group, Σj, Σz ^2, respectively Σzj za, The approximate straight lines that approximate the plot for each group are calculated by adding the values of ja, za ² , and zaja, respectively, by the least square method .

According to a fourth aspect of the present invention, in the road surface shape recognition device according to any one of the first to third aspects, the road surface shape model generation means calculates the intersection of the approximate lines of the selected two groups. The generated road surface shape model is corrected by replacing the approximate straight line with a relaxation curve having a tangent line .

A fifth invention, in road surface shape recognition apparatus,
A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Imaging means for capturing an image of the front of the host vehicle and acquiring an image;
A search is made on a horizontal line on the image to detect a pixel whose luminance difference between adjacent pixels is not less than a predetermined threshold as a lane candidate point, and the horizontal line to be searched is shifted in the vertical direction of the image while shifting the horizontal line. Lanes that detect lane candidate points, connect the detected lane candidate points, detect lanes on the image, and generate a road surface shape model as a shape on at least a virtual plane based on the detected lane information Detection means;
A processing unit that evaluates each of the road surface shape model generated by the road surface shape model generation unit and the road surface shape model generated by the lane detection unit, and selects any one of the road surface shape models;
It is characterized by providing .

A sixth aspect of the road surface shape recognition apparatus,
A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Imaging means for capturing an image of the front of the host vehicle and acquiring an image;
A search is made on a horizontal line on the image to detect a pixel whose luminance difference between adjacent pixels is not less than a predetermined threshold as a lane candidate point, and the horizontal line to be searched is shifted in the vertical direction of the image while shifting the horizontal line. Lanes that detect lane candidate points, connect the detected lane candidate points, detect lanes on the image, and generate a road surface shape model as a shape on at least a virtual plane based on the detected lane information Detection means;
The road surface shape model generated by the road surface shape model generation unit and the road surface shape model generated by the lane detection unit are each evaluated, and a weighted average of both the road surface shape models is generated to generate a road surface shape model. Processing means;
It is characterized by providing .

7th invention is the road surface shape recognition apparatus of 5th or 6th invention, The said process means is the said road surface shape model which the said road surface shape model produced | generated by the said road surface shape model production | generation means and the said lane detection means produced | generated. Model, the number of points of data used when detecting each road surface shape model, the range in which the data is detected, each road surface shape model generated in the past sampling period, and the behavior of the host vehicle thereafter. Of the difference between each position of the road surface and the position of each road surface shape model in the current sampling period estimated based on the variance or standard deviation of the data with respect to each approximate line constituting each road surface shape model It is characterized by evaluating each based on at least one of these .

Advantageously, in the road surface shape recognition apparatus,
A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
With
The representative distance detection means uses a class value of a class to which a mode value of the frequency of the histogram belongs as a statistical value in the histogram .

A ninth aspect of the road surface shape recognition apparatus,
A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
With
The representative distance detecting means performs processing in order from the lower horizontal line while shifting the horizontal line to be processed upward on the distance image generated by the distance image generating means,
The evaluation means includes
If the representative distance plotted later on the virtual plane is not farther than the previously plotted representative distance, the subsequent plot is evaluated as having no continuity with the previous plot. And excluding the subsequent plot from the virtual plane,
When the number of horizontal lines that have been continuously processed to exclude the plot reaches a predetermined number, the previous plot is excluded .

A tenth aspect of the invention is the road surface shape recognition device according to any one of the first to ninth aspects of the invention, wherein the representative distance detecting means includes past data among the position data existing on each horizontal line of the distance image. Only the data of the distance in the data of the position existing within a predetermined range from the position of the road surface in the current sampling period estimated based on the road surface shape model generated in the sampling period and the subsequent behavior of the host vehicle Are voted on the histogram .

An eleventh invention is the road surface shape recognition device according to any one of the first to tenth inventions, wherein the representative distance detecting means includes a traveling path of the host vehicle estimated from a behavior of the host vehicle in the distance image. A predetermined range is set, and only the distance data in the position data existing on each horizontal line within the predetermined range is voted on the histogram .

A twelfth aspect of the invention is the road surface shape recognition apparatus according to any one of the first to eleventh aspects of the invention,
Lane detection means for detecting a lane on the distance image,
The representative distance detection means sets a predetermined range based on the lane detected by the lane detection means among the horizontal lines of the distance image, and exists on each horizontal line within the predetermined range. Only the distance data in the position data to be voted on the histogram .

A thirteenth aspect of the present invention is the road surface shape recognition device according to any one of the first to twelfth aspects, wherein the road surface shape model generation means is not excluded by the evaluation means and remains on the virtual plane. Hough transform is performed on the plot, two approximate straight lines are calculated, and the road surface shape model is generated as a shape on at least the virtual plane .

According to the first , fifth, sixth, eighth, and ninth inventions, the distance data of each pixel on each horizontal line of the distance image is voted on the histogram for each horizontal line, and the statistical value is calculated for each Detect as the representative distance for each horizontal line, plot the representative distance on the virtual plane for each horizontal line, exclude non-continuous plots, and approximate all plots remaining on the virtual plane A straight line is calculated, and a road surface shape model is generated by selecting the most appropriate combination of approximate straight lines. Therefore, for example, as shown in FIG. 2 to be described later, it is possible to accurately and stably detect the shape of a road surface on which a lane is not marked on a road surface such as an unpaved road or a mountain road.

  Even if a mismatch occurs in stereo matching processing or abnormal data is obtained by a laser radar ranging device, the frequency of occurrence of such data on each horizontal line is smaller than the frequency of normal values. Therefore, such data is shaken off in the voting process for each histogram of the distance data of each pixel on each horizontal line of the distance image. Even if such data is adopted as the representative distance, the plot is excluded if the corresponding plot does not have continuity. Therefore, even when a mismatch occurs in the stereo matching process or abnormal data is obtained by the laser radar distance measuring device, the road surface shape can be accurately and stably detected.

  Further, on a road surface on which a lane or the like is marked, for example, data of a relatively large number of distances are effectively detected at the edge portion of the marking such as a lane illustrated in FIG. 16 to be described later or an arrow marked on the road surface. Therefore, it becomes possible to accurately and stably detect the road surface shape based on it.

Further, according to the first invention , in the actual road design, the shape of the road on the virtual plane is constituted by two straight lines, but the two approximate straight lines are appropriately changed while changing the group boundary. By searching for a combination, it is possible to generate a road surface shape model accurately and quickly using a combination of two approximate straight lines in accordance with the actual road design, and it is possible to accurately recognize the road surface shape. It becomes. Further, by generating a road surface shape model using a combination of approximate lines selected from a combination of two groups of approximate lines based on statistical values based on the approximate lines, the reliability of the generated road surface shape model can be improved. It becomes possible to improve more and the effect of the said invention is exhibited more correctly.

According to the second invention, in the first invention, an approximate line that approximates each plot for each group is calculated by the least square method, and a statistical value based on the approximate line is calculated with respect to the approximate line of each plot belonging to each group. By calculating as the total value of the variances or standard deviations, it is possible to accurately and easily select a combination of approximate straight lines, and to exhibit the effects of the first invention more accurately.

According to the third invention, in the second invention, each time the distance data Da of the boundary portion of each group is transferred from one group to the other group when the approximate straight line is calculated using the least square method. Since it is possible to calculate each equation of the approximate line simply by subtracting za corresponding to the distance data Da from each sum Σz etc. in one group and adding it to each sum Σz etc. in the other group. The approximate straight line can be calculated easily and at high speed, and the effect of the second aspect of the invention can be exhibited more accurately.

According to the fourth invention, in the actual road design, the shape of the road on the virtual plane is composed of two straight lines and a relaxation curve. It is possible to accurately generate a combination of two approximate straight lines and a relaxation curve, and it is possible to accurately recognize the road surface shape. Therefore, the effects of the above inventions can be more accurately exhibited.

According to the fifth aspect of the invention, a road surface shape model is generated based on the lane information detected on the captured image by the lane detection means separately from the road surface shape model generation means, and the road surface is generated by the processing means. By evaluating each of the road surface shape model generated by the shape model generating means and the road surface shape model generated by the lane detecting means, and selecting one of the road surface shape models, more accurate and reliable It is possible to accurately detect the actual road surface shape even on unpaved roads and mountain roads where lanes are not marked, as well as when lanes are marked on the road surface by selecting a high road surface shape model Thus, the effects of the respective inventions can be more accurately exhibited.

According to the sixth aspect of the invention, a road surface shape model is generated based on the information of the lane detected on the captured image by the lane detection means separately from the road surface shape model generation means, and the road surface shape model is generated by the processing means. The road surface shape model generated by the shape model generation means and the road surface shape model generated by the lane detection means are each evaluated, and both road surface shape models are weighted and averaged according to those evaluations, and a new road surface shape model is obtained. By configuring so that the lanes are generated, not only when the lanes are marked on the road surface by the newly generated road surface shape model, but also on roads such as unpaved roads and mountain roads where the lanes are not marked. The road surface shape can be accurately detected, and the effects of the inventions described above can be more accurately exhibited.

According to the seventh invention, the two road surface shape models, the number of data used when detecting each road surface shape model, the range in which the data is detected, each road surface generated in the past sampling period The difference between each position of the road surface and the position of each road surface shape model in the current sampling cycle estimated based on the shape model and the behavior of the host vehicle thereafter, the distribution of data for each approximate line constituting each road surface shape model, or By evaluating each based on at least one of the standard deviations, a more reliable road surface shape model is selected or a new road surface shape model is generated by weighted averaging of both road surface shape models according to the evaluation. It becomes possible, and it becomes possible to exhibit the effect of each said invention more correctly.

According to the eighth invention, the class value of the class to which the mode value of the frequency of the histogram belongs is used as the statistical value in the histogram, so that the class value of the distance that appears most frequently on the horizontal line is determined as the horizontal line. It is possible to accurately generate a road surface shape model based on the detected distance.

According to the ninth aspect, in the road surface shape, the representative distance on the side closer to the host vehicle than the upper horizontal line should be detected in the lower horizontal line of the distance image. Therefore, by performing the processing in the representative distance detection means in order from the horizontal line on the lower side of the distance image, when plotting them on the virtual plane, the representative distance of the plot plotted later is the same as that of the plot plotted earlier. If it is not on the far side from the representative distance, it is possible to easily and accurately evaluate the subsequent plot if it does not have continuity with the previous plot.

Therefore, in such a case, by configuring the subsequent plot to be excluded from the virtual plane, it becomes possible to accurately exclude a plot that is not suitable as a road surface shape, and to generate a road surface shape model. The effect of the invention can be exhibited more accurately, and the reliability of the generated road surface shape model can be further improved.

In the ninth invention, the subsequent plots are excluded. However, if the previous plots were originally plotted based on an abnormal representative distance, the subsequent normal plots are excluded one after another, which is inconvenient. is there. Therefore, when the number of horizontal lines that have been continuously excluded from the plot reaches a predetermined number, the previous plot is excluded, so that the abnormal previous plot is excluded and normal It is possible to make a simple plot as a target for generating a road surface shape model, and to exhibit the effects of the ninth invention more accurately.

According to the tenth invention, the distance existing within a predetermined range from the road surface position in the current sampling cycle estimated based on the road surface shape model generated in the past sampling cycle and the subsequent behavior of the host vehicle. By using only data as a target for voting on the histogram, the effects of the above-described inventions can be exhibited accurately, and a road surface shape model can be stably generated for each sampling period.

According to the eleventh aspect of the invention, in addition to the effects of the inventions described above, the search range of the data of the position existing on each horizontal line is set to a predetermined range including the traveling path of the host vehicle where the road surface is highly likely to exist. By limiting the speed, it is possible to speed up the processing, and it is possible to generate a road surface shape model without being affected by three-dimensional objects outside the traveling road, and stably generate the road surface shape model. It becomes possible to do.

According to the twelfth invention, in addition to the effects of the inventions described above, when the apparatus includes lane detection means, there is a high possibility that the road surface exists in the search range of the data of the position existing on each horizontal line. By limiting to a predetermined range including the lane, it is possible to speed up the processing, and it is possible to generate a road surface shape model without being affected by a solid object outside the traveling road. A shape model can be generated stably.

According to the thirteenth invention, the combination of approximate straight lines can be obtained by the Hough transform for all the plots remaining on the virtual plane without being excluded by the evaluation means. Therefore, by calculating two approximate straight lines by Hough transform and generating a road surface shape model, a road surface shape model can be generated by selecting a combination of approximate straight lines easily and accurately. It is possible to exert the effect accurately.

It is a block diagram which shows the structure of the road surface shape recognition apparatus which concerns on 1st Embodiment. It is a figure which shows the example of the reference | standard image imaged with an imaging means. It is a figure explaining the principle of stereo matching. It is a figure which shows the example of the produced distance image. It is a flowchart which shows the procedure of each process in a process part. It is a flowchart which shows the procedure of each process in a process part. It is a figure explaining the histogram etc. which are produced for every predetermined range and horizontal line which are set in a distance image. It is a figure explaining the example of the representative distance plotted on the virtual plane (zj plane). It is a figure explaining the example of the plot which does not exist in the far side rather than the representative distance plotted previously. It is a figure explaining the example of the plot after being excluded by the abnormal previous plot. It is a figure explaining that a previous plot is excluded when the process which excludes the subsequent plot in the plot of FIG. 10 is performed continuously. (A) It is a figure explaining the example of each plot which remain | survives on a virtual plane, (B) The boundary set between the plot whose representative distance is the 2nd most distant and the 3rd most distant plot in (A) FIG. It is a figure explaining the state which changed the plot of the boundary part, and set the boundary between the plot whose representative distance is the 3rd farthest and the 4th farthest. It is a figure showing the combination of the selected approximate line, a boundary, an intersection, etc. It is a figure showing the circular arc which is a relaxation curve which makes each approximate straight line a tangent. It is a figure which shows the example of the reference | standard image by which the lane etc. were imaged. It is a figure explaining the state where two peak values appeared in the frequency of the histogram in FIG. It is a block diagram which shows the structure of the road surface shape recognition apparatus which concerns on 2nd Embodiment. It is a figure explaining a horizontal line, a search area | region, a search start point, a search end point, etc. It is a graph showing the brightness | luminance of each pixel obtained by searching on a horizontal line. It is a figure showing the lane candidate point detected on the reference | standard image. It is a figure showing the straight line extracted as a result of a Hough conversion process. It is a figure showing the straight line selected as a straight line suitable for a lane from each straight line of FIG. It is a figure showing the straight line decided on the near side of the own vehicle. It is a figure showing the left and right lane detected on the reference | standard image. It is a figure which shows the example of the road surface model produced | generated by the lane detection means, (A) represents a horizontal shape model, (B) represents a road height model, ie, a road surface shape model.

  Embodiments of a road surface shape recognition apparatus according to the present invention will be described below with reference to the drawings.

[First Embodiment]
As shown in FIG. 1, the road surface shape recognition apparatus 1 according to the first exemplary embodiment of the present invention includes an imaging unit 2, a distance image generation unit 6, a representative distance detection unit 10, an evaluation unit 11, an approximate straight line calculation unit 12, A processing unit 9 having a road surface shape model generation unit 13 and the like is provided.

  The upstream configuration of the processing unit 9 including the distance image generating means 6 and the like is described in detail in the Patent Document 1 and the like previously filed by the applicant of the present application, and a detailed description of the configuration is given in the above. Leave it to the gazette. A brief description is given below.

  In the present embodiment, the image pickup means 2 includes a built-in image sensor such as a CCD or a CMOS sensor that is synchronized with each other, and is mounted in the vicinity of a vehicle rearview mirror, for example, at a predetermined interval in the vehicle width direction. This is a stereo camera composed of a main camera 2a on the passenger side and a sub camera 2b on the passenger seat side, and is configured to capture a predetermined sampling period and output a pair of images.

  In the following, one image as shown in FIG. 2 captured by the main camera 2a is referred to as a reference image T, and an image (not shown) captured by the sub camera 2b is referred to as a comparative image Tc. In the following, the case where the process in the approximate straight line calculation unit 12 or the like is performed on the reference image T will be described. However, the process is performed on the comparison image Tc or both the images T and Tc are processed. It is also possible.

  In the present embodiment, the main camera 2a and the sub camera 2b of the imaging unit 2 each acquire monochrome image data. However, color image data represented by RGB values or the like is captured. It is also possible to use an imaging means, and the present invention is also applied to such a case.

  When the reference image T or the like is picked up by the main camera 2a or the like, for example, as shown in FIG. 2, the image pickup starts from the leftmost pixel of each horizontal line J such as the reference image T and then sequentially scans in the right direction. To go. Further, the image data of the luminance of each pixel such as the reference image T is converted into the conversion means 3 in such a manner that the horizontal line J to be scanned is picked up while switching upward from the lowermost line in order. Are sent sequentially.

  The conversion unit 3 includes a pair of A / D converters 3a and 3b, and each image data for each pixel such as the reference image T captured by the main camera 2a and the sub camera 2b of the imaging unit 2 is 256, for example. The image data is sequentially converted into digital image data as the gray scale luminance of the gradation and output to the image correction unit 4.

  In addition, the image correction unit 4 sequentially performs image correction such as displacement and noise removal, luminance correction, etc. on each image data, and stores each image data in the image data memory 5 sequentially, and also in the processing unit 9. It is designed to transmit sequentially. Further, the image correction unit 4 sequentially transmits the image data of the reference image T and the comparison image Tc subjected to the image correction to the distance image generation unit 6.

  The distance image generating means 6 includes an image processor 7 and detects data of a position including a horizontal position x, a height y and a distance z in the real space on a road surface on which the host vehicle travels at a plurality of different points. In addition, a distance image Tz in which data at each position is represented on a two-dimensional plane is generated.

  The image processor 7 of the distance image generating means 6 sequentially performs stereo matching and filtering processing on the image data of the reference image T and the comparison image Tc so as to sequentially calculate the parallax dp for each pixel of the reference image T. It has become.

In stereo matching performed by the image processor 7, as shown in FIG. 3, the reference image T is divided into reference pixel blocks PB having a predetermined number of pixels such as 3 × 3 pixels and 4 × 4 pixels. Then, for each reference pixel block PB, each comparison pixel block PBc on the epipolar line EPL having the same J coordinate as that of the reference pixel block PB in the comparison image Tc is searched for.
SAD = Σ | p1st−p2st | (1)
And the comparison pixel block PBc having the smallest SAD value is specified.

  In Equation (1), p1st represents the luminance of each pixel in the reference pixel block PB, and p2st represents the luminance of each pixel in the comparison pixel block PBc. Further, the above sum is obtained when the reference pixel block PB and the comparison pixel block PBc are set as a 3 × 3 pixel region, for example, a range of 1 ≦ s ≦ 3, 1 ≦ t ≦ 3, and 4 × 4 pixels. When set as a region, calculation is performed for all pixels in the range of 1 ≦ s ≦ 4 and 1 ≦ t ≦ 4.

  In stereo matching, the parallax dp is sequentially obtained for each pixel of the reference image T from the position on the reference image T of the reference pixel block PB and the position on the comparison image Tc of the comparison pixel block PBc specified for the reference pixel block PB. It is calculated.

In real space, the point on the road surface directly below the center of the pair of cameras 2a, 2b is the origin, the vehicle width direction (ie, horizontal direction) of the host vehicle is the X-axis direction, and the vehicle height direction (ie, height direction). ) In the Y-axis direction and the vehicle length direction (that is, the distance direction) as the Z-axis direction, position data (x, y, z) including the horizontal position x, height y, and distance z in real space; The pixel coordinates (I, J) and the parallax dp on the reference image T can be associated one-to-one by coordinate conversion based on the principle of triangulation expressed by the following equations (2) to (4).
x = CD / 2 + z * PW * (I-IV) (2)
y = CH + z × PW × (J−JV) (3)
z = CD / (PW × (dp−DP)) (4)

  In the above equations, CD is the distance between the pair of cameras, PW is the viewing angle per pixel, CH is the mounting height of the pair of cameras, and IV and JV are I on the reference image T at the infinity point in front of the host vehicle. Coordinates, J coordinates, and DP represent vanishing point parallax.

  In the filtering process, the image processor 7 outputs the calculated parallax dp as invalid when the degree of similarity of the luminance pattern between the reference pixel block of the reference image T and the pixel block of the comparison image Tc is low. It has become.

  For this reason, in the present embodiment, the distance image generation unit 6 uses only the parallax dp calculated as valid by the image processor 7 among the parallaxes dp calculated for each pixel of the reference image T in this way (4 ), The calculated distance z in the real space is assigned to each corresponding pixel (I, J) of the reference image T, and the data at each position is placed on a two-dimensional plane as shown in FIG. The represented distance image Tz is generated.

  As described above, in stereo matching, one parallax dp is assigned to a reference pixel block PB of, for example, 3 × 3 pixels on the reference image T. Therefore, as described above, 3 × 3 pixels on the reference image T. The same parallax dp is assigned to these nine pixels.

  In the following process, the distance image Tz is generated so that one reference pixel block PB of 3 × 3 pixels, for example, on the reference image T is regarded as one pixel of the distance image Tz.

In this case, the coordinates (i, j) of each pixel of the distance image Tz and the coordinates (I, J) of each pixel of the reference image T are different, but the coordinates (i, j) of each pixel of the distance image Tz are different. ) And the parallax dp and the horizontal position x, height y, and distance z in the real space are associated one-to-one according to the following equations (5) to (7) similar to the above equations (2) to (4). be able to.
x = CD / 2 + z * PW * (i-IV) (5)
y = CH + z * PW * (j-JV) (6)
z = CD / (PW × (dp−DP)) (7)

  In this case, the meanings of CD, PW, CH, IV, JV, and DP are the same as those in the above formulas (2) to (4), but each numerical value is replaced for the distance image Tz. The coordinates (i, j) and parallax dp of each pixel of the distance image Tz, and the horizontal position x, height y, and distance z in the real space are in a one-to-one relationship according to the expressions (5) to (7). Is associated with.

  Further, instead of generating a distance image Tz by assigning a real space distance z to each pixel of the reference image T as in the present embodiment, a valid parallax dp is assigned to each pixel of the reference image T and the distance image is generated. It can also be configured to generate Tz.

  The distance image generation means 6 stores the distance image Tz generated in this way in the distance data memory 8 and sequentially transmits it to the processing unit 9.

  In the present embodiment, as described above, the distance image generation unit 6 calculates the distance z in the real space calculated by stereo matching or the like with respect to the reference image T and the comparison image Tc captured by the main camera 2a and the sub camera 2b. Although the case where it is comprised so that the distance image Tz may be produced | generated based on parallax dp is demonstrated, it is not limited to this.

  That is, the distance image generation means 6 can detect data of positions including the horizontal position x, height y, and distance z in real space on a road surface on which the host vehicle travels at a plurality of points different from each other. It suffices if the distance image Tz can be generated by representing the data of each position on a two-dimensional plane. For example, like the laser radar distance measuring device described above, it is possible to irradiate a laser beam in front of the host vehicle and detect position data on the road surface on which the host vehicle travels based on information of the reflected light at a plurality of points. Any detection method can be used, and the detection method is not limited to a specific method.

  In this embodiment, the processing unit 9 is configured by a computer in which a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), an input / output interface, and the like (not shown) are connected to a bus. The processing unit 9 includes a representative distance detection unit 10, an evaluation unit 11, an approximate straight line calculation unit 12, a road surface shape model generation unit 13, and a memory (not shown).

  Note that the processing unit 9 may be configured to perform other processing such as object detection for detecting an object such as a preceding vehicle. The processing unit 9 is connected to sensors Q such as a vehicle speed sensor, a yaw rate sensor, and a steering angle sensor for measuring the steering angle of the steering wheel. Measurement values such as a vehicle speed, a yaw rate, and a steering angle are input as needed. It has become so.

  Hereinafter, each process in the process part 9 is demonstrated according to the flowchart of FIG. 5 and FIG. 6, and the effect | action of the road surface shape recognition apparatus 1 which concerns on this embodiment is demonstrated.

  In the present embodiment, the representative distance detection unit 10 first calculates a travel path of the host vehicle that is estimated to travel in the future from the behavior of the host vehicle, that is, the vehicle speed and yaw rate of the host vehicle, in the distance image Tz. As shown in FIG. 7, a predetermined range R including the calculated traveling path is set on the distance image Tz (step S1). The predetermined range R need not always be set.

  Then, the representative distance detecting means 10 subsequently votes the data of the distance z in the real space existing on each horizontal line j of the distance image Tz to the histogram Hj created for each horizontal line j. (Step S2). At this time, in the present embodiment, the representative distance detecting unit 10 votes only the data of the distance z existing on the horizontal line j within the predetermined range R to the histogram Hj. This is because the data of the distance z in the real space other than the predetermined range R including the traveling path of the host vehicle is not necessarily required for detecting the road surface shape, and is intended to speed up the processing by limiting the processing range. .

  Further, in this embodiment, the representative distance detecting means 10 is a case where the position data existing on each horizontal line j, that is, the distance z in the position data is plotted on the virtual plane for each horizontal line j. The plot (z, j) exists within a predetermined range from the road surface position in the current sampling period estimated based on the road surface shape model (described later) generated in the past sampling period and the behavior of the host vehicle thereafter. Only data of such a distance z is voted on the histogram Hj. This is because data of a position that is far away from the position of the road surface estimated from a past road surface shape model or the like is not suitable for detection of the road surface shape.

  Further, as described above, in this embodiment, image data is transmitted from the imaging unit 2 in order from the lowest horizontal line j of the reference image T, and the real space of the distance image Tz generated by the distance image generation unit 6. The data of the upper distance z is also input in order from the lowermost horizontal line j. Therefore, the representative distance detection means 10 also votes the data of the distance z in the real space in order from the lower horizontal line j in the distance image Tz to the histogram Hj, and when the voting on the lower horizontal line j ends, The data of the distance z in the real space of the horizontal line j + 1 immediately above is voted for the histogram Hj + 1, and the data of the distance z in the real space is voted for each histogram Hj for each horizontal line j.

  In this embodiment, the representative distance detecting unit 10 shifts the horizontal line j to be processed on the distance image Tz generated by the distance image generating unit 6 in this way from the lower horizontal line j. Processing is performed in order.

  When the representative distance detection unit 10 completes voting to the histogram Hj of the data of the distance z in the real space existing within the predetermined range R of the horizontal line j in this way, the representative distance detection unit 10 in the histogram Hj of the horizontal line j A statistical value is calculated, and the statistical value is detected as a representative distance zj of the horizontal line j (step S3).

  In the present embodiment, as shown in FIG. 7, the representative distance detection means 10 uses the class value zj of the class to which the mode value of the frequency F of the histogram Hj belongs as a statistical value in the histogram Hj. The representative distance detecting means 10 detects the class value zj of the class to which the mode value of the frequency F belongs as the representative distance zj of the horizontal line j.

  Further, as shown in FIG. 8, the representative distance detecting means 10 plots the representative distance zj of the detected horizontal line j on a virtual plane (hereinafter referred to as zj plane) for each horizontal line j. (Step S4).

  Further, as described above, in this embodiment, the representative distance detection unit 10 shifts the horizontal line j to be processed upward, and performs plotting processing of the representative distance zj on the zj plane for each horizontal line j. To do.

  The evaluation means 11 evaluates the continuity of the plot of the representative distance z on the zj plane performed by the representative distance detection means 10 for each horizontal line j (step S5), and has continuity. The plot evaluated as not (step S5; NO) is excluded from the zj plane (step S6). As described above, there are not a few cases in which mismatching occurs in stereo matching processing, and if the plot does not have continuity corresponding to the road surface, the plot is not suitable for detection of the road surface shape.

  In the present embodiment, when the representative distance zjnew plotted later on the zj plane is not on the far side of the representative distance zjold plotted earlier, the evaluation unit 11 performs the subsequent plotting first. Therefore, the subsequent plot is excluded from the zj plane.

  Specifically, as shown in FIG. 9, the representative distance zjnew plotted later is closer to the representative distance zjold plotted earlier, or the representative distance zjold plotted earlier is used. Existing at the same distance zjold in the real space as the road surface shape is abnormal. Therefore, in such a case, the evaluation unit 11 evaluates the subsequent plot (zjnew, jnew) as not having continuity with the previous plot (zjold, jold), and the subsequent plot (zjnew). , Jnew) are excluded from the zj plane (step S6).

On the other hand, as shown in FIG. 10, for example, when an abnormal representative distance z0 is detected by the representative distance detecting means 10 due to a large number of mismatches occurring in the first horizontal line j (j = 0) or the like, Even if the normal representative distance zj is detected in the horizontal line j, if the above principle is followed, all plots having the normal representative distance zj are excluded, and the representative distance zj farther than the representative distance z0 is excluded. ^The plot on the zj plane is finally resumed when a plot (zj ^* , j ^* ) having ^* appears.

Further, in this state, as can be seen from FIG. 10, for example, the inclination of the plot (z0, 0) and the plot (zj ^* , j ^* ) with respect to the Z-axis direction and the plot (zj ^* , j ^* ) It is difficult to say that the slope continuity with respect to the Z-axis direction of each plot is the same slope, and it is difficult to say that the continuity of the plot representing the road surface shape is ensured.

  Therefore, in this embodiment, the evaluation unit 11 sets the horizontal line j in such a case, that is, the number of horizontal lines j obtained by continuously performing the process of excluding the plot by evaluating as not having continuity, for example, to 4 or the like. When the predetermined number is reached (step S7; YES), the previous plot is excluded (step S8).

  Therefore, for example, when the situation shown in FIG. 10 occurs, the evaluation unit 11 makes a plot (z0,0) corresponding to the horizontal line j (j = 0) on the zj plane as shown in FIG. ) Are plotted, and each plot corresponding to the horizontal lines j (j = 1), j (j = 2), j (j = 3), j (j = 4) is evaluated as having no continuity. When the processing of excluding z1,1), (z2,2), (z3,3), and (z4,4) is continuously performed, the plot (z0) corresponding to the horizontal line j (j = 0) , 0) is an abnormal plot, the plot (z0, 0) is excluded from the zj plane. In this case, the evaluation unit 11 restarts plotting on the zj plane from the plot (z5, 5) corresponding to the horizontal line j (j = 5).

  The approximate straight line calculation means 12 calculates approximate straight lines for all plots (zj, j) remaining on the zj plane (virtual plane) without being excluded by the evaluation means 11. . Since there is also a plot (zj, j) excluded by the evaluation means 11 as described above, in the following, the coordinates of the plot are simply expressed as (z, j), and the above representative distance zj is simply expressed as the representative distance z. .

  Hereinafter, first, an outline of processing in the approximate straight line calculation means 12 will be described.

In the present embodiment, the approximate straight line calculation unit 12 determines all the plots (z, j) that are not excluded by the evaluation unit 11 and remain on the zj plane based on the representative distance z. the plot and divides the side into the group G ₁ and the farther the group of the side G ₂ near the vehicle. Each time the plot of the boundary portion between the _two groups G ₁ and G ₂ is transferred from one group to the other group, approximate lines L ₁ and L ₂ that approximate the plots belonging to the groups G ₁ and G ₂ are approximated. Are calculated for each of the groups G ₁ and G ₂ .

Further, in the present embodiment, approximation line calculating means 12, using the least squares method, for each group _G 1, _{G 2,} the approximate straight line _L 1 which approximates the each plot belonging to each group _G 1, _{G 2} respectively, and calculates the L _2.

Each plot (z, j) is
j = a × z + b (8)
As is well known, in the least square method, a and b are as follows.
a = (nΣzj−Σz · Σj) / {nΣz ² − (Σz) ² } (9)
b = (Σz ² · Σj−Σzj · Σz) / {nΣz ² − (Σz) ² } (10)
It is calculated in the form of In the above equations (9) and (10), n represents the number of plots in each group, and the total sum Σ is performed for all plots in each group.

In the present embodiment, the division of the groups G ₁ and G ₂ and the calculation of the approximate lines L ₁ and L _{2 in} the approximate line calculation unit 12 are performed as follows.

It is assumed that each plot that is not excluded by the evaluation unit 11 remains on the zj plane as shown in FIG. In this case, it is assumed that Σz, Σj, Σz ² and Σzj necessary for the calculations of the above expressions (9) and (10) have already been calculated for all plots.

First, as shown in FIG. 12 (B), the approximate straight line calculation means 12 has a plot D _n−1 that is the second most distant plot D and a plot D that is the third furthest from among all remaining plots D. _The boundary DL is set between _n−2 , and all plots D are automatically composed of n−2 plots D from the plot D ₁ having the closest representative distance z to the plot D _n−2, which is the third most distant. It is far from the own vehicle (z = 0) consisting of the group G ₁ on the side closer to the vehicle (z = 0) and the two plots D, the plot D _n−1 having the second most distant representative distance z and the plot D _n farthest. divided into groups of side _{G 2.}

Then, for each group _G 1, _{G 2,} and calculates an approximate straight line _L 1, _{L 2} that approximates a plurality of plot D belonging to each group _G 1, _{G 2,} respectively. That is, in the group _{G 1,} performs linear approximation with respect to the plot _D 1 _{to D n-2} of n-2 pieces of plot D, the group _{G 2,} the two plots D plot _{D n-1, D n} A straight line approximation is performed on the lines to calculate approximate straight lines L ₁ and L ₂ , respectively.

At this time, for each plot D _{1 to} D _n-2 belonging to the group G ₁ , the plots D _n−1 and D _n of z, j of the group G ₂ from the already calculated Σz, Σj, Σz ² and Σzj. , Z ² , zj are subtracted, and their values are substituted into the above equations (9) and (10) to calculate a and b, and are substituted into the above equation (8) to obtain the approximate straight line L ₁ The equation j = a ₁ × z + b ₁ is calculated.

For each plot D _n−1 , D _n belonging to the group G ₂ , z, j, z ² , zj calculated for each plot D _n−1 , D _n are added, that is, Σz, Σj, Σz ^2, calculates the Shigumazj, those values above (9), (10) are substituted into equation a, calculates b, thereof equation (8) is substituted into the approximate straight line _{L 2} wherein j = a ₂ × z + b ₂ is calculated.

Subsequently, the plot _{D n-2} of the boundary DL portion transferred from a group _{G 1} in the group _{G 2,} the front side That is, as shown in FIG. 13 the position of the boundary DL for dividing the group _{G 1} and the group _{G 2} Move one vehicle toward the vehicle (z = 0). That is, the boundary DL is set between the plot D _n−2 where the representative distance z is the third furthest and the plot D _{n−3 which is} the fourth furthest.

Then, z _n corresponding to the plot D _n−2 to be transferred from each sum Σz, Σj, Σz ² , Σzj for each plot D _{1 to} D _n−2 belonging to the group G ₁ calculated as described above. ₋₂ , j _n−2 , z _n−2 ² , z _n−2 j _n−2 are subtracted, respectively, and the respective sums Σz, Σj, and the respective sums Σz, Σj for the plots D _{1 to} D _n-3 belonging to the group G ₁ Σz ² and Σzj are calculated.

As for the group _{G 2,} the sum Shigumaz for each plot _{D n-1, D n} which belongs to the group _{G 2} calculated as described above, Σj, Σz ^2, the Shigumazj, plot is transferred _{D n- 2} by adding the ^{_{z n-2, j n-}} 2, z n-2 2, z n-2 j n-2 corresponding, respective for each plot _{D n-2} to D _n which belongs to the group _{G 2} Sums Σz, Σj, Σz ² , and Σzj are calculated.

Then, to calculate the equation _{j = a 1 × z + b} 1 of the approximate straight line _{L 1} for each plot _D 1 _{~D n-3} belonging to the group _{G 1} in the same manner as described above, each plot _{D n-2} belonging to the group _{G 2} to D _n to calculate the equation _{j = a 2 × z + b} 2 of the approximate straight line _{L 2} on.

In the present embodiment, the approximate straight line calculation means 12 is based on the representative distance z for all plots D remaining on the zj plane (virtual plane) without being excluded by the evaluation means 11 in this way. All the plots D are divided into a group G ₁ on the side closer to the host vehicle and a group G ₂ on the far side, and the plot Da of the boundary DL portion between the _two groups G ₁ and G ₂ is changed from one group G ₁ to the other. Each time the transfer to the group G ₂ is performed, za, ja, za ² and zaja calculated from the plot Da (xa, ja) are subtracted from the respective sums Σz, Σj, Σz ² and Σzj in the group G ₁ , each sum Shigumaz in G ^_2, Σj, Σz _2, by adding the Shigumazj, has an approximate straight line _L 1, _{L 2} approximating each plot for each group _G 1, _{G 2} to be calculated.

According to this structure, the sum Shigumaz in each group _{G 1, G 2, Σj,} Σz 2, approximate line calculated easily and fast to each Shigumazj, approximating each plot for each group _G 1, _{G 2} L ₁ and L ₂ can be calculated easily and at high speed, respectively.

In the above configuration example, the position of the boundary DL that divides the group G ₁ and the group G ₂ is set on the farthest side from the own vehicle, and the approximate straight line L is moved while moving the position of the boundary DL in a direction approaching the own vehicle. ₁ and L ₂ have been described, but conversely, the position of the boundary DL is closest to the vehicle, that is, the plot D ₂ where the representative distance z is second closest and the plot D closest to third. It is also possible to set between ₃ and to calculate the approximate lines L ₁ and L ₂ while moving the position of the boundary DL in a direction away from the host vehicle.

Also in this case, every time the boundary portion plot Da is transferred from the group G ₂ to the group G ₁ , za, ja, za ² , zaja corresponding to the plot Da are converted into the sums Σz, Σj, Σz ² in the group G ₁ . It adds to Σzj, Σz the sum of the group _{G ^2, Σj,} Σz _2, only subtracted from Shigumazj, the sum Shigumaz in each group _{G 1, G 2, Σj,} Σz 2, Σzj easy and fast to each It is possible to calculate.

Therefore, as shown in the flowchart of FIG. 5, the approximate straight line calculating means 12 first calculates z−, in order to calculate Σz, Σj, Σz ² , Σzj necessary for the calculations of the above equations (9) and (10). Each time the plot on the j plane is evaluated by the evaluation means 11 as having continuity and plotted without being excluded (step S5; YES), the previously calculated Σz, Σj, Σz ² , Σzj Z, j, z ² and zj calculated from the current plot (z, j) are added (step S9).

If the plot on the z-j plane is evaluated as having no continuity by the evaluation unit 11 and excluded (step S8), the approximate straight line calculation unit 12 has already calculated Σz, Σj, Σz ² , If Σzj has been calculated, z, j, z ² and zj calculated from the current plot (z, j) are subtracted from Σz, Σj, Σz ² and Σzj, respectively (step S10).

  The representative distance detection unit 10, the evaluation unit 11, and the approximate line calculation unit 12 perform the above processing every time data of the distance z for each horizontal line j of the distance image Tz is transmitted from the distance image generation unit 6. If the processing has not been completed for all horizontal lines j (step S11; NO), the above processing is performed for the next horizontal line j + 1 (that is, the horizontal line j + 1 immediately above the horizontal line j in the present embodiment). It is supposed to repeat.

  In the present embodiment, as described above, the predetermined range R (see FIG. 7) is set on the distance image Tz (step S1), and the data of the distance z in the predetermined range R is processed. All the horizontal lines j in the determination process of step S11 indicate horizontal lines j in which a predetermined range R exists.

On the other hand, when the processing is completed for all the horizontal lines j (step S11; YES), the approximate straight line calculation means 12 makes all the plots D remaining on the zj plane (virtual plane) as described above. of, first, the representative distance z plot distant second D _n-1 and the third distant plot D _n-2 sets the boundary DL between to the group closer to all plots D in the vehicle G ₁ and the group G ₂ on the far side (step S12 in FIG. 6), approximate lines L ₁ and L ₂ that approximate the respective plots for each of the groups G ₁ and G ₂ are calculated, and the group G ₁ , G ₂ and the information of the approximate lines L ₁ , L ₂ of a ₁ , b ₁ , a ₂ , b _2, etc. for each of the groups G ₁ and G ₂ are stored in the memory in association with each other. (Step S13).

In addition, the road surface shape model generation unit 13 calculates a statistical value based on the approximate lines L ₁ and L ₂ calculated for each of the groups G ₁ and G ₂ by the approximate line calculation unit 12 transferring the plot D, and stores the statistical values. (Step S14).

In the present embodiment, the road surface shape model generating unit 13, approximated straight line _L 1, _L as ₂ based statistics, variance sigma ₁ for approximation lines _L 1, _{L 2} of each plot D belonging to each group _G 1, _{G 2} ² and σ ₂ ² are calculated according to the following equations (11) and (12), respectively.
σ ₁ ² = Σ {(a ₁ × z + b ₁ ) −j} ² / n ₁ (11)
σ ₂ ² = Σ {(a ₂ × z + b ₂ ) −j} ² / n ₂ (12)

Note that n ₁ and n ₂ represent the numbers of plots D belonging to the groups G ₁ and G ₂ , respectively. Moreover, as the statistical value, instead of the variance sigma ^2, it is also possible to constitute the standard deviation of each plot D belonging to each group G _1, G ₂ to calculate for each group G _1, G _2.

If the transfer of the plot Da of the boundary DL portion between the _two groups G ₁ and G ₂ to the group G ₂ on the far side from the group G ₁ on the far side of the host vehicle has not been completed (step S15; NO) ), a plot Da boundary DL portion transferred from a group _{G 1} in the group _{G 2} (step S16), and repeats the processing of steps S13, S14.

When the transfer of the plot Da of the boundary DL portion between the _two groups G ₁ and G ₂ to the group G ₂ on the far side from the group G ₁ on the far side is completed (step S15; YES), the road surface The shape model generation means 13 is, for example, a total value of σ ₁ ² and σ ₂ ² based on a statistical value calculated for each combination of the approximate lines L ₁ and L ₂ among the combinations of the approximate lines L ₁ and L _2. There are equal to select a combination of the approximation lines _L 1, _{L 2} having the smallest, two groups _G 1, _G approximate line _L 1 of _2, selects one from among the combinations of _{L 2} (step S17 ).

Then, the road surface shape model generation means 13 generates a road surface shape model as a shape on the zj plane (virtual plane) using the selected combination of the approximate lines L ₁ and L ₂ (step S18). ).

For example, in the case of a plot D performed on the zj plane as shown in FIG. 12A, the road surface shape model generation means 13 uses a combination of approximate straight lines L ₁ and L ₂ as shown in FIG. A road surface shape model is generated as a shape on the zj plane (virtual plane). That is, in this case, the vehicle (z = 0) approximate line L ₁ in the range of up to the boundary DL _from the distant range than the boundary DL road surface shape model to model an approximate straight line L ₂ is generated.

Further, in the present embodiment, the road surface shape model generation unit 13 further approximates each intersection C of the approximate lines L ₁ and L ₂ of the _two groups G ₁ and G ₂ selected as shown in FIG. The generated road surface shape models L ₁ and L ₂ are corrected by replacing the straight lines L ₁ and L ₂ with relaxation curves that are tangents.

In the present embodiment, an arc Ra as shown in FIG. 15 is used as the relaxation curve. It is also possible to use a quadratic curve or the like as the relaxation curve. When the arc Ra is used as the relaxation curve, the radius of curvature r may be a fixed value set in advance, and depends on the difference between the slopes a ₁ and a ₂ of the approximate straight lines L ₁ and L ₂ , etc. You may comprise so that it may change. Furthermore, the curvature radius r can be variously changed, and the optimum curvature radius r can be calculated based on the variance or the like of each plotted plot D with respect to the arc Ra.

The road surface shape model generation unit 13 calculates the road surface shape model calculated as described above, that is, information on the selected approximate straight lines L ₁ and L _{2 such} as a ₁ , b ₁ , a ₂ , and b ₂ and the position of the boundary DL. Information and the results calculated by each of the above-described means as necessary are stored in a memory and output to an external device.

In the example shown in FIG. 2, the case where there is an uphill or a flat road surface ahead of the host vehicle has been described, but the same processing is performed when there is a downhill ahead of the host vehicle. . Even in such a case, the boundary DL is accurately set, and the approximate straight lines L ₁ and L ₂ are selected accurately for the group G ₁ on the side closer to the host vehicle and the group G ₂ on the far side, respectively, and the road surface shape is selected. A model can be generated.

As described above, according to the road surface shape recognition device 1 according to the present embodiment, the data of the distance z of each pixel on each horizontal line j of the distance image Tz is voted on the histogram Hj for each horizontal line j, The statistical value is detected as the representative distance zj for each horizontal line j, the representative distance zj is plotted on the zj plane (virtual plane) for each horizontal line j, the plot having no continuity is excluded, and z− Approximate straight lines L ₁ and L ₂ are calculated for all the plots (z, j) remaining on the j plane, and the most appropriate combination of approximate straight lines L ₁ and L ₂ is selected to select a road surface shape model. Is generated.

  Therefore, in the road surface shape recognition apparatus 1 according to the present embodiment, for example, as shown in FIG. 2, even on a road surface on which a lane is not marked on a road surface such as an unpaved road or a mountain road, the shape is accurately and stably provided. It becomes possible to detect.

  Even when a mismatch occurs in the stereo matching process or abnormal data is obtained by the laser radar ranging device, the frequency of appearance of such data on each horizontal line j is higher than the frequency of appearance of normal values. Since it is small, such data is shaken off in the voting process for each histogram Hj of the data of the distance z of each pixel on each horizontal line j of the distance image Tz. Even if such data is adopted as the representative distance zj, the plot is excluded if the corresponding plot does not have continuity.

  Therefore, the road surface shape recognition device 1 according to the present embodiment accurately and stably detects the road surface shape even when a mismatch occurs in the stereo matching process or abnormal data is obtained by the laser radar distance measuring device. It becomes possible to do.

  Furthermore, on a road surface on which a lane or the like is marked, for example, data of a relatively large number of distances z are effectively detected at the edge of the marking such as the lane illustrated in FIG. 16 or an arrow marked on the road surface. Based on this, the road surface shape can be accurately and stably detected.

  As described above, according to the road surface shape recognition device 1 according to the present embodiment, it is possible to accurately detect the actual road surface shape. Of course, when the lane is marked on the road surface, the lane is marked. Even if it is not, the road surface shape can be accurately detected.

  In the present embodiment, the case where the predetermined range R including the traveling path of the host vehicle is set as the predetermined range R (see FIG. 7) set in the distance image Tz has been described. As described above, in a scene in which a lane is captured in the reference image T, as described above, a relatively large number of distance z data is effectively detected at the edge portion of the lane in the distance image Tz (not shown). The

  Therefore, for example, as in a second embodiment described later, lane detection means for detecting a lane is provided on the reference image T or the distance image Tz, and a predetermined range R set in the distance image Tz is detected as a lane detection. Based on the lane detected by the means, for example, a predetermined range including the lane may be set.

  In the present embodiment, as shown in FIG. 7, the representative distance detection unit 10 detects the class value zj of the class to which the mode value of the frequency F of the histogram Hj belongs as the representative distance zj of the horizontal line j. Although the case has been described, for example, as shown in FIG. 17, the class values zj1 and zj2 of the class to which a predetermined number of peak values such as two in the frequency F of the histogram Hj belong respectively are used as the statistical values in the histogram Hj. The class values zj1 and zj2 may be detected as the representative distances zj1 and zj2 of the horizontal line j, respectively.

  Further, a threshold value is set in advance for the frequency value or peak value of the frequency F of the histogram Hj, and the class value of the class to which the mode value or peak value belongs only when the mode value or peak value is equal to or greater than the threshold value. Can be detected as a representative distance zj of the horizontal line j.

Further, in this embodiment, the boundary DL is set for all plots D remaining on the zj plane (virtual plane) without being excluded by the evaluation unit 11 by the approximate line calculation unit 12, and the boundary While moving the DL, approximate straight lines L ₁ and L ₂ are calculated for the group G ₁ on the side closer to the host vehicle and the group G ₂ on the far side, respectively, and the road surface shape model generation means 13 calculates the optimum approximate lines L ₁ and L _2. A case has been described in which a road surface shape model is generated by selecting a combination.

However, the method for generating the road surface shape model is not limited to this. For example, the road surface shape model generation unit 13 plots the plot (zj, j) on the zj plane without the plot (zj, j) being excluded by the evaluation unit 11. Each time, the coordinates (zj, j) of the plot are voted on a Hough plane (not shown) to perform Hough transform, and by selecting two appropriate straight lines from the result, two approximate straight lines L ₁ , it is also possible to calculate the L ₂ configured to generate a road surface shape model described above.

[Second Embodiment]
As for the shape of the road surface on which the host vehicle travels, for example, a lane is detected from an image T captured by an imaging means such as a camera as shown in FIG. 16, and a road surface shape model is generated based on the detected lane. It is also possible to recognize. In the second embodiment of the present invention, in addition to the function of the road surface shape recognition device 1 according to the first embodiment, a road surface shape model is further generated based on the lane detected in the image T as described above. The road surface shape recognition device 20 having a function will be described.

  In the following, the road surface shape model generated by the road surface shape model generation unit 13 described above is referred to as a first road surface shape model, and is generated based on the lane detected in the image T by the lane detection unit 14 described later. The road surface shape model is referred to as a second road surface shape model. Further, as described in the first embodiment, the road surface shape model generated by the road surface shape model generation unit 13 based on the processing from the representative distance detection unit 10 to the approximate straight line calculation unit 12 is used as the first road surface shape model. That's it.

  As shown in FIG. 18, also in the road surface shape recognition apparatus 20 according to the second embodiment, the imaging unit 2, the distance image generation unit 6, the representative distance detection unit 10, the evaluation unit 11, the approximate straight line calculation unit 12, the road surface shape. A model generation unit 13 is provided, and these configurations are the same as those of the road surface shape recognition apparatus 1 according to the first embodiment.

  In the present embodiment, the processing unit 9 is further provided with a lane detection unit 14 and a processing unit 15, and the lane detection unit 14 is independent of the representative distance detection unit 10, the road surface shape model generation unit 13, and the like. Thus, lane detection and the like are performed simultaneously with these processes.

  In the present embodiment, the lane detection means 14 detects a lane on the reference image T obtained by imaging the front of the host vehicle. In the following, a case where a lane is detected on the reference image T will be described. However, a configuration may be adopted in which a lane is detected on the comparison image Tc. The configuration of the lane detection process in the lane detection means 14 is described in detail in Japanese Patent Application Laid-Open No. 2006-331389 previously filed by the applicant of the present application, and the detailed description is left to those publications.

  When the image data of the horizontal line J of the reference image T is transmitted from the imaging unit 2 via the image data memory 5 or the like, the lane detection unit 14 searches the horizontal line J and detects pixels adjacent to each other. Pixels having a difference Δp in luminance p that is equal to or greater than a predetermined threshold value Δpth1 and that are located on the road surface are detected as lane candidate points cr and cl. The lane candidate points cr and cl are detected while shifting the horizontal line J to be searched upward in the reference image T in the present embodiment, and the detected lane candidate points cr and cl are joined together. The lanes LR and LL are detected on the reference image T imaged at the sampling cycle.

  In the present embodiment, as described above, the luminance p data of each pixel of the captured reference image T from the imaging unit 2 is obtained for each horizontal line J having a pixel width and from the lower horizontal line J. Since the signals are transmitted in order, the lane detection means 14 is configured to detect the lane candidate points cr and cl while shifting the horizontal line J to be searched for upward in the reference image T as described above.

  Specifically, in this embodiment, the lane detection unit 14 first transmits the positions of the right lane LRlast and the left lane LLlast detected in the previous sampling cycle on the reference image T from the sensors Q. The correction is made based on the behavior of the host vehicle from the previous sampling cycle to the current sampling cycle calculated from the vehicle speed V, yaw rate γ, etc. of the incoming vehicle, and on the reference image T in the current sampling cycle Estimate each lane position.

  Then, for example, as shown in FIG. 19, with respect to each position of the lane on the reference image T in the estimated current sampling period, a range that is a predetermined distance apart in the real space in the left-right direction is set to each horizontal Search areas Sr and Sl for searching for lane candidate points cr and cl on the line J are set to the left and right of the host vehicle, respectively.

  As shown in FIG. 19, when the luminance p of each pixel of the horizontal line J having a width of one pixel is input from the imaging unit 2 in the captured reference image T, in the present embodiment, the lane detection unit 14 is used. First, a search is performed on the horizontal line J in the right search region Sr from the search start point isr at the left end of the region Sr to the search end point ier at the right end in the right direction. Then, as shown in FIG. 20, the difference (ie, edge strength) Δp between the luminance p of a certain pixel and the luminance p of an adjacent pixel is equal to or greater than a predetermined threshold Δpth1, and the pixel in which the luminance p greatly changes is obtained. It is detected as a lane candidate point cr.

  When the lane detection means 14 finishes the search in the right search area Sr on the horizontal line J, the lane detection means 14 subsequently searches the left search area S1 on the same horizontal line J for the search start point isl at the right end of the area Sl. To the left end search end point iel, and the lane candidate point cl is detected in the same manner as described above.

  The lane detection unit 14 repeats the above process every time the luminance p of each pixel of the horizontal line J having a width of one pixel of the captured reference image T is input from the imaging unit 2, and searches for the horizontal line J to be searched. In this embodiment, the lane candidate points cr and cl are detected while shifting upward in the reference image T.

  When the lane candidate points cr and cl detected in this way are plotted on the reference image T, as shown in FIG. 21, lane candidate points cr are respectively displayed in a region A near the right lane and a region B near the left lane. , Cl are plotted. In FIG. 21 and the following figures, it is described that only a dozen or so lane candidate points cr and cl on the right side and the left side are detected, but in reality, there are usually very many lane candidate points. cr and cl are detected.

  In this embodiment, for example, each time the lane detecting unit 14 detects a lane candidate point cr corresponding to the right lane, the coordinates (I, J) of the detected lane candidate point cr are voted on a Hough plane (not shown). When the detection of the lane candidate point cr on each horizontal line J is completed as described above, a Hough conversion process is performed based on the result of voting on the Hough plane, and a straight line suitable for the right lane is obtained. It comes to extract.

  If the Hough conversion process is similarly performed for the lane candidate point cl corresponding to the left lane, the straight lines r1 and r2 as straight lines corresponding to the right lane are obtained from the result shown in FIG. 21, for example, as shown in FIG. In addition, straight lines 11 and 12 are extracted as straight lines corresponding to the left lane, respectively. Therefore, the lane detection means 14 takes into consideration various conditions such as the parallelism with the traveling direction of the host vehicle and the parallelism of the left and right straight lines, as shown in FIG. 23, as straight lines suitable for the right and left lanes. The straight lines r1 and l1 are respectively selected.

  Then, the lane detection means 14 determines the portion on the near side of the selected straight lines r1 and l1 as a straight line suitable for the lane, as shown in FIG. As shown in FIG. 25, the lane candidate points cr and cl whose displacements in the I-axis direction and J-axis direction from the lower lane candidate points cr and cl are within a predetermined range are tracked and connected. In addition, the lanes LR and LL are detected on the reference image T, respectively.

  The lane detection means 14 stores the information on the lanes LR and LL detected in this way, the coordinates of the lane candidate points cr and cl that are the basis of the detection of the lanes LR and LL, and the like in a memory. .

  Further, the lane detection means 14 forms a three-dimensional model of the lane in the real space based on the information on the left and right lanes LR, LL detected in this way, thereby creating a road surface shape model (that is, the second road surface). (Shape model) is generated.

  In the present embodiment, the lane detecting means 14 approximates the detected lanes LL and LR with a three-dimensional approximate straight line for each predetermined section, for example, as shown in FIGS. The road surface model is formed by connecting and expressing. In FIG. 26 (A), lanes LR and LL are approximated by approximate lines in seven sections ahead of the host vehicle, and in FIG. 26 (B), lanes LR and LL are divided into two sections in front of the host vehicle. Although the case where it approximates with an approximate straight line is shown, respectively, it is not limited to this.

  In the present embodiment, the linear equation for each predetermined section is represented by the following equations (13) and (14), and is a horizontal road surface model on a virtual plane (that is, the ZX plane) shown in FIG. A shape model and a road height model which is a road surface model on a virtual plane (that is, a ZY plane) represented by the following equations (15) and (16) and shown in FIG. The detected road surface model information is stored in the memory.

[Horizontal shape model]
Left lane LL x = a _L · z + b _L (13)
Right lane LR x = a _R · z + b _R (14)
[Road height model]
Left lane LL y = c _L · z + d _L (15)
Right lane LR y = c _R · z + d _R (16)

  In FIG. 26 (B), the case of the right lane LR is representatively shown, but a road height model is similarly generated for the left lane LL. The road surface shape model 2 is calculated as an average value at each distance Z of the road height models of the left and right lanes LR and LL.

  In addition, the boundary Z0 divided into two sections ahead of the host vehicle in the road height model (second road surface shape model) shown in FIG. 26B is the boundary DL shown in the first embodiment and its real space. The upper distance z does not necessarily have to be set to be the same, and the position of the boundary Z0 may be a fixed position, or may be configured so that the position can be varied for each sampling period.

  The processing unit 15 includes a first road surface shape model generated by the road surface shape model generation unit 13 based on the processing from the representative distance detection unit 10 to the approximate straight line calculation unit 12 and a second road surface generated by the lane detection unit 14. Each road surface shape model is evaluated, one of the road surface shape models is selected, and output to an external device as necessary. Since the first road surface shape model is a model on the zj plane and the second road surface shape model is a model on the ZY plane as described above, in the evaluation, for example, the above (6) The evaluation is performed by appropriately converting j to y according to the formula or by appropriately converting y to j.

  In the present embodiment, the processing means 15 comprehensively evaluates the following viewpoints and selects one of the first and second road surface shape models.

(1) The number of data points used when detecting each road surface shape model.
The data score in the first road surface shape model is the number of plots (zj, j) remaining on the zj plane (virtual plane) without being excluded by the evaluation means 11. The number of data points in the second road surface shape model is the lane candidate points cr, cl used for detecting the lanes LR, LL, that is, on the detected lanes LR, LL or within a predetermined vicinity thereof. This is the number of lane candidate points cr and cl.

  Note that at most one plot (zj, j) is detected on each horizontal line j of the distance image Tz, while lane candidate points cr and cl are 2 on the left and right on each horizontal line J of the reference image T. Since there is a possibility that they will be detected one by one, the number of plots (zj, j) and the number of lane candidate points cr, cl are converted into a form that can be compared and compared. In addition, the higher the score of data, the higher the evaluation.

(2) A range in which data is detected.
In other words, the plot (zj, j) remaining on the zj plane (virtual plane) without being excluded by the evaluation means 11 or on the detected lanes LR and LL or within a predetermined vicinity thereof. This is a point of view regarding which lane candidate points cr and cl are detected from which distance in front of the host vehicle to which distance. The larger the range in which data is detected, the higher the evaluation.

(3) The difference between each road surface shape model generated in the past sampling cycle and the position of each road surface shape model in the current sampling cycle estimated based on the behavior of the host vehicle thereafter.
In this case, the smaller the position shift for each sampling period, the higher the evaluation that the road surface shape model is stably generated.

(4) Data variance or standard deviation for each approximate straight line constituting each road surface shape model.
As described in the first embodiment, the variances σ ₁ ² and σ ₂ of data (that is, plots) for the _two approximate groups L ₁ and L ₂ of the _two groups G ₁ and G ₂ divided by the boundary DL. ² is calculated by the above equations (11) and (12). In the case of using the standard deviation, the standard deviation can be calculated as the square root of the variance sigma ^2.

Further, the lane detecting means 14 is a variance σ of each detected data (that is, lane candidate points cr, cl) with respect to the approximate straight line represented by the above equations (15), (16) approximating the detected lanes LL, LR. ² or standard deviation is calculated according to the same formula as the above formulas (11) and (12). In this case, the smaller the variance or the standard deviation, the higher the evaluation.

  In addition, it is not necessary to comprise so that all said viewpoints may be evaluated, and a 1st road surface shape model and a 2nd road surface shape model are each evaluated based on at least 1 among those viewpoints.

  In addition, instead of configuring to select either the first road surface shape model or the second road surface shape model, the processing means 15 performs weighted averaging of the road surface shape models according to the evaluation and newly adds them. It can also be configured to generate a road surface shape model.

  In this case, for example, the processing means 15 assigns evaluation points s1 and s2 to the first and second road surface shape models from the above viewpoints, and weights that change depending on the evaluation points s1 and s2, respectively. The first and second road surface shape models can be averaged to generate a new road surface shape model.

  Next, the operation of the road surface shape recognition device 20 according to this embodiment will be described.

  For example, in a scene where an image T as shown in FIG. 16 is captured, for example, a relatively large number of distance z data is effectively detected at the edge portions of the markings such as lanes and arrows marked on the road surface. The road surface shape model generation means 13 and the lane detection means 14 can generate the road surface shape model accurately.

  Therefore, as described above, the processing unit 15 evaluates the first road surface shape model and the second road surface shape model from each of the above viewpoints, and selects or evaluates one of the road surface shape models. Accordingly, the road surface shape model that accurately reflects the shape of the road surface ahead of the host vehicle can be obtained by weighting and averaging the road surface shape models and generating a new road surface shape model.

  Further, for example, in the scene where the image T as shown in FIG. 2 is captured, the lane is not marked on the road surface, so the second road surface shape model cannot be generated by the lane detecting means 14, or Even if it is generated, it is possible to obtain only a road surface shape model that has a very low evaluation and poor reliability.

  However, even in such a case, in this embodiment, as described in the first embodiment, the distance z in the real space obtained for each pixel block by the stereo matching in the distance image generation unit 6 is obtained. Based on the distance image Tz to which the information is assigned, the representative distance detection unit 10 accurately calculates the representative distance zj, and after processing by the evaluation unit 11 and the approximate straight line calculation unit 12, the road surface shape model generation unit 13 accurately. A first road surface shape model can be generated.

  Therefore, in the processing means 15, the first road surface shape model is accurately selected from the above viewpoints, or the first road surface shape model is overwhelmingly weighted with respect to the second road surface shape model. In this case, it is possible to obtain a road surface shape model that accurately reflects the shape of the road surface ahead of the host vehicle.

  As described above, according to the road surface shape recognition device 20 according to the present embodiment, the effect of the road surface shape recognition device 1 according to the first embodiment is accurately exhibited, and the road surface shape model generation unit 13 performs the operation. Select one of the generated first road surface shape model and the second road surface shape model generated by the lane detecting means 14, or perform a new weighted average of these road surface shape models according to the evaluation. The road surface shape model is generated and the lanes LR and LL are marked on the road surface, as well as the road surface shape such as unpaved roads and mountain roads where the lanes LR and LL are not marked. Can be detected.

DESCRIPTION OF SYMBOLS 1,20 Road surface shape recognition apparatus 2 Imaging means 6 Distance image generation means 10 Representative distance detection means 11 Evaluation means 12 Approximate straight line calculation means 13 Road surface shape model generation means 14 Lane detection means 15 Processing means C Intersection cr, cl Lane candidate point D , (Z, j), (zj, j) Plot Da Plot of boundary portion of group DL Boundary F Frequency G ₁ Group G closer to own vehicle ₂ Group far from own vehicle Hj Histogram J Horizontal line j Horizontal line (Horizontal line position)
LR, LL lane L ₁ , L ₂ approximate straight line p brightness R predetermined range Ra arc (relaxation curve)
T Reference image (image)
Tz Distance image x Horizontal position in real space y Height in real space z Distance in real space, distance zj in real space of plot of boundary portion of representative distance za group Plot after representative distance (zjnew, jnew) (Zjold, jold) Previous plot Δp Luminance difference Δpth1 Predetermined threshold σ ₁ ² , σ ₂ ² variance (statistical value)

Claims (13)

A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Equipped with a,
The approximate straight line calculation means, for all the plots that remain on the virtual plane without being excluded by the evaluation means, based on the representative distance, the plots on the side closer to the host vehicle. Each time the plot of the boundary part of two groups is transferred from one group to the other group, an approximate straight line that approximates each plot for each group is divided into a group and a group on the far side. Respectively,
The road surface shape model generation means calculates a statistical value based on the approximate line for each group in which the plot is transferred, and based on the calculated statistical value based on the approximate lines of the two groups. A road surface shape recognition apparatus that selects one of the combinations of the approximate lines in the two groups and generates the road surface shape model using the selected combination of the approximate lines . The approximate straight line calculating means calculates the approximate straight line that approximates each plot for each group by a least square method,
The road surface shape model generation means calculates, for each group, a variance or a standard deviation of the plots belonging to each group with respect to the approximate line as a statistical value based on the approximate line, and the calculated two groups of the two groups The road surface shape recognition apparatus according to claim 1 , wherein a combination of the approximate lines that minimizes a total value of statistical values based on the approximate lines is selected. The approximate line calculation means represents the representative distance as z, the position of the horizontal line in the distance image as j, the representative distance in the plot of the boundary portion to be transferred as za, and the position of the horizontal line as ja. And when the sum of each group is expressed using Σ, each time the plot of the boundary portion is transferred from one group to the other group, Σz, Σj, Σz ² in the one group. , Σzj are respectively subtracted za, ja, za ² , zaja, and Σz, Σj, Σz ² , Σzj in the other group are respectively added to za, ja, za ² , zaja, The road surface shape recognition apparatus according to claim 2 , wherein the approximate straight line that approximates the plot is calculated for each group. The road surface shape model generation means corrects the generated road surface shape model by replacing the intersection of the approximate lines of the selected two groups with a relaxation curve having the approximate lines as tangents. The road surface shape recognition apparatus according to any one of claims 1 to 3 , wherein A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Imaging means for capturing an image of the front of the host vehicle and acquiring an image;
A search is made on a horizontal line on the image to detect a pixel whose luminance difference between adjacent pixels is not less than a predetermined threshold as a lane candidate point, and the horizontal line to be searched is shifted in the vertical direction of the image while shifting the horizontal line. Lanes that detect lane candidate points, connect the detected lane candidate points, detect lanes on the image, and generate a road surface shape model as a shape on at least a virtual plane based on the detected lane information Detection means;
A processing unit that evaluates each of the road surface shape model generated by the road surface shape model generation unit and the road surface shape model generated by the lane detection unit, and selects any one of the road surface shape models;
A road surface shape recognition apparatus comprising: A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Imaging means for capturing an image of the front of the host vehicle and acquiring an image;
A search is made on a horizontal line on the image to detect a pixel whose luminance difference between adjacent pixels is not less than a predetermined threshold as a lane candidate point, and the horizontal line to be searched is shifted in the vertical direction of the image while shifting the horizontal line. Lanes that detect lane candidate points, connect the detected lane candidate points, detect lanes on the image, and generate a road surface shape model as a shape on at least a virtual plane based on the detected lane information Detection means;
The road surface shape model generated by the road surface shape model generation unit and the road surface shape model generated by the lane detection unit are each evaluated, and a weighted average of both the road surface shape models is generated to generate a road surface shape model. Processing means;
A road surface shape recognition apparatus comprising: The processing means uses the road surface shape model generated by the road surface shape model generation means and the road surface shape model generated by the lane detection means to score the data used when detecting each road surface shape model. The range in which the data is detected, each road surface shape model generated in the past sampling cycle, and each position of the road surface in the current sampling cycle estimated based on the behavior of the own vehicle thereafter, and each road surface shape model the difference between the position of, or claim 5 or claim, characterized in that the respectively evaluated based on at least one of the variance or standard deviation of the data the of each approximate straight line constituting each road surface shape model Item 7. The road surface shape recognition device according to Item 6 . A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Equipped with a,
The road surface shape recognition apparatus characterized in that the representative distance detection means uses a class value of a class to which a mode value of the frequency of the histogram belongs as a statistical value in the histogram . A position image including a horizontal position, height and distance in real space on a road surface on which the vehicle travels is detected at a plurality of different points, and a distance image representing the data of each position on a two-dimensional plane is obtained. A distance image generating means for generating;
The distance data in the position data existing on each horizontal line of the distance image is voted for a histogram created for each horizontal line, and a statistical value in each histogram is calculated for each horizontal line. Representative distance detecting means for detecting as a representative distance and plotting the representative distance on a virtual plane for each horizontal line;
Evaluation means for evaluating the continuity of the plot of the representative distance on the virtual plane for each horizontal line and excluding the plot evaluated as having no continuity from the virtual plane;
Approximating straight line calculating means for calculating approximating straight lines for all the plots remaining on the virtual plane without being excluded by the evaluating means;
Road surface shape model generating means for generating a road surface shape model using a combination of the approximate straight lines;
Equipped with a,
The representative distance detecting means performs processing in order from the lower horizontal line while shifting the horizontal line to be processed upward on the distance image generated by the distance image generating means,
The evaluation means includes
If the representative distance plotted later on the virtual plane is not farther than the previously plotted representative distance, the subsequent plot is evaluated as having no continuity with the previous plot. And excluding the subsequent plot from the virtual plane,
The road surface shape recognition apparatus according to claim 1, wherein when the number of horizontal lines obtained by continuously performing the process of excluding the plot reaches a predetermined number, the previous plot is excluded . The representative distance detecting means is estimated based on the road surface shape model generated in the past sampling period and the subsequent behavior of the host vehicle among the position data existing on each horizontal line of the distance image. only data for the distance in the data of the position which exists from the position of the road surface at the current sampling cycle within a predetermined range, any of claims 1 to 9, characterized in that to vote for the histogram one The road surface shape recognition apparatus according to the item. The representative distance detecting means sets a predetermined range including a traveling path of the host vehicle estimated from the behavior of the host vehicle in the distance image, and the position existing on each horizontal line within the predetermined range. The road surface shape recognition apparatus according to any one of claims 1 to 10 , wherein only the distance data in the data is voted on the histogram. Lane detection means for detecting a lane on the distance image,
The representative distance detection means sets a predetermined range based on the lane detected by the lane detection means among the horizontal lines of the distance image, and exists on each horizontal line within the predetermined range. The road surface shape recognition apparatus according to any one of claims 1 to 11 , wherein only the distance data in the position data to be voted is voted on the histogram. The road surface shape model generation unit performs Hough transform on all the plots remaining on the virtual plane without being excluded by the evaluation unit, and calculates the two approximate straight lines to calculate the road surface shape The road surface shape recognition apparatus according to any one of claims 1 to 12 , wherein the model is generated as a shape on at least the virtual plane.

Images