JP5820774B2 - Road boundary estimation apparatus and program - Google Patents

Description

  The present invention relates to a road boundary estimation apparatus and program.

  Conventionally, a three-dimensional object existing in a road area based on the three-dimensional distance information obtained by acquiring three-dimensional distance information of the imaging area based on an image obtained by an image acquisition unit having two or more cameras that capture the road area The first road area in which the height of the three-dimensional object corresponding to the road boundary was detected and the height of the three-dimensional object corresponding to the road boundary could not be detected. For the two road areas, the image is converted to determine whether the three-dimensional object corresponding to the boundary of the first road area and the three-dimensional object corresponding to the boundary of the second road area are the same. Has been proposed, a road boundary detection / determination device that resets the second road area to the first road area has been proposed (see, for example, Patent Document 1).

  Also, feature points are extracted from the image in front of the host vehicle captured by the imaging unit, the moving speed of the pixels representing the feature points on the image is calculated, the pixels are grouped using the moving speed as an index, and the upper and lower ends in the group The coordinates of the pixel located at is converted into coordinates in the overhead coordinates of a predetermined range, the feature point is determined to be a plane object or a three-dimensional object based on the coordinates of the pixel after conversion, and based on the arrangement of the lower end point on the image A road boundary detection device that determines whether or not a road boundary is present has been proposed (see, for example, Patent Document 2).

  Further, a road shape recognition device that detects a road area of a road based on an image obtained by capturing the front side of the traveling direction of the vehicle and estimates the shape of the road based on the road area has been proposed (see, for example, Patent Document 3). ).

JP 2010-72807 A Japanese Patent Laid-Open No. 2007-316685 JP 2011-28659 A

  However, in the technique described in Patent Document 1, the road boundary is determined based on the detection result of the three-dimensional object. However, it is possible to detect a low step such as a curb using only three-dimensional distance information obtained by a stereo camera. Have difficulty. In particular, in a long-distance area where the distance from the camera is far, the error of the three-dimensional distance information becomes large. Therefore, it is not easy to detect a three-dimensional object buried in the error and the road boundary is far away. There is a problem that it is difficult to estimate with high accuracy. In addition, there is a problem in that road surface boundaries such as guardrails, fences, row trees, etc. with many gaps are likely to be undetected at the gaps, and the road boundary may not be estimated accurately.

  In the technique described in Patent Document 2, a three-dimensional object is detected by using an optical flow by a monocular camera. However, as in the technique described in Patent Document 1, it is easy to detect a three-dimensional object having a low height. In addition, since a non-detection is likely to occur even for a three-dimensional object having many gaps, the same problem as described above occurs.

  Further, in the technique described in Patent Document 3, the road area is estimated based on information on the luminance and color of the image. However, such a method is applicable when detecting a lane mark as a road boundary. However, it is difficult to apply when a physical road surface boundary such as a three-dimensional object or a step is to be detected, and conversely, there is a problem that a lane mark, a shadow, or the like causes a false detection.

  The present invention was made in order to solve the above-described problems, and even when there is a three-dimensional object with a low height or a large number of cuts, a road surface boundary that can accurately estimate the road surface boundary far away. An object is to provide an estimation device and a program.

  In order to achieve the above object, a road surface boundary estimation device of the present invention is mounted on a three-dimensional position based on the moving body at each of a plurality of points on an object existing around the moving body, and mounted on the moving body. An acquisition unit that acquires an image obtained by imaging the periphery of the moving object by an imaging unit, and a three-dimensional object region extraction that extracts a three-dimensional object region around the moving object based on the three-dimensional position acquired by the acquisition unit. Means, an edge line segment extraction means for extracting an edge line segment from the image acquired by the acquisition means, and a solid extracted by the solid object region extraction means from the edge line segment extracted by the edge line segment extraction means Line segment selection means for selecting an edge line segment whose distance from the object area is equal to or smaller than a predetermined value; and road surface boundary estimation means for estimating a road surface boundary based on the edge line segment selected by the line segment selection means. Including It is configured.

  According to the road surface boundary estimation apparatus of the present invention, the acquisition unit includes the three-dimensional position based on the moving body of each of a plurality of points on the object existing around the moving body, and the imaging unit mounted on the moving body. An image obtained by imaging the periphery of the moving body is acquired. Then, the three-dimensional object region extracting unit extracts a three-dimensional object region around the moving body based on the three-dimensional position acquired by the acquiring unit. Further, the edge line segment extraction unit extracts the edge line segment from the image acquired by the acquisition unit. Then, the line segment selection means selects an edge line segment whose distance from the solid object area extracted by the solid object area extraction means is a predetermined value or less from the edge line segment extracted by the edge line segment extraction means, and the road boundary An estimation means estimates a road surface boundary based on the edge line segment selected by the line segment selection means.

  In this way, since the road surface boundary is estimated by selecting an edge line segment whose distance from the solid object region is a predetermined value or less from the edge line segment extracted from the image, a solid object with a low height or a solid object with many gaps is used. Even when the road surface exists, the road surface boundary can be accurately estimated far away.

  Further, the road surface boundary estimation means may estimate the road surface boundary by integrating a boundary point between a road surface and a solid object indicated by an outer edge or a lower end of the solid object region and the selected edge line segment. it can. As a result, a road surface boundary can be estimated with high accuracy by interpolating a portion from which no edge line segment is extracted or a portion from which a three-dimensional object region is not extracted.

  Further, the three-dimensional object region extracting unit can extract the three-dimensional object region based on a plurality of three-dimensional positions acquired in time series by the acquiring unit. As a result, the three-dimensional object region can be extracted with higher accuracy.

  The road surface boundary estimation apparatus of the present invention can include a road surface area extraction unit that extracts a road surface area around the moving body based on the three-dimensional position acquired by the acquisition unit, and the line The minute selecting means can exclude edge line segments included in predetermined areas of the three-dimensional object area and the road surface area. Thereby, the edge line segment which becomes a candidate of a road surface boundary can be selected with higher accuracy.

  Further, the road surface area extracting unit can extract the road surface area based on a plurality of three-dimensional positions acquired in time series by the acquiring unit. Thereby, a road surface area can be extracted with higher accuracy.

  Further, the road surface boundary estimation device of the present invention can be configured to include road surface probability calculation means for calculating a road surface probability indicating the road surface probability at each point around the moving body based on the solid object region, The road surface boundary estimation means can estimate the road surface boundary by integrating the road surface probability calculated by the road surface probability calculation means and the selected edge line segment. Thereby, a road surface boundary can be estimated more accurately.

  Further, the road surface boundary estimation unit adds at least one of texture information obtained from the image acquired by the acquisition unit, dynamic background information based on a change in the pixel value of the image, and constraints based on the road surface structure. The road surface boundary can be estimated. By using such information useful for estimating the road surface boundary, the road surface boundary can be estimated with higher accuracy.

  Further, the line segment selection unit can change the predetermined value according to a distance between the moving body and a position corresponding to the edge line segment. Thereby, a more appropriate edge line segment can be selected.

Further, the line segment selecting unit calculates an evaluation value distance between the edge line and the three-dimensional object area is higher the closer, and compared with a predetermined threshold value and the evaluation value, the line segments map The edge line segment to be accumulated can be selected. Further, the line segment selection means can calculate the evaluation value by multiplying a coefficient that increases as the similarity between the moving direction vector of the moving object and the edge line segment increases. The selection means can add a weight based on at least one of the distance and the similarity between the boundary line based on the road surface structure and the edge line segment to the evaluation value. As a result, a highly accurate edge line segment can be selected as a candidate for an edge line segment indicating a road surface boundary.

  Further, the line segment selection means accumulates the selected edge line segment in a line segment map in which previously selected edge line segments are accumulated, and the road surface boundary estimation means is based on the line segment map, The road boundary can be estimated. In this way, by accumulating edge line segments extracted from images acquired at different times, it is possible to compensate for missing edge line segments due to differences in the appearance of edge line segments on the image. It is possible to accurately estimate the road surface boundary.

  Further, the road surface boundary estimation means can estimate the road surface boundary by thinning the edge line segments accumulated in the line segment map. Thereby, since the optimal road surface boundary is extracted from the accumulated edge line segment, the road surface boundary can be estimated with high accuracy.

  Further, the road surface boundary estimation program of the present invention is a program for causing a computer to function as each means constituting the road surface boundary estimation device.

  The storage medium for storing the program of the present invention is not particularly limited, and may be a hard disk or a ROM. Further, it may be a CD-ROM, a DVD disk, a magneto-optical disk or an IC card. Furthermore, the program may be downloaded from a server or the like connected to the network.

  As described above, according to the road surface boundary estimation apparatus and program of the present invention, the road surface boundary is estimated by selecting an edge line segment whose distance from the three-dimensional object region is a predetermined value or less from the edge line segment extracted from the image. Therefore, even when a three-dimensional object with a low height or a three-dimensional object with many cuts exists, the effect that the road surface boundary can be accurately estimated to a long distance can be obtained.

It is a block diagram which shows the road surface boundary estimation apparatus which concerns on 1st Embodiment. It is a figure which shows an example which imaged (a) picked-up image and (b) observation data. It is a figure for demonstrating the extraction method of the solid object area | region based on observation data. It is a figure which shows an example of a solid object map. It is a figure which shows an example of the extraction result of an edge line segment. It is a figure which shows an example of the part which projected the edge line segment on the solid object map. It is a figure for demonstrating evaluation value calculation of an edge line segment. It is a figure for demonstrating integration with a solid-object boundary and the selected edge line segment. It is a flowchart which shows the content of the road surface boundary estimation process routine in the road surface boundary estimation apparatus which concerns on 1st Embodiment. It is a figure which shows an example of the estimation result of a road surface boundary. It is a figure for demonstrating the other method of selection of an edge line segment. It is a block diagram which shows the road surface boundary estimation apparatus which concerns on 2nd Embodiment. It is a figure which shows an example which imaged the extraction result of (a) solid object area | region and (b) road surface area | region. It is a figure which shows the coordinate system regarding adaptation of a runway model. It is a figure for demonstrating the other example of integration of a solid-object boundary and the selected edge line segment. It is a flowchart which shows the content of the road surface boundary estimation process routine in the road surface boundary estimation apparatus which concerns on 2nd Embodiment. It is a block diagram which shows the road surface boundary estimation apparatus which concerns on 3rd Embodiment. It is a figure which shows an example of the line segment which shows the road surface boundary extracted from (a) line segment map and the line segment map of (b) and (a). It is a flowchart which shows the content of the road surface boundary estimation process routine in the road surface boundary estimation apparatus which concerns on 3rd Embodiment. It is a block diagram which shows the road surface boundary estimation apparatus which concerns on 4th Embodiment. It is a figure which shows an example of a road surface probability map. It is a figure for demonstrating calculation of the evaluation value of the edge line segment using the weight according to distance. It is a figure for demonstrating the evaluation value calculation of the edge line segment using the similarity with the own vehicle motion vector. It is a flowchart which shows the content of the road surface boundary estimation process routine in the road surface boundary estimation apparatus which concerns on 4th Embodiment. It is a figure for demonstrating the other example of integration of a solid-object boundary and the selected edge line segment.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In this embodiment, a case where the present invention is applied to a road surface boundary estimation device that is mounted on a vehicle and estimates a road surface boundary around the vehicle will be described as an example.

<First Embodiment>

  As shown in FIG. 1, a road surface boundary estimation apparatus 10 according to the first embodiment is a laser radar 12 that observes a three-dimensional position with respect to the host vehicle at a plurality of points on an object existing around the host vehicle. An imaging device 14 that outputs image data of an image obtained by imaging the periphery of the host vehicle, a computer 16 that executes processing for estimating a road surface boundary, and a display device 18 that displays an estimation result. Yes.

  The road surface boundary here refers to the boundary between the area where the host vehicle cannot physically travel and the area where the vehicle can travel due to the level difference between the road surface and the three-dimensional object, and other sites adjacent to the road surface (parking lots). And the like, including a boundary between a space and a road surface where normal traveling is not assumed. Further, the legal travel boundary identified by a white line or the like is not included in the road surface boundary in the present embodiment.

  The laser radar 12 is a device that is installed in front of the vehicle and observes the azimuth and distance at each point of an object existing in front of the vehicle with a viewing angle centered on the longitudinal direction of the vehicle. For example, the laser radar 12 scans lasers of a plurality of scanning lines to be output in the horizontal direction, and receives laser beams reflected at a plurality of points on the object surface existing in front of the host vehicle. The distance to the reflection point of the laser beam is calculated based on the time from when the laser beam is irradiated until it is received. Further, since the direction in which the laser beam is irradiated is known, the three-dimensional position of the reflection point of the laser beam can be observed.

  The observation data observed by the laser radar 12 is a set of three-dimensional coordinates representing the position of the laser light reflection point in front of the host vehicle. The observation processing by the laser radar 12 is executed in a constant cycle, and the laser radar 12 outputs observation data indicating a plurality of three-dimensional positions of the reflection point of the laser beam in front of the host vehicle at each time point to the computer 16. The three-dimensional position of each point indicated by the output observation data may be polar coordinates represented by the distance and orientation from the laser radar 12 to each reflection point, or on an orthogonal coordinate system centered on the laser radar 12. It may be a three-dimensional position coordinate.

  The imaging device 14 is installed in front of the vehicle, images the surrounding environment in an observation region common to the laser radar 12 at a viewing angle centered on the longitudinal direction of the vehicle body of the vehicle, and shows a captured image obtained by imaging Data is output to the computer 16. The image data output from the imaging device 14 may be a color image or a monochrome image.

  The laser radar 12 and the imaging device 14 need only have a viewing angle in a range in which a road surface boundary is desired to be estimated, and the number of sensors and the viewing angle are not particularly limited. Furthermore, since the installation positions and angles of the laser radar 12 and the imaging device 14 are known, the position of the object observed by the laser radar 12 corresponds to the position of the object in the captured image captured by the imaging device 14. Can be attached. The vehicle width direction is the X direction, the vertical direction is the Y direction, and the vehicle front-rear direction is the Z direction. That is, the observation data observed by the laser radar 12 is three-dimensional information in the XYZ space, the image data captured by the imaging device 14 is information on the XY plane, and a grid map such as a three-dimensional object map 50 described later is an XZ plane. Information.

  FIG. 2A shows an example of a picked-up image picked up by the image pickup device 14, and FIG. 2B shows an example of imaging observation data of the laser radar 12 that has observed the same region as the picked-up image of FIG. .

  The computer 16 includes a CPU, a ROM for storing various programs such as a program for executing a road surface boundary estimation processing routine and a ROM for storing various data, a RAM for temporarily storing data, a memory for storing various information, and the like. It is comprised including the bus which connects.

  If the computer 16 is described with function blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 1, a three-dimensional object area around the host vehicle is extracted from the observation data of the laser radar 12. A three-dimensional object region extracting unit 20, a three-dimensional object boundary estimating unit 22 that estimates a three-dimensional object boundary based on the three-dimensional object region, and an edge line segment extracting unit that extracts an edge line segment from a captured image captured by the imaging device 14. 24, a line segment selection unit 26 that selects an edge line segment based on the solid object region, and a road surface boundary estimation unit that estimates a road surface boundary by integrating the estimated solid object boundary and the selected edge line segment. 28.

  The three-dimensional object region extraction unit 20 acquires observation data observed by the laser radar 12 and extracts a three-dimensional object region indicating a three-dimensional object from the observation data. Specifically, as shown in FIG. 3, on the grid map of the two-dimensional XZ plane obtained by dividing the road surface plane in a grid pattern, the observation data of the laser radar 12 is a cell corresponding to the XZ plane position of the observation data. Project to. Then, a cell in which the variance in the height direction of the observation data group projected in each cell is equal to or greater than a predetermined threshold is extracted as a three-dimensional object region, and a three-dimensional object map 50 as shown in FIG. 4 is generated. In the three-dimensional object map 50 shown in FIG. 4, the cells indicating the three-dimensional object region are black, the cells that are not the three-dimensional object region (cells whose dispersion in the height direction is less than the predetermined threshold) are white, and observation data is not obtained. The cell is shown in gray.

  The three-dimensional object boundary estimation unit 22 estimates the outer edge of the three-dimensional object region as a three-dimensional object boundary indicating the boundary between the three-dimensional object and the road surface based on the three-dimensional object map 50 generated by the three-dimensional object region extraction unit 20.

  The edge line segment extraction unit 24 acquires image data from the imaging device 14, and uses the existing Sobel operator or Prewitt operator on the captured image indicated by the image data, to detect the edge of each pixel (the magnitude and direction of the luminance gradient). ). Then, the edge direction to be extracted is limited according to the position in the image, and the edge continuously extending in the vanishing point direction is extracted as an edge line segment. As a result, the edge of a three-dimensional object extending in the vertical direction can be removed. FIG. 5 shows an example of the edge line segment extracted on the captured image. Further, as shown in FIG. 5, an image processing range (ROI: Region of Interest) may be set based on the depression angle of the imaging device 14 or the amount of vehicle pitch fluctuation.

  The line segment selection unit 26 narrows down the edge line segments constituting the road surface boundary from the edge line segments extracted by the edge line segment extraction unit 24. Here, since a physical boundary such as a step or a barrier is assumed as the road surface boundary, an edge line segment existing in the vicinity of the three-dimensional object is selected as a candidate for the road surface boundary. Specifically, each of the edge line segments that are XY plane information extracted by the three-dimensional object region extraction unit 24 is projected onto the three-dimensional object map 50 that is XZ plane information. FIG. 6 shows an example of a part of an edge line segment projected on the three-dimensional object map 50. A cell indicating a three-dimensional object region is indicated by a circle, and a cell on which an edge line segment is projected (hereinafter referred to as an “edge line segment cell”) is shaded. The line segment selection unit 26 selects an edge line segment whose distance from the three-dimensional object region is a predetermined value or less. In the example of FIG. 6, since the edge line segment corresponding to the center line is far from the three-dimensional object, it is excluded from the edge line segment candidates indicating the road surface boundary.

  For example, as shown in FIG. 7, the evaluation of the distance between the three-dimensional object region and the edge line segment is performed with respect to each cell of the edge line segment as a neighboring cell in the positive / negative direction of X around the target cell (here, the target cell itself). And the evaluation value shown in the following formula (1) is calculated based on whether or not each of the two positive and negative cells is a cell indicating a three-dimensional object region.

Here, Eval _i is an evaluation value of the edge line segment i, eval _n is an evaluation value of the cell n of the edge line segment i, and m is the number of cells included in the edge line segment i. In the example of FIG. 7, since eval ₁ = 0, eval ₂ = 1, eval ₃ = 1, eval ₄ = 1, and eval ₅ = 0, Eval = 3. The line segment selection unit 26 determines and selects an edge line segment having the evaluation value Eval of the edge line segment calculated in this way larger than the threshold value thr as an edge line segment passing through the foot position of the three-dimensional object. The threshold value thr is set to a value such that the distance between the three-dimensional object region and the edge line segment is a predetermined value or less. The neighboring cells of each cell may be evaluated not only on the target cell column but also on a matrix including the positive and negative directions of Z.

  Note that the resolution of the captured image captured by the imaging device 14 and the observation data observed by the laser radar 12 becomes coarser as the region is farther from the device, and the detection accuracy of the edge line segment and the three-dimensional object region deteriorates. In other words, the calculation accuracy of the relative distance between the three-dimensional object region and the edge line segment also deteriorates in the far region. Therefore, the selection condition of the edge line segment (the threshold value of the distance to the three-dimensional object region. For example, the threshold value thr) may be changed according to the distance between the host vehicle and the position corresponding to the edge line segment. . For example, it can be changed according to the distance observed by the laser radar 12, or can be changed according to the vertical position in the image of the edge line segment.

  Further, the edge line segment selected by the line segment selection unit 26 does not necessarily have to be narrowed down to one. In addition to the distance from the three-dimensional object region, a plurality of candidates are considered in consideration of the edge strength, length, and inclination, and the position of the solid object boundary is estimated when the road surface boundary estimating unit 28 described later estimates the road surface boundary. The optimum edge line segment may be selected in consideration of the relationship and continuity from near to far.

  The road surface boundary estimation unit 28 integrates the solid object boundary estimated by the solid object boundary estimation unit 22 and the edge line segment selected by the line segment selection unit 26 to estimate a road surface boundary. At the time of integration, a constraint condition is provided so that road surface boundaries are continuously connected from the vicinity toward the distance. Specifically, as shown in FIG. 8A, the estimated solid object boundary and the selected edge line segment are mapped to the same XZ plane. Next, as shown in FIG. 2B, support points that constitute road surface boundaries are extracted at regular intervals in the depth direction of the course of the host vehicle. The support point is selected from the three-dimensional object boundary and the edge line segment that is closer to the planned traveling route of the host vehicle. Finally, as shown in FIG. 5C, a line passing through the extracted support point is estimated as a road surface boundary.

  The interval for extracting the support points is not limited to a fixed interval, and the interval may be changed according to the type of each boundary candidate (solid object boundary or edge line segment) and the distance from the host vehicle. For example, since the detection accuracy of the edge line segment and the three-dimensional object boundary is good in the vicinity region, the interval for extracting the support points can be made dense, and the interval can be made coarse in the far region. In addition, it is not limited to selecting the one close to the planned traveling route of the host vehicle among the three-dimensional object boundary and the edge line segment, and either the three-dimensional object boundary or the edge line segment is preferentially selected or the support point is selected. When there are a plurality of candidates, their median value, average value, or the like may be selected.

  Next, the operation of the road surface boundary estimation apparatus 10 according to the first embodiment will be described.

  First, the laser radar 12 scans the laser of a plurality of scanning lines in the horizontal direction in front of the host vehicle, and observes observation data specifying the three-dimensional position of each reflection point around the vehicle. In addition, in synchronization with the observation timing of the laser radar 12, image data indicating an image obtained by imaging the periphery of the vehicle by the imaging device 14 is output. Then, the computer 16 executes a road surface boundary estimation processing routine shown in FIG.

  In step 100, observation data observed by the laser radar 12 and image data output from the imaging device 14 are acquired.

  Next, in step 102, the three-dimensional object region extraction unit 20 projects the observation data acquired in step 100 onto a two-dimensional XZ plane grid map, and the height of the observation data group projected in each cell. A three-dimensional object map 50 is generated by extracting a cell whose direction variance is equal to or greater than a predetermined threshold as a three-dimensional object region.

  Next, in step 104, the three-dimensional object boundary estimation unit 22 estimates the outer edge of the three-dimensional object region as a three-dimensional object boundary indicating the boundary between the three-dimensional object and the road surface based on the three-dimensional object map 50 generated in step 102. To do.

  Next, in step 106, the edge line segment extraction unit 24 sets the processing range ROI for the captured image acquired in step 100, and the edge line segment continuously extending in the vanishing point direction from the image within the processing range ROI. To extract.

  Next, in step 108, the line segment selection unit 26 converts each edge line segment that is the XY plane information extracted in step 106 to the three-dimensional object map 50 that is the XZ plane information generated in step 102. Project. Then, an edge line segment whose distance from the three-dimensional object region is equal to or smaller than a predetermined value is selected as an edge line segment that passes through the foot of the three-dimensional object.

  Next, in step 110, the road surface boundary estimation unit 28 maps the three-dimensional object boundary estimated in step 104 and the edge line segment selected in step 108 to the same XZ plane, and the course of the host vehicle. The support points constituting the road boundary are extracted at regular intervals in the depth direction of the road, and a line passing through the support points is estimated as the road boundary.

  Next, in step 112, the road surface boundary estimation unit 28 controls the projection result of step 110 to be projected onto the XY plane corresponding to the captured image, combined with the captured image, and displayed on the display device 18. Then, the road surface boundary estimation process ends.

  FIG. 10 shows an example of a road surface boundary estimation result. The area where the edge line segment has not been extracted is assumed to be the solid object boundary (bottom position of the three-dimensional object area) as the road surface boundary, so the estimation result has some irregularities, but the physical road boundary can be estimated appropriately. ing. In addition, although there is a portion where the implantation is interrupted on the opposite lane side, the road surface boundary can be estimated without discontinuity even in the region where the solid object has a gap by using the edge line segment.

  As described above, according to the road surface boundary estimation apparatus 10 according to the first embodiment, it is determined that the vehicle passes through the foot of the three-dimensional object using the three-dimensional object map generated based on the observation data of the laser radar. Since an edge line segment is selected and the road surface boundary is determined based on the edge line segment, it is possible to accurately estimate the road surface boundary far away even when there are low-height solid objects or solid objects with many gaps. it can.

  In the first embodiment, the case where the selection of whether or not the edge line segment passes through the foot of the three-dimensional object is performed on the three-dimensional object map has been described, but this process may be performed on the captured image. Good. Specifically, first, in the three-dimensional object map 50 generated by the three-dimensional object region extraction unit 20, by projecting a cell indicating the three-dimensional object region onto the captured image, as shown in FIG. Recognizes a three-dimensional object region. Next, the three-dimensional object boundary estimation unit 22 estimates a point along the lower end of the three-dimensional object region recognized on the captured image as a three-dimensional object boundary. And it is good for the edge line segment extraction part 24 to select the edge line segment with the short distance with the estimated solid-object boundary.

<Second Embodiment>

  Next, a second embodiment will be described. In addition, about the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  As shown in FIG. 12, a road surface boundary estimation device 210 according to the second embodiment includes a laser radar 12, an imaging device 14, a computer 216 that executes processing for estimating a road surface boundary, and a display device 18. I have. If the computer 216 is described in terms of functional blocks divided for each function realization means determined based on hardware and software, as shown in FIG. 12, the three-dimensional object region extraction unit 20 and the observation data of the laser radar 12 are used. A road surface area extraction unit 21 that extracts a road surface area around the vehicle, a solid object boundary estimation unit 22, an edge line segment extraction unit 24, and a line segment selection unit that selects an edge line segment based on the solid object area and the road surface area 226 and a road surface boundary estimation unit 228 that integrates the estimated solid object boundary and the selected edge line segment and adapts the road model 52 to estimate the road surface boundary.

  The road surface area extraction unit 21 acquires observation data observed by the laser radar 12 and extracts a three-dimensional object area indicating a three-dimensional object from the observation data. Specifically, as in the three-dimensional object region extraction unit 20, a cell in which the variance in the height direction of the observation data group projected in each cell on the grid map is less than a predetermined threshold is designated as a road surface region (free space). ). When a three-dimensional object is detected in a certain direction as viewed from the host vehicle, no other three-dimensional object exists between the three-dimensional object and the host vehicle, and it can be said that this is a free space. And the solid object map 250 which integrated the information of the extracted road surface area | region and the information of the solid object area extracted by the solid object area extraction part 20 is produced | generated.

  The three-dimensional object region extraction unit 20 and the road surface region extraction unit 21 can divide the three-dimensional object region and the road surface region from the observation at each time step, but the observation is based on a plurality of observation data acquired in time series. It is possible to generate a highly accurate three-dimensional object map by integrating the results of the region division in consideration of the own vehicle motion between the two. For example, when projecting observation data, a corresponding cell is selected and projected from the three-dimensional object map 250 generated up to the previous time step in consideration of the vehicle motion. Then, comprehensively determine the values of past observation data accumulated in the cell and the observation data projected this time, and whether or not the cell is a three-dimensional object region or a road surface region Can be determined. FIG. 13A shows an example in which the extracted three-dimensional object region and the extracted road surface region are imaged in FIG. The estimation accuracy can be further improved by estimating the road surface boundary using a three-dimensional object map generated using a plurality of observation data, rather than a three-dimensional object map generated using one observation data.

  In the above description, the road surface area is extracted using the height information of the observation data of the laser radar 12. However, each reflection point is determined based on the pulse shape of the laser light and the signal intensity of the received laser light. The road surface area may be extracted by determining whether or not is a road surface reflection point. Further, it may be determined whether or not it is a road surface reflection point based on the arrangement (continuity) of each reflection point. For example, since the road surface is generally a flat surface, the road surface reflection points observed on the same scanning line are arcuate. Therefore, the reflection point that draws an arc that crosses the traveling direction in front of the vehicle when the amount of change in the height direction of the three-dimensional position between adjacent reflection points is small and the reflection points are connected is defined as a road surface reflection point. A region including the road surface reflection point can be extracted as a road surface region.

  The line segment selection unit 226 selects an edge line segment using the generated three-dimensional object map 250. Edge line segments indicating guard rail ridge lines can be easily excluded because they are included in the three-dimensional object region, and edge line segments indicating center lines and road markings can be easily excluded because they are included in the road surface region. it can. Also, white line edges close to a three-dimensional object on the road shoulder can be accurately excluded. Even when a plurality of candidate edge line segments appear at positions close to a three-dimensional object, such as white lines, asphalt boundaries, side grooves, and curbs, the edge line segments that are candidate road surface boundaries can be narrowed down with high accuracy. In this way, after narrowing down the edge line segment based on whether or not it is an edge line segment existing within a predetermined area of the solid object area and the road surface area (for example, an area inside the predetermined distance from the outer edge of each area), Similar to the embodiment, the edge line segment is selected based on the distance from the three-dimensional object region.

  The road surface boundary estimation unit 228 integrates the solid object boundary estimated by the solid object boundary estimation unit 22 and the edge line segment selected by the line segment selection unit 226 in the same manner as in the first embodiment. Estimate the boundary. Further, the traveling road model 52 is adapted to the estimated road surface boundary. Since the road surface boundary is continuously connected and does not change in the time series direction, the stability and accuracy of the estimation result are improved by adapting the road model 52 to the time series estimation result. Here, a clothoid curve of a cubic function shown in the following equation (2) is used as the road model 52, and the road surface boundary is estimated by adapting parameters so as to match the extracted left and right support points.

Here, C ₁ , C ₂ , and C ₃ are parameters indicating a yaw angle, a curvature, and a curvature change rate, respectively, and d ₁ and d ₂ represent offsets with respect to the left and right boundary lines. Moreover, the coordinate system of Formula (2) is shown in FIG. However, since there is no standard that roads follow clothoid curves on ordinary roads, other curves may be defined as the runway model 52 in advance. The road model 52 may be based on the road surface structure around the vehicle, and may have a road model 52 corresponding to the position of the host vehicle acquired by GPS or the like using map information. Furthermore, although the same curvature and yaw angle are assumed on the left and right in the above equation, separate parameters on the left and right may be fitted. Moreover, you may assume the runway model 52 containing a discontinuous point like a broken line model.

  Further, the road surface boundary estimation unit 228 may extract the support points when integrating the solid object boundary and the selected edge line segment in a stepwise manner in accordance with the distance from the host vehicle. For example, as shown in FIG. 15, the front area of the host vehicle is divided into a short distance area A, a middle distance area B, and a long distance area C. First, a support point is extracted from the short distance area A and the parameters of the road model 52 are estimated. Next, a boundary line that is a candidate for a road surface boundary is drawn using the estimated parameters, and a support point is extracted from an edge line segment close to the boundary line or a solid object boundary from the intermediate distance region B. The parameters of the road model 52 are estimated again by combining the support points extracted in each of the short distance area A and the intermediate distance area B. A support point is extracted from the long-distance region C in the same procedure, and a line segment indicated by the runway model 52 obtained from the finally estimated parameters is estimated as a road surface boundary. In this procedure, since the support points in the distance are selected based on reliable support points in the vicinity, the influence of the distant support points that are likely to increase the error is suppressed, and the estimation accuracy of the road boundary is further improved. can do.

  Next, a road surface boundary estimation processing routine in the second embodiment will be described with reference to FIG. In addition, about the process similar to the road surface boundary estimation process routine in 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  In step 100, observation data observed by the laser radar 12 and image data output from the imaging device 14 are acquired.

  Next, in step 200, the three-dimensional object region extraction unit 20 extracts a three-dimensional object region based on the observation data acquired in step 100, and the road surface region extraction unit 21 extracts a road surface region based on the observation data. Then, the three-dimensional object map 250 is generated.

  Next, in step 104, the three-dimensional object boundary estimation unit 22 estimates the solid object boundary, and in step 106, the edge line segment extraction unit 24 extracts edge line segments from the captured image.

  Next, in step 202, the line segment selection unit 226 projects each of the edge line segments extracted in step 106 onto the three-dimensional object map 250 generated in step 200, so that the three-dimensional object region and the road surface region are displayed. Exclude edge line segments in a predetermined area. Then, among the remaining edge line segments, an edge line segment whose distance from the three-dimensional object region is a predetermined value or less is selected as an edge line segment that passes through the foot of the three-dimensional object.

  Next, at step 204, the road surface boundary estimation unit 228 integrates the solid object boundary estimated at step 104 and the edge line segment selected at step 202, and adapts the integration result to the road model 52. Then, the road boundary is estimated. Next, in step 112, the estimation result is combined with the captured image and displayed on the display device 18, and the road boundary estimation process is terminated.

  As described above, according to the road surface boundary estimation device 10 according to the second embodiment, an edge that passes through the foot of a three-dimensional object more accurately by extracting a road surface region when generating a three-dimensional object map. A line segment can be selected, and the estimation accuracy of the road surface boundary is also improved. Further, when extracting the three-dimensional object region and the road surface region, the three-dimensional object region and the road surface region can be extracted with higher accuracy by using a plurality of observation data acquired in the past. In addition, a smooth road surface boundary can be estimated with higher accuracy by estimating the road surface boundary in accordance with the road model.

<Third Embodiment>

  Next, a third embodiment will be described. In addition, about the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  As shown in FIG. 17, a road surface boundary estimation device 310 according to the third embodiment includes a laser radar 12, an imaging device 14, a computer 316 that executes processing for estimating a road surface boundary, and a display device 18. I have. When this computer 316 is described by functional blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 17, a three-dimensional object region extracting unit 20, an edge line segment extracting unit 24, A configuration including a line segment selection unit 326 that selects an edge line segment based on the three-dimensional object map 50 and accumulates it in the line segment map 54, and a road surface boundary estimation unit 328 that estimates a road surface boundary based on the line segment map. Can be represented.

  Similar to the first embodiment, the line segment selection unit 326 selects an edge line segment whose distance from the three-dimensional object region is a predetermined value or less from the edge line segments extracted by the edge line segment extraction unit 24. Then, the selected edge line segment is projected and accumulated on the line segment map 54. Similar to the three-dimensional object map 50, the line segment map 54 is a grid map of a two-dimensional XZ plane obtained by dividing a road surface plane in a lattice shape. The line segment selection unit 326 projects the selected edge line segment onto the corresponding cell of the line segment map 54. At the time of projection, a predetermined occupation value (for example, 0.1 or 0.4) is assigned, and added to the occupation value of each cell accumulated up to the previous time step and accumulated.

  The road surface boundary estimation unit 328 takes a cell having a peak occupation value in the line segment map 54 and estimates it as a road surface boundary. The line segment estimated by taking the cell having the peak occupancy value is a line segment that is connected in the Z direction by selecting the cell having the maximum occupancy value for each column in the X direction of the line segment map 54. FIG. 18A shows an example of the line segment map 54, and FIG. 18B shows an example of the road boundary extracted from the line segment map 54 of FIG. In addition, the road surface boundary estimation unit 328 controls the extracted road surface boundary to be projected onto the XY plane corresponding to the captured image, combined with the captured image, and displayed on the display device 18.

  Next, a road boundary estimation processing routine in the third embodiment will be described with reference to FIG. In addition, about the process similar to the road surface boundary estimation process routine in 1st Embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  Through steps 100 to 108, an edge line segment that is a candidate for a road surface boundary is selected. Next, in step 300, the line segment selection unit 326 projects the edge line segment selected in step 108 onto the corresponding cell of the line segment map 54, and a predetermined occupancy value (for example, 0.1 or 0. 4) is added, and added to the occupied value of each cell accumulated up to the previous time step and accumulated.

  Next, in step 302, the road surface boundary estimation unit 328 estimates a line segment extracted by picking up a cell whose occupation value is peak in the line segment map 54 as a road surface boundary. Next, in step 112, the estimation result is imaged. Control is performed so that the image is combined with the image and displayed on the display device 18, and the road boundary estimation process is terminated. By repeating this routine for each time step, the edge line segments selected using the three-dimensional object map 50 are sequentially accumulated in the line segment map 54.

  As described above, according to the road surface boundary estimation device 310 according to the third embodiment, the edge line segment determined to pass the foot of the three-dimensional object using the three-dimensional object map 50 is accumulated in the line segment map 54. Since the road surface boundary is determined based on this line segment map 54, the lack of edge line segments generated in the sequential processing is complemented, and the road surface boundary can be estimated with higher accuracy. Also, by accumulating edge line segments, even if they are edge line segments in the same place, the appearance looks different on captured images captured at different times. A boundary with a space or the like is also likely to be extracted as a line segment having a part close to a three-dimensional object at any time, and an appropriate road surface boundary can be estimated.

  In the above description, the case has been described in which the information on the edge line segment accumulated in the line segment map 54 is thinned by picking up the cell whose occupation value is the peak in the line segment map 54 and extracting the road surface boundary. It is not limited to. For example, the road surface boundary may be extracted by thinning the edge line segment accumulated in the line segment map 54 by fitting with the least square approximation. Further, as in the second embodiment, the road surface boundary may be estimated from the line segment map 54 by fitting with the road model 52.

<Fourth embodiment>

  Next, a fourth embodiment will be described. In addition, about the structure similar to 1st and 3rd embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  As shown in FIG. 20, a road surface boundary estimation device 410 according to the fourth embodiment includes a laser radar 12, an imaging device 14, a computer 416 that executes processing for estimating a road surface boundary, and a display device 18. I have. When the computer 416 is described by function blocks divided for each function realizing means determined based on hardware and software, as shown in FIG. 20, a three-dimensional object region extraction unit 420, an edge line segment extraction unit 24, A line segment selection unit 426 that selects an edge line segment based on the three-dimensional object map 450 and stores it in the line segment map 54; a road surface probability calculation unit 32 that calculates a road surface probability map 56 based on the three-dimensional object map 450; A road surface boundary estimation unit 428 that estimates a road surface boundary based on the minute map 54 and the road surface probability map 56 can be used.

  The three-dimensional object region extraction unit 420 projects and accumulates the observation data of the laser radar 12 on the three-dimensional object map 450, similarly to the three-dimensional object region extraction unit 20 according to the first embodiment. In the three-dimensional object region extraction unit 420 according to the fourth embodiment, when the observation data is projected onto the three-dimensional object map 450, a predetermined occupation value (for example, 0.1 or 0.4) is given, and the previous time It accumulates by adding to the occupation value of each cell accumulated until the step. In addition, each cell of the three-dimensional object map 450 holds height information of observation data projected on the cell.

  The road surface probability calculation unit 32 is based on the height information and the occupancy value held by each cell of the three-dimensional object map 450 generated by the three-dimensional object region extraction unit 420, and indicates the road surface probability indicating the road surface likelihood at each point around the host vehicle. Is calculated. Further, the road surface probability map 56 is generated by storing the calculated road surface probability in each cell of the grid map having the same specifications as the three-dimensional object map 450. An example of imaging the road surface probability map 56 is shown in FIG. In the example of FIG. 21, the portion having a higher road surface probability is expressed in a darker color.

Similarly to the line segment selection unit 326 according to the third embodiment, the line segment selection unit 426 converts each of the edge line segments that are XY plane information extracted by the three-dimensional object region extraction unit 24 into XZ plane information. An evaluation value Eval _i for each edge line segment is calculated by projecting onto a solid object map 450. When calculating the evaluation value Eval _i , the line segment selection unit 426 according to the fourth embodiment multiplies the weight coefficient w according to the distance between each cell of the edge line segment and the cell indicating the three-dimensional object region. .

For example, as shown in FIG. 22, the weighting factor of the cell of interest is w ₀ , the weighting factor of the cell whose distance from the cell of interest is 1 is w ₁ , and the weighting factor of the cell whose distance from the cell of interest is 2 is w _2. , W ₀ = 1.0, w ₁ = 0.9, w ₂ = 0.8. As an example, when the cell indicating the solid object region and the cell of the edge line segment have a positional relationship as shown in FIG. 7, eval ₁₋ = 0, eval ₂ × w ₁ = 0.9, eval ₃ × w _{Since 2} = 0.8, eval ₄ × w ₁ = 0.9, and eval ₅ = 0, Eval = 2.6. The weighting factor may be a value that increases as the attention cell and the cell indicating the three-dimensional object region become closer, and may be determined based on, for example, a Gaussian distribution.

Further, the line segment selection unit 426 calculates an evaluation value Eval _{i obtained} by multiplying the similarity between the edge line segment, the vector direction of the own vehicle motion, and the edge line segment as a weight. For example, as shown in FIG. 23, the similarity can be set to cos θ (−π / 2 ≦ θ ≦ π / 2) when the angle formed by the vector of the own vehicle and the edge line segment is θ. . The vehicle motion vector can be obtained by acquiring sensor values such as a speed sensor and a yaw rate sensor (not shown) mounted on the vehicle.

An example of a formula for calculating the evaluation value Eval _i including the weighting factor w according to the distance and the similarity cos θ with the own vehicle motion vector is shown in the following formula (3).

  The line segment selection unit 426 also weights the boundary line segment estimated by the road model such as a clothoid curve in consideration of the weight coefficient corresponding to the distance from the edge line segment and the similarity to the own vehicle motion vector. May be added to the evaluation value.

  The road surface boundary estimation unit 428 integrates the line segment map 54, the road surface probability map 56, and other information, and estimates the road surface boundary by dynamic programming or the like. As other information, dynamic background information obtained by converting a change in luminance value of each pixel between frames of the captured image into a mosaic image, texture information obtained from the captured image, and the like can be used. These pieces of information are useful information for identifying the road surface area and the area other than the road surface on the captured image. Moreover, you may use the restrictions by road model, such as a clothoid curve. For example, a line segment map 54, a road surface probability map 56, and the like are integrated to extract a line segment having a certain probability of a road surface boundary of a predetermined value or more, and the degree of coincidence with the road structure model is the highest among the extracted line segments. The line segment can be estimated as the final road boundary.

  Next, a road boundary estimation processing routine in the fourth embodiment will be described with reference to FIG. In addition, about the process similar to the road surface boundary estimation process routine in 1st and 3rd embodiment, the same code | symbol is attached | subjected and detailed description is abbreviate | omitted.

  The solid object map 450 and the line segment map 54 are accumulated through steps 100 to 108 and 300. In step 102, the three-dimensional object region extraction unit 420 generates a three-dimensional object map 450 in which each cell holds the height information of the observation value of the laser radar 12 and a predetermined occupation value. Further, in step 108, the line segment selection unit 426 takes into account the weight according to the distance between the three-dimensional object region and the edge line segment, and the similarity between the vehicle motion vector and the edge line segment, for example, using equation (2). The evaluated value is calculated and an edge line segment is selected.

  Next, in step 400, the road surface probability calculation unit 32 stores the road surface probability calculated based on the height information and the occupancy value held in each cell of the three-dimensional object map 450 generated in step 102 in each cell. The generated road surface probability map 56 is generated.

  Next, in step 402, the road surface boundary estimation unit 428 integrates the line segment map 54, the road surface probability map 56, and other information, and estimates the road surface boundary by dynamic programming or the like. Thus, control is performed so that the estimation result is combined with the captured image and displayed on the display device 18, and the road surface boundary estimation processing is terminated.

  As described above, according to the road surface boundary estimation device 410 according to the fourth embodiment, the weight according to the distance between the three-dimensional object region and the edge line segment, and the similarity between the own vehicle motion vector and the edge line segment. Since the edge line segments taking the degree into consideration are selected and accumulated, the line segment map 54 with higher accuracy can be generated. Further, in addition to the line segment map 54, information useful for estimating the road surface boundary such as the road surface probability map 56, dynamic background information, texture information, and restrictions by the road structure model is integrated to estimate the road surface boundary. The road boundary can be estimated.

  Note that it is not necessary to use all of the dynamic background information, texture information, restrictions on the runway model, and the like, and any one or two of them may be selected and used. Further, other information may be further added and used.

  Also in the first and second embodiments, an edge line segment may be selected using an evaluation value as described in the fourth embodiment.

  Also in the third and fourth embodiments, as in the first and second embodiments, the solid object boundary is estimated, and the road surface boundary is estimated by integrating the solid object boundary and the line segment map. You may make it do.

  Furthermore, the integration method of the solid object boundary and the selected edge line segment is not limited to the method of extracting the support points as described above. For example, as shown in FIG. 25A, a likelihood map is generated by mapping the estimated solid object boundary and the selected edge line segment on the same XZ plane. Here, the likelihood is an index representing the likelihood of a road surface boundary at an arbitrary position (x, z). As shown in FIG. 6B, a likelihood map is generated such that the likelihood is large at the position of the extracted solid object boundary or edge line segment, and the likelihood is small as it is away from it. Note that FIG. 4B shows that the darker the region, the higher the likelihood of representing the road surface boundary. In addition, how much the likelihood decreases at what distance is a parameter at the time of generating the likelihood map and is experimentally determined. However, it may be changed according to the distance from the host vehicle and the type of boundary candidate. Good. For example, the likelihood can be set so that the likelihood decreases sharply as the distance from the boundary candidate increases in the short distance region, and gradually decreases in the far region. In addition, the likelihood map may be generated as a map representing the likelihood distribution in the two-dimensional plane, or only the likelihood change in the direction perpendicular to the course is generated at every predetermined distance interval in front of the host vehicle. Also good.

Then, as shown in FIG. 5C, a line indicating the road boundary by changing various parameters of the road model (for example, parameters C ₁ , C ₂ , C ₃ , d ₁ , d _{2 in} equation (2)). A segment is generated, a value obtained by accumulating the likelihood along the generated line segment is calculated as an evaluation value, and a line segment having the highest evaluation value is determined as a final road surface boundary. FIG. 4C is a schematic diagram when the parameters of the polygonal line model are adapted. The final road boundary will (as much as possible) pass through a place with high likelihood as shown.

  Further, in the above-described embodiment, a case has been described in which information for identifying the positions of a plurality of reflection points is observed from the periphery of a vehicle using a laser radar. However, the present invention is not limited to this, and electromagnetic waves such as millimeter waves are used. Also good. Further, for example, information specifying the positions of a plurality of points around the vehicle may be calculated by post-processing using an image captured by a stereo camera.

  Moreover, although the said embodiment demonstrated the case where an estimation result was synthesize | combined with a captured image and displayed on a display apparatus, you may make it display a road surface boundary by a point coordinate on a grid map. In addition, since the road boundary indicates the range that the vehicle should not deviate from, the system that alerts the driver when the vehicle is likely to deviate, or automatically steer to control to leave the road boundary You may make it utilize the estimation result of a road surface boundary as input, such as a system.

  In the above description, the case where the moving body is a vehicle has been described. However, the present invention is not limited to this, and the present invention can also be applied to other moving bodies such as a robot.

  Moreover, although the example which mounted the road surface boundary estimation apparatus of this invention in the vehicle was demonstrated above, the computer which comprises a road surface boundary estimation apparatus can also be comprised as an external server. In this case, the captured image acquired by the vehicle and the observation data of the laser radar are transmitted to the external server by the communication means, and the estimation result of the road surface boundary processed by the external server is returned to the information terminal mounted on the vehicle. That's fine.

10, 210, 310, 410 Road surface boundary estimation device 12 Laser radar 14 Imaging device 16, 216, 316, 416 Computer 18 Display device 20, 420 Solid object region extraction unit 21 Road surface region extraction unit 22 Solid object boundary estimation unit 24 Edge line Segment extraction unit 26, 226, 326, 426 Line segment selection unit 28, 228, 328, 428 Road boundary estimation unit 32 Road surface probability calculation unit 50, 250, 450 Solid object map 52 Road model 54 Line segment map 56 Road surface probability map

Claims (14)

A three-dimensional position of each of a plurality of points on an object existing around the moving body with reference to the moving body and an image obtained by imaging the periphery of the moving body by an imaging unit mounted on the moving body are acquired. Acquisition means;
A three-dimensional object region extracting unit that extracts a three-dimensional object region around the moving body based on the three-dimensional position acquired by the acquiring unit;
Edge line segment extraction means for extracting edge line segments from the image acquired by the acquisition means;
A line segment selection means for selecting an edge line segment having a predetermined distance or less from the edge line segment extracted by the edge line segment extraction means and the solid object area extracted by the solid object area extraction means;
Road surface boundary estimation means for estimating a road surface boundary based on the edge line segment selected by the line segment selection means;
Road surface boundary estimation apparatus including   The road surface boundary estimation unit estimates the road surface boundary by integrating a boundary point between a road surface and a solid object indicated by an outer edge or a lower end of the solid object region and the selected edge line segment. Road surface boundary estimation device.   The road surface boundary estimation apparatus according to claim 1, wherein the three-dimensional object region extracting unit extracts the three-dimensional object region based on a plurality of three-dimensional positions acquired in time series by the acquiring unit. Road surface area extraction means for extracting a road surface area around the moving body based on the three-dimensional position acquired by the acquisition means;
The road surface boundary estimation apparatus according to any one of claims 1 to 3, wherein the line segment selection unit excludes edge line segments included in predetermined regions of the three-dimensional object region and the road surface region.   5. The road surface boundary estimation device according to claim 4, wherein the road surface area extraction unit extracts the road surface area based on a plurality of three-dimensional positions acquired in time series by the acquisition unit. Based on the three-dimensional object region, including road surface probability calculating means for calculating a road surface probability indicating road surface likelihood at each point around the moving body,
The road surface boundary estimation unit estimates the road surface boundary by integrating the road surface probability calculated by the road surface probability calculation unit and the selected edge line segment. The road surface boundary estimation apparatus described.   The road surface boundary estimation unit adds at least one of texture information obtained from the image acquired by the acquisition unit, dynamic background information based on a change in a pixel value of the image, and a constraint based on a road surface structure, The road surface boundary estimation apparatus according to claim 1, wherein the road surface boundary is estimated.   The road surface boundary estimation device according to any one of claims 1 to 7, wherein the line segment selection unit changes the predetermined value according to a distance between the moving body and a position corresponding to the edge line segment. . The line segment selection means accumulates the selected edge line segment in a line segment map in which the edge line segments selected in the past are accumulated,
The road surface boundary estimation device according to any one of claims 1 to 8 , wherein the road surface boundary estimation unit estimates the road surface boundary based on the line segment map. The road surface boundary estimation device according to claim 9 , wherein the road surface boundary estimation unit estimates the road surface boundary by thinning edge line segments accumulated in the line segment map. The line segment selection means calculates an evaluation value that increases as the distance between the solid object region and the edge line segment decreases, compares the evaluation value with a predetermined threshold value, and stores the evaluation value in the line segment map. The road boundary estimation apparatus according to claim 9 or 10 , wherein an edge line segment to be selected is selected. 12. The road surface boundary estimation device according to claim 11 , wherein the line segment selection unit calculates the evaluation value by multiplying a coefficient that increases as the similarity between the moving direction vector of the moving object and the edge line segment increases. The road surface boundary estimation device according to claim 11 or 12 , wherein the line segment selection unit adds a weight based on at least one of a distance and a similarity between a boundary line based on a road surface structure and the edge line segment to the evaluation value.   The road surface boundary estimation program for functioning a computer as each means which comprises the road surface boundary estimation apparatus of any one of Claims 1-13.