JP6837262B2 - Travelable area detection device and travel support system - Google Patents

Description

The present invention relates to a travelable area detection device and a travel support system.

Conventionally, based on information obtained from cameras and radars mounted on vehicles such as automobiles that are moving objects, white lines and roadsides (sidewalks, roadside belts, curbs, etc.) on the road (roadway) on which the vehicle travels A system has been developed that detects the boundary of the vehicle and uses the detection result to assist the driver in driving.

Patent Document 1 discloses an in-vehicle system in which a roadside break is detected by an in-vehicle camera that images the periphery of the vehicle, and the target section is set as a travelable area from the edge information of the break section.

Japanese Unexamined Patent Publication No. 2016-18371

However, although the technique disclosed in Patent Document 1 detects roadside breaks, it is possible to calculate the distance of the roadside breaks and whether or not the region opposite to the roadway in the roadside breaks can travel. There is a problem that it is difficult to detect a highly reliable travelable area.

Therefore, a travelable area detection device capable of detecting a break in the roadside in front of the moving vehicle and determining whether or not there is a travelable region in which the vehicle can travel from the break. And a driving support system using it has been desired.

The present invention has been made in view of the above problems of the prior art, and an object of the present invention is to detect a roadside break in front of a moving body, and the moving body travels from the detected roadside break. It is to realize a travelable area detection device capable of determining whether or not a travelable area exists and a travel support system using the device.

In order to solve the above-mentioned problems, the present invention is configured as follows.

The travelable area detection device detects the roadside existing on the road on which the moving body travels based on the captured image, and outputs the break section of the roadside when it is determined that the roadside break exists. Based on the roadside detection unit, the search range determination unit that determines the search range based on the roadside interruption section detected by the roadside detection unit, and the search range determined by the search range determination unit. From the road surface gradient estimation unit that estimates the road surface gradient of the roadside runnability determination region when viewed from the moving body and at least the road surface gradient estimated by the road surface gradient estimation unit, the travelable determination region is determined by the moving body. It is provided with a travelable area determination unit for determining whether or not the vehicle is a travelable area.

Further, in the traveling support system, the roadside existing on the road on which the moving body travels is detected based on the imaging device that images the front of the moving body and the image captured by the imaging device, and the roadside of the moving body is detected . When it is determined that there is a break , the roadside detection unit that outputs the break section of the roadside and the search range determination unit that determines the search range based on the break section of the roadside detected by the roadside detection unit. Based on the search range determined by the search range determination unit, the road surface gradient estimation unit estimates the road surface gradient of the travelability determination region on the roadside side as viewed from the moving body, and at least the road surface gradient estimation unit estimates. A travelable area detection device having a travelable area determination unit for determining whether or not the travelable determination area is a travelable area based on the road surface gradient, and a travelable area detection device. A traveling support control device that controls the operation of the moving body according to the determination result to support traveling is provided.

According to the present invention, it is possible to detect a break in the roadside in front of the moving body and determine whether or not there is a travelable area in which the moving body can travel from the detected break in the roadside. It is possible to realize a travelable area detection device and a travel support system using the device.

It is a schematic block diagram of the travel support system to which the travelable area detection device which concerns on one Embodiment of this invention is applied. It is a flowchart of the travelable area determination process. It is a bird's-eye view which shows the roadside point group detected by the travelable area detection device. It is a figure which shows the roadside point group and the search range detected by the travelable area detection device.

Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

One embodiment of the present invention is an example when applied to a vehicle as a moving body.

FIG. 1 is a schematic configuration diagram of a travel support system to which a travelable area detection device according to an embodiment of the present invention is applied.

In FIG. 1, the traveling support system is mounted on, for example, a vehicle V (shown in FIG. 3) such as an automobile, and mainly from a plurality of (two in this embodiment) cameras that image the front of the vehicle V. A travelable area detection device 1 that detects a travelable area around the vehicle V from a plurality of images simultaneously captured by each camera of the stereo camera device (imaging device) 2 and the stereo camera device 2. Based on the detection result of the possible area detection device 1, various control devices (for example, accelerator control unit 31, brake control unit 32, speaker control unit 33, steering control unit 34, etc.) mounted on the vehicle V are controlled to control the corresponding vehicle. It is provided with a travel support control device 3 that controls the operation of V to support travel.

The stereo camera device 2 is installed in front of the upper part of the windshield in front of the vehicle V, for example, toward the front of the vehicle V, and serves as a pair of imaging units that capture an image of the front of the vehicle V to acquire image information. The left camera (left image acquisition unit) 10 and the right camera (right image acquisition unit) 11 are provided.

The left camera 10 and the right camera 11 each have an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and each has a vehicle's left-right direction (direction orthogonal to the vehicle's longitudinal direction). It is installed so as to image the front of the vehicle V from a position separated from each other.

The travelable area detection device 1 is a device that detects a travelable area based on the image information of the image target area in front of the vehicle V acquired in time series by the stereo camera device 2.

The travelable area detection device 1 detects a travelable area from a break in the road edge, which is a boundary with a sidewalk, a roadside zone, a curb, etc. on the side of the travel road, and outputs the detection result to the travel support control device 3. To do.

The travelable area detection device 1 controls a stereo camera device 2 (for example, controls the imaging timing and exposure amount of each camera), a RAM (Random Access Memory) which is a temporary storage area, and a RAM (Random Access Memory). ROM (Read Only Memory) that stores programs and various initial values, external IF (Interface) image processing LSI (Large Camera Integration) that outputs CPU (Central Processing Unit) recognition information that controls the entire system, etc. to the outside of the system. Each component is communicably connected via a bus.

As shown in FIG. 1, the travelable area detection device 1 includes a parallax image generation unit 12, a roadside detection unit 13, a search range determination unit 14, a road surface gradient estimation unit 15, and a travelable area determination unit 16. It has.

The left image acquisition unit 10 and the right image acquisition unit 11 of the stereo camera device 2 acquire the captured left image and right image, respectively, and transmit the acquired left image and right image to the parallax image generation unit 12.

The travel support control device 3 operates the speaker control unit 33 of the vehicle V based on the detection result (determination result) received from the travelable area detection device 1, and the accelerator control unit 3 for supporting the travel of the vehicle V. The control amount of 31, the control amount of the brake control unit 32, and the control amount of the steering control unit 34 are calculated to adjust the accelerator, brake, steering, and the like.

In the following, a case where the travelable area detection device 1 detects a travelable area existing in front of the vehicle V will be specifically described.

There is a change in height at the breaks in the roadsides (sidewalks, curbs, medians, etc.) on the vehicle's lane (for example, changes in roadside height and slope), so the height of the breaks. By using the change and the change in the slope of the road surface in the area opposite to the traveling lane in the interrupted section, it is possible to determine whether or not there is a travelable area in which the vehicle can travel from the interrupted section. It is thought that it can be done.

Next, the travelable area determination process executed by the travelable area detection device 1 will be described with reference to FIGS. 2, 3, and 4. FIG. 2 is a flowchart of the travelable area determination process, and FIG. 3 is a bird's-eye view showing a group of road end points detected by the travelable area detection device 1. Further, FIG. 4 is a diagram showing a group of road end points detected by the travelable area detection device 1 and a search range. The travelable area determination process will be described with reference to the flowchart shown in FIG. This process is started, for example, when the vehicle starts running.

Note that step S1 shown in FIG. 2 is a process of the parallax image generation unit 12, steps S2, S3, and S4 are processes of the roadside detection unit 13, and steps S5 and S6 are processes of the search range determination unit 14. is there.

Further, step S7 is a process of the road surface gradient estimation unit 15, and steps S8 and S9 are processes of the travelable area determination unit 16.

In step S1 of FIG. 2, a parallax image is generated using the left image and the right image acquired by the left image acquisition unit 10 and the right image acquisition unit 11, and the generated parallax image is processed in step S2.

Specifically, the parallax (positional deviation between images) is calculated by block matching processing based on the left image and the right image acquired by the left image acquisition unit 10 and the right image acquisition unit 11, and the calculated parallax is calculated. Based on this, a parallax image including three-dimensional distance information (spatial information) is generated.

More specifically, in step S1, in the block matching process, one image (for example, the left image) is divided by a first block having a predetermined shape so as to include a plurality of pixels, and the other image (for example, the right image) is separated. , Separated by the second block of the same size, shape and position as the first block.

Then, the second block is shifted laterally one pixel at a time to calculate the correlation values for the two luminance patterns in the first block and the second block at each position, and the correlation value is the smallest, that is, the correlation. Searches for the highest position (corresponding point search).

As a method for calculating the correlation value, for example, SAD (Sum of Absolute Difference), SSD (Sum of Squared Difference), NCC (Normalized Cross Correlation), gradient method and the like can be used.

In the case of search, when the position where the correlation is highest is specified, the distance between the specific pixel in the first block and the specific pixel in the second block at the specified position is calculated as parallax. A parallax image can be generated by executing the same step for all pixels with this as one step.

In step S2, the parallax image acquired in step S1 is used to extract roadside parts that can be candidates for roadsides (for example, sidewalks / curbs, guardrails, continuous poles, walls, etc.) on the road, and the extracted roadsides. The position information of is processed by step S3.

More specifically, step S2 will be described. In the parallax image acquired in step S1, the distance change in the vicinity of the object that is the roadside candidate is smaller than the distance change in the flat part, and the roadside candidate portion is used. Detect the feature points of. For example, the image data is scanned in the vertical direction to detect a step that can be regarded as a roadside object.

Next, in step S3, the detected roadside is distributed to the left side roadside and the right side roadside of the vehicle V.
Further, as shown in FIG. 3, the roadside point group 100 other than the roadside objects such as other vehicles and pedestrians (the illustrated example is the case of a vehicle) is excluded, and the roadside point group used for searching the travelable area is used. Only 101 is extracted, and only the extracted roadside information (output from the roadside detection unit 13) is processed in step S4.

Here, the roadside used for searching the travelable area is a boundary between the traveling lane and the roadside, excluding obstacles (for example, other vehicles) on the traveling lane of the vehicle V.

Next, in step S4, if there is a roadside break at the extracted roadside and it is determined that the roadside break is a candidate for the travelable area of the vehicle V, the extracted roadside position information is obtained. Processed in step S5. In the determination of the travelable area candidate of the vehicle V, when the distance between the roadside end point 103 and the roadside start point 102 of FIG. 3 is equal to or larger than the width of the vehicle V, the vehicle V is selected as the travelable area candidate.

In step S5, from the roadside break section (between the point 102 and the point 103) detected in step S4, the processing area used for estimating the road surface gradient, that is, the search range (the area range on the roadside side when viewed from the vehicle). ) Is determined, and the coordinates of the determined search range are processed in step S6.

Specifically, as shown in FIG. 4, a straight line 400 from the start point 102 of the break section of the roadside to the break end point 103, and the points 102 and 103 are drawn horizontally on the opposite side of the road. The search range is a rectangular region 600 formed by the straight lines 200 and 300 and the straight line 500 at the edge of the screen. The setting of the road surface gradient estimation filter size described in step S5 of FIG. 2 means the setting of the rectangular region 600.

Next, in step S6, if a shield (tree, guardrail, etc.) is detected within the range of the rectangular area 600 determined in step S5 and the slope of the road surface cannot be estimated, the road surface in step S17. The information in step S5 is processed in step S7 only when it is determined that the premises cannot be estimated and the road surface gradient can be estimated.

In step S7, the slope of the road surface of the rectangular area 600, which is the search area determined in step S5, is estimated, and the estimated slope information is processed in step S8.

In the specific estimation procedure in step S7, the rectangular region 600, which is a search region (travelable determination region), is scanned in the horizontal direction to acquire road surface height information. As one step, the height information of the road surface is acquired from the near side (V side of the own vehicle with respect to the interruption of the road edge), and the slope of the road surface in the rectangular region 600 is estimated.

In step S8, it is determined from the change in the slope of the road surface estimated in step S7 whether the road surface slope is a slope within the travelable range of the vehicle V, and if it is determined that the vehicle cannot travel, the vehicle cannot travel. The region and step S18 determine. Then, the information in step S7 is processed in step S9 only when it is determined that the rectangular region 600 can travel.

In step S9, when the vehicle V enters the roadside break section from the roadway, there is an area where the vehicle V can stop without protruding into the traveling lane, that is, an obstacle (wall, step, etc.). It is determined whether or not there is a hole, whether or not the vehicle can travel in the area.

If it is determined that an object that becomes an obstacle exists, a hole exists, or the area is not the area in which the own vehicle can travel, it is determined in step S18 that the area cannot be traveled.

Further, in step S8, when it is determined that there is no obstacle, no hole, and the area where the own vehicle can travel, the travelable area is determined in step S19, and the determination result is obtained. Is transmitted to the traveling support device 3.

Then, the travel support control device 3 determines that the road surface gradient cannot be estimated in step S17, determines the non-travelable area in step S18, and steps based on the shielding determination in step S6 and the determination results in steps S8 and S9. Based on the result of the travelable area determination in S19, the vehicle V is provided with travel support.

As described above, in the travelable area detection device according to the embodiment of the present invention, the position of the road edge of the travel area is detected from the parallax images generated based on the plurality of images acquired by the stereo camera device 2. Then, based on the detected position information of the roadside, the change in the height direction of the break between the roadsides 102 and 103 and the gradient change of the region extending from the break in the roadside to the opposite side of the traveling lane are estimated. This makes it possible to determine whether or not the roadside break is in the travelable area of the own vehicle V.

As a result, for example, even when a break in the roadside of the own vehicle V is detected, the travelable area can be estimated from the break in the roadside, and high-precision driving support for automatic driving is provided. can do.

That is, the travelable area detection capable of detecting the break in the roadside in front of the moving body and determining whether or not there is a travelable area in which the moving body can travel from the detected roadside break. It is possible to realize an apparatus and a traveling support system using the apparatus.

The above-mentioned example is an example in which the travelable area detection device of the present invention is mounted on a vehicle, but the present invention is applicable not only to the vehicle but also to other moving objects. For example, the present invention can be applied to a traveling robot (rescue robot) used for disasters and the like.

Further, in the above-described example, an image pickup element such as a CCD or CMOS is used as an example of the stereo camera device, but a laser radar can also be used as the image pickup device.

Further, in the above example, the parallax image generation unit 12 generates a parallax image and detects the roadside object, but the roadside object can be detected by an image recognition process other than the parallax image. Is.

1 ... Travelable area detection device, 2 ... Stereo camera device, 3 ... Travel support control device, 10 ... Left image acquisition unit, 11 ... Right image acquisition unit, 12 ... Parallax Image generation unit, 13 ... Roadside detection unit, 14 ... Search range determination unit, 15 ... Road surface gradient estimation unit, 16 ... Travelable area determination unit, 31 ... Accelerator control unit, 32 ... Brake control unit, 33 ... Speaker control unit, 34 ... Steering control unit, 100 ... Roadside point group other than roadside objects such as other vehicles and pedestrians, 101 ... Roadside point group, 102 ... Start point of the break section, 103 ... End point of the break section, 200, 300 ... Straight line drawn in the horizontal direction, 400 ... Straight line to the break end point, 500 ... Screen edge Straight line, 600 ... Rectangular area (search area)

Claims (12)

When the roadside existing on the road on which the moving body travels is detected based on the captured image and it is determined that the roadside break exists, the roadside detection unit that outputs the break section of the roadside and the roadside detection unit
A search range determination unit that determines a search range based on the break section of the roadside detected by the roadside detection unit, and a search range determination unit.
Based on the search range determined by the search range determination unit, the road surface gradient estimation unit that estimates the road surface gradient of the travelability determination region on the roadside side when viewed from the moving body, and the road surface gradient estimation unit.
From the road surface gradient estimated by at least the road surface gradient estimation unit, the travelable determination area includes a travelable area determination unit that determines whether or not the moving body can travel.
A travelable area detection device comprising. In the travelable area detection device according to claim 1,
A parallax image generation unit that generates a parallax image from a plurality of images is further provided, and the roadside detection unit detects a roadside existing on the road based on the parallax image generated by the parallax image generation unit. A travelable area detection device characterized by. In the travelable area detection device according to claim 1,
The travelable area detection device is characterized in that when the detected road surface gradient is larger than a predetermined value, the travelable area determination unit determines that the moving body cannot travel in the travelable area determination area. .. In the travelable area detection device according to claim 1,
The travelable area determination unit determines whether there is an obstacle, a hole, or an area in which the moving body can travel in the travelable determination area, and the obstacle exists. Or, when it is determined that there is a hole or the moving body is not in a travelable area, the moving body determines that the travelable determination area cannot be traveled. .. In the travelable area detection device according to claim 4,
In the travelable area determination unit, the detected road surface gradient is equal to or less than a predetermined value, and there are no obstacles, no holes, and the moving body in the travelable area determination area. In the case of a travelable area, the travelable area detection device is characterized in that the moving body determines that the travelability determination area is travelable. In the travelable area detection device according to any one of claims 1, 2, 3, 4, and 5.
A travelable area detection device characterized in that the moving body is a vehicle. An imaging device that images the front of a moving object,
Based on the captured image captured by the imaging device, the road edge existing on the road on which the moving body travels is detected, and when it is determined that the road edge break exists, the road that outputs the break section of the road edge. Movement based on the end detection unit, the search range determination unit that determines the search range based on the roadside break section detected by the roadside detection unit, and the search range determined by the search range determination unit. From the road surface gradient estimation unit that estimates the road surface gradient of the roadside runnability determination region when viewed from the body and at least the road surface gradient estimated by the road surface gradient estimation unit, the moving body travels in the travelability determination region. A travelable area detection device having a travelable area determination unit for determining whether or not the area is possible, and a travelable area detection device.
A travel support control device that controls the operation of the moving body according to the determination result of the travelable area detection device to support the travel, and a travel support control device.
A driving support system characterized by being equipped with. In the driving support system according to claim 7,
The travelable area detection device includes a parallax image generation unit that generates a parallax image from a plurality of images, and the roadside detection unit exists on the road based on the parallax image generated by the parallax image generation unit. A driving support control system characterized by detecting the roadside. In the driving support system according to claim 7,
The travelable area determination unit is a travel support system characterized in that when the detected road surface gradient is larger than a predetermined value, the travelable area determination area determines that the moving body cannot travel. In the driving support system set according to claim 7,
The travelable area determination unit determines whether there is an obstacle, a hole, or an area in which the moving body can travel in the travelable determination area, and the obstacle exists. A travel support system, characterized in that, when it is determined that a hole exists or the moving body is not in a travelable area, the moving body determines that the travelable determination area cannot be traveled. In the driving support system according to claim 10,
In the travelable area determination unit, the detected road surface gradient is equal to or less than a predetermined value, and there are no obstacles, no holes, and the moving body in the travelable area determination area. A traveling support system characterized in that the moving body determines that the traveling body is capable of traveling in the travelable area when the vehicle is in a travelable area. In the driving support system according to any one of claims 7, 8, 9, 10, and 11.
A traveling support system characterized in that the moving body is a vehicle.

Images