JP2023011400A - Lane detection apparatus, lane detection method, and lane detection program - Google Patents

Abstract

To improve accuracy of detecting a lane.SOLUTION: A lane detection apparatus includes a generation unit, a first integration processing unit, a second integration processing unit, and a detection unit. The generation unit generates an edge image from an image of a road surface captured by an in-vehicle camera. The first integration processing unit integrates, based on the edge image generated by the generation unit, line segments in the same dashed line constituting a lane, on the basis of a first integration condition based on a distance between the line segments and a difference of angle between the line segments. The second integration processing unit integrates, based on a result integrated by the first integration processing unit, the line segment between the dashed line constituting the lane on the basis of a second integration condition in which the distance between the line segments is longer and the difference of angle is smaller than those in the first integration condition. The detection unit detects a lane on the basis of a result integrated by the second integration processing unit.SELECTED DRAWING: Figure 1

Description

The present invention relates to a lane detection device, a lane detection method, and a lane detection program.

2. Description of the Related Art Conventionally, for example, there is a technique for detecting a lane in which a vehicle travels from an image captured by an in-vehicle camera. For example, in the technology for detecting lanes, lanes are detected by connecting a plurality of pixels having characteristics of lanes (see, for example, Patent Document 1).

JP 2015-090584 A

However, in the conventional technology, there is a possibility that the accuracy of lane detection is lowered when the lane is interrupted, for example, when the lane is blurred.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a lane detection device, a lane detection method, and a lane detection program capable of improving lane detection accuracy.

In order to solve the above-described problems and achieve the object, the lane detection device according to the present invention includes a generation section, a first integration processing section, a second integration processing section, and a detection section. The generation unit generates an edge image from a captured image of a road surface captured by an in-vehicle camera. Based on the edge image generated by the generation unit, the first integration processing unit determines line segments within the same dashed line that constitutes a lane under a first integration condition based on a distance between the line segments and an angle difference between the line segments. Integrate based on The second integration processing unit determines, based on the result of integration by the first integration processing unit, the line segments between the dashed lines that form the lanes so that the distance between the line segments is longer than the first integration condition, and the angle difference is is integrated based on the second integration condition where is small. The detection unit detects lanes based on the result of integration by the second integration processing unit.

ADVANTAGE OF THE INVENTION According to this invention, the detection accuracy of a lane can be improved.

FIG. 1 is a diagram showing an outline of a lane detection method. FIG. 2 is a block diagram of the lane detection device. FIG. 3A is a diagram illustrating an example of first integration conditions. FIG. 3B is a diagram illustrating an example of a first integration condition; FIG. 3C is a diagram illustrating an example of a second integration condition; FIG. 4 is a diagram showing an example of a predicted trajectory. FIG. 5 is a diagram showing an example of the peripheral area. FIG. 6 is a flow chart showing a processing procedure executed by the lane detection device.

Hereinafter, embodiments of a lane detection device, a lane detection method, and a lane detection program disclosed in the present application will be described in detail with reference to the accompanying drawings. In addition, this invention is not limited by embodiment shown below.

First, the outline of the lane detection method according to the embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an outline of a lane detection method. This lane detection method is executed by the lane detection device 10, which will be described later with reference to FIG. In addition, below, the case where the white line which comprises a lane is detected is demonstrated. Note that the target to be detected as a lane is not limited to the white line, and may be, for example, a “yellow line” or a “multiple line”.

A lane detection method according to an embodiment detects lane markings from an image captured by an in-vehicle camera. As shown in FIG. 1, in the lane detection method, an edge image is generated from a captured image G of a road surface captured by an in-vehicle camera (step S1). For example, in the lane detection method, an edge image is generated based on the luminance gradient of the captured image G and the like. In the edge image, for example, a line segment L appears as a boundary line between the white line portion and the road surface portion.

Here, for example, a method of detecting the dashed lines as one lane by integrating the line segments L between the dashed lines can be considered. However, if the line segment L is divided by an element other than the dashed line, such as when the white line is rubbed, the line segment L between the dashed lines may not be merged well.

Therefore, in the lane detection method according to the embodiment, the segmented line segments L are corrected by integrating the line segments L within the dashed lines by the first integration process, and then the line segments L between the dashed lines are corrected by the second integration process. decided to integrate

Specifically, in the lane detection method, a first integration process is performed to integrate the line segments L included in the edge image based on the first integration condition (step S2). Here, the first integration process is a process of integrating line segments L within the same dashed line, for example, a process of integrating line segments L of a worn white line, a white line having a zebra zone, or a ribbed white line. . In addition, for white lines that do not have broken lines, that is, solid lines, the entire white line may be regarded as one broken line.

For example, the line segment L of the rubbed white line depends on the degree of rubbing in inclination, length, and position, and has no regularity. .

Also, a white line having a zebra zone and a line segment L of a rib-type white line are interrupted and appear in the edge image. Further, for example, line segments L of a white line having a zebra zone or a rib type white line have line segments L periodically.

Therefore, for example, the next line segment L is likely to exist on the extension of the line segment L. Therefore, in the first integration process, the line segments L within the same dashed line are integrated using these as the first integration conditions. In FIG. 1, the processing result of the first integration processing is indicated as a first integration line Li1.

Subsequently, in the lane detection method, a second integration process is performed to integrate the line segments L included in the edge image based on the second integration condition (step S3). The second integration process is a process of integrating line segments L between dashed lines forming the same white line as one line segment.

In other words, the dashed white line alternates between areas where the line segment L exists and areas where the line segment L does not exist with a predetermined regularity. Therefore, in the second integration process, each line segment L is integrated according to these regularities. In addition, in the second integration process, the first integration line Li1 is also included in the target of the integration process.

For example, the lane detection method integrates line segments L having a predetermined regularity with features between dashed lines. For example, in the second integration process, line segments L that periodically exist and have inclination deviations or angle differences equal to or less than a predetermined value are integrated. As a result, the line segments L between the dashed lines can be integrated, and can be integrated as one white line. In addition, in FIG. 1, the processing result of the second integration processing is shown as a second integration line Li2.

Thereafter, in the lane detection method, lanes are detected based on the processing results of the first integration process and the second integration process (step S4). For example, in the lane detection method, two white lines corresponding to both ends of the lane are detected, and the lane is detected based on the detection results of the two white lines.

Thus, in the lane detection method according to the embodiment, the line segments L within the dashed lines are integrated by the first integration process, and the line segments L between the dashed lines are integrated by the second integration process. Thereby, in the lane detection method according to the embodiment, it is possible to improve the accuracy of the integration processing in the second integration processing.

Therefore, according to the lane detection method according to the embodiment, since the line segments L between the broken lines can be appropriately integrated, the white line detection accuracy can be improved, and the lane detection accuracy can be improved. .

Next, a configuration example of the lane detection device according to the embodiment will be described with reference to FIG. 2 . FIG. 2 is a block diagram of the lane detection device. Note that FIG. 2 also shows an in-vehicle camera 51 in addition to the lane detection device 10 .

Vehicle-mounted camera 51 is, for example, a drive recorder that captures the traveling direction of the vehicle. For example, the vehicle-mounted camera 51 is composed of various imaging elements and a lens such as a fisheye lens.

As shown in FIG. 2 , lane detection device 10 includes storage unit 20 and control unit 30 . The storage unit 20 is configured by a storage device such as a nonvolatile memory, data flash, or hard disk drive, for example.

In the example shown in FIG. 2, the storage unit 20 has an integration condition storage unit 21 and a detection history storage unit 22, and also stores various programs.

The integration condition storage unit 21 is a storage area that stores various types of information regarding the first integration condition and the second integration condition. Specific examples of the first integrated condition and the second integrated condition will now be described with reference to FIGS. 3A to 3C.

3A and 3B are diagrams showing an example of the first integration condition. FIG. 3C is a diagram illustrating an example of a second integration condition; As shown in FIG. 3A, for example, under the first integration condition, a circular first search area A1 is set at the end of the line segment L. As shown in FIG. Then, the line segment L for which the first search area A1 is set and the line segment L existing within the first search area A1 are integrated.

Further, as shown in FIG. 3B, under the first integration condition, a second search area A2 is set at the end of each line segment L. As shown in FIG. For example, the second search area A2 is an arc-shaped area. The second search area A2 has a longer radius and a narrower central angle than the first search area A1.

Also, the direction of the second search area A2 is set such that the center is on the extension of the line segment L. FIG. Therefore, the second search area A2 narrows the angle more than the first search area A1, and searches for the line segment L existing farther.

In addition, as shown in FIG. 3C, the third search area A3 is set at the end of each line segment L under the second integration condition. For example, the second search area A2 is a search area for integrating line segments L existing within the same dashed line in a zebra zone or a rib-type white line, whereas the third search area A3 is a search area between dashed lines. This is a search area for integrating line segments L. FIG.

For example, the third search area A3 has an arc shape like the second search area A2, and has a longer radius and a narrower center angle than the second search area A2. Also, the direction of the third search area A3 is set so that the center thereof is on the extension line of the line segment L. FIG.

As a result, the angle of the third search area A3 is narrowed down, and the line segment L that exists farther is searched. This is because, under the second integration condition, a line segment L located farther than under the first integration condition is searched. This is because there is a possibility that the line segment L of .

That is, by narrowing the angular range of the third search area A3 under the second integration condition compared to the first search area A1 and the second search area A2 under the first integration condition, the line segment L of the same white line is appropriately integrated. can do.

In general, dashed lines as lanes have a certain length (eg, 5 m) and are present at predetermined intervals (eg, 5 m intervals). Therefore, the third search area A3 has an exclusion area Af, which is excluded from the integration target, in the center. That is, the line segment L existing in the exclusion area Af is excluded from the processing targets of the second integration processing. As a result, from the viewpoint of periodicity, it is possible to suppress the unification of line segments L that are clearly not line segments L between broken lines. Note that the exclusion area Af may be similarly set for the second search area A2.

Returning to the description of FIG. 2, the detection history storage unit 22 will be described. The detection history storage unit 22 is a storage area that stores lane detection history. For example, the second integration condition is set based on the detection history stored in the detection history storage unit 22. Details of this point will be described later with reference to FIG.

The control unit 30 includes a generation unit 31, a first integration processing unit 32, a second integration processing unit 33, and a detection unit 34, and includes, for example, a CPU (Central Processing Unit), ROM (Read Only Memory), RAM (random access memory), hard disk drives, and computers and various circuits with input/output ports.

The CPU of the computer functions as the generation unit 31, the first integration processing unit 32, the second integration processing unit 33, and the detection unit 34 of the control unit 30 by reading and executing programs stored in the ROM, for example.

In addition, at least one part or all of the generation unit 31, the first integration processing unit 32, the second integration processing unit 33, and the detection unit 34 of the control unit 30 is an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). ) or the like.

The generation unit 31 generates an edge image from the captured image of the road surface captured by the in-vehicle camera 51 . The generation unit 31 acquires a captured image from the vehicle-mounted camera 51 and extracts an edge image based on the luminance gradient of each pixel forming the captured image. Further, the generation unit 31 passes information about the edge image to the first integration processing unit 32 each time an edge image is generated.

Based on the edge image generated by the generation unit 31, the first integration processing unit 32 applies the line segment L within the same dashed line forming the white line to the first integration condition based on the distance between the line segments and the angle difference between the line segments. Integrate based on First, the first integration processing unit 32 extracts each line segment L from the edge image.

Subsequently, the first integration processing unit 32 integrates the line segment L by switching the first integration condition according to the type of detected white line. Here, the types of white lines are broadly classified into, for example, worn white lines, white lines with zebra zones, and ribbed white lines.

For example, the first integration processing unit 32 first integrates line segments L of rubbed white lines, and then integrates line segments L of zebra zones and rib-type white lines. As a result, even if white lines with zebra zones or ribbed white lines are rubbed, the line segments L can be appropriately integrated.

Specifically, first, the first integration processing unit 32 generates a first integrated line Li1 (see FIG. 1) by integrating line segments L existing in the first search area A1. After that, the first integration processing unit 32 sets a second search area A2 (see FIG. 3) for the line segment L (hereinafter simply referred to as the line segment L) including the first integration line Li1.

The first integration processing unit 32 integrates, among the line segments L existing in the second search area A2, the line segments L whose angular difference from the target line segment L is equal to or less than a predetermined value. Also, at this time, the line segments L may be integrated by further considering the difference in length from the target line segment L. FIG.

That is, since the line segments L of the zebra zone or rib type white line have regularity in length, by integrating the line segments L whose length difference is equal to or less than a predetermined value, the zebra zone or rib type white line segment L is integrated. Line segments L corresponding to the line segment L of the white line can be selected and integrated.

After completing the first integration processing based on the first search area A1 and the second search area A2, the first integration processing unit 32 superimposes the respective first integration lines Li1 on the edge image, and sends the image to the second integration processing unit 33. hand over.

Based on the edge image received from the first integration processing unit 32, the second integration processing unit 33 performs a second integration process for integrating the line segments L between the dashed lines. First, the second integration processing section 33 sets a third search area A3 for each line segment L. As shown in FIG.

Subsequently, the second integration processing unit 33 integrates the line segments L whose angular difference from the target line segment L is equal to or less than a predetermined value among the line segments L existing in the third search area A3. The second integration processing unit 33 superimposes the second integration line Li<b>2 as the integration result on the edge image received from the first integration processing unit 32 , and passes the result to the detection unit 34 .

Further, the second integration processing unit 33 may set the second integration condition based on the white line detection history stored in the detection history storage unit 22, for example. For example, the second integration processing unit 33 calculates the predicted trajectory of the white line following the white line based on the detection history of the white line up to the past frame.

FIG. 4 is a diagram showing an example of the predicted trajectory of the white line. The predicted trajectory O shown in FIG. 4 is calculated, for example, by obtaining the curvature of the detected white line based on the detection history of the white line. For example, the second integration processing unit 33 sets the orientation of the third search area A3 along the predicted trajectory O.

Thereby, for example, even if the curve has a large R value, the direction of the third search area A3 can be set along the white line, so the line segments L between the dashed lines can be appropriately integrated. The second integration processing unit 33 may set the length of the third search area A3 based on the white line detection history up to the past frame.

That is, since dashed lines usually have regularity in length, the length of the dashed lines and the distance between the dashed lines are calculated based on the detection history, and the length of the third search area A3 is set. may Even in this case, the line segments L between the dashed lines can be appropriately integrated.

By the way, in the captured image G, pixels in the white line portion tend to have higher luminance values than pixels in the road surface portion. Therefore, the second integration processing unit 33 may integrate the line segment L in consideration of the difference in the brightness values of the peripheral regions sandwiching the line segment L.

FIG. 5 is a diagram showing an example of the peripheral area. As shown in FIG. 5, for example, if the line segment L is the edge of the white line, it becomes the boundary between the white line portion and the road surface portion. Therefore, the second integration processing unit 33 integrates the line segment L based on the luminance value difference between the first peripheral region f1 located on the left side of the line segment L and the second peripheral region f2 located on the left side of the line segment L. do.

More specifically, if the difference in brightness value between the first peripheral region f1 and the second peripheral region f2 is equal to or greater than a predetermined value, that is, if the brightness difference is sufficient, the line segment L is set as the integration target, If the difference in luminance value between the first peripheral region f1 and the second peripheral region f2 is less than a predetermined value, the line segment L is not integrated.

In this way, by further considering the luminance value, it is possible to integrate only the line segments L corresponding to the characteristics of the white line, thereby improving the detection accuracy of the white line.

Returning to the description of FIG. 2, the detection unit 34 will be described. The detection unit 34 detects lanes based on the processing result of the first integration processing unit 32 and the processing result of the second integration processing unit 33 . For example, the detection unit 34 detects white lines based on the edge image in which the first integrated line Li1 and the second integrated line Li2 are superimposed. Then, the detection unit 34 detects the lane based on the detection result of the white line.

For example, the information on the white line detected by the detection unit 34 is output to, for example, an automatic driving system that controls automatic driving of the vehicle or a warning system that warns when the vehicle deviates from the lane.

Next, a processing procedure executed by the lane detection device 10 according to the embodiment will be described with reference to FIG. 6 . FIG. 6 is a flow chart showing a processing procedure executed by the lane detection device 10. As shown in FIG. Note that the processing procedure described below is repeatedly executed by the control unit 30 .

As shown in FIG. 6, first, when the lane detection device 10 acquires the captured image G (step S101), it generates an edge image from the captured image G (step S102). Subsequently, the lane detection device 10 performs a first integration process of integrating the line segments L within the same dashed line (step S103).

Subsequently, the lane detection device 10 performs a second integration process for integrating the line segments L between the dashed lines based on the processing results up to step S103 (step S104). Subsequently, the lane detection device 10 detects lanes based on the processing results up to the second integrated processing (step S105), and ends the processing.

As described above, the lane detection device 10 according to the embodiment includes the generation unit 31, the first integration processing unit 32, the second integration processing unit 33, and the detection unit . The generating unit 31 generates an edge image from the image G captured by the vehicle-mounted camera 51 of the road surface. are integrated based on a first integration condition based on the distance between the line segments L and the angle difference between the line segments L.

Based on the result of integration by the first integration processing unit 32, the second integration processing unit 33 determines the line segments L between the dashed lines that make up the lanes so that the distance between the line segments L is longer than the first integration condition. are integrated based on the second integration condition in which the angle difference between is small. The detection unit 34 detects lanes based on the result of integration by the second integration processing unit 33 . Therefore, according to the lane detection device 10 according to the embodiment, it is possible to improve the lane detection accuracy.

Further effects and modifications can be easily derived by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments so shown and described. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept defined by the appended claims and equivalents thereof.

10 lane detection device 21 integration condition storage unit 22 detection history storage unit 31 generation unit 32 first integration processing unit 33 second integration processing unit 34 detection unit 51 in-vehicle camera G captured image

Claims (6)

a generator that generates an edge image from an image captured by an in-vehicle camera of a road surface;
A first integration that integrates line segments within the same dashed line forming a lane based on the edge image generated by the generation unit based on a first integration condition based on a distance between the line segments and an angle difference between the line segments. a processing unit;
Based on the result of integration by the first integration processing unit, the line segments between the dashed lines forming the lane are subjected to a second integration in which the distance between the line segments is longer and the angle difference between the line segments is smaller than the first integration condition. a second integration processing unit that integrates based on conditions;
A lane detection device comprising: a detection unit that detects a lane based on a result of integration by the second integration processing unit. The first integration processing unit
The lane detection device according to claim 1, wherein the line segments are integrated by switching the first integration condition according to the type of the lane. The second integration processing unit
The lane detection device according to claim 1 or 2, wherein the second integrated condition is set based on a detection history of the lane detected by the detector. The second integration processing unit
4. The lane detection device according to claim 1, wherein the line segments are integrated based on a difference in luminance value of a peripheral area sandwiching the line segment. A generation step of generating an edge image from an image captured by an in-vehicle camera of a road surface;
A first integration that integrates line segments within the same dashed line forming a lane based on the edge image generated by the generation step, based on a first integration condition based on a distance between the line segments and an angle difference between the line segments. a processing step;
second integration in which the distance between the line segments is longer and the angle difference between the line segments is smaller than the first integration condition, based on the integration result of the first integration processing step; a second integration processing step of integrating based on conditions;
and a detection step of detecting a lane based on the integration result of the second integration processing step. a generation procedure for generating an edge image from a captured image of a road surface captured by an in-vehicle camera;
A first integration that integrates line segments within the same dashed line forming a lane based on the edge image generated by the generation procedure based on a first integration condition based on a distance between the line segments and an angle difference between the line segments. a processing procedure;
second integration in which the distance between the line segments is longer and the angle difference between the line segments is smaller than the first integration condition, based on the result of integration by the first integration processing procedure; a second integration procedure that integrates based on conditions;
A lane detection program for causing a computer to execute a detection procedure for detecting lanes based on the result of integration by the second integration processing procedure.

Images