WO2015105239A1

WO2015105239A1 - Vehicle and lane position detection system and method

Info

Publication number: WO2015105239A1
Application number: PCT/KR2014/003288
Authority: WO
Inventors: 김동신
Original assignee: 삼성테크윈 주식회사
Priority date: 2014-01-13
Filing date: 2014-04-16
Publication date: 2015-07-16
Also published as: KR20150084234A; KR102016549B1

Abstract

The present invention provides a vehicle and lane position detection system and method. A vehicle and lane position detection system of the present invention may comprise: a segment unit for segmenting a road image obtained by a sensor with super pixels; a classification unit for classifying the super pixels into a road pixel configuring a road area and an off-road area which is close to the road area; a boundary detection unit for generating a plurality of vanishing lines in a boundary area where the road pixel and the off-road pixel coexist; and a position estimation unit for generating a candidate road area by a pair of vanishing lines selected from the plurality of vanishing lines and estimating a driving position from a type of the candidate road area.

Description

Vehicle and Lane Position Detection System and Method

The present invention relates to a vehicle and lane position detection system and method.

Recently, with the development of information and communication technology, the lane departure warning system (LDWS), the safe driving assistance system such as the lane maintenance, and the vehicle automatic control system have been developed, and the practical use is rapidly progressing.

An embodiment of the present invention provides a system and method for detecting a relative position on a road of a vehicle and a lane that can assist autonomous driving or driving by estimating the position of a lane in progress and a relative vehicle position on a road. To provide.

Lane position detection system according to an embodiment of the present invention, the segment unit for segmenting the road image obtained by the sensor into super pixels; A classification unit classifying the superpixels into a road pixel constituting a road area and a non-road pixel constituting a non-road area adjacent to the road area; A boundary detector for generating a plurality of vanishing lines in a boundary region where the road pixel and the non-road pixel are mixed; And a position estimator for generating a candidate road region by a pair of vanishing line selected from the plurality of vanishing lines, and estimating a driving position from a shape of the candidate road region.

The classifier may classify the superpixel into road pixels or non-road pixels based on at least one of visual and spatial features of the superpixel.

The classification unit may classify the unclassified super pixel into road pixels or non-road pixels using spatial information between adjacent super pixels.

The boundary detector may select a plurality of particle pairs from particles distributed in correspondence with a probability distribution set in the boundary area, and generate a plurality of vanishing lines by the plurality of particle pairs.

The boundary detector may select the particle pair so that the vanishing line does not pass through the road area.

The location estimating unit may estimate the driving position and the driving lane based on the entire road width and the lane width.

The position estimating unit may set a candidate group from a plurality of driving positions estimated from a plurality of vanishing line pairs, and estimate the driving position and the driving lane based on the total road width and the lane width with respect to the candidate group.

The position estimating unit may estimate a position having the highest density among the candidate groups as a driving position, and estimate a driving lane corresponding to the set driving position.

Lane position detection method according to an embodiment of the present invention, segmenting the road image obtained by the sensor into super pixels; Classifying the superpixels into road pixels constituting a road area and non-road pixels constituting a non-road area adjacent to the road area; Generating a plurality of vanishing lines in a boundary region where the road pixel and the non-road pixel are mixed; And generating a candidate road region by a pair of vanishing line selected from the plurality of vanishing lines, and estimating a driving position from the shape of the candidate road region.

The classifying step may include classifying the super pixel as a road pixel or a non-road pixel based on at least one of a visual feature and a spatial feature of the super pixel.

The classifying step may include classifying an unclassified super pixel into a road pixel or a non-road pixel using spatial information between adjacent super pixels.

The generating of the candidate vanishing line may include distributing particles in the boundary region corresponding to a probability distribution set in the boundary region; And selecting a plurality of particle pairs from the distributed particles, and generating a plurality of vanishing lines by the plurality of particle pairs.

The particle pair may be selected such that the vanishing line does not pass through the roadway region.

The estimating driving position may include estimating the driving position and the driving lane based on the entire road width and the lane width.

The position estimating may include: setting a candidate group from a plurality of driving positions estimated from a plurality of vanishing line pairs; And estimating the driving position and the driving lane with respect to the candidate group based on the entire road width and the lane width.

The estimating driving lane may include estimating a position having the highest density of the candidate group as a driving position and estimating a driving lane corresponding to the set driving position.

According to an embodiment of the present invention, by estimating the relative position of a vehicle and a driving lane, an expensive DGPS (Differential Global Positioning System) system, which is used for estimating a vehicle position, may be replaced with a low cost system. Can help with lane changes.

1 is a block diagram schematically illustrating a vehicle and a lane detection system according to an exemplary embodiment of the present invention.

2 is a diagram illustrating a road image input from a sensor.

FIG. 3 is a diagram illustrating a segment image obtained by segmenting the road image of FIG. 2 into super pixels.

4 is an exemplary view showing a road area and a non-road area according to an embodiment of the present invention.

5 is a diagram schematically illustrating a result classified into road pixels and non-road pixels according to an exemplary embodiment of the present invention.

6 is an exemplary view schematically showing a boundary area between a road area and a non-road area according to an embodiment of the present invention.

7 is an exemplary view showing vanishing lines generated in a boundary area according to an exemplary embodiment of the present invention.

8 is an exemplary diagram illustrating a plurality of driving positions estimated from a plurality of vanishing line pairs according to an exemplary embodiment of the present invention.

FIG. 9 is an exemplary diagram illustrating a result of determining a relative driving position in a road image and a relative position of a driving lane in a converted image according to an exemplary embodiment of the present invention.

10 is a flowchart illustrating a method of detecting a position of a vehicle and a lane in driving according to an embodiment of the present invention.

11 is a flowchart illustrating a pixel classification method of FIG. 10.

FIG. 12 is a flowchart for describing a method for generating candidate vanishing lines in FIG. 10.

As the inventive concept allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

In the following embodiments, terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are only used to distinguish one component from another.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the following embodiments, terms such as "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, one or more It is to be understood that it does not exclude in advance the possibility of the presence or addition of other features or numbers, steps, operations, components, components or combinations thereof.

Embodiments of the present invention can be represented by functional block configurations and various processing steps. Such functional blocks may be implemented in various numbers of hardware or / and software configurations that perform particular functions. For example, embodiments of the invention may be implemented directly, such as memory, processing, logic, look-up table, etc., capable of executing various functions by the control of one or more microprocessors or other control devices. Circuit configurations can be employed. Similar to the components of an embodiment of the present invention may be implemented in software programming or software elements, embodiments of the present invention include various algorithms implemented in combinations of data structures, processes, routines or other programming constructs. It may be implemented in a programming or scripting language such as C, C ++, Java, assembler, or the like. The functional aspects may be implemented with an algorithm running on one or more processors. In addition, embodiments of the present invention may employ the prior art for electronic configuration, signal processing, and / or data processing. Terms such as mechanism, element, means, configuration can be used broadly and are not limited to mechanical and physical configurations. The term may include the meaning of a series of routines of software in conjunction with a processor or the like.

A road-based autonomous driving system is a system that guarantees safe driving by correcting its position using lane information or inducing lane maintenance or change by determining a driving lane using a direct localization sensor such as GPS. . However, if the lane is blurred or erased, it is difficult to recognize the lane with only lane information. Accurate navigation devices are also expensive and accumulate in errors, and do not compensate themselves for errors. And even if a separate error correction algorithm is used, it is not robust as an algorithm that is not based on MAP.

The vehicle and lane position detection system 1 according to an exemplary embodiment of the present invention may detect the position of a moving object such as a vehicle or a mobile robot by recognizing a road area of a road currently being driven and a non-road area adjacent to the road area. Therefore, the embodiment of the present invention can infer the position of the movable body or the camera mounted on the movable body without using expensive equipment such as GPS.

Hereinafter, a vehicle will be described as an example of a moving object, which can be applied to other moving bodies in the same manner. The vehicle and lane position detection system 1 estimates approximate distance information from the left and right boundary of the road area to the position of the sensor 2 or the center position of the vehicle equipped with the sensor 2, that is, the current driving position of the vehicle. can do.

The sensor 2 is an image recognition device attached to a driving vehicle, and may be, for example, a camera that acquires a surrounding image of the vehicle. The sensor 2 may be installed at the front, rear, or side of the vehicle, and may be selectively mounted at an appropriate position so as to obtain an image of the road optimally. The range of the surrounding image obtained by the sensor 2 may vary according to the type of the sensor 2 mounted on the vehicle, and the entire width of the road or the width of the currently driving lane may be displayed on the image.

However, embodiments of the present invention are not limited thereto, and various sensors, such as an infrared sensor, a radar (RADAR), or a lidar (LIDAR) sensor, in addition to the image recognition apparatus, may be used alone or in a fused form.

The vehicle and lane position detection system 1 may provide position information of the vehicle to the driver through the information providing device 3.

The information providing apparatus 3 is a display device that provides a two-dimensional or three-dimensional image of an image obtained from the sensor 2 and a driving lane detection result from the vehicle and lane position detection system 1, and the vehicle and lane position detection. The system 1 may be connected by wire or wirelessly. The information providing apparatus 3 may provide information to the driver in various ways such as an image, a text, and a voice, and a display method (for example, lane and non-lane guide for each color, etc.) set in a manner agreed with the user when providing an image. ) So that users can easily recognize it. The information providing apparatus 3 may convert and provide a driving lane detection result into a bird view form.

The range of the road provided by the information providing device 3 may vary depending on the range of the image acquired by the sensor 2, and may provide the entirety of the acquired image or may provide only the road in the direction in which the vehicle is currently traveling. have.

Referring to FIG. 1, the vehicle and lane position detection system 1 may include a segment unit 11, a classification unit 13, a boundary detector 15, and a position estimator 17.

The segment unit 11 receives a road image including road information from a sensor 2 connected to the vehicle and the lane detection system 1. The road information may include lane marking drawn on the road surface.

The segment unit 11 may segment a road image composed of a plurality of pixels into super pixels. The segment part 11 may divide the road image into uniform areas having similar characteristics. In this case, the divided uniform region is called a super pixel. Each superpixel may be represented by a color distribution and a texture distribution representing a visual feature, and a central mass position representing a spatial feature. The segment part 11 may store the visual and spatial features of each super pixel in the memory 4.

FIG. 2 is a diagram illustrating a road image input from the sensor 2, and FIG. 3 is a diagram illustrating a segment image obtained by segmenting the road image of FIG. 2 into super pixels.

The classification unit 13 may learn the road area and the non-road area through samples of the road area and the non-road area under various environments and situations such as weather, lighting, and lane damage. The classifier 13 may be updated to reflect the lane position detection result.

Here, the non-road area is an area adjacent to the road area, not a remaining area other than the road area in the road image, as shown in FIG. 4. The area adjacent to the road may have unique visual features. In the embodiment of the present invention, the road area can be detected more accurately by using the features of the road area and the non-road area together.

The classifying unit 13 may classify each super pixel into a road pixel and a non-road pixel based on the visual and spatial characteristics of the road area and the non-road area according to the learning result. The classifier 13 may include an ANN classifier, an adaboost classifier, a naive bayes classifier, or a support vector machine (SVM) for classifying super pixels. . Embodiments of the present invention are not limited to the classifier described above, and various classifiers or algorithms that can classify super pixels based on features may be applied to the super pixel classification.

The classifying unit 13 may classify unclassified super pixels that are not classified as road pixels or non-road pixels as road pixels or non-road pixels based on a correlation with adjacent super pixels.

Super pixels that are spatially adjacent to each other can be expected to belong to the same area. The classifier 13 may triangulate the superpixels to indicate a relationship between the superpixels. For example, the classifier 13 may apply the Delaunay triangulation technique. However, the present invention is not limited to this, and of course, other techniques capable of defining spatial relationships can be used.

The classifying unit 13 converts the unclassified super pixel into a road pixel or a non-pixel in consideration of spatial correlation of the super pixels based on, for example, a Markov random field (MRF) or a conditional random field (CRF). Can be classified as pixels. However, the present invention is not limited to this, and of course, other techniques capable of performing clustering may be used.

The classifying unit 13 is sequentially or in parallel by a first classifier for classifying road pixels based on the characteristic information of the road area and a second classifier for classifying non-road pixels based on the characteristic information for the non-road area. Superpixels can be classified as road pixels or non-road pixels. Alternatively, the classifying unit 13 may classify the superpixels into road pixels or non-road pixels by one classifier based on the feature information about the road area and the non-road area.

When one super pixel is classified into road pixels and non-road pixels, the classifying unit 13 may classify the road pixel or the non-road pixel arbitrarily according to a predetermined classification criterion.

5 is an example schematically showing a result classified into road pixels and non-road pixels. The output of the classification unit 13 is a result of the classification of the super pixels, and the boundary between the road area and the non-road area is not clearly distinguished, and there is an area where the road pixel and the non-road pixel are mixed.

The boundary detector 15 may set a region where the road pixel and the non-road pixel are mixed as the boundary region and generate a plurality of vanishing lines in the boundary region. 6 is an exemplary view schematically showing a boundary area between a road area and a non-road area.

The boundary detector 15 may generate and distribute particles in the boundary region. In this case, the probability distribution may be set such that the center of the boundary region has the highest probability and the probability decreases as the distance from the center increases. Particles in the boundary region may be distributed corresponding to the set probability distribution.

The boundary detector 15 may generate a vanishing line (VL) by selecting any particle pair among the distributed particles. The boundary detector 15 may select the particle pair so that the vanishing line connecting the particle pair does not pass the road area. 7 is an exemplary diagram illustrating vanishing lines generated in a boundary region.

The position estimator 17 may select any vanishing line pair from the plurality of vanishing lines. The vanishing line pair may correspond to the boundary line pair of the left road boundary and the right road boundary. A vanishing point is predicted by the vanishing line pair, and a candidate road region in which the vanishing line pair is a road boundary pair may be determined. The position estimating unit 17 may estimate the position of the sensor 2, that is, the driving position of the vehicle, from the shape of the candidate road area. 8 is an exemplary diagram illustrating a plurality of driving positions CP estimated from a plurality of vanishing line pairs.

The location estimating unit 17 extracts road information including the total width of the road and the lane width by using the road map database in the memory 4, and uses the extracted road information to display the currently traveling lane corresponding to the driving position. It is possible to estimate the relative position of, i.e., in which of the lanes.

The position estimating unit 17 may set a driving position candidate group including a plurality of driving positions excluding a driving position corresponding to noise from the plurality of driving positions estimated from the plurality of vanishing line pairs. Since the driving position away from the region where the driving position is concentrated is unlikely to be the actual driving position, it may be determined as noise and may be excluded. The position estimating unit 17 may determine the driving lane to which the driving position belongs to the driving position candidate group based on the entire road width and the lane width. As another example, the position estimating unit 17 may set the position having the highest probability in the driving position candidate group, that is, the position with the highest density of the driving positions as the final driving position, and determine the driving lane to which the set driving position belongs.

The position estimator 17 may convert the road area represented by the vanishing line into the road area represented by the parallel boundary line. The information providing apparatus 3 may display the estimated driving position on the converted image. The position estimator 17 may perform image transformation by homography (or warping) using camera internal parameters and coordinate transformation. Since image conversion by homography or warping is a well-known technique, a detailed description thereof will be omitted.

FIG. 9 is an exemplary view illustrating a result of determining a relative position X ′ of a driving lane by reflecting a width L and a lane width of an actual road from a candidate driving position X in a road image. In the embodiment of Figure 9 it can be seen that the two lanes driving on the road of all three lanes.

10 is a flowchart illustrating a method of detecting a position of a vehicle and a lane in driving according to an embodiment of the present invention. 11 is a flowchart illustrating a pixel classification method of FIG. 10. FIG. 12 is a flowchart for describing a method for generating candidate vanishing lines in FIG. 10.

Referring to FIG. 10, the vehicle and lane position detection system may receive a road image from a sensor and segment the road image into super pixels (S101). The visual and spatial features of each super pixel can be stored in memory. The visual feature may include a color distribution and a texture distribution. The spatial feature may comprise a central mass position.

The vehicle and lane position detection system may classify the super pixels into road and non-road pixels based on the visual and spatial features of the super pixels (S103). Classified road pixels may constitute a road area, and non-road pixels may constitute a non-road area. The non-road area is an area adjacent to the road area, not a remaining area other than the road area.

Referring to FIG. 11, the vehicle and lane position detection system may classify road pixels from super pixels based on information about road areas acquired through training on road areas and non-road areas ( S113).

When the unclassified superpixels exist between the classified road pixels, the vehicle and lane position detection system detects the rough road area by classifying the unclassified superpixels as road pixels using spatial information between adjacent superpixels (S133). It may be (S153).

In addition, the vehicle and lane position detection system may classify the non-road pixels from the super pixels based on the information about the non-road area acquired through the training of the road area and the non-road area (S113).

If there is an unclassified superpixel between the classified nonroad pixels, the vehicle and lane position detection system classifies the unclassified superpixel into nonroad pixels using spatial information between adjacent superpixels (S173). The area can be detected (S193).

The vehicle and lane position detection system may classify a road pixel or a non-road pixel arbitrarily according to a predetermined classification criterion when one super pixel is classified into road pixels and non-road pixels. In addition, the vehicle and lane position detection system may convert the unclassified super pixel into a road pixel or a non-road pixel based on spatial information between adjacent super pixels or a predetermined classification criterion when an unclassified super pixel exists in an area where road pixels and non-road pixels are mixed. Can be classified as

Vehicle and lane position detection systems may classify road pixels and non-road pixels in parallel or sequentially.

The vehicle and lane position detection system may generate a plurality of vanishing lines in a boundary area where road pixels and non-road pixels are mixed (S105).

Referring to FIG. 12 together, the vehicle and lane position detection system may set a boundary area between a rough road area and a non-road area (S115). Since the detected rough road area and the non-road area are a collection of super pixels, the boundary between the road area and the non-road area is not clear, and the road area and the non-road pixel may be mixed in the boundary area. The vehicle and lane position detection system may set a predetermined area where road pixels and non-road pixels are mixed as a boundary area.

The vehicle and lane position detection system may distribute the particles in the boundary area (S135). The vehicle and lane position detection system may set the probability distribution in the boundary region, generate particles in the boundary region, and distribute the particles corresponding to the probability distribution. In this case, the highest probability may be set along the center of the boundary region, and the probability distribution may be set in the boundary region such that the probability decreases as the distance from the center increases.

The vehicle and lane position detection system may select any particle pair among the distributed particles and generate a vanishing line connecting the pair of particles (S155). At this time, the vehicle and lane position detection system may select the particle pair so that the vanishing line connecting the particle pair does not pass the road area.

The vehicle and lane position detection system may generate a candidate road area from the vanishing line pair, and estimate the driving position from the shape of the candidate road area (S107). The vehicle and lane position detection system may select any vanishing line pair from the plurality of vanishing lines. The vanishing line pair may correspond to the boundary line pair of the left road boundary and the right road boundary. The vanishing point is predicted by the pair of vanishing lines, and a candidate road region having the vanishing pairs as road boundaries may be generated. The vehicle and lane position detection system may estimate the position of the sensor, that is, the driving position of the vehicle, which obtained the road image from the shape of the candidate road area.

The vehicle and lane position detection system extracts road information including the total width and lane width of the road extracted from the road map database, and uses the extracted road information to determine the relative position of the currently driving lane corresponding to the driving position, that is, It is possible to estimate the number of lanes in the entire lane.

The vehicle and lane position detection system may set a driving position candidate group excluding the driving position corresponding to noise from the plurality of driving positions estimated from the plurality of vanishing line pairs. The vehicle and lane position detection system may determine the driving lane to which the driving position belongs based on the road width and the lane width for the driving position candidate group. As another example, the vehicle and lane position detection system may set the position having the highest density in the driving position candidate group as the driving position, and determine the driving lane corresponding to the set driving position.

As a result image determined by the vehicle and lane position detection system is displayed, the driver can grasp the vehicle position and the driving lane on the driving road, and thus can assist in driving such as determining lane departure or changing lanes.

In addition, in order to correct this when noise occurs in a value of a sensor used to know the current position of the vehicle during autonomous driving, the vehicle position information may be corrected by detecting the vehicle position according to an embodiment of the present invention.

An embodiment of the present invention may be applied to an indoor autonomous driving robot to perform line tracing.

The vehicle and lane position detection method according to the present invention can be embodied as computer readable codes on a computer readable recording medium. Computer-readable recording media include all kinds of recording devices that store data that can be read by a computer system. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disks, optical data storage devices, and the like. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. In addition, functional programs, codes, and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

Claims

A segment unit for segmenting the road image obtained by the sensor into super pixels;

A classification unit classifying the superpixels into a road pixel constituting a road area and a non-road pixel constituting a non-road area adjacent to the road area;

A boundary detector for generating a plurality of vanishing lines in a boundary region where the road pixel and the non-road pixel are mixed; And

And a position estimator for generating a candidate road region by a pair of vanishing line selected from the plurality of vanishing lines, and estimating a driving position from a shape of the candidate road region.
The method of claim 1, wherein the classification unit,

And classify the superpixel into road pixels or non-road pixels based on at least one of the visual and spatial features of the super pixel.
The method of claim 1, wherein the classification unit,

A vehicle and lane position detection system for classifying unclassified superpixels into road pixels or non-road pixels using spatial information between adjacent super pixels.
The method of claim 1, wherein the boundary detection unit,

And selecting a plurality of particle pairs from the particles distributed corresponding to the probability distribution set in the boundary area, and generating a plurality of vanishing lines by the plurality of particle pairs.
The method of claim 4, wherein the boundary detection unit,

And select the pair of particles such that the vanishing line does not pass through the roadway area.
The method of claim 1, wherein the position estimating unit,

A vehicle and lane position detection system for estimating the driving position and the driving lane on a road based on a road width and a lane width.
The method of claim 1, wherein the position estimating unit,

And a candidate group is set from a plurality of driving positions estimated from a plurality of vanishing line pairs, and the driving position and the traveling lane are estimated for the candidate group based on a total road width and a lane width.
The method of claim 7, wherein the position estimating unit,

And estimating a position having the highest density in the candidate group as a traveling position, and estimating a traveling lane corresponding to the set traveling position.
Segmenting the road image obtained by the sensor into super pixels;

Classifying the superpixels into road pixels constituting a road area and non-road pixels constituting a non-road area adjacent to the road area;

Generating a plurality of vanishing lines in a boundary region where the road pixel and the non-road pixel are mixed; And

And generating a candidate road region by a pair of vanishing line selected from the plurality of vanishing lines, and estimating a driving position from a shape of the candidate road region.
The method of claim 9, wherein the classification step,

And classifying the super pixel into road pixels or non-road pixels based on at least one of the visual and spatial features of the super pixel.
The method of claim 9, wherein the classification step,

And classifying the unclassified super pixel into road pixels or non-road pixels using spatial information between adjacent super pixels.
The method of claim 9, wherein the candidate vanishing line generation step,

Distributing particles in the boundary area corresponding to a probability distribution set in the boundary area; And

Selecting a plurality of particle pairs from the distributed particles, and generating a plurality of vanishing lines by the plurality of particle pairs.
The method of claim 12,

And the pair of particles are selected such that the vanishing line does not pass through the roadway area.
The method of claim 9, wherein the driving position estimation step,

Estimating the driving position and the driving lane based on a road width and a lane width.
The method of claim 9, wherein the step of estimating position,

Setting a candidate group from the plurality of travel positions estimated from the plurality of vanishing line pairs; And

And estimating the driving position and the driving lane based on the total road width and the lane width with respect to the candidate group.
The method of claim 15, wherein the determining the driving lane,

And estimating a position at which the density of the candidate group is the highest as a driving position, and estimating a driving lane corresponding to the set driving position.