KR102180036B1 - Method for generating predicted path data of objects, apparatus therefor, and unmanned ground vehicle - Google Patents

Abstract

According to an embodiment of the present invention, predicted path data in consideration of uncertainty according to the object's location, speed, direction angle information, and the type of the object is generated for an object around the unmanned vehicle obtained from environmental recognition information of the unmanned vehicle, By displaying collision risk information between an unmanned vehicle and an object on a continuous space-time on a predicted path through online learning, it is possible to generate an optimal path that minimizes the risk of collision with an object when planning a path of the unmanned vehicle.

Description

Method, apparatus, and unmanned vehicle for generating predicted path data of an object {METHOD FOR GENERATING PREDICTED PATH DATA OF OBJECTS, APPARATUS THEREFOR, AND UNMANNED GROUND VEHICLE}

The present invention relates to autonomous driving of an unmanned vehicle, and more particularly, to a method, apparatus, and an unmanned vehicle for generating predicted path data of an object around an unmanned vehicle.

In general, in order for an unmanned vehicle to minimize the risk of external collision and perform safe autonomous driving, collision risk information within several tens of meters, which changes every moment, must be expressed so that it can be easily reflected in route planning, and this must be serviced in real time. In addition, for stable autonomous driving in complex driving situations in which not only stationary obstacles but also moving obstacles exist, space-time prediction information including uncertainty of moving obstacles is reflected based on information about several moving obstacles acquired through environmental recognition every moment. It is essential to have a technology that can predict the possibility of collision in the image.

In this regard, conventionally, a technique has been proposed for recognizing the absolute position of an obstacle and an unmanned vehicle and calculating a moving path of a moving obstacle to avoid collision with a moving obstacle at a future point in time. Here, when calculating the moving path of the moving obstacle, the position of the moving obstacle was reliably predicted by approximating it with a cubic function.

However, since the conventional method as described above does not include uncertainty in the moving path of the moving obstacle, there is a problem that it is difficult to apply to a path optimization problem that minimizes the maximum risk due to the uncertainty.

(Patent Document)

Korean Patent Registration No. 10-1207903 (Registration date November 28, 2012)

Accordingly, in an embodiment of the present invention, predicted route data in consideration of uncertainty according to the object's location, speed, direction angle information, and the type of the object is generated for an object around the unmanned vehicle obtained from environmental recognition information of the unmanned vehicle, By displaying the collision risk information between the unmanned vehicle and the object in continuous space and time on the predicted route through online learning, the predicted route data of the object that generates the optimal route that minimizes the risk of collision with the object when planning the route of the unmanned vehicle is generated. It is intended to provide a method, apparatus and driverless vehicle for generating.

A method of generating predicted path data of objects according to an embodiment of the present invention described above, comprising motion information of at least one or more objects around the unmanned vehicle based on environmental recognition information on a driving path of an unmanned vehicle. And acquiring the type of the object, and generating predicted path data of the object based on the motion information of the object and uncertainty according to the type of the object.

In addition, the environmental recognition information may be obtained using an environment recognition sensor mounted on the unmanned vehicle.

In addition, the motion information of the object may include at least one of a position, a speed, and a direction angle of the object.

In addition, generating the predicted route data may include generating a plurality of predicted route data at predetermined time intervals.

In addition, when the type of the object is a pedestrian, an uncertainty coefficient related to the uncertainty is set to be lower than a predetermined reference value.

In addition, when the type of the object is a vehicle, an uncertainty coefficient related to the uncertainty is set to be larger than a predetermined reference value.

In addition, the generating of the prediction path data includes calculating an average and variance of the plurality of prediction path data generated at each predetermined time interval, and corresponding to the prediction path data based on the average and variance. It characterized in that it further comprises the step of generating and storing the label information indicating whether the risk in the absolute coordinates of the location.

In addition, the method further comprises calculating a collision probability between the unmanned vehicle and the object with respect to a continuous space between the absolute coordinates based on the label information.

)로 구성된 로지스틱 회귀 분류기를 이용하여 산출되는 것을 특징으로 한다.In addition, the collision probability is based on an absolute coordinate, a plurality of sparse kernels and a weight assigned to each sparse kernel (

) Is calculated using a logistic regression classifier.

In addition, the weight is updated through online learning using stochastic gradient descent (SGD).

In addition, it characterized in that it further comprises the step of generating and displaying the risk information of the object based on the collision probability.

In addition, as an apparatus for generating predicted route data of an object according to an embodiment of the present invention, based on environment recognition information on a driving route of an unmanned vehicle, motion information of at least one or more objects around the unmanned vehicle and the object And an object recognition unit that acquires a type of, and an object path prediction unit that generates predicted path data of the object based on motion information of the object and uncertainty according to the type of the object.

In addition, the motion information of the object may include at least one of a position, a speed, and a direction angle of the object.

In addition, the object path prediction unit is characterized in that it generates a plurality of prediction path data at predetermined time intervals.

In addition, the object path predictor may generate the predicted path data by setting an uncertainty coefficient related to the uncertainty to be lower than a predetermined reference value when the type of the object is a pedestrian.

In addition, when the type of the object is a vehicle, the object path predicting unit generates the predicted path data by setting an uncertainty coefficient related to the uncertainty to be greater than a predetermined reference value.

In addition, the object path prediction unit calculates an average and a variance of the plurality of prediction path data generated at each predetermined time interval, and based on the average and variance, at absolute coordinates of a location corresponding to the prediction path data It characterized in that it generates and stores label information indicating whether there is a danger of.

In addition, the apparatus may further include an object risk generation unit that calculates a collision probability between the unmanned vehicle and the object in a continuous space between the absolute coordinates based on the label information.

)로 구성된 로지스틱 회귀 분류기를 이용하여 상기 충돌 확률을 산출하는 것을 특징으로 한다.In addition, the object risk generation unit, with respect to the absolute coordinates, a plurality of sparse kernels (sparse kernel) and a weight assigned to each sparse kernel (

It characterized in that the collision probability is calculated using a logistic regression classifier consisting of ).

In addition, as an unmanned vehicle according to an embodiment of the present invention, the unmanned vehicle includes an apparatus for generating predicted route data of an object, and the apparatus for generating predicted route data of the object is provided in the driving route of the unmanned vehicle. Based on the environmental recognition information, based on an object recognition unit that obtains motion information of at least one object around the unmanned vehicle and a type of the object, and the motion information of the object and uncertainty according to the type of the object, And an object path predictor that generates predicted path data of the object.

According to an embodiment of the present invention, predicted path data in consideration of uncertainty according to the object's location, speed, direction angle information, and the type of the object is generated for an object around the unmanned vehicle obtained from environmental recognition information of the unmanned vehicle, By displaying collision risk information between an unmanned vehicle and an object on a continuous space-time on a predicted path through online learning, it is possible to generate an optimal path that minimizes the risk of collision with an object when planning a path of the unmanned vehicle.

1 is a detailed block diagram of an apparatus for generating predicted path data of an object according to an embodiment of the present invention.
2 to 6 are diagrams illustrating predicted movement positions and risk information of an object for each predicted time interval by an apparatus for generating predicted path data of an object according to an embodiment of the present invention.
7 is a flowchart of operation control in an apparatus for generating predicted path data of an object according to an embodiment of the present invention.

Hereinafter, the operating principle of the present invention will be described in detail with reference to the accompanying drawings. In the following description of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the present invention, which may vary depending on the intention or custom of users or operators. Therefore, the definition should be made based on the contents throughout this specification.

1 is a block diagram illustrating a detailed block configuration of an apparatus for generating predicted path data of an object according to an embodiment of the present invention.

Hereinafter, operations of each component of the apparatus 100 for generating predicted path data of objects according to an embodiment of the present invention will be described in detail with reference to FIG. 1.

First, the object recognition unit 110 recognizes an absolute position of an unmanned vehicle and at least one object located on a driving path of the unmanned vehicle, and recognizes motion information and a type of the object through environmental recognition information. At this time, the motion information on the object may include information such as the position, speed, and direction angle of the object, the type of the object may be a pedestrian, a vehicle, etc., and the environmental recognition information is a camera mounted on an unmanned vehicle. , Lidar, radar, etc. may be acquired through environmental recognition sensors such as a radar, and provided to the object recognition unit 110, but the present invention is not limited thereto.

The object path predictor 120 generates predicted path data of the object based on information such as the position, speed, and direction angle of the object acquired by the object recognition unit 110 and uncertainty associated with the type of the object.

In addition, in generating the predicted route data of the object, the object path predicting unit 120 may generate a plurality of predicted route data at each predetermined time interval.

In this case, the type of the object may be classified into a case where the object is a pedestrian and a case, but is not limited thereto.

In addition, in generating the predicted route data, the object route predictor 110 may generate predicted route data by setting an uncertainty coefficient related to the object's uncertainty to be lower than a predetermined reference value when the type of the object is a pedestrian. In addition, when the type of the object is a vehicle, predicted path data may be generated by setting an uncertainty coefficient related to the uncertainty of the object larger than a predetermined reference value. In this case, the vehicle has a relatively low probability of rapid changes in speed, direction angle, etc. at each predetermined time interval compared to the pedestrian, so that the uncertainty coefficient for the vehicle is set lower than a predetermined reference value.

In addition, the object path prediction unit 120 calculates an average and variance of the plurality of prediction path data generated at predetermined time intervals, and whether there is a risk in the absolute coordinates of the position corresponding to the prediction path data based on the average and variance. You can generate label information indicating.

Hereinafter, an operation of generating predicted path data of an object in consideration of uncertainty in the object path predictor 120 will be described in more detail.

The object path predictor 120 generates predicted path data of the object based on information such as the position, speed, and direction angle of the object acquired by the object recognition unit 110 and uncertainty associated with the type of the object.

스텝에서 대상체의 타입에 따라

개의 샘플 경로에 대해 불확실성을 고려한 예측 경로 데이터를 생성한 후, 샘플링된 예측 경로 데이터로부터 각 스텝에서의 평균과 분산을 계산하는 과정을 나타낸다. Pseudo codes as shown in the following [Table 1] are all divided by a preset time interval by the object path predictor 120

According to the type of object in the step

It shows a process of calculating the average and variance at each step from the sampled prediction path data after generating prediction path data in consideration of uncertainty for the sample paths.

[Table 1]

값으로

사이의 균등 난수를 생성하여 대상체의 예측 경로 데이터에 불확실성을 부여한다. 여기서,

는 불확실성 강도조절 계수이며 시간 간격

는 0.1초로 정한다. 이때, 본 발명의 일실시예에서는 위 시간 간격을 위에서 보여지는 바와 같이"0.1"로 설정한 것을 예를 들어 설명하나 이에 한정되는 것은 아니다.As shown in [Table 1] above, the object path prediction unit 120

By value

By generating an even random number between, uncertainty is added to the predicted path data of the object. here,

Is the uncertainty intensity control factor and the time interval

Is set to 0.1 seconds. In this case, in an embodiment of the present invention, the above time interval is set to "0.1" as shown above, but is not limited thereto.

번째 스텝에서의 평균과 분산에 따라 아래의 [수학식 1]에서와 같은 정규분포로부터

개의 대상체 각각에 대한 2차원 공간상에 위치

정보와 해당 위치에서의 무인 차량과 대상체간 충돌 등과 같은 위험 여부에 대한 레이블 정보

를 저장한다.In addition, the object path prediction unit 120

From the normal distribution as in [Equation 1] below according to the mean and variance in the first step

Position in 2D space for each of the two objects

Information and label information on whether there is a risk such as a collision between an unmanned vehicle and an object at the location

Save it.

2 to 6 are diagrams illustrating predicted movement position and risk information of an object for each time interval predicted by the apparatus 100 for generating predicted path data of an object according to an embodiment of the present invention.

2 shows the location and risk of the object acquired in the first step (0.1 second).

Referring to FIG. 2, the initial positions 200 and 210 of the first object in the two-dimensional space are x(east)=25, y(north)=22, the speed is 1m/s, and the direction angle is ??90ㅀ. And, it can be seen that the initial position of the second object in the 2D space is x(east)=12, y(north)=15, the speed is 4m/s, and the direction angle is +90°.

3 to 6 are predicted positions of the first and second objects in the 10th step (1 second), the 20th step (2 seconds), the 30th step (3 seconds), and the 40th step (4 seconds), respectively. (200', 210') and predicted path data and risk are shown.

6 of FIGS. 3 to 6 show the predicted route data and risk of the first and second subjects in the 40th step, and are predicted route data of each of the D objects indicated by black dots, and What is displayed in yellow light indicates the degree of risk based on the probability of a collision with an unmanned vehicle.

As shown in FIG. 6, in an embodiment of the present invention, in consideration of the uncertainty of each object, the predicted path data is randomly distributed from the initial position (200, 210) of each object to the expected position (200', 210'). Accordingly, when the above information is used, a path in which the unmanned vehicle does not collide with the object can be more accurately calculated.

Next, the object risk generation unit 130 performs online learning based on the predicted route data generated by the object route predicting unit 120 and label information indicating whether there is a danger in the absolute coordinates of the position corresponding to the predicted route data. Thus, a collision probability between an unmanned vehicle and an object in a continuous space between absolute coordinates is calculated, and a risk of a continuous space-time object is generated and displayed based on the collision probability. This risk may mean the probability of a collision between the unmanned vehicle and the object, and as shown in FIG. 6, it can be seen that the highest display can be made at the center point of the object, and the lower the display is toward the center point.

)로 구성된 로지스틱 회귀 분류기(Logistic regression classifier)를 이용하여 충돌 확률을 산출할 수 있다. 또한, 대상체 위험도 생성부(130)는 SGD(stochastic gradient descent)를 이용한 온라인 학습을 통해 가중치를 갱신할 수 있다.In this case, in calculating the collision probability, the object risk generation unit 130 includes a plurality of sparse kernels and weights allocated to each sparse kernel for absolute coordinates corresponding to a continuous space on the predicted path data.

The collision probability can be calculated using a logistic regression classifier consisting of ). Also, the object risk generator 130 may update the weight through online learning using stochastic gradient descent (SGD).

Hereinafter, an operation of generating a risk of an object on a driving route based on predicted route data generated from the object route predicting unit 120 by the object risk generating unit 130 will be described in more detail.

과 학습이 필요한 가중치(

)로 구성된 로지스틱 회귀 분류기를 통해 온라인 학습을 수행하여 무인 차량과 대상체간 충돌 확률을 산출하고, 충돌 확률을 기초로 연속적인 시공간상 대상체의 위험도를 표시한다. 이때, 대상체 위험도 생성부(130)는 연속 공간상에 충돌 확률에 기반한 위험도를 정의하기 위해 Hilbert maps 기법을 활용할 수 있으나, 이에 한정되는 것은 아니다.The object risk generation unit 130 is a sparse kernel as shown in [Equation 2] below, based on the predicted route data of the object and the label information indicating whether there is a collision risk on the predicted route data.

And weights that need to be learned (

), by performing online learning through a logistic regression classifier configured to calculate a collision probability between an unmanned vehicle and an object, and displays the risk of the object in continuous spatiotemporal based on the collision probability. In this case, the object risk generation unit 130 may use the Hilbert maps technique to define a risk based on a collision probability on a continuous space, but is not limited thereto.

은 아래의 [수학식 3]과 같이 나타낼 수 있다.Where, the sparse kernel

Can be expressed as [Equation 3] below.

)를 학습하기 위한 목적함수

를 아래의 [수학식 4]에서와 같이 Negative log-likelihood로 정의할 수 있다.In addition, the object risk generation unit 130 has a weight (

) Objective function for learning

Can be defined as Negative log-likelihood as in [Equation 4] below.

)를 점진적으로 갱신시킬 수 있다.Here, the object risk generation unit 130 uses Stochastic Gradient Descent (SGD) as shown in [Equation 5] below.

) Can be updated incrementally.

는 아래의 [수학식 6]과 같이 [수학식 7]의 공간에 대한 학습율과 [수학식 8]의 시간에 대한 학습율의 곱으로 정의될 수 있다.Where learning rate

Can be defined as the product of the learning rate for space in [Equation 7] and the learning rate for time in [Equation 8] as shown in [Equation 6] below.

는 강도조절 계수이고

는 감소 비율이다. 학습된

는 전체

개의 각

번째 스텝에서의 대상체의 위험도 정보는 무인 차량의 경로 계획 시 비용함수로 활용될 수 있다. here

Is the intensity control factor

Is the reduction rate. Learned

Is full

Each of the dogs

The risk information of the object in the first step may be used as a cost function when planning a route of an unmanned vehicle.

7 is a flowchart illustrating an operation control flow in an apparatus for generating predicted path data of an object according to an embodiment of the present invention. Hereinafter, an embodiment of the present invention will be described in detail with reference to FIGS. 1 to 7.

First, the apparatus 100 for generating predicted path data of an object recognizes the absolute position of an unmanned vehicle and at least one object located on a driving path of the unmanned vehicle (S700), and motion information about the object acquired through environment recognition information and Based on the uncertainty associated with the type of the object, predicted path data of the object is generated (S702).

In this case, the information on the object may include information such as the type, position, speed, and direction angle of the object, and the environmental recognition information as above is a camera, lidar, radar mounted on an unmanned vehicle. ) Can be obtained through environmental sensors such as, but is not limited thereto.

Here, in generating the predicted route data of the object, the apparatus 100 for generating predicted route data analyzes information such as the type, speed, and direction of the object based on environmental recognition information, and predicts a plurality of predictions at predetermined time intervals. Route data can be created.

In addition, the apparatus 100 for generating predicted route data of the object calculates an average and variance of a plurality of predicted route data generated at predetermined time intervals, and based on the average and variance, the absolute coordinates of the position corresponding to the predicted route data It is possible to generate label information indicating whether there is a danger of (S704). In this case, the above-described risk may represent a risk of collision between the unmanned vehicle and the object, but is not limited thereto.

Subsequently, the apparatus 100 for generating predicted path data of an object learns online based on the predicted path data generated by the object path predicting unit 120 and label information indicating whether there is a danger in the absolute coordinates of the position corresponding to the predicted path data. A collision probability between an unmanned vehicle and an object in a continuous space between absolute coordinates is calculated (S706), and a risk of a continuous space-time object is generated and displayed based on the collision probability (S708).

In this case, such a risk may mean a collision probability between the unmanned vehicle and the object, and may be displayed highest at the center point of the object as shown in FIG. 6, and may be displayed lower as the distance from the center point.

As described above, according to an embodiment of the present invention, a predicted path in consideration of the object's position, speed, direction angle information, and uncertainty according to the type of the object with respect to the object around the unmanned vehicle obtained from environmental recognition information of the unmanned vehicle By generating data and displaying information on the risk of collision between the unmanned vehicle and the object in continuous space and time on the predicted route through online learning, the optimal route can be created to minimize the risk of collision with the object when planning the route of the unmanned vehicle. have.

Combinations of each step of each flowchart attached to the present invention may be performed by computer program instructions. Since these computer program instructions can be mounted on the processor of a general purpose computer, special purpose computer or other programmable data processing equipment, the instructions executed by the processor of the computer or other programmable data processing equipment are the functions described in each step of the flowchart. Will create a means of doing things. These computer program instructions can also be stored in computer-usable or computer-readable memory that can be directed to a computer or other programmable data processing equipment to implement a function in a particular way, so that the computer-usable or computer-readable memory It is also possible to produce an article of manufacture containing instruction means for performing the functions described in each step of the flow chart. Since computer program instructions can also be mounted on a computer or other programmable data processing equipment, a series of operating steps are performed on a computer or other programmable data processing equipment to create a computer-executable process to create a computer or other programmable data processing equipment. It is also possible for the instructions to perform the processing equipment to provide steps for performing the functions described in each step of the flowchart.

In addition, each step may represent a module, segment, or part of code that contains one or more executable instructions for executing the specified logical function(s). Further, it should be noted that in some alternative embodiments, the functions mentioned in the steps may occur out of order. For example, two steps shown in succession may in fact be performed substantially simultaneously, or the steps may sometimes be performed in the reverse order depending on the corresponding function.

Meanwhile, in the above description of the present invention, specific embodiments have been described, but various modifications may be implemented without departing from the scope of the present invention. Therefore, the scope of the invention should not be determined by the described embodiments, but should be determined by the claims.

100: device for generating predicted path data of an object
110: object recognition unit
120: object path prediction unit
130: subject risk generation unit

Claims (20)

As a method of generating predicted path data of objects,
Obtaining motion information of at least one object around the unmanned vehicle and a type of the object based on environmental recognition information on the driving path of the unmanned vehicle;
Generating predicted path data of the object based on the motion information of the object and uncertainty according to the type of the object,
Generating the predicted path data,
Generating a plurality of predicted path data for the object at a predetermined time point; and
Calculating an average and variance of the plurality of predicted route data using the plurality of predicted route data generated at the predetermined time point;
And generating information indicating whether there is a risk in an absolute coordinate corresponding to the predicted path data determined based on the average and the variance.
A method of generating predicted path data of an object. The method of claim 1,
The environment recognition information is obtained using an environment recognition sensor mounted on the unmanned vehicle.
A method of generating predicted path data of an object. The method of claim 1,
The motion information of the object includes one or more of a position, a speed, and a direction angle of the object.
A method of generating predicted path data of an object. delete The method of claim 1,
When the type of the object is a pedestrian, the uncertainty coefficient related to the uncertainty is set lower than a predetermined reference value.
A method of generating predicted path data of an object. The method of claim 1,
When the type of the object is a vehicle, the uncertainty coefficient related to the uncertainty is set larger than a predetermined reference value.
A method of generating predicted path data of an object. delete The method of claim 1,
The step of calculating a collision probability between the unmanned vehicle and the object with respect to a continuous space between the absolute coordinates based on information indicating whether there is danger in the absolute coordinates.
A method of generating predicted path data of an object. The method of claim 8,
The collision probability is for absolute coordinates, a plurality of sparse kernels and a weight assigned to each sparse kernel (

) Calculated using a logistic regression classifier
A method of generating predicted path data of an object. The method of claim 9,
The weight is updated through online learning using stochastic gradient descent (SGD).
A method of generating predicted path data of an object. The method of claim 10,
Further comprising the step of generating and displaying risk information of the object based on the collision probability
A method of generating predicted path data of an object. An apparatus for generating predicted path data of an object,
An object recognition unit that acquires motion information of at least one object around the unmanned vehicle and the type of the object based on environmental recognition information on the driving path of the unmanned vehicle;
An object path predictor configured to generate predicted path data of the object based on the motion information of the object and uncertainty according to the type of the object,
The object path prediction unit,
At a predetermined time point, generating a plurality of predicted path data for the object,
Calculating an average and variance of the plurality of predicted route data using the plurality of predicted route data generated at the predetermined time point,
Generating information indicating whether there is a risk in absolute coordinates corresponding to the predicted path data determined based on the average and variance
Object prediction path generation device. The method of claim 12,
The motion information of the object includes one or more of a position, a speed, and a direction angle of the object.
Object prediction path generation device. delete The method of claim 12,
The object path predicting unit generates the predicted path data by setting an uncertainty coefficient related to the uncertainty to be lower than a predetermined reference value when the type of the object is a pedestrian.
Object prediction path generation device. The method of claim 12,
The object path prediction unit generates the predicted path data by setting an uncertainty coefficient related to the uncertainty to be greater than a predetermined reference value when the type of the object is a vehicle.
Object prediction path generation device. delete The method of claim 12,
The apparatus further includes an object risk generation unit that calculates a collision probability between the unmanned vehicle and the object with respect to a continuous space between the absolute coordinates, based on information indicating whether there is a danger in the absolute coordinates.
Object prediction path generation device. The method of claim 18,
The object risk generation unit, for absolute coordinates, a plurality of sparse kernels and weights allocated to each sparse kernel (

) To calculate the collision probability using a logistic regression classifier consisting of
Object prediction path generation device. As an unmanned vehicle,
The unmanned vehicle includes a device for generating predicted route data of an object,
An apparatus for generating predicted path data of the object,
An object recognition unit that acquires motion information of at least one object around the unmanned vehicle and a type of the object based on environmental recognition information on the driving path of the unmanned vehicle;
An object path predictor configured to generate predicted path data of the object based on the motion information of the object and uncertainty according to the type of the object,
The object path prediction unit,
At a predetermined time point, generating a plurality of predicted path data for the object,
Using the plurality of predicted route data generated at the predetermined time point, an average and variance of the plurality of predicted route data are calculated,
Generating information indicating whether there is a risk in absolute coordinates corresponding to the predicted path data determined based on the average and variance
Driverless vehicle.

Images