KR20230014008A - Method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network - Google Patents

Abstract

According to an embodiment, an apparatus for determining the possibility of collision of a driving vehicle by using an artificial neural network comprises: a preprocessing module which uses LIDAR and radar information received from a LIDAR and a radar as input information, and uses grid map information including driving information on a target vehicle and an ego vehicle existing in a preset area as output information; and a pre-trained collision detection artificial neural network module which uses the grid map information as input information and uses first collision prediction information including information on whether a collision has occurred between the target vehicle and the ego vehicle as output information. The first collision prediction information includes information on whether the same corresponds to one of predefined driving types of the target vehicle and the ego vehicle. The driving types may include a parallel driving type of the target vehicle and the ego vehicle, a lane movement type of the target vehicle, a lane movement type of the ego vehicle, and a simultaneous lane movement type of the target vehicle and the ego vehicle. Therefore, the apparatus can more accurately predict the possibility of collision with surrounding vehicles compared to an existing apparatus.

Description

Method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network

The present invention relates to a method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network, and more particularly, analyzes the relationship between the currently running vehicle and the driving vehicle based on information acquired by a lidar sensor and a vehicle sensor. Thus, the present invention relates to a technology for providing information on whether there is a possibility of future collision.

Autonomous driving can be defined as the operation of a moving object that can operate on its own from a starting point to a destination without driver intervention. Autonomous driving technology is divided into 6 levels in Europe and 5 levels in the United States, and each level of technology is defined.

Currently, the autonomous driving level of self-driving cars has reached level 3, conditional automation. In level 3 autonomous driving, the driving of the vehicle is generally left to the system, but when there is a request from the system or when the driver directly intervenes. It refers to a level of autonomous driving in which control of the vehicle is transferred from the system to the driver only for a limited time.

At level 3 of autonomous driving, the system identifies the traffic situation and operates autonomously. In the case of level 3, even driving is possible through control such as low-speed driving, highway driving, and automatic lane change in case of traffic congestion. However, in the case of level 3, there is a disadvantage in that it is impossible to determine the driving condition of the movement path or the entire road. Therefore, the current development direction of the autonomous driving system is focused on improving the technology of the autonomous driving system so that it can enter the level 4 stage where accurate judgment is possible even in such a situation.

In order for the level of autonomous driving to enter from level 3 to level 4, the self-driving system must accurately determine various factors around the vehicle on the moving route to enable perfect autonomous driving without the intervention of the driver. And in order to accurately execute this, the autonomous driving system does not simply recognize the surrounding objects of the vehicle, but considers the driving information of the operating vehicle to recognize the expected driving path of the surrounding moving objects, and through this, the collision between the moving vehicle and the surrounding objects information must be predictable.

However, the technologies proposed so far only predict the driving path of the surrounding vehicle based on the information of the surrounding vehicle acquired using a single sensor or the information acquired through communication with the surrounding vehicle. There is no technology that comprehensively analyzes the relative relationship and accurately provides information on the relationship between the driving vehicle and surrounding vehicles.

In addition, an accident while driving does not only occur between the vehicle in motion and the vehicle in the next lane, and there is a high possibility of an accident with a vehicle driving in the lane next to the vehicle in motion. A technology for determining the possibility of an accident with a nearby vehicle running in the next lane only exists, and only determines whether there is a possibility of a collision with a nearby vehicle driving in the lane next to the vehicle.

Republic of Korea Patent Publication No. 10-2018-0159606 - Object detection method using camera and lidar sensor fusion and apparatus therefor (published on December 12, 2018)

Therefore, a method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment is an invention designed to solve the above-described problems, and analyzes the relationship between a vehicle in motion and surrounding vehicles based on information acquired by a sensor. Its purpose is to provide information to the driver.

More specifically, a method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment predefine types that may occur between a driving vehicle and surrounding vehicles, and then based on information acquired by various sensors. The purpose of the present invention is to provide a technology that provides information on types that may occur in the future between the driving vehicle and surrounding vehicles through path analysis of the vehicle in motion and the surrounding vehicles.

An apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment uses lidar information and radar information received from lidar and radar as input information, and detects a target vehicle existing in a preset area and A pre-processing module that uses grid map information including driving information about the ego vehicle as output information and the grid map information as input information, and whether or not a collision between the target vehicle and the ego vehicle occurs and a pre-learned collision detection artificial neural network module having first collision prediction information containing information as output information, wherein the first collision prediction information is selected from among predefined driving types of the target vehicle and the self vehicle. and information on whether they correspond to one type, wherein the driving type includes a parallel driving type of the target vehicle and the subject vehicle, a lane movement type of the target vehicle, a lane movement type of the own vehicle, and the target vehicle and simultaneous lane movement types of the self vehicle.

The size of the preset area may include 10m in front of the vehicle, 10m in the rear, and 10m in the direction of the target vehicle, or the length of two lanes.

The driving information may include point information of a radar signal transmitted by the radar, strength information of the radar signal, and relative speed information between the target vehicle and the own vehicle.

The input information further includes vehicle information received from a sensor attached to the vehicle, and the vehicle information includes steering angle information, yaw rate information, longitudinal acceleration information, lateral acceleration information, and vehicle speed information of the vehicle. may contain at least one.

The driving information includes point information of a radar signal transmitted by the lidar, strength information of the radar signal, and point information of a lidar signal transmitted by the lidar, and the grid map information includes the vehicle information It may be information attached to the grid map in a dummy form.

The preprocessing module may include a time windowing module for generating integrated data in which the lidar information and the radar information are sequentially arranged a predetermined number of times during a predetermined time.

The preset time may include a time of 1 second to 2 seconds, and the preset number of times may include 5 to 10 times.

The pre-processing module is based on the integrated data, feature vector information including point information of the lidar signal transmitted by the lidar, strength information of the lidar signal, and relative speed information between the target vehicle and the self vehicle. It may include a grid map information generation module that generates.

The grid map information generation module may generate grid map information by projecting the feature vector information onto a grid map.

The device for determining the possibility of collision of a driving vehicle using the artificial neural network is a logic related to the four types of driving output from a fully-connected layer, which is a stage before the soft max layer of the collision detection artificial neural network module. After extracting vector (logit vector) information, adding information on exception types that cannot be determined as the driving type to the logic vector, and outputting second collision prediction information based on the summed logic vector information, post-processing It may contain more modules.

The second collision prediction information may include information on whether it corresponds to any one of the driving type and the exception type.

An apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment uses lidar information and radar information received from lidar and radar as input information, and targets a target vehicle and self existing in a preset area. A pre-processing module that uses as output information Grid Map information including driving information about the ego vehicle, and uses the grid map information as input information and information on whether or not a collision between the target vehicle and the self vehicle has occurred Receives vehicle information from a pre-learned collision detection artificial neural network module and a sensor attached to the vehicle, which takes the first collision prediction information containing as output information, and transmits the vehicle information to the fully connected layer in the collision detection artificial neural network module. and an intermediate input information transmission module that transmits intermediate input information to the first layer, wherein the first collision prediction information corresponds to one of predefined driving types of the target vehicle and the self vehicle. and outputs information on whether or not the driving type includes a parallel driving type of the target vehicle and the self vehicle, a lane movement type of the target vehicle, a lane movement type of the own vehicle, and a relationship between the target vehicle and the self vehicle. Can include simultaneous lane movement types.

The vehicle information may include at least one of steering angle information, yaw rate information, longitudinal acceleration information, lateral acceleration information, and vehicle speed information of the vehicle.

A method for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment uses lidar information and radar information received from lidar and radar as input information, and a target vehicle and self existing in a preset area A pre-processing step of using grid map information including driving information about the ego vehicle as output information and using the grid map information as input information, and information on whether or not a collision between the target vehicle and the self vehicle has occurred and an output information output step of outputting the first collision prediction information using a pre-learned collision detection artificial neural network module having first collision prediction information as output information, wherein the first collision prediction information includes includes information on whether it corresponds to any one of predefined driving types of the target vehicle and the own vehicle, wherein the driving type is a parallel driving type of the target vehicle and the own vehicle, the target vehicle A lane movement type of the vehicle, a lane movement type of the own vehicle, and a simultaneous lane movement type of the target vehicle and the own vehicle may be included.

The driving information may include point information of a radar signal transmitted by the radar, strength information of the radar signal, and relative speed information between the target vehicle and the own vehicle.

A method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment analyzes an expected driving path between a vehicle in motion and a nearby vehicle through artificial neural network learning based on data obtained using various sensors, There is an effect of predicting the possibility of collision with surrounding vehicles more accurately than the prior art.

A method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment classifies a currently driving vehicle, a vehicle existing in a lane next to a vehicle, and a driving situation that may occur in the future by type, and then learns based on this Because it infers, it is possible to provide the driver with information about the predicted situation in detail, and furthermore, situations that are difficult to accurately distinguish are excluded from the output result through a separate algorithm, which has the effect of increasing the accuracy of the inference result. do.

1 is a diagram explaining four types that may occur between a self vehicle and a target vehicle according to the present invention.
2 is a diagram showing some components of an apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment of the present invention.
Figure 3 (a) is a block diagram showing some components of the pre-processing module according to an embodiment of the present invention, Figure 3 (b) is a diagram showing information input to the pre-processing module.
4 is a diagram illustrating types of grid map information input to a preprocessing module according to an embodiment of the present invention.
5 is a diagram illustrating a learning session and an inference session of a collision detection artificial neural network module according to an embodiment of the present invention.
6 is a diagram illustrating components, input information, and output information of a collision detection artificial neural network module according to an embodiment of the present invention.
7 is a diagram for explaining a process of a collision detection artificial neural network module according to an embodiment of the present invention.
8 is a diagram for explaining a process of a post-processing module according to an embodiment of the present invention.
9 is a diagram for explaining a process of a collision detection artificial neural network module according to another embodiment of the present invention.
10 is a diagram showing information output by a collision detection artificial neural network module by type according to an embodiment of the present invention.
11 is a diagram illustrating a comparison of output results when a post-processing module is applied and when a post-processing module is not applied according to each embodiment of the present invention.
12 is a diagram showing, in numerical terms, experimental results for prediction accuracy of a collision possibility determination apparatus according to the present invention and a collision possibility determination apparatus according to the prior art.

Hereinafter, embodiments according to the present invention will be described with reference to the accompanying drawings. In adding reference numerals to the components of each drawing, it should be noted that the same components have the same numerals as much as possible even if they are displayed on different drawings. In addition, in describing an embodiment of the present invention, if it is determined that a detailed description of a related known configuration or function hinders understanding of the embodiment of the present invention, the detailed description thereof will be omitted. In addition, embodiments of the present invention will be described below, but the technical idea of the present invention is not limited or limited thereto and can be modified and implemented in various ways by those skilled in the art.

In addition, terms used in this specification are used to describe embodiments, and are not intended to limit and/or limit the disclosed invention. Singular expressions include plural expressions unless the context clearly dictates otherwise.

In this specification, terms such as "include", "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or the existence or addition of more other features, numbers, steps, operations, components, parts, or combinations thereof is not excluded in advance.

In addition, throughout the specification, when a part is said to be “connected” to another part, this is not only the case where it is “directly connected”, but also the case where it is “indirectly connected” with another element in the middle. Terms including ordinal numbers, such as "first" and "second" used herein, may be used to describe various components, but the components are not limited by the terms.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted.

In addition, the title of the present invention is described as 'a device for determining the possibility of collision of a driving vehicle using an artificial neural network', but in the following specification, for convenience of description, 'a device for determining the possibility of collision of a driving vehicle using an artificial neural network' is 'determining the possibility of collision. Let's refer to it as 'device' and explain it.

1 is a diagram explaining four types that may occur between a self vehicle and a target vehicle according to the present invention.

The present invention determines whether or not there is a possibility of collision between the self vehicle 10 on which the driver is currently driving and the target vehicle 20 that is the object of analysis because there is a possibility of collision with the self vehicle 10 while driving around the self vehicle 10 As an invention related to a technology for determining, more specifically, information on the possibility of collision by analyzing the possibility of collision between the self vehicle 10 and the target vehicle 20 driving in the next lane by pre-set type. It is an invention related to a technology that can provide a driver.

The expected driving relationship between the self vehicle 10 currently driving and the target vehicle 20 driving in the lane next to the target vehicle 20 can be classified into a total of four types as shown in FIG. 1 .

The first type is a case of parallel driving in which both the subject vehicle 10 and the target vehicle 20 drive in parallel without changing lanes in the current situation. In this case, it can be determined that there is no possibility of an accident. For convenience of explanation, the parallel driving is referred to as the first type for description.

The second type is a target vehicle lane changing type in which the subject vehicle 10 continues driving without changing lanes in the current situation, but the target vehicle 20 moves by one lane in the direction of the subject vehicle 10 . In the case of the second type, the possibility of an accident occurring immediately is low, but there is a possibility of an accident depending on the driving route in the future. For convenience of explanation, the lane change type of the target vehicle will be referred to as the second type.

The third type is a target vehicle lane changing type in which the target vehicle 20 continues driving without changing lanes in the current situation, but the subject vehicle 10 moves by one lane in the direction of the target vehicle 20 . In the case of the third type, the possibility of an accident occurring immediately is low, but there is a possibility of an accident depending on the driving route in the future. For convenience of description, the lane change type of the vehicle will be referred to as the third type.

The fourth type is a case in which both the self vehicle 10 and the target vehicle 20 move from one lane to the other vehicle one lane at a time in the current situation. Although this vehicle was not driving, it corresponds to the type of vehicle accident that may occur. For convenience of description, the lane change type of the subject vehicle and the target vehicle will be referred to as the fourth type.

That is, the apparatus 100 for determining the possibility of collision according to the present invention analyzes the vehicle currently driving and the vehicle traveling in the next lane and scenarios that may occur by type, and then both vehicles 10 based on the information obtained from the sensors. ,20) predicts what type of driving they will drive in the future, and then provides the predicted information to the driver. The components of the present invention will be described in detail through the following drawings.

2 is a diagram showing some components of an apparatus for determining the possibility of collision of a traveling vehicle using an artificial neural network according to an embodiment of the present invention, and FIG. 3 (a) is a diagram of a preprocessing module according to an embodiment of the present invention. It is a block diagram showing some components, FIG. 3(b) is a diagram showing information input to the preprocessing module, and FIG. 4 is a type of grid map information input to the preprocessing module according to an embodiment of the present invention. It is a drawing showing

Referring to FIG. 2, the apparatus 100 for determining the possibility of collision according to an embodiment of the present invention includes a pre-processing module 110 that performs a pre-processing process on data obtained by a sensor, and output by the pre-processing module 110 The collision detection artificial neural network module 120 and the collision detection artificial neural network module 120 output the data as input data and the first collision prediction information including information on whether or not a collision between the target vehicle and the self vehicle has occurred as output information. It may include a post-processing module 130 that infers whether a collision is detected or not based on one output information and a memory module 140 that stores data acquired by the sensor.

Referring to (a) of FIG. 3 , the preprocessing module 110 generates first input information 34 combining lidar information 31 and radar information 32 or lidar information 31 and radar information 32 and second input information 35 combined with vehicle information 33 as input information and grid map information 40 including driving information of the self vehicle 10 and the target vehicle 20 as output information. As, it includes a time window module 111 and a grid map information generating module 112.

LiDAR information 31 means information obtained from at least one lidar sensor attached to a vehicle, and specifically, after transmitting a lidar signal to the target vehicle 20, it is reflected from the target vehicle 20 and is reflected back to the lidar sensor. It may include point information about the signal input to , reflectance information, and information about the strength of the point intensity input to the LIDAR sensor.

Radar information 32 refers to information obtained from at least one radar sensor attached to a vehicle, and specifically, after transmitting a radar signal to the target vehicle 20, it is reflected from the target vehicle 20 and input to the radar sensor again. It may include point information about the detected signal, information about reflectivity, and information about the intensity of point intensity input to the radar sensor.

The vehicle information 33 means information about the own vehicle 10 acquired from a sensor attached to the own vehicle 10, and specifically, steering angle information of the own vehicle 10, yaw rate information, longitudinal acceleration information, Lateral acceleration information and vehicle speed information may be included.

The pre-processing module 110 according to the present invention acquires information from various sensors, and in processing it, acquires information about areas existing in a preset area, and then projects it onto a grid map to obtain grid map information 40. generate

Specifically, the grid map is a map made up of cells in a grid unit as shown in the drawing, and the length of each unit cell is 0.1 m in horizontal length (d1) and 1 m in vertical length (l1). Location information of the vehicle 10 and the target vehicle 20 may be displayed on a unit cell of the grid map.

The size of the grid map is determined by the size of a preset area, and the size of the preset area means an area to be viewed with interest in the relationship between the self vehicle 10 and the target vehicle 20 . Specifically, as shown in (b) of FIG. 3 , the total length L in the longitudinal direction based on the vehicle 10 may correspond to a total length of 20 m, 10 m in the front longitudinal direction and 10 m in the rear longitudinal direction. The total length D in the lateral direction may be set to include 10 m or the length of two lanes in the direction from the self vehicle 10 to the target vehicle 20. The length of the preset area according to the present invention is not limited to the example described above, and may be set to a range of various lengths according to the traffic environment.

The time window module 111 of the preprocessing module 110 generates integrated data by accumulating data over several hours using the time windowing method for the first input information 34 or the second input information 35. can do.

Specifically, the time window module 111 may generate integrated data obtained by acquiring data at preset intervals for a preset time and cumulatively accumulated. For example, after receiving the first input information 34 or the second input information 35 a total of 5 times at intervals of 0.5 seconds and 0.1 seconds, the received information may be generated as one data type. When information is generated by the time windowing technique, past location information and time information of the vehicle can be used, so there is an advantage in that information can be output with higher accuracy than simply inference based on current information. .

The grid map information generation module 112 generates a feature vector based on the information generated by the time window module 111, projects the generated feature vector information onto the grid map, and finally generates the grid map information 40. can

Specifically, the grid map information generation module 112 generates radar point information feature vectors, radar reflectivity information feature vectors, radar intensity information feature vectors, lidar point information feature vectors, etc. from the integrated data generated by the time window module 111. Information may be generated for each feature vector, and different types of feature vectors may be generated depending on the type of input information input to the preprocessing module 110 .

When the first input information 34 including lidar information 31 and radar information 32 is input as input information of the pre-processing module 110, the pre-processing module 110 generates first grid map information. can

As shown in (a) of FIG. 4, the first grid map information includes a feature vector including lidar point information (a), a feature vector including lidar intensity (intensity) information (b), and radar Information on a feature vector including relative speed information (c) between the self vehicle 10 and the target vehicle 20 based on the information may be included.

In generating each vector, the grid map information generation module 112 may generate as many vectors as the number of times the time window module 111 receives the first input information during a preset time. That is, for example, when the time window module 111 receives the first input information 34 a total of 10 times at intervals of 0.1 seconds for 1 second, as shown in FIG. 4, the grid map information 40 is each vector A total of 10 channels can be created for Accordingly, the first grid map information may include driving information including lidar point information, lidar strength information, and relative speed information.

When the second input information 35 including lidar information 31 and radar information 32 as well as vehicle information 33 is input as input information of the preprocessing module 110, the preprocessing module 110 2 Grid map information can be created.

As shown in (b) of FIG. 4, the second grid map information includes a feature vector including lidar point information (a), a feature vector including lidar intensity (intensity) information (b), and radar Information on the feature vector including the point information (d) may be included, and since the vehicle information 33 is not information that can be projected on the grid map itself, it is attached to the grid map in a dummy form and is attached to the grid map. information can be created.

In generating each vector, the grid map information generation module 112 may generate as many vectors as the number of times the time window module 111 receives the first input information during a preset time. That is, for example, when the time window module 111 receives the second input information 35 a total of 10 times at intervals of 0.1 seconds for 1 second, as shown in FIG. 4, the grid map information 40 is each vector A total of 10 channels can be created for Accordingly, the second grid map information may include driving information including lidar point information, lidar strength information, and radar point information, and the generated grid map information 40 is the artificial neural network module 200 It is used as input information for

5 is a diagram illustrating a learning session and an inference session of an artificial neural network module for collision detection according to an embodiment of the present invention, and FIG. 6 is a diagram showing components, input information, and output of the artificial neural network module for collision detection according to an embodiment of the present invention. It is a diagram showing information, and FIG. 7 is a diagram for explaining a process of a collision detection artificial neural network module according to an embodiment of the present invention.

5 to 7 , the collision detection artificial neural network module 120 uses the grid map information 40 as input information and based on the predicted driving paths for the target vehicle 20 and the self vehicle 10 A pre-learned collision detection artificial neural network module whose output information is the first collision prediction information 50, which includes information on whether a collision occurs between the target vehicle 20 and the self vehicle 10, as shown in FIG. As described above, it may include a learning session 121 that performs learning based on input information, output information, and reference information, and an inference session 122 that outputs output information based on the input information.

The collision detection artificial neural network module 120 may borrow an artificial neural network of a general Deep Neural Network (DNN) structure having a plurality of hidden layers, and the data input to the collision detection artificial neural network module 120 is grid map format data In view of this, a CNN (Convolution Neural Network) type of network may be employed.

As shown in FIGS. 6 and 7, the grid map information 40, which is input information, is extracted through a plurality of convolution layers and a max polling process, and a feature map including specific information is extracted. Output information may be output through a final soft max after passing through the hidden layers.

The output information 50 output by the collision detection artificial neural network module 120 according to the present invention may include information on whether it corresponds to any one of the four driving types discussed above and probability information for this information. . Accordingly, the fully-connected layer 45 immediately before passing through the final softmax layer includes probability information for each of the four types. That is, V1 means the first type probability information 51 including the first type probability information, and V2 means the second type probability information 52 including the second type probability information. V3 means type 3 probability information 53 including probability information about type 3, and V4 means probability information 54 of type 1 including probability information about type 4 And, the soft max layer can output information about the type with the highest probability among the four types as output information of the collision detection artificial neural network module 120.

8 is a diagram for explaining a process of a post-processing module according to an embodiment of the present invention.

Referring to FIG. 8 , the post-processing module 130 is a module that uses information generated by the collision detection artificial neural network module 120, and is specifically a step before the soft max layer of the collision detection prediction artificial neural network module. After extracting the logit vector information on the four preset driving types output from the fully connected layer 45, adding information on the exception type that cannot be determined as the four driving types to the logic vector, and then , means a module that outputs the second collision prediction information 60 based on the summed logic vector information.

That is, the post-processing module 400 converts the first type probability information 51, the second type probability information 52, the third type probability information 53, and the fourth type probability information 54 into the fully connected layer 45 ), a module that determines the possibility of collision again with a total of five types after making the exception types that cannot be determined into the above four types into a new class.

In the case of the present invention, as described above, since the output result is output in one of the preset 4 types, the first collision prediction information always outputs one of the four types. Although, most of the expected driving paths of the self vehicle 10 and the target vehicle 20 correspond to the four types described above, but ambiguous cases that do not correspond to the four types may occur in some cases. And this case, of course, does not correspond to any of the 4 types, but setting it as one type and learning or reasoning can rather cause a problem of reducing the prediction results for the 4 types. Therefore, since the post-processing module 130 according to the present invention classifies difficult-to-determine types into the fifth type and outputs them, the post-processing module 130 is not applied to the first to fourth types. Information with higher reliability can be output.

Here, the criterion for classifying the post-processing module 130 into the fifth type is when none of the probabilities for the four types exceed the preset standard (for example, 60%). In this case, the accuracy of the estimation can be guaranteed. Since this is an impossible situation, this case can be classified as a fifth type and output information can be generated as a separate type.

9 is a diagram for explaining a process of a collision detection artificial neural network module according to another embodiment of the present invention.

The basic structure of the collision detection artificial neural network module 120 according to FIG. 9 is similar to the artificial neural network module described in FIG. 7, but the information input to the collision detection artificial neural network module 120 includes lidar information 31 and radar information ( 32), which is the first grid map information generated based on the collision detection artificial neural network module 120. ), there is a difference in that the fully connected layer is input to the first fully connected layer 42 that starts first.

Here, the information input to the first fully connected layer 42 may include steering angle information, yaw rate information, longitudinal acceleration information, lateral acceleration information, and vehicle speed information of the vehicle 10, and collision detection artificial The neural network module 120 may output first collision prediction information 50 as output information by performing learning and inference using intermediate input information input in this way.

10 is a diagram showing information output by a collision detection artificial neural network module by type according to an embodiment of the present invention.

10(a) is a first type in which both the self vehicle 10 and the target vehicle 20 drive in parallel without changing lanes in the current situation, and as shown in the drawing, the predicted driving path for each vehicle It can be seen that is output in parallel in a straight line.

10 (b) corresponds to a target vehicle lane change type in which the subject vehicle 10 continues driving without changing lanes in the current situation, but the target vehicle 20 moves by one lane in the direction of the subject vehicle 10. As a second type, as shown in the figure, the predicted driving path for the subject vehicle 10 is output in a straight line, but it can be seen that the predicted driving path for the target vehicle 20 changes lanes. Even though the lane of the target vehicle 20 is changed by one lane, if the predicted driving path is not output in parallel with the predicted path of the self vehicle 10, there is a possibility of an accident, and therefore, risk information about this can be output to the driver. there is.

10(c) corresponds to the self vehicle lane change type in which the target vehicle 20 continues to drive without changing lanes in the current situation, but the self vehicle 10 moves by one lane in the direction of the target vehicle 20. As a third type, as shown in the figure, the predicted driving path for the target vehicle 20 is output in a straight line, but it can be seen that the predicted driving path for the vehicle 20 changes lanes. Even if the lane of the self vehicle 10 is changed by one lane, if the predicted driving path is not output in parallel with the predicted path of the target vehicle 20, there is a possibility of an accident, and therefore, risk information regarding this may be output to the driver. there is

10(d) is a case in which both the subject vehicle 10 and the target vehicle 20 move from one lane to the other vehicle in the current situation, and in the case of the fourth type, the subject vehicle 10 and the target vehicle 20 move one lane at a time. Since the predicted driving paths of 10) and the target vehicle 20 are directed to the second lane, this case corresponds to a case in which an accident is highly likely to occur. Accordingly, in the case of the fourth type, danger information about this may be output to the driver.

11 is a diagram showing a comparison of output results when a post-processing module is applied and when a post-processing module is not applied according to each embodiment of the present invention. 11(b) shows the predicted driving path output by the collision detection artificial neural network module 120 when the information output from the collision detection artificial neural network module 120 passes through the post-processing module 130 As shown in the drawing, it can be seen that the output result is output as information about the fifth type for the type that cannot be accurately determined, as shown in the drawing.

As described above, when the post-processing module 130 is not applied, the output result must be output in one of the 4 types, so the type must be selected even though it does not correspond to the 4 types. accuracy may decrease. However, when the post-processing module 130 is applied, since types that are difficult to determine are separately classified into the fifth type and output, more accurate information can be output for the first to fourth types.

Figure 12 is a diagram showing experimental results for the prediction accuracy of the collision possibility determination apparatus according to the present invention and the collision possibility determination apparatus according to the prior art, in numerical terms, Figure 12 (a) is a basic data set for the experiment ( data set), and FIG. 12(b) is a diagram showing the experimental results for each type.

In addition, FIG. 12(c) is a diagram showing a comparison of the results of various embodiments of the present invention. Among the table items, the vehicle information input item is when only the vehicle information 33 is entered as the input information of the preprocessing module, the time The windowing information item is input information when lidar information 31 and radar information 32 are input and then information to which the time window technique is applied by the time window module 111 is used, lidar information and radar information The item refers to a case in which only radar information 31 and radar information 32 are used as input information. The item of the present invention means a case in which lidar information 31, radar information 32, and vehicle information 33 are all used as input information and a time window technique is also applied. As shown in the table, it can be seen that the accuracy increases as the amount of data increases and the time window technique is applied.

So far, the components and operating principles of the apparatus 100 for determining the possibility of collision according to an embodiment have been described through drawings.

A method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment analyzes an expected driving path between a vehicle in motion and a nearby vehicle through artificial neural network learning based on data obtained using various sensors, There is an effect of predicting the possibility of collision with surrounding vehicles more accurately than the prior art.

A method and apparatus for determining the possibility of collision of a driving vehicle using an artificial neural network according to an embodiment classifies a currently driving vehicle, a vehicle existing in a lane next to a vehicle, and a driving situation that may occur in the future by type, and then learns based on this Because it infers, it is possible to provide the driver with information about the predicted situation in detail, and furthermore, situations that are difficult to accurately distinguish are excluded from the output result through a separate algorithm, which has the effect of increasing the accuracy of the inference result. do.

The devices described above may be implemented as hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. A processing device may also access, store, manipulate, process, and generate data in response to execution of software. For convenience of understanding, there are cases in which one processing device is used, but those skilled in the art will understand that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it can include. For example, a processing device may include a plurality of processors or a processor and a controller. Other processing configurations are also possible, such as parallel processors.

Software may include a computer program, code, instructions, or a combination of one or more of the foregoing, which configures a processing device to operate as desired or processes independently or collectively. You can command the device. Software and/or data may be any tangible machine, component, physical device, virtual equipment, computer storage medium or device, intended to be interpreted by or provide instructions or data to a processing device. can be embodied in Software may be distributed on networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer readable media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved. Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

10: self vehicle 20: target vehicle
30: preprocessing module input information 40: grid map information
50: first collision prediction information 60: second collision prediction information
100: collision possibility determination device 110: pre-processing module
111: time window module 112: grid map information generation module
120: collision detection artificial neural network module 121: learning session
122: inference session 130: post-processing module
140: memory module

Claims (15)

A grid map including lidar and lidar information and radar information received from the radar as input information and including driving information for a target vehicle and an ego vehicle existing in a preset area Map) information as output information pre-processing module; and
A pre-learned collision detection artificial neural network module that takes the grid map information as input information and first collision prediction information including information on whether or not a collision between the target vehicle and the self vehicle has occurred as output information,
The first collision prediction information includes information on whether or not the target vehicle and the self vehicle correspond to one of predefined driving types,
The driving type is
Including a parallel driving type of the target vehicle and the own vehicle, a lane movement type of the target vehicle, a lane movement type of the own vehicle, and a simultaneous lane movement type of the target vehicle and the own vehicle,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 1,
The size of the preset area is,
Including the length of two lanes or 10 m in the direction of the target vehicle, 10 m in front and 10 m in the rear based on the self vehicle,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 1,
The driving information is
Including point information of a radar signal transmitted by the radar, strength information of the radar signal, and relative speed information between the target vehicle and the self vehicle,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 1,
The input information further includes vehicle information received from a sensor attached to the vehicle.
The vehicle information includes at least one of steering angle information, yaw rate information, longitudinal acceleration information, lateral acceleration information, and vehicle speed information of the vehicle.
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 4,
The driving information is
Including point information of the radar signal transmitted by the lidar, strength information of the radar signal, and point information of the lidar signal transmitted by the lidar,
The grid map information,
The vehicle information is information attached to the grid map in a dummy form,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 1,
The preprocessing module,
A time windowing module for generating integrated data in which the radar information and the radar information received a preset number of times during a preset time are sequentially arranged; including,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 6,
The preset time includes a time of 1 second to 2 seconds,
The preset number of times includes 5 to 10 times,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 7,
The preprocessing module,
Grid map for generating feature vector information including point information of the lidar signal transmitted by the lidar, strength information of the lidar signal, and relative speed information between the target vehicle and the self vehicle based on the integrated data Including; information generating module;
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 8,
The grid map information generation module,
generating grid map information by projecting the feature vector information onto a grid map;
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 1,
After extracting log vector information about the driving type output from a fully-connected layer, which is a stage before the soft max layer of the collision detection artificial neural network module, A post-processing module that sums information on exception types that cannot be determined as driving types and then outputs second collision prediction information based on the summed logic vector information; further comprising:
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 10,
The second collision prediction information,
Including information on whether it corresponds to any one of the driving type and the exception type,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. A grid map including lidar and lidar information and radar information received from the radar as input information and including driving information for a target vehicle and an ego vehicle existing in a preset area Map) information as output information pre-processing module;
a pre-learned collision detection artificial neural network module that takes the grid map information as input information and first collision prediction information including information on whether or not a collision between the target vehicle and the own vehicle has occurred as output information; and
An intermediate input information transmission module that receives vehicle information from a sensor attached to the vehicle and transmits the vehicle information as intermediate input information to a layer where a fully connected layer first starts in the collision detection artificial neural network module,
The first collision prediction information outputs information on whether or not it corresponds to one of the predefined driving types of the target vehicle and the self vehicle;
The driving type is
Including a parallel driving type of the target vehicle and the own vehicle, a lane movement type of the target vehicle, a lane movement type of the own vehicle, and a simultaneous lane movement type of the target vehicle and the own vehicle,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. According to claim 12,
The vehicle information,
Including at least one of steering angle information, yaw rate information, longitudinal acceleration information, lateral acceleration information, and vehicle speed information of the vehicle,
A device for determining the possibility of collision of a driving vehicle using an artificial neural network. A grid map including lidar and lidar information and radar information received from the radar as input information and including driving information for a target vehicle and an ego vehicle existing in a preset area A pre-processing step of using Map) information as output information; and
Using a pre-learned collision detection artificial neural network module that takes the grid map information as input information and first collision prediction information including information on whether or not a collision between the target vehicle and the self vehicle has occurred as output information, Including; an output information output step of outputting first collision prediction information;
The first collision prediction information includes information on whether it corresponds to one of predefined driving types of the target vehicle and the self vehicle;
The driving type is
Including a parallel driving type of the target vehicle and the own vehicle, a lane movement type of the target vehicle, a lane movement type of the own vehicle, and a simultaneous lane movement type of the target vehicle and the own vehicle,
A method for determining the probability of collision of a driving vehicle using an artificial neural network. According to claim 14,
The driving information is
Including point information of a radar signal transmitted by the radar, strength information of the radar signal, and relative speed information between the target vehicle and the self vehicle,
A method for determining the probability of collision of a driving vehicle using an artificial neural network.

Images