KR20220084576A - Apparatus for building a database to support artificial intelligence-based autonomous driving - Google Patents

Abstract

The present invention relates to a database building device for an AI-based autonomous vehicle, which is a lidar sensor installed at the top of the front of the vehicle, a camera sensor installed in the upper front of the vehicle, a ToF sensor installed in the upper rear of the vehicle, and distributed on both sides of the rear of the vehicle an integrated sensing unit having two installed radar sensors to simultaneously acquire a plurality of sensing information; Integrated support for interworking with each of the lidar sensor, the camera sensor, the ToF sensor, and the radar sensor, converting the plurality of information into a preset data format, and integrating and outputting the information into one integrated sensing information integrated interfacing device; and a vehicle embedded processor that includes a plurality of vehicle scenarios subdivided according to object types and driving conditions, classifies and stores the integrated sensing information for each vehicle scenario, and builds a database for each vehicle scenario.

Description

Apparatus for building a database to support artificial intelligence-based autonomous driving}

The present invention relates to an apparatus for constructing a database for an artificial intelligence-based autonomous vehicle for acquiring and providing data necessary for learning an artificial intelligence network for autonomous vehicle driving.

Autonomous vehicles are recognized as an automobile market industrially, but recently, autonomous robots, energy, and related ecosystems such as the environment are expanding.

Connectivity through the development of communication technology expands the cognitive range of the body to prepare for dangerous moments in advance, and the automation function is software that mimics the brain of a driver who has been advanced through the convergence of artificial intelligence technology, low power and low heat generation of the system. Continuous technological improvement such as technology development is in progress.

In particular, autonomous vehicles employ artificial intelligence technology, and artificial intelligence technology for autonomous vehicles recognizes situations based on detected data and detection/recognition technology that detects pedestrians, vehicles, lanes, road markers, and signs based on images. to do it

In addition, in addition to image data, heterogeneous sensor data such as radar, lidar, and TOF should be adopted and utilized for precise judgment on driving.

However, in the prior art, the focus has only been on the technology for more effectively learning artificial intelligence based on the acquired data, and the technology for more efficiently and reliably acquiring and building the database required for learning the artificial intelligence network for autonomous vehicle driving. There was a limitation in which there was no consideration at all.

Accordingly, in order to solve the above problems, the present invention is to provide an artificial intelligence-based database construction apparatus for autonomous vehicles that can acquire multi-sensor information such as images, radar, lidar, and TOF in an optimal state.

In addition, by building and providing a database for each vehicle scenario, it is intended to provide an artificial intelligence-based database construction device for autonomous vehicles that can be used more effectively in learning artificial intelligence networks for vehicle autonomous driving.

The object of the present invention is not limited to the object mentioned above, and other objects not mentioned will be clearly understood by those of ordinary skill in the art from the description below.

As a means for solving the above problems, according to an embodiment of the present invention, a lidar sensor installed at the top of the front of the vehicle, a camera sensor installed on the front of the vehicle, a ToF sensor installed on the upper side of the rear of the vehicle, and distributed on both sides of the rear of the vehicle an integrated sensing unit having two installed radar sensors to simultaneously acquire a plurality of sensing information; Integrated support for interworking with each of the lidar sensor, the camera sensor, the ToF sensor, and the radar sensor, converts the plurality of information into a preset data format, and then integrates and outputs the integrated sensing information integrated interfacing device; and a vehicle embedded processor having a plurality of vehicle scenarios subdivided according to object type and driving situation, classifying and storing the integrated sensing information for each vehicle scenario, and building a database for each vehicle scenario. A building device is provided.

The integrated sensing unit remotely senses the front of the vehicle at an angle of view of 70 degrees to 110 degrees through the lidar sensor, and remotely shoots the front of the vehicle with an angle of view of 2 degrees to 56 degrees through the camera sensor, and uses the ToF sensor Sensing the rear of the vehicle at an angle of view of 80 to 100 degrees through an ultra-short distance sensing, and sensing the rear side of the vehicle at an angle of view of 5 to 20 degrees through the two radar sensors, or short-distance sensing with an angle of view of 40 to 50 degrees do it with

The vehicle embedded processor may further include a function of generating and providing a large amount of learning data in which a correlation between the vehicle scenario and the integrated sensing data is defined based on the database for each vehicle scenario.

The learning data is characterized by having characteristic information of the integrated sensing data as an input condition and an object type and driving situation of a vehicle scenario as an output condition.

The characteristic information of the integrated sensing data includes information on the type, shape, distance, and direction of the object extracted from the output of the lidar sensor, the type of object extracted from the output of the camera sensor, information on the location, the ToF sensor and It is characterized in that it is composed of information about the distance to the object extracted from the output of the radar sensor.

The present invention makes it possible to more reliably monitor object types and driving conditions by installing images, radar, lidar, and TOF in optimal positions to acquire integrated sensor information.

In addition, it is possible to automatically build and provide a database for each vehicle scenario by classifying and storing the integrated sensing data according to the vehicle scenario, thereby eliminating the need to perform a separate data classification operation when learning the AI network. In other words, it makes it possible to automatically build and provide a database so that it can be used more effectively for artificial intelligence network learning for vehicle autonomous driving.

1 is a diagram illustrating a configuration diagram of an apparatus for constructing a database for an AI-based autonomous vehicle according to an embodiment of the present invention.
2 and 3 are diagrams illustrating a camera sensor and a sensor arrangement diagram of an apparatus for constructing learning data according to an embodiment of the present invention.
4 is a diagram for explaining a method of constructing a database for each vehicle scenario according to an embodiment of the present invention.

The following is merely illustrative of the principles of the invention. Therefore, those skilled in the art will be able to devise various devices that, although not explicitly described or shown herein, embody the principles of the present invention and are included within the spirit and scope of the present invention. Further, it is to be understood that all conditional terms and examples listed herein are, in principle, expressly intended solely for the purpose of enabling the concept of the present invention to be understood, and not limited to the specifically enumerated embodiments and states as such. should be

Moreover, it is to be understood that all detailed description reciting the principles, aspects, and embodiments of the invention, as well as specific embodiments, are intended to cover structural and functional equivalents of such matters. It should also be understood that such equivalents include not only currently known equivalents, but also equivalents developed in the future, i.e., all devices invented to perform the same function, regardless of structure.

Thus, for example, the block diagrams herein are to be understood as representing conceptual views of illustrative circuitry embodying the principles of the invention. Similarly, all flowcharts, state transition diagrams, pseudo code, etc. may be tangibly embodied on computer-readable media and be understood to represent various processes performed by a computer or processor, whether or not a computer or processor is explicitly shown. should be

1 is a diagram illustrating a configuration diagram of an apparatus for constructing a database for an AI-based autonomous vehicle according to an embodiment of the present invention.

Referring to FIG. 1 , the learning data building apparatus of the present invention may be largely composed of an integrated sensing unit 100 , an integrated interfacing device 200 , and a vehicle embedded processor 300 .

The integrated sensing unit 100 includes a lidar sensor 110 for remote recognition, a camera sensor 120 for object recognition, a ToF sensor 130 for parking assistance, and two radar sensors 141 and 142 for rear-side warning. ) and the like, and through them, sensing information about the vehicle surroundings can be simultaneously acquired in various forms.

At this time, the lidar sensor 110 obtains and provides a three-dimensional image of the far front, the camera sensor obtains and provides a color image of the far front, and the ToF sensor 130 obtains and provides a depth image of the rear of the vehicle ultra-short distance and the radar sensor senses and provides a distance to an obstacle on the rear side of the vehicle.

The integrated interfacing device 200 has various vehicle communication interfaces such as CAN, LIN, FlexRay, Ethernet, and Serial, and through these, the lidar sensor 110, the camera sensor 120, the ToF sensor 130, and two Interworking with each of the radar sensors 141 and 142 is integrated and supported. In addition, the data format required by the user can be preset, and the data format of each of the pieces of information obtained through the integrated sensing unit 100 is batch-converted, and then integrated into one integrated sensing data and output.

The vehicle embedded processor 300 is implemented as an embedded device that can be installed inside a vehicle. This includes a plurality of vehicle scenarios in which object types and driving conditions are subdivided, and by classifying and storing the integrated sensing data of the integrated interfacing device 200 for each vehicle scenario, a database for each vehicle scenario is constructed.

In addition, when the lidar sensor 110 , the camera sensor 120 , and the ToF sensor 130 are installed or positioned, the vehicle body is detected by performing an object detection operation for each of the lidar sensing information, the camera image, and the ToF sensing information. It is checked whether the vehicle body is detected, and warning information for requesting reinstallation of the sensor can be generated and provided. That is, it is possible to prevent in advance that the sensing range is violated or reduced by the vehicle body.

In addition, the vehicle embedded processor 300 also generates and provides data for learning the artificial intelligence network for autonomous driving based on the database for each vehicle scenario. That is, it is possible to directly generate and provide large-capacity learning data in which the correlation between the vehicle scenario and the integrated sensing data is defined.

2 and 3 are diagrams illustrating a camera sensor and a sensor arrangement diagram of an apparatus for constructing learning data according to an embodiment of the present invention.

As shown in FIG. 2 , in the present invention, a lidar sensor 110 , a camera sensor 120 , a Time Of Flight (ToF) sensor 130 , and two radar sensors 141 and 142 in consideration of the camera sensor and sensor characteristics. ) are distributed.

The lidar sensor 110 is a device that uses light to image the surrounding environment in three dimensions. It is installed at the top of the front of the vehicle (eg, on the roof of the vehicle) and senses the front of the vehicle with an angle of view of 70 degrees to 110 degrees (for example, , 100~500M).

The camera sensor 120 is a device that acquires an image using visible light, which is installed on the upper side of the front of the vehicle (eg, behind the vehicle front rearview mirror) to take long-distance photography (eg, the front of the vehicle with an angle of view of 42 to 56 degrees) For example, 80~120M).

The ToF sensor 130 is a device that outputs a depth image using the TOF method (that is, a method of calculating a distance by measuring a time when light is emitted and reflected), and is a device for outputting a depth image at the upper rear side of the vehicle (eg, the vehicle It is installed on the upper side of the trunk) to take ultra-short-range shooting (eg, 0.5 to 6M) of the rear of the vehicle with an angle of view of 80 to 100 degrees.

The two radar sensors 141 and 142 are devices for detecting forward obstacles using electromagnetic waves, and each of the radar sensors detects both sides of the rear of the vehicle (for example, the rear lights of the vehicle) at an angle of view of 5 to 20 degrees (150 ~ 200M), or short-range sensing (50-80M) with an angle of view of 40 to 50 degrees on both sides of the rear of the vehicle.

That is, the present invention integrates the lidar sensor 110, the camera sensor 120, the ToF sensor 130, and the two radar sensors 141 and 142, and provides short-distance and long-distance maps for vehicle surroundings under various vehicle scenarios. to create and provide

4 is a diagram for explaining a method of constructing a database for each vehicle scenario according to an embodiment of the present invention.

First, a plurality of vehicle scenarios in which object types and driving situations are subdivided are created (S1). In this case, the object type may be subdivided into a person, a vehicle, an animal, a building, and the like, and the driving situation may be subdivided into a vehicle speed, a road situation, weather, illuminance, and the like.

In addition, while changing at least one of an object type and a driving situation according to a vehicle scenario, the vehicle using the lidar sensor 110 , the camera sensor 120 , the ToF sensor 130 , and both the radar sensors 141 and 142 at the same time The surrounding situation is integrated and monitored in various forms (S2).

And the data format of the data acquired and provided by the lidar sensor 110, the camera sensor 120, the ToF sensor 130, and the two radar sensors 141 and 142, respectively, is converted into one integrated data format. After collecting the sensing data (S3), by classifying and storing by vehicle scenario, a database for each vehicle scenario is constructed (S4).

In this way, in a state where the database for each vehicle scenario is built, when an external device connects through the integrated interfacing device 200 and requests the provision of a database corresponding to a specific vehicle scenario (S5), the vehicle embedded processor 300 is In response, a database corresponding to the corresponding vehicle scenario is searched and immediately provided to an external device (S5).

Although it has been described in the above description that the integrated sensing data is stored as it is, in order to improve data storage efficiency, the characteristic information extraction operation for the integrated sensing data is first performed to detect the characteristic information, and based on this, a database for each vehicle scenario can be built. make it possible

In addition, in this case, it is possible to generate a large amount of learning data in which the correlation between the vehicle scenario and the integrated sensing data is defined by using the characteristic information of the integrated sensing data read from the database for each vehicle scenario as it is, and as a result, it is possible to generate a large amount of learning data The required processing load can also be minimized.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications may be made by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or perspective of the present invention.

Claims (4)

Equipped with a lidar sensor installed at the top of the front of the vehicle, a camera sensor installed on the upper front of the vehicle, a ToF sensor installed on the upper rear of the vehicle, and two radar sensors distributed on both sides of the rear of the vehicle, multiple sensing information is simultaneously acquired integrated sensing unit;
Integrated support for interworking with each of the lidar sensor, the camera sensor, the ToF sensor, and the radar sensor, converts the plurality of information into a preset data format, and then integrates and outputs the integrated sensing information integrated interfacing device; and
Building a database for an AI-based autonomous vehicle including a vehicle embedded processor that has a plurality of vehicle scenarios subdivided according to object type and driving situation, classifies and stores the integrated sensing information for each vehicle scenario, and builds a database for each vehicle scenario Device. According to claim 1, wherein the integrated sensing unit
The front of the vehicle is remotely sensed with an angle of view of 70 to 110 degrees through the lidar sensor, the front of the vehicle is remotely photographed with an angle of view of 2 to 56 degrees through the camera sensor, and the rear of the vehicle is captured through the ToF sensor Ultra-short-range sensing with an angle of view of 80 to 100 degrees, and long-distance sensing with an angle of view of 5 to 20 degrees on both sides of the vehicle rear side through the two radar sensors, or short-range sensing with an angle of view of 40 to 50 degrees Artificial intelligence, characterized in that Database building device for self-driving cars. According to claim 1, wherein the vehicle embedded processor
AI, characterized in that it further comprises a function of performing an object detection operation for each of the lidar sensing information, the camera image, and the ToF sensing information, and generating and providing warning information for requesting reinstallation of the sensor when the vehicle body is detected. Database building device for self-driving cars. According to claim 1, wherein the vehicle embedded processor
Based on the database for each vehicle scenario, the AI-based self-driving vehicle database construction apparatus, characterized in that it further comprises a function to generate and provide a large amount of learning data in which the correlation between the vehicle scenario and the integrated sensing data is defined.

Images