WO2023282466A1 - Training data collection method using laser preview, and computer program recorded on recording medium to execute same - Google Patents

Abstract

The present invention proposes a training data collection method using a laser preview, which collects data for machine training of artificial intelligence (AI). The method may comprise the steps of: a training data collection apparatus receiving detected data from a radar fixedly installed on a vehicle; the training data collection apparatus identifying an object from the received detected data; and the training data collection apparatus controlling, in correspondence to the distance between the vehicle and the identified object, LiDar that is fixedly installed on the vehicle and is to radiate laser pulses. According to the present invention described above, it is possible to reduce data having relatively low importance from among pieces of data for machine training of artificial intelligence (AI) by controlling a data collection period of multiple sensors installed in a vehicle, which collect data for machine training artificial intelligence (AI) that may be used for autonomous driving, or the quality of data to be collected.

Description

Learning data collection method using laser preview and computer program recorded on a recording medium to execute

The present invention relates to the processing of artificial intelligence (AI) learning data. More specifically, in collecting data for machine learning of artificial intelligence (AI), it relates to a learning data collection method using a laser preview and a computer program recorded on a recording medium to execute the method.

Artificial intelligence (AI) refers to a technology that artificially implements some or all of human learning abilities, reasoning abilities, and perception abilities using computer programs. In relation to artificial intelligence (AI), machine learning refers to learning to optimize parameters with given data using a model composed of multiple parameters. Such machine learning is classified into supervised learning, unsupervised learning, and reinforcement learning according to the form of learning data.

In general, designing data for artificial intelligence (AI) learning proceeds in the steps of data structure design, data collection, data refinement, data processing, data expansion, and data verification.

To describe each step in more detail, data structure design is performed through ontology definition, classification system definition, and the like. Data collection is performed by collecting data through direct filming, web crawling, or associations/professional organizations. Data purification is performed by removing redundant data from collected data and de-identifying personal information. Data processing is performed by performing annotation and inputting metadata. Data extension is performed by performing ontology mapping and supplementing or extending the ontology as needed. In addition, data verification is performed by verifying validity according to the set target quality using various verification tools.

On the other hand, autonomous driving of a vehicle refers to a system that can judge and drive a vehicle by itself. Such autonomous driving may be classified into gradual stages from non-automation to complete automation according to the degree of involvement of the system in driving and the degree of control of the vehicle by the driver. In general, the level of autonomous driving is divided into six levels classified by the Society of Automotive Engineers (SAE) International. According to the six levels classified by the International Association of Automotive Engineers, level 0 is non-automation, level 1 is driver assistance, level 2 is partial automation, level 3 is conditional automation, level 4 is highly automated, and level 5 The steps are fully automated steps.

Autonomous driving of vehicles is performed through mechanisms of perception, localization, path planning, and control. Currently, several companies are developing to implement recognition and path planning among autonomous driving mechanisms using artificial intelligence (AI).

Data used for machine learning of artificial intelligence (AI) that can be used for autonomous driving of these vehicles is collected by various types of sensors installed in the vehicle. For example, the data used for machine learning of artificial intelligence (AI) that can be used for autonomous driving of a vehicle are lidar, camera, radar, and ultrasonic sensor fixed to the vehicle. ) may be acquired, photographed, or sensed data by, but is not limited thereto.

However, not all data continuously acquired, photographed, or measured by various sensors installed in a vehicle include an object to be learned by artificial intelligence (AI). That is, among all data continuously acquired, photographed, or measured by various sensors, a large number of data that are meaningless for machine learning of artificial intelligence (AI) that can be used for autonomous driving are included.

In addition, various sensors for acquiring, photographing, or measuring data used for machine learning of artificial intelligence (AI) cannot be physically installed at the same point, but must be spaced apart from each other and installed separately. Accordingly, data acquired, photographed, or measured by various sensors inevitably have differences depending on the location where each sensor is installed.

One object of the present invention is to provide a method for collecting learning data using laser preview in collecting data for machine learning of artificial intelligence (AI) that can be used for autonomous driving of a vehicle.

Another object of the present invention is to provide a computer program recorded on a recording medium to execute a learning data collection method using laser preview in collecting data for machine learning of artificial intelligence (AI).

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the technical problem as described above, the present invention proposes a method for effectively controlling multiple sensors for collecting data for machine learning of artificial intelligence (AI). The method includes the steps of receiving, through the transceiver, sensing data from a ray in which a learning data collection device is fixed to a vehicle, and a radar in which the processor is fixed to the vehicle; identifying, by the processor, an object from the received sensing data; And it may be a computer program recorded on a recording medium so that the processor executes the step of controlling a lidar that is fixedly installed in the vehicle and emits laser pulses in response to the distance from the identified object. there is.

Details of other embodiments include receiving sensing data from detailed descriptions and radars; identifying an object from the received sensing data by the learning data collection device; and controlling, by the learning data collection device, a lidar fixedly installed in the vehicle to emit laser pulses in response to a distance between the vehicle and the identified object. In this case, the detection data may be information on points at which the electromagnetic wave signal emitted by the radar toward the driving direction of the vehicle is reflected.

In detail, in the identifying the object, the object may be identified by extracting points forming a crowd within a preset threshold range from points included in the sensed data.

In the controlling of the lidar, when no object is identified from the sensing data, the lidar may be controlled not to emit the laser pulse.

In the controlling of the lidar, when the distance between the vehicle and the object is greater than a preset lidar recognition range, the lidar does not emit the laser pulse or emits the laser pulse only with a preset intensity. It can be controlled to emit.

In the step of controlling the lidar, when the distance between the vehicle and the object is within the lidar recognition range, the laser pulse radiation period of the lidar is lengthened or shortened in proportion to the distance from the object. You can control it. In the controlling of the lidar, when the distance between the vehicle and the object is within a predetermined range, a plurality of cameras fixed to the vehicle may be controlled to capture 2D images. In this case, the specific possible range corresponds to a narrower range than the lidar recognition range.

In the controlling of the LIDAR, a resolution of a 2D image to be photographed by the plurality of cameras may be differently set in correspondence to a distance between the vehicle and the object.

The controlling of the lidar includes estimating a movement path of the object based on a time-sequential change in position of the object included in the sensing data, and photographing a direction corresponding to the estimated movement path among the plurality of cameras. The shooting cycle of the camera can be set shorter than that of cameras shooting in other directions.

In the controlling of the lidar, when the distance between the vehicle and the object is within a preset contactable range, a plurality of ultrasonic sensors fixed to the vehicle may be controlled to emit ultrasonic waves.

In order to achieve the technical problem as described above, the present invention proposes a computer program recorded on a recording medium to execute a method capable of effectively controlling multiple sensors. The computer program may include a memory; transceiver; and a processor configured to process instructions resident in the memory.

According to embodiments of the present invention, by controlling the data collection period of multiple sensors installed in a vehicle or the quality of data to be collected that collects data for machine learning artificial intelligence (AI) that can be used for autonomous driving, artificial intelligence (AI) Among AI) data for machine learning, data of relatively low importance may be reduced. In this way, by reducing data of relatively low importance, it is possible to lower the burden of data processing for machine learning of artificial intelligence (AI).

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a block diagram of an artificial intelligence learning system according to an embodiment of the present invention.

2 is an exemplary view for explaining multiple sensors according to an embodiment of the present invention.

3 is a logical configuration diagram of a learning data collection device according to an embodiment of the present invention.

4 is a hardware configuration diagram of a learning data collection device according to an embodiment of the present invention.

5 to 7 are exemplary diagrams for explaining a process of controlling multiple sensors according to an embodiment of the present invention.

8 and 9 are exemplary diagrams for explaining a process of post-processing collected data according to an embodiment of the present invention.

10 to 12 are exemplary diagrams for explaining a process of correcting errors of multiple sensors according to an embodiment of the present invention.

13 is a flowchart illustrating a data collection method according to an embodiment of the present invention.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in this specification should be interpreted in terms commonly understood by those of ordinary skill in the art to which the present invention belongs, unless specifically defined otherwise in this specification, and are overly inclusive. It should not be interpreted in a positive sense or in an excessively reduced sense. In addition, when the technical terms used in this specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that those skilled in the art can correctly understand. In addition, general terms used in the present invention should be interpreted as defined in advance or according to context, and should not be interpreted in an excessively reduced sense.

Also, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "consisting of" or "having" should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or steps are included. It should be construed that it may not be, or may further include additional components or steps.

Also, terms including ordinal numbers such as first and second used in this specification may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. On the other hand, when a component is referred to as “directly connected” or “directly connected” to another component, it should be understood that no other component exists in the middle.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings. The spirit of the present invention should be construed as extending to all changes, equivalents or substitutes other than the accompanying drawings.

As described above, not all data continuously acquired, photographed, or measured by various sensors installed in a vehicle include an object to be learned by artificial intelligence (AI). That is, among all data continuously acquired, photographed, or measured by various sensors, a large number of data that are meaningless for machine learning of artificial intelligence (AI) that can be used for autonomous driving are included.

In addition, various sensors for acquiring, photographing, or measuring data used for machine learning of artificial intelligence (AI) cannot be physically installed at the same point, but must be spaced apart from each other and installed separately. Accordingly, data acquired, photographed, or measured by various sensors inevitably have differences depending on the location where each sensor is installed.

In order to solve these difficulties, the present invention can effectively control multiple sensors that collect data for machine learning of artificial intelligence (AI), reduce meaningless data, and minimize errors between data collected by multiple sensors. We would like to suggest a variety of means available.

1 is a block diagram of an artificial intelligence learning system according to an embodiment of the present invention.

As shown in FIG. 1, the artificial intelligence learning system according to an embodiment of the present invention includes a learning data collection device 100, a learning data generating device 200, a plurality of annotation devices 300, and an artificial intelligence learning device ( 400) may be configured.

Since the components of the artificial intelligence learning system according to an embodiment are merely functionally distinct elements, two or more components are integrated and implemented in an actual physical environment, or one component is implemented in an actual physical environment. may be implemented separately from each other.

Describing each component, the learning data collection device 100 is a device that can be used to collect data for machine learning of artificial intelligence (AI) that can be used for autonomous driving of a vehicle.

That is, the learning data collection device 100 may obtain, capture, or sense data by controlling multiple sensors. And, the learning data collection device 100 may transmit acquired, photographed, or sensed data to the learning data generating device 200 so that they can be processed.

In this case, the multiple sensors that are controlled by the learning data collection device 100 may include one or more of a lidar, a camera, a radar, and an ultrasonic sensor. It is not limited.

Such multiple sensors that are controlled by the learning data collection device 100 will be described in more detail later with reference to FIG. 2 .

Characteristically, the learning data collection apparatus 100 according to embodiments of the present invention effectively controls multiple sensors for collecting data, reduces meaningless data, and reduces errors between data collected by multiple sensors. It has features that can be minimized.

Such characteristics of the learning data collection device 100 will be described in more detail later with reference to FIGS. 3 and 4 .

With the following configuration, the learning data generating device 200 is a device that can be used to design and process data for machine learning of artificial intelligence (AI) that can be used for autonomous driving of vehicles.

Specifically, the learning data generating device 200 may receive attributes of a project related to artificial intelligence (AI) learning from the artificial intelligence learning device 400 . The learning data generating device 200 is based on the user's control and the properties of the received project, designing a data structure for artificial intelligence (AI) learning, refining collected data, processing data, expanding data, and verification can be performed.

First of all, the learning data generating device 200 may design a data structure for artificial intelligence (AI) learning. For example, the learning data generating device 200 forms an ontology for artificial intelligence (AI) learning and a data classification system for artificial intelligence (AI) learning based on the attributes of the user's control and the received project. can define

The learning data generating device 200 may collect data for artificial intelligence (AI) learning based on the designed data structure. To this end, the learning data generating device 200 may receive sensing data, 3D point cloud data, 2D images, and sensing data from the learning data collecting device 100 . However, the learning data generation device 200 is not limited thereto, and may perform web crawling or download data from an external organization's device.

The learning data generation apparatus 200 may remove redundant or extremely similar data from among the collected sensing data, 3D point cloud data, 2D images, and sensing data. The learning data generation apparatus 200 may de-identify personal information included in the collected sensing data, 3D point cloud data, and 2D images.

The learning data generating device 200 may distribute and transmit collected and refined sensing data, 3D point cloud data, 2D images, and sensing data to a plurality of annotation devices 300 . In this case, the learning data generating device 200 generates sensing data, 3D point cloud data, 2D images, and sensing data in response to a pre-allocated amount for an operator (ie, labeler) of the annotation device 300. can be distributed

The learning data generating device 200 may receive annotation work results from each annotation device 300 . The learning data generating device 200 may generate AI learning data by packaging the received annotation work result. And, the learning data generating device 200 may transmit the generated artificial intelligence (AI) learning data to the artificial intelligence learning device 400 .

The learning data generation device 200 having such characteristics transmits and receives data to and from the learning data collection device 100, the annotation device 300, and the artificial intelligence learning device 400, and performs calculations based on the transmitted and received data. Any device that can do this is acceptable. For example, the learning data generating device 200 may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto.

With the following configuration, the annotation device 300 is a device that can be used to annotate the sensed data distributed by the learning data generating device 200, 3D point cloud data, 2D images, and sensing data. All or part of the annotation device 300 may be a device for performing annotation work by an annotation worker through a clouding service.

Specifically, the annotation device 300 selects one sensed data, 3D point cloud data, It can be output to 2D image or sensing data display.

The annotation device 300 may select a tool according to a signal input from a user through an input/output device. Here, the tool is a tool for setting a bounding box that specifies one or more objects included in sensing data, 3D point cloud data, 2D image, or sensing data.

The annotation device 300 may receive coordinates according to the selected tool through an input/output device. In addition, the annotation device 300 may specify an object included in sensing data, 3D point cloud data, 2D image, or sensing data by setting a bounding box based on the input coordinates.

Here, the bounding box is an area for specifying an object to be learned by artificial intelligence (AI) among objects included in sensing data, 3D point cloud data, 2D image, or sensing data. Such a bounding box may have a rectangle or cube shape, but is not limited thereto.

For example, the annotation device 300 receives two coordinates through an input/output device, and based on the inputted two coordinates, the coordinates and the upper left vertex within sensing data, 3D point cloud data, 2D image, or sensing data. An object can be specified by setting a bounding box based on a rectangle with the coordinates of the lower right vertex. In this case, the two coordinates may be set by the user inputting one type of input signal twice (eg, mouse click) or by the user inputting two types of input signal once (eg, mouse drag). It may, but is not limited thereto.

The annotation device 300 generates sensing data, 3D point cloud data, 2D image, sensing data, or metadata for a set object to be annotated according to a signal input from a user through an input/output device. can Here, the metadata is sensing data, 3D point cloud data, 2D image, sensing data, or data for describing an object. Such metadata includes the category of the specified object, the rate at which the object is clipped by the angle of view, the rate at which the object is obscured by other objects or objects, the tracking ID of the object, the time the image was taken, and the weather conditions on the day the image was taken. may include, but are not limited to, file size, image size, copyright holder, resolution, bit value, aperture transmittance, exposure time, ISO sensitivity, focal length, aperture value, angle of view, white balance, RGB depth, class name , tag, shooting location, type of road, road surface information, or traffic jam information may be further included.

The annotation device 300 may generate an annotation work result based on the specified object and generated metadata. In this case, the annotation work result may have a JSON (Java Script Object Notation) file format, but is not limited thereto. The annotation device 300 may transmit the generated annotation work result to the learning data generating device 200 .

Any device capable of transmitting and receiving data with the learning data generating device 200 and performing an operation based on the transmitted and received data may be used as the annotation device 300 having such characteristics. For example, the annotation device 100 may be a stationary computing device such as a desktop, workstation, or server, or a smartphone, laptop, tablet, or tablet. It may be any one of mobile computing devices such as a phablet, a portable multimedia player (PMP), a personal digital assistant (PDA), or an e-book reader.

With the following configuration, the artificial intelligence learning device 400 is a device that can be used for machine learning of artificial intelligence (AI) that can be used for autonomous driving of a vehicle.

Specifically, the artificial intelligence learning device 400 may transmit requirements for achieving the purpose of artificial intelligence (AI) that can be used for autonomous driving of a vehicle to the learning data generating device 200 . The artificial intelligence learning device 400 may receive artificial intelligence (AI) learning data from the learning data generating device 200 . In addition, the artificial intelligence learning apparatus 400 may perform machine learning on artificial intelligence (AI) that can be used for autonomous driving of a vehicle using the received artificial intelligence (AI) learning data.

As such, the artificial intelligence learning device 400 may be any device capable of transmitting and receiving data to and from the learning data generating device 200 and performing calculations using the transmitted and received data. For example, the artificial intelligence learning device 400 may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto.

As described above, the learning data collection device 100, the learning data generation device 200, a plurality of annotation devices 300, and the artificial intelligence learning device 400 are connected directly to each other through a secure line, common Data may be transmitted and received using a network in which one or more of a wired communication network or a mobile communication network is combined.

For example, public wired communication networks may include Ethernet, x Digital Subscriber Line (xDSL), Hybrid Fiber Coax (HFC), and Fiber To The Home (FTTH). It may be, but is not limited thereto. In addition, in the mobile communication network, Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), High Speed Packet Access (HSPA), Long Term Evolution, LTE) and 5th generation mobile telecommunication may be included, but is not limited thereto.

Hereinafter, multiple sensors to be controlled by the learning data collection device 100 as described above will be described in more detail.

2 is one of the present invention in the examples for explaining multiple sensors according to is an example .

As shown in FIG. 2, the learning data collection device 100 according to an embodiment of the present invention includes a radar 20, a lidar 30, a camera 40, and an ultrasonic sensor ( 50), it is possible to acquire, photograph or sense data for machine learning of artificial intelligence (AI).

Here, the vehicle 10 is a vehicle equipped with a radar 20, a lidar 30, a camera 40, and an ultrasonic sensor 50 for collecting basic data for machine learning of artificial intelligence (AI). It can be distinguished from vehicles that perform autonomous driving by intelligence (AI).

The radar 20 is fixedly installed in the vehicle 10 and emits electromagnetic waves toward the driving direction of the vehicle 10, and the electromagnetic waves reflected by an object located in front of the vehicle 10 and returned. By sensing, the vehicle 10 may generate sensing data corresponding to an image of the front side.

In other words, the sensing data is information on points at which electromagnetic waves emitted by the radar 20 fixedly installed in the vehicle 10 toward the driving direction of the vehicle are reflected. Accordingly, the coordinates of the points included in the sensing data may have values corresponding to the position and shape of an object located in front of the vehicle 10 . This sensed data may be 2D information, but is not limited thereto and may be 3D information.

The lidar 30 is fixedly installed on the vehicle 10, radiates a laser pulse around the vehicle 10, and detects light reflected back by an object located around the vehicle 10. , 3D point cloud data corresponding to a 3D image of the surroundings of the vehicle 10 may be generated.

In other words, the 3D point cloud data is three-dimensional information on points obtained by reflecting laser pulses emitted around the vehicle by the LIDAR 30 fixedly installed in the vehicle 10 . Accordingly, coordinates of points included in the 3D point cloud data may have values corresponding to the location and formation of objects located around the vehicle 10 .

The camera 40 is fixedly installed in the vehicle 10 and can capture a two-dimensional image of the surroundings of the vehicle 10 . As such, a plurality of cameras 40 may be configured according to the angle of view. For example, although FIG. 2 shows an example in which six cameras 40 are installed in the vehicle 10, the number of cameras 40 that can be installed in the vehicle 10 can be configured according to the present invention. It will be obvious to those skilled in the art.

In other words, the 2D image is an image captured by the camera 40 fixed to the vehicle 10 . Accordingly, the 2D image may include color information of an object located in a direction in which the camera 40 faces.

The ultrasonic sensor 50 is fixedly installed in the vehicle 50, emits ultrasonic waves around the vehicle 10, detects sound waves reflected by an object positioned adjacent to the vehicle 10 and returns, Distance information corresponding to the distance between the ultrasonic sensor 50 installed in (10) and the object may be generated. In general, a plurality of ultrasonic sensors 50 may be configured and fixedly installed at the front, rear, front and rear sides of the vehicle 10, which are easily contacted with objects.

In other words, the distance information is information about a distance from an object detected by the ultrasonic sensor 50 fixedly installed in the vehicle 10 .

Hereinafter, the configuration of the learning data collection device 100 as described above will be described in more detail.

3 is one of the present invention in the examples It is a logical configuration diagram of the learning data collection device according to FIG.

As shown in FIG. 3, the learning data collection device 100 includes a communication unit 105, an input/output unit 110, a multi-sensor control unit 115, a data post-processing unit 120, an error correcting unit 125, and a data control unit. It may be configured to include study 130.

Since the components of the learning data collection device 100 are only functionally distinct elements, two or more components are integrated and implemented in an actual physical environment, or one component is mutually exclusive in an actual physical environment. It could be implemented separately.

Describing each component, the communication unit 105 may transmit/receive data with multiple sensors and the learning data generating device 200 .

Specifically, the communication unit 105 transmits detection data, 3D point cloud data, 2D image, and distance information from the radar 20, lidar 30, camera 40, and ultrasonic sensor 50 fixedly installed in the vehicle 10. can receive

To this end, the learning data collection device 100 and the multiple sensors may be directly connected to each other by cables for data transmission and reception, but are not limited thereto.

Further, the communication unit 105 may transmit the post-processing and error correction-performed sensing data, 3D point cloud data, 2D image, and distance information to the learning data generating device 200 under the control of the data providing unit 130. .

With the following configuration, the input/output unit 110 may receive a signal from a user through a user interface (UI) or output an operation result to the outside.

Specifically, the input/output unit 110 may receive input of a threshold range from a user. Here, the critical range is a size range for determining a crowd of points that can be identified as objects among points included in the sensing data or 3D point cloud data, A value determined according to the type may be input.

The input/output unit 110 may receive an input of a lidar recognition range from a user. Here, the lidar recognition range is a range in which a laser pulse emitted from the lidar 30 arrives and an object can be recognized, and a value determined according to the output or type of the lidar 30 may be input.

The input/output unit 110 may receive input of a specific possible range from the user. Here, the specific possible range is a range in which an object can be specified from a 2D image captured by the camera 40, and includes the type of object to be targeted for machine learning of artificial intelligence (AI) and the number of minimum required pixels. A value determined according to, etc. may be input.

The input/output unit 110 may receive an input of a contactable range from the user. Here, the contactable range is a range in which the vehicle 10 may have a possibility of contact with another object when it behaves within a common sense range, and a value determined according to the goal of machine learning of artificial intelligence (AI) may be input. there is.

The input/output unit 110 may receive input of a sampling table from a user. Here, the sampling table is a table in which reference information for sampling only some data from 3D point cloud data obtained by lidar and the distance between the vehicle 10 and the object are mapped to each other. The first sampling rate included in the sampling data is information about the number of 3D point cloud data to be sampled per unit time, and the second sampling rate is information about the number of points to configure one 3D point cloud data.

With the following configuration, the multi-sensor controller 115 may control the radar 20, lidar 30, camera 40, and ultrasonic sensor 50 fixedly installed in the vehicle 10.

Specifically, the multi-sensor controller 115 may receive one or more of sensing data, 3D point cloud data, 2D image, and distance information through the communication unit 105 . For example, the multi-sensor control unit 115 may receive sensing data through the communication unit 105 and may additionally receive one or more of 3D point cloud data, 2D image, and distance information according to circumstances.

Here, the sensing data is information on points at which electromagnetic waves emitted toward the driving direction of the vehicle by the radar 20 fixedly installed in the vehicle 10 are reflected. The 3D point cloud data is three-dimensional information on points obtained by reflecting laser pulses emitted around the vehicle by the LIDAR 30 fixedly installed in the vehicle 10 . The 2D image is an image captured by a camera 40 fixedly installed in the vehicle 10 . Further, the distance information is information about a distance from an object detected by the ultrasonic sensor 50 fixedly installed in the vehicle 10 .

The multi-sensor controller 115 may identify an object from the received sensing data. For example, the multi-sensor control unit 125 may identify an object from the sensing data by extracting points forming a cluster within a threshold range previously set by the input/output unit 110 from among the points included in the sensing data. there is.

The multi-sensor control unit 115 corresponds to the distance between the vehicle 10 and the object identified from the sensing data, the lidar 30 fixed to the vehicle 10 to emit laser pulses, and the camera to take a 2D image ( 40) and at least one of the ultrasonic sensor 50 to emit ultrasonic waves.

In more detail, the multi-sensor controller 115 may control the lidar 30 not to interrupt the laser pulse when no object is identified from the sensing data.

When the distance between the vehicle 10 and the object is greater than the lidar recognition range previously set through the input/output unit 110, the multi-sensor control unit 115 prevents the lidar 30 from emitting a laser pulse, or It can be controlled to emit laser pulses only with the intensity set in .

When the distance between the vehicle 10 and the object is greater than the LIDAR recognition range, the multi-sensor control unit 115 does not capture 2D images from the plurality of cameras 40 fixed to the vehicle 10, and the ultrasonic sensor 50 ) can be controlled so that it does not emit ultrasonic waves.

When the distance between the vehicle 10 and the object is within the lidar recognition range, the multi-sensor control unit 115 lengthens or shortens the laser pulse radiation period of the lidar 30 in proportion to the distance from the object. You can control it.

When the distance between the vehicle 10 and the object is within a specific possible range previously set through the input/output unit 110, the multi-sensor control unit 115 detects a 2D image by a plurality of cameras 40 fixed to the vehicle 10. can be controlled to shoot. In this case, the specific possible range may correspond to a narrower range than the LIDAR recognition range.

The multi-sensor controller 115 may differently set resolutions of 2D images to be captured by the plurality of cameras 40 in correspondence to the distance between the vehicle 10 and the object. For example, the multi-sensor controller 115 may set the resolution of a 2D image to be captured by the plurality of cameras 40 to increase as the distance between the vehicle 10 and the object decreases.

In addition, the multi-sensor control unit 115 may estimate the moving path of the object based on the time-sequential position change of the object included in the sensing data. The multi-sensor controller 115 may identify a camera that photographs a direction corresponding to the estimated movement path of the object from among the plurality of cameras 40 . In addition, the multi-sensor controller 115 may set a shorter shooting cycle of a camera that captures a direction corresponding to the moving path of the object than a camera that captures other directions.

In addition, the multi-sensor control unit 115 provides a plurality of ultrasonic sensors 50 fixed to the vehicle 10 when the distance between the vehicle 10 and the object is within a contactable range preset through the input/output unit 110. They can be controlled to emit ultrasonic waves. In this case, the possible contact range may correspond to a narrower range than the specific possible range.

With the following configuration, the data post-processing unit 120 collects detection data, 3D point cloud data, 2D image, and distance information from the radar 20, lidar 30, camera 40, and ultrasonic sensor 50 as targets. Post-processing can be done.

Specifically, the data post-processing unit 120 may receive one or more of sensing data, 3D point cloud data, 2D images, and distance information through the communication unit 105 . For example, the data post-processing unit 120 may receive sensing data and 3D point cloud data through the communication unit 105, and may additionally receive one or more of 2D images and distance information depending on circumstances.

The data post-processing unit 120 may identify an object from the received sensing data. For example, the data post-processing unit 125 may identify an object from the sensing data by extracting points forming a cluster within a threshold range previously set by the input/output unit 110 from among the points included in the sensing data. there is.

In addition, the data post-processing unit 120 selects an area to which the laser pulse of the LIDAR 30 can reach, centered on the area where the identified object is located among the coordinates included in the 3D point cloud data, as a region of interest (ROI). interest) and regions of uninterested.

The data post-processing unit 120 corresponds to the distance between the vehicle 10 and the object identified from the sensing data, the plurality of 3D point cloud data obtained by the lidar 30 and the plurality of points photographed by the camera 40 One or more of the 2D images of and distance information sensed by the ultrasonic sensor 50 may be filtered.

In more detail, the data post-processing unit 120 may identify a first sampling rate corresponding to the distance between the vehicle 10 and the object from the sampling table previously set by the input/output unit 110. . The data post-processing unit 120 may reduce the number of 3D point cloud data per unit time by sampling a plurality of 3D point cloud data according to the identified first sampling rate.

Also, the data post-processing unit 120 may identify a second sampling rate corresponding to the distance between the vehicle 10 and the object from the sampling table. The data post-processing unit 120 may reduce the number of points constituting each 3D point cloud data by sampling a plurality of 3D point cloud data according to the identified second sampling rate.

On the other hand, the data post-processing unit 120 converts 2D images captured by the camera 40 that captures the region of interest from among a plurality of 2D images to 2D images captured by the camera 40 that captures the region of interest. It can be recompressed to a lower resolution.

The data post-processing unit 120 may reduce the number of 2D images per unit time by sampling 2D images captured by the camera 40 that captures the region of interest among a plurality of 2D images.

The data post-processing unit 120 expands the size of chroma subsampling for compressing color information of 2D images captured by the camera 40 that captures the region of interest among a plurality of 2D images, and then reproduces the data post-processing unit 120. can be compressed.

Further, the data post-processing unit 120 discards the distance information detected by the ultrasonic sensor 50 when the distance between the vehicle 10 and the object is longer than a specific possible range previously set through the input/output unit 110. may be

With the following configuration, the error correcting unit 125 may correct errors according to installation positions of the radar 20, lidar 30, camera 40, and ultrasonic sensor 50 fixedly installed in the vehicle 10. .

Specifically, the error correction unit 125 may receive one or more of sensing data, 3D point cloud data, 2D images, and distance information through the communication unit 105 . For example, the data post-processing unit 120 may receive sensing data and 3D point cloud data through the communication unit 105, and may additionally receive one or more of 2D images and distance information depending on circumstances.

The error correction unit 125 may identify a first point where the radar 20 fixedly installed in the vehicle 10 emits an electromagnetic wave signal. That is, the first point means a point where the radar 20 is fixedly installed in the vehicle 10 . If a plurality of radars 20 are installed in the vehicle 10, the number of first points may also be plural.

The error correction unit 125 may identify a second point where the LIDAR 30 fixedly installed in the vehicle 10 emits a laser pulse. That is, the second point means a point where the lidar 30 is fixedly installed in the vehicle 10 . If a plurality of lidars 30 are installed in the vehicle 10, the number of second points may also be plural.

Such a first point and a second point may be set in advance according to the model and size of the vehicle 10 in which the radar 20 and the lidar 30 are fixedly installed, the learning goal of artificial intelligence (AI), and the like, , but not limited thereto.

The error correction unit 125 may correct coordinates of points included in the sensed data and coordinates of points included in the 3D point cloud data so that the identified first and second points may be recognized as the same position.

In more detail, the error correction unit 125 may identify an object from sensing data. For example, the error correction unit 125 may identify an object from the sensing data by extracting points forming a cluster within a threshold range previously set by the input/output unit 110 from among the points included in the sensing data. there is.

The error correction unit 125 corrects the coordinates of the points included in the sensed data and the coordinates of the points included in the 3D point cloud data only when an object is identified from the sensed data, and when no object is identified from the sensed data, the error corrector 125 corrects the coordinates of the points included in the sensed data The coordinates of the points included in the data and the coordinates of the points included in the 3D point cloud data can be maintained as they are.

Meanwhile, the error correction unit 125 sets a virtual standard point having three-dimensional coordinates, and then coordinates of points included in the sensing data so that the first point and the second point are recognized as being located at the reference point. and coordinates of points included in 3D point cloud data can be corrected.

For example, after setting the center point of the vehicle 10 as a reference point, the error compensating unit 125 recognizes that the first and second points are located at the reference point, so that the points included in the detected data The coordinates of points and the coordinates of points included in 3D point cloud data can be corrected.

On the other hand, the sensing data obtained by the radar 20 and the 3D point cloud data obtained by the LIDAR 30 are composed of coordinates of points where electromagnetic waves or laser pulses are reflected, and observation points (ie, observation points) through correction of coordinate values. , the first point and the second point) can be matched. However, since the 2D image captured by the camera 40 is composed of color information, observation points cannot be matched. Therefore, the error correction unit 125 provides a formula for correcting the first point of the radar 20 and the second point of the LIDAR 30 to the point where the camera 40 captures the 2D image. can do.

In more detail, the error correcting unit 125 may calculate 3D vector values between respective photographing points where the plurality of cameras 40 are installed and a reference point. The error correction unit 125 calculates a formula for correcting sensing data or 3D point cloud data so that the first point or the second point can be recognized as being located at each imaging point, based on the calculated 3D vector values. can be derived.

In addition, the error correcting unit 125 provides a 3D image regardless of whether the lidar 30 fixed to the vehicle 10 corresponds to a static lidar or a rotative lidar. It can be calibrated to handle point cloud data.

More specifically, the error compensating unit 125 is a case where the lidar 30 fixedly installed in the vehicle 10 is composed of a plurality of fixed lidar devices installed spaced apart from each other at different points in the vehicle 10, a plurality of lidar By combining the coordinates of the points included in the 3D point cloud data acquired by the device, one 3D point cloud data having a spherical shape obtained by a rotary lidar that rotates around a reference point and emits laser pulses is generated can do.

Conversely, the error correction unit 125 is configured as a rotary lidar device in which the lidar 30 fixedly installed in the vehicle 10 rotates around the second point and emits laser pulses. By dividing the coordinates of the points included in one 3D point cloud data obtained by the laser pulse into a plurality of points based on the direction in which the laser pulse was emitted, a plurality of 3D point cloud data each having a form obtained by a plurality of fixed lidar devices is obtained. can create

Meanwhile, the error correction unit 125 may correct distance values included in the distance information so that each sensing point where the plurality of ultrasonic sensors 50 are installed is recognized as being located at a reference point.

With the following configuration, the data providing unit 130 may provide basic data that can be used for machine learning of artificial intelligence (AI) to the learning data generating device 200 .

Specifically, the data providing unit 130 provides sensing data, 3D point cloud data, After the 2D image and distance information are collected, and the collected sensing data, 3D point cloud data, 2D image, and distance information are post-processed by the data post-processing unit 120, the post-processed sensing data, 3D point cloud data, 2D image, and When the distance information is corrected by the error correction unit 125, the corrected sensing data, 3D point cloud data, 2D image, and distance information may be transmitted to the learning data generating device 200 through the communication unit 105.

Hereinafter, hardware for implementing the above-described logical components of the learning data collection device 100 will be described in more detail.

4 is one of the present invention in the examples It is a hardware configuration diagram of the learning data collection device according to FIG.

As shown in FIG. 4, the learning data collection device 100 includes a processor (150), a memory (155), a transceiver (160), an input/output device (165), and a data bus. (Bus, 170) and storage (Storage, 175) can be configured.

The processor 150 may implement the operation and function of the learning data collection device 100 based on instructions according to the software 180a in which the method according to the embodiments of the present invention is resident in the memory 155. . Software 180a in which a method according to embodiments of the present invention is implemented may be loaded in the memory 155 . The transceiver 160 may transmit and receive data to and from the radar 20 , lidar 30 , camera 40 , ultrasonic sensor 50 , and learning data generating device 200 . The input/output device 165 may receive data required for operation of the learning data collection device 100 and output collected sensing data, 3D point cloud data, 2D image, and distance information. The data bus 170 is connected to the processor 150, the memory 155, the transceiver 160, the input/output device 165, and the storage 175, and is a movement path for transferring data between each component. role can be fulfilled.

The storage 175 stores an application programming interface (API), a library file, a resource file, etc. necessary for the execution of the software 180a in which the method according to the embodiments of the present invention is implemented. can be saved The storage 175 may store software 180b in which a method according to embodiments of the present invention is implemented. Also, the storage 175 may store information necessary for performing a method according to embodiments of the present invention.

According to an embodiment of the present invention, the software 180a, 180b for implementing a learning data collection method using a laser preview resident in the memory 155 or stored in the storage 175 is provided by the processor 150 in the vehicle 10 ) receiving sensing data from a radar 20 fixedly installed through the transceiver 160, identifying an object from the received sensing data by a processor 150, and identifying the identified object by a processor 150 Corresponding to the distance from, it may be a computer program recorded on a recording medium to execute the step of controlling the lidar 30 that is fixedly installed in the vehicle and emits laser pulses.

According to another embodiment of the present invention, the software (180a, 180b) for implementing a data processing method resident in the memory 155 or stored in the storage 175 is a radar in which the processor 150 is fixed to the vehicle 10 Receiving the sensing data obtained by (20) and a plurality of 3D point cloud data obtained by the LIDAR 30 fixedly installed in the vehicle 10 through the transceiver 160, the processor 150 Identifying an object from the received sensing data, and the processor 150 corresponding to the distance between the vehicle 10 and the object, a plurality of 3D point cloud data acquired by the lidar 30 It may be a computer program recorded on a recording medium to execute the step of filtering them.

According to another embodiment of the present invention, the software (180a, 180b) for implementing the error correction method resident in the memory 155 or stored in the storage 175 is a processor 150 fixedly installed in the vehicle 10 Receiving sensing data obtained by the radar 20 and 3D point cloud data obtained by the LIDAR 30 fixedly installed in the vehicle 10 through the transceiver 160, wherein the processor 150 performs the Identifying a first point where the radar 20 emits an electromagnetic wave signal and identifying a second point where the lidar 30 emits a laser pulse, and the processor 150 determines the first point and the It may be a computer program recorded on a recording medium to execute the step of correcting the coordinates of the points included in the sensed data and the coordinates of the points included in the 3D point cloud data so that the second points are recognized as the same position. there is.

More specifically, the processor 150 may include an Application-Specific Integrated Circuit (ASIC), another chipset, a logic circuit, and/or a data processing device. The memory 155 may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and/or other storage devices. The transceiver 160 may include a baseband circuit for processing wired/wireless signals. The input/output device 165 includes an input device such as a keyboard, a mouse, and/or a joystick, and a Liquid Crystal Display (LCD), an Organic LED (OLED), and/or a liquid crystal display (LCD). Alternatively, an image output device such as an active matrix OLED (AMOLED) may include a printing device such as a printer or a plotter.

When the embodiments included in this specification are implemented as software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described functions. A module may reside in memory 155 and be executed by processor 150 . The memory 155 may be internal or external to the processor 150 and may be connected to the processor 150 by various well-known means.

Each component shown in FIG. 4 may be implemented by various means, eg, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), FPGAs ( Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded. Here, the recording medium may include program commands, data files, data structures, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disks), floptical It includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, such as a floptical disk, and ROM, RAM, flash memory, and the like. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. These hardware devices may be configured to operate as one or more pieces of software to perform the operations of the present invention, and vice versa.

Hereinafter, features of the artificial intelligence learning system according to various embodiments of the present invention as described above will be described in detail with reference to the drawings.

Fig. 5 degree 7 is one of the present invention in the examples It is an exemplary view for explaining a process of controlling multiple sensors according to the

5 to 7, the learning data collection device 100 according to an embodiment of the present invention includes a radar 20, a lidar 30, a camera 40 and The ultrasonic sensor 50 may be controlled.

Specifically, the learning data collection device 100 receives one or more of sensing data, 3D point cloud data, 2D image, and distance information from the radar 20, lidar 30, camera 40, and ultrasonic sensor 50 can do.

Here, the sensing data is information on points at which electromagnetic waves emitted toward the driving direction of the vehicle by the radar 20 fixedly installed in the vehicle 10 are reflected. The 3D point cloud data is three-dimensional information on points obtained by reflecting laser pulses emitted around the vehicle by the LIDAR 30 fixedly installed in the vehicle 10 . The 2D image is an image captured by a camera 40 fixedly installed in the vehicle 10 . Further, the distance information is information about a distance from an object detected by the ultrasonic sensor 50 fixedly installed in the vehicle 10 .

The learning data collection device 100 may identify the object 60 from the received sensing data. For example, the learning data collection apparatus 100 may identify the object 60 from the sensing data by extracting points forming a cluster within a preset threshold range from points included in the sensing data.

The learning data collection device 100 corresponds to the distance d between the vehicle 10 and the object 60, and is fixedly installed in the vehicle 10 to radiate laser pulses, LiDAR 30, and to capture 2D images. At least one of the camera 40 and the ultrasonic sensor 50 to emit ultrasonic waves may be controlled.

In more detail, in the learning data collection device 100, when the distance d between the vehicle 10 and the object 60 is greater than the lidar recognition range 31, the lidar 30 does not emit laser pulses. Alternatively, the laser pulse may be controlled to be emitted only with a preset intensity. In addition, in the learning data collection device 100, when the distance d between the vehicle 10 and the object 60 is greater than the lidar recognition range 31, the plurality of cameras 40 do not capture 2D images, , the ultrasonic sensor 50 may be controlled not to emit ultrasonic waves.

When the distance d between the vehicle 10 and the object 60 is within the lidar recognition range 31, the learning data collection device 100 is a lidar in proportion to the distance d from the object 60. The laser pulse radiation period of (30) can be controlled to be long or short.

When the distance d between the vehicle 10 and the object 60 is within a specific possible range 41, the learning data collection device 100 generates a 2D image by a plurality of cameras 40 fixed to the vehicle 10. can be controlled to shoot. In this case, the specific possible range 41 may correspond to a narrower range than the lidar recognition range 31 .

In addition, the learning data collection apparatus 100 may set different resolutions of 2D images to be captured by the plurality of cameras 40 in correspondence to the distance d between the vehicle 10 and the object 60 . The learning data collection device 100 estimates the movement path of the object 60 based on the time-series position change of the object 60 included in the sensing data, and photographs a direction corresponding to the estimated movement path of the object 60. It is possible to identify a camera that captures the object 60 and set a shorter shooting cycle of a camera that shoots a direction corresponding to the moving path of the object 60 than a camera that shoots another direction.

When the distance d between the vehicle 10 and the object 60 is within the contactable range 51, the learning data collection device 100 uses a plurality of ultrasonic sensors 50 fixedly installed in the vehicle 10 to transmit ultrasonic waves. can be controlled to fire. In this case, the possible contact range 51 may correspond to a narrower range than the specific possible range 41 .

Therefore, the learning data collection apparatus 100 according to an embodiment of the present invention controls the data collection period of multiple sensors or the quality of data to be collected, so that among data for machine learning of artificial intelligence (AI), the importance is relatively low. data can be reduced.

8 and 9 are one of the present invention in the examples About the data collected according to to post-process It is an example diagram to explain the process.

8 and 9, the learning data collection device 100 according to an embodiment of the present invention includes a radar 20, a lidar 30, a camera 40 and Post-processing may be performed on the sensing data, 3D point cloud data, 2D image, and distance information collected from the ultrasonic sensor 50 .

Specifically, the learning data collection apparatus 100 detects sensing data, 3D point cloud data, and 2D images from the radar 20, lidar 30, camera 40, and ultrasonic sensor 50. and distance information may be received.

The learning data collection device 100 may identify the object 60 from the received sensing data. For example, the learning data collection apparatus 100 may identify the object 60 from the sensing data by extracting points forming a cluster within a preset threshold range from points included in the sensing data.

The learning data collection apparatus 100 sets the region to which the laser pulse of the lidar 30 can reach, centered on the region where the identified object 60 is located among the coordinates included in the 3D point cloud data, as the region of interest ( ROI) and non-interest regions.

The learning data collection apparatus 100 corresponds to the distance between the vehicle 10 and the object 60, a plurality of 3D point cloud data obtained by the lidar 30, and a plurality of points photographed by the camera 40. One or more of 2D images and distance information sensed by the ultrasonic sensor 50 may be filtered.

More specifically, the learning data collection device 100 identifies a first sampling rate corresponding to the distance between the vehicle 10 and the object 60 from the sampling table, and generates a plurality of 3D images according to the identified first sampling rate. The number of 3D point cloud data per unit time may be reduced by sampling the point cloud data.

In addition, the learning data collection apparatus 100 identifies a second sampling rate corresponding to the distance between the vehicle 10 and the object 60 from the sampling table, and generates a plurality of 3D point cloud data according to the identified second sampling rate. It is possible to reduce the number of points constituting each 3D point cloud data by sampling them.

On the other hand, the learning data collection apparatus 100 selects the 2D images captured by the camera 40 that captures the region of interest among a plurality of 2D images captured by the plurality of cameras 40 as a region of interest (ROI). The 2D images captured by the camera 40 may be recompressed to a resolution lower than that of the 2D images.

The training data collection apparatus 100 may reduce the number of 2D images per unit time by sampling 2D images captured by the camera 40 that captures a region of interest among a plurality of 2D images.

The learning data collection apparatus 100 may recompress after expanding the size of chroma subsampling for compressing color information of 2D images captured by the camera 40 that captures a region of non-interest among a plurality of 2D images. there is.

Also, when the distance between the vehicle 10 and the object 60 is greater than a specific possible range, the learning data collection device 100 may discard distance information detected by the ultrasonic sensor 50 .

Therefore, the learning data collection apparatus 100 according to an embodiment of the present invention filters data collected by multiple sensors to reduce data of relatively low importance among machine learning data of artificial intelligence (AI). can As a result, by reducing the amount of data of relatively low importance, it is possible to lower the burden of data processing for machine learning of artificial intelligence (AI).

Fig. 10 degree 12 is one of the present invention in the examples the error of multiple sensors according to correcting It is an example diagram to explain the process.

10 to 12, the learning data collection device 100 according to an embodiment of the present invention includes a radar 20, a lidar 30, a camera 40 and An error according to the installation position of the ultrasonic sensor 50 may be corrected.

Specifically, the learning data collection apparatus 100 detects sensing data, 3D point cloud data, and 2D images from the radar 20, lidar 30, camera 40, and ultrasonic sensor 50. and distance information may be received.

The learning data collection device 100 may identify a first point 25 from which the radar 20 fixedly installed in the vehicle 10 emits an electromagnetic wave signal. That is, the first point 25 means a point where the radar 20 is fixedly installed in the vehicle 10 . If a plurality of radars 20 are installed in the vehicle 10, the number of first points 25 may also be plural.

The learning data collection device 100 may identify a second point 35 where the LIDAR 30 fixedly installed in the vehicle 10 emits a laser pulse. That is, the second point 35 means a point where the LIDAR 30 is fixedly installed in the vehicle 10 . If a plurality of lidars 30 are installed in the vehicle 10, the number of second points 35 may also be plural.

The learning data collection device 100 determines the coordinates of the points included in the sensing data and the points included in the 3D point cloud data so that the identified first point 25 and the second point 35 can be recognized as the same position. Coordinates can be corrected. More specifically, after setting a virtual standard point having three-dimensional coordinates, the learning data collection device 100 recognizes that the first point 25 and the second point 35 are located at the reference point. The coordinates of the points included in the sensing data and the coordinates of the points included in the 3D point cloud data may be corrected as much as possible.

On the other hand, the learning data collection apparatus 100 calculates 3D vector values between each of the photographing points 45 where the plurality of cameras 40 are installed and the reference point, and based on the calculated 3D vector values, An equation capable of correcting sensed data or 3D point cloud data may be derived so that the first point 25 or the second point 35 may be recognized as being located at each imaging point 45 .

On the other hand, when the learning data collection device 100 is composed of a plurality of fixed lidar devices installed at different points in the vehicle 10 and spaced apart from each other in the lidar 30 fixed to the vehicle 10, a plurality of lidar devices Combining the coordinates of the points included in the 3D point cloud data acquired by can

Conversely, the learning data collection device 100 is a rotary lidar device when the lidar 30 fixed to the vehicle 10 rotates around the second point and emits laser pulses. The coordinates of the points included in one 3D point cloud data obtained by are separated into a plurality of points based on the direction in which the laser pulse was emitted, and a plurality of 3D point cloud data each having a form obtained by a plurality of fixed lidar devices. can also create

Therefore, the learning data collection device 100 according to an embodiment of the present invention can directly apply the location of an object recognized by a specific sensor to data obtained from other sensors by correcting errors according to the installation positions of multiple sensors. there will be As a result, it is possible to integrate and manage annotation work results for data individually obtained by multiple sensors.

Hereinafter, operations of the learning data collection apparatus 100 according to various embodiments of the present invention as described above will be described in detail with reference to the drawings.

13 is one of the present invention in the examples to explain data collection methods according to is a flowchart .

Referring to FIG. 13, the learning data collection apparatus 100 according to an embodiment of the present invention detects data and 3D point cloud data from the radar 20, lidar 30, camera 40, and ultrasonic sensor 50. , 2D image and distance information may be received (S100).

The learning data collection device 100 identifies an object from the received sensing data, and is fixedly installed in the vehicle 10 to emit laser pulses in response to the distance between the identified object and the vehicle 10 (30). ), at least one of the camera 40 to capture a 2D image and the ultrasonic sensor 50 to emit ultrasonic waves (S200).

A detailed description of the process of controlling the lidar 30, the camera 40, and the ultrasonic sensor 50 by the learning data collection device 100 is the same as that described with reference to FIGS. 5 to 7, so it will not be described repeatedly. .

The learning data collection device 100 performs post-processing on detection data, 3D point cloud data, 2D image, and distance information received from the radar 20, lidar 30, camera 40, and ultrasonic sensor 50. It can be performed (S300).

A detailed description of the process of post-processing the sensing data, 3D point cloud data, 2D image, and distance information by the learning data collection device 100 is the same as that described with reference to FIGS. 8 and 9, so it will not be described repeatedly.

The learning data collection device 100 may correct errors according to installation positions of the radar 20, lidar 30, camera 40, and ultrasonic sensor 50 (S400).

The learning data collection device 100 detects errors in detection data, 3D point cloud data, 2D image, and distance information according to the installation positions of the radar 20, lidar 30, camera 40, and ultrasonic sensor 50. Since the detailed description of the correction process is the same as that described with reference to FIGS. 10 to 12, it will not be described repeatedly.

The learning data collection device 100 may transmit the corrected sensing data, 3D point cloud data, 2D image, and distance information to the learning data generating device 200 (S500).

As described above, although preferred embodiments of the present invention have been disclosed in the present specification and drawings, it is in the technical field to which the present invention belongs that other modified examples based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein. It is self-evident to those skilled in the art. In addition, although specific terms have been used in the present specification and drawings, they are only used in a general sense to easily explain the technical content of the present invention and help understanding of the present invention, but are not intended to limit the scope of the present invention. Accordingly, the foregoing detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be selected by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (10)

1. As a method of collecting data for machine learning of artificial intelligence (AI) that can be used for autonomous driving,

   receiving, by a learning data collection device, sensing data from a radar fixed to a vehicle;

   identifying, by the learning data collection device, an object from the received sensing data; and

   Learning using laser preview, comprising controlling, by the learning data collection device, a lidar fixed to the vehicle to emit laser pulses, corresponding to the distance between the vehicle and the identified object. Data collection method.
2. According to claim 1,

   The detection data is information on points at which electromagnetic wave signals emitted by the radar toward the driving direction of the vehicle are reflected,

   Identifying the object

   A learning data collection method using laser preview, characterized in that for identifying the object by extracting points forming a crowd within a preset threshold range from points included in the sensing data.
3. The method of claim 2, wherein controlling the lidar

   Characterized in that when the distance between the vehicle and the object is longer than the preset lidar recognition range, the lidar does not emit the laser pulse or controls to emit the laser pulse only with a preset intensity , Learning data collection method using laser preview.
4. The method of claim 3, wherein controlling the lidar

   When the distance between the vehicle and the object is within the lidar recognition range, the laser pulse emission period of the lidar is lengthened or shortened in proportion to the distance from the object. Learning data collection method using Preview.
5. The method of claim 3, wherein controlling the lidar

   Controlling a plurality of cameras fixedly installed in the vehicle to capture a 2D image when the distance between the amount and the object is within a predetermined specific possible range,

   Characterized in that the specific possible range corresponds to a narrower range than the lidar recognition range, learning data collection method using laser preview.
6. The method of claim 5, wherein controlling the lidar

   In response to the distance between the vehicle and the object, the method of collecting learning data using laser preview, characterized in that the resolution of the 2D image to be captured by the plurality of cameras is set differently.
7. The method of claim 6, wherein controlling the lidar

   A movement path of the object is estimated based on a time-sequential position change of the object included in the sensing data, and a shooting period of a camera that photographs a direction corresponding to the estimated movement path among the plurality of cameras is set in a different direction. Learning data collection method using laser preview, characterized in that it is set shorter than the shooting camera.
8. The method of claim 5, wherein controlling the lidar

   When the distance between the vehicle and the object is within a preset contactable range, a plurality of ultrasonic sensors fixed to the vehicle are controlled to emit ultrasonic waves.
9. The method of claim 2, wherein controlling the lidar

   Characterized in that, when no object is identified from the sensing data, the LIDAR is controlled not to emit the laser pulse, learning data collection method using laser preview.
10. memory;

    transceiver; and

    In combination with a computing device configured to include a processor for processing instructions resident in the memory,

    receiving, by the processor, sensing data from a radar fixed to the vehicle through the transceiver;

    identifying, by the processor, an object from the received sensing data; and

    A computer program recorded on a recording medium that causes the processor to execute a step of controlling a lidar that is fixedly installed in the vehicle and emits laser pulses in response to the distance from the identified object.

Images