KR102613162B1 - Method for annotation for 3D point cloud data, and computer program recorded on record-medium for executing method thereof - Google Patents

Abstract

The present invention proposes an annotation method for specifying objects included in 3D point cloud data acquired by LIDAR in relation to data annotation work for machine learning artificial intelligence. The method includes an annotation device receiving 3D point group data collected from a lidar, the annotation device setting the 3D point group data to at least one view point, and , generating a plurality of guide points for specifying an object at each of the set at least one view point, and the annotation device creates an object based on the selected plurality of guide points. It may include the step of performing annotation to specify.

Description

An annotation method for 3D point cloud data and a computer program recorded on a recording medium to execute the same {Method for annotation for 3D point cloud data, and computer program recorded on record-medium for executing method thereof}

The present invention relates to the processing of artificial intelligence (AI) learning data. More specifically, in relation to the annotation task of data for machine learning artificial intelligence (AI), an annotation method for specifying objects included in 3D point cloud data acquired by lidar and the same It relates to a computer program recorded on a recording medium for execution.

Artificial intelligence (AI) refers to a technology that artificially implements some or all of human learning, reasoning, and perception abilities using computer programs. In relation to artificial intelligence (AI), machine learning refers to learning to optimize parameters with given data using a model composed of multiple parameters. According to the type of learning data, machine learning is divided into supervised learning, unsupervised learning, and reinforcement learning.

In general, the design of data for artificial intelligence (AI) learning progresses through the stages of data structure design, data collection, data purification, data processing, data expansion, and data verification.

To explain each step in more detail, the design of the data structure is accomplished through the definition of ontology and classification system. Data collection is done through direct photography, web crawling, or through associations/professional organizations. Data cleaning is accomplished by removing duplicate data from the collected data and de-identifying personal information. Data processing is accomplished by performing annotation and entering metadata. Data expansion is accomplished by performing ontology mapping and supplementing or expanding the ontology as needed. In addition, data verification is accomplished by verifying the validity according to the set target quality using various verification tools.

Meanwhile, automatic driving of a vehicle refers to a system that allows the vehicle to drive by making its own decisions. As such, autonomous driving can be divided into progressive stages from non-automation to full automation, depending on the degree to which the system participates in driving and the degree to which the driver controls the vehicle. In general, the stages of autonomous driving are divided into six levels classified by SAE (Society of Automotive Engineers) International. According to the six levels classified by the International Association of Automotive Engineers, level 0 is non-automation, level 1 is driver assistance, level 2 is partial automation, level 3 is conditional automation, level 4 is high automation, and level 5. The step is fully automated.

Autonomous driving is performed through mechanisms of perception, localization, path planning, and control. Additionally, various companies are developing to implement recognition and path planning in autonomous driving mechanisms using artificial intelligence (AI).

Recently, in addition to the data used for artificial intelligence (AI) machine learning of autonomous driving as described above, 3D point cloud data acquired from lidar in various fields has been used for artificial intelligence machine learning. The demand for this is increasing.

In order to process 3D point cloud data obtained from LIDAR, annotation work must be performed by effectively grouping the points corresponding to each object in 3D space.

Conventionally, annotation work on 3D point cloud data was performed by the operator specifying a 2D bounding box corresponding to the object at a specific view point.

However, as annotation work was performed through a 2D bounding box at a specific point in 3D space, there was a problem in that noise points that did not correspond to the object to be specified were selected.

Republic of Korea Patent Publication No. 10-2230144, ‘Artificial intelligence deep learning target detection and speed potential field algorithm-based obstacle avoidance and autonomous driving method and device’, (announced on March 15, 2021)

One purpose of the present invention is to provide an annotation method for specifying objects included in 3D point cloud data acquired by LIDAR in relation to annotation work on data for machine learning artificial intelligence.

Another object of the present invention is to use a computer recorded on a recording medium to execute an annotation method for specifying objects included in 3D point cloud data acquired by LiDAR, in relation to data annotation work for machine learning artificial intelligence. To provide a program.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

In order to achieve the technical challenges described above, the present invention proposes an annotation method for specifying objects included in 3D point cloud data acquired by LIDAR in relation to annotation work on data for machine learning artificial intelligence. do. The method includes an annotation device receiving 3D point group data collected from a lidar, the annotation device setting the 3D point group data to at least one view point, and , generating a plurality of guide points for specifying an object at each of the set at least one view point, and the annotation device creates an object based on the generated plurality of guide points. It may include the step of performing annotation to specify.

Specifically, generating the guide points includes generating at least one point board, which is a plane crossing the object, from the 3D point cloud data, and specifying the object within the at least one point board. It is characterized in that it includes the step of selecting a plurality of guide points for.

In addition, the point board is characterized in that it is a coordinate plane in a 3D orthogonal coordinate system preset in the 3D point cloud data.

In addition, in the step of creating the point board, when a specific point of the object is selected under the control of an operator performing annotation, the 3D Cartesian coordinate system is created with the selected point as the origin, and the created 3D Cartesian coordinate system is It is characterized in that the coordinate plane created based on each pair of coordinate axes in is generated as a point board.

The step of generating the point board is characterized by generating the at least one point board, and generating a plurality of additional point boards that are parallel to each point board and spaced apart at a predetermined interval across the object.

The step of generating the point board may include generating a plurality of additional point boards by moving the at least one point board in parallel under the control of the operator.

The step of generating the point board includes identifying an outline of the object in each of the generated at least one point board, generating a virtual line crossing the object, and generating a virtual line having the largest width among the generated virtual lines. Characterized by setting the size of each of the at least one point board created based on a line.

The step of selecting the guide points includes selecting a plurality of guide points for specifying the object in each of the at least one point boards, identifying an outline of the object in each of the at least one point boards, and The plurality of guide points are spaced apart from the identified outline by a preset distance.

The step of selecting the guide points includes identifying an outline of the object in each of the at least one point board, automatically generating a plurality of guide points spaced at regular intervals along the outline of the identified object, and automatically generating a plurality of guide points spaced apart from each other at regular intervals along the outline of the identified object. The plurality of guide points created are spaced apart from the outline of by a preset distance.

The step of performing the annotation is characterized by dividing the space by connecting the guide points into triangles, and performing 3D modeling based on the triangles created by dividing the triangles so that the minimum value of the interior angle is maximum.

The step of performing the annotation identifies a point cloud corresponding to the object based on the density of the point cloud in the 3D point cloud data, extracts a point located outside the 3D modeled area from among the identified point cloud, and extracts the point cloud. It is characterized by re-performing 3D modeling based on the point made.

The step of performing the annotation is characterized by creating a guide point at a point located outside the 3D modeled area and re-performing the 3D modeling based on the generated guide point.

In order to achieve the technical task described above, the present invention implements an annotation method to specify objects included in 3D point cloud data acquired by LIDAR in relation to annotation work on data for machine learning artificial intelligence. To do this, we propose a computer program recorded on a recording medium. The computer program may be combined with a computing device that includes a memory and a processor that processes instructions resident in the memory. And, the computer program includes the steps of the processor receiving 3D point group data collected from a lidar, the processor converting the 3D point group data into at least one view point. setting and generating a plurality of guide points for specifying an object at each of the at least one view point, and the processor creates an object based on the selected plurality of guide points. It may be a computer program recorded on a recording medium to execute the step of performing annotation to specify.

Specific details of other embodiments are included in the detailed description and drawings.

According to embodiments of the present invention, by creating a guide point for object specification in the 3D space of 3D point cloud data and performing annotation, it is possible to precisely specify the object and accurately specify the points corresponding to the object. can do.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description of the claims.

1 and 2 are configuration diagrams showing an artificial intelligence learning system according to various embodiments of the present invention.
Figure 3 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.
Figure 4 is a hardware configuration diagram of an annotation device according to an embodiment of the present invention.
Figure 5 is a flowchart for explaining an annotation method according to an embodiment of the present invention.
6 to 10 are exemplary diagrams for explaining an annotation method according to an embodiment of the present invention.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, the technical terms used in this specification, unless specifically defined in a different way in this specification, should be interpreted as meanings generally understood by those skilled in the art in the technical field to which the present invention pertains, and are not overly comprehensive. It should not be interpreted in a literal or excessively reduced sense. Additionally, if the technical terms used in this specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted according to the definition in the dictionary or according to the context, and should not be interpreted in an excessively reduced sense.

Additionally, as used herein, singular expressions include plural expressions, unless the context clearly dictates otherwise. In this application, terms such as “consists of” or “have” should not be construed as necessarily including all of the various components or steps described in the specification, and only some of the components or steps are included. It may not be possible, or it should be interpreted as including additional components or steps.

Additionally, terms including ordinal numbers, such as first, second, etc., used in this specification may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component without departing from the scope of the present invention.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected to or connected to the other component, but other components may also exist in between. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of the reference numerals, and duplicate descriptions thereof will be omitted. Additionally, when describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the attached drawings. The spirit of the present invention should be construed as extending to all changes, equivalents, or substitutes other than the attached drawings.

Meanwhile, in order to process 3D point cloud data obtained from LIDAR, annotation work must be performed by effectively grouping the points corresponding to each object in 3D space.

Conventionally, annotation work on 3D point cloud data was performed by the operator specifying a 2D bounding box corresponding to the object at a specific view point.

However, as annotation work was performed through a 2D bounding box at a specific point in 3D space, there was a problem in that even points that did not correspond to the object to be specified were selected.

In order to overcome these limitations, the present invention seeks to propose various means for specifying objects included in 3D point cloud data acquired by LIDAR.

1 and 2 are configuration diagrams showing an artificial intelligence learning system according to various embodiments of the present invention.

As shown in Figure 1, the artificial intelligence learning system according to an embodiment of the present invention includes a learning data generating device 100 and one or more annotation devices (200-1, 200-2, ..., 200-n; 200). , may be configured to include a learning data verification device 300 and a learning device 400.

In addition, as shown in Figure 2, the artificial intelligence learning system according to another embodiment of the present invention includes one or more annotation devices (200-a, 200-b, ..., 200-m; 200) and a learning data verification device ( A plurality of groups (Group-a, Group-b..., Group-m) consisting of 300-a, 300-b, ..., 300-m; 300), a learning data generating device 100, and a learning device 400 It may be configured to include.

Since the components of the artificial intelligence learning system according to various embodiments of this kind merely represent functionally distinct elements, two or more components are integrated and implemented with each other in the actual physical environment, or one component is implemented in the actual physical environment. may be implemented separately from each other.

In terms of each component, the learning data generating device 100 is a device that can be used to design and generate data for machine learning artificial intelligence (AI) that can be used for autonomous driving of a vehicle.

In this way, the learning data generation device 100 is basically a separate device from the learning data verification device 300, but in an actual physical environment, the learning data generation device 100 and the learning data verification device 300 are one device. It can also be integrated and implemented.

Specifically, the learning data design device 100 may receive the properties of a project related to artificial intelligence (AI) learning from the learning device 400. The learning data design device 100 designs a data structure for artificial intelligence (AI) learning, purifies the collected data, processes the data, expands the data, and Verification can be performed.

First, the learning data design device 100 can design a data structure for artificial intelligence (AI) learning. For example, the learning data design device 100 creates an ontology for artificial intelligence (AI) learning and a classification system for data for artificial intelligence (AI) learning based on the user's control and the properties of the received project. can be defined.

The learning data design device 100 can collect data for artificial intelligence (AI) learning based on the designed data structure. To this end, the learning data design device 100 may receive 3D point cloud data from the outside, collect 3D point cloud data by performing web crawling, or download 3D point cloud data from a device of an external organization. .

Here, the 3D point cloud data is data acquired by a lidar fixed to a vehicle. Lidar fixed to a vehicle can emit laser pulses, detect light reflected by objects located around the vehicle, and generate 3D point cloud data corresponding to a 3D image of the vehicle's surroundings. In other words, the point cloud that constitutes 3D point cloud data refers to a set of points that reflect laser pulses fired into 3D space by LIDAR.

The learning data generating device 100 may remove duplicate or extremely similar data from the collected 3D point cloud data.

The learning data generating device 100 may distribute and transmit the collected and refined 3D point cloud data to a plurality of annotation devices 200. In this case, the learning data generating device 100 may distribute 3D point cloud data according to a pre-allocated amount to the operator (i.e., labeler) of the annotation device 200.

The learning data generating device 100 may receive annotation work results directly from the annotation device 200, or may receive annotation work results and inspection results from the learning data verification device 300.

The learning data generating device 100 may generate artificial intelligence (AI) learning data by packaging the received annotation work results. Additionally, the learning data generating device 100 may transmit the generated artificial intelligence (AI) learning data to the learning device 400 .

Having these characteristics, the learning data generating device 100 can transmit and receive data with the annotation device 200, the learning data verification device 300, and the learning device 400, and perform calculations based on the transmitted and received data. Any device that exists can be permitted.

For example, the learning data generating device 100 may be any one of fixed computing devices such as a desktop, workstation, or server, but is not limited thereto.

The annotation device 200 is a device that can be used to perform annotation work on images provided from the learning data generating device 100.

Specifically, the annotation task may include the process of specifying an object within 3D point cloud data and inputting class information including attribute information of the specified object.

In particular, the annotation device 200 according to an embodiment of the present invention receives 3D point group data collected from lidar and converts the 3D point group data into at least one view point. It is possible to set and create a plurality of guide points for specifying an object at each of at least one set view point.

Additionally, the annotation device 200 may perform annotation to specify an object based on a plurality of selected guide points.

Meanwhile, specific details regarding the annotation device 200 will be described later with reference to FIGS. 3 and 4.

As such, the annotation device 200 can be any device that can transmit and receive data with the learning data generating device 100 or the learning device 400 and perform calculations using the transmitted and received data.

For example, the annotation device 200 may be any one of fixed computing devices such as a desktop, workstation, or server, but is not limited thereto, and may be a smart phone, Devices such as laptops, tablets, phablets, Portable Multimedia Players (PMPs), Personal Digital Assistants (PDAs), or E-book readers. It may be any one of the portable computing devices.

With the following configuration, the learning data verification device 300 is a device that can be used to verify artificial intelligence (AI) learning data. In other words, the learning data verification device 300 verifies whether the annotation work product generated by the annotation device 200 meets the preset target quality or whether the annotation work product is valid for artificial intelligence (AI) learning. It is a device that can do this.

Specifically, the learning data verification device 300 may receive annotation work results from the annotation device 200. Here, the annotation work result may include coordinates of an object specified from 3D point cloud data and metadata about the image or object. The metadata of the annotation work result includes the category of the specified object, the rate at which the object is truncated by the angle of view of the 2D image (truncation), the rate at which the object is obscured by another object or object (occlusion), and the tracking ID of the object. identifier), the time when the image was taken, weather conditions on the day the image was taken, etc., but are not limited to this. The result of such annotation work may have a JSON (Java Script Object Notation) file format, but is not limited to this.

The learning data verification device 300 can inspect the received annotation work results. To this end, the learning data verification device 300 may inspect the annotation work results using a script. Here, the script is a code for verifying whether the annotation work results meet the preset target quality or whether the data is valid.

Additionally, the learning data verification device 300 may transmit the annotation work results and inspection results received from the annotation devices 200 to the learning data generation device 100.

The learning data verification device 300, which has the characteristics described above, can transmit and receive data with the annotation device 200 and the learning data generation device 100, and can perform operations based on the transmitted and received data. Any device may be permitted. For example, the learning data verification device 300 may be any one of fixed computing devices such as a desktop, workstation, or server, but is not limited thereto.

In the following configuration, the learning device 400 is a device that can be used to machine learn artificial intelligence (AI) through 3D point cloud data provided from the learning data generating device 100.

As such, the learning device 400 can be any device that can transmit and receive data with the learning data generating device 100 and perform calculations using the transmitted and received data. For example, the learning device 400 may be any one of stationary computing devices such as a desktop, workstation, or server, but is not limited thereto.

As described above, the learning data generating device 100, one or more annotation devices 200, the learning data verification device 300, and the learning device 400 are connected directly between the devices through a secure line, a public wired communication network, or a mobile communication network. Data can be transmitted and received using a network that combines one or more of the following.

For example, public wired networks may include Ethernet, xDigital Subscriber Line (xDSL), Hybrid Fiber Coax (HFC), and Fiber To The Home (FTTH). It may be possible, but it is not limited to this. In addition, mobile communication networks include Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), High Speed Packet Access (HSPA), and Long Term Evolution. LTE) and 5th generation mobile telecommunication may be included, but are not limited thereto.

Hereinafter, the logical configuration of the annotation device according to an embodiment of the present invention will be described in detail.

Figure 3 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.

As shown in Figure 3, the annotation device 200 according to an embodiment of the present invention includes a communication unit 205, an input/output unit 210, a point creation unit 215, an object specification unit 220, and a storage unit ( 225).

Since the components of the annotation device 200 merely represent functionally distinct elements, two or more components may be implemented integrated with each other in the actual physical environment, or one component may be separated from each other in the actual physical environment. It could be implemented.

To describe each component, the communication unit 205 may transmit the annotation work result to the learning data generation device 100 or the learning data verification device 300.

Here, the work result may include the coordinates of the work result and object attribute information of the annotation set according to the operator's control. Additionally, the work result may be in a JSON file format, but is not limited thereto.

Additionally, the communication unit 250 may receive project properties, image properties, or worker properties from the learning data generating device 100.

Here, the properties of the project may include the learning purpose for the project related to artificial intelligence (AI) learning, the learning period, the properties of the object to be identified in the 3D point cloud data required for learning, annotation rules, etc., but are not limited to this. no.

The worker's attributes may include, but are not limited to, the worker's name, identification number, assigned work amount, cost according to the work, and work result evaluation.

In the following configuration, the input/output unit 210 can receive signals from the user through a user interface (UI) or output calculation results to the outside.

Specifically, the input/output unit 210 may input a signal from an operator through a user interface (UI) or output a calculated result to the outside.

Here, a worker refers to a person who performs annotation work. As such, the worker may be referred to as a user, performer, labeler, or data labeler, but is not limited thereto.

Specifically, the input/output unit 210 may output 3D point cloud data that is the target of annotation work. The input/output unit 210 can receive a control signal for annotation work from an operator.

In the following configuration, the point generator 215 sets the 3D point cloud data to at least one view point and provides a plurality of guide points for specifying an object at each of the set at least one view point. points) can be created.

Specifically, the point generator 215 may generate at least one point board, which is a plane crossing an object, from 3D point cloud data.

Here, the point board may be a coordinate plane in a 3D orthogonal coordinate system preset in 3D point cloud data.

For example, the point board can be an xy coordinate plane, a yz coordinate plane, and an xz coordinate plane in a Cartesian coordinate system that includes the x, y, and z axes.

That is, when a specific point of an object is selected according to the control of the operator performing the annotation, the point generator 215 generates a 3D Cartesian coordinate system with the selected point as the origin, and each pair of coordinate axes in the created 3D Cartesian coordinate system. The coordinate plane created based on can be created as a point board.

Additionally, the point generator 215 may generate a plurality of additional point boards by moving at least one point board in parallel under the control of the operator.

For example, the point generator 215 may receive a point located at the top of the initial object from the operator and generate a 3D Cartesian coordinate system with the selected point as the origin. Thereafter, the point generator 215 may move the point board in parallel in the z-axis direction, and finally move the point board to the bottom of the object.

However, it is not limited to this, and the point generator 215 may generate a plurality of additional point boards that are parallel to each point board and spaced at a predetermined interval across the object.

For example, the point generator 215 may receive an object selection from an operator, and when an object is selected, the point generator 215 may identify the outline of the selected object. In addition, the point generator 215 generates a point board, which is a coordinate plane, based on a preset 3D orthogonal coordinate system, and, based on the outline of the identified object, a plurality of points parallel to each point board and spaced at a certain interval across the object. Additional point boards can be created.

When creating a point board, the point creation unit 215 sets the size of the point board based on the outline of the object, thereby enabling the worker to easily input guide points on the point board.

That is, the point generator 215 identifies the outline of an object in each of at least one generated point board, generates a virtual line crossing the object, and uses the virtual line with the largest width among the generated virtual lines as a reference. You can set the size of each of at least one point board created.

Thereafter, the point generator 215 may select a plurality of guide points for specifying an object within at least one point board.

Specifically, the point generator 215 selects a plurality of guide points for specifying an object in each of at least one point board, identifies the outline of the object in each of the at least one point board, and Multiple guide points can be spaced apart by a preset distance.

In addition, the point generator 215 identifies the outline of an object in each of at least one point board, automatically generates a plurality of guide points spaced apart at regular intervals along the outline of the identified object, and automatically generates a plurality of guide points spaced apart from the outline of the identified object. The generated plurality of guide points can be spaced apart by a preset distance.

That is, when the object specification unit 220, which will be described later, performs annotation by connecting a plurality of selected guide points, if the plurality of guide points are connected without receiving sufficient guide points from the operator, the object specification unit 220, which will be described later, connects the plurality of guide points without receiving sufficient guide points from the operator, within the 3D modeled area. A problem may arise in which not all point clouds for an object are included.

Accordingly, the point generator 215 can support the point cloud of the object to be completely included within the 3D modeled area by separating the guide points selected by the operator from the object.

For example, the point generator 215 estimates the center point of the identified object, forms an virtual line connecting the guide points based on the estimated center point, and creates a guide point from the outside of the object based on the formed virtual line. They can be separated by a preset distance.

With the following configuration, the object specification unit 220 can perform annotation to specify an object based on a plurality of selected guide points.

Specifically, the object specification unit 220 connects a plurality of selected guide points, performs 3D modeling based on the point cloud of the object in the 3D point cloud data, and creates a 3D model including the point cloud of the object for specification. there is.

For example, the object specification unit 220 may divide the space by connecting guide points into triangles and perform 3D modeling based on the triangles created by dividing the space so that the minimum value of the interior angle of the triangles is maximum.

That is, the object specification unit 220 can 3D model the object through Delaunay triangulation. In other words, the object specification unit 220 forms a super triangle including all of the selected plurality of guide points, and if there are no other points within the circle containing the three points for each point in the initial triangle, We can add that triangle as a Delaunay triangle. At this time, the initial triangle can be deleted once all Delaney triangle network construction processes have been performed.

In addition, the object specification unit 220 identifies a point cloud corresponding to an object based on the density of the point cloud in the 3D point cloud data, extracts points located outside the 3D modeled area from among the identified point clouds, and sets the extracted points as the basis. You can re-perform 3D modeling.

For example, the object specification unit 220 may create a guide point at a point located outside the 3D modeled area and re-perform 3D modeling based on the generated guide point.

Hereinafter, hardware for implementing the logical components of the annotation device 200 as described above will be described in more detail.

Figure 4 is a hardware configuration diagram of an annotation device according to an embodiment of the present invention.

As shown in FIG. 4, the annotation device 200 includes a processor (Processor, 250), a memory (Memory, 255), a transceiver (260), an input/output device (265), and a data bus (Bus). , 270) and storage (Storage, 275).

The processor 250 may implement the operations and functions of the annotation device 200 based on instructions according to the software 280a in which the method according to the embodiments of the present invention is implemented, which resides in the memory 255. Software 280a implementing methods according to embodiments of the present invention may be loaded in the memory 255. The transceiver 260 can transmit and receive data with the learning data generating device 100, the learning data verification device 300, and the artificial intelligence learning device 400. The input/output device 265 can receive data required for the operation of the annotation device 200 or output learning results. The data bus 270 is connected to the processor 250, memory 255, transceiver 260, input/output device 265, and storage 275, and forms a moving path for transferring data between each component. can perform its role.

The storage 275 stores an application programming interface (API), library files, resource files, etc. necessary for executing the software 280a in which the method according to the embodiments of the present invention is implemented. You can save it. The storage 275 may store software 280b in which methods according to embodiments of the present invention are implemented. Additionally, the storage 275 may store information necessary for performing methods according to embodiments of the present invention.

According to one embodiment of the present invention, software 280a, 280b for implementing a learning method resident in memory 255 or stored in storage 275 allows the processor 250 to use 3D data collected from lidar. Receiving point cloud data (3D point group), the processor 250 sets the 3D point cloud data to at least one view point and specifies an object at each of the at least one view point. In order to execute the step of creating a plurality of guide points and the step of the processor 250 performing an annotation to specify an object based on the selected plurality of guide points, record It can be a computer program recorded on a medium.

More specifically, the processor 250 may include an application-specific integrated circuit (ASIC), another chipset, a logic circuit, and/or a data processing device. The memory 255 may include read-only memory (ROM), random access memory (RAM), flash memory, a memory card, a storage medium, and/or other storage devices. The transceiver 260 may include a baseband circuit for processing wired and wireless signals. The input/output device 265 includes input devices such as a keyboard, mouse, and/or joystick, a liquid crystal display (LCD), an organic light emitting diode (OLED), and/ Alternatively, it may include an image output device such as an active matrix OLED (AMOLED), a printing device such as a printer, a plotter, etc.

When the embodiments included in this specification are implemented as software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described function. The module resides in memory 255 and can be executed by processor 250. Memory 255 may be internal or external to processor 250 and may be coupled to processor 250 by a variety of well-known means.

Each component shown in FIG. 4 may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, an embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and FPGAs ( Field Programmable Gate Arrays), processor, controller, microcontroller, microprocessor, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded Here, the recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the recording medium may be those specifically designed and configured for the present invention, or may be known and available to those skilled in the art of computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROM (Compact Disk Read Only Memory) and DVD (Digital Video Disk), and floptical media. It includes magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, etc. Examples of program instructions may include machine language code such as that created by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Such hardware devices may be configured to operate as one or more software to perform the operations of the present invention, and vice versa.

Figure 5 is a flowchart for explaining an annotation method according to an embodiment of the present invention.

Referring to FIG. 5, in step S100, the annotation device may receive 3D point group data collected from lidar.

Next, in step S200, the annotation device may generate at least one point board, which is a plane that traverses the object from the 3D point cloud data.

Here, the point board may be a coordinate plane in a 3D orthogonal coordinate system preset in 3D point cloud data.

In other words, when the annotation device selects a specific point of an object under the control of the operator performing the annotation, it creates a 3D Cartesian coordinate system with the selected point as the origin, and creates it based on each pair of coordinate axes in the created 3D Cartesian coordinate system. The coordinate plane can be created as a point board.

At this time, the annotation device may generate at least one point board, and may generate a plurality of additional point boards that are parallel to each point board and spaced apart at a predetermined interval across the object.

However, the annotation device is not limited to this, and may create a plurality of additional point boards by moving the at least one point board in parallel under the control of the operator.

Additionally, the annotation device identifies the outline of an object in each of at least one generated point board, generates a virtual line crossing the object, and generates at least one virtual line based on the virtual line with the largest width among the generated virtual lines. You can set the size of each point board.

Next, in step S300, the annotation device may select a plurality of guide points for specifying an object within at least one point board.

Specifically, the annotation device selects a plurality of guide points for specifying an object in each of at least one point board, identifies the outline of the object in each of the at least one point board, and generates a plurality of guide points from the identified outline. Points can be spaced apart by a preset distance.

In addition, the annotation device identifies the outline of an object in each of at least one point board, automatically generates a plurality of guide points spaced at regular intervals along the outline of the identified object, and generates a plurality of guide points from the outline of the identified object. Guide points can be spaced apart by a preset distance.

And, in step S400, the annotation device may perform annotation to specify an object based on a plurality of selected guide points.

Specifically, the annotation device may connect a plurality of selected guide points, perform 3D modeling based on the point cloud of the object in 3D point cloud data, and create a 3D model including the point cloud of the object for specification.

For example, the annotation device may divide space by connecting guide points into triangles, and perform 3D modeling based on the triangles created by dividing the triangles so that the minimum value of the interior angle is maximum.

In addition, the annotation device identifies the point cloud corresponding to the object based on the density of the point cloud in the 3D point cloud data, extracts points located on the 3D modeled outskirts among the identified point clouds, and re-executes 3D modeling based on the extracted points. It can be done.

At this time, the annotation device may create a guide point at a point located on the outside of the 3D modeled area and re-perform 3D modeling based on the generated guide point.

Additionally, the annotation device can change the color of the point cloud of an object located within a 3D modeled area so that workers can easily identify it.

6 to 10 are exemplary diagrams for explaining an annotation method according to an embodiment of the present invention.

Referring to FIGS. 6 to 10 , the annotation device according to an embodiment of the present invention may generate at least one point board, which is a plane crossing an object, from 3D point cloud data.

Here, the point board may be a coordinate plane in a 3D orthogonal coordinate system preset in 3D point cloud data.

In other words, when the annotation device selects a specific point of an object under the control of the operator performing the annotation, it creates a 3D Cartesian coordinate system with the selected point as the origin, and creates it based on each pair of coordinate axes in the created 3D Cartesian coordinate system. The xy coordinate plane (a), yz coordinate plane (b), and xz coordinate plane (c) can be created as a point board.

As shown in FIG. 6, when a point board (a, b, c) is created, the annotation device selects a plurality of guide points to specify an object on at least one of the created point boards (a, b, c). You can receive it.

For example, the operator can select one of the point boards (a, b, c) while changing the view point of the 3D point cloud data. Also, when the worker selects one of the point boards (a, b, c) and designates a specific point on the selected point board, a guide point can be created on the point board.

At this time, the annotation device can move the point boards (a, b, c) in parallel according to the operator's control. That is, as shown in FIG. 7, the annotation device can move the point boards (a, b, c) in parallel in the z-axis direction under the control of the operator.

That is, the operator creates guide points by placing the point board (a) corresponding to the xy coordinate plane at the top of the object, and adds guide points by moving the point board (a) to the bottom of the object in parallel. It can be created with

Finally, as shown in FIG. 8, the operator can create guide points after placing the point board (a) at the bottom of the object.

As shown in FIG. 9, when the operator's guide point creation task is completed, the annotation device can perform annotation to specify an object based on a plurality of selected guide points.

Specifically, the annotation device can divide the space by connecting guide points into triangles so that the minimum value of the interior angles of the triangles is maximum.

And, as shown in FIG. 10, the annotation device can perform 3D modeling by deleting guide points selected by the operator and forming an external surface based on the generated triangle.

As described above, although preferred embodiments of the present invention have been disclosed in the specification and drawings, it is known in the technical field to which the present invention belongs that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein. It is self-evident to those with ordinary knowledge.

In addition, although specific terms are used in the specification and drawings, they are merely used in a general sense to easily explain the technical content of the present invention and aid understanding of the invention, and are not intended to limit the scope of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: Learning data generation device 200: Annotation device
300: Learning data verification device 400: Artificial intelligence learning device
205: communication unit 210: input/output unit
215: Point creation unit 220: Object specific unit
225: storage unit

Claims (10)

An annotation device receiving 3D point group data collected from lidar;
The annotation device sets the 3D point cloud data to at least one view point and generates a plurality of guide points for specifying an object at each of the set at least one view point. steps; and
performing, by the annotation device, an annotation to specify an object based on the plurality of guide points created; Including,
The step of creating the guide points is
generating at least one point board, which is a plane crossing the object, from the 3D point cloud data; and
A step of selecting a plurality of guide points for specifying the object within the at least one point board,
The point board is
Characterized in that it is a coordinate plane in a 3D orthogonal coordinate system preset in the 3D point cloud data,
The step of creating the point board is
When a specific point of the object is selected under the control of the operator performing annotation, the 3D Cartesian coordinate system is created with the selected point as the origin, and the 3D Cartesian coordinate system is generated based on each pair of coordinate axes in the created 3D Cartesian coordinate system. Characterized by creating a coordinate plane as a point board,
The step of creating the point board is
An annotation method, characterized in that generating the at least one point board and generating a plurality of additional point boards that are parallel to each point board and spaced at a predetermined interval across the object.
The method of claim 1, wherein the step of creating the point board is
An annotation method, characterized in that a plurality of additional point boards are created by moving the at least one point board in parallel under the control of the operator.
The method of claim 1, wherein the step of selecting the guide points is
A plurality of guide points for specifying the object in each of the at least one point boards are selected, an outline of the object in each of the at least one point boards is identified, and the plurality of guide points are selected from the identified outline. An annotation method characterized by separating points by a preset distance.
The method of claim 1, wherein the step of selecting the guide points is
The step of creating the point board is
Identifying the outline of the object in each of the generated at least one point board, generating a virtual line crossing the object, and generating at least one virtual line based on the virtual line having the largest width among the generated virtual lines. An annotation method characterized by setting the size of each point board.
The method of claim 1, wherein the step of selecting the guide points is
Identifying the outline of the object in each of the at least one point board, automatically generating a plurality of guide points spaced at regular intervals along the outline of the identified object, and automatically generating a plurality of guide points created from the outline of the identified object. An annotation method characterized by spacing guide points by a preset distance.
The method of claim 1, wherein performing the annotation includes
An annotation method characterized in that the space is divided by connecting the guide points into triangles, and 3D modeling is performed based on the triangles created by dividing the triangles so that the minimum value of the interior angle is maximum.
The method of claim 6, wherein the step of performing the annotation is
Identify a point cloud corresponding to the object based on the density of the point cloud in the 3D point cloud data, extract points located outside the 3D modeled area from among the identified point clouds, and perform 3D modeling based on the extracted points. An annotation method, characterized in that re-performing.
The method of claim 7, wherein the step of performing the annotation is
An annotation method characterized by creating a guide point at a point located outside the 3D modeled area and re-performing 3D modeling based on the created guide point.
memory;
transceiver; and
Combined with a computing device configured to include a processor that processes instructions resident in the memory,
Receiving, by the processor, 3D point group data collected from lidar;
The processor sets the 3D point cloud data to at least one view point and generates a plurality of guide points for specifying an object at each of the at least one view point. ; and
The processor performs an annotation to specify an object based on the generated plurality of guide points.
The step of creating the guide points is
generating at least one point board, which is a plane crossing the object, from the 3D point cloud data; and
A step of selecting a plurality of guide points for specifying the object within the at least one point board,
The point board is
Characterized in that it is a coordinate plane in a 3D orthogonal coordinate system preset in the 3D point cloud data,
The step of creating the point board is
When a specific point of the object is selected under the control of the operator performing the annotation, the 3D Cartesian coordinate system is created with the selected point as the origin, and the 3D Cartesian coordinate system is generated based on each pair of coordinate axes in the created 3D Cartesian coordinate system. Characterized by creating a coordinate plane as a point board,
The step of creating the point board is
A computer program recorded on a recording medium, characterized in that it generates the at least one point board and generates a plurality of additional point boards parallel to each point board and spaced at a predetermined interval across the object.
The method of claim 9, wherein the step of performing the annotation is
A computer program recorded on a recording medium, characterized in that space is divided by connecting the guide points into triangles, and 3D modeling is performed based on the triangles created by dividing the triangles so that the minimum value of the interior angle is maximum.

Images