KR20230149255A - Method for annotation using boundary designation - Google Patents

Abstract

The present invention proposes an annotation method through boundary designation that can easily specify overlapping objects included in an image. The method uses an annotation device, under operator control, to specify the outlines of a plurality of objects arranged to overlap each other contained in an image that is the target of annotation work for artificial intelligence (AI) learning. Step, the annotation device designates a boundary line between the plurality of objects within the designated outline, and the annotation device identifies each of the plurality of objects based on a plurality of areas divided based on the designated boundary line. It may include steps.

Description

Annotation method using boundary designation {Method for annotation using boundary designation}

The present invention relates to processing data for artificial intelligence (AI) machine learning. More specifically, it relates to an annotation method through boundary designation that can easily specify overlapping objects included in an image when annotating artificial intelligence (AI) learning data.

Artificial intelligence (AI) refers to a technology that artificially implements some or all of human learning, reasoning, and perception abilities using computer programs. In relation to artificial intelligence (AI), machine learning refers to learning to optimize parameters with given data using a model composed of multiple parameters. According to the type of learning data, machine learning is divided into supervised learning, unsupervised learning, and reinforcement learning.

Generally, the design of data for artificial intelligence (AI) learning progresses through the stages of data structure design, data collection, data purification, data processing, data expansion, and data verification.

To explain each step in more detail, the design of the data structure is accomplished through the definition of ontology and classification system. Data collection is done through direct photography, web crawling, or through associations/professional organizations. Data cleaning is accomplished by removing duplicate data from the collected data and de-identifying personal information. Data processing is accomplished by entering meta data and performing annotation. Data expansion is accomplished by performing ontology mapping and supplementing or expanding the ontology as needed. In addition, data verification is accomplished by verifying the validity according to the set target quality using various verification tools.

Meanwhile, automatic driving of a vehicle refers to a system that allows the vehicle to drive by making its own decisions. As such, autonomous driving can be divided into progressive stages from non-automation to full automation depending on the degree to which the system participates in driving and the degree to which the driver controls the vehicle. In general, the stages of autonomous driving are divided into six levels classified by SAE (Society of Automotive Engineers) International. According to the six levels classified by the Society of Automotive Engineers (SAE), level 0 is no automation, level 1 is driver assistance, level 2 is partial automation, and level 3 is. The levels are conditional automation, level 4 is high automation, and level 5 is full automation.

Autonomous driving of a vehicle is performed through mechanisms of perception, localization, path planning, and control. Currently, several companies are developing autonomous driving mechanisms to implement recognition and path planning using artificial intelligence (AI). And, the data used for machine learning of artificial intelligence (AI) that can be used in autonomous driving consists of a large number ranging from a few thousand to as many as millions.

Data used for machine learning of artificial intelligence (AI) that can be used for autonomous driving of vehicles is collected by various types of sensors installed in the vehicle. For example, data used for machine learning of artificial intelligence (AI) that can be used for autonomous driving of vehicles comes from lidar, cameras, radar, and ultrasonic sensors fixed to the vehicle. ) and GPS (Global Positioning System), etc., may be data acquired, photographed, or sensed, but are not limited thereto.

Generally, the collection of learning data is carried out on a project basis. At this time, the control tower of each project receives data from multiple vehicles equipped with data collection devices.

At this time, in the process of uploading data collected from multiple vehicles, there was a problem in which the same image was uploaded repeatedly from each vehicle or the same image was uploaded repeatedly from different vehicles.

In addition, the manager of each vehicle collects data from the control tower according to a guide specifying collection conditions and uploads the collected data. At this time, there was a problem in which data that did not meet the standards was randomly uploaded depending on the subjective viewpoint of the manager of each vehicle, the collection environment, and errors in the collection device.

To prevent the above problems, the control tower deploys inspectors to manually inspect the uploaded data. However, there is a need for a method to prevent unnecessary waste of resources due to deploying inspectors.

Meanwhile, the annotation work in the data processing stage is performed by identifying objects included in the image using bounding boxes, polygons, etc., and inputting attribute information of the identified objects. This kind of annotation work is also referred to as data labeling. And, the dataset corresponding to the annotation work result is produced in the form of a JSON (Java Script Object Notation) file. Since this annotation work is performed on a large number of data ranging from a few thousand to as many as millions of pieces of data, a large number of workers perform the annotation work.

Therefore, it is a difficult task to calculate the overall work cost of a project that requires numerous workers to perform numerous annotation tasks. In the past, there was a problem in that the total work cost of a project related to annotation work was calculated simply depending on the number of data to be worked on or the difficulty of the work predicted according to the intuition of the person in charge.

In addition, the polygon technique during annotation work is a method in which workers identify objects by creating a plurality of points along the outline of the object included in the image. This polygon technique has the advantage of being able to precisely select the outline of an atypical object such as a car or person, allowing the size and shape of the object to be accurately recognized.

However, when objects are placed overlapping in an image, in order to identify each object using the polygon technique, the image must be enlarged and a double point must be created at the same point of the boundary line. At this time, if a point was not created exactly at the same point of the boundary line, there was a problem that a space was created between the overlapping objects.

Republic of Korea Patent Publication No. 10-2020-0042629, ‘Touch-based annotation and image generation method and device for mobile devices for artificial intelligence learning’, (published on April 24, 2020)

One purpose of the present invention is to provide an annotation method through boundary designation that can easily specify overlapping objects included in an image when annotating artificial intelligence learning data.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the description below.

In order to achieve the technical problems described above, the present invention proposes an annotation method through boundary designation that can easily specify overlapping objects included in an image. The method uses an annotation device, under operator control, to specify the outlines of a plurality of objects arranged to overlap each other contained in an image that is the target of annotation work for artificial intelligence (AI) learning. Step, the annotation device designates a boundary line between the plurality of objects within the designated outline, and the annotation device identifies each of the plurality of objects based on a plurality of areas divided based on the designated boundary line. It may include steps.

Specifically, the step of specifying the outline is characterized in that designating the entire outline of a plurality of objects including at least a first object and a second object arranged to overlap each other, and the step of specifying the boundary line is characterized by designating the entire outline of a plurality of objects arranged to overlap each other, and the step of specifying the border line Characterized in that designating an overlapping boundary line between the first object and the second object within the boundary line, wherein the boundary line forms an outline of a portion of the first object overlapping with the second object and the second object It is characterized by forming an outline of a portion of the object that overlaps with the first object.

The step of specifying the outline is characterized by receiving a plurality of points as input along an outline including the plurality of objects from the operator, connecting the plurality of points, and generating outlines for the plurality of objects.

The step of specifying the outline includes extracting an edge of the image, identifying at least one object based on the extracted edge, receiving a plurality of objects selected from the identified objects by the operator, and extracting the edge. It is characterized by generating outlines for the plurality of selected objects based on the selected edges.

The step of specifying the outline is to select a plurality of point clouds with a certain depth among the points present inside the designated outline based on point cloud data acquired from lidar at the same time as the image. identifying, receiving a plurality of objects selected from the plurality of point clouds by the operator, and generating outlines for the selected plurality of objects based on the identified plurality of point clouds.

The step of specifying the outline includes setting a partial area including the plurality of objects as a bounding box, extracting the edge of the object from the area inside the bounding box, and dividing the object based on the extracted edge. It is characterized by distinguishing between a background and a background, deleting the background, and specifying an outline of the plurality of objects.

The step of specifying the boundary line is characterized by identifying a boundary line within the outline of the plurality of objects based on the extracted edges.

The step of specifying the boundary line is characterized in that a plurality of points located inside the outline of the plurality of objects are input from the operator, and the boundary line is created by connecting the plurality of input points.

The step of specifying the boundary line identifies a plurality of point clouds with a distance within a certain range among the points present inside the designated outline based on the point cloud data acquired from the lidar simultaneously with the image, and the distance between the plurality of point clouds is identified. It is characterized in that the boundary line is designated as a boundary line between the plurality of objects.

The step of specifying the boundary line generates groups with RGB values similar to preset values based on the RGB (Red, Green, Blue) values of pixels located inside the designated outline, and each created group is It is characterized by recognizing it as an object and creating a boundary line of the recognized object.

The step of specifying the boundary line identifies a plurality of point clouds having a certain distance within a certain range among the points present inside the designated outline based on the point cloud data acquired from the lidar simultaneously with the image, and the plurality of point clouds are identified from the operator. One object from a point cloud is selected, and the boundary line is generated based on the point cloud of the selected object.

The step of specifying the boundary line includes extracting an edge inside the designated outline, identifying at least one object based on the extracted edge, selecting one of the identified objects from the operator, and selecting the extracted object. The boundary line is generated based on an edge.

The step of specifying the boundary line generates a plurality of points having a preset interval along the created boundary line, and modifies the boundary line by moving at least one of the plurality of points under the control of the operator. Do it as

In order to achieve the technical problems described above, the present invention proposes a computer program recorded on a recording medium to implement an annotation method through boundary designation that can easily specify overlapping objects included in an image. The computer program includes memory; and a processor that processes instructions resident in the memory. In addition, the computer program generates the outlines of a plurality of objects arranged to overlap each other included in the image that is the subject of annotation work for artificial intelligence (AI) learning, according to the control of the operator. A step of designating, wherein the processor designates a boundary line between the plurality of objects within the designated outline, and wherein the processor selects each of the plurality of objects based on a plurality of areas divided based on the designated boundary line. In order to execute the identification step, it may be a computer program recorded on a recording medium.

Specific details of other embodiments are included in the detailed description and drawings.

According to embodiments of the present invention, when performing annotation, an outline for a plurality of objects is specified for a plurality of objects included in an image and arranged to overlap each other, and then the boundary line of each object is designated. By distinguishing objects, objects placed overlapping can be easily identified.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the description of the claims.

1 is a configuration diagram of an artificial intelligence learning system according to an embodiment of the present invention.
Figure 2 is a logical configuration diagram of a learning data generating device according to an embodiment of the present invention.
Figure 3 is a hardware configuration diagram of a learning data generating device according to an embodiment of the present invention.
Figure 4 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.
Figure 5 is a hardware configuration diagram of an annotation device according to an embodiment of the present invention.
Figure 6 is a flowchart for explaining a data classification method according to an embodiment of the present invention.
Figure 7 is a flowchart for explaining a data classification method according to another embodiment of the present invention.
Figure 8 is a flowchart illustrating a method for predicting work costs according to an embodiment of the present invention.
Figure 9 is a flowchart for explaining an annotation method according to an embodiment of the present invention.
10 to 16 are exemplary diagrams for explaining an annotation method according to an embodiment of the present invention.
Figure 17 is a flowchart for explaining an annotation method according to another embodiment of the present invention.
18 to 20 are exemplary diagrams for explaining an annotation method according to another embodiment of the present invention.
Figure 21 is an example diagram for explaining a data classification method according to an embodiment of the present invention.
Figure 22 is an example diagram for explaining a data classification method according to another embodiment of the present invention.
Figures 23 and 24 are exemplary diagrams for explaining an annotation method according to another embodiment of the present invention.
Figures 25 and 26 are example diagrams for explaining outlines of a plurality of overlapping objects.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, the technical terms used in this specification, unless specifically defined in a different way in this specification, should be interpreted as meanings generally understood by those skilled in the art in the technical field to which the present invention pertains, and are not overly comprehensive. It should not be interpreted in a literal or excessively reduced sense. Additionally, if the technical terms used in this specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that can be correctly understood by those skilled in the art. In addition, general terms used in the present invention should be interpreted according to the definition in the dictionary or according to the context, and should not be interpreted in an excessively reduced sense.

Additionally, as used herein, singular expressions include plural expressions, unless the context clearly dictates otherwise. In this application, terms such as “consists of” or “have” should not be construed as necessarily including all of the various components or steps described in the specification, and only some of the components or steps are included. It may not be possible, or it should be interpreted as including additional components or steps.

Additionally, terms including ordinal numbers, such as first, second, etc., used in this specification may be used to describe various components, but the components should not be limited by the terms. The above terms are used only for the purpose of distinguishing one component from another. For example, a first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component without departing from the scope of the present invention.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected to or connected to the other component, but other components may also exist in between. On the other hand, when it is mentioned that a component is “directly connected” or “directly connected” to another component, it should be understood that there are no other components in between.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. However, identical or similar components will be assigned the same reference numbers regardless of the reference numerals, and duplicate descriptions thereof will be omitted. Additionally, when describing the present invention, if it is determined that a detailed description of related known technologies may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the attached drawings. The spirit of the present invention should be construed as extending to all changes, equivalents, or substitutes other than the attached drawings.

Meanwhile, the collection of learning data is carried out on a project basis. At this time, the control tower of each project receives data from multiple vehicles equipped with data collection devices.

At this time, in the process of uploading data collected from multiple vehicles, there was a problem in which the same image was uploaded repeatedly from each vehicle or the same image was uploaded repeatedly from different vehicles.

In addition, the manager of each vehicle collects data from the control tower according to a guide specifying collection conditions and uploads the collected data. At this time, there was a problem in which data that did not meet the standards was randomly uploaded depending on the subjective viewpoint of the manager of each vehicle, the collection environment, and errors in the collection device.

To prevent the above problems, the control tower deploys inspectors to manually inspect the uploaded data. However, there is a need for a method to prevent unnecessary waste of resources due to deploying inspectors.

Additionally, estimating the overall cost of a project that requires numerous workers to perform numerous annotation tasks is a difficult task. In the past, there was a problem in that the total work cost of a project related to annotation work was calculated simply depending on the number of data to be worked on or the difficulty of the work predicted according to the intuition of the person in charge.

Also, when objects are placed overlapping in an image, in order to identify each object using the polygon technique, the image must be enlarged and a double point must be created at the same point of the boundary line. At this time, if a point was not created exactly at the same point of the boundary line, there was a problem that a space was created between the overlapping objects.

In order to overcome these limitations, the present invention seeks to propose various means for collecting data for artificial intelligence machine learning and purifying unnecessary data among the collected data.

In addition, the present invention seeks to propose various means for reasonably predicting the overall work cost of a project related to annotation work on artificial intelligence learning data.

Additionally, the present invention seeks to provide various means for easily specifying overlapping objects included in images when annotating artificial intelligence learning data.

1 is a configuration diagram of an artificial intelligence learning system according to an embodiment of the present invention.

As shown in Figure 1, the artificial intelligence learning system according to an embodiment of the present invention includes a plurality of learning data collection devices (100a, 100b, ..., 100n; 100), a learning data generation device 200, and a plurality of annotations. It may be configured to include devices 300a, 300b, ..., 300n; 300 and an artificial intelligence learning device 400.

Since the components of the artificial intelligence learning system according to this embodiment merely represent functionally distinct elements, two or more components are integrated and implemented with each other in the actual physical environment, or one component is implemented in the actual physical environment. may be implemented separately from each other.

To describe each component, the learning data collection device 100 includes a lidar and a camera (lidar) installed in the vehicle to collect data for machine learning artificial intelligence (AI) that can be used for autonomous driving. It is a device that collects data in real time from one or more of a camera, radar, ultrasonic sensor, rain sensor, position measurement sensor, and speed detection sensor.

The learning data collection device 100 may receive a request from the learning data generation device 100 to collect data for machine learning of artificial intelligence. At this time, the learning data collection device 100 may receive guide information including data collection conditions from the learning data generating device 100.

Here, the guide information may include the object class, data extension, and image resolution, which are collection conditions. At this time, the learning data collection device 100 may receive guide information through a sample image.

The types of sensors that are controlled by the learning data collection device 100 and installed in the vehicle to acquire, photograph, or detect data for machine learning include lidar, camera, radar, and ultrasonic sensors. One or more of an ultrasonic sensor, a rain sensor, a position measurement sensor, and a speed sensor may be included, but are not limited thereto. In addition, the sensor that is the control object of the learning data collection device 100 and is installed in the vehicle to acquire, photograph, or detect data for machine learning is not limited to being provided one for each type, and may be provided in plural even for the same type of sensor. It can be.

In the following configuration, the learning data generating device 200 is a device that can be used to design and generate data for machine learning artificial intelligence (AI) that can be used for autonomous driving of a vehicle.

Characteristically, the learning data generating device 200 according to an embodiment of the present invention requests at least one learning data collection device 100 to collect images for machine learning of artificial intelligence (AI), and , images may be received from at least one learning data collection device 100. The learning data generating device 200 may extract color information from received images and classify noise images based on color information between images.

In addition, the learning data generation device 200 according to another embodiment of the present invention transmits guide information including image collection conditions for machine learning of artificial intelligence to at least one learning data collection device 100, and at least Images can be received from one learning data collection device 100. The learning data generating device 200 may extract image information corresponding to collection conditions from images, compare the extracted image information with guide information, and classify noise images according to the collection environment.

In addition, the learning data generation device 200 according to another embodiment of the present invention transmits guide information including image collection conditions for machine learning of artificial intelligence to at least one learning data collection device 100, and at least Images can be received from one learning data collection device 100. The learning data generation device 200 extracts image information corresponding to the collection conditions from the images, compares the extracted image information with guide information, and classifies noise images according to physical factors of the learning data collection device 100. there is.

In addition, the learning data generating device 200 according to another embodiment of the present invention receives at least one sample data for performing a project related to annotation work scheduled to be performed for artificial intelligence learning, and Existing data included in a plurality of projects can be compared with the sample data, and at least one project containing existing data whose similarity to the sample data is higher than a preset value can be extracted. The learning data generating device 200 may predict the total work cost required to perform a project related to annotation work to be performed based on at least one extracted project.

The learning data generation device 200, which has these characteristics, transmits and receives data with the learning data collection device 100, the annotation device 300, and the artificial intelligence learning device 400, and performs calculations based on the transmitted and received data. Any device that can do this is acceptable. For example, the learning data generating device 200 may be any one of fixed computing devices such as a desktop, workstation, or server, but is not limited thereto.

As described above, the specific configuration and operation of the learning data generating device 200 will be described later with reference to FIGS. 2 and 3.

In the following configuration, the annotation device 300 is a device that can be used to annotate images provided from the learning data generating device 200.

As such, users of the annotation device 300 may be divided into labelers, reviewers, inspectors, and trainees.

Here, the labeler is a person who performs annotation work on images. A reviewer is a person who visually verifies the image on which the annotation work has been performed. An inspector is a person who verifies the annotation work results using a script. Additionally, a trainer is a person who receives training to perform the annotation task.

Specifically, the annotation device 300 can perform annotation work as follows under the control of the user corresponding to the labeler.

The annotation device 300 may output images received from the learning data generating device 200 according to user control.

The annotation device 300 can select a tool according to user control. Here, the tool is a tool for specifying one or more objects included in the image. The annotation device 300 can receive coordinates input according to user control using a selected tool. The annotation device 300 can identify objects based on input coordinates.

Meanwhile, the annotation device 300 according to an embodiment of the present invention uses a polygon technique in which an operator identifies an object by creating a plurality of points along the outline of an object included in the image. Objects can be identified. However, it is not limited to this, and the annotation device 300 may use techniques such as bounding box, polyline, point, cuboid, and semantic segmentation. there is.

In particular, the annotation device 300 according to an embodiment of the present invention, under the control of the operator, selects a first object among a plurality of objects arranged to overlap each other included in an image that is the target of annotation work for artificial intelligence learning. An outline can be designated, a borderline between a first object and a second object arranged to overlap can be designated, and the borderline can be set as part of the outline of the second object. The annotation device 300 can specify the outline of the second object based on the set boundary line.

In addition, the annotation device 300 according to another embodiment of the present invention specifies the outlines of a plurality of objects arranged to overlap each other included in the image that is the target of the annotation task for artificial intelligence learning, under the control of the operator. , you can specify boundaries between multiple objects within the designated outline. The annotation device 300 can identify a plurality of objects based on a plurality of areas divided based on a designated boundary line.

The annotation device 300 can set attribute information of a specified object. Here, the object property information is information for specifying the properties of the object that is the target of artificial intelligence (AI) learning. In this way, the object property information includes information about the annotation type (type), class name (class), classification items (tags), whether the object is truncated (truncated), major classification, small classification, or upper level (instance upper). It can be done, but is not limited to this.

The annotation device 300 may generate an annotation work result including coordinates according to the location and size of the object set by the user and set attribute information. Such work results may have a JSON (Java Script Object Notation) file format, but are not limited thereto.

And, the annotation device 300 may transmit the generated annotation work result to the learning data generation device 200.

Meanwhile, a detailed description regarding the annotation device 300 will be described later with reference to FIGS. 4 and 5.

As such, the annotation device 300 can be any device that can transmit and receive data with the learning data generating device 200 and perform operations using the transmitted and received data.

For example, the annotation device 300 may be any one of fixed computing devices such as a desktop, workstation, or server, but is not limited thereto, and may be a smart phone, Devices such as laptops, tablets, phablets, Portable Multimedia Players (PMPs), Personal Digital Assistants (PDAs), or E-book readers. It may be any one of the portable computing devices.

In the following configuration, the artificial intelligence learning device 400 is a device that can be used to perform machine learning of artificial intelligence based on artificial intelligence learning data.

Specifically, the artificial intelligence learning device 400 may transmit sample data related to a project to be performed to the learning data generating device 200. Here, sample data is a sample related to annotation work scheduled to be performed for artificial intelligence learning. Such sample data may be an image that is the target of annotation work, or may be an annotation work result, but is not limited thereto.

The artificial intelligence learning device 400 may receive the total work cost required to perform the project scheduled to be performed from the learning data generating device 200. The artificial intelligence learning device 400 can output the total received job cost.

This total work cost can be utilized to conclude a contract related to project performance between the operating entity of the artificial intelligence learning device 400 and the operating entity of the learning data generating device 200.

After a contract related to project performance is concluded between the operating entity of the artificial intelligence learning device 400 and the operating entity of the learning data generating device 200, the artificial intelligence learning device 400 is packaged from the learning data generating device 200. You can receive the annotation work results. And, the artificial intelligence learning device 400 can perform artificial intelligence (AI) machine learning based on the received annotation work results.

As such, the artificial intelligence learning device 400 can be any device that can transmit and receive data with the learning data generating device 200 and perform calculations using the transmitted and received data. For example, the artificial intelligence learning device 400 may be any one of fixed computing devices such as a desktop, workstation, or server, but is not limited thereto.

As described above, one or more learning data collection device 100, learning data generation device 200, annotation device 300, and artificial intelligence learning device 400 are directly connected between the devices through a secure line, a public wired communication network, or Data can be transmitted and received using a network that combines one or more of the mobile communication networks.

For example, public wired networks may include Ethernet, xDigital Subscriber Line (xDSL), Hybrid Fiber Coax (HFC), and Fiber To The Home (FTTH). It may be possible, but it is not limited to this. In addition, mobile communication networks include Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), High Speed Packet Access (HSPA), and Long Term Evolution. LTE) and 5th generation mobile telecommunication may be included, but are not limited thereto.

Hereinafter, the configuration of the learning data generating device 200 as described above will be described in more detail.

Figure 2 is a logical configuration diagram of a learning data generating device according to an embodiment of the present invention.

Referring to FIG. 2, the learning data generating device 200 includes a communication unit 205, an input/output unit 210, a data design unit 215, a data collection unit 220, a data preprocessing unit 225, and a data delivery unit 230. ) and a storage unit 235.

Since the components of the learning data generating device 200 merely represent functionally distinct elements, two or more components may be implemented integrated with each other in the actual physical environment, or one component may be integrated with each other in the actual physical environment. It could be implemented separately.

Specifically, the communication unit 205 may transmit guide information including image collection conditions for machine learning of artificial intelligence (AI) to at least one learning data collection device.

In addition, the communication unit 205 receives images captured by a camera, point cloud data obtained from a lidar, and data detected from a position measurement sensor and a speed detection sensor from the learning data collection device 100. can do.

Additionally, the communication unit 205 may transmit one or more images that are the subject of annotation work to the annotation device 300. Additionally, the communication unit 205 may receive annotation work results from the annotation device 300.

Additionally, the communication unit 205 may receive at least one sample data from the artificial intelligence learning device 400. The communication unit 205 may transmit to the artificial intelligence learning device 400 the total work cost required to perform the project to be performed, which is predicted by the data design unit 215 or input by the user.

In the following configuration, the input/output unit 210 can input a signal from the user through a user interface (UI) or output the calculated result to the outside.

Specifically, the input/output unit 210 may receive guide information including collection conditions of the learning data collection device 100 from the user. Guide information may include, but is not limited to, the learning purpose, learning period, number of images required for learning, properties of the object to be identified in the image, image resolution, image extension, etc.

Additionally, the input/output unit 210 can receive sample data from the user. Additionally, the input/output unit 210 may receive input of decomposition components, weights, and guide information from the user. Here, the decomposition component is an element used to predict the overall work cost of the project among the elements that make up the annotation work result. For example, decomposition components may include, but are not limited to, classes and tools.

The input/output unit 210 may output the total work cost required to perform the project to be performed, predicted by the data design unit 215. The input/output unit 210 may receive a revised total job cost input from the user.

With the following configuration, the data design unit 215 can predict the total work cost required to perform the project scheduled to be performed and provide the estimate to the artificial intelligence learning device 400.

Specifically, the data design unit 215 may receive at least one sample data from the artificial intelligence learning device 400 to perform a project related to annotation work scheduled to be performed for artificial intelligence learning. Here, the sample data is a sample related to annotation work scheduled to be performed for artificial intelligence (AI) learning. Such sample data may be an image that is the target of annotation work, or may be an annotation work result, but is not limited thereto.

The data design unit 215 may compare existing data included in a plurality of previously performed projects with sample data and extract at least one project containing existing data whose similarity to the sample data is higher than a preset value.

At this time, the data design unit 215 may identify one or more decomposed components that constitute the existing data. Here, the decomposition component is an element used to predict the overall work cost of the project among the elements that make up the annotation work result. For example, decomposition components may include, but are not limited to, classes and tools.

For example, when the sample data corresponds to an image subject to annotation work, the data design unit 215 may perform annotation work on the image corresponding to the sample data under user control. In addition, the data design unit 215 can identify the class of the object specified from the image through annotation work and the tool used to specify the object as a decomposition component of the sample data.

Additionally, the data design unit 215 may extract at least one project containing existing data whose similarity to sample data is higher than a preset value. At this time, the data design unit 215 may receive weights for the decomposed components of the sample image from the artificial intelligence learning device 400 and evaluate the degree of similarity with the existing image by considering the input weights. For example, when a higher weight is given to the class of the object by the artificial intelligence learning device 400 and the class of the object among the tools used to specify the object, the data design unit 215 focuses on the class of the object. You can extract similar existing data and extract projects containing the existing data.

The data design unit 215 extracts the edge of the sample image, detects the object included in the sample data based on the extracted edge, and stores the RGB (Red, Green, Blue) value of the object included in the existing data. Similarity can be evaluated by comparing with the RGB values of the object.

For example, the data design unit 215 may generate an RGB (Red, Green, Blue) histogram for the pixel of the detected object, and calculate the degree of similarity by comparing the generated RGB histograms. Here, the RGB histogram is a graph showing the brightness distribution of each primary color (RGB) in the image. For example, in an RGB histogram, the horizontal axis represents the brightness level of a color, the vertical axis represents the number of pixels assigned to the color's brightness level, the more pixels deviated to the left, the darker and less vivid the color, and the more pixels deviated to the right, the darker and less vivid the color is. The greater the number of pixels, the brighter and darker the color can be expressed. In this way, the data design unit 215 can calculate the similarity by comparing the color saturation, gradation state, white balance tendency, etc. of the object included in the sample image and the object included in the existing image through the RGB histogram. However, it is not limited to this, and the data design unit 215 may calculate the degree of similarity by comparing the moments about the edges of the extracted objects.

In addition, the data design unit 215 compares representative images pre-stored for each of the plurality of previously performed projects with sample images, and transmits a plurality of representative images with a similarity to the sample image higher than a preset value to the artificial intelligence learning device. , one of a plurality of representative images can be selected from the artificial intelligence learning device 400. Here, the representative image may be an image with the highest similarity to a sample image received to carry out each of the plurality of projects among the images collected in the process of performing the plurality of projects that have already been performed.

At this time, the data design unit 215 transmits a plurality of representative images whose similarity to the sample image is higher than a preset value to the artificial intelligence learning device 400, identifies objects included in the representative images, and classifies the identified objects. If is pre-registered as confidential information, the identified object can be de-identified and transmitted to the artificial intelligence learning device 400. The data design unit 215 may perform de-identification processing by blurring objects corresponding to classes registered as confidential information.

In other words, images collected for each project may contain confidential information. Here, the confidential information may be confidential information of each company designated by the company that requested each project, or may include personal information such as face or license plate. This confidential information may be set from the artificial intelligence learning device 400 that requested each of a plurality of projects that have already been performed, or may be set in advance by the learning data generating device 200.

The data design unit 215 performs de-identification processing by blurring some of the objects designated as confidential information, extracts landmarks from the identified objects, and performs blurring on the extracted landmarks. can be performed. For example, when the identified object is a person, the data design unit 215 can extract the eyes, nose, and mouth corresponding to the landmarks of the person, and selectively perform blurring only on the extracted eyes, nose, and mouth.

Additionally, the data design unit 215 may predict the total work cost required to perform a project related to annotation work to be performed based on at least one extracted project. At this time, the data design unit 215 can receive the data quantity of the project related to the annotation work to be performed and predict the total work cost by considering the extracted project cost and data quantity.

For example, the data design unit 215 detects the data quantity and work cost for at least one extracted project. In addition, the data design unit 215 receives the data quantity of the project to be performed, and predicts the total work cost by adding or subtracting the work cost according to the input data quantity in proportion to the data quantity and work cost of the extracted project.

In addition, the data design unit 215 can extract a plurality of projects containing existing data whose similarity to sample data is higher than a preset value, and predict the average work cost of the extracted plurality of projects as the total work cost of the project to be performed. there is.

And, the data design unit 215 can output the predicted total work cost through the input/output unit 210. The data design unit 215 may modify the overall work cost according to user control input through the input/output unit 210. In addition, the data design unit 215 may transmit the predicted or revised total work cost to the artificial intelligence learning device 400 through the communication unit 205.

In the following configuration, the data collection unit 220 can collect artificial intelligence (AI) learning data for the project when a contract related to project performance is concluded with the operator of the artificial intelligence learning device 400.

Specifically, the data collection unit 220 may request at least one learning data collection device 100 to collect images for machine learning of artificial intelligence. To this end, the data collection unit 220 may transmit guide information including image collection conditions to the learning data collection device 100. Here, the guide information may include the object class, data extension, image resolution, etc., which are collection conditions. At this time, the data collection unit 220 may provide guide information through a sample image. That is, the data collection unit 220 may transmit the sample image received at the time of the project contract to the learning data collection device 100 to recognize the collection conditions of the learning data collection device 100.

Additionally, the data collection unit 220 may receive images from at least one learning data collection device 100. At this time, the data collection unit 220 may assign an identifier to each of the at least one learning data collection device 100 and store the images received for each assigned identifier.

With the following configuration, the data purification unit 225 can extract color information of received images. Here, the color information may be an RGB (Red, Green, Blue) value or a color code value for a pixel. The data purification unit 225 may store the file name and color information of the images in the storage unit 235 based on an identifier assigned to each of at least one learning data collection device.

The data purification unit 225 may classify noise images based on color information between images. That is, the data purification unit 225 may classify at least one of the images in which the similarity of color information is higher than a preset value as a noise image.

At this time, the data purification unit 225 may resample the images to a preset resolution and evaluate the degree of similarity between the images by comparing the color information of pixels present at the same coordinates of the resampled images.

Additionally, when there are multiple images with the same file name and the same identifier, the data purification unit 225 may classify at least one of the images with the same file name as a noise image. When there are multiple file names with different identifiers but the same file name, the data purification unit 225 may classify at least one of the images with the same file name as a noise image. That is, the data purification unit 225 can prevent the case in which files with the same name are repeatedly registered with the same identifier or the same file is repeatedly registered with different identifiers.

In addition, the data purification unit 225 lists the images in chronological order, generates sequence data by grouping the listed images into a preset number, and compares the color information of the images included in each sequence data to create a noise image. can be classified.

Specifically, the data purification unit 225 may extract the edge of an object included in each image of the generated sequence data. The data purification unit 225 may evaluate the similarity of images based on the amount of edge change between consecutive images for each sequence data.

Here, an edge is a place where pixel values change rapidly within an image. The data purification unit 225 may determine the edge based on the size of a gradient vector obtained by differentiating the image. For example, the data purification unit 225 may extract edges on the image through an edge extraction algorithm such as the Sobel edge detection algorithm and the Canny edge detection algorithm.

Additionally, the data purification unit 225 may calculate the sharpness of images whose similarity is higher than a preset value, and classify the images other than the image with the highest calculated sharpness as noise images.

In other words, if a plurality of similar images exist, all but one of the plurality of images must be removed based on a specific criterion. To this end, the data purification unit 225 may classify the remaining images, excluding the image with the highest clarity among the selected images, as noise images and delete them.

Additionally, the data purification unit 225 may calculate the similarity between consecutive images for each sequence data and determine the number of frames per second for each sequence data based on the calculated similarity.

That is, if the similarity between consecutive images in one sequence data is higher than a preset value, the data purification unit 225 determines that the speed of the vehicle that collected the image is high, and sets the number of frames per second for each sequence data. By making this decision, the volume of the image included in the sequence data can be reduced.

Additionally, the data purification unit 225 may extract image information corresponding to collection conditions from the images. Here, the image information may include at least one of a file extension, image resolution, RGB (Red, Green, Blue) value for a pixel, and a color code value.

Here, the data purification unit 225 may compare the extracted image information with guide information and classify noise images according to errors in the collection environment or the learning data collection device 100.

Specifically, the data purification unit 225 extracts sample image information including at least one of the file extension, image resolution, RGB value for the pixel, and color code value of the sample image, and extracts the extracted sample image information from the images. It can be compared with extracted image information.

At this time, the data purification unit 225 may classify an image whose similarity to the sample image is lower than a preset value as a noise image. That is, the data purification unit 225 determines whether the sample image and the file extension or image resolution are different, or the similarity of the RGB (Red, Green, Blue) values and color code values for the pixel is set to a preset value. If it is lower, the image can be classified as a noise image.

The data purification unit 225 may arrange images in chronological order, generate sequence data by grouping the listed images into a preset number, and classify noise images by sequence data. At this time, the data purification unit 225 compares the similarity between the before and after images for a specific image among the sequence data, and determines that the similarity between the before and after images is higher than a preset value, but the similarity between the before and after images and the specific image is at a preset value. If it is lower, a specific image may be judged to be a noise image.

That is, the data purification unit 225 compares the images before and after a specific image, and if only the specific image has low similarity, it determines that the specific image is an image taken in the process of crossing a speed bump, and classifies the image as a noise image. there is. Here, the data purification unit 225 may resample the images to a preset resolution and evaluate the degree of similarity between the images by comparing the color information of pixels present at the same location in the resampled images.

In addition, the data purification unit 225 extracts the edge of the object included in each of the images, detects the object included in each of the images, and selects an image in which the position change value of the detected object is higher than a preset value as a noise image. It can be classified as: That is, the data purification unit 225 can determine whether a specific image is an image taken in the process of crossing a speed bump based on the degree of movement of an object included in the image.

Additionally, the data purification unit 225 may receive meta information of each image from at least one learning data collection device 100 through the communication unit 205. Here, the meta information may include location information and speed information of the learning data collection device 100 at the time of capturing each of the images.

Using the meta information provided from the learning data collection device 100, the data purification unit 225 uses the location information of the speed bump included in the map information including the path traveled by the learning data collection device 100. By comparing with the meta information, the image taken at the location of the speed bump can be classified as a noise image.

In addition, the data purification unit 225 compares the location information of the curved road included in the map information including the path traveled by the learning data collection device 100 with meta information, and determines the location of the curved road. The generated image can be classified as a noise image.

In addition, the data purification unit 225 compares the similarity between consecutive images in the sequence data, and when images with a similarity lower than a preset value are continuously detected, the detected images are images taken on a curved road. It is possible to determine and classify the detected images as noise images.

Additionally, the data purification unit 225 may classify noise images based on the similarity of consecutive images for each sequence data and estimate the type of error for each classified noise image.

Specifically, the data purification unit 225 compares the similarity between consecutive images among the sequence data, and determines that the similarity between the first image and the second consecutive image is lower than a preset value, and the similarity between the first image and the second consecutive image is lower than a preset value, and the 3 If the similarity between the image and the second image is higher than the preset value, it can be determined as an error in which the camera angle of the camera that captured the image included in the sequence data has changed. That is, when the image is rapidly changed and the image in the changed state is continuously collected, the data purification unit 225 may determine that the data after the image change is noise data due to an error in the camera angle change.

In addition, the data purification unit 225 compares the similarity between consecutive images in the sequence data, and when the number of images with similarity lower than a preset value exceeds the preset number, the camera that took the images included in the sequence data It can be judged to be an error due to poor binding. In other words, if the image continuously changes, the data purification unit 225 may determine it to be an error due to poor binding of the camera.

The data purification unit 225 may delete data corresponding to the estimated error or include the type of the estimated error in meta information so that the inspector can check it.

Additionally, the data purification unit 225 may match received images and pre-stored images based on global positioning system (GPS) coordinates, compare similarities between the matched images, and classify noise images. That is, the data purification unit 225 matches the images previously collected at the corresponding location among the previously performed projects with the currently collected images, and compares the similarity between the matched images. If the similarity is lower than a preset value, The image can be classified as a noise image.

In addition, the data purification unit 225 selects each image from the first side and the second side based on the similarity of RGB values between the pixels constituting the first side and the pixels constituting the second side. One vertex can be identified, and a line segment connecting the two identified vertices can be extracted from the first side and the second side.

That is, the data purification unit 225 evaluates the similarity of pixels existing at both ends of the image to identify objects that statically exist in the image, such as roads, soundproof walls, and guardrails, and connects both ends of the image. can be identified. Additionally, the object connecting both ends of the image can be recognized as a static object.

If the data purification unit 225 is outside a preset error range based on at least one of the length and angle of the line segment extracted from each matched image, it may be determined as an error in which the camera angle has changed.

However, it is not limited to this, and the data purification unit 225 extracts an edge from each matched image, identifies objects included in each matched image based on the extracted edge, and changes the position of the identified object. Noisy images can be classified based on their values.

In addition, the data purification unit 225 may further receive images and 3D point group data acquired through lidar simultaneously through the communication unit 205. The data purification unit 225 determines whether the type of object detected in each matched image is a moving object or a static object based on the distance information included in the 3D point cloud data, and determines the position change value of the static object among the detected objects. The noise image can be classified based on .

That is, among the objects included in the image, there may be moving objects such as cars, bicycles, and people, and static objects such as roads, buildings, and guide rails. Accordingly, the data purification unit 225 may classify an image in which the position change value of a static object between the sample image and the matching image is higher than a preset value as a noise image.

In the following configuration, the data delivery unit 230 can distribute one or more annotation work objects (ie, images) to the annotation devices 300. Additionally, the data delivery unit 230 may verify the annotation work result and then deliver it to the artificial intelligence learning device 400.

With the following configuration, the storage unit 235 can store data necessary for the operation of the learning data generating device 200. The storage unit 235 can store data necessary for designing data for artificial intelligence (AI) learning.

Specifically, the storage unit 235 may store images that are the target of annotation work. The storage unit 235 may store project properties, image properties, or worker properties.

Hereinafter, the hardware for implementing the logical components of the learning data generating device 200 as described above will be described in more detail.

Figure 3 is a hardware configuration diagram of a learning data generating device according to an embodiment of the present invention.

The learning data generating device 200 includes a processor (250), a memory (255), a transceiver (260), an input/output device (265), a data bus (Bus), and storage (270). It can be configured to include Storage, 275).

The processor 250 may implement the operations and functions of the learning data generating device 200 based on instructions according to the software 280a residing in the memory 255. Software 280a implementing the method according to the present invention may be loaded in the memory 255. The transceiver 260 can transmit and receive data with the learning data collection device 100, the annotation device 300, and the artificial intelligence learning device 400. The input/output device 265 may receive data required for the operation of the learning data design device 200 and output classified noise images, predicted total work costs, etc. The data bus 270 is connected to the processor 250, memory 255, transceiver 260, input/output device 265, and storage 275, and forms a moving path for transferring data between each component. can perform its role.

The storage 275 may store an application programming interface (API), a library file, a resource file, etc. required to execute the software 280a in which another method according to the present invention is implemented. Storage 275 may store software 280b in which the method according to the present invention is implemented. Additionally, the storage 275 may store information necessary to perform a method of generating data for artificial intelligence learning. In particular, the storage 275 may include a database 285 that stores project properties, image properties, worker properties, information on a plurality of previously performed projects, and a pool of workers.

According to one embodiment of the present invention, the software 280a, 280b residing in the memory 255 or stored in the storage 275 allows the processor 250 to perform machine learning of artificial intelligence (AI). requesting collection of images for at least one learning data collection device, the processor 250 receiving images from the at least one learning data collection device, the processor 250 receiving color information of the received images It may be a computer program recorded on a recording medium in order to execute the step of extracting and the processor 250 classifying the noise image based on color information between images.

According to another embodiment of the present invention, the software 280a, 280b residing in the memory 255 or stored in the storage 275 is used by the processor 250 to perform machine learning of artificial intelligence (AI). ) transmitting guide information including collection conditions for at least one collection device, the processor 250 receiving images from the at least one learning data collection device, the processor 250, collecting conditions and In order to execute the step of extracting corresponding image information from the images and the step of the processor 250 comparing the image information with the guide information and classifying the noise image according to the collection environment, a computer program recorded on the recording medium is used. It can be.

According to another embodiment of the present invention, the software 280a, 280b residing in the memory 255 or stored in the storage 275 is used by the processor 250 to perform machine learning of artificial intelligence (AI). ) transmitting guide information including image collection conditions for to at least one learning data collection device, the processor 250 receiving images from the at least one learning data collection device, the processor 250 Executing a step of extracting image information corresponding to the collection conditions from the images and a step of the processor 250 comparing the extracted image information with the guide information and classifying noise images according to collection device errors among the images. For this purpose, it may be a computer program recorded on a recording medium.

According to another embodiment of the present invention, the software 280a, 280b resident in the memory 255 or stored in the storage 275 is scheduled to be performed by the processor 250 for artificial intelligence (AI) learning. Receiving at least one sample data for performing a project related to an annotation task from an artificial intelligence learning device, the processor 250 compares existing data included in a plurality of previously performed projects with the sample data, and , extracting at least one project containing existing data whose similarity to sample data is higher than a preset value, and the processor 250 performs a project related to annotation work to be performed based on the extracted at least one project. It may be a computer program recorded on a recording medium to execute the steps of predicting the total cost of work required to do so.

More specifically, the processor 250 may include an application-specific integrated circuit (ASIC), another chipset, a logic circuit, and/or a data processing device. The memory 255 may include read-only memory (ROM), random access memory (RAM), flash memory, a memory card, a storage medium, and/or other storage devices. The transceiver 260 may include a baseband circuit for processing wired and wireless signals. The input/output device 265 includes input devices such as a keyboard, mouse, and/or joystick, a liquid crystal display (LCD), an organic light emitting diode (OLED), and/ Alternatively, it may include an image output device such as an active matrix OLED (AMOLED), a printing device such as a printer, a plotter, etc.

When the embodiments included in this specification are implemented as software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described function. The module resides in memory 255 and can be executed by processor 250. Memory 255 may be internal or external to processor 250 and may be coupled to processor 250 by a variety of well-known means.

Each component shown in FIG. 3 may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, an embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and FPGAs ( Field Programmable Gate Arrays), processor, controller, microcontroller, microprocessor, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded Here, the recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the recording medium may be those specifically designed and constructed for the present invention, or may be known and available to those skilled in the art of computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROM (Compact Disk Read Only Memory) and DVD (Digital Video Disk), and floptical media. It includes magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, etc. Examples of program instructions may include machine language code such as that created by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Such hardware devices may be configured to operate as one or more software to perform the operations of the present invention, and vice versa.

Hereinafter, the logical configuration of the annotation device 300 according to an embodiment of the present invention will be described in detail.

Figure 4 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.

Referring to FIG. 4, the annotation device 300 according to an embodiment of the present invention includes a communication unit 305, an input/output unit 310, a storage unit 315, an object identification unit 320, and an object property setting unit 325. ) and a result generating unit 330.

Since the components of the annotation device 300 merely represent functionally distinct elements, two or more components may be implemented integrated with each other in the actual physical environment, or one component may be separated from each other in the actual physical environment. It could be implemented.

To describe each component, the communication unit 305 can transmit and receive data with the learning data generating device 200.

Specifically, the communication unit 305 may receive an image from the learning data generating device 200.

Here, the image is an image that is the target of annotation work for artificial intelligence (AI) learning. In this way, images subject to annotation work may be received individually, or a plurality of images may be received in batches, according to a data processing plan designed by the learning data generating device 200.

Additionally, the communication unit 305 may transmit the annotation work result to the learning data generating device 200.

Here, the work product may include the coordinates of the bounding box and object attribute information set according to the operator's control. Additionally, the work result may be in a JSON file format, but is not limited thereto.

Additionally, the communication unit 305 may receive project properties, image properties, or worker properties from the learning data generating device 200.

Here, the properties of the project may include the learning purpose for the project related to artificial intelligence (AI) learning, learning period, number of images required for learning, properties of the object to be identified in the image, polygon setting rules, etc. It is not limited.

Image properties include image file name, image size (width, height), resolution, bit level, compression format, shooting device name, exposure time, ISO sensitivity, focal length, aperture opening value, and shooting location coordinates (GPS latitude, longitude). , shooting time, etc. may be included, but are not limited thereto.

The worker's attributes may include, but are not limited to, the worker's name, identification number, assigned work amount, cost according to the work, and work result evaluation.

In the following configuration, the input/output unit 310 can input a signal from an operator through a user interface (UI) or output the calculated result to the outside.

Here, a worker refers to a person who performs annotation work. As such, the worker may be referred to as a user, performer, labeler, or data labeler, but is not limited thereto.

Specifically, the input/output unit 310 may output an image that is the target of an annotation task. The input/output unit 310 may receive a control signal for specifying an object from the operator. Additionally, the input/output unit 310 can overlay and output an area designated by the user on the image.

Additionally, the input/output unit 310 may receive a control signal for setting attribute information of an object from an operator.

Object property information is information for specifying the properties of an object that is the target of artificial intelligence (AI) learning. In this way, the object property information includes information about the annotation type (type), class name (class), classification items (tags), whether the object is truncated (truncated), major classification, small classification, or upper level (instance upper). It can be done, but is not limited to this.

With the following configuration, the storage unit 315 can store the image received through the communication unit 305. The storage unit 315 may store project properties, image properties, or worker properties received through the communication unit 305.

In the following configuration, the object identification unit 320 can specify the outline of a first object among a plurality of objects arranged to overlap each other included in an image that is the target of an annotation task for artificial intelligence (AI) learning. .

At this time, the object identification unit 320 may receive a plurality of points along the outline of the first object from the operator and connect the plurality of points to form the outline of the first object. That is, the object identification unit 320 can create a polygon-shaped area by connecting points designated by the operator.

At this time, when the object identification unit 320 selects an arbitrary first and second point among the plurality of points from the operator and designates at least one new third point between the first point and the second point, By connecting the first point, the second point, and the third point, the outline of the first object can be modified. That is, after receiving a plurality of points, the object identification unit 320 selects two points corresponding to the area requiring correction, and when specifying a new point between the two points, deletes the previously designated point and , a new area can be created by connecting multiple points based on a new point.

However, it is not limited to this, and the object identification unit 320 may modify the outline of the first object based on the moved point when any point among the plurality of points is dragged and moved according to the operator's control. You can.

In addition, the object identification unit 320 extracts an edge of the image, identifies at least one object based on the extracted edge, and, upon receiving one point from the operator, identifies the object including the input point. The edge can be designated as the outline of the first object.

That is, the object identification unit 320 can automatically identify the object and designate the outline of the object without receiving a plurality of points input from the operator and generating the outline of the object. That is, when identifying an object based on an edge, multiple objects may be identified in the image. At this time, when the operator selects a specific point, the object identification unit 320 may determine that the edge containing the point is the edge of the object to be identified, and may recognize the edge of the object as an outline.

After designating the outline of the first object, the object identification unit 320 may designate a boundary line between the first object and the second object arranged to overlap.

Specifically, the object identification unit 320 generates a plurality of points having a preset interval along the outline of the first object, selects at least one point from the plurality of generated points, and selects at least one point selected. The above boundary line can be specified as a basis.

At this time, rather than receiving all points corresponding to the boundary line from the user, the object identification unit 320 selects two random points among a plurality of points and selects at least one point that exists between the two selected points. You can create connecting lines and designate the created lines as boundary lines.

Here, when the object identification unit 320 selects an arbitrary first point and a second point among a plurality of points and selects a third point that exists between the first point and the second point, the object identification unit 320 selects the first point and the second point. A line connecting two points and a third point can be created. That is, when two points are selected, there are two lines forming the outline of the object based on the two selected points. Accordingly, the object identification unit 320 can select a point that is different between two points and clearly recognize the boundary line.

After designating the boundary line, the object identification unit 320 may set the boundary line as part of the outline of the second object. At this time, the object identification unit 320 may output the boundary line in a different color to distinguish it from the outline of the first object.

Additionally, the object identification unit 320 may designate the outline of the second object based on the set boundary line.

At this time, the object identification unit 320 extracts the edge of the image, identifies at least one object based on the extracted edge, and selects the remaining objects excluding the first object among the identified at least one object including the boundary line. The edge of can be designated as the outline of the second object.

Additionally, the object identification unit 320 may receive a plurality of points along the outline of the second object from the operator and connect the boundary line and the plurality of points to form the outline of the second object.

Then, the object identification unit 320 changes the width of the borderline to the width of the sum of the widths of the outline of the first object and the outline of the second object, divides the borderline with the changed width into two adjacent lines, and , each of the two divided lines can be connected to the outline of the first object and the outline of the second object. That is, the object identification unit 320 can generate one input boundary line into two boundary lines arranged in close contact with each other, and connect the two created boundary lines to each object.

Meanwhile, the object identification unit 320 can identify objects as follows in addition to the above-described method.

Specifically, the object identification unit 320 may specify the outlines of a plurality of objects arranged to overlap each other included in an image that is the target of an annotation task for artificial intelligence (AI) learning.

At this time, the object identification unit 320 may receive a plurality of points input from the operator along an outline including a plurality of objects, connect the plurality of points, and generate outlines for the plurality of objects.

In addition, the object identification unit 320 extracts an edge of the image, identifies at least one object based on the extracted edge, receives a plurality of objects selected from the identified objects by the operator, and selects the extracted edge. As a basis, you can create outlines for multiple selected objects excluding boundary lines.

Additionally, the object identification unit 320 may receive point cloud data obtained from lidar simultaneously with the image through the communication unit 305. The object identification unit 320 identifies a plurality of point clouds with a certain depth within a certain range among the points present inside the designated outline based on the point cloud data acquired from the lidar simultaneously with the image, and receives the information from the operator. A plurality of objects from among a plurality of point clouds may be selected, and outlines for the selected objects may be generated based on the identified plurality of point clouds.

Then, the object identification unit 320 sets a partial area containing a plurality of objects as a bounding box, extracts the edge of the object from the area inside the bounding box, and identifies the object based on the extracted edge. You can specify the outlines of multiple objects by distinguishing between the background and the background and deleting the background.

After designating outlines for a plurality of objects, the object identification unit 320 may designate boundaries between the plurality of objects within the designated outlines.

Here, the object identification unit 320 may identify a boundary line within the outline of a plurality of objects based on the extracted edges.

Additionally, the object identification unit 320 may receive input from an operator as a plurality of points located inside the outline of a plurality of objects and connect the plurality of input points to create a boundary line.

In addition, the object identification unit 320 identifies a plurality of point clouds with a certain distance within a certain range among the points present inside the designated outline based on the point cloud data acquired from the lidar simultaneously with the image, and the boundary line between the plurality of point clouds. can be designated as a boundary line between multiple objects.

In addition, the object identification unit 320 creates a group with RGB values similar to a preset value based on the RGB (Red, Green, Blue) values of pixels located inside the designated outline, and the created group is Each can be recognized as an object and the boundary line of the recognized object can be created.

In addition, the object identification unit 320 identifies a plurality of point clouds with a certain distance within a certain range among the points present inside the designated outline based on the point cloud data acquired from the lidar simultaneously with the image, and selects a plurality of point clouds from the operator among the plurality of point clouds. One object can be selected and a boundary line can be created based on the point cloud of the selected object.

Then, the object identification unit 320 extracts an edge inside the specified outline, identifies at least one object based on the extracted edge, selects one object among the identified objects from the operator, and selects an object based on the extracted edge. You can create a boundary line with . At this time, the object identification unit 320 may generate a plurality of points having a preset interval along the generated boundary line, and modify the boundary line by moving at least one of the plurality of points under the control of the operator.

After designating the boundary line, the object identification unit 320 may identify a plurality of objects based on a plurality of areas divided based on the designated boundary line.

In the following configuration, the object property setting unit 325 can receive a control signal for setting property information of an object from an operator through the input/output unit 310.

When one piece of information is selected from the list of recommended information under the operator's control, the object attribute setting unit 325 may provide feedback according to the type of object corresponding to the selected information.

In one embodiment, the object property setting unit 325 may change the user interface (UI) related to the area inside the object by reflecting differently set colors or transparency depending on the type of object corresponding to the selected information. there is.

With the following configuration, the result generator 330 can generate an annotation work result and transmit it to the learning data generation device 200.

Hereinafter, hardware for implementing the logical components of the annotation device 300 as described above will be described in more detail.

Figure 5 is a hardware configuration diagram of an annotation device according to an embodiment of the present invention.

As shown in FIG. 5, the annotation device 300 includes a processor (350), a memory (355), a transceiver (360), an input/output device (365), and a data bus (Bus). , 370) and storage (Storage, 375).

The processor 350 may implement the operations and functions of the annotation device 300 based on instructions according to the software 380a in which the annotation method is implemented, which resides in the memory 355. Software 380a implementing an annotation method may be loaded in the memory 355. The transceiver 360 can transmit and receive data with the learning data generating device 200. The input/output device 365 can receive data required for the operation of the annotation device 300 and output an image. The data bus 370 is connected to the processor 350, memory 355, transceiver 360, input/output device 365, and storage 375, and forms a moving path for transferring data between each component. can perform its role.

The storage 375 may store an application programming interface (API), a library file, a resource file, etc. required to execute the software 180a in which the annotation method is implemented. The storage 375 may store software 380b in which the annotation method is implemented. Additionally, the storage 375 may store information necessary to perform the annotation method. In particular, the storage 375 may include a database 385 that stores images that are subject to annotation work.

According to one embodiment of the present invention, the software 380a, 380b for implementing the annotation method resident in the memory 355 or stored in the storage 375 is operated by the processor 350 under the control of the operator, using artificial intelligence (AI). A step of specifying the outline of a first object among a plurality of objects arranged overlapping each other included in an image that is the target of an annotation task for Artificial Intelligence (AI) learning, where the processor 350 A step of specifying a boundary line between overlapping second objects, where the processor 350 sets the boundary line as part of the outline of the second object, and where the processor 350 sets the outline of the second object based on the boundary line. It may be a computer program recorded on a recording medium to execute the specified steps.

According to another embodiment of the present invention, the software 380a, 380b for implementing the annotation method resident in the memory 355 or stored in the storage 375 allows the processor 350 to perform artificial intelligence learning under the control of the operator. A step of specifying the outlines of a plurality of objects arranged to overlap each other included in an image that is the target of an annotation work for, a step of the processor 350 designating a boundary line between the plurality of objects within the specified outline, and the processor ( 350) may be a computer program recorded on a recording medium to execute the step of identifying a plurality of objects based on a plurality of areas divided based on a designated boundary line.

More specifically, the processor 350 may include an application-specific integrated circuit (ASIC), another chipset, a logic circuit, and/or a data processing device. The memory 355 may include read-only memory (ROM), random access memory (RAM), flash memory, a memory card, a storage medium, and/or other storage devices. The transceiver 360 may include a baseband circuit for processing wired and wireless signals. The input/output device 365 includes input devices such as a keyboard, mouse, and/or joystick, a liquid crystal display (LCD), an organic light emitting diode (OLED), and/ Alternatively, it may include an image output device such as an active matrix OLED (AMOLED), a printing device such as a printer, a plotter, etc.

When the embodiments included in this specification are implemented as software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described function. The module resides in memory 355 and can be executed by processor 350. Memory 355 may be internal or external to processor 350 and may be coupled to processor 350 by a variety of well-known means.

Each component shown in FIG. 5 may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In the case of implementation by hardware, an embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and FPGAs ( Field Programmable Gate Arrays), processor, controller, microcontroller, microprocessor, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded Here, the recording medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the recording medium may be those specifically designed and constructed for the present invention, or may be known and available to those skilled in the art of computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROM (Compact Disk Read Only Memory) and DVD (Digital Video Disk), and floptical media. It includes magneto-optical media such as floptical disks, and hardware devices specially configured to store and execute program instructions such as ROM, RAM, flash memory, etc. Examples of program instructions may include machine language code such as that created by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc. Such hardware devices may be configured to operate as one or more software to perform the operations of the present invention, and vice versa.

Hereinafter, a data classification method according to an embodiment of the present invention will be described.

Figure 6 is a flowchart for explaining a data classification method according to an embodiment of the present invention.

Referring to FIG. 6, first, in step S110, the learning data generating device may request image collection from at least one learning data collecting device.

Next, in step S120, the learning data generating device may receive images from at least one learning data collecting device.

Next, in step S130, the learning data generating device may extract color information of the received images. Here, the color information may be an RGB (Red, Green, Blue) value or a color code value for a pixel.

Here, the learning data generating device may store the file name and color information of the images based on an identifier assigned to each of the at least one learning data collection device.

Next, in step S140, the learning data generating device may classify noise images based on color information between images. That is, the learning data generating device may classify at least one of the images in which the similarity of color information is higher than a preset value as a noise image.

Specifically, when a plurality of images with the same file name and the same identifier exist, the learning data generating device may classify at least one of the images with the same file name as a noise image.

Additionally, when there are multiple file names with different identifiers but the same file name, the learning data generating device may classify at least one of the images with the same file name as a noise image.

In other words, the learning data generating device can prevent the case in which files with the same name are repeatedly registered with the same identifier or the same file is repeatedly registered with different identifiers.

In addition, the learning data generation device lists images in chronological order, generates sequence data by grouping the listed images into a preset number, and classifies noise images by comparing the color information of the images included in each sequence data. can do.

Specifically, the learning data generating device may extract the edge of an object included in each image of the generated sequence data. The learning data generating device can evaluate the similarity of images based on the amount of edge change between consecutive images for each sequence data.

In addition, the learning data generating device may calculate the sharpness of images whose similarity is higher than a preset value and classify the images other than the image with the highest calculated sharpness as noise images.

Additionally, the learning data generating device can calculate the similarity between consecutive images for each sequence data and determine the number of frames per second for each sequence data based on the calculated similarity.

Then, in step S150, the learning data generating device may list and output the noise images classified in step S140 or delete the images classified as noise images.

Hereinafter, a data classification method according to another embodiment of the present invention will be described.

Figure 7 is a flowchart for explaining a data classification method according to another embodiment of the present invention.

Referring to FIG. 7, first, in step S210, the learning data generating device may request image collection from at least one learning data collecting device. At this time, the learning data generating device may transmit guide information including collection conditions to at least one learning data collection device.

Next, in step S220, the learning data generating device may receive images from at least one learning data collecting device.

Next, in step S230, the learning data generating device may extract image information corresponding to the collection conditions from the images. Here, the image information may include at least one of a file extension, image resolution, RGB (Red, Green, Blue) value for a pixel, and a color code value.

Next, in step S240, the learning data generating device may compare the extracted image information with guide information to classify noise images according to errors in the collection environment or the learning data collection device.

Specifically, the learning data generating device extracts sample image information including at least one of the file extension of the sample image, image resolution, RGB value for a pixel, and color code value, and uses the extracted sample image information extracted from the images. It can be compared with image information. At this time, the learning data generating device may classify an image whose similarity to the sample image is lower than a preset value as a noise image. In other words, the learning data generating device has a different file extension or image resolution from the sample image, or the similarity of the RGB (Red, Green, Blue) values and color code values for the pixel is lower than the preset value. In this case, the image can be classified as a noise image.

Additionally, the learning data generating device may list images in chronological order, generate sequence data by grouping the listed images into a preset number, and classify noise images for each sequence data.

At this time, the learning data generating device compares the similarity of the before and after images for a specific image among the sequence data, and the similarity between the before and after images is higher than the preset value, but the similarity between the before and after images and the specific image is lower than the preset value. In this case, a specific image may be determined to be a noise image.

In addition, the learning data generating device extracts the edges of the objects included in each of the images, detects the objects included in each of the images, and noises the images in which the position change value of the detected object is higher than the preset value. It can be classified as an image. In other words, the learning data generating device can determine whether a specific image is an image taken in the process of crossing a speed bump through the degree of movement of an object included in the image.

Additionally, the learning data generating device may receive meta information for each of the images from at least one learning data collection device. Here, the meta information may include location information and speed information of the learning data collection device at the time of capturing each image.

The learning data generation device can classify the image taken at the location of the speed bump as a noise image by comparing it with meta information based on the location information of the speed bump included in the map information including the path traveled by the learning data collection device. there is.

In addition, the learning data generation device compares the meta information based on the location information of the curved road included in the map information including the path traveled by the learning data collection device, and noises the image generated at the location of the curved road. It can be classified as an image.

In addition, the learning data generation device compares the similarity between consecutive images in the sequence data, and when images with a similarity lower than a preset value are continuously detected, it determines that the detected images are images taken on a curved road. , the detected images can be classified as noise images.

Additionally, the learning data generating device can classify noise images based on the similarity of consecutive images for each sequence data and estimate the type of error for each classified noise image.

Specifically, the learning data generating device compares the similarity between consecutive images among the sequence data, and the similarity between the first image and the second consecutive image is lower than a preset value, and the similarity between the second image and the third consecutive image is lower than the preset value. If the similarity between and the second image is higher than a preset value, it can be determined as an error in which the camera angle of the camera that captured the image included in the sequence data has changed.

In addition, the learning data generation device compares the similarity between consecutive images in the sequence data, and if the number of images with similarity lower than a preset value exceeds the preset number, the camera that captured the images included in the sequence data is bound. It can be judged as an error due to a defect.

Additionally, the learning data generating device may match received images and pre-stored images based on global positioning system (GPS) coordinates, compare the similarity between the matched images, and classify noise images.

In addition, the learning data generating device generates one data from the first side and the second side based on the similarity of the RGB values between the pixels constituting the first side and the pixels constituting the second side of each image. A vertex may be identified, and a line segment connecting the two identified vertices may be extracted from the first side and the second side.

If the learning data generating device deviates from a preset error range based on at least one of the length and angle of the line segment extracted from each matched image, it may be determined as an error in which the camera angle has changed.

However, it is not limited to this, and the learning data generating device extracts an edge from each matched image, identifies the object included in each matched image based on the extracted edge, and calculates the position change value of the identified object. Based on this, noise images can be classified.

Additionally, the learning data generating device may further receive 3D point group data acquired through lidar simultaneously with the images. The learning data generation device determines whether the type of object detected in each matched image is a moving object or a static object based on the distance information included in the 3D point cloud data, and based on the position change value of the static object among the detected objects. The noise image can be classified as follows.

Then, in step S250, the learning data generating device may list and output the classified noise images or delete the images classified as noise images.

Hereinafter, a method for predicting work costs according to an embodiment of the present invention will be described.

Figure 8 is a flowchart illustrating a method for predicting work costs according to an embodiment of the present invention.

Referring to FIG. 8, in step S310, the learning data generating device may receive from the artificial intelligence learning device 400 at least one sample data for performing a project related to annotation work scheduled to be performed for artificial intelligence learning.

Next, in step S320, the learning data generation device compares existing data included in a plurality of previously performed projects with sample data, and selects at least one project containing existing data whose similarity to the sample data is higher than a preset value. It can be extracted.

At this time, the learning data generating device may target existing data and identify one or more decomposed components that constitute the existing data.

Specifically, when the sample data corresponds to an image that is the target of annotation work, the learning data generating device may perform annotation work on the image corresponding to the sample data under user control. In addition, the learning data generation device can identify the class of the object specified from the image through annotation work and the tool used to specify the object as a decomposition component of the sample data.

Additionally, the learning data generating device may extract at least one project containing existing data whose similarity to sample data is higher than a preset value. At this time, the learning data generating device may receive weights for the decomposition components of the sample image from the artificial intelligence learning device and evaluate the similarity with the existing image by considering the input weights.

In addition, the learning data generation device extracts the edge of the sample image, detects the object included in the sample data based on the extracted edge, and includes the RGB (Red, Green, Blue) value of the object in the existing data. Similarity can be evaluated by comparing with the RGB values of the object.

In addition, the learning data generation device compares representative images pre-stored for each of the plurality of previously performed projects with sample images, and transmits a plurality of representative images whose similarity to the sample image is higher than a preset value to the artificial intelligence learning device, One of multiple representative images can be selected from the artificial intelligence learning device.

At this time, the learning data generation device transmits a plurality of representative images with a similarity to the sample image higher than a preset value to the artificial intelligence learning device, identifies objects included in the representative images, and classifies the identified objects as confidential information. If pre-registered, the identified object can be de-identified and transmitted to the artificial intelligence learning device 400.

And, in step S330, the learning data generation device may predict the total work cost required to perform a project related to annotation work to be performed based on at least one extracted project. At this time, the learning data generating device can receive the data quantity of the project related to the annotation work to be performed and predict the total work cost by considering the cost and data quantity of the extracted project.

In addition, the learning data generation device extracts a plurality of projects containing existing data whose similarity to sample data is higher than a preset value, and predicts the average work cost of the extracted multiple projects as the total work cost of the project to be performed. .

And, the learning data generating device can output the predicted total work cost. The learning data generating device may modify the overall task cost under user control. And, the learning data generating device can transmit the predicted or corrected total task cost to the artificial intelligence learning device.

Hereinafter, an annotation method according to an embodiment of the present invention will be described.

Figure 9 is a flow chart for explaining an annotation method according to an embodiment of the present invention, and Figures 10 to 16 are exemplary diagrams for explaining an annotation method according to an embodiment of the present invention.

First, as shown in FIG. 10, in step S410, the annotation device, under the control of the operator, includes a plurality of overlapping arrays included in the image that are the target of the annotation task for artificial intelligence (AI) learning. The outline of the first object (A) among the objects (A, B) can be specified.

At this time, as shown in FIG. 11, the annotation device receives a plurality of points along the outline of the first object (A) from the operator and connects the plurality of points to create an outline of the first object (A). can be formed. In other words, the annotation device can create a polygon-shaped area by connecting points specified by the operator.

Next, in step S420, the annotation device may designate the outline of the first object (A) and then designate a borderline between the first object (A) and the overlapping second object (A).

Specifically, the annotation device may select at least one point among a plurality of points created along the outline of the first object (A) and designate a boundary line based on the at least one selected point.

At this time, rather than receiving all points corresponding to the borderline from the user, as shown in FIG. 12, the annotation device selects two arbitrary points among a plurality of points, and as shown in FIG. 13, the selected You can create a line connecting at least one point that exists between two points, and designate the created line as a boundary line.

Next, as shown in FIG. 14, in step S430, the annotation device may specify the boundary line and then set the boundary line as part of the outline of the second object (B). At this time, the annotation device can output the boundary line to be distinguished from the outline of the first object (A).

Next, as shown in FIG. 15, in step S440, the annotation device can specify the outline of the second object (B) based on the set boundary line.

At this time, the annotation device may receive a plurality of points along the outline of the second object (B) from the operator and connect the boundary line and the plurality of points to form the outline of the second object (B).

In addition, the annotation device extracts the edge of the image, identifies at least one object based on the extracted edge, and identifies the remaining objects except the first object (A) among the objects including the boundary line among the at least one identified object. The edge can be designated as the outline of the second object (B).

And, as shown in FIG. 16, in step S450, the annotation device can identify the designated first object (A) and second object (B), respectively.

Hereinafter, an annotation method according to another embodiment of the present invention will be described.

Figure 17 is a flow chart for explaining an annotation method according to another embodiment of the present invention, and Figures 18 to 21 are exemplary diagrams for explaining an annotation method according to another embodiment of the present invention.

First, as shown in FIG. 18, in step S510, the annotation device creates the outlines of a plurality of objects (A, B) arranged to overlap each other included in the image that is the target of the annotation task for artificial intelligence (AI) learning. You can specify. At this time, the outline of the plurality of objects may be an outline excluding the border line.

At this time, the annotation device may receive a plurality of points as input along an outline including a plurality of objects from the operator, connect the plurality of points, and generate outlines for the plurality of objects.

In addition, the annotation device extracts the edge of the image, identifies at least one object based on the extracted edge, receives a plurality of objects selected from the identified objects by the operator, and draws a boundary line based on the extracted edge. You can create outlines for multiple selected objects, excluding them.

In addition, based on point cloud data acquired from lidar simultaneously with the image, the annotation device identifies a plurality of point clouds with a certain depth range among the points that exist inside the designated outline, and identifies the operator. A plurality of objects may be selected from among a plurality of point clouds, and outlines for the selected plurality of objects may be generated based on the identified plurality of point clouds.

Then, the annotation device sets a part of the area containing a plurality of objects as a bounding box, extracts the edge of the object from the area inside the bounding box, and creates the object and background based on the extracted edge. ), you can specify the outlines of multiple objects by deleting the background.

Next, as shown in FIG. 19, the annotation device can specify boundaries between a plurality of objects within a designated outline.

Here, the annotation device may identify a boundary line within the outline of a plurality of objects based on the edge extracted in step S510.

Additionally, the annotation device can receive input from a worker as a plurality of points located inside the outline of a plurality of objects and connect the plurality of input points to create a boundary line.

In addition, based on point cloud data acquired from LiDAR simultaneously with the image, the annotation device identifies a plurality of point clouds with a certain distance within a certain range among the points that exist inside the designated outline, and marks the boundary between the plurality of point clouds as a plurality of objects. It can be specified as a boundary line between them.

In addition, the annotation device creates groups with RGB values similar to preset values based on the RGB (Red, Green, Blue) values of pixels located inside the specified outline, and recognizes each created group as an object. And the boundary line of the recognized object can be created.

In addition, based on point cloud data acquired from LiDAR simultaneously with the image, the annotation device identifies a plurality of point clouds with a certain distance within a certain range among the points that exist inside the designated outline, and selects one object from the plurality of point clouds from the operator. A boundary line can be created based on the point cloud of the selected object.

Then, the annotation device extracts the edge inside the specified outline, identifies at least one object based on the extracted edge, selects one object among the identified objects from the operator, and creates a boundary line based on the extracted edge. can do. At this time, the annotation device can generate a plurality of points with a preset interval along the generated borderline, and modify the borderline by moving at least one of the plurality of points under the control of the operator.

And, as shown in FIG. 20, the annotation device can each identify a plurality of objects based on a plurality of areas divided based on a designated boundary line.

Figure 21 is an example diagram for explaining a data classification method according to an embodiment of the present invention.

Referring to FIG. 21, the learning data generation device lists images in chronological order, generates sequence data by grouping the listed images into a preset number, and color information of the images included for each sequence data. You can classify noise images by comparing each.

Specifically, the learning data generating device generates a color histogram for the RGB values of each of the consecutive first images (image A) and the second image (image B), and creates a color histogram for the first image (image A) based on the generated color histogram. ) and the similarity of the second image (image B) can be determined.

When the similarity between the first image (image A) and the second image (image B) is higher than a preset value, the learning data generating device noises at least one of the first image (image A) and the second image (image B). You can judge by the image.

At this time, the learning data generating device may calculate the sharpness of images whose similarity is higher than a preset value and classify the remaining images as noise images except for the image with the highest calculated sharpness.

In other words, if the similarity between the first image (image A) and the second image (image B) is determined to be high, one of the first image (image A) and the second image (image B) must be removed based on specific criteria. . To this end, the learning data generating device may classify the remaining images, excluding the image with high clarity among the selected first image (image A) and second image (image B), as noise images and delete them.

Figure 22 is an example diagram for explaining a data classification method according to another embodiment of the present invention.

Referring to FIG. 22, the learning data generating device matches received images and pre-stored images based on GPS (global positioning system) coordinates, compares the similarity between the matched images, and classifies noise images. .

That is, as shown in (A), the learning data generating device determines the first side and One vertex (point A, B) can be identified from each second side. At this time, each identified vertex can be a guide rail, which is a static object.

The learning data generating device may extract a line segment (line A) connecting two identified vertices (points A and B) from the first side and the second side, respectively.

And, as shown in (B), the learning data generating device measures the similarity of RGB values between the pixels constituting the first side and the pixels constituting the second side in the image existing at the same location as the pre-stored image. Based on , one vertex (point A, C) can be identified from the first side and the second side, respectively.

The learning data generating device may extract a line segment (line B) connecting two identified vertices (points A and C) from the first side and the second side, respectively.

Additionally, if the learning data generating device is outside a preset error range based on at least one of the length and angle of the extracted line segment, it may be determined as an error in which the camera angle has changed.

Figures 23 and 24 are exemplary diagrams for explaining an annotation method according to another embodiment of the present invention.

Referring to Figures 23 and 24, the annotation device according to another embodiment of the present invention has a plurality of trunk lines established by continuously connecting two points included in the outline and a plurality of points input under the control of the operator. You can specify a borderline by including .

In specifying the boundary line between objects, the annotation device specifies the boundary line between objects based on an edge connecting a plurality of points input from the operator, and interpolates the points between the boundary lines specified by the operator based on the edge of the object. (interpolation) can be performed.

Specifically, as shown in FIG. 24, the annotation device extracts the edge of the object located inside the specified outline based on the RGB (Red, Green, Blue) values of the pixel located inside the specified outline, and , you can create a virtual line perpendicular to the main line based on the center point of the line.

Additionally, the annotation device may interpolate points included in the trunk line where the distance (x) between the center point and the point (c) where the generated virtual line and the extracted edge intersect is greater than a preset value.

However, the annotation device is not limited to this, and may interpolate points included in a plurality of trunk lines whose lengths exceed a preset value. That is, based on the length of the trunk line, the annotation device may determine that an trunk line whose length is longer than a preset value is an trunk line that requires interpolation, and may perform interpolation for points included in the trunk line.

In one embodiment, the annotation device may add a new point for interpolation at the point (c) where the virtual line and the extracted edge intersect.

Additionally, the annotation device can correct points input from the operator based on the user's profile.

Here, the user's profile may include information about the input tool for the worker to input points and the worker's dominant hand for using the input tool.

Specifically, the annotation device can obtain the worker's profile and identify an input tool for inputting a plurality of points and a dominant hand for using the input tool based on the obtained profile.

For example, the annotation device can identify that the worker uses a mouse as an input tool and uses the right hand as the dominant hand.

The annotation device acquires correction values corresponding to the identified input tool and dominant hand from a pre-stored correction table according to the input tool and dominant hand, and batches a plurality of points input from the operator based on the obtained correction values. It can be corrected.

For example, when a worker uses a mouse and uses the right hand as the dominant hand, the points entered by the worker tend to be biased toward the bottom left during input with the mouse button.

Accordingly, the annotation device can pre-store a correction table according to the profile and collectively correct a plurality of points input according to the identified profile.

Additionally, the annotation device can collectively correct a plurality of points input from the operator based on the correction history accumulated for the operator.

For example, the annotation device may correct a plurality of input points based on a correction table according to a profile and adjust the correction strength by assigning weights based on accumulated correction history.

Additionally, the annotation device can extract a 3D model corresponding to the type of the identified object and verify a plurality of points input from the operator.

Specifically, the annotation device may estimate the type of the object based on the outline of the identified object and extract a 3D model stored in advance for each type of the estimated object.

For example, the annotation device may estimate the type of object as a specific car based on the shape of the outline of the identified object and extract a 3D model corresponding to the specific car.

The annotation device can rotate the extracted 3D model in 3D to identify the shooting direction of the object, and identify preset essential components for each object in response to the identified shooting direction.

For example, while rotating the extracted 3D model in 3D, the annotation device can identify a point in time when the similarity of the outline with the identified object is higher than a preset value in the shooting direction of the object. If the identified object is a car, Tires, bonnet, side mirrors, trunk, etc. can be identified as essential components.

Afterwards, the annotation device can determine whether essential components exist within the closed area of the identified object and verify the input points under the operator's control.

At this time, if an essential component does not exist in the identified object, the annotation device can display a specific symbol in the corresponding area so that the operator can intuitively recognize the area and correct the points entered in the area.

For example, the annotation device can transplant the outlines of essential components identified in the 3D model to the recognized object and display them in different colors or thicknesses.

Figures 25 and 26 are example diagrams for explaining outlines of a plurality of overlapping objects.

First, as shown in FIG. 25, in step S510, the annotation device may specify the outlines of a plurality of objects arranged to overlap each other included in an image that is the target of an annotation task for artificial intelligence (AI) learning. At this time, the outline of the plurality of objects may be the outline excluding the borderline.

Hereinafter, an example will be given where at least part of the first object (A), second object (B), and third object (C) are arranged to overlap each other.

Next, the annotation device can specify an overlapped borderline between the first object (A), the second object (B), and the third object (C) within the designated outline.

To be described in detail with reference to FIG. 25, when there is a first object (A), a second object (B), and a third object (C) overlapping with each other in the image that is the target of the annotation work, step S510 is the first object (A), the second object (B), and the third object (C). Specify a thick line (outline) including the object (A), the second object (B), and the third object (C), and in step S520, the first object (A) and the second object (B) are drawn within the thick line. , You can specify an overlapping thin line (borderline) of the third object (C).

Here, the thin line (borderline) is, as shown in FIG. 26, the dashed line border line BL1 at the overlap between the first object A and the third object C, the first object A and It can be designated as a solid line boundary line (BL2) at the portion where the second object (B) overlaps, and a dotted line boundary line (BL3) at the portion where the second object (B) and the third object (C) overlap. One of the boundaries (BL1, BL2, BL3) is formed by the outlines of at least two objects (A, B, C).

That is, the dashed line border BL1 can form the outline of the first object A and at the same time the outline of the third object C, and the solid line border BL2 can form the outline of the first object A. It is possible to form the outline of the second object (B) at the same time as forming the outline, and the dotted border line (BL3) can form the outline of the second object (B) and the outline of the third object (C) at the same time. there is.

Through this, the present invention creates a plurality of overlapping boundary lines between a plurality of objects using a polygon technique (e.g., a solid line boundary line (BL2), which is the boundary line between the first object (A) and the second object (B), of each object. If it is specified twice as an outline, the problem that the twice-designated solid line boundary line (BL2) does not match can be solved.

As described above, although preferred embodiments of the present invention have been disclosed in the specification and drawings, it is known in the technical field to which the present invention belongs that other modifications based on the technical idea of the present invention are possible in addition to the embodiments disclosed herein. It is self-evident to those with ordinary knowledge. In addition, although specific terms are used in the specification and drawings, they are merely used in a general sense to easily explain the technical content of the present invention and aid understanding of the invention, and are not intended to limit the scope of the present invention. Accordingly, the above detailed description should not be construed as restrictive in all respects and should be considered illustrative. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: Learning data collection device 200: Learning data generation device
300: Annotation device 400: Artificial intelligence learning device
205: communication unit 210: input/output unit
215: data design department 220: data collection department
225: data purification unit 230: data delivery unit
235: storage unit 305: communication unit
310: input/output unit 315: storage unit
320: Object identification unit 325: Object property setting unit
330: Result generation unit

Claims (10)

A step in which the annotation device specifies an outline containing a plurality of objects arranged to overlap each other included in an image that is the target of annotation work for artificial intelligence (AI) learning, under the control of the operator. ;
specifying, by the annotation device, a boundary line between the plurality of objects to distinguish the objects included within the designated outline from each other; and
identifying, by the annotation device, each of the plurality of objects for each of a plurality of areas divided based on the designated boundary line; Including,
According to claim 1,
The step of specifying the outline is
Characterized by specifying the entire outline of a plurality of objects including at least a first object and a second object arranged to overlap each other,
The step of specifying the boundary line is
Characterized by specifying an overlapped boundary line between the first object and the second object within the designated outline,
The boundary line is,
An annotation method characterized by forming an outline of a portion of the first object that overlaps with the second object and simultaneously forming an outline of a portion of the second object that overlaps with the first object.
According to claim 1,
The step of specifying the boundary line is
An annotation method, characterized in that the boundary line is designated by including a plurality of trunk lines established by continuously connecting two points included in the outline and a plurality of points input under the control of the operator.
The method of claim 1, wherein the step of designating the outline is
An annotation method, characterized in that: receiving a plurality of points along an outline including the plurality of objects from the worker, connecting the plurality of points, and generating an outline for the plurality of objects.
The method of claim 1, wherein the step of designating the outline is
Set a partial area containing the plurality of objects as a bounding box, extract the edge of the object from the area inside the bounding box, and select the object and background based on the extracted edge. An annotation method, characterized in that designating the outline of the plurality of objects by distinguishing them and deleting the background.
The method of claim 1, wherein the step of designating the boundary line is
The edge of the object located inside the specified outline is extracted based on the RGB (Red, Green, Blue) value of the pixel located inside the specified outline, and the edge of the object located inside the specified outline is perpendicular to the edge based on the center point of each of the plurality of edges. An annotation method that generates a virtual line and interpolates points included in the edge where the distance between the center point and the point where the generated virtual line and the extracted edge intersect is greater than a preset value.
The method of claim 6, wherein the step of designating the boundary line is
An annotation method that adds a new point for interpolation at the point where the virtual line and the extracted edge intersect.
The method of claim 6, wherein the step of designating the boundary line is
An annotation method, characterized in that interpolating points included in an edge whose length of each of the plurality of edges exceeds a preset value.
The method of claim 1, wherein the step of designating the boundary line is
Obtain a profile of the worker, identify an input tool for inputting the plurality of points and a dominant hand for using the input tool based on the obtained profile, and identify the identified input tool and the An annotation method characterized by obtaining a correction value corresponding to the dominant hand and collectively correcting a plurality of points input from the operator based on the obtained correction value.
The method of claim 1, wherein the identifying step
Estimating the type of object based on the outline of the identified object, extracting a 3D model stored in advance for each type of the estimated object, rotating the extracted 3D model in 3D to identify the shooting direction of the object, and Characterized by identifying essential components for each preset object in response to the identified shooting direction, determining whether essential components exist within the closed area of the identified object, and verifying the input points under the operator's control. Annotation method.

Images