WO2022265262A1 - Method for extracting data for artificial intelligence training based on big data, and computer program recorded on recording medium in order to execute same - Google Patents

Abstract

Proposed is a method whereby data for artificial intelligence training based on big data can be extracted. The method may comprise a step in which a training data generation device receives a plurality of annotation task results, classifies the plurality of annotation task results on the basis of metadata, and generates data for artificial intelligence (AI) training.

Description

Data extraction method for artificial intelligence learning based on big data and computer program recorded on a recording medium to execute it

The present invention relates to the processing of artificial intelligence (AI) learning data. More specifically, it relates to a method for extracting artificial intelligence learning data that meets needs based on preformed big data and a computer program recorded on a recording medium to execute the method.

Artificial intelligence (AI) refers to a technology that artificially implements some or all of human learning abilities, reasoning abilities, and perception abilities using computer programs. In relation to artificial intelligence (AI), machine learning refers to learning to optimize parameters with given data using a model composed of multiple parameters. Such machine learning is classified into supervised learning, unsupervised learning, and reinforcement learning according to the form of learning data.

In general, designing data for artificial intelligence (AI) learning proceeds in the steps of data structure design, data collection, data refinement, data processing, data expansion, and data verification.

To describe each step in more detail, data structure design is performed through ontology definition, classification system definition, and the like. Data collection is performed by collecting data through direct filming, web crawling, or associations/professional organizations. Data purification is performed by removing redundant data from collected data and de-identifying personal information. Data processing is performed by performing annotation and inputting metadata. Data extension is performed by performing ontology mapping and supplementing or extending the ontology as needed. In addition, data verification is performed by verifying validity according to the set target quality using various verification tools.

Data processing is also referred to as data labeling. More specifically, the annotation work of data processing is performed by a worker processing a bounding box for an object included in an image. Here, the bounding box is an area for specifying the position and shape of an object included in the image.

In addition, the metadata input operation of data processing is performed by a worker inputting image information, copyright information, shooting condition information, annotated object information, environment information, and the like. For example, image information may include a file name, file size, and image size. Copyright information may include information about the copyright holder of the image. The photographing condition information may include resolution, bit value, aperture transmittance, exposure time, ISO sensitivity, focal length, aperture value, angle of view, white balance, and RGB depth. Object information may include annotation type, coordinate value, area size, class name, tag item, clipping, major classification, middle classification, small classification, and instance information. In addition, the environment information may include a shooting date and time, a shooting location, and weather information.

The number and type of information that can be input in the metadata input process of data processing may vary depending on the purpose of artificial intelligence (AI) learning. For example, in the case of artificial intelligence (AI) learning data for performing automatic driving, environment information may further include the type of road on which an image is captured, road surface information, traffic congestion information, and the like. can

As described above, data processing is performed on a large number of data ranging from a few thousand to several million. Accordingly, various means capable of more easily processing a large number of data and more easily verifying the processed data are required.

One object of the present invention is to provide a method capable of extracting data for artificial intelligence (AI) learning that meets a specific purpose using metadata from big data.

Another object of the present invention is to provide a computer program recorded on a recording medium to execute a method capable of generating data for artificial intelligence (AI) learning that meets a specific purpose from big data.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the technical problem as described above, the present invention proposes a method for generating artificial intelligence (AI) learning data that meets a specific purpose from big data using metadata. The method includes the steps of distributing, by a learning data generating device, a plurality of images to be annotated; A plurality of annotation devices specify the coordinates of an object included in each image for the distributed plurality of images, and generate coordinates of the specified object in the image and metadata for the image. and generating an annotation work result by including the generated metadata; and receiving, by the learning data generating device, the plurality of annotation job results, classifying the plurality of annotation job results based on the metadata, and generating data for artificial intelligence (AI) learning. can do.

In the step of generating data for artificial intelligence (AI) learning, big data is built using the annotation work result classified based on the metadata, and when a requested value is input from the outside, the big data is mined. An annotation work result corresponding to the requested value may be extracted by mining or filtering, and the extracted annotation work result may be packaged to generate the artificial intelligence (AI) learning data. In this case, the requested value may be a value indicating the type of information to be included in the AI learning data.

For example, the big data includes a hash table composed of a plurality of buckets according to the type of the metadata, and each of the buckets includes a link to an annotation work result. It may be configured to include more than one slot (slot) of. In addition, the step of generating the AI learning data is to search for one or more slots by substituting the requested value into a hash function previously set in correspondence with the hash table, and to search for one or more slots. You can extract one or more annotation work outputs through the included links.

In the step of generating artificial intelligence (AI) learning data, if the loading density of the hash table exceeds a preset critical density in the process of constructing the big data, the link included in the slot is maintained. In one state, after doubling the number of buckets according to the type of metadata, the capacity of the hash table may be increased by readjusting the home bucket of the slot.

In the step of generating data for artificial intelligence (AI) learning, when the type of information indicated by the request value is not included in the metadata, the data structure is such that the type of information not included is included in the metadata. After redesigning the data structure, the plurality of images may be redistributed to the plurality of annotation devices.

In the step of generating artificial intelligence (AI) learning data, the big data is divided into a plurality of layers, and in the process of constructing the big data, based on the job evaluation grade given to the user of the annotation device, the A layer to which the annotation work result belongs may be determined, and the annotation work result may be extracted from each layer according to a ratio corresponding to the request value input from the outside.

On the other hand, in the step of generating the annotation work result, when the plurality of images correspond to images continuously photographed by the same camera, environmental information previously given to an image photographed in advance among the plurality of images is followed by It is possible to generate metadata for an image taken later by including it as environment information of a captured image. In this case, the environment information may include information on the date and time of image capture, location, weather information, road type, road surface, and traffic congestion.

In the step of generating the annotation work result, if there are a plurality of previously photographed images, environment information previously assigned to an image including an object to which the same tracking identifier is assigned among the plurality of images is selected. It can be included as environment information of images taken later. In this case, the tracking ID may be a unique identifier of an object assigned to track the object.

The step of generating the annotation work result is based on a result of comparing the background area obtained by removing the area occupied by the object to which the same ID is assigned for each of the preceding and succeeding images, It may be determined whether or not the captured image can be applied as the environment information of the captured image by following the environment information given to the captured image.

In order to achieve the technical problem as described above, the present invention proposes a computer program recorded on a recording medium to execute the verification method as described above. The computer program may include a memory; transceiver; and a processor configured to process instructions resident in the memory. The computer program may further include distributing, by the processor, a plurality of images to be annotated through the transceiver; receiving, by the processor, the plurality of annotation job results through the transceiver, the annotation job result including coordinates of an object in the image for each of the plurality of images and metadata for the image; and generating data for artificial intelligence (AI) learning by the processor classifying the plurality of annotation work results based on the metadata, which may be a computer program recorded on a recording medium.

Details of other embodiments are included in the detailed description and drawings.

According to conventional means, when AI learning data is requested from a customer who wants to learn AI, after that, data structure design, data collection, data refinement, data processing, It took a relatively long time to expand data, verify data, generate artificial intelligence (AI) learning data based on the results, and deliver it to customers.

In contrast, according to the embodiments of the present invention, big data built in advance in response to the needs of customers who want to learn artificial intelligence (AI) is simply mined or filtered for artificial intelligence (AI) learning. By generating data and immediately delivering the generated AI learning data, the period required to generate and deliver AI learning data can be greatly shortened.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 and 2 are configuration diagrams of an artificial intelligence learning system according to various embodiments of the present invention.

3 is a logical configuration diagram of an apparatus for generating learning data according to an embodiment of the present invention.

4 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.

5 is a hardware configuration diagram of an annotation device according to an embodiment of the present invention.

6 is a signal flow diagram for explaining a method of generating data for artificial intelligence (AI) learning according to an embodiment of the present invention.

7 is a conceptual diagram for explaining the concept of generating data for artificial intelligence (AI) learning according to an embodiment of the present invention.

8 is an exemplary view for explaining the concept of a tracking ID according to an embodiment of the present invention.

9 is an exemplary diagram for explaining a user interface (UI) for checking a tracking ID according to an embodiment of the present invention.

10 and 11 are exemplary diagrams for explaining a process of determining environment information in consecutive images according to some embodiments of the present invention.

12 and 13 are exemplary diagrams for explaining a process of automatically determining environment information according to some embodiments of the present invention.

14 is an exemplary diagram for explaining a user interface (UI) for inspecting metadata according to an embodiment of the present invention.

15 is a flowchart illustrating an operation process of an annotation device according to an embodiment of the present invention.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in this specification should be interpreted in terms commonly understood by those of ordinary skill in the art to which the present invention belongs, unless specifically defined otherwise in this specification, and are overly inclusive. It should not be interpreted in a positive sense or in an excessively reduced sense. In addition, when the technical terms used in this specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that those skilled in the art can correctly understand. In addition, general terms used in the present invention should be interpreted as defined in advance or according to context, and should not be interpreted in an excessively reduced sense.

Also, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "consisting of" or "having" should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or steps are included. It should be construed that it may not be, or may further include additional components or steps.

Also, terms including ordinal numbers such as first and second used in this specification may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. On the other hand, when a component is referred to as “directly connected” or “directly connected” to another component, it should be understood that no other component exists in the middle.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings. The spirit of the present invention should be construed as extending to all changes, equivalents or substitutes other than the accompanying drawings.

Processing of images to create data for artificial intelligence (AI) learning is performed on a large number of images ranging from a few thousand to several million. Therefore, according to the conventional means, it took a relatively long time to generate AI learning data and deliver it to the customer after receiving the customer's request to learn artificial intelligence (AI).

In order to solve these difficulties, the present invention intends to propose various means capable of more easily processing a large number of images and more easily verifying the processed data.

1 and 2 are various of the present invention in the examples It is a configuration diagram of an artificial intelligence learning system according to

As shown in FIG. 1, the artificial intelligence learning system according to an embodiment of the present invention includes a learning data generating device 100, one or more annotation devices 200-1, 200-2, ..., 200-n; 200 , It may be configured to include a learning data verification device 300 and an artificial intelligence learning device 400.

In addition, as shown in FIG. 2, the artificial intelligence learning system according to another embodiment of the present invention includes one or more annotation devices (200-a, 200-b, ..., 200-m; 200) and a learning data verification device ( 300-a, 300-b, ..., 300-m; 300) consisting of a plurality of groups (Group-a, Group-b ..., Group-m), the learning data generating device 100 and the artificial intelligence learning device ( 400) may be configured.

Since the components of the artificial intelligence learning system according to various embodiments are merely functionally distinct elements, two or more components are integrated and implemented in the actual physical environment, or one component is implemented in the actual physical environment. may be implemented separately from each other.

Each component will be described. The learning data generating device 100 is a device that can be used to design and generate artificial intelligence (AI) learning data. As such, the learning data generating device 100 is basically a device that is distinguished from the learning data verifying device 300, but in an actual physical environment, the learning data generating device 100 and the learning data verifying device 300 are integrated into one device. It may be integrated and implemented.

Characteristically, the learning data generating device 100 according to an embodiment of the present invention preemptively generates artificial intelligence (AI) data before receiving a request for artificial intelligence (AI) learning data from the artificial intelligence learning device 400. It is possible to build big data that can generate data for learning.

In addition, when the learning data generating device 100 receives a request value according to the customer's needs from the artificial intelligence learning device 400, it mines or filters the preemptively built big data to generate artificial intelligence (AI). Data for learning may be generated, and the generated data for artificial intelligence (AI) learning may be transmitted to the artificial intelligence learning device 400 .

The learning data generation device 100 having such characteristics transmits and receives data with the annotation device 200, the learning data verification device 300, and the artificial intelligence learning device 400, and performs calculations based on the transmitted and received data. Any device that can do this is acceptable. For example, the learning data generating device 100 may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto.

A detailed configuration and operation of the learning data generating device 100 will be described later with reference to FIG. 3 .

With the following configuration, the annotation device 200 is a device that can be used to annotate images distributed by the learning data generating device 100.

Characteristically, the annotation device 200 according to an embodiment of the present invention tracks a specific object from each image distributed by the learning data generating device 100 (ie, an annotation target image). A tracking identifier can be assigned. In addition, the annotation device 200 may automatically assign environment information to each image distributed by the learning data generating device 100.

The annotation device 200 having such characteristics may be any device capable of transmitting and receiving data to and from the learning data generating device 100 and the learning data verifying device 300 and performing calculations based on the transmitted and received data. may be allowed For example, the annotation device 100 may be a stationary computing device such as a desktop, workstation, or server, or a smart phone, a laptop, a tablet, a phablet, or a portable multimedia player. (Portable Multimedia Player, PMP), personal digital assistants (PDAs), or e-book readers (E-book reader).

In addition, all or part of the annotation device 200 may be a device in which an annotation worker performs annotation through a clouding service.

A detailed configuration and operation of the annotation device 200 will be described with reference to FIGS. 4 and 5 later.

With the following configuration, the learning data verification device 300 is a device that can be used to verify artificial intelligence (AI) learning data. That is, the learning data verification device 300 verifies whether the annotation task result generated by the annotation device 200 meets a pre-set target quality or whether the annotation task result is valid for artificial intelligence (AI) learning. It is a device that can

Specifically, the learning data verification device 300 may receive an annotation job result from the annotation device 200 . Here, the annotation work result may include coordinates of an object specified by an annotation worker and metadata about an image or object. The metadata of the annotation work result may include image information, copyright information, photographing condition information, specific object information, and environment information, but is not limited thereto. Such an annotation work result may have a JSON (Java Script Object Notation) file format, but is not limited thereto.

The learning data verification device 300 may inspect the received annotation work result. To this end, the learning data verification device 300 may perform inspection using a script for annotation work results. Here, the script is a code for verifying whether or not the target quality previously set for the annotation work result meets or whether the data is valid.

Characteristically, the learning data verification device 300 according to the embodiments of the present invention can provide a user interface (UI) that can more easily inspect the tracking ID and environment variables of objects included in the annotation work result. .

Specifically, the learning data verification apparatus 300 outputs a user interface (UI) capable of verifying the metadata of the annotation work result, a cropped image of an object to which the same tracking ID is assigned from images included in the image group. After collecting (cropped images), the cropped images collected on the user interface (UI) may be sequentially arranged and output according to the order of the images taken.

Here, the image group is a set of annotated images. Such image groups may be formed for each annotation device 200 or for each project. For example, the image group includes images taken consecutively in time series by the same camera as the camera that took the image to be annotated, and other images arranged adjacent to the camera that took the image to be annotated. Images spatially continuously photographed by a camera may be included, but are not limited thereto.

Also, the cropped image is an image obtained by cropping a part of an image that is an annotation work target based on coordinates set to specify an object by annotation.

In addition, the learning data verification apparatus 300 compares patterns of RGB (Red, Green, Blue) values and patterns of edges of objects included in each of the cropped images to crop images of objects to which the same tracking ID is assigned. In cross-contrast, it is possible to self-inspect whether the same tracking ID is assigned to each object.

Conversely, the learning data verification apparatus 300 cross-contrasts the RGB value pattern and the edge pattern of objects to which different tracking IDs have been assigned, targeting cropped images of objects to which different tracking IDs have been assigned. It is also possible to check whether different tracking IDs are assigned to each other.

In this case, the pattern of RGB values means a pattern of a ratio of RGB values to pixels in an image and an arrangement order of RGB values. And, the pattern of the edge means a pattern for the number of pixels included in an enclosure closed by the edge extracted from the image and the relative position where the edge is located in the image.

In addition, when the learning data verification device 300 determines that it is incorrect to assign the same tracking ID or to assign different tracking IDs, the tracking ID that is determined to be incorrect is displayed on the user interface (UI). Information about the assigned object can be output.

In this way, the learning data verification device 300 outputs a user interface (UI) in which cropped images of a plurality of objects to which the same tracking ID is assigned are successively arranged, and the result of self-checking whether the tracking ID of the objects is correctly assigned By outputting a user interface (UI) including the user interface (UI), the user of the training data verification device 300 can easily check and inspect a plurality of objects determined to be the same at a glance.

On the other hand, in outputting a user interface (UI) capable of verifying the metadata of the annotation work result, the learning data verification apparatus 300 has a figure having a preset color corresponding to a value included in each metadata. ), each of the images included in the image group may be graphicalized and included in the user interface (UI).

For example, the learning data verification apparatus 300 may express each of the images included in the image group as a single line having a color corresponding to a value included in metadata. The training data verification apparatus 300 may construct bar-shaped graphics by continuously arranging each line segment in the direction of the width of the line segment according to the sequence of images captured. In addition, the training data verification device 300 may output the configured bar-shaped graphic by including it in the user interface (UI).

In this way, the learning data verification apparatus 300 outputs a user interface (UI) in which a plurality of images are simply graphic according to the value of the metadata, so that the user of the learning data verification apparatus 300 can view the metadata of the plurality of images. can be easily checked at a glance, and an image having a specific metadata value unlike other images can be intuitively identified.

Also, the learning data verification device 300 may transmit annotation work results and inspection results received from the annotation devices 200 to the learning data generating device 100 .

The learning data verification device 300 having the above characteristics can transmit and receive data with the annotation device 200 and the learning data generating device 100 and perform calculations based on the transmitted and received data. Any device may be acceptable. For example, the learning data verification device 300 may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto.

With the following configuration, the artificial intelligence learning device 400 is a device that can be used to develop artificial intelligence (AI).

Specifically, the artificial intelligence learning device 400 provides the learning data generating device 100 with a request value including requirements that AI learning data must satisfy in order for AI to achieve its development purpose. can transmit The artificial intelligence learning device 400 may receive artificial intelligence (AI) learning data from the learning data generating device 100 . In addition, the artificial intelligence learning device 400 may perform machine learning on artificial intelligence (AI) to be developed using the received artificial intelligence (AI) learning data.

As such, the artificial intelligence learning device 400 may be any device capable of transmitting and receiving data to and from the learning data generating device 100 and performing calculations using the transmitted and received data. For example, the artificial intelligence learning device 400 may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto.

As described above, the learning data generating device 100, one or more annotation devices 200, the learning data verification device 300, and the artificial intelligence learning device 400 are directly connected to each other through a security line and a public wired communication network. Alternatively, data may be transmitted and received using a network in which one or more of the mobile communication networks are combined.

For example, public wired communication networks may include Ethernet, x Digital Subscriber Line (xDSL), Hybrid Fiber Coax (HFC), and Fiber To The Home (FTTH). It may be, but is not limited thereto. In addition, in the mobile communication network, Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), High Speed Packet Access (HSPA), Long Term Evolution, LTE) and 5th generation mobile telecommunication may be included, but is not limited thereto.

Hereinafter, the configuration of the learning data generating device 100 as described above will be described in more detail.

3 is one of the present invention in the examples It is a logical configuration diagram of the learning data generating device according to FIG.

As shown in FIG. 3, the learning data generating device 300 includes a communication unit 105, an input/output unit 110, a storage unit 115, a data structure design unit 120, a data collection and refinement unit 125, and a data It may include a processing unit 130, a big data management unit 135, and a learning data generation unit 140.

Since the components of the learning data generating device 100 are merely functionally distinct elements, two or more components are integrated and implemented in an actual physical environment, or one component is integrated with each other in an actual physical environment. It could be implemented separately.

Describing each component, the communication unit 105 may transmit/receive data with one or more of the annotation device 200, the learning data verification device 300, and the artificial intelligence learning device 400.

Specifically, the communication unit 105 may transmit a plurality of images to be annotated to the annotation device 200 . The communication unit 105 may receive annotation work results and inspection results from the learning data verification device 300 .

The communication unit 105 may receive a request value from the artificial intelligence learning device 400 . Here, the required value is a value related to requirements that AI learning data must satisfy in order for AI to achieve its development purpose. That is, the requested value is a value indicating the type of information to be included in the data for AI learning. Also, the communication unit 105 may transmit artificial intelligence (A) learning data to the artificial intelligence learning device 400 .

With the following configuration, the input/output unit 110 may receive a signal from a user through a user interface (UI) or output an operation result to the outside.

Specifically, the input/output unit 110 may receive a control signal for designing a data structure for artificial intelligence (AI) learning from a user. The input/output unit 110 may receive data (ie, images) for artificial intelligence (AI) learning from the outside. The input/output unit 110 may receive an input of an allocation amount for distributing annotation work to a plurality of annotation devices 200 from a user.

The input/output unit 110 may output state information of the big data managed by the big data management unit 135 . Also, the input/output unit 110 may output the requested value received from the artificial intelligence learning device 400 through the communication unit 105 .

With the following configuration, the storage unit 115 may store data for generating artificial intelligence (AI) learning data.

Specifically, the storage unit 115 may store a data structure designed by the data structure design unit 120 . The storage unit 115 may store data (ie, images) for artificial intelligence (AI) learning collected by the data collection and refinement unit 125 . The storage unit 115 may store annotation work results received from the annotation device 200 or the learning data verification device 300 .

Also, the storage unit 115 may store information about big data being generated and managed by the big data management unit 135 . The storage unit 115 according to another embodiment of the present invention may store information about big data being generated and managed by the big data management unit 135 through an external storage device.

With the following configuration, the data structure design unit 120 may design a data structure for artificial intelligence (AI) learning.

Specifically, the data structure design unit 120 may design a data structure for artificial intelligence (AI) learning based on user control input through the input/output unit 110 . For example, the data structure design unit 120 may define an ontology for artificial intelligence (AI) learning and a data classification system for artificial intelligence (AI) learning based on user control. In this case, the user of the learning data generating device 100 can control the structure of data for artificial intelligence (AI) learning to have versatility in consideration of the needs of the main business partner or customer.

With the following configuration, the data collection and refinement unit 125 may collect and refine data for artificial intelligence (AI) learning based on the data structure designed by the data structure design unit 120 .

Specifically, the data collection and refinement unit 125 may receive an image from the outside through the input/output unit 110 based on the designed data structure. The data collection and refinement unit 125 may collect images by performing web crawling through the communication unit 105 . Also, the data collection and refinement unit 125 may download an image from an external institution's device having image data.

The data collection and refinement unit 125 may remove redundant or extremely similar images from among collected images. Also, the data collection and refinement unit 125 may de-identify personal information included in the collected images.

With the following configuration, the data processing unit 130 may process the collected and refined data (ie, the image to be annotated) using the annotation device 200 .

Specifically, the data processing unit 130 distributes the collected and refined data (ie, images to be annotated) to the plurality of annotation devices 200 based on the quota set through the input/output unit 110 can be sent.

The data processing unit 130 may receive annotation work results directly from the annotation device 200 or receive annotation work results and inspection results from the learning data verification device 300 through the communication unit 105 .

With the following configuration, the big data management unit 135 may construct and manage big data related to a plurality of annotation work results based on metadata.

Specifically, the big data management unit 135 performs a plurality of annotation tasks received from the annotation device 200 or the learning data verification device 300 through the data processing unit 130 based on the metadata included in the annotation task result. Results can be categorized.

Here, the metadata included in the annotation work result may include image information, copyright information, shooting condition information, annotated object information, and environment information, but is not limited thereto. Image information included in metadata may include a file name, file size, and image size. Copyright information may include information about the copyright holder of the image. The photographing condition information may include resolution, bit value, aperture transmittance, exposure time, ISO sensitivity, focal length, aperture value, angle of view, white balance, and RGB depth. Object information may include annotation type, coordinate value, area size, class name, tag item, clipping, major classification, middle classification, small classification, and instance information. The environmental information may include shooting date and time, shooting location, weather information, road type, road surface information, and traffic jam information. In addition, the number and type of information that may be included in metadata is not limited thereto.

The big data management unit 135 may build big data using a plurality of annotation work results classified based on metadata. The big data management unit 135 may include a plurality of annotation work results received after the big data is pre-constructed in the pre-constructed big data.

In this way, the big data constructed by the big data management unit 135 may include a hash table. A hash table included in big data may be composed of a plurality of buckets according to the type of metadata. Each of the plurality of buckets constituting the hash table may include one or more slots. In addition, each slot included in the bucket may include a link to an annotation work result.

On the other hand, the big data management unit 135, in the process of constructing big data, when the load density of the hash table exceeds a preset critical density, maintains the links included in each slot and stores the metadata. The capacity of the hash table can be increased by doubling the number of buckets according to the type of , and then readjusting only the home bucket of each slot.

Unlike this, the big data constructed by the big data management unit 135 may be partitioned into a plurality of layers. In this case, the big data management unit 135, based on the job evaluation grade given to the user of the annotation device 200 in the process of constructing big data, the annotation work result generated by the corresponding annotation device 200 belongs to. A layer of big data may be determined, and a corresponding annotation work result may be added to the determined layer of big data.

With the following configuration, the learning data generating unit 140 may generate artificial intelligence (AI) learning data using the big data built by the big data management unit 135 .

Specifically, when a request value is input or received from the outside through the input/output unit 110 or the communication unit 105, the learning data generator 140 mines or filters the built big data to obtain the requested value. Annotation work results corresponding to can be extracted.

Here, the required value is a value related to requirements that AI learning data must satisfy in order for AI to achieve its development purpose. That is, the requested value is a value indicating the type of information to be included in the data for AI learning.

For example, the learning data generating unit 140 may search for one or more slots by substituting a requested value into a hash function set in advance corresponding to a hash table of big data. The training data generator 140 may extract one or more annotation work results through links respectively included in one or more searched slots.

Alternatively, the learning data generation unit 140 may extract annotation work results from each layer of the big data composed of a plurality of layers according to a preset ratio corresponding to the requested value.

In addition, the learning data generation unit 140 may generate AI learning data by packaging the extracted annotation work result. And, the learning data generation unit 140 may transmit the generated artificial intelligence (AI) learning data to the artificial intelligence learning device 400 through the communication unit 105 .

On the other hand, when the type of information indicated by the request value is not included in the metadata of the annotation work result, the learning data generator 140 allows the type of information not included in the metadata to be included in the metadata, The data structure may be redesigned through the data structure design unit 120 . Also, the learning data generation unit 140 may redistribute the plurality of images to the plurality of annotation devices 200 through the data processing unit 130 to redistribute the annotation work.

Hereinafter, the configuration of the annotation device 200 as described above will be described in more detail.

4 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.

As shown in FIG. 4, the annotation device 200 includes a communication unit 205, an input/output unit 210, a storage unit 215, an object specification unit 220, a metadata generation unit 225, and a result generation unit ( 230) may be configured.

Since the components of the annotation device 200 are merely functionally distinct elements, two or more components are integrated and implemented in an actual physical environment, or one component is separated from each other in an actual physical environment. could be implemented.

Describing each component, the communication unit 205 may transmit/receive data with the learning data generating device 100 and the learning data verifying device 300 .

Specifically, the communication unit 205 may receive a plurality of images distributed by the learning data generating device 100 . Here, the image is an image that is a target of annotation work for artificial intelligence (AI) learning. Such a plurality of images may be received individually or collectively.

And, the communication unit 205 may transmit the annotation work result to the learning data verification device 300 . Such an annotation work result may have a JSON file format, but is not limited thereto.

With the following configuration, the input/output unit 210 may receive a signal from a user through a user interface (UI) or output an operation result to the outside. Here, the user means a person who performs annotation work. Such a user may be referred to as a worker, performer, labeler, data labeler, and the like, but is not limited thereto.

Specifically, the input/output unit 210 may output an image to be an annotation work. The input/output unit 110 may receive a control signal for setting a bounding box from a user. Also, the input/output unit 110 may overlay and output a bounding box on the image.

Also, the input/output unit 110 may receive a control signal for setting metadata of an image from a user.

With the following configuration, the storage unit 215 may store data required for annotation work.

Specifically, the storage unit 215 may store an image received through the communication unit 205 . Also, the storage unit 215 may store project properties, image properties, or user properties received through the communication unit 205 .

With the following configuration, the object specifying unit 220 may specify an object included in an image to be annotated.

Specifically, the object specifying unit 220 may load an image to be an annotation work into memory. The object specifying unit 220 may select a tool according to a user's control signal input through the input/output unit 210 . Here, the tool is a tool for setting a bounding box that specifies one or more objects included in the image.

The object specifying unit 220 may receive coordinates through the selected tool according to a control signal input through the input/output unit 210 . The object specifying unit 220 may specify an object included in the image by setting a bounding box based on the input coordinates.

Here, the bounding box is an area for specifying an object to be learned by artificial intelligence (AI) among objects included in the image. Such a bounding box may have a rectangle or polygon shape, but is not limited thereto.

For example, the object specification unit 220 receives two coordinates from the user through the input/output unit 210, and has the input two coordinates as the coordinates of the upper left vertex and the coordinates of the lower right vertex in the image. An object included in an image can be specified by setting a bounding box based on a rectangle.

In this case, the two coordinates may be set by the user inputting one type of input signal twice (eg, mouse click) or by the user inputting two types of input signal once (eg, mouse drag). It may, but is not limited thereto.

With the following configuration, the metadata generating unit 225 may generate metadata for an image that is a target of annotation work.

Here, the metadata may include image information, copyright information, shooting condition information, annotated object information, and environment information, but is not limited thereto. Image information included in metadata may include a file name, file size, and image size. Copyright information may include information about the copyright holder of the image. The photographing condition information may include resolution, bit value, aperture transmittance, exposure time, ISO sensitivity, focal length, aperture value, angle of view, white balance, and RGB depth. Object information may include annotation type, coordinate value, area size, class name, tag item, clipping, major classification, middle classification, small classification, and instance information. The environmental information may include shooting date and time, shooting location, weather information, road type, road surface information, and traffic jam information. In addition, the number and type of information that may be included in metadata is not limited thereto.

The metadata generator 225 according to an embodiment of the present invention may set specific values to be included in the metadata according to a control signal input from the user, but characteristically, the metadata generator 225 responds to the user's control signal. It is possible to automatically determine or assign the object's tracking ID and environment information without relying on it. In other words, the metadata generator 225 may generate metadata by including coordinates in the image of the specified object, an automatically assigned tracking ID, and automatically determined environment information.

Specifically, the metadata generation unit 225 may automatically assign a tracking ID to the object specified by the object specification unit 220 . Here, the tracking ID is a unique identifier of an object assigned to track an object from an image group. An image group is a set of annotated images. For example, the image group includes images taken consecutively in time series by the same camera as the camera that took the image to be annotated, and other images arranged adjacent to the camera that took the image to be annotated. Images spatially continuously photographed by a camera may be included, but are not limited thereto.

The metadata generating unit 225 according to an embodiment of the present invention may assign a tracking ID based on the color of the object. Specifically, the metadata generating unit 225 may identify a pattern of RGB values for an area occupied by an object in an image. The metadata generation unit 225 may generate an identifier corresponding to the identified pattern of RGB values and assign it as the tracking ID of the object specified by the object specification unit 220 . Here, the pattern of RGB values means a pattern of ratios of RGB values to pixels and arrangement order of RGB values.

In this case, the metadata generating unit 225, before generating a new identifier, determines the pattern of RGB values for objects specified from the images included in the image group and the object specified by the object specifying unit 220. Similar objects may be searched for by comparing the identified patterns of RGB values. Also, when a similar object is found from the images included in the image group, the metadata generator 225 assigns the tracking ID assigned to the searched similar object to the tracking ID of the object specified by the object specifying unit 220. can do.

The metadata generator 225 according to another embodiment of the present invention may assign a tracking ID based on the shape of an object. Specifically, the metadata generating unit 225 may extract an edge of an image (edge detection). The metadata generation unit 225 may identify only the edge of the object specified by the object specification unit 220 from all edges of the extracted image. The metadata generating unit 225 may generate an identifier corresponding to the edge pattern of the identified object and assign it as the tracking ID of the object specified by the object specifying unit 220 . Here, the pattern of the edge means a pattern for the number of pixels included in the area closed by the edge extracted from the image and the relative position of the edge within the image.

In this case, the metadata generation unit 225 identifies edge patterns for objects specified from the images included in the image group and the object identified by the object specification unit 220 before newly generating the identifier. Similar objects may be searched for by comparing the edge patterns of objects with each other. Also, when a similar object is found from the images included in the image group, the metadata generator 225 assigns the tracking ID assigned to the searched similar object to the tracking ID of the object specified by the object specifying unit 220. can do.

The metadata generator 225 according to another embodiment of the present invention may assign a tracking ID based on symbols or characters included in objects. Specifically, the metadata generating unit 225 may identify whether symbols or characters are included in the area occupied by the object in the image. The metadata generation unit 225 targets objects specified from the images included in the image group, and similar objects including the same symbol or character as the symbol or character identified from the object specified by the object specifying unit 220. can be searched for. Also, when a similar object is found from the images included in the image group, the metadata generator 225 assigns the tracking ID assigned to the searched similar object to the tracking ID of the object specified by the object specifying unit 220. can do.

Meanwhile, the metadata generating unit 225 may automatically determine environment information about an image that is a target of annotation work. Here, the environment information refers to information related to a place or time point at which an image was captured by a camera. Such environmental information may include, but is not limited to, information on the date and time of image capture, location, weather information, road type, road surface, and traffic congestion. It doesn't.

The metadata generation unit 225 according to an embodiment of the present invention compares an existing image with an image to be annotated, and converts environmental information given to the existing image to the environment of the image to be annotated. information can be applied.

More specifically, the metadata generation unit 225 compares the existing image with the image to be annotated, and determines whether environmental information given to the existing image can be applied as environmental information of the image to be annotated. can determine whether Here, the existing image means an image taken prior to the image to be annotated by the same camera as the camera that captured the image to be annotated.

If there are a plurality of images taken prior to the image to be annotated, the metadata generation unit 225 assigns the same tracking ID as the object specified by the object specification unit 220 among the plurality of images. You can select an image that contains an object that has been deleted as an existing image.

In contrast, when there are a plurality of images taken prior to the image to be annotated, the metadata generation unit 225 selects an image whose shooting time is closest to the image to be annotated from among the plurality of images. You can also choose by image.

Thereafter, the metadata generating unit 225 may identify a background area occupied by an object to which the same tracking ID as an object of the existing image is removed from the image to be annotated. The metadata generating unit 225 may identify a background area occupied by an object to which the same tracking ID as the object of the image to be annotated is removed from the selected existing image.

Further, the metadata generation unit 225 converts environment information pre-assigned to the existing image to a target of annotation based on a result of comparing the background area of the image to be annotated with the background area of the existing image. It can be determined whether or not it is applicable as environment information of the image to be applied.

That is, the metadata generating unit 225 is an area occupied by an object to which the same ID has been assigned to each of the previously photographed image (i.e., the previous image) and the subsequent photographed image (annotation target image). The background areas from which are removed are compared with each other, and based on the comparison result, it is determined whether or not the environmental information previously assigned to the previously photographed image can be applied as the environment information of the subsequent photographed image.

As an example of a specific determination, the metadata generation unit 225 determines the existing image based on the pattern of RGB values for the background area of the image to be annotated and the pattern of RGB values for the background area of the existing image. It is possible to determine whether pre-given environmental information can be applied as environmental information of an image to be annotated.

Unlike this, the metadata generation unit 225 is based on the pattern of the edge of the background area of the image to be annotated and the pattern of the edge of the background area of the existing image, the environment given to the existing image. It is possible to determine whether the information can be applied as environmental information of an image to be annotated.

Alternatively, the metadata generator 225 may set the average brightness value or average gamma value of the background area of the image to be annotated, and the average brightness value or average brightness value of the background area of the existing image. Based on the average gamma value, it may be determined whether environmental information given to an existing image can be applied as environmental information of an image to be annotated.

In addition, when it is determined that the metadata generation unit 225 can apply the environmental information previously assigned to the existing image as the environmental information of the image to be annotated, the metadata generation unit 225 annotates the environmental information previously assigned to the existing image. Metadata can be created by including it as the environment information of the target image.

In summary, when a plurality of images correspond to images continuously photographed by the same camera, the metadata generation unit 225 assigns previously given environment information to the preceding image among the plurality of images to be photographed. By including it as environment information of the image, it is possible to automatically generate metadata for images taken later.

On the other hand, if it is determined that the metadata generation unit 225 cannot apply the environment information already given to the existing image as the environmental information of the image to be annotated, one or more sets in the image to be annotated. Based on the characteristics of the reference point, environmental information about an image to be annotated can be automatically determined.

More specifically, the metadata generating unit 225 may set one or more reference points in an image to be annotated. A number of images used in machine learning have a relatively similar composition so that artificial intelligence (AI) can achieve a specific purpose. A reference point according to an embodiment of the present invention may be one or more points in an image set to correspond to the learning purpose of artificial intelligence (AI). In addition, the metadata generator 225 may determine environment information of an image to be annotated based on the color, brightness, or gamma of each pixel of one or more set reference points.

The metadata generator 225 according to an embodiment of the present invention may extract an edge of an image to be annotated, and divide the image into a plurality of regions based on the extracted edge. The metadata generating unit 225 may set one or more reference points for each of the plurality of divided areas. The metadata generation unit 225 may identify a background area by removing an area occupied by an object from an image to be annotated based on the extracted edge. The metadata generating unit 225 may determine the orientation of the object in the image based on the class assigned to the object. For example, when the class assigned to the object is a vehicle, the metadata generation unit 225 may determine the direction in which the wheels of the object are heading in the image as the ground. The metadata generator 225 may identify a ground area and an air area from among a plurality of areas included in the identified background area, based on the determined object directionality. Here, the ground area is an area closest to the ground among a plurality of areas divided by edges. The aerial area is an area furthest from the ground among a plurality of areas divided by edges. Further, the metadata generation unit 225 determines road surface information constituting the environment information based on the pixels of the reference point set in the ground area, and determines the environment information based on the pixels of the reference point set in the air area. It is possible to determine constituting weather information.

The metadata generator 225 according to another embodiment of the present invention may set one or more reference points for each of the four line segments constituting the border of the image to be annotated. The metadata generating unit 225 may determine the orientation of the object in the image based on the class assigned to the object. The metadata generation unit 225 may identify a ground segment and an air segment among four line segments based on the determined object direction. Here, the ground segment is a line segment closest to the ground among four line segments constituting the edge of the image. The mid-air segment is the farthest line segment from the ground among the four line segments constituting the edge of the image. Then, the metadata generator 225 determines road surface information constituting environmental information based on pixels of a reference point set in a ground segment, and weather information constituting environmental information based on pixels of a reference point set in an aerial segment can decide

Meanwhile, the metadata generation unit 225 sets a reference point at the same location for each of the image to be annotated and the existing image included in the image group, and the pixel of the reference point set in the image to be annotated is compared with the existing image. Contrasting the pixels of reference points identically set in the image, if the color, brightness, and gamma values of the pixels are within the preset allowable range, the preset environment information of the existing image will be determined as the environment information of the image to be annotated. may be If the difference between the color, brightness, and gamma values of the pixels of the reference point set identically in the image to be annotated and the existing image is out of the allowable range, the metadata generator 225 sets the environment information manually. A message informing that it should be done may be output through a user interface (UI).

With the following configuration, the result generation unit 230 may generate an annotation work result based on the metadata generated by the metadata generation unit 225 . In this case, the annotation work result may have a JSON file format, but is not limited thereto.

In addition, the result generation unit 230 may directly transmit the generated annotation work result to the learning data generation device 100 or to the learning data verification device 300 through the communication unit 205 .

Hereinafter, hardware for implementing the above-described logical components of the annotation device 200 will be described in more detail.

5 is one of the present invention in the examples followed annotation It is a hardware configuration diagram of the device.

As shown in FIG. 5, the annotation device 200 includes a processor 250, a memory 255, a transceiver 260, an input/output device 265, and a data bus. , 270) and storage (Storage, 275).

The processor 250 may implement operations and functions of the annotation device 200 based on instructions according to the software 280a in which the method according to the embodiments of the present invention is resident in the memory 255 . Software 280a in which a method according to embodiments of the present invention is implemented may be loaded in the memory 255 . The transceiver 260 may transmit and receive data to and from the learning data generating device 100 and the learning data verifying device 300 . The input/output device 265 may receive data necessary for the operation of the annotation device 200 and output an image, a bounding box, and metadata to be an annotation work. The data bus 270 is connected to the processor 250, the memory 255, the transceiver 260, the input/output device 265, and the storage 275, and is a movement path for transferring data between each component. role can be fulfilled.

The storage 275 stores an application programming interface (API), a library file, a resource file, etc. necessary for the execution of the software 280a in which the method according to the embodiments of the present invention is implemented. can be saved The storage 275 may store software 280b in which a method according to embodiments of the present invention is implemented. Also, the storage 275 may store information necessary for performing a method according to embodiments of the present invention.

According to one embodiment of the present invention, the software (280a, 280b) for implementing the method of assigning and verifying the tracking ID resides in the memory 255 or stored in the storage 275, the processor 250 is artificial intelligence (AI) Step of specifying an object included in an image that is a target of annotation work for learning, step of giving a tracking ID to the specified object by the processor 250, and step of the processor 250 specifying the image of the specified object It may be a computer program recorded on a recording medium to execute the step of generating metadata for the image by including the inside coordinates and the assigned tracking ID.

According to another embodiment of the present invention, the software (280a, 280b) for implementing the method of generating and checking metadata resident in the memory 255 or stored in the storage 275 is the processor 250 artificial intelligence (AI) Specifying an object included in an image that is a target of annotation work for learning, the processor 250 compares the image with an existing image, and converts environment information given to the existing image into environment information of the image. Determining whether or not it is applicable, and if the processor 250 determines that the environment information given to the existing image is applicable as the environment information of the image, the coordinates of the specified object in the image and the existing image It may be a computer program recorded on a recording medium to execute the step of generating metadata for the image by including the given environment information.

According to another embodiment of the present invention, the software (280a, 280b) for implementing a method for automatically generating metadata residing in the memory 255 or stored in the storage 275 is provided by the processor 250 using artificial intelligence (AI) Specifying an object included in an image that is a target of annotation work for learning, the processor 250 setting one or more reference points in the image, and based on the color, brightness, or gamma of a pixel of the set reference point In order to execute the step of determining environmental information about an image, and the step of generating metadata about the image by including the coordinates of the specified object in the image and the determined environment information by the processor 250, the recording medium It can be a recorded computer program.

More specifically, the processor 250 may include an Application-Specific Integrated Circuit (ASIC), another chipset, a logic circuit, and/or a data processing device. The memory 255 may include read-only memory (ROM), random access memory (RAM), flash memory, a memory card, a storage medium, and/or other storage devices. The transceiver 260 may include a baseband circuit for processing wired/wireless signals. The input/output device 265 includes an input device such as a keyboard, a mouse, and/or a joystick, and a Liquid Crystal Display (LCD), an Organic LED (OLED), and/or a liquid crystal display (LCD). Alternatively, an image output device such as an active matrix OLED (AMOLED) may include a printing device such as a printer or a plotter.

When the embodiments included in this specification are implemented as software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described functions. A module may reside in memory 255 and be executed by processor 250 . The memory 255 may be internal or external to the processor 250 and may be connected to the processor 250 by various well-known means.

Each component shown in FIG. 5 may be implemented by various means, eg, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), FPGAs ( Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded. Here, the recording medium may include program commands, data files, data structures, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disks), floptical It includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, such as a floptical disk, and ROM, RAM, flash memory, and the like. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. These hardware devices may be configured to operate as one or more pieces of software to perform the operations of the present invention, and vice versa.

Hereinafter, features of the artificial intelligence learning system according to various embodiments of the present invention as described above will be described in detail with reference to the drawings.

6 is one of the present invention in the examples This is a signal flow diagram to explain how to generate data for artificial intelligence (AI) learning.

As shown in FIG. 6 , the learning data generation apparatus 100 of the artificial intelligence learning system according to an embodiment of the present invention may design a data structure for artificial intelligence (AI) learning (S105).

Specifically, the learning data generating device 100 may define an ontology for AI learning and a classification system of data for AI learning based on a user's control. In this case, the user of the learning data generating device 100 can control the structure of data for artificial intelligence (AI) learning to have versatility in consideration of the needs of the main business partner or customer.

The learning data generating device 100 may collect and refine data for artificial intelligence (AI) learning based on the designed data structure (S110).

Specifically, the learning data generating device 100 may receive images from the outside, collect images by performing web crawling, or download images from an external institution's device that holds image data.

Also, the learning data generation device 100 may remove duplicate or extremely similar images from among the collected images and de-identify personal information included in the images.

The learning data generating device 100 may distribute and transmit the collected and refined data (ie, the image to be annotated) to a plurality of annotation devices 200 (S115).

Each of the plurality of annotation devices 200 of the artificial intelligence learning system according to an embodiment of the present invention may specify an object included in an image received from the learning data generating device 100 and to be annotated ( S120).

Specifically, the annotation device 200 may receive coordinates through a tool according to a user's control signal, and set a bounding box based on the input coordinates to specify an object included in the image. Here, the bounding box is an area for specifying an object to be learned by artificial intelligence (AI) among objects included in the image.

The annotation device 200 may generate metadata for an image to be an annotation work (S125).

Characteristically, the annotation device 200 according to an embodiment of the present invention may set specific values to be included in metadata according to a control signal input from a user, but the tracking ID and environment of the object are not dependent on the user's control signal. Information can be determined or assigned automatically.

The annotation device 200 may generate an annotation work result based on the generated metadata and transmit the generated annotation work result to the learning data verification device 300 (S130).

The learning data verification apparatus 300 of the artificial intelligence learning system according to an embodiment of the present invention may inspect annotation work results received from the annotation apparatuses 200 (S135).

Specifically, the learning data verification device 300 may perform inspection using a script for annotation work results. Here, the script is a code for verifying whether or not the target quality previously set for the annotation work result meets or whether the data is valid.

Characteristically, the learning data verification device 300 according to an embodiment of the present invention can provide a user interface (UI) that can more easily inspect tracking IDs and environment variables of objects included in annotation work results. .

The learning data verification device 300 may transmit annotation work results and inspection results to the learning data generating device 100 (S140).

The learning data generating apparatus 100 may build big data about a plurality of annotation work results based on the metadata of the annotation work results (S145).

After the big data is constructed as described above, the learning data generating device 100 may receive a data request including a request value from the artificial intelligence learning device 400 (S150). Here, the required value is a value related to requirements that AI learning data must satisfy in order for AI to achieve its development purpose. That is, the requested value is a value indicating the type of information to be included in the data for AI learning.

The learning data generating apparatus 100 may mine or filter the constructed big data to extract an annotation work result corresponding to a requested value. The learning data generating device 100 may generate artificial intelligence (AI) learning data by packaging the extracted annotation work result (S155).

And, the learning data generating device 100 may transmit the generated artificial intelligence (AI) learning data to the artificial intelligence learning device 400 .

Claims (10)

1. generating a plurality of annotation work results by a plurality of annotation devices; and

   A learning data generating device receiving the plurality of annotation job results, classifying the plurality of annotation job results based on the metadata, and generating artificial intelligence (AI) learning data , Data extraction method for artificial intelligence learning based on big data.
2. The method of claim 1, wherein the generating data for artificial intelligence (AI) learning

   Big data is built using the annotation work result classified based on the metadata, and when a request value is input from the outside, the big data is mined or filtered to obtain the requested value. Extracting the corresponding annotation work result, and packaging the extracted annotation work result to generate the artificial intelligence (AI) learning data,

   The request value is a value indicating the type of information to be included in the artificial intelligence (AI) learning data.
3. The method of claim 2, wherein the big data

   A hash table composed of a plurality of buckets according to the type of the metadata, and each bucket includes one or more slots including a link to an annotation work result. is composed of,

   The step of generating data for artificial intelligence (AI) learning

   Searching for one or more slots by substituting the requested value into a hash function set in advance corresponding to the hash table, and extracting one or more annotation work results through a link included in the searched one or more slots. A data extraction method for artificial intelligence learning based on big data.
4. The method of claim 3, wherein the generating data for artificial intelligence (AI) learning

   In the process of constructing the big data, if the loading density of the hash table exceeds a preset critical density, the number of buckets according to the type of metadata while maintaining the link included in the slot Big data-based data extraction method for artificial intelligence learning, characterized in that the capacity of the hash table is increased by readjusting the home bucket of the slot after doubling.
5. The method of claim 2, wherein the step of generating data for artificial intelligence (AI) learning

   When the type of information indicated by the request value is not included in the metadata, after redesigning the data structure to include the type of information not included in the metadata, Characterized in that redistributing the plurality of images to the annotation device, big data-based artificial intelligence learning data extraction method.
6. The method of claim 2, wherein the step of generating data for artificial intelligence (AI) learning

   The big data is divided into a plurality of layers, and in the process of constructing the big data, the layer to which the annotation work result belongs is determined based on the task evaluation grade given to the user of the annotation device, and the external Big data-based artificial intelligence learning data extraction method, characterized in that for extracting the annotation work result from each layer according to the ratio corresponding to the requested value input from.
7. The method of claim 1, wherein generating the annotation work result

   When the plurality of images correspond to images consecutively photographed by the same camera, environment information previously assigned to an image photographed earlier among the plurality of images is included as environment information of an image photographed later, and the subsequent image is captured. to generate metadata for the captured image,

   Characterized in that the environment information includes information on the date and time of image capture, location, weather information, road type, road surface, and traffic congestion. A data extraction method for artificial intelligence learning based on big data.
8. 8. The method of claim 7, wherein generating the annotation work result

   If there are a plurality of images photographed in advance, environment information of an image photographed subsequent to an image including an object to which the same tracking identifier is assigned among the plurality of images is converted to environmental information. It is characterized by including

   The tracking ID is a data extraction method for artificial intelligence learning based on big data, characterized in that the unique identifier of the object given to track the object.
9. 9. The method of claim 8, wherein generating the annotation work result

   Based on a result of comparing the background areas obtained by removing the area occupied by the object to which the same ID is assigned to each of the preceding images and the succeeding images, an environment assigned to the preceding images is determined. A data extraction method for learning artificial intelligence based on big data, characterized in that it determines whether the information can be applied as environmental information of the image taken after the information.
10. memory;

    transceiver; and

    In combination with a computing device configured to include a processor for processing instructions resident in the memory,

    receiving, by the processor, a plurality of annotation work results through the transceiver, the annotation work results including coordinates of an object in the image for each of the plurality of images and metadata for the image; and

    A computer program recorded on a recording medium so that the processor executes the step of classifying the plurality of annotation work results based on the metadata and generating data for artificial intelligence (AI) learning.

Images