WO2023054832A1 - Annotation method for easily designating object, and computer program recorded in recording medium in order to perform method - Google Patents

Abstract

The present invention proposes an annotation method for easily designating an object included in an image. The method may comprise the steps of: setting, in accordance with controlling by a worker, one area in an image which is subjected to an annotation work for artificial intelligence (AI) learning, as a bounding box; identifying an object which is completely included in the inner area of the bounding box or overlaps one or more lines from among a plurality of lines forming the bounding box; expanding or reducing the size of the bounding box such that the size corresponds to the outer line of the identified object; and, if the location and the size of the bounding box which has been expanded or reduced by the worker's controlling is determined, generating a work result of the annotation by including coordinates in accordance with the determined location and size of the bounding box.

Description

An annotation method that can easily designate an object and a computer program recorded on a recording medium to execute the annotation method

The present invention relates to data design for artificial intelligence (AI) learning. More specifically, in annotating artificial intelligence (AI) learning data, it relates to an annotation method that can easily designate an object included in an image and a computer program recorded on a recording medium to execute the annotation method.

Artificial intelligence (AI) refers to a technology that artificially implements some or all of human learning abilities, reasoning abilities, and perception abilities using computer programs. In relation to artificial intelligence (AI), machine learning refers to learning to optimize parameters with given data using a model composed of multiple parameters. Such machine learning is classified into supervised learning, unsupervised learning, and reinforcement learning according to the form of learning data.

In general, the design of artificial intelligence (AI) learning data proceeds in the steps of data structure design, data collection, data refinement, data processing, data expansion, and data verification.

To describe each step in more detail, data structure design is performed through ontology definition, classification system definition, and the like. Data collection is performed by collecting data through direct filming, web crawling, or associations/professional organizations. Data purification is performed by removing redundant data from collected data and de-identifying personal information. Data processing is performed by inputting meta data and performing annotation. Data extension is performed by performing ontology mapping and supplementing or extending the ontology as needed. In addition, data verification is performed by verifying validity according to the set target quality using various verification tools.

In general, annotation in the data processing step is performed by processing a bounding box for an object included in an image and inputting property information of the object included in the image. Such annotations are also referred to as data labeling. And, a dataset corresponding to the work result of the annotation is calculated in the form of a JSON (Java Script Object Notation) file.

Specifically, in the process of processing the bounding box in the annotation, a worker manually processes the bounding box according to the outline of the object for each object to be recognized in the image.

However, the bounding box processing must follow various rules determined in advance to ensure an artificial intelligence (AI) learning effect (ie, recognition rate of an object in an image). For example, the bounding box processing must follow rules such as the minimum size value of an area in which an object can be recognized and the maximum number of pixels away from the object's outline. Therefore, it is not easy for a worker to perfectly comply with all rules determined in advance in the process of manually processing bounding boxes for a plurality of objects.

In addition, the process of inputting attribute information in annotations is also performed by a worker manually inputting information for each bounding box processed object.

However, the attribute information to be entered by the operator includes various information such as the type of annotation, class name (class), classification item (tags), whether the object is truncated (truncated), large category, small category, and instance upper level. should be included Therefore, it is also not easy for an operator to manually and accurately input all attribute information for a plurality of objects one by one.

One object of the present invention is to provide an annotation method that can easily designate an object included in an image when annotating data for artificial intelligence (AI) learning.

Another object of the present invention is to provide a computer program recorded on a recording medium to execute an annotation method capable of easily designating an object included in an image.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to achieve the above technical problem, the present invention proposes an annotation method that can easily designate an object included in an image. The method includes the steps of setting a partial region in an image, which is a target of annotation work for artificial intelligence (AI) learning, as a bounding box under the control of an operator; identifying an object completely included in an inner region of the bounding box or spanning one or more line segments among a plurality of lines constituting the bounding box; enlarging or reducing the size of the bounding box to have a size corresponding to the outline of the identified object; and when the position and size of the expanded or reduced bounding box are determined under the operator's control, generating an annotation work result including coordinates according to the determined position and size of the bounding box. .

In the setting of the bounding box, the bounding box may be set based on a rectangle having two coordinates input from the operator as coordinates of an upper left vertex and a coordinate of a lower right vertex in the image.

In the identifying the object, it may be determined whether the object spans the line segment based on continuity between pixels included in the inner area of the bounding box and pixels in the outer area of the bounding box. Here, the continuity may be determined based on whether a difference in color, saturation, or brightness between two adjacent pixels is within a preset range.

In the identifying of the object, when there are a plurality of objects identified in the inner region of the bounding box, only one object may be selected from among the plurality of objects according to a preset priority. In this case, the priority may be a criterion set according to the size, position, or shape of an object occupied in the image.

According to one embodiment, in the step of enlarging or reducing the size, when the identified object is completely included in the inner region of the bounding box, it is present on the outline of the object for each of a plurality of line segments constituting the bounding box. The line segment may be moved toward the center of the bounding box until the distance from the closest point to the point is within a preset maximum number of pixels apart.

According to another embodiment, the step of enlarging or reducing the size may include, when the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, the identified object is located in an inner area of the bounding box. A line segment spanning the object may be moved toward an outside direction of the bounding box until it is completely included.

According to another embodiment, the step of enlarging or reducing the size may include, when the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, the identified object is located in an inner area of the bounding box. The size of the bounding box may be expanded in all directions while moving the center of the bounding box in the direction of the line segment on which the object spans until it is completely included.

According to another embodiment, the step of enlarging or reducing the size may include dividing the bounding box into a plurality of detailed areas when the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, and then , From among the divided detailed areas, only the size of the detailed area spanning the object may be enlarged until the identified object is completely included in the inner area of the bounding box.

According to another embodiment, the step of enlarging or reducing the size may include, when the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, the identified object is merged inside the bounding box. A process of newly setting an additional area having a predetermined size outside the line segment where the object spans and merging the newly set additional area and the bounding box may be repeatedly performed until the object is completely included in the area.

In order to achieve the technical problem as described above, the present invention proposes a computer program recorded on a recording medium to execute the annotation method. The computer program may include a memory; input output device; and a processor configured to process instructions resident in the memory. In addition, the computer program may include setting, by the processor, a partial region in an image to be an annotation work target for artificial intelligence (AI) learning as a bounding box according to operator control input through the input/output device; identifying, by the processor, an object completely included in an inner region of the bounding box or spanning one or more line segments among a plurality of line segments constituting the bounding box; enlarging or reducing, by the processor, the size of the bounding box to have a size corresponding to the outline of the identified object; and when the location and size of the expanded or reduced bounding box are confirmed by the processor under the operator's control input through the input/output device, annotation work including coordinates according to the location and size of the determined bounding box. In order to execute the step of generating the result, it can be recorded on a recording medium.

Details of other embodiments are included in the detailed description and drawings.

According to the embodiments of the present invention, in performing annotation on each of a plurality of images for artificial intelligence (AI) learning, the size of the bounding box is automatically enlarged or reduced according to the shape of an object included in the image, so that the annotation A worker who performs can more easily set the bounding box, and the annotation work result can have a more uniform quality.

As a result, according to embodiments of the present invention, it is possible to generate artificial intelligence (AI) learning data capable of constantly guaranteeing a recognition rate of an object included in an image.

The effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 to 3 are configuration diagrams of an artificial intelligence learning system according to various embodiments of the present invention.

4 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.

5 is a hardware configuration diagram of an annotation device according to an embodiment of the present invention.

6 to 9 are exemplary diagrams for explaining a process of setting a bounding box according to an embodiment of the present invention.

10 and 11 are exemplary diagrams for explaining a process of inputting object attributes according to an embodiment of the present invention.

12 is a flowchart for explaining an annotation method according to an embodiment of the present invention.

13 is a flowchart for explaining a method of setting a bounding box according to an embodiment of the present invention.

14 is a flowchart illustrating a method of inputting attribute information of an object according to an embodiment of the present invention.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in this specification should be interpreted in terms commonly understood by those of ordinary skill in the art to which the present invention belongs, unless specifically defined otherwise in this specification, and are overly inclusive. It should not be interpreted in a positive sense or in an excessively reduced sense. In addition, when the technical terms used in this specification are incorrect technical terms that do not accurately express the spirit of the present invention, they should be replaced with technical terms that those skilled in the art can correctly understand. In addition, general terms used in the present invention should be interpreted as defined in advance or according to context, and should not be interpreted in an excessively reduced sense.

Also, singular expressions used in this specification include plural expressions unless the context clearly indicates otherwise. In this application, terms such as "consisting of" or "having" should not be construed as necessarily including all of the various components or steps described in the specification, and some of the components or steps are included. It should be construed that it may not be, or may further include additional components or steps.

Also, terms including ordinal numbers such as first and second used in this specification may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention.

When a component is referred to as being “connected” or “connected” to another component, it may be directly connected or connected to the other component, but other components may exist in the middle. On the other hand, when a component is referred to as “directly connected” or “directly connected” to another component, it should be understood that no other component exists in the middle.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components are given the same reference numerals regardless of reference numerals, and redundant description thereof will be omitted. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may obscure the gist of the present invention, the detailed description will be omitted. In addition, it should be noted that the accompanying drawings are only for easily understanding the spirit of the present invention, and should not be construed as limiting the spirit of the present invention by the accompanying drawings. The spirit of the present invention should be construed as extending to all changes, equivalents or substitutes other than the accompanying drawings.

On the other hand, in relation to annotations for processing artificial intelligence (AI) learning data, it is not easy for workers to perfectly comply with all rules determined in advance in the process of manually processing bounding boxes for multiple objects. In addition, it is also not easy for a worker to manually and accurately input all attribute information for a plurality of objects.

In order to overcome these difficulties, the present invention intends to propose means capable of providing various conveniences to operators in the process of setting a bounding box and inputting object attribute information.

*

Figure 1 degree 3 is a variety of the present invention in the examples It is a configuration diagram of an artificial intelligence learning system according to

As shown in Figure 1, the artificial intelligence learning system according to an embodiment of the present invention includes one or more annotation devices (100-1, 100-2, 100-3, ..., 100-n; 100) and artificial intelligence learning. It may be configured to include device 300 .

As shown in FIG. 2, the artificial intelligence learning system according to another embodiment of the present invention may be configured to additionally include a learning data design device 200 in addition to the annotation device 100 and the artificial intelligence learning device 300. .

In addition, as shown in FIG. 3, the artificial intelligence learning system according to another embodiment of the present invention has a network ( They can also be connected to each other through a public network). In this case, a part of the annotation device 100 may be a device that performs annotation by a clouding service.

Since the components of the artificial intelligence learning system according to various embodiments are merely functionally distinct elements, two or more components are integrated and implemented in the actual physical environment, or one component is implemented in the actual physical environment. may be implemented separately from each other.

Each component is described, the annotation device 100 is a device that can be used to annotate an image provided from the learning data design device 200 or the artificial intelligence learning device 300.

In particular, the annotation device 100 according to the present invention has a feature of providing various conveniences to the operator in the process of setting a bounding box and inputting object attribute information in relation to annotation work.

As such, the annotation device 100 may be any device capable of transmitting/receiving data with the learning data design device 200 or the artificial intelligence learning device 300 and performing calculations using the transmitted/received data. there is.

For example, the annotation device 100 may be any one of a fixed computing device such as a desktop, a workstation, or a server, but is not limited thereto, and may include a smart phone, such as laptops, tablets, phablets, Portable Multimedia Players (PMPs), Personal Digital Assistants (PDAs) or E-book readers. It may be any one of the mobile computing devices.

As described above, the specific configuration and operation of the annotation device 100 will be described with reference to FIGS. 4 to 14 later.

With the following configuration, the learning data design device 200 is a device that can be used to design and generate artificial intelligence (AI) learning data. As such, the learning data design device 200 is basically a device that is distinguished from the artificial intelligence learning device 300, but may be implemented by being integrated with the artificial intelligence learning device 300 in a real physical environment.

Specifically, the learning data design device 200 may receive attributes of a project related to artificial intelligence (AI) learning from the artificial intelligence learning device 300 . The learning data design device 200 designs data structures for artificial intelligence (AI) learning, refines collected data, processes data, expands data, and verifies data based on the user's control and the attributes of the project. can be done

In particular, the learning data design device 200 may transmit an image to be annotated to the annotation device 100 for data processing for artificial intelligence (AI) learning. The learning data design device 200 may receive an annotation work result from the annotation device 100 . In this case, the annotation work result may have a JSON (Java Script Object Notation) file format. Alternatively, the learning data design device 200 may generate a JSON file based on the received result after receiving a result of annotation work in a format different from JSON. In addition, the learning data design device 200 may inspect the received or created JSON file, package it, and transmit it to the artificial intelligence learning device 300.

As such, the learning data design device 200 may be any device capable of transmitting/receiving data with the annotation device 100 and the artificial intelligence learning device 300 and performing calculations using the transmitted/received data. there is. For example, the learning data design device 200 may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto.

With the following configuration, the artificial intelligence learning device 300 is a device that can be used to perform machine learning of artificial intelligence (AI) based on data for artificial intelligence (AI) learning.

Specifically, the artificial intelligence learning device 300 may receive a JSON file packaged directly from the annotation device 100 or from the learning data design device 200 . And, the artificial intelligence learning device 300 may perform artificial intelligence (AI) machine learning using the received JSON file.

As such, the artificial intelligence learning device 300 may be any device capable of transmitting and receiving data to and from the annotation device 100 or the learning data design device 200 and performing calculations using the transmitted and received data. there is. For example, the artificial intelligence learning device 300 may be any one of a fixed computing device such as a desktop, workstation, or server, but is not limited thereto.

As described above, one or more of the annotation device 100, the learning data design device 200, and the artificial intelligence learning device 300 are a combination of one or more of a security line, a common wired communication network, or a mobile communication network directly connected between the devices. Data can be transmitted and received using a network.

For example, public wired communication networks may include Ethernet, x Digital Subscriber Line (xDSL), Hybrid Fiber Coax (HFC), and Fiber To The Home (FTTH). It may be, but is not limited thereto. In addition, in the mobile communication network, Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), High Speed Packet Access (HSPA), Long Term Evolution, LTE) and 5th generation mobile telecommunication may be included, but is not limited thereto.

Hereinafter, the configuration of the annotation device 100 as described above will be described in more detail.

4 is a logical configuration diagram of an annotation device according to an embodiment of the present invention.

As shown in FIG. 4, the annotation device 100 includes a communication unit 105, an input/output unit 110, a storage unit 115, a bounding box setting unit 120, an object property setting unit 125, and a result generation unit. It may be configured to include (130).

Since the components of the annotation device 100 are merely functionally distinct elements, two or more components are integrated and implemented in an actual physical environment, or one component is separated from each other in an actual physical environment. could be implemented.

The communication unit 105 can transmit/receive data with the learning data design device 200 and the artificial intelligence learning device 300 when each component is described.

Specifically, the communication unit 105 may receive one or more images from the learning data design device 200 or the artificial intelligence learning device 300 .

Here, the image is an image that is a target of annotation work for artificial intelligence (AI) learning. According to the data processing plan designed by the learning data design device 200 or the artificial intelligence learning device 300, the images to be annotated are individually received, or a plurality of images are collectively received. can receive

The communication unit 105 may transmit the annotation work result to the learning data design device 200 or the artificial intelligence learning device 300 .

Here, the work result may include the coordinates of the bounding box set under the operator's control and property information of the object. In addition, the work result may have a JSON file format, but is not limited thereto.

Also, the communication unit 150 may receive project properties, image properties, or worker properties from the learning data design device 200 or the artificial intelligence learning device 300 .

Here, the properties of the project may include the learning purpose, learning period, number of images required for learning, object properties to be identified in images, bounding box setting rules, etc. for a project related to artificial intelligence (AI) learning. It is not limited to this.

Image properties include image file name, image size (width, height), resolution, bit level, compression format, shooting device name, exposure time, ISO speed, focal length, aperture value, and shooting location coordinates (GPS latitude, longitude). , shooting time, etc. may be included, but is not limited thereto.

The worker's attributes may include, but are not limited to, the worker's name, identification number, assigned work amount, cost according to the work, work result evaluation, and the like.

In addition, the bounding box setting rule is a rule that a worker must follow in a process of setting a bounding box for an object in an image according to a project property, an image property, or a worker property. These bounding box setting rules may include, but are not limited to, a minimum size value of an area and a maximum number of pixels separated from an object's outline.

With the following configuration, the input/output unit 110 may input a signal from an operator through a user interface (UI) or output a calculated result to the outside.

Here, the worker means a person who performs annotation work. Such an operator may be referred to as a user, performer, labeler, or data labeler, but is not limited thereto.

Specifically, the input/output unit 110 may output an image to be an annotation work. The input/output unit 110 may receive a control signal for setting a bounding box from an operator. Also, the input/output unit 110 may overlay and output a bounding box on the image.

Here, the bounding box is an area for specifying an object to be learned by artificial intelligence (AI) among objects included in the image. Such a bounding box may have a rectangle or polygon shape, but is not limited thereto.

The input/output unit 110 may output a list of recommendation information that can be used as object attribute information. Also, the input/output unit 110 may receive a control signal for setting object attribute information from an operator.

Here, the proposed information list is a list of information that is composed of information that is likely to be included in object attribute information and can be utilized as object attribute information in the process of performing annotation by a worker.

In addition, object attribute information is information for specifying the attribute of an object that is an artificial intelligence (AI) learning target. Such object attribute information includes information on the type of annotation, class name (class), classification items (tags), truncated state of the object, major classification, subclassification, or upper level (instance upper). may be, but is not limited thereto.

With the following configuration, the storage unit 115 may store data required for annotation work.

Specifically, the storage unit 115 may store an image received through the communication unit 105 . The storage unit 115 may store project properties, image properties, or worker properties received through the communication unit 105 .

The storage unit 115 may temporarily store the location and size of the bounding box set according to the control signal input through the input/output unit 110 . The storage unit 115 may temporarily store attributes of objects input through the input/output unit 110 .

With the following configuration, the bounding box setting unit 120 may set a bounding box for specifying an object to be studied by artificial intelligence (AI) among objects included in an image. In particular, the bounding box setting unit 120 according to the present invention automatically adjusts the size of the bounding box according to the shape of the object included in the image so that the bounding box primarily set by the operator has a size corresponding to the outline of the object. It has a feature that can be enlarged or reduced.

First of all, the bounding box setting unit 120 may set a partial region in an image to be an annotation work target for learning as a bounding box according to control of an operator input through the input/output unit 110 .

Specifically, the bounding box setting unit 120 receives two coordinates from the operator through the input/output unit 110, and has the input two coordinates as the coordinates of the upper left vertex and the coordinates of the lower right vertex in the image. You can set a bounding box based on a rectangle. In this case, the two coordinates may be set by the operator inputting one type of input signal twice (eg, mouse click) or by the operator inputting two types of input signals once (eg, mouse drag). It may, but is not limited thereto.

Alternatively, the bounding box setting unit 120 may receive three or more line segments from an operator through the input/output unit 110 and set a bounding box based on a polygon composed of the input three or more line segments. In this case, three or more line segments may be input by an operator at one time using a polygon tool or by inputting three or more segments by an operator using a straight line tool, but are not limited thereto.

Next, the bounding box setting unit 120 may identify an object that is completely included in the inner region of the bounding box set by the operator's control or spans one or more line segments among a plurality of line segments constituting the bounding box.

Specifically, the bounding box setting unit 120 may identify all or some of the objects included in the inner region of the bounding box set under the operator's control. To this end, the bounding box setting unit 120 may perform image processing on an area inside the bounding box.

For example, the bounding box setting unit 120 may divide the inner region of the bounding box into three images according to RGB (Red, Green, Blue). The bounding box setting unit 120 may extract an edge of each image divided into three images. In more detail, the bounding box setting unit 120 may use either a Laplacian of Gaussian (LoG) algorithm or a Difference of Gaussian (DoG) algorithm to extract an edge of each image.

When using the LoG algorithm, the bounding box setting unit 120 may remove noise existing in the image using a Gaussian filter. The bounding box setting unit 120 may apply a Laplacian filter to the noise-removed image. Also, the bounding box setting unit 120 may extract an edge by detecting zerocrossing in the image to which the Laplacian filter is applied.

When using the DoG algorithm, the bounding box setting unit 120 generates two Gaussian masks having different variances from the image. The bounding box setting unit 120 subtracts another mask from one created mask. Also, the bounding box setting unit 120 may extract an edge by applying the subtracted mask to the image.

The bounding box setting unit 120 may identify one or more enclosures by edges extracted from each image. In this case, the bounding box setting unit 120 may first process binarization on each image in order to clarify whether the edge area is closed. Also, the bounding box setting unit 120 may identify the identified closed area as an object.

If a plurality of objects are identified in the inner region of the bounding box, the bounding box setting unit 120 may select only one object from among the identified plurality of objects according to a preset priority. In this case, the priority may be a criterion set according to the size, position or shape of the object occupying the image.

Next, the bounding box setting unit 120 determines whether the identified object is one or more of a plurality of line segments constituting the bounding box based on continuity between pixels included in the inner region of the bounding box and pixels in the outer region of the bounding box. It can be determined whether or not it crosses a line segment. Here, continuity may be determined based on whether a difference in color, saturation, or brightness between two adjacent pixels is within a preset range.

The bounding box setting unit 120 may determine that the object is completely included in the inner region of the bounding box when the identified object does not span any line segment among a plurality of line segments constituting the bounding box.

Characteristically, according to an embodiment of the present invention, when it is determined that the object is completely included in the inner region of the bounding box, the bounding box setting unit 120 bounds the object so that it closely fits the inner region of the bounding box. You can reduce the size of the box.

In one embodiment, the bounding box setting unit 120, for each of a plurality of line segments constituting the bounding box, until the distance from the nearest point existing on the outline of the object is within a preset maximum number of spaced apart pixels. , the size of the bounding box can be reduced by moving the line segment toward the center of the bounding box.

Characteristically, according to one embodiment of the present invention, the bounding box setting unit 120, when it is determined that the object spans one or more line segments among a plurality of line segments constituting the bounding box, sets the object to be completely included within the bounding box. The size of the bounding box can be enlarged. After expanding the size of the bounding box, the bounding box setting unit 120 may reduce the size of the bounding box so that the object completely fits the inner region of the bounding box.

In one embodiment, the bounding box setting unit 120 may, when the object spans one or more line segments among a plurality of line segments constituting the bounding box, until the object is completely included in the inner area of the bounding box, in the outer direction of the bounding box. The size of the bounding box can be enlarged by moving the line segment on which the object spans toward.

In another embodiment, the bounding box setting unit 120 determines the center of the bounding box until the object is completely included in the inner region of the bounding box when the object spans one or more line segments among a plurality of line segments constituting the bounding box. The size of the bounding box can be expanded in all directions by moving the object in the direction of the line segment that the object spans.

In another embodiment, the bounding box setting unit 120 divides the bounding box into a plurality of subregions when the object spans one or more line segments among a plurality of line segments constituting the bounding box, and then the object is placed inside the bounding box. It is possible to enlarge only the size of the detailed region where the object spans among the divided detailed regions until it is completely included in the region. In this case, the size of the detailed region may be determined in advance according to the resolution of the image or the resolution of the object to be identified. Also, the bounding box enlarged according to the shape of the detailed region may have a polygonal shape.

In another embodiment, the bounding box setting unit 120 may, when the object spans one or more line segments among a plurality of line segments constituting the bounding box, until the object is completely included in the inner region of the merged bounding box, the object A process of newly setting an additional area outside the overlapped line segment and merging the newly set additional area and the bounding box may be repeatedly performed.

With the following configuration, the object property setting unit 125 may set the properties of the object specified by the bounding box. In particular, the object property setting unit 125 according to the present invention may preemptively provide a list of recommended information that can be used as object property information before a worker directly inputs property information.

First of all, the object property setting unit 125 may create a list of recommendation information that can be used as property information of an object corresponding to the bounding box set by the bounding box setting unit 120 .

In one embodiment, the object property setting unit 125 provides information that may be used as property information of an object according to the property of a project related to artificial intelligence (AI) learning, the property of an image, or the property of a worker set in advance. You can include it to create a list of recommended information.

As another embodiment, the object property setting unit 125 may calculate the average color, saturation, or brightness of pixels included in the bounding box set by the bounding box setting 120 . The object property setting unit 125 may select one recommended object type according to the calculated average color, saturation, or brightness based on the probability distribution of color, saturation, or brightness configured for each type of object. In addition, the object property setting unit 125 may create a list of recommendation information by including information related to the type of the selected recommendation object.

As another embodiment, the object property setting unit 125 identifies the input frequency of the property information previously input by the operator from the operator's work log stored in the local storage of the annotation device 100. can do. Further, the object property setting unit 125 may create a list of recommended information by including property information with a high input frequency based on the identified input frequency.

Next, the object property setting unit 125 may filter a pre-created list of recommended information.

Specifically, the object property setting unit 125 includes information related to the type of object that cannot be set by the tool used by the operator in the process of setting the bounding box, among information included in the list of pre-created recommendation information. (ie, information irrelevant to the object) can be removed.

As an embodiment, when an operator uses a tool capable of specifying a rectangular area in the process of setting a bounding box, the object property setting unit 125 determines the type of object corresponding to a living organism from a list of pre-created recommendation information and Relevant information can be removed.

In another embodiment, when a worker uses a tool capable of designating a skeleton structure of an object in the course of setting a bounding box, the object property setting unit 125 may select an object that does not have a skeleton from a list of pre-created recommended information. Information related to the type of may be removed.

The object property setting unit 125 may remove information unrelated to the object from the list of recommendation information and sort remaining information.

As an embodiment, the object property setting unit 125 may arrange the information included in the list of recommended information according to a pre-set sorting criterion based on the position or size occupied by the bounding box in the image.

In another embodiment, the object property setting unit 125 compares the shape of the object included in the bounding box with the basic shape provided in advance for each type of object, and based on the similarity of the two shapes included in the list of recommended information. Information can also be sorted.

Next, the object property setting unit 125 may output the created and filtered recommendation information list to the input/output unit 110 . Also, the object property setting unit 125 may receive a control signal for setting object property information from an operator through the input/output unit 110 .

When one piece of information is selected from a list of recommended information under the control of an operator, the object property setting unit 125 may provide feedback according to the type of object corresponding to the selected information.

In one embodiment, the object property setting unit 125 reflects the color or transparency set differently according to the type of object corresponding to the selected information, and the user interface related to the area inside the bounding box or the area inside the object. , UI) can be changed.

With the following configuration, the result generation unit 130 may generate an annotation work result and transmit it to the learning data design device 200 or the artificial intelligence learning device 300.

Specifically, when the position and size of the expanded or reduced bounding box is determined under the operator's control, and the object property information is determined under the operator's control based on the recommended information list, the result generation unit 130 determines the finalized bounding box. Annotation work results including coordinates according to the location and size of the bounding box and determined attribute information can be created. Such a work result may have a JSON file format, but is not limited thereto.

In addition, the output generation unit 130 may transmit the generated annotation work result to the learning data design device 200 or the artificial intelligence learning device 300 through the communication unit 105 .

Hereinafter, hardware for implementing the above-described logical components of the annotation device 100 will be described in more detail.

5 is one of the present invention in the examples followed annotation It is a hardware configuration diagram of the device.

As shown in FIG. 5, the annotation device 100 includes a processor 150, a memory 155, a transceiver 160, an input/output device 165, and a data bus , 170) and storage (Storage, 175).

The processor 150 may implement operations and functions of the annotation device 100 based on instructions according to the software 180a in which the annotation method resident in the memory 155 is implemented. Software 180a in which the annotation method is implemented may be loaded in the memory 155 . The transceiver 160 may transmit and receive data with the learning data design device 200 and the artificial intelligence learning device 300 . The input/output device 165 may receive data necessary for the operation of the annotation device 100 and output a list of images, bounding boxes, and recommendation information. The data bus 170 is connected to the processor 150, the memory 155, the transceiver 160, the input/output device 165, and the storage 175, and is a movement path for transferring data between each component. role can be fulfilled.

The storage 175 may store an application programming interface (API), a library file, a resource file, and the like required for execution of the software 180a in which the annotation method is implemented. The storage 175 may store software 180b in which the annotation method is implemented. Also, the storage 175 may store information necessary for performing the annotation method. In particular, the storage 175 may include a database 185 for storing an image subject to an annotation work, a project attribute, an image attribute, a worker attribute, and a worker's work log.

According to the first embodiment of the present invention, the software (180a, 180b) for implementing the annotation method resident in the memory 155 or stored in the storage 175 is input by the processor 150 through the input/output device 165. Setting a partial region in an image to be an annotation task for artificial intelligence (AI) learning as a bounding box under the control of an operator, wherein the processor 150 is completely included in the inner region of the bounding box or Identifying an object spanning one or more line segments among a plurality of line segments, enlarging or reducing the size of the bounding box so that the processor 150 has a size corresponding to the outline of the identified object, and the processor 150 ) is input through the input/output device 165, when the location and size of the expanded or reduced bounding box are confirmed, an annotation work result is generated including the coordinates according to the location and size of the determined bounding box. It may be a computer program recorded on a recording medium to execute the steps.

According to the second embodiment of the present invention, the software (180a, 180b) for implementing the annotation method resident in the memory 155 or stored in the storage 175 is input by the processor 150 through the input/output device 165. A step of setting a region in an image, which is an object of annotation work for artificial intelligence (AI) learning, as a bounding box under the control of an operator, which is used as attribute information of an object corresponding to the set bounding box by the processor 150. Creating a list of recommended information that can be recommended, and if the property information of the object is determined by the operator's control based on the list of recommended information created by the processor 150, coordinates according to the location and size of the bounding box and the determined It may be a computer program recorded on a recording medium to execute a step of generating a work result of annotation including attribute information.

More specifically, the processor 150 may include an Application-Specific Integrated Circuit (ASIC), another chipset, a logic circuit, and/or a data processing device. The memory 155 may include read-only memory (ROM), random access memory (RAM), flash memory, memory cards, storage media, and/or other storage devices. The transceiver 160 may include a baseband circuit for processing wired/wireless signals. The input/output device 165 includes an input device such as a keyboard, a mouse, and/or a joystick, and a Liquid Crystal Display (LCD), an Organic LED (OLED), and/or a liquid crystal display (LCD). Alternatively, an image output device such as an active matrix OLED (AMOLED) may include a printing device such as a printer or a plotter.

When the embodiments included in this specification are implemented as software, the above-described method may be implemented as a module (process, function, etc.) that performs the above-described functions. A module may reside in memory 155 and be executed by processor 150 . The memory 155 may be internal or external to the processor 150 and may be connected to the processor 150 by various well-known means.

Each component shown in FIG. 5 may be implemented by various means, eg, hardware, firmware, software, or a combination thereof. In the case of hardware implementation, one embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), FPGAs ( Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, etc.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is stored on a recording medium readable through various computer means. can be recorded. Here, the recording medium may include program commands, data files, data structures, etc. alone or in combination. Program instructions recorded on the recording medium may be those specially designed and configured for the present invention, or those known and usable to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs (Compact Disk Read Only Memory) and DVDs (Digital Video Disks), floptical It includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, such as a floptical disk, and ROM, RAM, flash memory, and the like. Examples of program instructions may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine language codes generated by a compiler. These hardware devices may be configured to operate as one or more pieces of software to perform the operations of the present invention, and vice versa.

Hereinafter, features of the artificial intelligence learning system according to various embodiments of the present invention as described above will be described in detail with reference to the drawings.

Fig. 6 degree 9 is one of the present invention in the examples according to bounding It is an exemplary diagram for explaining the process of setting a box.

Referring to FIG. 6 , the annotation device 100 of the artificial intelligence learning system according to an embodiment of the present invention loads an image 10, which is a target of annotation work for artificial intelligence (AI) learning, from an operator. A control signal for specifying one object 20 included in (10) can be input.

6 illustrates a situation in which two coordinates 30-1 and 30-2 are designated according to a control signal of an operator, but embodiments of the present invention are not limited thereto.

Referring to FIG. 7 , the annotation device 100 may set a partial region in the image 10 as a bounding box 40 according to a control signal input from a person in charge.

7 sets the bounding box 40 to have a rectangular shape having two coordinates input from the operator as the upper left vertex 30-1 and the lower right vertex 30-2, but can be set according to the present invention The bounding box 40 may be various, such as a polygon, a skeleton, and the like, in addition to a rectangle.

Referring to FIG. 8 , the annotation device 100 is an object that is completely included in the inner region of the bounding box 40 or spans one or more line segments 40-1 among a plurality of line segments constituting the bounding box 40 ( 20) can be identified.

In addition, when the annotation device 100 determines that the identified object 20 spans one or more line segments 40-1 among a plurality of line segments constituting the bounding box 40, the object within the bounding box 40 The size of the bounding box can be enlarged to include (20).

In one embodiment, the annotation apparatus 100 may include a line segment 40-1 along which the object 20 extends toward the outside of the bounding box 40 until the object 20 is completely included in the inner region of the bounding box 40. ) may be moved to enlarge the size of the bounding box 40 .

In another embodiment, the annotation apparatus 100 sets the center of the bounding box 40 to the line segment 40-1 along which the object 20 spans until the object 20 is completely included in the inner region of the bounding box 40. It is possible to increase the size of the bounding box 40 in all directions while moving in the direction of .

In another embodiment, the annotation device 100 divides the bounding box 40 into a plurality of detailed regions, and then until the object 20 is completely included in the inner region of the bounding box 40, the object 20 is selected from among the divided detailed regions. Only the size of the detailed area spanned by (20) can be enlarged.

In another embodiment, the annotation device 100 newly adds an additional area to the outer area of the line segment 40-1 spanned by the object 20 until the object 20 is completely included in the inner area of the bounding box 40. The process of setting and merging the newly set additional area and the bounding box 40 may be repeatedly performed.

Referring to FIG. 9 , when the object 20 does not span any line segment among a plurality of line segments constituting the bounding box 40, the annotation apparatus 100 indicates that the object 20 is located in an area inside the bounding box 40. It is determined that it is completely included, and the size of the bounding box 40 may be reduced so that the object 20 closely fits the inner region of the bounding box 40 .

In one embodiment, the annotation device 100 determines that, for each of a plurality of line segments constituting the bounding box 40, the distance from the nearest point existing on the outline of the object 20 is within a preset maximum number of pixels apart. The size of the bounding box 40 may be reduced by moving the segments 40-2 and 40-3 toward the center of the bounding box 40 until

According to the embodiment of the present invention as described above, in performing annotation on each of the plurality of images 10 for artificial intelligence (AI) learning, according to the shape of the object 20 included in the image 10 Since the size of the bounding box 40 is automatically enlarged or reduced, the operator can more easily set the bounding box 40, and the annotation work result can have a more uniform quality.

10 and 11 are one of the present invention in the examples It is an exemplary diagram for explaining the process of inputting the properties of an object according to FIG.

Referring to FIG. 10, in the annotation device 100 of the artificial intelligence learning system according to an embodiment of the present invention, before an operator directly inputs attribute information of an object 20 specified by a bounding box 40, A list 50 of recommended information that can be used as attribute information of the object 20 can be provided in advance.

To this end, the annotation device 100 creates a list 50 of recommended information based on the attributes of a project related to artificial intelligence (AI) learning, the attributes of an image, or the attributes of a worker, or included in the bounding box 40. The recommended information list 50 may be prepared based on the average color, saturation, or brightness of the selected pixels, or the recommended information list 50 may be created based on the input frequency of the worker's work log stored in the local storage device. .

After that, the annotation device 100 removes information irrelevant to the type of object 20 from the list of previously created recommendation information, and includes it in the list 50 according to the bounding box 40 or the characteristics of the object 20. information can be sorted.

Also, the annotation device 100 may receive a control signal for setting object attribute information 51 from an operator.

Referring to FIG. 11 , when one piece of information 51 is selected from a list of recommended information 50 under the control of an operator, the annotation device 100 selects the type of object 20 corresponding to the selected information 50. You can provide feedback accordingly.

In one embodiment, the annotation device 100 reflects the color or transparency set differently according to the type of the object 20 corresponding to the selected information 51, and the area inside the bounding box 40 or the object 20 A user interface (UI) related to the internal area of the can be changed.

As described above, according to the embodiment of the present invention, in performing annotation on each of the plurality of images 10 for artificial intelligence (AI) learning, in response to the object 20 set as the bounding box 40 By providing the recommended information 50, the operator can more easily input the attribute information 51 of the object 20, and can accurately recognize the attribute information of the object 20 input by the worker through feedback. there will be

Hereinafter, the operation of the annotation device 100 as described above will be described in more detail.

12 is one of the present invention in the examples followed annotation to explain how is a flowchart .

In describing the annotation method according to an embodiment of the present invention with reference to FIG. 12, the same description as described with reference to FIGS. 6 to 11 will not be repeated.

Referring to FIG. 12 , the annotation apparatus 100 may receive coordinates for setting a partial region in an image to be annotated as a bounding box from an operator (S100).

The annotation device 100 may set a bounding box based on the coordinates input by the operator (S200).

In particular, according to the shape of the object included in the image, the annotation device 100 according to an embodiment of the present invention has a size of the bounding box that is primarily set by the operator to have a size corresponding to the outline of the object. can be automatically enlarged or reduced. A detailed description of this will be described later with reference to FIG. 13 .

The annotation apparatus 100 may input attribute information of an object specified by the bounding box (S300).

In particular, the annotation device 100 according to an embodiment of the present invention may preemptively provide a list of recommended information that can be used as object attribute information before an operator directly inputs object attribute information. . A detailed description of this will be described later with reference to FIG. 14 .

When the location and size of the expanded or reduced bounding box are determined under the control of the operator, and the property information of the object is determined under the control of the operator based on the recommended information list, the annotation apparatus 100 determines the location of the bounding box. And it is possible to create an annotation work result including coordinates according to the size and determined attribute information (S400). Such a work result may have a JSON file format, but is not limited thereto.

In addition, the annotation device 100 may transmit the generated annotation work result to the learning data design device 200 or the artificial intelligence learning device 300.

13 is one of the present invention in the examples according to bounding It is a flowchart to explain how to set a box.

Referring to FIG. 13 , the annotation apparatus 100 according to an embodiment of the present invention may set a bounding box based on coordinates input from an operator (S210).

As an embodiment, the annotation apparatus 100 may set a bounding box based on a rectangle having two coordinates input from an operator as coordinates of an upper left vertex and a coordinate of a lower right vertex in an image. As another embodiment, the annotation apparatus 100 may set a bounding box based on a polygon composed of three or more line segments input from an operator.

The annotation device 100 may identify all or part of the objects included in the area inside the set bounding box (S220). To this end, the annotation apparatus 100 may perform image processing such as edge extraction, binarization, and closed region identification on the region inside the bounding box.

The annotation device 100 may determine whether a plurality of objects are included in the inner region of the bounding box (S230). As a result of the determination, if there are a plurality of objects identified in the inner region of the bounding box, the annotation apparatus 100 may select only one object from among the identified plurality of objects according to a preset priority (S240).

The annotation apparatus 100 may determine whether the identified object is completely included in the inner region of the bounding box (S250). To this end, the annotation apparatus 100 determines whether an identified object is assigned to one or more line segments among a plurality of line segments constituting the bounding box, based on continuity between pixels included in the inner area of the bounding box and pixels in the outer area of the bounding box. It can be judged whether or not there is

As a result of the determination, when it is determined that the object spans one or more line segments among a plurality of line segments constituting the bounding box, the annotation apparatus 100 may enlarge the size of the bounding box so that the object is completely included in the bounding box (S260). .

As an embodiment, the annotation apparatus 100 may enlarge the size of the bounding box by moving a line segment spanning the object toward the outside of the bounding box until the object is completely included in the inner region of the bounding box.

As another embodiment, the annotation apparatus 100 may expand the size of the bounding box in all directions while moving the center of the bounding box in the direction of the line segment where the object spans until the object is completely included in the inner region of the bounding box.

In another embodiment, the annotation apparatus 100 divides the bounding box into a plurality of detailed regions, and then enlarges only the size of the detailed region spanning the object among the divided detailed regions until the object is completely included in the inner region of the bounding box. can make it

In another embodiment, the annotation apparatus 100 newly sets an additional area outside the line segment where the object spans, and merges the newly set additional area and the bounding box until the object is completely included in the inner area of the bounding box. may be repeated.

As a result of the determination, when it is determined that the object is completely included in the inner region of the bounding box, the annotation apparatus 100 may reduce the size of the bounding box so that the object closely fits the inner region of the bounding box (S270).

In an embodiment, the annotation apparatus 100 performs bounding, for each of a plurality of line segments constituting the bounding box, until the distance from the nearest point existing on the outline of the object is within a preset maximum number of pixels apart. You can reduce the size of the bounding box by moving the line segment toward the center of the box.

14 is one of the present invention in the examples It is a flowchart to explain how to input object property information according to

Referring to FIG. 14 , the annotation apparatus 100 according to an embodiment of the present invention may create a list of recommendation information that can be used as attribute information of an object corresponding to a bounding box (310).

As an embodiment, the annotation device 100 may create a list of recommended information by including information that may be used as object attribute information according to project attributes, image attributes, or worker attributes.

As another embodiment, the annotation apparatus 100 may calculate the average color, saturation, or brightness of pixels included in the bounding box. The annotation apparatus 100 may select one recommended object type according to the calculated average color, saturation, or brightness based on a probability distribution of color, saturation, or lightness configured for each type of object. Also, the annotation device 100 may create a recommendation information list by including information related to the type of the selected recommendation object.

As another embodiment, the annotation device 100 may identify an input frequency of attribute information previously input by a worker from a worker's work log stored in a local storage device. Also, based on the identified input frequency, the annotation apparatus 100 may create a list of recommended information by including attribute information having a high input frequency.

The annotation device 100 may filter a pre-created list of recommendation information (S320). In this case, filtering of the recommended information list may be performed by removing noise and sorting.

Specifically, the annotation apparatus 100 may remove information related to the type of object that cannot be set by the work tool used by the operator in setting the bounding box, from among the information included in the list of recommendation information.

Also, the annotation device 100 may sort information remaining in the list of recommendation information.

As an embodiment, the annotation apparatus 100 may arrange information included in the list of recommended information according to a pre-set sorting criterion based on the position or size occupied by the bounding box in the image.

In another embodiment, the annotation device 100 compares the shape of the object included in the bounding box with the basic shape provided in advance for each object type, and compares the information included in the list of recommended information based on the similarity of the two shapes. can also be sorted.

The annotation device 100 may output a list of filtered recommendation information (S330). Also, the annotation apparatus 100 may receive a control signal for setting object property information from an operator (S340).

Then, when one piece of information is selected from the list of recommended information by an operator, the annotation apparatus 100 may provide feedback according to the type of object corresponding to the selected information (S350).

Specifically, when one piece of information is selected from the list of recommended information, the annotation device 100 reflects a color or transparency set differently according to the type of object corresponding to the selected information, so that the area inside the bounding box or the inside of the object You can change the user interface (UI) related to the area.

As described above, although preferred embodiments of the present invention have been disclosed in the present specification and drawings, it is in the technical field to which the present invention belongs that other modified examples based on the technical spirit of the present invention can be implemented in addition to the embodiments disclosed herein. It is self-evident to those skilled in the art. In addition, although specific terms have been used in the present specification and drawings, they are only used in a general sense to easily explain the technical content of the present invention and help understanding of the present invention, but are not intended to limit the scope of the present invention. Accordingly, the foregoing detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the present invention should be selected by reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

Claims (10)

1. Setting a partial region in an image, which is a target of annotation for artificial intelligence (AI) learning, as a bounding box under operator control;

   identifying an object completely included in an inner region of the bounding box or spanning one or more line segments among a plurality of lines constituting the bounding box;

   enlarging or reducing the size of the bounding box to have a size corresponding to the outline of the identified object; and

   When the position and size of the expanded or reduced bounding box are determined under the control of the operator, generating an annotation work result including coordinates according to the determined position and size of the bounding box, the annotation method comprising: .
2. The method of claim 1, wherein setting the bounding box

   The annotation method characterized in that the bounding box is set based on a rectangle having two coordinates input from the operator in the image as the coordinates of the upper left vertex and the coordinates of the lower right vertex.
3. The method of claim 1 , wherein identifying the object comprises:

   determining whether the object spans the line segment based on continuity between pixels included in the inner area of the bounding box and pixels in the outer area of the bounding box;

   The continuity is determined based on whether a difference in color, saturation, or brightness of two adjacent pixels is within a preset range.
4. 4. The method of claim 3, wherein identifying the object comprises:

   When there are a plurality of objects identified in the inner region of the bounding box, only one object is selected from among the plurality of objects according to a preset priority,

   Characterized in that the priority is a criterion set according to the size, position or shape of the object occupied in the image, the annotation method.
5. The method of claim 3, wherein the step of enlarging or reducing the size

   When the identified object is completely included in the inner region of the bounding box, the maximum number of pixels at which the distance from the nearest point existing on the outline of the object is set in advance for each of a plurality of line segments constituting the bounding box characterized in that, moving the line segment toward the center of the bounding box until it is within the bounding box.
6. The method of claim 3, wherein the step of enlarging or reducing the size

   When the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, the object toward the outer direction of the bounding box until the identified object is completely included in the inner area of the bounding box. Annotation method characterized by moving the line segment spanned by .
7. The method of claim 3, wherein the step of enlarging or reducing the size

   When the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, the center of the bounding box is moved until the identified object is completely included in the inner area of the bounding box. Annotation method characterized by moving in the direction of a line segment and expanding the size of the bounding box in all directions.
8. The method of claim 3, wherein the step of enlarging or reducing the size

   If the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, the identified object will be completely included in the inner area of the bounding box after dividing the bounding box into a plurality of detailed areas. characterized in that only the size of the detailed region spanning the object is enlarged among the divided detailed regions.
9. The method of claim 3, wherein the step of enlarging or reducing the size

   When the identified object spans one or more line segments among a plurality of line segments constituting the bounding box, until the identified object is completely included in the inner area of the merged bounding box, the object extends to the outer area of the line segment. An annotation method characterized by repeatedly performing a process of newly setting an additional region having a predetermined size and merging the newly set additional region and the bounding box.
10. memory;

    input output device; and

    In combination with a computing device configured to include a processor for processing instructions resident in the memory,

    setting, by the processor, a partial region in the image to be an object of annotation work for artificial intelligence (AI) learning as a bounding box according to operator control input through the input/output device;

    identifying, by the processor, an object completely included in an inner region of the bounding box or spanning one or more line segments among a plurality of line segments constituting the bounding box;

    enlarging or reducing, by the processor, the size of the bounding box to have a size corresponding to the outline of the identified object; and

    When the processor determines the location and size of the expanded or reduced bounding box by the operator's control input through the input/output device, an annotation work result including the coordinates according to the determined location and size of the bounding box A computer program recorded on a recording medium in order to execute the steps of generating a.

Images